Both R and Python make it reasonably easy to work with compiled
extensions. But how to access objects in one environment from the other
and share state or (non-trivial) objects remains trickier.
Recently (and while r-forge was ‘resting’ so we opened GitHub
Discussions) a question was asked concerning R
and Python object pointer exchange.
This lead to a pretty decent discussion including arrow interchange demos (pretty
ideal if dealing with data.frame-alike objects), but once the focus is
on more ‘library-specific’ objects from a given (C or C++, say) library
it is less clear what to do, or how involved it may get.
R has external pointers, and these make it feasible to instantiate
the same object in Python. To demonstrate, I created a pair of
(minimal) packages wrapping a lovely (small) class from the excellent spdlog library by Gabi Melman, and more specifically
in an adapted-for-R version (to avoid some R CMD check
nags) in my RcppSpdlog
package. It is essentially a nicer/fancier C++ version of the
tic() and tic() timing scheme. When an object
is instantiated, it ‘starts the clock’ and when we accessing it later it
prints the time elapsed in microsecond resolution. In Modern C++ this
takes little more than keeping an internal chrono
object.
Which makes for a nice, small, yet specific object to pass to Python.
So the R side of
the package pair instantiates such an object, and accesses its
address. For different reasons, sending a ‘raw’ pointer across does not
work so well, but a string with the address printed works fabulously
(and is a paradigm used around other packages so we did not invent
this). Over on the Python side of the
package pair, we then take this string representation and pass it to
a little bit of pybind11 code to
instantiate a new object. This can of course also expose functionality
such as the ‘show time elapsed’ feature, either formatted or just
numerically, of interest here.
And that is all that there is! Now this can be done from R as well
thanks to reticulate
as the demo() (also shown on the package README.md)
shows:
> library(chronometre)
> demo("chronometre", ask=FALSE)
demo(chronometre)
---- ~~~~~~~~~~~
> #!/usr/bin/env r
>
> stopifnot("Demo requires 'reticulate'" = requireNamespace("reticulate", quietly=TRUE))
> stopifnot("Demo requires 'RcppSpdlog'" = requireNamespace("RcppSpdlog", quietly=TRUE))
> stopifnot("Demo requires 'xptr'" = requireNamespace("xptr", quietly=TRUE))
> library(reticulate)
> ## reticulate and Python in general these days really want a venv so we will use one,
> ## the default value is a location used locally; if needed create one
> ## check for existing virtualenv to use, or else set one up
> venvdir <- Sys.getenv("CHRONOMETRE_VENV", "/opt/venv/chronometre")
> if (dir.exists(venvdir)) {
+ > use_virtualenv(venvdir, required = TRUE)
+ > } else {
+ > ## create a virtual environment, but make it temporary
+ > Sys.setenv(RETICULATE_VIRTUALENV_ROOT=tempdir())
+ > virtualenv_create("r-reticulate-env")
+ > virtualenv_install("r-reticulate-env", packages = c("chronometre"))
+ > use_virtualenv("r-reticulate-env", required = TRUE)
+ > }
> sw <- RcppSpdlog::get_stopwatch() # we use a C++ struct as example
> Sys.sleep(0.5) # imagine doing some code here
> print(sw) # stopwatch shows elapsed time
0.501220
> xptr::is_xptr(sw) # this is an external pointer in R
[1] TRUE
> xptr::xptr_address(sw) # get address, format is "0x...."
[1] "0x58adb5918510"
> sw2 <- xptr::new_xptr(xptr::xptr_address(sw)) # cloned (!!) but unclassed
> attr(sw2, "class") <- c("stopwatch", "externalptr") # class it .. and then use it!
> print(sw2) # `xptr` allows us close and use
0.501597
> sw3 <- ch$Stopwatch( xptr::xptr_address(sw) ) # new Python object via string ctor
> print(sw3$elapsed()) # shows output via Python I/O
datetime.timedelta(microseconds=502013)
> cat(sw3$count(), "\n") # shows double
0.502657
> print(sw) # object still works in R
0.502721
>
The same object, instantiated in R is used in Python and thereafter
again in R. While this object here is minimal in features, the
concept of passing a pointer is universal. We could use it for
any interesting object that R can access and Python too can instantiate.
Obviously, there be dragons as we pass pointers so one may want to
ascertain that headers from corresponding compatible versions are used
etc but principle is unaffected and should just work.
Both parts of this pair of packages are now at the corresponding
repositories: PyPI
and CRAN.
As I commonly do here on package (change) announcements, I include the
(minimal so far) set of high-level changes for the R package.
Changes in version 0.0.2
(2026-02-05)
Changes in version 0.0.1
(2026-01-25)
- Initial version and CRAN upload
Questions, suggestions, bug reports, … are welcome at either the (now
awoken from the R-Forge slumber) Rcpp
mailing list or the newer Rcpp
Discussions.
This post by Dirk
Eddelbuettel originated on his Thinking inside the box
blog. If you like this or other open-source work I do, you can sponsor me at
GitHub.
/code/chronometre |
permanent link
