Skip to content
Tutorials Center
My learning

Topics

Lesson 32 of 40 ยท Testing and Tooling

Calling Python, C and R

Use code from C, Python and R inside Julia: ccall and shared libraries, PythonCall.jl, RCall.jl, type conversion, array layout, and common pitfalls.

No language has every library. Julia's answer is to make calling other languages cheap and easy, so you can use a decades-old C numerical library, a Python machine-learning package, or an R statistics routine without rewriting it. Julia can call C directly with almost no overhead, and packages built on top of that give you Python and R.

This lesson shows the three routes: ccall for C, PythonCall.jl for Python and RCall.jl for R. The last section covers the pitfalls that catch everyone: array layout, indexing, copying, and memory ownership. Julia, C, Python and R are not available to run on this site, so all of the interop code is read-only. The runnable examples are Python stand-ins that demonstrate the two ideas that matter most: memory layout and struct layout.

Calling C with ccall

C functions live in shared libraries (.so on Linux, .dylib on macOS, .dll on Windows). ccall calls one by name, and you tell Julia the return type and the argument types, because C libraries carry no type information at run time:

JULIA
# double sin(double) from the C math library
ccall(:sin, Cdouble, (Cdouble,), 1.0)      # 0.8414709848078965

# size_t strlen(const char *) from the C standard library
ccall(:strlen, Csize_t, (Cstring,), "hello")   # 0x0000000000000005

# a function from your own library
ccall((:add_ints, "libmylib"), Cint, (Cint, Cint), 2, 3)   # 5

The syntax is ccall((function_name, library), return_type, (argument_types...), arguments...). The built-in C types are Cint, Cdouble, Cchar, Csize_t, Cstring, Ptr{T} and so on. They map to Julia types of the right size, for example Cint is Int32. Because the call is compiled to a normal machine call, there is no per-call conversion cost for plain numbers.

Pass an array by giving a Ptr (or a typed array, which converts automatically) and make sure it stays alive during the call:

JULIA
# void scale(double *x, size_t n, double factor);  in libmylib
function scale!(x::Vector{Float64}, factor::Float64)
    ccall((:scale, "libmylib"), Cvoid,
          (Ptr{Cdouble}, Csize_t, Cdouble), x, length(x), factor)
    return x
end

Julia arrays are contiguous blocks, so a Vector{Float64} has the same layout as a C double[]. Get a path to a library with Libdl.dlopen, or better, use a prebuilt "JLL" package (such as OpenBLAS_jll) that ships the binary for every platform. Never hard-code a path.

For C structs, define a Julia struct with the same fields in the same order; Julia uses the C layout for isbits structs:

JULIA
struct Point        # struct Point { double x; double y; }
    x::Cdouble
    y::Cdouble
end

ccall((:length, "libgeometry"), Cdouble, (Point,), Point(3.0, 4.0))

If you want to call C++, wrap it with extern "C" functions, or use a tool like CxxWrap.jl.

Three bridges

ccall Built in. Calls C libraries directly.
PythonCall.jl Import and run Python packages.
RCall.jl Run R code and move data both ways.
1 / 2

Calling Python with PythonCall.jl

PythonCall manages its own Python environment through the package manager (it uses CondaPkg.jl), so installing a Python library is also reproducible. In Julia:

JULIA
using PythonCall

math = pyimport("math")
math.sqrt(2)                      # Python: 1.4142135623730951
# the result is a Python object (Py); convert it to a Julia value:
pyconvert(Float64, math.sqrt(2))  # 1.4142135623730951

np = pyimport("numpy")
a = np.arange(6).reshape(2, 3)    # a Python NumPy array, wrapped as a Py
size(a)                           # (2, 3)

Results of Python calls come back wrapped in a Py object, and you ask for a Julia value with pyconvert(Type, x). Wrapped objects support attribute access (a.shape), calls, indexing and iteration using familiar Julia syntax. Note that Python indexing stays 0-based on Py objects: a[0] works on the Python object. You can run Python source with pyexec and pyeval.

The reverse direction, calling Julia from Python, uses the juliacall package. A third option is PyCall.jl, an older package that converts more automatically (with some surprises); new code typically uses PythonCall.

Calling R with RCall.jl

RCall runs an embedded R session. The R"..." string macro evaluates R code, and $name in the string pastes in a Julia variable:

JULIA
using RCall

x = [1.0, 2.0, 3.0, 4.0, 10.0]

R"mean($x)"                    # RObject with value 4.0
R"sd($x)"                      # 3.535534

y = rcopy(R"cumsum($x)")       # convert to a Julia Vector{Float64}
# [1.0, 3.0, 6.0, 10.0, 20.0]

R"fit <- lm(y ~ x, data = data.frame(x = $x, y = $x .^ 2))"

Switch to R's own prompt from the Julia REPL by pressing $, which is handy for plots with ggplot2. Use rcopy to copy an R result into Julia, and @rput x and @rget y to move named variables in and out.

Pitfalls that catch everyone

The calls are easy. The surprises are in the data.

Array layout. Julia, R, Fortran and MATLAB store matrices in column-major order (the first index changes fastest in memory). C and NumPy default to row-major. When a matrix crosses a boundary, one of two things happens: it is copied and rearranged (safe, slower) or shared (fast, but the "shape" may be the transpose of what you expect). For a C routine, that means you must say whether it expects rows or columns contiguous. A LAPACK routine from Fortran expects column-major, which is why Julia can pass its matrices to it without copying.

Indexing. Julia and R count from 1. Python and C count from 0. Off-by-one bugs happen when you pass indices across a boundary, for example the position returned by Python's argmax is 0-based and needs + 1 before use as a Julia index.

Copies versus views. Converting a large array with pyconvert or rcopy copies it. For repeated calls on big data, prefer wrappers that share memory (PythonCall can wrap NumPy arrays without copying through PyArray), and avoid converting back and forth inside a loop.

Ownership and lifetimes. If C allocates memory, C must free it; if Julia allocates, the garbage collector may move or free it when the array is no longer referenced. When you pass a pointer to C, wrap the code in GC.@preserve x begin ... end so the array is not collected during the call.

Strings and encoding. C strings are NUL-terminated bytes; Julia strings are UTF-8 and may contain NUL. Cstring conversion throws an error if the Julia string has a NUL inside it.

Dependencies and threads. Embedding Python or R brings their own runtimes. Python's global interpreter lock (GIL) means Python calls from multiple Julia threads do not run in parallel. Mixing conflicting versions of shared libraries (for example two copies of OpenMP) can crash. Pin your interop environment in Project.toml and CondaPkg.toml, as described in Packages and Environments.

Prefer native Julia when it exists. Calling out is the right choice for a library that has no Julia equivalent. It is the wrong choice for a simple task in a hot loop, because every crossing has overhead and hides types from the compiler (the result is a Py or RObject, not a concrete type), so the code after it can be type-unstable.

Stand-in: memory layout

Python's NumPy lets you choose the memory layout, which makes it a good place to see the column-major versus row-major difference that causes most interop surprises. The array has the same values either way, but the order of the bytes differs:

Python Row-major and column-major

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Julia's [1 2 3; 4 5 6] has the memory order 1, 4, 2, 5, 3, 6, the second line above. A library that reads the buffer assuming row-major order would see a 3 by 2 matrix full of the wrong values.

Stand-in: struct layout

When Julia passes a struct to C, both sides must agree on field order, sizes and padding. Python's ctypes has the same C struct model, so it can show padding that the compiler inserts:

Python C struct layout and padding

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A Padded struct holds one byte of data plus eight for the double, but takes sixteen bytes because the compiler pads for alignment. Julia's struct Padded; flag::Int8; value::Float64; end has the same layout, and fieldoffset(Padded, 2) reports 0x0000000000000008.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What must you give ccall that a normal Julia call does not need?

    C shared libraries carry no type information at run time, so you declare the return type and a tuple of argument types.
  2. 2. How does PythonCall hand back the result of a Python function?

    Results are Py objects. Call pyconvert(Type, x) to get a Julia value.
  3. 3. A matrix is passed to a C routine that expects row-major data, but the matrix is a Julia Matrix. What is the risk?

    Julia stores the first index fastest in memory. Without a copy or a transpose the C code sees scrambled rows and columns.
  4. 4. Why wrap a ccall that receives a pointer in GC.@preserve?

    Julia only knows an array is in use while a reference to it exists. A raw pointer is not a reference, so you preserve the array explicitly.
  5. 5. Which is the best reason to call Python or R from Julia?

    Interop is for reusing existing libraries. For ordinary numeric work, native Julia is faster and keeps the types visible to the compiler.

Summary

  • ccall((:name, "lib"), RetType, (ArgTypes...), args...) calls C with no per-call overhead; use JLL packages to ship the library.
  • PythonCall.jl returns Py objects (convert with pyconvert); RCall.jl uses R"...", rcopy, @rput and @rget.
  • Watch for column-major versus row-major layout, 1-based versus 0-based indexing, and silent copies.
  • Use GC.@preserve when passing pointers, and remember who owns and frees memory.
  • Prefer native Julia code when it exists; keep interop at the edges of your program, not inside hot loops.

Next: put the tools together in Project: Numerical Simulation.