Skip to content
Tutorials Center
My learning

Topics

Lesson 72 of 134 ยท Metaprogramming

Native Extensions and Fiddle

Call C code from Ruby. Learn how native extensions are built, how Fiddle loads shared libraries, maps types, handles pointers, structs and callbacks, and when to choose each approach.

Ruby is written in C, and it can talk to C. Sometimes you need that: a library for image decoding, cryptography, compression or a database driver already exists in C, and rewriting it in Ruby would be slow and risky. Sometimes you need speed for a tiny hot loop that the interpreter cannot match. And sometimes you simply want to call one function from the operating system's own library.

Ruby gives you two paths. A native extension is a compiled shared library written against Ruby's C API; it becomes part of Ruby when you require it. Fiddle is a standard library that loads any shared library at run time and calls its functions directly from Ruby code, with no C compiler needed. This lesson explains both, runs Fiddle live, and shows the C side as read-only code, because our runner cannot compile extensions to load into itself. It builds on the ideas in How Ruby Runs and the object model from Metaprogramming.

Two ways to cross the boundary

flowchart LR
  R["Your Ruby code"] --> A["Native extension<br/>compiled .so, uses Ruby C API"]
  R --> B["Fiddle<br/>loads any .so, no compile step"]
  A --> L["C library or C code"]
  B --> L
  L --> O["Operating system"]

A native extension is compiled for Ruby; Fiddle talks to existing libraries from plain Ruby.

Native extension Fiddle (or the ffi gem)
You write C (or Rust, C++) against ruby.h Ruby only
Needs a compiler Yes, at install time No
Speed of a call Fastest, almost like a built-in method Slower; every call converts values
Ruby objects Can create and inspect any VALUE directly Only plain C types and pointers
Crash risk A bug can segfault Ruby A wrong signature can segfault Ruby
Best for Gems, performance, deep integration Quick wrappers, scripts, prototypes

Whichever you choose, one rule stays: C does not check anything for you. A wrong type, a missing byte of a string, or a pointer used after free can crash the whole Ruby process with a segmentation fault instead of raising a tidy exception.

How a native extension works

A native extension is a shared library (.so on Linux, .bundle on macOS, .dll on Windows) that exports one special function named Init_<name>. When you require "fastmath", Ruby searches $LOAD_PATH for fastmath.rb or fastmath.so, loads the library into the process, and calls Init_fastmath exactly once. That function registers modules, classes and methods with the interpreter.

sequenceDiagram
  participant U as Your script
  participant R as Ruby require
  participant D as Dynamic loader
  participant I as Init_fastmath
  U->>R: require "fastmath"
  R->>D: find fastmath.so on the load path
  D-->>R: library loaded
  R->>I: call Init_fastmath
  I-->>R: FastMath module and methods defined
  R-->>U: true

Loading an extension: the library is mapped in, then its Init function defines the Ruby API.

Here is a complete, tiny extension. This is a read-only example; it needs a C compiler and the Ruby headers.

C
/* ext/fastmath/fastmath.c  (read-only example) */
#include <ruby.h>

static VALUE fastmath_sum(VALUE self, VALUE array) {
  Check_Type(array, T_ARRAY);               /* raises TypeError if it is not an Array */
  long total = 0;
  for (long i = 0; i < RARRAY_LEN(array); i++) {
    total += NUM2LONG(rb_ary_entry(array, i)); /* Ruby Integer -> C long */
  }
  return LONG2NUM(total);                   /* C long -> Ruby Integer */
}

void Init_fastmath(void) {
  VALUE mod = rb_define_module("FastMath");
  rb_define_module_function(mod, "sum", fastmath_sum, 1);  /* 1 argument */
}

Everything Ruby sees as an object is a C value of type VALUE. The macros NUM2LONG and LONG2NUM convert between Ruby integers and C integers, and Check_Type raises the proper Ruby exception for a wrong argument type. The file that tells Ruby how to build it is called extconf.rb:

Ruby
# ext/fastmath/extconf.rb  (read-only example)
require "mkmf"
create_makefile("fastmath/fastmath")

mkmf ("make makefile") inspects the machine, finds the Ruby headers and writes a Makefile. You then run ruby extconf.rb && make, and a fastmath.so appears. For a gem, you list the file in the gemspec (spec.extensions = ["ext/fastmath/extconf.rb"]), and RubyGems builds it during gem install. That is why installing gems like nokogiri or pg sometimes prints a long compile log, and why they fail with "failed to build gem native extension" when system headers are missing. The gem tooling is covered in RubyGems and Bundler, and projects usually add the rake-compiler gem to automate builds with Rake.

Three C-extension topics deserve respect, because they cause most bugs:

  • Garbage collection. If you keep a Ruby object in a C struct, you must tell the collector (a mark function), or it may free the object while you still use it. Structs that wrap C data use TypedData_Wrap_Struct with mark and free callbacks.
  • The global VM lock (GVL). Only one thread runs Ruby code at a time. A long C call that does not release the lock blocks every other thread. Extensions release it around slow work with rb_thread_call_without_gvl.
  • Errors. Use rb_raise to signal failure from C. Never exit or abort from inside an extension.

Today you do not have to use C at all. Rust gems built with the magnus and rb-sys crates give the same capabilities with memory safety, and many popular gems now ship Rust or Go parts. The loading model above is the same.

Fiddle: calling a library without compiling anything

Fiddle wraps the libffi and dlopen facilities. You open a shared library, look up a function by name, describe its signature (the C types of arguments and result), and call it like a Ruby method. Passing nil to Fiddle.dlopen gives you the symbols that are already loaded in the current process, which includes the C standard library.

The C types have Ruby constants: TYPE_INT, TYPE_LONG, TYPE_SIZE_T, TYPE_DOUBLE, TYPE_VOIDP for pointers (a Ruby string passed as TYPE_VOIDP is handed over as a pointer to its bytes) and TYPE_VOID for "returns nothing".

Ruby Calling C library functions with Fiddle

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Two results are worth noticing. (The TypeError is the lucky case: the declared type is int, and a string cannot become one. Passing the integer 12 where a const char* is expected is not refused: C would treat 12 as a memory address and the process would crash.) strlen("hรฉllo") returns 6 while Ruby's length returns 5: C counts bytes and the accented letter is two bytes in UTF-8. And atoi stops at the first character that is not a digit, as the C function always did. The sizes at the end differ between platforms (a long is 8 bytes on 64-bit Linux and macOS, 4 on 64-bit Windows), so code that depends on them should read the constants instead of hard-coding numbers.

Loading your own or other libraries

Fiddle.dlopen("libm.so.6") opens a named library on Linux, libm.dylib is the macOS equivalent and msvcrt.dll-style names apply on Windows. The Fiddle::Importer accepts the library name directly:

Ruby
# read-only example: library names differ per platform
require "fiddle/import"

module Sqlite
  extend Fiddle::Importer
  dlload "libsqlite3.so.0"          # "libsqlite3.dylib" on macOS
  extern "const char* sqlite3_libversion()"
end

puts Sqlite.sqlite3_libversion

If the library is missing, dlopen raises Fiddle::DLError, a rescue-able error you should turn into a friendly message. The ffi gem offers a similar but more portable API with extras such as attach_function, enums and layouts; most of what you learn here transfers.

Memory, pointers and structs

C functions often work with memory instead of Ruby objects. Fiddle::Pointer represents an address plus an optional size. You can allocate a block with Pointer.malloc, write and read bytes by slicing (ptr[offset, length]), and use String#pack / unpack to convert between Ruby numbers and raw bytes. The Fiddle::Importer.struct helper declares a C struct layout, so you get typed accessors without counting offsets by hand.

Ruby Pointers, raw bytes and a C struct

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Memory you get from Pointer.malloc without the RUBY_FREE argument is yours to release with free. Releasing it twice, or reading it after it was freed, is the classic way to crash. Pass Fiddle::RUBY_FREE when you want Ruby's garbage collector to free the block for you, and never keep a pointer longer than the object that owns the memory.

Callbacks: C calling back into Ruby

Some C functions take a function pointer. The standard qsort sorts any array and asks you to supply the comparison function. Fiddle can turn a Ruby block into such a pointer with Fiddle::Closure::BlockCaller: you give the return type, the argument types and a block. The block receives the arguments as raw addresses, so you read the values from memory with Fiddle::Pointer.

Ruby qsort calling a Ruby block as its comparison

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Callbacks are the hardest part of foreign calls: the closure must stay alive for as long as C may call it (keep it in a variable), and exceptions raised inside it do not travel through C code safely, so rescue them inside the block.

Choosing an approach

  • Just need a Ruby library? Look for a gem first. Someone has probably wrapped your C library already.
  • One or two functions, a script, a prototype? Use Fiddle or ffi. No build step, easy to share.
  • Performance critical, or a public gem with a lot of C? Write a native extension (or a Rust one) and ship it with extconf.rb and rake-compiler.
  • Slow Ruby code? Do not reach for C yet. Re-read Performance Tips and YJIT and Ruby Performance Internals: a better algorithm or fewer allocations is almost always cheaper than a foreign-language rewrite.

Finally, remember the portability cost. A native extension must be compiled for each platform and Ruby version, and a Fiddle wrapper bakes in library names and type sizes. Test on every operating system you claim to support.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does Ruby do when you require a compiled extension named fastmath?

    The shared library is loaded into the Ruby process, and its Init_fastmath function registers the modules, classes and methods.
  2. 2. Which tool lets you call a function in an existing shared library without writing or compiling any C?

    Fiddle (like the ffi gem) opens a library at run time and calls functions using signatures you describe in Ruby. The others are part of building extensions.
  3. 3. strlen.call("hรฉllo") returns 6 but "hรฉllo".length is 5. Why?

    C strings are byte arrays. strlen counts bytes up to the terminating zero, while Ruby's length counts characters.
  4. 4. What is the main danger of getting a Fiddle signature or pointer wrong?

    C performs no type or bounds checks. Wrong argument types, wrong sizes or freed pointers corrupt memory and can take the whole process down.
  5. 5. Why must you keep a Fiddle::Closure in a variable while C code may call it?

    The function pointer given to C refers to the closure object. If Ruby collects it, C would call freed memory.

Summary

  • A native extension is a shared library with an Init_<name> function that defines Ruby classes and methods using the C API; extconf.rb and mkmf generate the Makefile that builds it.
  • Extensions must cooperate with the garbage collector (mark functions), release the GVL for slow work and report failures with rb_raise.
  • Fiddle loads existing shared libraries at run time: Fiddle.dlopen, Fiddle::Function, or the shorter Fiddle::Importer with extern.
  • C types map to TYPE_* constants; strings are byte arrays, so lengths can differ from Ruby's.
  • Fiddle::Pointer gives raw memory access, pack/unpack convert numbers and bytes, and Importer.struct declares C structs.
  • A Fiddle::Closure::BlockCaller turns a Ruby block into a C function pointer; keep it referenced while C might call it.
  • Prefer a gem, then Fiddle or ffi, then your own extension; optimize your Ruby before leaving it.

Next: Regular Expressions