Binding C libraries in OCaml with the dune ctypes stanza

The OCaml ctypes library along with new support for ctypes stanzas in the dune build system make it a snap to bind C libraries.

In this article, we will walk through how to prepare your project to use an external C library. But first, I’d like to try to convince you not to do this.

Stop using C code!

By using OCaml in your project you have almost certainly reaped incredible productivity and maintainability gains with hardly a sacrifice. While true, OCaml is a garbage-collected (GC) language and this adds a performance cost, it’s one of the fastest GC’d languages ever designed.

Further, if the GC is indeed getting in your way, you can write non-idiomatic OCaml which minimizes allocation to further narrow (or eliminate) the gap. My experience writing OCaml for more than ten years is that with fairly modest (but informed) effort it’s often possible to approach 2.5x the running time of carefully optimized C code. There are typically more opportunities to further refine performance, but at that point one can usually find greater economic wins elsewhere.

You needn’t only trust my opinion on this. Very fast performance is achievable in the real world with non-idiomatic OCaml code and even performant but non-idiomatic OCaml code is still a huge maintenance win over a body of C code.

Finally, if you still feel tugged towards dropping out of OCaml for low-level performance reasons, consider adding Rust to your OCaml project instead.

Rust is a hands-down better C and if you’ve never used it, this would be a good opportunity to see what all of the hype is about.

I can’t stress the win of OCaml and the loss of adding C code enough. Jane Street’s codebase has very little C code in it (and no, it’s not for the performance critical bits, as mentioned in the link above, but for the interoperability); if our team ever came across costly SIGSEGV bugs or other strange undefined behavior that were difficult to track down, it was almost always because of bugs in C code (at the bindings or the underlying libraries). We were regularly reminded of the starkness of the cost difference between maintaining even a tiny amount of C code versus millions of lines of OCaml. We only used C when we absolutely had to, and most of the time it was imposed on us by either the operating environment or vendors from outside of the company.

Which brings us to the next point.

Okay fine use C, but don’t write C stubs by hand!

Alas, sometimes you really need to use a third party library to bring in functionality, and you haven’t been able to find a suitable one written in pure OCaml (or Rust!), so you have no choice but to wrap a C library.

OCaml provides a way to bind a C library by writing “C stubs”, glue code that translates from the OCaml world into the C world. For most of OCaml’s life this was the only option. The C stubs had to be written by hand, and would come with the same set of maintenance issues that you can expect from writing new C code. Nowadays though, you can wrap C library without writing any new C code at all, through the magic of ctypes.

The ctypes library

The ctypes library gives you a way of calling C functions by simply describing the call with OCaml types and values. See this chapter in Real World OCaml (RWO) for more information on how to dispatch these C calls.

This may be everything you need to call functions defined in C libraries, though it has some limitations. The approach described in RWO relies on the ctypes “foreign” mode to dynamically bind C functions at runtime. If you try to call a missing function, or specify the wrong parameter dimensions, you’ll get errors very late in the development loop.

Using the ctypes “stub generation” (cstubs) mode lets you generate bindings to C library functions at compile time, complete with all of the type checking that can be wrestled out of C code with extra OCaml type-checking pixie dust on top.

Using cstubs is described here by Simon Beaumont, near the end. In this link, you may notice something: you need to write a lot of boilerplate build rules to compile all of this stuff.

The reason for the boilerplate is because the workflow for stub generation is complicated. You’re essentially writing intermediate OCaml programs that you compile, link and run and the output of those programs when run must be compiled and linked and run again. The output of that second pass is included in your application (or library). The build system rules to set this up can be pretty gnarly. This is actually even a simplification, it’s more gnarly than I described.

Or, at least, it used to be pretty gnarly until dune 3.0!

The dune ctypes stanza

Here is a toy example for using the new in dune 3.0 build system’s ctypes stanza support to bind a system-installed C library called libfoo.

You need dune 3.0 or later installed. Do dune --version to double check it’s at least 3.0. Run opam update and opam upgrade dune if you need to catch up.

To begin, you must declare the experimental ctypes extension in your dune-project file:

; dune-project
(lang dune 3.0)
(using ctypes 0.1)

Next, here is a dune file you can use to define an OCaml program that binds a C system library called libfoo, which offers the foo.h in a standard location.

; dune
(executable
 (name foo)
 (libraries core)
 ; ctypes backward compatibility shims warn sometimes; suppress them
 (flags (:standard -w -9-27))
 (ctypes
  (external_library_name libfoo)
  (build_flags_resolver pkg_config)
  (headers (include "foo.h"))
  (type_description
   (instance Type)
   (functor Type_description))
  (function_description
   (concurrency sequential)
   (instance Functions)
   (functor Function_description))
  (generated_types Types_generated)
  (generated_entry_point C)))

This stanza will introduce a module named C into your project, with the sub-modules Types and Functions that will have your fully-bound C types, constants, and functions.

A few brief notes on the above stanza (full docs here):

• Don’t let the word “functor” intimidate you. You don’t have to know how they work at all or write new functor calls yourself for basic usage of ctypes stub generation.
• The concurrency directive controls whether the OCaml runtime lock is held or released during the C call. More about parallel execution here.
• The build_flags_resolver directive explains how to figure out the C compile and link flags; pkg-config tells dune to probe for the flags using the pkg-config tool. Other options are to provide the flags directly, which you may use if you’ve vendored the library in your project.

Given libfoo with the C header file foo.h:

/* foo.h */
#define FOO_VERSION 1

int foo_init(void);

int foo_fnubar(char *);

void foo_exit(void);

You would create a type_description.ml file like this:

(* type_description.ml *)
open Ctypes

module Types (F : Ctypes.TYPE) = struct
  open F

  let foo_version = constant "FOO_VERSION" int
end

You would create a function_description.ml file like this:

(* function_description.ml *)
open Ctypes

(* This Types_generated module is an instantiation of the Types
   functor defined in the type_description.ml file. It's generated by
   a C program that Dune creates and runs behind the scenes.
   The name 'Types_generated' below is provided in the 'generated_types'
   field in the dune ctypes stanza above. *)
module Types = Types_generated

module Functions (F : Ctypes.FOREIGN) = struct
  open F

  let foo_init = foreign "foo_init" (void @-> returning int)

  let foo_fnubar = foreign "foo_fnubar" (string_opt @-> returning int)

  let foo_exit = foreign "foo_exit" (void @-> returning void)
end

Finally, the entry point of your executable named above (in the dune executable name stanza), foo.ml, demonstrates how to access the bound C library functions and values:

(* foo.ml *)
let () =
  if C.Types.foo_version <> 1 then
    failwith "foo only works with libfoo version 1";

  let err_code = C.Functions.foo_init () in
  if err_code <> 0 then
    failwith (Printf.sprintf "foo_init: %d" err_code);

  C.Functions.foo_fnubar (Some "fnubar!");
  C.Functions.foo_exit ()
;;

From here, one only needs to run dune build ./foo.exe to generate the stubs and build and link the example foo.exe program.

In practice these C.Functions.foo_* functions are a bit low-level, and you would probably wrap them in an OCaml module to provide more a more robust interface to the rest of your application.

Wrapping up

This isn’t the whole story of course. Handling structs and doing more complicated calls with pointers is still not a total cakewalk, but it’s generally much, much easier with ctypes. For now, one of the best ways to learn how to use more advanced ctypes is to explore projects that use ctypes. Perhaps the most sophisticated usage is found in the OCaml libuv bindings.

See also the complete dune ctypes stanza reference.

Finally, if you get stuck, feel free to post your question to the OCaml Discuss forum.