Binding C libraries in OCaml with the dune ctypes stanza
ctypes library along with new support for
ctypes stanzas in the
dune build system make it a snap to bind C libraries.
In this article, we will walk through how to prepare your project to use an external C library. But first, I’d like to try to convince you not to do this.
Stop using C code!
By using OCaml in your project you have almost certainly reaped incredible productivity and maintainability gains with hardly a sacrifice. While true, OCaml is a garbage-collected (GC) language and this adds a performance cost, it’s one of the fastest GC’d languages ever designed.
Further, if the GC is indeed getting in your way, you can write non-idiomatic OCaml which minimizes allocation to further narrow (or eliminate) the gap. My experience writing OCaml for more than ten years is that with fairly modest (but informed) effort it’s often possible to approach 2.5x the running time of carefully optimized C code. There are typically more opportunities to further refine performance, but at that point one can usually find greater economic wins elsewhere.
You needn’t only trust my opinion on this. Very fast performance is achievable in the real world with non-idiomatic OCaml code and even performant but non-idiomatic OCaml code is still a huge maintenance win over a body of C code.
Finally, if you still feel tugged towards dropping out of OCaml for low-level performance reasons, consider adding Rust to your OCaml project instead.
Rust is a hands-down better C and if you’ve never used it, this would be a good opportunity to see what all of the hype is about.
I can’t stress the win of OCaml and the loss of adding C code enough. Jane Street’s codebase has very little C code in it (and no, it’s not for the performance critical bits, as mentioned in the link above, but for the interoperability); if our team ever came across costly SIGSEGV bugs or other strange undefined behavior that were difficult to track down, it was almost always because of bugs in C code (at the bindings or the underlying libraries). We were regularly reminded of the starkness of the cost difference between maintaining even a tiny amount of C code versus millions of lines of OCaml. We only used C when we absolutely had to, and most of the time it was imposed on us by either the operating environment or vendors from outside of the company.
Which brings us to the next point.
Okay fine use C, but don’t write C stubs by hand!
Alas, sometimes you really need to use a third party library to bring in functionality, and you haven’t been able to find a suitable one written in pure OCaml (or Rust!), so you have no choice but to wrap a C library.
OCaml provides a way to bind a C library by writing “C stubs”, glue code that
translates from the OCaml world into the C world. For most of OCaml’s life this
was the only option. The C stubs had to be written by hand, and would come with
the same set of maintenance issues that you can expect from writing new C code.
Nowadays though, you can wrap C library without writing any new C code at all,
through the magic of
The ctypes library
ctypes library gives you a way of calling C functions by simply describing
the call with OCaml types and values. See this chapter in Real World OCaml (RWO) for more information on how
to dispatch these C calls.
This may be everything you need to call functions defined in C libraries, though it has some limitations. The approach described in RWO relies on the ctypes “foreign” mode to dynamically bind C functions at runtime. If you try to call a missing function, or specify the wrong parameter dimensions, you’ll get errors very late in the development loop.
ctypes “stub generation” (
cstubs) mode lets you generate bindings to C library
functions at compile time, complete with all of the type checking that can
be wrestled out of C code with extra OCaml type-checking pixie dust on top.
cstubs is described here by Simon
Beaumont, near the end. In this
link, you may notice something: you need to write a lot of boilerplate build rules to
compile all of this stuff.
The reason for the boilerplate is because the workflow for stub generation is complicated. You’re essentially writing intermediate OCaml programs that you compile, link and run and the output of those programs when run must be compiled and linked and run again. The output of that second pass is included in your application (or library). The build system rules to set this up can be pretty gnarly. This is actually even a simplification, it’s more gnarly than I described.
Or, at least, it used to be pretty gnarly until
The dune ctypes stanza
Here is a toy example for using the new in
dune 3.0 build system’s
support to bind a system-installed C library called
dune 3.0 or later installed. Do
dune --version to double check it’s at least 3.0.
opam update and
opam upgrade dune if you need to catch up.
To begin, you must declare the experimental ctypes extension in your
; dune-project (lang dune 3.0) (using ctypes 0.1)
Next, here is a dune file you can use to define an OCaml program that binds a C
system library called
libfoo, which offers the
foo.h in a standard location.
; dune (executable (name foo) (libraries core) ; ctypes backward compatibility shims warn sometimes; suppress them (flags (:standard -w -9-27)) (ctypes (external_library_name libfoo) (build_flags_resolver pkg_config) (headers (include "foo.h")) (type_description (instance Type) (functor Type_description)) (function_description (concurrency sequential) (instance Functions) (functor Function_description)) (generated_types Types_generated) (generated_entry_point C)))
This stanza will introduce a module named
C into your project, with the
Functions that will have your fully-bound C types,
constants, and functions.
A few brief notes on the above stanza (full docs here):
- Don’t let the word “functor” intimidate you. You don’t have to know how they
work at all or write new functor calls yourself for basic usage of
concurrencydirective controls whether the OCaml runtime lock is held or released during the C call. More about parallel execution here.
build_flags_resolverdirective explains how to figure out the C compile and link flags;
duneto probe for the flags using the pkg-config tool. Other options are to provide the flags directly, which you may use if you’ve vendored the library in your project.
libfoo with the C header file
/* foo.h */ #define FOO_VERSION 1 int foo_init(void); int foo_fnubar(char *); void foo_exit(void);
You would create a
type_description.ml file like this:
(* type_description.ml *) open Ctypes module Types (F : Ctypes.TYPE) = struct open F let foo_version = constant "FOO_VERSION" int end
You would create a
function_description.ml file like this:
(* function_description.ml *) open Ctypes (* This Types_generated module is an instantiation of the Types functor defined in the type_description.ml file. It's generated by a C program that Dune creates and runs behind the scenes. The name 'Types_generated' below is provided in the 'generated_types' field in the dune ctypes stanza above. *) module Types = Types_generated module Functions (F : Ctypes.FOREIGN) = struct open F let foo_init = foreign "foo_init" (void @-> returning int) let foo_fnubar = foreign "foo_fnubar" (string_opt @-> returning int) let foo_exit = foreign "foo_exit" (void @-> returning void) end
Finally, the entry point of your executable named above (in the
executable name stanza),
foo.ml, demonstrates how to access the bound C
library functions and values:
(* foo.ml *) let () = if C.Types.foo_version <> 1 then failwith "foo only works with libfoo version 1"; let err_code = C.Functions.foo_init () in if err_code <> 0 then failwith (Printf.sprintf "foo_init: %d" err_code); C.Functions.foo_fnubar (Some "fnubar!"); C.Functions.foo_exit () ;;
From here, one only needs to run
dune build ./foo.exe to generate the stubs
and build and link the example
In practice these
C.Functions.foo_* functions are a bit low-level, and you
would probably wrap them in an OCaml module to provide more a more robust interface
to the rest of your application.
This isn’t the whole story of course. Handling structs and doing more complicated
calls with pointers is still not a total cakewalk, but it’s generally much,
much easier with
ctypes. For now, one of the best ways to learn how to use
ctypes is to explore projects that use
ctypes. Perhaps the
most sophisticated usage is found in the OCaml libuv
See also the complete dune ctypes stanza reference.
Finally, if you get stuck, feel free to post your question to the OCaml Discuss forum.