Wrapping callbacks without userdata
— Rust
It’s easy to find articles describing how to wrap Rust closures to pass to C libraries that will carry a void* to your callback, but what can you do when no userdata value is provided? This post is based on work I’ve done for the lua crate.
The Situation
Let’s imagine that the C library provides an API roughly analogous to the following:
type ffi_Object = ...; // opaque type representing foreign Object
type ffi_Callback = unsafe extern fn(*mut ffi_Object) -> u32;
extern {
fn ffi_push_callback(_: *mut ffi_Object, _: ffi_Callback);
}
We currently have a safe wrapper type named Object and we want to enable consumers of our library to push callbacks onto it without writing any unsafe code, but how can we do that without a userdata pointer? While the lack of a userdata pointer prevents us from carrying any state with the callback (ruling out closures), it’s still possible to wrap ordinary functions.
The Solution
First, let’s think about the signature we want the users of our wrapper to conform to. The FFI demands unsafe extern fn(*mut ffi_Object) -> u32, but we want to allow such callbacks to be totally safe. This means no unsafe or extern, and we should replace *mut ffi_Object with a safe analogue. Since the callback does not take ownership, this is &mut Object.
Now we’re going to need to define a function that converts from our safe signature to the unsafe signature. At first, you might try to write:
fn wrap_callback(callback: fn(&mut Object) -> u32) -> ffi_Callback { ... }
but this leads to a dead end. A fn(&mut Object) -> u32 is a function pointer, a runtime value, state! Exactly what we can’t pack into an ffi_Callback. We’re going to need to build, at compile time, a different ffi_Callback for each function we’re ever passed. Luckily, Rust has the useful property that each function gets its own unique type, so we can use a type parameter - our wrap_callback function is going to be generic. These unique types can’t be written directly, so let’s add a dummy parameter _: F to allow inference to fill them in for us.
fn wrap_callback<F: Fn(&mut Object) -> u32>(_: F) -> ffi_Callback {
Now that we know the function’s signature, what about its contents? First, let’s not forget to reintroduce a restriction we just accidentally loosened: this signature accepts closures with state, and it’s not possible for us to support that. Let’s be sure we’re being passed ordinary functions (or equally-valid stateless closures):
assert!(mem::size_of::<F>() == 0);
It’s a little unfortunate that we have to runtime assert this and can’t check it at compile time, but it’s the best we can do in current Rust. Make sure this requirement is displayed in your library’s documentation.
Alright, now what? Well, we need to define a function with the signature demanded by ffi_Callback. Unlike wrap_callback, which accepts a function as a type parameter and returns an ffi_Callback, this function will accept the same and be an ffi_Callback.
unsafe extern fn wrapped<F: Fn(&mut Object) -> u32>(ptr: *mut ffi_Object) -> u32 {
First, let’s convert our *mut ffi_Object into a safe Object we can use:
let mut object = Object::from_ptr(ptr);
Now we need to obtain a value of type F in order to call it. We just asserted that F is zero-sized, so there’s exactly one possible value, but the compiler doesn’t know that. We’ll have to unsafely construct that value somehow - probably the simplest way is mem::zeroed. Then, we’re going to call that function with a reference to the Object, and save the result:
// | generate value || call it |
let result = mem::zeroed::<F>()(&mut object);
Lastly, since our Object is an owned type, we need to avoid dropping it, then return the result we just generated. A future post will cover how to handle this in a less risky way.
mem::forget(object);
result
}
Now we’re back in wrap_callback, ready to return the function satisfying ffi_Closure we just built:
wrapped::<F>
}
And that’s it! Using it is simple, and as we hoped, contains no use of unsafe:
fn our_callback(object: &mut Object) -> u32 { ... }
fn push_our_callback(object: &mut Object) {
let wrapped: ffi_Callback = wrap_callback(our_callback);
object.push_callback(wrapped);
}
Depending on your needs, you can make wrap_callback public or use it internally.
Full Example
Here’s the code in this article all put together, including a dummy version of Object which I omitted for brevity. If you want to experiment, you can use this code on the Rust Playground. I’m interested in feedback and can accept it by email or the comments section of wherever you found this post.
#![allow(non_camel_case_types)]
use std::mem;
// The Situation
struct ffi_Object;
type ffi_Callback = unsafe extern fn(*mut ffi_Object) -> u32;
extern fn ffi_push_callback(_: *mut ffi_Object, _: ffi_Callback) {}
// Dummy wrapper
struct Object {
ptr: *mut ffi_Object
}
impl Object {
fn new() -> Object {
// initialize ffi_Object here
Object { ptr: std::ptr::null_mut() }
}
fn from_ptr(ptr: *mut ffi_Object) -> Object {
Object { ptr: ptr }
}
fn push_callback(&mut self, cb: ffi_Callback) {
ffi_push_callback(self.ptr, cb);
}
}
impl Drop for Object {
fn drop(&mut self) {
// destroy ffi_Object here
}
}
// The Solution
fn wrap_callback<F: Fn(&mut Object) -> u32>(_: F) -> ffi_Callback {
assert!(mem::size_of::<F>() == 0);
unsafe extern fn wrapped<F: Fn(&mut Object) -> u32>(ptr: *mut ffi_Object) -> u32 {
let mut object = Object::from_ptr(ptr);
let result = mem::transmute::<_, &F>(&())(&mut object);
mem::forget(object);
result
}
wrapped::<F>
}
// Usage
fn our_callback(_: &mut Object) -> u32 {
0
}
fn main() {
let mut object = Object::new();
let wrapped: ffi_Callback = wrap_callback(our_callback);
object.push_callback(wrapped);
}
Addendum: Safer, Less Usable
If you’re not satisfied with the runtime assert, it can be eliminated if you’re willing to sacrifice usability:
// Definition
trait Callback {
fn call(object: &mut Object) -> i32;
}
fn wrap_callback<F: Callback>() -> ffi_Callback {
unsafe extern fn wrapped<F: Callback>(object: *mut ffi_Object) -> i32 {
let mut object = Object::from_ptr(ptr);
let result = F::call(&mut object);
mem::forget(object);
result
}
wrapped::<F>
}
// Usage
struct OurCallback;
impl Callback for OurCallback {
fn call(object: &mut Object) -> u32 {
...
}
}
fn push_our_callback(object: &mut Object) {
object.push_callback(wrap_callback::<OurCallback>());
}
Thanks to Lalaland on #rust for this suggestion.
Addendum: The Past
There was once a time when those unique per-function types we made use of were not zero-sized! This was just an implementation flaw and has since been resolved, but it was possible to work around it using closures that captured no state, and were zero-sized:
fn push_our_callback(object: &mut Object) {
object.push_callback(wrap_callback(|s| our_callback(s)));
}