rustlang dynamic library part 2

A previous post was about getting a simple rust dynamic library working. The next thing I wanted to do was try passing a more complex type from the dynamic library to the executable using it.

First, I changed dynamiclib to:

#[derive(Debug)]
pub struct Tuple (u32,);

#[derive(Debug)]
pub struct MyStruct {
    s1: &'static str,
    s2: String,
    t: Tuple,
}

#[no_mangle]
pub extern "C" fn test() -> MyStruct {
    MyStruct {
        s1: "hello",
        s2: "world".to_string(),
        t: Tuple(47),
    }
}

And dynamic to:

#[link(name="dynamiclib")]
extern {
    fn test() -> MyStruct;
}

fn main() {
    let retval = unsafe { test() };
    println!("Got: {:?}", retval);
}

And (somewhat surprisingly) it worked as expected. Although you get the following error:

warning: found struct without foreign-function-safe representation annotation in foreign module, consider adding a #[repr(C)] attribute to the type, #[warn(improper_ctypes)] on by default

At first I considered appeasing the warning, but if it's a library written in rust and called from rust, shouldn't the default

repr(Rust)

be safe/correct?

Next, I decided to try loading and re-loading the shared library using this more complex struct... and I started getting SIGSEGV the second time I attempted to use the library.

I came across libloading (crates.io) which purports to be safer wrapper for working with dyanmic libraries, and if nothing else has much better documentation.

A quick change to dynamic/src/main.rs (devoid of error-checking):

    let lib = Library::new("libdynamiclib.dylib").unwrap();
    let test: Symbol MyStruct> = unsafe {
        lib.get(b"test\0").unwrap()
    };
    println!("Got: {:?}", test());

And it works correctly even when re-loading the library. So, I was either using stale symbols, or something else. Still several things that I obviously don't fully understand. Hopefully pouring over Rustonomicon will reveal the secrets to me.

rustlang dynamic library

Was wanting to figure out how to do a rust dynamic library because I've been thinking about how I could apply software updates to a running system. I came across dynamic_lib which is apparently deprecated and replaced with dylib (with the same api), but there's still fairly little documentation. Pretty much the only simple working example I was able to find was this StackOverflow question. But it's from over a year ago and predates even rust 1.0 (I'm currently using 1.7).

To start off, use cargo to setup two projects; one for the library itself, another for the executable that uses it:

cargo new dynamiclib
cargo new --bin dynamic
cd dynamiclib

Add the following to dynamiclib's Cargo.toml:

[lib]
crate-type = ["dylib"]

Setting the crate-type to "dylib" means a dynamic library will be built (rather than an executable or static library). Save the following as src/lib.rs:

#[no_mangle]
pub extern "C" fn test() -> u32 {
    47
}

And build the library by running:

cargo build

Next, we need to create the executable that loads and uses the library we just created. Add the following to dynamic/Cargo.toml:

[dependencies]
dylib = "0.0.2"

And save the following as dynamic/src/main.rs:

extern crate dylib;
use dylib::DynamicLibrary;
use std::path::Path;

fn main() {
    // I'm on OSX, obviously
    match DynamicLibrary::open(Some(Path::new("libdynamiclib.dylib"))) {
        Ok(lib) => {
            let test = unsafe {
                let ptr = lib.symbol::("test").unwrap();
                println!("Found it: {:?}", ptr);
                std::mem::transmute::<_, fn() -> u32>(ptr)
            };
            println!("Got: {}", test());
        },
        Err(e) => println!("Failed: {}", e)
    }
}

I eventually came across this post that's about calling rust from Node.js, and it made me realise that dylib is just a wrapper around Foreign Function Interface (FFI) (as opposed to some special/magical functionality provided by the runtime or a core library). This is easily confirmed if you check the dylib source; it just uses libc to call dlopen and kin. That means that rather than using prepend_search_path to ensure the dynamic library can be found, you can set LD_LIBRARY_PATH to the output path of dynamiclib (e.g. $HOME/dynamiclib/target/release). Allowing you to build and run the executable with:

LD_LIBRARY_PATH=$HOME/dynamiclib/target/release crate run

I'm admittedly a little bit disappointed that dynamic libraries are so painfully close to native interop preventing idiomatic implementations that transparently work like native code. Something a bit more like the magic in .Net- but that would bring the runtime nastiness that rust tries so hard to avoid.

It's worth noting that if you have no intention of re-loading the dynamic library and you simply want to link with it, I could have changed dynamic/src/main.rs to:

#[link(name="dynamiclib")]
extern {
    fn test() -> u32;
}

fn main() {
    println!("Got: {}", unsafe { test() });
}

Now dynamiclib needs to be in one of the locations that rust searches for libraries (e.g. dynamic/target/debug) at build time.