Create a Python module in RustDaniele Esposti's Blog, in 18 September 2016
Rust is a new language which aims to be fast a C/C++ but safer and more expressive. Writing code in Rust is not just fun but it also can be useful to write modules for Python to replace CPU-bound code with it’s counterpart in Rust.
Thanks to the rust-cpython project it’s possible to execute Python code from Rust and vice-versa build a module in Rust for Python. However the given examples and documentation shows you only how to execute Python from Rust, where in this post I’ll show you how to build a module in Rust to be called by Python code.
The code examples in this post uses Python 2.7 or 3.x indifferently and Rust 1.11+.
If you need to compile this code for Python 2.7 a small change must be made in the
Cargo.toml file, it will be explained further down in the post.
The first trivial example
Let’s start with a simple example, a function which return an “Hello World” string, implemented in Rust and saved in
The first notable thing is that all the functions which will be called by the Python code needs to receive as the first parameter an instance of the current Python interpreter (argument
py of type
Python and if they return a value it should be wrapped in a
PyResult type (an alias to the
Result type). Other functions not exposed to the Python code don’t need these constraints.
The second thing is that the return value is a Python string and not a Rust
str type, this is possible because the
rust-cpython crate expose to you the Python built-in types in Rust so you don’t need to return a C string and convert into into a Python string later. This is a big boost in performances because the compiler will optimise the creation of
PyString instance and the Python code can use the instance as is without any overhead.
Now we need to expose this function as part of the module, this can be done with the
To conclude the setup let’s define the
Now we are ready to compile our dynamic library and call the
hello() function from Python:
As you can see to use our
example module is the same as importing any other Python module, no difference at all except the fact that the executed code is native C code.
A more complete example
Now that we know hot to degine, implement and call a function written in Rust from Python code let’s move to something a little more complex which involves data convertion betwfrom Python to Rust and vice-versa and a little bit of error handling.
For this example I’m going to implement a function
greetings() which accept a string as parameter and returns a formatted greeting; all the strings will be Unicode strings and if the string passed as function’s argument contains an invalid codepoint an
UnicodeDecodeError will be raised. Here the implementation:
As you notice the conversion from Python’s string type to a Rust’s String type is done by pattern matching.
Ok() case we format the
String instance into the greetings phrase and we convert the result back into a
PyString instance because the API of the
PyString type doesn’t expose any method to perform string concatenation nor formatting.
Err() case we just propagate the error out of the function and up into the Python code; as per documentation of the
PyString::to_string() method the error will be a Python’s
UnicodeDecodeError exception which can be catch and handled by the Python code.
The last step is to expose the
greetings() function as part of the Python module (here alongside the previous
Compiling the library, importing it and calling the function, including calling it with an invalid Unicode codepoint will raise the Python exception as expected:
Targeting different Python version
rust-cpython compiles against Python 3.4 or 3.5 but it’s possible to compile it agains Python 2.7 as well. To be able to do that we need to specify the correct feature for the
rust-cpython crate in our
Rust is a very promising system language which gives you the ability to produce very fast binary code with a relatively easy syntax. Using Rust to replace CPU-bound Python code give you a boost in performace with no overhead at all on calling the Rust code from Python code; instead of calling C functions using
ctypes and convert the C data types into Python data types
rust-cpython provides Python data types in Rust directly. Optimisations applied by the compiler also generates optimal code in term of speed and memory usage.
Building a Python module is pretty easy as well and projects like rust-python-ext are trying to integrate the compilation of the Rust code with Python’s setuptools to make the entire distribution and deploy process smoother as possible.
All the code in this post is available on GitHub.