Now Reading
Autogenerating Rust-JS bindings with UniFFI – Mozilla Hacks

Autogenerating Rust-JS bindings with UniFFI – Mozilla Hacks

2023-08-09 21:22:20

I work on the Firefox sync staff at Mozilla. 4 years in the past, we wrote a blog post describing our strategy to ship cross-platform Rust components for syncing and storage on all our platforms. The imaginative and prescient was to consolidate the separate implementations of options like historical past, logins, and syncing that existed on Firefox Desktop, Android, and iOS.

We’d exchange these implementations with a core written in Rust and a set of hand-written international language wrappers for every platform: JavaScript for Desktop, Kotlin for Android, and Swift for iOS.

Since then, we’ve realized some classes and needed to modify our technique. It seems that creating hand-written wrappers in a number of languages is a big time-sink. The wrappers required a big period of time to jot down, however extra importantly, they have been accountable for many severe bugs.

These bugs have been straightforward to overlook, onerous to debug, and sometimes led to crashes. One of many largest advantages of Rust is reminiscence security, however these hand-written wrappers have been negating a lot of that profit.

To resolve this downside, we developed UniFFI: a Rust library for auto-generating international language bindings. UniFFI allowed us to create wrappers shortly and safely, however there was one concern: UniFFI supported Kotlin and Swift, however not JavaScript, which powers the Firefox Desktop front-end. UniFFI helped us ship shared parts for Firefox Android and iOS, however Desktop remained out of attain.

This modified with Firefox 105 once we added help for producing JavaScript bindings through UniFFI which enabled us to proceed pushing ahead on our single part imaginative and prescient. This challenge validated some core ideas which have been in UniFFI from the beginning but additionally required us to increase UniFFI in a number of methods. This weblog submit will stroll by a number of the points that arose alongside the way in which and the way we dealt with them.

Prior Artwork

This challenge has already been tried at the very least as soon as earlier than at Mozilla. The staff was in a position to get a number of the performance supported, however some elements remained out of attain. One of many first issues we realized was that the overall strategy the earlier makes an attempt took would in all probability not help the UniFFI options we have been utilizing in our parts.

Does this imply the earlier work was a failure? Completely not. The staff left behind an exquisite trove of design paperwork, discussions, and code that we made positive to review and steal from. Specifically, there was an ADR that mentioned completely different approaches which we studied, in addition to a working C++/WebIDL code that we repurposed for our challenge.

Calling the FFI features

UniFFI bindings reside on high of an FFI layer utilizing the C ABI that we name “the scaffolding.” Then the consumer API is outlined on high of the scaffolding layer, within the international language. This permits the consumer API to help options circuitously expressible in C and in addition permits the generated API to really feel idiomatic and pure. Nonetheless, JavaScript complicates this image as a result of it doesn’t have help for calling C features. Privileged code in Firefox can use the Mozilla js-ctypes library, however its use is deprecated.

The earlier challenge solved this downside through the use of C++ to name into the scaffolding features, then leveraged the Firefox WebIDL code generation tools to create the JavaScript API. That code technology software is sort of good and allowed us to outline the consumer API utilizing a mix of WebIDL and C++ glue code. Nonetheless, it was restricted and didn’t help all UniFFI options.

Our staff determined to make use of the identical WebIDL code technology software, however to generate simply the scaffolding layer as a substitute of the complete consumer API. Then we used JavaScript to outline the consumer API on high of that, similar to for different languages. We have been pretty assured that the code technology software would now not be a limiting issue, for the reason that scaffolding layer is designed to be minimalistic and expressible in C.

Async features

The threading mannequin for UniFFI interfaces just isn’t very versatile: all operate and technique calls are blocking. It’s the caller’s duty to make sure that calls don’t block the unsuitable thread. Sometimes this implies executing UniFFI calls in a thread pool.

The threading mannequin for Firefox frontend JavaScript code is equally rigid: you should by no means block the primary thread. The primary JavaScript thread is accountable for all UI updates and blocking it means an unresponsive browser. Moreover, the one technique to begin one other thread in JavaScript is utilizing Web Workers, however these usually are not at the moment utilized by the frontend code.

To resolve the unstoppable drive vs. immovable object state of affairs we discovered ourselves in, we merely reversed the UniFFI mannequin and made all calls asynchronous. Which means all features return a promise fairly than their return worth immediately.

See Also

The “all features are async” mannequin appears cheap, at the very least for the primary few tasks we intend to make use of with UniFFI. Nonetheless, not all features actually must be async – some are fast sufficient that they aren’t blocking. Finally, we plan so as to add a approach for customers to customise which features are blocking and that are async. This can in all probability occur alongside some basic work for async UniFFI, since we’ve discovered that async execution is a matter for a lot of parts utilizing UniFFI.

How has it been working?

Since touchdown UniFFI help in Firefox 105, we’ve slowly began including some UniFFI’ed Rust parts to Firefox. In Firefox 108 we added the Rust distant tabs syncing engine, making it the primary part shared by Firefox on all three of our platforms. The brand new tabs engine makes use of UniFFI to generate JS bindings on Desktop, Kotlin bindings on Android, and Swift bindings on iOS.

We’ve additionally been persevering with to advance our shared part technique on Cellular. Firefox iOS has traditionally lagged behind Android by way of shared part adoption, however the Firefox iOS 116 launch will use our shared sync supervisor part. Which means each cellular browsers can be utilizing all the shared parts we’ve written to this point.

We additionally use UniFFI to generate bindings for Glean, a Mozilla telemetry library, which was a little bit of an uncommon case. Glean doesn’t generate JS bindings; it solely generates the scaffolding API, which results in the GeckoView library that powers Firefox Android. Firefox Android can then eat Glean through the generated Kotlin bindings which hyperlink to the scaffolding in Geckoview.

In case you’re on this challenge or UniFFI normally, please be a part of us in #uniffi on the Mozilla Matrix chat.

More articles by Ben Dean-Kawamura…

Source Link

What's Your Reaction?
Excited
0
Happy
0
In Love
0
Not Sure
0
Silly
0
View Comments (0)

Leave a Reply

Your email address will not be published.

2022 Blinking Robots.
WordPress by Doejo

Scroll To Top