The bane of my existence: Supporting each async and sync code in Rust

2024-01-19 16:00:01

The third attempt relies on
a crate referred to as maybe_async
. I bear in mind foolishly pondering it was
the right resolution again once I found it.

Anyway, the concept is that with this crate you’ll be able to robotically take away the
async and .await occurrences in your code with a procedural macro,
basically automating the copy-pasting strategy. For instance:

#[maybe_async::maybe_async]
async fn endpoint() { /* stuff */ }

Generates the next code:

#[cfg(not(feature = "is_sync"))]
async fn endpoint() { /* stuff */ }

#[cfg(feature = "is_sync")]
fn endpoint() { /* stuff with `.await` eliminated */ }

You’ll be able to configure whether or not you need asynchronous or blocking code by toggling the
maybe_async/is_sync characteristic when compiling the crate. The macro works for
features, traits and impl blocks. If one conversion isn’t as straightforward as eradicating
async and .await, you’ll be able to specify customized implementations with the
async_impl and sync_impl procedural macros. It does this splendidly, and
we’ve already been utilizing it for Rspotify for some time now.

In truth, it labored so effectively that I made Rspotify http-client agnostic, which is
much more versatile than being async/sync agnostic. This permits us to help
a number of HTTP purchasers like reqwest
and ureq
,
independently of whether or not the consumer is asynchronous or synchronous.

Being http-client agnostic isn’t that onerous to implement in case you have
maybe_async round. You simply have to outline a trait for the
HTTP
client, after which implement it for every of the purchasers you need to help:

#[maybe_async]
trait HttpClient {
    async fn get(&self) -> String;
}

#[sync_impl]
impl HttpClient for UreqClient {
    fn get(&self) -> String { ureq::get(/* ... */) }
}

#[async_impl]
impl HttpClient for ReqwestClient {
    async fn get(&self) -> String { reqwest::get(/* ... */).await }
}

struct SpotifyClient<Http: HttpClient> {
    http: Http
}

#[maybe_async]
impl<Http: HttpClient> SpotifyClient<Http> {
    async fn endpoint(&self) { self.http.get(/* ... */) }
}

Then, we may prolong it in order that whichever consumer they need to use might be
enabled with characteristic flags of their Cargo.toml. For instance, if client-ureq
is enabled, since ureq is synchronous, it will allow maybe_async/is_sync.
In flip, this could take away the async/.await and the #[async_impl] blocks,
and the Rspotify consumer would use ureq‘s implementation internally.

This resolution has not one of the downsides I listed in earlier makes an attempt:

No code duplication in any respect
No overhead neither at runtime nor at compile time. If the consumer needs a
blocking consumer, they will use ureq, which doesn’t pull tokio and mates
Fairly straightforward to grasp for the consumer; simply configure a flag in you
Cargo.toml

Nonetheless, cease studying for a few minutes and check out to determine why you
shouldn’t do that. In truth, I’ll offer you 9 months, which is how lengthy it took me
to take action…

The issue

Properly, the factor is that options in Rust have to be additive: “enabling a
characteristic shouldn’t disable performance, and it ought to often be secure to
allow any mixture of options”. Cargo might merge options of a crate when
it’s duplicated within the dependency tree as a way to keep away from compiling the identical
crate a number of occasions.
The
reference explains this quite well, if you want more details.

This optimization implies that mutually unique options might break a dependency
tree. In our case, maybe_async/is_sync is a toggle characteristic enabled by
client-ureq. So if you happen to attempt to compile it with client-reqwest additionally enabled,
it can fail as a result of maybe_async can be configured to generate synchronous
operate signatures as an alternative. It’s not possible to have a crate that will depend on
each sync and async Rspotify both straight or not directly, and the entire
idea of maybe_async is at present unsuitable in response to the Cargo reference.

The characteristic resolver v2

A standard false impression is that that is fastened by the “characteristic resolver v2”,
which
the
reference also explains quite well. It has been enabled by default for the reason that
2021 version, however you’ll be able to specify it inside your Cargo.toml in earlier ones.
This new model, amongst different issues, avoids unifying options in some particular
circumstances, however not in ours:

Options enabled on platform-specific dependencies for targets not at present
being constructed are ignored.

Construct-dependencies and proc-macros don’t share options with regular
dependencies.

Dev-dependencies don’t activate options except constructing a goal that wants
them (like assessments or examples).

Simply in case, I attempted to breed this myself, and it did work as I anticipated.
This repository is an
instance of conflicting options, which breaks with any characteristic resolver.

Different fails

There have been a number of crates that additionally had this downside:

arangors
and aragog
: wrappers for ArangoDB. Each
use maybe_async to modify between async and sync (arangors‘s creator is
the identical particular person, actually) [5] [6].
inkwell
: a wrapper for LLVM. It helps a number of variations of
LLVM, which aren’t suitable with eachother [7].
k8s-openapi
: a wrapper for Kubernetes, with the identical difficulty as
inkwell [8].

Fixing `maybe_async`

As soon as the crate began to achieve reputation, this difficulty was opened in
maybe_async, which explains the scenario and showcases a repair:

async and sync in the same program
fMeow/maybe-async-rs#6

maybe_async would now have two characteristic flags: is_sync and is_async. The
crate would generate the features in the identical method, however with a _sync or
_async suffix appended to the identifier in order that they wouldn’t be conflicting.
For instance:

#[maybe_async::maybe_async]
async fn endpoint() { /* stuff */ }

Would now generate the next code:

#[cfg(feature = "is_async")]
async fn endpoint_async() { /* stuff */ }

#[cfg(feature = "is_sync")]
fn endpoint_sync() { /* stuff with `.await` eliminated */ }

Nonetheless, these suffixes introduce noise, so I puzzled if it will be potential
to do it in a extra ergonomic method. I forked maybe_async and gave it a attempt,
about which you’ll be able to learn extra
in this
series of comments. In abstract, it was too difficult, and I finally gave
up.

The one technique to repair this edge case could be to worsen the usability of Rspotify
for everybody. However I’d argue that somebody who will depend on each async and sync is
unlikely; we haven’t really had anybody complaining but. Not like reqwest,
rspotify is a “excessive stage” library, so it’s arduous to think about a situation the place
it seems greater than as soon as in a dependency tree within the first place.

Maybe we may ask the Cargo devs for assist?

Official Help

Rspotify is much from being the primary who has been by way of this downside, so it
is perhaps attention-grabbing to learn earlier discussions about it:

This now-closed RFC for the Rust
compiler steered including the oneof configuration predicate (suppose
#[cfg(any(…))] and similars) to help unique options. This solely
makes it simpler to have conflicting options for circumstances the place there’s no
alternative, however options ought to nonetheless be strictly additive.
The earlier RFC began
some
discussion within the context of permitting unique options in Cargo itself, and
though it has some attention-grabbing data, it didn’t go too far.
This issue in Cargo explains a
comparable case with the Home windows API. The dialogue contains extra examples and
resolution concepts, however none have made it to Cargo but.
Another issue in Cargo asks
for a technique to check and construct with mixtures of flags simply. If options are
strictly additive, then cargo check --all-features will cowl the whole lot. However
in case it doesn’t, the consumer has to run the command with a number of mixtures
of characteristic flags, which is kind of cumbersome. That is already potential
unofficially due to cargo-hack.
A very completely different strategy
based
on the Keyword Generics Initiative. It appears to be the latest tackle
fixing this, however it’s in an “exploration” part, and
no
RFCs are available as of this writing.

In response to
this old
comment, it’s not one thing the Rust staff has already discarded; it’s nonetheless
being mentioned.

Though unofficial, one other attention-grabbing strategy that could possibly be explored additional
in Rust is “Sans I/O”. It is a Python
protocol that abstracts away using community protocols like HTTP in our case,
thus maximizing reusability. An current instance in Rust could be
tame-oidc.

Source Link