What are Senders Good For, Anyway? – Eric Niebler
Some colleagues of mine and I’ve been working for the previous few years to present C++ an ordinary async programming mannequin. That effort has resulted in a proposal, P2300, that has been design-approved for C++26. If you already know me or in the event you observe me on social media, you already know I’m embarrassingly excited about this work and its potential affect. I’m conscious, nevertheless, that not everyone shares my enthusiasm. I hear this stuff loads nowadays:
Why would I wish to use it?
Why do we want senders when C++ has coroutines?
It’s all simply too sophisticated!
The P2300 crew have collectively performed a horrible job of constructing this work accessible. On the coronary heart of P2300 is a straightforward, elegant (IMHO) core that brings many advantages, however it’s onerous to see that forest for all of the bushes.
So let’s make this concrete. On this put up, I’ll present tips on how to deliver a crusty previous C-style async API into the world of senders, and why you would possibly wish to try this.
In a previous life I did numerous Win32 programming. Win32 has a number of async fashions to select from, however the easiest is the great ol’ callback. Async IO APIs like ReadFileEx are formed like this:
/// Outdated-style async C API with a callback
/// (like Win32's ReadFileEx)
struct overlapped {
// ...OS internals right here...
};
utilizing overlapped_callback =
void(int standing, int bytes, overlapped* person);
int read_file(FILE*, char* buffer, int bytes,
overlapped* person, overlapped_callback* cb);
The protocol is fairly easy: you name read_file passing within the ordinary arguments plus two further: a pointer to an “overlapped” construction and a callback. The OS will use the overlapped struct for its personal functions, however the person can stuff knowledge there too that the callback can later use. It appears to be like like this:
struct my_overlapped : overlapped {
// ... my further knowledge goes right here ...
};
void my_callback(int standing, int bytes, overlapped* knowledge) {
auto* my_data = static_cast<my_overlapped*>(knowledge);
// ...use the additional stuff we put within the `my_overlapped`
// object...
delete my_data; // clear up
}
void enqueue_read(FILE* pfile) {
// Allocate and initialize my_data...
auto* my_data =
new my_overlapped{{}, /* my knowledge goes right here */ };
int standing =
read_file(pfile, buff, bytes, my_data, my_callback);
// ...
}
What occurs is that this: read_file causes the OS to enqueue an IO operation, saving the overlapped and overlapped_callback pointers with it. When the IO completes, the OS invokes the callback, passing within the pointer to the overlapped struct. I’ve written code like this a whole bunch of instances. You most likely have too.
It’s easy. It really works. Why make this extra sophisticated, proper?
There’s nothing incorrect with the callback API. What’s incorrect is that each library that exposes asynchrony makes use of a barely totally different callback API. If you wish to chain two async operations from two totally different libraries, you’re going to want to put in writing a bunch of glue code to map this async abstraction to that async abstraction. It’s the Tower of Babel downside.
Pieter Brueghel the Elder, Public area, by way of Wikimedia Commons“Look, they’re one folks, and so they have all one language, and that is solely the start of what they may do; nothing that they suggest to do will now be unattainable for them. Come, allow us to go down and confuse their language there, in order that they won’t perceive each other’s speech.”
— Genesis 11:6–7
So, when there are too many incompatible methods to do a factor, what can we do? We make one other in fact.
The way in which to flee this entice is for the C++ commonplace to endorse one async abstraction. Then all of the libraries that expose asynchrony can map their abstractions to the usual one, and we’ll all have the ability to speak to one another. Babel solved.
So that’s the reason the C++ Standardization Committee is on this downside. Virtually talking, it’s an issue that may solely be solved by the usual.
Which brings us to senders, the subject of P2300. There’s loads I may say about them (they’re environment friendly! they’re structured! they compose!), however as a substitute I’m going to point out some code and let the code do the speaking.
If we have a look at the read_file API above, we will establish some totally different components:
- The allocation of any assets the async operation wants,
- The knowledge that should stay at a secure handle at some stage in the operation (i.e., the
overlappedconstruction), - The initiation of the async operation that enqueues the async IO, and the dealing with of any initiation failure,
- The user-provided continuation that’s executed after the async operation completes (i.e., the callback).
- The reclamation of any assets allotted in step 1.
Senders have all these items too, however in a uniform form that makes it doable to work with them generically. In actual fact, the one significant distinction between senders and the C-style API is that as a substitute of 1 callback, in senders there are three: one every for fulfillment, failure, and cancellation.
Step 1: The Allocation
Our re-imagined read_file API will seem like this:
read_file_sender
async_read_file(FILE* file, char* buffer, int measurement)
{
return {file, buffer, measurement};
}
The one job of this operate is to place the arguments right into a sender-shaped object, which appears to be like as follows (clarification after the break)[*]:
[*]: Or slightly, it will seem like this after P2855 is accepted.
namespace stdex = std::execution;
struct read_file_sender
{
utilizing sender_concept = stdex::sender_t; // (1)
utilizing completion_signatures = // (2)
stdex::completion_signatures<
stdex::set_value_t( int, char* ),
stdex::set_error_t( int ) >;
auto join( stdex::receiver auto rcvr ) // (3)
{
return read_file_operation{{}, {}, pfile, buffer,
measurement, std::transfer(rcvr)};
}
FILE* pfile;
char* buffer;
int measurement;
};
The job of a sender is to explain the asynchronous operation. (It is usually a manufacturing unit for the operation state, however that’s step 2.) On the road marked “(1)”, we declare this kind to be a sender. On the road marked “(2)”, we declare the methods wherein this asynchronous operation can full. We do that utilizing an inventory of operate varieties. This:
stdex::set_value_t( int, char* )
… declares that this async operation might full efficiently by passing an int and a char* to the worth callback. (Keep in mind, there are three callbacks.) And this:
stdex::set_error_t( int )
… declares that this async operation might full in error by passing an int to the error callback. (If this async operation have been cancelable, it will declare that with stdex::set_stopped_t().)
Step 2: The Knowledge
On the road marked “(3)” above, the join member operate accepts a “receiver” and returns an “operation state”. A receiver is an amalgamation of three callbacks: worth, error, and stopped (canceled, roughly). The results of connecting a sender and a receiver is an operation state. The operation state, just like the overlapped struct within the C API, is the info for async operation. It should stay at a secure handle for the length.
The join operate returns a read_file_operation object. The caller of join assumes accountability for guaranteeing that this object stays alive and doesn’t transfer till one of many callbacks is executed. The read_file_operation sort appears to be like like this (clarification after the break):
struct immovable {
immovable() = default;
immovable(immovable&&) = delete;
};
template <class Receiver>
struct read_file_operation : overlapped, immovable // (1)
{
static void _callback(int standing, int bytes, // (2)
overlapped* knowledge)
{
auto* op =
static_cast<read_file_operation*>(knowledge); // (3)
if (standing == OK)
stdex::set_value(std::transfer(op->rcvr), // (4)
bytes, op->buffer);
else
stdex::set_error(std::transfer(op->rcvr),
standing);
}
void begin() noexcept // (5)
{
int standing =
read_file(pfile, buffer, measurement, this, &_callback);
if (standing != OK)
stdex::set_error(std::transfer(rcvr), standing);
}
FILE* pfile;
char* buffer;
int measurement;
Receiver rcvr;
};
The operation state shops the arguments wanted to provoke the async operation in addition to the receiver (the three callbacks). Let’s break this down by line.
- “(1)”: The operation state inherits from
overlappedso we will cross a pointer to it intoread_file. It additionally inherits from animmovablestruct. Though not strictly vital, this ensures we don’t transfer the operation state by chance. - “(2)”: We outline the
overlapped_callbackthat we’ll cross toread_fileas a categorystaticoperate. - “(3)”: Within the callback, we down-cast the
overlappedpointer again right into a pointer to theread_file_operationobject. - “(4)”: Within the callback, we verify the
standingto see if the operation accomplished efficiently or not, and we nameset_valueorset_erroron the receiver as applicable. - “(5)”: Within the
begin()operate — which all operation states should have — we really provoke the learn operation. If the initiation fails, we cross the error to the receiver instantly because the callback won’t ever execute.
Step 3: The Initiation
You’ll discover that once we name the sender-ified async_read_file operate, we’re simply developing a sender. No precise work is began. Then we name join with a receiver and get again an operation state, however nonetheless no work has been began. We’ve simply been lining up our geese, ensuring every little thing is in place at a secure handle so we will provoke the work. Work isn’t initiated till .begin() is named on the operation state. Solely then can we make a name to the C-style read_file API, thereby enqueuing an IO operation.
All this hoop-jumping turns into vital as soon as we begin constructing pipelines and process graphs of senders. Separating the launch of the work from the development of the operation state lets us mixture heaps of operation states into one which accommodates all the info wanted by your complete process graph, swiveling every little thing into place earlier than any work will get began. Meaning we will launch numerous async work with advanced dependencies with solely a single dynamic allocation or, in some instances, no allocations in any respect.
Now I’ve to fess up. I fibbed a little bit after I stated that the caller of join must hold the operation state alive at a secure handle till one of many callbacks is executed. That solely turns into true as soon as .begin() has been known as on it. It’s completely acceptable to attach a sender to a receiver after which drop the operation state on the ground so long as .begin() hasn’t been known as but. However with .begin() you’re dedicated. .begin() launches the rockets. There’s no calling them again.
OK, we’ve constructed the operation state, and we’ve known as .begin() on it. Now the proverbial ball is within the working system’s courtroom.
Step 4: The Continuation
The working system does its IO magic. Time passes. When the IO operation is completed, it should invoke the _callback operate with a standing code, a pointer to the overlapped struct (our read_file_operation) and, if profitable, the variety of bytes learn. The _callback passes the completion data to the receiver that was join-ed to the sender, and the circle is full.
However wait, what about “Step 5: Deallocation”? We by no means actually allotted something within the first place! The join operate returned the operation state by worth. It’s as much as the caller of join, whoever that’s, to maintain it alive. They could try this by placing it on the heap, wherein case they’re liable for cleansing it up. Or, if this async operation is a part of a process graph, they could try this by aggregating the operation state into a bigger one.
Step 6: Revenue!
At this level you might be questioning what’s the purpose to all of this. Senders and receivers, operation states with fiddly lifetime necessities, join, begin, three totally different callbacks — who desires to handle all of this? The C API was approach easier. It’s true! So why am I so unreasonably enthusiastic about all of this?
The caller of
async_read_filedoesn’t have to care about any of that.
The top person, the caller of async_read_file, will not be going to be mucking about with receivers and operation states. They’ll be awaiting senders in coroutines. Look! The next code makes use of a coroutine process sort from the stdexec library.
exec::process< std::string > process_file( FILE* pfile )
{
std::string str;
str.resize( MAX_BUFFER );
auto [bytes, buff] =
co_await async_read_file(pfile, str.knowledge(), str.measurement());
str.resize( bytes );
co_return str;
}
What wizardry is that this? We wrote a sender, not an awaitable, proper? However that is working code! See for your self: https://godbolt.org/z/Kj1dPnerx.
That is the place we get to reap the advantages of programming to an ordinary async mannequin. Generic code, whether or not from the usual library or from third social gathering libraries, will work with our senders. Within the case above, the process<> sort from the stdexec library is aware of tips on how to await something that appears like a sender. When you’ve got a sender, you possibly can co_await it with out doing any further work.
P2300 comes with a small assortment of generic async algorithms for widespread async patterns — issues like chaining (then), dynamically choosing the following process (let_value), grouping senders into one (when_all), and blocking till a sender is full (sync_wait). It’s a paltry set to make sure, however it should develop with future requirements. And as third social gathering libraries start to undertake this mannequin, increasingly more async code will work collectively. Some day very quickly you’ll have the ability to provoke some file IO, learn from the community, wait on a timer, pay attention for a cancellation from the person, anticipate all of them to finish, after which switch execution to a thread pool to do extra work — whew! — even when every sender and the thread pool got here from totally different libraries.
Senders, FTW!
So, again to these pernicious questions people hold asking me.
Why would I wish to use it?
You wish to use senders as a result of then you possibly can sew your async operations along with different operations from different libraries utilizing generic algorithms from nonetheless different libraries. And so you possibly can co_await your async operations in coroutines with out having to put in writing an extra line of code.
Why do we want senders when C++ has coroutines?
I hope you understand by now that this isn’t an both/or. Senders are a part of the coroutine story. In case your library exposes asynchrony, then returning a sender is a superb alternative: your customers can await the sender in a coroutine in the event that they like, or they’ll keep away from the coroutine body allocation and use the sender with a generic algorithm like then() or when_all(). The dearth of allocations makes senders an particularly sensible choice for embedded builders.
It’s all simply too sophisticated!
I’ll grant that implementing senders is extra concerned than utilizing odd C-style callbacks. However consuming senders is as simple as typing co_await, or so simple as passing one to an algorithm like sync_wait(). Opting in to senders is opting into an ecosystem of reusable code that may develop over time.
THAT is why I’m enthusiastic about senders.
(And be sincere, was wrapping read_file in a sender actually all that arduous, in any case?)