C++ and Julia 1.9 Integration

On this tutorial we exhibit name Julia libraries with a number of threads from C++. With the introduction of Julia 1.9 in Might 2023, the runtime can dynamically “undertake” exterior threads, enabling the mixing of Julia libraries into multi-threaded codebases written in different languages, equivalent to C++. This text is written in collaboration with Evangelos Paradas, the maestro of algorithm deployment at ASML. Evangelos has been answerable for closely testing and debugging this multi-threading characteristic. I humbly repeated the ultimate outcomes after his many trial-and-error makes an attempt and summarized all the pieces for you on this article.

Julia in manufacturing

Julia is a general-purpose language designed for scientific and numerical computing, putting a stability between pace and ease. The adoption of Julia within the business is rising yearly, however advanced circumstances require enhanced deployment capabilities within the core of the language. One such essential enchancment we would have liked was the flexibility to name Julia libraries with a number of threads from one other language. Luckily, that is now attainable in Julia model 1.9. Since we’ve got been concerned in testing this new characteristic extensively, we want to share this tutorial with you to speed up your journey with exterior threads in Julia.

Weaving threads throughout a number of programming languages is an excessive sport in software program engineering. You achieve this at your personal threat. Incorrect utilization of this expertise will crash your manufacturing techniques. You could have been duly warned.

Earlier than beginning, ensuring you might be working with Julia 1.9, both by utilizing juliaup or downloading Julia 1.9 manually and including it to your path.

Introduction to C++ embedding

Up to now, I’ve spent fairly a while writing a tutorial about how to embed Julia libraries into C++. It is not trivial. Excessive degree the steps concerned are:

• Create a Julia package deal with the Julia c-interface capabilities

• Write the C++ code that may name these Julia capabilities

• Compile the Julia code to a library with PackageCompiler.jl

• Compile C++ and hyperlink it to the Julia library

I will not delve into all of the specifics above, so in case you want to reproduce the outcomes of this text, it is advisable to first learn my earlier article. Previous to embarking on a multi-threaded journey, just remember to are intimately aware of embedding in a single-threaded method. Having a number of C++ threads name into Julia is an exceptionally superior topic, notably when you have restricted prior expertise with C++ and multi-threading. Take your time to be taught the ropes.

Julia code instance

We wrote a quite simple Julia perform that throws an error relying on the enter worth. The precise Julia performance does not matter on this article. As talked about, you may learn the in depth blog post on my previous blog for particulars, however listed below are the vital highlights for making a Julia perform prepared for C/C++ embedding:

• Use Base.@ccallable to verify the Julia perform will be referred to as from C/C++

• Use C sorts on the interface. On this instance I solely use Cint sorts. Observe that Cint is an alias for Int32, so Julia integers and C integers even have the identical reminiscence structure.

Base.@ccallable perform divide_function(enter::Cint)::Cint
    if enter > 10
        throw(ErrorException("You can not divide by greater than 10"))
    finish
    outputValue::Cint = div(12, enter)
    return outputValue
finish

Once you place this perform inside a Julia package deal, you may compile it to a library with PackageCompiler. An instance construct script will be present in my github repository that accompanies this text.

Initializing Julia

Listed below are a number of the interfaces which might be vital for initializing the Julia library within the appropriate method for accepting/adopting exterior threads from C++. We requested advise to make use of many of those capabilities, as we’re not consultants on this both. The C API of the Julia runtime (these jl_* capabilities) may undoubtedly use some extra documentation.

• The init_julia perform comes from a header file that’s created collectively together with your compiled Julia library. Nothing particular right here.

• The code with jl_is_initialized has to enter a strive/catch block as a result of when Julia just isn’t initialized this variable just isn’t accessible within the reminiscence and returns a segfault. A stunning gotcha.

• Make certain to lock and unlock the initialization of Julia, in order that no different thread can by accident attempt to begin Julia as nicely, whereas this thread is busy initializing Julia.

• jl_adopt_thread permits this C++ thread for use by Julia. That is an important C API perform to recollect for exterior multi-threading. It’s available since Julia 1.9.

• the job of jl_gc_safe_enter is to mark the thread as secure, in order that the rubbish collector (GC) can run concurrently to that thread. By utilizing this perform, you make a promise to not do any GC seen work, equivalent to allocating new reminiscence. The usage of parentheses around the function is solely to keep away from confusion with a function-like macro.

• jl_enter_threaded_region units Julia to multi-threading mode, I imagine. This perform can be used for instance by the Julia @threads macro, however lacks any documentation.

In response to the link with news about thread adoption says that @ccallable Julia perform will robotically undertake threads. That is true, however what in case you execute a Julia perform or macro earlier than the @ccallable perform? In that case you get a segmentation fault, as a result of this thread just isn’t but adopted. For instance, while you need to seize Julia errors, you’ll want to name the JL_TRY macro earlier than the @ccallable. Within the subsequent part, we’ll present use such macros inside a multithread surroundings. On this initialization part, we present the most secure method is to carry out the thread adoption by calling jl_adopt_thread explicitly.

All collectively we use these capabilities to initialize the Julia compiled library as follows. I’ve saved the code instance concise to spotlight what issues.

#embody "julia_init.h"

bool is_julia_initialized()
{
    strive
    {
        return jl_is_initialized() != 0;
    }
    catch (...)
    {
        return false;
    }
}

void initialize_julia(int argc, char *argv[])
{
    mtx.lock();

    if (!is_julia_initialized())
    {
        init_julia(argc, argv);
        jl_adopt_thread();
        (jl_gc_safe_enter)();
        jl_enter_threaded_region();
    }

    mtx.unlock();
}

The principle C++ code

Let’s write a easy wrapper round our beautiful c-callable Julia perform and present you catch any errors thrown by Julia. All in a multi-threaded method. Keep in mind, the Julia perform divide_function is a trivial perform that makes use of integers and throws an exception when the enter integer is bigger than 10.

We use jl_get_pgcstack to examine if a thread is already adopted by Julia. In case you try to undertake a thread twice, you’ll encounter a segmentation fault. That is one method to keep away from making that mistake by accident.

The JL_TRY macro will examine if an error occurred within the adopted thread. This macro solely works if the thread is definitely adopted, else you get one more segmentation fault. Contained in the macro we name the perform from the Julia library.

If you wish to retrieve the precise Julia error contained in the JL_CATCH, you’ll need to name into the Julia runtime. I’ve some instance code in a earlier article about catching Julia exceptions from C++ on my private weblog. Within the instance right here, we saved it easy and simply printed a message.

void call_and_catch(int x)
{
    
    if (jl_get_pgcstack() == NULL)
        jl_adopt_thread();    

    
    JL_TRY
    {
        divide_function(x); 
        std::cout << "Succeeded for x = " << x << std::endl;
    }
    JL_CATCH
    {
        std::cout << "Caught error for x = " << x << std::endl;
    }
}

We are able to now write a bit of multi-threaded C++ code and name our Julia perform. The simplest method is to first create a pool of threads. If you wish to make this instance extra sophisticated, you will need to be taught a bit extra about C++, which is past the scope of this text. However this can be a good instance to get you began.

int principal()
{
    const size_t n_of_threads(15);
    initialize_julia();

    
    std::thread all_threads[n_of_threads];
    for(int i=0; i<n_of_threads; i++)
        all_threads[i] = std::thread(call_and_catch, i+1);

    
    for(auto& thread : all_threads)
        thread.be part of();

    return 0;
}

Compiling

Make certain so as to add the -lpthread flag, this can be a system library that’s required for C++ threads. I’ve already added this flag to the MakeFile in my repository. Apart from that, compilation is an identical to regular Julia embedding in C++.

After compiling with the makefile, I can run the generated executable, and we see 15 printed messages, as anticipated. They seem in considerably random order, as a result of nature of multi-threading, however the erroring threads seem final, in all probability as a result of the error dealing with takes extra time.

In case you ever handle to reach at this identical level, please congratulate your self! That is difficult enterprise.

Succeeded for x = 2 
Succeeded for x = 1
Succeeded for x = 4
Succeeded for x = 3
Succeeded for x = 5
Succeeded for x = 8
Succeeded for x = 10
Succeeded for x = 9
Succeeded for x = 7
Succeeded for x = 6
Caught error for x = 15
Caught error for x = 12
Caught error for x = 13
Caught error for x = 14
Caught error for x = 11

Pitfalls to keep away from

Usually multi-threading requires a whole lot of consideration because of many attainable pitfalls, equivalent to thread-safety points, deadlocks, race situations and rather more. Including exterior multi-threading to the combination makes all the pieces much more sophisticated. Contemplate fastidiously whether or not you actually need to go down this route with a number of languages. If you wish to proceed, this is just a few complexities we encountered alongside the way in which, however bear in mind that you could be discover many extra.

We encountered some points with BLAS and different libraries. It is best to set the variety of threads to at least one by way of LinearAlgebra.BLAS.set_num_threads(1), else each thread in Julia spawns a number of threads within the BLAS library. Identical for MKL and some other third get together library you utilize. Issues may fit tremendous, however your efficiency may not be optimum. You in all probability don’t desire your 4 exterior C++ threads by accident spawning 16 BLAS threads or extra.

Usually, make sure to take a look at each binary artifact you need to use in manufacturing and take into account the implications in your multi-threading setup. That is good recommendation for any software program improvement undertaking you undertake, unbiased of Julia.

We encountered a pitfall with Java, when embedding our library into Spark. On this article, we won’t go into the main points of passing Java threads (by way of C++) to Julia, however we seen some points with the Java sign handler. Guarantee that your library is explicitly conscious of the Java sign dealing with library, for instance by way of export LD_PRELOAD=/path/to/libjsig.so . In any other case Julia will produce a segmentation fault and your utility will crash. That is some form of language interoperability problem that we needed to circumvent.

Massive lesson discovered from the above: by no means ever disable the Julia sign handler, as a result of else Java is simply dealing with the indicators. These indicators are working system indicators, equivalent to segfaults or sigabort or the well-known sigkill (while you hit ctrl+c to kill one thing). If Julia can not deal with these indicators, you’ve got obtained a major problem. We made this error whereas determining the earlier pitfall.

Integrating C++ and Julia with a number of threads generally is a advanced process, nevertheless it presents highly effective capabilities for incorporating Julia libraries into multi-threaded C++ codebases. By fastidiously initializing the Julia runtime and dealing with potential pitfalls, builders can efficiently mix these two languages for improved efficiency and performance. Nevertheless, it is essential to be conscious of sophisticated multi-threading challenges to make sure the reliability of the ultimate product.