Separation of Issues in Cross-Compilation

Managing advanced C++ tasks throughout a number of platforms usually finally ends up being a
irritating and time-consuming job.
Nevertheless, this frequent problem confronted additionally by probably the most skilled software program
builders doesn’t must be an inevitable wrestle.
Think about a world the place cross-compilation is not only possible, but additionally environment friendly
and fewer cumbersome.
I’ve seen in a number of previous pleasant discussions with different engineers that
many are in no way acquainted with the kind of Separation of Issues that Nix
didn’t invent but additionally makes use of.

The sooner article C++ with Nix in 2023, Part 2: Package Generation and
Cross-Compilation
concentrated extra on the creation of a brand new package deal from scratch and explaining
mkDerivation and so on., whereas on this article, we’re going to look from a bit of
bit larger stage at how cross-compilation is dealt with so elegantly in Nix, that
it may well enhance general undertaking well being and growth tempo and on the similar time
scale back prices.

What makes Cross-Compilation Exhausting?

Cross-compilation ought to generally not be exhausting:
We simply want a compiler whose backend generates code for the goal structure
and let it hyperlink the compiled code towards libraries which might be additionally compiled for
that focus on structure.

So why does it transform exhausting each time it’s being achieved for large real-life
tasks?

It sometimes goes downhill in steps which have much less to do with cross-compilation
itself however extra with managing dependencies:

• Product:
  We assume a undertaking that’s already large and has an advanced construct system
• Cross-compiler:
  We’d like one
• Exterior libraries:
  We’d like cross-compiled variants of them
• Distribution:
  If we use shared libraries, we might want to present a method to package deal these
  together with the app.
  Alternatively, we use static linking to get one large binary that’s simpler to
  distribute.

The steps are then:

1. The cross-compiler is now obtained in certainly one of two methods:
   1. Somebody creates a compiler bootstrap script that goals to be Linux-distro
      unbiased. This script will go right into a
      Dockerfile or proper
      into the construct system.
   2. A cross-compiler package deal is used. As this ties the undertaking to a sure
      package deal distribution, we are actually compelled to this distro or use Docker.
2. The package deal supervisor doesn’t allow us to set up native and foreign-target
   libraries on the identical system. And builders additionally use totally different package deal
   managers. So we sometimes find yourself constructing exterior libraries for the goal
   platform ourselves:
   1. These go into the identical Docker picture.
   2. Or the undertaking construct system additionally builds them for us.
3. As we don’t have package deal distribution infrastructure, we sometimes go the
   static constructing route and find yourself additionally fiddling static linkage into our
   undertaking construct system.

The result’s then certainly one of these two:

1. An enormous, difficult, monolithic construct system that manages not solely the construct
   of our precise undertaking, but additionally the construct of the compiler and all of the
   libraries.
   • It’s painful to arrange and the code takes perpetually to compile.
   • Solely senior builders are allowed to the touch the delicate components.
   • Sophisticated upgrading.
2. A Docker picture that properly abstracts the components away which might be totally different from
   regular non-cross-compilation.

Variant B appears to be the cleanest, however a clear execution of it appears to be
uncommon within the business.
A minimum of in my expertise, growth groups find yourself creating an enormous pile of
complexity within the type of Variant A.

Particularly together with static linking, builders usually resolve that the
construct system ought to solely construct static binaries, as a result of sustaining each dynamic
and static linking on the similar time makes the construct system too advanced.

CMake and meson can typically
each be used accurately to maintain the undertaking description agnostic of the linking
technique after which merely choose the tactic with command line parameters.
Nevertheless, I’ve not seen many large industrial real-life tasks the place this was
nonetheless potential with out a lot trouble.

I usually requested myself “Why do these construct techniques do all the things fully
in another way than urged within the official documentation and tutorials of the
construct techniques?”
With my expertise of at the moment, I believe the reply is straightforward:

Separation of Issues.

Construct techniques will not be designed to handle dependencies.
Not understanding it higher, builders attempt to do it anyway.
The result’s exhausting to vary, prolong, keep, and improve.

As promised within the introduction, we are going to take a look at how Nix makes it simple
to vary this for the higher.

The Instance App

Let’s construct an instance app that is determined by OpenSSL
and Boost at run-time.
It merely reads a personality stream from normal enter and makes use of OpenSSL
to calculate the SHA256 hash.
We use the increase dependency to cease the time – the
C++ STL may have achieved that for
us, too, however then we wouldn’t have one other good dependency on an enormous exterior
lib.

This system is roughly 60 LOC quick/lengthy.
I uploaded the code to this GitHub repository:
https://github.com/tfc/cpp-cross-compilation-example

Let’s name this app minisha256sum and write a
CMakeLists.txt file for it:

cmake_minimum_required(VERSION 3.27)
project(minisha256sum)

find_package(OpenSSL REQUIRED)
find_package(Boost REQUIRED COMPONENTS chrono)

add_executable(minisha256sum src/main.cpp)
target_link_libraries(minisha256sum Boost::chrono OpenSSL::SSL)
set_property(TARGET minisha256sum PROPERTY CXX_STANDARD 20)

install(TARGETS minisha256sum DESTINATION bin)

CMake provides good standard facilities for finding external libraries.
This fashion, the construct system might stay easy (it nonetheless appears comparatively noisy
in comparison with different language ecosystems as a result of that’s how C++ construct techniques look
like).

This undertaking can now be constructed through the standard CMake dance:

$ mkdir build
$ cd build
$ cmake ..
$ cmake --build .

$ ./minisha256sum < src/main.cpp
cb8829956b86a05cd4bf374e95f4ae3928644f4b79cef82ef89529c2ef65f004 0 milliseconds
$ sha256sum < src/main.cpp
cb8829956b86a05cd4bf374e95f4ae3928644f4b79cef82ef89529c2ef65f004  -

It provides the same hash as the
sha256sum app from GNU coreutils,
which must be adequate.
The app isn’t actually optimized however that won’t be a matter for the remainder of
this text.

Packaging it with Nix

To get a pleasant nix construct and nix run workflow, we have to present just a few nix
expressions.
Let’s begin with a package deal.nix that already displays our dependency construction:

# file: package.nix
{ stdenv, lib, cmake, boost, openssl }:

stdenv.mkDerivation {
  name = "minisha256sum";

  src = lib.fileset.toSource {
    root = ./.;
    fileset = lib.fileset.unions [
      ./src
      ./CMakeLists.txt
    ];
  };

  nativeBuildInputs = [ cmake ];
  buildInputs = [ boost openssl ];
}

nativeBuildInputs means “compile-time dependency on the building host” and
buildInputs means “run-time dependency on the target”.
The nixpkgs documentation describes this in more detail.

To create a buildable, installable, and runnable package deal from this expression,
we have to apply the callPackage perform which is a
well-known pattern in the Nix sphere:
It mechanically fills out all of the perform parameters from what’s accessible in
pkgs that we will see within the first line of package deal.nix, which occur to be
our dependencies.

minisha256sum = pkgs.callPackage ./package.nix { }

In the project’s flake.nix file, we
make this call in this line.
With this in place, we will now run it with out dealing with the construct instructions
manually (I’m not hiding code right here: mkDerivation typically is aware of how one can construct
CMake tasks when CMake was talked about as a dependency).
After pushing it to a repository, we will even do that from a special pc
with out cloning the repo first:

See Also

Folklore.org: Scrooge McDuck

$ nix build github:tfc/cpp-cross-compilation-example
$ file result/bin/minisha256sum
result/bin/minisha256sum:
ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), dynamically linked,
interpreter /nix/store/7jiqcrg061xi5clniy7z5pvkc4jiaqav-glibc-2.38-27/lib/ld-linux-x86-64.so.2,
for GNU/Linux 3.10.0, not stripped

# Run directly over via `nix run` without building it first:

$ result/bin/minisha256sym < ~/some-file.zip
f7cb5ade7906f364a0d4d11478a4b9f25c86d0b3381a5b3907e2c49b31a00fee 3 milliseconds

$ nix run github:tfc/cpp-cross-compilation-example < ~/some-file.zip
f7cb5ade7906f364a0d4d11478a4b9f25c86d0b3381a5b3907e2c49b31a00fee 3 milliseconds

static = pkgs.pkgsStatic.callPackage ./package.nix { };
aarch64 = pkgs.pkgsCross.aarch64-multiplatform.callPackage ./package.nix { };
aarch64-static = pkgs.pkgsCross.aarch64-multiplatform.pkgsStatic.callPackage ./package.nix { };

There’s also pkgs.pkgsCross.mingwW64, which compiles binaries for
Microsoft Windows utilizing the
minimalist GNU environment for Windows mingw.

At any time when we use a type of specialised callPackage implementations to name
our package deal.nix perform, this occurs:

As a result of we structurally break up the dependencies between compile-time dependencies
and run-time dependencies, the cross-callPackage perform can now fill the
package deal dependencies with the proper variations of every.

This fashion, the construct system doesn’t must be educated quite a bit about what
occurs:
Nix creates the right construct setting for the given package deal variant (comparable
to the strategy with the clear Docker picture, however solely with precisely the wanted
dependencies and with much less overhead for outlining it), the construct system merely
makes use of the given compiler and locates the given dependencies through
CMake/meson-specific setting variables (which were set by Nix), and
builds the undertaking.
(It additionally works with construct system mixtures like
GNU Automake/Autoconf and
GNUMake and others)

I examined this instance with the next mixtures:

Curiously, if we “cross-compile” from the identical structure to the identical
structure on Linux, we get precisely the identical package deal like for the native
pkgs.callPackage model, so Nix doesn’t even trouble to rebuild it.

The image ✅ consists of static/dynamic linkage in all instances however not for Home windows.

The entries with the ❌ image within the desk will not be applied within the Nixpkgs
repository.
This could possibly be achieved if wanted.
Sometimes, firms both implement performance and upstream it or present
funding to make it occur.

Abstract

The demonstrated trick reveals that we successfully separated the
Dependency Administration from the Construct System.
Many builders I talked to about this have by no means thought of this
separation.
The rationale could be easy:
As a result of it’s not simple to implement with out a good know-how for dependency
administration.
I really feel like Docker has extra of a spot in deployment than in growth.

The benefits of this separation are large:

• An easier construct system that’s simple to increase even for non-seniors
• Free selection between dynamic and static linking per construct system parameter
• Vast help for compiling from/to totally different host architectures and working
  techniques
• Not managing compiler, deps, and undertaking in a single construct system makes all the things
  modular:
  • Much less work with updates
  • Quicker setup time per developer
  • Cacheable dependencies
  • Simpler reuse of particular person modules in different tasks

We solely constructed binaries this time, no
container images,
systemd-nspawn images, VMs, or disk photographs.
That is nevertheless easy so as to add on prime.

If you wish to consider Nix and even use it in your actual tasks, don’t hesitate
to give us a call!
We now have numerous expertise, particularly with low-level C++ tasks.
Contact us and see how the complexity of your tasks may be simplified.