Tremendous Colliding Nix Shops: Nix Flakes for Thousands and thousands of Builders
We’ve teamed up with Obsidian Systems and Tweag to allow Nix to merge a number of (possibly-remote) Nix shops, bringing Nix Flakes and growth surroundings portability to tens of millions of Replit customers.
Nix is an open-source cross-platform package deal supervisor and construct software that allows you to entry probably the most up-to-date and full repository of software program packages on the earth. Nix’s method to package deal constructing ensures that software program and growth environments will at all times work the identical means, regardless of the place it’s deployed.
Replit has bet big on Nix as a result of we consider in a future the place builders are free from the drudgery of establishing their growth and manufacturing environments. A world the place onboarding a co-worker is so simple as forking a Repl. Whenever you use Nix to explain your surroundings, you don’t need to redo your setup in all places you need your code to work. As Mitchell Hashimoto, founder of Hashicorp, said:
one huge profit is that after you undertake Nix, you will get a constant surroundings throughout growth (on each Linux and Mac), CI, and manufacturing.
Since early final yr, all new Repls have been powered by Nix. Quick-forward to at the moment, we’re offering almost one million software program artifacts for fast set up, all with out counting towards your Repl’s storage limits.
However that was just the start, we need to provide you with extra energy to configure Repls how you prefer to and to extend Repl portability with different platforms. We need to provide you with entry to a good bigger assortment of Nix packages: ones that had been printed years in the past, and the newest ones printed at the moment. We additionally need you to have the ability to use Nix Flakes on Replit: it shouldn’t require further work to configure growth environments on each platform. Write a Nix Flake as soon as and have the identical reproducible surroundings in all places.
To attain this, we’re going to want a technique to merge Nix shops! However first let’s dive into the main points of how we use Nix to create reproducible growth environments.
Improvement environments at Replit
Each Repl has a replit.nix file which lets you set up tens of hundreds of packages obtainable within the Nixpkgs collection. For instance, right here is the one for C++:
{ pkgs }: {
deps = [
pkgs.clang_12
pkgs.ccls
pkgs.gdb
pkgs.gnumake
];
}It installs a compiler (clang), language server (ccls), debugger (gdb), and construct software (make). When this file is modified, we add all these packages to your Repl’s shell surroundings. Behind the scenes we do one thing near what would occur in your pc if you happen to copied the next right into a default.nix file and ran nix-shell in a terminal in the identical listing.
{pkgs ? import <nixpkgs> {}}:
pkgs.mkShell {
packages = [
pkgs.clang_12
pkgs.ccls
pkgs.gdb
pkgs.gnumake
];
}Whenever you run nix-shell, Nix evaluates this code and determines that it must construct a shell surroundings as a package deal. For instance, if I run nix-build in the identical listing, I see:
$ nix-build
this derivation shall be constructed:
/nix/retailer/ni73sa8sh3jl99nz06z3af5kkp9xl1ws-nix-shell.drv
constructing '/nix/retailer/ni73sa8sh3jl99nz06z3af5kkp9xl1ws-nix-shell.drv'...
constructing
/nix/retailer/ylhzwsmznmjw8g39wvysg1rsh9ld2il7-nix-shellNix has found out what to construct and wrote an in depth reproducible construct plan within the Derivation file /nix/retailer/ni73sa8sh3jl99nz06z3af5kkp9xl1ws-nix-shell.drv into the /nix/retailer/ listing after which it adopted the construct plan producing the file /nix/retailer/ylhzwsmznmjw8g39wvysg1rsh9ld2il7-nix-shell which is a shell script that describes what surroundings variables to set to supply this growth surroundings. Right here’s a modified excerpt of that file:
declare -x PATH=
"/nix/retailer/7c4759gi42c2bhfgxixbq1hilv0m1g4i-clang-wrapper-12.0.1/bin:
/nix/retailer/k33vhsd3js5ri12prl1q309fs5l73c1p-clang-12.0.1/bin:
/nix/retailer/lyvhsvwp2pzy74fkcn7qbs5vcgy5d7vl-glibc-2.37-8-bin/bin:
/nix/retailer/ahkfdxq8mcpsb5kvdvgqr1wv8zjngbh4-coreutils-9.1/bin..."
declare -x SHELL=
"/nix/retailer/rhvbjmcfnkg8i2dxpzr114cp1ws7f667-bash-5.2-p15/bin/bash"The script units a bunch of surroundings variables to ensure your growth surroundings has precisely what it wants in it.
When it builds the shell surroundings efficiently, it additionally provides an entry to a SQLite database at /nix/var/nix/db/db.sqlite:
sqlite>
choose *
from ValidPaths
the place
path = "/nix/retailer/ylhzwsmznmjw8g39wvysg1rsh9ld2il7-nix-shell";
333859|/nix/retailer/ylhzwsmznmjw8g39wvysg1rsh9ld2il7-nix-shell|
sha256:450a0340bb5cf76d350c1c7bee48c3403c3df6076c3d66f592f31499ac4f2f5e|
1684944022|
/nix/retailer/ni73sa8sh3jl99nz06z3af5kkp9xl1ws-nix-shell.drv|7936|1||Nix data this info within the database for each file-system-consistency causes and efficiency causes.
There’s one final step for establishing Repl growth environments that we’ll get to after we be taught in regards to the overlay filesystem we use to allow instantaneous package deal set up.
How our overlay filesystem at the moment works
If we allow you to use Nix the very same means you do in your native pc, you’d use plenty of area and initially obtain loads from the Nix caches, which may take many minutes.
As a substitute, we now have ~16 TB Persistent Disks (Massive Disks) on Google Cloud that we connect to every machine that serves Repls. The Massive Disks have a Nix Retailer (a set of packages in a /nix/retailer listing, alongside the SQLite database) containing virtually one million packages. Just like the nix-shell instance above, the SQLite database has a row for every package deal.
Inside Repls, the Massive Disk is mounted in an overlay filesystem stack. The Massive Disk is a decrease disk and an higher scratch disk lets Nix builds occur contained in the Repl. Critically, this higher disk is not persistent.
Below this setup, if we endured the higher retailer, you wouldn’t see the up to date decrease retailer database the following time we added a package deal to the Massive Disk.
Which means that at the moment everytime you do an everyday Nix construct (or something with Nix Flakes) inside a Repl, the construct consequence will not be saved between periods and is proscribed to the dimensions allotted to the scratch disks.
Revisiting growth environments
Because the higher retailer will not be persistent, how can we keep away from rebuilding a Repl’s growth surroundings each session? Caching.
We save all of the surroundings variables to .cache/replit/nix/env.json together with metadata that helps us decide if the cache is stale. When a Repl begins, we search for this cache, and if it isn’t stale we use it as a substitute of rebuilding the Nix surroundings. See our earlier submit Faster Nix Repl Startup for extra particulars.
With out a persistent higher retailer, all the things we cache needs to be current on the Massive Disk. If you wish to add a brand new Nix package deal to the Massive Disk, you replace nixpkgs-replit (Replit’s nixpkgs overlay) and watch for the disks to be rebuilt and deployed.
Unlocking the complete energy of Nix
If we may persist the higher retailer, we’d deliver the complete energy of Nix to Replit. Listed below are among the advantages:
Configuration reusability: tasks with present Nix configurations can deliver them to Replit with out modification. Repls with customary Nix configuration might be downloaded to your native pc and work. Write one growth surroundings configuration, reuse in all places (native, Repls, CI, and production deployments).
Higher caching: your growth surroundings might be cached and able to go when your Repl begins. You possibly can even cache your construct artifacts throughout completely different variations of your code.
Nix Flakes: get entry to the improved person expertise and code sharing supplied by Nix Flakes.
Utilizing different Nix tasks: not all Nix code is accessible in nixpkgs, this unlocks entry to the broader ecosystem.
Construct your mission: Nix is a common construct software with a number of functions. It may construct tasks in virtually any language together with Python, Go, Rust, and C++. It may additionally construct containers, and digital machine pictures.
Layered Retailer to the rescue!
By modifying Nix to pay attention to the layered overlay filesystem, we will combine instantaneous set up with the complete energy of persistent Nix builds. Moderately than utilizing an overlay filesystem for /nix/retailer and the database, we solely overlay the /nix/retailer directories.
When Nix is able to construct, it first appears to be like into the decrease retailer database to see if it was already constructed. If obtainable, it skips constructing and provides an entry to the higher retailer database indicating the package deal is accessible. By way of the facility of the overlay filesystem, the construct merchandise are already in place!
With the overlay filesystem between the higher and decrease databases eliminated, we will persist the higher retailer and add packages to the decrease retailer database with out interfering with the higher retailer database.
Different functions for the Layered Retailer
The Layered Retailer is an fascinating primitive that we predict could possibly be helpful in quite a lot of conditions:
- Utilizing a community filesystem to serve a Nix retailer
- A construct farm or CI server that maintains a fast-access higher Nix retailer whereas nonetheless accessing a networked filesystem decrease retailer for reusing huge or costly construct merchandise.
- Something composing greater than 2 shops (2 shops is the higher restrict of an overlay filesystem with out a Layered Retailer)
We’d love to listen to your concepts for a way you’d use it.
Improvement
To make this a actuality, we’ve teamed up with Obsidian Systems and Tweag. Within the coming weeks, we shall be creating the Layered Retailer options, releasing a Nix Neighborhood RFC, and dealing to upstream it into Nix.
Wish to be one of many first ones to strive it out? Add yourself to our beta list.
Work at Replit
Are you interested by Nix and would like to see it within the palms of extra individuals? Come work with us on making instantaneous software program setup and deployment a actuality for everybody.