The Hitchhiker's Guide to NixOS

Posted on March 23, 2021
Table of Contents

Like cargo build, but for your OS.

Brave New World

Infrastructure as Code

How you build your software?

git clone https://github.com/xxx/yyy.git
{stack,cargo,cabal,yarn,go,gradlew,sbt,mvn} {build,compile}

or maybe just

{python,lua,ruby,perl} xxx.{py,lua,rb,pl}

How you build your OS?

Entering nix flake

git clone https://github.com/contrun/dotfiles.git
cd dotfiles
git checkout c524c7316967049b444e84e6fba925426b94a112
nixos-rebuild build --flake '.#x86_64-linux'

Demo time

nix build '.#nixosConfigurations.x86_64-linux.config.system.build.toplevel'
nix build '.#nixosConfigurations.x86_64-linux.config.system.build.vm'
nix path-info '.#nixosConfigurations.x86_64-linux.config.system.build.toplevel' -sSh
QEMU_NET_OPTS="hostfwd=tcp::2222-:22" ./result/bin/run-*-vm
ssh -o GlobalKnownHostsFile=/dev/null -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -p 2222 e@localhost

Show off my PATH, systemctl status, i3wm, xmonad.

Three pillars of nix super power

I’ve just demonstrated packer + vagrant. Nix is far more powerful.

Reproducible

Build it solely from a git repo, with all the desired packages, configurations.

Declarative

A minimalist language to do all those things, without diving into all the details.

Reliable

Easily roll back to the previous state.

Devil in the Nix City

Let’s see who you really are

nix build '.#nixosConfigurations.x86_64-linux.config.system.build.toplevel' --json | jq
nix show-derivation $(nix build '.#nixosConfigurations.x86_64-linux.config.system.build.toplevel' --json | jq -r '.[0].drvPath') | jq -r '.. | .buildCommand? | select(.)'

Anatomy of a flake file

{
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-20.09";

  outputs = { self, nixpkgs }: {

    nixosConfigurations.container = nixpkgs.lib.nixosSystem {
      system = "x86_64-linux";
      modules =
        [ ({ pkgs, ... }: {
            boot.isContainer = true;

            # Let 'nixos-version --json' know about the Git revision
            # of this flake.
            system.configurationRevision = nixpkgs.lib.mkIf (self ? rev) self.rev;

            # Network configuration.
            networking.useDHCP = false;
            networking.firewall.allowedTCPPorts = [ 80 ];

            # Enable a web server.
            services.httpd = {
              enable = true;
              adminAddr = "morty@example.org";
            };
          })
        ];
    };

  };
}
cd "$(mktemp -d)"
git init
git add .
nixos-container create flake-test --flake .
nixos-container start flake-test
curl http://flake-test/

cf Nix Flakes, Part 3: Managing NixOS systems

Anatomy of a nix derivation

Demo time

cat ~/Workspace/nixpkgs/pkgs/applications/misc/hello/default.nix
nix-shell '<nixpkgs>' -A hello
declare -f genericBuild
unpackPhase

Conclusion

Key components

Nix, the language

Nix, the interpreter

Nixpkgs, the package set

  1. A large library of functions to build std derivations

    buildPythonPackage, buildGoPackage, buildRustPackage, snapTools.makeSnap, lib.nixosSystem (thus nixos itself is nothing special)

  2. A large library of nixos modules

    kernel customization, zsh, emacs, xmonad

  3. Some helper functions

    fixed point calculation, string manipulation

The Perks of Being a Stoic

Me compiling, github microsoft crying

Compiling

Don’t cry for me, github and cachix. cf nix build github actions

Github actions to build my packages

Cachix to cache my build results

nixos test

Integration tests

You are a distributed system hacker. You have cassandra, pomerium, a blockchain. You don’t emulate distributed system with seperated processes.

Demo time

nix-build ./nixos/tests/pomerium.nix
nix-instantiate --strict --eval -E 'with import ./nixos/tests/pomerium.nix {}; driver.outPath'
$(nix-instantiate --strict --eval -E 'with import ./nixos/tests/pomerium.nix {}; driver.outPath' | xargs)/bin/nixos-test-driver

backend.wait_for_unit("nginx")
backend.wait_for_open_port(80)
pomerium.wait_for_unit("pomerium")
pomerium.wait_for_open_port(80)
pomerium.succeed("curl --resolve my.website:80:127.0.0.1 http://my.website | grep -q 'hello world'")

Complex build environment

Cross compilation

You need to cross compile a package. I can haz a good cross compilation toolchain without scratch my heads?

Static linking

You need to statically link programs, your program depends on some c library which your package manager does not pack static libraries, e.g. openssl, rocksdb, or the library is not static-linking friendly, e.g. glibc.

Demo time

go build ./cmd/ipvpnd/
ldd ipvpnd
go build -ldflags '-extldflags "-static"' ./cmd/ipvpnd/
nix-shell -E 'with import <nixpkgs> {}; pkgsCross.musl64.clangStdenv.mkDerivation { name = "clang-nix-shell"; }'
go build -ldflags '-extldflags "-static"' ./cmd/ipvpnd/
nix-build -E 'with import <nixpkgs> {}; pkgsCross.aarch64-multiplatform-musl.hello'
file ./result/bin/hello
qemu-aarch64 ./result/bin/hello
./result/bin/hello # courtesy of binfmt.emulatedSystems = [ "aarch64-linux" ];

Build farms

Outsource cpu-intensive tasks to faster machines. See the above nix-build -E 'with import <nixpkgs> {}; pkgsCross.aarch64-multiplatform-musl.hello'.

dev shells, direnv for projects

Atomic updates, Easy rollback

realpath /nix/var/nix/profiles/system
realpath /run/current-system/
nixos-rebuild switch --rollback xxx
ls /run/secrets*

containers

See nixos-container create flake-test --flake .

nixops

Great Expectations

Nice-to-have improvements

Beginner-friendliness

Enterprise-grade applications

Versatility

All your base are belong to us

devos

kubernix

nixops

deploy-rs

Hercules CI

trustix

Flox