Monday, August 29, 2022

Stop calling everything "Nix"

nix-terminology

One of my pet peeves is when people abuse the term “Nix” without qualification when trying to explain the various components of the Nix ecosystem.

As a concrete example, a person might say:

“I hate Nix’s syntax”

… and when you dig into this criticism you realize that they’re actually complaining about the Nixpkgs API, which is not the same thing as the syntax of the Nix expression language.

So one of the goals of this post is to introduce some unambiguous terminology that people can use to refer to the various abstraction layers of the Nix ecosystem in order to avoid confusion. I’ll introduce each abstraction layer from the lowest level abstractions to the highest level abstractions.

Another reason I explain “Nix” in terms of these abstraction layers is because this helps people consult the correct manual. The Nix ecosystem provides three manuals that you will commonly need to refer to in order to become more proficient:

… and I hope by the end of this post it will be clearer which manual interests you for any given question.

Edit: Domen Ko┼żar pointed out that there is an ongoing effort to standardize terminology here:

I’ll update the post to match the agreed-upon terminology when that is complete.

Layer #0: The Nix store

I use the term “Nix store” to mean essentially everything you can manage with the nix-store command-line tool.

That is the simplest definition, but to expand upon that, I mean the following files:

  • Derivations: /nix/store/*.drv
  • Build products: /nix/store/* without a .drv extension
  • Log files: /nix/var/log/nix/drvs/**
  • Garbage collection roots: /nix/var/nix/gcroots/**

… and the following operations:

  • Realizing a derivation

    i.e. converting a .drv file to the corresponding build products using nix-store --realise

  • Adding static files to the /nix/store

    i.e. nix-store --add

  • Creating GC roots for build products

    i.e. the --add-root option to nix-store

  • Garbage collecting derivations not protected by a GC root

    i.e. nix-store --gc

There are other things the Nix store supports (like profile management), but these are the most important operations.

CAREFULLY NOTE: the “Nix store” is independent of the “Nix language” (which we’ll define below). In other words, you could replace the front-end Nix programming language with another language (e.g. Guile scheme, as Guix does). This is because the Nix derivation format (the .drv files) and the nix-store command-line interface are both agnostic of the Nix expression language. I have a talk which delves a bit more into this subject:

Layer #1: The Nix language

I use the term “Nix language” to encompass three things:

  • The programming language: source code we typically store in .nix files
  • Instantiation: the interpretation of Nix code to generate .drv files
  • Flakes: pure evaluation and instantiation caching

To connect this with the previous section, the typical pipeline for converting Nix source code to a build product is:

Nix source code (*.nix)            │ Nix language
      ↓ Instantiation              ├─────────────
Nix derivation (/nix/store/*.drv)  │
      ↓ Realization                │ Nix store
Nix build product (/nix/store/*)   │

In isolation, the Nix language is “just” a purely functional programming language with simple language constructs. For example, here is a sample Nix REPL session:

nix-repl> 2 + 2
4

nix-repl> x = "world"   

nix-repl> "Hello, " + x  
"Hello, world"

nix-repl> r = { a = 1; b = true; }

nix-repl> if r.b then r.a else 0
1

However, as we go up the abstraction ladder the idiomatic Nix code we’ll encounter will begin to stray from that simple functional core.

NOTE: Some people will disagree with my choice to include flakes at this abstraction layer since flakes are sometimes marketed as a dependency manager (similar to niv). I don’t view them in this way and I treat flakes as primarily as mechanism for purifying evaluation and caching instantiation, as outlined in this post:

… and if you view flakes in that capacity then they are a feature of the Nix language since evaluation/instantiation are the primary purpose of the programming language.

Layer #2: The Nix build tool

This layer encompasses the command-line interface to both the “Nix store” and the “Nix language”.

This includes (but is not limited to):

  • nix-store (the command, not the underlying store)
  • nix-instantiate
  • nix-build
  • nix-shell
  • nix subcommands, including:
    • nix build
    • nix run
    • nix develop
    • nix log
    • nix flake

I make this distinction because the command-line interface enables some additional niceties that are not inherent to the underlying layers. For example, the nix build command has some flake integration so that you can say nix build someFlake#somePackage and this command-line API nicety is not necessarily inherent to flakes (in my view).

Also, many of these commands operate at both Layer 0 and Layer 1, which can blur the distinction between the two. For example the nix-build command can accept a layer 1 Nix program (i.e. a .nix file) or a layer 0 derivation (i.e. a .drv file).

Another thing that blurs the distinction is that the Nix manual covers all three of the layers introduced so far, ranging from the Nix store to the command-line interface. However, if you want to better understand these three layers then that is correct place to begin:

Layer #3: Nixpkgs

Nixpkgs is a software distribution (a.k.a. “distro”) for Nix. Specifically, all of the packaging logic for Nixpkgs is hosted on GitHub here:

This repository contains a large number of Nix expressions for building packages across several platforms. If the “Nix language” is a programming language then “Nixpkgs” is a gigantic “library” authored within that language. There are other Nix “libraries” outside of Nixpkgs but Nixpkgs is the one you will interact with the most.

The Nixpkgs repository establishes several widespread idioms and conventions, including:

  • The standard environment (a.k.a. stdenv) for authoring a package
    • There are also language-specific standard-environments, too
  • A domain-specific language for overriding individual packages or sets of packages

When people complain about “Nix’s syntax”, most of the time they’re actually complaining about Nixpkgs and more specifically complaining about the Nixpkgs system for overriding packages. However, I can see how people might mistake the two.

The reason for the confusion is that the Nixpkgs support for overrides is essentially an embedded domain-specific language, meaning that you still express everything in the Nix language (layer 1), but the ways in which you express things is fundamentally different than if you were simply using low-level Nix language features.

As a contrived example, this “layer 1” Nix code:

let
  x = 1;

  y = x + 2;

… would roughly correspond to the following “layer 3” Nixpkgs overlay:

self: super: {
  x = 1;

  y = self.x + 2;
}

The reason why Nixpkgs doesn’t do the simpler “layer 1” thing is because Nixpkgs is designed to support “late binding” of expressions, meaning that everything can be overridden, even dependencies deep within the dependency tree. Moreover, this overriding is done in such a way that everything “downstream” of the overrride (i.e. all reverse dependencies) pick up the change correctly.

As a more realistic example, the following program:

let
  pkgs = import <nixpkgs> { };

  fast-tags =
    pkgs.haskell.lib.justStaticExecutables pkgs.haskellPackages.fast-tags;

  fast-tags-no-tests =
    pkgs.haskell.lib.dontCheck fast-tags;

in
  fast-tags-no-tests

… is simpler, but is not an idiomatic use of Nixpkgs because it is not using the overlay system and therefore does not support late binding. The more idiomatic analog would be:

let
  overlay = self: super: {
    fast-tags =
      self.haskell.lib.justStaticExecutables self.haskellPackages.fast-tags;

    fast-tags-no-tests =
      self.haskell.lib.dontCheck self.fast-tags;
  };

  pkgs = import <nixpkgs> { overlays = [ overlay ]; };

in
  pkgs.fast-tags-no-tests

You can learn more about this abstraction layer by consulting the Nixpkgs manual:

Layer #4: NixOS

NixOS is an operating system that is (literally) built on Nixpkgs. Specifically, there is a ./nixos/ subdirectory of the Nixpkgs repository for all of the NixOS-related logic.

NixOS is based on the NixOS module system, which is yet another embedded domain-specific language. In other words, you configure NixOS with Nix code, but the idioms of that Nix code depart even more wildly from straightforward “layer 1” Nix code.

NixOS modules were designed to look more like Terraform modules than Nix code, but they are still technically Nix code. For example, this is what the NixOS module for the lorri service looks like at the time of this writing:

{ config, lib, pkgs, ... }:

let
  cfg = config.services.lorri;
  socketPath = "lorri/daemon.socket";
in {
  options = {
    services.lorri = {
      enable = lib.mkOption {
        default = false;
        type = lib.types.bool;
        description = lib.mdDoc ''
          Enables the daemon for `lorri`, a nix-shell replacement for project
          development. The socket-activated daemon starts on the first request
          issued by the `lorri` command.
        '';
      };
      package = lib.mkOption {
        default = pkgs.lorri;
        type = lib.types.package;
        description = lib.mdDoc ''
          The lorri package to use.
        '';
        defaultText = lib.literalExpression "pkgs.lorri";
      };
    };
  };

  config = lib.mkIf cfg.enable {
    systemd.user.sockets.lorri = {
      description = "Socket for Lorri Daemon";
      wantedBy = [ "sockets.target" ];
      socketConfig = {
        ListenStream = "%t/${socketPath}";
        RuntimeDirectory = "lorri";
      };
    };

    systemd.user.services.lorri = {
      description = "Lorri Daemon";
      requires = [ "lorri.socket" ];
      after = [ "lorri.socket" ];
      path = with pkgs; [ config.nix.package git gnutar gzip ];
      serviceConfig = {
        ExecStart = "${cfg.package}/bin/lorri daemon";
        PrivateTmp = true;
        ProtectSystem = "strict";
        ProtectHome = "read-only";
        Restart = "on-failure";
      };
    };

    environment.systemPackages = [ cfg.package ];
  };
}

You might wonder how NixOS relates to the underlying layers. For example, if Nix is a build system, then how do you “build” NixOS? I have another post which elaborates on that subject here:

Also, you can learn more about this abstraction layer by consulting the NixOS manual:

Nix ecosystem

I use the term “Nix ecosystem” to describe all of the preceding layers and other stuff not mentioned so far (like hydra, the continuous integration service).

This is not a layer of its own, but I mention this because I prefer to use “Nix ecosystem” instead of “Nix” to avoid ambiguity, since the latter can easily be mistaken for an individual abstraction layer (especially the Nix language or the Nix build tool).

However, when I do hear people say “Nix”, then I generally understand it to mean the “Nix ecosystem” unless they clarify otherwise.

Conclusion

Hopefully this passive aggressive post helps people express themselves a little more precisely when discussing the Nix ecosystem.

If you enjoy this post, you will probably also like this other post of mine:

… since that touches on the Nixpkgs and NixOS embedded domain-specific languages and how they confound the user experience.

I’ll conclude this post with the following obligatory joke:

I’d just like to interject for a moment. What you’re refering to as Nix, is in fact, NixOS, or as I’ve recently taken to calling it, Nix plus OS. Nix is not an operating system unto itself, but rather another free component of a fully functioning ecosystem made useful by the Nix store, Nix language, and Nix build tool comprising a full OS as defined by POSIX.

Many Guix users run a modified version of the Nix ecosystem every day, without realizing it. Through a peculiar turn of events, the operating system based on Nix which is widely used today is often called Nix, and many of its users are not aware that it is basically the Nix ecosystem, developed by the NixOS foundation.

There really is a Nix, and these people are using it, but it is just a part of the system they use. Nix is the expression language: the program in the system that specifies the services and programs that you want to build and run. The language is an essential part of the operating system, but useless by itself; it can only function in the context of a complete operating system. Nix is normally used in combination with an operating system: the whole system is basically an operating system with Nix added, or NixOS. All the so-called Nix distributions are really distributions of NixOS!

Sunday, August 28, 2022

Incrementally package a Haskell program using Nix

incremental-nix

This post walks through how to take a standalone Haskell file and progressively package the file using Nix. In other words, we will tour a spectrum of packaging options ranging from simple to fancy.

The running example will be the following standalone single-file Haskell program:

I won’t go into detail about what that program does, although you can study the program if you are curious. Essentially, I’m planning to deliver a talk based on that program at this year’s MuniHac and I wanted to package it up so that other people could collaborate on the program with me during the hackathon.

When I began writing this post, there was no packaging logic for this program; it’s a standalone Haskell file. However, this file has several dependencies outside of Haskell’s standard library, so up until now I needed some way to obtain those dependencies for development.

Stage 0: ghc.withPackages

The most low-tech way that you can hack on a Haskell program using Nix is to use nix-shell to obtain a transient development environment (this is what I had done up until now).

Specifically, you can do something like this:

$ nix-shell --packages 'ghc.withPackages (pkgs: [ pkgs.mtl pkgs.MemoTrie pkgs.containers pkgs.pretty-show ])'

… where pkgs.mtl and pkgs.MemoTrie indicate that I want to include the mtl and MemoTrie packages in my Haskell development environment.

Inside of that development environment I can build and run the file using ghc. For example, I can use ghc -O to build an executable to run:

[nix-shell]$ ghc -O Spire.hs
[nix-shell]$ ./Spire

… or if I don’t care about optimizations I can interpret the file using runghc:

$ runghc Spire.hs

Stage 1: IDE support

Once I’m inside a Nix shell I can begin to take advantage of integrated development environment (IDE) support.

The two most common tools Haskell developers use for rapid feedback are ghcid and haskell-language-server:

  • ghcid provides a command-line interface for fast type-checking feedback but doesn’t provide other IDE-like features

  • haskell-language-server is more of a proper IDE that you use in conjunction with some editor

I can obtain either tool by exiting from the shell and creating a new shell that includes the desired tool.

For example, if I want to use ghcid then I recreate the nix-shell using the following command:

$ nix-shell --packages ghcid 'ghc.withPackages (pkgs: [ pkgs.mtl pkgs.MemoTrie pkgs.containers pkgs.pretty-show ])'

… and then I can tell ghcid to continuously type-check my file using:

[nix-shell]$ ghcid Spire.hs

If I want to use haskell-language-server, then I recreate the nix-shell using this command:

$ nix-shell --packages haskell-language-server 'ghc.withPackages (pkgs: [ pkgs.mtl pkgs.MemoTrie pkgs.containers pkgs.pretty-show ])'

… and then I can explore the code in any editor that supports the language server protocol.

Note that if you use VSCode as your editor then you may need to install some additional plugins:

… and the next section will show how to install VSCode and those plugins using Nix.

However, once you do install those plugins then you can open the file in VSCode from within the nix-shell using:

[nix-shell]$ code Spire.hs

… and once you trust the file the IDE features will kick in.

Stage 2: Global development environment

Sometimes I like to globally install development tools that are commonly shared between projects. For example, if I use ghcid or haskell-language-server across all my projects then I don’t want to have to explicitly enumerate that tool in each project’s Nix shell.

Moreover, my tool preferences might not be shared by other developers. If I share my nix-shell with other developers for a project then I probably don’t want to add editors/IDEs or other command-line tools to that environment because then they have to download those tools regardless of whether they plan to use them.

However, I don’t want to globally install development tools like this:

$ nix-env --install --file '<nixpkgs>' --attr ghcid
$ nix-env --install --file '<nixpkgs>' --attr haskell-language-server

Part of the reason I use Nix is to avoid imperatively managing my development environment. Fortunately, though, nix-env supports a more declarative way of managing dependencies.

What you can do instead is save a file like this to ~/default.nix:

let
  # For VSCode
  config = { allowUnfree = true; };

  overlay = pkgsNew: pkgsOld: {
    # Here's an example of how to use Nix to install VSCode with plugins managed
    # by Nix, too
    vscode-with-extensions = pkgsOld.vscode-with-extensions.override {
      vscodeExtensions = [
        pkgsNew.vscode-extensions.haskell.haskell
        pkgsNew.vscode-extensions.justusadam.language-haskell
      ]; 
    };
  };

  pkgs = import <nixpkgs> { inherit config; overlays = [ overlay ]; };

in      
  { inherit (pkgs)
      # I included some sample useful development tools for Haskell.  Feel free
      # to customize.
      cabal-install
      ghcid
      haskell-language-server
      stylish-haskell
      vscode-with-extensions 
    ; 
  }     

… and once you create that file you have two options.

The first option is that you can set your global development environment to match the file by running:

$ nix-env --remove-all --install --file ~/default.nix

NOTE: At the time of this writing you may also need to add --system x86_64-darwin if you are trying out these examples on an M1 Macbook. For more details, see:

Carefully note the --remove-all, which resets your development environment to match the file, so that nothing from your old development environment is accidentally carried over into your new development environment. This makes our use of the nix-env command truly declarative.

The second option is that you can change the file to create a valid shell, like this:

let
  config = { allowUnfree = true; };

  overlay = pkgsNew: pkgsOld: {
    vscode-with-extensions = pkgsOld.vscode-with-extensions.override {
      vscodeExtensions = [
        pkgsNew.vscode-extensions.haskell.haskell
        pkgsNew.vscode-extensions.justusadam.language-haskell
      ];
    };
  };

  pkgs = import <nixpkgs> { inherit config; overlays = [ overlay ]; };

in
  pkgs.mkShell {
    packages = [
      pkgs.ghcid
      pkgs.haskell-language-server
      pkgs.stylish-haskell
      pkgs.vscode-with-extensions
      pkgs.cabal-install
    ];
  }

… and then run:

$ nix-shell ~/default.nix

Or, even better, you can rename the file to ~/shell.nix and then if you’re already in your home directory (e.g. you just logged into your system), then you can run:

$ nix-shell

… which will select ~/shell.nix by default. This lets you get a completely transient development environment so that you never have to install anything development tools globally.

These nix-shell commands stack, so you can first run nix-shell to obtain your global development environment and then use nix-shell a second time to obtain project-specific dependencies.

My personal preference is to use the declarative nix-env trick for installing global development tools. In my opinion it’s just as elegant as nix-shell and slightly less hassle.

Stage 3: Cabal

Anyway, enough about global development tools. Back to our Haskell project!

So ghc.withPackages is a great way to just start hacking on a standalone Haskell program when you don’t want to worry about packaging up the program. However, at some point you might want to share the program with the others or do a proper job of packaging if you’re trying to productionize the code.

That brings us to the next step, which is packaging our Haskell program with a Cabal file (a Haskell package manifest). We’ll need the cabal-install command-line tool before we proceed further, so you’ll want to add that tool to your global development environment (see the previous section).

To create our .cabal file we can run the following command from the top-level directory of our Haskell project:

$ cabal init --interactive
Should I generate a simple project with sensible defaults? [default: y] n

… and follow the prompts to create a starting point for our .cabal file.

After completing those choices and trimming down the .cabal file (to keep the example simple), I get a file that looks like this:

cabal-version:      2.4
name:               spire
version:            1.0.0
license:            BSD-3-Clause
license-file:       LICENSE

executable spire
    main-is:          Spire.hs
    build-depends:    base ^>=4.14.3.0
    default-language: Haskell2010

The only thing I’m going change for now is to add dependencies to the build-depends section and increase the upper bound on base::

cabal-version:      2.4
name:               spire
version:            1.0.0
license:            BSD-3-Clause
license-file:       LICENSE

executable spire
    main-is:          Spire.hs
    build-depends:    base >=4.14.3.0 && < 5
                    , MemoTrie
                    , containers
                    , mtl
                    , pretty-show
                    , transformers
    default-language: Haskell2010

Stage 4: cabal2nix --shell

Adding a .cabal file suffices to share our Haskell package with other Haskell developers if they’re not using Nix. However, if we want to Nix-enable package our package then we have a few options.

The simplest option is to run the following command from the top-level of the Haskell project:

$ cabal2nix --shell . > shell.nix

That will create something similar to the following shell.nix file:

{ nixpkgs ? import <nixpkgs> {}, compiler ? "default", doBenchmark ? false }:

let

  inherit (nixpkgs) pkgs;

  f = { mkDerivation, base, containers, lib, MemoTrie, mtl
      , pretty-show, transformers
      }:
      mkDerivation {
        pname = "spire";
        version = "1.0.0";
        src = ./.;
        isLibrary = false;
        isExecutable = true;
        executableHaskellDepends = [
          base containers MemoTrie mtl pretty-show transformers
        ];
        license = lib.licenses.bsd3;
      };

  haskellPackages = if compiler == "default"
                       then pkgs.haskellPackages
                       else pkgs.haskell.packages.${compiler};

  variant = if doBenchmark then pkgs.haskell.lib.doBenchmark else pkgs.lib.id;

  drv = variant (haskellPackages.callPackage f {});

in

  if pkgs.lib.inNixShell then drv.env else drv

… and if you run nix-shell within the same directory the shell environment will have the Haskell dependencies you need to build and run project using cabal:

$ nix-shell
[nix-shell]$ cabal run

… and tools like ghcid and haskell-language-server will also work within this shell, too. The only difference is that ghcid now takes no arguments, since it will auto-detect the cabal project in the current directory:

[nix-shell]$ ghcid

Note that this nix-shell will NOT include cabal by default. You will need to globally install cabal (see the prior section on “Global development environment”).

This cabal2nix --shell workflow is sufficiently lightweight that you can Nixify other people’s projects on the fly when hacking on them locally. A common thing I do if I need to make a change to a person’s project is to clone their repository, run:

$ cabal2nix --shell . > shell.nix
$ nix-shell

… and start hacking away. I don’t even need to upstream the shell.nix file I created in this way; I just keep it around locally for my own hacking.

In fact, I typically don’t want to upstream such a shell.nix file (even if the upstream author were receptive to Nix), because there are more robust Nix expressions we can upstream instead.

Stage 5: Custom shell.nix file

One disadvantage of cabal2nix --shell is that you have to re-run the command any time your dependencies change. However, if you’re willing to hand-write your own shell.nix file then you can create something more stable:

let
  overlay = pkgsNew: pkgsOld: {
    haskellPackages = pkgsOld.haskellPackages.override (old: {
      overrides = pkgsNew.haskell.lib.packageSourceOverrides {
        spire = ./.;
      };
    });
  };

  pkgs = import <nixpkgs> { overlays = [ overlay ]; };

in
  pkgs.haskellPackages.spire.env

The packageSourceOverrides is the key bit. Under the hood, that essentially runs cabal2nix for you any time your project changes and then generates your development environment from the result. You can also use packageSourceOverrides to specify non-default versions of dependencies, too:

let
  overlay = pkgsNew: pkgsOld: {
    haskellPackages = pkgsOld.haskellPackages.override (old: {
      overrides = pkgsNew.haskell.lib.packageSourceOverrides {
        spire = ./.;

        # Example of how to pin a dependency to a non-defaul version
        pretty-show = "1.9.5";
      };
    });
  };

  pkgs = import <nixpkgs> { overlays = [ overlay ]; };

in
  pkgs.haskellPackages.spire.env

… although that will only work for packages that have been released prior to the version of Nixpkgs that you’re depending on.

If you want something a bit more robust, you can do something like this:

let
  overlay = pkgsNew: pkgsOld: {
    haskellPackages = pkgsOld.haskellPackages.override (old: {
      overrides =
        pkgsNew.lib.fold
          pkgsNew.lib.composeExtensions
          (old.overrides or (_: _: { }))
          [ (pkgsNew.haskell.lib.packageSourceOverrides {
              spire = ./.;
            })
            (pkgsNew.haskell.lib.packagesFromDirectory {
              directory = ./packages;
            })
          ];
    });
  };

  pkgs = import <nixpkgs> { overlays = [ overlay ]; };

in
  pkgs.haskellPackages.spire.env

… and then you have the option to also depend on any dependency that cabal2nix knows how to generate:

$ mkdir packages

$ # Add the following file to version control to preserve the directory
$ touch packages/.gitkeep

$ cabal update

$ cabal2nix cabal://${PACKAGE_NAME}-${VERSION} > ./packages/${PACKAGE_NAME}.nix

… and that works even on bleeding-edge Haskell packages that Nixpkgs hasn’t picked up, yet.

Stage 6: Pinning Nixpkgs

All of the prior examples are “impure”, meaning that they depend on the ambient nixpkgs channel installed on the developer’s system. This nixpkgs channel might vary from system to system, meaning that each system might have different versions of nixpkgs installed, and then you run into issues reproducing each other’s builds.

For example, if you have a newer version of nixpkgs installed your Nix build for the above Haskell project might succeed, but then another developer might attempt to build your project with an older version of nixpkgs, which might select an older incompatible version of one of your Haskell dependencies.

Or, vice versa, the examples in this blog post might succeed at the time of this writing for the current version of nixpkgs but then as time goes on the examples might begin to fail for future versions of nixpkgs.

You can fix that by pinning Nixpkgs, which this post covers:

For example, we could pin nixpkgs for our global ~/default.nix like this:

let
  nixpkgs = builtins.fetchTarball {
    url    = "https://github.com/NixOS/nixpkgs/archive/0ba2543f8c855d7be8e90ef6c8dc89c1617e8a08.tar.gz";
    sha256 = "14ann7vz7qgfrw39ji1s19n1p0likyf2ag8h7rh8iwp3iv5lmprl";
  };

  config = { allowUnfree = true; };

  overlay = pkgsNew: pkgsOld: {
    vscode-with-extensions = pkgsOld.vscode-with-extensions.override {
      vscodeExtensions = [
        pkgsNew.vscode-extensions.haskell.haskell
        pkgsNew.vscode-extensions.justusadam.language-haskell
      ];
    };
  };

  pkgs = import nixpkgs { inherit config; overlays = [ overlay ]; };

in
  { inherit (pkgs)
      cabal-install
      ghcid
      haskell-language-server
      stylish-haskell
      vscode-with-extensions
    ;
  }

… which pins us to the tip of the release-22.05 branch at the time of this writing.

We can likewise pin nixpkgs for our project-local shell.nix like this:

let
  nixpkgs = builtins.fetchTarball {
    url    = "https://github.com/NixOS/nixpkgs/archive/0ba2543f8c855d7be8e90ef6c8dc89c1617e8a08.tar.gz";
    sha256 = "14ann7vz7qgfrw39ji1s19n1p0likyf2ag8h7rh8iwp3iv5lmprl";
  };

  overlay = pkgsNew: pkgsOld: {
    haskellPackages = pkgsOld.haskellPackages.override (old: {
      overrides = pkgsNew.haskell.lib.packageSourceOverrides {
        spire = ./.;
      };
    });
  };

  pkgs = import nixpkgs { overlays = [ overlay ]; };

in
  pkgs.haskellPackages.spire.env

Flakes

The final improvement we can make is the most important one of all: we can convert our project into a Nix flake:

There are two main motivations for flake-enabling our project:

  • To simplify managing inputs that we need to lock (e.g. nixpkgs)
  • To speed up our shell

To flake-enable our project, we’ll save the following code to flake.nix:

{ inputs = {
    nixpkgs.url = github:NixOS/nixpkgs/release-22.05;

    utils.url = github:numtide/flake-utils;
  };

  outputs = { nixpkgs, utils, ... }:
    utils.lib.eachDefaultSystem (system:
      let
        config = { };

        overlay = pkgsNew: pkgsOld: {
          spire =
            pkgsNew.haskell.lib.justStaticExecutables
              pkgsNew.haskellPackages.spire;

          haskellPackages = pkgsOld.haskellPackages.override (old: {
            overrides = pkgsNew.haskell.lib.packageSourceOverrides {
              spire = ./.;
            };
          });
        };

        pkgs =
          import nixpkgs { inherit config system; overlays = [ overlay ]; };

      in
        rec {
          packages.default = pkgs.haskellPackages.spire;

          apps.default = {
            type = "app";

            program = "${pkgs.spire}/bin/spire";
          };

          devShells.default = pkgs.haskellPackages.spire.env;
        }
    );
}

… and then we can delete our old shell.nix because we don’t need it anymore.

Now we can obtain a development environment by running:

$ nix develop

… and the above flake also makes it possible to easily build and run the program, too:

$ nix run    # Run the program
$ nix build  # Build the project

In fact, you can even run a flake without having to clone a repository. For example, you can run the example code from this blog post by typing:

$ nix run github:Gabriella439/spire

Moreover, we no longer have to take care of managing hashes for, say, Nixpkgs. The flake machinery takes care of that automatically for you and generates a flake.lock file which you can then add to version control. For example, the lock file I got was:

{
  "nodes": {
    "nixpkgs": {
      "locked": {
        "lastModified": 1661617163,
        "narHash": "sha256-NN9Ky47j8ohgPhA9JZyfkYIbbAo6RJkGz+7h8/exVpE=",
        "owner": "NixOS",
        "repo": "nixpkgs",
        "rev": "0ba2543f8c855d7be8e90ef6c8dc89c1617e8a08",
        "type": "github"
      },
      "original": {
        "owner": "NixOS",
        "ref": "release-22.05",
        "repo": "nixpkgs",
        "type": "github"
      }
    },
    "root": {
      "inputs": {
        "nixpkgs": "nixpkgs",
        "utils": "utils"
      }
    },
    "utils": {
      "locked": {
        "lastModified": 1659877975,
        "narHash": "sha256-zllb8aq3YO3h8B/U0/J1WBgAL8EX5yWf5pMj3G0NAmc=",
        "owner": "numtide",
        "repo": "flake-utils",
        "rev": "c0e246b9b83f637f4681389ecabcb2681b4f3af0",
        "type": "github"
      },
      "original": {
        "owner": "numtide",
        "repo": "flake-utils",
        "type": "github"
      }
    }
  },
  "root": "root",
  "version": 7
}

… and you can easily upgrade to, say, a newer revision of Nixpkgs if you need to.

Additionally, all of the Nix commands are now faster. Specifically, the first time you run a command Nix still needs to download and/or build dependencies, but subsequent runs are faster because Nix can skip the instantiation phase. For more details, see:

Conclusion

Flakes are our final destination, so that’s as far as this post will go. There are technically some more ways that we can overengineer things, but in my experience the idioms highlighted in this post are the ones that provide the highest power-to-weight ratio.

The key thing to take away is that the Nixpkgs Haskell infrastructure lets you smoothly transition from simpler approaches to more powerful approaches, and even the final flake-enabled approach is actually not that complicated.