Friday, December 30, 2022

Nixpkgs support for Linux builders running on macOS

macos-builder

I recently upstreamed a derivation for a Linux builder into Nixpkgs that’s easy to deploy on macOS. The significance of this work is that you can now run the following command on macOS:

$ nix run nixpkgs#darwin.builder

… and that will launch a Linux builder that you can delegate builds to. For full details, read the corresponding section of the Nixpkgs manual.

In this post, I wanted to provide some of the background and motivation for this work to help contextualize it.

Background - NixOS qemu VMs on MacOS

I wasn’t originally trying to create a Linux builder for macOS when I began this project. I was actually working on making it as easy as possible to experiment interactively with (non-builder) NixOS qemu VMs on macOS.

While searching for prior art related to this I stumbled across the following Nixpkgs issue requesting exactly this same feature: Allowing NixOS VM’s to be run on macOS.

Even better, by the time I discovered that issue several people had already done most of the work, culminating in the following repository demonstrating how all of the features were supposed to fit together: YorikSar/nixos-vm-on-macos.

In fact, the flake for that repository also came with a binary cache, so if you just ran:

$ nix run github:YorikSar/nixos-vm-on-macos

… then you could run the sample NixOS VM from that repository on macOS without requiring access to an Linux builder because it would download all the Linux build products from the matching cache. Pretty neat!

However, this still didn’t completely satisfy my use case for reasons already noted by someone else: it doesn’t work well if you want to run a NixOS VM that differs even slightly from the included sample VM. Any difference requires Linux build products to be rebuilt which requires access to a Linux builder because those build products will not be cached ahead of time.

Background - linuxkit-nix

The need for a Linux builder wasn’t a showstopper for me because there was already prior art for bootstrapping a Linux builder on macOS, which was the linuxkit-nix project. So what I could have done was:

  • Launch a (non-NixOS) linuxkit VM on macOS for use as a Linux builder
  • Use the linuxkit builder to build the desired NixOS qemu VM
  • Run that NixOS qemu VM on macOS

However, I was curious if I could use a NixOS VM for the first step, too! In other words:

  • Launch a cached NixOS qemu VM on macOS for use as a Linux builder
  • Use the qemu builder to build the desired (non-builder) NixOS qemu VM
  • Run that NixOS qemu VM on macOS

The only difference between the two approaches is the first step: instead of using linuxkit to create the Linux builder we use qemu to create a NixOS builder. This works because the qemu builder’s NixOS configuration doesn’t need to change so hydra.nixos.org can build and cache the NixOS qemu builder ahead of time.

There were a few reasons I took interest in this approach:

  • linuxkit-nix appears to not work on aarch64-darwin (i.e. Apple Silicon)

    This seems like it is potentially fixable, but I wasn’t yet ready to volunteer to do that work.

  • It’s easier to customize a NixOS builder

    linuxkit-nix doesn’t use NixOS for the builder and instead creates a bespoke builder for this purpose. This means that you can’t use the NixOS module system to more easily customize the behavior of the builder.

  • The qemu-based solution is simpler than linuxkit-nix

    I think the easiest way to explain this is for me to link to the macos-builder.nix NixOS module, which has the entirety of the code that I contributed, which is significantly simpler than linuxkit-nix.

    The main reason that the qemu-based solution is simpler than linuxkit-nix is because it is reusing more infrastructure that has already been upstreamed into Nixpkgs (most notably, NixOS and qemu VMs).

  • linuxkit-nix appears to be unmaintained

    There was a nascent attempt to upstream linuxkit-nix into Nixpkgs, but that stalled because it seems like linuxkit-nix appears to have been abandoned a couple of years ago.

    I could have restored that effort, but personally I was fine with using the simpler qemu-based approach. I haven’t given up on the idea of reviving linuxkit-nix, but it’s not on my immediate roadmap.

There is one notable downside to using qemu over linuxkit, which is that qemu is supposed to be slower than linuxkit

Note: I have not actually verified this claim since I can’t run linuxkit-nix on my M1 Mac, but this is purportedly the reason that the authors of linuxkit-nix did not opt to use qemu for their approach according to this PR description.

qemu performance hasn’t been an issue for me (yet), but that could change, especially if I try to make use of this at work, where performance could potentially matter more.

Motivation

As I mentioned above, the long-term goal for all of this is to run NixOS VMs on macOS. There are two main reasons I’m interested in this:

  • I’m working on a NixOS book

    … and I wanted macOS users to be able to test-drive example NixOS configurations on their local machine without requiring them to own and operate a separate Linux machine.

  • I’m interested in running NixOS tests on macOS

    … primarily for work-related reasons. At work developers have to install postgres on their development machines for integration testing, and it would be much nicer if we could restructure our integration tests as NixOS tests (which run inside of qemu VMs instead of running on the host).

    However, at the time of this writing this would still require additional work which is in progress on this draft pull request.

Monday, December 19, 2022

Nixpkgs support for incremental Haskell builds

incremental

The context for this post is that at work I recently implemented Nix ecosystem support for “incrementally” building Haskell packages. By “incrementally” I mean that these Nix builds only need to build what changed since the last full build of the package so that the package doesn’t need to be built from scratch every time.

The pull requests implementing this feature have not yet been approved or merged at the time of this writing, but I figured that I would explain the motivation, design, results, and limitations of this work to hopefully persuade people that this work should be merged.

If you’re not interested in the design then you can skip straight to the Demo section below.

Background

I work on Mercury’s Backend Development User Experience team and we support developers contributing to a large Haskell monolith consisting of 3000+ modules. That may seem like a lot but the vast majority of these modules are small and the whole codebase takes ~14 minutes to compile in CI if we disable optimizations (although we still build with optimizations enabled for deployment).

In my experience, that’s pretty good for a Haskell project of this size, thanks not only to the work of our team but also other teams who also contribute to improving the development experience. In fact, the pioneering work for this “incremental builds” feature actually originated from two engineers outside our team.

First, Harry Garrood improved GHC’s change detection algorithm so that GHC would use the hash of the file to detect changes instead of using the timestamp. In this post he explains how you can make use of this to implement incremental builds for traditional CI services (e.g. GitHub actions) where each build reuses the intermediate build products from the prior build instead of building from scratch.

That alone would not be enough for us to use this at work since we use Nix where this sort of build impurity doesn’t fly. However, Harry and Jade Lovelace prototyped using this feature in Nixpkgs so that Nix builds of Haskell packages could also reuse intermediate build products from prior builds to save work. You can find their prototype here.

The basic idea behind the prototype Nixpkgs integration is that you split a Haskell package build into two separate builds:

  • A “full build” that builds the Haskell package from scratch

    This full build exports its intermediate build products (i.e. the dist directory) which can then be reused by:

  • An “incremental build” that only builds what changed since the full build

    This incremental build imports the intermediate build products from the corresponding full build so that it doesn’t have to build the package from scratch.

So you might wonder: if that was already implemented then what work still remained for me to do?

Problem

The main issue with the initial Nixpkgs integration is that it does not provide any support for selecting which Git revision to use as the basis for the full build. The existing solutions require some out-of-band process to automatically select and lock the appropriate git revision to use for the older (full) build.

Non-solution #0: Rolling rebuilds

The first non-solution is for each revision to always reuse the build products from the previous revision. This doesn’t work well with Nix because it would create an increasingly-long chain of dependent derivations; in order to build the most recent revision you’d have to build all preceding revisions.

The dilemma here is that Nix is forcing us to confront something that other build tools gloss over: if you’re always reusing build products from the last build then you can’t accurately reproduce the most recent build from scratch without reproducing all prior builds. You’ve essentially “contaminated” the current build with all prior builds by doing things in this way.

So what we really want is something more like this:

Periodically do a full build from scratch and then make each incremental build relative to the last full rebuild.

That’s much more compatible with Nix because then we only need to do two builds of our project if we rebuild things from scratch, instead of one build for every revision in our project’s history.

There’s also another issue with rolling rebuilds when you’re not using Nix, which is that most naïve attempts to do this don’t ensure that the starting build products came from the parent commit. You can end up with contamination of build products across branches if you’re not careful, which further complicates reproducibility.

Non-solution #1: Lockfile

Okay, so suppose you periodically do a full build of the project from scratch and then each incremental build is relative to the last full build. You would need to do a full rebuild frequently enough so that the incremental builds stay quick. If you wait too long in between full rebuilds then the project will evolve to the point where the incremental builds can no longer reuse most of the build products from the last full build and in the extreme case the incremental builds degenerate into full builds if they can’t reuse any old build products.

For example, at our work we currently do a full build of our large package once a day, so we need some way to update the full build to point to the last revision from the preceding day.

One existing approach to solving this involved using Nix flakes to manage the git revision for the older build. The idea is that you periodically run nix flake update to update the revision used for the full build and you might even automate this process by having some recurring cron job generate a pull request or commit to bump this revision on the main development branch. You don’t have to use flakes for this purpose, but flakes are probably the most ergonomic solution along these lines.

However, there are a few issues with this approach:

  • It only works well for short-lived pull requests

    In other words, if you update the revision used for the full build once a day then typically only pull requests that are less than a day old will benefit from incremental builds.

    Specifically, what we’d really like is “branch-local” incremental builds. In other words if a longer-lived development branch were to deposit a few commits a day we’d like there to be a full rebuild once a day on that branch so that incremental builds against the tip of that development branch remain snappy.

  • It pollutes the git history

    If you bump the lockfile, say, once per day then that’s one junk commit that you’ve added to your git history every day.

  • It’s difficult to open source any useful automation around this

    If the solution requires out-of-band machinery (e.g. some recurring cron job) to bump the lockfile you can’t provide a great user experience for open source projects. It only really works well for proprietary projects that can tolerate that complexity.

That last point was the most important one for me. Generally, when I design something (even something intended for internal, proprietary use) I try to design it in such a way that it works well in an open source context, too. In my experience, doing things in this way tends to improve the design, quality, and user experience of software that I build.

In particular, I wanted a solution where all the automation could be implemented entirely within the Nix language. However, this is not possible in Nix’s present form!

Non-solution #2: Rollback derivation

So what I really wanted was a Nix function (which I will call “truncate”) that would take any git repository and roll it back in time to the last commit before some repeating time boundary (where the time boundary might be, say, an hour, or day, or week). For simplicity, let’s just say that the desired time interval is one day so I want to roll back the repository to the last revision from the day before.

If I had such a truncate function then it would be easy to automatically select which revision to use for the full build. I would:

  • extract the source git repository from the current Haskell package build

  • truncate that git repository to the last revision from the day before

  • Use that “truncated” revision as the source for the full build

  • Use that full build as the input to the current (incremental) build

Then if I built multiple revisions for the same day they would all share the same full build since they would all get “truncated” to the same revision from the previous day.

However, there isn’t a great way to implement this truncate function in Nix. To see why, consider the following (wrong) solution:

  • extract the source git repository from the current Haskell package build

    Let’s call the derivation for this git repository “src

  • create a new Nix derivation (“src2”) that rolls back src

    In other words, this would be a trivial Nix derivation that begins from src and runs something like:

    $ git checkout $(git rev-list -1 --before '1 day ago' HEAD)

    … and stores that as the result

  • Use src2 as the input to the full build

Do you see the problem with that approach?

The above wrong solution doesn’t allow multiple incremental builds from the same day to share the same full build from the prior day. This is because src2 depends on src and since each incremental build has a different src repository then each also have a different src2 derivation and therefore a different full build. That in turn defeats the purpose of incremental builds if we have to do a new full rebuild for each incremental build.

For this to work we would need a way to roll back a git repository to an older revision that less sensitive to the current revision.

Non-solution #3: Plain fetchGit

The builtins.fetchGit utility almost does what we want! This primitive function lets you fetch a git repository at evaluation time, like this:

nix-repl> builtins.fetchGit { url = ~/proj/turtle; revision = "837f52d2101368bc075d382774460a717904d2ab"; }
{ lastModified = 1655501878; lastModifiedDate = "20220617213758"; narHash = "sha256-Ic4N2gzm0hYsPCynkzETJv7lpAWO1KM+FO+r3ov60y0="; outPath = "/nix/store/ygznanxv6rmbxw5gkgk7axfxazhsa93z-source"; rev = "837f52d2101368bc075d382774460a717904d2ab"; revCount = 566; shortRev = "837f52d"; submodules = false; }

The above result is the same no matter what revision I currently have checked out at ~/proj/turtle because Nix’s fetchGit function produces a content-addressed derivation. In other words, if two invocations of fetchGit generate the same final repository state then they share the same outPath. This is exactly the behavior we want: we need the source repository for the full build to be content-addressed so that multiple incremental builds can share the same full build.

However, the problem is that I don’t exactly know which revision I want. What I really want to be able to say is “get me the last revision from the day before this other revision”. fetchGit does not expose any way to do something like that.

That brings us to the actual solution:

Solution

The solution I went with was the following two pull requests:

  • Add optional date argument to builtins.fetchGit

    This amends builtins.fetchGit to allow a date specification, which can either be a relative date (e.g. 1 day ago) or an absolute date (e.g. 2020-01-01T00:00:00 or a Unix timestamp like 1671388622). Basically, this argument accepts anything git accepts as a date specification (which is a lot since git is pretty flexible in this regard).

    The cool thing about this change is that it doesn’t compromise the purity of builtins.fetchGit. If a given fetchGit specification was pure then adding a date specification preserves that purity.

  • Add haskell.lib.incremental utility

    This pull request actually does two separate things:

    • This polishes and upstreams the prototype support for incremental builds

      In other words, this upstreams Harry and Jade’s work to split a Haskell build into two builds: a full build and incremental build

    • This uses the fetchGit patch to automate the full build selection

      There’s a new pkgs.haskell.lib.incremental utility which uses builtins.fetchGit to automatically update the full build for you and it has all the desired behaviors (including branch-local incrementalism).

    I could have split this into two separate pull request (and I still might) but for internal testing purposes it was easier to do everything on one branch. I’m waiting for a decision on the other pull request before deciding whether or not to split up this branch.

Demo

I’ll use my turtle package as the running example for the demo. If you clone the gabriella/incremental branch of my turtle repository:

$ git clone --branch gabriella/incremental \
    https://github.com/Gabriella439/turtle.git
$ cd turtle

… you’ll find the following default.nix file making use of the Nixpkgs support for incremental Haskell builds:

{ interval ? 24 * 60 * 60 }:

let
  nixpkgs = builtins.fetchTarball {
    url    = "https://github.com/MercuryTechnologies/nixpkgs/archive/696e0820b03e8ea7ad6a9ba21a00a79c91efc580.tar.gz";
    sha256 = "1k3swii3absl154154lmk6zjw11vzzqx8skaiw1250armgfyv9v8";
  };

  # We need GHC 9.4 or newer for this feature to work
  compiler ="ghc94";

  overlay = self: super: {
    haskell = super.haskell // {
      packages = super.haskell.packages // {
        "${compiler}" =
          super.haskell.packages."${compiler}".override (old: {
            overrides =
              self.lib.fold
                self.lib.composeExtensions
                (old.overrides or (_: _: { }))
                [ (self.haskell.lib.packageSourceOverrides {
                    turtle = ./.;
                  })

                  (hself: hsuper: {
                    turtle-incremental =
                      self.haskell.lib.compose.incremental
                        { inherit interval;

                          makePreviousBuild =
                            truncate: (import (truncate ./.) { }).turtle;
                        }
                        hsuper.turtle;
                  })
                ];
          });
      };
    };
  };

  pkgs = import nixpkgs { config = { }; overlays = [ overlay ]; };

in
  { inherit (pkgs.haskell.packages."${compiler}")
      turtle
      turtle-incremental
    ;
  }

However, that alone is not enough to make use of incremental builds. If you attempt to build that (at the time of this writing) you’ll get an error message like this:

$ nix build --file ./default.nix turtle-incremental
error: evaluation aborted with the following error message:
'pkgs.haskell.lib.incremental requires Nix version 2.12.0pre20221128_32c182b or
newer'
(use '--show-trace' to show detailed location information)

The Nixpkgs support for incremental builds depends on a matching change to the Nix interpreter, so you actually have to run:

$ nix run github:Gabriella439/nix/gabriella/fetchGit -- \
    build --file ./default.nix turtle-incremental

… or if you don’t yet have flakes enabled, then use this pedantically complete command:

$ nix --option extra-experimental-features 'nix-command flakes' \
    run github:Gabriella439/nix/gabriella/fetchGit -- \
    build --file ./default.nix turtle-incremental

… and that will definitely work.

Once the build is complete you can inspect the logs and you should see something like the following buildPhase:

$ nix log ./result
…
@nix { "action": "setPhase", "phase": "buildPhase" }
building
Preprocessing library for turtle-1.6.1..
Building library for turtle-1.6.1..
Preprocessing test suite 'regression-broken-pipe' for turtle-1.6.1..
Building test suite 'regression-broken-pipe' for turtle-1.6.1..
[2 of 2] Linking dist/build/regression-broken-pipe/regression-broken-pipe [Libr>
Preprocessing test suite 'regression-masking-exception' for turtle-1.6.1..
Building test suite 'regression-masking-exception' for turtle-1.6.1..
[2 of 2] Linking dist/build/regression-masking-exception/regression-masking-exc>
Preprocessing test suite 'tests' for turtle-1.6.1..
Building test suite 'tests' for turtle-1.6.1..
[2 of 2] Linking dist/build/tests/tests [Library changed]
Preprocessing test suite 'system-filepath-tests' for turtle-1.6.1..
Building test suite 'system-filepath-tests' for turtle-1.6.1..
[2 of 2] Linking dist/build/system-filepath-tests/system-filepath-tests [Librar>
Preprocessing test suite 'cptree' for turtle-1.6.1..
Building test suite 'cptree' for turtle-1.6.1..
[2 of 2] Linking dist/build/cptree/cptree [Library changed]
…

This is shows that the incremental builds are indeed working. We still have to re-link some executables (for reasons that are still not clear to me), but none of the Haskell modules needed to be rebuilt since nothing has changed (yet) since the last rebuild.

Now let’s test that by making a small whitespace change to one of the Turtle modules:

$ echo >> src/Turtle/Prelude.hs 

Then if we rebuild the package we’ll see the following build phase:

$ nix --option extra-experimental-features 'nix-command flakes' \
    run github:Gabriella439/nix/gabriella/fetchGit -- \
    build --file ./default.nix --print-build-logs
…
turtle> building
turtle> Preprocessing library for turtle-1.6.1..
turtle> Building library for turtle-1.6.1..
turtle> [ 7 of 10] Compiling Turtle.Prelude   ( src/Turtle/Prelude.hs, dist/build/Turtle/Prelude.o, dist/build/Turtle/Prelude.dyn_o ) [Source file changed]
turtle> src/Turtle/Prelude.hs:319:1: warning: [-Wunused-imports]
turtle>     The import of ‘Data.Monoid’ is redundant
turtle>       except perhaps to import instances from ‘Data.Monoid’
turtle>     To import instances alone, use: import Data.Monoid()
turtle>     |
turtle> 319 | import Data.Monoid ((<>))
turtle>     | ^^^^^^^^^^^^^^^^^^^^^^^^^
turtle> Preprocessing test suite 'regression-broken-pipe' for turtle-1.6.1..
turtle> Building test suite 'regression-broken-pipe' for turtle-1.6.1..
turtle> [2 of 2] Linking dist/build/regression-broken-pipe/regression-broken-pipe [Library changed]
turtle> Preprocessing test suite 'regression-masking-exception' for turtle-1.6.1..
turtle> Building test suite 'regression-masking-exception' for turtle-1.6.1..
turtle> [2 of 2] Linking dist/build/regression-masking-exception/regression-masking-exception [Library changed]
turtle> Preprocessing test suite 'tests' for turtle-1.6.1..
turtle> Building test suite 'tests' for turtle-1.6.1..
turtle> [2 of 2] Linking dist/build/tests/tests [Library changed]
turtle> Preprocessing test suite 'system-filepath-tests' for turtle-1.6.1..
turtle> Building test suite 'system-filepath-tests' for turtle-1.6.1..
turtle> [2 of 2] Linking dist/build/system-filepath-tests/system-filepath-tests [Library changed]
turtle> Preprocessing test suite 'cptree' for turtle-1.6.1..
turtle> Building test suite 'cptree' for turtle-1.6.1..
turtle> [2 of 2] Linking dist/build/cptree/cptree [Library changed]
…

Our package only built the “diff” (the Turtle.Prelude module we just changed)!

Benchmarks

For the turtle package the speed-up is not a huge deal because the package doesn’t take long time to compile, but the benefit for our main project at work is dramatic!

As I mentioned in the introduction, our work project normally takes ~14 minutes to build and after this change builds can be as fast as ~3.5 minutes. In fact, they could even be faster except for the presence of a Paths_* module that is rebuilt each time and triggers a large number of gratuitous downstream rebuilds (we’re working on fixing that).

Limitations

There is one major issue with this work, which is that it does not work well with flakes.

Specifically, if you try to turn the above default.nix into the equivalent flake the build will fail because Nix’s flake mechanism will copy the project into the /nix/store but without the .git history, so builtins.fetchGit will fail to to fetch the current repository’s history necessary to truncate the build to the previous day.

I believe this can be fixed with a change to flakes to support something like a ?shallow=false or ?allRefs=true addendum to git URLs, but I have not implemented that, yet.

Monday, October 24, 2022

How to correctly cache build-time dependencies using Nix

caching

Professional Nix users often create a shared cache of Nix build products so that they can reuse build products created by continuous integration (CI). For example, CI might build Nix products for each main development branch of their project or even for every pull request and it would be nice if those build products could be shared with all developers via a cache.

However, uploading build products to a cache is a little non-trivial if you don’t already know the “best” solution, which is the subject of this post.

The solution described in this post is:

  • Simple

    It only takes a few lines of Bash code because we use the Nix command-line interface idiomatically

  • Efficient

    It is very cheap to compute which build products to upload and requires no additional builds nor an exorbitant amount of disk space

  • Accurate

    It uploads the build products that most people would intuitively want to upload

Note: Throughout this post I will be using the newer Nix command-line interface and flakes, which requires either adding this line to your nix.conf file:

extra-experimental-features = nix-command flakes

… and restarting your Nix daemon (if you have a multi-user Nix installation), or alternatively adding these flags to the beginning of all nix commands throughout this post:

$ nix --option extra-experimental-features 'nix-command flakes'

Wrong solution #0

As a running example, suppose that our CI builds a top-level build product using a command like this:

$ nix build .#example

The naïve way to upload that to the cache would be:

$ nix store sign --key-file "${KEY_FILE}" --recursive .#example

$ nix copy --to s3://cache.example.com .#example

Note: You will need to generate a KEY_FILE using the nix-store --generate-binary-cache-key command if you haven’t already. For more details, see the following documentation from the manual:

Click to expand to see the documentation
Operation --generate-binary-cache-key
   Synopsis
       nix-store --generate-binary-cache-key key-name secret-key-file
       public-key-file

   Description
       This command generates an Ed25519 key pair (http://ed25519.cr.yp.to/)
       that can be used to create a signed binary cache. It takes three
       mandatory parameters:

       1.     A key name, such as cache.example.org-1, that is used to look up
              keys on the client when it verifies signatures. It can be
              anything, but it’s suggested to use the host name of your cache
              (e.g.  cache.example.org) with a suffix denoting the number of the
              key (to be incremented every time you need to revoke a key).

       2.     The file name where the secret key is to be stored.

       3.     The file name where the public key is to be stored.

That seems like a perfectly reasonable thing to do, right? However, the problem with that is that it is incomplete, meaning that the cache would still be missing several useful build products that developers would expect to be there.

Specifically, the above command only copies the “run-time” dependencies of our build product whereas most developers expect the cache to also include “build-time” dependencies, and I’ll explain the distinction between the two.

Run-time vs. Build-time

Many paths in the /nix/store are not “valid” in isolation. They typically depend on other paths within the /nix/store.

For example, suppose that I build the GNU hello package, like this:

$ nix build nixpkgs#hello

I can query all of the other paths within the /nix/store that the hello package transitively depends on at run-time using this command:

$ nix-store --query --requisites ./result
/nix/store/jzid7pfrhv6gpiwqbx6763v0g9c3bdzb-libobjc-11.0.0
/nix/store/9rb5qaba71mkgfgd8wfqg03cmi46xarg-apple-framework-CoreFoundation-11.0>
/nix/store/akjp4x41jjx5hzgzrschwqzr8qfsdpys-hello-2.12.1

… or I can print the same information in tree form like this:

$ nix-store --query --tree ./result
/nix/store/akjp4x41jjx5hzgzrschwqzr8qfsdpys-hello-2.12.1
└───/nix/store/9rb5qaba71mkgfgd8wfqg03cmi46xarg-apple-framework-CoreFoundation-11.0.0
    └───/nix/store/jzid7pfrhv6gpiwqbx6763v0g9c3bdzb-libobjc-11.0.0

On my macOS machine, it has two run-time dependencies (other than itself) within the /nix/store: libobjc and apple-framework-CoreFoundation-11.0.

Note: there might be other run-time dependencies, because I believe Nixpkgs support for macOS requires some impure system dependencies, but I’m not an expert on this so I could be wrong.

These are called “run-time” dependencies because we cannot run our hello executable without them.

Nix prevents us from getting into situations where a /nix/store path is missing its run-time dependencies. For example, if I were to nix copy the hello build product to any cache, then Nix would perform the following steps, in order:

  • Copy libobjc to the cache

    … since that has no dependencies

  • Copy apple-framework-CoreFoundation to the cache

    … since its libobjc dependency is now satisfied within the cache

  • Copy hello to the cache

    … since its apple-framework-CoreFoundation dependency is now satisfied within the cache

However, Nix also has a separate notion of “build-time” dependencies, which are dependencies that we need to in order to build the hello package.

Note: The reason we’re interested in build-time dependencies for our project is that we want developers to be able to rebuild the project if they make any changes to the source code. If we were to only cache the run-time dependencies of our project that wouldn’t cache the development environment that developers need.

In order to query these dependencies I need to first get the “derivation” (.drv file) for hello:

$ DERIVATION="$(nix path-info --derivation nixpkgs#hello)"

$ declare -p DERIVATION
typeset DERIVATION=/nix/store/4a78f0s4p5h2sbcrrzayl5xas2i7zq1m-hello-2.12.1.drv

You can think of a derivation file as a build recipe that contains instructions for how to build the corresponding build product (the hello package in this case).

I can query the direct dependencies of that derivation using this command:

$ nix-store --query --references "${DERIVATION}"
/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh
/nix/store/labgzlb16svs1z7z9a6f49b5zi8hb11s-bash-5.1-p16.drv
/nix/store/cdk3pz11mvhqpphr0197wwmzhqppn7rl-stdenv-darwin.drv
/nix/store/hwymznwkd1kgf5ldcldjl9bnc1wz2azb-hello-2.12.1.tar.gz.drv

Many of these dependencies are themselves derivations (.drv files), meaning that they represent other packages that Nix might have to build or fetch from a cache.

Note: the .drv files are actually not the build-time dependencies, but rather the instructions for building them. You can convert any .drv file to the matching product it is supposed to build using the same nix build command, like this:

$ nix build /nix/store/labgzlb16svs1z7z9a6f49b5zi8hb11s-bash-5.1-p16.drv

Does that mean that these build-time dependencies are on our machine if we built nixpkgs#hello? Not necessarily. In fact, in all likelihood the nixpkgs#hello build was cached, meaning that nix build nixpkgs#hello only downloaded hello and its run-time dependencies and no build-time dependencies were required nor installed by Nix.

However, I could in principle force Nix to build the hello package instead of downloading it from a cache, like this:

$ nix build nixpkgs#hello --rebuild

… and that would download the direct build-time dependencies of the hello package in order to rebuild the package.

Wrong solution #1

By this point you might suppose that you have enough information to come up with a better set of /nix/store paths to cache. Your solution might look like this:

  • Get the derivation for the top-level build product

  • Get the direct build-time dependencies of that derivation

  • Build the top-level build product and its direct build-time dependencies

  • Cache the top-level build product and its direct build-time dependencies

In other words, something like this Nix code:

$ DERIVATION="$(nix path-info --derivation "${BUILD}")"

$ DEPENDENCIES=($(nix-store --query --references "${DERIVATION}"))

$ nix build "${BUILD}" "${DEPENDENCIES[@]}"

$ nix store sign --key-file "${KEY_FILE}" --recursive "${BUILD}" "${DEPENDENCIES[@]}"

$ nix copy --to "${CACHE}" "${BUILD}" "${DEPENDENCIES[@]}"

This is better, but still not good enough!

The problem with this solution is that it only works well if your dependencies never change and you only modify your top-level project. If you upgrade or patch any of your direct build-time dependencies then you need to have their build-time dependencies cached so that you can quickly rebuild them.

In fact, going two layers deep is still not enough; in practice you can’t easily anticipate in advance how deep in the build-time dependency tree you might need to patch or upgrade things. For example, you might need to patch or upgrade your compiler, which is really deep in your build-time dependency tree.

Wrong solution #2

Okay, so maybe we can try to build and cache all of our build-time dependencies?

Wrong again. There are way too many of them. You can query them by replacing --references with --requisites and you’ll a giant list of results, even for “small” packages. For example:

$ DERIVATION=$(nix path-info --derivation nixpkgs#hello)

$ nix-store --query --requisites "${DERIVATION}"
/nix/store/8djp1rizc1dblv8svnb0mpa0c3lwvc17-drop-comments.patch
/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh
/nix/store/3glray2y14jpk1h6i599py7jdn3j2vns-mkdir.drv
/nix/store/50ql5q0raqkcydmpi6wqvnhs9hpdgg5f-cpio.drv
/nix/store/81xahsrhpn9mbaslgi5sz7gsqra747d4-unpack-bootstrap-tools-aarch64.sh
/nix/store/fzbk4fnbjqhr0l1scx5fspsx5najbrbm-bootstrap-tools.cpio.bz2.drv
/nix/store/gxzl4vmccqj89yh7kz62frkxzgdpkxmp-sh.drv
 🌺 500+ derivations later 🌺 …
/nix/store/i0zc5mm4vpj3lviyydb9s73j53mypkrg-nghttp2-1.49.0.drv
/nix/store/w37b5s734m53gxnzqyb5v0v98mhdfg2i-coreutils-9.1.drv
/nix/store/mmsrbggvzn6rwlx1ijw90sw3wvhzj18j-openssl-3.0.5.drv
/nix/store/n7iibs6b818v402j0yczf4mgy73sbzpv-libssh2-1.10.0.drv
/nix/store/z074ki54p77r7db3wsgxh9p18f67xnv8-curl-7.85.0.drv
/nix/store/hwymznwkd1kgf5ldcldjl9bnc1wz2azb-hello-2.12.1.tar.gz.drv
/nix/store/4a78f0s4p5h2sbcrrzayl5xas2i7zq1m-hello-2.12.1.drv
Click to expand and see the full list of build-time dependencies
/nix/store/8djp1rizc1dblv8svnb0mpa0c3lwvc17-drop-comments.patch
/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh
/nix/store/3glray2y14jpk1h6i599py7jdn3j2vns-mkdir.drv
/nix/store/50ql5q0raqkcydmpi6wqvnhs9hpdgg5f-cpio.drv
/nix/store/81xahsrhpn9mbaslgi5sz7gsqra747d4-unpack-bootstrap-tools-aarch64.sh
/nix/store/fzbk4fnbjqhr0l1scx5fspsx5najbrbm-bootstrap-tools.cpio.bz2.drv
/nix/store/gxzl4vmccqj89yh7kz62frkxzgdpkxmp-sh.drv
/nix/store/pjbpvdy0gais8nc4sj3kwpniq8mgkb42-bzip2.drv
/nix/store/7kcayxwk8khycxw1agmcyfm9vpsqpw4s-bootstrap-tools.drv
/nix/store/1i5y55x4b4m9qkx5dqbmr1r6bvrqbanw-multiple-outputs.sh
/nix/store/59jmzisg8fkm9c125fw384dqq1np602l-move-docs.sh
/nix/store/bnj8d7mvbkg3vdb07yz74yhl3g107qq5-patch-shebangs.sh
/nix/store/cickvswrvann041nqxb0rxilc46svw1n-prune-libtool-files.sh
/nix/store/ckzrg0f0bdyx8rf703nc61r3hz5yys9q-builder.sh
/nix/store/fyaryjvghbkpfnsyw97hb3lyb37s1pd6-move-lib64.sh
/nix/store/g8xg0i02aqwhgxwd2vnp5ax3d6lrkg1v-strip.sh
/nix/store/jngr4r80x5jn482ckqrfh08ljrx1k86f-setup.sh
/nix/store/kd4xwxjpjxi71jkm6ka0np72if9rm3y0-move-sbin.sh
/nix/store/kxw6q8v6isaqjm702d71n2421cxamq68-make-symlinks-relative.sh
/nix/store/m54bmrhj6fqz8nds5zcj97w9s9bckc9v-compress-man-pages.sh
/nix/store/ngg1cv31c8c7bcm2n8ww4g06nq7s4zhm-set-source-date-epoch-to-latest.sh
/nix/store/wlwcf1nw2b21m4gghj70hbg1v7x53ld8-reproducible-builds.sh
/nix/store/nbxwxwqwcr9rrmxb6gb532f18102815x-bootstrap-stage0-stdenv-darwin.drv
/nix/store/ycwm35msmsdi2qgjax1slmjffsmwy8am-write-mirror-list.sh
/nix/store/i65va14cylqc74y80ksgnrsaixk39mmh-mirrors-list.drv
/nix/store/lphxcbw5wqsjskipaw1fb8lcf6pm6ri6-builder.sh
/nix/store/bgp77z9z42x35vmwyfywqaiqsmnb3ffa-patchutils-0.3.3.tar.xz.drv
/nix/store/3lhw0v2wyzimzl96xfsk6psfmzh38irh-bash51-007.drv
/nix/store/3p62kw9mpkcp0grhirfn46i9afhqf0c9-bash51-015.drv
/nix/store/3za6mykjk49sr616w80lvmy5xcmbkrp3-bash51-006.drv
/nix/store/5lv0fbn6ajwdzw04nz88cc0qqrgrvnp1-bash51-016.drv
/nix/store/4dq81yma6drk9701h17h64zx47r7p5n8-utils.sh
/nix/store/ds0q1li2i96dy7yp6n8zkbakz7m7d5l8-bootstrap-stage0-stdenv-darwin.drv
/nix/store/vcbpmcxpir9nyy480vx5sxb4pm2v0dps-bootstrap-stage0-sigtool.drv
/nix/store/5xk8j72a1dddq1gxh39amgifknwb0lvm-signing-utils.drv
/nix/store/fmd71yqpgwrkdbidzxwmfasqp39zzf8r-CLTools_macOSNMOS_SDK.pkg.drv
/nix/store/ijdrwgv6kv4k17qb5jvqkbs1ccqrlalb-bootstrap-stage0-pbzx.drv
/nix/store/z727aawh6imz3hpmviqyf4zlgprfn8zf-bootstrap-stage0-cpio.drv
/nix/store/fbhkqzn4wc69pa4rvndin5h4b8k02f5c-MacOSX-SDK-11.0.0.drv
/nix/store/72fl7wcbnl6srm5dxb7xbnn20zr07011-libobjc-11.0.0.drv
/nix/store/93n8xmgm6x65mh88amw15wps3h18yqn8-bootstrap-stage0-libcxx.drv
/nix/store/cyk47lyfswsxgn7z2qnhkp0919nhdd3b-config.sub?id=6faca61810d335c7837f320733fe8e15a1431fc2.drv
/nix/store/lmdwlh09b7g3kzga0i2hprcfxszg3ghz-config.guess?id=6faca61810d335c7837f320733fe8e15a1431fc2.drv
/nix/store/9vh7qbpb8chmx5r5cswrix00hngy7vky-gnu-config-2021-01-25.drv
/nix/store/2idjp2xdckqyrdk3hd8msp4cfdi6b8if-utils.bash
/nix/store/47gspvwaxra868q4rsmva0g5nz7zac6p-add-clang-cc-cflags-before.sh
/nix/store/797k3249lr4rx8wslf7kcsa3hv6fis3x-role.bash
/nix/store/bybz12bxjbk94hm58zc70sc0xhj2dxif-add-darwin-ldflags-before.sh
/nix/store/c1vmxz359mfljs3cdimhd2fr6fw0n99s-add-hardening.sh
/nix/store/civvq4xh4p0mj04l7k73p1xbsq1rs9bc-darwin-install_name_tool-wrapper.sh
/nix/store/dk3ly72kiv27srcj1mjr5n4112vx2hfk-setup-hook.sh
/nix/store/khkfggnk4h14f2spsjcjrxlf8himd4vj-bootstrap-stage0-rewrite-tbd.drv
/nix/store/ff3yqdllxmlp1r8mfkfgjf671r8flf8j-libSystem-11.0.0.drv
/nix/store/gvr0mz9wfz4g0z9w6477ikywmcivk1mh-bootstrap-stage0-coreutils.drv
/nix/store/jravmbdjn0md7cnn6rbqwf3aflw72msb-post-link-sign-hook.drv
/nix/store/js1lic1bmif60d4mlffkpbvz5jim34n3-darwin-strip-wrapper.sh
/nix/store/m0ww06j4y2v3jdsabrr8n0y9d4nnvqn3-bootstrap-stage0-gnugrep.drv
/nix/store/mrzpfh0ml9k07sw019ydagbb2z1q4sxz-add-flags.sh
/nix/store/vicx6qnvvxb96y0iwnya977viira2apc-ld-wrapper.sh
/nix/store/dj5xngrf61x9isyp2r5d84h3i43wg65l-bootstrap-stage0-binutils-wrapper-.drv
/nix/store/gn1b4gh07byi8xnahgc800jznyarqin0-bootstrap-stage0-clang.drv
/nix/store/hc3z14jhqg35x5286hw5wmc3injg62fa-add-hardening.sh
/nix/store/ji2yrl1na00bwav65hh1vr0nc0s1xzvz-add-flags.sh
/nix/store/lck6bijpq64zjvmkwdi081v6wm2r8nyx-bootstrap-stage0-libcxxabi.drv
/nix/store/li62b4bvg51zikbni9xadg08za340k71-cc-wrapper.sh
/nix/store/nc4bvrgb8jxh4k1fq2zgzh4mmxqavp54-setup-hook.sh
/nix/store/xbfsjb46np040h2aph1k76iybq9rzd7x-bootstrap-stage0-compiler-rt.drv
/nix/store/cz1x2bgvnzi0qc39hjwm0ppdvqwkfybl-bootstrap-stage0-clang-wrapper-11.1.0.drv
/nix/store/00qr10y7z2fcvrp9b2m46710nkjvj55z-update-autotools-gnu-config-scripts.sh
/nix/store/qzvw98z9qwv3vasfc9lwcq0d1sgfin6v-hook.drv
/nix/store/vp49i9krzqf282vj6bqr6rxs96d2a1fv-apple-framework-CoreFoundation-11.0.0.drv
/nix/store/lkjwmgmnm4f7d3iiglxglrgll1c8vdkf-bootstrap-stage1-stdenv-darwin.drv
/nix/store/r09r8wpshjqdkx0xwkin79km812nbgp3-m4-1.4.19.tar.bz2.drv
/nix/store/7rxh71ny4xrckw1ip50hv44mylpxpd5v-gnum4-1.4.19.drv
/nix/store/h1s43lrwjjf98dmfhayn6cmax2h19qz9-bison-3.8.2.tar.gz.drv
/nix/store/85f1q5rm0qzqay7fk3935h6kvzfqdcfv-http-tiny-verify-ssl-by-default.patch
/nix/store/dm81j9qdcdr4c458pqbc9wvq9ymgzk4m-setup-hook.sh
/nix/store/mb381gpm9k5wdl37l5ad5pp8w11qzhjg-no-sys-dirs-5.31.patch
/nix/store/n315a3g9bcxlypqzbm36nzrrg40h6lcj-cpp-precomp.patch
/nix/store/ppbkcbhzwzwpqaad6whhr9qgxjvj82gj-perl-5.36.0.tar.gz.drv
/nix/store/0phjl3yzr4m1gady21l21h4brn140hjm-CVE-2022-37434.patch
/nix/store/jw5f744vn0nv6q0ms6s5q0w0kkml2a8v-fix-configure-issue-cross.patch
/nix/store/mg7z3nsc96864cfkx2jwda7wmpdn71dl-zlib-1.2.12.tar.gz.drv
/nix/store/xniqbx8mh8lx06bw267g6hkfkp2c79kn-comprehensive-crc-validation-for-wrong-implementations.patch
/nix/store/vblzwf77rk1mxxk6qjhgaqp43z36j992-zlib-1.2.12.drv
/nix/store/x0ll5lnagy6lg7hgvqb26il3qnmsqisd-sw_vers.patch
/nix/store/m9h3rw3jlzf8hapgiw29i5fjyzw7r9v4-perl-5.36.0.drv
/nix/store/5nxxxmzb23y9pzp673grmfs95jrx14rx-bison-3.8.2.drv
/nix/store/13i5k09s5g6gdkaj731rqsi1qc70khka-bootstrap-stage1-stdenv-darwin.drv
/nix/store/jik02mkz72r2f6hhxnlhp6h5f0fi89gw-expand-response-params.c
/nix/store/g9h3q3y44p4ycn8vdai1s9aw1f0s7icy-expand-response-params.drv
/nix/store/734l1nlc4d2zbksafqvp5436ynp26z3g-bash51-014.drv
/nix/store/crar8b49gsplly9x0v279bibvixmj9gc-bash51-003.drv
/nix/store/d7i8a7c8mb0qmsb1c3rj8g38m071vmaf-bash51-004.drv
/nix/store/gapa7v6rz1gzjsn1kckpb4fj07pmznw6-bash-5.1.tar.gz.drv
/nix/store/ig7d802vp43ap4ga26r2vd174h2c1bk7-bash51-010.drv
/nix/store/jj3jm1bjqnlwz4yassc9h70mwmrxy442-bash51-008.drv
/nix/store/krhwn1440gxvry5gw4wmzk6y83fm4bdw-bash51-001.drv
/nix/store/n92wsf556wqp4dbf0rxwx1b175m0vwyn-bash51-013.drv
/nix/store/nkzvvyvmckwv4a8r84iwyfbivvw05nvc-bash51-009.drv
/nix/store/nm5946y3ffkynrz67vz47ik0ygvg6byn-pgrp-pipe-5.1.patch
/nix/store/pkd2vxq8w5fhrfk6k89hi29d5ldbbj29-bash51-005.drv
/nix/store/q6jpdv2j5dh4nbsbmkzy6za1vfpb2l30-bash51-011.drv
/nix/store/w8q9jdvb96sayxyi25amd27yki283zg9-bash51-002.drv
/nix/store/wg9gacyfgldnwh3gc23nr49n4j0b58sv-bash51-012.drv
/nix/store/4kpfa2fhfdb6r9icym3q8n3w1s8mfv6z-bash-5.1-p16.drv
/nix/store/k9bkzj7nhqxnvbmz5vk8pmzxpmym0qa6-file-5.43.tar.gz.drv
/nix/store/hgps4nk9p01z9zngmbnbv42pqmzg9nhg-file-5.43.drv
/nix/store/4cmjzk8yr6i5vls5d2050p653zzdvmvp-setup-hook.sh
/nix/store/4hcdpxjmr4nh625ry840g70xp00vdf5a-2.71-fix-race.patch
/nix/store/kxdvjnq8cl9jd5g9jndhhl5a17h2xbih-autoconf-2.71.tar.xz.drv
/nix/store/irpqw60zcy00lj3hjia8gr15z984x9xn-texinfo-6.8.tar.xz.drv
/nix/store/19ndr8g629l0xzzs1x7xz4z0fbkwpgcj-xz-5.2.6.tar.bz2.drv
/nix/store/kas3n4g0jyrz3rhl1znjcxqmpyddk8sw-xz-5.2.6.drv
/nix/store/5pj1w63j668yqssrxb4dykpzpm1yhx9q-libiconv-50.tar.gz.drv
/nix/store/xrvbfm0ahaiik1vlxzn3vsp5gyhksy2n-setup-hook.sh
/nix/store/5xx3gba361rf696b27r3jfa0q7rmlyh9-libiconv-50.drv
/nix/store/ny42y6hs4p294rvnrwbmrpwzqghw2816-gettext-setup-hook.sh
/nix/store/p2fp6i7hjx9af1wbwr32k217wp2dxmiw-absolute-paths.diff
/nix/store/sihz0cdcajckxnhjm8n4p652sbd5xwxd-gettext-0.21.tar.gz.drv
/nix/store/ri729qz1iq3iqr9bdvb406izklj1fpfa-gettext-0.21.drv
/nix/store/yl6g04fsr6p9lkvccqjjl8q1xnnmw49s-fix-glibc-2.34.patch
/nix/store/v94cwmp6hs6dvrw6v4jmqk9jwl0ik75q-texinfo-6.8.drv
/nix/store/ih062232k706fpydng4xlk75fpzraxmn-autoconf-2.71.drv
/nix/store/r5kd1di71n7xk9kjvrgyy75cz0c0skay-automake-1.16.5.tar.xz.drv
/nix/store/hqf9nllss9z0i3gmi8sv9kjcm9qhvf8q-automake-1.16.5.drv
/nix/store/ipdgmqcfn56mhgmq3hv3yy5xj2kj2ri7-libtool-2.4.7.tar.gz.drv
/nix/store/06nfwja5j8c36n780jvlwjv8gs2m7i3m-gettext-1.07.tar.gz.drv
/nix/store/qf3mzpvsmkrw963xchbivcci06078n13-builder.sh
/nix/store/j7jrmh8zz3jjcdsa050f1ja19nm07vz7-perl5.36.0-gettext-1.07.drv
/nix/store/lac4y6zxaqqhyf4r60p3ag68k3ckifvv-help2man-1.49.2.tar.xz.drv
/nix/store/k9223a4j2ar7j76zpjs8fmlfl9q76a76-help2man-1.49.2.drv
/nix/store/9gns0vpn7fajyzm3w2rsdbcz9pqgfsrm-libtool-2.4.7.drv
/nix/store/ghdamd4hl6yi7jysh1x3436fj1v9yvjb-autoreconf.sh
/nix/store/8byqd66k9p5zbiggz2a9frki47xqy28r-hook.drv
/nix/store/ing5qg4q87wdvm1h455c3xn889m2bbj0-coreutils-9.1.tar.xz.drv
/nix/store/66ayp7hm682rxhlxd061fvprrmf2zx3c-gmp-6.2.1.tar.bz2.drv
/nix/store/h96aykl4imzhfapnmavplr49v43c8apx-6.2.1-CVE-2021-43618.patch
/nix/store/k49s9lr0lw5zhwsz4ni4rf2643byqrak-gmp-with-cxx-6.2.1.drv
/nix/store/jp5jbqfxjazf38w63bipqf68h7hyq4v0-coreutils-9.1.drv
/nix/store/322332kbmj7ig7ii3cwlfjcg4mf5xgz6-grep-3.7.tar.xz.drv
/nix/store/m2qc8a9c4yr5xmqck50fpzzwzpqggbbw-stacksize-detection.patch
/nix/store/qk3pnajspb378zx6c8g0anj2d7z31a88-pcre-8.45.tar.bz2.drv
/nix/store/pgzgn3knxdn335qci805a49bvlnl4ppa-pcre-8.45.drv
/nix/store/l2bbsavfww8zv5a9ncldh83c9vsz3y01-gnugrep-3.7.drv
/nix/store/1igs2sr5j99180z801426ivzzbikxi7q-CVE-2019-13232-2.patch.drv
/nix/store/1k1wn8807yizgz3ghnbd4k6zsc0dzfkr-CVE-2014-9913.patch
/nix/store/2cq4hsc1v8ylccspw8351r72s56w1fia-CVE-2015-7697.diff
/nix/store/6np2acjv1nxcg0xzsv9a76wyrpxznkna-CVE-2014-8141.diff
/nix/store/6zqn6w9rwkgfa6z1hpagnh5xhz2dag6m-CVE-2015-7696.diff
/nix/store/97d26l91h0db8h0qkmhxwi5d8shrilv6-CVE-2016-9844.patch
/nix/store/p67r2s04xw8plqs0552rc5qyj5016wxb-bzip2-1.0.8.tar.gz.drv
/nix/store/ra4ikm6b0nxgi0sil4mk3ikkingm80x0-bzip2-1.0.6.2-autoconfiscated.patch.drv
/nix/store/awrvprhqr5j0n6n8zqss2i5cyv25wis3-bzip2-1.0.8.drv
/nix/store/bqwhj80hz1z49365miaqcxp4c2mzsfp3-unzip60.tar.gz.drv
/nix/store/cciw7lgkldvx25d77cxpjhh1iw4xghd9-setup-hook.sh
/nix/store/d9b2qrrq32jzdsdx4y33inzrra5n5z5n-CVE-2014-8140.diff
/nix/store/ly8k93l59mlzlgnq679vcp54nqpi4sdc-06-initialize-the-symlink-flag.patch.drv
/nix/store/mwkywhh9wvym79lzlk8gsanw5swhfb8w-CVE-2019-13232-3.patch.drv
/nix/store/p46prhgmv7ibjh9igfkbc6zfxbbi6sk5-dont-hardcode-cc.patch
/nix/store/p55a764pi2f4kkx3adb43bxb2dnb4z6r-CVE-2018-18384.patch
/nix/store/pdcj2chp5c2gvm2jc3shbajfc62kbx1i-CVE-2014-9636.diff
/nix/store/rdkdki1f24q8mqgnbsyk7gmh28c027ks-CVE-2014-8139.diff
/nix/store/wx62rx7078a86mpimgg5gfsiisx3qk5l-CVE-2019-13232-1.patch.drv
/nix/store/zxpjddlgb46cdynbgbgq41i6k9a40wfg-unzip-6.0.drv
/nix/store/cnds925pfnac943p1f516pxwrbqy0gdi-source.drv
/nix/store/2cqnhs2ds0gn7xrq0zcwbj7gsv6l5xr7-use-etc-ssl-certs-darwin.patch
/nix/store/2yk6ab4c6j9y00g3x3cixqgi9jmgiwzd-openssl-3.0.5.tar.gz.drv
/nix/store/6hyy4ngzkxdsg71zmryvy3xkw0ydag21-openssl-disable-kernel-detection.patch
/nix/store/sq4h6bqjx12v9whvm65pjss25hg1538q-nix-ssl-cert-file.patch
/nix/store/imc07k6kg26rcam012pkgxba3mj0c0xq-openssl-3.0.5.drv
/nix/store/1ksmnsr3m6paw8gs7jp9b623agzdrqi2-add-flags.sh
/nix/store/9whvblgb7bgnmqsv7y5xqs1w8xv9aa1d-pkg-config-wrapper.sh
/nix/store/c4akajrb4jg50k72jw7zfbyv8z139ri0-setup-hook.sh
/nix/store/f4bvwqvj0y3z6blvh0knz71a8yq1c45p-requires-private.patch
/nix/store/qycyfzgf92i96ygij2b5bz27ll6mkyyw-pkg-config-0.29.2.tar.gz.drv
/nix/store/dq5y5c22p6ixswcqaxb6s0ymvi2bwmlm-pkg-config-0.29.2.drv
/nix/store/ssvy1s8p78q1jfy7bap0kl49sm8ad0m3-pkg-config-wrapper-0.29.2.drv
/nix/store/xz6rbdix12jn1bd3lydd1cagnvr94yf4-sigtool-0.1.2.drv
/nix/store/gp50m2w3aab5w1pgx9h18yky5x5jzzyq-signing-utils.drv
/nix/store/gz78khbgfz3rh5phvq1bavdp4v0bcimn-post-link-sign-hook.drv
/nix/store/zp333xdrvsgn3mjicwcbfiqd259ix6gd-MacOSX-SDK-11.0.0.drv
/nix/store/x3wxdzi7f36fnb1sryr1ifgafsycasks-libSystem-11.0.0.drv
/nix/store/v77imik30yacsdbfqkkdv0djyk1hsdgh-bootstrap-stage0-binutils-wrapper-.drv
/nix/store/6cqn1ln7v306ymslgmvcy77pbq30pjy1-bootstrap-stage0-clang-wrapper-11.1.0.drv
/nix/store/3inq067xw7bic7dy828bgjjzx54kav9v-install-shell-files.sh
/nix/store/70pf3jk5jc64jc82hqck1jx6z5l42xj3-install-shell-files.drv
/nix/store/zwlzlqvh3x2nw1421cvx2210bzz3xgfn-libev-4.33.tar.gz.drv
/nix/store/gn80i99rcc85d7l687d1jdmcm0yl51yv-libev-4.33.drv
/nix/store/h60a8qk9wqy8gbdvl9sf0qcnz63by9w4-nghttp2-1.49.0.tar.bz2.drv
/nix/store/09ba5xcaaaybqdsvljh5skzzl6blmmw4-c-ares-1.18.1.tar.gz.drv
/nix/store/l0niywhmkdqv97i8rlxzr1yqbn9k7sc1-c-ares-1.18.1.drv
/nix/store/36rbachr8ra090v3m6s86603sfp382k4-nghttp2-1.49.0.drv
/nix/store/fgr4mizzn2y712bqlk895lk8wpws27ir-dyld-433.5.tar.gz.drv
/nix/store/7c2sh9fyqz113rrpx5mdnmkk8mdrhb47-dyld-433.5.drv
/nix/store/81gwh57vnrm6qpw3gxmdmrjsp26dxi7p-find-xml-catalogs.sh
/nix/store/1vaq58n8rvn8bbr9mcm3k30zkr63480d-hook.drv
/nix/store/68bzcamhirzd95vsh05wilz8p8vfvyb6-expat-2.4.9.tar.xz.drv
/nix/store/0vyw0ywxhng31zvxqia2y23ljjmzsdj6-expat-2.4.9.drv
/nix/store/345r2zz7pgiyk91j89qlf7mhs95jrv6f-no-ldconfig.patch
/nix/store/zi0m9pfmvy5lw89x7a8x674rm99i8qiq-setup-hook.sh
/nix/store/4aakllhrfd3r87a67g0hb6r37nk4ahqf-python-setup-hook.sh.drv
/nix/store/4j7gbzbahn5by6vvck1gcpjc95k6vpz6-Python-3.10.7.tar.xz.drv
/nix/store/9m54l1bi5814x9cqznwlga7yfs5ipi6h-nuke-refs.sh
/nix/store/pm08hy0dzswr0wj3n7nczm5pbzzjxdh6-darwin-sign-fixup.sh
/nix/store/4qmyys335vfbmyx2q8ii3md77yaswsim-nuke-references.drv
/nix/store/57kclla9vza2n87xgwg1ap54d20cz6lb-fix-finding-headers-when-cross-compiling.patch
/nix/store/aafa965mg7grhivxa01mmbqksz6c3w77-python-3.x-distutils-C++.patch
/nix/store/cv1ynpzvjjr0s72jkbblbzz3ymr87lpi-0001-On-all-posix-systems-not-just-Darwin-set-LDSHARED-if.patch
/nix/store/dkb2rjyj7lwmvsn4zzwx85kx8r61nk9w-darwin-libutil.patch
/nix/store/w7hflmy5kddj6p1kvbkgag7qjs44217d-libffi-3.4.3.tar.gz.drv
/nix/store/nv7ln6adf9vv2c81rw1rv8sarp2w3dbp-libffi-3.4.3.drv
/nix/store/r112dk8w7zvdjipki58ch00m825li7fq-virtualenv-permissions.patch
/nix/store/x6aiw4vay2b63slqz5byimn0alhg5b1s-darwin-tcl-tk.patch
/nix/store/70zdg3iypjrxjyvj602ai92j36j8l4rp-python3-minimal-3.10.7.drv
/nix/store/1qrnbw8xsww3vydd71lwfp32ylgx9i8g-make-wrapper.sh
/nix/store/819fzxfwzp7zhhi4wy5nkapimkb1bsx5-die.sh
/nix/store/csphakh9ck533qnjfqmrh5ybg7amwvwv-hook.drv
/nix/store/scgakk3jkkkqfzxp5v9575163dj03c2y-hook.drv
/nix/store/8p4sgrqajxwzcm9l02m45qvpg6ncr0h9-patchutils-0.3.3.drv
/nix/store/baz2nahq7z7xxya1gi7in6a4msmvkbly-5b2d07a72670513e41b481a9d922c983a64027ca.patch.drv
/nix/store/frr6f2hq56j6b4wc0bsklaqskjsagqc0-utf8-xmlErrorFuncHandler.patch
/nix/store/vyh448kgn8qx2frx4g42f64b1akic218-libxml2-2.10.0.tar.xz.drv
/nix/store/lyl7058saady3i75h1w0177qfzfr1zf0-ncurses-6.3-20220507.tgz.drv
/nix/store/wy5amsi09scnbxgsbm1fzzq9963zlg9m-ncurses-6.3-p20220507.drv
/nix/store/x1fa0bns4szjkbb7f8bsflcxmzas41h0-4ad71c2d72beef0d10cf75aa417db10d77846f75.patch.drv
/nix/store/9gql9xbn6mfd56lxxchd0q7qzyf7cdby-libxml2-2.10.0.drv
/nix/store/1nv6k9zyc7pj74yshdzm4bmjfv6k86l9-source.drv
/nix/store/34qn4by9lqrri323ahm5vizv6bgsbhfn-006-darwin-always-set-runtime-c-flag.diff
/nix/store/chhz1kdhnql7gshwrns13lawgm75an7c-adv_cmds-119.tar.gz.drv
/nix/store/bylhfm7hj9sm4qb5n9mnsvj71xhmi6rx-flex-2.6.4.tar.gz.drv
/nix/store/vila3sxd48ha6r4yhdbfzqlvfl1jn2bw-glibc-2.26.patch.drv
/nix/store/lnb1468vfblvbc6lqgpigypspa4lzl68-flex-2.6.4.drv
/nix/store/6s064qz2lzpi1ml9z4vx0ajmk7fwafv6-bsdmake-24.tar.gz.drv
/nix/store/pps2jxig0cgkz81qwhy9gqar360mbbdb-bsdmake-24.drv
/nix/store/x1dh5wnl7pf81iq0gx0gqj6i9vnz89vv-source.drv
/nix/store/icl9b9dikbnwsrb5agnjmfryxyjnfp1n-adv_cmds-119.drv
/nix/store/6wbmq1k5x7q9lchs986bjq2qf1ip7b41-ps-adv_cmds-119.drv
/nix/store/7qslhycy1d0ag13xn4rfxfwdm8f7afb1-001-search-path.diff
/nix/store/ar1p4gcvlqf8dwbkfrcb01srbywinaj5-check-pc-files-hook.sh
/nix/store/ha21dnn4nw858g8d0wrkvhyvp7zmqgnz-setup-hook.sh
/nix/store/jmif3w9zsykc13zrhw4y8pynnnxg27zn-cmake-3.24.2.tar.gz.drv
/nix/store/k845qxmkbra9fc4628lpi0q8yjmc1sx9-005-remove-systemconfiguration-dep.diff
/nix/store/ri7qddqm02y0w2g43zqqz8lzy2pbdgrk-002-application-services.diff
/nix/store/rq41inv92jszcs5x62nc5cci5hapbjpw-003-libuv-application-services.diff
/nix/store/23dflh1nkw215df7bfyy9ml3jd5ranrw-cmake-boot-3.24.2.drv
/nix/store/c14fh88kpi7gc627s1l19gdlipfrwd59-brotli-1.0.9.drv
/nix/store/rv3w6kch2d0l70r6h3kk2r5d8ca216hz-libobjc-11.0.0.drv
/nix/store/f07bc31w87jm2rjk8zap5xyf92ach79p-apple-framework-CoreFoundation-11.0.0.drv
/nix/store/k8xvsl4226pzy73ywglgx46h7scffpjq-gnu-config-2021-01-25.drv
/nix/store/h9gslm9dppn38plwa811yr5c03s5lw9w-ICU-66108.tar.gz.drv
/nix/store/mqvcwr74dcsqlib5crdcawk16dmdgj9h-ICU-66108.drv
/nix/store/rkr3wamhhf9ha9n89yimjwyazvf3ar6j-hook.drv
/nix/store/f0qv4kz20212qcnd9wsck36zk3r8isl9-curl-7.85.0.tar.bz2.drv
/nix/store/gdqnvkbp6nnrvww108adb7nvjgrpdxzb-7.79.1-darwin-no-systemconfiguration.patch
/nix/store/3bld52y6l8fg58gaj9b937qg6m7zbm16-krb5-1.20.tar.gz.drv
/nix/store/3xk9ps0qz073k641b88swpa4fgx3hzwg-nix-ssl-cert-file.patch
/nix/store/n3vnxwsnf75gz250yayipdga4ziidwpi-openssl-1.1.1q.tar.gz.drv
/nix/store/82cq1qadbgv5d6gxb11zgkxl530wvzxj-openssl-1.1.1q.drv
/nix/store/18vddqgdxg6xcv5iws1vv7fd152rag04-bootstrap_cmds-121.tar.gz.drv
/nix/store/jyci5k74rnj58fkalyil3pj28x7xnr3m-bootstrap_cmds-121.drv
/nix/store/hj7zxy2r8ib6s6fa669kr6hqdiv4l6s1-libkrb5-1.20.drv
/nix/store/z3h1d8wp61892ydalxldvb496ar0hiz6-libssh2-1.10.0.tar.gz.drv
/nix/store/zrh6il3gp9xa58ldg40d57kwgsvljyb1-openssl_add_support_for_libressl_3_5.patch
/nix/store/yf97cpb80lyvqgd0lnq82c4qkx7kpb9f-libssh2-1.10.0.drv
/nix/store/w5683845xkfzwlp8sgbax0farj5fzhpv-curl-7.85.0.drv
/nix/store/9rkj0y4cl1sbbzndzg01crdamv4813pg-bootstrap-stage2-stdenv-darwin.drv
/nix/store/labgzlb16svs1z7z9a6f49b5zi8hb11s-bash-5.1-p16.drv
/nix/store/hvl8g8b6n8m8dk16bdavvpg31g8zmk96-bootstrap-stage2-stdenv-darwin.drv
/nix/store/9lzpvga5gm5klwg28iv1hgf01g0hpfaa-hook.drv
/nix/store/96fgzfyknjaj6fc85ai2n68qfndbps02-hook.drv
/nix/store/6wky968nz63ndx7z6kppcada8cdj4hg8-bsdmake-24.drv
/nix/store/0w9k3ah3f9c0j7k4vxka74pi07x01bbv-adv_cmds-119.drv
/nix/store/2n3wbbh9sk778vkl1lsz58inmyvk2g3r-file-5.43.drv
/nix/store/bcv1p9lpghn2l1zcw21z7401qc8fnmvr-gnu-config-2021-01-25.drv
/nix/store/47msbw943yjc7hm6d13r9jl5cz0ih9x9-hook.drv
/nix/store/24a06br9jvy87sbanym8hijfq4j2dsqm-no-threads.patch
/nix/store/4a0sl25fn3ymdmaalxwvjk6a2xajy3cb-libcxxabi-11.1.0.src.tar.xz.drv
/nix/store/ccf2fg1l77aqbw3v8hiql7a4c54dr8lh-bootstrap-stage0-clang-wrapper-11.1.0.drv
/nix/store/bhmp58x5m3lkscnxs9zwscc4rxbz7fc4-bootstrap-stage2-stdenv-darwin.drv
/nix/store/db7hhqin9klk8qx344r2b3dhb4xrfiq6-llvm-11.1.0.src.tar.xz.drv
/nix/store/il3mvxijf7cc9ys09pcblff47ia93kk2-gnu-install-dirs.patch
/nix/store/z3pj61rk5h9ffbj5anhwbp3ah2qysvqf-libcxx-11.1.0.src.tar.xz.drv
/nix/store/fg68n6bycig5c23sw8yfn7nmgbj85v6x-libcxxabi-11.1.0.drv
/nix/store/wjgikzdk0fcbldspzlkyh0dvd8wa2say-patchutils-0.3.3.drv
/nix/store/3v9a47xl4w8kc8bff4daxvbg50008imh-19bc9ea480b60b607a3e303f20c7a3a2ea553369.patch.drv
/nix/store/c9w69kl4caarcs2j84hck0icrdj9jqr6-fix-darwin-dylib-names.sh
/nix/store/g958ikb42h89wl8rgx597l5h6k9n2cfx-fix-darwin-dylib-names-hook.drv
/nix/store/ym4y16msxvvpbcsc0s829has6v8mxg56-gnu-install-dirs.patch
/nix/store/hhn96pp2rk2bq1hipcr147hqfrgh80gk-libcxx-11.1.0.drv
/nix/store/rizgpw7ndpwy12kyr03mnlhhkfwzk75a-pcre-8.45.drv
/nix/store/m1pm94jj46gsmks3d11p44qdhqml9inm-gnugrep-3.7.drv
/nix/store/r2mi8209hbvfhyhjcxy6qqvyawf8s4k3-expand-response-params.drv
/nix/store/6yjiri44dy8c8lnjn0f14w8nvgs2fhf0-bootstrap-stage3-stdenv-darwin.drv
/nix/store/dlm9y4dfkjmcqrysf37kxfhlds9r9jng-hook.drv
/nix/store/q6x1zg4xsbzw7qv3qdcq2rny1n4pr5xk-hook.drv
/nix/store/iraif23i0p01sx6qq6jkry21v7g84wvi-bootstrap-stage0-clang-wrapper-11.1.0.drv
/nix/store/wpni47ni5xf8qms83in475fyn8z2ikf6-bootstrap-stage3-stdenv-darwin.drv
/nix/store/40ri4k2mfvs5wjwry47iqv30587p1jh0-patchutils-0.3.3.drv
/nix/store/0w90wllxcb2wyjbkxjacm0m3q2wfz702-uops-CMOV16rm-noreg.diff.drv
/nix/store/09q0yxa6ixb030mkw96j2za7h8dpbpc7-dejagnu-1.6.3.tar.gz.drv
/nix/store/fk7p458jm8ra2d6zf2y4nw1ykckvmnrr-tcl8.6.11-src.tar.gz.drv
/nix/store/5z49kw0iq4qyga5zxmmhw1fh2l6jiwjz-tcl-8.6.11.drv
/nix/store/g4c3jbhc8ag6db5py0xk2sicfy0hrpmw-tcl-package-hook.sh
/nix/store/a1ipqs2qcpbqyfmxgk6yi3yyl2f6pd62-tcl-package-hook.drv
/nix/store/4vq3350zc6sqnibkqpgic2d6cvi1r9hq-substitute-all.sh
/nix/store/5241l4i852qd9imqz3jgv9rv9gv701gk-fix-cross-compilation.patch
/nix/store/dhba38jjxia0b5snjrhvcl9dbjbdb3k0-fix-cross-compilation.patch.drv
/nix/store/gdag5rjmiv9iwgj8gnibcvzic5f1kwbp-0001-enable-cross-compilation.patch.drv
/nix/store/qcmyh5mgqv62zip6gkp2xjriklpbm0xv-expect5.45.4.tar.gz.drv
/nix/store/xpdirxij63a69jk43pgajjrgmv7gaajd-hook.drv
/nix/store/iag2icih50jb6wxn15a71jy9pix7jd15-expect-5.45.4.drv
/nix/store/nharms54shj3rhwzw2ywk4alfrgm5k1m-dejagnu-1.6.3.drv
/nix/store/2hc51kpv7vkbfpifjq28jbhm49wlphzw-libffi-3.4.3.drv
/nix/store/6wv8acdd7bm7npixgr3rk86mf55fj47c-llvm-config-link-static.patch
/nix/store/9fcwwdr92g5wzpj2wxf86f0zy5mn0h1v-zlib-1.2.12.drv
/nix/store/9fd3xdcfv1dw93y99a18lw735fx8kvgy-gnu-install-dirs.patch
/nix/store/g05jynqyglkf9if6y70822l2r6y3nkj5-ncurses-6.3-p20220507.drv
/nix/store/2mys4xnihsf6r5y0sbby0y3qcn22ggjw-4ad71c2d72beef0d10cf75aa417db10d77846f75.patch.drv
/nix/store/2nyygvchsc258gkqymnbzmh741wsizrf-libiconv-50.drv
/nix/store/i68qy2hwsvqx3haf36smqi3n4lg1wavm-5b2d07a72670513e41b481a9d922c983a64027ca.patch.drv
/nix/store/w8ma9ddjcfc3l3z4ng6rlwljxiw85fhv-hook.drv
/nix/store/j4h65frq4nx4kpl06sj5c3cz2lc9fdrz-libxml2-2.10.0.drv
/nix/store/kl8l1ci2ycc6y23vkxvjhlbz2p93zp3z-polly-11.1.0.src.tar.xz.drv
/nix/store/n1sl525i7qanfd75l8s1bbqn32fasma0-dfaemitter-gcc-12.patch.drv
/nix/store/qcb7ljbcrmgdf6xv0irgvdv6rivgj3md-nvptx-gcc-12.patch.drv
/nix/store/yzac3dpyb5cyndxjl0d9nrpwqmcz18dk-b498303066a63a203d24f739b2d2e0e56dca70d1.patch.drv
/nix/store/8vqfrpgwki8j8nk9j08g04vh1iqcm9bw-llvm-11.1.0.drv
/nix/store/2jlsizsg3hsj6p10cm11fx9rshsjwwri-clang-tools-extra-11.1.0.src.tar.xz.drv
/nix/store/34zhl915l308dpa7v2786z3xgbvy6398-purity.patch
/nix/store/akc6qlr586k4r5lwmj70i0mbb8wxz709-clang-11.1.0.src.tar.xz.drv
/nix/store/v2az1r12rfivd43pvq2s0brp60n54mqa-clang-11-12-LLVMgold-path.patch
/nix/store/bhdlpz3fgkzcrh94rsp76m0n0gwb6fxx-clang-11-12-LLVMgold-path.patch.drv
/nix/store/f91k74rxz9c8miqg843jf6dfq35jhpka-fix-darwin-dylib-names-hook.drv
/nix/store/yrskcbyfm56qm4vs9dnacg8mqpqx7qwx-gnu-install-dirs.patch
/nix/store/arqis7aqh6b9p3a90idcjnr21fp1d0c4-clang-11.1.0.drv
/nix/store/79xln4yb0zql3j1zvc72yfi1y6333crl-disable-rpath.patch
/nix/store/dvdql179kllliqznwkd307bvdny2h96h-source.drv
/nix/store/5ks4wyqmh20wzz3i2yyqky7g0g8jhmsk-libtapi-1100.0.11.drv
/nix/store/aipzgrzywrh1qgw2l3bigbnnwpyr61kn-source.drv
/nix/store/fz56qb3cymbf2acghsqmdgjwwklc411f-install-shell-files.drv
/nix/store/r9iy2wdkq3w0n1hbyg92c2sqg3vcbkhl-ld-ignore-rpath-link.patch
/nix/store/3jjd65b6nyg7g30ixszjcsgb08j5mmhn-0001-Add-useless-descriptions-to-AC_DEFINE.patch
/nix/store/f3ygraph3msfdkyrqg91j7smx1c78qnd-bzip2-1.0.8.drv
/nix/store/j3frsgc0zsrg2pg7w9pqmxjh6qd01aqs-0002-Use-pkg-config-for-libxml2.patch
/nix/store/y26ic2b9n1g1cng3s68753qcxd6fvqrq-xz-5.2.6.drv
/nix/store/yqdd6m0r80c5pn3z7wc1mhn1wqmyli4i-xar-1.6.1.tar.gz.drv
/nix/store/1m30bskfc8z8r4dhb2qv9sd7jyl92srx-xar-1.6.1.drv
/nix/store/7vs1bbfsk0w6p6n5c4bqs3mp7q6n9lyd-source.drv
/nix/store/paybkfdbyh40jzyv6l5ywsbaabqy1sav-pbzx-1.0.2.drv
/nix/store/vwlpvn89jh6h45dj8q9nin0iv7rw85qb-MacOSX-SDK-11.0.0.drv
/nix/store/vb56s92027cg2c6573dahs6830fhz9xk-libobjc-11.0.0.drv
/nix/store/zr59kiajvksmks3vl1hja2ybix9z2zqk-ld-rpath-nonfinal.patch
/nix/store/ldj50kzrjbzh8dzcb7mqqa5gqsmc33v9-cctools-port-949.0.1.drv
/nix/store/0df8rz15sp4ai6md99q5qy9lf0srji5z-0001-Revert-libtool.m4-fix-nm-BSD-flag-detection.patch
/nix/store/k1cgpjsl77c8wfaq0w7n6k1i7fvxr5p8-gas-dwarf-zero-PR29451.patch
/nix/store/lgniihp1bk6mkd5nn9y5ikfim2ignr52-0001-libtool.m4-update-macos-version-detection-block.patch
/nix/store/pa83jbilxjpv5d4f62l3as4wg2fri7r7-always-search-rpath.patch
/nix/store/dqgzlxvbzq4dih9ska9k0y91sc1kv7d9-autoconf-2.69.tar.xz.drv
/nix/store/s8wb99pw1w8yspcz26zfadsy0j1k70ww-autoconf-2.69.drv
/nix/store/sqbhaaayam0xw3a3164ks1vvbrdhl9vq-deterministic.patch
/nix/store/xrw086zw3xqsvy9injgil8n2qdkvkpff-0001-Revert-libtool.m4-fix-the-NM-nm-over-here-B-option-w.patch
/nix/store/zbdl2p9amxdkr9cqjq0yv6h0mr55lm3l-binutils-2.39.tar.bz2.drv
/nix/store/zki7kfvf2f0xdksq9hp004xz1hsxklz3-texinfo-6.8.drv
/nix/store/q1kz46q80wj4bfc314g5p3sylpilpv0i-binutils-2.39.drv
/nix/store/00fij0grbvf9svcxvyk5ys51qcpmk7sa-cctools-binutils-darwin-949.0.1.drv
/nix/store/17agldwh525770zc3w11sdkpdxq8xwjy-gnu-config-2021-01-25.drv
/nix/store/0kkx3whrs8zb85qwwvl42ax99hmk0xpz-pcre-8.45.drv
/nix/store/3qkad2sv1xwngyn4gmx473mzrjhs2jx8-gnugrep-3.7.drv
/nix/store/xn8bjkw4fmyr2xvk9higd54g9s7q2kdz-gmp-with-cxx-6.2.1.drv
/nix/store/hbby7876a2qhk93hlx5ppr3jpgm1lwkl-coreutils-9.1.drv
/nix/store/vj0ai85srlbfn0yvd6mdql2nxan24naa-post-link-sign-hook.drv
/nix/store/vsd7hdcfnzn5n4q44flcdiaa2lv58d6d-signing-utils.drv
/nix/store/z4gcxwcrzc0mkx0zgha8k4jr506kgns7-expand-response-params.drv
/nix/store/vnxdn7rjmsm0naz6sgb98flh437iff1g-cctools-binutils-darwin-wrapper-949.0.1.drv
/nix/store/5gm6pb695hmb0q26cyvmm0ish0p5yig0-clang-wrapper-11.1.0.drv
/nix/store/2cw54rrcb7plvq6v5hxsm3sb42kq6fk5-bootstrap-stage3-stdenv-darwin.drv
/nix/store/3ic95d8vv23cdj4vq7634zh5zcdsda1p-normalize-var.patch
/nix/store/3m53ki7pr92hacq8mghsldg1wc4wrifm-codesign.patch
/nix/store/706qcmh667cmjwffc4vbjs9c745c7hs0-gnu-install-dirs.patch
/nix/store/skwzly3754w50q8mzdpadz8bsfmn6hs8-compiler-rt-11.1.0.src.tar.xz.drv
/nix/store/y87vwjq5v1fi0d077xqrsnkqrax3p4iz-libsanitizer-no-cyclades-11.patch
/nix/store/ylv0v02l6panidz2hkh756fz363yc68h-X86-support-extension.patch
/nix/store/1ffkl3b9gb1qyvmz2r1633vzkhd5bxn2-compiler-rt-libc-11.1.0.drv
/nix/store/9k3fqaqk3k7m510nn57ynd4ngx2x4rxs-clang-wrapper-11.1.0.drv
/nix/store/mmdgxk0mpsq2lzlja8g6mrdjzxpc3wmq-hook.drv
/nix/store/s9rdswvsx68yjwfk57vlw5a34hbjsl2v-source.drv
/nix/store/hrp64mrc7ss3zz8gfl36jfq8fj8qwk2b-libyaml-0.2.5.drv
/nix/store/y7wqq0qnhnhd6hr5d2hxl967fjssfrny-source.drv
/nix/store/vy20n6yc3nd8d1yk0sqa6cah7dpyhv9s-rewrite-tbd-20201114.drv
/nix/store/sn11j0j89hflj7snivdlxvxawvrpzp01-apple-framework-CoreFoundation-11.0.0.drv
/nix/store/aawhsgywk753j170584pf3r6rlismpi6-bootstrap-stage4-stdenv-darwin.drv
/nix/store/qwxrck5ibwhhhsgyblc711vnvw5rx2ib-bootstrap-stage4-stdenv-darwin.drv
/nix/store/bs639bs7frzx60c8bh3nyzmak49zbc3k-hook.drv
/nix/store/cfkvf8l9jw3vrvkss3c5i1ccgnp86nv6-hook.drv
/nix/store/skm4ysx9pk4nnwx75fpk9vghzc130887-perl-5.36.0.drv
/nix/store/ma5p31xpqrcyk0z70l14m7cklw2zkb5s-patchutils-0.3.3.drv
/nix/store/lapsha9pivk9wvrwlrc482i2biq06gw5-CVE-2021-38185-2.patch.drv
/nix/store/ncfbi3qgbflyph4x2ngcnz2584kykzqa-CVE-2021-38185-1.patch.drv
/nix/store/pklid951p82izlw5f06w5yvpps1zwgxw-CVE-2021-38185-3.patch.drv
/nix/store/sp2alvzdl09796wpg2wdf68akiha4d4g-fno-common-fix.patch.drv
/nix/store/zmadr12vmal7mwlgy1w4w4x70lss6j73-cpio-2.13.tar.bz2.drv
/nix/store/d54hwx5g50niakv2lpb9lkp1jjk910q1-cpio-2.13.drv
/nix/store/4yn89klg688jxj8sidmzc84hl7ndcwkm-pkg-config-0.29.2.drv
/nix/store/416vhsxki7508q1ijs8n415fqmvjxyky-pkg-config-wrapper-0.29.2.drv
/nix/store/a736nzx1nigikprmk4ignqzndmz7ls0m-gettext-0.21.drv
/nix/store/2szis9v9c9dzazajlflfdc9jl3b0pihg-gnum4-1.4.19.drv
/nix/store/gbznai4iy45kxmlnrn56fn5m0x2rlspc-texinfo-6.8.drv
/nix/store/f33p3mlclvl3hhnnjsnzw70bshn7criz-autoconf-2.71.drv
/nix/store/dw19klga9vppaq2f6lzj9h7bl4c29mga-automake-1.16.5.drv
/nix/store/hizv414b6ky026kw5f4hwgm2lbxzh08y-perl5.36.0-gettext-1.07.drv
/nix/store/ccwg6bkak9j236lc6k0n713iyn12k996-help2man-1.49.2.drv
/nix/store/jx8mci3vwqi9qv1wkbrfpjya00nl285j-file-5.43.drv
/nix/store/lahdhba85rpm60wxmhxanq1dqi8sa5kb-libtool-2.4.7.drv
/nix/store/g69zz30gq1rb85b6kdz5iidxxbwp1bda-hook.drv
/nix/store/hb2yk99cw015si2ry1l1fygp365as523-openssl-3.0.5.drv
/nix/store/cv65milx0bs2fk1xikgrgrhvkafkpqdk-xar-1.6.1.drv
/nix/store/9wfpm5alc542isr7232gg4qg5njdsgdq-unzip-6.0.drv
/nix/store/ggp35l799v2ggwsk73sz61y7wg3kpqy1-source.drv
/nix/store/msvf870j9sr6lwzakv6m263gk0r7126z-pbzx-1.0.2.drv
/nix/store/0b2hxys6lxyh567j82addkw2k36qjzgz-MacOSX-SDK-11.0.0.drv
/nix/store/0c1ijhg1cq01zcrvnhg24vhm5qn47kwd-libev-4.33.drv
/nix/store/0fqp7dz3rlrrhqjfc69wkmii3j5y0g7d-make-4.3.tar.gz.drv
/nix/store/bqlikyzhzhsjfirhzgmps9p99mhvn9az-bison-3.8.2.drv
/nix/store/c3kdwmns3lyigqqm6c4czisv0n226dy7-source.drv
/nix/store/hp3krbr0v290hwgrcskls6kk545virpz-flex-2.6.4.drv
/nix/store/mdihpwmpbkv1wg4yw03d7wy3fbs0l45w-bsdmake-24.drv
/nix/store/ln3xll7z1avhm03k3m8cdllln7wdgrrk-adv_cmds-119.drv
/nix/store/gs103r2mxcsjs3sw0ibvlnsv54qcw9q6-ps-adv_cmds-119.drv
/nix/store/94y4s4a30p8qqfg8bxr9rgwkdc9m3610-cmake-boot-3.24.2.drv
/nix/store/n91acyjrlchm0snw0w16i4683pf788ax-playtests-darwin.patch
/nix/store/vfb2ll7c9aq63mlkkvmvfq4ibiinq5nh-source.drv
/nix/store/vhp9nf4r8328m91l9l5c8fd9wlmb4bnd-fix-darwin-dylib-names-hook.drv
/nix/store/0w2r0sw68fwxqqki50mqx83iz1q6clgq-zstd-1.5.2.drv
/nix/store/4rj3r6gga5ipdfkiw9ahmrj9yr411ry6-libobjc-11.0.0.drv
/nix/store/vc3jlishkiy0qsw95smzzzr9vcgafi56-source.drv
/nix/store/jhs384scb8wk3sn9dw92khjpay91x9mg-libtapi-1100.0.11.drv
/nix/store/ylsnxqqfn3gpp0wr133z4ksj2bhklrg7-source.drv
/nix/store/z8gyza1abwd7jh3pv10r6kcrgphi2h2r-install-shell-files.drv
/nix/store/0xwx3m2lvpw92w4j45n1772f3aimskxg-cctools-port-949.0.1.drv
/nix/store/0y5flakfvnf813cwrr8rygf1jnk0gfnc-CVE-2019-13636.patch
/nix/store/4r8s8hcwyvvvnpcncps09zscqkh5qapx-no-install-statedir.patch
/nix/store/bljrd66ff2vp1zqikdfrz5x0k90kaw81-findutils-4.9.0.tar.xz.drv
/nix/store/10zrhmiqirncfbxyac8xrjg6p8mqf30k-findutils-4.9.0.drv
/nix/store/53d5wfhiifvxzgj847fva6q6py02m1g1-expand-response-params.drv
/nix/store/c2yrfg597pjcl1867pyir9jiq4fw3jl4-source.drv
/nix/store/9h8631c24qp90y7w0fvsl3c0dv8pv6pi-sigtool-0.1.2.drv
/nix/store/anpnd9wprrqhf1fdcwy96j66vb5fcsii-signing-utils.drv
/nix/store/5xzxmr5xpnqx0b0ar0kpy7kw0282a1jc-autoconf-2.69.drv
/nix/store/a921f1jk651ahri5f05gca9rwdiq3rlz-binutils-2.39.drv
/nix/store/laack7baw4rfxgvyqfrfydbn1v8zz9b6-clang-11-12-LLVMgold-path.patch.drv
/nix/store/d7h3zix26zzgdp0rzsyb19hrif8cbzmz-clang-11.1.0.drv
/nix/store/jxpxgb5hph4lw3s12g7lyqz0ci8a37ra-cctools-binutils-darwin-949.0.1.drv
/nix/store/w3hhid4crxafa1j03iq4a1y30d8p0sk2-post-link-sign-hook.drv
/nix/store/5w0k8f3ialhwr7p5g0y94zny3j74ryzn-cctools-binutils-darwin-wrapper-949.0.1.drv
/nix/store/74fhpxplxsc5qg1c67hb5picw88flx3d-libssh2-1.10.0.drv
/nix/store/76nc36kgrvhx8n9m8jz7ywrrqbcdkr48-gnu-config-2021-01-25.drv
/nix/store/9wyqdma0i671db7l4m0a3qbp6jpza2vi-tar-1.34.tar.xz.drv
/nix/store/90phqk429ip6kbi5jlm0lcg4h2xxaq2k-gnutar-1.34.drv
/nix/store/2j0fmwgm5ybgnc8jprc4ypcxw6s4r2nv-Allow_input_files_to_be_missing_for_ed-style_patches.patch
/nix/store/7mq3l9rhjbmpf06fwnrp51q6sy1l6g9j-patch-2.7.6.tar.xz.drv
/nix/store/8p3z4jsrxr5ck92iasc9bc7bmapb5mmg-CVE-2018-6951.patch
/nix/store/h2fcbw7ghgn3i4qadszdp272w4dab7ln-lzip-setup-hook.sh
/nix/store/vvciv7wkw3z9x0bj4jszb31crk0lix8y-lzip-1.23.tar.gz.drv
/nix/store/6gj38dbipcd2vbjcsv028jmpnn6bv1sz-lzip-1.23.drv
/nix/store/bv0xxgk72g693vdgs3w2w3d252hlxys9-ed-1.18.tar.lz.drv
/nix/store/lmssb21nd3zkv5gssngmk92bdf0q4h1w-ed-1.18.drv
/nix/store/npqvgz8c8w9kpj1gdma5bbn0pdkisyzp-CVE-2018-6952.patch
/nix/store/rxgi2l6jrgd5xmrrsbcv5cwi558lb36m-CVE-2019-13638-and-CVE-2018-20969.patch
/nix/store/sz6rhpf50kqh75fhqwl75q6dm6fr9xyd-CVE-2018-1000156.patch
/nix/store/aqmy225ay8m4yg51mib0bkpz3r1w2z7j-patch-2.7.6.drv
/nix/store/2k52bklbjhhq47dn35gm833vlh06fgfn-0001-No-impure-bin-sh.patch
/nix/store/6cc64ayl3fd2nc28ffw47cqsqi2bg1sn-0002-remove-impure-dirs.patch
/nix/store/avhna3r651j0frjk7jhy771za84mlh4j-gnumake-4.3.drv
/nix/store/b7mkkj1kbaxpihqh3k12s4a3viz9pdvk-hook.drv
/nix/store/dy63w31j33lknbd95fl81f24sndlgf35-source.drv
/nix/store/bl8jncx9dy4rr54cn8p1vwpf3wa01yyj-brotli-1.0.9.drv
/nix/store/rp4wqqv22pjl2235ra7ag3nb7yy0b5kr-diffutils-3.8.tar.xz.drv
/nix/store/d554acfihg4ssgij79ybd6ls0ww1p14x-diffutils-3.8.drv
/nix/store/szfjhkhmsxfyc289vz39882d3l866888-fix-error-darwin.patch
/nix/store/z8912zv98rl5yv104mf99j3k72xva3nr-libidn2-2.3.2.tar.gz.drv
/nix/store/qhs49gbxssyr8im8h2xc058gh4kdndv6-libunistring-1.0.tar.gz.drv
/nix/store/zipavm2mq94fsw09kx9mw7pi5n8xhkp4-libunistring-1.0.drv
/nix/store/gzy4kvn6djdm9b631fcm3g76pijsvvq7-libidn2-2.3.2.drv
/nix/store/hvp22x3rmpwfj6kcf0hj9dcv8lrkd4hs-clang-wrapper-11.1.0.drv
/nix/store/xhmwfi4xij5ryg9x2j0n6067ki55dscr-c-ares-1.18.1.drv
/nix/store/qpsbvsn7dl8gmlsmh474m2h9gmh8mg7i-nghttp2-1.49.0.drv
/nix/store/lwhmzpa1py1k244hfd3l026kc7r7b6ib-sed-4.8.tar.xz.drv
/nix/store/rvq62srcss99303nbgr94bq4av9kjv8q-gnused-4.8.drv
/nix/store/pid9is7y02wzdplvk6jcw2n0vfdh2y32-openssl-1.1.1q.drv
/nix/store/pz4hlrs0xa3q9knmn83v8v3wpf88iigz-bootstrap_cmds-121.drv
/nix/store/6m3w5kmj2m73nj3pakh6kaqjs2k64p5m-libkrb5-1.20.drv
/nix/store/sa6lxl61bh1wmr9rbpbyiq5wg2y5kbf2-curl-7.85.0.drv
/nix/store/vff2y41m68f5garsjmjxc0xfjrvw9pfm-setup-hook.drv
/nix/store/w99jp1rsykvc8rb09hr03c0rakw2dgzc-gzip-1.12.tar.xz.drv
/nix/store/swj6qdzryq4ln3h172s4h45wf0ks7g4j-gzip-1.12.drv
/nix/store/v994hjy8kmwi7g5li2lrpss92ldj0a54-ICU-66108.drv
/nix/store/pmxi9k28qls2yr7jhfnz3qp1fjchy64m-gawk-5.1.1.tar.xz.drv
/nix/store/x9ndmqlkrngf4jdy4zmvmal9ma7gh3z0-gawk-5.1.1.drv
/nix/store/5pfijbmkmy5kc81yzp0lpm4gpm2aq5rk-source.drv
/nix/store/zf7v9n0hxgb302wf08y762s0rhsbnqd2-source.drv
/nix/store/qr3r1g7n6xppyxb88726z7yr6r246nzd-libyaml-0.2.5.drv
/nix/store/91bfp6zivp9jq9sqq6iqq7vdpghdaffa-rewrite-tbd-20201114.drv
/nix/store/ya29pfy418vy1l0i5symnyprdb80c8ha-apple-framework-CoreFoundation-11.0.0.drv
/nix/store/cdk3pz11mvhqpphr0197wwmzhqppn7rl-stdenv-darwin.drv
/nix/store/26z459l0k3znhr99dsshkzj0il8dhwxx-perl-5.36.0.drv
/nix/store/sbcibnd6hym9c2rlbfnyhrbmlvgmsa59-stdenv-darwin.drv
/nix/store/sr9iyw9n2awaikjzvjfgwhvvn6vimf5w-pkg-config-0.29.2.drv
/nix/store/3zmkapnjbnajncxw7cix0lmj1fbspwa0-pkg-config-wrapper-0.29.2.drv
/nix/store/7dlkjg6cyd8d47qwiamxi77hld2z5360-mirrors-list.drv
/nix/store/i0zc5mm4vpj3lviyydb9s73j53mypkrg-nghttp2-1.49.0.drv
/nix/store/w37b5s734m53gxnzqyb5v0v98mhdfg2i-coreutils-9.1.drv
/nix/store/mmsrbggvzn6rwlx1ijw90sw3wvhzj18j-openssl-3.0.5.drv
/nix/store/n7iibs6b818v402j0yczf4mgy73sbzpv-libssh2-1.10.0.drv
/nix/store/z074ki54p77r7db3wsgxh9p18f67xnv8-curl-7.85.0.drv
/nix/store/hwymznwkd1kgf5ldcldjl9bnc1wz2azb-hello-2.12.1.tar.gz.drv
/nix/store/4a78f0s4p5h2sbcrrzayl5xas2i7zq1m-hello-2.12.1.drv

The above command not only lists the build-time dependencies for the hello package, but also their transitive build-time dependencies. In other words, these are all the derivations needed to build the hello package “from scratch” in the absence of any cache products. We can see the complete tree of build-time dependencies like this:

$ nix-store --query --tree "${DERIVATION}"
/nix/store/4a78f0s4p5h2sbcrrzayl5xas2i7zq1m-hello-2.12.1.drv
├───/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh
├───/nix/store/labgzlb16svs1z7z9a6f49b5zi8hb11s-bash-5.1-p16.drv
   ├───/nix/store/7kcayxwk8khycxw1agmcyfm9vpsqpw4s-bootstrap-tools.drv
   │   ├───/nix/store/3glray2y14jpk1h6i599py7jdn3j2vns-mkdir.drv
   │   ├───/nix/store/50ql5q0raqkcydmpi6wqvnhs9hpdgg5f-cpio.drv
   │   ├───/nix/store/81xahsrhpn9mbaslgi5sz7gsqra747d4-unpack-bootstrap-tools->
   │   ├───/nix/store/fzbk4fnbjqhr0l1scx5fspsx5najbrbm-bootstrap-tools.cpio.bz>
   │   ├───/nix/store/gxzl4vmccqj89yh7kz62frkxzgdpkxmp-sh.drv
   │   └───/nix/store/pjbpvdy0gais8nc4sj3kwpniq8mgkb42-bzip2.drv
   ├───/nix/store/3lhw0v2wyzimzl96xfsk6psfmzh38irh-bash51-007.drv
   │   ├───/nix/store/7kcayxwk8khycxw1agmcyfm9vpsqpw4s-bootstrap-tools.drv [..>
   │   ├───/nix/store/nbxwxwqwcr9rrmxb6gb532f18102815x-bootstrap-stage0-stdenv>
   │   │   ├───/nix/store/1i5y55x4b4m9qkx5dqbmr1r6bvrqbanw-multiple-outputs.sh
   │   │   ├───/nix/store/59jmzisg8fkm9c125fw384dqq1np602l-move-docs.sh
   │   │   ├───/nix/store/7kcayxwk8khycxw1agmcyfm9vpsqpw4s-bootstrap-tools.drv>
   │   │   ├───/nix/store/bnj8d7mvbkg3vdb07yz74yhl3g107qq5-patch-shebangs.sh
   │   │   ├───/nix/store/cickvswrvann041nqxb0rxilc46svw1n-prune-libtool-files>
   │   │   ├───/nix/store/ckzrg0f0bdyx8rf703nc61r3hz5yys9q-builder.sh
   │   │   ├───/nix/store/fyaryjvghbkpfnsyw97hb3lyb37s1pd6-move-lib64.sh
   │   │   ├───/nix/store/g8xg0i02aqwhgxwd2vnp5ax3d6lrkg1v-strip.sh
   │   │   ├───/nix/store/jngr4r80x5jn482ckqrfh08ljrx1k86f-setup.sh
   │   │   ├───/nix/store/kd4xwxjpjxi71jkm6ka0np72if9rm3y0-move-sbin.sh
   │   │   ├───/nix/store/kxw6q8v6isaqjm702d71n2421cxamq68-make-symlinks-relat>
   │   │   ├───/nix/store/m54bmrhj6fqz8nds5zcj97w9s9bckc9v-compress-man-pages.>
   │   │   ├───/nix/store/ngg1cv31c8c7bcm2n8ww4g06nq7s4zhm-set-source-date-epo>
   │   │   └───/nix/store/wlwcf1nw2b21m4gghj70hbg1v7x53ld8-reproducible-builds>
   │   ├───/nix/store/i65va14cylqc74y80ksgnrsaixk39mmh-mirrors-list.drv
   │   │   ├───/nix/store/7kcayxwk8khycxw1agmcyfm9vpsqpw4s-bootstrap-tools.drv>
   │   │   ├───/nix/store/nbxwxwqwcr9rrmxb6gb532f18102815x-bootstrap-stage0-st>
   │   │   └───/nix/store/ycwm35msmsdi2qgjax1slmjffsmwy8am-write-mirror-list.sh
   │   └───/nix/store/lphxcbw5wqsjskipaw1fb8lcf6pm6ri6-builder.sh

If we were to build and cache all of these build-time dependencies then our local /nix/store and cache would explode in size. Also, we do not need to do this because there is a better solution …

Correct solution

The solution that provides the best value is to cache all transitive build-time dependencies that are present within the current /nix/store after building the top-level build product. In other words, don’t bother to predict which build-time dependencies we need; instead, empirically infer which ones to cache based on which ones Nix installed and used along the way.

This is not only more accurate, but it’s also more efficient: we don’t need to build or download anything new because we’re only caching things we already locally installed.

As a matter of fact, the nix-store command already supports this use case quite well. If you consult the documentation for the --requisites flag, you’ll find this gem:

       • --requisites; -R
         Prints out the closure (../glossary.md) of the store path paths.

         This query has one option:


         • --include-outputs Also include the existing output paths of store
           derivations, and their closures.

         This query can be used to implement various kinds of deployment. A
         source deployment is obtained by distributing the closure of a store
         derivation. A binary deployment is obtained by distributing the closure
         of an output path. A cache deployment (combined source/binary
         deployment, including binaries of build-time-only dependencies) is
         obtained by distributing the closure of a store derivation and
         specifying the option --include-outputs.

We’re specifically interested in a “cache deployment”, so we’re going to do exactly what the documentation says and use the --include-outputs flag in conjunction with the --requisites flag. In other words, the --include-outputs flag was expressly created for this use case!

So here is the simplest, but least robust, version of the script for computing the set of build-time dependencies to cache, as a Bash array:

$ # Continue reading before using this code; there's a more robust version later

$ # Optional: Perform the build if you haven't already
$ nix build "${BUILD}"

$ DERIVATION="$(nix path-info --derivation "${BUILD}")"

$ DEPENDENCIES=($(nix-store --query --requisites --include-outputs "${DERIVATION}"))

$ nix store sign --key-file "${KEY_FILE}" --recursive "${DEPENDENCIES[@]}"

$ nix copy --to "${CACHE}" "${DEPENDENCIES[@]}"

The above code is simple and clear enough to illustrate the idea, but we’re going to make a few adjustments to make this code more robust.

Specifically, we’re going to:

  • Change the code to support an array of build targets

    i.e. BUILDS instead of BUILD

  • Use mapfile instead of ($(…)) to create intermediate arrays

    See: SC2207

  • Use xargs to handle command line length limits

… which gives us:

$ # Optional: Perform the build if you haven't already
$ echo "${BUILDS[@]}" | xargs nix build

$ mapfile -t DERIVATIONS < <(echo "${BUILDS[@]}" | xargs nix path-info --derivation)

$ mapfile -t DEPENDENCIES < <(echo "${DERIVATIONS[@]}" | xargs nix-store --query --requisites --include-outputs)

$ echo "${DEPENDENCIES[@]}" | xargs nix store sign --key-file "${KEY_FILE}" --recursive

$ echo "${DEPENDENCIES[@]}" | xargs nix copy --to "${CACHE}"

… where you:

  • replace BUILDS with a Bash array containing what you want to build

    e.g. .#example or nixpkgs#hello

  • replace CACHE with whatever store you use as your cache

    e.g. s3://cache.example.com

  • replace KEY_FILE with the path to your cache signing key

Conclusion

That last script is the pedantically robust way to do this in Bash if you want to be super paranoid. The above script might not work in other shells, but hopefully this post was sufficiently clear that you can adapt the script to your needs.

If I made any mistakes in the above post, let me know and I can fix them.