111 lines
4.7 KiB
Markdown
111 lines
4.7 KiB
Markdown
+++
|
|
date = "2022-06-15"
|
|
draft = false
|
|
path = "/blog/building-nix-derivations-manually"
|
|
tags = ["nix"]
|
|
title = "Building Nix derivations manually in nix-shell"
|
|
+++
|
|
|
|
When writing, debugging, or otherwise working with Nix expressions, it is often
|
|
useful to run a build part of the way through, or to run it manually in a
|
|
shell. For instance, perhaps you want to test some development version of a
|
|
tool against a package and want to iterate quickly. Or, a build is broken and
|
|
you want to look at it more closely than `nix-build --keep-failed` makes
|
|
convenient.
|
|
|
|
Most packages are built with the generic builder, regardless of language. The
|
|
language specific builders are then written in terms of it. For instance,
|
|
Haskell packages are [built with this builder][haskell-builder], customizing it
|
|
by adding `setupCompilerEnvironmentPhase` and overriding many phases.
|
|
|
|
[haskell-builder]: https://github.com/nixos/nixpkgs/blob/master/pkgs/development/haskell-modules/generic-builder.nix
|
|
|
|
The way `nix-build` typically builds packages with the generic builder is
|
|
documented in the [`stdenv` chapter][stdenv-docs] of the nixpkgs documentation.
|
|
More or less, it runs [various "phases"][phases] in order in order to build the
|
|
package. This is roughly analogous to the `build()`, `check()`, etc functions
|
|
in an Arch Linux or Alpine Linux `PKGBUILD` file.
|
|
|
|
[stdenv-docs]: https://nixos.org/manual/nixpkgs/stable/#chap-stdenv
|
|
[phases]: https://nixos.org/manual/nixpkgs/stable/#sec-stdenv-phases
|
|
|
|
The generic builder is made of function definitions in the file
|
|
[`pkgs/stdenv/generic/setup.sh`][setup.sh] in the nixpkgs repository, and it is
|
|
usually `source`d by the actual build script,
|
|
[`pkgs/stdenv/generic/default-builder.sh`][default-builder.sh], which is
|
|
trivial:
|
|
|
|
```sh
|
|
source $stdenv/setup
|
|
genericBuild
|
|
```
|
|
|
|
[setup.sh]: https://github.com/nixos/nixpkgs/blob/master/pkgs/stdenv/generic/setup.sh
|
|
[default-builder.sh]: https://github.com/nixos/nixpkgs/blob/master/pkgs/stdenv/generic/default-builder.sh
|
|
|
|
`default-builder.sh` is not always used: `setup.sh` is more or less a shell
|
|
script library and can equally be used in a custom build script that defines
|
|
some functions then calls `genericBuild`.
|
|
|
|
*By default*, `genericBuild` will run the default phases, the most notable of
|
|
which are `unpackPhase`, `configurePhase`, `buildPhase`, `checkPhase` (tests),
|
|
and `installPhase`. There are also hooks that can run. These phases are either
|
|
the functions of that name, which perform the default C `./configure; make;
|
|
make install` thing, environment variables containing scripts overriding
|
|
them, or shell functions defined in the file that sources `setup.sh`.
|
|
|
|
Another feature supported by the generic builder is "hooks" which are basically
|
|
little shell functions, possibly defined by environment variables, which can be
|
|
run at various points in the build process if present.
|
|
|
|
More phases can also be added for more complicated things to build. In
|
|
practice, all of these extensibility features mean that the generic builder is
|
|
reused for most language ecosystems.
|
|
|
|
## Running a build manually
|
|
|
|
Because of the complication of possible custom phases, hooks, etc, it is not
|
|
really prudent to just run the phase functions directly because it essentially
|
|
means you are reimplementing `genericBuild` in your head.
|
|
|
|
When debugging, what you probably want to do is consult the full phases list
|
|
and run `genericBuild` with those phases. We can see that the default phases are
|
|
the following (from [setup.sh][setupsh-phases]):
|
|
|
|
[setupsh-phases]: https://github.com/nixos/nixpkgs/blob/1e2a288f0e84b7064020554cd89415932b458c1b/pkgs/stdenv/generic/setup.sh#L1335-L1340
|
|
|
|
```sh
|
|
if [ -z "${phases:-}" ]; then
|
|
phases="${prePhases:-} unpackPhase patchPhase ${preConfigurePhases:-} \
|
|
configurePhase ${preBuildPhases:-} buildPhase checkPhase \
|
|
${preInstallPhases:-} installPhase ${preFixupPhases:-} fixupPhase installCheckPhase \
|
|
${preDistPhases:-} distPhase ${postPhases:-}";
|
|
fi
|
|
```
|
|
|
|
A manual build of a package might look like this:
|
|
|
|
First, open a nix-shell with the derivation of the package in question. For
|
|
instance, `nix-shell '<nixpkgs>' -A hello`.
|
|
|
|
Then do:
|
|
|
|
```sh
|
|
phases="${prePhases:-} unpackPhase patchPhase" genericBuild
|
|
```
|
|
|
|
which will dump you into a shell in the sources directory for the package.
|
|
|
|
Once you've done the stuff you'll run once, you can run the build as many times
|
|
as you'd like:
|
|
|
|
```sh
|
|
out=$(pwd)/out phases="${preConfigurePhases:-} configurePhase ${preBuildPhases:-} buildPhase" genericBuild
|
|
```
|
|
|
|
The `$out` variable has to be specified because it defaults to somewhere in the
|
|
nix store, which usually breaks builds since is not available for writing if
|
|
you are not the actual sandboxed nix builder. If there are other outputs such
|
|
as `doc`, these also need to be specified here.
|
|
|
|
|