Finding things in nixpkgs and NixOS source

Jade Lovelace <jade at jade dot fyi>

Get the slides at https://jade.fyi/nixcon2023/slides

# whoami * Author of nix-doc (not to be confused with nixdoc) * Most involved in Nix for Haskell and Rust packaging * I use NixOS as a dev workstation and server

# the sources are for everyone * We do need to improve our documentation * Nix is uniquely participatory: a *lot* of us write modules and packages, which means reading sources * Navigating the source is a bit of witchcraft, and there's a lot of it * With good tooling, the sources of the docs sometimes are faster than the docs

# motivation (2.94 million lines of it) <pre><code class="text"><script type="text/template"> dev/nixpkgs » tokei -t nix . ======================================= Language Files Lines ======================================= Nix 30400 2941526 ======================================= Total 30400 2941526 ======================================= </script></code></pre>

# roadmap * nix * static analysis * nixd * ctags * dynamic analysis * using the repl * using nix-doc * nixos * ctags * repl usage

# Static analysis * Any method that takes 15 seconds or less and doesn't involve running any code * I am showing the fancy ways of doing it * ripgrep is static analysis too; it's a good fallback

# nixd Newish language server, using Nix as a library. In nixpkgs as `nixd`. Good for: * need go-to-definition * interactive help writing nix code Limitations: * does not eval everything * needs some setup per-project * completions are imperfect but they exist * not sure how to get goto definition to go past args for some things

# nixd setup Install from nixpkgs and configure in your editor. Then: `.nixd.json`: <pre><code class="json"><script type="text/template"> { "eval": { "target": { "args": [ "--expr", "with import ./. {}; nix-doc" ] } } } </script></code></pre>

# demo (nixd) FIXME

# ctags * Very old and simple source code index format * Usually gets through abstraction by being too naive to get broken * Typically generated by simple parse-tree traversal * Wide editor support ([VSCode extension](https://marketplace.visualstudio.com/items?itemName=jaydenlin.ctags-support), built into vim/emacs) Sample: ``` boot.initrd.luks.reusePassphrases nixos/modules/system/boot/luksroot.nix 533 ```

# ctags on nix * `nix-doc tags .` in nixpkgs, then in vim you can `:tj fixedPoints`, `C-]`, etc. * [WIP ctags for NixOS options](https://github.com/NixOS/nixpkgs/pull/249243)

# demo (ctags on nix) FIXME

# summary (static analysis)

# dynamic analysis * Static analysis often doesn't cut it, power tools time! * `nix repl` is your friend

# using the repl (nix) * `nix repl -f .`, `nix repl -f '<nixpkgs>` * `:lf flake-ref` (more on this later) * `:e derivation-or-function` to open in editor

# getting docs in the repl <pre><code class="text"><script type="text/template"> nix-repl> :doc builtins.head Synopsis: builtins.head list Return the first element of a list; abort evaluation if the argument isn’t a list or is an empty list. You can test whether a list is empty by comparing it with []. </script></code></pre>

# getting docs in the repl (ii) <pre><code class="text"><script type="text/template"> nix-repl> :doc haskell.lib.overrideSrc error: value does not have documentation </script></code></pre> Oops. Related: * [RFC 0145](https://github.com/NixOS/rfcs/pull/145) * [nix#3904](https://github.com/NixOS/nix/issues/3904)

# nix-doc? <pre><code class="nix"><script type="text/template"> { pkgs, ... }: { nix.extraOptions = '' plugin-files = ${pkgs.nix-doc}/lib/libnix_doc_plugin.so ''; } </script></code></pre>

# using nix-doc with the repl <pre><code class="text"><script type="text/template"> nix-repl> builtins.doc haskell.lib.overrideSrc Override the sources for the package and optionally the version. This also takes of removing editedCabalFile. func = drv: src: ... # /nix/store/wl5m5xfayd69ycyspzyd4rilfgl6wmh0-source/pkgs/development/haskell-modules/lib/default.n ix:285 null </script></code></pre>

# `builtins.unsafeGetAttrPos` Probably the biggest dynamic analysis hammer. <pre><code class="text"><script type="text/template"> nix-repl> builtins.doc pkgs.fetchFromGitHub error: value is a set while a lambda was expected nix-repl> :e pkgs.fetchFromGitHub error: package 'fetchFromGitHub' has no source location information nix-repl> builtins.unsafeGetAttrPos "fetchFromGitHub" pkgs { column = 3; file = "/nix/store/wl5m5xfayd69ycyspzyd4rilfgl6wmh0-source/pkgs/top-level/all-package s.nix"; line = 1193; } </script></code></pre>

# summary (dynamic analysis) * `:e function/package` * \+ works pretty well * \+ built-in * \- issues with wrappers (e.g. `fetchFromGitHub`: `lib.makeOverridable`) * \- doesn't help for attr sets * `builtins.doc` (nix-doc) * \+ concise output on functions * \- issues with wrappers * `builtins.unsafeGetAttrPos` * \+ can get you close to the source if other things fail due to abstraction * \- unergonomic args and return value for interactive use

# NixOS ctags ([PR](https://github.com/NixOS/nixpkgs/pull/249243)) With the PR checked out: <pre><code class="text"><script type="text/template"> $ nix build -f nixos/release.nix optionsCtags -o opts-tags </script></code></pre> Then add `opts-tags` to ctags search path (`:set tags+=opts-tags` in vim).

# NixOS ctags (demo) FIXME

# NixOS in the repl Getting the configuration into the repl: * Non-flakes: `nix repl -I nixos-config=/path/to/configuration.nix -f <nixpkgs/nixos>` * Flakes: `nix repl`, `:lf .`, `nixosConfigurations.xx.{....}`

# NixOS in the repl (demo) FIXME <!-- nix-repl> config.documentation.man.generateCaches true nix-repl> options.documentation.man.generateCaches { __toString = «lambda @ /nix/store/wl5m5xfayd69ycyspzyd4rilfgl6wmh0-source/lib/modules.nix:807:22» ; _type = "option"; declarations = [ ... ]; default = false; definitions = [ ... ]; definitionsWith Locations = [ ... ]; description = "Whether to generate the manual page index caches.\nThis allows searching for a page or\nkeyword using utilities like {manpage}`apropos(1)`\nand the `-k` option of \n{manpage}`man(1)`.\n"; files = [ ... ]; highestPrio = 1000; isDefined = true; loc = [ ... ]; opti ons = [ ... ]; type = { ... }; value = true; } nix-repl> options.documentation.man.generateCaches.definitions [ true ] nix-repl> options.documentation.man.generateCaches.definitionsWithLocations [ { ... } ] nix-repl> :p options.documentation.man.generateCaches.definitionsWithLocations [ { file = "/nix/store/wl5m5xfayd69ycyspzyd4rilfgl6wmh0-source/nixos/modules/programs/fish.nix"; va lue = true; } ] -->