• Nix Core Ecosystem, Nix, NixOS, Nix Lang, Nixpkgs are all distinctly different; related things which can be confusing for beginners this article explains them.

• nixpkgs: Vast package repository

• How Nix Works

• Nix Reference Manual Data Types The main Data Types you'll come across in the Nix ecosystem

• NixOS Wiki

• nix.dev: Has become the top respected source of information in my opinion. There is a lot of great stuff in here, and they actively update the information.

Is a language feature where a string, path, or attribute name can contain expressions enclosed in ${ }. This construct is called interpolated string, and the expression inside is an interpolated expression.

string interpolation.

Rather than writing:

let path = "/usr/local"; in "--prefix=${path}"

• This evaluates to "--prefix=/usr/local". Interpolated expressions must evaluate to a string, path, or an attribute set with an outPath or __toString attribute.

• In Nix, the process of managing software starts with package definitions. These are files written in the Nix language that describe how a particular piece of software should be built. These package definitions, when processed by Nix, are translated into derivations.

• At its core, a derivation in Nix is a blueprint or a recipe that describes how to build a specific software package or any other kind of file or directory. It's a declarative specification of:

• Inputs: What existing files or other derivations are needed as dependencies.

• Build Steps: The commands that need to be executed to produce the desired output.

• Environment: The specific environment (e.g., build tools, environment variables) required for the build process.

• Outputs: The resulting files or directories that the derivation produces.

Think of a package definition as the initial instructions, and the derivation as the detailed, low-level plan that Nix uses to actually perform the build."

Again, a derivation is like a blueprint that describes how to build a specific software package or any other kind of file or directory.

Key Characteristics of Derivations:

• Declarative: You describe the desired outcome and the inputs, not the exact sequence of imperative steps. Nix figures out the necessary steps based on the builder and args.

• Reproducible: Given the same inputs and build instructions, a derivation will always produce the same output. This is a cornerstone of Nix's reproducibility.

• Tracked by Nix: Nix keeps track of all derivations and their outputs in the Nix store. This allows for efficient management of dependencies and ensures that different packages don't interfere with each other.

• Content-Addressed: The output of a derivation is stored in the Nix store under a unique path that is derived from the hash of all its inputs and build instructions. This means that if anything changes in the derivation, the output will have a different path.

Here's a simple Nix derivation that creates a file named hello in the Nix store containing the text "Hello, World!":

{pkgs ? import <nixpkgs> {}}:
pkgs.stdenv.mkDerivation {
  name = "hello-world";

  dontUnpack = true;

  # No need for src = null; when dontUnpack = true;
  # src = null;

  buildPhase = ''
     # Create a shell script that prints "Hello, World!"
    echo '#!${pkgs.bash}/bin/bash' > hello-output-file # Shebang line
    echo 'echo "Hello, World!"' >> hello-output-file # The command to execute
    chmod +x hello-output-file # Make it executable
  '';

  installPhase = ''
    mkdir -p $out/bin
    cp hello-output-file $out/bin/hello # Copy the file from build directory to $out/bin
  '';

  meta = {
    description = "A simple Hello World program built with Nix";
    homepage = null;
    license = pkgs.lib.licenses.unfree; # Ensure this is pkgs.lib.licenses.unfree
    maintainers = [];
  };
}

And a default.nix with the following contents:

{ pkgs ? import <nixpkgs> {} }:

import ./hello.nix { pkgs = pkgs; }

• { pkgs ? import <nixpkgs> {} }: This is a function that takes an optional argument pkgs. We need Nixpkgs to access standard build environments like stdenv.

• pkgs.stdenv.mkDerivation { ... }: This calls the mkDerivation function from the standard environment (stdenv). mkDerivation is the most common way to define software packages in Nix.

• name = "hello-world";: Human-readable name of the derivation

• The rest are the build phases and package metadata.

To use the above derivation, save it as a .nix file (e.g. hello.nix). Then build the derivation using,:

nix-build
this derivation will be built:
  /nix/store/9mc855ijjdy3r6rdvrbs90cg2gf2q160-hello-world.drv
building '/nix/store/9mc855ijjdy3r6rdvrbs90cg2gf2q160-hello-world.drv'...
Running phase: patchPhase
Running phase: updateAutotoolsGnuConfigScriptsPhase
Running phase: configurePhase
no configure script, doing nothing
Running phase: buildPhase
Running phase: installPhase
Running phase: fixupPhase
shrinking RPATHs of ELF executables and libraries in /nix/store/2ydxh5pd9a6djv7npaqi9rm6gmz2f73b-hello-world
checking for references to /build/ in /nix/store/2ydxh5pd9a6djv7npaqi9rm6gmz2f73b-hello-world...
patching script interpreter paths in /nix/store/2ydxh5pd9a6djv7npaqi9rm6gmz2f73b-hello-world
stripping (with command strip and flags -S -p) in  /nix/store/2ydxh5pd9a6djv7npaqi9rm6gmz2f73b-hello-world/bin
/nix/store/2ydxh5pd9a6djv7npaqi9rm6gmz2f73b-hello-world

• Nix will execute the buildPhase and installPhase

• After a successful build, the output will be in the Nix store. You can find the exact path by looking at the output of the nix build command (it will be something like /nix/store/your-hash-hello-world).

Run the "installed" program:

./result/bin/hello

• This will execute the hello file from the Nix store and print "Hello, World!".

• nix.dev nixlang-basics

• learn nix in y minutes

• nix onepager

• awesome-nix

• zero-to-nix nix lang

• nix-pills basics of nixlang

The Nix expression evaluator has a bunch of functions and constants built in:

• toString e: (Convert the expression e to a string)

• import path: (Load, parse and return the Nix expression in the file path)

• throw x: (Throw an error message x. Usually stops evaluation)

• map f list: (Apply the function f to each element in the list)

• Built-in Functions

• Nix Operators

Because of currying, you can apply arguments to a Nix function one at a time. This is called partial application. When you provide only some of the expected arguments, you get a new function that "remembers" the provided arguments and waits for the rest.

Example:

Using our greeting function again:

nix repl
nix-repl> greeting = prefix: name: "${prefix}, ${name}!";
nix-repl> helloGreeting = greeting "Hello";
nix-repl> helloGreeting "Alice"
"Hello, Alice"

• helloGreeting is now a new function. It has already received the prefix argument ("Hello"), when we provide the second argument we get "Hello, Alice!"

Benefits of Partial Application:

• Creating Specialized Functions: You can create more specific functions from general ones by fixing some of their parameters.

• Adapting to Higher-Order Functions: Many functions that operate on other functions (like map and filter) expect functions with a certain number of arguments. Partial application allows you to adapt existing functions to fit these requirements.

• nix.dev Nix Lang Basics

• nix pills Functions and Imports

• zero-to-nix Nix Lang

• A tour of Nix "Functions"

• learn Nix in y minutes

• noogle function library

• An attribute set is a collection of name-value pairs called attributes:

• Attribute sets are written enclosed in curly braces {}. Attribute names and attribute values are separated by an equal sign =. Each value can be an arbitrary expression, terminated by a semicolon ;.

Example:nix.dev reference This defines an attribute set with attributes named:

• x with the value 123, an integer
• text with the value "Hello", a string
• y where the value is the result of applying the function f to the attribute set {bla = 456; }

{
 x = 123;
 text = "Hello";
 y = f { bla = 456; };
}

{ a = "Foo"; b = "Bar"}.a
 "Foo"

• Attributes can appear in any order. An attribute name may only occur once in each attribute set.

❗ Remember {} is a valid attribute set in Nix.

• The following is a function with an attribute set argument, remember that anytime you see a : in Nix code it means this is a function. To the left is the function arguments and to the right is the function body:

{ a, b }: a + b

• The simplest possible NixOS Module:

{ ... }:
{
}

{
  config,
  lib,
  pkgs,
  ...
}

• Inputs: The module takes the above inputs and ... (catch-all for other args)

  • config: Allows the module to read option values (e.g. config.programs.vim.enable). It provides access to the evaluated configuration.

  • lib: The Nixpkgs library, giving us helper functions like mkEnableOption , mkIf, and mkOverride.

  • pkgs: The Nixpkgs package set, used to access packages like pkgs.vim

  • ...: Allows the module to accept additional arguments, making it flexible for extension in the future.

Key Takeaways: A NixOS module is typically a function that can include config, lib, and pkgs, but it doesn’t require them. The ... argument ensures flexibility, allowing a module to accept extra inputs without breaking future compatibility. Using lib simplifies handling options (mkEnableOption, mkIf, mkOverride) and helps follow best practices. Modules define options, which users can set in their configuration, and config, which applies changes based on those options.

2. Local Configuration Reference:

let
  cfg = config.programs.vim;
in

• This is a local alias. Instead of typing config.programs.vim over and over, the module uses cfg.

3. Option Declaration

options.programs.vim = {
  enable = lib.mkEnableOption "Vi IMproved, an advanced text";
  defaultEditor = lib.mkEnableOption "vim as the default editor";
  package = lib.mkPackageOption pkgs "vim" { example = "vim-full"; };
};

This defines three user-configurable options:

• enable: Turns on Vim support system-wide.

• defaultEditor: Sets Vim as the system's default $EDITOR.

• package: lets the user override which Vim package is used.

mkPackageOption is a helper that defines a package-typed option with a default (pkgs.vim) and provides docs + example.

4. Conditional Configuration

config = lib.mkIf (cfg.enable || cfg.defaultEditor) {

• This block is only activated if either programs.vim.enable or defaultEditor is set.

5. Warnings

warnings = lib.mkIf (cfg.defaultEditor && !cfg.enable) [
  "programs.vim.defaultEditor will only work if programs.vim.enable is enabled,
   which will be enforced after the 24.11 release"
];

• Gives you a soft warning if you try to set defaultEditor = true without also enabling Vim.

6. Actual System Config Changes

environment = {
  systemPackages = [ cfg.package ];
  variables.EDITOR = lib.mkIf cfg.defaultEditor (lib.mkOverride 900 "vim");
  pathsToLink = [ "/share/vim-plugins" ];
};

• It adds Vim to your systemPackages, sets $EDITOR if defaultEditor is true, and makes /share/vim-plugins available in the environment.

Traditionally, NixOS configurations often implicitly rely on the version of Nixpkgs available when nixos-rebuild is run. However, for more robust and reproducible setups, especially in collaborative environments or when rolling back to specific configurations, explicitly locking these dependencies to specific versions becomes crucial.

In the following example, we'll explore how to use a tool called npins to manage and lock the dependencies of a NixOS configuration, ensuring a more predictable and reproducible system. This will involve setting up a project structure and using npins to pin the specific version of Nixpkgs our configuration relies on.

This is the file structure:

❯ tree
.
├── configuration.nix
├── default.nix
├── desktop.nix
└── npins
    ├── default.nix
    └── sources.json

This uses npins for dependency locking. Install it and run this in the project

directory:

npins init

Create a default.nix with the following:

# default.nix
{ system ? builtins.currentSystem, sources ? import ./npins, }:
let
  pkgs = import sources.nixpkgs {
    config = { };
    overlays = [ ];
  };
  inherit (pkgs) lib;
in lib.makeScope pkgs.newScope (self: {

  shell = pkgs.mkShell { packages = [ pkgs.npins self.myPackage ]; };

    # inherit lib;

  nixosSystem = import (sources.nixpkgs + "/nixos") {
    configuration = ./configuration.nix;
  };

  moduleEvale = lib.evalModules {
    modules = [
      # ...
    ];
  };
})

A configuration.nix with the following:

# configuration.nix
{
  boot.loader.grub.device = "nodev";
  fileSystems."/".device = "/devst";
  system.stateVersion = "25.05";

  # declaring options means to declare a new option
  # defining options means to define a value of an option
  imports = [
    # ./main.nix
     ./desktop.nix # Files
    # ./minimal.nix
  ];

  # mine.desktop.enable = true;
}

And a desktop.nix with the following:

# desktop.nix
{ pkgs, lib, config, ... }:

{
  imports = [];

  # Define an option to enable or disable desktop configuration
  options.mine.desktop.enable = lib.mkEnableOption "desktop settings";

  # Configuration that applies when the option is enabled
  config = lib.mkIf config.mine.desktop.enable {
    environment.systemPackages = [ pkgs.git ];
  };
}

mkEnableOption defaults to false. Now in your configuration.nix you can uncomment mine.desktop.enable = true; to enable the desktop config and vice-versa.

You can test that this works by running:

nix-instantiate -A nixosSystem.system

• nix-instantiate performs only the evaluation phase of Nix expressions. During this phase, Nix interprets the Nix code, resolves all dependencies, and constructs derivations but does not execute any build actions. Useful for testing.

To check if this worked and git is installed in systemPackages you can load it into nix repl but first you'll want lib to be available so uncomment this in your default.nix:

# default.nix
inherit lib;

Rerun nix-instantiate -A nixosSystem.system

Then load the repl and check that git is in systemPackages:

nix repl -f .
nix-repl> builtins.filter (pkg: lib.hasPrefix "git" pkg.name) nixosSystem.config.environment.systemPackages

This shows the path to the derivation

Check that mine.desktop.enable is true

nix-repl> nixosSystem.config.mine.desktop.enable
true

As demonstrated with npins, explicitly managing the dependencies of your NixOS modules is a powerful technique for ensuring the long-term stability and reproducibility of your system configurations. By pinning specific versions of Nixpkgs and other resources, you gain greater control over your environment and reduce the risk of unexpected changes due to upstream updates.

{
  inherit (pkgs) rustup evcxr nix-prefetch-git;
}

is equivalent to:

{
  rustup = pkgs.rustup;
  evcxr = pkgs.evcxr;
  nix-prefetch-git = pkgs.nix-prefetch-git;
}

Applying builtins.attrValues produces:

[ pkgs.evcxr pkgs.nix-prefetch-git pkgs.rustup ]

• As you can see only the values are included in the list, not the keys. This is more explicit and declarative but can be more complicated, especially for a beginner.

• builtins.attrValues returns the values of all attributes in the given set, sorted by attribute name. The above expression turns into something like the following avoiding bringing every attribute name from nixpkgs into scope.

A more straightforward example:

attrValues {c = 3; a = 1; b = 2;}
=> [1 2 3]

• WritingNixOsModules

• NixWikiNixOSModules

• nix.dev A basic module

• ModuleSystemDeepDive

• xeiaso Nixos Modules for fun & profit

• NixOS Flakes Book Module System

Videos

NixHour Writing NixOS modules -- This example is from this video infinisilModules

tweagModuleSystemRecursion

Flake references (flakerefs) are a way to specify the location of a flake. They have two different formats:

Attribute set representation:

{
  type = "github";
  owner = "NixOS";
  repo = "nixpkgs";
}

or URL-like syntax:

github:NixOS/nixpkgs

These are used on the command line as a more convenient alternative to the attribute set representation. For instance, in the command

nix build github:NixOS/nixpkgs#hello

github:NixOS/nixpkgs is a flake reference (while hello is an output attribute). They are also allowed in the inputs attribute of a flake, e.g.

inputs.nixpkgs.url = "github:NixOS/nixpkgs";

is equivalent to

inputs.nixpkgs = {
  type = "github";
  owner = "NixOS";
  repo = "nixpkgs";
};

-- nix.dev flake-references

nix flake provides subcommands for creating, modifying and querying Nix Flakes. Flakes are the unit for packaging Nix code in a reproducible and discoverable way. They can have dependencies on other flakes, making it possible to have multi-repository Nix projects.

— From nix.dev Reference Manual

• The main thing to note here is that nix flake is used to manage Nix flakes and that Flake commands are whitespace separated rather than hyphen - separated.

• Flakes do provide some advantages when it comes to discoverability of outputs.

• For Example, two helpful commands to inspect a flake are:

  • nix flake show command: Show the outputs provided by a flake.

  • nix flake check command: check whether the flake evaluates and run its tests.

  • Any Nix CLI command that is run against a flake -- like nix build, nix develop, nix flake show -- generate a flake.lock file for you.

    • The flake.lock file ensures that all flake inputs are pinned to specific revisions and that Flakes have purely deterministic outputs.

• Attribute sets are fundamental in Nix. They are simply collections of name-value pairs wrapped in curly braces {}.

  • Example, (click to see Output):

  let
    my_attrset = { foo = "bar"; };
  in
  my_attrset.foo
   "bar"
  

• Top-Level Attributes of a Flake:

  • Flakes have specific top-level attributes that can be accessed directly (without dot notation). The most common ones are inputs, outputs, and nixConfig.

• practical-nix-flakes

• Nix Flakes an Introduction

• tweag nix-flakes

• NixOS-wiki Flakes

• nix.dev flakes

• flakes-arent-real

• wombats-book-of-nix

• zero-to-nix flakes

• nixos-and-flakes-book

• FlakeHub

In a NixOS system, everything is built from a single "system derivation." The command nix-build '<nixpkgs/nixos>' -A system initiates this build process.

The -A system part tells Nix to focus on the system attribute defined in the '<nixpkgs/nixos>' file (which is essentially ./default.nix within the Nixpkgs repository).

This system attribute is specifically the NixOS option system.build.toplevel . Think of system.build.toplevel as the very top of the configuration hierarchy for your entire NixOS system. Almost every setting you configure eventually influences this top-level derivation, often through a series of intermediate steps.

Key Takeaway: system.build.toplevel is the ultimate output that defines your entire NixOS system.

Here's an example of how a high-level option can lead down to a low-level system configuration:

• You enable Nginx with services.nginx.enable = true;.
• This setting influences the lower-level option systemd.services.nginx.
• Which, in turn, affects the even lower-level option systemd.units."nginx.service".
• Ultimately, this leads to the creation of a systemd unit file within environment.etc."systemd/system".
• Finally, this unit file ends up as result/etc/systemd/system/nginx.service within the final system.build.toplevel derivation.

1. Importing Modules: It recursively finds and includes all modules specified in imports = [ ... ]; statements.

2. Declaring Options: It collects all option declarations defined using options = { ... }; from all the modules and merges them. If the same option is declared in multiple modules, the module system handles this (details omitted for simplicity).

3. Defining Option Values: For each declared option, it gathers all the value assignments (defined using config = { ... }; or directly at the top level if no options or config are present) from all modules and merges them according to the option's defined type.

Important Note: Option evaluation is lazy, meaning an option's value is only computed when it's actually needed. It can also depend on the values of other options.

Let's look at an example of what not to do:

{ pkgs, lib, config, ... }:
{
imports = [];

# Defining an option at the top level

options.mine.desktop.enable = lib.mkEnableOption "desktop settings";

# This will cause an error because 'environment' and 'appstream'

# are not 'options' and 'config' is also present at the top level.

environment.systemPackages =
lib.mkIf config.appstream.enable [ pkgs.git ];

appstream.enable = true;
}

This will result in the error: error: Module has an unsupported attribute 'appstream' This is caused by introducing a top-level 'config' or 'options' attribute. Add configuration attributes immediately on the top level instead, or move all of them into the explicit 'config' attribute.

Key Takeaway: When you have options or config at the top level, all value assignments need to go inside the config block.

The Correct Way): Using the config Attribute

To fix the previous example, you need to move the value assignments for environment.systemPackages and appstream.enable inside the config attribute:

{ pkgs, lib, config, ... }:
{
imports = [];

# Defining an option at the top level

options.mine.desktop.enable = lib.mkEnableOption "desktop settings";

config = {
environment.systemPackages =
lib.mkIf config.appstream.enable [ pkgs.git ];

    appstream.enable = true;

};
}

Now, Nix knows that you are declaring an option (options.mine.desktop.enable) and then setting values for other options (environment.systemPackages, appstream.enable) within the config block.

Key Takeaway: The config attribute is used to define the values of options.

Implicit config: When options is Absent

If your module does not define either options or config at the top level, then any attributes you define directly at the top level are implicitly treated as being part of the config.

For example, this is valid:

{ pkgs, lib, config, ... }:
{
environment.systemPackages =
lib.mkIf config.appstream.enable [ pkgs.git ];

appstream.enable = true;
}

Nix will implicitly understand that environment.systemPackages and appstream.enable are configuration settings.

Key Takeaway: If no explicit options or config are present, top-level attributes are automatically considered part of the configuration.

Removing an Option: What Happens to config

Even if you remove the options declaration from a module that has a config section, the config = { environment.systemPackages = ... }; part will still function correctly, assuming the option it's referencing (appstream.enable in this case) is defined elsewhere (e.g., in an imported module).

1. Package Definition:

   • This is essentially a function written in the Nix language.

   • Nix language shares similarities with JSON but includes the crucial addition of functions.

   • It acts as the blueprint for creating a package.

2. Derivation:

   • When the package definition is evaluated by Nix, it results in a derivation.

   • A derivation is a concrete and detailed build plan.

   • It outlines the exact steps Nix needs to take: fetching source code, building dependencies, compiling code, and ultimately producing the desired output (the package).

3. Realization (Building the Package):

   • You don't get a pre-built "package" directly from the definition or the derivation.

   • The package comes into being when Nix executes the derivation. This process is often referred to as "realizing" the derivation.

Analogy: Think of a package definition as an architectural blueprint, the derivation as the detailed construction plan, and the realized package as the finished building.

1 Function Structure:

• The file starts with a function taking an attribute set of dependencies from Nixpkgs: { lib, setuptools, buildPythonApplication, fetchFromGitHub, slurp } :.

2. Derivation Creation:

• It calls buildPythonApplication, a specialized helper for Python packages (similar to stdenv.mkDerivation but pre-configured for Python). The rec keyword allows attributes within the derivation to refer to each other.

3. Package Metadata:

• pname and version define the package's name and version.

• The meta attribute provides standard package information like the homepage, description, license, maintainers, and supported platforms.

4. Source Specification:

• The src attribute uses fetchFromGitHub to download the source code from the specified repository and revision, along with its sha256 hash for verification.

5. Build and Runtime Dependencies:

• nativeBuildInputs: Lists tools required during the build process (e.g., setuptools for Python).

• propagatedBuildInputs: Lists dependencies needed at runtime (e.g., slurp).

6. Build Format:

• format = "pyproject"; indicates that the package uses a pyproject.toml file for its Python build configuration.

Integration within Nixpkgs

• Location: The swaytools definition resides in pkgs/tools/wayland/swaytools/default.nix.

• Top-Level Inclusion: It's made available as a top-level package in pkgs/top-level/all-packages.nix like this:

# all-packages.nix
swaytools = python3Packages.callPackage ../tools/wayland/swaytools { };

• python3Packages.callPackage is used here because swaytools is a Python package, and it ensures the necessary Python-related dependencies are correctly passed to the swaytools definition.

• To understand how derivations work, let's create a very basic example using a bash script as our builder.

Why a Builder Script?

• The builder attribute in a derivation tells Nix how to perform the build steps. A simple and common way to define these steps is with a bash script.

The Challenge with Shebangs in Nix

• In typical Unix-like systems, you might start a bash script with a shebang (#!/bin/bash or #!/usr/bin/env bash) to tell the system how to execute it. However, in Nix derivations, we generally avoid this.

• Reason: Nix builds happen in an isolated environment where the exact path to common tools like bash isn't known beforehand (it resides within the Nix store). Hardcoding a path or relying on the system's PATH would break Nix's stateless property.

The Importance of Statelessness in Nix

• Stateful Systems (Traditional): When you install software traditionally, it often modifies the core system environment directly. This can lead to dependency conflicts and makes rollbacks difficult.

• Stateless Systems (Nix): Nix takes a different approach. When installing a package, it creates a unique, immutable directory in the Nix store. This means:

  • No Conflicts: Different versions of the same package can coexist without interfering with each other.
  • Reliable Rollback: You can easily switch back to previous versions without affecting system-wide files.
  • Reproducibility: Builds are more likely to produce the same result across different machines if they are "pure" (don't rely on external system state).

Our builder Script

• For our first derivation, we'll create a simple builder.sh file in the current directory:

# builder.sh
declare -xp
echo foo > $out

• The command declare -xp lists exported variables (it's a bash builtin function).

• Nix needs to know where the final built product (the "cake" in our earlier analogy) should be placed. So, during the derivation process, Nix calculates a unique output path within the Nix store. This path is then made available to our builder script as an environment variable named $out. The .drv file, which is the recipe, contains instructions for the builder, including setting up this $out variable. Our builder script will then put the result of its work (in this case, the "foo" file) into this specific $out directory.

• As mentioned earlier we need to find the nix store path to the bash executable, common way to do this is to load Nixpkgs into the repl and check:

nix-repl> :l <nixpkgs>
Added 3950 variables.
nix-repl> "${bash}"
"/nix/store/ihmkc7z2wqk3bbipfnlh0yjrlfkkgnv6-bash-4.2-p45"

So, with this little trick we are able to refer to bin/bash and create our derivation:

nix-repl> d = derivation { name = "foo"; builder = "${bash}/bin/bash";
 args = [ ./builder.sh ]; system = builtins.currentSystem; }
nix-repl> :b d
[1 built, 0.0 MiB DL]

this derivation produced the following outputs:
  out -> /nix/store/gczb4qrag22harvv693wwnflqy7lx5pb-foo

• The contents of the resulting store path (/nix/store/...-foo) now contain the file foo, as intended. We have successfully built a derivation!

• Derivations are the primitive that Nix uses to define packages. “Package” is a loosely defined term, but a derivation is simply the result of calling builtins.derivation.

• NixPillsOurFirstDerivation

• NixPills-WorkingDerivation

• nix.dev-Derivations

• nix.dev-packagingExistingSoftware

• howToLearnNix-MyFirstDerivation

• howToLearnNix-DerivationsInDetail

• Sparky/blog-creatingASuperSimpleDerivation # How to learn Nix

• Sparky/blog-Derivations102

• ScriveNixWorkshop-nixDerivationBasics

• zeroToNix-Derivations

• Tweag-derivationOutputs

• theNixLectures-Derivations

• bmcgee-whatAreFixed-OutputDerivations

• A key benefit of Nix Flakes is their default enforcement of pure evaluation.

• In Nix, an impure operation depends on something outside its explicit inputs. Examples include:

  • User's system configuration
  • Environment variables
  • Current time

• Impurity leads to unpredictable builds that may differ across systems or time.

1. Setup:

   mkdir hello && cd hello/
   

2. Create flake.nix (Initial Impure Example):

   # flake.nix
   {
     outputs = { self, nixpkgs }: {
       myHello = (import nixpkgs {}).hello;
     };
   }
   

   • Note: Flakes don't have access to builtins.currentSystem directly.

3. Impure Build (Fails):

   nix build .#myHello
   

   • This fails because Flakes enforce purity by default.

4. Force Impure Build:

   nix build .#myHello --impure
   

5. Making the Flake Pure:

   # flake.nix
   {
     inputs = {
       nixpkgs.url = "github:NixOS/nixpkgs";
       flake-utils.url = "github:numtide/flake-utils";
     };
   
     outputs = { self, nixpkgs, flake-utils }:
       flake-utils.lib.eachDefaultSystem (system:
         let
           pkgs = nixpkgs.legacyPackages.${system};
         in {
           packages.myHello = pkgs.hello;
         }
       );
   }
   

   • flake-utils simplifies making flakes system-agnostic and provides the system attribute.

6. Pure Build (Success):

   nix build .#myHello
   

1. Setup:

   mkdir hello2 && cd hello2/
   

2. Create default.nix (Initial Impure Example):

   # default.nix
   { myHello = (import <nixpkgs> { }).hello; }
   

3. Build (Impure):

   nix-build -A myHello
   

4. Impurity Explained:

   nix repl
   nix-repl> <nixpkgs>
   /nix/var/nix/profiles/per-user/root/channels/nixos
   

   • <nixpkgs> depends on the user's environment (Nixpkgs channel), making it impure. Even with channels disabled, it relies on a specific Nixpkgs version in the store.

5. Achieving Purity: Using fetchTarball

   • GitHub allows downloading repository snapshots at specific commits, crucial for reproducibility.

   • Get Nixpkgs Revision from flake.lock (from the Flake example):

   # flake.lock
   "nixpkgs": {
     "locked": {
       "lastModified": 1746372124,
       "narHash": "sha256-n7W8Y6bL7mgHYW1vkXKi9zi/sV4UZqcBovICQu0rdNU=",
       "owner": "NixOS",
       "repo": "nixpkgs",
       "rev": "f5cbfa4dbbe026c155cf5a9204f3e9121d3a5fe0",
       "type": "github"
     },
   

6. Modify default.nix for Purity:

   # default.nix
   let
     nixpkgs = fetchTarball {
       url = "[https://github.com/NixOS/nixpkgs/archive/f5cbfa4dbbe026c155cf5a9204f3e9121d3a5fe0.tar.gz](https://github.com/NixOS/nixpkgs/archive/f5cbfa4dbbe026c155cf5a9204f3e9121d3a5fe0.tar.gz)";
       sha256 = "0000000000000000000000000000000000000000000000000000"; # Placeholder
     };
   in {
     myHello = (import nixpkgs {}).hello;
   }
   

   • Replace <nixpkgs> with fetchTarball and a specific revision. A placeholder sha256 is used initially.

7. Build (Nix provides the correct sha256):

   nix-build -A myHello
   

8. Verification: Both Flake and Traditional Nix builds now produce the same output path.

9. Remaining Impurities in Traditional Nix:

   • Default arguments to import <nixpkgs> {} can introduce impurity:
     • overlays: ~/.config/nixpkgs/overlays (user-specific)
     • config: ~/.config/nixpkgs/config.nix (user-specific)
     • system: builtins.currentSystem (machine-specific)

10. Making Traditional Nix Fully Pure:

    # default.nix
    {system ? builtins.currentSystem}:
    let
      nixpkgs = fetchTarball {
        url =
          "[https://github.com/NixOS/nixpkgs/archive/0243fb86a6f43e506b24b4c0533bd0b0de211c19.tar.gz](https://github.com/NixOS/nixpkgs/archive/0243fb86a6f43e506b24b4c0533bd0b0de211c19.tar.gz)";
        sha256 = "1qvdbvdza7hsqhra0yg7xs252pr1q70nyrsdj6570qv66vq0fjnh";
      };
    in {
      myHello = (import nixpkgs {
        overlays = [];
        config = {};
        inherit system;
      }).hello;
    }
    

    • Override impure defaults for overlays, config, and make system an argument.

11. Building with a Specific System:

    nix-build -A myHello --argstr system x86_64-linux
    

12. Pure Evaluation Mode in Traditional Nix:

    nix-instantiate --eval --pure-eval --expr 'fetchGit { url = ./.; rev = "b4fe677e255c6f89c9a6fdd3ddd9319b0982b1ad"; }'
    

    • Example of using --pure-eval.

    nix-build --pure-eval --expr '(import (fetchGit { url = ./.; rev = "b4fe677e255c6f89c9a6fdd3ddd9319b0982b1ad"; }) { system = "x86_64-linux"; }).myHello'
    

    • Building with a specific revision and system.

nix flake update

nix build .#myHello --override-input nixpkgs github:NixOS/nixpkgs/nixos-24.11

nix-shell -p niv
niv init

# default.nix
{ system ? builtins.currentSystem,
  sources ? import nix/sources.nix,
  nixpkgs ? sources.nixpkgs,
  pkgs ? import nixpkgs {
    overlays = [ ];
    config = { };
    inherit system;
  }, }: {
  myHello = pkgs.hello;
}

And build it with:

nix-build -A myHello

niv update nixpkgs --branch=nixos-unstable
nix-build -A myHello

# flake.nix
{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs";
    flake-utils.url = "github:numtide/flake-utils";
    home-manager.url = "github:nix-community/home-manager";
  };

  outputs = { self, nixpkgs, flake-utils, home-manager, ... }:
    flake-utils.lib.eachDefaultSystem (system:
      let pkgs = nixpkgs.legacyPackages.${system};
      in {
        packages.myHello = pkgs.hello;
        packages.x86_64-linux.homeManagerDocs =
          home-manager.packages.x86_64-linux.docs-html;
      });
}

nix flake update
nix flake show github:nix-community/home-manager

home-manager.inputs.follows = "nixpkgs";

Adding Home-Manager with Traditional Nix

nix repl
nix-repl> s = import ./nix/sources.nix
nix-repl> s.home-manager

{ system ? builtins.currentSystem, sources ? import nix/sources.nix
  , nixpkgs ? sources.nixpkgs, pkgs ? import nixpkgs {
    overlays = [ ];
    config = { };
    inherit system;
  }, }: {
  homeManagerDocs = (import sources.home-manager { pkgs = pkgs; }).docs;

  myHello = pkgs.hello;
}

nix-build -A homeManagerDocs

Conclusion

In this chapter, we've explored the key differences between traditional Nix and Nix Flakes, particularly focusing on how each approach handles purity, dependency management, and project structure. We've seen that while traditional Nix can achieve purity with careful configuration, Flakes enforce it by default, offering a more robust and standardized way to build reproducible environments. Flakes also streamline dependency management and provide a more structured project layout compared to the often ad-hoc nature of traditional Nix projects.

However, regardless of whether you're working with Flakes or traditional Nix, understanding how to debug and trace issues within your Nix code is crucial. When things go wrong, you'll need tools and techniques to inspect the evaluation process, identify the source of errors, and understand how your modules and derivations are being constructed.

In our next chapter, Debugging and Tracing Modules, we will delve into the world of Nix debugging. We'll explore various techniques and tools that can help you understand the evaluation process, inspect the values of expressions, and trace the execution of your Nix code, enabling you to effectively troubleshoot and resolve issues in both Flake-based and traditional Nix projects.

Chapter 9

• Debugging and Tracing NixOS Modules
• Example 2
• Example 3
• Example 4
• Summary
• More Functionality between modules
  • Infinite recursion error
  • Example 5
  • Tests
  • Test 2
  • Key Takeaways for Debugging NixOS Modules
• Conclusion

Debugging and Tracing NixOS Modules

• Other related post if you haven't read my previous post on modules, that may be helpful before reading this one:

  • nix-modules-explained

  • This post is my notes following Nix Hour 40. If it seems a little chaotic, try watching one. They are hard to follow if you're not extremely familiar with the concepts.

  • Nix Hour 40

Nix Code is particularly hard to debug because of (e.g. lazy evaluation, declarative nature, layered modules)

• The following simple Nix code snippet illustrates a basic NixOS module definition and how options are declared and configured. We'll use this example to demonstrate fundamental debugging techniques using nix-instantiate.

let
  lib = import <nixpkgs/lib>;
in
lib.evalModules {
  modules = [
    ({ lib, ... }: {
      options.foo = lib.mkOption {
        # type = lib.types.raw;
        type = lib.types.anything;
        # default = pkgs;
      };
      config.foo = {
        bar = 10;
        list = [1 2 3 ];
        baz = lib.mkDefault "baz";
      };
    })
    {
      foo.baz = "bar";
    }
  ];
}

• In the above code, adding lib to the function arguments isn't required but if you were to move the module to another file it would fail without it because lib comes from outside of it. So it's good practice to refer to lib in the modules themselves.

• You should always assign a type to your options, if you don't know which type to use you could use raw. raw is a type that doesn't do any processing. So if you were to assign the entire packages set to the option e.g. default = pkgs; it wouldn't recurseinto all the packages and try to evaluate them. There is also anything, that is useful if you do want to recurse into the values.

• The following is an example of how you would run this inside vim/neovim, the rest of the examples will be from the command line:

:!nix-instantiate --eval -A config.foo --strict

Output:

{ bar = 10; baz = "bar"; list = [ 1 2 3 ]; }

error: The option 'foo' is defined multiple times while it's expected to be
unique

nix-instantiate --eval --strict -A config.foo

nix-instantiate --eval --strict -A config.foo --show-trace

Example 2

# configuration.nix
{ lib, ... }: {
  boot.loader.grub.device = "nodev";
  fileSystems."/".device = "/devst";
  system.stateVersion = "24.11";
}

nix-instantiate '<nixpkgs/nixos>' --arg configuration ./configuration.nix -A system

/nix/store/kfcwvvpdbsb3xcks1s76id16i1mc3l5k-nixos-system-nixos-25.05pre-git.drv

{ lib, ... }: {
  boot.loader.grub.device = "nodev";
  fileSystems."/".device = "/devst";
  system.stateVersion = builtins.genList "24.11" null;
}

(stack trace truncated; use '--show-trace' to show the full, detailed trace)
error: expected an integer but found null: null

nix-instantiate '<nixpkgs/nixos>' --arg configuration ./configuration.nix -A system --show-trace

{lib, ...}: {
  boot.loader.grub.device = "nodev";
  fileSystems."/".device = "/devst";
  system.stateVersion = builtins.addErrorContext "AAAAAAAAAAAAAAAAA" (builtins.genList "24.11" null);
}

nix-instantiate '<nixpkgs/nixos>' --arg configuration ./configuration.nix -A system --show-trace`

 … while evaluating the attribute 'value'
     at /nix/store/ccfwxygjrarahgfv5865x2f828sjr5h0-source/lib/modules.nix:770:21:
      769|             inherit (module) file;
      770|             inherit value;
         |                     ^
      771|           }) module.config

   … AAAAAAAAAAAAAAAAA

   … while calling the 'genList' builtin
     at /home/jr/tests/configuration.nix:4:71:
        3|   fileSystems."/".device = "/devst";
        4|   system.stateVersion = builtins.addErrorContext "AAAAAAAAAAAAAAAAA"
         (builtins.genList "24.11" null);
         |                                                                       ^
        5| }

   … while evaluating the second argument passed to builtins.genList

   error: expected an integer but found null: null

Example 3

# default.nix
let
  lib = import <nixpkgs/lib>;
in
lib.evalModules {
  modules = [
    ({ lib, ... }: {
      options.ints = lib.mkOption {
        type = lib.types.attrsOf lib.types.int;
      };
      options.strings = lib.mkOption {
        type = lib.types.string;
        # type = lib.types.attrsOf lib.types.string;
        default = "foo";
      };
    })
  ];
}

nix-instantiate --eval --strict -A config.strings

evaluation warning: The type `types.string` is deprecated.
See https://github.com/NixOS/nixpkgs/pull/66346 for better alternative types.
"foo"

export NIX_PATH=nixpkgs=channel:nixpkgs-unstable
echo $NIX_PATH

nixpkgs=channel:nixpkgs-unstable

nix-instantiate --find-file nixpkgs

/nix/store/ydrgwsibghsyx884qz97zbs1xs93yk11-source

nix-instantiate --eval channel:nixpkgs-unstable -A path

/nix/store/ydrgwsibghsyx884qz97zbs1xs93yk11-source

# default.nix
let
  lib = import <nixpkgs/lib>;
in
  lib.evalModules {
    modules = [
      ({lib, ...}: {
        options.ints = lib.mkOption {
          type = lib.types.attrsOf lib.types.int;
        };
        options.strings = lib.mkOption {
          # type = lib.types.string;
          type = lib.types.attrsOf lib.types.string;
          default = {
            x = "foo";
          };
        };
        config = {
          strings = lib.mkOptionDefault {
            x = "bar";
          };
        };
      })
    ];
  }

nix-instantiate --eval --strict -A config.strings

evaluation warning: The type `types.string` is deprecated. See https://github.
com/NixOS/nixpkgs/pull/66346 for better alternative types.
{ x = "foobar"; }

# default.nix
let
  lib = import <nixpkgs/lib>;
in
  lib.evalModules {
    modules = [
      ({lib, ...}: {
        options.ints = lib.mkOption {
          type = lib.types.attrsOf lib.types.int;
        };
        options.strings = lib.mkOption {
          # type = lib.types.string;
          type = lib.types.attrsOf lib.types.str;
          # Sets the value with a lower priority: lib.mkOptionDefault
          default = {
            x = "foo";
          };
        };
        config = {
          strings = lib.mkOptionDefault {
            x = "bar";
          };
        };
      })
    ];
  }

error:
… while evaluating the attribute 'x'

… while evaluating the attribute 'value'
 at /nix/store/ydrgwsibghsyx884qz97zbs1xs93yk11-source/lib/modules.nix:1148:41:
 1147|
 1148|     optionalValue = if isDefined then { value = mergedValue; } else { };
     |                                         ^
 1149|   };

… while calling the 'foldl'' builtin
 at /nix/store/ydrgwsibghsyx884qz97zbs1xs93yk11-source/lib/options.nix:508:8:
  507|     else
  508|       (foldl' (
     |        ^
  509|         first: def:

(stack trace truncated; use '--show-trace' to show the full, detailed trace)

error: The option `strings.x' has conflicting definition values:
- In `<unknown-file>': "foo"
- In `<unknown-file>': "bar"
Use `lib.mkForce value` or `lib.mkDefault value` to change the priority on any of these definitions.

shell returned 1

Summary

• So types in the module system aren't just types in the conventional sense but they also specify the emerging behavior of these values.

• If we switch the type in the above example to types.lines you get this returned, { x = "foo\nbar"; }

• mkOptionDefault isn't typically something you should generally use, instead options have a default setting

• If you want to make sure that you set a default but if the user specifies it, it shouldn't get overridden. You should not set it in the following:

options.strings = lib.mkOption {
  type = lib.types.attrsOf lib.types.lines;
  default = {
    x = "foo";
  };
}

Because the above uses mkOptionDefault but instead in under the config attribute like the following:

# ...snip...
options.strings = lib.mkOption {
  type = lib.types.attrsOf lib.types.lines;
  # default = {
    # x = "foo";
  # };
};
config = {
  strings = {
    x = lib.mkDefault "foo";
  };
};
# ...snip...

let
  lib = import <nixpkgs/lib>;
in
  lib.evalModules {
    modules = [
      ({lib, ...}: {
        options.ints = lib.mkOption {
          type = lib.types.attrsOf lib.types.int;
        };
        options.strings = lib.mkOption {
          # type = lib.types.string;
          type = lib.types.attrsOf lib.types.str;
          # Sets the value with a lower priority: lib.mkOptionDefault
          #default = {
          #  x = "foo";
          #};
        };
        config.strings = {
          x = "foo";
        };
      })
      {
        config.strings = {
          y = "bar";
        };
      }
    ];
  }

Output:

• This works now because there's no difference between x and y

{ x = "foo"; y = "bar"; }

More Functionality between modules

let
  lib = import <nixpkgs/lib>;
in
  lib.evalModules {
    modules = [
      ({lib, ...}: {
        options.ints = lib.mkOption {
          type = lib.types.attrsOf lib.types.int;
        };
        options.strings = lib.mkOption {
          # type = lib.types.string;
          type = lib.types.attrsOf lib.types.str;
          # Sets the value with a lower priority: lib.mkOptionDefault
          #default = {
          #  x = "foo";
          #};
        };
        config.strings = {
          x = lib.mkDefault "foo";
        };
      })
      {
        config.strings = {
          x = "x";
          y = "bar";
        };
      }
    ];
  }

• The above command would cause a conflict without the x = lib.mkDefault foo And this is typically what you want to do for defaults and modules in things like nested configuration.

Output:

{ x = "x"; y = "bar"; }

Infinite recursion error

1. A common pitfall is to introduce a hard to debug error infinite recursion when shadowing a name. The simplest example for this is:

let a = 1; in rec { a = a; }

💡TIP: Avoid rec. Use let ... in Example:

let
 a = 1;
in {
 a = a;
 b = a + 2;
}

# default.nix
let
  lib = import <nixpkgs/lib>;
in
  lib.evalModules {
    modules = [
      ./module.nix
    ];
  }

# module.nix
{ lib, pkgs, ...}: {
  options.etc = lib.mkOption {
    type = lib.types.attrsOf lib.types.path;
    default = { };
    description = ''
      Specifies which paths are is /etc/
    '';
  };

  config._module.args.pkgs = import <nixpkgs> {
    config = {};
    overlays = [];
  };
  config.etc.foo = pkgs.writeText "foo" ''
    foo configuration
  '';
}

nix-instantiate --eval --strict -A config.etc

nix repl
nix-repl> :l <nixpkgs>
nix-repl> hello.out.out.out

«derivation /nix/store/b1vcpm321dwbwx6wj4n13l35f4y2wrfv-hello-2.12.1.drv»

nix-instantiate -A config.etc

warning: you did not specify '--add-root'; the result might be removed by the garbage collector
/nix/store/abyfp1rxk73p0n5kfilv7pawxwvc7hsg-foo.drv

# module.nix
{
  lib,
  pkgs,
  ...
}: {
  options.etc = lib.mkOption {
    type = lib.types.attrsOf (lib.types.attrsOf lib.types.path);
    default = {};
    description = ''
      Specifies which paths are in /etc/
    '';
  };

  config._module.args.pkgs = import <nixpkgs> {
    config = {};
    overlays = [];
  };
  config.etc.foo.bar = pkgs.writeText "foo" ''
    foo configuration
  '';
}

nix repl -f default.nix
nix-repl> config.etc
{
  foo = { ... };
}
nix-repl> config.etc.foo
{
  bar = «derivation /nix/store/abyfp1rxk73p0n5kfilv7pawxwvc7hsg-foo.drv»;
}

Example 5

# default.nix
let
  lib = import <nixpkgs/lib>;
in
  lib.evalModules {
    modules = [
      ./module.nix
    ];
  }

# module.nix
{
  lib,
  pkgs,
  config,
  ...
}: {
  options.etc = lib.mkOption {
    type = lib.types.attrsOf (lib.types.attrsOf lib.types.path);
    default = {};
    description = ''
      Specifies which paths are in /etc/
    '';
  };
  options.etcCombined = lib.mkOption {
    type = lib.types.package;
    default =
      pkgs.linkFarm "etc"
      (lib.mapAttrsToList (name: value: {
        name = name;
        path = value;
      }) config.etc);
  };

  config._module.args.pkgs = import <nixpkgs> {
    config = {};
    overlays = [];
  };
  config.etc.foo = pkgs.writeText "foo" ''
    foo configuration
  '';
  config.etc.bar = pkgs.writeText "bar" ''
    bar configuration
  '';
}

nix-instantiate -A config.etcCombined

/nix/store/3da61nmfk546qn2zpxsm57mq6vz6fjx8-etc.drv

nix-build -A config.etcCombined

these 3 derivations will be built:
/nix/store/41yfxq4af1vrs0rrgfk5gc36kmjc7270-bar.drv
/nix/store/abyfp1rxk73p0n5kfilv7pawxwvc7hsg-foo.drv
/nix/store/3da61nmfk546qn2zpxsm57mq6vz6fjx8-etc.drv
building '/nix/store/41yfxq4af1vrs0rrgfk5gc36kmjc7270-bar.drv'...
building '/nix/store/abyfp1rxk73p0n5kfilv7pawxwvc7hsg-foo.drv'...
building '/nix/store/3da61nmfk546qn2zpxsm57mq6vz6fjx8-etc.drv'...
/nix/store/ca3wyk5m3qhy8n1nbn0181m29qvp1klp-etc

nix-build -A config.etcCombined && ls result/ -laa

/nix/store/ca3wyk5m3qhy8n1nbn0181m29qvp1klp-etc
dr-xr-xr-x - root 31 Dec  1969  .
drwxrwxr-t - root 16 May 15:13  ..
lrwxrwxrwx - root 31 Dec  1969  bar -> /nix/store/1fsjyc2hmilab1qw6jfkf6cb767kz858-bar
lrwxrwxrwx - root 31 Dec  1969  foo -> /nix/store/wai5dycp0zx1lxg0rhpdxnydhiadpk05-foo

# linkFarm "myexample" [ { name = "hello-test"; path = pkgs.hello; }
# { name = "foobar"; path = pkgs.stack; } ]

Tests

mkdir passthru-tests && cd passthru-tests

# default.nix
let
  pkgs = import <nixpkgs> {};

  package = pkgs.runCommand "foo" {
    passthru.tests.simple = pkgs.runCommand "foo-test" {} ''
      if [[ "$(cat ${package})" != "foo" ]]; then
        echo "Result is not foo"
        exit 1
      fi
      touch $out
  '';
  } ''
    echo foo > $out
  '';
in
package

nix-build

nix-build -A passthru.tests

this derivation will be built:
/nix/store/pqpqq9x1wnsabzbsb52z4g4y4zy6p7yx-foo-test.drv
building '/nix/store/pqpqq9x1wnsabzbsb52z4g4y4zy6p7yx-foo-test.drv'...
/nix/store/7bbw2ban0mgkh4d59yz3cnai4aavwvb6-foo-test

let
  pkgs = import <nixpkgs> {};

  package =
    pkgs.runCommand "foo" {
      passthru.tests.simple = pkgs.runCommand "foo-test" {} ''
        if [[ "$(cat ${package})" != "foo" ]]; then
          echo "Result is not foo"
          exit 1
        fi
        touch $out
      '';

      passthru.tests.version = pkgs.testers.testVersion {
         package = package;
         version = "1.2";
     };

      # pkgs.writeShellApplication
      script = ''
        #!${pkgs.runtimeShell}
        echo "1.2"
      '';
      passAsFiles = [ "script" ];

    } ''
      cp "$scriptPath" "$out"
    '';
in
  package

nix-build -A passthru.tests

these 3 derivations will be built:
  /nix/store/lyz86bd78p7f3yjy1qky6annmggymcwd-foo.drv
  /nix/store/s4iawjy5zpv89dbkc3zz7z3ngz4jq2cv-foo-test.drv
  /nix/store/z3gi4pb8jn2h9rvk4dhba85fiphp5g4z-foo-test-version.drv
building '/nix/store/lyz86bd78p7f3yjy1qky6annmggymcwd-foo.drv'...
cp: cannot stat '': No such file or directory
error: builder for '/nix/store/lyz86bd78p7f3yjy1qky6annmggymcwd-foo.drv'
 failed with exit code 1;
     last 1 log lines:
     > cp: cannot stat '': No such file or directory
     For full logs, run:
       nix log /nix/store/lyz86bd78p7f3yjy1qky6annmggymcwd-foo.drv
error: 1 dependencies of derivation '/nix/store/z3gi4pb8jn2h9rvk4dhba85fiphp5g4z
-foo-test-version.drv' failed to build
error: build of '/nix/store/s4iawjy5zpv89dbkc3zz7z3ngz4jq2cv-foo-test.drv',
 '/nix/store/z3gi4pb8jn2h9rvk4dhba85fiphp5g4z-foo-test-version.drv' failed

nix-build

nix derivation show /nix/store/lyz86bd78p7f3yjy1qky6annmggymcwd-foo.drv | jq '.[].env'

{
  "__structuredAttrs": "",
  "buildCommand": "cp \"$scriptPath\" \"$out\"\n",
  "buildInputs": "",
  "builder": "/nix/store/xg75pc4yyfd5n2fimhb98ps910q5lm5n-bash-5.2p37/bin/bash",
  "cmakeFlags": "",
  "configureFlags": "",
  "depsBuildBuild": "",
  "depsBuildBuildPropagated": "",
  "depsBuildTarget": "",
  "depsBuildTargetPropagated": "",
  "depsHostHost": "",
  "depsHostHostPropagated": "",
  "depsTargetTarget": "",
  "depsTargetTargetPropagated": "",
  "doCheck": "",
  "doInstallCheck": "",
  "enableParallelBuilding": "1",
  "enableParallelChecking": "1",
  "enableParallelInstalling": "1",
  "mesonFlags": "",
  "name": "foo",
  "nativeBuildInputs": "",
  "out": "/nix/store/9mcrnddb6lf1md14v4lj6s089i99l5k7-foo",
  "outputs": "out",
  "passAsFile": "buildCommand",
  "passAsFiles": "script",
  "patches": "",
  "propagatedBuildInputs": "",
  "propagatedNativeBuildInputs": "",
  "script": "#!/nix/store/xg75pc4yyfd5n2fimhb98ps910q5lm5n-bash-5.2p37/bin/bash\necho \"1.2\"\n",
  "stdenv": "/nix/store/lgydi1gl5wqcw6k4gyjbaxx7b40zxrsp-stdenv-linux",
  "strictDeps": "",
  "system": "x86_64-linux"
}

nix derivation show /nix/store/lyz86bd78p7f3yjy1qky6annmggymcwd-foo.drv | jq
 '.[].env.buildCommand'

"cp \"$scriptPath\" \"$out\"\n"

nix derivation show /nix/store/lyz86bd78p7f3yjy1qky6annmggymcwd-foo.drv | jq
 '.[].env.buildCommand' -r

cp "$scriptPath" "$out"

let
  pkgs = import <nixpkgs> {};

  package =
    pkgs.runCommand "foo" {
      #passthru.tests.simple = pkgs.runCommand "foo-test" {} ''
      #  if [[ "$(cat ${package})" != "foo" ]]; then
      #    echo "Result is not foo"
      #    exit 1
      #  fi
      #  touch $out
      #'';

      passthru.tests.version = pkgs.testers.testVersion {
        package = package;
        version = "1.2";
      };

      # pkgs.writeShellApplication
      script = ''
        #!${pkgs.runtimeShell}
        echo "1.2"
      '';
      passAsFile = ["script"];
    } ''
      mkdir -p "$out/bin"
      cp "$scriptPath" "$out/bin/foo"
      chmod +x "$out/bin/foo"
    '';
in
  package

nix-build -A passthru.tests

these 2 derivations will be built:
  /nix/store/lqrlcd64dmpzkggcfzlnsnwjd339czd3-foo.drv
  /nix/store/c3kw4xbdlrig08jrdm5wis1dmv2gnqsd-foo-test-version.drv
building '/nix/store/lqrlcd64dmpzkggcfzlnsnwjd339czd3-foo.drv'...
building '/nix/store/c3kw4xbdlrig08jrdm5wis1dmv2gnqsd-foo-test-version.drv'...
1.2
/nix/store/zsbk5zawak68ailvkwi2gad2bqbqmdz9-foo-test-version

Key Takeaways for Debugging NixOS Modules

• nix-instantiate is Your Friend: Use nix-instantiate to evaluate your NixOS modules and pinpoint errors.

• Unlock Details with --show-trace: When errors occur, always append --show-trace to get a comprehensive stack trace, revealing the origin of the problem. Remember that in newer Nix versions, the most relevant parts of the trace are often at the bottom.

• Understand Option Types: Nix option types (raw, anything, string/str, lines, attrsOf) are not just about data types; they also dictate how values are merged and processed within the module system.

• Be Mindful of mkOptionDefault: While useful in specific scenarios, mkOptionDefault sets a lower priority default. For standard defaults that can be overridden by user configuration, define them directly within the config attribute using lib.mkDefault.

• Use builtins.addErrorContext: Enhance your custom error messages by providing specific context relevant to your module's logic using builtins.addErrorContext.

• Derivations vs. Evaluation: Be aware of the difference between evaluating expressions (--eval --strict) and instantiating derivations (nix-instantiate). Strict evaluation can trigger infinite recursion if it encounters unevaluated derivations with cyclic dependencies during attribute access.

• Explore with nix repl: The nix repl allows you to interactively explore Nix expressions and the outputs of derivations, providing insights into the structure and values within Nixpkgs.

Conclusion

This chapter has equipped you with essential techniques for debugging and tracing NixOS modules. We've explored how to use nix-instantiate and --show-trace to pinpoint errors, how to interpret Nix's often-verbose error messages, and how to leverage the nix repl for interactive exploration. Understanding option types and the nuances of mkOptionDefault is crucial for writing robust and predictable modules. We've also touched upon the distinction between evaluation and instantiation, and how that impacts debugging.

While these tools and techniques are invaluable for understanding and troubleshooting your own Nix configurations, they also become essential when you want to contribute to or modify the vast collection of packages and modules within Nixpkgs itself. Nixpkgs is where the majority of Nix packages and NixOS modules reside, and learning how to navigate and contribute to it opens up a whole new level of control and customization within the Nix ecosystem.

In the next chapter, Working with Nixpkgs Locally, we'll shift our focus to exploring and modifying Nixpkgs. We'll cover how to clone Nixpkgs, how to make changes to package definitions, and how to test those changes locally before contributing them back upstream. This chapter will empower you to not just use existing Nix packages, but also to customize and extend them to fit your specific needs.

Chapter 10

• Working with Nixpkgs Locally: Benefits and Best Practices
• I. Why Work with Nixpkgs Locally?
  • A. Faster Development Cycle
  • B. Enhanced Version Control
  • C. Flexible Debugging Capabilities
  • D. Streamlined Contribution Workflow
  • E. Up-to-Date Documentation Source
  • F. Optimized Storage and Performance
• II. Flake vs. Non-Flake Syntax for Local Nixpkgs
  • A. Flake Syntax (nix build .#<package>)
  • B. Non-Flake Syntax (nix-build -f . <package> or nix build -f . <package>)
  • III. Setting Up a Local Nixpkgs Repository Efficiently
  • A. Initial Clone: Shallow Cloning
  • B. Managing Branches with Worktrees
  • A. Online Search with search.nixos.org
  • B. Local Source Code Search with rg (ripgrep)
• V. Local Derivation Search with nix-locate
• VI. Key Benefits of Working with Nixpkgs Locally (Recap)
• VII. Best Practices and Tips from the Community

Working with Nixpkgs Locally: Benefits and Best Practices

• Nixpkgs, the package repository for NixOS, is a powerful resource for building and customizing software.
• Working with a local copy enhances development, debugging, and contribution workflows.
• This post covers setting up a local Nixpkgs repository, searching for dependencies, and leveraging its advantages, incorporating tips from the Nix community.

I. Why Work with Nixpkgs Locally?

• A local Nixpkgs repository offers significant advantages for Nix developers:

  A. Faster Development Cycle

  • Local searches for packages and dependencies are significantly quicker than querying remote repositories or channels.
  • This speedup is crucial for efficient debugging and rapid prototyping of Nix expressions.

  B. Enhanced Version Control

  • By pinning your local repository to specific commits or branches (e.g., nixos-unstable), you ensure build reproducibility.
  • This prevents unexpected issues arising from upstream changes in Nixpkgs.

  C. Flexible Debugging Capabilities

  • You can directly test and modify package derivations within your local copy.
  • This allows for quick fixes to issues like missing dependencies without waiting for upstream updates or releases.

  D. Streamlined Contribution Workflow

  • Developing and testing new packages or patches locally is essential before submitting them as pull requests to Nixpkgs.
  • A local setup provides an isolated environment for experimentation.

  E. Up-to-Date Documentation Source

  • The source code and comments within the Nixpkgs repository often contain the most current information about packages.
  • This can sometimes be more up-to-date than official, external documentation.

  F. Optimized Storage and Performance

  • Employing efficient cloning strategies (e.g., shallow clones) and avoiding unnecessary practices (like directly using Nixpkgs as a flake for local development) minimizes disk usage and build times.

II. Flake vs. Non-Flake Syntax for Local Nixpkgs

• When working with Nixpkgs locally, the choice between Flake and non-Flake syntax has implications for performance and storage:

  A. Flake Syntax (nix build .#<package>)

  • Treats the current directory as a flake, requiring evaluation of flake.nix.
  • For local Nixpkgs, this evaluates the flake definition in the repository root.
  • Performance and Storage Overhead: Flakes copy the entire working directory (including Git history if present) to /nix/store for evaluation. This can be slow and storage-intensive for large repositories like Nixpkgs.

  B. Non-Flake Syntax (nix-build -f . <package> or nix build -f . <package>)

  • -f . specifies the Nix expression (e.g., default.nix or a specific file) in the current directory.
  • Efficiency: Evaluates the Nix expression directly without copying the entire worktree to /nix/store. This is significantly faster and more storage-efficient for local development on large repositories.

III. Setting Up a Local Nixpkgs Repository Efficiently

IV. Debugging Missing Dependencies: A Practical Example

nix-build -A icat
# ... (Error log showing "fatal error: X11/Xlib.h: No such file or directory")

V. Local Derivation Search with nix-locate

VI. Key Benefits of Working with Nixpkgs Locally (Recap)

• Speed: Faster searches and builds compared to remote operations.
• Control: Full control over the Nixpkgs version.
• Up-to-Date Information: Repository source often has the latest details.

VII. Best Practices and Tips from the Community

Chapter 11

• Nix Pull Requests
• Build and Test the Changes
• Next Steps
• Conclusion

Nix Pull Requests

Pull requests communicate changes to a branch in a repository. Once a pull request is opened, you can review changes with collaborators and add follow-up commits.

• A pull request is a proposal to merge a set of changes from one branch into another. In a pull request, collaborators can review and discuss the proposed set of changes before they integrate the changes into the main codebase.

• Pull requests display the differences, or diffs, between the content in the source branch and the content in the target branch.

graph LR
    A[Your Local Repository] --> B(Feature Branch);
    B --> C{GitHub Repository};
    C -- "Open Pull Request" --> D[Pull Request on GitHub];
    D -- "Review & Discussion" --> D;
    D -- "Merge" --> E(Main Branch on GitHub);
    E --> F[Nixpkgs Users];

Explanation of the Diagram:

Flakes often rely on having access to the full history of the Git repository to correctly determine dependencies, identify specific revisions of inputs, and evaluate the flake. Not in all situations will a shallow clone work and this is one of them.

If you have any changes to your local copy of Nixpkgs make sure to stash them before the following:

git stash -u

• This command saves your uncommited changes (including staged files) temporarily. You can restore them later with git stash pop

Step 1 Clone Nixpkgs Locally

If you don't have Nixpkgs locally, you'll need to clone it:

git clone https://github.com/NixOS/nixpkgs.git

Step 2 Find a Relevant Pull Request

To find specifig commits and releases:

status.nixos.org provides the latest tested commits for each release - use when pinning to specific commits. List of active release channels - use when tracking latest channel versions.

The complete list of channels is available at nixos.org/channels

To find a relevant PR you can go to:

• Nixpkgs Pull Requests

• The following example actually uses the Nix Pull Requests the process is the same, but that is an important distinction.

• In the Filters enter stack trace for this example.

• The pull request I chose was 8623

Step 3 Add the Remote Repository (if necessary)

If the pull request is from a different repository than your local clone (as in the case of the nix PR while working in a nixpkgs clone), you need to add that repository as a remote. It's common to name the main Nixpkgs remote origin and other related repositories like nix as upstream.

Assuming you are in your nixpkgs clone and want to test a PR from the nix repository:

git remote add upstream https://github.com/NixOS/nix.git

Step 4 Fetch the Pull Request Changes

Fetch the Pull Request Information:

git fetch upstream refs/pull/8623/head:pr-8623

• This command fetches the branch named head from the pull request 8623 in the upstream remote and creates a local branch named pr-8623 that tracks it.

Output:

remote: Enumerating objects: 104651, done.
remote: Counting objects: 100% (45/45), done.
remote: Compressing objects: 100% (27/27), done.
remote: Total 104651 (delta 33), reused 20 (delta 18), pack-reused 104606 (from 1)
Receiving objects: 100% (104651/104651), 61.64 MiB | 12.56 MiB/s, done.
Resolving deltas: 100% (74755/74755), done.
From https://github.com/NixOS/nix
 * [new ref]             refs/pull/8623/head -> pr-8623
 * [new tag]             1.0                 -> 1.0
 * [new tag]             1.1                 -> 1.1
 * [new tag]             1.10                -> 1.10
 * [new tag]             1.11                -> 1.11
 * [new tag]             1.11.1              -> 1.11.1
 * [new tag]             1.2                 -> 1.2
 * [new tag]             1.3                 -> 1.3
 * [new tag]             1.4                 -> 1.4
 * [new tag]             1.5                 -> 1.5
 * [new tag]             1.5.1               -> 1.5.1
 * [new tag]             1.5.2               -> 1.5.2
 * [new tag]             1.5.3               -> 1.5.3
 * [new tag]             1.6                 -> 1.6
 * [new tag]             1.6.1               -> 1.6.1
 * [new tag]             1.7                 -> 1.7
 * [new tag]             1.8                 -> 1.8
 * [new tag]             1.9                 -> 1.9
 * [new tag]             2.0                 -> 2.0
 * [new tag]             2.2                 -> 2.2

Step 5 Checkout the Local Branch:

git checkout pr-8623

Or with the gh cli:

gh pr checkout 8623

Build and Test the Changes

• Now we want to see if the code changes introduced by the pull request actually build correctly within the Nix ecosystem.

nix build

Output:

error: builder for '/nix/store/rk86daqgf6a9v6pdx6vcc5b580lr9f09-nix-2.20.0pre20240115_20b4959.drv' failed with exit code 2;
   last 25 log lines:
   >
   >         _NIX_TEST_ACCEPT=1 make tests/functional/lang.sh.test
   >
   >     to regenerate the files containing the expected output,
   >     and then view the git diff to decide whether a change is
   >     good/intentional or bad/unintentional.
   >     If the diff contains arbitrary or impure information,
   >     please improve the normalization that the test applies to the output.
   > make: *** [mk/lib.mk:90: tests/functional/lang.sh.test] Error 1
   > make: *** Waiting for unfinished jobs....
   > ran test tests/functional/selfref-gc.sh... [PASS]
   > ran test tests/functional/store-info.sh... [PASS]
   > ran test tests/functional/suggestions.sh... [PASS]
   > ran test tests/functional/path-from-hash-part.sh... [PASS]
   > ran test tests/functional/gc-auto.sh... [PASS]
   > ran test tests/functional/path-info.sh... [PASS]
   > ran test tests/functional/flakes/show.sh... [PASS]
   > ran test tests/functional/fetchClosure.sh... [PASS]
   > ran test tests/functional/completions.sh... [PASS]
   > ran test tests/functional/build.sh... [PASS]
   > ran test tests/functional/impure-derivations.sh... [PASS]
   > ran test tests/functional/build-delete.sh... [PASS]
   > ran test tests/functional/build-remote-trustless-should-fail-0.sh... [PASS]
   > ran test tests/functional/build-remote-trustless-should-pass-2.sh... [PASS]
   > ran test tests/functional/nix-profile.sh... [PASS]
   For full logs, run:
     nix log /nix/store/rk86daqgf6a9v6pdx6vcc5b580lr9f09-nix-2.20.0pre20240115_20b4959.drv

• nix build (Part of the Nix Unified CLI):

  • Declarative: when used within a Nix flake (flake.nix), nix build is a bit more declarative. It understands the outputs defined in your flake.

  • Clearer Output Paths: nix build typically places build outputs in the ./result directory by default (similar to nix-build's result symlink)

  • Better Error Reporting: It gives more informative error messages.

  • Future Direction

Benefits of using nix build:

• Flake Integration: nix build naturally understands the flake's outputs.

• Development Shells: When you are in a nix develop shell, nix build is the more idiomatic way to build packages defined in your dev environment.

• Consistency: Using the unified CLI promotes a more consistent workflow.

Next Steps

As you can see this build failed, as for why the build failed, the key part of the error message is:

make: *** [mk/lib.mk:90: tests/functional/lang.sh.test] Error 1

• This suggests that one of the functional tests (lang.sh.test) failed. This happens when the expected output of the test doesn't match the actual output.

This can heppen when:

1. The test expectations are outdated due to changes in the codebase.

2. The test captures environment-specific or transient outputs that are not properly normalized.

3. The test includes impure or non-deterministic information, making it hard to verify.

To address this, _NIX_TEST_ACCEPT=1 is used as an override mechanism that tells the test framework: > "Accept whatever output is generated as the new expected result."

The message advises running:

_NIX_TEST_ACCEPT=1 make tests/functional/lang.sh.test

• This will regenerate the expected output files, allowing you to inspect what changed with git diff:

git diff tests/functional/lang.sh.test

• Verifies if Changes are Intentional: If the difference is reasonable and expected (due to a legitimate update in the logic), you can commit these changes to update the test suit. If not, you have to refine the test normalization process further.

If the changes seem valid, commit them:

git add tests/functional/lang.sh.test
git commit -m "Update expected test output for lang.sh.test"

Running the following will provide the full logs:

nix log /nix/store/rk86daqgf6a9v6pdx6vcc5b580lr9f09-nix-2.20.0pre20240115_20b4959.drv

Conclusion

Testing Nixpkgs pull requests is a vital part of contributing to a healthy and reliable Nix ecosystem. By following these steps, you can help ensure that changes are well-vetted before being merged, ultimately benefiting all Nix users. Your efforts in testing contribute significantly to the quality and stability of Nixpkgs.

Chapter 12

• Intro to Nushell on NixOS
• The Good
• The Bad
• Key Differences Between Nushell & Bash
• The Beautiful and Powerful
• Using Just and Justfiles
• Resources

Intro to Nushell on NixOS

• TL;DR:I recently switched default shells from zsh to nushell, this post is about some of the challenges and advantages of using nushell with NixOS.

• While the average user might not immediately see significant advantages, those who frequently work with structured data formats like JSON, YAML, and CSV – such as developers interacting with APIs, system administrators managing configurations, and data professionals – will likely find Nushell's native data handling and powerful pipeline capabilities a plus. Additionally, users who value a more consistent and safer scripting experience might appreciate Nushell's language-first design and features like strong typing.

• I'll start with some of the unique build design choices and unique features that I think make Nushell special, then show an example using Nushell to manipulate JSON data. Finally, I will highlight some of the visually appealing aspects of Nushell and lastly I share some resources for learning more.

The Good

• Nushell borrows concepts from many shells and languages and is itself both a programming language and a shell. Because of this, it has its own way of working with files, directories, websites, and more.

• Nushell is powerful and has many essential commands built directly into the shell ("internal" commands) rather than a link to an executable. You can use this set of commands across different operating systems, having this consistency is helpful when creating cross-platform code.

• When internal Nushell commands (like ls, open, where, get, sort-by, etc.) produce output, they generally do so in Nushell's structured data format (tables or records). This is the shell's native way of representing information.

• Beyond these foundational strengths, Nushell offers a range of unique features that enhance its functionality and make it particularly well-suited for data-heavy tasks. Here are some highlights that showcase its versatility.

Some Unique Features:

• Besides the built-in commands, Nushell has a standard library Nushell operates on structured data. You could call it a "data-first" shell and programming language.

• Also included, is a full-featured dataframe processing engine using Polars if you want to process large data efficiently directly in your shell, check out the Dataframes-Docs

• Multi-Line Editing:

• When writing a long command you can press Enter to add a newline and move to the next line. For example:

ls            |    # press enter
where name =~ |    # press enter, comments after pipe ok
get name      |    # press enter
mv ...$in ./backups/

• This allows you to cycle through the entire multi-line command using the up and down arrow keys and then customize different lines or sections of the command.

• You can manually insert a newline using Alt+Enter or Shift+Enter.

• The Reedline-Editor is powerful and provides good vi-mode or emacs support built in.

• It's default Ctrl+r history command is nice to work with out of the box.

• The explore command, is nu's version of a table pager, just like less but for table structured data:

$nu | explore --peek

• With the above command you can navigate with vim keybinds or arrow keys.

• These features demonstrate Nushell’s user-friendly interface, but what truly sets it apart is its underlying design as a structured data scripting language. This “language-first” approach powers many of its distinctive capabilities.

Unique design:

• Fundamentally designed as a structured data scripting language: and then it acts as a shell on top of that foundation. This "language first" approach is what gives it many of its distinctive features and makes it a powerful scripting language. I reiterate this here because of the implications of this. A few of those features are:

  • Pipelines of structured data: Unlike traditional shells that primarily deal with plain text streams, Nushell pipelines operate on tables of structured data. Each command can understand and manipulate this structured data directly.

  • Consistent syntax: Its syntax is more consistent and predictable compared to the often quirky syntax of Bash and Zsh, drawing inspiration from other programming languages.

  • Strong typing Nushell has a type system, which helps catch errors early and allows for more robust scripting.

  • First-class data types: It treats various data formats (like JSON, CSV, TOML) as native data types, making it easier to work with them. Because of this, Nushell aims to replace the need for external tools like jq, awk, sed, cut, and even some uses of grep and curl.

• Variables are Immutable by Default: Nushell's commands are based on a functional-style of programming which requires immutability, sound familiar?

• Nushell's Environment is Scoped: Nushell takes many design cues from compiled languages, one is that languages should avoid global mutable state. Shells have commonly used global mutation to update the environment, Nushell attempts to steer clear of this increasing reproducability.

• Single-use Environment Variables:

FOO=BAR $env.FOO
# => BAR

• Permanent Environment Variables: In your config.nu

# config.nu
$env.FOO = 'BAR'

• Coming-From-Bash

• These design principles make Nushell a powerful tool for scripting, but they’re best understood through a hands-on example. Let’s see how Nushell’s structured data capabilities shine in a common task: processing a JSON file.

Example: I wanted to provide a practical example to illustrate some of these "Good" features in action. And break it down for better understanding.

• Let's consider a common task: processing data from a JSON file. Imagine you have a file containing a list of users with their names and ages. With traditional shells, you'd likely need to rely on external tools like jq to parse and filter this data. However, Nushell can handle this directly within its own commands.

• For this example you could create a test directory and move to it:

mkdir test ; cd test

• Create a users.json with the following contents:

[
  { "name": "Alice", "age": 25 },
  { "name": "Bob", "age": 30 },
  { "name": "Charlie", "age": 20 }
]

• And create the following filter.nu that first converts users.json into its own internal structured data format with the open command, then to filters out people under 21 with the where control flow construct, then selects the name and age columns, sorts them by age, and finally converts them back to json and saves them to a file called filtered_users.json. A lot happening in a 6 line script.

open users.json           # Read JSON file into structured data
| where age > 21         # Filter users older than 21
| select name age        # Select only name and age columns
| sort-by age            # Sort by age
| to json                # Convert back to JSON
| save filtered_users.json # Save result to a new file

• The open command takes data from a file (or even a URL in some cases) and parses it and converts it into Nushells own internal structured data format. So this command isn't just showing you the contents of users.json but doing a conversion to Nu's special structured format.

open users.json
╭───┬─────────┬─────╮
│ # │  name   │ age │
├───┼─────────┼─────┤
│ 0 │ Alice   │  25 │
│ 1 │ Bob     │  30 │
│ 2 │ Charlie │  20 │
╰───┴─────────┴─────╯

• The source command in Nushell is used to execute the commands within a script file (like filter.nu) in the current Nushell environment. It's similar to running the script directly in the shell, but keeps the shell open for further use. In this example, source filter.nu runs the commands inside filter.nu, processing the users.json file and creating the filtered_users.json file:

source filter.nu
bat filtered_users.json
───────┬──────────────────────────────────────────────────────────────────────────────────────
       │ File: filtered_users.json
───────┼──────────────────────────────────────────────────────────────────────────────────────
   1   │ [
   2   │   {
   3   │     "name": "Alice",
   4   │     "age": 25
   5   │   },
   6   │   {
   7   │     "name": "Bob",
   8   │     "age": 30
   9   │   }
  10   │ ]
───────┴───────────────────────────────────────────────────────────────────────────────────

• As you can see, without needing any external tools, Nushell was able to read, filter, select, sort, and then re-serialize JSON data using a clear and concise pipeline. This demonstrates its power in handling structured data natively, making common data manipulation tasks within the shell significantly more streamlined and readable compared to traditional approaches.

In the filter.nu example:

open users.json           # Read JSON file into structured data
| where age > 21         # Filter users older than 21
| select name age        # Select only name and age columns
| sort-by age            # Sort by age
| to json                # Convert back to JSON
| save filtered_users.json # Save result to a new file

The Bad

• While the project is still maturing, the active community and ongoing improvements are promising. Don't get too discouraged by the following, there would be a bad section for any shell imo.

• There are many similarities so it can be easy to forget that some Bash (and POSIX in general) style constructs just won't work in Nushell. Considering that NixOS seems to have been designed for bash, even Zsh isn't fully compatable you may want to think twice before you choose Nushell as your default.

• The documentation is incomplete and written by devs for devs imo, it is quite a bit different from anything else I've seen so there is a bit of a learning curve. Nushell is generally still considered to be in a stage where it might not be the most seamless or trouble-free experience as a daily driver default shell for most users, especially on a system like NixOS known for its unique approach.

• The any-nix-shell project doesn't include Nushell as with many others because of it's lack of maturity.

• The following addition comes from Joey_McKur's sugggestion, on mentioning the job command as one of the biggest criticisms against Nu because it doesn't support background tasks. I should also note that Nushell's team is aware of these criticisms and actively working on improving job control.

Limited Feature Set Compared to Traditional Job Control:

• Lack of Full POSIX Job Control: Nushell's job control doesn't yet fully implement all the features and signals defined by POSIX job control (e.g., more nuanced signal handling, stopped jobs). While it covers the basics, users accustomed to advanced Bash job control might find it lacking.

• Foregrounding Behavior: There have been criticisms about how foregrounding jobs interacts with the terminal and potential issues with signal propagation.

Output Handling Challenges:

• Interleaved Output: Managing the output of multiple backgrounded jobs can sometimes be messy, with output from different jobs potentially interleaving in the terminal. While Nushell tries to handle this, it's not always as clean as desired.

• Redirection Complexity: Redirecting the input and output of backgrounded jobs can be less straightforward than in Bash, sometimes requiring more explicit handling.

Integration with Pipelines:

• Backgrounding Pipelines: Backgrounding complex pipelines with multiple stages can sometimes lead to unexpected behavior or difficulties in managing the entire pipeline as a single job.

Error Reporting:

• Difficult to Track Errors in Background Jobs: Identifying and debugging errors in backgrounded jobs can be less direct than with foreground processes, and the job command's output might not always provide sufficient information for troubleshooting.

• Many of Nushell’s challenges stem from its departure from traditional shell conventions, particularly those of Bash, which NixOS heavily relies on. To better understand these differences and how they impact your workflow, let’s compare Nushell’s static, structured approach to Bash’s dynamic, text-based model.

Key Differences Between Nushell & Bash

• && doesn't work use ; instead.

• > is used as the greater-than operator for comparisons:

"hello" | save output.txt

is equivalent to the following in bash:

echo "hello" > output.txt

• If you notice above the nushell command doesn't require an echo prefix, this is because Nushell has Implicit Return:

"Hello, World" == (echo "Hello, World")
# => true

• The above example shows that the string, "Hello, World" is equivalent to the output value from echo "Hello, World"

• Every Command Returns a Value:

let p = 7
print $p  # 7
$p * 6    # 42

• Understanding these differences highlights why Nushell feels so distinct from Bash, but it’s the shell’s advanced features and integrations that truly make it shine. Let’s dive into some of the beautiful and powerful tools and custom commands that elevate Nushell for NixOS users.

The Beautiful and Powerful

• Ctrl+t List Commands with carapace and fzf:

  

• Carapace Carapace-Bin Install:

  

• Carapace man example:

  

Custom Nushell Commands

• The following command allows you to choose which input to update interactively with fzf.

# nix.nu
# upgrade system packages
# `nix-upgrade` or `nix-upgrade -i`
def nix-upgrade [
  flake_path: string = "/home/jr/flake", # path that contains a flake.nix
  --interactive (-i) # select packages to upgrade interactively
]: nothing -> nothing {
  let working_path = $flake_path | path expand
  if not ($working_path | path exists) {
    echo "path does not exist: $working_path"
    exit 1
  }
  let pwd = $env.PWD
  cd $working_path
  if $interactive {
    let selections = nix flake metadata . --json
    | from json
    | get locks.nodes
    | columns
    | str join "\n"
    | fzf --multi --tmux center,20%
    | lines
    # Debug: Print selections to verify
    print $"Selections: ($selections)"
    # Check if selections is empty
    if ($selections | is-empty) {
      print "No selections made."
      cd $pwd
      return
    }
    # Use spread operator to pass list items as separate arguments
    nix flake update ...$selections
  } else {
    nix flake update
  }
  cd $pwd
  nh os switch $working_path
}

• The ns command is designed to search for Nix packages using nix search and present the results in a cleaner format, specifically removing the architecture and operating system prefix that nix search often includes.

def ns [
    term: string # Search target.
] {

    let info = (
        sysctl -n kernel.arch kernel.ostype
        | lines
        | {arch: ($in.0|str downcase), ostype: ($in.1|str downcase)}
    )

    nix search --json nixpkgs $term
        | from json
        | transpose package description
        | flatten
        | select package description version
        | update package {|row| $row.package | str replace $"legacyPackages.($info.arch)-($info.ostype)." ""}
}

• nufetch command:

# `nufetch` `(nufetch).packages`
def nufetch [] {
{
"kernel": $nu.os-info.kernel_version,
"nu": $env.NU_VERSION,
"packages": (ls /etc/profiles/per-user | select name | prepend [[name];
["/run/current-system/sw"]] | each { insert "number" (nix path-info --recursive
 ($in | get name) | lines | length) | insert "size" ( nix path-info -S
 ($in | get name) | parse -r '\s(.*)' | get capture0.0 | into filesize) | update
 "name" ($in | get name | parse -r '.*/(.*)' | get capture0.0 | if $in == "sw"
 {"system"} else {$in}) | rename "environment"}),
"uptime": (sys host).uptime
}
}

• duf command, I have mine aliased to df:

• ps command:

• Adding the following to your configuration.nix will show you the diff of the closures on rebuild:

# configuration.nix
# During system activation, compare the closure size difference between the
# current and new system and display a formatted table if significant changes are
# detected.
system.activationScripts.diff = ''
  if [[ -e /run/current-system ]]; then
    ${pkgs.nushell}/bin/nu -c "let diff_closure = (${pkgs.nix}/bin/nix store
     diff-closures /run/current-system '$systemConfig'); let table =
     (\$diff_closure | lines | where \$it =~ KiB | where \$it =~ → | parse -r
     '^(?<Package>\S+): (?<Old>[^,]+)(?:.*) → (?<New>[^,]+)(?:.*), (?<DiffBin>.*)$'
     | insert Diff { get DiffBin | ansi strip | into filesize } | sort-by -r Diff
     | reject DiffBin); if (\$table | get Diff | is-not-empty) { print \"\"; \$table
    | append [[Package Old New Diff]; [\"\" \"\" \"\" \"\"]] | append [[Package Old
     New Diff]; [\"\" \"\" \"Total:\" (\$table | get Diff | math sum) ]]
    | print; print \"\" }"
  fi
'';

• nix-list-system command lists all installed packages:

# list all installed packages
def nix-list-system []: nothing -> list<string> {
  ^nix-store -q --references /run/current-system/sw
  | lines
  | filter { not ($in | str ends-with 'man') }
  | each { $in | str replace -r '^[^-]*-' '' }
  | sort
}

• These custom Nushell commands showcase its flexibility, but sometimes you need to work around Nushell’s limitations, like compatability with certain NixOS tools. This is where just and justfiles come in, simplifying complex workflows and bridging gaps in Nushell’s functionality.

Using Just and Justfiles

• The following is my justfile that I keep right next to my flake.nix it simplifies some commands and makes things work that weren't working with nushell for my case, you'll have to change it to match your configuration. It's not perfect but works for my use case, take whats useful and leave the rest.

• You'll first need to install just to make use of justfiles.

# nix shell nixpkgs#just nixpkgs#nushell
set shell := ["nu", "-c"]
flake_path := "/home/jr/flake"
hostname := "magic"
home_manager_output := "jr@magic"

utils_nu := absolute_path("utils.nu")

default:
    @just --list
# Rebuild
[group('nix')]
fr:
    nh os switch --hostname {{hostname}} {{flake_path}}

# Flake Update
[group('nix')]
fu:
    nh os switch  --hostname {{hostname}} --update {{flake_path}}

# Update specific input
# Usage: just upp nixpkgs
[group('nix')]
upp input:
    nix flake update {{input}}
# Test
[group('nix')]
ft:
    nh os test --hostname {{hostname}} {{flake_path}}
# Collect Garbage
[group('nix')]
ncg:
    nix-collect-garbage --delete-old ; sudo nix-collect-garbage -d ; sudo /run/current-system/bin/switch-to-configuration boot

[group('nix')]
cleanup:
    nh clean all

• To list available commands type, (you must be in the same directory as the justfile): just

• So just fmt will run nix fmt.

• A lot of the .nu files came from this repo by BlindFS:

  • modern-dot-files he uses Nix Darwin so there are a few changes for NixOS. I found this through this_week_in_nu.

  • my-nu-config

  • The examples use this starship configAylur-dotfiles The logic on the bottom enables starship for Nushell, Zsh, and Bash!

  • If you wan't to use my config you'll have to enable the experimental-feature pipe-operators in the same place you enable flakes and nix-command.

• There are still situations where I need to switch to zsh or bash to get something to work i.e. nix-shell and a few others.

• From custom commands to justfile integrations, Nushell offers a wealth of tools to enhance your NixOS experience, even if occasional workarounds are needed. To dive deeper into Nushell and tailor it to your needs, here are some valuable resources to explore, from official documentation to community-driven configurations.

Resources