Help with `buildDhallPackage`


#1

I’m trying to generate a YAML file from a Dhall expression in Nix. The Dhall file that I pass to dhall-to-yaml is shown below and I will refer to it as “dhall source”

let Alacritty =
      https://raw.githubusercontent.com/cideM/dhall-alacritty/master/linux.dhall sha256:c9cf010f9ef1a7da4590d70571c1fa5114743f9096b8387b180bfb02f5cdffb1

let mono = ./mono.dhall with size = 12.0

in  Alacritty.Config::{
    , font = mono
    , shell = Some { program = "/usr/bin/fish", args = [ "-l" ] }
    , colors = ./papercolor.dhall
    , key_bindings = ./keys_common.dhall
    }
  with window.decorations = Alacritty.Window.Decoration.full
  with window.dynamic_padding = True
  with window.padding = { x = +10, y = +10 }

I made sure that everything in the dhall-alacritty dependency has a sha256 for the Dhall Prelude, so that I can use buildDhallPackage to load these things into the cache.

Here’s the Nix code

let
  sources = import ./nix/sources.nix;
  pkgs = import sources.nixpkgs {};

  prelude = pkgs.dhallPackages.buildDhallPackage {
    name = "dhall-lang-prelude";
    code = "${sources.dhall-lang}/Prelude/package.dhall";
  };

  # Fix in source
  preludeMap = pkgs.dhallPackages.buildDhallPackage {
    name = "dhall-lang-prelude";
    code = "${sources.dhall-lang}/Prelude/Map/Type";
  };


  linux = pkgs.dhallPackages.buildDhallPackage {
    name = "dhall-alacritty";
    code = "${sources.dhall-alacritty}/linux.dhall";
    dependencies = [
      prelude
      preludeMap
    ];
  };

  # config = pkgs.dhallPackages.buildDhallPackage {
  #   name = "alacritty_linux";
  #   code = ./src/linux.dhall;
  #   dependencies = [ 
  #     linux
  #   ];
  # };

in derivation {
  name = "alacritty_linux.yml";
  builder = "${pkgs.bash}/bin/bash";
  args = [ ./builder.sh ];
  dhalljson = pkgs.haskellPackages.dhall-json;
  coreutils = pkgs.coreutils;
  inherit linux;
  # inherit config;
  src = ./src;
  system = builtins.currentSystem;
}

The idea is to use that derivation in Home Manager to generate an alacritty.yml file. The problem is that if I uncomment alacritty_linux it complains about missing ./mono.dhall. So in other words, the dhall source file expects other files to be available in the same folder. If I run this file through buildDhallPackage it ends up in the Nix store, without the rest of the sources. But if I comment out the call to buildDhallPackage, my cache doesn’t have the necessary dependencies, it tries to make an HTTP call and that fails for obvious reasons in the Nix builder.

I’m missing a piece of the puzzle here and I don’t know what it is. My idea is that I should uncomment the commented out lines, and then pass inherit config to derivation but while also adding the necessary, local (~ relative) sources to name = "alacritty_linux"


#2

@yuuki: What’s happening is that Nix has special language support for path literals that interferes with what you are trying to do.

Specifically, when you write something of the form:

  config = pkgs.dhallPackages.buildDhallPackage {
    name = "alacritty_linux";
    code = ./src/linux.dhall;
    dependencies = [ 
      linux
    ];
  };

… Nix will first add ./src/linux.dhall to the Nix store in a process that is essentially the same as running this command:

$ nix-store --add ./src/linux.dhall
/nix/store/…-linux.dhall

… and then it will replace the path literal with the path that it added to the Nix store, like this:

  config = pkgs.dhallPackages.buildDhallPackage {
    name = "alacritty_linux";
    code = "/nix/store/…-linux.dhall";
    dependencies = [ 
      linux
    ];
  };

… and that will then fail to resolve the ./mono.dhall relative import because ./mono.dhall was not also added to the Nix store. Also, even if mono.dhall had been added to the /nix/store the relative import would fail because it’s going to be looking for /nix/store/mono.dhall when it resolves that relative import and not /nix/store/…-mono.dhall.

Fortunately, there are two ways you can work around this.

The first solution, is to add the entire ./src directory to the Nix store instead of the file, like this:

  config = pkgs.dhallPackages.buildDhallPackage {
    name = "alacritty_linux";
    code = "${./src}/linux.dhall";
    dependencies = [ 
      linux
    ];
  };

That will create a /nix/store/…-src directory containing both linux.dhall and mono.dhall underneath it, so that relative references work correctly. I’m guessing that this is probably the solution that you want.

However, for completeness, the second solution is that you can protect ./mono.dhall with an integrity check and then add a package that builds ./mono.dhall as a dependency of the package that builds linux.dhall. In other words, the cache resolution trick that was created for remote imports also works for local imports. For example, we do this at work because we have a few Dhall packages in disparate parts of our monorepo, and rather than add the entire repository to the /nix/store we just protect “distant” imports with integrity checks so that they can be resolved via Nix dependencies.


#3

You are the most helpful person on the entire internet. Thanks for the very well written reply. I was ultimately able to make things work thanks to your help. I also needed to change my builder.sh from

"$dhalljson/bin/dhall-to-yaml" --file $src/linux.dhall --output $out

to

"$dhalljson/bin/dhall-to-yaml" --file $config/source.dhall --output $out

so it refers to the built Dhall package source. For that, I also added the source = true; parameter. The difference is that $src/linux.dhall would be a source file starting with let Alacritty = https://raw.githubusercontent.com/cideM/dhall-alacritty/master/linux.dhall sha256:c9cf010f9ef1a7da4590d70571c1fa5114743f9096b8387b180bfb02f5cdffb1. My assumption was that this would work, since the import should be cached. But the call to dhall-to-yaml seems to still try to fetch this via remote.

I then concluded that I need to use the buildDhallPackage infrastructure to prebuild everything, so that the final source passed to dhall-to-yaml in the builder has no remote calls. And indeed, if I add a simple cat $config/source.dhall inside the builder, I get something like

{ colors =
  { bright =
    { black = "0x949494"
    , blue = "0x4271ae"
    , cyan = "0x3e999f"
    , green = "0x718c00"
    , magenta = "0x8959a8"
    , red = "0xd7005f"
    , white = "0xf5f5f5"
    , yellow = "0xd75f00"
    }
...

Which is the output you’d get if every import and every function call is evaluated. The only step missing here is to actually serialize this to YAML, which is then done by dhall-to-yaml. I checked and what lands in the Nix store is indeed my alacritty.yml.

I’ll update this post with a link to the whole folder once I’ve cleaned some stuff up and pushed it.

Thanks again for the help!

Here’s a permalink to the final Nix file: https://github.com/cideM/dotfiles/blob/79eff97b97d7278ef3e5d4b5bd3b36011dd2f94f/src/nixpkgs/.config/nixpkgs/alacritty/default.nix#L1

The folder structure here is super messy but the concept works.


#4

You’re welcome! :slightly_smiling_face: