Skip to content

nix

5 posts with the tag “nix”

Using Tmux with Nix

TL;DR

I throw all my tmux configuration (along with some basic alacritty and starship configuration) into a nix module. I really enjoy using it so far. I hope it can give you some ideas about how to potentially improve your workflow

Longer version

Before using Nix, I had very limited experience with Tmux. I think I only used once when I ssh into some ec2 box for doing mutlipy long running jobs. After adpating Nix, I started to try out Alacirtty (I was having a hard time to get Kitty build on MacOS with Nix), I really like Alacirtty, but it doesn’t support tabs or split, the community recommends using window manager or terminal multiplexer. The idea didn’t bother me too much. (I brought a tmux 2 book long before that, never read it), I thought to myself: maybe this is a good oppurity to learn tmux. So I skip through the book, and set up tmux with basic configuration using nix home manager, and back to everyday work. I didn’t leverage tmux too much, and I often feel like I should spend some more time to tweak my configuration, so it suits my use case better. One day I came across waylonwalker’s blog, which introduce Chirs Toomy’s thoughtbot course on tmux to me. These materials really give me lots of ideas, and after some embarrassing long hours, I finally manage to put all my tmux configuration into a single nix module.

Here are some lessons and tricks I learned:

display-popup and display-menu

Most of tmux material I came cross are little bit of dated. The latest version of tmux at the moment of writing is 3.2a. I think “new” (I am not sure how new are they) commands like display-popup and display-menu are really cool. If you are using tmux, and not aware of them, I think you should give them a try. They might helps you to improve your workflow. waylonwalker’s blog has some cool ideas on how to use display-popup. There is an example how I use display-menu and display-popup

Basically diplay-menu allow you to display a menu on a specific position with a title. You choose items from Menu using arrow keys or shortcut, usually item is tmux command. You can optionally add an visual divider between items.

bind-key Tab display-menu -T "#[align=centre]Sessions" "Switch" . 'choose-session -Zw' Last l "switch-client -l" ${tmuxMenuSeperator} \
"Open Main Workspace" m "display-popup -E \" td ${cfg.mainWorkspaceDir} \"" "Open Sec Workspace" s "display-popup -E \" td ${cfg.secondaryWorkspaceDir} \"" ${tmuxMenuSeperator} \
"Kill Current Session" k "run-shell 'tmux switch-client -n \; tmux kill-session -t #{session_name}'" "Kill Other Sessions" o "display-popup -E \"tkill \"" ${tmuxMenuSeperator} \
Random r "run-shell 'tat random'" Emacs e "run-shell 'temacs'" ${tmuxMenuSeperator} \
Exit q detach"

Have a visual cue on tmux prefix press

You might want to hit whether you currently press tmux prefix key or not. I found this nice solution

Without press prefix

with prefix press

Mouse or no mouse

Maybe you think the point of tmux is to do mouse-free workflow, to enable mouse in tmux might seems wrong. But there are certain tasks like resizing panel are easier with Mouse. You can even set a command to toggle enabling mouse.

Use oh-my-zsh tmux plugin to start tmux automatically

I am using zsh and oh-my-zsh, it has a tmux plugin.

A single nix module

Nix module allows us to group all tmux related configurations (bash script, zsh and tmux) into a single place.

Integrate git hooks with treefmt and devshell

Background

I was looking for a way to integrate pre-commit-hooks.nix and numtide/devshell, I came across this github issue. It seems zimbatm added git.hooks extra module to support git hook integration in devshell. So I decide to give a try.

Code

I am configuring my devshell using nix, rather than toml file.

I think for toml version, you should just do

devshell.toml

imports = ["git/hooks"]
git.hooks.enable = true
git.hooks.pre-comment = "treefmt"

assuming you already added devshell overlay, we need to import git extra module, and enable git.hooks and add script for the hook we want to use. Here i am using numtide/treefmt.

devShell = pkgs.devshell.mkShell {
imports = [ (pkgs.devshell.extraModulesDir + "/git/hooks.nix") ];
git.hooks.enable = true;
git.hooks.pre-commit.text = "${pkgs.treefmt}/bin/treefmt";
}

assuming you have a treefmt.toml in your project root directory, you should be to ready to go

Getting started with Nix Flakes and devshell

Introduction

I finally converted my blog project to use Nix Flakes and numtide/devshell. I want to write down what I learnt about Nix Flakes and devshell.

What is Nix Flakes and Why you might care about it

Nix Flakes are a set of experimental features in the Nix package manager.

If you are not familiar with Flakes yet, here is a list of resources on it.

Some of goals of Flakes are

  • Standardized how we compose nix files and provide a single entry-point (You don’t have to have default.nix, ci.nix, shell.nix, of course you can break down your flake file into smaller nix files).

  • Standardized nix packages’ dependency management (I think with Flakes, one doesn’t need niv to pin down dependencies version. Although niv is great, and its commands are more user friendly than what Flakes offers right now)

  • a set of more user friendly nix commands (nix run, nix develop)

  • better reproducibility

How to install/uninstall Flakes

install

Right now, Nix Flakes is not enabled by default. We need to explicitly enable it.

NixOS

adding the following in the configuration.nix

{ pkgs, ... }: {
nix = {
package = pkgs.nixFlakes;
extraOptions = ''
experimental-features = nix-command flakes
'';
};
}

non-NixOS

Terminal window
nix-env -iA nixpkgs.nixFlakes

and add

Terminal window
experimental-features = nix-command flakes

to ~/.config/nix/nix.conf (if current shell user is nix trusted users) or /etc/nix/nix.conf

Install Nix Flakes installer I am not sure whether this step is still needed

Terminal window
sh <(curl -L https://github.com/numtide/nix-flakes-installer/releases/download/nix-2.4pre20210126_f15f0b8/install)

You can type nix-env --version to verify. The Flakes version should looks like nix-env (Nix) 2.4pre20210126_f15f0b8. (the version was 3.0, and version rollbacked to 2.4)

uninstall

NixOS

just revert the change in configuration.nix and do nixos-rebuild switch

non-NixOS

nix-env -iA nixpkgs.nix should bring out nix to the mainline version, and we need to revert the nix.conf change. Of course, multi-user version needs to restart nix-daemon.

How to bootstrap a Nix Flakes project

use nix flake init to generate the flake.nix, nix flake update to generate flake.lock file.

An important thing about Flakes, to improve the reproducibility, Flakes requires us to git staging all the flake.nix changes.

(Selective) Anatomy of flake.nix

Beside description, flake.nix has 2 top-level attributes

  • inputs (the dependency management part)
  • outputs the function takes the all inputs we defined and evaluate a set of attributes. (Usually our build artifacts).

inputs

a typical input might look like

{
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
flake-utils.url = "github:numtide/flake-utils/master";
};
}

here, it declares two dependencies nixpkgs and flake-utils. We can use nix flake update to lock down dependencies.

We can point to a branch: inputs.nixpkgs.url = "github:Mic92/nixpkgs/master";.

or revision: inputs.nix-doom-emacs.url = "github:vlaci/nix-doom-emacs?rev=238b18d7b2c8239f676358634bfb32693d3706f3";

for non-Flakes dependency, we need to declare that.

{
inputs.bar.url = "github:foo/bar/branch";
inputs.bar.flake = false;
}

Further, we can override a Flake dependency’s input

{
inputs.sops-nix.url = "github:Mic92/sops-nix";
inputs.sops-nix.inputs.nixpkgs.follows = "nixpkgs";
}

outputs

Schema

I skipped all the nixos related attributes.

{ self, ... }@inputs:
{
# Executed by `nix flake check`
checks."<system>"."<attr>" = derivation;
# Executed by `nix build .#<name>`
packages."<system>"."<attr>" = derivation;
# Executed by `nix build .`
defaultPackage."<system>" = derivation;
# Executed by `nix run .#<name>`
apps."<system>"."<attr>" = {
type = "app";
program = "<store-path>";
};
# Executed by `nix run . -- <args?>`
defaultApp."<system>" = { type = "app"; program = "..."; };
}

where

  • <system> is the name of the platform, such as “x8664-linux”, “x8664-darwin”
  • <attr> is the attribute name (package name)
  • <store-path> is a /nix/store... path

So for each <attr>, we can have

  • check (prerequisites for build the package)
  • package
  • app (executable)

and we can define a default <attr>.

flake-utils

flake-utils ,as its name indicates, is a utility package help us write flake.

For example, it has eachDefaultSystem function take a lambda and iterate through all the systems supported by nixpkgs an hydra. So we can reuse the same lambda to build for different systems.

Using flake-utils.lib.eachSystem [ "x86_64-linux" ], you target fewer systems.

flattenTree takes a tree of attributes and flatten them into a one level key-value (attribute to derivation), which is what Flakes packages outputs expects.

flattenTree { hello = pkgs.hello; gitAndTools = pkgs.gitAndTools }

returns

{
hello = «derivation»;
"gitAndTools/git" = «derivation»;
"gitAndTools/hub" = «derivation»;
}

mkApp is a helper function to construct nix app.

here is an example

{
description = "Flake utils demo";
inputs.flake-utils.url = "github:numtide/flake-utils";
outputs = { self, nixpkgs, flake-utils }:
flake-utils.lib.eachDefaultSystem (system:
let pkgs = nixpkgs.legacyPackages.${system};
in rec {
packages = flake-utils.lib.flattenTree {
hello = pkgs.hello;
gitAndTools = pkgs.gitAndTools;
};
defaultPackage = packages.hello;
apps.hello = flake-utils.lib.mkApp { drv = packages.hello; };
defaultApp = apps.hello;
});
}

Case Study 1: nix-tree

utdemir has this nice and concise example using Flakes with a Haskell project. I think it is a great starting point to understand Flakes.

in nix-tree, the outputs looks likes

{
outputs = { self, nixpkgs, flake-utils }: # list out the dependencies
let
overlay = self: super: { # a pattern of bring build artifacts to pkgs
haskellPackages = super.haskellPackages.override {
overrides = hself: hsuper: {
nix-tree = hself.callCabal2nix "nix-tree"
(self.nix-gitignore.gitignoreSourcePure [
./.gitignore
"asciicast.sh"
"flake.nix"
] ./.) { };
};
};
nix-tree =
self.haskell.lib.justStaticExecutables self.haskellPackages.nix-tree;
};
in {
inherit overlay;
} // flake-utils.lib.eachDefaultSystem (system: # leverage flake-utils
let
pkgs = import nixpkgs {
inherit system;
overlays = [ overlay ];
};
in {
defaultPackage = pkgs.nix-tree;
devShell = pkgs.haskellPackages.shellFor { # development environment
packages = p: [ p."nix-tree" ];
buildInputs = with pkgs.haskellPackages; [
cabal-install
ghcid
ormolu
hlint
pkgs.nixpkgs-fmt
];
withHoogle = false;
};
});
}

Let’s break down the function a little bit. The outputs have 2 dependencies nixpkgs and flake-utils.

First thing, it construct an overlay contains the local nix-tree as Haskell package and a derivation for the executable.

Next, for eachDefaultSystem, it initialize the new nixpkgs with relevant system and overlay, and construct defaultPackage and devShell. devShell is Nix Flakes’ version of nix-shell (without -p capability, if you want to use nix-shell -p, there is nix shell). We can start a development shell by nix develop command. There is nix develop integration with direnv

How to use non-flake dependency

Let’s say if I want to use easy-purescript-nix in my project. First I need to add it as inputs

{
inputs.easy-ps = {
url = "github:justinwoo/easy-purescript-nix/master";
flake = false;
};
}

there are more than one packages in easy-purescript-nix. I can added them into an overlay and add the overlay into the pkgs.

{
outputs = {self, nixpkgs, easy-ps}: {
overlay = final: prev: {
purs = (prev.callPackage easy-ps {}).purs;
spago = (prev.callPackage easy-ps {}).spago;
} // (
flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs {
inherit system;
overlays = [self.overlay];
};
in rec {
devShell = {
packages = [
pkgs.purs
pkgs.spago
];
};
};
));

On the another hand, you can use flake-compat to use Flakes project from mainline (legacy) Nix.

Case Study 2: todomvc-nix

todomvc-nix is a much more complex example. It needs to build Haskell (even ghcjs, which usually is more chanlleing to build) and rust source code.

You can checkout the code yourself to see how one can override different haskell packages and using numtide/devshell to customize the nix develop experience.

numtide/devshell

devshell (not to confuse with Nix Flakes devShell) is numtide project to customize per-project developer environments. The marketing slogan is “like virtualenv, but for all the languages”.

I think it is fair to say that devshel is still early stage of development. (Although one can argue almost every thing mentioned in this article is in the early stage of development.) Lots of usages are subject to future changes. Using devshell probably requires you to read throught the source code. But I think devshell is a really exicting project.

How to “install” devshell

devshell does aim to support non-Flakes and Flakes Nix. I am only going to cover the Flakes version, the non-Flakes usage is covered at the devshell’s doc.

First thing is to declare devshell as an input, and we need to import devshell overlay into our instance of nixpkgs.

flake-utils.lib.eachSystem [ "x86_64-darwin" ] (system:
let
pkgs = import nixpkgs {
inherit system;
overlays = [ devshell.overlay overlay ];
};

the overlay would bring devshell attribute into the pkgs. devshell has functions like mkShell and fromTOML. fromTOML allows us to configure the devshell using TOML file.

{
# assuming we import devshell.overlay
# and there is devshell.toml file
devShell = pkgs.devshell.fromTOML ./devshell.toml;
}

mkShell allows us to use Nix experssions.

{ devShell = pkgs.devshell.mkShell { name = "blog-dev-shell"; }; }

Here is My devshell config. devshell document page has a list of configuration options.

environments variables

This is kind of like shellHook in the old mkShell function. We can define environment variables in our devshell.

TOML version looks like

[[env]]
name = "GO111MODULE"
value = "on"

Nix version looks little verbose

{
devshell = pkgs.devshell.mkShell {
env = [{
name = "NODE_ENV";
value = "development";
}];
};
}

packages

Of course, we can define packages for our devshell TOML version

[devshell]
packages = [
"go"
]

Nix counterpart is more flexible, imagine I have a custom haskellPackages with lots of overlays, I can reference it in flake.nix pretty easily.

{
devshell =
pkgs.devshell.mkShell { packages = [ myHaskellEnv pkgs.nixpkgs-fmt ]; };
}

commands

I think this is a cool feature in devshell. Using Nix expressions we can define some common commands for your project.

{
devshell = pkgs.devshell.mkShell {
commands = [
{
name = "cssWatch";
category = "css";
command =
"ls tailwind/*.css | ${pkgs.entr}/bin/entr ${pkgs.yarn}/bin/yarn build";
}
{
name = "siteClean";
category = "static site";
help = "clean static site files";
command = "${pkgs.blog}/bin/blog clean";
}
{
name = "yarn";
category = "javascript";
package = "yarn";
}
];
};
}

Everytime, you can enter devshell, all commands and a motd (message of the day) will be displayed. the commands are grouped by their category. packages won’t show up in there.

Terminal window
🔨 Welcome to blog-dev-shell
[css]
cssWatch
[general commands]
menu - prints this menu
[javascript]
node - Event-driven I/O framework for the V8 JavaScript engine
yarn - Fast, reliable, and secure dependency management for javascript
[purescript]
purs
spago
[static site]
siteClean - clean static site files
siteWatch - Watch static site files
[utility]
entr - Run arbitrary commands when files change

modules

Right now, all the build-in modules are in devshell/extra directory.

  • git hook
  • locale
  • c
  • go
  • rust

One can write custom module. For example, nixpkgs haskell-modules has a nice shellFor function, we can turn it into a haskell module for devshell.

Push Nix-built container image to GCR using github action

Intro

I have been trying out google cloud run with a simple haskell + purescript web app. In order to deploy the app, I need to containerize the app and push the docker image to google container registry. I wanted to figure out whether this can be done with nix and some simple github actions. Turns out: with skopep, it is pretty easy!

skopeo copy

After some searching, I found this discourse thread mentions skopeo, and I found this repo using nix flakes and skopeo to push docker image into a private docker registry in github action. So I decided to try out skopeo.

skopeo can push an image from one location to another. It supports Google Container Registry (GCR)

So we can push our image like this

skopeo copy docker-image://$(nix-build -A my-image --no-out-link) docker://gcr.io/google-cloud-project-name/service:tag

install skopeo in github action

I am using install-nix-action in my build step already, so I added a shell script bin in my nix flake, and make it as nix flake app using flakes-utils, which can be run using nix run ".#script"

upload-script = pkgs.writeShellScriptBin "upload-image" ''
set -eu
OCI_ARCHIVE=$(nix-build --no-out-link)
DOCKER_REPOSITORY="docker://gcr.io/$GOOGLE_CLOUD_PROJECT_NAME/$GOOGLE_CLOUD_RUN_SERVICE_NAME:$GITHUB_SHA"
${pkgs.skopeo}/bin/skopeo copy --dest-creds="_json_key:$GCR_DEVOPS_SERVICE_ACCOUNT_KEY" "docker-archive:$OCI_ARCHIVE" "$DOCKER_REPOSITORY"
'';
apps.upload-script = flake-utils.lib.mkApp { drv = upload-script; };

Authenticate skopeo with GCR

I was not able to find too much documentation on this step. So I went to some trail and error.

GCR supports different authentication methods. I thought the easiest would to create a service account key with Cloud Storage Admin role, and upload the key content as a secret in github actions.

docker login -u _json_key -p "$(cat keyfile.json)" https://HOSTNAME

Of course, we want to replace docker with skopeo. This works locally, but not in github action due to how skope login works.

But turns out, we don’t have to do skope login, we can just using skopeo copy --dest-creds.

Put everything together

So I put my service account key, google cloud project name, and the name of the GCR docker image name into github actions as secret, and pass them as env variable in the step, and do nix run ".#upload-script"

name: CI
on: [push, pull_request]
jobs:
build:
name: Build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2.4.0
- uses: cachix/install-nix-action@v16
with:
extra_nix_config: |
access-tokens = github.com=${{ secrets.GITHUB_TOKEN }}
- name: Push Image
run: |
nix run ".#upload-script"
env:
GCR_DEVOPS_SERVICE_ACCOUNT_KEY: ${{ secrets.GCR_DEVOPS_SERVICE_ACCOUNT_KEY }}
GOOGLE_CLOUD_PROJECT_NAME: ${{ secrets.GOOGLE_CLOUD_PROJECT_NAME }}
GOOGLE_CLOUD_RUN_SERVICE_NAME: ${{ secrets.GOOGLE_CLOUD_RUN_SERVICE_NAME}}
if: github.ref == 'refs/heads/master'

Works like a charm. Thanks for reading

References

Build a blog with Nix

Vision

I have been learning nix for a while. Overall, I am really happy with Nix ecosystem. With Nix, home-manager, cachix, you have setup a reliable and efficient development environment and CI. For demonstration, I want to show you how I use nix to publish my blog:

  • compile a Hakyll program
  • generate the static site using the Hayll program
  • Upload the site to Firebase

If you are new to Nix, I hope you can find something useful in this blog.

Why Nix ecosystem

Hopefully, at this point you are already sold on nix. If not, maybe checkout build with nix or nix.dev. For me, the biggest selling point of is: you can use a declarative language to set up a reproducible environment for your local development and CI.

Why Hakyll

This is probably a tough sell if you don’t care about Haskell. Nowadays, you use tools like Hugo, WordPress, or Jekyll to effortless setup a static site. I love Haskell, and Hakyll gives me an opportunity to use Haskell.

Why Firebase

There are lots of decent alternative to Firebase for hosting a static site. Just to list few

  • GitHub page
  • aws s3
  • netlify seems really nice. probably worth trying out
  • Google cloud storage can host a static site, but currently it does not support SSL.

Required Tools

  • nix (either single user or multi-user installation would work)
  • direnv or nix-direnv recommend using home-manager to get it)
  • lorri (optional, only if direnv by itself doesn’t work well enough for you)
  • cachix (optional)

Consider using nix project template

Both templates are very opinionated. getting-started-nix-template is a more generic nix template. hakyll-nix-template is Hakyll specified, it has sitemap, tag support.

Terminal window
.
├── content
| ├── about.org
| ├── contact.org
| ├── css
| ├── images
| ├── index.html
| ├── posts
| └── templates
├── default.nix
├── nix
| ├── default.nix
| ├── sources.json
| └── sources.nix
├── shell.nix
└── src

scaffold hakyll project using hakyll-init

Hakyll comes with a command line too haskyll-init to scaffold a Hakyll project. Since we only need haskell-init once, normal nix workflow would be

Terminal window
nix-shell -p pkgs.haskellPackages.hakyll

but depend on which nix channel is your default channel, you might get an error about pkgs.haskellPackages.hakyll is broken. We could use -I flag to pin our one-time shell package to stable version. The pattern is -I nixpkgs=https://github.com/NixOS/nixpkgs-channels/archive/{rev}.tar.gz~

Terminal window
~nix-shell -p pkgs.haskellPackages.hakyll -I nixpkgs=https://github.com/NixOS/nixpkgs-channels/archive/14006b724f3d1f25ecf38238ee723d38b0c2f4ce.tar.gz~
Terminal window
hakyll-init . #scaffold at current directory
hakyll-init content #scaffold at ./content directory

The project is just a plain cabal project. We can edit project or executable name. We can use normal Nix Haskell development workflow.

Nix Haskell Workflow

Currently, there are two major Nix Haskell workflows: iohk-haskell.nix and your traditional nixpgs. People usually recommend iohk one for complex project, I haven’t used it at all.

Terminal window
nix-shell -p cabal2nix # unless you already installed cabal2nix globally
cabal2nix . > blog.nix
{ project ? import ./nix {}
}:
let
haskellPackages = project.pkgs.haskellPackages;
packages = haskellPackages.callCabal2nix "blog" ./blog.cabal {};
in
haskellPackages.shellFor {
withHoogle = true;
packages = p: [ packages ];
buildInputs = builtins.attrValues project.devTools;
shellHook = ''
${project.ci.pre-commit-check.shellHook}
'';
}

https://discourse.nixos.org/t/nix-haskell-development-2020/6170 hoogle server --local -p 3000 -n

How to find certain (Haskell) package’s version

Terminal window
nix repl
nix-repl> sources = import ./nix/sources.nix
nix-repl> pkgs = import sources.nixpkgs {}
nix-repl> pkgs.haskellPackages.hakyll.version
"4.13.0.1"
nix-repl> :q

How to customize Hakyll

This is probably beyond the scope of this blog, Robert Pearce has an on-going serial on the topic. https://robertwpearce.com/hakyll-pt-1-setup-and-initial-customization.html

Here is a list of Hakyll projects I often check

Hakyll website has a more comphersive list

GitHub Action

Build Step

Most of the YAML configuration is copied from getting-started-nix-template. My default.nix only build the Hakyll program, it doesn’t generate the site. So I added result/bin/site build in run command. (site is the name of my Hakyll executable). We need pass generated site directory as artifacts between build steps

- name: Archive Production Artifact
uses: actions/upload-artifact@master
with:
name: dist
path: dist

dist is the directory name for the generated site, by default Hakyll uses _site.

Publish to Firebase

I use w9jds firebase action to publish the generated static site directory to Firebase. There are publish actions for netlify and Github Page. Of course, we have to store our Firebase token as encrypted secret and pass them as environment variables into the build step.

Enable cachix cache (Optional)

Current version of GitHub Action YAML

name: CI
on:
push:
branches:
- master
jobs:
build:
name: Build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2.3.4
- uses: cachix/install-nix-action@v12
- uses: cachix/cachix-action@v8
with:
name: yuanw-blog
signingKey: ${{ secrets.CACHIX_SIGNING_KEY }}
- name: Nix build
run: |
nix-build
result/bin/site build
- name: Archive Production Artifact
uses: actions/upload-artifact@master
with:
name: dist
path: dist
deploy:
name: Deploy
needs: build
runs-on: ubuntu-latest
steps:
- name: Checkout Repo
uses: actions/checkout@v2.3.2
- name: Download Artifact
uses: actions/download-artifact@master
with:
name: dist
path: dist
- name: Deploy to Firebase
uses: w9jds/firebase-action@v1.5.0
with:
args: deploy --message '${{github.event.head_commit.message}}' --only hosting
env:
FIREBASE_TOKEN: ${{ secrets.FIREBASE_TOKEN }}
PROJECT_ID: ${{secrets.FIREBASE_PROJECT_ID}}

Result

Right now, the whole CI steps averagely takes 4 min to run. I am pretty happy with the setup.

References

About Nix in general

Nix Haskell development

Hakyll

Github Action