Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions flake.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions flake.nix
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@
flake-utils.url = "github:numtide/flake-utils";
git-hooks.inputs.nixpkgs.follows = "nixpkgs";
git-hooks.url = "github:cachix/git-hooks.nix";
nix-darwin.inputs.nixpkgs.follows = "nixpkgs";
nix-darwin.url = "github:nix-darwin/nix-darwin";
nix-editor.inputs.nixpkgs.follows = "nixpkgs";
nix-editor.inputs.utils.follows = "flake-utils";
nix-editor.url = "github:snowfallorg/nix-editor";
Expand Down Expand Up @@ -47,6 +49,7 @@
nix/devShells.nix
nix/fmt.nix
nix/hooks.nix
nix/hosts.nix
nix/nixpkgs.nix
nix/packages
nix/overlays
Expand Down
189 changes: 189 additions & 0 deletions nix/docs/nixos-tests-on-macos.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,189 @@
## Prerequisites

Running NixOS tests on macOS requires a Linux builder VM because NixOS tests need a Linux environment.
This project includes a nix-darwin configuration that sets up a linux-builder VM automatically.

You need:
- macOS with Apple Silicon (aarch64-darwin)
- Nix installed (see [Getting Started](start-here.md)) preferably a recent version (2.30+)

## Setup

Run the setup script to configure nix-darwin with the linux-builder:

```bash
nix run .#setup-darwin-linux-builder
```

Note that you don't have to checkout the repository to run the setup, you can run it directly from GitHub:

```bash
nix run github:supabase/postgres#setup-darwin-linux-builder
```

This command will:
- Back up existing system files (`/etc/nix/nix.conf`, `/etc/bashrc`, `/etc/zshrc`)
- Configure nix-darwin with the linux-builder VM
- Install helper scripts for managing the builder

The linux-builder VM is configured with:
- 6 CPU cores
- 8GB RAM
- 40GB disk
- Support for both x86_64-linux and aarch64-linux builds
- The `nixos-test` feature required for running NixOS tests

After setup completes, restart your shell to access the helper commands.

## Verify the setup

The setup script runs verification automatically after configuration.
You can also run verification manually at any time:

```bash
nix run .#verify-darwin-linux-builder
```

Or after setup, use the installed command:

```bash
verify-darwin-linux-builder
```

The verification script checks:

1. Launchd service status (running vs loaded-but-stopped)
2. Nix configuration via `nix config show` (substituters, trusted keys, experimental features)
3. Builder features (`/etc/nix/machines` includes `nixos-test`)
4. Builder responsiveness (test build of `nixpkgs#hello` for aarch64-linux)

Each check reports pass/fail with actionable guidance on failures.

You can also manually test that the linux-builder is working by building a simple package for Linux:

```bash
nix build --system x86_64-linux nixpkgs#hello
nix build --system aarch64-linux nixpkgs#hello
```

If both commands succeed, the linux-builder is ready for NixOS tests.

## Running NixOS tests

NixOS tests are defined in `nix/ext/tests/` and exposed as flake checks.
To run a test on macOS, use the `aarch64-darwin` system attribute:

```bash
nix build .#checks.aarch64-darwin.ext-pgjwt -L
```

The `-L` flag shows logs during the build, which is helpful for seeing test progress and debugging failures.

If the nix build exit immediately with success, it means that the result was fetched from cache and the test passed previously.
To force a re-run of the test, use the `--rebuild` flag:
Comment on lines +82 to +83
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Fix grammar and improve clarity.

The sentence has a grammatical error ("exit" should be "exits") and could be clearer.

📝 Suggested fix
-If the nix build exit immediately with success, it means that the result was fetched from cache and the test passed previously.
+If the nix build exits immediately, the result was fetched from cache and the test passed previously.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
If the nix build exit immediately with success, it means that the result was fetched from cache and the test passed previously.
To force a re-run of the test, use the `--rebuild` flag:
If the nix build exits immediately, the result was fetched from cache and the test passed previously.
To force a re-run of the test, use the `--rebuild` flag:
🧰 Tools
🪛 LanguageTool

[style] ~60-~60: ‘with success’ might be wordy. Consider a shorter alternative.
Context: ...res. If the nix build exit immediately with success, it means that the result was fetched f...

(EN_WORDINESS_PREMIUM_WITH_SUCCESS)

🤖 Prompt for AI Agents
In `@nix/docs/nixos-tests-on-macos.md` around lines 60 - 61, Fix the grammatical
error and improve clarity in the sentence that currently reads "If the nix build
exit immediately with success, it means that the result was fetched from cache
and the test passed previously." Change "exit" to "exits" and rephrase for
clarity (for example: "If the nix build exits immediately with a success, it
means the result was fetched from cache and the test passed previously. To force
a re-run of the test, use the `--rebuild` flag:") so the meaning is unambiguous
and flows smoothly.


```bash
nix build .#checks.aarch64-darwin.ext-pgjwt -L --rebuild
```

### Available tests

List all available checks with:

```bash
nix flake show --json 2>/dev/null | jq -r '.checks["aarch64-darwin"] | keys[]' | sort
```

Extension tests follow the naming pattern `ext-<extension_name>`:

```bash
nix build .#checks.aarch64-darwin.ext-pgjwt -L
nix build .#checks.aarch64-darwin.ext-postgis -L
nix build .#checks.aarch64-darwin.ext-vector -L
nix build .#checks.aarch64-darwin.ext-pg_graphql -L
```

## Managing the linux-builder VM

The setup installs two helper commands for controlling the VM:

```bash
stop-linux-builder # Stop the VM (pauses resource usage)
start-linux-builder # Start the VM again
```

As the VM can consume significant resources, you may want to stop it when not running tests using `stop-linux-builder`.
When stopped with `stop-linux-builder`, the service is unloaded to prevent automatic restart.
Use `start-linux-builder` to re-enable and start the service.

### Checking VM status

```bash
sudo launchctl list | grep linux-builder
```

If the VM is running, you'll see a line containing `org.nixos.linux-builder`.

## Troubleshooting

### Tests fail with "builder not available"

Ensure the linux-builder is running:

```bash
start-linux-builder
```

Then verify with a simple build:

```bash
nix build --system aarch64-linux nixpkgs#hello
```

### VM won't start after reboot

If the VM doesn't start automatically, run:

```bash
start-linux-builder
```

The VM is configured as ephemeral, meaning it's recreated fresh on each start.
This ensures a clean environment but requires re-downloading cached build artifacts.

### Slow first build

The first NixOS test run may download significant data.
Subsequent runs benefit from the Nix store cache and the project's binary cache at `nix-postgres-artifacts.s3.amazonaws.com`.

## How it works

The linux-builder is a QEMU virtual machine managed by nix-darwin.
When you run a build targeting Linux (like NixOS tests), Nix automatically delegates the build to this VM.

Key configuration from `nix/hosts/darwin-nixostest/darwin-configuration.nix`:

```nix
nix.linux-builder = {
enable = true;
ephemeral = true;
maxJobs = 4;
supportedFeatures = [
"kvm"
"benchmark"
"big-parallel"
"nixos-test" # Required for NixOS integration tests
];
config = {
virtualisation = {
darwin-builder = {
diskSize = 40 * 1024; # 40GB
memorySize = 8 * 1024; # 8GB
};
cores = 6;
};
};
};
```

The `nixos-test` supported feature is what enables running NixOS VM tests from macOS.
11 changes: 11 additions & 0 deletions nix/hosts.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{ inputs, self, ... }:
{
flake = {
darwinConfigurations = {
darwin-nixostest = inputs.nix-darwin.lib.darwinSystem {
specialArgs = { inherit self; };
modules = [ ./hosts/darwin-nixostest/darwin-configuration.nix ];
};
};
};
}
135 changes: 135 additions & 0 deletions nix/hosts/darwin-nixostest/darwin-configuration.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
{
lib,
pkgs,
self,
...
}:
let
start-linux-builder = pkgs.writeShellApplication {
name = "start-linux-builder";
text = ''
echo "Starting linux-builder..."

if sudo launchctl list | grep -q org.nixos.linux-builder; then
echo "linux-builder is already running"
exit 0
fi

# Use load instead of start to re-enable the service
if sudo launchctl load -w /Library/LaunchDaemons/org.nixos.linux-builder.plist 2>/dev/null; then
echo "linux-builder started successfully"
else
echo "Error: Could not start linux-builder"
echo "Make sure nix-darwin is configured with linux-builder enabled"
exit 1
fi

# Check if it's running
sleep 2
if sudo launchctl list | grep -q org.nixos.linux-builder; then
echo "linux-builder is now running"
else
echo "Warning: linux-builder may not have started properly"
fi
'';
Comment on lines +8 to +34
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🌐 Web query:

launchctl list output format PID field macOS how to detect running job

💡 Result:

launchctl list prints three columns (header often shows PID Status Label):

  • PID: the job’s process ID if it is currently running; otherwise it’s shown as -. [1]
  • Status: the job’s last exit status (negative values mean it was killed by a signal; e.g. -15 = SIGTERM). [1]
  • Label: the launchd job label. [1]

Detect whether a specific job is running (using launchctl list)

If the job appears in the output and the PID column is not -, it’s running. [1]

Example:

label="com.example.myjob"
pid="$(launchctl list | awk -v l="$label" '$3==l {print $1}')"
if [ -n "$pid" ] && [ "$pid" != "-" ]; then
  echo "running (pid=$pid)"
else
  echo "not running (loaded but not running, or not loaded)"
fi

Note on stability / better check

Apple’s launchctl(1) warns the output is not an API and may change. [2] For more reliable state/details, use launchctl print <domain>/<label> (e.g., gui/$UID/<label> or system/<label>). [3]

Sources: [1] launchctl list description (man page excerpt) via Stack Overflow/Server Fault quoting man launchctl; [2] Apple launchctl(1) man page warning; [3] launchctl print description. [1] (stackoverflow.com) [2] (manp.gs) [3] (stackoverflow.com)

Citations:


🌐 Web query:

launchctl print alternative to launchctl list macOS check if job running

💡 Result:

Use launchctl print (Apple calls it the recommended alternative to the legacy launchctl list).[2]

Check whether a specific job is running

Pick the right domain, then look for a PID / running state in the output:

  • LaunchDaemon (system-wide):
launchctl print system/<label>
  • LaunchAgent (GUI login session for your user):
launchctl print gui/$(id -u)/<label>

If the job is running, print will include its current state and typically a PID (where applicable), since print “includes … its current state”.[1]

Practical one-liners

  • “Is it loaded at all?” (errors if not loaded):
launchctl print gui/$(id -u)/<label> >/dev/null 2>&1 && echo loaded || echo not-loaded
  • “Show just state/PID lines”:
launchctl print gui/$(id -u)/<label> 2>/dev/null | egrep 'state =|pid ='

Domain target forms you’ll commonly use are system/<service-name> and gui/<uid>/<service-name>.[1]

Sources:
[1] launchctl(1) man page (domain/service targets; print shows current state) (manp.gs)
[2] launchctl(1) man page (list recommended alternative: print; list PID column meaning) (keith.github.io)

Citations:


Start script may exit even when the job is loaded but not running.

launchctl list | grep matches loaded jobs regardless of running state (PID is "-" when not running). The script can incorrectly skip starting a stopped service. Check the PID field to ensure the job is actually running.

🐛 Proposed fix (awk-based)
-      if sudo launchctl list | grep -q org.nixos.linux-builder; then
+      if sudo launchctl list | awk '$3=="org.nixos.linux-builder" && $1 != "-" {found=1} END{exit !found}'; then
           echo "linux-builder is already running"
           exit 0
       fi

Alternatively, use the more robust launchctl print (Apple's recommended replacement for launchctl list):

if sudo launchctl print system/org.nixos.linux-builder 2>/dev/null | grep -q "pid ="; then
🤖 Prompt for AI Agents
In `@nix/hosts/darwin-nixostest/darwin-configuration.nix` around lines 3 - 29, The
start-linux-builder shell app currently uses `launchctl list | grep
org.nixos.linux-builder` which matches loaded but non-running jobs; update the
start logic in the start-linux-builder text to verify the service PID (so it
only treats the job as running when PID is numeric) or switch to `launchctl
print system/org.nixos.linux-builder` and check for a "pid =" line; modify the
initial existence check and the post-load check (both in the start-linux-builder
block) to use one of these PID-aware approaches so the script will attempt to
load the service when it is loaded but not running and only exit 0 when a real
PID is present.

};
stop-linux-builder = pkgs.writeShellApplication {
name = "stop-linux-builder";
text = ''
echo "Stopping linux-builder..."

# Use unload instead of stop because KeepAlive=true will restart it
if sudo launchctl unload -w /Library/LaunchDaemons/org.nixos.linux-builder.plist 2>/dev/null; then
echo "linux-builder stopped successfully"
else
echo "Warning: Could not stop linux-builder (it may not be running)"
fi

# Check if it's still running
sleep 1
if sudo launchctl list | grep -q org.nixos.linux-builder; then
echo "Warning: linux-builder is still running"
STATUS=$(sudo launchctl list | grep org.nixos.linux-builder || true)
echo "Current status: $STATUS"
Comment on lines +50 to +53
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Same launchctl list | grep detection issue in stop script.

The stop script has the same problem: a loaded-but-not-running job will trigger the "still running" warning incorrectly. Consider using the same PID-aware check suggested for the start script.

🐛 Proposed fix
-      if sudo launchctl list | grep -q org.nixos.linux-builder; then
+      if sudo launchctl list | awk '$3=="org.nixos.linux-builder" && $1 != "-" {found=1} END{exit !found}'; then
           echo "Warning: linux-builder is still running"
🤖 Prompt for AI Agents
In `@nix/hosts/darwin-nixostest/darwin-configuration.nix` around lines 45 - 48,
The stop script currently uses "sudo launchctl list | grep -q
org.nixos.linux-builder" which treats a loaded-but-stopped job as "still
running"; change the detection to be PID-aware by querying the specific job
(e.g., sudo launchctl list org.nixos.linux-builder) and parse the PID column to
ensure it's a positive integer before warning. Update the conditional that sets
STATUS and prints "Warning: linux-builder is still running" to only trigger when
the parsed PID > 0, and keep writing the STATUS variable (from the launchctl
list output) for debugging when present.

else
echo "linux-builder is not running"
fi
'';
};
verify-darwin-linux-builder = self.packages.aarch64-darwin.verify-darwin-linux-builder;
in
{
nixpkgs.hostPlatform = "aarch64-darwin";

# Install builder control scripts
environment.systemPackages = [
start-linux-builder
stop-linux-builder
verify-darwin-linux-builder
];

nix.settings = {
experimental-features = [
"nix-command"
"flakes"
];
always-allow-substitutes = true;
max-jobs = "auto";
trusted-users = [ "@admin" ];
extra-substituters = [ "https://nix-postgres-artifacts.s3.amazonaws.com" ];
extra-trusted-substituters = [ "https://nix-postgres-artifacts.s3.amazonaws.com" ];
extra-trusted-public-keys = [
"nix-postgres-artifacts:dGZlQOvKcNEjvT7QEAJbcV6b6uk7VF/hWMjhYleiaLI="
];
};

nix.extraOptions = ''
!include nix.custom.conf
'';

# accept existing nix.custom.conf
system.activationScripts.checks.text = lib.mkForce "";
system.activationScripts.nix-daemon.text = lib.mkForce ''
if ! diff /etc/nix/nix.conf /run/current-system/etc/nix/nix.conf &> /dev/null || ! diff /etc/nix/machines /run/current-system/etc/nix/machines &> /dev/null; then
echo "reloading nix-daemon..." >&2
launchctl kill HUP system/org.nixos.nix-daemon
fi
max_wait=30
waited=0
while ! nix-store --store daemon -q --hash ${pkgs.stdenv.shell} &>/dev/null; do
if [ $waited -ge $max_wait ]; then
echo "ERROR: nix-daemon failed to start after $max_wait seconds" >&2
exit 1
fi
echo "waiting for nix-daemon" >&2
launchctl kickstart system/org.nixos.nix-daemon
sleep 1
waited=$((waited + 1))
done
'';

nix.linux-builder = {
enable = true;
ephemeral = true;
maxJobs = 4;
supportedFeatures = [
"kvm"
"benchmark"
"big-parallel"
"nixos-test"
];
config = {
virtualisation = {
darwin-builder = {
diskSize = 40 * 1024;
memorySize = 8 * 1024;
};
cores = 6;
};
};
};

nix.distributedBuilds = true;

system.stateVersion = 6;
}
1 change: 1 addition & 0 deletions nix/mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ nav:
- Adding Tests: adding-tests.md
- Migration Tests: migration-tests.md
- Testing PG Upgrade Scripts: testing-pg-upgrade-scripts.md
- NixOS Tests on macOS: nixos-tests-on-macos.md
- References: references.md

validation:
Expand Down
Loading
Loading