Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Flake Allocations — Cross-Host Cluster Configuration

Purpose

The flake allocations module declares cluster-wide configuration options at the flake level. Rather than configuring each NixOS system independently, allocations let you declare concerns like which machines have accelerators, which server acts as the IO Coordinator, and which servers act as distributed builders — in a single place — and then propagate those values into every host configuration.

Entry Point

Architecture / Services / Scope

The allocation system has three layers:

  1. Option Definitions (modules/flake/allocations.nix) — declares the available allocation options.
  2. Configuration (flake/nixos/flake-module.nix) — sets the actual values for those options.
  3. Apply Modules (modules/flake/apply/) — propagate allocation values into each NixOS or Home-Manager configuration via specialArgs.

Data Flow

allocations.nix          flake-module.nix            apply/system.nix
┌──────────────┐   ┌──────────────────────┐   ┌───────────────────────┐
│ Define opts  │──▶│ Set values           │──▶│ Map to NixOS options  │
│ (types,      │   │ (which host has what)│   │ per system via        │
│  defaults)   │   │                      │   │ specialArgs           │
└──────────────┘   └──────────────────────┘   └───────────────────────┘

When a NixOS configuration is built, it receives the allocations attribute set and passes it as a specialArgs argument. The apply module then conditionally maps those allocations to NixOS module options based on the host’s device type.

Allocation Options

allocations.accelerators
Typeattribute set of list of (one of "cuda", "rocm")
Default{ }
Example{ cudaAndRocmHost = [ "cuda" "rocm" ]; nothing = [ ]; onlyRocm = [ "rocm" ]; }

Define hardware accelerators allocated to a machine by hostname.

The attribute names are hostnames, and the values are lists of accelerator types assigned to that host.

allocations.hostTypes
Typeattribute set of list of string
Default{ desktop = [ "nixmi" ]; server = [ "nixai" "nixarr" "nixcloud" "nixdev" "nixio" "nixmon" "nixserv" ]; }
Example{ desktop = [ "workstation1" ]; server = [ "nixbuild1" "nixbuild2" ]; }

An Attribute set defining hostnames by their device type. The attribute names are device types, and the values are lists of hostnames assigned to that device type.

allocations.server.distributedBuilders
Typelist of (one of "nixai", "nixarr", "nixcloud", "nixdev", "nixio", "nixmon", "nixserv")
Default[ ]
Example[ "nixbuild1" "nixbuild2" ]

List of servers that will act as remote builders for server-side distributed builds.

allocations.server.ioPrimaryCoordinator
Typeone of "nixai", "nixarr", "nixcloud", "nixdev", "nixio", "nixmon", "nixserv"

Designate a server to act as the Primary I/O coordinator

allocations.server.monitoringPrimaryHost
Typeone of "nixai", "nixarr", "nixcloud", "nixdev", "nixio", "nixmon", "nixserv"

Designate a server to act as the primary monitoring collector.

This host will run Prometheus, Loki, Grafana, and Alertmanager for centralized observability of the entire server cluster.

allocations.accelerators

Maps hostnames to their available hardware accelerators (cuda, rocm). Used by the builder system to configure nixpkgs with the correct cudaSupport / rocmSupport flags per host. Hosts not listed default to no accelerators. The builder reads allocations.accelerators.${hostname} and sets the corresponding nixpkgs config flags.

allocations.hostTypes

Read-only attribute set mapping device types to their hostnames. Auto-populated from the host-discovery function, which scans the hosts/ directory structure.

Server primary-host allocations

These options each designate a specific server as the coordinator for a cluster role (IO Coordinator, Monitoring Coordinator, Database Coordinator, Storage Coordinator, Identity Coordinator). The type of each is constrained to an enum of server hostnames, automatically derived from hostTypes.server. Values flow through apply/system.nix into the corresponding server.* option on each server configuration:

  • allocations.server.ioPrimaryCoordinator → server.ioPrimaryHost (IO Coordinator)
  • allocations.server.monitoringPrimaryHost → server.monitoringPrimaryHost (Monitoring Coordinator)
  • allocations.server.databasePrimaryHost → server.databasePrimaryHost (Database Coordinator)
  • allocations.server.storagePrimaryHost → server.storagePrimaryHost (Storage Coordinator)
  • allocations.server.authPrimaryHost → server.authPrimaryHost (Identity Coordinator)

allocations.server.distributedBuilders

List of servers that act as remote builders for distributed builds. Flows into server.distributedBuilds.builders on each server configuration.

Apply Modules

The apply modules bridge flake-level allocations to per-system NixOS options.

apply/system.nix is imported during system construction. It receives allocations and deviceType via specialArgs and maps the server allocations onto the corresponding server.* options. It uses optionalAttrs to only apply server-specific options when deviceType == "server", preventing errors on non-server systems.

apply/home-manager.nix is imported by the Home-Manager builder. It is currently a no-op — a placeholder for future home-manager-level allocations.

References