Core - llmhop

services.llmhop.enable

Whether to enable llmhop reverse proxy.

Type: boolean

Default:

false

Example:

true

services.llmhop.package

The llmhop package to use.

Type: package

Default:

pkgs.callPackage ./package.nix { }

Credentials granted to llmhop through systemd. A path outside the Nix store uses LoadCredential=. The attribute form can select LoadCredentialEncrypted= for a systemd-creds encrypted source, which must be encrypted under the same name, or omit source to import the credential of that name from the system credential store. That store is shared by every service, so prefix an imported name with its service, as in llmhop.client-token.

Reference them from settings as ${cred:<name>}, the same spelling the model backends use: llmhop reads its own config, so the reference expands to the credential’s contents rather than to its path.

Type: attribute set of ((submodule) or absolute path convertible to it)

Default:

{ }

Example:

{
  api-keys = "/run/secrets/api-keys";
  tls-key = {
    source = "/run/secrets/tls-key.cred";
    encrypted = true;
  };
  "llmhop.hf-token" = { };
}

services.llmhop.credentials.<name>.encrypted

Whether to load and decrypt source with LoadCredentialEncrypted=. Imported credentials are decrypted as needed.

Type: boolean

Default:

false

services.llmhop.credentials.<name>.source

File or socket from which systemd loads the credential. null imports the credential of the same name with ImportCredential= from the system credential store, such as /etc/credstore and /etc/credstore.encrypted, and from the credentials passed to the system.

Type: null or absolute path not in the Nix store

Default:

null

services.llmhop.gid

GID of the declared group. null lets NixOS allocate one.

Type: null or (unsigned integer, meaning >=0)

Default:

config.services.llmhop.uid

services.llmhop.group

Primary group of user. The module declares it while it keeps its default name, any other group is the deployer’s to declare.

Type: string

Default:

config.services.llmhop.user

services.llmhop.host

IP address to bind port to. The default binds every interface, leaving access control to the firewall. IPv6 literals are written plain (e.g. ::1).

Type: string

Default:

""

Example:

"127.0.0.1"

services.llmhop.listen

Addresses llmhop serves besides the default listener, which the top-level port, host, socket, socketUser, socketGroup and socketMode options define. Each is a llmhop-<name>.socket unit handing its socket to llmhop through socket activation. systemd applies the ownership and mode of a unix socket and removes it on stop.

Type: attribute set of (submodule)

Default:

{ }

Example:

{
  caddy = {
    socketGroup = "caddy";
  };
}

services.llmhop.listen.<name>.host

IP address to bind port to. The default binds every interface, leaving access control to the firewall. IPv6 literals are written plain (e.g. ::1).

Type: string

Default:

""

Example:

"127.0.0.1"

services.llmhop.listen.<name>.port

TCP port to listen on, registered in the global port registry so a backend reusing it fails evaluation. null listens on the unix socket socket instead.

Type: null or 16 bit unsigned integer; between 0 and 65535 (both inclusive)

Default:

null

services.llmhop.listen.<name>.socket

Unix socket to listen on while port is null.

Type: string

Default:

"${config.services.llmhop.socketDirectory}/‹name›.sock"

services.llmhop.listen.<name>.socketGroup

Group of socket, typically the one of a reverse proxy in front of llmhop.

Type: string

Default:

"root"

Example:

"caddy"

services.llmhop.listen.<name>.socketMode

Mode of socket. Connecting needs write permission.

Type: string

Default:

"0660"

services.llmhop.listen.<name>.socketUser

Owner of socket.

Type: string

Default:

"root"

services.llmhop.openFirewall

Whether to open the port of every TCP listener in the host firewall.

Type: boolean

Default:

false

services.llmhop.port

TCP port to listen on, registered in the global port registry so a backend reusing it fails evaluation. null listens on the unix socket socket instead.

Type: null or 16 bit unsigned integer; between 0 and 65535 (both inclusive)

Default:

services.llmhop.settings

Configuration written to the JSON config file passed to llmhop. See the upstream Config struct for available fields. host and port only apply outside socket activation, so the module’s listeners come from the options of the same name and listen instead.

The generated file is validated at build time by the binary itself, so unknown keys and malformed model URLs fail nixos-rebuild rather than the service.

Type: JSON value

Default:

{ }

Example:

{
  models = {
    gpt-4 = {
      url = "https://api.openai.com";
    };
  };
}

services.llmhop.socket

Unix socket to listen on while port is null.

Type: string

Default:

"${config.services.llmhop.socketDirectory}/default.sock"

services.llmhop.socketDirectory

Directory of every unix socket llmhop serves or connects to: the default path of each socket listener, and one directory per workload without a port. Those are RuntimeDirectory=s except under a rootless Quadlet user, hence the /run prefix. Each path component must start with a letter, digit, or underscore and contain only those characters, dots, and hyphens.

Type: string matching the pattern /run(/[[:alnum:]][[:alnum:].-]*)+

Default: