Docker deployment

When to use

The Docker Compose stack runs the whole control plane as containers: the manager (REST + admin UI + FlightSQL edge) plus its Postgres metastore, with every Quack node (a qod-node process running DuckDB, bundled in the image) spawned as a child process inside the manager container. It is the right choice for:

• A single-host deployment where you want the manager and its Postgres bundled and supervised together, with persistent state on the host.
• A reproducible demo or evaluation host that boots with one command and no Java, sbt, or Node toolchain on the box.
• Running off a dedicated data disk or NFS mount: the persistent folders are bind-mounted and each one accepts an external host path (see Data folders below).

For first-boot-to-first-query, see the Quickstart. For a single manager container pointed at an external Postgres you already run, see the Docker section of Installation. For multi-host or orchestrated deployments, use the Kubernetes backend.

What comes up

docker compose up (or the scripts/docker/run-docker-compose.sh wrapper) starts two services:

Service	Role
postgres	Control-plane database qod (the qodstate_* tables) plus every tenant database (${tenant}_${tenantDb}, e.g. tpch_tpch1) the manager provisions. DuckLake's __ducklake_* catalog lives inside each tenant database.
quack	The manager: REST + admin UI on :20900, the FlightSQL edge on :31338, the native Quack front door on :9494 (DuckDB ATTACH), and the child Quack node port range.

Three optional profiles add services only when you ask for them: seaweedfs (an in-network S3 object store), observability (Prometheus + Grafana), and starflow (the Starflow UI, signed in from the admin UI). See Profiles below.

Boot it

The scripts/docker/run-docker-compose.sh wrapper handles port preflight, profile auto-detection, optional demo seeding, and readiness waiting. QOD_VERSION picks where the image comes from: a published tag (default latest) is pulled, BUILD builds it from the repo's Dockerfile, and LOCAL reuses the image a previous BUILD produced:

# Pull the published image starlakeai/quack-on-demand and start (default).
./scripts/docker/run-docker-compose.sh

# Pin a version.
QOD_VERSION=0.3.2 ./scripts/docker/run-docker-compose.sh

# Build the image from this repo's Dockerfile instead of pulling.
QOD_VERSION=BUILD ./scripts/docker/run-docker-compose.sh

# Reuse the image from a previous BUILD, without pulling or rebuilding.
QOD_VERSION=LOCAL ./scripts/docker/run-docker-compose.sh

Settings come from a .env file (copied from .env.example on first run). Edit it before booting, or pass the same keys as environment variables. The admin UI is at http://localhost:20900/ui/; log in with ADMIN_USERNAME / ADMIN_PASSWORD (defaults admin / admin).

Plain docker compose up --build also works if you do not want the wrapper's preflight and seeding.

Data folders (external bind mounts)

All persistent state is bind-mounted onto the host. Each mount defaults to a repo-relative folder, but every host path accepts an external override (an absolute path such as /data/qod/pgdata, or a relative one) so the data can live off the repo on a dedicated disk or NFS mount. Set the override in .env or as an environment variable:

Host path (default)	Container path	Override	Holds
./pgdata	/var/lib/postgresql/data	PGDATA_DIR	Postgres data: the qod control plane plus every tenant database.
./ducklake	/app/ducklake	DUCKLAKE_DIR	DuckLake Parquet data files, one subdirectory per tenant database.
./certs	/app/certs	CERTS_DIR	Self-signed FlightSQL TLS cert and key, auto-generated on first boot.
./seaweedfs	/data	SEAWEEDFS_DIR	SeaweedFS object data (only with the seaweedfs profile).
./seaweedfs-config	/etc/seaweedfs	SEAWEEDFS_CONFIG_DIR	SeaweedFS S3 credentials (only with the seaweedfs profile).
./starflow	/app/starlake/projects, /app/starlake/dags	STARFLOW_DIR	Starflow projects and generated DAGs (only with the starflow profile). Its metadata lives in the starlake database under ./pgdata.

For example, to keep the metastore and DuckLake data under /data:

PGDATA_DIR=/data/qod/pgdata DUCKLAKE_DIR=/data/qod/ducklake \
  ./scripts/docker/run-docker-compose.sh

When DuckLake writes to an S3-compatible bucket instead of the filesystem (QOD_DUCKLAKE_DATA_PATH=s3://...), the ./ducklake mount is unused; the catalog persists the s3:// URL and every Quack node resolves it identically regardless of host. See Object storage.

caution

NUKE=1 ./scripts/docker/run-docker-compose.sh asks you to type the project name on a terminal before wiping (non-tty runs skip the prompt) and only wipes the repo-relative defaults (./pgdata, ./ducklake, ./certs, ./seaweedfs, ./seaweedfs-config, ./starflow, plus a legacy ./rustfs left by checkouts that ran the briefly-bundled RustFS). External folders you point these overrides at are not auto-wiped; remove them by hand.

Do not mix a Docker run and a native-jar run against the same catalog database. DuckLake records the absolute data path in Postgres metadata: inside the container it is /app/ducklake/<db>, natively it is <host-cwd>/ducklake/<db>. Use a different control-plane database name per mode, or wipe the data between switches.

Ports

The wrapper exposes these host ports (override in .env):

Env var	Default	Purpose
MANAGER_PORT	20900	REST + admin UI
EDGE_PORT	31338	FlightSQL edge
QUACK_PORT	9494	Native Quack front door (DuckDB ATTACH 'quack:host:9494')
PG_PORT	5432 (15432 in .env.example)	Postgres. With 5432, the wrapper bumps it to 15432 in .env when the host port is busy.
QUACK_MIN_PORT / QUACK_MAX_PORT	21900 / 22500	Child Quack node range
STARFLOW_PORT	9900	Starflow UI (only with the starflow profile)

Fresh start and seeding

# Tear down the stack and wipe the repo-relative state folders, then boot clean.
NUKE=1 ./scripts/docker/run-docker-compose.sh

# Seed the demo tenants before the manager is ready.
# Numeric value = scale factor for TPC-H, TPC-DS, and the SSB star schema.
# (SF=1 is roughly 6M lineitem rows for TPC-H.)
LOAD_TPC=1 ./scripts/docker/run-docker-compose.sh

# Both flags combine.
NUKE=1 LOAD_TPC=1 ./scripts/docker/run-docker-compose.sh

# Single-DuckDB profile: one tenant, one pool, one dual node.
NUKE=1 DEMO=minimal LOAD_TPCH=1 ./scripts/docker/run-docker-compose.sh

LOAD_TPC=1 seeds two demo tenants inside the container: acme loaded with TPC-H (8 tables in schema tpch1, database acme_tpch) plus the SSB star schema derived from it (5 tables in schema ssb1, same database), and globex loaded with TPC-DS (24 tables in schema tpcds1, database globex_tpcds). Use LOAD_TPCH / LOAD_TPCDS / LOAD_SSB to seed each independently. The bundled manifest bootstrap-demo.yaml declares the tenants, pools, roles, groups, and users; QOD_BOOTSTRAP_YAML=classpath:bootstrap-demo.yaml is injected into .env automatically so the manager imports it on startup.

DEMO=minimal injects bootstrap-demo-minimal.yaml instead - the shape for fronting a single DuckDB/DuckLake database: one tenant (acme), one pool (bi), one dual node serving both reads and writes, and the analyst RLS/CLS demo. DEMO=full is the default and covers the multi-tenant, multi-pool, and federation demos. The profile is only consulted when a LOAD_* flag is set and QOD_BOOTSTRAP_YAML is unset; bootstrap only imports into a fresh control plane, so switch profiles with NUKE=1. DEMO=minimal with LOAD_TPCDS warns and skips the TPC-DS loader (no globex tenant in this profile).

You can also seed a running stack directly:

docker compose exec -e PG_HOST=postgres -e DB_NAME=acme_tpch -e SCHEMA_NAME=tpch1 \
  -e DATA_PATH=/app/ducklake/acme_tpch -e SF=1 quack /app/scripts/load-tpch-dbgen.sh
docker compose exec -e PG_HOST=postgres -e DB_NAME=globex_tpcds -e SCHEMA_NAME=tpcds1 \
  -e DATA_PATH=/app/ducklake/globex_tpcds -e SF=1 quack /app/scripts/load-tpcds-dbgen.sh

Optional profiles

Activate with --profile on docker compose, or via PROFILES=... on the wrapper.

Object storage (S3-compatible)

Set QOD_DUCKLAKE_DATA_PATH to an s3:// URL and fill in the QOD_S3_* credentials to write DuckLake Parquet to a bucket instead of the ./ducklake bind mount. Two options:

• Bundled SeaweedFS (no external dependency). Bring it up with docker compose --profile seaweedfs up -d; the wrapper auto-activates this profile when QOD_S3_ENDPOINT (exported, or in .env) points at seaweedfs:8333. Pair with QOD_DUCKLAKE_DATA_PATH=s3://${S3_BUCKET}/<db>. SeaweedFS auto-creates buckets on first write, so no separate bucket step is needed. The Filer UI on :8888 doubles as a file browser.
• External S3 (AWS, R2, MinIO, GCS HMAC). Leave the profile off and set the bucket plus credentials; omit QOD_S3_ENDPOINT to use the AWS default.

The .env.example file has ready-to-uncomment blocks for both. The spawn-quack-node.sh and TPC-H loader detect the s3:// scheme, install DuckDB's httpfs, and CREATE SECRET scoped to each database's data path, so every node reads and writes that database's Parquet in the bucket. Those keys sign nothing else: users who read or write their own files register S3 keys of their own (qod profile s3 add), and an endpoint other than AWS's (SeaweedFS, MinIO) must then be listed in QOD_USER_S3_ALLOWED_ENDPOINTS.

Observability

docker compose --profile observability up -d
# or: PROFILES=observability ./scripts/docker/run-docker-compose.sh

Prometheus scrapes the in-network quack service; Grafana serves a pre-provisioned dashboard on :3000 (anonymous admin, so do not expose it to a public network as-is). See the Resilience and metrics references for what is scraped.

Starflow

For other setups, see Run Starflow with QoD.

The starflow profile runs Starflow (the Starlake API + UI) next to the manager, paired with it over SSO: the admin UI's Workbench entry opens Starflow already signed in, and Starflow queries the gateway as the signed-in QoD user. It is the Docker equivalent of qod start --with-starflow.

STARFLOW_ENABLED=true ./scripts/docker/run-docker-compose.sh

The wrapper activates the profile, generates the shared session secret SESSION_JWT_SECRET into .env when it is missing (printed by name only), picks the edge URL Starflow uses from TLS, and waits for Starflow before printing its URL (http://localhost:9900 by default). The first run builds the Starflow image locally (docker/starflow), which downloads a Starflow release of several hundred MB.

With plain docker compose, set the secret yourself; it must stay stable across restarts, or SSO and Starflow's calls to the manager break:

cat >> .env <<EOF
STARFLOW_ENABLED=true
COMPOSE_PROFILES=starflow
SESSION_JWT_SECRET=$(openssl rand -hex 48)
EOF
docker compose up -d --build

A one-shot starflow-db-init service creates the starlake metadata database on the compose Postgres, then the starflow service starts. If STARFLOW_ENABLED or SESSION_JWT_SECRET is missing, starflow-db-init exits with an error naming it and Starflow does not start. Without the profile, the manager runs with the Starlake integration off, exactly as before.

Env var	Default	Purpose
STARFLOW_ENABLED	false	Turns the profile on for the wrapper, and the Workbench entry on in the manager.
SESSION_JWT_SECRET	(generated by the wrapper)	Signs QoD sessions and the tokens Starflow mints for the edge, so QoD and Starflow must share it. Starflow lists the pools a user may reach with that user's own session (GET /api/profile/pools).
STARFLOW_VERSION	1.8.8	Starflow release baked into the image.
STARFLOW_PORT	9900	Host port of the Starflow UI.
STARFLOW_URL / STARFLOW_DOMAIN	http://localhost:9900 / localhost	Browser-facing Starflow URL and its host, for clients on other machines.
QOD_PUBLIC_URL	http://localhost:20900	Browser-facing manager URL Starflow links back to.
STARFLOW_DB	starlake	Starflow metadata database on the compose Postgres.
STARFLOW_QOD_FLIGHT_URL	http://quack:31338	Edge URL Starflow queries through; the wrapper switches it to https:// when TLS=true.

Fleet mode (manager and workers as containers)

docker-compose.fleet.yml layers on the base file and runs the stack with the fleet runtime: the manager schedules nodes onto two worker containers (fleet-worker-1 on port 21901, fleet-worker-2 on 21902) instead of spawning them itself. It needs a join token in .env and the SeaweedFS profile; it sets QOD_INTERNAL_URL=http://host.docker.internal:20901 and publishes that port, the manager's verdict listener, so the worker nodes can call the manager back for their authorization verdicts (fleet mode refuses to boot without it):

echo "FLEET_JOIN_TOKEN=$(openssl rand -hex 24)" >> .env
docker compose -f docker-compose.yml -f docker-compose.fleet.yml --profile seaweedfs up -d --build

Corporate proxy

When the host needs an HTTP proxy to reach the public internet (for DuckDB extension downloads from extensions.duckdb.org), set HTTP_PROXY / HTTPS_PROXY / NO_PROXY in .env. Compose forwards them into both the build and the runtime, and the manager translates them into DuckDB's SET http_proxy before INSTALL. The in-network hostnames (postgres, seaweedfs) are always added to NO_PROXY so intra-stack traffic and S3 PUTs bypass the proxy. Note that docker pull itself reads the Docker daemon's proxy config, not these variables.

Before exposing beyond localhost

The defaults are tuned for a fast local smoke test. Change these before any non-local exposure:

Setting	Env var	Insecure default
Admin password	ADMIN_PASSWORD (or QOD_ADMIN_PASSWORD, which wins)	admin, set on the first boot only; afterwards rotate with qod auth change-password
Postgres password	PG_PASSWORD	azizam
FlightSQL TLS	TLS	false when TLS is unset (plain docker compose up with no .env). A wrapper run with no .env copies .env.example, which sets TLS=true, so the edge starts with TLS on

There is no REST API key to set: the REST API takes qod login sessions and personal access tokens (qod auth pat create) only, and an API_KEY line left in .env by an earlier release is ignored with a warning. AUTH=false boots only with another FlightSQL authenticator and a way to sign in to the admin UI (see .env.example); otherwise the manager refuses to boot. The session signing secret needs no setting: when SESSION_JWT_SECRET is unset, the first boot generates one and stores it in the control-plane database, so sessions survive restarts.

Enable edge TLS with TLS=true (clients then connect with grpc+tls://...); see TLS. For the full set of tunable keys, see the Configuration reference.

Secure in production

• Set the values in Before exposing beyond localhost (ADMIN_PASSWORD, PG_PASSWORD, TLS=true) in .env before the first boot: the admin password is read on that boot only.
• Remove the ports entry of the postgres service unless outside tools must reach it: it publishes PG_PORT on the host, and the manager logs in as the bundled Postgres's own superuser (PG_USER, default postgres). For tenants you do not fully trust, run an external Postgres with a non-superuser CREATEDB role instead.
• Block the node port range (QUACK_MIN_PORT..QUACK_MAX_PORT, 21900 to 22500) at the host firewall: the compose file publishes it, and only the manager, inside the same container, calls the nodes.
• Keep :9494 (QUACK_PORT) closed to remote clients, or turn its TLS on: the native front door speaks plain HTTP by default, and the compose file does not pass QOD_QUACK_TLS_ENABLED, so add it to the quack service's environment (see TLS).
• Put a TLS-terminating reverse proxy in front of MANAGER_PORT (20900) for remote browsers and CLIs: the REST port speaks plain HTTP.
• Leave the observability profile off on a host others can reach, or turn off Grafana's anonymous admin first (see Observability); with the seaweedfs profile, firewall its published ports (8333, 9333, 8080, 8888).
• Do not seed with LOAD_TPC / LOAD_TPCH / LOAD_TPCDS / LOAD_SSB on a shared host: the demo manifest creates users whose passwords are published in it.
• Treat PGDATA_DIR (./pgdata) and its backups as secrets: the control plane stores catalog passwords, object-store keys, the node-key secret and, unless SESSION_JWT_SECRET is set, the session-signing secret.
• With every database's data on object storage (QOD_DUCKLAKE_DATA_PATH=s3://...), add QOD_NODE_ALLOW_LOCAL_FILES: "false" to the quack service's environment: nodes then open no local file for anyone, superusers included, from their next start. The compose file does not pass it from .env. On the default /app/ducklake/data bind mount, leave it unset, since a database on a local path is then blocked; tenant users still reach no local file by path, because the compose file does not pass QOD_TENANT_FILES_ROOT either. See Local files off.

See Security hardening for the controls to add before tenants you do not fully trust connect.