Metrics

The manager registers these series through Micrometer. They are emitted to whichever sink is active (QOD_METRICS_SINK); under the default Prometheus sink they appear at GET :20900/metrics. For how to scrape, push to a cloud monitor, or import the Grafana dashboard, see Observability.

Application metrics

Metric	Type	Labels	Meaning
statements_total	counter	tenant, pool, status	Statements executed, partitioned by outcome status. Drives QPS and error-rate panels.
statement_duration_seconds	histogram	tenant, pool	Statement execution latency; the source of the p50/p95/p99 percentiles.
flightsql_sessions_active	gauge	(none)	Currently open FlightSQL sessions.
pool_nodes	gauge	tenant, pool, role	Node count per pool, broken down by node role.
node_healthy	gauge	tenant, pool, node_id, role	1 when the node is healthy, 0 otherwise.
node_draining	gauge	tenant, pool, node_id, role	1 when the node is draining in-flight work before shutdown.
node_in_flight	gauge	tenant, pool, node_id, role	Statements currently executing on the node.
node_ewma_latency_seconds	gauge	tenant, pool, node_id, role	EWMA of completed-statement latency, the signal the router uses to pick the least-loaded node.

Routing metrics

Emitted per routed statement by the cache-aware placement layer. The locality series (routing_tables_total, routing_placements_total) run on any routing policy and stay populated even with QOD_ROUTING_CACHE_AWARE=false; the decision and load series describe placement outcomes.

Metric	Type	Labels	Meaning
routing_tables_total	counter	tenant, pool, result	Table references seen, split into new (first time this table is routed) and repeat (seen before). A low repeat rate means locality has little to exploit.
routing_placements_total	counter	tenant, pool, result	Repeat-table routings by whether the table stayed on its last node or switched to another. The switch rate is the locality-loss (scatter) signal: high means placement is being destroyed.
routing_decisions_total	counter	tenant, pool, outcome	Placement decisions by outcome: claim, sticky-fresh, sticky-stale, overflow-new-home, overflow-evict-home, pinned-sticky, pinned-move, no-refs-fallback, not-eligible, flag-off. Frequent overflow-evict-home on a pool means three homes are not enough for its hottest tables. pinned-sticky / pinned-move mark statements a transaction pin or soft preferredNode placed rather than the scorer, so they are excluded from the overflow signal.
routing_load_ratio	summary	tenant, pool	Chosen-node in-flight count over the pool average. Should stay under routing.loadCapFactor; a sustained excess means the load cap is not binding.

Maintenance metrics

Emitted per finished managed-maintenance run. These series carry the qod_ prefix; the legacy series above predate the prefix convention.

Metric	Type	Labels	Meaning
qod_maint_runs_total	counter	tenant, tenant_db, result	Maintenance runs by outcome (succeeded, failed, partial). A rising partial count usually means the pinned-file guard is firing.
qod_maint_bytes_reclaimed_total	counter	tenant, tenant_db	Catalog bytes released by runs (physical deletion lags by the cleanup grace window).
qod_maint_files_compacted_total	counter	tenant, tenant_db	Files touched by the merge and rewrite steps.
qod_maint_snapshots_expired_total	counter	tenant, tenant_db	Snapshots expired (pinned snapshots are skipped, never counted here).
qod_maint_duration_seconds	timer	tenant, tenant_db	End-to-end run duration, including maintenance-node spawn time.

DuckDB engine metrics

Scraped from each node's DuckDB engine (duckdb_memory(), duckdb_temporary_files()) by the background health probe, one extra round-trip per node per QOD_HEALTH_CHECK_INTERVAL_SEC tick. A node that has never been scraped successfully publishes no row (rather than a misleading zero); a failed scrape keeps the previous sample until the next tick.

Metric	Type	Labels	Meaning
node_duckdb_memory_used_bytes	gauge	tenant, pool, node_id, role	Buffer-manager memory in use, summed across all consumers (base tables, hash tables, parquet readers, ...). Compare against the node's memory_limit to spot memory pressure before latency degrades.
node_duckdb_temp_storage_bytes	gauge	tenant, pool, node_id, role	Bytes the buffer manager has moved to temporary storage.
node_duckdb_spill_files	gauge	tenant, pool, node_id, role	Live spill-to-disk files. Non-zero means queries are exceeding the memory budget and spilling.
node_duckdb_spill_bytes	gauge	tenant, pool, node_id, role	Total size of live spill files.

JVM and process metrics

Registered by the Micrometer JVM and process binders:

Metric	Type	Meaning
jvm_memory_used_bytes	gauge	Heap and non-heap memory in use.
jvm_gc_pause_seconds_sum	counter	Cumulative GC pause time.
jvm_threads_live_threads	gauge	Live thread count.
process_uptime_seconds	gauge	Process uptime since manager start.

Common labels

Every series can carry static deployment labels when these are set, useful for separating environments in a shared Grafana:

Variable	Label	Example
QOD_METRICS_DEPLOYMENT	deployment	prod-eu
QOD_METRICS_REGION	region	eu-west-1