Starlake Environment Variables

Starlake uses YAML-based environment files to manage configuration across different environments (DEV, PROD, etc.). Global variables are defined in metadata/env.sl.yml. Environment-specific overrides live in files like env.DEV.sl.yml and env.PROD.sl.yml. Variables are referenced using ${} or {{}} syntax and resolved at runtime based on the SL_ENV setting.

How Variable Resolution Works

Starlake resolves variables in the following order:

1. Global defaults -- Variables defined in metadata/env.sl.yml apply to all environments.
2. Environment overrides -- Variables in metadata/env.<ENV>.sl.yml override the global defaults for the corresponding environment.
3. Runtime substitution -- When Starlake runs, it replaces {{variable_name}} and ${variable_name} references in domain and job YAML files with the resolved values.

The ${} and {{}} syntaxes are interchangeable. Both produce the same result.

Define Global Variables in env.sl.yml

Define variables that apply across all environments in metadata/env.sl.yml. These serve as default values that can be overridden per environment.

metadata/env.sl.yml

env:
  root_path: "/tmp/default"
  activeConnection: "duckdb"

Define Environment-Specific Variables (DEV, PROD)

Create separate files for each environment in the metadata/ folder. Each file overrides specific variables for that environment.

Example: A sales domain references a variable for its directory:

load:
  name: "sales"
  directory: "{{root_path}}/sales"
  ack: "ack"

The DEV environment file sets root_path to a local directory:

metadata/env.DEV.sl.yml

env:
  root_path: "/tmp/userguide"

The PROD environment file sets root_path to a cluster directory:

metadata/env.PROD.sl.yml

env:
  root_path: "/cluster/userguide"

When Starlake runs with SL_ENV=DEV, the sales domain directory resolves to /tmp/userguide/sales. With SL_ENV=PROD, it resolves to /cluster/userguide/sales.

Set the Active Environment with SL_ENV

Set the SL_ENV environment variable before running Starlake to select which override file to load:

# For development
export SL_ENV=DEV

# For production
export SL_ENV=PROD

Or inline with a command:

SL_ENV=DEV starlake load

Starlake System Environment Variables

Starlake uses several system-level environment variables to control its behavior:

Variable	Description
SL_ROOT	Root directory of the Starlake project
SL_ENV	Active environment name (e.g., DEV, PROD, BQ, DUCKDB)
SL_DATASETS	Path to the datasets directory
SL_METRICS_ACTIVE	Enable or disable metrics collection
SL_API_HTTP_PORT	Port for the Starlake web UI (default: 9900)
SL_API_DOMAIN	Domain or IP for remote web UI access

These variables can be set in your shell, in a .env file, or passed as Docker environment variables.

Predefined Date/Time Variables

Starlake provides built-in variables set to the current date and time at execution. Use them in any YAML configuration file with {{sl_date}} or ${sl_date} syntax.

Variable	Format
sl_date	yyyyMMdd
sl_datetime	yyyyMMddHHmmss
sl_year	yyyy
sl_month	MM
sl_day	dd
sl_hour	HH
sl_minute	mm
sl_second	ss
sl_milli	SSS
sl_epoch_second	Number of seconds since 1/1/1970
sl_epoch_milli	Number of milliseconds since 1/1/1970

Complete Variable Substitution Example

This example shows the full flow from variable definition to resolved output.

1. Define a global default in env.sl.yml:

metadata/env.sl.yml

env:
  root_path: "/tmp/default"
  report_date: "{{sl_date}}"

2. Override for production in env.PROD.sl.yml:

metadata/env.PROD.sl.yml

env:
  root_path: "/data/warehouse"

3. Reference variables in a load configuration:

metadata/load/sales.sl.yml

load:
  name: "sales"
  directory: "{{root_path}}/sales/{{report_date}}"

4. Result at runtime with SL_ENV=PROD on January 15, 2026:

The directory resolves to: /data/warehouse/sales/20260115

Frequently Asked Questions

How do I define environment-specific variables in Starlake?

Create a file named env.<ENV>.sl.yml in the metadata folder (e.g., env.DEV.sl.yml, env.PROD.sl.yml). Define variables under the env: key. Set SL_ENV=DEV or SL_ENV=PROD before running Starlake to apply the correct values.

What is the variable substitution syntax in Starlake?

Variables can be referenced with ${} or {{}} syntax in domain and job YAML files. Example: directory: "{{root_path}}/sales".

Where do I define global variables in Starlake?

Define global variables in metadata/env.sl.yml. These apply across all environments but can be overridden by environment-specific files (env.DEV.sl.yml, env.PROD.sl.yml).

What predefined variables does Starlake provide?

Starlake provides date/time variables: sl_date (yyyyMMdd), sl_datetime (yyyyMMddHHmmss), sl_year, sl_month, sl_day, sl_hour, sl_minute, sl_second, sl_milli, sl_epoch_second, sl_epoch_milli. These are automatically set to the current date/time.

How does environment override priority work in Starlake?

Environment-specific variables (defined in env.<ENV>.sl.yml) override global variables (defined in env.sl.yml). The active environment is selected by setting the SL_ENV environment variable.

Are the ${} and {{}} syntaxes interchangeable?

Yes. Both ${variable_name} and {{variable_name}} produce the same result. You can use either syntax in domain and job YAML files.

Related

• Configure Database Connections -- define connections that use environment variables for credentials
• Load Tutorial -- use SL_ENV to target different warehouses when loading data
• Transform Tutorial -- use predefined date variables like sl_date in SQL transforms