Skip to main content

Install Starlake CLI

Starlake CLI is the command-line interface for the Starlake data pipeline platform. It runs on Linux, macOS, and Windows, and is also available as a Docker image. The CLI requires Java 17 or higher; the setup script installs an embedded JDK automatically if none is found, and the Docker image includes all dependencies. Once installed, the CLI provides commands to bootstrap projects, load data, run SQL transformations, and start the Cockpit.

Install Starlake on Linux, macOS, Windows, or Docker​

Choose the method that matches your environment. Docker is the fastest option because it does not require Java.

Step-by-Step Overview​

  1. Choose an installation method -- Docker (no Java needed), Linux/macOS, or Windows (PowerShell 5.0+). If no Java 17+ is found, the installer provisions an embedded JDK for you.
  2. Run the installation command -- See the platform-specific tabs below.
  3. Verify the installation -- Run starlake --version. The output should display the version number.
  4. Start the Cockpit (optional) -- Run starlake serve and open http://localhost:9900.

Prerequisites for Linux and macOS​

  • Java 17+ is recommended but not required: if java -version reports nothing or a version below 17, the installer downloads an embedded portable Temurin 17 JDK into the install directory (starlake/jdk), with no root rights needed. The starlake launcher picks it up automatically whenever JAVA_HOME is not set.

Install Starlake on Linux or macOS​

Run the one-line installation script:

bash <(curl -sL https://starlake.ai/setup.sh)

The script asks for an installation directory, then shows the most recent release versions to choose from.

To script the installation, both prompts can be answered with flags:

bash <(curl -sL https://starlake.ai/setup.sh) --target=$HOME/starlake --version=1.7.1

--version accepts any released version, not just the ones shown in the menu. The installer fetches every component (core JAR, connectors, Spark runtime) at that exact release's pinned versions, so there is no need to install the latest version first and downgrade afterwards.

The script creates the following directory structure:

starlake/
├── bin/
│   ├── deps/      # Extra dependencies (e.g., Oracle client)
│   ├── sl         # Starlake CLI
│   └── spark      # Embedded Spark runtime

Verify Your Starlake Installation​

Run the following command to confirm the CLI is installed:

starlake --version

Expected output (the exact versions depend on the release you installed):

Starlake 1.8.1
Spark 4.1.3
...

The command also lists the versions of the bundled components (Spark, Hadoop, and the enabled connector drivers).

Start the Starlake Cockpit​

Launch the Cockpit with:

starlake serve

Open http://localhost:9900 in your browser.

To use a different port, set the SL_API_HTTP_PORT environment variable:

SL_API_HTTP_PORT=11000 starlake serve

To access the Cockpit from a remote machine, set SL_API_DOMAIN to your server's domain or IP address:

SL_API_DOMAIN=your.domain.com starlake serve
Store data outside the installation folder

When creating projects through the Cockpit, projects and the management database are stored in the starlake installation folder by default. To ensure they survive reinstalls and upgrades, configure an external location using these environment variables before creating any projects:

  • SL_API_PROJECTS_ROOT: path where projects are stored
  • SL_API_JDBC_URL: JDBC URL for the user and project management database (PostgreSQL)
export SL_API_PROJECTS_ROOT=/path/to/my/projects
export SL_API_JDBC_URL=jdbc:postgresql://localhost:5432/starlake

Starlake VS Code Extension​

Starlake provides an official VS Code extension for editing project configuration files.

Install the VS Code Extension​

  1. Open VS Code
  2. Go to Extensions (Ctrl+Shift+X)
  3. Search for "Starlake"
  4. Install "Starlake Data Pipeline Tools"

The extension provides:

  • Syntax highlighting for Starlake YAML configurations
  • Schema validation for application.sl.yml and domain files
  • SQL transformation snippets
  • Pipeline visualization
  • Starlake Skills for AI-assisted data pipeline development in Claude Code, GitHub Copilot, and other AI coding assistants
note

The VS Code extension requires the Starlake Cockpit to be running in the background. Start it with starlake serve before using the extension.

Upgrade Starlake to a Newer Version​

Run the upgrade command to switch an existing installation to another version:

starlake upgrade

For scripted or CI usage, pass the version directly instead of using the interactive menu (the SL_UPGRADE_VERSION environment variable works too):

starlake upgrade --version 1.7.1

The upgrade process performs four steps:

  1. Version selection -- Retrieves the most recent release versions from GitHub Releases. You choose which one to install, or pass --version to skip the menu.
  2. Launcher self-update -- Replaces the starlake script itself with the latest published one.
  3. Runtime check -- Reads the target release's own version pins and replaces the embedded Spark runtime in ./bin/spark when the target pins a different Spark; otherwise it is kept as-is.
  4. Re-provisioning -- Downloads the target release's installer and lets it replace the core JAR, the API, and the connector JARs at that release's pinned versions, then regenerates versions.sh (or versions.cmd) from scratch.

Upgrades work in both directions: selecting an older version downgrades the installation, including the Spark runtime (for example moving from a 1.8.x release on Spark 4 back to 1.7.x on Spark 3.5). Only the connectors that are already installed are refreshed; a selective installation stays selective.

Where artifacts live

All Starlake artifacts are distributed from GitHub Releases: each release tag vX.Y.Z carries the starlake-core assembly jar and the starlake-api zip with .sha256 checksum companions. Development snapshots are published the same way, as a rolling pre-release tagged v<version>-SNAPSHOT whose assets are overwritten on every snapshot build. Snapshots never appear in the upgrade menu; to install one explicitly, set SL_VERSION to the snapshot version (for example SL_VERSION=1.7.1-SNAPSHOT) before running the setup script. Nothing is published to Maven Central or Sonatype anymore.

Troubleshooting Starlake Installation​

Common installation issues and solutions:

  1. Java version mismatch

    # Check Java version
    java -version
    # Install Java 17+ if needed from Oracle JDK or OpenJDK

  2. Permission issues on Linux/macOS

    chmod +x bin/starlake

  3. Docker issues

    # Verify Docker installation
    docker --version

    # Test Docker permissions
    docker run hello-world
Extra Dependencies

Any additional libraries (e.g., Oracle client) should be copied to the bin/deps folder.

Next Steps​

The exhaustive list of configuration options can be found in the api-env-vars.md file.

Continue with the CLI by following the tutorials below:

Frequently Asked Questions​

What are the prerequisites to install Starlake?​

None beyond a shell: the setup script checks for Java 17 or higher and, if none is found, installs an embedded portable JDK inside the Starlake directory (no root or administrator rights needed). Windows additionally needs PowerShell 5.0 or later. The Docker installation requires only Docker to be installed and running.

Can I install Starlake using Docker?​

Yes, Docker is the fastest way to get started with Starlake. Pull the image with docker pull starlakeai/starlake:latest and run commands by mounting your project directory to /starlake in the container.

Which Java version does Starlake require?​

Starlake requires Java 17 or higher. If your machine has none, the setup script installs an embedded portable Temurin 17 JDK inside the Starlake directory automatically. You can also use your own Oracle JDK or OpenJDK; verify with java -version.

How do I upgrade Starlake to a newer version?​

Run starlake upgrade and pick a version from the menu, or run starlake upgrade --version X.Y.Z non-interactively. The command works in both directions (upgrade and downgrade) and re-provisions the core JAR, connectors, and the Spark runtime when the target release pins a different Spark.

How do I install a specific (older) version of Starlake?​

Pass the version to the setup script directly, for example bash <(curl -sL https://starlake.ai/setup.sh) --version=1.7.1 on Linux/macOS. On an existing installation, starlake upgrade --version 1.7.1 switches to that version. In both cases every component (core JAR, connectors, Spark runtime) is installed at that release's pinned versions; there is no need to install the latest version first.

Can I install Starlake without Java?​

Yes. Use the Docker installation method. The Docker image includes all dependencies, so you do not need to install Java separately.