Quick Start Guide
Get started with Starlake in minutes. Choose your preferred installation method below or our advanced UI. To install Starlake with the advanced user interface, follow the Starlake UI Installation Guide.
Installation Methods
- 🐳 Docker (Recommended)
- 🖥️ Linux/MacOS
- 🪟 Windows
Prerequisites
- Docker installed and running
- No other dependencies required - everything is included in the container
Installation
The fastest way to get started with Starlake:
# Pull the latest stable image
docker pull starlakeai/starlake:latest
# Verify installation
docker run -it starlakeai/starlake:latest help
Custom Build Instructions
Build a specific version or branch:
# Clone repository
git clone [email protected]:starlake-ai/starlake.git
cd starlake
# Build with specific version
docker build -t starlakeai/starlake:VERSION \
--build-arg SL_VERSION=1.2.0 .
Running Starlake with Docker
- Basic Usage
- AWS Integration
- GCP Integration
- Azure Integration
- Advanced Options
# Mount current directory and run Starlake
docker run -it \
-v $(pwd):/starlake \
starlakeai/starlake:latest <command>
# Example: Run help command
docker run -it \
-v $(pwd):/starlake \
starlakeai/starlake:latest help
# Run with AWS credentials
docker run -it \
-v $(pwd):/starlake \
-e SL_ROOT=/starlake \
-e AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID \
-e AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY \
-e AWS_SESSION_TOKEN=$AWS_SESSION_TOKEN \
-e AWS_REGION=$AWS_REGION \
starlakeai/starlake:latest <command>
# Run with GCP credentials
docker run -it \
-v $(pwd):/starlake \
-v $HOME/.config/gcloud:/root/.config/gcloud \
-e SL_ROOT=/starlake \
-e GOOGLE_APPLICATION_CREDENTIALS=/root/.config/gcloud/application_default_credentials.json \
starlakeai/starlake:latest <command>
# Run with Azure credentials
docker run -it \
-v $(pwd):/starlake \
-e SL_ROOT=/starlake \
-e AZURE_TENANT_ID=$AZURE_TENANT_ID \
-e AZURE_CLIENT_ID=$AZURE_CLIENT_ID \
-e AZURE_CLIENT_SECRET=$AZURE_CLIENT_SECRET \
starlakeai/starlake:latest <command>
# Run with custom configuration
docker run -it \
-v $(pwd):/starlake \
-e SL_ROOT=/starlake \
-e SL_ENV=production \
-e SL_METRICS_ACTIVE=true \
-e JAVA_OPTS="-Xmx4g" \
-e SPARK_DRIVER_MEMORY=4g \
starlakeai/starlake:latest <command>
Common environment variables:
SL_ROOT
: Project root directory inside containerSL_ENV
: Environment (dev, test, prod)SL_METRICS_ACTIVE
: Enable/disable metrics collectionJAVA_OPTS
: JVM optionsSPARK_DRIVER_MEMORY
: Spark driver memory allocation
Always mount your project directory to /starlake
in the container to ensure proper file access.
For production environments, consider using mounted credential files or secret management services instead of environment variables.
Prerequisites
- Java 11+ installed
# Verify Java version
java -version - If needed, install Java from Oracle JDK or OpenJDK
Installation
One-line installation script:
sh <(curl https://raw.githubusercontent.com/starlake-ai/starlake/master/distrib/setup.sh)
The script will create the following directory structure:
starlake/
├── bin/
│ ├── deps/ # Extra dependencies (e.g., Oracle client)
│ ├── sl # Starlake CLI
│ └── spark # Embedded Spark runtime
Development Tools
VS Code Extension
Prerequisites
- Visual Studio Code installed
- GraphViz (required for pipeline visualization)
Install GraphViz based on your platform:
- Linux
- MacOS
- Windows
- Docker
# Debian/Ubuntu
sudo apt install graphviz
# RHEL/CentOS
sudo yum install graphviz
brew install graphviz
Download the installer from GraphViz Official Website
GraphViz comes pre-installed in the Starlake Docker image.
Installation
Enhance your development experience with our official VS Code extension:
- Open VS Code
- Go to Extensions (Ctrl+Shift+X)
- Search for "Starlake"
- Install "Starlake Data Pipeline Tools"
Features:
- Syntax highlighting for Starlake configurations
- Schema validation
- SQL transformation snippets
- Pipeline visualization
Verify Installation
Test your installation:
starlake --version
You should see output like:
Starlake Version 1.2.0
Next Steps
Troubleshooting
Common installation issues and solutions:
-
Java Version Mismatch
# Check Java version
java -version
# Install correct version if needed -
Permission Issues
# Fix permissions on Linux/MacOS
chmod +x bin/sl -
Docker Issues
# Verify Docker installation
docker --version
# Test Docker permissions
docker run hello-world
Any additional libraries (e.g., Oracle client) should be copied to the bin/deps
folder.