# Deploy Starlake Pipelines

> Step-by-step guide to deploying Starlake data pipelines to production: configure environment variables, copy metadata to cloud storage, and deploy generated DAGs to your orchestrator using the Starlake Docker container.

<script type="application/ld+json">
    {JSON.stringify({
      "@context": "https://schema.org",
      "@type": "HowTo",
      "name": "How to Deploy Starlake Pipelines to Production",
      "description": "Step-by-step guide to deploying Starlake data pipelines to cloud environments using Docker and an orchestrator.",
      "tool": [
        {"@type": "HowToTool", "name": "Docker"},
        {"@type": "HowToTool", "name": "Starlake CLI"}
      ],
      "step": [
        {
          "@type": "HowToStep",
          "name": "Set environment variables",
          "text": "Configure SL_ENV and SL_ROOT environment variables in your Starlake Docker container to match the target environment.",
          "position": 1
        },
        {
          "@type": "HowToStep",
          "name": "Copy metadata to cloud storage",
          "text": "Copy your metadata folder to a cloud storage bucket so the Starlake Docker container can access it from any cloud provider.",
          "position": 2
        },
        {
          "@type": "HowToStep",
          "name": "Deploy DAGs to your orchestrator",
          "text": "Copy the contents of the metadata/dags/generated folder to your orchestrator's DAGs folder to schedule and run your pipelines.",
          "position": 3
        },
        {
          "@type": "HowToStep",
          "name": "Verify deployment",
          "text": "Run a dry-run on the deployed DAGs to verify the pipeline works as expected before enabling scheduling.",
          "position": 4
        }
      ]
    })}
  </script>
  <script type="application/ld+json">
    {JSON.stringify({
      "@context": "https://schema.org",
      "@type": "FAQPage",
      "mainEntity": [
        {
          "@type": "Question",
          "name": "How to deploy a Starlake project to production?",
          "acceptedAnswer": {
            "@type": "Answer",
            "text": "Copy the metadata folder to a cloud storage bucket, then run load/transform commands from the Starlake Docker container on any cloud provider."
          }
        },
        {
          "@type": "Question",
          "name": "What environment variables need to be configured?",
          "acceptedAnswer": {
            "@type": "Answer",
            "text": "SL_ENV (target environment) and SL_ROOT (project root directory) must be defined in the Starlake Docker container."
          }
        },
        {
          "@type": "Question",
          "name": "How to deploy DAGs to the orchestrator?",
          "acceptedAnswer": {
            "@type": "Answer",
            "text": "Copy the contents of the metadata/dags/generated folder to your orchestrator's DAGs folder."
          }
        },
        {
          "@type": "Question",
          "name": "Does Starlake work with any cloud provider?",
          "acceptedAnswer": {
            "@type": "Answer",
            "text": "Yes. The Starlake Docker container can run on GCP, AWS, Azure, or on-premise."
          }
        },
        {
          "@type": "Question",
          "name": "How to abstract differences between environments?",
          "acceptedAnswer": {
            "@type": "Answer",
            "text": "Use environment variables to parameterize your project. Environment configuration is described in the Environment section of the documentation."
          }
        },
        {
          "@type": "Question",
          "name": "What is the typical deployment flow?",
          "acceptedAnswer": {
            "@type": "Answer",
            "text": "1. Configure SL_ENV and SL_ROOT. 2. Copy metadata to cloud storage. 3. Copy the generated DAGs to the orchestrator. 4. The orchestrator runs Starlake commands via the Docker container."
          }
        }
      ]
    })}
  </script>



Deploying a Starlake project to production involves three steps: setting environment variables, copying metadata to cloud storage, and deploying generated DAGs to your orchestrator. The Starlake Docker container runs on GCP, AWS, Azure, or on-premise with no vendor lock-in.

## Deployment overview

1. **Set environment variables** -- Configure `SL_ENV` and `SL_ROOT` in your Starlake Docker container.
2. **Copy metadata to cloud storage** -- Upload your `metadata` folder to a cloud storage bucket.
3. **Deploy DAGs to your orchestrator** -- Copy `metadata/dags/generated` to your orchestrator's DAGs folder.
4. **Verify** -- Run a dry-run to confirm the pipeline works before enabling scheduling.

## Set environment variables

Configure the Starlake Docker container with the target environment:

:::note
Use environment variables to abstract your project from the target environment as described in the [Environment configuration](../../configuration/environment) guide.

Set `SL_ENV` to the target environment name and `SL_ROOT` to the project root directory inside the container.
:::

## Copy metadata to cloud storage

Copy your `metadata` folder to a cloud storage bucket accessible by the Starlake Docker container. The container runs load and transform commands against the metadata stored in cloud storage.

This approach works identically across cloud providers. The Starlake Docker container reads the metadata from the bucket and executes the pipeline commands.

## Deploy DAGs to your orchestrator

Copy the contents of the `metadata/dags/generated` folder to your orchestrator's DAGs folder. The orchestrator schedules and triggers Starlake commands via the Docker container.

For DAG generation details, see the [Orchestration Tutorial](../orchestrate/tutorial). For customization, see [Customizing DAG Generation](../orchestrate/customization).

## Related guides

- [Environment configuration](../../configuration/environment) -- Parameterize your project for multiple environments.
- [Orchestration Tutorial](../orchestrate/tutorial) -- Generate and deploy DAGs.
- [Unit Testing](../unit-tests/concepts) -- Test your pipelines before deployment.

## Frequently Asked Questions

### How to deploy a Starlake project to production?

Copy the metadata folder to a cloud storage bucket, then run load/transform commands from the Starlake Docker container on any cloud provider.

### What environment variables need to be configured?

`SL_ENV` (target environment) and `SL_ROOT` (project root directory) must be defined in the Starlake Docker container.

### How to deploy DAGs to the orchestrator?

Copy the contents of the `metadata/dags/generated` folder to your orchestrator's DAGs folder.

### Does Starlake work with any cloud provider?

Yes. The Starlake Docker container can run on GCP, AWS, Azure, or on-premise.

### How to abstract differences between environments?

Use environment variables to parameterize your project. Environment configuration is described in the [Environment](../../configuration/environment) section of the documentation.

### What is the typical deployment flow?

1. Configure `SL_ENV` and `SL_ROOT`.
2. Copy metadata to cloud storage.
3. Copy the generated DAGs to the orchestrator.
4. The orchestrator runs Starlake commands via the Docker container.