---
title: "STACKIT Data Engineer"
description: "Comprehensive learning path for data engineers: SQL, dimensional modeling, pipelines with Airflow and Spark, and data lakehouses with Iceberg and Dremio on STACKIT."
hideBreadcrumbs: true
sidebar:
  hidden: true
scfTrail:
  maintainers:
    - user: "tobias.mueller"
  tags: ["STACKIT University", "Data Lakehouse", "Airflow", "Spark", "Dremio", "Kubernetes"]
  steps:
    - style: "compass"
      title: "Cloud Computing Fundamentals"
      trailContext: "Service and deployment models, and core cloud architecture patterns."
      assetId: "data-and-ai/assetcontainer/stackit/cloud-computing-fundamentals"
    - style: "hut"
      title: "STACKIT Platform Fundamentals"
      trailContext: "The STACKIT Portal, core IaaS and PaaS services, and the project and resource model."
      assetId: "architecture/assetcontainer/stackit/stackit-platform-fundamentals"
    - style: "stairs"
      title: "Data Engineering Fundamentals"
      trailContext: "The data engineer's role, the data lifecycle, and data governance and security."
      assetId: "data-and-ai/assetcontainer/stackit/data-engineer-fundamentals"
    - style: "t-bar"
      title: "SQL and Relational Database Fundamentals"
      trailContext: "Relational database concepts, SQL querying, and optimization."
      assetId: "data-and-ai/assetcontainer/stackit/sql-relational-database-fundamentals"
    - style: "chart"
      title: "Data Modeling and Warehousing"
      trailContext: "Star and snowflake schemas, fact and dimension tables, and slowly changing dimensions."
      assetId: "data-and-ai/assetcontainer/stackit/data-modeling-warehousing"
    - style: "gondola"
      title: "Data Lakehouse Fundamentals"
      trailContext: "Open table formats and lakehouse architecture."
      assetId: "data-and-ai/assetcontainer/stackit/data-lakehouse-fundamentals"
    - style: "shield"
      title: "Apache Iceberg Fundamentals"
      trailContext: "Table architecture, schema evolution, and time travel."
      assetId: "data-and-ai/assetcontainer/stackit/apache-iceberg-fundamentals"
    - style: "chairlift"
      title: "Data Pipelines with Apache Airflow"
      trailContext: "Orchestration and DAGs."
      assetId: "data-and-ai/assetcontainer/stackit/data-pipelines-airflow"
    - style: "crevasse"
      title: "Big Data Processing with Apache Spark"
      trailContext: "PySpark, structured streaming, and performance tuning."
      assetId: "data-and-ai/assetcontainer/stackit/big-data-processing-spark"
    - style: "chart"
      title: "Data Lakehouse with Dremio (DeepDive)"
      trailContext: "Query federation, Apache Arrow, and Project Nessie."
      assetId: "data-and-ai/assetcontainer/stackit/data-lakehouse-dremio"
    - style: "gondola"
      title: "Docker Fundamentals"
      trailContext: "Building images, running containers, and Docker Compose."
      assetId: "architecture/assetcontainer/stackit/docker-fundamentals"
    - style: "stairs"
      title: "Kubernetes Fundamentals"
      trailContext: "The foundation for containerized applications."
      assetId: "architecture/assetcontainer/stackit/kubernetes-fundamentals"
    - style: "t-bar"
      title: "Kubernetes for Data Engineering"
      trailContext: "Stateful workloads, Airflow and Spark on Kubernetes, and Kafka."
      assetId: "data-and-ai/assetcontainer/stackit/kubernetes-data-engineering"
    - style: "chairlift"
      title: "STACKIT Dremio: Getting Started"
      trailContext: "Instance setup, SSO/OIDC, and connecting data sources."
      assetId: "data-and-ai/assetcontainer/stackit/stackit-dremio-getting-started"
    - style: "summit"
      title: "STACKIT Dremio: Hands-On Lab"
      trailContext: "Semantic layer and federated queries on STACKIT."
      assetId: "data-and-ai/assetcontainer/stackit/stackit-dremio-hands-on-lab"
source_url: "https://framework.stackit.cloud/data-and-ai/trails/stackit/data-engineer/"
source_file: "docs/data-and-ai/trails/stackit/data-engineer.mdx"
---

## Steps

### 1. Cloud Computing Fundamentals

Stage: `compass`

Service and deployment models, and core cloud architecture patterns.

Asset: [/data-and-ai/assetcontainer/stackit/cloud-computing-fundamentals/](/data-and-ai/assetcontainer/stackit/cloud-computing-fundamentals/) — source: [/raw/data-and-ai/assetcontainer/stackit/cloud-computing-fundamentals.md](/raw/data-and-ai/assetcontainer/stackit/cloud-computing-fundamentals.md)

### 2. STACKIT Platform Fundamentals

Stage: `hut`

The STACKIT Portal, core IaaS and PaaS services, and the project and resource model.

Asset: [/architecture/assetcontainer/stackit/stackit-platform-fundamentals/](/architecture/assetcontainer/stackit/stackit-platform-fundamentals/) — source: [/raw/architecture/assetcontainer/stackit/stackit-platform-fundamentals.md](/raw/architecture/assetcontainer/stackit/stackit-platform-fundamentals.md)

### 3. Data Engineering Fundamentals

Stage: `stairs`

The data engineer's role, the data lifecycle, and data governance and security.

Asset: [/data-and-ai/assetcontainer/stackit/data-engineer-fundamentals/](/data-and-ai/assetcontainer/stackit/data-engineer-fundamentals/) — source: [/raw/data-and-ai/assetcontainer/stackit/data-engineer-fundamentals.md](/raw/data-and-ai/assetcontainer/stackit/data-engineer-fundamentals.md)

### 4. SQL and Relational Database Fundamentals

Stage: `t-bar`

Relational database concepts, SQL querying, and optimization.

Asset: [/data-and-ai/assetcontainer/stackit/sql-relational-database-fundamentals/](/data-and-ai/assetcontainer/stackit/sql-relational-database-fundamentals/) — source: [/raw/data-and-ai/assetcontainer/stackit/sql-relational-database-fundamentals.md](/raw/data-and-ai/assetcontainer/stackit/sql-relational-database-fundamentals.md)

### 5. Data Modeling and Warehousing

Stage: `chart`

Star and snowflake schemas, fact and dimension tables, and slowly changing dimensions.

Asset: [/data-and-ai/assetcontainer/stackit/data-modeling-warehousing/](/data-and-ai/assetcontainer/stackit/data-modeling-warehousing/) — source: [/raw/data-and-ai/assetcontainer/stackit/data-modeling-warehousing.md](/raw/data-and-ai/assetcontainer/stackit/data-modeling-warehousing.md)

### 6. Data Lakehouse Fundamentals

Stage: `gondola`

Open table formats and lakehouse architecture.

Asset: [/data-and-ai/assetcontainer/stackit/data-lakehouse-fundamentals/](/data-and-ai/assetcontainer/stackit/data-lakehouse-fundamentals/) — source: [/raw/data-and-ai/assetcontainer/stackit/data-lakehouse-fundamentals.md](/raw/data-and-ai/assetcontainer/stackit/data-lakehouse-fundamentals.md)

### 7. Apache Iceberg Fundamentals

Stage: `shield`

Table architecture, schema evolution, and time travel.

Asset: [/data-and-ai/assetcontainer/stackit/apache-iceberg-fundamentals/](/data-and-ai/assetcontainer/stackit/apache-iceberg-fundamentals/) — source: [/raw/data-and-ai/assetcontainer/stackit/apache-iceberg-fundamentals.md](/raw/data-and-ai/assetcontainer/stackit/apache-iceberg-fundamentals.md)

### 8. Data Pipelines with Apache Airflow

Stage: `chairlift`

Orchestration and DAGs.

Asset: [/data-and-ai/assetcontainer/stackit/data-pipelines-airflow/](/data-and-ai/assetcontainer/stackit/data-pipelines-airflow/) — source: [/raw/data-and-ai/assetcontainer/stackit/data-pipelines-airflow.md](/raw/data-and-ai/assetcontainer/stackit/data-pipelines-airflow.md)

### 9. Big Data Processing with Apache Spark

Stage: `crevasse`

PySpark, structured streaming, and performance tuning.

Asset: [/data-and-ai/assetcontainer/stackit/big-data-processing-spark/](/data-and-ai/assetcontainer/stackit/big-data-processing-spark/) — source: [/raw/data-and-ai/assetcontainer/stackit/big-data-processing-spark.md](/raw/data-and-ai/assetcontainer/stackit/big-data-processing-spark.md)

### 10. Data Lakehouse with Dremio (DeepDive)

Stage: `chart`

Query federation, Apache Arrow, and Project Nessie.

Asset: [/data-and-ai/assetcontainer/stackit/data-lakehouse-dremio/](/data-and-ai/assetcontainer/stackit/data-lakehouse-dremio/) — source: [/raw/data-and-ai/assetcontainer/stackit/data-lakehouse-dremio.md](/raw/data-and-ai/assetcontainer/stackit/data-lakehouse-dremio.md)

### 11. Docker Fundamentals

Stage: `gondola`

Building images, running containers, and Docker Compose.

Asset: [/architecture/assetcontainer/stackit/docker-fundamentals/](/architecture/assetcontainer/stackit/docker-fundamentals/) — source: [/raw/architecture/assetcontainer/stackit/docker-fundamentals.md](/raw/architecture/assetcontainer/stackit/docker-fundamentals.md)

### 12. Kubernetes Fundamentals

Stage: `stairs`

The foundation for containerized applications.

Asset: [/architecture/assetcontainer/stackit/kubernetes-fundamentals/](/architecture/assetcontainer/stackit/kubernetes-fundamentals/) — source: [/raw/architecture/assetcontainer/stackit/kubernetes-fundamentals.md](/raw/architecture/assetcontainer/stackit/kubernetes-fundamentals.md)

### 13. Kubernetes for Data Engineering

Stage: `t-bar`

Stateful workloads, Airflow and Spark on Kubernetes, and Kafka.

Asset: [/data-and-ai/assetcontainer/stackit/kubernetes-data-engineering/](/data-and-ai/assetcontainer/stackit/kubernetes-data-engineering/) — source: [/raw/data-and-ai/assetcontainer/stackit/kubernetes-data-engineering.md](/raw/data-and-ai/assetcontainer/stackit/kubernetes-data-engineering.md)

### 14. STACKIT Dremio: Getting Started

Stage: `chairlift`

Instance setup, SSO/OIDC, and connecting data sources.

Asset: [/data-and-ai/assetcontainer/stackit/stackit-dremio-getting-started/](/data-and-ai/assetcontainer/stackit/stackit-dremio-getting-started/) — source: [/raw/data-and-ai/assetcontainer/stackit/stackit-dremio-getting-started.md](/raw/data-and-ai/assetcontainer/stackit/stackit-dremio-getting-started.md)

### 15. STACKIT Dremio: Hands-On Lab

Stage: `summit`

Semantic layer and federated queries on STACKIT.

Asset: [/data-and-ai/assetcontainer/stackit/stackit-dremio-hands-on-lab/](/data-and-ai/assetcontainer/stackit/stackit-dremio-hands-on-lab/) — source: [/raw/data-and-ai/assetcontainer/stackit/stackit-dremio-hands-on-lab.md](/raw/data-and-ai/assetcontainer/stackit/stackit-dremio-hands-on-lab.md)