---
title: "Architecture Asset: Real-Time Streaming Analytics on STACKIT"
description: 'Reference architecture for real-time streaming analytics on STACKIT using Pub/Sub messaging, Kubernetes stream processing and OpenSearch live dashboards.'
sidebar:
  badge:
    text: "STACKIT"
    variant: success
scfAsset:
  maintainers:
    - user: "tobias.mueller"
  managed: false
  category: 'blueprint'
  external: false
  tags: ["foundation", "streaming", "analytics", "real-time", "kubernetes", "wip"]
source_url: "https://framework.stackit.cloud/data-and-ai/assetcontainer/stackit/streaming-analytics-pipeline/"
source_file: "docs/data-and-ai/assetcontainer/stackit/streaming-analytics-pipeline.mdx"
---

## Overview

This pattern targets near-real-time analytics. Events stream through a messaging layer, are processed on Kubernetes, persisted to Object Storage and indexed in OpenSearch for live dashboards and alerting.

## Typical use case

- **Operational insight**: monitor events, transactions or telemetry in near real time.
- **Event-driven processing**: trigger downstream actions from streaming data.
- **Replayable history**: keep raw events in Object Storage for reprocessing.

## Architecture diagram

```d2
vars: {
  d2-config: {
    pad: 32
  }
}

style.font-size: 22
direction: right
grid-columns: 1

Producers: "Event Producers" {
  icon: ../../../../../../public/stackit-icons/developer-docs/git.svg
}

Stream: "Streaming Platform" {
  direction: down
  grid-columns: 1

  Bus: "Messaging (Pub/Sub)" {
    icon: ../../../../../../public/stackit-icons/messaging/pubsub.svg
    link: https://docs.stackit.cloud/products/messaging/rabbitmq/
  }

  Processor: "Stream Processing (Kubernetes)" {
    icon: ../../../../../../public/stackit-icons/runtime/kubernetes.svg
    link: https://docs.stackit.cloud/products/runtime/kubernetes-engine/
  }
}

Sinks: "Storage & Serving" {
  direction: down
  grid-columns: 1

  Raw: "Raw Events (Object Storage)" {
    icon: ../../../../../../public/stackit-icons/computing/object-storage.svg
    link: https://docs.stackit.cloud/products/storage/object-storage/
  }
  Index: "Live Index (OpenSearch)" {
    icon: ../../../../../../public/stackit-icons/databases/opensearch.svg
    link: https://docs.stackit.cloud/products/databases/opensearch/
  }
}

Obs: "Observability" {
  icon: ../../../../../../public/stackit-icons/logging-monitoring/observability.svg
  link: https://docs.stackit.cloud/products/logging-and-monitoring/observability/
}

Producers -> Stream.Bus
Stream.Bus -> Stream.Processor
Stream.Processor -> Sinks.Raw
Stream.Processor -> Sinks.Index
Stream.Processor -> Obs
```

## Design best practices

- **Decouple producers and consumers**: use the messaging layer as a durable buffer.
- **Make processing idempotent**: design for at-least-once delivery and safe retries.
- **Persist raw before transform**: keep replayable raw events for reprocessing and audit.
- **Right-size retention and indexes**: balance query speed against storage cost.

## Related data & AI assets

- <LinkChip href="/data-and-ai/assetcontainer/stackit/lakehouse-data-platform/">Sovereign lakehouse data platform</LinkChip>