Skip to content
Beta

Architecture Asset: Real-Time Streaming Analytics on STACKIT

Last updated on

Overview

Section titled “Overview”

This pattern targets near-real-time analytics. Events stream through a messaging layer, are processed on Kubernetes, persisted to Object Storage and indexed in OpenSearch for live dashboards and alerting.

Typical use case

Section titled “Typical use case”
  • Operational insight: monitor events, transactions or telemetry in near real time.
  • Event-driven processing: trigger downstream actions from streaming data.
  • Replayable history: keep raw events in Object Storage for reprocessing.

Architecture diagram

Section titled “Architecture diagram”
Event ProducersStreaming PlatformStorage & ServingObservabilityMessaging (Pub/Sub)Stream Processing (Kubernetes)Raw Events (Object Storage)Live Index (OpenSearch)

Design best practices

Section titled “Design best practices”
  • Decouple producers and consumers: use the messaging layer as a durable buffer.
  • Make processing idempotent: design for at-least-once delivery and safe retries.
  • Persist raw before transform: keep replayable raw events for reprocessing and audit.
  • Right-size retention and indexes: balance query speed against storage cost.

Related data & AI assets

Section titled “Related data & AI assets”
Asset historyActive 6 of the last 12 weeksTMUpdatedNo updates · 1 bar = 1 week i
Maintainers
TMTobias M.Head of STACKIT Cloud Framework · STACKITOwnerActive 12 of the last 12 weeks · 168 updatesSTACKITwww.linkedin.com/in/tobias-müller-011304172Contributed in STACKIT
Show full history (8 more)