# Data pipeline monitoring 101: Tracking health and performance across the data stack

DevFeed: [Data pipeline monitoring 101: Tracking health and performance across the data stack](<https://devfeed.tech/articles/data-pipeline-monitoring-101-tracking-health-and-performance-across-the-data-stack-2253.md>)

Original publisher: [Read original article](<https://www.datadoghq.com/blog/data-pipeline-monitoring/>)

Author: Aaron Kaplan; Ryan Warrier

Published: 2026-08-14T00:00:00Z

Content type: article

Language: en

Sources: [Datadog | The Monitor blog](<https://devfeed.tech/sources/datadog-the-monitor-blog.md>)

Topics: [Monitoring](<https://devfeed.tech/topics/monitoring.md>), [data observability](<https://devfeed.tech/topics/data-observability.md>), [observability](<https://devfeed.tech/topics/observability.md>), [data](<https://devfeed.tech/topics/data.md>), [data streams monitoring](<https://devfeed.tech/topics/data-streams-monitoring.md>), [Kafka](<https://devfeed.tech/topics/kafka.md>), [Apache Spark](<https://devfeed.tech/topics/spark.md>), [event driven](<https://devfeed.tech/topics/event-driven.md>), [Streaming](<https://devfeed.tech/topics/streaming.md>)

Tags: [ai-ml](<https://devfeed.tech/tags/ai-ml.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [data](<https://devfeed.tech/tags/data.md>), [data-analytics](<https://devfeed.tech/tags/data-analytics.md>), [data-engineering](<https://devfeed.tech/tags/data-engineering.md>), [data-observability](<https://devfeed.tech/tags/data-observability.md>), [data-streams-monitoring](<https://devfeed.tech/tags/data-streams-monitoring.md>), [event-driven](<https://devfeed.tech/tags/event-driven.md>), [kafka](<https://devfeed.tech/tags/kafka.md>), [learn](<https://devfeed.tech/tags/learn.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [observability](<https://devfeed.tech/tags/observability.md>), [performance](<https://devfeed.tech/tags/performance.md>), [spark](<https://devfeed.tech/tags/spark.md>), [streaming](<https://devfeed.tech/tags/streaming.md>)

## AI overview

This article introduces end-to-end monitoring for modern data pipelines. It explains how to track pipeline health, performance, data quality, and availability across varied architectures and technology layers, with examples including Kafka, Flink, Apache Spark, data lakes, warehouses, and lakehouses.

## Source excerpt

Learn about monitoring the end-to-end health and performance of modern data pipelines.