# data observability

Published articles for data observability.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Data pipeline monitoring 101: Tracking health and performance across the data stack

DevFeed: [Data pipeline monitoring 101: Tracking health and performance across the data stack](<https://devfeed.tech/articles/data-pipeline-monitoring-101-tracking-health-and-performance-across-the-data-stack-2253.md>)

Original publisher: [Read original article](<https://www.datadoghq.com/blog/data-pipeline-monitoring/>)

Author: Aaron Kaplan; Ryan Warrier

Published: 2026-08-14T00:00:00Z

Content type: article

Language: en

Sources: [Datadog | The Monitor blog](<https://devfeed.tech/sources/datadog-the-monitor-blog.md>)

Topics: [Monitoring](<https://devfeed.tech/topics/monitoring.md>), [data observability](<https://devfeed.tech/topics/data-observability.md>), [observability](<https://devfeed.tech/topics/observability.md>), [data](<https://devfeed.tech/topics/data.md>), [data streams monitoring](<https://devfeed.tech/topics/data-streams-monitoring.md>), [Kafka](<https://devfeed.tech/topics/kafka.md>), [Apache Spark](<https://devfeed.tech/topics/spark.md>), [event driven](<https://devfeed.tech/topics/event-driven.md>), [Streaming](<https://devfeed.tech/topics/streaming.md>)

Tags: [ai-ml](<https://devfeed.tech/tags/ai-ml.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [data](<https://devfeed.tech/tags/data.md>), [data-analytics](<https://devfeed.tech/tags/data-analytics.md>), [data-engineering](<https://devfeed.tech/tags/data-engineering.md>), [data-observability](<https://devfeed.tech/tags/data-observability.md>), [data-streams-monitoring](<https://devfeed.tech/tags/data-streams-monitoring.md>), [event-driven](<https://devfeed.tech/tags/event-driven.md>), [kafka](<https://devfeed.tech/tags/kafka.md>), [learn](<https://devfeed.tech/tags/learn.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [observability](<https://devfeed.tech/tags/observability.md>), [performance](<https://devfeed.tech/tags/performance.md>), [spark](<https://devfeed.tech/tags/spark.md>), [streaming](<https://devfeed.tech/tags/streaming.md>)

### AI overview

This article introduces end-to-end monitoring for modern data pipelines. It explains how to track pipeline health, performance, data quality, and availability across varied architectures and technology layers, with examples including Kafka, Flink, Apache Spark, data lakes, warehouses, and lakehouses.

### Source excerpt

Learn about monitoring the end-to-end health and performance of modern data pipelines.

## Get reliable answers to business questions with Bits Data Analysis

DevFeed: [Get reliable answers to business questions with Bits Data Analysis](<https://devfeed.tech/articles/get-reliable-answers-to-business-questions-with-bits-data-analysis-2234.md>)

Original publisher: [Read original article](<https://www.datadoghq.com/blog/bits-data-analysis/>)

Author: Jonathan Morin; Jonathan Parisot; Harel Shein

Published: 2026-06-09T00:00:00Z

Content type: article

Language: en

Sources: [Datadog | The Monitor blog](<https://devfeed.tech/sources/datadog-the-monitor-blog.md>)

Topics: [Data analysis](<https://devfeed.tech/topics/data-analysis.md>), [data observability](<https://devfeed.tech/topics/data-observability.md>), [semantic-layer](<https://devfeed.tech/topics/semantic-layer.md>), [SIEM, Security, Observability](<https://devfeed.tech/topics/siem-security-observability.md>), [Model Context Protocol](<https://devfeed.tech/topics/model-context-protocol.md>), [ai-coding](<https://devfeed.tech/topics/ai-coding.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-coding](<https://devfeed.tech/tags/ai-coding.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [bits-ai](<https://devfeed.tech/tags/bits-ai.md>), [data-analytics](<https://devfeed.tech/tags/data-analytics.md>), [data-engineering](<https://devfeed.tech/tags/data-engineering.md>), [data-observability](<https://devfeed.tech/tags/data-observability.md>), [digital-experience-monitoring](<https://devfeed.tech/tags/digital-experience-monitoring.md>), [mcp](<https://devfeed.tech/tags/mcp.md>), [observability](<https://devfeed.tech/tags/observability.md>), [root-cause-analysis](<https://devfeed.tech/tags/root-cause-analysis.md>), [semantic-layer](<https://devfeed.tech/tags/semantic-layer.md>)

### AI overview

Bits Data Analysis, now in Preview, helps teams answer business questions with governed context from their data stack and Datadog telemetry. It uses metric definitions, lineage, freshness, quality signals, application telemetry, and source code to select appropriate data and provide confidence indicators with links to the definitions and tables used.

### Source excerpt

Learn how Bits Data Analysis answers business questions using governed data context from Datadog.

## Chainguard customers safe from elementary-data compromise

DevFeed: [Chainguard customers safe from elementary-data compromise](<https://devfeed.tech/articles/chainguard-customers-safe-from-elementary-data-compromise-12937.md>)

Original publisher: [Read original article](<https://www.chainguard.dev/unchained/chainguard-customers-safe-from-elementary-data-compromise>)

Published: 2026-04-25T00:00:00Z

Content type: article

Language: en

Sources: [Chainguard: Unchained](<https://devfeed.tech/sources/chainguard-unchained.md>)

Topics: [chainguard](<https://devfeed.tech/topics/chainguard.md>), [Malware](<https://devfeed.tech/topics/malware.md>), [data observability](<https://devfeed.tech/topics/data-observability.md>), [Data Quality](<https://devfeed.tech/topics/data-quality.md>), [chainguard containers](<https://devfeed.tech/topics/chainguard-containers.md>), [chainguard libraries](<https://devfeed.tech/topics/chainguard-libraries.md>)

Tags: [chainguard](<https://devfeed.tech/tags/chainguard.md>), [chainguard-containers](<https://devfeed.tech/tags/chainguard-containers.md>), [chainguard-customers](<https://devfeed.tech/tags/chainguard-customers.md>), [chainguard-libraries](<https://devfeed.tech/tags/chainguard-libraries.md>), [data-observability](<https://devfeed.tech/tags/data-observability.md>), [data-quality](<https://devfeed.tech/tags/data-quality.md>), [elementary-data-compromise](<https://devfeed.tech/tags/elementary-data-compromise.md>), [malware](<https://devfeed.tech/tags/malware.md>), [pypi](<https://devfeed.tech/tags/pypi.md>), [pypi-malware](<https://devfeed.tech/tags/pypi-malware.md>)

### AI overview

Chainguard reports that customers using its Python Libraries and Container images were unaffected by the compromised elementary-data 0.23.3 package on PyPI. Chainguard detected malicious patterns before building the package, while the compromised release was quarantined and related GitHub and Docker artifacts were removed.

### Source excerpt

Malicious elementary-data version hit PyPI. Chainguard customers stayed protected by detecting malware pre-build and serving only verified safe versions.

## April 2025 Newsletter

DevFeed: [April 2025 Newsletter](<https://devfeed.tech/articles/april-2025-newsletter-4888.md>)

Original publisher: [Read original article](<https://clickhouse.com/blog/202504-newsletter>)

Author: Mark Needham

Published: 2025-04-16T08:29:42Z

Content type: article

Language: en

Sources: [ClickHouse Blog](<https://devfeed.tech/sources/clickhouse-blog.md>)

Topics: [clickhouse](<https://devfeed.tech/topics/clickhouse.md>), [real-time](<https://devfeed.tech/topics/real-time.md>), [Data Infrastructure](<https://devfeed.tech/topics/data-infrastructure.md>), [data observability](<https://devfeed.tech/topics/data-observability.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [airflow](<https://devfeed.tech/topics/airflow.md>)

Tags: [2025](<https://devfeed.tech/tags/2025.md>), [ai](<https://devfeed.tech/tags/ai.md>), [airflow](<https://devfeed.tech/tags/airflow.md>), [aws](<https://devfeed.tech/tags/aws.md>), [clickhouse](<https://devfeed.tech/tags/clickhouse.md>), [data](<https://devfeed.tech/tags/data.md>), [data-infrastructure](<https://devfeed.tech/tags/data-infrastructure.md>), [data-observability](<https://devfeed.tech/tags/data-observability.md>), [newsletter](<https://devfeed.tech/tags/newsletter.md>), [openai](<https://devfeed.tech/tags/openai.md>), [real-time](<https://devfeed.tech/tags/real-time.md>)

### AI overview

The April 2025 ClickHouse newsletter covers CloudQuery's experience with ClickHouse, the new query condition cache in version 25.3, ClickHouse's Rust development, the acquisition of HyperDX, community news, upcoming events, training, and ClickHouse's role in data observability and AI applications.

### Source excerpt

Welcome to the April ClickHouse newsletter, which will round up what's happened in real-time data warehouses over the last month.

## Why Astronomer chose ClickHouse to power its new data observability platform, Astro Observe

DevFeed: [Why Astronomer chose ClickHouse to power its new data observability platform, Astro Observe](<https://devfeed.tech/articles/why-astronomer-chose-clickhouse-to-power-its-new-data-observability-platform-astro-observe-5657.md>)

Original publisher: [Read original article](<https://clickhouse.com/blog/why-astronomer-chose-clickhouse-to-power-its-new-data-observability-platform-astro-observe>)

Author: Julian LaNeve

Published: 2025-03-10T15:41:46Z

Content type: article

Language: en

Sources: [ClickHouse Blog](<https://devfeed.tech/sources/clickhouse-blog.md>)

Topics: [telemetry](<https://devfeed.tech/topics/telemetry.md>)

Tags: [airflow](<https://devfeed.tech/tags/airflow.md>), [astro](<https://devfeed.tech/tags/astro.md>), [clickhouse](<https://devfeed.tech/tags/clickhouse.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [data-observability](<https://devfeed.tech/tags/data-observability.md>), [monitor](<https://devfeed.tech/tags/monitor.md>), [workflows](<https://devfeed.tech/tags/workflows.md>)

### AI overview

Astronomer chose ClickHouse Cloud for Astro Observe, an observability platform for monitoring and troubleshooting Apache Airflow workflows at scale.

### Source excerpt

Read how Astronomer chose ClickHouse Cloud to power Astro Observe, its new observability platform helping data teams proactively monitor and troubleshoot Apache Airflow workflows at scale with lightning-fast queries and minimal overhead.

## Data quality testing

DevFeed: [Data quality testing](<https://devfeed.tech/articles/data-quality-testing-11744.md>)

Original publisher: [Read original article](<https://incident.io/blog/data-quality-testing>)

Author: Lambert Le Manh

Published: 2024-09-04T16:30:00Z

Content type: article

Language: en

Sources: [The incident.io Blog](<https://devfeed.tech/sources/the-incident-io-blog.md>)

Topics: [Data Quality](<https://devfeed.tech/topics/data-quality.md>), [data observability](<https://devfeed.tech/topics/data-observability.md>), [DataOps](<https://devfeed.tech/topics/dataops.md>), [ci](<https://devfeed.tech/topics/ci.md>), [Transactions](<https://devfeed.tech/topics/transactions.md>)

Tags: [ci](<https://devfeed.tech/tags/ci.md>), [data-observability](<https://devfeed.tech/tags/data-observability.md>), [data-quality](<https://devfeed.tech/tags/data-quality.md>), [flaky](<https://devfeed.tech/tags/flaky.md>), [incident](<https://devfeed.tech/tags/incident.md>), [incident-channel](<https://devfeed.tech/tags/incident-channel.md>), [incident-management](<https://devfeed.tech/tags/incident-management.md>), [incident-response](<https://devfeed.tech/tags/incident-response.md>), [observability](<https://devfeed.tech/tags/observability.md>), [outage](<https://devfeed.tech/tags/outage.md>), [post-mortem](<https://devfeed.tech/tags/post-mortem.md>), [slack-incident](<https://devfeed.tech/tags/slack-incident.md>), [testing](<https://devfeed.tech/tags/testing.md>), [transactions](<https://devfeed.tech/tags/transactions.md>), [workflow](<https://devfeed.tech/tags/workflow.md>)

### AI overview

The article explains how incident.io uses dbt's native testing features for data quality testing and data observability. It describes integrating validation into production and CI workflows, including custom tests and relationship tests, and addresses flaky failures caused by ingestion and transformation pipelines running at different times.

### Source excerpt

Our data observability workflow uses data quality testing to ensure data meets accuracy, consistency, and reliability standards, enabling confident, data-driven decisions. See how we built it, the common challenges we encountered, and the solutions.