# Single pane of glass for pipelines running on various orchestrators

DevFeed: [Single pane of glass for pipelines running on various orchestrators](<https://devfeed.tech/articles/single-pane-of-glass-for-pipelines-running-on-various-orchestrators-80404.md>)

Original publisher: [Read original article](<https://dlthub.com/blog/single-pane-glass>)

Author: Adrian Brudaru

Published: 2024-02-21T00:00:00Z

Content type: article

Language: en

Sources: [DLT Hub](<https://devfeed.tech/sources/dlt-hub.md>)

Topics: [Data pipelines](<https://devfeed.tech/topics/data-pipelines.md>), [data observability](<https://devfeed.tech/topics/data-observability.md>), [Service fabric](<https://devfeed.tech/topics/service-fabric.md>), [Apache Kafka](<https://devfeed.tech/topics/apache-kafka.md>)

Tags: [aws](<https://devfeed.tech/tags/aws.md>), [data](<https://devfeed.tech/tags/data.md>), [data-teams](<https://devfeed.tech/tags/data-teams.md>), [gcp](<https://devfeed.tech/tags/gcp.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [pipelines](<https://devfeed.tech/tags/pipelines.md>), [product](<https://devfeed.tech/tags/product.md>)

## AI overview

The article describes how data teams can operate pipelines across different infrastructure and orchestrators while improving operational visibility. It discusses Slack alerts for pipeline failures, runtime metrics such as row counts, and schema annotations and column-level lineage to support data governance and deletion of personally identifiable information. It presents standardized data ingestion as a way to support monitoring across orchestrators.

## Source excerpt

## The challenge in the discussion In large organisations, there are often many data teams that serve different departments. These data teams usually cannot agree on where to run their infrastructure, and everyone ends up doing something else. For example: - 40 generated GCP projects with various services used on each - Native AWS services under no particular orchestrator - That on-prem machine that's the only gateway to some strange corporate data - and of course that SaaS orchestrator from...