# Cloud Dataflow

Published articles for Cloud Dataflow.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## New data pipeline management platform at Khan Academy

DevFeed: [New data pipeline management platform at Khan Academy](<https://devfeed.tech/articles/new-data-pipeline-management-platform-at-khan-academy-27388.md>)

Original publisher: [Read original article](<http://engineering.khanacademy.org/posts/khanalytics.htm>)

Author: Khan Academy

Published: 2018-04-30T22:00:00Z

Content type: article

Language: en

Sources: [Khan Academy](<https://devfeed.tech/sources/khan-academy.md>)

Topics: [data](<https://devfeed.tech/topics/data.md>), [DataOps](<https://devfeed.tech/topics/dataops.md>), [systems](<https://devfeed.tech/topics/systems.md>)

Tags: [batch](<https://devfeed.tech/tags/batch.md>), [bigquery](<https://devfeed.tech/tags/bigquery.md>), [cloud-dataflow](<https://devfeed.tech/tags/cloud-dataflow.md>), [data](<https://devfeed.tech/tags/data.md>), [data-pipelines](<https://devfeed.tech/tags/data-pipelines.md>), [debugging](<https://devfeed.tech/tags/debugging.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [news](<https://devfeed.tech/tags/news.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [scheduling](<https://devfeed.tech/tags/scheduling.md>)

### AI overview

Khan Academy developed Khanalytics to manage its growing collection of data pipelines. The platform provides a sandboxed environment for batch jobs, a web interface, automatic parallelization, centralized logs, and pipeline scheduling with dependencies.

### Source excerpt

By Ragini Gupta Data is very crucial to Khan Academy and is itself an internal product for the ... Read more

## Implementing ETL on GCP

DevFeed: [Implementing ETL on GCP](<https://devfeed.tech/articles/implementing-etl-on-gcp-22998.md>)

Original publisher: [Read original article](<https://bravenewgeek.com/implementing-etl-on-gcp/>)

Author: Deepmala

Published: 2020-07-15T20:53:17Z

Content type: tutorial

Language: en

Sources: [Brave New Geek](<https://devfeed.tech/sources/brave-new-geek.md>)

Topics: [DataOps](<https://devfeed.tech/topics/dataops.md>), [Cloud](<https://devfeed.tech/topics/cloud.md>), [BigQuery](<https://devfeed.tech/topics/bigquery.md>), [data loss prevention](<https://devfeed.tech/topics/data-loss-prevention.md>), [Low code](<https://devfeed.tech/topics/low-code.md>), [No-code](<https://devfeed.tech/topics/no-code.md>), [olap](<https://devfeed.tech/topics/olap.md>)

Tags: [analytics](<https://devfeed.tech/tags/analytics.md>), [analytics-pipeline](<https://devfeed.tech/tags/analytics-pipeline.md>), [bi-tools](<https://devfeed.tech/tags/bi-tools.md>), [bigquery](<https://devfeed.tech/tags/bigquery.md>), [cdap](<https://devfeed.tech/tags/cdap.md>), [cloud-data-loss-prevention](<https://devfeed.tech/tags/cloud-data-loss-prevention.md>), [cloud-dataflow](<https://devfeed.tech/tags/cloud-dataflow.md>), [cloud-dataprep](<https://devfeed.tech/tags/cloud-dataprep.md>), [cloud-dataproc](<https://devfeed.tech/tags/cloud-dataproc.md>), [cloud-pub-sub](<https://devfeed.tech/tags/cloud-pub-sub.md>), [cloud-storage](<https://devfeed.tech/tags/cloud-storage.md>), [cloud-tasks](<https://devfeed.tech/tags/cloud-tasks.md>), [data-analytics](<https://devfeed.tech/tags/data-analytics.md>), [data-fusion](<https://devfeed.tech/tags/data-fusion.md>), [data-lake](<https://devfeed.tech/tags/data-lake.md>), [data-loss-prevention](<https://devfeed.tech/tags/data-loss-prevention.md>), [elt](<https://devfeed.tech/tags/elt.md>), [etl](<https://devfeed.tech/tags/etl.md>), [gcp](<https://devfeed.tech/tags/gcp.md>), [no-code](<https://devfeed.tech/tags/no-code.md>), [serverless](<https://devfeed.tech/tags/serverless.md>), [sql](<https://devfeed.tech/tags/sql.md>)

### AI overview

A practical guide to building ETL pipelines on Google Cloud Platform using Google-managed services. It explains a two-phase architecture with Cloud Storage as a data lake, Cloud Data Loss Prevention for sensitive-data detection or redaction, and BigQuery as the curated data warehouse, with attention to low-code and no-code approaches.

### Source excerpt

ETL (Extract-Transform-Load) processes are an essential component of any data analytics program. This typically involves loading data from disparate sources, transforming or enriching it, and storing the curated data in a data warehouse for consumption by different users or systems. An example of this would be taking customer data from operational databases, joining it with data from Salesforce and Google Analytics, and writing it to an OLAP database or BI engine.