# ClickPy at 2 Trillion rows: Scaling ingestion and fixing the past

DevFeed: [ClickPy at 2 Trillion rows: Scaling ingestion and fixing the past](<https://devfeed.tech/articles/clickpy-at-2-trillion-rows-scaling-ingestion-and-fixing-the-past-5184.md>)

Original publisher: [Read original article](<https://clickhouse.com/blog/clickpy-2-trillion-rows>)

Author: Lionel Palacin

Published: 2026-01-21T10:48:32Z

Content type: article

Language: en

Sources: [ClickHouse Blog](<https://devfeed.tech/sources/clickhouse-blog.md>)

Topics: [clickhouse](<https://devfeed.tech/topics/clickhouse.md>), [data](<https://devfeed.tech/topics/data.md>), [Python](<https://devfeed.tech/topics/python.md>), [BigQuery](<https://devfeed.tech/topics/bigquery.md>), [dataset](<https://devfeed.tech/topics/dataset.md>), [cloud-infrastructure](<https://devfeed.tech/topics/cloud-infrastructure.md>), [Code](<https://devfeed.tech/topics/code.md>), [Scripting](<https://devfeed.tech/topics/scripting.md>)

Tags: [bigquery](<https://devfeed.tech/tags/bigquery.md>), [blog](<https://devfeed.tech/tags/blog.md>), [blog-post](<https://devfeed.tech/tags/blog-post.md>), [clickhouse](<https://devfeed.tech/tags/clickhouse.md>), [cloud-storage](<https://devfeed.tech/tags/cloud-storage.md>), [code](<https://devfeed.tech/tags/code.md>), [data](<https://devfeed.tech/tags/data.md>), [database](<https://devfeed.tech/tags/database.md>), [operational](<https://devfeed.tech/tags/operational.md>), [python](<https://devfeed.tech/tags/python.md>), [scale](<https://devfeed.tech/tags/scale.md>), [scripting](<https://devfeed.tech/tags/scripting.md>), [storage](<https://devfeed.tech/tags/storage.md>)

## AI overview

ClickPy has grown to more than 2 trillion Python package-download rows dating back to 2011. This article describes replacing a custom, cron-based ingestion script with ClickPipes and correcting historical data while keeping the service available and ingestion uninterrupted.

## Source excerpt

A look at how ClickPy handles over 2 trillion Python package downloads, from ingestion redesign to fixing historical data at scale.