# Processing 1 TB with DuckDB in less than 30 seconds

DevFeed: [Processing 1 TB with DuckDB in less than 30 seconds](<https://devfeed.tech/articles/processing-1-tb-with-duckdb-in-less-than-30-seconds-27250.md>)

Original publisher: [Read original article](<https://blog.dataexpert.io/p/i-processed-1-tb-with-duckdb-in-30>)

Author: Matt Martin

Published: 2025-12-23T19:58:37Z

Content type: article

Language: en

Sources: [DataExpert.io Newsletter](<https://devfeed.tech/sources/dataexpert-io-newsletter.md>)

Topics: [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [data](<https://devfeed.tech/topics/data.md>), [parquet](<https://devfeed.tech/topics/parquet.md>), [Python](<https://devfeed.tech/topics/python.md>), [Processes](<https://devfeed.tech/topics/processes.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [data](<https://devfeed.tech/tags/data.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [parquet](<https://devfeed.tech/tags/parquet.md>), [process](<https://devfeed.tech/tags/process.md>), [python](<https://devfeed.tech/tags/python.md>)

## AI overview

This article benchmarks DuckDB on progressively larger datasets, reporting that it read approximately 200 GB in under 10 seconds and 500 GB in about 40 seconds. It also describes generating a 1 TB dataset in roughly 70 minutes on an M2 Pro Mac with 16 GB of RAM using 10 parallel workers.

## Source excerpt

And so can you