# parquet Apache Parquet is an open-source, column-oriented data file format designed for efficient data storage and retrieval. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## ClickHouse release 26.8 DevFeed: [ClickHouse release 26.8]() Original publisher: [Read original article]() Author: ClickHouse Published: 2026-09-10T00:00:00Z Content type: release Language: en Sources: [ClickHouse Blog]() Topics: [clickhouse](), [parquet](), [data]() Tags: [bug](), [clickhouse](), [data](), [feature](), [parquet](), [performance](), [release](), [sql](), [tokenizers]() ### AI overview ClickHouse 26.8 is an LTS release that adds background queries, pipelined SQL, text tokenizers, and expanded data lake integrations, alongside performance improvements for Parquet, aggregations, and joins. ### Source excerpt ClickHouse 26.8 LTS introduces background queries, pipelined SQL, new text tokenizers, expanded data lake integrations, and faster Parquet, aggregation, and join queries. ## Loading Parquet data into MySQL with ClickHouse DevFeed: [Loading Parquet data into MySQL with ClickHouse]() Original publisher: [Read original article]() Author: Mark Needham Published: 2026-09-10T00:00:00Z Content type: tutorial Language: en Sources: [ClickHouse Blog]() Topics: [clickhouse](), [MySQL](), [parquet](), [Databases](), [Amazon S3]() Tags: [clickhouse](), [mysql](), [parquet](), [s3](), [sql]() ### AI overview A tutorial on using ClickHouse to load Parquet data into MySQL and query MySQL through ClickHouse table functions. ### Source excerpt Use ClickHouse to load Parquet files into MySQL, explore remote data, and run MySQL queries with table functions and named collections. ## Parquet File Write Support, Bloom Filters, Improved Performance: Hardwood 1.1.0.Beta1 Is Out DevFeed: [Parquet File Write Support, Bloom Filters, Improved Performance: Hardwood 1.1.0.Beta1 Is Out]() Original publisher: [Read original article]() Published: 2026-08-31T19:36:00Z Content type: release Language: en Sources: [Gunnar Morling]() Topics: [parquet](), [Library](), [Parser](), [Command-line interface](), [Compression]() Tags: [apache-parquet](), [api](), [cli](), [compression](), [library](), [parquet](), [performance](), [release]() ### AI overview The first beta of Hardwood 1.1 introduces initial Parquet file-writing support through record-based and batch-oriented APIs. The release also adds Bloom filters, dictionary-based row-group pruning, performance improvements, and CLI enhancements. ### Source excerpt Table of Contents Write Support Query Evaluation: Bloom Filters and Dictionary-Based Row-Group Pruning Performance Improvements Hardwood CLI Closing Thoughts "When is write support gonna land in Hardwood?" That's probably the most common question I got over the last few months. As of today, I am very happy to share that the answer has changed from "It's coming soon" to "A first cut is there, give it a try" -- the first Beta of Hardwood 1.1 is out! This is a major milestone for the project, marking the first step in evolving Hardwood from being solely a Parquet parser to a complete library for this widely used columnar file format. But there's more. This release also comes with significant enhancements to the query layer (Bloom filters, dictionary-based row-group pruning), many performance improvements such as a fast path for effectively fixed-length list columns, an even snappier CLI, and much more. Let's dig into some of the new features and changes! ## A Fast Path for Fixed-Length Lists in Parquet DevFeed: [A Fast Path for Fixed-Length Lists in Parquet]() Original publisher: [Read original article]() Published: 2026-07-22T05:30:00Z Content type: article Language: en Sources: [Gunnar Morling]() Topics: [parquet](), [Embeddings](), [Optimization](), [data]() Tags: [embedding](), [embeddings](), [encoding](), [optimization](), [parquet](), [performance](), [retrieval](), [vector]() ### AI overview The article explains why Apache Parquet handles fixed-length lists inefficiently through Dremel encoding and describes a Hardwood optimization that detects effectively fixed-length data pages and bypasses regular record reconstruction. Benchmarks show speed-ups of up to 3.7x for 768-element lists, reaching performance comparable to a flat column. ### Source excerpt Table of Contents Parquet's Dremel Encoding Reading Effectively-Fixed-Length Lists Faster Performance Gains Summary In its current form Apache Parquet isn't a great fit for storing fixed-length lists, such as coordinates, RGB(A) colors, or--an increasingly common case--vector embeddings driving search and retrieval workloads. A 768-dimensional embedding is just a list of floats that always has the same length, yet Parquet's Dremel machinery encodes it as if that length could vary from row to row, spelling out and reconstructing each vector's structure on read. That costs roughly 3x more than a purely flat columnar representation of the same data (see apache/arrow#34510). ## How We Refresh Razorpay's Data Warehouse 10x Faster with Graphs and Indexes DevFeed: [How We Refresh Razorpay's Data Warehouse 10x Faster with Graphs and Indexes]() Original publisher: [Read original article]() Author: Amit Prabhu Published: 2026-07-14T14:06:16Z Content type: article Language: en Sources: [Razorpay Engineering - Medium]() Topics: [data](), [airflow](), [Apache Spark](), [Amazon S3](), [Apache Iceberg](), [microservices architecture](), [parquet](), [Architecture & Design]() Tags: [airflow](), [analytics](), [apache-iceberg](), [architecture](), [batch](), [data](), [graphs](), [infrastructure](), [parquet](), [pipelines](), [razorpay](), [spark](), [trino](), [warehouse]() ### AI overview Razorpay describes its data warehouse refresh pipeline, which builds wide denormalized Facts by joining data from multiple microservices. The article covers the original Airflow- and Spark-based full-refresh process, the underlying lake formats and query layer, and the scaling challenges that led the team to reconsider refresh strategy, data layout, and high-cardinality dimensions. ### Source excerpt Contributors: Utkarsh Koppikar Rohan Background Razorpay provides the payment infrastructure for millions of merchants globally. Behind every payment, settlement, and refund is a microservices architecture where each service owns its own database. While this keeps services independent and scalable, it creates a challenge for stakeholders who need to see across those boundaries. The Data Platform team manages the infrastructure that bridges this gap. Transactional data flows into the lake via CDC pipelines, ingested onto S3 in Delta Lake, Apache Iceberg, or plain Parquet formats. On top of the lake, we build domain-specific warehouse tables -- wide, pre-joined tables that co-locate all the data a consumer needs, queryable via Trino. These power two use cases: Analytics (internal dashboards on Tableau and Superset) and Reporting (merchants and regulated entities who download structured data exports; Razorpay generates nearly a million such reports per month). The warehouse tables that power both use cases are called Facts. A Fact is a flat denormalised table on S3, produced by joining 10 to 30 microservice tables and materialising the result once. A settlement Fact, for example, merges payments, refunds, adjustments, and card details into a single wide row so that a dashboard or report reads from a single table instead of joining across services in real time. It is closer to a domain-specific materialised view than a classical data warehouse fact table. We maintain over 50 such Facts, and approximately 40% of all merchant reports are served directly from them. As data volumes and the number of entities per fact grew, the batch generation pipeline began to show its limits, prompting us to rethink the refresh strategy, the data layout, and how to handle high-cardinality dimensions. The rest of this post covers that journey. The Full Refresh Pipeline: Our Baseline and the Pain The original full-refresh pipeline was straightforward. Schedule: Airflow schedules Spark jobs o ## Tableflow: Turn Kafka Topics into Iceberg Tables DevFeed: [Tableflow: Turn Kafka Topics into Iceberg Tables]() Original publisher: [Read original article]() Author: Mohtasham Sayeed Mohiuddin Published: 2026-07-10T15:36:14Z Content type: tutorial Language: en Sources: [Confluent: Data in motion]() Topics: [Apache Iceberg](), [Kafka](), [schema-evolution](), [Confluent Cloud](), [parquet](), [Amazon S3](), [Streaming](), [AWS Glue](), [Amazon Redshift](), [BigQuery]() Tags: [amazon-redshift](), [amazon-s3](), [apache-iceberg](), [aws-glue](), [bigquery](), [confluent-cloud](), [kafka](), [parquet](), [schema-evolution](), [streaming](), [technologies](), [tutorial]() ### AI overview This tutorial explains how Confluent Cloud Tableflow continuously materializes Apache Kafka topics as Apache Iceberg or Delta Lake tables. It covers automatic schema handling, type conversion, schema evolution, Parquet conversion, catalog publishing, and table maintenance for querying streaming data with analytics engines and warehouses. ### Source excerpt Learn how Confluent Tableflow turns Kafka topics into Iceberg tables for zero-ETL analytics with automatic schema evolution and open catalog access. ## Running Low-Latency Analytical Workloads with GPU-Accelerated Presto on NVIDIA GB200 NVL72 DevFeed: [Running Low-Latency Analytical Workloads with GPU-Accelerated Presto on NVIDIA GB200 NVL72]() Original publisher: [Read original article]() Author: Tanya Lenz Published: 2026-07-08T16:05:25Z Content type: article Language: en Sources: [NVIDIA Developer](), [NVIDIA Technical Blog]() Topics: [GPU](), [benchmarking](), [SQL](), [Open Source](), [datasets](), [Nvidia](), [GPUDirect](), [NVIDIA DGX](), [NVLink](), [parquet](), [C++](), [Optimization]() Tags: [benchmarking](), [benchmarks](), [c-plus-plus](), [cache](), [communication](), [cpu](), [data](), [data-analytics](), [data-science](), [datasets](), [developer-tools-techniques](), [featured](), [gb200](), [gpu](), [gpudirect](), [latency](), [low-latency](), [multi-gpu](), [nvidia](), [nvidia-dgx](), [nvl72](), [open-source](), [performance]() ### AI overview This article presents GPU-accelerated Presto for low-latency analytical SQL workloads on large datasets. It compares single-node multi-GPU execution on NVIDIA DGX B200 and multinode NVIDIA GB200 NVL72 systems with CPU-based Presto, highlighting NVLink communication, GPUDirect Storage, cuDF algorithms, Parquet data, and benchmark results. ### Source excerpt Presto is an open source, distributed SQL engine for running fast, interactive queries on very large datasets. On NVIDIA GPUs, Presto delivers peak performance... ## The Creator of Pandas on AI, Apache Arrow, and the Future of Software Engineering DevFeed: [The Creator of Pandas on AI, Apache Arrow, and the Future of Software Engineering]() Original publisher: [Read original article]() Author: Daniel Beach Published: 2026-07-08T12:16:09Z Content type: article Language: en Sources: [Data Engineering Central]() Topics: [pandas](), [data-engineering](), [Artificial Intelligence](), [DuckDB](), [data-platforms](), [software-development](), [future of software](), [Apache Spark](), [Hadoop](), [Open Source](), [parquet]() Tags: [ai](), [apache-arrow](), [arrow](), [data](), [data-engineering](), [data-platforms](), [duckdb](), [hadoop](), [large-language-models](), [open-source](), [pandas](), [parquet](), [software-development]() ### AI overview An interview with Wes McKinney covers the origins of pandas and Apache Arrow, the evolution of modern data engineering from Hadoop to lakehouse architectures, and the roles of tools such as Parquet, DuckDB, DataFusion, and Spark. McKinney also discusses how AI affects software development, arguing that it can improve experienced engineers' productivity but does not replace software engineering, architecture, or judgment. ### Source excerpt interview with Wes McKinney ## Hardwood 1.0: A Fast, Lightweight Apache Parquet Reader for the JVM DevFeed: [Hardwood 1.0: A Fast, Lightweight Apache Parquet Reader for the JVM]() Original publisher: [Read original article]() Published: 2026-06-25T05:30:00Z Content type: release Language: en Sources: [Gunnar Morling]() Topics: [parquet](), [releases](), [Java](), [Library](), [Open Source](), [Maven Central]() Tags: [apache-parquet](), [compatibility](), [compression](), [cpu](), [java](), [jvm](), [library](), [maven-central](), [open-source](), [parquet](), [performance](), [releases]() ### AI overview The article announces Hardwood 1.0, an open-source Apache Parquet library for Java 21 or newer and the JVM. It focuses on fast, multi-threaded reading with no mandatory dependencies, supports local and remote files, and is available from Maven Central. ### Source excerpt Table of Contents Why Hardwood What's in Hardwood 1.0 Performance The Hardwood CLI Building Open-Source With AI A Big Thank You What's Ahead Hardwood is a new Parquet library for the JVM, written from scratch to do one thing well: read (and soon, write) Apache Parquet files fast, with no mandatory dependencies. It is performance-focused and multi-threaded at its core, fanning page decoding out across all your CPU cores by default. Today, Hardwood reaches 1.0. After five preview releases since the start of the year (Alpha1, Beta1, Beta2, CR1, CR2), we now consider Hardwood ready for production, and its public API will evolve with a strong focus on backwards compatibility going forward. Hardwood targets Java 21 or newer, is open-source (Apache License 2.0), and is available from Maven Central. ## Bridge Queries in Redpanda SQL DevFeed: [Bridge Queries in Redpanda SQL]() Original publisher: [Read original article]() Author: Paul Wilkinson Published: 2026-06-23T00:00:00Z Content type: article Language: en Sources: [Redpanda]() Topics: [Apache Iceberg](), [SQL](), [Streaming](), [parquet](), [Compression](), [real-time](), [data](), [dashboards]() Tags: [analytics](), [compression](), [data](), [engineering](), [parquet](), [real-time](), [sql](), [storage](), [streaming]() ### AI overview This article introduces bridge queries in Redpanda SQL, which combine historical data from Iceberg tables with recent messages from Redpanda topics through a single virtual SQL table. By reading the freshness gap directly from the topic, teams can flush data to Iceberg less frequently, producing better-sized Parquet files while retaining near-real-time query results and reducing compaction overhead. ### Source excerpt Stop choosing between fresh data and robust Parquet files. Redpanda SQL bridge queries let you query live streaming topics and historical Iceberg tables together, without the compaction overhead. ## Cisco Talos empowers threat researchers while reducing TCO by 75% with ClickHouse Cloud DevFeed: [Cisco Talos empowers threat researchers while reducing TCO by 75% with ClickHouse Cloud]() Original publisher: [Read original article]() Author: ClickHouse Published: 2026-06-15T00:00:00Z Content type: article Language: en Sources: [ClickHouse Blog]() Topics: [clickhouse](), [Cisco Talos](), [Threat Research](), [Cybersecurity](), [data](), [Cloud](), [databricks](), [Amazon S3](), [parquet](), [Amazon Web Services](), [Kubernetes](), [Serverless]() Tags: [api](), [aws](), [cisco-talos](), [clickhouse](), [cybersecurity](), [data](), [databricks](), [event-driven](), [kubernetes](), [parquet](), [s3](), [serverless]() ### AI overview Cisco Talos migrated a nearly 2-trillion-row, 2 PB threat-intelligence dataset from self-managed ClickHouse to ClickHouse Cloud on AWS with zero downtime and a complete data match. The move reduced storage costs by about 90% and total cost of ownership by about 75%. ### Source excerpt Cisco Talos runs a threat intelligence reputation service on ClickHouse Cloud, classifying file hashes so researchers can make fast, accurate security decisions. ## Improved Column Reader API, First Cut of Geospatial Support: Hardwood 1.0.0.CR1 Is Available DevFeed: [Improved Column Reader API, First Cut of Geospatial Support: Hardwood 1.0.0.CR1 Is Available]() Original publisher: [Read original article]() Published: 2026-05-31T20:36:00Z Content type: release Language: en Sources: [Gunnar Morling]() Topics: [parquet](), [Command-line interface](), [API]() Tags: [apache-parquet](), [cli](), [documentation](), [geospatial](), [improvements](), [parquet](), [release]() ### AI overview Hardwood 1.0.0.CR1 introduces a substantially improved column reader API for Apache Parquet files, initial support for GEOMETRY and GEOGRAPHY column types, and other improvements to the core library and CLI. The columnar API remains experimental. ### Source excerpt Table of Contents Reworked ColumnReader API Geospatial Support Documentation Overhaul Further Fixes and Improvements I am happy to announce the release of Hardwood 1.0.0.CR1! This first candidate release of Hardwood 1.0 brings a substantially improved API for columnar access to Apache Parquet files, initial support for Parquet's GEOMETRY/GEOGRAPHY column types, and many other improvements to the core library as well as the Hardwood CLI. ## VARIANT Support, Interactive Parquet File TUI: Hardwood 1.0.0.Beta2 Is Out DevFeed: [VARIANT Support, Interactive Parquet File TUI: Hardwood 1.0.0.Beta2 Is Out]() Original publisher: [Read original article]() Published: 2026-04-29T17:45:00Z Content type: release Language: en Sources: [Gunnar Morling]() Topics: [parquet](), [Text-based user interface](), [Command-line interface](), [Parser](), [API](), [Amazon S3]() Tags: [apache-parquet](), [cli](), [compression](), [parquet](), [performance](), [release](), [s3](), [screen](), [storage]() ### AI overview Hardwood 1.0.0.Beta2 adds support for VARIANT columns, an interactive text-based UI for examining Parquet files, improved performance, and more efficient object-storage reads. ### Source excerpt Table of Contents VARIANT Support Hardwood CLI TUI Unified Reader API Performance Improvements Wrapping Up I am happy to announce the release of Hardwood 1.0.0.Beta2! The latest version of this new parser for Apache Parquet comes with support for VARIANT columns, an interactive text-based UI (TUI) for examining and analysing the structure of Parquet files, significantly improved performance, more efficient reading of files from object storage, and much more. ## Do you still need Elasticsearch for log analytics? ClickHouse says no. DevFeed: [Do you still need Elasticsearch for log analytics? ClickHouse says no.]() Original publisher: [Read original article]() Author: Tom Schreiber; Lionel Palacin Published: 2026-04-23T00:00:00Z Content type: article Language: en Sources: [ClickHouse Blog]() Topics: [clickhouse](), [benchmarking](), [elasticsearch](), [log management](), [observability](), [OpenTelemetry](), [telemetry](), [parquet](), [GitHub]() Tags: [analytics](), [benchmark](), [clickhouse](), [data](), [elasticsearch](), [github](), [logs](), [observability](), [opentelemetry](), [parquet]() ### AI overview ClickHouse presents redesigned full-text search integrated with its analytical engine and benchmarks it against Elasticsearch for log analytics. Across OpenTelemetry workloads of up to 50 billion rows, the article reports 2-6x faster full-text analytical workloads and more compact storage for ClickHouse. ### Source excerpt ClickHouse now brings full-text search and large-scale analytics together in one engine, making it a powerful alternative to Elasticsearch for log analytics. This benchmark shows why. ## Hardwood Reaches Beta: S3, Predicate Push-Down, CLI, and More DevFeed: [Hardwood Reaches Beta: S3, Predicate Push-Down, CLI, and More]() Original publisher: [Read original article]() Published: 2026-04-02T18:20:00Z Content type: release Language: en Sources: [Gunnar Morling]() Topics: [parquet](), [Amazon S3](), [Command-line interface]() Tags: [apache-parquet](), [cli](), [parquet](), [rest-api](), [s3]() ### AI overview Hardwood 1.0.0.Beta1 is a new Apache Parquet parser with an S3 backend, predicate push-down for local and remote files, Avro bindings, and a CLI for inspecting Parquet files. Its S3 support uses Java's built-in HTTP client and adds no mandatory dependencies. ### Source excerpt Table of Contents S3 Backend Predicate Push-Down Avro Bindings hardwood-cli Wrapping Up I am pleased to announce the release of Hardwood 1.0.0.Beta1! Hardwood is a new parser for Apache Parquet, optimized for minimal dependencies and great performance. Since the project's initial release just a few weeks back, a small yet very active community has come together and evolved Hardwood significantly. Today, we are shipping an S3 backend, allowing to parse files directly from object storage, predicate pushdown for both local and remote files, Avro bindings, a CLI for inspecting Parquet files, and much more. We're also excited to launch a website for the project, hardwood.dev, which contains the documentation and API reference. Let's dig in. ## ClickHouse is data lake ready DevFeed: [ClickHouse is data lake ready]() Original publisher: [Read original article]() Author: Karolina Ruiz Rogelj; Melvyn Peignon Published: 2026-03-25T00:00:00Z Content type: article Language: en Sources: [ClickHouse Blog]() Topics: [clickhouse](), [Apache Iceberg](), [parquet](), [Data Space](), [real-time](), [Cloud](), [Amazon S3](), [Azure]() Tags: [analytics](), [apache-iceberg](), [azure](), [caching](), [clickhouse](), [cloud](), [concurrency](), [data](), [interoperability](), [open-source](), [parquet](), [performance](), [real-time](), [s3](), [scale](), [storage]() ### AI overview ClickHouse is presented as data lake ready, with direct querying of Iceberg and Delta Lake data, support for cloud catalogs and Parquet optimizations, and the option to load data into native storage for faster high-concurrency analytics while writing results back to open formats. ### Source excerpt ClickHouse now supports direct querying of Iceberg and Delta Lake formats across major cloud catalogs without requiring data migration. ## Announcing chDB 4: write Pandas, run ClickHouse, now on Hex DevFeed: [Announcing chDB 4: write Pandas, run ClickHouse, now on Hex]() Original publisher: [Read original article]() Author: Ryadh Dahimene; Auxten Wang Published: 2026-03-11T10:52:16Z Content type: release Language: en Sources: [ClickHouse Blog]() Topics: [clickhouse](), [pandas](), [Data Science](), [Optimization](), [parquet](), [API](), [Compression](), [JIT](), [Jupyter Notebook]() Tags: [clickhouse](), [code](), [data](), [data-science](), [how-to](), [jit](), [learn](), [optimization](), [parquet]() ### AI overview chDB 4, launched with Hex integration, adds a Pandas-like DataStore API that executes familiar Pandas code on the in-process ClickHouse OLAP engine. The article explains how this approach addresses Pandas's eager execution and memory overhead by leveraging ClickHouse optimizations for analytical workflows. ### Source excerpt Today, we're launching chDB 4 together with HexEX, where it is available along with the new Pandas-like DataStore API as a native integration ## Understanding Parquet Format for beginners DevFeed: [Understanding Parquet Format for beginners]() Original publisher: [Read original article]() Author: Zach Wilson Published: 2026-03-03T21:52:56Z Content type: tutorial Language: en Sources: [DataExpert.io Newsletter]() Topics: [parquet]() Tags: [beginners](), [format](), [parquet]() ### AI overview A beginner-oriented walkthrough of the Parquet file format. ### Source excerpt A walk through of the most important file format to ever exist ## Hardwood: A New Parser for Apache Parquet DevFeed: [Hardwood: A New Parser for Apache Parquet]() Original publisher: [Read original article]() Published: 2026-02-26T12:30:00Z Content type: release Language: en Sources: [Gunnar Morling]() Topics: [parquet](), [Parser](), [Java](), [Open Source](), [Compression](), [DuckDB]() Tags: [apache-parquet](), [compression](), [dependencies](), [java](), [maven-central](), [open-source](), [parquet](), [parsing](), [performance](), [release]() ### AI overview The article announces Hardwood, an open-source Java 21+ parser for Apache Parquet. It is designed to minimize dependencies and improve parsing performance through a multithreaded decoding pipeline, with optional libraries for Parquet compression algorithms. The article also introduces Maven setup and row-oriented and columnar APIs. ### Source excerpt Table of Contents Why Hardwood? Hello, Hardwood! Parsing Performance Built With AI, Not By AI What's Next? Today, it's my great pleasure to announce the first public release of Hardwood, a new parser for the Apache Parquet file format, optimized for minimal dependencies and great performance. Hardwood is open-source (Apache License 2.0) and supports Java 21 or newer. You can grab it from Maven Central and start parsing your Parquet files with ease and efficiency. ## How Scribd's Content Crush consolidates S3 objects for storage cost management DevFeed: [How Scribd's Content Crush consolidates S3 objects for storage cost management]() Original publisher: [Read original article]() Author: R Tyler Croy Published: 2026-01-12T00:00:00Z Content type: article Language: en Sources: [Scribd Tech]() Topics: [Amazon S3](), [Amazon Web Services](), [data](), [parquet](), [cloud-infrastructure]() Tags: [apache-parquet](), [aws](), [aws-s3](), [cost](), [data](), [featured](), [object-storage](), [retrieval](), [rust](), [s3](), [storage]() ### AI overview Scribd describes Content Crush, a system that consolidates the many objects generated from uploaded documents and presentations into a single Apache Parquet file in Amazon S3 while preserving virtualized keyspace and retrieval semantics. The system addresses storage-management complexity and the cost of S3 Intelligent-Tiering monitoring at Scribd's scale. ### Source excerpt Scribd and Slideshare have been using AWS S3 for almost twenty years and store hundreds of billions of objects making storage management quite a challenge. My focus at Scribd has generally been around data and storage but only in the past twelve months have I started to really focus on one of our hardest technology problems: cost-effective storage and availability for the hundreds of billions of objects that represent our content library. ## Processing 1 TB with DuckDB in less than 30 seconds DevFeed: [Processing 1 TB with DuckDB in less than 30 seconds]() Original publisher: [Read original article]() Author: Matt Martin Published: 2025-12-23T19:58:37Z Content type: article Language: en Sources: [DataExpert.io Newsletter]() Topics: [DuckDB](), [Benchmark](), [data](), [parquet](), [Python](), [Processes]() Tags: [benchmark](), [data](), [duckdb](), [parquet](), [process](), [python]() ### AI overview This article benchmarks DuckDB on progressively larger datasets, reporting that it read approximately 200 GB in under 10 seconds and 500 GB in about 40 seconds. It also describes generating a 1 TB dataset in roughly 70 minutes on an M2 Pro Mac with 16 GB of RAM using 10 parallel workers. ### Source excerpt And so can you ## chDB Kernel Upgrade Journey: Upgrading ClickHouse to v25.8.2.29 DevFeed: [chDB Kernel Upgrade Journey: Upgrading ClickHouse to v25.8.2.29]() Original publisher: [Read original article]() Author: Victor Gao Published: 2025-11-19T00:00:00Z Content type: article Language: en Sources: [ClickHouse Blog]() Topics: [clickhouse](), [Python](), [SQL](), [C++](), [out-of-process](), [parquet](), [CSV](), [JSON]() Tags: [blog-post](), [c-plus-plus](), [clickhouse](), [compression](), [json](), [parquet](), [python](), [sql](), [upgrade]() ### AI overview This blog post describes chDB's upgrade of the embedded ClickHouse kernel from v25.5 to v25.8.2.29. It explains the in-process architecture, zero-copy data transfer between Python and C++, support for multiple data formats, and the dual dynamic-library design used to maintain compatibility across Python versions. ### Source excerpt chDB recently upgraded the ClickHouse kernel from v25.5 to v25.8.2.29. This blog post goes through that journey. ## LeRobot v0.4.0: Supercharging OSS Robot Learning DevFeed: [LeRobot v0.4.0: Supercharging OSS Robot Learning]() Original publisher: [Read original article]() Author: Steven Palma; Michel Aractingi; Pepijn Kooijmans; Caroline Pascal; Jade Choghari; Francesco Capuano; Adil Zouitine; Martino Russi; Thomas Wolf Published: 2025-10-24T00:00:00Z Content type: release Language: en Sources: [Hugging Face - Blog]() Topics: [lerobot](), [Robotics](), [datasets](), [parquet](), [Streaming](), [Command-line interface](), [hugging face](), [Benchmark](), [Open Source]() Tags: [cli](), [datasets](), [hugging-face](), [lerobot](), [oss](), [parquet](), [robotics](), [simulation](), [streaming]() ### AI overview LeRobot v0.4.0 upgrades open-source robot learning with LeRobotDataset v3.0, supporting chunked episodes, large-scale datasets, efficient video storage and streaming, unified Parquet metadata, faster loading, and dataset editing utilities. The release also adds LIBERO and Meta-World simulation support, simplified multi-GPU training, new VLA models, a plugin system for hardware integration, and a Hugging Face Robot Learning Course. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## Unlock the power of images with AI Sheets DevFeed: [Unlock the power of images with AI Sheets]() Original publisher: [Read original article]() Author: Ame Vi; Daniel Vila; Francisco Aranda; Damián Pumar; Leandro von Werra; Thomas Wolf Published: 2025-10-21T00:00:00Z Content type: release Language: en Sources: [Hugging Face - Blog]() Topics: [hugging face](), [AI Development](), [AI Models](), [inference-providers](), [datasets](), [Open Source](), [CSV](), [parquet]() Tags: [ai](), [ai-models](), [code](), [datasets](), [hugging-face](), [inference-providers](), [open-source](), [parquet](), [release](), [vision]() ### AI overview Hugging Face has released a major update to AI Sheets, an open-source spreadsheet tool that uses AI models to build, transform, and enrich datasets without code. The update adds vision support for analyzing images, extracting structured information, generating visuals, and editing images, with access to thousands of open models through Inference Providers. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. [Next page]()