# apache-parquet Published articles for apache-parquet. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Parquet File Write Support, Bloom Filters, Improved Performance: Hardwood 1.1.0.Beta1 Is Out DevFeed: [Parquet File Write Support, Bloom Filters, Improved Performance: Hardwood 1.1.0.Beta1 Is Out]() Original publisher: [Read original article]() Published: 2026-08-31T19:36:00Z Content type: release Language: en Sources: [Gunnar Morling]() Topics: [parquet](), [Library](), [Parser](), [Command-line interface](), [Compression]() Tags: [apache-parquet](), [api](), [cli](), [compression](), [library](), [parquet](), [performance](), [release]() ### AI overview The first beta of Hardwood 1.1 introduces initial Parquet file-writing support through record-based and batch-oriented APIs. The release also adds Bloom filters, dictionary-based row-group pruning, performance improvements, and CLI enhancements. ### Source excerpt Table of Contents Write Support Query Evaluation: Bloom Filters and Dictionary-Based Row-Group Pruning Performance Improvements Hardwood CLI Closing Thoughts "When is write support gonna land in Hardwood?" That's probably the most common question I got over the last few months. As of today, I am very happy to share that the answer has changed from "It's coming soon" to "A first cut is there, give it a try" -- the first Beta of Hardwood 1.1 is out! This is a major milestone for the project, marking the first step in evolving Hardwood from being solely a Parquet parser to a complete library for this widely used columnar file format. But there's more. This release also comes with significant enhancements to the query layer (Bloom filters, dictionary-based row-group pruning), many performance improvements such as a fast path for effectively fixed-length list columns, an even snappier CLI, and much more. Let's dig into some of the new features and changes! ## Hardwood 1.0: A Fast, Lightweight Apache Parquet Reader for the JVM DevFeed: [Hardwood 1.0: A Fast, Lightweight Apache Parquet Reader for the JVM]() Original publisher: [Read original article]() Published: 2026-06-25T05:30:00Z Content type: release Language: en Sources: [Gunnar Morling]() Topics: [parquet](), [releases](), [Java](), [Library](), [Open Source](), [Maven Central]() Tags: [apache-parquet](), [compatibility](), [compression](), [cpu](), [java](), [jvm](), [library](), [maven-central](), [open-source](), [parquet](), [performance](), [releases]() ### AI overview The article announces Hardwood 1.0, an open-source Apache Parquet library for Java 21 or newer and the JVM. It focuses on fast, multi-threaded reading with no mandatory dependencies, supports local and remote files, and is available from Maven Central. ### Source excerpt Table of Contents Why Hardwood What's in Hardwood 1.0 Performance The Hardwood CLI Building Open-Source With AI A Big Thank You What's Ahead Hardwood is a new Parquet library for the JVM, written from scratch to do one thing well: read (and soon, write) Apache Parquet files fast, with no mandatory dependencies. It is performance-focused and multi-threaded at its core, fanning page decoding out across all your CPU cores by default. Today, Hardwood reaches 1.0. After five preview releases since the start of the year (Alpha1, Beta1, Beta2, CR1, CR2), we now consider Hardwood ready for production, and its public API will evolve with a strong focus on backwards compatibility going forward. Hardwood targets Java 21 or newer, is open-source (Apache License 2.0), and is available from Maven Central. ## Improved Column Reader API, First Cut of Geospatial Support: Hardwood 1.0.0.CR1 Is Available DevFeed: [Improved Column Reader API, First Cut of Geospatial Support: Hardwood 1.0.0.CR1 Is Available]() Original publisher: [Read original article]() Published: 2026-05-31T20:36:00Z Content type: release Language: en Sources: [Gunnar Morling]() Topics: [parquet](), [Command-line interface](), [API]() Tags: [apache-parquet](), [cli](), [documentation](), [geospatial](), [improvements](), [parquet](), [release]() ### AI overview Hardwood 1.0.0.CR1 introduces a substantially improved column reader API for Apache Parquet files, initial support for GEOMETRY and GEOGRAPHY column types, and other improvements to the core library and CLI. The columnar API remains experimental. ### Source excerpt Table of Contents Reworked ColumnReader API Geospatial Support Documentation Overhaul Further Fixes and Improvements I am happy to announce the release of Hardwood 1.0.0.CR1! This first candidate release of Hardwood 1.0 brings a substantially improved API for columnar access to Apache Parquet files, initial support for Parquet's GEOMETRY/GEOGRAPHY column types, and many other improvements to the core library as well as the Hardwood CLI. ## VARIANT Support, Interactive Parquet File TUI: Hardwood 1.0.0.Beta2 Is Out DevFeed: [VARIANT Support, Interactive Parquet File TUI: Hardwood 1.0.0.Beta2 Is Out]() Original publisher: [Read original article]() Published: 2026-04-29T17:45:00Z Content type: release Language: en Sources: [Gunnar Morling]() Topics: [parquet](), [Text-based user interface](), [Command-line interface](), [Parser](), [API](), [Amazon S3]() Tags: [apache-parquet](), [cli](), [compression](), [parquet](), [performance](), [release](), [s3](), [screen](), [storage]() ### AI overview Hardwood 1.0.0.Beta2 adds support for VARIANT columns, an interactive text-based UI for examining Parquet files, improved performance, and more efficient object-storage reads. ### Source excerpt Table of Contents VARIANT Support Hardwood CLI TUI Unified Reader API Performance Improvements Wrapping Up I am happy to announce the release of Hardwood 1.0.0.Beta2! The latest version of this new parser for Apache Parquet comes with support for VARIANT columns, an interactive text-based UI (TUI) for examining and analysing the structure of Parquet files, significantly improved performance, more efficient reading of files from object storage, and much more. ## Hardwood Reaches Beta: S3, Predicate Push-Down, CLI, and More DevFeed: [Hardwood Reaches Beta: S3, Predicate Push-Down, CLI, and More]() Original publisher: [Read original article]() Published: 2026-04-02T18:20:00Z Content type: release Language: en Sources: [Gunnar Morling]() Topics: [parquet](), [Amazon S3](), [Command-line interface]() Tags: [apache-parquet](), [cli](), [parquet](), [rest-api](), [s3]() ### AI overview Hardwood 1.0.0.Beta1 is a new Apache Parquet parser with an S3 backend, predicate push-down for local and remote files, Avro bindings, and a CLI for inspecting Parquet files. Its S3 support uses Java's built-in HTTP client and adds no mandatory dependencies. ### Source excerpt Table of Contents S3 Backend Predicate Push-Down Avro Bindings hardwood-cli Wrapping Up I am pleased to announce the release of Hardwood 1.0.0.Beta1! Hardwood is a new parser for Apache Parquet, optimized for minimal dependencies and great performance. Since the project's initial release just a few weeks back, a small yet very active community has come together and evolved Hardwood significantly. Today, we are shipping an S3 backend, allowing to parse files directly from object storage, predicate pushdown for both local and remote files, Avro bindings, a CLI for inspecting Parquet files, and much more. We're also excited to launch a website for the project, hardwood.dev, which contains the documentation and API reference. Let's dig in. ## Hardwood: A New Parser for Apache Parquet DevFeed: [Hardwood: A New Parser for Apache Parquet]() Original publisher: [Read original article]() Published: 2026-02-26T12:30:00Z Content type: release Language: en Sources: [Gunnar Morling]() Topics: [parquet](), [Parser](), [Java](), [Open Source](), [Compression](), [DuckDB]() Tags: [apache-parquet](), [compression](), [dependencies](), [java](), [maven-central](), [open-source](), [parquet](), [parsing](), [performance](), [release]() ### AI overview The article announces Hardwood, an open-source Java 21+ parser for Apache Parquet. It is designed to minimize dependencies and improve parsing performance through a multithreaded decoding pipeline, with optional libraries for Parquet compression algorithms. The article also introduces Maven setup and row-oriented and columnar APIs. ### Source excerpt Table of Contents Why Hardwood? Hello, Hardwood! Parsing Performance Built With AI, Not By AI What's Next? Today, it's my great pleasure to announce the first public release of Hardwood, a new parser for the Apache Parquet file format, optimized for minimal dependencies and great performance. Hardwood is open-source (Apache License 2.0) and supports Java 21 or newer. You can grab it from Maven Central and start parsing your Parquet files with ease and efficiency. ## How Scribd's Content Crush consolidates S3 objects for storage cost management DevFeed: [How Scribd's Content Crush consolidates S3 objects for storage cost management]() Original publisher: [Read original article]() Author: R Tyler Croy Published: 2026-01-12T00:00:00Z Content type: article Language: en Sources: [Scribd Tech]() Topics: [Amazon S3](), [Amazon Web Services](), [data](), [parquet](), [cloud-infrastructure]() Tags: [apache-parquet](), [aws](), [aws-s3](), [cost](), [data](), [featured](), [object-storage](), [retrieval](), [rust](), [s3](), [storage]() ### AI overview Scribd describes Content Crush, a system that consolidates the many objects generated from uploaded documents and presentations into a single Apache Parquet file in Amazon S3 while preserving virtualized keyspace and retrieval semantics. The system addresses storage-management complexity and the cost of S3 Intelligent-Tiering monitoring at Scribd's scale. ### Source excerpt Scribd and Slideshare have been using AWS S3 for almost twenty years and store hundreds of billions of objects making storage management quite a challenge. My focus at Scribd has generally been around data and storage but only in the past twelve months have I started to really focus on one of our hardest technology problems: cost-effective storage and availability for the hundreds of billions of objects that represent our content library. ## Out with the old file system DevFeed: [Out with the old file system]() Original publisher: [Read original article]() Author: Manfred Moser, David Phillips, Mateusz Gajewski Published: 2025-02-10T00:00:00Z Content type: article Language: en Sources: [Trino Blog]() Topics: [Filesystems](), [Apache Iceberg](), [Hadoop](), [Amazon S3](), [Azure](), [Google Cloud Platform (GCP)]() Tags: [amazon-s3](), [apache-iceberg](), [apache-parquet](), [azure](), [compression](), [data-lake](), [deprecated](), [google-cloud](), [hadoop](), [maintenance](), [release]() ### AI overview Trino 470 deprecated its legacy Hadoop-based file system support, which will be removed in a future release. The article explains Trino's move toward custom file system implementations for cloud storage and describes the migration path for users, including catalog-level file system configuration and warnings for deprecated properties. ### Source excerpt What a long journey it has been! From the start Trino supported querying Hive data and used libraries from the Hive and Hadoop ecosystem. With the release of Trino 470 we mark another milestone to more features and better performance for data lake and lakehouse querying with Trino. We deprecated the legacy file system support, and will permanently remove them in an upcoming release. ## A behind-the-scenes look at building interactive analysis capabilities in Benchling DevFeed: [A behind-the-scenes look at building interactive analysis capabilities in Benchling]() Original publisher: [Read original article]() Author: Wonja Fairbrother Published: 2024-06-11T13:01:25Z Content type: article Language: en Sources: [Benchling]() Topics: [data-processing](), [web applications](), [Amazon EKS](), [Amazon S3](), [API](), [Back end](), [Front end](), [JSON]() Tags: [apache-arrow](), [apache-parquet](), [api](), [architecture](), [backend](), [data](), [data-analysis](), [data-transformation](), [data-visualization](), [duckdb](), [frontend](), [json](), [real-time](), [s3](), [storage](), [systems]() ### AI overview This article explains the architecture and design decisions behind Benchling's Interactive Analysis capabilities. The system lets scientists select data from multiple sources, transform and analyze it in real time, and visualize results within Benchling. Its architecture uses the Benchling web application, a stateless service running on EKS, temporary S3 storage, and a JSON-based transformation API. ### Source excerpt Authors: Wonja Fairbrother and Eli Levine Science is iterative. To design the next experiment, scientists need to analyze the results of previous ones. Interactive Analysis in Benchling allows scientists to perform real-time data transformation, visualization, and analysis without having to transfer it into other systems. In this post we will describe the architecture behind interactive analysis capabilities in Benchling and give a peek into the decision journey we took along the way¹. Interactive Analysis allows scientists to: 1. Select data from many sources: Benchling entity and results data Instrument data Notebook tables Data upload via both API and UI 2. Transform, visualize, and analyze data in real time, without leaving Benchling: Data transformations: filtering, aggregations, window functions, etc. Visualizations: line chart, bar chart, scatter plot, etc. Scientific analysis methods: IC50 and various curve fitting functions Overall architecture The architecture backing Interactive Analysis consists of: The Benchling web application An auto-scaling stateless internal service running on EKS that performs the transformations Temporary S3 storage locations for input and output data, shared between the web app and the service The frontend of the application is responsible for taking in input datasets and transformation configurations from users. The backend of the web application collects all the input data from the appropriate sources, serializes and uploads the data to S3, and sends a synchronous transformation request to the service. The service's API consists of one main endpoint that takes in a JSON payload of transformation parameters. The service can accept a single transformation, or a list of many transformations to perform. In this endpoint, the service downloads and deserializes the input data, performs the transformation with an analysis engine, and serializes and uploads the resulting data to S3. Each request spins up its own self-contained in-memor ## How to Reliably Scale Your Data Platform for High Volumes DevFeed: [How to Reliably Scale Your Data Platform for High Volumes]() Original publisher: [Read original article]() Author: Arbab Ahmed Published: 2020-12-08T17:30:29Z Content type: article Language: en Sources: [Shopify Engineering](), [Shopify Engineering - Shopify Engineering]() Topics: [data](), [Platform Engineering](), [Kafka](), [MySQL](), [parquet](), [Apache Spark](), [systems]() Tags: [analytics](), [apache](), [apache-parquet](), [apache-spark](), [data](), [data-platform-engineering](), [infrastructure](), [insights](), [kafka](), [parquet](), [platform](), [scale](), [spark](), [streaming](), [systems]() ### AI overview Shopify's Data Platform Engineering team describes how it prepared the data platform to handle the high-volume Black Friday and Cyber Monday event. The platform experienced an average throughput increase of 150 percent and processes data through ingestion, batch or stream processing, and delivery to merchants, partners, and internal teams. The article covers the use of Apache Parquet, Apache Spark, dbt, MySQL, Kafka, and tiered services to prioritize reliability and infrastructure investment. ### Source excerpt In this post, we'll outline the approach we took to reliably scale our data platform in preparation for Black Friday and Cyber Monday.