# A Fast Path for Fixed-Length Lists in Parquet

DevFeed: [A Fast Path for Fixed-Length Lists in Parquet](<https://devfeed.tech/articles/a-fast-path-for-fixed-length-lists-in-parquet-18817.md>)

Original publisher: [Read original article](<https://www.morling.dev/blog/fast-path-for-fixed-length-lists-in-parquet/>)

Published: 2026-07-22T05:30:00Z

Content type: article

Language: en

Sources: [Gunnar Morling](<https://devfeed.tech/sources/gunnar-morling.md>)

Topics: [parquet](<https://devfeed.tech/topics/parquet.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [Optimization](<https://devfeed.tech/topics/optimization.md>), [data](<https://devfeed.tech/topics/data.md>)

Tags: [embedding](<https://devfeed.tech/tags/embedding.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [encoding](<https://devfeed.tech/tags/encoding.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [parquet](<https://devfeed.tech/tags/parquet.md>), [performance](<https://devfeed.tech/tags/performance.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [vector](<https://devfeed.tech/tags/vector.md>)

## AI overview

The article explains why Apache Parquet handles fixed-length lists inefficiently through Dremel encoding and describes a Hardwood optimization that detects effectively fixed-length data pages and bypasses regular record reconstruction. Benchmarks show speed-ups of up to 3.7x for 768-element lists, reaching performance comparable to a flat column.

## Source excerpt

Table of Contents Parquet's Dremel Encoding Reading Effectively-Fixed-Length Lists Faster Performance Gains Summary In its current form Apache Parquet isn't a great fit for storing fixed-length lists, such as coordinates, RGB(A) colors, or--an increasingly common case--vector embeddings driving search and retrieval workloads. A 768-dimensional embedding is just a list of floats that always has the same length, yet Parquet's Dremel machinery encodes it as if that length could vary from row to row, spelling out and reconstructing each vector's structure on read. That costs roughly 3x more than a purely flat columnar representation of the same data (see apache/arrow#34510).