# Using Materialized Views and Derived Datasets to Optimize Data Queries

DevFeed: [Using Materialized Views and Derived Datasets to Optimize Data Queries](<https://devfeed.tech/articles/you-gotta-push-if-you-wanna-pull-18893.md>)

Original publisher: [Read original article](<https://www.morling.dev/blog/you-gotta-push-if-you-wanna-pull/>)

Published: 2025-12-07T09:05:00Z

Content type: article

Language: en

Sources: [Gunnar Morling](<https://devfeed.tech/sources/gunnar-morling.md>)

Topics: [data-architecture](<https://devfeed.tech/topics/data-architecture.md>), [Data Management](<https://devfeed.tech/topics/data-management.md>), [Database](<https://devfeed.tech/topics/database.md>), [datasets](<https://devfeed.tech/topics/datasets.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [elasticsearch](<https://devfeed.tech/topics/elasticsearch.md>), [parquet](<https://devfeed.tech/topics/parquet.md>)

Tags: [data](<https://devfeed.tech/tags/data.md>), [data-lake](<https://devfeed.tech/tags/data-lake.md>), [data-management](<https://devfeed.tech/tags/data-management.md>), [database](<https://devfeed.tech/tags/database.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [elasticsearch](<https://devfeed.tech/tags/elasticsearch.md>), [latency](<https://devfeed.tech/tags/latency.md>), [parquet](<https://devfeed.tech/tags/parquet.md>), [performance](<https://devfeed.tech/tags/performance.md>)

## AI overview

The article explains how pull-based queries retrieve matching records at query time and why this can create performance, data-format, data-shape, and data-location challenges. It presents materialized views and derived datasets as a way to precompute query results and store them in an optimized format, shape, and location.

## Source excerpt

Table of Contents Materialized Views Embracing Data Duplication Streams for machines, tables for humans Historically, data management systems have been built around the notion of pull queries: users query data which, for instance, is stored in tables in an RDBMS, Parquet files in a data lake, or a full-text index in Elasticsearch. When a user issues a query, the engine will produce the result set at that point in time by churning through the data set and finding all matching records (oftentimes sped up by utilizing indexes).