# Rockset

Rockset is a real-time analytics database that ingests data with 1-2 seconds of freshness and executes analytical SQL queries, including joins and aggregations, in milliseconds.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## What I learned from a month with WalShadow on ClickHouse Managed Postgres

DevFeed: [What I learned from a month with WalShadow on ClickHouse Managed Postgres](<https://devfeed.tech/articles/what-i-learned-from-a-month-with-walshadow-on-clickhouse-managed-postgres-121767.md>)

Original publisher: [Read original article](<https://clickhouse.com/blog/a-month-with-walshadow-clickhouse-managed-postgres>)

Author: Hidehiko ASAHI

Published: 2026-11-06T16:16:22Z

Content type: article

Language: en

Sources: [ClickHouse Blog](<https://devfeed.tech/sources/clickhouse-blog.md>)

Topics: [database-consistency](<https://devfeed.tech/topics/database-consistency.md>), [Rockset](<https://devfeed.tech/topics/rockset.md>), [self-hosted databases vs managed services](<https://devfeed.tech/topics/self-hosted-databases-vs-managed-services.md>)

Tags: [aggregates](<https://devfeed.tech/tags/aggregates.md>), [cdc](<https://devfeed.tech/tags/cdc.md>), [changes](<https://devfeed.tech/tags/changes.md>), [clickhouse](<https://devfeed.tech/tags/clickhouse.md>), [data](<https://devfeed.tech/tags/data.md>), [latency](<https://devfeed.tech/tags/latency.md>), [managed](<https://devfeed.tech/tags/managed.md>), [postgres](<https://devfeed.tech/tags/postgres.md>), [preview](<https://devfeed.tech/tags/preview.md>), [private](<https://devfeed.tech/tags/private.md>), [replication](<https://devfeed.tech/tags/replication.md>), [schema](<https://devfeed.tech/tags/schema.md>), [testing](<https://devfeed.tech/tags/testing.md>)

### AI overview

In this September-October 2026 private-preview test, WalShadow's median replication latency stayed around 0.4 seconds from 1,000 to 200,000 rows per second, while ClickPipes processed about 50,000 rows per second: its latency followed the configured sync interval below that rate, then grew as writes exceeded capacity. Row counts and checksums matched across source and destinations. With WalShadow, the author also found that application writes, schema changes, restarts, and resizes continued replicating without manual intervention. Practical caveats include temporary cross-table inconsistency within a transaction, handling row versions and deletes in materialized views, fixed metadata values on initial-load rows, and delays behind large transactions. ClickPipes supports external Postgres sources and offers pause/resume, logs, and destination-stop email alerts; WalShadow was tested only on Managed Postgres and required checking console status to detect stopped replication. Results reflect private-preview behavior and may change.

### Source excerpt

Hidehiko ASAHI shares a month of WalShadow private-preview testing, comparing replication latency with ClickPipes and checking data correctness, schema changes, and recovery.

## Does your application need local data or faster Postgres?

DevFeed: [Does your application need local data or faster Postgres?](<https://devfeed.tech/articles/does-your-application-need-local-data-or-faster-postgres-115491.md>)

Original publisher: [Read original article](<https://clickhouse.com/blog/does-your-application-need-local-data-or-faster-postgres>)

Author: Lionel Palacin; Gülçin Yıldırım Jelínek

Published: 2026-11-03T14:55:13Z

Content type: article

Language: en

Sources: [ClickHouse Blog](<https://devfeed.tech/sources/clickhouse-blog.md>)

Topics: [database-optimization](<https://devfeed.tech/topics/database-optimization.md>), [copy](<https://devfeed.tech/topics/copy.md>), [consistency](<https://devfeed.tech/topics/consistency.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>), [Rockset](<https://devfeed.tech/topics/rockset.md>), [API](<https://devfeed.tech/topics/api.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [api](<https://devfeed.tech/tags/api.md>), [application](<https://devfeed.tech/tags/application.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [clickhouse](<https://devfeed.tech/tags/clickhouse.md>), [complexity](<https://devfeed.tech/tags/complexity.md>), [connection-pooling](<https://devfeed.tech/tags/connection-pooling.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [copy](<https://devfeed.tech/tags/copy.md>), [nvme](<https://devfeed.tech/tags/nvme.md>), [postgres](<https://devfeed.tech/tags/postgres.md>), [synchronization](<https://devfeed.tech/tags/synchronization.md>)

### AI overview

The article recommends measuring whether a central Postgres database meets latency and throughput needs before adding local synchronization. Sync engines use logical replication to maintain declared subsets in local stores, making covered reads local, but add lifecycle and write-reconciliation work; they suit offline use and predictable, reused working sets. AI agents make subsets harder to predict, so broad or changing sync can transfer unused data or put remote fetches back in the read path. ClickHouse Managed Postgres is presented as a faster central option using attached NVMe, with read replicas for scaling reads. NVMe does not fix inefficient queries, so pooling, query tuning, and concurrency controls still matter. For durability, up to two cross-zone standbys are supported: two use synchronous replication, while one is asynchronous; WAL archiving and backups support recovery. For analytics, WalShadow is in private preview with roughly 200 ms commit-to-visible latency, and pg_clickhouse requires configured foreign tables and supported query pushdown; it does not automatically route every query.

### Source excerpt

Local synchronization makes reads fast, but adds another data layer to manage. Explore how AI agents change the trade-off and when NVMe-backed Postgres can simplify the architecture.

## Run Large PlanetScale Queries in Laravel With DB::olap()

DevFeed: [Run Large PlanetScale Queries in Laravel With DB::olap()](<https://devfeed.tech/articles/run-large-planetscale-queries-in-laravel-with-db-olap-89997.md>)

Original publisher: [Read original article](<https://laravel-news.com/laravel-planetscale-olap>)

Author: Paul Redmond

Published: 2026-10-19T02:22:57Z

Content type: news

Language: en

Sources: [Laravel](<https://devfeed.tech/sources/laravel.md>)

Topics: [olap](<https://devfeed.tech/topics/olap.md>), [Laravel](<https://devfeed.tech/topics/laravel.md>), [Rockset](<https://devfeed.tech/topics/rockset.md>), [Eloquent ORM](<https://devfeed.tech/topics/eloquent.md>)

Tags: [added](<https://devfeed.tech/tags/added.md>), [csv](<https://devfeed.tech/tags/csv.md>), [customer](<https://devfeed.tech/tags/customer.md>), [data](<https://devfeed.tech/tags/data.md>), [database](<https://devfeed.tech/tags/database.md>), [databases](<https://devfeed.tech/tags/databases.md>), [groupby](<https://devfeed.tech/tags/groupby.md>), [join](<https://devfeed.tech/tags/join.md>), [laravel](<https://devfeed.tech/tags/laravel.md>), [laravel-tutorials](<https://devfeed.tech/tags/laravel-tutorials.md>), [mysql](<https://devfeed.tech/tags/mysql.md>), [olap](<https://devfeed.tech/tags/olap.md>), [orders](<https://devfeed.tech/tags/orders.md>), [php](<https://devfeed.tech/tags/php.md>), [planetscale](<https://devfeed.tech/tags/planetscale.md>), [query](<https://devfeed.tech/tags/query.md>), [vitess](<https://devfeed.tech/tags/vitess.md>)

### AI overview

Laravel 13.36 adds DB::olap() to run a callback's queries in Vitess OLAP mode, then restore OLTP mode. The method handles both read and write connections and does nothing special for non-Vitess databases. The article recommends it for occasional large exports, backfills, and admin reports, while cautioning against routine application queries.

### Source excerpt

Laravel's new DB::olap() method runs PlanetScale and Vitess queries in OLAP mode to get past the 100,000 row limit, then switches the connection back. The post Run Large PlanetScale Queries in Laravel With DB::olap() appeared first on Laravel News. Join the Laravel Newsletter to get Laravel articles like this directly in your inbox.

## ClickHouse Cloud and BigQuery benchmarked for real-time analytics performance and cost

DevFeed: [ClickHouse Cloud and BigQuery benchmarked for real-time analytics performance and cost](<https://devfeed.tech/articles/why-bigquery-can-t-match-clickhouse-cloud-for-real-time-analytics-77070.md>)

Original publisher: [Read original article](<https://clickhouse.com/blog/clickhouse-vs-bigquery-real-time-performance-per-dollar>)

Author: Tom Schreiber; Lionel Palacin

Published: 2026-10-08T10:49:15Z

Content type: comparison

Language: en

Sources: [ClickHouse Blog](<https://devfeed.tech/sources/clickhouse-blog.md>)

Topics: [Rockset](<https://devfeed.tech/topics/rockset.md>), [BigQuery](<https://devfeed.tech/topics/bigquery.md>), [streaming data analytics](<https://devfeed.tech/topics/streaming-data-analytics.md>), [Redpanda](<https://devfeed.tech/topics/redpanda-data-streaming.md>)

Tags: [analytics](<https://devfeed.tech/tags/analytics.md>), [bigquery](<https://devfeed.tech/tags/bigquery.md>), [clickhouse](<https://devfeed.tech/tags/clickhouse.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [real-time-analytics](<https://devfeed.tech/tags/real-time-analytics.md>)

### AI overview

CostBench compares ClickHouse Cloud and BigQuery while both systems ingest stock-market quotes and serve queries. In this workload, ClickHouse Cloud had lower modeled preparation and query costs and shorter accumulated query runtime, producing scores reported as 438x and 512x better performance per dollar under BigQuery Capacity and On-demand pricing. The article details the workload, pricing assumptions, measurement boundaries, and how each system prepares and serves fresh data.

### Source excerpt

ClickHouse Cloud delivered 438x better performance per dollar than BigQuery in CostBench. We trace the gap from fresh data arriving to fast answers coming back.

## Importing Data using Java Table Functions

DevFeed: [Importing Data using Java Table Functions](<https://devfeed.tech/articles/importing-data-using-java-table-functions-64792.md>)

Original publisher: [Read original article](<https://duckdb.org/2026/10/05/import-data-with-java.html>)

Author: Guest Author, Geertjan Wielenga, Alex Kasko

Published: 2026-10-05T00:00:00Z

Content type: tutorial

Language: en

Sources: [DuckDB](<https://devfeed.tech/sources/duckdb.md>)

Topics: [Rockset](<https://devfeed.tech/topics/rockset.md>), [Java](<https://devfeed.tech/topics/java.md>), [batch insert](<https://devfeed.tech/topics/batch-insert.md>)

Tags: [batch-insert](<https://devfeed.tech/tags/batch-insert.md>), [data](<https://devfeed.tech/tags/data.md>), [database](<https://devfeed.tech/tags/database.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [java](<https://devfeed.tech/tags/java.md>), [using-duckdb](<https://devfeed.tech/tags/using-duckdb.md>)

### AI overview

The article describes importing vulnerability data from MongoDB into DuckDB to speed analytical queries and exports. It compares a MongoDB community extension with Java JDBC batch inserts, DuckDB's Appender, and a custom Java table function. The table function had the lowest overhead among the Java approaches; the production pipeline used a non-parallel version with sorting in the same table creation query, achieving import times on par with the extension.

### Source excerpt

When the analytics screen running on our main operational database got too slow, we moved a year of data into DuckDB on the same server. Getting the data in was the hard part. This is the story of every method we tried, and why a table function written in pure Java is the one we shipped.

## Jev for analytics: turn text columns into gold, in plain SQL

DevFeed: [Jev for analytics: turn text columns into gold, in plain SQL](<https://devfeed.tech/articles/jev-for-analytics-turn-text-columns-into-gold-in-plain-sql-78940.md>)

Original publisher: [Read original article](<https://motherduck.com/blog/jev-for-analytics>)

Author: Mehdi Ouazza

Published: 2026-09-29T00:00:00Z

Content type: tutorial

Language: en

Sources: [MotherDuck](<https://devfeed.tech/sources/motherduck.md>)

Topics: [SQL](<https://devfeed.tech/topics/sql.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [Rockset](<https://devfeed.tech/topics/rockset.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [about](<https://devfeed.tech/tags/about.md>), [account](<https://devfeed.tech/tags/account.md>), [agent](<https://devfeed.tech/tags/agent.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [alert](<https://devfeed.tech/tags/alert.md>), [all](<https://devfeed.tech/tags/all.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [column](<https://devfeed.tech/tags/column.md>), [consumer](<https://devfeed.tech/tags/consumer.md>), [cost](<https://devfeed.tech/tags/cost.md>), [jev](<https://devfeed.tech/tags/jev.md>), [llm](<https://devfeed.tech/tags/llm.md>), [plus](<https://devfeed.tech/tags/plus.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [sql](<https://devfeed.tech/tags/sql.md>), [text](<https://devfeed.tech/tags/text.md>), [trust](<https://devfeed.tech/tags/trust.md>), [with](<https://devfeed.tech/tags/with.md>)

### AI overview

`prompt_jev()` turns free text into a typed choice from labels you supply, letting repeated classification run once and leaving later analysis to ordinary SQL. In the excerpt's 100,000 CFPB complaints, it classified seven requested-outcome categories in 82.2 seconds; the launch benchmark reports $0.50 per 100,000 rows versus $1.58 for gpt-5-nano and $37.58 for GPT-5.6 Terra. The labels must be constant choices, though a DuckDB variable can hold a taxonomy stored in a table. Its confidence scores can flag uncertain rows for review, but high confidence does not guarantee correctness; a sample comparison also showed disagreement between models around fuzzy category boundaries. Use an LLM to develop categories or generate explanations, and test Jev on a sample before scaling. The excerpt says text is sent to TypeSafe for inference and advises checking data-handling requirements for sensitive data.

### Source excerpt

Turn a free-text column into typed SQL columns with prompt_jev(): 100,000 consumer complaints classified in 82 seconds, at about 1% of a frontier LLM's cost, plus how to check the labels before you trust them.

## Core dump epidemiology: fixing an 18-year-old bug

DevFeed: [Core dump epidemiology: fixing an 18-year-old bug](<https://devfeed.tech/articles/core-dump-epidemiology-fixing-an-18-year-old-bug-6359.md>)

Original publisher: [Read original article](<https://openai.com/index/core-dump-epidemiology-data-infrastructure-bug>)

Published: 2026-06-30T00:00:00Z

Content type: article

Language: en

Sources: [OpenAI News](<https://devfeed.tech/sources/openai-news.md>)

Topics: [bug](<https://devfeed.tech/topics/bug.md>), [OpenAI](<https://devfeed.tech/topics/openai.md>), [Rockset](<https://devfeed.tech/topics/rockset.md>), [C++](<https://devfeed.tech/topics/c-plus-plus.md>), [Data Infrastructure](<https://devfeed.tech/topics/data-infrastructure.md>), [Azure](<https://devfeed.tech/topics/azure.md>), [Hardware](<https://devfeed.tech/topics/hardware.md>), [race-condition](<https://devfeed.tech/topics/race-condition.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [Streaming](<https://devfeed.tech/topics/streaming.md>)

Tags: [azure](<https://devfeed.tech/tags/azure.md>), [bug](<https://devfeed.tech/tags/bug.md>), [c-plus-plus](<https://devfeed.tech/tags/c-plus-plus.md>), [data-infrastructure](<https://devfeed.tech/tags/data-infrastructure.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [openai](<https://devfeed.tech/tags/openai.md>), [race-condition](<https://devfeed.tech/tags/race-condition.md>), [software](<https://devfeed.tech/tags/software.md>)

### AI overview

OpenAI engineers analyzed a large population of core dumps to investigate rare crashes in the Rockset service. The investigation uncovered two unrelated causes: silent CPU corruption on an Azure host and an 18-year-old race condition in GNU libunwind.

### Source excerpt

OpenAI engineers used large-scale core dump analysis to debug rare infrastructure crashes, uncovering both a hardware fault and a long-standing software bug.

## I don't care about your database benchmarks (and neither should you)

DevFeed: [I don't care about your database benchmarks (and neither should you)](<https://devfeed.tech/articles/i-don-t-care-about-your-database-benchmarks-and-neither-should-you-81418.md>)

Original publisher: [Read original article](<https://stack.convex.dev/on-competitive-benchmarks>)

Author: Stack

Published: 2026-02-25T16:50:00Z

Content type: opinion

Language: en

Sources: [Stack](<https://devfeed.tech/sources/stack.md>)

Topics: [Database](<https://devfeed.tech/topics/database.md>), [Rockset](<https://devfeed.tech/topics/rockset.md>), [consistency](<https://devfeed.tech/topics/consistency.md>), [acid](<https://devfeed.tech/topics/acid.md>), [transaction-log](<https://devfeed.tech/topics/transaction-log.md>), [read-replica](<https://devfeed.tech/topics/read-replica.md>), [Internets of interest](<https://devfeed.tech/topics/internets-of-interest.md>)

Tags: [acid](<https://devfeed.tech/tags/acid.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [buffer](<https://devfeed.tech/tags/buffer.md>), [caching](<https://devfeed.tech/tags/caching.md>), [class](<https://devfeed.tech/tags/class.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cpus](<https://devfeed.tech/tags/cpus.md>), [crash](<https://devfeed.tech/tags/crash.md>), [customer](<https://devfeed.tech/tags/customer.md>), [data](<https://devfeed.tech/tags/data.md>), [database](<https://devfeed.tech/tags/database.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [development](<https://devfeed.tech/tags/development.md>), [durability](<https://devfeed.tech/tags/durability.md>), [failure](<https://devfeed.tech/tags/failure.md>), [fsync](<https://devfeed.tech/tags/fsync.md>), [perspectives](<https://devfeed.tech/tags/perspectives.md>)

### AI overview

Convex argues that database benchmark charts often compare systems with different durability, consistency, workload, and configuration assumptions, making headline performance rankings misleading. It recommends learning from real customer workloads and reporting representative latency, cost, and performance trends to help customers evaluate capacity and scaling.

### Source excerpt

Consider this our declaration of permanent disengagement from "bar chart wars." It's good dev marketing, but it's a waste of time.

## How to Join Two Collections with a SQL Derivation in Estuary

DevFeed: [How to Join Two Collections with a SQL Derivation in Estuary](<https://devfeed.tech/articles/how-to-join-two-collections-with-a-sql-derivation-in-estuary-82087.md>)

Original publisher: [Read original article](<https://estuary.dev/blog/derivations-join-collections-sql>)

Author: Dani Pálma

Published: 2024-09-26T20:48:12Z

Content type: tutorial

Language: en

Sources: [Estuary](<https://devfeed.tech/sources/estuary.md>)

Topics: [JOIN](<https://devfeed.tech/topics/join.md>), [Rockset](<https://devfeed.tech/topics/rockset.md>), [SQL](<https://devfeed.tech/topics/sql.md>), [data-engineering](<https://devfeed.tech/topics/data-engineering.md>), [concatenation](<https://devfeed.tech/topics/concatenation.md>)

Tags: [count](<https://devfeed.tech/tags/count.md>), [data-engineering](<https://devfeed.tech/tags/data-engineering.md>), [derivations](<https://devfeed.tech/tags/derivations.md>), [estuary](<https://devfeed.tech/tags/estuary.md>), [join](<https://devfeed.tech/tags/join.md>), [snowflake](<https://devfeed.tech/tags/snowflake.md>), [sql](<https://devfeed.tech/tags/sql.md>)

### AI overview

This tutorial shows how to define an Estuary SQL derivation that combines artist and album collections by artist_id, sums album plays, and keeps results updated as data arrives. It covers previewing the derivation with flowctl, publishing it, and materializing the output to Snowflake.

### Source excerpt

Learn to write and publish a SQL derivation in Estuary that joins two collections on artist_id, previews results with flowctl, and materializes to Snowflake.

## Rockset Migration: Choosing an Alternative and Preparing Real-Time Analytics Pipelines

DevFeed: [Rockset Migration: Choosing an Alternative and Preparing Real-Time Analytics Pipelines](<https://devfeed.tech/articles/rockset-migration-guide-how-to-migrate-and-thrive-in-with-real-time-analytics-after-rockset-82357.md>)

Original publisher: [Read original article](<https://estuary.dev/blog/rockset-migration-guide>)

Author: Rob Meyer

Published: 2024-07-11T16:20:06Z

Content type: tutorial

Language: en

Sources: [Estuary](<https://devfeed.tech/sources/estuary.md>)

Topics: [Rockset](<https://devfeed.tech/topics/rockset.md>), [schema-evolution](<https://devfeed.tech/topics/schema-evolution.md>), [data migrations](<https://devfeed.tech/topics/data-migrations.md>), [clickhouse](<https://devfeed.tech/topics/clickhouse.md>)

Tags: [migration](<https://devfeed.tech/tags/migration.md>), [migration-guide](<https://devfeed.tech/tags/migration-guide.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [setup](<https://devfeed.tech/tags/setup.md>), [strategies](<https://devfeed.tech/tags/strategies.md>)

### AI overview

This guide discusses migrating from Rockset, comparing alternative analytics databases and recommending resilient data pipelines. It emphasizes preserving source data, supporting schema evolution, and running replacement destinations side by side to ease migration and prepare for future vendor changes.

### Source excerpt

Struggling with your Rockset migration? Discover proven strategies, Rockset alternatives, and tools to streamline the process and build a resilient real-time analytics setup.

## Teradata and Knoware join forces to help New Zealand and Australia customers accelerate data analytics investments

DevFeed: [Teradata and Knoware join forces to help New Zealand and Australia customers accelerate data analytics investments](<https://devfeed.tech/articles/teradata-and-knoware-join-forces-to-help-new-zealand-and-australia-customers-accelerate-data-analytics-investments-81106.md>)

Original publisher: [Read original article](<https://www.teradata.com/press-releases/2024/teradata-and-knoware-join-forces>)

Author: Jennifer Donahue

Published: 2024-07-05T07:00:00Z

Content type: news

Language: en

Sources: [Teradata News](<https://devfeed.tech/sources/teradata-news.md>)

Topics: [Data Management](<https://devfeed.tech/topics/data-management.md>), [databricks](<https://devfeed.tech/topics/databricks.md>), [Rockset](<https://devfeed.tech/topics/rockset.md>)

Tags: [article](<https://devfeed.tech/tags/article.md>), [australia](<https://devfeed.tech/tags/australia.md>), [data](<https://devfeed.tech/tags/data.md>), [data-analytics](<https://devfeed.tech/tags/data-analytics.md>), [new-zealand](<https://devfeed.tech/tags/new-zealand.md>)

### AI overview

Teradata and New Zealand-based consultancy Knoware announced a partnership to help customers in New Zealand and Australia advance data and analytics initiatives. Knoware will provide Teradata VantageCloud and ClearScape Analytics services, including advice, implementation, training and support.

### Source excerpt

Teradata and Knoware announced a partnership to empower customers in New Zealand and Australia to deliver on outcomes with their data and analytics investments.

## Introducing Omni aggregate awareness

DevFeed: [Introducing Omni aggregate awareness](<https://devfeed.tech/articles/introducing-omni-aggregate-awareness-82461.md>)

Original publisher: [Read original article](<https://omni.co/blog/aggregate-awareness>)

Author: Steven Talbot

Published: 2024-05-24T16:00:00Z

Content type: release

Language: en

Sources: [Omni Blog](<https://devfeed.tech/sources/omni-blog.md>)

Topics: [Rockset](<https://devfeed.tech/topics/rockset.md>), [Cache](<https://devfeed.tech/topics/cache.md>)

Tags: [aggregate-awareness](<https://devfeed.tech/tags/aggregate-awareness.md>), [dbt](<https://devfeed.tech/tags/dbt.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>)

### AI overview

Omni's aggregate awareness feature uses metadata to match precomputed aggregate tables to queries and route suitable queries to those smaller tables, while queries needing other columns continue to use the underlying data. The article describes a dbt integration for materializing rollup tables and says the approach can make eligible queries 10 to 100 times faster.

### Source excerpt

Today, we're excited to bring together two key workflows in Omni with aggregate awareness. Using aggregate awareness, you can leverage preexisting, clean data (perhaps transformed by a tool, like dbt to optimize your queries dynamically and help them run between 10x and 100x faster.

## Queries that scale

DevFeed: [Queries that scale](<https://devfeed.tech/articles/queries-that-scale-81428.md>)

Original publisher: [Read original article](<https://stack.convex.dev/queries-that-scale>)

Author: Stack

Published: 2024-02-13T20:00:52Z

Content type: tutorial

Language: en

Sources: [Stack](<https://devfeed.tech/sources/stack.md>)

Topics: [Rockset](<https://devfeed.tech/topics/rockset.md>), [Transactions](<https://devfeed.tech/topics/transactions.md>), [Skiplists](<https://devfeed.tech/topics/skiplists.md>)

Tags: [batching](<https://devfeed.tech/tags/batching.md>), [caching](<https://devfeed.tech/tags/caching.md>), [database](<https://devfeed.tech/tags/database.md>), [patterns](<https://devfeed.tech/tags/patterns.md>), [queries](<https://devfeed.tech/tags/queries.md>), [scaling](<https://devfeed.tech/tags/scaling.md>)

### AI overview

The article explains how to keep Convex database queries efficient as an app grows. It covers indexing to avoid full table scans, limiting or paginating results, and reducing unnecessary cache invalidation by separating frequently updated data and batching online status updates. It also cautions against optimizing before an app has meaningful user growth.

### Source excerpt

As your app grows from tens to hundreds to thousands of users, there are some techniques that will keep your database queries snappy and efficient. I've put together a short list of the most common techniques used by teams scaling on Convex.

## Under the hood of Omni's intelligent cache

DevFeed: [Under the hood of Omni's intelligent cache](<https://devfeed.tech/articles/under-the-hood-of-omni-s-intelligent-cache-82618.md>)

Original publisher: [Read original article](<https://omni.co/blog/under-the-hood-of-omnis-intelligent-cache>)

Author: Jamie Davidson

Published: 2024-02-01T16:00:00Z

Content type: article

Language: en

Sources: [Omni Blog](<https://devfeed.tech/sources/omni-blog.md>)

Topics: [software-architecture](<https://devfeed.tech/topics/software-architecture.md>), [Rockset](<https://devfeed.tech/topics/rockset.md>), [Cache](<https://devfeed.tech/topics/cache.md>), [self-hosted databases vs managed services](<https://devfeed.tech/topics/self-hosted-databases-vs-managed-services.md>), [Load Balancing](<https://devfeed.tech/topics/load-balancing.md>)

Tags: [allows](<https://devfeed.tech/tags/allows.md>), [cache](<https://devfeed.tech/tags/cache.md>), [cost](<https://devfeed.tech/tags/cost.md>), [data-infrastructure](<https://devfeed.tech/tags/data-infrastructure.md>), [database](<https://devfeed.tech/tags/database.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [intelligent-cache](<https://devfeed.tech/tags/intelligent-cache.md>)

### AI overview

Omni describes how its analytics platform combines in-memory and in-database query execution with an intelligent cache. The cache reuses aggregated results and routes queries to data at an appropriate granularity, aiming to reduce recomputation, network latency, and data warehouse costs. Omni contrasts this approach with BI tools that rely on extracts or direct queries.

### Source excerpt

Omni's unique architecture represents a step function improvement over the query execution of legacy analytical platforms. We believe the real magic comes from optimizing across each level, and we've obsessed over fine-tuning our execution pipeline.