# DuckDB

DuckDB is an open-source analytical SQL database system that runs in-process and can be deployed from edge devices to servers.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Announcing DuckDB 2.0.1 LTS

DevFeed: [Announcing DuckDB 2.0.1 LTS](<https://devfeed.tech/articles/announcing-duckdb-2-0-1-lts-146570.md>)

Original publisher: [Read original article](<https://duckdb.org/2026/11/20/announcing-duckdb-201.html>)

Author: The DuckDB team

Published: 2026-11-20T00:00:00Z

Content type: release

Language: en

Sources: [DuckDB](<https://devfeed.tech/sources/duckdb.md>)

Topics: [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [patches](<https://devfeed.tech/topics/patches.md>), [least privilege](<https://devfeed.tech/topics/least-privilege.md>), [compressing data with Parquet](<https://devfeed.tech/topics/compressing-data-with-parquet.md>), [Linux](<https://devfeed.tech/topics/linux.md>)

Tags: [bugfixes](<https://devfeed.tech/tags/bugfixes.md>), [custom](<https://devfeed.tech/tags/custom.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [duckdb-2-0](<https://devfeed.tech/tags/duckdb-2-0.md>), [extension](<https://devfeed.tech/tags/extension.md>), [improvements](<https://devfeed.tech/tags/improvements.md>), [lts](<https://devfeed.tech/tags/lts.md>), [patch](<https://devfeed.tech/tags/patch.md>), [performance](<https://devfeed.tech/tags/performance.md>), [release](<https://devfeed.tech/tags/release.md>), [repositories](<https://devfeed.tech/tags/repositories.md>), [security](<https://devfeed.tech/tags/security.md>), [security-patches](<https://devfeed.tech/tags/security-patches.md>)

### AI overview

DuckDB 2.0.1 is the first patch release in the 2.0 LTS line, with bug and security fixes, performance improvements, and a new way to register custom extension repositories by hostname. Host-based registration verifies the host's repository pointer over HTTPS, then pins signing keys from the destination repository; it avoids a central registry and lets an organization use its domain while serving extensions elsewhere. The default repository policy now permits host-based repositories, while prefix-based repositories require a broader setting; `forbidden` disables custom repositories. Review host and bucket `SECRET` scopes, which now require exact host or bucket names (and explicit ports), and `MERGE` or `UPDATE ... FROM`, which again update once per source match and may produce non-deterministic final values. Other fixes address 1.x database compatibility, external-access controls, memory handling on affected Linux kernels, and numerous query, storage, and extension issues. The announcement recommends upgrading from 1.4 LTS, whose community support ended November 17, 2026; 2.0 LTS is supported until at least October 2027.

### Source excerpt

Today we are releasing DuckDB 2.0.1, the first patch release of our 2.0 LTS line. This release ships bugfixes, performance improvements, and security patches. In addition, a capability is added to define custom extension repositories with only their hostname.

## Orchestrate DuckDB analytics pipelines with Amazon MWAA

DevFeed: [Orchestrate DuckDB analytics pipelines with Amazon MWAA](<https://devfeed.tech/articles/orchestrate-duckdb-analytics-pipelines-with-amazon-mwaa-141881.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/big-data/orchestrate-duckdb-analytics-pipelines-with-amazon-mwaa/>)

Author: Vinod Jayendra

Published: 2026-11-18T20:57:09Z

Content type: tutorial

Language: en

Sources: [AWS Big Data Blog](<https://devfeed.tech/sources/aws-big-data-blog.md>)

Topics: [Data pipelines](<https://devfeed.tech/topics/data-pipelines.md>), [Analytics Pipeline](<https://devfeed.tech/topics/analytics-pipeline.md>), [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [Amazon S3 Tables](<https://devfeed.tech/topics/amazon-s3-tables.md>), [Amazon S3](<https://devfeed.tech/topics/amazon-s3.md>), [Parquet files in Amazon S3](<https://devfeed.tech/topics/parquet-files-in-amazon-s3.md>), [parquet](<https://devfeed.tech/topics/parquet.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [airflow](<https://devfeed.tech/tags/airflow.md>), [amazon](<https://devfeed.tech/tags/amazon.md>), [amazon-managed-workflows-for-apache-airflow-amazon-mwaa](<https://devfeed.tech/tags/amazon-managed-workflows-for-apache-airflow-amazon-mwaa.md>), [amazon-s3](<https://devfeed.tech/tags/amazon-s3.md>), [amazon-s3-tables](<https://devfeed.tech/tags/amazon-s3-tables.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [apache](<https://devfeed.tech/tags/apache.md>), [apache-iceberg](<https://devfeed.tech/tags/apache-iceberg.md>), [apache-iceberg-tables](<https://devfeed.tech/tags/apache-iceberg-tables.md>), [data](<https://devfeed.tech/tags/data.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [effective](<https://devfeed.tech/tags/effective.md>), [embedded](<https://devfeed.tech/tags/embedded.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [iceberg](<https://devfeed.tech/tags/iceberg.md>), [learn-how-to](<https://devfeed.tech/tags/learn-how-to.md>), [model](<https://devfeed.tech/tags/model.md>), [parquet](<https://devfeed.tech/tags/parquet.md>), [pipelines](<https://devfeed.tech/tags/pipelines.md>), [retry](<https://devfeed.tech/tags/retry.md>), [s3](<https://devfeed.tech/tags/s3.md>), [technical-how-to](<https://devfeed.tech/tags/technical-how-to.md>)

### AI overview

Amazon MWAA can orchestrate bounded SQL analytics by running DuckDB inside Airflow worker tasks, without a separate query cluster. The walkthrough reads S3 Parquet files, deduplicates events, uses ASOF LEFT JOINs to apply rates and ownership effective at each event time, and merges results into three Iceberg tables in Amazon S3 Tables. A second task validates counts, uniqueness, and allocation states. On supported MWAA Provisioned environments running Airflow 3.3.1, DuckDB and its providers plus the duckdb_default AWS-aware connection are preinstalled; the execution role supplies temporary credentials. The example's unchanged-input reruns retained 300,000 unique events and duplicate-free summaries, but the excerpt says this does not establish recovery from partial failures or concurrent external writes. Each task shares one worker's CPU, memory, and storage, so limits and concurrency need tuning with representative data. Use Glue or EMR for steps needing horizontal scaling or more capacity than one worker provides. Three table commits are not one transaction, and Iceberg update/delete support can vary by version.

### Source excerpt

Learn how to orchestrate embedded DuckDB analytics pipelines with Amazon MWAA. This post shows how to run DuckDB inside Airflow tasks to read Parquet data from Amazon S3, apply time-effective model rates, and publish Apache Iceberg tables to Amazon S3 Tables, with retry-safe publishing and worker-sizing considerations.

## DuckDB User Group Workshop on AI-Powered Review Analysis with Hugging Face

DevFeed: [DuckDB User Group Workshop on AI-Powered Review Analysis with Hugging Face](<https://devfeed.tech/articles/duckdb-user-group-workshop-on-ai-powered-review-analysis-with-hugging-face-138566.md>)

Original publisher: [Read original article](<https://duckdb.org/2026/11/17/duckdb-user-group-workshop-ai-powered-review-analysis.html>)

Author: Geertjan Wielenga

Published: 2026-11-17T00:00:00Z

Content type: news

Language: en

Sources: [DuckDB](<https://devfeed.tech/sources/duckdb.md>)

Topics: [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [spaces](<https://devfeed.tech/topics/spaces.md>), [jobs](<https://devfeed.tech/topics/jobs.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [2023](<https://devfeed.tech/tags/2023.md>), [2026](<https://devfeed.tech/tags/2026.md>), [22](<https://devfeed.tech/tags/22.md>), [3](<https://devfeed.tech/tags/3.md>), [about](<https://devfeed.tech/tags/about.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-powered](<https://devfeed.tech/tags/ai-powered.md>), [all](<https://devfeed.tech/tags/all.md>), [amsterdam](<https://devfeed.tech/tags/amsterdam.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [first](<https://devfeed.tech/tags/first.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [november-2026](<https://devfeed.tech/tags/november-2026.md>), [review](<https://devfeed.tech/tags/review.md>), [session](<https://devfeed.tech/tags/session.md>), [user](<https://devfeed.tech/tags/user.md>), [using-duckdb](<https://devfeed.tech/tags/using-duckdb.md>), [with](<https://devfeed.tech/tags/with.md>), [workshop](<https://devfeed.tech/tags/workshop.md>)

### AI overview

The workshop pipeline used DuckDB to identify products with falling ratings in 2.1 million Amazon appliance reviews, then sent only the 40 most helpful recent complaints for the selected product to an LLM for counted themes, quotes, and prioritized safety issues. Attendees developed on a 22 MB sample, then ran the same ingestion files on the full dataset through a Hugging Face Job, storing Parquet output in a Storage Bucket. DuckDB's ranking compared recent and earlier verified-review averages, requiring at least 20 recent and 50 earlier reviews; attendees could adjust those thresholds. Gradio Workflow connected the steps, and deployment to a Hugging Face Space exposed each node as an API endpoint. The workshop required SQL and basic Python, but no ML experience.

### Source excerpt

On 11 November 2026, the DuckDB User Group held its first workshop in Amsterdam, together with Hugging Face. Pratik Patel from Hugging Face led the session.

## DuckDB just shipped its biggest release ever. Its own team says not to run it in production just yet.

DevFeed: [DuckDB just shipped its biggest release ever. Its own team says not to run it in production just yet.](<https://devfeed.tech/articles/duckdb-just-shipped-its-biggest-release-ever-its-own-team-says-not-to-run-it-in-production-just-yet-136128.md>)

Original publisher: [Read original article](<https://thenewstack.io/duckdb-storage-compatibility-agents/>)

Author: Jennifer Riggins

Published: 2026-11-16T18:22:41Z

Content type: news

Language: en

Sources: [The New Stack](<https://devfeed.tech/sources/the-new-stack.md>)

Topics: [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [project release](<https://devfeed.tech/topics/project-release.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>)

Tags: [agent-mode](<https://devfeed.tech/tags/agent-mode.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [ai-coding-agents](<https://devfeed.tech/tags/ai-coding-agents.md>), [amazon](<https://devfeed.tech/tags/amazon.md>), [amazon-quick](<https://devfeed.tech/tags/amazon-quick.md>), [amsterdam](<https://devfeed.tech/tags/amsterdam.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [andy-jassy](<https://devfeed.tech/tags/andy-jassy.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-marketplace](<https://devfeed.tech/tags/aws-marketplace.md>), [aws-storage](<https://devfeed.tech/tags/aws-storage.md>), [bug-fixes](<https://devfeed.tech/tags/bug-fixes.md>), [bugs](<https://devfeed.tech/tags/bugs.md>), [ceo](<https://devfeed.tech/tags/ceo.md>), [data](<https://devfeed.tech/tags/data.md>), [database](<https://devfeed.tech/tags/database.md>), [databases](<https://devfeed.tech/tags/databases.md>), [developers](<https://devfeed.tech/tags/developers.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [podcast](<https://devfeed.tech/tags/podcast.md>), [production](<https://devfeed.tech/tags/production.md>), [release](<https://devfeed.tech/tags/release.md>), [sponsor-amazon-web-services-aws](<https://devfeed.tech/tags/sponsor-amazon-web-services-aws.md>), [sponsor-aws-marketplace](<https://devfeed.tech/tags/sponsor-aws-marketplace.md>), [sponsored](<https://devfeed.tech/tags/sponsored.md>), [sponsored-podcast](<https://devfeed.tech/tags/sponsored-podcast.md>), [video](<https://devfeed.tech/tags/video.md>)

### AI overview

DuckDB 2.0, codenamed Cinnamon Teal, adds server operation, faster queries, an extensible SQL parser enabled by default, and first-class support for the Quack remote protocol. The parser may break things, and DuckLabs' Gábor Szárnyas advises against putting a .0 release into production. The team is working on 2.0.1, focused on bug fixes, small performance tweaks, and user feedback. The release also includes Agent Mode: compact Markdown table output, structured JSON errors, no hidden truncation, and safeguards against runaway queries, designed to help coding agents interpret failures and correct SQL.

### Source excerpt

DuckDB's developers had more than a few ideas for improving how their database stores data, and some of those ideas The post DuckDB just shipped its biggest release ever. Its own team says not to run it in production just yet. appeared first on The New Stack.

## LuthorSQL: A One-Pass Text-to-SQL Model as System 1 for a Slower LLM Agent

DevFeed: [LuthorSQL: A One-Pass Text-to-SQL Model as System 1 for a Slower LLM Agent](<https://devfeed.tech/articles/luthorsql-a-one-pass-text-to-sql-model-as-system-1-for-a-slower-llm-agent-135111.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.17780>)

Author: Ibrahim Memon, Armin Karamzade, M. Hanif Jhaveri, Roy Fox

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Databases](<https://devfeed.tech/sources/arxiv-databases.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [text2sql](<https://devfeed.tech/topics/text2sql.md>), [AI for SQL query generation](<https://devfeed.tech/topics/ai-for-sql-query-generation.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [quantization](<https://devfeed.tech/topics/quantization.md>), [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [SQL](<https://devfeed.tech/topics/sql.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [autoregressive](<https://devfeed.tech/tags/autoregressive.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [bigquery](<https://devfeed.tech/tags/bigquery.md>), [cascade](<https://devfeed.tech/tags/cascade.md>), [claude](<https://devfeed.tech/tags/claude.md>), [claude-opus-5-5](<https://devfeed.tech/tags/claude-opus-5-5.md>), [confidence](<https://devfeed.tech/tags/confidence.md>), [cost](<https://devfeed.tech/tags/cost.md>), [costs](<https://devfeed.tech/tags/costs.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-db](<https://devfeed.tech/tags/cs-db.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [databases](<https://devfeed.tech/tags/databases.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [execution](<https://devfeed.tech/tags/execution.md>), [external](<https://devfeed.tech/tags/external.md>), [llm](<https://devfeed.tech/tags/llm.md>), [model](<https://devfeed.tech/tags/model.md>), [sql](<https://devfeed.tech/tags/sql.md>), [system](<https://devfeed.tech/tags/system.md>), [text](<https://devfeed.tech/tags/text.md>)

### AI overview

LuthorSQL is a local 7B text-to-SQL model designed to handle confident queries in one forward pass and send uncertain ones to a slower LLM agent. It scores query-plan slots under a bounded grammar, runs up to 32 candidate plans, and selects with a trained validator. On BIRD's hidden test set it achieved 68.03% execution accuracy at 0.84 seconds per question on one GPU; 8-bit quantization preserved accuracy in 9 GB, and zero-shot results on five other databases were within 3.1 points of SQLite. In a pre-registered cascade with Claude Opus 5.5 on 227 post-cutoff database questions, it answered nearly half locally and reduced API cost 33.3% at the same accuracy. The abstract also reports substantial verbatim reproduction of public benchmark queries by Opus, while its tool-assisted accuracy held on renamed schemas and reached 95.6-97.4% on post-cutoff databases.

### Source excerpt

arXiv:2611.17780v1 Announce Type: cross Abstract: Frontier LLM agents that query databases with tools are among the most accurate text-to-SQL systems, but per-query calls to a hosted model are slow, costly, and expose sensitive organizational queries to external APIs. For example, a Claude Opus 5.5 agent serving 10,000 queries a day costs over $70,000 a year at list price. To what extent can local hardware already owned by an organization offload this burden without compromising the agent's accuracy? We present LuthorSQL, a non-autoregressive 7B text-to-SQL model that serves as a rapid System 1 gate for a slower LLM agent (System 2), escalating queries only when uncertain. LuthorSQL makes a single forward pass to predict slot scores over a bounded grammar for query plans, compiles and runs up to 32 candidate plans, and chooses an answer using a trained validator. On its own, LuthorSQL reaches 68.03% execution accuracy on BIRD's hidden test set at 0.84 s per question on a single GPU, maintains its accuracy in 9 GB with 8-bit quantized weights, and transfers zero-shot to Snowflake, BigQuery, MySQL, PostgreSQL, and DuckDB within 3.1 points of its native SQLite accuracy. Combined with Claude Opus 5.5 in a pre-registered confidence cascade over 227 questions on databases published after Opus's training cutoff, LuthorSQL answers nearly half of all questions locally and cuts API cost by 33.3% at the same accuracy. Finally, contamination probing shows that Opus reproduces 48-92% of public benchmark gold queries verbatim without seeing the schema; however, its tool-assisted accuracy holds on renamed schemas and reaches 95.6-97.4% on post-cutoff databases. Code and data: https://github.com/IAMIbrahimmemon/luthorsql

## DuckDB has an answer to "bloated" database messages

DevFeed: [DuckDB has an answer to "bloated" database messages](<https://devfeed.tech/articles/duckdb-has-an-answer-to-bloated-database-messages-134453.md>)

Original publisher: [Read original article](<https://thenewstack.io/duckdb-quack-database-protocol/>)

Author: Jennifer Riggins

Published: 2026-11-15T15:05:56Z

Content type: news

Language: en

Sources: [The New Stack](<https://devfeed.tech/sources/the-new-stack.md>)

Topics: [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [Database](<https://devfeed.tech/topics/database.md>), [SQL](<https://devfeed.tech/topics/sql.md>), [Transactions](<https://devfeed.tech/topics/transactions.md>)

Tags: [2-0-0](<https://devfeed.tech/tags/2-0-0.md>), [amsterdam](<https://devfeed.tech/tags/amsterdam.md>), [application](<https://devfeed.tech/tags/application.md>), [argument](<https://devfeed.tech/tags/argument.md>), [authentication](<https://devfeed.tech/tags/authentication.md>), [aws-marketplace](<https://devfeed.tech/tags/aws-marketplace.md>), [backend-development](<https://devfeed.tech/tags/backend-development.md>), [batches](<https://devfeed.tech/tags/batches.md>), [big-data](<https://devfeed.tech/tags/big-data.md>), [cli](<https://devfeed.tech/tags/cli.md>), [client](<https://devfeed.tech/tags/client.md>), [clients](<https://devfeed.tech/tags/clients.md>), [column](<https://devfeed.tech/tags/column.md>), [columnar](<https://devfeed.tech/tags/columnar.md>), [command-line](<https://devfeed.tech/tags/command-line.md>), [commands](<https://devfeed.tech/tags/commands.md>), [data](<https://devfeed.tech/tags/data.md>), [database](<https://devfeed.tech/tags/database.md>), [databases](<https://devfeed.tech/tags/databases.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [messages](<https://devfeed.tech/tags/messages.md>), [podcast](<https://devfeed.tech/tags/podcast.md>), [protocols](<https://devfeed.tech/tags/protocols.md>), [results](<https://devfeed.tech/tags/results.md>), [sponsor-amazon-web-services-aws](<https://devfeed.tech/tags/sponsor-amazon-web-services-aws.md>), [sponsor-aws-marketplace](<https://devfeed.tech/tags/sponsor-aws-marketplace.md>), [sponsored](<https://devfeed.tech/tags/sponsored.md>), [sponsored-podcast](<https://devfeed.tech/tags/sponsored-podcast.md>), [video](<https://devfeed.tech/tags/video.md>)

### AI overview

Quack is DuckDB's HTTP-based client/server protocol, released in DuckDB 2.0.0 LTS, designed to reduce message overhead and stream query results as columnar data. Rather than serializing rows with repeated column details or waiting for a full query to finish, it sends DuckDB data chunks as the engine produces them, so clients can consume results while execution continues. The article says this can keep memory use small. The release adds SQL CONNECT for remote servers, -serve and -connect CLI options, and concurrent client writes using DuckDB's MVCC isolation. Further server-side stability improvements were expected in patch releases.

### Source excerpt

Older database protocols were built before big data. Some still send results row by row, repeating details like column names The post DuckDB has an answer to "bloated" database messages appeared first on The New Stack.

## Local Analytics Doesn't Need a Database Server

DevFeed: [Local Analytics Doesn't Need a Database Server](<https://devfeed.tech/articles/local-analytics-doesn-t-need-a-database-server-129683.md>)

Original publisher: [Read original article](<https://duckdb.org/2026/11/12/local-analytics-without-a-database-server.html>)

Author: Tim Cox

Published: 2026-11-12T00:00:00Z

Content type: article

Language: en

Sources: [DuckDB](<https://devfeed.tech/sources/duckdb.md>)

Topics: [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [Geographic Information System](<https://devfeed.tech/topics/gis.md>), [AI for interactive applications](<https://devfeed.tech/topics/ai-for-interactive-applications.md>), [Database](<https://devfeed.tech/topics/database.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-assistants](<https://devfeed.tech/tags/ai-assistants.md>), [ai-assisted](<https://devfeed.tech/tags/ai-assisted.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [app](<https://devfeed.tech/tags/app.md>), [application](<https://devfeed.tech/tags/application.md>), [based](<https://devfeed.tech/tags/based.md>), [blocks](<https://devfeed.tech/tags/blocks.md>), [browser](<https://devfeed.tech/tags/browser.md>), [case-study](<https://devfeed.tech/tags/case-study.md>), [database](<https://devfeed.tech/tags/database.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [graph](<https://devfeed.tech/tags/graph.md>), [interactive](<https://devfeed.tech/tags/interactive.md>), [local](<https://devfeed.tech/tags/local.md>), [machine](<https://devfeed.tech/tags/machine.md>), [maps](<https://devfeed.tech/tags/maps.md>), [server](<https://devfeed.tech/tags/server.md>), [toolkit](<https://devfeed.tech/tags/toolkit.md>), [user](<https://devfeed.tech/tags/user.md>), [using-duckdb](<https://devfeed.tech/tags/using-duckdb.md>)

### AI overview

DuckDB can run analytics on a user's machine through WebAssembly in a browser or natively in a desktop app, avoiding a remote database server for the described workflows. Flowmap City loads Parquet data from S3 through presigned URLs and runs SQL in the browser as users interact with maps, reducing server round trips. SQLRooms packages reusable React components for data loading, SQL, tables, visualizations, and AI assistants around shared DuckDB connections and workspace state. In Foursquare Spatial Desktop, DuckDB's spatial extension generates map tiles on demand from local data, sampling dense features and simplifying geometry by zoom level; production code also handles invalid geometry, filters small shapes, and caches tiles. Cosmograph uses DuckDB-Wasm in a background worker to query graph data while keeping the interface responsive. SQLRooms' AI assistant generates SQL and chart specifications for local execution, though schemas and selected results may be sent to the AI provider. The described command-line notebook is still evolving.

### Source excerpt

From interactive maps to graph exploration, DuckDB makes it possible to analyze data without a database server. Ilya Boyandin shares how this approach led to SQLRooms, a toolkit for building analytics applications that bring data, visualizations, and AI-assisted analysis together on the user's machine.

## Heavy MySQL Reports in Seconds: Open Source HTAP with DBTrail and Percona Server for MySQL

DevFeed: [Heavy MySQL Reports in Seconds: Open Source HTAP with DBTrail and Percona Server for MySQL](<https://devfeed.tech/articles/heavy-mysql-reports-in-seconds-open-source-htap-with-dbtrail-and-percona-server-for-mysql-127457.md>)

Original publisher: [Read original article](<https://www.percona.com/blog/heavy-mysql-reports-in-seconds-open-source-htap-with-dbtrail-and-percona-server-for-mysql/>)

Author: Vadim Tkachenko

Published: 2026-11-10T21:04:03Z

Content type: article

Language: en

Sources: [Percona](<https://devfeed.tech/sources/blog-percona.md>)

Topics: [Percona Server for MySQL](<https://devfeed.tech/topics/percona-server-for-mysql.md>), [MySQL](<https://devfeed.tech/topics/mysql.md>), [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [MariaDB](<https://devfeed.tech/topics/mariadb.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [apache-2-0](<https://devfeed.tech/tags/apache-2-0.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [column](<https://devfeed.tech/tags/column.md>), [columnar](<https://devfeed.tech/tags/columnar.md>), [docker](<https://devfeed.tech/tags/docker.md>), [docker-compose](<https://devfeed.tech/tags/docker-compose.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [hammerdb](<https://devfeed.tech/tags/hammerdb.md>), [history](<https://devfeed.tech/tags/history.md>), [htap](<https://devfeed.tech/tags/htap.md>), [insight-for-developers](<https://devfeed.tech/tags/insight-for-developers.md>), [mysql](<https://devfeed.tech/tags/mysql.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [percona](<https://devfeed.tech/tags/percona.md>), [percona-server-for-mysql](<https://devfeed.tech/tags/percona-server-for-mysql.md>), [reports](<https://devfeed.tech/tags/reports.md>), [uncategorized](<https://devfeed.tech/tags/uncategorized.md>)

### AI overview

In a DBTrail project benchmark, 12 of 14 analytical reports ran faster on DBTrail's columnar copy than on a Percona Server for MySQL replica; the slowest fell from 225 seconds to 7.94 seconds, while the two lightest reports favored Percona Server. Across 4,436 paired rounds, the copy was 6.5x faster by geometric mean, but it lagged the source by one to five minutes. DBTrail follows the database and uses query plans to route most SELECTs between the server and its copy without requiring rewritten or tagged queries. Its read-only port can be paired with ProxySQL to send writes and transactions to the source. In a separate MariaDB workload test with reports running alongside 400 transactions per second, routing through DBTrail kept p95 latency near 13 ms versus 6,960 ms with reports on the database, and 22x more reports completed. These are project-run results on specific workloads; the excerpt recommends benchmarking one's own reports.

### Source excerpt

In Part 1 we ran analytical reports in DuckDB on a DBTrail copy instead of on MySQL. This time I want to look at a different question: what if the application does not need to know the copy exists? Most MySQL servers have a few queries that make someone wait. The monthly report that aggregates ... Continued The post Heavy MySQL Reports in Seconds: Open Source HTAP with DBTrail and Percona Server for MySQL appeared first on Percona.

## How to Validate LLM Output in a DuckDB and dbt Pipeline

DevFeed: [How to Validate LLM Output in a DuckDB and dbt Pipeline](<https://devfeed.tech/articles/how-to-validate-llm-output-in-a-duckdb-and-dbt-pipeline-127370.md>)

Original publisher: [Read original article](<https://towardsdatascience.com/how-to-validate-llm-output-in-a-duckdb-and-dbt-pipeline/>)

Author: Ibrahim Salami

Published: 2026-11-10T13:30:00Z

Content type: tutorial

Language: en

Sources: [Towards Data Science](<https://devfeed.tech/sources/towards-data-science.md>)

Topics: [Analytics Pipeline](<https://devfeed.tech/topics/analytics-pipeline.md>), [Data Quality](<https://devfeed.tech/topics/data-quality.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [Encoding](<https://devfeed.tech/topics/encoding.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [AI Prompt Templates](<https://devfeed.tech/topics/ai-prompt-templates.md>)

Tags: [dbt](<https://devfeed.tech/tags/dbt.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [gemini-flash-lite](<https://devfeed.tech/tags/gemini-flash-lite.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-evaluation](<https://devfeed.tech/tags/llm-evaluation.md>), [llmops](<https://devfeed.tech/tags/llmops.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [python](<https://devfeed.tech/tags/python.md>), [tests](<https://devfeed.tech/tags/tests.md>)

### AI overview

The pipeline uses Gemini Flash-Lite to extract skills, seniority, experience requirements, Nigeria eligibility, and a role summary from RemoteOK postings, then validates results in Python and dbt. The author added job location after an initial Nigeria eligibility answer lacked key context, and required supporting quotes for yes/no answers; code checks each quote against the posting or location before saving. Python rejects invalid responses, while dbt checks allowed categories, evidence, skill names, encoding problems, and unread-job coverage, with failures for bad data and warnings for suspicious conditions. A strict quote check exposed pre-existing mojibake in one location value; an early repair let that job pass, though the author could not establish where the corruption began. In this 18-job sample, 17 records initially passed, and the final results showed one Nigeria-eligible job and SQL as the most requested skill. The author cautions that the small, analyst-heavy sample does not represent broader data-engineering demand.

### Source excerpt

A Gemini Flash-Lite step pulls skills, seniority, and Nigeria eligibility from 18 RemoteOK postings, with Python checks and dbt tests guarding every answer. The post How to Validate LLM Output in a DuckDB and dbt Pipeline appeared first on Towards Data Science.

## Ducks in a Column

DevFeed: [Ducks in a Column](<https://devfeed.tech/articles/ducks-in-a-column-127523.md>)

Original publisher: [Read original article](<https://duckdb.org/2026/11/10/ducks-in-a-column.html>)

Author: The DuckDB team

Published: 2026-11-10T00:00:00Z

Content type: article

Language: en

Sources: [DuckDB](<https://devfeed.tech/sources/duckdb.md>)

Topics: [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [parquet](<https://devfeed.tech/topics/parquet.md>), [database-optimization](<https://devfeed.tech/topics/database-optimization.md>), [table-partitioning](<https://devfeed.tech/topics/table-partitioning.md>), [CSV](<https://devfeed.tech/topics/csv.md>), [data lake](<https://devfeed.tech/topics/data-lake.md>)

Tags: [aggregation](<https://devfeed.tech/tags/aggregation.md>), [columnar](<https://devfeed.tech/tags/columnar.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [parquet](<https://devfeed.tech/tags/parquet.md>), [querying](<https://devfeed.tech/tags/querying.md>), [sql](<https://devfeed.tech/tags/sql.md>), [using-duckdb](<https://devfeed.tech/tags/using-duckdb.md>), [window-functions](<https://devfeed.tech/tags/window-functions.md>), [writing](<https://devfeed.tech/tags/writing.md>)

### AI overview

DuckDB's columnar storage lets analytical queries read only named columns, and Parquet adds row-group statistics that can skip data excluded by filters. The article demonstrates querying CSV, Parquet, remote files and DataFrames in place; using EXCLUDE, REPLACE and COLUMNS() for column-level SQL; and simplifying analysis with GROUP BY ALL, FILTER, arg_max, PIVOT/UNPIVOT, window functions, QUALIFY and ASOF JOIN. For faster later reads, it recommends partitioning on commonly filtered columns, sorting before writing Parquet, choosing narrow types, and checking EXPLAIN ANALYZE to see which files and columns were scanned. It also notes that approximate aggregates can use less memory than exact counterparts, while giving a small-table example where an approximate distinct count differs from the exact count.

### Source excerpt

DuckDB is fast among other reasons because it stores data by column, so a query reads only the columns it names. This post covers DuckDB features that take advantage of that design choice: querying files in place, column-level SQL, aggregation shortcuts, window functions, ASOF joins, and writing well-organized Parquet.

## WooCommerce Analytics with DuckDB

DevFeed: [WooCommerce Analytics with DuckDB](<https://devfeed.tech/articles/woocommerce-analytics-with-duckdb-119603.md>)

Original publisher: [Read original article](<https://www.percona.com/blog/woocommerce-analytics-with-duckdb/>)

Author: Juan Arruti

Published: 2026-11-05T16:31:19Z

Content type: article

Language: en

Sources: [Percona](<https://devfeed.tech/sources/blog-percona.md>)

Topics: [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [read-replica](<https://devfeed.tech/topics/read-replica.md>), [Percona Server for MySQL](<https://devfeed.tech/topics/percona-server-for-mysql.md>), [data type](<https://devfeed.tech/topics/data-type.md>), [Data pipelines](<https://devfeed.tech/topics/data-pipelines.md>), [Percona](<https://devfeed.tech/topics/percona.md>)

Tags: [analytics](<https://devfeed.tech/tags/analytics.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [columnar](<https://devfeed.tech/tags/columnar.md>), [columnar-databases](<https://devfeed.tech/tags/columnar-databases.md>), [database-trends](<https://devfeed.tech/tags/database-trends.md>), [databases](<https://devfeed.tech/tags/databases.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [innodb](<https://devfeed.tech/tags/innodb.md>), [mysql](<https://devfeed.tech/tags/mysql.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [percona](<https://devfeed.tech/tags/percona.md>), [percona-server-for-mysql](<https://devfeed.tech/tags/percona-server-for-mysql.md>), [storage-engine](<https://devfeed.tech/tags/storage-engine.md>), [woocommerce](<https://devfeed.tech/tags/woocommerce.md>)

### AI overview

Columnar DuckDB replicas can speed WooCommerce analytics while the primary MySQL InnoDB server handles transactions. In a test with 400,000 orders, asynchronous replication to AliSQL's DuckDB engine reduced database size from 7.3 GiB to 601 MiB. Built-in reports ran roughly 10-63 times faster, segmented reports 14-83 times faster, and all eleven reports from an unmodified customer analytics plugin were faster; a query against live order tables took about 4.1 seconds on DuckDB versus 4 minutes 16 seconds on InnoDB. The setup used a second copy of WooCommerce pointed at the replica because WordPress uses one database connection for analytics and store traffic. Results have caveats: some MySQL-specific SQL failed on DuckDB, text grouping can differ because DuckDB ignores column collations, and floating-point decimal handling caused sub-cent differences. DuckDB also exposes less monitoring detail. The article notes that Percona does not offer or support a production DuckDB build.

### Source excerpt

MySQL and DuckDB Unlike row-oriented databases, which store data for a single record sequentially on disk, favoring single-row lookups, columnar databases store all the values from a column contiguously in storage, allowing only the selected column to be read from disk. The default MySQL engine is InnoDB, which is ACID-compliant and great for OLTP (Online ... Continued The post WooCommerce Analytics with DuckDB appeared first on Percona.

## DataGrip 2026.3: Agentic Development With JetBrains Air, Cross-Database Queries, DuckDB Support, and Major UX Improvements

DevFeed: [DataGrip 2026.3: Agentic Development With JetBrains Air, Cross-Database Queries, DuckDB Support, and Major UX Improvements](<https://devfeed.tech/articles/datagrip-2026-3-agentic-development-with-jetbrains-air-cross-database-queries-duckdb-support-and-major-ux-improvements-119340.md>)

Original publisher: [Read original article](<https://blog.jetbrains.com/datagrip/2026/11/05/datagrip-2026-3-agentic-development-with-jetbrains-air-cross-database-queries-duckdb-support-and-major-ux-improvements/>)

Author: Richie Mitish

Published: 2026-11-05T14:43:00Z

Content type: release

Language: en

Sources: [JetBrains](<https://devfeed.tech/sources/the-jetbrains-blog.md>)

Topics: [Tool](<https://devfeed.tech/topics/tool.md>), [SQL](<https://devfeed.tech/topics/sql.md>), [Database](<https://devfeed.tech/topics/database.md>), [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [2026-3](<https://devfeed.tech/tags/2026-3.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-development](<https://devfeed.tech/tags/agentic-development.md>), [datagrip](<https://devfeed.tech/tags/datagrip.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [jetbrains](<https://devfeed.tech/tags/jetbrains.md>), [jetbrains-air](<https://devfeed.tech/tags/jetbrains-air.md>), [queries](<https://devfeed.tech/tags/queries.md>), [ux-improvements](<https://devfeed.tech/tags/ux-improvements.md>)

### AI overview

DataGrip 2026.3 adds JetBrains Air by default, where chats can run SQL against a selected data source, reference database objects, and display sortable, searchable CSV data. A DuckDB-based Workspace data source enables queries that join data across sources and DBMSs without copying it first; DuckDB also gains a dedicated dialect, completion, and introspection. Connecting from an untrusted project now requires trusting it, while safe mode disables connections. Other changes include a Database Output window for query results, data-editor tools for opening results and their original SQL, column pinning and management, and improved transaction-state handling in supported database consoles. Database support includes MariaDB sequences, initial package support, and PostgreSQL 19 recognition. The excerpt also notes PostgreSQL transaction fixes and that automatic savepoints are disabled by default outside the PostgreSQL family.

### Source excerpt

DataGrip 2026.3, the third major update of the year, is here! Let's take a look at what's inside. AI JetBrains Air is now available by default in DataGrip. You can run SQL, explore data, and reference database objects directly in chats with AI agents. SQL snippets use your IDE's syntax highlighting and formatting settings. You [...]

## How JetBrains Built Cross-Database Queries into DataGrip with DuckDB

DevFeed: [How JetBrains Built Cross-Database Queries into DataGrip with DuckDB](<https://devfeed.tech/articles/how-jetbrains-built-cross-database-queries-into-datagrip-with-duckdb-119804.md>)

Original publisher: [Read original article](<https://duckdb.org/2026/11/05/jetbrains-datagrip.html>)

Author: Tim Cox

Published: 2026-11-05T00:00:00Z

Content type: article

Language: en

Sources: [DuckDB](<https://devfeed.tech/sources/duckdb.md>)

Topics: [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [Database](<https://devfeed.tech/topics/database.md>), [jetbrains](<https://devfeed.tech/topics/jetbrains.md>)

Tags: [apache](<https://devfeed.tech/tags/apache.md>), [case-study](<https://devfeed.tech/tags/case-study.md>), [custom](<https://devfeed.tech/tags/custom.md>), [database](<https://devfeed.tech/tags/database.md>), [databases](<https://devfeed.tech/tags/databases.md>), [datagrip](<https://devfeed.tech/tags/datagrip.md>), [developer-tools](<https://devfeed.tech/tags/developer-tools.md>), [developers](<https://devfeed.tech/tags/developers.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [duckdb-2-0](<https://devfeed.tech/tags/duckdb-2-0.md>), [embedded](<https://devfeed.tech/tags/embedded.md>), [functions](<https://devfeed.tech/tags/functions.md>), [h2](<https://devfeed.tech/tags/h2.md>), [ide](<https://devfeed.tech/tags/ide.md>), [integration](<https://devfeed.tech/tags/integration.md>), [jetbrains](<https://devfeed.tech/tags/jetbrains.md>), [join](<https://devfeed.tech/tags/join.md>), [queries](<https://devfeed.tech/tags/queries.md>), [query](<https://devfeed.tech/tags/query.md>), [query-engine](<https://devfeed.tech/tags/query-engine.md>), [using-duckdb](<https://devfeed.tech/tags/using-duckdb.md>)

### AI overview

DataGrip embeds DuckDB to let users join data across database connections in one query, avoiding the need for JetBrains to build and maintain its own cross-database engine. DuckDB's custom table functions connect it to DataGrip's existing query execution; JetBrains says this was simpler than the alternatives it evaluated, while offering more capabilities than lighter options such as H2, including projection pushdown. Cross-source queries use DuckDB's SQL dialect, which DataGrip identifies for users and supports with syntax highlighting. The feature is an initial release: JetBrains plans to observe how users apply it before adding more query optimizations, and is considering using DuckDB to query file-based data sources. The excerpt says the approach works across supported databases without extra infrastructure.

### Source excerpt

JetBrains users had long asked to join data across different database connections in a single query. Rather than build its own cross-database query engine, JetBrains embedded DuckDB in DataGrip, using DuckDB's custom table functions to connect it to the IDE's existing query execution. This is why DuckDB won out over the alternatives, and where the integration goes next.

## Bringing Local Analytics to the Quant's Desktop

DevFeed: [Bringing Local Analytics to the Quant's Desktop](<https://devfeed.tech/articles/bringing-local-analytics-to-the-quant-s-desktop-115975.md>)

Original publisher: [Read original article](<https://duckdb.org/2026/11/03/bringing-local-analytics-to-the-quants-desktop.html>)

Author: Tim Cox

Published: 2026-11-03T00:00:00Z

Content type: article

Language: en

Sources: [DuckDB](<https://devfeed.tech/sources/duckdb.md>)

Topics: [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [streaming data to S3 with Parquet](<https://devfeed.tech/topics/streaming-data-to-s3-with-parquet.md>), [streaming data analytics](<https://devfeed.tech/topics/streaming-data-analytics.md>), [analytics stack](<https://devfeed.tech/topics/analytics-stack.md>), [Analytics Pipeline](<https://devfeed.tech/topics/analytics-pipeline.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-assisted](<https://devfeed.tech/tags/ai-assisted.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [application](<https://devfeed.tech/tags/application.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [banks](<https://devfeed.tech/tags/banks.md>), [cache](<https://devfeed.tech/tags/cache.md>), [case-study](<https://devfeed.tech/tags/case-study.md>), [database](<https://devfeed.tech/tags/database.md>), [desktop](<https://devfeed.tech/tags/desktop.md>), [desktop-application](<https://devfeed.tech/tags/desktop-application.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [embedded](<https://devfeed.tech/tags/embedded.md>), [excel](<https://devfeed.tech/tags/excel.md>), [local](<https://devfeed.tech/tags/local.md>), [pulse](<https://devfeed.tech/tags/pulse.md>), [quant](<https://devfeed.tech/tags/quant.md>), [using-duckdb](<https://devfeed.tech/tags/using-duckdb.md>)

### AI overview

DuckDB lets smaller teams run local analytics without first setting up a server, and it now underpins several of Ryan Hamilton's tools. In QStudio, analysts can save costly remote query results as Parquet and query them locally later, avoiding reruns; the Java integration improved after its Appender was rebuilt. Pulse uses DuckDB to snapshot and cache data, while keeping features such as styled Excel exports in the application UI, partly because bank users may be unable to install extensions. For PeachQ, Hamilton pairs the Rayforce in-memory engine with DuckDB-backed historical storage to build an open-source Q-language system. He describes DuckDB as sufficient for many smaller teams, though it may not scale to every workload.

### Source excerpt

After years building kdb+-based data platforms for banks and hedge funds, Ryan Hamilton wanted an analytical engine he could recommend to smaller teams without specialized infrastructure or expensive licensing. DuckDB gave him a small, embedded way to bring fast local analytics into QStudio and Pulse, and it is now shaping his next open-source project, PeachQ.

## Polars 2.0 vs DuckDB 2.0

DevFeed: [Polars 2.0 vs DuckDB 2.0](<https://devfeed.tech/articles/polars-2-0-vs-duckdb-2-0-112907.md>)

Original publisher: [Read original article](<https://dataengineeringcentral.substack.com/p/polars-20-vs-duckdb-20>)

Author: Daniel Beach

Published: 2026-11-02T13:11:02Z

Content type: comparison

Language: en

Sources: [Data Engineering Central](<https://devfeed.tech/sources/data-engineering-central.md>)

Topics: [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [Apache Iceberg](<https://devfeed.tech/topics/apache-iceberg.md>), [apache-arrow](<https://devfeed.tech/topics/apache-arrow.md>)

Tags: [aggregation](<https://devfeed.tech/tags/aggregation.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [duckdb-2-0](<https://devfeed.tech/tags/duckdb-2-0.md>), [polars](<https://devfeed.tech/tags/polars.md>), [vs](<https://devfeed.tech/tags/vs.md>)

### AI overview

The author's 2 GB Linode test processed about 23 GB of CSV data on S3 with aggregation and window operations. DuckDB 2.0 completed in 837 seconds. Polars 2.0 failed even when the machine was increased to 8 GB; the author attributes this to then-missing spill-to-disk support for group-bys, which the cited release notes said was planned. After Polars 2.1 added group-by and window spill support, it completed the same test on 2 GB in 438 seconds, roughly half DuckDB's reported time. The comparison reflects this specific setup and query. The text also describes Polars 2.0's streaming default, SQL support, and schema checks, and DuckDB 2.0's Quack networking protocol, asynchronous Parquet/CSV I/O, VARIANT type, and triggers.

### Source excerpt

who's been up to what?

## I Benchmarked Databricks Against My iPhone | Blog | FIvetran

DevFeed: [I Benchmarked Databricks Against My iPhone | Blog | FIvetran](<https://devfeed.tech/articles/i-benchmarked-databricks-against-my-iphone-blog-fivetran-136801.md>)

Original publisher: [Read original article](<https://wf.fivetran.com/blog/i-benchmarked-databricks-against-my-iphone>)

Author: George Fraser

Published: 2026-10-30T21:16:27Z

Content type: opinion

Language: en

Sources: [Fivetran Blog](<https://devfeed.tech/sources/fivetran-blog.md>)

Topics: [databricks](<https://devfeed.tech/topics/databricks.md>), [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [iphone](<https://devfeed.tech/topics/iphone.md>), [data lake](<https://devfeed.tech/topics/data-lake.md>), [High-Performance Computing](<https://devfeed.tech/topics/high-performance-computing.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [amazon](<https://devfeed.tech/tags/amazon.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [analytics-workloads](<https://devfeed.tech/tags/analytics-workloads.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [aws](<https://devfeed.tech/tags/aws.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [big-data](<https://devfeed.tech/tags/big-data.md>), [business](<https://devfeed.tech/tags/business.md>), [business-analytics](<https://devfeed.tech/tags/business-analytics.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cloud-compute](<https://devfeed.tech/tags/cloud-compute.md>), [cluster](<https://devfeed.tech/tags/cluster.md>), [clusters](<https://devfeed.tech/tags/clusters.md>), [compute](<https://devfeed.tech/tags/compute.md>), [databricks](<https://devfeed.tech/tags/databricks.md>), [fivetran](<https://devfeed.tech/tags/fivetran.md>), [iphone](<https://devfeed.tech/tags/iphone.md>)

### AI overview

The article reports that DuckDB on an iPhone ran TPC-H faster than Databricks Serverless SQL clusters at 25, 50, and 100 GB and competitively at 200 GB. It attributes the result to query planning and compilation overhead on workloads small for modern CPUs, plus the shuffle costs Databricks incurs for joins and group-bys across nodes; single-node execution can avoid those costs when data fits. The author argues many business analytics workloads could use single-node engines such as DuckDB or Polars, while distributing separate users' queries across workers. The proposed migration is to write ingested data to Iceberg, configure transformations to output Iceberg, then move selected notebook and ad-hoc read workloads to local compute. The article cautions that an iPhone is a stunt and can lose about 30% of performance to thermal throttling.

### Source excerpt

The device in your pocket easily runs most practical analytics workloads.

## What the Heck is Hardcharger?

DevFeed: [What the Heck is Hardcharger?](<https://devfeed.tech/articles/what-the-heck-is-hardcharger-107729.md>)

Original publisher: [Read original article](<https://hackernoon.com/what-the-heck-is-hardcharger?source=rss>)

Author: Shawn Gordon

Published: 2026-10-30T05:48:03Z

Content type: article

Language: en

Sources: [HackerNoon](<https://devfeed.tech/sources/hackernoon.md>)

Topics: [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [apache-arrow](<https://devfeed.tech/topics/apache-arrow.md>), [Streaming](<https://devfeed.tech/topics/streaming.md>), [TLS (Transport Layer Security)](<https://devfeed.tech/topics/tls.md>), [Query (disambiguation)](<https://devfeed.tech/topics/query.md>), [cURL](<https://devfeed.tech/topics/curl.md>), [OpenAPI Specification](<https://devfeed.tech/topics/openapi.md>), [Deno](<https://devfeed.tech/topics/deno.md>), [Documentation](<https://devfeed.tech/topics/documentation.md>)

Tags: [access-control](<https://devfeed.tech/tags/access-control.md>), [admin](<https://devfeed.tech/tags/admin.md>), [administrator](<https://devfeed.tech/tags/administrator.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [apache-2-0](<https://devfeed.tech/tags/apache-2-0.md>), [apache-arrow](<https://devfeed.tech/tags/apache-arrow.md>), [arrow](<https://devfeed.tech/tags/arrow.md>), [audit](<https://devfeed.tech/tags/audit.md>), [chargewire-protocol](<https://devfeed.tech/tags/chargewire-protocol.md>), [cloud-computing](<https://devfeed.tech/tags/cloud-computing.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [duckdb-2-0](<https://devfeed.tech/tags/duckdb-2-0.md>), [gizmosql-successor](<https://devfeed.tech/tags/gizmosql-successor.md>), [hackernoon-top-story](<https://devfeed.tech/tags/hackernoon-top-story.md>), [hardcharger-explained](<https://devfeed.tech/tags/hardcharger-explained.md>), [http](<https://devfeed.tech/tags/http.md>), [license](<https://devfeed.tech/tags/license.md>), [license-keys](<https://devfeed.tech/tags/license-keys.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [protocol](<https://devfeed.tech/tags/protocol.md>), [server](<https://devfeed.tech/tags/server.md>), [sql](<https://devfeed.tech/tags/sql.md>)

### AI overview

Hardcharger is an Apache 2.0 DuckDB 2.0 server that replaces GizmoSQL with a from-scratch implementation and includes features that previously required GizmoSQL Enterprise licenses. Its Chargewire protocol carries Apache Arrow IPC data over ordinary HTTP/1.1 and TLS, with JSON control messages; it supports one-request short queries, asynchronous long queries, resumable downloads, and parallel result partitions. The article contrasts this with Arrow Flight SQL's gRPC transport, which the author says caused proxy and load-balancer problems and requires generated client code. Hardcharger's described security includes identity-bound handles, bearer tokens, OIDC and certificate authentication, plan-based access checks, and blocking access to sensitive host files. Included capabilities also cover granular roles and grants, row policies, masking, audit logs, monitoring, query controls, a web UI, and drivers for several languages. The author reports using its cloud service to share a DuckDB file and query it through an AI/MCP integration with a read-only token. Automatic startup on requests and running data collection apps on the server are described as desired future features.

### Source excerpt

Hardcharger is GizmoData's open-source DuckDB 2.0 server, built on Warp SQL, an HTTP protocol streaming Apache Arrow. Every feature included; no license keys.

## Tessera: Fast Join-Aggregates over Skewed Sparse Relations in Database Engines

DevFeed: [Tessera: Fast Join-Aggregates over Skewed Sparse Relations in Database Engines](<https://devfeed.tech/articles/tessera-fast-join-aggregates-over-skewed-sparse-relations-in-database-engines-108348.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.40620>)

Author: Gyuyeong Kim

Published: 2026-10-30T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Databases](<https://devfeed.tech/sources/arxiv-databases.md>)

Topics: [JOIN](<https://devfeed.tech/topics/join.md>), [database-optimization](<https://devfeed.tech/topics/database-optimization.md>), [DuckDB](<https://devfeed.tech/topics/duckdb.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [addition](<https://devfeed.tech/tags/addition.md>), [aggregate](<https://devfeed.tech/tags/aggregate.md>), [aggregates](<https://devfeed.tech/tags/aggregates.md>), [also](<https://devfeed.tech/tags/also.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [avoid](<https://devfeed.tech/tags/avoid.md>), [based](<https://devfeed.tech/tags/based.md>), [block](<https://devfeed.tech/tags/block.md>), [cells](<https://devfeed.tech/tags/cells.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [cs-db](<https://devfeed.tech/tags/cs-db.md>), [database](<https://devfeed.tech/tags/database.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [either](<https://devfeed.tech/tags/either.md>), [engine](<https://devfeed.tech/tags/engine.md>), [engines](<https://devfeed.tech/tags/engines.md>), [execution](<https://devfeed.tech/tags/execution.md>), [fast](<https://devfeed.tech/tags/fast.md>), [join](<https://devfeed.tech/tags/join.md>), [over](<https://devfeed.tech/tags/over.md>)

### AI overview

Tessera targets join-aggregate queries over skewed, sparse relations, where conventional plans can materialize billions of intermediate rows. It computes sparse matrix products directly from engine rows, emits only nonzero result cells, and prunes values that cannot meet the query threshold before multiplication. For sufficiently dense pairs of frequent values, it offloads that block to a CPU matrix accelerator. The abstract reports Tessera is 2.8x-18.8x faster than the fastest existing system and up to 1,802x faster than DuckDB execution plans; it does not provide further experimental details.

### Source excerpt

arXiv:2610.40620v1 Announce Type: new Abstract: Join-aggregate queries count or sum matching rows for each pair of values, often over skewed sparse relations in which a few values are frequent and most pairs of values never co-occur. Conventional execution plans materialize billions of intermediate join rows, taking minutes to tens of minutes per query. Recent matrix-based solutions reformulate these queries as matrix multiplication to reduce or avoid this materialization. However, they remain slow because they either store result cells for non-matching pairs or still materialize intermediate join rows for infrequent values. We present Tessera, a skew-aware join-aggregate operator for database engines that avoids both sources of overhead. Specifically, Tessera computes a sparse matrix product directly over the engine's rows, emits only nonzero cells as result rows, and prunes values that cannot meet the query's threshold before multiplication begins. It also offloads the dense block formed by pairs of the most frequent values to a CPU matrix accelerator when that block is dense enough. Experimental results show that Tessera is faster than the fastest existing system by 2.8x to 18.8x. In addition, Tessera outperforms DuckDB's execution plans by up to 1,802x.

## Little's Law Says Part of Your Sales Pipeline Is Dead. Here's the SQL to Find It.

DevFeed: [Little's Law Says Part of Your Sales Pipeline Is Dead. Here's the SQL to Find It.](<https://devfeed.tech/articles/little-s-law-says-part-of-your-sales-pipeline-is-dead-here-s-the-sql-to-find-it-106125.md>)

Original publisher: [Read original article](<https://hackernoon.com/littles-law-says-part-of-your-sales-pipeline-is-dead-heres-the-sql-to-find-it?source=rss>)

Author: Malek Damiri

Published: 2026-10-29T08:02:06Z

Content type: article

Language: en

Sources: [HackerNoon](<https://devfeed.tech/sources/hackernoon.md>)

Topics: [Query (disambiguation)](<https://devfeed.tech/topics/query.md>), [SQL](<https://devfeed.tech/topics/sql.md>), [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [Analytics Pipeline](<https://devfeed.tech/topics/analytics-pipeline.md>)

Tags: [data-analytics](<https://devfeed.tech/tags/data-analytics.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [little](<https://devfeed.tech/tags/little.md>), [operations-research](<https://devfeed.tech/tags/operations-research.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [revops](<https://devfeed.tech/tags/revops.md>), [sales](<https://devfeed.tech/tags/sales.md>), [sales-operations](<https://devfeed.tech/tags/sales-operations.md>), [sales-pipeline](<https://devfeed.tech/tags/sales-pipeline.md>), [sql](<https://devfeed.tech/tags/sql.md>), [startups](<https://devfeed.tech/tags/startups.md>), [supply-chain](<https://devfeed.tech/tags/supply-chain.md>)

### AI overview

Little's Law (open deals ~= deals created per week x average weeks to resolution) can flag pipeline buildup that ordinary CRM value counts conceal. In the article's simulated data, a clean CRM has 190 open deals against 183 expected; when 15% of lost deals remain open, it has 510 against 182. The author then uses the 90th percentile of resolved-deal cycle time to group open deals by age: $6.4 million of $11.8 million is more than twice that threshold, versus one deal in the clean CRM's oldest bucket. Suggested steps are to exclude deals older than 2x p90 from coverage unless requalified with a dated next step, report expected versus actual open deals monthly, and track deals resolved per week. The check assumes a reasonably stable system; sudden volume changes add noise, and cycle time from closed deals may understate W if sales cycles are lengthening. Treat it as a diagnostic, then inspect the deals.

### Source excerpt

Malek Damiri applies Little's Law (L = λW) from warehouse operations to a sales pipeline to measure dead deals, with SQL you can run in DuckDB.

## From Spreadsheets to DuckLake: A Data Engineer's Road to DuckDB

DevFeed: [From Spreadsheets to DuckLake: A Data Engineer's Road to DuckDB](<https://devfeed.tech/articles/from-spreadsheets-to-ducklake-a-data-engineer-s-road-to-duckdb-107420.md>)

Original publisher: [Read original article](<https://duckdb.org/2026/10/29/from-spreadsheets-to-ducklake.html>)

Author: Tim Cox

Published: 2026-10-29T00:00:00Z

Content type: article

Language: en

Sources: [DuckDB](<https://devfeed.tech/sources/duckdb.md>)

Topics: [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [SQL](<https://devfeed.tech/topics/sql.md>), [Data Infrastructure](<https://devfeed.tech/topics/data-infrastructure.md>), [big-data](<https://devfeed.tech/topics/big-data.md>), [Data pipelines](<https://devfeed.tech/topics/data-pipelines.md>), [AWS Glue](<https://devfeed.tech/topics/aws-glue.md>), [deltalake](<https://devfeed.tech/topics/deltalake.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>)

Tags: [2016](<https://devfeed.tech/tags/2016.md>), [2022](<https://devfeed.tech/tags/2022.md>), [amazon-s3](<https://devfeed.tech/tags/amazon-s3.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [apache-iceberg](<https://devfeed.tech/tags/apache-iceberg.md>), [apache-spark](<https://devfeed.tech/tags/apache-spark.md>), [asynchronous](<https://devfeed.tech/tags/asynchronous.md>), [automated](<https://devfeed.tech/tags/automated.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-glue](<https://devfeed.tech/tags/aws-glue.md>), [big-data](<https://devfeed.tech/tags/big-data.md>), [case-study](<https://devfeed.tech/tags/case-study.md>), [complexity](<https://devfeed.tech/tags/complexity.md>), [data](<https://devfeed.tech/tags/data.md>), [data-infrastructure](<https://devfeed.tech/tags/data-infrastructure.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [ducklake](<https://devfeed.tech/tags/ducklake.md>), [engineer](<https://devfeed.tech/tags/engineer.md>), [martin](<https://devfeed.tech/tags/martin.md>), [sql](<https://devfeed.tech/tags/sql.md>), [tester](<https://devfeed.tech/tags/tester.md>), [using-duckdb](<https://devfeed.tech/tags/using-duckdb.md>)

### AI overview

Matt Martin's experience suggests many analytical workloads can use DuckDB and DuckLake without the operational overhead he encountered with Spark. DuckDB lets him query local files, S3 data, and combinations of CSV, Parquet, and Iceberg using familiar SQL. For DuckLake, metadata resides in a SQL database; Martin reports catalog operations under a millisecond in his experience, compared with seconds for Iceberg, and says setup on object storage takes two statements. In the v2.0 alpha, he highlights dynamic ATTACH expressions and asynchronous I/O, which cut his S3 file-reading benchmark times by 67%-70% without query changes. He also describes Quack for serving DuckLake remotely. These are his experience and benchmarks; he argues infrastructure should match workload requirements rather than assuming distributed systems are always needed.

### Source excerpt

After nearly two decades working with increasingly sophisticated data infrastructure, data engineer [Matt Martin](https://www.linkedin.com/in/mattmartin14/) discovered DuckDB could handle many of the analytical tasks he cared about without the operational complexity he had come to associate with big data. The same preference for familiar SQL and simpler infrastructure drew him to DuckLake, and he has since become an active tester and advocate of DuckDB's evolving capabilities.

## Bringing the Big Ducks

DevFeed: [Bringing the Big Ducks](<https://devfeed.tech/articles/bringing-the-big-ducks-105858.md>)

Original publisher: [Read original article](<https://motherduck.com/blog/bigger-ducks>)

Author: Jordan Tigani

Published: 2026-10-28T00:00:00Z

Content type: article

Language: en

Sources: [MotherDuck](<https://devfeed.tech/sources/motherduck.md>)

Topics: [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [big-data](<https://devfeed.tech/topics/big-data.md>), [Query (disambiguation)](<https://devfeed.tech/topics/query.md>), [Ground truth / benchmark quality](<https://devfeed.tech/topics/ground-truth-benchmark-quality.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [amazon](<https://devfeed.tech/tags/amazon.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [architectures](<https://devfeed.tech/tags/architectures.md>), [arithmetic](<https://devfeed.tech/tags/arithmetic.md>), [azure](<https://devfeed.tech/tags/azure.md>), [azure-synapse](<https://devfeed.tech/tags/azure-synapse.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [bigquery](<https://devfeed.tech/tags/bigquery.md>), [claude](<https://devfeed.tech/tags/claude.md>), [client](<https://devfeed.tech/tags/client.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [cpus](<https://devfeed.tech/tags/cpus.md>), [data-warehouse](<https://devfeed.tech/tags/data-warehouse.md>), [database](<https://devfeed.tech/tags/database.md>), [databases](<https://devfeed.tech/tags/databases.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [motherduck](<https://devfeed.tech/tags/motherduck.md>)

### AI overview

MotherDuck's results suggest its single-node Giga instance can handle TPC-DS workloads at 1 TB and 10 TB scale while remaining competitive with distributed warehouses. On DuckDB 1.5.6, Giga's geometric mean query time was 690 ms at 1 TB and 5.6 seconds at 10 TB; a smaller Mega instance took 2 seconds at 1 TB. The 10 TB benchmark scans 168 TB across 99 queries. Comparisons use vendor or third-party results published in 2022, so they do not establish current speed rankings. MotherDuck ran unmodified DuckDB TPC-DS queries, reporting the best of three iterations per query and the geometric mean of hot query times. The author says some workloads still struggle to scale up and may need distributed support; DuckDB 2.0 improvements were not included.

### Source excerpt

The pursuit of all things duck-related has led us to some interesting places. In this blog, MotherDuck CEO Jordan Tigani explores the bigger, darker corners of benchmarks on top DuckDB 1.5 (and teases what is possible in DuckDB 2.0).

## dbt without the warehouse (or the bill): DuckDB end to end

DevFeed: [dbt without the warehouse (or the bill): DuckDB end to end](<https://devfeed.tech/articles/dbt-without-the-warehouse-or-the-bill-duckdb-end-to-end-102644.md>)

Original publisher: [Read original article](<https://www.getdbt.com/blog/dbt-duckdb-warehouse-free-data-transformation>)

Author: Daniel Poppy

Published: 2026-10-27T17:36:00Z

Content type: news

Language: en

Sources: [dbt Labs](<https://devfeed.tech/sources/dbt-labs.md>)

Topics: [DuckDB](<https://devfeed.tech/topics/duckdb.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [about](<https://devfeed.tech/tags/about.md>), [adapter](<https://devfeed.tech/tags/adapter.md>), [agent](<https://devfeed.tech/tags/agent.md>), [aggregation](<https://devfeed.tech/tags/aggregation.md>), [all](<https://devfeed.tech/tags/all.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [analytics-engineering](<https://devfeed.tech/tags/analytics-engineering.md>), [bill](<https://devfeed.tech/tags/bill.md>), [built-in](<https://devfeed.tech/tags/built-in.md>), [change](<https://devfeed.tech/tags/change.md>), [dbt](<https://devfeed.tech/tags/dbt.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [learn](<https://devfeed.tech/tags/learn.md>), [offline](<https://devfeed.tech/tags/offline.md>), [profile](<https://devfeed.tech/tags/profile.md>), [small](<https://devfeed.tech/tags/small.md>), [v2](<https://devfeed.tech/tags/v2.md>), [warehouse](<https://devfeed.tech/tags/warehouse.md>), [with](<https://devfeed.tech/tags/with.md>)

### AI overview

dbt v2 includes a built-in DuckDB adapter, and connecting it requires setting the profile output to duckdb and specifying a file path; omitting the path creates a temporary in-memory database. In a live demonstration, Hannes Mühleisen ran dbt fully offline over about 10 GB of compressed Wikipedia page-view CSVs stored locally. A build with window-function calculations across millions of article titles and an aggregation over roughly 300 million rows finished in about three seconds. DuckDB can query CSV, Parquet, and Iceberg files directly without importing them, and includes SQL features such as GROUP BY ALL, UNION BY NAME, and PIVOT. The excerpt presents DuckDB as a single-machine option for many datasets, while previewing DuckDB 2.0 features expected around October 2026.

### Source excerpt

dbt v2 ships with a built-in DuckDB adapter, so connecting dbt to a local, offline warehouse takes only a small profile change.

## Apache Iceberg on Snowflake got a lot simpler in 2026, and dbt's new catalog config is what makes it click

DevFeed: [Apache Iceberg on Snowflake got a lot simpler in 2026, and dbt's new catalog config is what makes it click](<https://devfeed.tech/articles/apache-iceberg-on-snowflake-got-a-lot-simpler-in-2026-and-dbt-s-new-catalog-config-is-what-makes-it-click-102635.md>)

Original publisher: [Read original article](<https://www.getdbt.com/blog/apache-iceberg-snowflake-dbt-catalog-configuration>)

Author: Daniel Poppy

Published: 2026-10-27T17:31:00Z

Content type: news

Language: en

Sources: [dbt Labs](<https://devfeed.tech/sources/dbt-labs.md>)

Topics: [Apache Iceberg](<https://devfeed.tech/topics/apache-iceberg.md>), [DuckDB](<https://devfeed.tech/topics/duckdb.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [agent](<https://devfeed.tech/tags/agent.md>), [ai](<https://devfeed.tech/tags/ai.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [analytics-engineering](<https://devfeed.tech/tags/analytics-engineering.md>), [apache](<https://devfeed.tech/tags/apache.md>), [apache-iceberg](<https://devfeed.tech/tags/apache-iceberg.md>), [aws](<https://devfeed.tech/tags/aws.md>), [catalog](<https://devfeed.tech/tags/catalog.md>), [cli](<https://devfeed.tech/tags/cli.md>), [clustering](<https://devfeed.tech/tags/clustering.md>), [column](<https://devfeed.tech/tags/column.md>), [compatibility](<https://devfeed.tech/tags/compatibility.md>), [compute](<https://devfeed.tech/tags/compute.md>), [config](<https://devfeed.tech/tags/config.md>), [cortex](<https://devfeed.tech/tags/cortex.md>), [dbt](<https://devfeed.tech/tags/dbt.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [iceberg](<https://devfeed.tech/tags/iceberg.md>), [learn](<https://devfeed.tech/tags/learn.md>), [snowflake](<https://devfeed.tech/tags/snowflake.md>)

### AI overview

The described 2026 changes make Snowflake-managed Iceberg tables easier to configure and query outside Snowflake. Snowflake's default Horizon catalog and external writes remove the need, as described, for a separate Polaris account or a tool such as Glue. In dbt, `catalog_name` in `dbt_project.yaml` points to catalog definitions in `catalogs.yml`; dbt write-compatibility options and an upstream DuckDB library change enabled the end-to-end workflow. The demo uses temporary, narrowly scoped vended credentials to write from dbt, then queries the same governed tables directly with DuckDB, avoiding Snowflake warehouse compute while retaining row-level policies and column masking. The speaker says Horizon remains more integrated for teams already on Snowflake; DuckLake requires a separately maintained Postgres catalog. These are claims from the supplied talk excerpt.

### Source excerpt

Changes across Snowflake, dbt, and DuckDB made Apache Iceberg simple enough for a small team to set up in an afternoon.

## Gems of DuckDB 2.0.0

DevFeed: [Gems of DuckDB 2.0.0](<https://devfeed.tech/articles/gems-of-duckdb-2-0-0-101870.md>)

Original publisher: [Read original article](<https://duckdb.org/2026/10/26/gems-of-duckdb-200.html>)

Author: Geertjan Wielenga

Published: 2026-10-26T00:00:00Z

Content type: article

Language: en

Sources: [DuckDB](<https://devfeed.tech/sources/duckdb.md>)

Topics: [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [table-partitioning](<https://devfeed.tech/topics/table-partitioning.md>), [data type](<https://devfeed.tech/topics/data-type.md>), [aio](<https://devfeed.tech/topics/aio.md>)

Tags: [1-4](<https://devfeed.tech/tags/1-4.md>), [1-5](<https://devfeed.tech/tags/1-5.md>), [2-0-0](<https://devfeed.tech/tags/2-0-0.md>), [2018](<https://devfeed.tech/tags/2018.md>), [2019](<https://devfeed.tech/tags/2019.md>), [2020](<https://devfeed.tech/tags/2020.md>), [2021](<https://devfeed.tech/tags/2021.md>), [2023](<https://devfeed.tech/tags/2023.md>), [announcement](<https://devfeed.tech/tags/announcement.md>), [asynchronous](<https://devfeed.tech/tags/asynchronous.md>), [blog](<https://devfeed.tech/tags/blog.md>), [blog-post](<https://devfeed.tech/tags/blog-post.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [features](<https://devfeed.tech/tags/features.md>), [i-o](<https://devfeed.tech/tags/i-o.md>), [improvements](<https://devfeed.tech/tags/improvements.md>), [post](<https://devfeed.tech/tags/post.md>), [protocol](<https://devfeed.tech/tags/protocol.md>), [release](<https://devfeed.tech/tags/release.md>), [support](<https://devfeed.tech/tags/support.md>)

### AI overview

DuckDB 2.0.0 adds practical improvements across its Python client, CLI, SQL, and data export. Python gains an appender that buffers rows and flushes them when its context block exits; for data already in a DataFrame or Arrow table, `INSERT INTO ... SELECT` remains faster. The CLI adds formatted schema output, function help via `.manual`, calculator expressions, and queryable command history. SQL changes include `SECURE VIEW` to hide underlying rows from statistics and query plans, `SHOW` for settings, sequence reset with `setval`, and prepared-statement fallback to session variables. Export features include custom partition paths and ordering, plus GeoJSON and GeoJSONL output. Extensions can define window functions, and extension pages now list version and platform compatibility. The text also notes that `SHOW` for describing tables is deprecated in favor of `DESCRIBE`, and that not every extension is available on every version and platform.

### Source excerpt

We released DuckDB 2.0.0 last week with an improved Quack protocol and `VARIANT` support, asynchronous I/O and many other features. In this blog post, we highlight a few smaller features and improvements that did not make it into the release announcement.

[Next page](<https://devfeed.tech/topics/duckdb.md?cursor=WyIyMDI2LTEwLTI2VDAwOjAwOjAwKzAwOjAwIiwgImI0MjUxYzlkLTZkZmQtNDk1ZC05YjJjLTU1Yjk1NDFjMmFmZiJd>)