# Apache Iceberg Published articles for Apache Iceberg. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Run DuckDB analytics on your Amazon DynamoDB data with zero-ETL DevFeed: [Run DuckDB analytics on your Amazon DynamoDB data with zero-ETL]() Original publisher: [Read original article]() Author: Lee Hannigan Published: 2026-09-11T14:53:57Z Content type: tutorial Language: en Sources: [AWS Database Blog]() Topics: [Databases](), [Amazon S3](), [Apache Iceberg](), [AWS Lambda](), [AWS IAM](), [Latency]() Tags: [advanced-300](), [amazon-dynamodb](), [amazon-s3-tables](), [analytics](), [apache-iceberg](), [aws-lambda](), [duckdb](), [how-to](), [iam](), [integration](), [sql](), [technical-how-to]() ### AI overview This tutorial explains how to run ad hoc SQL analytics on Amazon DynamoDB data with DuckDB through a zero-ETL replication flow. ### Source excerpt Run ad hoc SQL analytics on your Amazon DynamoDB data with DuckDB. A zero-ETL integration replicates your table into Apache Iceberg tables on Amazon S3 Tables, and an AWS Lambda function running DuckDB serves SQL queries through an IAM-authorized function URL. ## Netflix Reworks Conductor for 420 Million Monthly Workflow Executions and 10X Larger Workflows DevFeed: [Netflix Reworks Conductor for 420 Million Monthly Workflow Executions and 10X Larger Workflows]() Original publisher: [Read original article]() Author: Leela Kumili Published: 2026-09-11T14:17:00Z Content type: news Language: en Sources: [InfoQ]() Topics: [Orchestration](), [Amazon S3](), [Kafka](), [Apache Iceberg]() Tags: [amazon-s3](), [apache-iceberg](), [apache-kafka](), [architecture-design](), [asynchronous-architecture](), [cassandra](), [cloud-architecture](), [concurrency](), [development](), [devops](), [distributed-systems](), [elasticsearch](), [java-operator-sdk](), [kafka](), [latency](), [microservices](), [netflix](), [netflix-conductor-4-workflow](), [news](), [orchestration](), [s3](), [scalability](), [windows-workflow-foundation](), [workflow](), [workflow-bpm](), [workflow-foundation]() ### AI overview Netflix reworked Conductor 4.0 to scale workflow orchestration to roughly 200,000 definitions and 420 million monthly executions. The redesign raises supported workflow size to 30,000 tasks and reports a roughly 40% reduction in p99 evaluation latency by loading only task data needed for each decision. ### Source excerpt Netflix has reworked its Conductor workflow orchestration engine to handle larger workloads, increasing supported workflow size from about 2,500 to 30,000 tasks and reducing p99 workflow evaluation latency by about 40%. Conductor 4.0 separates workflow metadata from task data, moves evaluation to asynchronous processing, and introduces dynamic worker allocation and concurrency controls. By Leela Kumili ## AWS Glue 6.0 now available with 30% lower price and full Apache Iceberg v3 support DevFeed: [AWS Glue 6.0 now available with 30% lower price and full Apache Iceberg v3 support]() Original publisher: [Read original article]() Author: Channy Yun (윤석찬) Published: 2026-08-21T18:53:26Z Content type: release Language: en Sources: [AWS News Blog]() Topics: [Apache Iceberg](), [Amazon Web Services](), [Apache Spark](), [Python](), [Scala](), [Geographic Information System](), [Internet of things]() Tags: [analytics](), [apache](), [apache-iceberg](), [aws](), [aws-glue](), [launch](), [news](), [performance](), [pricing](), [python-3-13](), [real-time](), [release](), [scala](), [spark](), [streaming]() ### AI overview AWS Glue 6.0 is generally available with 30% lower pricing, a modernized Spark 4.1 runtime, Python 3.13 and Scala 2.13 support, and full Apache Iceberg v3 support. The release adds improved handling of semi-structured data, declarative ETL pipelines, faster PySpark execution, and real-time streaming with single-digit millisecond latency. ### Source excerpt AWS Glue 6.0 is built on a fully modernized runtime, Apache Spark 4.1, Python 3.13, and Scala 2.13, delivering 30% lower pricing than previous AWS Glue versions. ## How Jump Trading uses ClickHouse with Iceberg for analytics DevFeed: [How Jump Trading uses ClickHouse with Iceberg for analytics]() Original publisher: [Read original article]() Author: ClickHouse Published: 2026-07-30T00:00:00Z Content type: article Language: en Sources: [ClickHouse Blog]() Topics: [clickhouse](), [Apache Iceberg](), [log management](), [data](), [Latency](), [Open Source](), [real-time](), [Database]() Tags: [analytics](), [apache](), [apache-iceberg](), [business](), [clickhouse](), [data](), [database](), [industry](), [latency](), [logging](), [logs](), [low-latency](), [observability](), [open](), [open-source](), [performance](), [real-time](), [scale](), [technology](), [trading]() ### AI overview Jump Trading uses a self-managed ClickHouse platform to capture and analyze petabyte-scale financial trading logs. The platform ingests hundreds of terabytes daily with a sub-20-second p99 and supports real-time analytics across hundreds of billions of events. To support large-scale batch reporting and research without affecting the real-time cluster, Jump added a parallel Apache Iceberg pipeline. ### Source excerpt Jump Trading captures petabyte-scale financial trading logs on a self-managed ClickHouse platform, where zero data loss and low latency are critical requirements. ## How We Refresh Razorpay's Data Warehouse 10x Faster with Graphs and Indexes DevFeed: [How We Refresh Razorpay's Data Warehouse 10x Faster with Graphs and Indexes]() Original publisher: [Read original article]() Author: Amit Prabhu Published: 2026-07-14T14:06:16Z Content type: article Language: en Sources: [Razorpay Engineering - Medium]() Topics: [data](), [airflow](), [Apache Spark](), [Amazon S3](), [Apache Iceberg](), [microservices architecture](), [parquet](), [Architecture & Design]() Tags: [airflow](), [analytics](), [apache-iceberg](), [architecture](), [batch](), [data](), [graphs](), [infrastructure](), [parquet](), [pipelines](), [razorpay](), [spark](), [trino](), [warehouse]() ### AI overview Razorpay describes its data warehouse refresh pipeline, which builds wide denormalized Facts by joining data from multiple microservices. The article covers the original Airflow- and Spark-based full-refresh process, the underlying lake formats and query layer, and the scaling challenges that led the team to reconsider refresh strategy, data layout, and high-cardinality dimensions. ### Source excerpt Contributors: Utkarsh Koppikar Rohan Background Razorpay provides the payment infrastructure for millions of merchants globally. Behind every payment, settlement, and refund is a microservices architecture where each service owns its own database. While this keeps services independent and scalable, it creates a challenge for stakeholders who need to see across those boundaries. The Data Platform team manages the infrastructure that bridges this gap. Transactional data flows into the lake via CDC pipelines, ingested onto S3 in Delta Lake, Apache Iceberg, or plain Parquet formats. On top of the lake, we build domain-specific warehouse tables -- wide, pre-joined tables that co-locate all the data a consumer needs, queryable via Trino. These power two use cases: Analytics (internal dashboards on Tableau and Superset) and Reporting (merchants and regulated entities who download structured data exports; Razorpay generates nearly a million such reports per month). The warehouse tables that power both use cases are called Facts. A Fact is a flat denormalised table on S3, produced by joining 10 to 30 microservice tables and materialising the result once. A settlement Fact, for example, merges payments, refunds, adjustments, and card details into a single wide row so that a dashboard or report reads from a single table instead of joining across services in real time. It is closer to a domain-specific materialised view than a classical data warehouse fact table. We maintain over 50 such Facts, and approximately 40% of all merchant reports are served directly from them. As data volumes and the number of entities per fact grew, the batch generation pipeline began to show its limits, prompting us to rethink the refresh strategy, the data layout, and how to handle high-cardinality dimensions. The rest of this post covers that journey. The Full Refresh Pipeline: Our Baseline and the Pain The original full-refresh pipeline was straightforward. Schedule: Airflow schedules Spark jobs o ## Evaluating Cloudflare as a Data Platform DevFeed: [Evaluating Cloudflare as a Data Platform]() Original publisher: [Read original article]() Author: Daniel Beach Published: 2026-07-13T12:39:09Z Content type: article Language: en Sources: [Data Engineering Central]() Topics: [Cloudflare](), [data](), [Apache Iceberg]() Tags: [apache-iceberg](), [cloudflare](), [data](), [data-platform]() ### AI overview The article examines Cloudflare's data products and whether they can support a complete data stack, including how to build with them and what they cost. The supplied excerpt establishes the article's scope but does not provide its final findings. ### Source excerpt new kid on the block ## Tableflow: Turn Kafka Topics into Iceberg Tables DevFeed: [Tableflow: Turn Kafka Topics into Iceberg Tables]() Original publisher: [Read original article]() Author: Mohtasham Sayeed Mohiuddin Published: 2026-07-10T15:36:14Z Content type: tutorial Language: en Sources: [Confluent: Data in motion]() Topics: [Apache Iceberg](), [Kafka](), [schema-evolution](), [Confluent Cloud](), [parquet](), [Amazon S3](), [Streaming](), [AWS Glue](), [Amazon Redshift](), [BigQuery]() Tags: [amazon-redshift](), [amazon-s3](), [apache-iceberg](), [aws-glue](), [bigquery](), [confluent-cloud](), [kafka](), [parquet](), [schema-evolution](), [streaming](), [technologies](), [tutorial]() ### AI overview This tutorial explains how Confluent Cloud Tableflow continuously materializes Apache Kafka topics as Apache Iceberg or Delta Lake tables. It covers automatic schema handling, type conversion, schema evolution, Parquet conversion, catalog publishing, and table maintenance for querying streaming data with analytics engines and warehouses. ### Source excerpt Learn how Confluent Tableflow turns Kafka topics into Iceberg tables for zero-ETL analytics with automatic schema evolution and open catalog access. ## Scaling Grab's Data Lake: Our journey to Apache Iceberg adoption DevFeed: [Scaling Grab's Data Lake: Our journey to Apache Iceberg adoption]() Original publisher: [Read original article]() Author: Rahul Penti Published: 2026-07-10T00:00:00Z Content type: article Language: en Sources: [Grab Tech]() Topics: [Apache Iceberg](), [data](), [Amazon S3](), [parquet](), [Apache Spark](), [Latency](), [Concurrency]() Tags: [apache-iceberg](), [compute](), [concurrency](), [data](), [database](), [engineering](), [latency](), [parquet](), [performance](), [s3](), [scale](), [spark]() ### AI overview Grab describes its transition from Hive Parquet and a directory-based Hive Metastore architecture toward Apache Iceberg and a table-centric lakehouse. The article outlines scalability problems involving metadata latency, small files, S3 request overhead, and operational management, and mentions a unified Spark catalog built to hide table-format differences. ### Source excerpt Introduction: The evolution of Grab's Data Lake At Grab's scale, managing petabytes of data across billions of S3 objects demands more than a storage layer. It demands a robust architectural primitive that supports the high-concurrency needs of a modern "Lakehouse." Our goal is full storage-compute separation, leveraging S3 as an elastic foundation for both near-real-time metrics and large-scale batch transformations. For years, the vast majority of our tables were Hive Parquet, managed through the Hive Metastore with a directory-based layout. This model served us well, but as data volume grew, the directory-and-metastore approach became the limiting factor. We are now transitioning to a table-centric architecture built on modern table formats, treating data as a first-class primitive to ensure consistency and performance across our internal data transformation platforms: Slide, which powers batch transformations, and Hugo, which handles online-to-data-lake ingestion. Along the way, we also built the UnifiedSparkCatalog, a unified Spark catalog that hides table-format differences from users entirely, which we are open-sourcing alongside this post. The catalyst for change: Challenges with Hive Parquet For years, Hive Parquet was the backbone of our Data Lake, representing the vast majority of our tables. However, as data volume scaled, the architectural limitations of directory-based storage became apparent. We identified four primary bottlenecks: Catalog latency: The Hive Metastore (HMS) became a centralized failure point. High concurrency during metadata access led to O(n) listing overhead, where query planning time scaled linearly with partition count, crippling throughput. The small file problem: The directory layout left us with severe file fragmentation. Certain Machine Learning (ML) datasets had an average file size under 1 MB, with thousands of files in each partition. At this scale, the overhead of S3 object listing and metadata request latency drove up Appl ## Announcing Apache Iceberg 1.11.0 DevFeed: [Announcing Apache Iceberg 1.11.0]() Original publisher: [Read original article]() Author: Google Open Source (noreply@blogger.com) Published: 2026-05-27T18:30:00Z Content type: release Language: en Sources: [Google Open Source Blog]() Topics: [Apache Iceberg](), [Apache Spark](), [apache-flink](), [schema-evolution](), [Streaming](), [Encryption](), [Cryptography]() Tags: [apache-flink](), [apache-iceberg](), [apache-spark](), [encryption](), [kms](), [lakehouse](), [releases](), [schema-evolution](), [streaming]() ### AI overview Apache Iceberg 1.11.0 adds support for Apache Spark 4.1 and Apache Flink 2.1, makes both default build targets, and introduces changes including server-side REST catalog scan planning, partition statistics APIs, built-in envelope encryption with Google KMS support, and Google Storage Analytics integration. ### Source excerpt by Alex Stephen & Talat Uyarer, Lakehouse Apache Iceberg project has just launched version 1.11.0! A lot has happened since the last version. Iceberg 1.11.0 adds support for Apache Spark 4.1 and Apache Flink 2.1, the latest releases of the two engines and makes both the default build targets The rest are more structural. The REST catalog learns to plan scans server-side, shifting metadata work off the query engine. A new partition statistics scan API gives optimizers a clean, supported way to read a table's shape. Built-in table encryption arrives with envelope encryption and Google KMS support. And Google Storage Analytics library integration makes your Iceberg workloads faster than before. Let's take a look at some of the biggest changes. Spark & Flink Updates As Spark and Flink are moving forward, the 1.11.0 release is pushing forward for new version support in both. Spark 4.1 & DSv2 Migration: Spark 4.1 unlocks is MERGE INTO with automatic schema evolution: Spark's newer MERGE syntax accepts a WITH SCHEMA EVOLUTION clause, so a MERGE whose source carries columns the target table lacks can add those columns to the table within the same statement, with no separate ALTER TABLE round trip. Beyond the version bump, the 1.11 Spark connector also modernizes against Spark's newer DataSource V2 APIs and adds an asynchronous micro-batch planner that speeds up Structured Streaming. Flink Ecosystem Updates: Initial work for Flink 2.1 support has landed in the core repository, continuing Iceberg's promise of providing first-class, low-latency streaming sink capabilities. The centerpiece of the Flink work is the DynamicIcebergSink, an experimental sink that breaks the old one-sink-per-table model: a single sink routes each record to a table chosen at runtime, creating tables on demand and evolving their schemas and partition specs on the fly as the input changes including dropping columns once you opt in with dropUnusedColumns. In addition to DynamicIcebergSInk work Flink sta ## How Partition Access Visualizations Reduced our Data Lake S3 Cost by 33% DevFeed: [How Partition Access Visualizations Reduced our Data Lake S3 Cost by 33%]() Original publisher: [Read original article]() Author: Nick Del Nano, Data Streaming Published: 2026-05-21T00:00:00Z Content type: article Language: en Sources: [Yelp]() Topics: [data](), [data lake](), [Apache Iceberg](), [Amazon S3](), [AWS IAM]() Tags: [analytics](), [apache-iceberg](), [aws](), [data](), [data-lake](), [iam](), [partition](), [s3]() ### AI overview Yelp describes visualizations that map partition keys against access-event timestamps to reveal daily batch jobs, backfills, and ad hoc queries. The resulting usage attribution supported Apache Iceberg migration and storage-efficiency work that reduced the cost of its petabyte-scale data lake by 33%. ### Source excerpt Introduction In large analytics environments, data teams often struggle to answer deceptively simple questions, like who their stakeholders are and how their data is being used. At Yelp, we address this by visualizing access patterns, plotting time-based partition key values against access event timestamps. These visualizations reveal distinct usage signatures - ad hoc queries, daily batch jobs, and periodic backfills - allowing data owners to understand their stakeholders and use cases. This deeper insight into data usage has enabled high-impact platform initiatives including migrating thousands of tables to Apache Iceberg format and identifying storage efficiencies which reduced the cost of... ## Powering self-driving vehicle analytics at Avride with ClickHouse Cloud DevFeed: [Powering self-driving vehicle analytics at Avride with ClickHouse Cloud]() Original publisher: [Read original article]() Author: ClickHouse Published: 2026-05-11T00:00:00Z Content type: article Language: en Sources: [ClickHouse Blog]() Topics: [data](), [Apache Iceberg](), [telemetry](), [Grafana]() Tags: [analytics](), [apache-iceberg](), [autonomous](), [aws](), [clickhouse](), [cloud](), [data-infrastructure](), [driving](), [grafana](), [latency](), [lidar](), [metrics](), [robots](), [storage](), [streams](), [telemetry](), [tooling](), [visualization]() ### AI overview Avride uses ClickHouse Cloud as the data backbone for its autonomous vehicles and delivery robots, supporting ride-data indexing, metrics, analytics, and internal tooling. Its migration from Apache Iceberg reduced index lookup and ingestion latency. ### Source excerpt Avride replaced Apache Iceberg with ClickHouse Cloud, cutting index lookup latency from 20 seconds to under 100ms and ingestion from hours to seconds. ## ClickHouse integrates with Lakehouse Runtime Catalog DevFeed: [ClickHouse integrates with Lakehouse Runtime Catalog]() Original publisher: [Read original article]() Author: Melvyn Peignon Published: 2026-04-22T13:27:18Z Content type: release Language: en Sources: [ClickHouse Blog]() Topics: [clickhouse](), [Apache Iceberg](), [Google](), [BigQuery](), [Apache Spark](), [Cloud](), [API](), [data](), [Authentication]() Tags: [analytics](), [apache-iceberg](), [api](), [authentication](), [bigquery](), [clickhouse](), [cloud](), [data](), [google](), [google-cloud](), [integration](), [spark]() ### AI overview ClickHouse announces a beta integration with Google's Lakehouse Runtime Catalog, enabling direct querying of Google Cloud Lakehouse Apache Iceberg tables through the Iceberg REST Catalog. The integration supports ClickHouse 26.2 and later, ClickHouse Cloud, Google Application Default Credentials, and querying or loading data without data movement or metadata synchronization. ### Source excerpt We're excited to announce a new integration between ClickHouse and Google's Lakehouse Runtime CatalogCatalog ## ClickHouse is data lake ready DevFeed: [ClickHouse is data lake ready]() Original publisher: [Read original article]() Author: Karolina Ruiz Rogelj; Melvyn Peignon Published: 2026-03-25T00:00:00Z Content type: article Language: en Sources: [ClickHouse Blog]() Topics: [clickhouse](), [Apache Iceberg](), [parquet](), [Data Space](), [real-time](), [Cloud](), [Amazon S3](), [Azure]() Tags: [analytics](), [apache-iceberg](), [azure](), [caching](), [clickhouse](), [cloud](), [concurrency](), [data](), [interoperability](), [open-source](), [parquet](), [performance](), [real-time](), [s3](), [scale](), [storage]() ### AI overview ClickHouse is presented as data lake ready, with direct querying of Iceberg and Delta Lake data, support for cloud catalogs and Parquet optimizations, and the option to load data into native storage for faster high-concurrency analytics while writing results back to open formats. ### Source excerpt ClickHouse now supports direct querying of Iceberg and Delta Lake formats across major cloud catalogs without requiring data migration. ## Introducing Iceberg output for Redpanda Connect DevFeed: [Introducing Iceberg output for Redpanda Connect]() Original publisher: [Read original article]() Author: Prakhar Garg Published: 2026-03-05T00:00:00Z Content type: release Language: en Sources: [Redpanda]() Topics: [Apache Iceberg](), [Redpanda-Connect](), [Streaming](), [Routing (disambiguation)](), [schema-evolution](), [YAML](), [data]() Tags: [apache-iceberg](), [connectors](), [integration](), [pii](), [redpanda-connect](), [routing](), [schema-evolution](), [streaming](), [yaml]() ### AI overview Redpanda introduces an Iceberg output for Redpanda Connect that writes streaming data directly to Apache Iceberg tables through declarative YAML pipelines. The component supports transformations, enrichment, PII removal, and content-based routing across Redpanda Connect inputs and processors, enabling data from sources such as HTTP webhooks, Postgres CDC streams, and GCP Pub/Sub to reach lakehouse tables with less infrastructure and configuration overhead. ### Source excerpt Land any data into Apache Iceberg with Redpanda Connect. Maximize infra density, automate schema evolution, and scale 1-to-many routing without the toil. ## Data-to-Production: Bridging the Gap Between Iceberg and Live Microservices DevFeed: [Data-to-Production: Bridging the Gap Between Iceberg and Live Microservices]() Original publisher: [Read original article]() Author: Wix Engineering Published: 2026-02-17T11:04:08Z Content type: article Language: en Sources: [Wix Engineering]() Topics: [Apache Iceberg](), [data observability](), [Microservice](), [clickhouse](), [Amazon S3](), [Architecture & Design](), [Apache Spark](), [airflow](), [Back end](), [API]() Tags: [airflow](), [amazon-s3](), [apache-iceberg](), [api](), [architecture](), [clickhouse](), [data](), [data-engineering](), [json](), [microservices](), [spark]() ### AI overview Wix describes Data-to-Production, a platform that activates data from Amazon S3 and Apache Iceberg for backend microservices. The system ingests Iceberg data into ClickHouse and serves it through a type-safe JSON API, using metadata governance and an Airflow and Python ingestion engine. ### Source excerpt At Wix, our Data Warehouse (DWH) is a massive repository of insights. Built on Amazon S3 using Apache Iceberg table formats, and populated by Trino and Spark jobs, it houses petabytes of data--from user segmentation and logs to AI chat analytics. However, storage is only half the battle. The real challenge--and the "holy grail" for many data engineering teams--is Activation : taking that petabyte-scale data and exposing it to backend microservices with millisecond latency, high availability, and... ## Build a real-time lakehouse architecture with Redpanda and Databricks DevFeed: [Build a real-time lakehouse architecture with Redpanda and Databricks]() Original publisher: [Read original article]() Author: Peter Henn Published: 2026-01-06T00:00:00Z Content type: article Language: en Sources: [Redpanda]() Topics: [Apache Iceberg](), [real-time](), [databricks](), [Streaming](), [data](), [Cloud]() Tags: [analytics](), [apache-iceberg](), [cloud-native](), [databricks](), [governance](), [platforms](), [real-time](), [real-time-lakehouse-architecture]() ### AI overview This article explains how Redpanda and Databricks can support a real-time lakehouse architecture in which streaming data flows directly into governed, analytics-ready tables. It presents Apache Iceberg as the open, cloud-native table foundation that combines data-lake flexibility with warehouse-style governance and reliability, while describing how this approach reduces batch processing and operational complexity. ### Source excerpt Learn how Redpanda's Iceberg Topics and Databricks Unity Catalog enable real-time, analytics-ready tables without batch, orchestration, or babysitting. ## Introducing iceberg-js: A JavaScript Client for Apache Iceberg DevFeed: [Introducing iceberg-js: A JavaScript Client for Apache Iceberg]() Original publisher: [Read original article]() Author: Katerina Skroumpelou Published: 2025-12-08T07:00:00Z Content type: article Language: en Sources: [Supabase Blog]() Topics: [Apache Iceberg](), [JavaScript](), [REST API](), [TypeScript](), [API](), [Open Source]() Tags: [apache-iceberg](), [api](), [fetch](), [javascript](), [libraries](), [open-source](), [parquet](), [rest](), [typescript]() ### AI overview Supabase introduces iceberg-js, a minimal, vendor-agnostic JavaScript client for the Apache Iceberg REST Catalog API. It gives JavaScript and TypeScript developers a type-safe way to manage Iceberg namespaces and tables across catalog implementations. ### Source excerpt A minimal, vendor-agnostic JavaScript client for the Apache Iceberg REST Catalog API, providing type-safe catalog management for JavaScript and TypeScript developers. ## Introducing Analytics Buckets DevFeed: [Introducing Analytics Buckets]() Original publisher: [Read original article]() Author: Fabrizio Fenoglio Published: 2025-12-02T07:00:00Z Content type: release Language: en Sources: [Supabase Blog]() Topics: [Apache Iceberg](), [Amazon S3]() Tags: [amazon-s3](), [analytics](), [apache-iceberg](), [parquet](), [postgres](), [s3](), [scale](), [schema-evolution](), [storage](), [time-series]() ### AI overview Supabase introduces Analytics Buckets for storing large analytical datasets in Supabase Storage. The service uses Apache Iceberg, Amazon S3, and columnar Parquet files, while Postgres remains suited to transactional application data. ### Source excerpt Use Analytics Buckets to store huge datasets in Supabase Storage with Apache Iceberg and columnar Parquet format, optimized for analytical workloads. ## ClickHouse Announces Integration with Microsoft OneLake for Federated Analytics DevFeed: [ClickHouse Announces Integration with Microsoft OneLake for Federated Analytics]() Original publisher: [Read original article]() Author: Melvyn Peignon Published: 2025-11-18T00:00:00Z Content type: release Language: en Sources: [ClickHouse Blog]() Topics: [clickhouse](), [interoperability](), [Microsoft](), [data lake](), [real-time](), [Apache Iceberg]() Tags: [apache-iceberg](), [clickhouse](), [data-lake](), [databases](), [fabric](), [integration](), [interoperability](), [latency](), [microsoft](), [real-time]() ### AI overview ClickHouse announced an integration with Microsoft OneLake, the unified data lake service within Microsoft Fabric. The integration exposes OneLake Iceberg tables to ClickHouse so users can query and analyze data across ClickHouse and Fabric, supporting real-time analytical workloads. ### Source excerpt ClickHouse today announced the availability of a powerful new integration with Microsoft OneLake, the unified data lake service within Microsoft Fabric. ## ClickHouse integrates with Microsoft OneLake Table APIs DevFeed: [ClickHouse integrates with Microsoft OneLake Table APIs]() Original publisher: [Read original article]() Author: Melvyn Peignon Published: 2025-11-18T00:00:00Z Content type: article Language: en Sources: [ClickHouse Blog]() Topics: [clickhouse](), [Apache Iceberg](), [Microsoft](), [interoperability](), [data](), [AI Platform]() Tags: [analytics](), [apache](), [apache-iceberg](), [apis](), [clickhouse](), [data](), [genai](), [integration](), [interoperability](), [microsoft]() ### AI overview ClickHouse announces a beta integration with Microsoft OneLake through the OneLake Tables APIs and Apache Iceberg. The integration will enable direct querying of Iceberg tables in OneLake, supporting unified analytics and AI use cases across data stored in different systems. ### Source excerpt We're excited to announce a new integration between ClickHouse and Microsoft OneLake, powered by the OneLake Tables APIs and Apache Iceberg. ## Are open-table-formats + lakehouses the future of observability? DevFeed: [Are open-table-formats + lakehouses the future of observability?]() Original publisher: [Read original article]() Author: Melvyn Peignon; Dale McDiarmid Published: 2025-10-16T00:00:00Z Content type: article Language: en Sources: [ClickHouse Blog]() Topics: [telemetry](), [Apache Iceberg]() Tags: [apache-iceberg](), [compression](), [cost](), [logs](), [metrics](), [observability](), [open](), [parquet](), [scale](), [telemetry](), [traces]() ### AI overview The article assesses whether lakehouses built on open table formats can support observability workloads at scale. It outlines benefits such as low-cost object storage, database-like semantics, compression, and flexible schemas, while examining remaining challenges and emerging improvements. ### Source excerpt Can open table formats like Iceberg and Delta Lake really power observability at scale, and if not, what is missing today and which innovations could unlock low cost open observability? ## 3 demos to get started with Redpanda on Apache Iceberg™ DevFeed: [3 demos to get started with Redpanda on Apache Iceberg™]() Original publisher: [Read original article]() Author: Paul Wilkinson Published: 2025-10-07T00:00:00Z Content type: article Language: en Sources: [Redpanda]() Topics: [Apache Iceberg](), [Apache Iceberg integration](), [Kafka](), [Kubernetes](), [DuckDB](), [Streaming](), [Development]() Tags: [apache-iceberg](), [apache-iceberg-integration](), [config](), [development](), [duckdb](), [integration](), [kafka](), [kubernetes](), [no-code]() ### AI overview This article presents three demos for developing with Redpanda's Iceberg Topics in a local Kubernetes environment. The project demonstrates streaming data from Redpanda topics into Apache Iceberg tables without external cloud services or ETL, using Polaris, MinIO, and DuckDB for the end-to-end flow. ### Source excerpt Apache Iceberg integration made simple. Get started with three no-fluff demos using Redpanda's Iceberg Topics. One config. Endless scale. ## ClickHouse Release 25.7 DevFeed: [ClickHouse Release 25.7]() Original publisher: [Read original article]() Author: ClickHouse Published: 2025-08-08T00:00:00Z Content type: release Language: en Sources: [ClickHouse Blog]() Topics: [clickhouse](), [SQL](), [Apache Iceberg](), [Artificial Intelligence]() Tags: [ai](), [apache-iceberg](), [bug-fixes](), [clickhouse](), [community](), [contributors](), [features](), [performance](), [release](), [sql](), [updates]() ### AI overview ClickHouse 25.7 is a monthly release featuring lightweight standard SQL UPDATE and DELETE operations, AI-powered SQL generation, ingestion into Apache Iceberg, performance improvements, bug fixes, and community contributions. Its lightweight patch-part mechanism applies changes efficiently during background merges, making declarative updates substantially faster with minimal query impact. ### Source excerpt ClickHouse 25.7 is available. In this post, you will learn about new features, including lightweight updates, AI-powered SQL generation, ingests into Apache Iceberg, and more! ## IoT for fun and Prophet: Scaling IoT and predicting the future DevFeed: [IoT for fun and Prophet: Scaling IoT and predicting the future]() Original publisher: [Read original article]() Author: Bryan Wood Published: 2025-07-22T00:00:00Z Content type: tutorial Language: en Sources: [Redpanda]() Topics: [Apache Iceberg](), [Internet of things](), [Redpanda-Connect](), [Streaming](), [Time Series](), [ESP32](), [Amazon Web Services](), [Amazon S3](), [schema-evolution](), [event driven](), [Hardware](), [Kafka]() Tags: [apache-iceberg](), [apache-iceberg-for-scalable-storage](), [aws](), [core](), [devices](), [esp32-mqtt-iot-example](), [event-driven](), [hardware](), [integrating-iceberg-and-redpanda](), [iot](), [iot-data-pipeline-architecture](), [iot-predictive-analytics](), [iot-sensor-data-forecasting](), [predictive-analysis-with-iot](), [prophet-forecasting-for-iot](), [real-time-iot-data-streaming](), [redpanda-connect](), [redpanda-connect-for-iot](), [s3](), [scaling-iot-with-prophet](), [schema-evolution](), [schema-management-in-iot](), [sensor](), [streaming](), [thought-leadership]() ### AI overview This tutorial presents a scalable IoT data pipeline for real-time streaming and forecasting. It combines Redpanda and Redpanda Connect for ingestion and messaging, Apache Iceberg for scalable time-series data storage, AWS IoT for device-to-cloud communication, and ESP32 hardware for affordable connected-device experiments and deployments. ### Source excerpt Check out this real-world example of scaling IoT for predictive analysis with Redpanda, Iceberg, and Prophet--without high costs or complexity. [Next page]()