# data-architecture A technical framework for designing and governing an organization's data collection, management, movement, storage, and use. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## OWC Acquires OpenDrives, Adding Atlas, Astraeus, and Edge to the Jellyfish Shared Storage Line DevFeed: [OWC Acquires OpenDrives, Adding Atlas, Astraeus, and Edge to the Jellyfish Shared Storage Line]() Original publisher: [Read original article]() Author: Harold Fritts Published: 2026-09-15T16:55:08Z Content type: news Language: en Sources: [StorageReview.com]() Topics: [data-architecture](), [cloud-infrastructure](), [Hardware](), [data-processing]() Tags: [acquisition](), [cloud-native](), [company](), [enterprise](), [enterprise-storage](), [hybrid-cloud](), [infrastructure](), [products](), [storage]() ### AI overview Other World Computing (OWC) has acquired OpenDrives, bringing the Atlas, Astraeus, and Edge platforms into its Jellyfish shared storage portfolio. The companies say the deal expands OWC's capabilities in enterprise data management, hybrid cloud orchestration, and edge workflows; financial terms were not disclosed. ### Source excerpt Other World Computing (OWC) has acquired OpenDrives, the Los Angeles software-defined storage company whose Atlas platform has sat behind Hollywood studios, post houses, and live broadcast networks since 2011. The deal brings OpenDrives' Atlas, Astraeus, and Edge platforms into OWC's shared storage portfolio alongside the Jellyfish line, and OWC says it extends that line into The post OWC Acquires OpenDrives, Adding Atlas, Astraeus, and Edge to the Jellyfish Shared Storage Line appeared first on StorageReview.com. ## Aiven, Confluent, Redpanda, StreamNative and Ververica Form Streamhouse Working Group DevFeed: [Aiven, Confluent, Redpanda, StreamNative and Ververica Form Streamhouse Working Group]() Original publisher: [Read original article]() Author: Streamhouse Working Group Published: 2026-09-15T07:00:00Z Content type: release Language: en Sources: [Confluent: Data in motion]() Topics: [data-architecture](), [real-time](), [data](), [Streaming](), [Artificial Intelligence]() Tags: [agents](), [ai](), [architectures](), [confluent](), [data](), [distributed](), [news](), [real-time](), [streaming]() ### AI overview Aiven, Confluent, Redpanda, StreamNative, and Ververica announced the Streamhouse Working Group and published a vendor-neutral definition of Streamhouse. The proposed data architecture is designed to keep business context continuously available to production applications, analytics, and AI agents, with real-time, production-native, and decentralized attributes. ### Source excerpt New industry initiative establishes an open category for data architectures that power real-time applications and AI agents ## How to build a service marketplace with provider profiles in Webflow Cloud DevFeed: [How to build a service marketplace with provider profiles in Webflow Cloud]() Original publisher: [Read original article]() Author: Ismail Ajagbe Published: 2026-09-12T00:00:00Z Content type: tutorial Language: en Sources: [Webflow Blog]() Topics: [data-architecture](), [Database](), [SQLite](), [Next.js](), [Node.js](), [Authentication](), [Content Management System]() Tags: [data-modeling](), [database](), [guide](), [guides](), [how-to](), [next-js](), [node-js](), [sqlite](), [webflow]() ### AI overview A guide to building the discovery side of a service marketplace on Webflow Cloud. It explains why provider profiles should be modeled as database rows rather than CMS content when they require owner editing, flexible filtering, and approval before publication. ### Source excerpt Learn how to build the discovery half of a service marketplace on Webflow Cloud. ## "Six tools, one harness": Salesforce loops together a six-pack of favorites DevFeed: ["Six tools, one harness": Salesforce loops together a six-pack of favorites]() Original publisher: [Read original article]() Author: Adrian Bridgwater Published: 2026-09-10T20:03:25Z Content type: article Language: en Sources: [The New Stack]() Topics: [Architecture & Design](), [Orchestration](), [data-architecture](), [Vibe coding](), [Claude](), [SDKs]() Tags: [agent](), [ai](), [ai-agents](), [api](), [claude](), [data](), [enterprise](), [integration](), [orchestration](), [salesforce](), [security](), [vibe-coding]() ### AI overview Salesforce introduced an Enterprise AI Harness that combines platform capabilities through a common, composable architecture and an AI control plane. The article describes its use of data, integration, APIs, agent orchestration, analytics, security and compliance to reduce siloed enterprise automation. ### Source excerpt Salesforce introduced its Salesforce Enterprise AI Harness on Thursday as a formalized amalgamation of the AI harness concepts and infrastructure The post "Six tools, one harness": Salesforce loops together a six-pack of favorites appeared first on The New Stack. ## A Unified Data Architecture For Sovereign Agentic AI With VMware Tanzu And VMware vSAN DevFeed: [A Unified Data Architecture For Sovereign Agentic AI With VMware Tanzu And VMware vSAN]() Original publisher: [Read original article]() Author: arnab chakraborty Published: 2026-09-03T23:27:50Z Content type: article Language: en Sources: [VMware Blogs]() Topics: [Agentic AI Architecture](), [AI Architecture](), [data-architecture](), [AI Agent](), [cloud-infrastructure](), [Latency](), [real-time](), [Security](), [data-processing]() Tags: [agentic-ai](), [agentic-ai-architecture](), [ai](), [ai-architecture](), [blog](), [cloud](), [data-architecture](), [efficiency](), [infrastructure](), [latency](), [modern-apps](), [real-time](), [security]() ### AI overview The article presents a unified, on-premises data architecture for sovereign agentic AI using VMware Tanzu, VMware Tanzu Greenplum, VMware Cloud Foundation, and VMware vSAN. It argues that placing AI compute close to enterprise data can improve performance and cost while reducing latency, data-transfer fees, and compliance risks. ### Source excerpt By combining VMware Tanzu Greenplum with VMware vSAN, organizations can bring their AI compute directly to their data storage layer for improved cost and latency. The post A Unified Data Architecture For Sovereign Agentic AI With VMware Tanzu And VMware vSAN appeared first on Tanzu. The post A Unified Data Architecture For Sovereign Agentic AI With VMware Tanzu And VMware vSAN appeared first on VMware Blogs. ## Machine vs. machine: The new reality of cybersecurity in ANZ DevFeed: [Machine vs. machine: The new reality of cybersecurity in ANZ]() Original publisher: [Read original article]() Author: Jeremy Pell Published: 2026-08-26T00:00:00Z Content type: article Language: en Sources: [Elastic Blog - Elasticsearch, Kibana, and ELK Stack]() Topics: [Cybersecurity](), [Frontier AI](), [Threat Research](), [threat detection](), [data-architecture](), [AI Platforms/Deployment](), [Security & Privacy]() Tags: [agentic-ai-cybersecurity-security-research](), [australia](), [compliance](), [cybersecurity](), [data-architecture](), [endpoint-security-siem-security](), [frontier-ai](), [policy](), [research](), [threat-detection]() ### AI overview Frontier AI is accelerating cyberattacks across Australia and New Zealand to machine speed, while defensive capabilities, policy, data visibility, and operational security are struggling to keep pace. Survey findings from more than 850 IT and cybersecurity professionals highlight gaps between regulatory intent and real-world protection, as well as the need for searchable, unified data architectures to support reliable AI-enabled defence. ### Source excerpt Frontier AI has accelerated cyber threats to machine speed, leaving many ANZ organisations vulnerable. Our latest research reveals how fragmented data and visibility gaps hinder defence and why a unified platform is essential to battle threats. ## AI success depends on data foundations DevFeed: [AI success depends on data foundations]() Original publisher: [Read original article]() Author: James Heward Published: 2026-08-14T14:19:00Z Content type: article Language: en Sources: [Scott Logic]() Topics: [data-architecture](), [AI Strategy](), [AI Development]() Tags: [agentic-ai](), [ai](), [ai-readiness](), [artificial-intelligence](), [data](), [data-engineering](), [governance](), [integration](), [legacy-it](), [legacy-modernisation](), [prototypes](), [quality](), [silos]() ### AI overview Strong data foundations--discoverable, high-quality, accessible, governed, and integrated--are presented as essential for reliable AI outcomes, especially as organisations adopt more autonomous agentic systems. ### Source excerpt As organisations invest in AI, many discover that their biggest challenges are not AI-related at all. In this post, I explore why strong data foundations, from quality and accessibility to governance and integration, are essential for turning AI ambition into reliable, production-ready outcomes. ## The 4 Failure Modes of Agent Context in Production DevFeed: [The 4 Failure Modes of Agent Context in Production]() Original publisher: [Read original article]() Author: Jeff Mills Published: 2026-07-28T00:00:00Z Content type: article Language: en Sources: [Redis Blog]() Topics: [AI Agent](), [Artificial Intelligence](), [data](), [data-architecture](), [Orchestration](), [real-time](), [systems]() Tags: [agent](), [agentic](), [agentic-ai](), [ai](), [ai-agent](), [amazon-simple-storage-service-s3](), [data-management](), [gartner](), [orchestration](), [real-time](), [tech-de]() ### AI overview The article examines four infrastructure failure modes that can undermine production AI agents: fragmented context, opacity, speed degradation, and non-accumulation. It explains how incomplete or stale information drawn from enterprise systems can produce confident but incorrect answers, and presents a real-time context layer as part of the solution. ### Source excerpt A production AI agent depends heavily on the context layer that tells it what to know at the moment it acts. It can pass every staging test, answer questions, call the right tools, and demo beautifully, then hit production and confidently offer a re... ## What Matters Most for NoSQL Migrations DevFeed: [What Matters Most for NoSQL Migrations]() Original publisher: [Read original article]() Author: Cynthia Dunlop Published: 2026-07-21T13:44:46Z Content type: tutorial Language: en Sources: [ScyllaDB]() Topics: [NoSQL](), [Database Migration](), [migration](), [data-architecture](), [data](), [App]() Tags: [apps](), [community](), [complexity](), [data](), [data-modeling](), [database](), [database-migration](), [databases](), [how-to](), [migration](), [offline](), [performance](), [scale](), [validation](), [what-matters]() ### AI overview This article presents practical guidance for planning, executing, and de-risking NoSQL database migrations. It compares online and offline approaches and emphasizes schema migration, data movement, and validation, with attention to performance, scale, complexity, and disruption. ### Source excerpt How to prioritize the things that matter most for planning, executing and de-risking your NoSQL database migration ## You Don't Need to Build It to Design It DevFeed: [You Don't Need to Build It to Design It]() Original publisher: [Read original article]() Author: Humayun Tanwar Published: 2026-07-06T11:05:34Z Content type: tutorial Language: en Sources: [Mindorks - Medium]() Topics: [systems](), [data-architecture](), [Latency](), [Requirements]() Tags: [business](), [code](), [compliance](), [data-architecture](), [design](), [latency](), [machine-learning](), [patterns](), [requirements](), [software-engineering](), [systems](), [tech-stack](), [technology]() ### AI overview This article explains why solutions engineers can contribute meaningfully to system design without writing the implementation. It connects integration scoping, data ownership, real-time communication choices, and failure handling to core design concepts, then presents five questions about scale, constraints, dependencies, failure tolerance, and success criteria. ### Source excerpt System design is one of those topics that feels like it belongs exclusively to engineers who write the code. The diagrams, the architecture reviews, the database choices , it all looks like backend territory from the outside. You're Already Doing This Solutions engineers sit at an interesting intersection. You're deep enough in the technical conversation to understand how systems connect, and close enough to the customer to understand what they actually need from those systems. That combination gives you a view that's genuinely useful in design conversations, and one that's worth developing deliberately. When you scope an integration and map which system owns which data, that's data architecture. When you recommend a webhook approach over polling because the customer needs real-time updates, that's a latency and reliability trade-off. When you ask a customer how they handle failures before a build starts, that's systems thinking. The formal vocabulary of system design is mostly a structured way of describing what you're already doing in the field. Building that vocabulary, and understanding the patterns underneath it, makes it easier to contribute in technical conversations, ask the right questions earlier, and connect your field experience to the broader design decisions being made around you. The Five Questions That Structure Any Design Good design conversations don't start with a diagram. They start with these. Scale. What volume are you designing for, not today but at the ceiling that changes the architecture? A thousand requests a day is a different system from a thousand per second. That number changes your database choice, whether you need a queue, and how retries need to behave. Constraints. What's already fixed? Existing tech stack, compliance requirements, vendor contracts. Every real system gets built inside constraints. Name them before drawing anything. Dependencies. What does this system talk to, and what talks to it? Systems that look simple in a conv ## Data Engineering Weekly #276 DevFeed: [Data Engineering Weekly #276]() Original publisher: [Read original article]() Author: Ananth Packkildurai Published: 2026-06-29T03:52:17Z Content type: article Language: en Sources: [Data Engineering Weekly]() Topics: [data-engineering](), [schema-evolution](), [data-architecture](), [Amazon OpenSearch Service](), [Resilience](), [data observability]() Tags: [architecture](), [compatibility](), [data-engineering](), [engineering](), [observability](), [opensearch](), [resilience](), [schema-evolution]() ### AI overview Data Engineering Weekly #276 is a newsletter roundup covering data platform fundamentals, storage and workload architecture, schema evolution in Pinterest's ingestion framework, zone-failure-resilient OpenSearch at Uber, AI modernization, and stateful reasoning systems for notebooks. ### Source excerpt The Weekly Data Engineering Newsletter ## Data Engineering Weekly #274 DevFeed: [Data Engineering Weekly #274]() Original publisher: [Read original article]() Author: Ananth Packkildurai Published: 2026-06-15T05:29:03Z Content type: news Language: en Sources: [Data Engineering Weekly]() Topics: [data-engineering](), [Data Quality](), [anthropic](), [data-architecture](), [AI, ML & Data Engineering](), [Claude](), [Messaging]() Tags: [anthropic](), [architecture](), [claude](), [data](), [data-architecture](), [data-engineering](), [data-quality](), [engineering](), [software]() ### AI overview Data Engineering Weekly #274 covers data platform fundamentals, Anthropic's use of Claude for self-service analytics, Airbnb's data architecture and ownership conventions, Uber's data abstraction layer, and semantic search for AI agents. ### Source excerpt The Weekly Data Engineering Newsletter ## ScyllaDB Customer Experience Spotlight: Faisal Saeed DevFeed: [ScyllaDB Customer Experience Spotlight: Faisal Saeed]() Original publisher: [Read original article]() Author: Cynthia Dunlop Published: 2026-06-11T12:01:56Z Content type: article Language: en Sources: [ScyllaDB]() Topics: [ScyllaDB Cloud](), [data-architecture](), [migration](), [Automation](), [Database](), [on-prem](), [MariaDB]() Tags: [app](), [automation](), [blog](), [cloud](), [community](), [customer](), [cx-profiles](), [data-modeling](), [database](), [demo](), [india](), [migration](), [on-prem](), [scale](), [scylladb](), [scylladb-cloud](), [testing](), [tool]() ### AI overview Faisal Saeed, Principal Customer Engineer at ScyllaDB, describes his work supporting existing customers, evaluating deployments, data modeling, and migrations to ScyllaDB Enterprise or ScyllaDB Cloud. He also discusses the ScyllaDB Automation Framework, which automates cluster operations, workload execution, and stress testing, and highlights ScyllaDB Cloud and a large-scale customer use case in India. ### Source excerpt Meet Faisal Saeed, Principal Customer Engineer on the Customer Experience team here at ScyllaDB. ## Scaling beyond one: How Airbnb evolved its data architecture for a multi-product world DevFeed: [Scaling beyond one: How Airbnb evolved its data architecture for a multi-product world]() Original publisher: [Read original article]() Author: Patrick Lam Published: 2026-06-09T17:01:02Z Content type: article Language: en Sources: [The Airbnb Tech Blog - Medium]() Topics: [data-architecture](), [Data Infrastructure](), [data-modeling](), [data]() Tags: [analytics](), [architecture](), [data](), [data-architecture](), [data-engineering](), [data-infrastructure](), [data-modeling](), [engineering](), [offline](), [post](), [technology]() ### AI overview Airbnb's data and analytics engineering teams evolved a decade-old offline data warehouse to support Homes, Experiences, and Services. The article examines the trade-offs between separate product-specific data models and a unified monolithic model while describing the need for a consistent, flexible, and scalable data foundation. ### Source excerpt How Airbnb's data engineers and analytics engineers built a consistent and flexible data modeling framework to support the expansion into Homes, Experiences, and Services. By: Patrick Lam, Namrata Lamba, Jamie Stober With the May 2025 Summer Release, Airbnb redesigned its app, relaunched Experiences, and debuted Services, pushing us beyond our traditional Homes focus. For the data teams, this meant rapidly evolving a decade-old infrastructure to integrate two brand-new product pillars. Our data engineers and analytics engineers rose to the challenge by building a consistent and flexible framework to serve as a robust and scalable data foundation for the next decade of growth. But getting there wasn't straightforward. This fundamental shift surfaced a critical question for our data organization: How do you evolve your offline data architecture to support new product lines without introducing disorder in vital analytics services? We knew the approach we took would have long-lasting implications. A fragmented strategy risked creating data silos, inconsistent analytics, and a tangled web of technical debt that would likely slow down future innovation. In this post, we'll take you behind the scenes to share key decisions that we made, the framework that emerged, and the lessons that helped reshape our offline data warehouse for the future. Note that we focus specifically on our offline data warehouse (the analytics-oriented data infrastructure owned by our data engineers and analytics engineers) rather than the online data systems that serve the app directly, as the two domains have fundamentally different requirements, constraints, and design philosophies that warrant separate treatment. The core dilemma: separate vs. monolithic The first and most critical question was how to structure offline data for the new, three-product world, with Homes, a refreshed Experiences product, and the new Services offering. This involved a trade-off between two main approaches: Separate ## Agentic AI removes human safeguards from enterprise data feedback loops DevFeed: [Agentic AI removes human safeguards from enterprise data feedback loops]() Original publisher: [Read original article]() Author: flashdba Published: 2026-04-20T22:04:35Z Content type: opinion Language: en Sources: [flashdba]() Topics: [Artificial Intelligence](), [data-architecture](), [data-engineering](), [rate-limiting](), [audit trail]() Tags: [agentic](), [agentic-ai](), [ai](), [audit-trail](), [cloud](), [data-architecture](), [data-engineering](), [databases](), [databases-and-agentic-ai](), [inferencing](), [performance](), [rate-limiting]() ### AI overview This opinion article argues that enterprise data architectures traditionally relied on human involvement as an implicit safeguard. Delays, approvals and handoffs helped absorb errors, create audit points and limit the speed of changes. Agentic AI can close the feedback loop at machine speed, potentially removing those protections along with the human decision point. ### Source excerpt Classic enterprise data architecture had an implicit safeguard built into it. The human in the loop provided error absorption, audit accretion and natural rate-limiting - none of which were ever specified. Agentic AI removes the human. It removes all of those protections simultaneously. ## Introducing the NUMBER data type DevFeed: [Introducing the NUMBER data type]() Original publisher: [Read original article]() Author: Piotr Findeisen, Starburst Data Published: 2026-03-25T00:00:00Z Content type: article Language: en Sources: [Trino Blog]() Topics: [data](), [SQL](), [data-architecture](), [Databases](), [Oracle Database](), [PostgreSQL](), [MySQL](), [MariaDB]() Tags: [data](), [database](), [integration](), [mysql](), [oracle](), [postgresql](), [sql](), [types]() ### AI overview Trino 480 introduces the NUMBER data type, enabling high-precision, variable-scale decimal values beyond the existing 38-digit DECIMAL limit. The feature improves access to numeric data across Oracle, PostgreSQL, MySQL, MariaDB, and SingleStore systems while avoiding skipped columns or lossy rounding. ### Source excerpt One of Trino's core strengths is breaking down data silos--enabling data engineers to query diverse data sources through a single SQL interface. However, when those sources use high-precision numeric types beyond Trino's 38-digit DECIMAL limit, that promise breaks down. Users faced an impossible choice: skip the columns entirely and lose access to critical data, or accept lossy rounding that compromises data integrity. This challenge required a new approach: a dedicated data type for high-precision, variable-scale decimals. ## Two Betting Platforms, One Lesson: The Database Matters DevFeed: [Two Betting Platforms, One Lesson: The Database Matters]() Original publisher: [Read original article]() Author: Becca Weng Published: 2026-02-24T00:00:00Z Content type: article Language: en Sources: [Cockroach Labs]() Topics: [CockroachDB](), [Databases](), [Scalability](), [real-time](), [Resilience](), [Replication](), [Concurrency](), [consistency](), [data-architecture]() Tags: [cockroachdb](), [concurrency](), [consistency](), [data-architecture](), [database](), [distributed](), [real-time](), [replication](), [resilience](), [scalability]() ### AI overview The article describes how betting and gaming platforms face sharp traffic spikes, high transaction volumes, and reliability demands. It presents Newton and Kaizen Gaming as examples of operators using CockroachDB and distributed SQL to support global scale, real-time activity, replication, resilience, and consistency. ### Source excerpt In betting and gaming, growth is rarely gradual. Traffic spikes around major sporting events. Millions of real-time transactions hit your platform at once, and every one of them has to be correct. When the database falters, the consequences aren't abstract. Revenue stops. Players lose trust. Regulators start asking questions. ## BigQuery connector for ClickPipes is now in Private Preview DevFeed: [BigQuery connector for ClickPipes is now in Private Preview]() Original publisher: [Read original article]() Author: Marta Paes Published: 2026-01-28T16:38:57Z Content type: news Language: en Sources: [ClickHouse Blog]() Topics: [BigQuery](), [clickhouse](), [migration](), [real-time](), [Performance Testing](), [data-architecture](), [datasets](), [Cloud](), [Replication]() Tags: [architecture](), [bigquery](), [clickhouse](), [cloud](), [data](), [datasets](), [migration](), [open-source](), [performance](), [performance-testing](), [real-time](), [replication]() ### AI overview The article announces a private-preview BigQuery connector for ClickPipes, enabling users to load BigQuery data into ClickHouse Cloud for rapid exploration, prototyping, and performance testing. It presents ClickHouse as a low-latency speed layer alongside BigQuery's batch-processing and warehousing workloads, while describing automated data movement and future continuous-ingestion capabilities. ### Source excerpt Load data from BigQuery into ClickHouse Cloud in a few clicks for fast exploration and prototyping. The connector simplifies data migration for testing ClickHouse's real-time query performance on your BigQuery datasets. ## Rethinking the Global Reporting Platform DevFeed: [Rethinking the Global Reporting Platform]() Original publisher: [Read original article]() Author: Alex Seriy Published: 2026-01-28T00:00:00Z Content type: article Language: en Sources: [Cockroach Labs]() Topics: [CockroachDB](), [data-architecture](), [Database](), [dashboards](), [real-time](), [Software]() Tags: [cockroachdb](), [dashboards](), [data-architecture](), [database](), [real-time](), [reporting](), [software]() ### AI overview This article examines why traditional reporting architectures are struggling as reporting becomes more operational, distributed, governed, and increasingly real-time. It presents CockroachDB as a foundation for modern reporting platforms by embedding correctness, availability, and governance in the operational database. ### Source excerpt Data doesn't just serve the business anymore. Data is the business. Every transaction, every decision, every customer experience depends on having timely, trustworthy insight - not weekly, not daily, but continuously. ## How ClickHouse Cloud enabled LaunchDarkly to build and ship features faster DevFeed: [How ClickHouse Cloud enabled LaunchDarkly to build and ship features faster]() Original publisher: [Read original article]() Author: ClickHouse Published: 2026-01-22T00:00:00Z Content type: article Language: en Sources: [ClickHouse Blog]() Topics: [clickhouse](), [Data Infrastructure](), [data-architecture](), [product analytics](), [real-time](), [Low Latency](), [observability](), [Streaming](), [systems]() Tags: [analytics](), [clickhouse](), [data-architecture](), [data-infrastructure](), [development](), [latency](), [low-latency](), [observability](), [real-time](), [scale](), [storage](), [streaming]() ### AI overview LaunchDarkly uses ClickHouse Cloud to analyze raw feature evaluation events in real time for experimentation, product analytics, and observability. The migration replaced parts of its streaming and batch-oriented architecture, reducing data latency to seconds and enabling faster feature development. ### Source excerpt "With ClickHouse, we can serve fresh data, query raw events directly, lean on materialized views when needed, and build new capabilities in weeks instead of months or quarters." Joe Karayusuf, Software Engineer at LaunchDarkly ## Using Materialized Views and Derived Datasets to Optimize Data Queries DevFeed: [Using Materialized Views and Derived Datasets to Optimize Data Queries]() Original publisher: [Read original article]() Published: 2025-12-07T09:05:00Z Content type: article Language: en Sources: [Gunnar Morling]() Topics: [data-architecture](), [Data Management](), [Database](), [datasets](), [Latency](), [elasticsearch](), [parquet]() Tags: [data](), [data-lake](), [data-management](), [database](), [datasets](), [elasticsearch](), [latency](), [parquet](), [performance]() ### AI overview The article explains how pull-based queries retrieve matching records at query time and why this can create performance, data-format, data-shape, and data-location challenges. It presents materialized views and derived datasets as a way to precompute query results and store them in an optimized format, shape, and location. ### Source excerpt Table of Contents Materialized Views Embracing Data Duplication Streams for machines, tables for humans Historically, data management systems have been built around the notion of pull queries: users query data which, for instance, is stored in tables in an RDBMS, Parquet files in a data lake, or a full-text index in Elasticsearch. When a user issues a query, the engine will produce the result set at that point in time by churning through the data set and finding all matching records (oftentimes sped up by utilizing indexes). ## Building an Enterprise Data Warehouse on Heroku: From Complex ETL to Seamless Salesforce Integration DevFeed: [Building an Enterprise Data Warehouse on Heroku: From Complex ETL to Seamless Salesforce Integration]() Original publisher: [Read original article]() Author: Sudarshan Hiray Published: 2025-11-05T20:05:38Z Content type: article Language: en Sources: [Heroku]() Topics: [Heroku](), [data-architecture](), [Databases](), [Monitoring]() Tags: [analytics](), [applications](), [article](), [building](), [data](), [ecosystems](), [engineering](), [enterprise](), [etl](), [heroku](), [heroku-postgres](), [integration](), [platforms](), [salesforce](), [services](), [systems](), [tools](), [warehouse]() ### AI overview Heroku describes an enterprise data warehouse architecture that unifies Salesforce and data from multiple application databases for real-time analytics. The approach uses Heroku Connect, Heroku Postgres follower databases, and a unified analytics platform to address API limits, ETL complexity, production database load, and fragmented monitoring. ### Source excerpt Modern businesses don't just run on Salesforce--they run on entire ecosystems of applications. At Heroku, we operate dozens of services alongside our Salesforce instance such as billing systems, user management platforms, analytics engines, and support tools. Traditional approaches to unifying this data create more problems than they solve. In this article, we'll see how we [...] The post Building an Enterprise Data Warehouse on Heroku: From Complex ETL to Seamless Salesforce Integration appeared first on Heroku. ## Governed autonomy: The path to enterprise Agentic AI DevFeed: [Governed autonomy: The path to enterprise Agentic AI]() Original publisher: [Read original article]() Author: Tyler Akidau Published: 2025-10-28T00:00:00Z Content type: article Language: en Sources: [Redpanda]() Topics: [Artificial Intelligence](), [data-architecture](), [Model Context Protocol](), [MCP Server](), [systems]() Tags: [agentic](), [agentic-ai](), [agents](), [ai](), [ai-agents](), [announcement](), [architecture](), [connectivity](), [external](), [governance](), [mcp](), [mcp-server]() ### AI overview Redpanda presents the Agentic Data Plane as a governed data control plane for connecting autonomous AI agents with enterprise data and systems. It combines connectivity, governance policies, access controls, open standards such as MCP and A2A, focused MCP servers, and unified auditing and lineage to help enterprises scale and trust agentic workforces. ### Source excerpt The Agentic Data Plane by Redpanda is how enterprises will connect and control AI agents, and how they'll prove those agents act responsibly. Sign up for early access. ## "Like night and day": How Auditzy made queries 33x faster by switching from Postgres to ClickHouse DevFeed: ["Like night and day": How Auditzy made queries 33x faster by switching from Postgres to ClickHouse]() Original publisher: [Read original article]() Author: Mayank Joshi, Co-Founder and CTO, Auditzy Published: 2025-10-09T00:00:00Z Content type: article Language: en Sources: [ClickHouse Blog]() Topics: [clickhouse](), [data-architecture](), [Latency](), [Back end](), [data](), [real-time](), [datasets](), [Amazon Web Services](), [Azure](), [browser](), [Operating system]() Tags: [architecture](), [aws](), [azure](), [backend](), [browser](), [clickhouse](), [compression](), [data](), [data-architecture](), [datasets](), [geolocation](), [golang](), [india](), [latency](), [metrics](), [performance](), [postgres](), [real-time](), [scalable-architecture](), [switching]() ### AI overview Auditzy migrated from a Postgres-based architecture to ClickHouse after query latency and ingestion problems emerged as data volumes grew. The migration delivered queries that were 33x faster and 10x better compression, supporting scalable, real-time website performance analytics. ### Source excerpt When Mumbai-based startup Auditzy hit Postgres performance limits, they switched to ClickHouse--and saw queries run 33x faster with 10x better compression. [Next page]()