# Lakeflow

Published articles for Lakeflow.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Query Databricks Lakeflow Pipeline Tables from Snowflake, Without Copying Data

DevFeed: [Query Databricks Lakeflow Pipeline Tables from Snowflake, Without Copying Data](<https://devfeed.tech/articles/query-databricks-lakeflow-pipeline-tables-from-snowflake-without-copying-data-121423.md>)

Original publisher: [Read original article](<https://medium.com/snowflake/query-databricks-lakeflow-pipeline-tables-from-snowflake-without-copying-data-dde301ede202?source=rss----34b6daafc07---4>)

Author: Vikneshwara Balakrishnan

Published: 2026-11-06T15:01:02Z

Content type: tutorial

Language: en

Sources: [Snowflake Builders](<https://devfeed.tech/sources/snowflake-builders.md>)

Topics: [Apache Iceberg](<https://devfeed.tech/topics/apache-iceberg.md>), [databricks](<https://devfeed.tech/topics/databricks.md>), [Data pipelines](<https://devfeed.tech/topics/data-pipelines.md>), [OpenID connect (OIDC)](<https://devfeed.tech/topics/oidc.md>)

Tags: [apache-iceberg](<https://devfeed.tech/tags/apache-iceberg.md>), [apache-spark](<https://devfeed.tech/tags/apache-spark.md>), [aws](<https://devfeed.tech/tags/aws.md>), [data](<https://devfeed.tech/tags/data.md>), [data-engineering](<https://devfeed.tech/tags/data-engineering.md>), [databricks](<https://devfeed.tech/tags/databricks.md>), [lakeflow](<https://devfeed.tech/tags/lakeflow.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [snowflake](<https://devfeed.tech/tags/snowflake.md>)

### AI overview

The excerpt reports that all 13 selected Lakeflow pipeline tables were read in place from Snowflake on AWS after enabling `pipelines.externalMetadata.enabled`, Iceberg v3 compatibility, and Unity Catalog external access. Individual Iceberg tables served raw and curated data: they allow a precise table list and refresh only when the pipeline event log indicates changes, but require explicit refreshes. A catalog-linked database served the gold schema with automatic discovery and refresh; it is read-only here, exposes whole namespaces, and can propagate drops to Databricks if writes are allowed. The setup requires OAuth catalog integrations, vended credentials, and grants for the Snowflake service principal. Reported pitfalls include disabled metastore external access, missing pipeline metadata opt-in, and Iceberg UUID mismatches after full refreshes; the author handled the latter by recreating affected Snowflake Iceberg objects. The author describes pipeline-table external access as Public Preview and reports limitations including unsupported external writes, row-level security, and masking. Results come from one environment, and some explanations and fixes are explicitly unverified.

### Source excerpt

Streaming tables and materialized views, read in place as Iceberg: the settings that matter, what the logs show, and the Snowflake errors I hit Most guides to reading Databricks from Snowflake start with a plain Delta table. Mine are different: every table here is built by Lakeflow Declarative Pipelines (formerly Delta Live Tables), so they are streaming tables and materialized views, which an outside engine cannot read out of the box. I got Snowflake to read all 13 of them in place, as Apache Iceberg, with no copy: three layers on AWS, each in its own Unity Catalog catalog. I tried two Snowflake features on them: Iceberg tables over Unity Catalog's REST endpoint, and a catalog-linked database. This article covers how each works on Lakeflow tables, the Snowflake checks that confirm it, and the four Snowflake errors that cost me time. In short: Lakeflow tables need one opt-in (pipelines.externalMetadata.enabled) to be readable from Snowflake; with it, both Iceberg tables over the REST endpoint (raw, curated) and a catalog-linked database (gold) read all 13 in place. Disclaimer: This is my own work, not affiliated with or endorsed by Snowflake, Databricks or AWS. Parts were established by trial and error in one environment, so verify against the official documentation.Why Lakeflow tables are different A Lakeflow pipeline publishes streaming tables (incremental) and materialized views (recomputed queries). Both are Unity Catalog managed tables, but neither is a plain Delta table you can point another engine at. Three things follow, and they shaped everything below. They need an opt-in. Per Databricks' documentation, they appear externally "as managed tables" with the same name, but only with pipelines.externalMetadata.enabled = 'true', UniForm for Iceberg and Iceberg v3 (Databricks Runtime 17.3 or later). Skip the property and Snowflake rejects the views (Error 2). The pipeline owns them. Its run-as identity owns every table, so everyone else, including Snowflake's ser

## Unify your productivity and AI data with Lakeflow Connect

DevFeed: [Unify your productivity and AI data with Lakeflow Connect](<https://devfeed.tech/articles/unify-your-productivity-and-ai-data-with-lakeflow-connect-101129.md>)

Original publisher: [Read original article](<https://www.databricks.com/blog/unify-your-productivity-and-ai-data-lakeflow-connect>)

Author: Sonia Bendre; Giselle Goicochea

Published: 2026-10-26T23:20:12Z

Content type: article

Language: en

Sources: [Databricks](<https://devfeed.tech/sources/databricks.md>)

Topics: [Data pipelines](<https://devfeed.tech/topics/data-pipelines.md>), [data-engineering](<https://devfeed.tech/topics/data-engineering.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [AI connector platforms](<https://devfeed.tech/topics/ai-connector-platforms.md>)

Tags: [about](<https://devfeed.tech/tags/about.md>), [access-control](<https://devfeed.tech/tags/access-control.md>), [accounting](<https://devfeed.tech/tags/accounting.md>), [adoption](<https://devfeed.tech/tags/adoption.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-bricks](<https://devfeed.tech/tags/agent-bricks.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [ai-data](<https://devfeed.tech/tags/ai-data.md>), [ai-foundation-models](<https://devfeed.tech/tags/ai-foundation-models.md>), [connect](<https://devfeed.tech/tags/connect.md>), [data](<https://devfeed.tech/tags/data.md>), [lakeflow](<https://devfeed.tech/tags/lakeflow.md>), [managed](<https://devfeed.tech/tags/managed.md>), [platform](<https://devfeed.tech/tags/platform.md>), [post](<https://devfeed.tech/tags/post.md>), [product](<https://devfeed.tech/tags/product.md>), [productivity](<https://devfeed.tech/tags/productivity.md>), [series](<https://devfeed.tech/tags/series.md>), [this](<https://devfeed.tech/tags/this.md>), [with](<https://devfeed.tech/tags/with.md>)

### AI overview

Lakeflow Connect ingests productivity and AI data from SaaS tools into governed Delta tables in Unity Catalog. Connectors are configured through a point-and-click UI or API and manage incremental ingestion without customer-provisioned infrastructure. The excerpt describes handling unstructured files, pages, and messages in source-dependent formats, with structured files such as CSVs and Excel files also loadable as Delta tables for downstream document processing and retrieval. It presents use cases including grounding knowledge assistants in current documentation, combining Jira delivery data with Confluence documentation for analytics, and tracking foundation-model usage. Customer examples include Toyota Finance Australia using Jira ingestion, VaxCare grounding a chatbot in Confluence, and Orizon syncing over one million Jira tickets daily for dashboards and natural-language queries. The excerpt says Unity Catalog provides shared governance, lineage, and access control; it does not enumerate specific connector coverage or detailed setup steps.

### Source excerpt

This is the second post in our series on how Lakeflow Connect brings fully managed...