# pipeline Published articles for pipeline. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Why human oversight is shifting from writing code to defining requirements DevFeed: [Why human oversight is shifting from writing code to defining requirements]() Original publisher: [Read original article]() Author: Naseeb Ahmed Mian Published: 2026-09-17T13:00:00Z Content type: opinion Language: en Sources: [The New Stack]() Topics: [Requirements](), [AI Agent](), [Code](), [Availability]() Tags: [agent](), [agents](), [ai-agents](), [ai-engineering](), [andela](), [automated](), [availability](), [code](), [pipeline](), [post-contributed](), [requirements](), [review](), [software-development](), [sponsor-andela](), [sponsored-post-contributed](), [tests]() ### AI overview The article argues that human oversight of AI-assisted software development should focus on defining and reviewing requirements, not only checking whether generated code conforms to them. It illustrates the risk with a flawed availability-related requirement that passed specification review, generated six passing tests, traceability checks, and automated QA while violating the feature's intended outcome. ### Source excerpt This walks through the pipeline our agents operate inside--from a recorded scoping meeting through unit specs, spec review, generated code, The post Why human oversight is shifting from writing code to defining requirements appeared first on The New Stack. ## OpenTelemetry everywhere: Migrating a metrics platform at scale DevFeed: [OpenTelemetry everywhere: Migrating a metrics platform at scale]() Original publisher: [Read original article]() Author: Iris Grace Endozo, Farzad Vazirnia and Albert Kerr, Atlassian Published: 2026-09-17T11:00:00Z Content type: article Language: en Sources: [Cloud Native Computing Foundation]() Topics: [OpenTelemetry](), [migration](), [observability](), [telemetry](), [atlassian](), [Open Source]() Tags: [atlassian](), [blog](), [collector](), [metrics](), [migration](), [observability](), [opentelemetry](), [pipeline](), [scale](), [sidecar]() ### AI overview Atlassian describes replacing a large-scale gostatsd metrics pipeline with OpenTelemetry while preserving the existing StatsD interface for service teams. The migration uses purpose-built OpenTelemetry Collector distributions across collection, ingest, aggregation, and forwarding stages, with support for both StatsD and OTLP during the transition. ### Source excerpt Why we did this at all For most of the last decade our metrics pipeline ran on gostatsd, the open-source StatsD implementation we maintain. It primarily did two jobs: as sidecar on every host and the... ## Build a serverless PII redaction pipeline with Amazon Bedrock Data Automation DevFeed: [Build a serverless PII redaction pipeline with Amazon Bedrock Data Automation]() Original publisher: [Read original article]() Author: Samantha Stuart Published: 2026-09-16T15:17:37Z Content type: tutorial Language: en Sources: [Artificial Intelligence]() Topics: [Amazon Bedrock](), [pii](), [Serverless](), [AWS Lambda](), [AWS Step Functions](), [Artificial Intelligence](), [Amazon Web Services](), [Ansible]() Tags: [advanced-300](), [amazon-bedrock](), [amazon-bedrock-data-automation](), [aws](), [aws-lambda](), [aws-step-functions](), [how-to](), [pii-redaction](), [pipeline](), [precision](), [serverless](), [technical-how-to]() ### AI overview This tutorial presents a serverless AWS pipeline for detecting and redacting personally identifiable information in scanned documents and images. It uses Amazon Bedrock Data Automation with a custom blueprint, AWS Step Functions, and AWS Lambda, with a token-matching quality check to improve recall on degraded and handwritten documents. ### Source excerpt Learn how to automate end-to-end PII detection and redaction from scanned documents at scale using Amazon Bedrock Data Automation with a custom blueprint, AWS Step Functions, and AWS Lambda. A custom blueprint redacts sensitive fields with field-level precision, and a token matching quality check raises recall across degraded and handwritten documents. ## New data pipeline management platform at Khan Academy DevFeed: [New data pipeline management platform at Khan Academy]() Original publisher: [Read original article]() Author: Khan Academy Published: 2018-04-30T22:00:00Z Content type: article Language: en Sources: [Khan Academy]() Topics: [data](), [DataOps](), [systems]() Tags: [batch](), [bigquery](), [cloud-dataflow](), [data](), [data-pipelines](), [debugging](), [engineering](), [infrastructure](), [news](), [pipeline](), [scheduling]() ### AI overview Khan Academy developed Khanalytics to manage its growing collection of data pipelines. The platform provides a sandboxed environment for batch jobs, a web interface, automatic parallelization, centralized logs, and pipeline scheduling with dependencies. ### Source excerpt By Ragini Gupta Data is very crucial to Khan Academy and is itself an internal product for the ... Read more ## ML based ranking using Nrtsearch DevFeed: [ML based ranking using Nrtsearch]() Original publisher: [Read original article]() Author: Mohammad Mohtasham (Software Engineer); Tao Yu (Software Engineer) Published: 2026-09-16T00:00:00Z Content type: article Language: en Sources: [Yelp]() Topics: [Inference](), [Scalability](), [Latency](), [recommendation systems](), [Open Source]() Tags: [bridge](), [inference](), [machine-learning](), [overhead](), [performance](), [pipeline](), [scalability](), [service]() ### AI overview Yelp's Nrtsearch Inference Plugin embeds machine-learning ranking directly in the search layer. The article explains the scoring workflow, including model configuration, feature extraction, candidate ranking, and application-specific business logic. It describes how co-locating feature storage and inference reduces network transfer, serialization overhead, and latency compared with a standalone inference service. ### Source excerpt We've extended Nrtsearch with the Inference Plugin, which embeds ML-based ranking directly in the search layer -- eliminating the need for a standalone scoring service. We use Nrtsearch (read more information on the blog post), a Lucene-based open-source search engine built by Yelp, to power a variety of applications such as business search, reviews search, ad delivery and photo search. In this blog post, we give a high-level overview of the Machine Learning (ML) based scoring workflow in Nrtsearch. We'll show how ML models are configured and loaded, and how different applications use custom business logic to develop, test, and... ## Run contained pen testing as chaos experiments securely DevFeed: [Run contained pen testing as chaos experiments securely]() Original publisher: [Read original article]() Author: Uma Mukkara Published: 2026-09-09T22:42:00Z Content type: article Language: en Sources: [Harness Blog]() Topics: [Resilience](), [Security](), [Testing]() Tags: [chaos](), [permission](), [pipeline](), [production](), [resilience](), [security](), [testing](), [tool]() ### AI overview The article describes contained pen-testing experiments that run inside a release pipeline. Each experiment targets one service, one failure or intrusion condition, and one pipeline stage, producing an immutable audit record while avoiding broad production access. ### Source excerpt | Blog ## GHarchive data has become unreliable for measuring GitHub activity DevFeed: [GHarchive data has become unreliable for measuring GitHub activity]() Original publisher: [Read original article]() Author: KD (noreply@blogger.com) Published: 2026-09-03T16:00:00Z Content type: opinion Language: en Sources: [Google Open Source Blog]() Topics: [data](), [datasets](), [GitHub](), [Crawler](), [GitHub API](), [Open Source](), [GraphQL]() Tags: [collect](), [data](), [dataset](), [datasets](), [github](), [google](), [google-open-source](), [graphql](), [open-data-sets](), [open-source](), [oss](), [pipeline](), [pull-requests](), [retention](), [stream](), [volume]() ### AI overview This article examines the reliability of open source data, focusing on GHarchive's coverage of GitHub events. It argues that GHarchive should not be used for real-time or volume-based metrics because event retention has declined and some activity is omitted by the GitHub Event stream and API limitations. ### Source excerpt by Sophia Vargas, Google Open Source & Andrew Nesbitt, Ecosyste.ms Every second, open source contribution quietly shapes the software we rely on, and yet our view of this open ecosystem is surprisingly opaque. Open source development is performed in public spaces -- we can see the commits, issues and comments, the APIs and endpoints are free to use -- the logs are just sitting there, so why can't we just collect all of the data? ...Said every researcher, everywhere. However in most cases of open source related data, we are only looking at part of the whole. Why am I writing this post? Because many of us (including many business decision-makers) are too comfortable with unsubstantiated data. We've gotten used to it. Our models assume that it's smelly and we adjust the logic and weights to compromise. When it comes to open source, our confidence is even lower, even though our resulting decisions can directly impact individuals whom we collectively depend on. Let's consider one of my favorite datasets: GHarchive. Started as a hobby project in 2011, this crawler has amassed more than 15 years of event data from GitHub. While this source provides a historical record of open source development on GitHub, as a real-time or comprehensive source of metrics, it's unreliable and should not be a source for volume-based metrics. In 2025, GHarchive captured 14% fewer events than in 2024, despite steady growth in platform adoption. Since 2025, we estimate that data retention in GHarchive has fallen to ~50% and in 2026 it may be as low as 20% for some event types (see figure below). Prior to 2025, you could make the general assumption that the majority of events would be represented in this pipeline. Since 2025, we must now assume we may be missing at least half of events and possibly more -- not to mention all of the additional activity that's left out of the event API (see GitHub's GraphQL API.) The crawler logic behind this dataset is simple: give me all the events from the GitHub Ev ## Building Trust in AI DevOps: Validating the Harness Knowledge Graph DevFeed: [Building Trust in AI DevOps: Validating the Harness Knowledge Graph]() Original publisher: [Read original article]() Author: Vikram Sahu Published: 2026-08-31T18:37:00Z Content type: article Language: en Sources: [Harness Blog]() Topics: [DevOps](), [Testing](), [Graphs](), [Artificial Intelligence](), [Software]() Tags: [ai](), [ai-evals](), [api](), [automated](), [data](), [devops](), [evals](), [graph](), [knowledge-graph](), [lifecycle](), [operational](), [other](), [pipeline](), [schema](), [sdlc](), [security](), [services](), [software](), [software-delivery](), [validation]() ### AI overview This article explains how Harness validates answers from its SDLC Knowledge Graph. Its multi-layered approach combines AI evaluations, schema traversal, API checks, direct product verification, production data, and shift-left testing to improve reliability. ### Source excerpt Discover our multi-layered validation approach combining AI evals to ensure reliable AI-powered software delivery insights. | Blog ## CircleCI Smarter Testing: Stop running tests that don't matter DevFeed: [CircleCI Smarter Testing: Stop running tests that don't matter]() Original publisher: [Read original article]() Author: Nathan Fish Published: 2026-08-26T19:00:00Z Content type: article Language: en Sources: [The CircleCI Blog Feed | CircleCI]() Topics: [Testing](), [CI/CD]() Tags: [accelerate](), [auto-rerun-failed-tests](), [ci-cd](), [circleci](), [circleci-news](), [developer-productivity](), [dynamic-test-splitting](), [engineering-productivity](), [flaky](), [intelligent-test-selection](), [parallel](), [pipeline](), [quality](), [reduce](), [smarter-testing](), [test-impact-analysis](), [testing](), [tests](), [time]() ### AI overview CircleCI describes Smarter Testing, a set of features designed to reduce CI/CD test execution time by skipping tests unaffected by changes, balancing parallel nodes, and retrying flaky tests. The article says early users have seen test runs up to four times faster. ### Source excerpt Testing eats up to half your pipeline time. See how CircleCI Smarter Testing skips unaffected tests, rebalances parallel nodes, and retries flaky tests. ## CodeScanner.scan(): Barcode Scanning Without Rebuilding the Camera Pipeline DevFeed: [CodeScanner.scan(): Barcode Scanning Without Rebuilding the Camera Pipeline]() Original publisher: [Read original article]() Author: Shai Almog Published: 2026-08-26T00:00:00Z Content type: release Language: en Sources: [CodeName One]() Topics: [Barcode](), [QR Code](), [webcam](), [Code]() Tags: [camera](), [code](), [component](), [on-device](), [pipeline](), [vision]() ### AI overview Codename One adds CodeScanner and VisionCameraView, higher-level APIs for full-screen barcode scanning and embedded live vision analysis. The update also introduces typed results, coordinate helpers, image bridging, and analyzer-specific native dependency selection. ### Source excerpt CodeScanner and VisionCameraView put full-screen scanning and embedded live analysis above Codename One's on-device vision APIs, with typed results and simulator scripting. ## Using Data Contracts to Coordinate Data Evolution at Enterprise Scale DevFeed: [Using Data Contracts to Coordinate Data Evolution at Enterprise Scale]() Original publisher: [Read original article]() Author: Keerthipriyan Published: 2026-08-25T20:22:17Z Content type: article Language: en Sources: [Walmart Global Tech]() Topics: [data-platforms](), [Data Quality](), [DataOps](), [systems]() Tags: [architecture](), [data](), [data-engineering](), [data-governance](), [data-platforms](), [data-quality](), [data-science](), [enterprise](), [organizational](), [pipeline](), [schema](), [software-engineering](), [teams](), [technology]() ### AI overview The article explains how data contracts help large enterprises coordinate changes across independently evolving data teams and downstream consumers. It argues that schema validation alone cannot identify ownership, downstream impact, or migration responsibilities, and presents data contracts as machine-enforceable coordination agreements. ### Source excerpt Coauthored by Satyajeet Coordinating Data Evolution at Enterprise Scale When you operate data platforms on a global enterprise scale, hundreds of engineering teams ship improvements every week, each moving independently to deliver value at the pace of the business demands. This velocity is a competitive advantage. The challenge: How do you enable hundreds of teams to evolve their data products independently while maintaining reliability for thousands of downstream consumers? Traditional coordination methods (messages, wiki updates, shared spreadsheets) work at small scale but break at Walmart scale. A source team ships an enhancement, perfectly valid within their domain, but that change ripples through fifteen downstream pipelines owned by different teams with different release schedules. Without a formal coordination mechanism, you discover the impact after it reaches production. The gap isn't technical debt or fragile systems. It's the absence of machine-enforceable agreements that scale with organizational complexity. Data contracts solve this: enabling teams to move fast independently while maintaining coordinated reliability across organizational boundaries. Here's the architecture we built. Why Schema Validation Alone Isn't Enough When data quality issues surface in production, the first instinct is often added to more schema validation. If a field is missing or has the wrong type, the pipeline catches it. This works for many data quality problems, but not all of them. Consider a scenario where a source team enhances their data model by restructuring field names to support new business capabilities. The schema still validates perfectly: every field exists; every type is correct; the data is well formed. But downstream consumers who depend on the original field names now receive empty results. Schema validation checks whether data has the right shape. It tells you that a field is missing. It does not tell you who owns that field, which downstream teams will bre ## Building Menu Vision: Real-Time Dish Recognition DevFeed: [Building Menu Vision: Real-Time Dish Recognition]() Original publisher: [Read original article]() Author: Arpitha Dudi, Growth Tech Lead Published: 2026-08-20T00:00:00Z Content type: article Language: en Sources: [Yelp]() Topics: [Computer vision](), [Artificial Intelligence](), [Android](), [cameraX](), [ML Kit](), [Hackathon](), [Development](), [iOS]() Tags: [ai](), [android](), [camerax](), [data-pipeline](), [development](), [hackathon](), [ios](), [ml-kit](), [pipeline](), [prototype](), [real-time](), [recognition](), [vision]() ### AI overview Yelp describes Menu Vision, a feature that uses a phone camera, AI, augmented reality, and text recognition to identify dishes on restaurant menus and surface related user photos and reviews. The article covers its hackathon-origin Android prototype and the production system's on-device machine learning, real-time computer vision, fuzzy matching, and distributed data pipeline. ### Source excerpt Menus aren't just lists, they're a window into a restaurant's unique offerings, specialties, and personality, shaping where and what we choose to eat. But here's the challenge: reading "Kung Pao Chicken - Stir-fried chicken with peanuts in spicy sauce" doesn't tell you what the portions look like, whether other diners loved it, or if it matches your expectations. At Yelp, we knew we had the solution sitting in our user-generated content: hundreds of millions of photos, reviews, and prices for dishes. The problem? Users had to manually search for each dish, an experience that doesn't work well when you're at... ## From Weeks to Hours: Inside Wix's Autonomous Bug-Fixing System DevFeed: [From Weeks to Hours: Inside Wix's Autonomous Bug-Fixing System]() Original publisher: [Read original article]() Author: Wix Engineering Published: 2026-08-19T10:17:19Z Content type: article Language: en Sources: [Wix Engineering]() Topics: [bug](), [Code review]() Tags: [architecture](), [autonomous](), [bug](), [code-review](), [context](), [database](), [jira](), [logs](), [pipeline](), [production](), [repo](), [support]() ### AI overview Wix describes the architecture behind Wix Orchestrator, an autonomous bug-fixing system designed to reduce the time from user reports to production fixes. The system automates investigation, context gathering, implementation, and code review, while requiring a human engineer to approve every fix before deployment. ### Source excerpt A user opens a support ticket. Their subscription cancellation stopped working. Somewhere inside a system with thousands of services and millions of users, something broke. Before we built Wix Orchestrator, here's what happened next: the ticket joined a queue. Days later, a support engineer picked it up, tried to reproduce the issue, contacted the customer to gather more context, and opened a Jira ticket routed to R&D. We triaged it, prioritized it against everything else on our plate,... ## Online index migration and shard scaling in OpenSearch with the AOSC plugin DevFeed: [Online index migration and shard scaling in OpenSearch with the AOSC plugin]() Original publisher: [Read original article]() Author: Arpit Singla Published: 2026-08-18T21:56:28Z Content type: article Language: en Sources: [OpenSearch]() Topics: [Amazon OpenSearch Service](), [migration](), [Open Source](), [Routing (disambiguation)]() Tags: [blog](), [mapping](), [migration](), [open-source](), [opensearch](), [pipeline](), [plugin](), [production](), [reconciliation](), [routing](), [schema](), [technical](), [workflow]() ### AI overview This article introduces Automatic Online Schema Change (AOSC), an open-source OpenSearch plugin for migrating live indexes to pre-created targets with different mappings, settings, shard counts, or document shapes. It backfills existing documents, replays operations made during migration, and switches an alias after a short write block, while documenting its scaling behavior and limitations. ### Source excerpt Learn how the open-source AOSC plugin migrates live OpenSearch indexes--changing mappings, settings, or shard counts--without losing writes or requiring downtime. The post Online index migration and shard scaling in OpenSearch with the AOSC plugin appeared first on OpenSearch. ## Rerun Only the Jobs That Failed DevFeed: [Rerun Only the Jobs That Failed]() Original publisher: [Read original article]() Author: Pete Miloravac Published: 2026-08-12T10:31:39Z Content type: release Language: en Sources: [Semaphore Engineering]() Topics: [ci](), [jobs](), [Development]() Tags: [2026](), [ci](), [cost](), [flag](), [jobs](), [pipeline](), [product-news](), [rebuilds](), [run](), [workflow]() ### AI overview Semaphore now reruns only failed jobs in a pipeline, carrying over successful jobs and preserving the workflow topology. This reduces rerun time and billable usage, while a YAML flag preserves the previous full-block rebuild behavior. ### Source excerpt Your pipeline fails on one job, and until now you had to rebuild the entire block to recover. Not anymore. Semaphore now reruns only the jobs that actually failed, so you get feedback faster and pay less to get it. What Shipped: Job Rerun When a pipeline failed, the old behavior rebuilt every job inside [...] The post Rerun Only the Jobs That Failed appeared first on Semaphore. ## Q2 2026 Product Update: Harness Pipeline DevFeed: [Q2 2026 Product Update: Harness Pipeline]() Original publisher: [Read original article]() Author: Vishal Vishwaroop Published: 2026-08-06T00:00:00Z Content type: release Language: en Sources: [Harness Blog]() Topics: [ci](), [Git](), [opa](), [configuration](), [Developer experience](), [Monitoring](), [API]() Tags: [2026](), [api](), [ci](), [developer-experience](), [git](), [opa](), [pipeline](), [pipelines](), [update]() ### AI overview Harness's Q2 2026 Pipeline update introduces beta DAG pipelines, allowing stages to declare explicit dependencies. It also adds looping strategies for chained pipelines, earlier matrix exclusions, template and governance improvements, Git-backed entity enforcement, execution visibility, and a pipeline dry-run API. ### Source excerpt DAG pipelines, template overrides, OPA enforcement on Git-backed entities, Git Experience monitoring, and 20 pipeline improvements -- Q2 2026 in review. | Blog ## Deploy Apps and Database Migrations in One CD Pipeline DevFeed: [Deploy Apps and Database Migrations in One CD Pipeline]() Original publisher: [Read original article]() Author: Animesh Pathak Published: 2026-08-05T00:00:00Z Content type: tutorial Language: en Sources: [Harness Blog]() Topics: [CI/CD](), [Database Migration](), [Deployment](), [Databases](), [DevOps]() Tags: [ci-cd](), [database](), [deployment](), [devops](), [governance](), [migrations](), [pipeline](), [visibility]() ### AI overview The article explains how Harness CD and Database DevOps can combine application deployments and database schema migrations in one CI/CD pipeline. It describes how this unified workflow can improve release consistency, visibility, governance, verification, and rollback safety while reducing deployment risk and operational overhead. ### Source excerpt Learn how Harness CD and Database DevOps automate application deployments and database schema migrations in a unified CI/CD pipeline. | Blog ## How Lawmatics Cut CI Compute Cost by 39.3% and Shortened Pipeline Time by 15.8% DevFeed: [How Lawmatics Cut CI Compute Cost by 39.3% and Shortened Pipeline Time by 15.8%]() Original publisher: [Read original article]() Author: Christian Gómez Alonso Published: 2026-07-27T15:23:37Z Content type: article Language: en Sources: [Semaphore Engineering]() Topics: [ci](), [Optimization](), [cpu](), [browser]() Tags: [browser](), [ci](), [ci-cd](), [compute](), [cost](), [cpu](), [optimization](), [parallel](), [parallelism](), [pipeline](), [time]() ### AI overview This case study describes how Lawmatics optimized its CI pipeline by changing the machine class used for parallel browser end-to-end tests while initially preserving parallelism. The reported result was a 39.3% reduction in average application compute cost per pipeline and a 15.8% reduction in active pipeline duration. ### Source excerpt CI optimization is easiest to reason about when the problem is concrete: one pipeline, one critical path, and one cost model. Lawmatics reached out to us with that kind of problem. Their application pipeline was already parallelized and already using a sensible CI structure. The remaining question was whether the most expensive part of the [...] The post How Lawmatics Cut CI Compute Cost by 39.3% and Shortened Pipeline Time by 15.8% appeared first on Semaphore. ## Building an AI Agent for LinkedIn Scraping & Candidate Sourcing DevFeed: [Building an AI Agent for LinkedIn Scraping & Candidate Sourcing]() Original publisher: [Read original article]() Author: Pranshu Raj Published: 2026-07-23T00:00:00Z Content type: tutorial Language: en Sources: [Pranshu Raj - blog on backend systems, performance and sidequests]() Topics: [AI Agent](), [Artificial Intelligence](), [FastAPI](), [Python](), [Large Language Model](), [GitHub](), [data]() Tags: [agent](), [ai](), [ai-agent](), [async](), [fastapi](), [github](), [llms](), [pipeline](), [python](), [recruiting](), [scraping]() ### AI overview A developer describes building an AI-powered LinkedIn candidate-sourcing agent for a hackathon. The Python and FastAPI application combines LinkedIn and GitHub profile discovery, six-factor candidate scoring, and Llama-powered personalized outreach, using external services for search and LinkedIn data retrieval. ### Source excerpt How I built an AI agent that scrapes LinkedIn, scores candidates, and generates personalized outreach, a full recruiting sourcing pipeline in Python and FastAPI. ## Drag, drop, done: a visual composer for Redpanda Connect DevFeed: [Drag, drop, done: a visual composer for Redpanda Connect]() Original publisher: [Read original article]() Author: Prakhar Garg Published: 2026-07-23T00:00:00Z Content type: article Language: en Sources: [Redpanda]() Topics: [Redpanda-Connect](), [Canvas](), [stream-processing](), [Caching](), [Routing (disambiguation)](), [HTTP](), [n8n](), [workflow automation]() Tags: [api](), [automation](), [building](), [cache](), [caching](), [events](), [front-end](), [http](), [n8n](), [pipeline](), [redpanda-connect](), [routing](), [stream-processing](), [workflow]() ### AI overview Redpanda introduces a visual Pipeline Builder for Redpanda Connect, using a drag-and-drop canvas to compose pipelines from inputs, processors, and outputs. The builder generates YAML while making routing, enrichment, caching, error handling, and retries easier to understand and configure. The article contrasts it with n8n, emphasizing Redpanda Connect's suitability for high-volume event streaming with ordering, backpressure, and exactly-once semantics. ### Source excerpt A first look at the new visual Pipeline Builder for Redpanda Connect (preview), plus a roundup of recent CDC and connector updates. ## Why I needed Durable Execution to read a toy manual DevFeed: [Why I needed Durable Execution to read a toy manual]() Original publisher: [Read original article]() Author: Shy Ruparel Published: 2026-07-13T00:00:00Z Content type: opinion Language: en Sources: [Temporal Blog]() Topics: [Artificial Intelligence](), [Software](), [Python]() Tags: [ai](), [cleanup](), [japanese](), [ocr](), [pipeline](), [retries](), [temporal-voices](), [translation]() ### AI overview Shy Ruparel describes building Toku Solutions, an AI pipeline that translates Japanese collectible toy manuals into editable static sites. Durable Execution handles OCR, retries, and cleanup so processing does not have to restart from the beginning. ### Source excerpt Shy Ruparel built an AI pipeline to translate Japanese toy manuals. Durable Execution keeps OCR, retries, and cleanup from starting over. ## How Rapidflare built a million+ document ingestion pipeline for Agents on Temporal DevFeed: [How Rapidflare built a million+ document ingestion pipeline for Agents on Temporal]() Original publisher: [Read original article]() Author: Vasanth Asokan Published: 2026-07-08T00:00:00Z Content type: article Language: en Sources: [Temporal Blog]() Topics: [Agent Harness](), [Documentation](), [reliability](), [data](), [Graphs]() Tags: [agent-harness](), [agents](), [community](), [knowledge-graph](), [pipeline](), [pipelines](), [reliability](), [scale](), [technical]() ### AI overview Rapidflare explains how it uses Temporal to build durable, observable document-ingestion pipelines for technical sales agents processing knowledge bases at million-document scale. The article describes the need to ingest and pre-structure extensive technical literature into proprietary knowledge-graph formats, while preserving completeness and reliability. ### Source excerpt Rapidflare explains how it uses Temporal to run durable, observable document ingestion pipelines for technical sales Agents at million-document scale. ## Stream Oracle changes to ClickHouse in real time DevFeed: [Stream Oracle changes to ClickHouse in real time]() Original publisher: [Read original article]() Author: Prakhar Garg Published: 2026-07-07T00:00:00Z Content type: tutorial Language: en Sources: [Redpanda]() Topics: [clickhouse](), [Redpanda-Connect](), [Oracle Database](), [Kafka](), [real-time](), [data](), [Database](), [YAML]() Tags: [clickhouse](), [connectors](), [data](), [database](), [event](), [integration](), [kafka](), [oracle](), [pipeline](), [real-time](), [redpanda-connect](), [schema](), [yaml]() ### AI overview A hands-on walkthrough shows how to stream Oracle change data capture events through Redpanda Connect into ClickHouse in real time. It uses Oracle LogMiner and YAML-configured pipelines, with a Redpanda topic serving as an intermediate store before data is delivered to ClickHouse or other destinations. ### Source excerpt A hands-on walkthrough of the new Oracle input in Redpanda Connect. No Debezium, no Kafka Connect runtime, no JVM. ## Real-time Change Data Capture with Redpanda Connect and MySQL DevFeed: [Real-time Change Data Capture with Redpanda Connect and MySQL]() Original publisher: [Read original article]() Author: Chandler Mayo Published: 2026-07-02T00:00:00Z Content type: tutorial Language: en Sources: [Redpanda]() Topics: [MySQL](), [Kafka](), [Databases](), [Docker Compose](), [real-time](), [Docker](), [SQL]() Tags: [apache-kafka](), [connectors](), [container](), [docker](), [event](), [how-to](), [kafka](), [mysql](), [network](), [pipeline](), [real-time](), [sql](), [tutorial](), [volume]() ### AI overview A step-by-step tutorial for building a real-time change data capture pipeline with MySQL and Redpanda Connect. It explains how row-level inserts, updates, and deletes become downstream events without polling, using compatibility with the Kafka Connect ecosystem. ### Source excerpt A step-by-step tutorial on how to stream every insert, update, and delete from your database using MySQL and a faster, simpler alternative to Kafka Connect. [Next page]()