# Apache Beam for Search: Getting Started by Hacking Time

DevFeed: [Apache Beam for Search: Getting Started by Hacking Time](<https://devfeed.tech/articles/apache-beam-for-search-getting-started-by-hacking-time-1294.md>)

Original publisher: [Read original article](<https://shopify.engineering/apache-beam-for-search-getting-started-by-hacking-time>)

Author: Doug Turnbull

Published: 2021-01-08T15:00:01Z

Content type: tutorial

Language: en

Sources: [Shopify Engineering](<https://devfeed.tech/sources/shopify-engineering.md>), [Shopify Engineering - Shopify Engineering](<https://devfeed.tech/sources/shopify-engineering-shopify-engineering.md>)

Topics: [streaming-data-processing](<https://devfeed.tech/topics/streaming-data-processing.md>), [data-processing](<https://devfeed.tech/topics/data-processing.md>), [AI Bots](<https://devfeed.tech/topics/ai-bots.md>)

Tags: [apache](<https://devfeed.tech/tags/apache.md>), [apis](<https://devfeed.tech/tags/apis.md>), [batch](<https://devfeed.tech/tags/batch.md>), [data-processing](<https://devfeed.tech/tags/data-processing.md>), [event](<https://devfeed.tech/tags/event.md>), [events](<https://devfeed.tech/tags/events.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [search](<https://devfeed.tech/tags/search.md>), [shopify](<https://devfeed.tech/tags/shopify.md>), [spark](<https://devfeed.tech/tags/spark.md>), [statistics](<https://devfeed.tech/tags/statistics.md>), [storage](<https://devfeed.tech/tags/storage.md>), [stream-processing](<https://devfeed.tech/tags/stream-processing.md>), [streaming-data-processing](<https://devfeed.tech/tags/streaming-data-processing.md>), [workflows](<https://devfeed.tech/tags/workflows.md>)

## AI overview

An introduction to using Apache Beam for search-related streaming data processing. It explains how unified batch and streaming workflows can process clickstream data for real-time relevance tuning, and introduces event time, delayed events, and out-of-order data as core challenges.

## Source excerpt

To create relevant search, processing clickstream data is key: you frequently want to promote search results that are being clicked on and purchased, and demote those things users don't love. Typically search systems think of processing clickstream data as a batch job run over historical data, perhaps using a system like Spark.