# Building Smarter Search Products: 3 Steps for Evaluating Search Algorithms

DevFeed: [Building Smarter Search Products: 3 Steps for Evaluating Search Algorithms](<https://devfeed.tech/articles/building-smarter-search-products-3-steps-for-evaluating-search-algorithms-1382.md>)

Original publisher: [Read original article](<https://shopify.engineering/evaluating-search-algorithms>)

Author: Jodi Sloan

Published: 2021-04-01T18:45:00Z

Content type: article

Language: en

Sources: [Shopify Engineering](<https://devfeed.tech/sources/shopify-engineering.md>), [Shopify Engineering - Shopify Engineering](<https://devfeed.tech/sources/shopify-engineering-shopify-engineering.md>)

Topics: [Algorithms, Complexity](<https://devfeed.tech/topics/algorithms-complexity.md>), [Data Science](<https://devfeed.tech/topics/data-science.md>), [Ground truth / benchmark quality](<https://devfeed.tech/topics/ground-truth-benchmark-quality.md>), [Kafka](<https://devfeed.tech/topics/kafka.md>), [Shopify](<https://devfeed.tech/topics/shopify.md>)

Tags: [algorithms](<https://devfeed.tech/tags/algorithms.md>), [data](<https://devfeed.tech/tags/data.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [kafka](<https://devfeed.tech/tags/kafka.md>), [search](<https://devfeed.tech/tags/search.md>), [shopify](<https://devfeed.tech/tags/shopify.md>), [tests](<https://devfeed.tech/tags/tests.md>), [the-challenge](<https://devfeed.tech/tags/the-challenge.md>)

## AI overview

Shopify describes a three-step framework for evaluating experimental search algorithms. The process uses Kafka and annotation to collect data, establishes a ground-truth relevance dataset, and measures algorithm performance through offline and online A/B tests. The article illustrates the approach by comparing Vanilla Pagerank with Query-specific Pagerank.

## Source excerpt

The three-step framework Shopify's Data Science & Engineering team built for evaluating new search algorithms.