# From Problem to Fix in Minutes: Automating Search Relevance Tuning with the Amazon OpenSearch Service Relevance Agent

DevFeed: [From Problem to Fix in Minutes: Automating Search Relevance Tuning with the Amazon OpenSearch Service Relevance Agent](<https://devfeed.tech/articles/from-problem-to-fix-in-minutes-automating-search-relevance-tuning-with-the-amazon-opensearch-service-relevance-agent-104239.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/big-data/from-problem-to-fix-in-minutes-automating-search-relevance-tuning-with-the-amazon-opensearch-service-relevance-agent/>)

Author: Aruna Govindaraju

Published: 2026-10-27T21:31:20Z

Content type: article

Language: en

Sources: [AWS Big Data Blog](<https://devfeed.tech/sources/aws-big-data-blog.md>)

Topics: [Amazon OpenSearch Service](<https://devfeed.tech/topics/amazon-opensearch-service.md>), [opensearch](<https://devfeed.tech/topics/opensearch.md>), [hybrid-search](<https://devfeed.tech/topics/hybrid-search.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [ai](<https://devfeed.tech/tags/ai.md>), [all](<https://devfeed.tech/tags/all.md>), [amazon](<https://devfeed.tech/tags/amazon.md>), [amazon-opensearch-service](<https://devfeed.tech/tags/amazon-opensearch-service.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [better](<https://devfeed.tech/tags/better.md>), [chat](<https://devfeed.tech/tags/chat.md>), [data](<https://devfeed.tech/tags/data.md>), [experiment](<https://devfeed.tech/tags/experiment.md>), [insights](<https://devfeed.tech/tags/insights.md>), [intermediate-200](<https://devfeed.tech/tags/intermediate-200.md>), [leaving](<https://devfeed.tech/tags/leaving.md>), [minutes](<https://devfeed.tech/tags/minutes.md>), [offline](<https://devfeed.tech/tags/offline.md>), [opensearch](<https://devfeed.tech/tags/opensearch.md>), [query](<https://devfeed.tech/tags/query.md>), [relevance](<https://devfeed.tech/tags/relevance.md>), [search](<https://devfeed.tech/tags/search.md>), [service](<https://devfeed.tech/tags/service.md>), [technical-how-to](<https://devfeed.tech/tags/technical-how-to.md>), [with](<https://devfeed.tech/tags/with.md>)

## AI overview

The Amazon OpenSearch Service Search Relevance Agent automates an offline tuning loop: it analyzes User Behavior Insights (UBI) data, proposes a revised query DSL configuration, and uses Search Relevance Workbench (SRW) experiments to compare it with the baseline. Its three skills cover behavior analysis, hypothesis generation, and evaluation; deterministic tools calculate metrics, and the proposed changes and experiment artifacts can be inspected before a person decides whether to deploy. In the example, wireless earbuds searches had a 57.14% CTR (8 clicks from 14 searches); evaluation raised NDCG with minor MAP and Precision tradeoffs. The agent tunes configurations for systematic relevance issues; it does not rewrite queries at runtime or intercept requests. The article requires an OpenSearch Service domain running OpenSearch 3.5 or later, the OpenSearch UI application, and instrumented behavioral records. For hybrid search, SRW also has a separate optimizer for testing normalization, combination, and lexical/neural weight settings.

## Source excerpt

The Amazon OpenSearch Service Search Relevance Agent turns a low NDCG score into a better search query in minutes. Describe the problem through the Ask AI chat, and the agent analyzes your User Behavior Insights data, generates an improved query, and validates it with an offline Search Relevance Workbench experiment, all without leaving the UI.