# Fix evaluation bias to unlock better models on Amazon Bedrock

DevFeed: [Fix evaluation bias to unlock better models on Amazon Bedrock](<https://devfeed.tech/articles/fix-evaluation-bias-to-unlock-better-models-on-amazon-bedrock-102789.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/fix-evaluation-bias-to-unlock-better-models-on-amazon-bedrock/>)

Author: Khaled Thabit

Published: 2026-10-27T16:53:02Z

Content type: article

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Ground truth / benchmark quality](<https://devfeed.tech/topics/ground-truth-benchmark-quality.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [Amazon Bedrock](<https://devfeed.tech/topics/amazon-bedrock.md>), [vlm](<https://devfeed.tech/topics/vlm.md>), [Dataset development](<https://devfeed.tech/topics/dataset-development.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-evaluation](<https://devfeed.tech/tags/ai-evaluation.md>), [amazon-bedrock](<https://devfeed.tech/tags/amazon-bedrock.md>), [bias](<https://devfeed.tech/tags/bias.md>), [cost](<https://devfeed.tech/tags/cost.md>), [customer-solutions](<https://devfeed.tech/tags/customer-solutions.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [image](<https://devfeed.tech/tags/image.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [prompt-optimization](<https://devfeed.tech/tags/prompt-optimization.md>), [property](<https://devfeed.tech/tags/property.md>), [quality](<https://devfeed.tech/tags/quality.md>), [vision](<https://devfeed.tech/tags/vision.md>)

## AI overview

Property Finder found that 96.8% of its evaluation labels came from its incumbent model, biasing comparisons against alternatives. It replaced those labels with independent expert annotations for 356 UAE images, used multi-label classification for overlapping room types, and tuned prompts through error analysis and iteration. On an evaluation set that included the 256 images used for tuning, Llama 4 Maverick 17B Instruct scored Micro F1 0.80 versus 0.56 for the incumbent; the article cautions these results are optimistic. The selected model matched or exceeded the incumbent on all 11 attributes, with equal photo-classification performance. The reported 86% cost reduction is an estimate against the previous setup's cost analysis, and the reported 40-60% latency reduction comes from Property Finder's production monitoring.

## Source excerpt

Property Finder found that 96.8% of their vision-AI evaluation labels were inherited from the incumbent model, hiding its true accuracy. See how they built unbiased ground truth, applied evolutionary prompt optimization, and selected a more accurate, cost-effective model on Amazon Bedrock, cutting cost by 86% across 11 image-quality attributes.