# How We Built an AI Agent to Clean Up Dead Code After A/B Tests

DevFeed: [How We Built an AI Agent to Clean Up Dead Code After A/B Tests](<https://devfeed.tech/articles/how-we-built-an-ai-agent-to-clean-up-dead-code-after-a-b-tests-26513.md>)

Original publisher: [Read original article](<https://medium.com/engineering-housing/how-we-built-an-ai-agent-to-clean-up-dead-code-after-a-b-tests-a5519af4892e?source=rss----3a69e32e2594---4>)

Author: Aseem Upadhyay

Published: 2026-06-23T10:54:43Z

Content type: article

Language: en

Sources: [Housing.com](<https://devfeed.tech/sources/housing-com.md>)

Topics: [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [experiments](<https://devfeed.tech/topics/experiments.md>), [Code](<https://devfeed.tech/topics/code.md>), [Pull Request](<https://devfeed.tech/topics/pull-request.md>), [context](<https://devfeed.tech/topics/context.md>), [Android](<https://devfeed.tech/topics/android.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [ai-agents-in-action](<https://devfeed.tech/tags/ai-agents-in-action.md>), [automated](<https://devfeed.tech/tags/automated.md>), [code](<https://devfeed.tech/tags/code.md>), [concurrent](<https://devfeed.tech/tags/concurrent.md>), [experiment](<https://devfeed.tech/tags/experiment.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [jira](<https://devfeed.tech/tags/jira.md>), [llm](<https://devfeed.tech/tags/llm.md>)

## AI overview

Housing.com describes building an AI-agent pipeline to help clean up code after A/B tests. The workflow interprets experiment tickets, reports experiment status, and applies instructions to code, while using scripts for deterministic steps and an LLM where judgment is required.

## Source excerpt

Photo by Microsoft Copilot on Unsplash At Housing.com, running product experiments is a continuous cycle. A/B tests go live, collect data, and eventually reach a conclusion. That's the exciting part. Then comes the mundane reality where someone has to clean up the code .i.e. remove a feature flag, promote a winning variant or revert the loser variant and finally raise a change request. Sounds simple? Maybe Is it tedious and quietly expensive? Yes! lifecycle of a taskThe Problem Worth Solving An experiment conclusion ticket typically lands on an engineer's desk looking something like this: Experiment: show_listing_map_widget Platform: Android Result: Negative - revert to control The job of the assigned engineer involves four distinct steps: Find every reference to the flag across the codebase. Delete the losing variant's code path. Trace every side-effect that only existed to support that variant Commit, open a PR, and comment on the Jira ticket. Step 3 is where the trap lies. Be it applying or removing a change, changing all the infrastructure code dependent on it could increase the complexity and risk of creating technical debt. But what if we automated a part of it? The AI Agent Pipelineupdated AI enabled lifecycle The problem statement became simple: Let stakeholders own the trigger. We built two agents to make it happen, 1. to interpret tickets and report experiment status 2. to take the instructions and code. Then came the hard part. Navigating Roadblocks The real complexity lies in building an AI agent that runs autonomously and serves different users across different use cases We found ourselves wrestling with questions we hadn't fully anticipated: How do we optimise on the tokens used per request? How do we handle concurrent requests? How do we ensure that the consistency in the output? So we went looking for answers.. Optimising Tokens per request Not every step needs AI. At each point in the workflow, we asked one question: is this operation deterministic