# Games as Model Eval: 1-Click Deploy AI Town on Fly.io

DevFeed: [Games as Model Eval: 1-Click Deploy AI Town on Fly.io](<https://devfeed.tech/articles/games-as-model-eval-1-click-deploy-ai-town-on-fly-io-1698.md>)

Original publisher: [Read original article](<https://fly.io/blog/games-as-model-eval/>)

Published: 2025-08-11T00:00:00Z

Content type: opinion

Language: en

Sources: [The Fly Blog](<https://devfeed.tech/sources/the-fly-blog.md>)

Topics: [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Kaggle](<https://devfeed.tech/topics/kaggle.md>), [fly.io](<https://devfeed.tech/topics/fly-io.md>)

Tags: [ai-models](<https://devfeed.tech/tags/ai-models.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cdn](<https://devfeed.tech/tags/cdn.md>), [close-to-users](<https://devfeed.tech/tags/close-to-users.md>), [deploy-app-servers](<https://devfeed.tech/tags/deploy-app-servers.md>), [docker](<https://devfeed.tech/tags/docker.md>), [elixir](<https://devfeed.tech/tags/elixir.md>), [eval](<https://devfeed.tech/tags/eval.md>), [fly](<https://devfeed.tech/tags/fly.md>), [fly-io](<https://devfeed.tech/tags/fly-io.md>), [games](<https://devfeed.tech/tags/games.md>), [heroku-alternative](<https://devfeed.tech/tags/heroku-alternative.md>), [heroku-competitor](<https://devfeed.tech/tags/heroku-competitor.md>), [hosting](<https://devfeed.tech/tags/hosting.md>), [i](<https://devfeed.tech/tags/i.md>), [networking](<https://devfeed.tech/tags/networking.md>), [postgresql-clusters](<https://devfeed.tech/tags/postgresql-clusters.md>), [servers](<https://devfeed.tech/tags/servers.md>)

## AI overview

The article argues that games can make AI model evaluation more rigorous and engaging. It highlights the limits of conventional benchmarks and subjective output comparisons, points to Google's Kaggle Game Arena, and presents game environments as tests of strategic reasoning, long-term planning, and dynamic adaptation.

## Source excerpt

Recently, I suggested that The Future Isn't Model Agnostic, that it's better to pick one model that works for your project and build around it, rather than engineering for model flexibility. If you buy that, you also have to acknowledge how important comprehensive model evaluation becomes. Benchmarks tell us almost nothing about how a model will actually behave in the wild, especially with long contexts, or when trusted to deliver the tone and feel that defines the UX we're shooting for. Even the best evaluation pipelines usually end in subjective, side-by-side output comparisons. Not especially rigorous, and more importantly, boring af. Can we gamify model evaluation? Oh yes. And not just because we get to have some fun for once. Google backed me up this week when it announced the Kaggle Game Arena. A public platform where we can watch AI models duke it out in a variety of classic games. Quoting Google; "Current AI benchmarks are struggling to keep pace with modern models... it can be hard to know if models trained on internet data are actually solving problems or just remembering answers they've already seen." When models boss reading comprehension tests, or ace math problems, we pay attention. But when they fail to navigate a simple conversation with a virtual character or completely botch a strategic decision in a game environment, we tell ourselves we're not building a game anyway and develop strategic short-term memory loss. Just like I've told my mom a thousand times, games are great at testing brains, and it's time we take this seriously when it comes to model evaluation. Why Games Don't Lie Games provide what benchmarks can't, "a clear, unambiguous signal of success." They give us observable behavior in dynamic environments, the kind that would be extremely difficult (and tedious) to simulate with prompt engineering alone. Games force models to demonstrate the skills we actually care about; strategic reasoning, long-term planning, and dynamic adaptation in in