# DigitalOcean Evaluations: Production Model and Router Testing for the Inference Stack

DevFeed: [DigitalOcean Evaluations: Production Model and Router Testing for the Inference Stack](<https://devfeed.tech/articles/digitalocean-evaluations-production-model-and-router-testing-for-the-inference-stack-19920.md>)

Original publisher: [Read original article](<https://www.digitalocean.com/blog/now-available-evaluations>)

Author: Grace Morgan

Published: 2026-07-01T15:41:47Z

Content type: release

Language: en

Sources: [DigitalOcean](<https://devfeed.tech/sources/digitalocean.md>)

Topics: [Digital Ocean](<https://devfeed.tech/topics/digital-ocean.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [Testing](<https://devfeed.tech/topics/testing.md>)

Tags: [configuration](<https://devfeed.tech/tags/configuration.md>), [cost](<https://devfeed.tech/tags/cost.md>), [data](<https://devfeed.tech/tags/data.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [digitalocean](<https://devfeed.tech/tags/digitalocean.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [latency](<https://devfeed.tech/tags/latency.md>), [llm](<https://devfeed.tech/tags/llm.md>), [metric](<https://devfeed.tech/tags/metric.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [performance](<https://devfeed.tech/tags/performance.md>), [pii](<https://devfeed.tech/tags/pii.md>), [product-updates](<https://devfeed.tech/tags/product-updates.md>), [production](<https://devfeed.tech/tags/production.md>), [quality](<https://devfeed.tech/tags/quality.md>), [router](<https://devfeed.tech/tags/router.md>), [testing](<https://devfeed.tech/tags/testing.md>), [token](<https://devfeed.tech/tags/token.md>)

## AI overview

DigitalOcean Evaluations adds production testing for models and inference router configurations in the DigitalOcean Inference Engine. Teams can run LLM-as-a-Judge evaluations on their own prompts and data, compare quality, latency, and cost, and use built-in or custom rubrics across models, imports, and router setups.

## Source excerpt

Choosing the right model or inference router for production means more than reading a leaderboard. It means validating any model or routing configuration on your own data using your prompts and your evaluation criteria before it ever reaches production, and comparing quality, latency, and cost in one place. Evaluations, now available on the DigitalOcean Inference Engine, lets teams validate any model or inference router configuration on their own data before production. Run structured LLM-as-a-Judge evaluations across catalog models, fine-tuned models, BYOM imports, and router setups without stitching together a separate evaluation stack. DigitalOcean Evaluations Capabilities Evaluations provide everything teams need to validate model and router performance before production. LLM-as-a-Judge scoring runs across any candidate in your inference stack and returns per-item scores with judge rationale, plus latency, token, and cost tracking per run. Six pre-built metrics cover the most common evaluation needs out of the box. For teams that need full control: custom rubrics, reusable presets, MCP support, and full dataset management -- all in the same platform as the inference endpoints you use in production. View YouTube video Pre-Built and Custom Rubrics: Score Against Criteria That Match Your Domain The six pre-built metrics, correctness, completeness, faithfulness, PII, toxicity, and bias, cover common evaluation needs. For specialized domains, custom rubrics let teams define their own judge instructions and scoring criteria directly in the judge prompt. The judge evaluates responses against these criteria and returns per-item scores with rationale. Custom rubrics can also adapt the built-in correctness metric to different data formats instead of relying on a default interpretation. Evaluation Presets: Save Configurations and Re-Run Without Rebuilding Without saved configurations, every re-run becomes a rebuild with different judge models, parameters, or prompts, making