# The Open Evaluation Standard: Benchmarking NVIDIA Nemotron 3 Nano with NeMo Evaluator

DevFeed: [The Open Evaluation Standard: Benchmarking NVIDIA Nemotron 3 Nano with NeMo Evaluator](<https://devfeed.tech/articles/the-open-evaluation-standard-benchmarking-nvidia-nemotron-3-nano-with-nemo-evaluator-7393.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/nvidia/nemotron-3-nano-evaluation-recipe>)

Author: Seph Mard; Isabel Hulseman; Besmira Nushi; Piotr Januszewski; Grzegorz Chlebus; VivienneZhang; Wojciech Prazuch; Pablo Ribalta; Nik Spirin; Ferenc Galko

Published: 2025-12-17T13:22:18Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [NeMo](<https://devfeed.tech/topics/nemo.md>), [Nemotron](<https://devfeed.tech/topics/nemotron.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [configuration](<https://devfeed.tech/topics/configuration.md>), [Library](<https://devfeed.tech/topics/library.md>), [Inference](<https://devfeed.tech/topics/inference.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [build](<https://devfeed.tech/tags/build.md>), [configuration](<https://devfeed.tech/tags/configuration.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [inference](<https://devfeed.tech/tags/inference.md>), [library](<https://devfeed.tech/tags/library.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [nemo](<https://devfeed.tech/tags/nemo.md>), [nemotron](<https://devfeed.tech/tags/nemotron.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [open](<https://devfeed.tech/tags/open.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [pipelines](<https://devfeed.tech/tags/pipelines.md>), [reuse](<https://devfeed.tech/tags/reuse.md>), [validation](<https://devfeed.tech/tags/validation.md>), [verify](<https://devfeed.tech/tags/verify.md>)

## AI overview

This article presents an open, reproducible evaluation recipe for NVIDIA Nemotron 3 Nano 30B A3B using the NeMo Evaluator library. It explains how complete configurations, prompts, runtime settings, harness versions, logs, and artifacts enable independent verification and consistent model comparisons across inference backends and infrastructure.

## Source excerpt

NVIDIA released Nemotron 3 Nano 30B A3B with an explicitly open evaluation approach to make that distinction clear. Alongside the model card, we are publishing the complete evaluation recipe used to generate the results, built with the NVIDIA NeMo Evaluator library, so anyone can rerun the evaluation pipeline, inspect the artifacts, and analyze the outcomes independently. We believe that open innovation is the foundation of AI progress.