# math-verify

Math-Verify is a mathematical expression evaluation system for assessing large language model outputs, with parsing and verification functionality.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Fixing Open LLM Leaderboard with Math-Verify

DevFeed: [Fixing Open LLM Leaderboard with Math-Verify](<https://devfeed.tech/articles/fixing-open-llm-leaderboard-with-math-verify-7345.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/math_verify_leaderboard>)

Author: Hynek Kydlicek; Alina Lozovskaya; Nathan Habib; Clémentine Fourrier

Published: 2025-02-14T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [math-verify](<https://devfeed.tech/topics/math-verify.md>), [open-llm-leaderboard](<https://devfeed.tech/topics/open-llm-leaderboard.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [math](<https://devfeed.tech/topics/math.md>), [Parser](<https://devfeed.tech/topics/parser.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [evals](<https://devfeed.tech/tags/evals.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [leaderboard](<https://devfeed.tech/tags/leaderboard.md>), [llm](<https://devfeed.tech/tags/llm.md>), [math-verify](<https://devfeed.tech/tags/math-verify.md>), [open-llm-leaderboard](<https://devfeed.tech/tags/open-llm-leaderboard.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [science](<https://devfeed.tech/tags/science.md>)

### AI overview

The article explains how Math-Verify was used to re-evaluate 3,751 models submitted to the Open LLM Leaderboard. It addresses answer-format, symbolic parsing, and comparison problems in the previous MATH-Hard evaluator, leading to substantially revised leaderboard results.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Open R1: Update #2

DevFeed: [Open R1: Update #2](<https://devfeed.tech/articles/open-r1-update-2-7422.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/open-r1/update-2>)

Author: Loubna Ben Allal; Lewis Tunstall; Anton Lozhkov; Elie Bakouch; Guilherme Penedo; Hynek Kydlicek; Gabriel Martín Blázquez

Published: 2025-02-10T16:10:47Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [deepseek](<https://devfeed.tech/topics/deepseek.md>), [synthetic-data](<https://devfeed.tech/topics/synthetic-data.md>), [datasets](<https://devfeed.tech/topics/datasets.md>), [Chain-of-thought](<https://devfeed.tech/topics/chain-of-thought.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [math](<https://devfeed.tech/topics/math.md>), [llama](<https://devfeed.tech/topics/llama.md>), [qwen](<https://devfeed.tech/topics/qwen.md>), [sglang](<https://devfeed.tech/topics/sglang.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [math-verify](<https://devfeed.tech/topics/math-verify.md>), [Parser](<https://devfeed.tech/topics/parser.md>)

Tags: [chain-of-thought](<https://devfeed.tech/tags/chain-of-thought.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [deepseek](<https://devfeed.tech/tags/deepseek.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [llama](<https://devfeed.tech/tags/llama.md>), [math](<https://devfeed.tech/tags/math.md>), [math-verify](<https://devfeed.tech/tags/math-verify.md>), [qwen](<https://devfeed.tech/tags/qwen.md>), [sglang](<https://devfeed.tech/tags/sglang.md>), [synthetic-data](<https://devfeed.tech/tags/synthetic-data.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

### AI overview

Open R1 Update #2 presents OpenR1-Math-220k, a large-scale mathematical reasoning dataset created to help reconstruct parts of the DeepSeek R1 training pipeline and synthetic-data process. The article describes reasoning-trace generation, local inference with vLLM and SGLang, automated filtering with Math Verify, and the use of Llama3.3-70B-Instruct as a judge. It also discusses distillation and fine-tuning of Qwen and Llama models using reasoning traces.

### Source excerpt

We are now two weeks into the Open R1 project which aims to reconstruct the missing pieces of DeepSeek R1--specifically, the training pipeline and synthetic data. In this post, we are happy to share the construction of OpenR1-Math-220k: our first large-scale dataset for mathematical reasoning!