# math-verify

Published articles for math-verify.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Fixing Open LLM Leaderboard with Math-Verify

DevFeed: [Fixing Open LLM Leaderboard with Math-Verify](<https://devfeed.tech/articles/fixing-open-llm-leaderboard-with-math-verify-7345.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/math_verify_leaderboard>)

Author: Hynek Kydlicek; Alina Lozovskaya; Nathan Habib; Clémentine Fourrier

Published: 2025-02-14T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [math-verify](<https://devfeed.tech/topics/math-verify.md>), [open-llm-leaderboard](<https://devfeed.tech/topics/open-llm-leaderboard.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [math](<https://devfeed.tech/topics/math.md>), [Parser](<https://devfeed.tech/topics/parser.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [evals](<https://devfeed.tech/tags/evals.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [leaderboard](<https://devfeed.tech/tags/leaderboard.md>), [llm](<https://devfeed.tech/tags/llm.md>), [math-verify](<https://devfeed.tech/tags/math-verify.md>), [open-llm-leaderboard](<https://devfeed.tech/tags/open-llm-leaderboard.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [science](<https://devfeed.tech/tags/science.md>)

### AI overview

The article explains how Math-Verify was used to re-evaluate 3,751 models submitted to the Open LLM Leaderboard. It addresses answer-format, symbolic parsing, and comparison problems in the previous MATH-Hard evaluator, leading to substantially revised leaderboard results.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Open R1: Update #2

DevFeed: [Open R1: Update #2](<https://devfeed.tech/articles/open-r1-update-2-7422.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/open-r1/update-2>)

Author: Loubna Ben Allal; Lewis Tunstall; Anton Lozhkov; Elie Bakouch; Guilherme Penedo; Hynek Kydlicek; Gabriel Martín Blázquez

Published: 2025-02-10T16:10:47Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [deepseek](<https://devfeed.tech/topics/deepseek.md>), [synthetic-data](<https://devfeed.tech/topics/synthetic-data.md>), [datasets](<https://devfeed.tech/topics/datasets.md>), [Chain-of-thought](<https://devfeed.tech/topics/chain-of-thought.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [math](<https://devfeed.tech/topics/math.md>), [llama](<https://devfeed.tech/topics/llama.md>), [qwen](<https://devfeed.tech/topics/qwen.md>), [sglang](<https://devfeed.tech/topics/sglang.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [math-verify](<https://devfeed.tech/topics/math-verify.md>), [Parser](<https://devfeed.tech/topics/parser.md>)

Tags: [chain-of-thought](<https://devfeed.tech/tags/chain-of-thought.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [deepseek](<https://devfeed.tech/tags/deepseek.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [llama](<https://devfeed.tech/tags/llama.md>), [math](<https://devfeed.tech/tags/math.md>), [math-verify](<https://devfeed.tech/tags/math-verify.md>), [qwen](<https://devfeed.tech/tags/qwen.md>), [sglang](<https://devfeed.tech/tags/sglang.md>), [synthetic-data](<https://devfeed.tech/tags/synthetic-data.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

### AI overview

Open R1 Update #2 presents OpenR1-Math-220k, a large-scale mathematical reasoning dataset created to help reconstruct parts of the DeepSeek R1 training pipeline and synthetic-data process. The article describes reasoning-trace generation, local inference with vLLM and SGLang, automated filtering with Math Verify, and the use of Llama3.3-70B-Instruct as a judge. It also discusses distillation and fine-tuning of Qwen and Llama models using reasoning traces.

### Source excerpt

We are now two weeks into the Open R1 project which aims to reconstruct the missing pieces of DeepSeek R1--specifically, the training pipeline and synthetic data. In this post, we are happy to share the construction of OpenR1-Math-220k: our first large-scale dataset for mathematical reasoning!