# Evaluating Audio Reasoning with Big Bench Audio

DevFeed: [Evaluating Audio Reasoning with Big Bench Audio](<https://devfeed.tech/articles/evaluating-audio-reasoning-with-big-bench-audio-7128.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/big-bench-audio-release>)

Author: Micah Hill-Smith; George Cameron

Published: 2024-12-20T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [ai observability](<https://devfeed.tech/topics/ai-observability.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>)

Tags: [analysis](<https://devfeed.tech/tags/analysis.md>), [arena](<https://devfeed.tech/tags/arena.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [audio](<https://devfeed.tech/tags/audio.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [collaboration](<https://devfeed.tech/tags/collaboration.md>), [community](<https://devfeed.tech/tags/community.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [gemini](<https://devfeed.tech/tags/gemini.md>), [gpt](<https://devfeed.tech/tags/gpt.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [leaderboard](<https://devfeed.tech/tags/leaderboard.md>), [models](<https://devfeed.tech/tags/models.md>), [performance](<https://devfeed.tech/tags/performance.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [research](<https://devfeed.tech/tags/research.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-to-speech](<https://devfeed.tech/tags/speech-to-speech.md>), [synthetic](<https://devfeed.tech/tags/synthetic.md>), [testing](<https://devfeed.tech/tags/testing.md>), [text-to-speech](<https://devfeed.tech/tags/text-to-speech.md>)

## AI overview

Big Bench Audio is a 1,000-question dataset for evaluating audio language-model reasoning across speech and text configurations. Initial results report a speech reasoning gap for GPT-4o between text-only and speech-to-speech performance.

## Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.