# 🇨🇿 BenCzechMark - Can your LLM Understand Czech?

DevFeed: [🇨🇿 BenCzechMark - Can your LLM Understand Czech?](<https://devfeed.tech/articles/benczechmark-can-your-llm-understand-czech-7126.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/benczechmark>)

Author: Martin Fajčík; Hynek Kydlicek; Martin Dočekal; Jan Doležal; Jakub Štětina; Alexander Polok; Zuzana Nevěřilová; Ales Horak; Michal Štefánik; Adam Jirkovský

Published: 2024-10-01T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [LLMs](<https://devfeed.tech/topics/llms.md>), [Human-AI evaluation](<https://devfeed.tech/topics/human-ai-evaluation.md>), [datasets](<https://devfeed.tech/topics/datasets.md>), [Open Source Models & Datasets](<https://devfeed.tech/topics/open-source-models-datasets.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [datasets](<https://devfeed.tech/tags/datasets.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [leaderboard](<https://devfeed.tech/tags/leaderboard.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llms](<https://devfeed.tech/tags/llms.md>), [metrics](<https://devfeed.tech/tags/metrics.md>), [models](<https://devfeed.tech/tags/models.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [performance](<https://devfeed.tech/tags/performance.md>), [research](<https://devfeed.tech/tags/research.md>)

## AI overview

BenCzechMark is an evaluation suite and leaderboard for assessing how well language models handle Czech. It covers 50 tasks across nine categories, using mostly native Czech content and metrics such as accuracy, exact match, AUROC, and word-level perplexity to compare more than 25 open-source models.

## Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.