# LAVE: Zero-shot VQA Evaluation on Docmatix with LLMs - Do We Still Need Fine-Tuning?

DevFeed: [LAVE: Zero-shot VQA Evaluation on Docmatix with LLMs - Do We Still Need Fine-Tuning?](<https://devfeed.tech/articles/lave-zero-shot-vqa-evaluation-on-docmatix-with-llms-do-we-still-need-fine-tuning-7574.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/zero-shot-vqa-docmatix>)

Author: Dana Aubakirova; Andres Marafioti

Published: 2024-07-25T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [LLMs](<https://devfeed.tech/topics/llms.md>), [datasets](<https://devfeed.tech/topics/datasets.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [community](<https://devfeed.tech/tags/community.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [llms](<https://devfeed.tech/tags/llms.md>), [metrics](<https://devfeed.tech/tags/metrics.md>), [research](<https://devfeed.tech/tags/research.md>), [synthetic-data](<https://devfeed.tech/tags/synthetic-data.md>), [vlm](<https://devfeed.tech/tags/vlm.md>), [vlms](<https://devfeed.tech/tags/vlms.md>), [vqa](<https://devfeed.tech/tags/vqa.md>), [zero-shot](<https://devfeed.tech/tags/zero-shot.md>)

## AI overview

This article examines zero-shot visual question answering evaluation on the synthetic Docmatix dataset using large language models and vision-language models. It explains why traditional VQA Accuracy can undervalue semantically correct answers in out-of-distribution settings and discusses the trade-off between fine-tuning models and developing metrics that better reflect human judgment.

## Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.