# VLMs Published articles for VLMs. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Building Federated Multimodal AI Workflows with NVIDIA FLARE DevFeed: [Building Federated Multimodal AI Workflows with NVIDIA FLARE]() Original publisher: [Read original article]() Author: Tanya Lenz Published: 2026-08-19T17:50:47Z Content type: article Language: en Sources: [NVIDIA Developer](), [NVIDIA Technical Blog]() Topics: [vlm](), [SDKs]() Tags: [agentic-ai-generative-ai](), [ai](), [data-science](), [edge-computing](), [featured](), [federated-learning](), [multimodal](), [multimodal-ai](), [nvidia](), [python](), [streaming](), [training](), [training-ai-models](), [updates](), [vlms](), [workflows]() ### AI overview The article explains how NVIDIA FLARE supports federated training for multimodal and vision-language models when data remains distributed across sites. It focuses on deciding which model state to exchange and on efficiently transferring and aggregating large updates through externalization, tensor streaming, and disk-backed aggregation. ### Source excerpt Modern vision-language models (VLMs) can support tasks such as visual question answering, captioning, and image-text reasoning. In practice, however, the data... ## MindTopo reveals VLMs' spatial reasoning abilities DevFeed: [MindTopo reveals VLMs' spatial reasoning abilities]() Original publisher: [Read original article]() Author: Yunfei Ge, Anbang Liu, Qineng Wang, Johnalbert Garnica, Zihan Wang, Reuben Tan, Jianfeng Gao, Ruohan Zhang, Yining Hong, Jiajun Wu, Manling Li Published: 2026-08-12T16:00:00Z Content type: article Language: en Sources: [Microsoft Research]() Topics: [Benchmark](), [LLM evaluation / benchmarking](), [multimodal-ai](), [Language models](), [Robotics]() Tags: [ai](), [benchmark](), [large-language-models](), [multimodal](), [reasoning](), [research](), [research-blog](), [testing](), [vlms]() ### AI overview MindTopo is a benchmark for evaluating whether multimodal large language models can understand and manipulate topological relationships such as connectivity, enclosure, order, separation, and knots. It compares static recognition with interactive planning and finds that current models often lose track of structural relationships during sequences of actions. ### Source excerpt A path, a fence, a knot. MindTopo sets a new benchmark for testing how AI understands topological relationships and highlights new opportunities to strengthen spatial reasoning and planning. The post MindTopo reveals VLMs' spatial reasoning abilities appeared first on Microsoft Research. ## Introducing CARE-X: Towards Clinically Useful Radiology VLMs with Auxiliary Supervision, Reward-Aligned Learning, and Tool-Augmented Measurement DevFeed: [Introducing CARE-X: Towards Clinically Useful Radiology VLMs with Auxiliary Supervision, Reward-Aligned Learning, and Tool-Augmented Measurement]() Original publisher: [Read original article]() Author: Mercy Ranjit, Nikhilesh E, Dr. Abhyuday Kumara Swamy, Tanuja Ganu Published: 2026-08-11T16:00:00Z Content type: article Language: en Sources: [Microsoft Research]() Topics: [vlm](), [Artificial Intelligence](), [Reinforcement learning](), [data]() Tags: [ai](), [data](), [generation](), [reasoning](), [reinforcement-learning](), [research](), [research-blog](), [tools](), [vlms]() ### AI overview CARE-X is a research chest X-ray vision-language model that combines free-text report generation, structured diagnostic prediction, and reinforcement learning for multi-task clinical interpretation. The article also describes a separate experiment using deterministic measurement tools with Qwen3-VL-4B-Instruct and reports validation on real-world Indian clinical data, while emphasizing that CARE-X is not approved for clinical use. ### Source excerpt Radiology AI is evolving beyond report generation. CARE-X explores a unified approach that combines flexible reasoning, calibrated predictions, and measurement-based tools for chest X-ray interpretation. The post Introducing CARE-X: Towards Clinically Useful Radiology VLMs with Auxiliary Supervision, Reward-Aligned Learning, and Tool-Augmented Measurement appeared first on Microsoft Research. ## Meta is back with Muse Glimmer: local, agentic, multimodal, and open source DevFeed: [Meta is back with Muse Glimmer: local, agentic, multimodal, and open source]() Original publisher: [Read original article]() Author: Pedro Cuenca; merve; ben burtenshaw; Aritra Roy Gosthipaty Published: 2026-08-10T00:00:00Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [vlm](), [multimodal](), [Artificial Intelligence](), [hugging face](), [Benchmark](), [coding]() Tags: [agentic](), [artificial-intelligence](), [benchmark](), [hub](), [hugging-face](), [images](), [llama-cpp](), [llms](), [local](), [meta](), [model](), [multimodal](), [muse](), [open-source](), [videos](), [vllm](), [vlm](), [vlms]() ### AI overview Hugging Face presents Muse Glimmer, a local, agentic, multimodal, open-source 30B-parameter vision-language model developed with Meta. The article outlines its vision and language architecture, benchmark context, optional speculative decoding for faster generation, and support for both images and videos. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## NVIDIA Ising Enables Fully Automated Quantum Computer Calibration with Enhanced In-Context Learning DevFeed: [NVIDIA Ising Enables Fully Automated Quantum Computer Calibration with Enhanced In-Context Learning]() Original publisher: [Read original article]() Author: Tanya Lenz Published: 2026-07-27T16:00:00Z Content type: article Language: en Sources: [NVIDIA Developer](), [NVIDIA Technical Blog]() Topics: [Ising](), [Quantum Computing](), [vlm](), [Nvidia](), [Open Source](), [Benchmark](), [benchmarking](), [NVFP4](), [DGX Spark]() Tags: [agentic-ai-generative-ai](), [ai-agent](), [benchmark](), [benchmarking](), [data-science](), [dgx-spark](), [featured](), [ising](), [nvfp4](), [nvidia](), [open-source](), [quantum](), [quantum-computing](), [simulation-modeling-design](), [training-ai-models](), [vlm](), [vlms]() ### AI overview NVIDIA Ising Calibration 1.5 is an open-source vision-language model for interpreting quantum-processor diagnostics and recommending calibration actions. The article highlights zero-shot and in-context learning evaluation on QCalEval, plus an NVFP4-quantized version for local deployment. ### Source excerpt NVIDIA Ising Calibration is an open source vision language model (VLM) designed to interpret diagnostic outputs from quantum processors and determine how they... ## How to Run an Autoresearch Workflow with RL Agent Skills and NVIDIA NeMo DevFeed: [How to Run an Autoresearch Workflow with RL Agent Skills and NVIDIA NeMo]() Original publisher: [Read original article]() Author: Tanya Lenz Published: 2026-07-14T16:00:00Z Content type: tutorial Language: en Sources: [NVIDIA Developer](), [NVIDIA Technical Blog]() Topics: [AI Bots](), [debugging]() Tags: [agent-skill](), [agent-skills](), [agentic-ai-generative-ai](), [ai-agent](), [ai-agents](), [codex](), [coding](), [debugging](), [developer-tools-techniques](), [featured](), [gpu](), [how-to](), [machine-learning](), [nemo](), [nvidia](), [python](), [reinforcement-learning](), [research](), [rl](), [synthetic-data-generation](), [training](), [vlms](), [workflow]() ### AI overview A tutorial on running a skill-based autoresearch workflow in which coding AI agents set up, debug, run, monitor, and iterate on reinforcement-learning experiments using NVIDIA NeMo RL and NeMo Gym. ### Source excerpt Coding AI agents are becoming practical operators for long-running machine learning (ML) workflows. They can inspect repositories, set up runtimes, resolve... ## AI Model Co-Design: Hardware-Friendly LLM Design DevFeed: [AI Model Co-Design: Hardware-Friendly LLM Design]() Original publisher: [Read original article]() Author: Elizabeth Goodman Published: 2026-07-10T16:36:02Z Content type: article Language: en Sources: [NVIDIA Developer](), [NVIDIA Technical Blog]() Topics: [AI Chat]() Tags: [agentic-ai-generative-ai](), [ai](), [ai-inference](), [data-center-cloud](), [deployment](), [design](), [developer-tools-techniques](), [developers](), [featured](), [hardware](), [inference-performance](), [latency](), [llm](), [llms](), [model](), [performance](), [training](), [training-ai-models](), [vlms]() ### AI overview A practical primer on designing LLMs for modern hardware while balancing accuracy, throughput, and responsiveness. It explains how context length and latency or throughput goals change the importance of attention, feed-forward layers, and parallelism. ### Source excerpt AI performance comes down to three dimensions: Accuracy: How well the model reasons and produces outputs Throughput: How many tokens per second a... ## ScreenSuite - The most comprehensive evaluation suite for GUI Agents! DevFeed: [ScreenSuite - The most comprehensive evaluation suite for GUI Agents!]() Original publisher: [Read original article]() Author: Amir Mahla; Aymeric Roucher; Thomas Wolf Published: 2025-06-06T00:00:00Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [screensuite](), [gui-agents](), [computer-use](), [Benchmark](), [benchmarking](), [AI Chat](), [Claude]() Tags: [agents](), [artificial-intelligence](), [benchmarking](), [computer-use](), [docker](), [evaluation](), [gui](), [gui-agents](), [open-source](), [screensuite](), [smolagents](), [virtual-machines](), [vlm](), [vlms]() ### AI overview ScreenSuite is an open benchmarking and evaluation suite for GUI agents. It unifies 13 benchmarks covering perception, grounding, single-step actions, and multi-step agentic capabilities for Vision Language Models, with support for remote desktop sandboxes and Ubuntu or Android virtual machines launched in Docker. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## Holo1: New family of GUI automation VLMs powering GUI agent Surfer-H DevFeed: [Holo1: New family of GUI automation VLMs powering GUI agent Surfer-H]() Original publisher: [Read original article]() Author: Mats L Richter; Pierre-Louis Cedoz Published: 2025-06-03T13:27:59Z Content type: release Language: en Sources: [Hugging Face - Blog]() Topics: [Artificial Intelligence](), [computer-use](), [browsers](), [Website](), [Benchmark](), [Transformers](), [hugging face](), [Localization (l10n)]() Tags: [agents](), [ai](), [automation](), [benchmarks](), [browsers](), [cost](), [gui](), [models](), [modular](), [open-source](), [performance](), [transformers](), [vlms](), [web]() ### AI overview Holo1 is an open-source family of Action VLMs for understanding web interfaces and locating UI elements precisely. Holo1-3B and Holo1-7B are released on Hugging Face with the WebClick benchmark; Holo1-7B reports 76.2% average accuracy on common UI localization benchmarks. The article also describes Surfer-H, a browser-based web automation agent built with separate policy, localization, and validation components. ### Source excerpt Surfer-H, a web-native agent that interacts with browsers like a human relies on the Holo1. Holo1 is the first family of open-source Action VLMs designed specifically for deep web UI understanding and precise localization. The family includes Holo1-3B and Holo1-7B models, with the latter achieving 76.2% average accuracy on common UI localization benchmarks--the highest among small-size models. ## The Transformers Library: standardizing model definitions DevFeed: [The Transformers Library: standardizing model definitions]() Original publisher: [Read original article]() Author: Lysandre; Arthur Zucker; Pedro Cuenca; Julien Chaumond Published: 2025-05-15T00:00:00Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [Transformer](), [LLMs](), [interoperability](), [sglang](), [vllm](), [llama.cpp](), [tgi](), [MLX]() Tags: [announcement](), [community](), [interoperability](), [library](), [llama-cpp](), [llms](), [mlx](), [open-source](), [sglang](), [tgi](), [transformers](), [vllm](), [vlms]() ### AI overview The article presents Transformers as a standard library for defining and supporting machine learning model architectures. It describes its broad ecosystem integrations, including training frameworks and inference engines, and highlights interoperability with vLLM, SGLang, TGI, llama.cpp, and MLX. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## Vision Language Models (Better, faster, stronger) DevFeed: [Vision Language Models (Better, faster, stronger)]() Original publisher: [Read original article]() Author: merve; Sergio Paniego; Aritra Roy Gosthipaty; Pedro Cuenca; Andres Marafioti Published: 2025-05-12T00:00:00Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [vlm](), [multimodal](), [Open Source](), [qwen](), [Meta](), [deepseek]() Tags: [architecture](), [community](), [deepseek](), [embeddings](), [multimodal](), [open-source](), [qwen](), [research](), [streaming](), [vision](), [vlm](), [vlms]() ### AI overview This article reviews major developments in vision-language models over the past year, including smaller and more capable models, new architectures, reasoning, agency, long-video understanding, multimodal Retrieval Augmented Generation, and multimodal agents. It also explains any-to-any models and discusses examples including Chameleon, Lumina-mGPT, Qwen 2.5 Omni, MiniCPM-o 2.6, and Janus-Pro-7B. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## Introducing AutoRound: Intel's Advanced Quantization for LLMs and VLMs DevFeed: [Introducing AutoRound: Intel's Advanced Quantization for LLMs and VLMs]() Original publisher: [Read original article]() Author: wenhua cheng; Haihao Shen; weiweiz1; Heng Guo; Huang, Tai; Ke Ding; Ilyas Moutawwakil; Marc Sun; Mohamed Mekkouri Published: 2025-04-29T00:00:00Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [quantization](), [Large Language Model](), [LLMs](), [intel](), [GPU](), [hugging face](), [deepseek](), [llama](), [qwen]() Tags: [architectures](), [artificial-intelligence](), [cpu](), [cuda](), [deepseek](), [efficiency](), [gpu](), [hugging-face](), [inference](), [intel](), [language-models](), [llama](), [llms](), [model](), [offline](), [open-source](), [post-training](), [precision](), [quantization](), [qwen](), [vlms]() ### AI overview This article introduces AutoRound, Intel's weight-only post-training quantization method for LLMs and VLMs. It uses signed gradient descent to optimize weight rounding and clipping ranges, supports low-bit formats from INT2 to INT8, and aims to preserve accuracy with efficient quantization. The article describes its model, hardware, export-format, calibration, and tuning-recipe support, including performance on low-bit benchmarks. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## SigLIP 2: A better multilingual vision language encoder DevFeed: [SigLIP 2: A better multilingual vision language encoder]() Original publisher: [Read original article]() Author: Aritra Roy Gosthipaty; merve; Pavel Iakubovskii Published: 2025-02-21T00:00:00Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [Artificial Intelligence](), [computer-use](), [object-detection](), [Google AI](), [Open Source]() Tags: [artificial-intelligence](), [multimodal](), [object-detection](), [open-source](), [performance](), [retrieval](), [train](), [training](), [vision](), [vlms](), [zero-shot]() ### AI overview Google's SigLIP 2 is a multilingual vision-language encoder family that extends SigLIP's sigmoid-loss training with additional objectives for semantic understanding, localization, and dense visual features. The models improve on SigLIP across scales and core capabilities including zero-shot classification, image-text retrieval, and visual representation transfer, with a dynamic-resolution variant for resolution- and aspect-ratio-sensitive tasks. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## PaliGemma 2 Mix - New Instruction Vision Language Models by Google DevFeed: [PaliGemma 2 Mix - New Instruction Vision Language Models by Google]() Original publisher: [Read original article]() Author: merve; Aritra Roy Gosthipaty; Andreas P. Steiner Published: 2025-02-19T00:00:00Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [vlm](), [Fine-tuning](), [Google](), [gemma](), [object-detection](), [datasets]() Tags: [artificial-intelligence](), [datasets](), [demo](), [fine-tuning](), [gemma](), [google](), [llm](), [multimodal](), [object-detection](), [ocr](), [open-source](), [vision](), [vlms]() ### AI overview Google's PaliGemma 2 mix models are fine-tuned on a mixture of vision-language tasks, including OCR, image captioning, visual question answering, document understanding, object detection, and image segmentation. The article explains how the mix models indicate the performance of pretrained PaliGemma 2 checkpoints after fine-tuning and describes their prompting approach. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## We now support VLMs in smolagents! DevFeed: [We now support VLMs in smolagents!]() Original publisher: [Read original article]() Author: Aymeric Roucher; merve; Albert Villanova del Moral Published: 2025-01-24T00:00:00Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [smolagents](), [Artificial Intelligence](), [AI Development](), [web browser](), [document ai]() Tags: [agentic](), [agents](), [artificial-intelligence](), [document-ai](), [multimodal](), [open-source](), [smolagents](), [vision](), [vlm](), [vlms](), [web-browser]() ### AI overview smolagents now supports vision-language models natively in agentic pipelines. The article explains how agents can receive images at initialization or dynamically through memory callbacks, enabling visual web browsing, Document AI workflows, and processing long PDFs with visual elements. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## Visual Document Retrieval Goes Multilingual DevFeed: [Visual Document Retrieval Goes Multilingual]() Original publisher: [Read original article]() Author: Marco Cimolai; Logan Markewich Published: 2025-01-10T00:00:00Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [datasets](), [Embeddings](), [hugging face](), [llamaindex](), [multimodal](), [Open Source](), [Inference](), [data](), [Query (disambiguation)]() Tags: [artificial-intelligence](), [benchmarks](), [cv](), [data](), [datasets](), [embeddings](), [hugging-face](), [inference](), [integrations](), [llamaindex](), [multimodal](), [nlp](), [open-source](), [open-source-collab](), [retrieval](), [synthetic](), [vector](), [vlms]() ### AI overview The article introduces a multilingual embedding model for visual document retrieval and its English-only counterpart. The models encode document page screenshots into dense single-vector representations, enabling visual search across languages without OCR or document-chunking pipelines. The article also presents a 500,000-sample open-source multilingual synthetic dataset, reports faster inference and lower VRAM usage, and describes cross-lingual retrieval and Matryoshka Representation Learning capabilities. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## Introducing TextImage Augmentation for Document Images DevFeed: [Introducing TextImage Augmentation for Document Images]() Original publisher: [Read original article]() Author: Dana Aubakirova; Pablo Montalvo; Vladimir Iglovikov Published: 2024-08-06T00:00:00Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [data augmentation](), [albumentations](), [multimodal](), [Fine-tuning](), [datasets](), [Artificial Intelligence]() Tags: [ai](), [albumentations](), [data-augmentation](), [dataset](), [document-ai](), [fine-tuning](), [multimodal](), [research](), [synthetic-data](), [text-generation](), [training](), [vision](), [vlms]() ### AI overview The article introduces a multimodal data augmentation pipeline for document images used in Vision Language Model fine-tuning. Developed with Albumentations AI, it modifies document images and their text annotations together while aiming to preserve text quality. The methods include text insertion, deletion, swapping, and stopword replacement, followed by image masking and inpainting. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## LAVE: Zero-shot VQA Evaluation on Docmatix with LLMs - Do We Still Need Fine-Tuning? DevFeed: [LAVE: Zero-shot VQA Evaluation on Docmatix with LLMs - Do We Still Need Fine-Tuning?]() Original publisher: [Read original article]() Author: Dana Aubakirova; Andres Marafioti Published: 2024-07-25T00:00:00Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [LLMs](), [datasets](), [Benchmark](), [Artificial Intelligence]() Tags: [benchmark](), [community](), [dataset](), [evaluation](), [fine-tuning](), [llms](), [metrics](), [research](), [synthetic-data](), [vlm](), [vlms](), [vqa](), [zero-shot]() ### AI overview This article examines zero-shot visual question answering evaluation on the synthetic Docmatix dataset using large language models and vision-language models. It explains why traditional VQA Accuracy can undervalue semantically correct answers in out-of-distribution settings and discusses the trade-off between fine-tuning models and developing metrics that better reflect human judgment. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## Preference Optimization for Vision Language Models DevFeed: [Preference Optimization for Vision Language Models]() Original publisher: [Read original article]() Author: Quentin Gallouédec; Shengyi Costa Huang; merve; Kashif Rasul Published: 2024-07-10T00:00:00Z Content type: tutorial Language: en Sources: [Hugging Face - Blog]() Topics: [vlm](), [Language models]() Tags: [data](), [dpo](), [fine-tuning](), [gpu](), [images](), [language-models](), [multimodal](), [optimization](), [rlhf](), [training](), [trl](), [vlm](), [vlms]() ### AI overview A tutorial on training vision-language models with TRL's direct preference optimization support, covering preference data formatting and memory considerations. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## Going multimodal: How Prezi is leveraging the Hub and the Expert Support Program to accelerate their ML roadmap DevFeed: [Going multimodal: How Prezi is leveraging the Hub and the Expert Support Program to accelerate their ML roadmap]() Original publisher: [Read original article]() Author: Violette; Jeff Boudier; Moritz Borrett-Laurer (formerly Laurer); Máté Börcsök Published: 2024-06-19T00:00:00Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [AI, ML & Data Engineering](), [AI search](), [Language models]() Tags: [case-studies](), [expert-support](), [images](), [language-models](), [llm](), [machine-learning](), [multimodal](), [open-source](), [search](), [vision](), [vlms]() ### AI overview Prezi describes using Hugging Face expert support to improve its multimodal machine-learning workflow for generating presentation drafts. The workflow combines vision, text, and vision-language models, and uses an open-source re-ranker to select presentation assets. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science.