# multimodal-ai Published articles for multimodal-ai. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Building Pinterest's VLM Serving Stack on NVIDIA Dynamo DevFeed: [Building Pinterest's VLM Serving Stack on NVIDIA Dynamo]() Original publisher: [Read original article]() Author: Pinterest Engineering Published: 2026-09-10T23:08:16Z Content type: article Language: en Sources: [Pinterest Engineering Blog - Medium]() Topics: [vlm](), [Inference](), [model-deployment](), [Orchestration](), [Latency](), [Caching](), [Open Source Models & Datasets]() Tags: [blackwell](), [cache](), [dynamo](), [embeddings](), [engineering](), [inference](), [latency](), [llm](), [multimodal](), [multimodal-ai](), [nvidia](), [nvidia-blackwell](), [open-source](), [orchestration](), [performance](), [pinterest](), [vllm](), [vlm](), [vlm-serving]() ### AI overview Pinterest describes its VLM serving stack built on NVIDIA Blackwell GPUs and NVIDIA Dynamo. The stack addresses multimodal inference demands such as image processing, variable prefill costs, KV-cache pressure, routing, and cache offloading. ### Source excerpt Lei Pan | Senior Software Engineer; Salina Wu | Senior Software Engineer; Cristian Lopez | Software Engineer I; Guangtong Bai | Staff Software Engineer; Soam Acharya | Principal Engineer; Saurabh Vishwas Joshi | Principal Engineer; Chia-Wei Chen | Staff Software Engineer; Ambud Sharma | Principal Engineer Why VLM Serving Matters at Pinterest Pinterest is a visual search and discovery platform, so its AI systems must reason over both language and visual content. Vision-language models (VLMs), which can interpret images, compare visual candidates, and respond naturally to user intent, are becoming the foundation for the next generation of Pinterest experiences: Pinterest Assistant, hybrid search, multimodal reranking, content understanding, signal generation, content safety, and more. This direction also reflects Pinterest's broader strategy to customize open-source models to meet its product & scale needs. Pinterest Assistant is a standout example. This multi-turn conversational experience covers both user language and visual content. Serving it requires low-latency VLM inference over rich multimodal context as well as reworking Qwen3-VL with proprietary multimodal embeddings to cut runtime cost while improving performance. Serving VLMs, however, introduces more challenges compared to text-only LLM workloads. Requests may carry multiple images, require extra vision encoder computation, incur larger and more variable prefill cost, and create higher KV cache pressure. To support this new class of models & product experiences, we built Pinterest's VLM serving stack on top of NVIDIA Blackwell GPUs and NVIDIA Dynamo. Blackwell GPUs incorporate many architectural innovations that are uniquely positioned for today's most demanding AI workloads -- including higher BF16/FP8 compute throughput, increased memory bandwidth, and larger HBM memory capacity -- that enable dramatically higher performance for inference. Dynamo provides a distributed inference orchestration layer that g ## Building Federated Multimodal AI Workflows with NVIDIA FLARE DevFeed: [Building Federated Multimodal AI Workflows with NVIDIA FLARE]() Original publisher: [Read original article]() Author: Tanya Lenz Published: 2026-08-19T17:50:47Z Content type: article Language: en Sources: [NVIDIA Developer](), [NVIDIA Technical Blog]() Topics: [vlm](), [SDKs]() Tags: [agentic-ai-generative-ai](), [ai](), [data-science](), [edge-computing](), [featured](), [federated-learning](), [multimodal](), [multimodal-ai](), [nvidia](), [python](), [streaming](), [training](), [training-ai-models](), [updates](), [vlms](), [workflows]() ### AI overview The article explains how NVIDIA FLARE supports federated training for multimodal and vision-language models when data remains distributed across sites. It focuses on deciding which model state to exchange and on efficiently transferring and aggregating large updates through externalization, tensor streaming, and disk-backed aggregation. ### Source excerpt Modern vision-language models (VLMs) can support tasks such as visual question answering, captioning, and image-text reasoning. In practice, however, the data... ## BIY: Preparing a Dataset and Benchmarking AI Models for Scatterplot-Related Tasks DevFeed: [BIY: Preparing a Dataset and Benchmarking AI Models for Scatterplot-Related Tasks]() Original publisher: [Read original article]() Author: João Palmeiro Published: 2026-01-19T14:52:55Z Content type: article Language: en Sources: [Feedzai]() Topics: [Accessibility](), [LLM evaluation / benchmarking](), [multimodal-ai](), [Canvas](), [AI Models](), [dataset](), [SVG]() Tags: [accessibility](), [ai-models](), [artificial-intelligence](), [benchmarking](), [canvas](), [clustering](), [data-visualization](), [dataset](), [large-language-models](), [multimodal](), [multimodal-ai](), [outlier-detection](), [svg]() ### AI overview The article introduces Benchmark It Yourself (BIY), an effort to prepare a dataset and benchmark multimodal AI models for scatterplot-related tasks. It examines generating distribution-aware descriptions from scatterplot images to improve the accessibility of canvas charts, including direct description generation and structured-data extraction for predefined templates. Initial results were mixed. ### Source excerpt Benchmark It Yourself (BIY): Preparing a Dataset and Benchmarking AI Models for Scatterplot-Related Tasks When we need to visualize and interact with millions, or even just thousands, of individual points while analyzing data, we typically resort to rendering them in the browser using a canvas. The other common approach for the web, SVG, doesn't scale when the number of individual elements increases to such quantities. However, while solving one problem, canvas charts introduce a new challenge: accessibility. Although SVG charts are not accessible by default, they can be by design. Each part of an SVG chart has a corresponding element on the web page, allowing for a programmable, accessible experience for screen reader users. We can simply think of SVG as HTML. On the other hand, a canvas chart is just like a PNG image. If a screen reader user tries to learn more about a canvas chart, unless the developer has prepared a detailed description of it, they will just hear the word "image". There's no way to get an idea of what one of these charts represents, let alone extract any insights. For static charts, the solution can be as simple as preparing a description and integrating it into the rendered chart. However, for platforms leveraging dynamic, large datasets, automatically generating these descriptions is not a simple task, especially for charts like scatterplots where data distributions can assume countless forms. At Feedzai, we started exploring ways to generate data distribution-aware descriptions for scatterplots from their respective images using recent multimodal AI models. When the raw data is not available, or the datasets are composed of several thousand or million instances, relying on chart images and these models becomes tempting. This combination has the potential to generate such descriptions and serve them alongside their respective charts, significantly improving the accessibility of canvas charts. That said, we focused on two main directions: using ## StreetReaderAI: Towards making street view accessible via context-aware multimodal AI DevFeed: [StreetReaderAI: Towards making street view accessible via context-aware multimodal AI]() Original publisher: [Read original article]() Published: 2025-10-29T16:38:00Z Content type: article Language: en Sources: [The latest research from Google]() Topics: [Accessibility](), [multimodal-ai](), [Artificial Intelligence](), [Google](), [Image](), [alt text](), [real-time]() Tags: [accessibility](), [ai](), [alt-text](), [audio](), [gaming](), [generative-ai](), [human-computer-interaction-and-visualization](), [machine-intelligence](), [multimodal](), [multimodal-ai](), [natural-language-processing](), [navigation](), [real-time](), [research](), [screen](), [vision](), [voice]() ### AI overview Google Research introduces StreetReaderAI, a proof-of-concept accessible street view prototype that uses context-aware, real-time multimodal AI to interpret streetscape imagery. Designed with blind and sighted accessibility researchers, it provides audio-led exploration, keyboard and voice navigation, directional feedback, movement guidance, and descriptions of nearby places and geographic information. ### Source excerpt Generative AI ## Introducing Mistral AI Studio. DevFeed: [Introducing Mistral AI Studio.]() Original publisher: [Read original article]() Published: 2025-10-24T12:00:00Z Content type: release Language: en Sources: [Mistral AI Blog]() Topics: [AI Platform](), [Artificial Intelligence](), [MLOps](), [Security](), [DevOps](), [data](), [Large Language Model](), [Deployment](), [Prompt Engineering](), [on-prem](), [VPC]() Tags: [agents](), [ai](), [ai-adoption](), [ai-assistants](), [ai-platform](), [ai-studio](), [autonomous-agents](), [compliance](), [infrastructure](), [multimodal-ai](), [on-prem](), [privacy](), [provenance](), [vpc](), [workflows]() ### AI overview Mistral introduces AI Studio, an enterprise AI platform for customizing, fine-tuning, evaluating, governing, and deploying AI assistants, autonomous agents, and multimodal AI with open models. It addresses the gap between AI prototypes and production by providing evaluation, feedback and dataset workflows, provenance and versioning, governance, and flexible deployment across hybrid, VPC, and on-prem infrastructure. ### Source excerpt The most powerful AI platform for enterprises. Customize, fine-tune, and deploy AI assistants, autonomous agents, and multimodal AI with open models. ## Image and audio models from fal now available on DigitalOcean DevFeed: [Image and audio models from fal now available on DigitalOcean]() Original publisher: [Read original article]() Author: Grace Morgan Published: 2025-10-23T12:30:00Z Content type: release Language: en Sources: [DigitalOcean]() Topics: [AI Platform](), [Digital Ocean](), [multimodal-ai](), [API](), [stable-diffusion](), [flux](), [Serverless](), [voice ai]() Tags: [ai-platform](), [api](), [audio](), [digitalocean](), [flux](), [multimodal](), [multimodal-ai](), [product-updates](), [serverless](), [stable-diffusion]() ### AI overview DigitalOcean announces four multimodal AI models from fal in public preview on the Gradient AI Platform through Serverless Inference. The models support image generation, audio generation, and multilingual text-to-speech through an API. ### Source excerpt We're excited to announce the launch of four multimodal AI models from fal on the DigitalOcean Gradient™ AI Platform, now available in public preview through Serverless Inference. These models allow you to generate images and audio directly via API, without worrying about infrastructure, scaling, or vendor management. With this release, building AI-powered applications that include visual and audio content is easier than ever. Explore the new models The fal models, now in public preview, cover a variety of modalities, enabling you to experiment, prototype, and deploy multimodal AI features quickly: Image generation: Stable Diffusion XL fast (fal-ai/fast-sdxl) - High-resolution image generation FLUX.1 (schnell) (fal-ai/flux/schnell) - Fast image generation for quick prototyping Audio generation: Stable Audio (fal-ai/stable-audio-25/text-to-audio) - Convert text into natural-sounding audio ElevenLabs TTS Multilingual v2 9 (fal-ai/elevenlabs/tts/multilingual-v2) - Multilingual text-to-speech These models are available via Serverless Inference, letting you generate images and audio through the same simple API-driven workflow you already use on Gradient AI Platform. Try it out You can start using these models through the Serverless Inference API (https://inference.do-ai.run) after opting in to the public preview in the DigitalOcean console. Here's a quick look at how to interact with them: First, opt in to the public preview to access the fal models on the Gradient AI Platform. Once opting in, it should take about 10 to 15 minutes for your access to be granted. Example: Generate an Image export MODEL_ACCESS_KEY="YOUR_KEY" curl -sS -X POST 'https://inference.do-ai.run/v1/async-invoke' \ -H "Authorization: Bearer $MODEL_ACCESS_KEY" \ -H "Content-Type: application/json" \ -d '{ "model_id": "fal-ai/flux/schnell", "input": { "prompt": "A high-quality photo of a futuristic city at sunset" } }' Example: Generate an Image with Customized Parameters export MODEL_ACCESS_KEY="YOUR_ ## TimeScope: How Long Can Your Video Large Multimodal Model Go? DevFeed: [TimeScope: How Long Can Your Video Large Multimodal Model Go?]() Original publisher: [Read original article]() Author: Orr Zohar; Rui Li; Andres Marafioti; Xiaohan Wang Published: 2025-07-23T00:00:00Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [Benchmark](), [vlm](), [long-context](), [multimodal-ai](), [hugging face]() Tags: [artificial-intelligence](), [benchmark](), [datasets](), [hugging-face](), [language-models](), [long-context](), [measurement](), [multimodal](), [multimodal-ai](), [open-source](), [reasoning](), [retrieval](), [video]() ### AI overview TimeScope is an open-source benchmark for evaluating how vision-language models understand long videos. It inserts short video clips into videos ranging from one minute to eight hours and measures localized retrieval, information synthesis, and fine-grained temporal perception. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## Mistral Compute DevFeed: [Mistral Compute]() Original publisher: [Read original article]() Published: 2025-06-11T11:30:00Z Content type: article Language: en Sources: [Mistral AI Blog]() Topics: [AI Infrastructure](), [AI Platform](), [Sovereign AI](), [GPU](), [Orchestration](), [Hardware](), [Tooling](), [Frontier AI]() Tags: [agents](), [ai](), [ai-infrastructure](), [ai-platform](), [autonomous-agents](), [frontier-ai](), [gpu](), [hardware](), [infrastructure](), [models](), [multimodal-ai](), [open](), [orchestration]() ### AI overview Mistral AI announces Mistral Compute, a private and integrated AI infrastructure offering spanning GPUs, orchestration, APIs, products, and services. Customers can deploy it from bare-metal servers to fully managed PaaS and use it to build, train, and serve AI workloads while retaining control of their environments. ### Source excerpt The most powerful AI platform for enterprises. Customize, fine-tune, and deploy AI assistants, autonomous agents, and multimodal AI with open models. ## Visual Salamandra: Pushing the Boundaries of Multimodal Understanding DevFeed: [Visual Salamandra: Pushing the Boundaries of Multimodal Understanding]() Original publisher: [Read original article]() Author: Iñigo Pikabea; Jaume Lozano Published: 2025-04-11T14:21:56Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [multimodal-ai](), [vlm](), [Artificial Intelligence](), [Large Language Model](), [datasets](), [Embeddings](), [mlp](), [AI Research]() Tags: [ai](), [blog](), [blog-post](), [datasets](), [embeddings](), [grounding](), [hugging-face](), [language](), [llm](), [mlp](), [models](), [multimodal](), [multimodal-ai](), [ocr](), [research](), [training](), [vision](), [vqa]() ### AI overview Visual Salamandra is a multilingual multimodal model built by extending the Salamandra Instructed 7B model with Google's SigLIP image encoder, an MLP projector, and late-fusion techniques. It processes text, images, and videos, with training focused on visual grounding, document understanding, mathematical reasoning, OCR, and European-language coverage. ### Source excerpt A Blog post by Language Technologies Laboratory @ Barcelona Supercomputing Center on Hugging Face ## SmolVLM - small yet mighty Vision Language Model DevFeed: [SmolVLM - small yet mighty Vision Language Model]() Original publisher: [Read original article]() Author: Andres Marafioti; merve; Miquel Farré; Elie Bakouch; Pedro Cuenca Published: 2024-11-26T00:00:00Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [smolvlm](), [vlm](), [multimodal-ai](), [Transformers](), [Fine-tuning](), [Inference](), [Open Source Models & Datasets](), [GPU]() Tags: [apache](), [fine-tuning](), [gpu](), [inference](), [llm](), [multimodal](), [multimodal-ai](), [nlp](), [on-device](), [open](), [open-source](), [smolvlm](), [transformers](), [trl](), [vision]() ### AI overview This article introduces SmolVLM, a family of small, fast, memory-efficient 2B vision-language models released fully open source under the Apache 2.0 license. It describes the model variants, architecture, training resources, Transformers integration, demo, fine-tuning script, and efficient local or on-device deployment. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science.