# post-training Post-training is the set of training stages applied after base pretraining to shape a language model's behavior, including instruction following, preference alignment, and safety shaping before deployment. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## PPO vs GRPO, Simply Explained DevFeed: [PPO vs GRPO, Simply Explained]() Original publisher: [Read original article]() Author: Dr. Ashish Bamania Published: 2026-09-17T11:47:38Z Content type: tutorial Language: en Sources: [Into AI]() Topics: [Large Language Model](), [post-training](), [Reinforcement learning](), [Algorithms]() Tags: [algorithm](), [alignment](), [human-feedback](), [llm](), [llm-training](), [post-training](), [reinforcement-learning](), [training]() ### AI overview A tutorial comparing PPO and GRPO as reinforcement learning algorithms used in LLM post-training. It explains PPO, including RLHF, policy-gradient updates, and clipped token-probability changes intended to keep model behavior close to its previous version. ### Source excerpt A simple lesson on two important LLM post-training algorithms. ## Salesforce Announces Koa Reasoning Model Built on NVIDIA Nemotron 3 Super DevFeed: [Salesforce Announces Koa Reasoning Model Built on NVIDIA Nemotron 3 Super]() Original publisher: [Read original article]() Author: Brian Caulfield Published: 2026-09-15T22:24:34Z Content type: news Language: en Sources: [NVIDIA Blog]() Topics: [Koa](), [Nemotron](), [Artificial Intelligence](), [Nvidia](), [post-training](), [ai safety]() Tags: [agentic](), [agentic-ai](), [ai](), [cloud](), [corporate](), [events](), [model](), [nemotron](), [nvidia](), [nvidia-nemo](), [post-training](), [safety](), [security](), [software]() ### AI overview At Salesforce Dreamforce, NVIDIA CEO Jensen Huang discussed AI infrastructure, safety, and enterprise adoption alongside Salesforce CEO Marc Benioff. The event coincided with the announcement of Koa, Salesforce's first CRM reasoning model, built by post-training NVIDIA Nemotron 3 Super on a proprietary synthetic dataset derived from nearly three decades of enterprise CRM deployments. ### Source excerpt Know everything. Do anything. That was the message NVIDIA founder and CEO Jensen Huang brought to Salesforce Dreamforce Tuesday, joining CEO Marc Benioff onstage in an appearance that coincided with the announcement of Koa -- Salesforce's first CRM reasoning model, built on NVIDIA Nemotron 3 Super. Huang didn't just take the stage. He walked into [...] ## Autonomous LLM post-training with Tunix on TPUs DevFeed: [Autonomous LLM post-training with Tunix on TPUs]() Original publisher: [Read original article]() Author: Wei Wei Published: 2026-09-12T11:04:33.891311Z Content type: article Language: en Sources: [Google Developers Blog]() Topics: [post-training](), [AI Agent](), [Fine-tuning](), [Large Language Model](), [grpo](), [lora](), [gemma](), [Google](), [Reinforcement learning](), [dataset](), [Command-line interface]() Tags: [autonomous](), [cli](), [fine-tuning](), [gemma](), [git](), [google](), [grpo](), [llm](), [lora](), [post-training](), [reinforcement-learning](), [training]() ### AI overview This article presents autofinetune, an autonomous research loop for LLM post-training. Using AI agents and Google's AI stack, including Tunix, Gemma, Cloud TPUs, Antigravity CLI, and Gemini Flash 3.7, it automates supervised fine-tuning and reinforcement learning with GRPO, exploring hyperparameters such as LoRA configurations, learning rates, batch sizes, and rollout settings. ### Source excerpt Imagine going to sleep after writing a single Markdown specification and waking up to find that an A... ## Deploying AI You Control Doesn't Need to be So Hard DevFeed: [Deploying AI You Control Doesn't Need to be So Hard]() Original publisher: [Read original article]() Author: Jeetu Patel Published: 2026-09-10T09:00:50Z Content type: news Language: en Sources: [Cisco Blogs]() Topics: [Artificial Intelligence](), [AI Platforms/Deployment](), [AI Factory](), [Nvidia](), [Critical Infrastructure](), [post-training](), [data](), [Nemotron](), [on-prem](), [Fine-tuning](), [Cloud](), [Benchmark]() Tags: [ai](), [ai-factory](), [ai-infrastructure](), [ai-strategy](), [cisco-cloud-control-framework](), [cisco-secure-ai-factory](), [cloud](), [critical-infrastructure](), [data](), [executive-platform](), [fine-tuning](), [infrastructure](), [model](), [models](), [nemotron](), [nvidia](), [on-prem](), [post-training]() ### AI overview Cisco announces a collaboration with Palantir to deliver Palantir's Ontology for Cybersecurity through Cisco's Secure AI Factory, using NVIDIA as a preferred full-stack foundation for Palantir's Sovereign AI OS. The article argues that enterprise AI decisions should balance intelligence, cost, and control, including custom evaluations, post-training with proprietary data, and deployment in the cloud, at the edge, or on-premises. ### Source excerpt Announcing a collaboration with Palantir to deliver Cisco's Secure AI Factory with NVIDIA as a preferred full-stack foundation for Palantir's Sovereign AI OS. ## On-Policy Distillation, Simply Explained DevFeed: [On-Policy Distillation, Simply Explained]() Original publisher: [Read original article]() Author: Dr. Ashish Bamania Published: 2026-09-09T19:22:38Z Content type: tutorial Language: en Sources: [Into AI]() Topics: [LLM Techniques](), [Large Language Model](), [post-training](), [Reinforcement learning]() Tags: [fine-tuning](), [gpu](), [llms](), [post-training](), [training]() ### AI overview This tutorial explains on-policy distillation (OPD), a post-training method for large language models. A student model generates its own responses, which a stronger teacher scores token by token; the student is trained to reduce the reverse KL divergence between their next-token distributions. The article distinguishes OPD from reinforcement learning, conventional knowledge distillation, and supervised fine-tuning. ### Source excerpt On-Policy Distillation (OPD) has become a popular algorithm for post-training LLMs, and almost all recent open-weight LLMs (Qwen3, GLM-5.3, and Nemotron-Cascade 2) have used it to achieve amazing performance. ## This smart boxing band takes advantage of the new Arduino Nesso N1 DevFeed: [This smart boxing band takes advantage of the new Arduino Nesso N1]() Original publisher: [Read original article]() Author: Arduino Team Published: 2026-09-09T15:36:33Z Content type: article Language: en Sources: [Arduino Blog]() Topics: [Arduino](), [Hardware](), [data](), [real-time](), [Artificial Intelligence](), [post-training]() Tags: [ai](), [arduino](), [boxing](), [boxing-training](), [data](), [hardware](), [imu](), [nesso-n1](), [performance](), [post-training](), [real-time](), [smart-boxing-band](), [training]() ### AI overview An Arduino Nesso N1-based smart boxing band combines punch recognition, biometric tracking, and real-time feedback. It uses a pulse sensor and the board's IMU to relate punch types and movement to heart rate, while logging data for post-training analysis. ### Source excerpt Data is now a huge part of many sports, from F1 racing to football. Presenting that data to fans is secondary to the real purpose: helping teams and athletes maximize performance. No matter what sport you enjoy, you can benefit from that kind of quantified training. Manivannan proved that by building his Smart AI Boxing [...] The post This smart boxing band takes advantage of the new Arduino Nesso N1 appeared first on Arduino Blog. ## Developing Nemotron 3.5 Lightning NVFP4 with QAD Using NVIDIA Model Optimizer DevFeed: [Developing Nemotron 3.5 Lightning NVFP4 with QAD Using NVIDIA Model Optimizer]() Original publisher: [Read original article]() Author: Tanya Lenz Published: 2026-08-17T18:12:48Z Content type: tutorial Language: en Sources: [NVIDIA Developer](), [NVIDIA Technical Blog]() Topics: [Nemotron](), [NVFP4](), [quantization](), [Post-training optimization](), [Nvidia](), [post-training](), [Mamba]() Tags: [agentic-ai-generative-ai](), [benchmarks](), [compute](), [developer-tools-techniques](), [developers](), [edge-computing](), [evaluation](), [featured](), [latency](), [mamba](), [megatron](), [memory](), [model](), [model-optimizer](), [models](), [nemotron](), [nvfp4](), [nvidia](), [open](), [post-training](), [quantization](), [speed](), [training](), [training-ai-models]() ### AI overview This tutorial explains how quantization-aware distillation (QAD) creates the Nemotron 3.5 Lightning NVFP4 checkpoint using NVIDIA Model Optimizer. It covers post-training quantization, teacher-student distillation, and evaluation, showing how QAD can recover accuracy while reducing memory usage and increasing throughput. ### Source excerpt Teams customize their models to hit their targets for latency, speed, memory, and compute. With the open NVIDIA Nemotron family of models, developers can find... ## Beyond VLAs: How World Action Models Reshape Robot Manipulation DevFeed: [Beyond VLAs: How World Action Models Reshape Robot Manipulation]() Original publisher: [Read original article]() Author: Michelle Horton Published: 2026-08-04T16:00:00Z Content type: article Language: en Sources: [NVIDIA Developer](), [NVIDIA Technical Blog]() Topics: [Robotics](), [World models](), [vlm](), [post-training](), [Cosmos](), [Nvidia](), [NVIDIA Research]() Tags: [agentic-ai-generative-ai](), [cosmos](), [featured](), [nvidia](), [nvidia-research](), [physical-ai](), [post-training](), [robot-manipulation](), [robotics](), [simulation-modeling-design](), [thor](), [vlm](), [world-model](), [zero-shot]() ### AI overview The article explains how World Action Models (WAMs) use video world models as backbones for robot policies, addressing the physical-generalization limitations of vision-language-action models. It discusses post-training WAMs into specialized policies and presents NVIDIA Cosmos 3 as a foundation for building them. ### Source excerpt A central challenge in robotics is building policies that generalize beyond the demonstrations they're trained on. A policy that succeeds in a training scene... ## Data Engineering Weekly #281 DevFeed: [Data Engineering Weekly #281]() Original publisher: [Read original article]() Author: Ananth Packkildurai Published: 2026-08-03T12:34:40Z Content type: article Language: en Sources: [Data Engineering Weekly]() Topics: [data-engineering](), [AI, ML & Data Engineering](), [Loop Engineering](), [Artificial Intelligence](), [Large Language Model](), [post-training]() Tags: [agent-harness](), [ai](), [data-engineering](), [feature-engineering](), [genai](), [netflix](), [newsletter](), [post-training](), [weekly]() ### AI overview Data Engineering Weekly #281 covers building data platforms, emerging approaches to AI workflow architecture, data modernization, Netflix's GenRec recommendation system, AI infrastructure modernization, and evaluation practices for generative AI at scale. ### Source excerpt The Weekly Data Engineering Newsletter ## Post-Train NVIDIA Cosmos 3 in One Day Using Agent Skills DevFeed: [Post-Train NVIDIA Cosmos 3 in One Day Using Agent Skills]() Original publisher: [Read original article]() Author: Tanya Lenz Published: 2026-07-14T16:00:00Z Content type: tutorial Language: en Sources: [NVIDIA Developer](), [NVIDIA Technical Blog]() Topics: [Cosmos](), [Agent Skills](), [post-training](), [lora](), [Nvidia]() Tags: [agent-skills](), [agentic-ai-generative-ai](), [ai-agent](), [computer-vision-video-analytics](), [cosmos](), [featured](), [fine-tuning](), [lora](), [physical-ai](), [post-training](), [simulation-modeling-design](), [training-ai-models]() ### AI overview A tutorial on post-training NVIDIA Cosmos 3 Nano for video question answering with coding-agent skills, LoRA, and TAO AutoML configuration sweeps. ### Source excerpt What if autonomous coding AI agents could push your vision reasoning models above 90% accuracy with almost no manual effort? When adapting vision reasoning... ## Introducing OpenRL: A self-hosted post-training API for fine-tuning LLMs DevFeed: [Introducing OpenRL: A self-hosted post-training API for fine-tuning LLMs]() Original publisher: [Read original article]() Author: Google Open Source (noreply@blogger.com) Published: 2026-06-11T18:30:00Z Content type: release Language: en Sources: [Google Open Source Blog]() Topics: [API](), [Fine-tuning](), [Large Language Model](), [Self-hosted](), [post-training](), [Kubernetes](), [Orchestration](), [reliability]() Tags: [api](), [fine-tuning](), [gke](), [kubernetes](), [llms](), [open-source](), [orchestration](), [post-training](), [reinforcement-learning](), [reliability](), [rl](), [self-hosted](), [sft]() ### AI overview This article announces OpenRL, an open-source research preview from GKE Labs. OpenRL is a self-hosted training API for fine-tuning LLMs on a Kubernetes cluster, designed to separate post-training infrastructure from AI research workflows. The article describes potential benefits including concurrent reinforcement-learning jobs, improved GPU utilization, and simpler researcher workflows. ### Source excerpt by Sunil Arora, Shuby Mishra & Chuang Wang, GKE We are pleased to share a research preview of OpenRL, a new open-source project coming out of GKE Labs. OpenRL is a self-hosted training API for fine-tuning LLMs on your own Kubernetes cluster. Why we built it If you look at agentic RL on LLMs, it is incredibly easy to get bogged down in system complexity. To run a single RL loop, you have to coordinate a dozen different things: selecting and cleaning datasets, choosing RL environments, debugging training loops, managing reward signals, handling inference mismatches, allocating hardware, and managing infrastructure. Picture looks something like this: Figure shows an AI researcher and an infrastructure engineer staring at the hurdles in post training along the way to the summit. Each of these is a hard problem. But what makes it more complex is how tightly AI research and infrastructure concerns are mixed together in today's tooling and frameworks. We believe decoupling the infrastructure from AI research can make these problems more tractable so that infrastructure engineers and AI researchers can independently tackle them. We have seen this pattern with Kubernetes where Kubernetes abstracted out the infrastructure and made application developers and SREs life easier. So, can you abstract out post training infrastructure? We believe so and drew huge inspiration/validation from Tinker (from Thinking Machines). The Tinker APIs for post training hit that Goldilocks zone where it hides all the post training infrastructure behind four key APIs: Figure shows high level components and their interaction in a OpenRL based RL workflow So the end result of this abstraction is that AI Researchers get full flexibility on their RL loop and infrastructure engineers can focus on scaling, orchestration, and reliability. OpenRL allows you to run the same training APIs but on your own infrastructure. And this decoupling has other interesting benefits. Sharing GPUs Traditional RL loops ar ## TRL v1.0: Post-Training Library Built to Move with the Field DevFeed: [TRL v1.0: Post-Training Library Built to Move with the Field]() Original publisher: [Read original article]() Author: Quentin Gallouédec; Steven Liu; Pedro Cuenca; Sergio Paniego Published: 2026-03-31T00:00:00Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [trl](), [post-training](), [dpo](), [grpo](), [rlvr](), [Artificial Intelligence](), [Code]() Tags: [announcement](), [artificial-intelligence](), [code](), [dpo](), [grpo](), [open-source](), [post-training](), [reinforcement-learning](), [rlvr](), [software](), [trl](), [v1]() ### AI overview TRL v1.0 presents a post-training library designed to remain useful as algorithms, models, and paradigms change. The article explains the evolution from PPO to DPO-style preference optimization and RLVR methods such as GRPO, emphasizing flexible abstractions and practical comparison over a supposedly permanent design. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## Keep the Tokens Flowing: Lessons from 16 Open-Source RL Libraries DevFeed: [Keep the Tokens Flowing: Lessons from 16 Open-Source RL Libraries]() Original publisher: [Read original article]() Author: Amine Dirhoussi; Quentin Gallouédec; Kashif Rasul; Lewis Tunstall; Edward Beeching; Albert Villanova del Moral; Nouamane Tazi; Leandro von Werra; Sergio Paniego Published: 2026-03-10T00:00:00Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [distributed-training](), [Reinforcement learning](), [distributed-systems](), [Orchestration](), [NCCL](), [post-training](), [lora](), [Mixture of Experts (MoE)](), [grpo](), [Chain-of-thought](), [Open Source Models & Datasets]() Tags: [agentic](), [analysis](), [chain-of-thought](), [distributed-training](), [grpo](), [llm](), [lora](), [mixture-of-experts](), [nccl](), [open-source](), [post-training](), [rl](), [tokens]() ### AI overview This article surveys 16 open-source libraries for asynchronous reinforcement-learning training. It explains how separating inference and training across GPU pools, using rollout buffers, and synchronizing weights asynchronously can reduce training-GPU idle time. The comparison covers orchestration, buffering, weight synchronization, staleness management, partial rollouts, LoRA, and distributed-training backends, highlighting Ray, NCCL broadcasts, limited LoRA support, and distributed MoE as an emerging differentiator. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## 20x Faster TRL Fine-tuning with RapidFire AI DevFeed: [20x Faster TRL Fine-tuning with RapidFire AI]() Original publisher: [Read original article]() Author: Kamran Bigdely; Arun Kumar; Quentin Gallouédec Published: 2025-11-21T00:00:00Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [Fine-tuning](), [trl](), [Large Language Model](), [post-training](), [Multi-GPU](), [Orchestration](), [dashboards](), [GPU](), [data]() Tags: [dashboards](), [data](), [dpo](), [experimentation](), [fine-tuning](), [gpu](), [grpo](), [llm](), [llms](), [logs](), [metrics](), [multi-gpu](), [orchestration](), [post-training](), [rapidfireai](), [trl]() ### AI overview RapidFire AI accelerates LLM fine-tuning and post-training experimentation by running multiple TRL configurations concurrently, including on a single GPU. Its adaptive chunk-based scheduling, live metrics dashboard, multi-GPU orchestration, and interactive controls help teams compare configurations sooner, stop weak runs, and clone promising ones. The article cites internal benchmarks reporting approximately 16-24x higher experimentation throughput than sequential comparison. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## Smol2Operator: Post-Training GUI Agents for Computer Use DevFeed: [Smol2Operator: Post-Training GUI Agents for Computer Use]() Original publisher: [Read original article]() Author: Amir Mahla; merve; Sergio Paniego; Vaibhav Srivastav; Lewis Tunstall Published: 2025-09-23T00:00:00Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [gui-agents](), [computer-use](), [post-training](), [vlm](), [Artificial Intelligence](), [Open Source](), [Benchmark](), [datasets](), [Fine-tuning]() Tags: [agentic](), [agents](), [artificial-intelligence](), [benchmark](), [blog](), [community](), [computer-use](), [datasets](), [fine-tuning](), [gui](), [gui-agents](), [language-models](), [open-source](), [post-training](), [science](), [training](), [vision](), [vlm]() ### AI overview This article presents a multi-phase method for post-training vision-language models to perform GUI automation and computer-use tasks. It covers data processing, action standardization, grounding, supervised fine-tuning, agentic reasoning, and evaluation with the ScreenSpot-v2 benchmark, using SmolVLM2-2.2B-Instruct as the baseline model. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## NVIDIA Releases 6 Million Multi-Lingual Reasoning Dataset DevFeed: [NVIDIA Releases 6 Million Multi-Lingual Reasoning Dataset]() Original publisher: [Read original article]() Author: Jane Polak Scowcroft; Dhruv Nathawani; Shuoyang Ding; Oleksii Kuchaiev; Vitaly Lavrukhin Published: 2025-08-20T22:13:18Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [datasets](), [Nemotron](), [post-training](), [Open Source Models & Datasets](), [Mamba](), [Transformer](), [Large Language Model](), [Nvidia]() Tags: [datasets](), [japanese](), [llama](), [mamba](), [model-development](), [nemotron](), [nvidia](), [open](), [post-training](), [reasoning]() ### AI overview NVIDIA announces a 6-million-example multilingual reasoning dataset translated into French, Spanish, German, Italian, and Japanese. The article also presents Nemotron Nano 2 9B, an edge-oriented model using a hybrid Transformer-Mamba architecture, configurable thinking budgets, and open model weights and training resources. ### Source excerpt NVIDIA continues releasing permissive datasets in support of the open ecosystem with 6 Million Multilingual Reasoning Dataset. Continuing the success of the recent Nemotron Post-Training Dataset v1 release used in Llama Nemotron Super model, and our Llama Nemotron Post-Training Dataset release earlier this year, we're excited to release the reasoning dataset translated into five target languages: French, Spanish, German, Italian, and Japanese. ## Measuring Open-Source Llama Nemotron Models on DeepResearch Bench DevFeed: [Measuring Open-Source Llama Nemotron Models on DeepResearch Bench]() Original publisher: [Read original article]() Author: Jay Rodge Published: 2025-08-04T19:51:50Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [Nvidia](), [Open Source](), [hugging face](), [llama](), [Nemotron](), [Large Language Model](), [Open Source Models & Datasets](), [long-context](), [Inference](), [post-training](), [GPU](), [Reinforcement learning]() Tags: [agentic](), [ai](), [gpu](), [hallucination-detection](), [hugging-face](), [inference](), [llama](), [llm](), [long-context](), [nemotron](), [nvidia](), [open-source](), [post-training](), [reinforcement-learning]() ### AI overview The article presents NVIDIA's AI-Q Blueprint as a portable, open deep research agent that reached the top of Hugging Face's "LLM with Search" leaderboard on DeepResearch Bench. It combines Llama 3.3-70B Instruct with Llama-3.3-Nemotron-Super-49B-v1.5 for retrieval, reasoning, synthesis, query planning, and tool use, and incorporates NVIDIA NeMo Retriever and the NVIDIA NeMo Agent toolkit. The architecture emphasizes long-context, multimodal, low-latency search over local and web data, privacy, compliance, on-premise deployment, transparent model lineage, and efficient inference on GPUs. ### Source excerpt NVIDIA's AI-Q Blueprint--the leading portable, open deep research agent--recently climbed to the top of the Hugging Face "LLM with Search" leaderboard on DeepResearch Bench. This is a significant step forward for the open-source AI stack, proving that developer-accessible models can power advanced agentic workflows that rival or surpass closed alternatives. What sets AI-Q apart? ## Post-Training Isaac GR00T N1.5 for LeRobot SO-101 Arm DevFeed: [Post-Training Isaac GR00T N1.5 for LeRobot SO-101 Arm]() Original publisher: [Read original article]() Author: You Liang Tan; Fengyuan Hu; Oyindamola Omotuyi; Oluwaseun Doherty; Chitoku Yato; Shane Reetz Published: 2025-06-11T18:27:22Z Content type: tutorial Language: en Sources: [Hugging Face - Blog]() Topics: [Isaac](), [post-training](), [lerobot](), [Fine-tuning](), [foundation-models](), [datasets](), [multimodal](), [Open Source](), [Python](), [Terminal]() Tags: [data](), [fine-tuning](), [foundation-models](), [hardware](), [isaac](), [lerobot](), [open-source](), [post-training](), [python](), [robots](), [training](), [tutorial]() ### AI overview This tutorial demonstrates how to post-train, or fine-tune, NVIDIA Isaac GR00T N1.5 for a LeRobot SO-101 robot arm using teleoperation data. It covers dataset preparation, GR00T compatibility, installation prerequisites, and running the fine-tuning workflow, with a default requirement of approximately 25 GB of VRAM. ### Source excerpt NVIDIA Isaac GR00T (Generalist Robot 00 Technology) is a research and development platform for building robot foundation models and data pipelines, designed to accelerate the creation of intelligent, adaptable robots. Today, we announced the availability of Isaac GR00T N1.5, the first major update to Isaac GR00T N1, the world's first open foundation model for generalized humanoid robot reasoning and skills. ## Welcoming Llama Guard 4 on Hugging Face Hub DevFeed: [Welcoming Llama Guard 4 on Hugging Face Hub]() Original publisher: [Read original article]() Author: merve; Aritra Roy Gosthipaty; Sergio Paniego; Pedro Cuenca Published: 2025-04-29T00:00:00Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [llama](), [multimodal](), [hugging face](), [Artificial Intelligence](), [large-language-models](), [moe](), [post-training](), [Open Source Models & Datasets]() Tags: [artificial-intelligence](), [gpu](), [hugging-face](), [language-models](), [large-language-models](), [llama](), [llm](), [mixture-of-experts-moe](), [multimodal](), [post-training](), [safety](), [vision](), [vlm]() ### AI overview Hugging Face introduces Llama Guard 4, a multimodal safety model for detecting unsafe image and text inputs and outputs. The dense 12B model can run on a single 24 GB GPU, supports multilingual content, and classifies hazards for configurable moderation pipelines. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## Open R1: Update #4 DevFeed: [Open R1: Update #4]() Original publisher: [Read original article]() Author: Leandro von Werra; Vaibhav Srivastav; Daniel Vila; Yacine Jernite Published: 2025-03-26T18:47:29Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [deepseek](), [Benchmark](), [Code](), [coding](), [math](), [post-training](), [Front end]() Tags: [benchmark](), [benchmarks](), [code](), [coding](), [deepseek](), [function-calling](), [math](), [post-training](), [release](), [update]() ### AI overview The article reports on a new DeepSeek model that updates DeepSeek-V3, retains its architecture, and adopts an MIT license. It describes improvements in instruction following, coding, mathematics, front-end web development, writing, search, and function calling, supported by stronger benchmark results. ### Source excerpt This week, a new model from DeepSeek silently landed on the Hub. It's an updated version of DeepSeek-V3, the base model underlying the R1 reasoning model. There isn't much information shared yet on this new model, but we do know a few things! The model has the same architecture as the original DeepSeek-V3 and now also comes with an MIT license, while the previous V3 model had a custom model license. ## π0 and π0-FAST: Vision-Language-Action Models for General Robot Control DevFeed: [π0 and π0-FAST: Vision-Language-Action Models for General Robot Control]() Original publisher: [Read original article]() Author: Dana Aubakirova; Pablo Montalvo; Mustafa Shukor; Remi Cadene Published: 2025-02-04T00:00:00Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [foundation-models](), [Artificial Intelligence](), [lerobot](), [Machine learning](), [datasets](), [multimodal](), [data](), [post-training](), [PyTorch](), [sim-to-real transfer robotics]() Tags: [ai](), [artificial-intelligence](), [community](), [datasets](), [foundation-models](), [lerobot](), [models](), [multimodal](), [post-training](), [pytorch](), [research](), [robotics](), [robots](), [training](), [vision]() ### AI overview The article introduces π0 and π0-FAST, prototype Vision-Language-Action models and learning frameworks for generalist robot control. π0 uses large-scale pretraining and flow matching-based action generation, trained on data from seven robotic platforms and 68 tasks to support dexterous manipulation across different robot embodiments. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## Predictive Human Preference: From Model Ranking to Model Routing DevFeed: [Predictive Human Preference: From Model Ranking to Model Routing]() Original publisher: [Read original article]() Author: Chip Huyen Published: 2024-02-28T00:00:00Z Content type: article Language: en Sources: [Chip Huyen]() Topics: [Model Routing](), [AI Models](), [Model Development](), [post-training](), [AI Chat]() Tags: [ai-models](), [anthropic](), [chatgpt](), [dpo](), [gemini](), [latency](), [model](), [model-routing](), [ranking](), [rlhf]() ### AI overview The article examines predictive human preference: predicting which AI model users will prefer for a specific prompt. It describes model routing as a use case, where prompts could be directed to a preferred model to potentially reduce cost and latency, and discusses using preference predictions to interpret model strengths and weaknesses. It also outlines evaluating predictions against Chatbot Arena and building a preference predictor. ### Source excerpt A challenge of building AI applications is choosing which model to use. What if we don't have to? What if we can predict the best model for any prompt? Predictive human preference aims to predict which model users might prefer for a specific query. Human preference has emerged to be both the Northstar and a powerful tool for AI model development. Human preference guides post-training techniques including RLHF and DPO. Human preference is also used to rank AI models, as used by LMSYS's Chatbot Arena. Chatbot Arena aims to determine which model is generally preferred. I wanted to see if it's possible to predict which model is preferred for each query. One use case of predictive human preference is model routing. For example, if we know in advance that for a prompt, users will prefer Claude Instant's response over GPT-4, and Claude Instant is cheaper/faster than GPT-4, we can route this prompt to Claude Instant. Model routing has the potential to increase response quality while reducing costs and latency. Another use case of predictive human preference is interpretability. Mapping out a model's performance on different prompts can help us understand this model's strengths and weaknesses. See section Experiment results for examples. Here's what predictive human preference for different model pairs looks like for the prompt "What's the best way to cluster text embeddings?". The predictions were generated by my toy preference predictor. The bright yellow color for the (GPT-4, GPT-3.5-Turbo) cell means that my predictor thinks GPT-4's response is very likely to be preferred to that of GPT-3.5-Turbo's for this prompt. This post first discusses the correctness of Chatbot Arena, which will then be used as a baseline to evaluate the correctness of preference predictions. It then discusses how to build a preference predictor and the initial results. Ranking Models Using Human Preference Using preferential signals (comparisons) to rank models has grown in popularity in the last