# ai safety AI safety is an area of machine learning research focused on identifying unintended behavior and developing technical solutions to ensure AI systems operate safely and reliably. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Salesforce Announces Koa Reasoning Model Built on NVIDIA Nemotron 3 Super DevFeed: [Salesforce Announces Koa Reasoning Model Built on NVIDIA Nemotron 3 Super]() Original publisher: [Read original article]() Author: Brian Caulfield Published: 2026-09-15T22:24:34Z Content type: news Language: en Sources: [NVIDIA Blog]() Topics: [Koa](), [Nemotron](), [Artificial Intelligence](), [Nvidia](), [post-training](), [ai safety]() Tags: [agentic](), [agentic-ai](), [ai](), [cloud](), [corporate](), [events](), [model](), [nemotron](), [nvidia](), [nvidia-nemo](), [post-training](), [safety](), [security](), [software]() ### AI overview At Salesforce Dreamforce, NVIDIA CEO Jensen Huang discussed AI infrastructure, safety, and enterprise adoption alongside Salesforce CEO Marc Benioff. The event coincided with the announcement of Koa, Salesforce's first CRM reasoning model, built by post-training NVIDIA Nemotron 3 Super on a proprietary synthetic dataset derived from nearly three decades of enterprise CRM deployments. ### Source excerpt Know everything. Do anything. That was the message NVIDIA founder and CEO Jensen Huang brought to Salesforce Dreamforce Tuesday, joining CEO Marc Benioff onstage in an appearance that coincided with the announcement of Koa -- Salesforce's first CRM reasoning model, built on NVIDIA Nemotron 3 Super. Huang didn't just take the stage. He walked into [...] ## AI safety does not stop at the model DevFeed: [AI safety does not stop at the model]() Original publisher: [Read original article]() Author: Samar Abbas Published: 2026-09-15T00:00:00Z Content type: opinion Language: en Sources: [Temporal Blog]() Topics: [ai safety](), [Agent Harness](), [reliability](), [retry](), [Security]() Tags: [agent](), [agent-harness](), [ai](), [reliability](), [retry](), [safety](), [security](), [temporal-voices]() ### AI overview The article argues that AI safety extends beyond model behavior to the application layer, where companies must control agent authority and enforce approvals, policies, and credential limits. It emphasizes that these controls must remain reliable through crashes, timeouts, and retries, and presents Temporal Agent Harness as an execution-layer control point around an agent SDK. ### Source excerpt Auditing what an agent did is only half the job. Companies also have to control what agents may do, and make those limits hold when systems fail. ## Deploy NeMo Guardrails on Red Hat OpenShift AI DevFeed: [Deploy NeMo Guardrails on Red Hat OpenShift AI]() Original publisher: [Read original article]() Author: Rob Geada Published: 2026-09-09T15:05:01Z Content type: tutorial Language: en Sources: [Red Hat](), [Red Hat Developer]() Topics: [NeMo](), [Deployment](), [ai safety](), [Open Source](), [prompt injection]() Tags: [article](), [deployment](), [developer](), [how-to](), [nemo](), [open-source](), [operator](), [platform](), [prompt-injection](), [red-hat](), [safety](), [security]() ### AI overview This tutorial explains how to deploy an evaluated NeMo Guardrails configuration on a Red Hat OpenShift AI cluster. It covers prerequisites, creating a ConfigMap from the configuration files, and creating a NeMoGuardrails custom resource for a managed deployment. ### Source excerpt This is part 3 in a 3-part series on local guardrail development and evaluation. In the 1st article, I looked at how to design and develop a guardrail configuration on a local machine, and then tried some manual testing. In the 2nd article, I explored how to rigorously test a guardrail against popular large-scale risk datasets. The post Deploy NeMo Guardrails on Red Hat OpenShift AI appeared first on Red Hat Developer. ## Perturbation Probing: A New Diagnostic for the Fragility of LLM Safety DevFeed: [Perturbation Probing: A New Diagnostic for the Fragility of LLM Safety]() Original publisher: [Read original article]() Author: Tony Li, Hongliang Liu and Yuhao Wu Published: 2026-08-28T22:00:07Z Content type: article Language: en Sources: [Unit 42]() Topics: [Large Language Model](), [ai safety](), [Machine Learning, Security Attacks](), [Security](), [Artificial Intelligence](), [Reinforcement learning](), [human feedback]() Tags: [ai](), [ai-safety](), [benchmark](), [benchmarks](), [external](), [general](), [insights](), [internals](), [jailbreak](), [llm](), [llms](), [model](), [optimization](), [research](), [safety](), [security]() ### AI overview The article presents perturbation probing, a low-cost method for identifying neurons causally responsible for targeted behaviors in aligned large language models. It reports that very small neuron subsets control refusal or false-agreement behaviors, suggesting that LLM safety can be fragile and concentrated rather than broadly distributed. ### Source excerpt New research reveals that AI safety refusal lives in a thin neural layer, highlighting the critical need for external, multi-layered security. The post Perturbation Probing: A New Diagnostic for the Fragility of LLM Safety appeared first on Unit 42. ## Ensuring Safety in the Generative AI Ecosystem: Protecting Users from Non-Consensual Intimate Content DevFeed: [Ensuring Safety in the Generative AI Ecosystem: Protecting Users from Non-Consensual Intimate Content]() Original publisher: [Read original article]() Author: Android Developers (noreply@blogger.com) Published: 2026-08-25T17:00:00Z Content type: article Language: en Sources: [Android Developers Blog](), [Android Developers Blog]() Topics: [Generative AI](), [Responsibility & Safety](), [ai safety](), [Artificial Intelligence](), [Google](), [LineageOS]() Tags: [ai](), [ai-safety](), [android](), [best-practices](), [generative-ai](), [google](), [policy](), [safety](), [testing]() ### AI overview This article outlines Google Play's safety expectations for generative AI apps, with a focus on preventing non-consensual intimate imagery and other harmful content. It describes platform safeguards, app-lifecycle reviews, developer transparency, adversarial testing, and Android safety practices. ### Source excerpt Posted by Ron Aquino, Senior Director, Trust & Safety, Chrome, Android, and Play At Google Play, user safety and developer success go hand in hand. We continue to see growth in apps with AI generated features, and indeed, adding generative AI into your apps is a great way to unlock incredible creative possibilities. However, AI features also bring new safety challenges - such as the rise of AI-facilitated generation of non-consensual intimate imagery (NCII). Google Play's policies prohibit the facilitation, creation, or distribution of non-consensual sexual content. Harmful applications designed to target, harass, or exploit individuals have absolutely no place on Google Play, and we are committed to enforcing our policies to keep the store a safe space for developers to thrive. We know that the vast majority of you are dedicated to building positive, ethical tools. To protect both your hard work and our shared user base, we are investing heavily in platform protections, technical defenses, and developer resources to stop abuse. How we're safeguarding our shared ecosystem Protecting the platform is a continuous effort. Bad actors attempt to exploit distribution channels, monetization paths, and model boundaries. To help keep the ecosystem fair and safe, we've put a multi-layered defense strategy in place: Safeguards across the app lifecycle: Generative AI features are dynamic and can be less predictable, so safety isn't just a one-time check when you submit your app. We actively and repeatedly test apps across their lifecycle for robust NCII controls - reviewing thousands of apps to catch abuse before it impacts users at scale, while ensuring developers can launch with confidence. Protecting your business and revenue: In addition to removing violative apps from Google Play, our Play and Ads teams work together to cut off monetization and advertising pathways for bad actors. Apps that are suspended or removed for attempting to generate or monetize harmful content suc ## Ensuring Safety in the Generative AI Ecosystem: Protecting Users from Non-Consensual Intimate Content DevFeed: [Ensuring Safety in the Generative AI Ecosystem: Protecting Users from Non-Consensual Intimate Content]() Original publisher: [Read original article]() Author: Android Developers (noreply@blogger.com) Published: 2026-08-25T17:00:00Z Content type: article Language: en Sources: [Android Developers Blog]() Topics: [ai safety](), [Generative AI](), [Google Play](), [trust & safety](), [LineageOS]() Tags: [ai](), [ai-safety](), [generative-ai](), [google-play](), [policy](), [safety](), [testing](), [trust-safety]() ### AI overview Google Play outlines safety requirements and safeguards for generative AI apps to help prevent non-consensual intimate imagery and other harmful content. The guidance covers app-lifecycle reviews, guardrail visibility, adversarial testing, and Android safety practices. ### Source excerpt Posted by Ron Aquino, Senior Director, Trust & Safety, Chrome, Android, and Play At Google Play, user safety and developer success go hand in hand. We continue to see growth in apps with AI generated features, and indeed, adding generative AI into your apps is a great way to unlock incredible creative possibilities. However, AI features also bring new safety challenges - such as the rise of AI-facilitated generation of non-consensual intimate imagery (NCII). Google Play's policies prohibit the facilitation, creation, or distribution of non-consensual sexual content. Harmful applications designed to target, harass, or exploit individuals have absolutely no place on Google Play, and we are committed to enforcing our policies to keep the store a safe space for developers to thrive. We know that the vast majority of you are dedicated to building positive, ethical tools. To protect both your hard work and our shared user base, we are investing heavily in platform protections, technical defenses, and developer resources to stop abuse. How we're safeguarding our shared ecosystem Protecting the platform is a continuous effort. Bad actors attempt to exploit distribution channels, monetization paths, and model boundaries. To help keep the ecosystem fair and safe, we've put a multi-layered defense strategy in place: Safeguards across the app lifecycle: Generative AI features are dynamic and can be less predictable, so safety isn't just a one-time check when you submit your app. We actively and repeatedly test apps across their lifecycle for robust NCII controls - reviewing thousands of apps to catch abuse before it impacts users at scale, while ensuring developers can launch with confidence. Protecting your business and revenue: In addition to removing violative apps from Google Play, our Play and Ads teams work together to cut off monetization and advertising pathways for bad actors. Apps that are suspended or removed for attempting to generate or monetize harmful content suc ## Announcing Capital One's 2026 UIUC AI Awardees DevFeed: [Announcing Capital One's 2026 UIUC AI Awardees]() Original publisher: [Read original article]() Author: Capital One Tech Published: 2026-07-21T14:45:44Z Content type: release Language: en Sources: [Capital One Tech]() Topics: [Artificial Intelligence](), [ai safety](), [Generative AI](), [Large Language Model](), [Code generation](), [Cybersecurity]() Tags: [2026](), [academic-research](), [agentic-ai](), [agents](), [ai](), [ai-research](), [ai-safety](), [code-generation](), [cybersecurity](), [generative-ai](), [large-language-models-llms](), [research](), [science]() ### AI overview Capital One announces its 2026-2027 research and fellowship awardees from the University of Illinois. The featured projects address LLM reasoning faithfulness, code reasoning, imaginative LLM agents, and agentic AI safety. ### Source excerpt Meet the University of Illinois researchers and fellows advancing Agentic AI through our academic partnership.Announcing the Center for Generative AI Safety, Knowledge Systems, and Cybersecurity (ASKS) 2026-2027 Capital One Research Awardees from the University of Illinois As we look toward the 2026-2027 academic year, we continue to partner with institutions that lead the global conversation on the future of intelligence. We are thrilled to announce this year's cohort of research and fellowship awardees from the University of Illinois, whose pioneering work addresses the most critical frontier in technology today: Agentic AI. From the way machines "think" and "imagine" to the hardware that powers them and the safety protocols that govern them, these five projects represent Capital One's holistic push toward AI that is not only powerful but also faithful, safe and creative. The 2026-2027 research awardees1. Ensuring Intellectual Honesty Advancing LLM Reasoning Faithfulness without Faithfulness Rewards Faculty: Hao Peng Professor Peng is tackling the "hallucination" problem at its core. By developing methods to ensure large language models (LLMs) follow a logical, faithful reasoning path-without relying on traditional, often biased, reward systems-this work ensures that when an AI gives an answer, the "why" behind it is actually true, a critical component for trustworthy financial applications. TrACE-Contrast: Faithful & Consistent Code Reasoning via Trace-Aware Contrastive Learning Faculty: Talia Ringer and Reyhan Jabaarvand Writing code is one thing; understanding how it executes is another. Professor Ringer's project uses contrastive learning to align a model's code generation with its actual execution "trace." This ensures that AI-generated software is verified and logically sound, which is vital for maintaining the integrity of our core technology systems. 2. Bridging the Gap: Agency & Imagination Creative LLM Agents based on Thinking with Imagination Faculty: H ## From Campus to Community Part 3: What Universities Uniquely Bring DevFeed: [From Campus to Community Part 3: What Universities Uniquely Bring]() Original publisher: [Read original article]() Author: Nithya Ruff Published: 2026-07-21T14:14:34Z Content type: opinion Language: en Sources: [Linux Foundation - Blog]() Topics: [Artificial Intelligence](), [ai safety](), [Responsibility & Safety](), [LLM evaluation / benchmarking](), [openssf](), [linux foundation](), [Nvidia]() Tags: [ai](), [ai-safety](), [benchmarking](), [evaluation](), [nvidia](), [openssf](), [policy](), [security]() ### AI overview This opinion article argues that universities have a distinctive role in shaping safe and fair AI through independent benchmarking, adversarial evaluation, governance standards, and longer-term research. It describes ongoing debates over open AI models and cites Linux Foundation initiatives and related communities. ### Source excerpt This is a series from Linux Foundation Board Chair, Nithya Ruff. Read Part One and Two. ## GPT-Red: OpenAI's Internal Model for Testing Prompt-Injection Defenses DevFeed: [GPT-Red: OpenAI's Internal Model for Testing Prompt-Injection Defenses]() Original publisher: [Read original article]() Author: RisingStack Engineering Published: 2026-07-16T13:05:35Z Content type: opinion Language: en Sources: [RisingStack]() Topics: [prompt injection](), [ai safety](), [AI Development](), [OpenAI]() Tags: [agent](), [ai](), [ai-safety](), [openai](), [prompt-injection](), [red-teaming]() ### AI overview The article explains indirect prompt injection and discusses GPT-Red, an internal OpenAI red-teaming model trained to attack other AI models, mainly through prompt injection. It argues that application permissions and tool access are important factors in the resulting security risk. ### Source excerpt Prompt injection is still one of the least comfortable problems in AI development. You can improve your system prompt, restrict tools, validate outputs, and add approval steps before sensitive actions. Still, the model eventually has to read data you do not control. It might browse a webpage, process an email, inspect a repository, or use [...] The post GPT-Red: OpenAI Is Training Models to Break Other Models appeared first on RisingStack Engineering. ## GPT-Red: Unlocking Self-Improvement for Robustness DevFeed: [GPT-Red: Unlocking Self-Improvement for Robustness]() Original publisher: [Read original article]() Published: 2026-07-15T10:00:00Z Content type: article Language: en Sources: [OpenAI News]() Topics: [Artificial Intelligence](), [ai safety](), [OpenAI](), [Vulnerabilities](), [Responsibility & Safety](), [Training AI Models](), [Model Development](), [Deployment](), [Monitoring](), [Bug Bounty](), [browsers](), [data]() Tags: [ai](), [ai-safety](), [attacks](), [browsers](), [code](), [data](), [deployment](), [gpt](), [model](), [models](), [monitoring](), [openai](), [prompt-injection](), [safety](), [tool](), [tools](), [train](), [training](), [vulnerabilities]() ### AI overview OpenAI describes GPT-Red, an automated red-teaming model that uses adversarial attacks and self-play to find vulnerabilities, generate training data, and improve the robustness of future AI systems against prompt injection. The approach complements human and third-party red-teaming, layered safeguards, and real-time monitoring. ### Source excerpt Explore GPT-Red, OpenAI's automated red teaming system that uses self-play to improve AI safety, alignment, and prompt injection robustness. ## New method aims to keep kids safe from illegal AI-generated content DevFeed: [New method aims to keep kids safe from illegal AI-generated content]() Original publisher: [Read original article]() Author: Adam Zewe | MIT News Published: 2026-07-13T04:00:00Z Content type: news Language: en Sources: [MIT AI News]() Topics: [Artificial Intelligence](), [ai safety](), [AI Research](), [Open Source]() Tags: [ai](), [ai-deepfakes](), [ai-safety](), [algorithms](), [artificial-intelligence](), [ashia-wilson](), [computer-science-and-technology](), [csam](), [ethics](), [generative](), [generative-ai](), [human-computer-interaction](), [institute-for-medical-engineering-and-science-imes](), [laboratory-for-information-and-decision-systems-lids](), [machine-learning](), [marzyeh-ghassemi](), [mit-schwarzman-college-of-computing](), [models](), [online-child-safety](), [open-source](), [privacy](), [public-health](), [research](), [safety](), [school-of-engineering](), [vinith-suriyakumar]() ### AI overview MIT researchers and Thorn developed an auditing technique that assesses whether a generative AI model has been specialized to produce child sexual abuse material without generating illegal outputs. In testing, the procedure identified specialized model variants with 100 percent accuracy. ### Source excerpt Researchers developed an auditing technique to test generative AI models for malicious capabilities, without prompting them for illegal outputs. ## Claude Fable 5 Restrictions Exposed Availability Risks for AI Developers DevFeed: [Claude Fable 5 Restrictions Exposed Availability Risks for AI Developers]() Original publisher: [Read original article]() Author: RisingStack Engineering Published: 2026-07-06T13:42:50Z Content type: article Language: en Sources: [RisingStack]() Topics: [Claude](), [ai safety](), [Availability](), [API](), [context window](), [long-context]() Tags: [ai](), [ai-safety](), [api](), [availability](), [claude](), [context-window](), [models](), [tool-use]() ### AI overview The article examines how restrictions on Anthropic's Claude Fable 5 and Claude Mythos 5 exposed a new availability risk for software teams. It describes a temporary global suspension of access, Fable 5's return with updated safeguards, and Mythos 5's continued limited availability. The article argues that developers building on frontier APIs must account for government decisions, identity rules, safeguards, and provider policies alongside conventional operational dependencies. ### Source excerpt For years, most AI safety debates focused on model outputs. Could a model generate malware, explain a dangerous process, produce convincing misinformation, or comply with a jailbreak? Claude Fable 5 pushed that discussion toward a harder question: what happens when a model can inspect large codebases, use tools, maintain context across long-running tasks, and work [...] The post Claude Fable 5 Was Restricted for What It Could Do appeared first on RisingStack Engineering. ## Advancing youth safety and opportunity through global leadership DevFeed: [Advancing youth safety and opportunity through global leadership]() Original publisher: [Read original article]() Published: 2026-06-02T07:00:00Z Content type: article Language: en Sources: [OpenAI News]() Topics: [Artificial Intelligence](), [ai safety](), [OpenAI](), [Responsibility & Safety](), [ChatGPT]() Tags: [ai](), [ai-safety](), [chatgpt](), [collaboration](), [global](), [global-affairs](), [leadership](), [learn](), [openai](), [work]() ### AI overview OpenAI calls for coordinated global action on youth AI safety, including stronger default safeguards, family support, higher industry standards, and an international institute to sustain research and practical guidance. ### Source excerpt OpenAI calls for global action on youth AI safety, proposing an international institute to strengthen safeguards, standards, and opportunities for young people. ## Our views on AI policy and political advocacy DevFeed: [Our views on AI policy and political advocacy]() Original publisher: [Read original article]() Published: 2026-06-01T17:00:00Z Content type: article Language: en Sources: [OpenAI News]() Topics: [Artificial Intelligence](), [Responsibility & Safety](), [ai safety](), [OpenAI]() Tags: [ai](), [ai-safety](), [company](), [global-affairs](), [openai](), [policy](), [safety]() ### AI overview OpenAI explains its views on AI policy and political advocacy, including its lack of political donations or an employee-funded PAC, the distinction between employees' personal political activities and the company's positions, and its support for transparent advocacy, thoughtful regulation, rigorous testing, and strong safety standards. ### Source excerpt Our approach to AI policy and political advocacy, transparency, support for thoughtful regulation and AI safety, and that no outside political group speaks on the company's behalf. ## OpenAI's Frontier Governance Framework DevFeed: [OpenAI's Frontier Governance Framework]() Original publisher: [Read original article]() Published: 2026-05-28T00:00:00Z Content type: article Language: en Sources: [OpenAI News]() Topics: [OpenAI](), [Frontier AI](), [Responsibility & Safety](), [Security](), [Incident response](), [ai safety]() Tags: [ai](), [ai-safety](), [eu](), [frontier-ai](), [incident-response](), [legal](), [openai](), [safety](), [safety-security](), [security](), [update]() ### AI overview OpenAI publishes its Frontier Governance Framework, describing how its safety, security, and risk-management practices align with emerging legal requirements, including the EU AI Act and California's Transparency in Frontier AI Act. ### Source excerpt Explore OpenAI's Frontier Governance Framework and how our AI safety, security, and risk practices align with emerging EU and California regulations. ## Improving instruction hierarchy in frontier LLMs DevFeed: [Improving instruction hierarchy in frontier LLMs]() Original publisher: [Read original article]() Published: 2026-03-10T11:00:00Z Content type: article Language: en Sources: [OpenAI News]() Topics: [Large Language Model](), [ai safety](), [prompt injection](), [Reinforcement learning](), [Security](), [OpenAI]() Tags: [ai-safety](), [llms](), [openai](), [prompt-injection](), [reinforcement-learning](), [research](), [safety](), [security]() ### AI overview IH-Challenge is a training dataset designed to improve frontier LLMs' instruction hierarchy, safety steerability, and robustness against prompt-injection attacks. The article explains how models should prioritize system, developer, user, and tool instructions according to their trust level, and describes reinforcement learning as a way to train this behavior. ### Source excerpt IH-Challenge trains models to prioritize trusted instructions, improving instruction hierarchy, safety steerability, and resistance to prompt injection attacks. ## Agent Safety is a Box DevFeed: [Agent Safety is a Box]() Original publisher: [Read original article]() Author: Marc Brooker Published: 2026-01-12T00:00:00Z Content type: opinion Language: en Sources: [Marc Brooker's Blog](), [Marc Brooker's Blog]() Topics: [AI Agent](), [ai safety](), [Responsibility & Safety](), [Model Context Protocol]() Tags: [agent](), [ai-agent](), [ai-agents](), [ai-safety](), [mcp]() ### AI overview The article argues that AI agents should be constrained by a strong, deterministic control layer outside the agent--a "box"--that limits which tools they can call and what they can do. It distinguishes safety from prompting, steering, and context management, which may improve success rates and cost but cannot provide sufficient guarantees for flexible and adaptive agents acting through side effects. ### Source excerpt Agent Safety is a Box Keep a lid on it. Before we start, let's cover some terms so we're thinking about the same thing. This is a post about AI agents, which I'll define (riffing off Simon Willison1) as: An AI agent runs models and tools in a loop to achieve a goal. Here, goals can include coding, customer service, proving theorems, cloud operations, or many other things. These agents can be interactive or one-shot; called by humans, other agents, or traditional computer systems; local or cloud; and short-lived or long-running. What they don't tend to be is pure. They typically achieve their goals by side effects. Side effects including modifying the local filesystem, calling another agent, calling a cloud service, making a payment, or starting a 3D print. The topic of today's post is those side-effects. Simply, what agents can do. We should also be concerned with what agents can say, and I'll touch on that topic a bit as I go. But the focus is on do. Agents do things with tools. These could be MCP-style tools, powers, skills, or one of many other patterns for tool calling. But, crucially, the act of doing inference doesn't do anything. Without the do, the think seems less important. The right way to control what agents do is to put them in a box. The box is a strong, deterministic, exact, layer of control outside the agent which limits which tools it can call, and what it can do with those tools. Why a Box? The most important one of those properties is outside the agent. Alignment and other AI safety topics are important. Steering, careful prompting, and context management help a lot. These techniques have a lot of value for liveness (success rate, cost, etc), but are insufficient for safety. They're insufficient for safety for the same reason we're building agents in the first place: because they're flexible, adaptive, creative2 problem solvers. Traditional old-school workflows are great. They're cheap, predictable, deterministic, understandable, and well understo ## Gemma Scope 2: helping the AI safety community deepen understanding of complex language model behavior DevFeed: [Gemma Scope 2: helping the AI safety community deepen understanding of complex language model behavior]() Original publisher: [Read original article]() Author: Language Model Interpretability team Published: 2025-12-16T10:14:24Z Content type: article Language: en Sources: [Google DeepMind News]() Topics: [ai safety](), [gemma](), [Large Language Model](), [AI Research](), [Open Source](), [Development]() Tags: [ai-models](), [ai-research](), [ai-safety](), [debug](), [demo](), [gemma](), [language-models](), [open-source](), [responsibility-safety](), [safety]() ### AI overview Google DeepMind announces Gemma Scope 2, an open suite of interpretability tools covering all Gemma 3 model sizes from 270M to 27B parameters. The tools use sparse autoencoders and transcoders to help researchers examine language model behavior, investigate safety risks, and study issues such as jailbreaks, hallucinations, and sycophancy. ### Source excerpt Open interpretability tools for language models are now available across the entire Gemma 3 family with the release of Gemma Scope 2. ## Deepening our partnership with the UK AI Security Institute DevFeed: [Deepening our partnership with the UK AI Security Institute]() Original publisher: [Read original article]() Author: William Isaac; Owen Larter Published: 2025-12-11T00:06:40Z Content type: article Language: en Sources: [Google DeepMind News]() Topics: [ai safety](), [Google](), [Responsibility & Safety](), [Security](), [AI Models](), [AI Research](), [Chain-of-thought]() Tags: [ai](), [ai-models](), [ai-safety](), [ai-security](), [chain-of-thought](), [collaboration](), [google](), [publications](), [reports](), [research](), [responsibility-safety](), [safety](), [technical](), [techniques](), [uk]() ### AI overview Google DeepMind announces an expanded partnership with the UK AI Security Institute focused on foundational AI security and safety research. The collaboration includes model testing, shared research resources, joint publications, and work on monitoring AI reasoning processes. ### Source excerpt Google DeepMind and UK AI Security Institute (AISI) strengthen collaboration on critical AI safety and security research ## The Alignment Trap: AI Safety as Path to Power DevFeed: [The Alignment Trap: AI Safety as Path to Power]() Original publisher: [Read original article]() Published: 2024-10-27T12:25:00Z Content type: opinion Language: en Sources: [Thomas Young]() Topics: [Artificial Intelligence](), [ai safety](), [Responsibility & Safety](), [trust]() Tags: [ai](), [ai-safety](), [alignment](), [networks](), [process](), [safety](), [surveillance](), [systems](), [trust]() ### AI overview The article argues that AI safety measures designed to make systems controllable and aligned with human values could also give governments, corporations, or other powerful organizations more effective tools for concentrating power. It compares AI-enabled control with historical limits on dictatorship, including unreliable loyalty, information distortion, cognitive constraints, and administrative friction. ### Source excerpt Recent discussions about artificial intelligence safety have focused heavily on ensuring AI systems remain under human control. While this goal seems laudable on its surface, we should carefully examine whether some proposed safety measures could paradoxically enable rather than prevent dangerous concentrations of power. The Control Paradox The fundamental tension lies in how we define "safety." Many current approaches to AI safety focus on making AI systems more controllable and aligned with human values. But this raises a critical question: controllable by whom, and aligned with whose values? When we develop mechanisms to control AI systems, we are essentially creating tools that could be used by any sufficiently powerful entity - whether that's a government, corporation, or other organization. The very features that make an AI system "safe" in terms of human control could make it a more effective instrument of power consolidation. Natural Limits on Human Power Historical examples reveal how human nature itself acts as a brake on totalitarian control. Even the most powerful dictatorships have faced inherent limitations that AI-enhanced systems might easily overcome: The Trust Problem: Stalin's paranoia about potential rivals wasn't irrational - it reflected the real difficulty of ensuring absolute loyalty from human subordinates. Every dictator faces this fundamental challenge: they can never be entirely certain of their underlings' true thoughts and loyalties. Information Flow: The East German Stasi, despite maintaining one of history's most extensive surveillance networks, still relied on human informants who could be unreliable, make mistakes, or even switch allegiances. Human networks inherently leak and distort information. Cognitive Limitations: Hitler's micromanagement of military operations often led to strategic blunders because no human can effectively process and control complex operations at scale. Human dictators must delegate, creating opportunities ## How NIST is changing standards to safeguard AI DevFeed: [How NIST is changing standards to safeguard AI]() Original publisher: [Read original article]() Published: 2024-08-19T00:00:00Z Content type: article Language: en Sources: [Chainguard: Unchained]() Topics: [Security](), [Responsibility & Safety](), [ai safety](), [AI Strategy](), [Microsoft](), [ChatGPT]() Tags: [chatgpt](), [copilot](), [gemini](), [generative-ai](), [nist](), [responsible-ai](), [risk-management](), [security](), [standards]() ### AI overview The article explains how NIST is developing AI safety and security guidelines in response to the White House's October 2023 Executive Order. It highlights generative AI risks such as disinformation, phishing, malware development, and sensitive-data leaks, and describes planned companion resources for the AI Risk Management Framework and Secure Software Development Framework. ### Source excerpt New NIST AI standards address generative AI risks with a focus on risk management, security practices, and recognizing synthetic content.