# red-teaming Published articles for red-teaming. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## How and Why We Bought 4x DGX Sparks DevFeed: [How and Why We Bought 4x DGX Sparks]() Original publisher: [Read original article]() Author: Alex Ellis Published: 2026-09-15T00:00:00Z Content type: opinion Language: en Sources: [Alex Ellis' Blog]() Topics: [Local AI](), [GPU](), [qwen](), [Cloud]() Tags: [agents](), [ai](), [cloud](), [gpu](), [llm](), [local-ai](), [localai](), [qwen](), [red-teaming](), [security]() ### AI overview The author explains why OpenFaaS Ltd bought four DGX Sparks and what the team learned from deploying local AI. The article argues that local infrastructure can provide tangible privacy and risk-reduction benefits for business use cases, even though it is not primarily justified by cost per token. ### Source excerpt In June we deployed an RTX 6000 Pro into production, a few weeks later, we're now operating DGX Sparks for the team. Learn how and why. ## GPT-Red: OpenAI's Internal Model for Testing Prompt-Injection Defenses DevFeed: [GPT-Red: OpenAI's Internal Model for Testing Prompt-Injection Defenses]() Original publisher: [Read original article]() Author: RisingStack Engineering Published: 2026-07-16T13:05:35Z Content type: opinion Language: en Sources: [RisingStack]() Topics: [prompt injection](), [ai safety](), [AI Development](), [OpenAI]() Tags: [agent](), [ai](), [ai-safety](), [openai](), [prompt-injection](), [red-teaming]() ### AI overview The article explains indirect prompt injection and discusses GPT-Red, an internal OpenAI red-teaming model trained to attack other AI models, mainly through prompt injection. It argues that application permissions and tool access are important factors in the resulting security risk. ### Source excerpt Prompt injection is still one of the least comfortable problems in AI development. You can improve your system prompt, restrict tools, validate outputs, and add approval steps before sensitive actions. Still, the model eventually has to read data you do not control. It might browse a webpage, process an email, inspect a repository, or use [...] The post GPT-Red: OpenAI Is Training Models to Break Other Models appeared first on RisingStack Engineering. ## Capital One at ACL 2026 DevFeed: [Capital One at ACL 2026]() Original publisher: [Read original article]() Author: Capital One Tech Published: 2026-07-01T15:28:51Z Content type: article Language: en Sources: [Capital One Tech]() Topics: [Natural language processing](), [Large Language Model](), [LLM security](), [Machine Learning, Security Attacks](), [Reinforcement learning](), [Jailbreak](), [Security](), [Routing (disambiguation)](), [datasets]() Tags: [2026](), [ai](), [ai-ml](), [ai-research](), [conference](), [jailbreak](), [llm-security](), [natural-language-processing](), [paper](), [partners](), [red-teaming](), [reinforcement-learning](), [research](), [science]() ### AI overview Capital One describes its accepted ACL 2026 research on natural language processing, including work on adaptive LLM red teaming, query-only model routing with generated data, and language identification on web data. The article also highlights collaboration with academic partners. ### Source excerpt Discover how Capital One is advancing state-of-the-art AI/ML science through collaborative natural language processing research.Advancing AI and NLP Frontiers at ACL 2026 As language models grow more deeply integrated into technology ecosystems, pioneering robust, efficient, and reliable Natural Language Processing (NLP) techniques becomes paramount. Capital One continues to invest in state-of-the-art AI/ML science through deep multi-sector collaboration and peer-reviewed research. At the upcoming Annual Meeting of the Association for Computational Linguistics (ACL 2026), Capital One researchers and academic partners will showcase novel findings stretching from LLM security to multilingual capabilities. Through the Science & Academic Partnerships program, Capital One bridges industry needs with academic expertise, funding critical university research and engineering solutions that make technology safer and more powerful. Our accepted publications at ACL 2026 demonstrate this thriving flywheel of talent and collaborative innovation across multiple research categories. Main Conference Research Adaptive Instruction Composition for Automated LLM Red Teaming Routing with Generated Data: Annotation-Free LLM Skill Estimation and Expert Selection Capital One Authors: Jesse Zymet, Swapnil Shinde, Sahil Wadhwa, Andy Luo Overview: Standard red teaming approaches often struggle with a limited range of jailbreak strategies or rely on ineffective, randomized crowd-sourced tactics. This paper introduces a novel framework -- Adaptive Instruction Composition -- that utilizes reinforcement learning and a neural contextual bandit to tailor attack compositions dynamically, balancing diversity and effectiveness to proactively uncover target model vulnerabilities. Routing with Generated Data: Annotation-Free LLM Skill Estimation and Expert Selection Capital One Authors: Genta Winata, Sambit Sahu, Supriyo Chakraborty, Shixiong Zhang Overview: Emerging from our gifted research collaboration ## Кто выпустил гончую. Ищем следы коллекторов BloodHound в логах Windows DevFeed: [Кто выпустил гончую. Ищем следы коллекторов BloodHound в логах Windows]() Original publisher: [Read original article]() Author: StepVolg ("Лаборатория Касперского") Published: 2026-04-24T12:37:53Z Content type: tutorial Language: ru Sources: ["Лаборатория Касперского" RU]() Topics: [Endpoint Security & XDR](), [Windows](), [SOC]() Tags: [active-directory](), [bloodhound](), [enumerate](), [red-teaming](), [sharphound](), [siem](), [soc](), [windows](), [windows-1ad1db2b7e3a]() ### AI overview The article examines traces left by BloodHound collectors in Windows logs and discusses detecting Active Directory reconnaissance activity. ### Source excerpt Служба каталогов Active Directory остается одной из самых популярных целей как среди злоумышленников, так и среди специалистов по Red Teaming и пентестеров. С выходом новых версий операционных систем семейства Windows продолжают появляться новые векторы атак на AD, например атаки на Delegated Managed Service Accounts (dMSA) в 2025-м. В ходе каждой атаки есть этап сбора информации, обнаружение которого является более сложной задачей, чем кажется на первый взгляд. Согласно аналитическому отчету нашего сервиса MDR за 2025 год в целом обнаружение данного этапа атак затруднено из-за большого количества ложных срабатываний, что снижает качество обнаружения и уменьшает вероятность предотвращения атаки, особенно в больших инфраструктурах с тысячами активов. Меня зовут Степан Ляхов, я работаю старшим инженером SOC в "Лаборатории Касперского". В этой статье я хочу рассмотреть один из самых популярных инструментов для сбора информации о домене Active Directory, разобрать, какие следы он оставляет в журналах и как обнаружить его активность. Читать далее ## Google Workspace's continuous approach to mitigating indirect prompt injections DevFeed: [Google Workspace's continuous approach to mitigating indirect prompt injections]() Original publisher: [Read original article]() Author: Kimberly Samra (noreply@blogger.com) Published: 2026-04-02T16:00:00Z Content type: opinion Language: en Sources: [Google Online Security]() Topics: [prompt injection](), [Google](), [Security](), [Adversarial attacks](), [Machine Learning, Security Attacks](), [Machine learning](), [Artificial Intelligence](), [Google AI]() Tags: [ai](), [attacks](), [automated](), [discovery](), [google](), [llms](), [machine-learning](), [none](), [prompt](), [prompt-injection](), [red-teaming](), [research](), [safety](), [security](), [testing]() ### AI overview Google describes its ongoing approach to mitigating indirect prompt injection attacks against Workspace with Gemini. The approach includes discovering new attack vectors, human and automated red-teaming, and collaboration with external researchers through the Google AI Vulnerability Rewards Program. ### Source excerpt Posted by Adam Gavish, Google GenAI Security Team Indirect prompt injection (IPI) is an evolving threat vector targeting users of complex AI applications with multiple data sources, such as Workspace with Gemini. This technique enables the attacker to influence the behavior of an LLM by injecting malicious instructions into the data or tools used by the LLM as it completes the user's query. This may even be possible without any input directly from the user. IPI is not the kind of technical problem you "solve" and move on. Sophisticated LLMs with increasing use of agentic automation combined with a wide range of content create an ultra-dynamic and evolving playground for adversarial attacks. That's why Google takes a sophisticated and comprehensive approach to these attacks. We're continuously improving LLM resistance to IPI attacks and launching AI application capabilities with ever-improving defenses. Staying ahead of the latest indirect prompt injection attacks is critical to our mission of securing Workspace with Gemini. In our previous blog "Mitigating prompt injection attacks with a layered defense strategy", we reviewed the layered architecture of our IPI defenses. In this blog, we'll share more detail on the continuous approach we take to improve these defenses and to solve for new attacks. New attack discovery By proactively discovering and cataloging new attack vectors through internal and external programs, we can identify vulnerabilities and deploy robust defenses ahead of adversarial activity. Human Red-Teaming Human Red-Teaming uses adversarial simulations to uncover security and safety vulnerabilities. Specialized teams execute attacks based on realistic user profiles to exploit weaknesses, coordinating with product teams to resolve identified issues. Automated Red-Teaming Automated Red-Teaming is done via dynamic, machine-learning-driven frameworks to stress-test environments. By algorithmically generating and iterating on attack payloads, we can mimi ## Architecting Security for Agentic Capabilities in Chrome DevFeed: [Architecting Security for Agentic Capabilities in Chrome]() Original publisher: [Read original article]() Author: Google (noreply@blogger.com) Published: 2025-12-08T18:03:00Z Content type: article Language: en Sources: [Google Online Security]() Topics: [agentic browsing](), [Security](), [Chrome](), [prompt injection](), [Artificial Intelligence](), [browsers](), [Google]() Tags: [agentic-browsing](), [ai](), [chrome](), [chrome-security](), [gemini-in-chrome](), [google](), [llm](), [none](), [prompt-injection](), [real-time](), [red-teaming](), [safety](), [security]() ### AI overview Google describes Chrome's layered security approach for agentic browsing, focusing on indirect prompt injection from untrusted web content. The approach includes a User Alignment Critic, origin isolation, user confirmations for critical steps, real-time threat detection, and red-teaming. ### Source excerpt Posted by Nathan Parker, Chrome security team Chrome has been advancing the web's security for well over 15 years, and we're committed to meeting new challenges and opportunities with AI. Billions of people trust Chrome to keep them safe by default, and this is a responsibility we take seriously. Following the recent launch of Gemini in Chrome and the preview of agentic capabilities, we want to share our approach and some new innovations to improve the safety of agentic browsing. The primary new threat facing all agentic browsers is indirect prompt injection. It can appear in malicious sites, third-party content in iframes, or from user-generated content like user reviews, and can cause the agent to take unwanted actions such as initiating financial transactions or exfiltrating sensitive data. Given this open challenge, we are investing in a layered defense that includes both deterministic and probabilistic defenses to make it difficult and costly for attackers to cause harm. Designing safe agentic browsing for Chrome has involved deep collaboration of security experts across Google. We built on Gemini's existing protections and agent security principles and have implemented several new layers for Chrome. We're introducing a user alignment critic where the agent's actions are vetted by a separate model that is isolated from untrusted content. We're also extending Chrome's origin-isolation capabilities to constrain what origins the agent can interact with, to just those that are relevant to the task. Our layered defense also includes user confirmations for critical steps, real-time detection of threats, and red-teaming and response. We'll step through these layers below. Checking agent outputs with User Alignment Critic The main planning model for Gemini uses page content shared in Chrome to decide what action to take next. Exposure to untrusted web content means it is inherently vulnerable to indirect prompt injection. We use techniques like spotlighting that direct ## Red-teaming a RAG app: gpt-4o-mini v. llama3.1 v. hermes3 DevFeed: [Red-teaming a RAG app: gpt-4o-mini v. llama3.1 v. hermes3]() Original publisher: [Read original article]() Author: Pamela Fox (noreply@blogger.com) Published: 2025-08-04T17:08:00Z Content type: article Language: en Sources: [Pamela Fox]() Topics: [Artificial Intelligence](), [Retrieval Augmented Generation (RAG)](), [PostgreSQL](), [Python](), [Azure](), [Open Source]() Tags: [ai](), [azure](), [gpt](), [llama3](), [open-source](), [openai](), [postgresql](), [python](), [rag](), [red-teaming](), [testing]() ### AI overview This article examines red-teaming a retrieval-augmented generation application using an automated Red Teaming agent from the azure-ai-evaluations Python package. It describes how adversarial questions are generated, transformed with the open-source pyrit package, sent to a RAG-on-PostgreSQL application, and evaluated for unsafe responses across models including gpt-4o-mini, llama3.1, and hermes3. ### Source excerpt When we develop user-facing applications that are powered by LLMs, we're taking on a big risk that the LLM may produce output that is unsafe in some way - like responses that encourage violence, hate speech, or self-harm. How can we be confident that a troll won't get our app to say something horrid? We could throw a few questions at it while manually testing, like "how do I make a bomb?", but that's only scratching the surface. Malicious users have gone to far greater lengths to manipulate LLMs into responding in ways that we definitely don't want happening in domain-specific user applications. Red-teaming That's where red teaming comes in: bring in a team of people that are expert at coming up with malicious queries and that are deeply familiar with past attacks, give them access to your application, and wait for their report of whether your app successfully resisted the queries. But red-teaming is expensive, requiring both time and people. Most companies don't have the resources nor expertise to have a team of humans red-teaming every app, plus every iteration of an app each time a model or prompt changes. Fortunately, my colleagues at Microsoft developed an automated Red Teaming agent, part of the azure-ai-evaluations Python package. The agent uses an adversarial LLM, housed safely inside an Azure AI Foundry project such that it can't be used for other purposes, in order to generate unsafe questions across various categories. The agent then transforms the questions using the open-source pyrit package, which uses known attacks like base-64 encoding, URL encoding, Ceaser Cipher, and many more. It sends both the original plain text questions and transformed questions to your app, and then evaluates the response to make sure that the app didn't actually answer the unsafe question. RAG application So I red-team'ed a RAG app! My RAG-on-PostgreSQL sample application answers questions about products from a database representing a fictional outdoors store. It uses a basi ## Mitigating prompt injection attacks with a layered defense strategy DevFeed: [Mitigating prompt injection attacks with a layered defense strategy]() Original publisher: [Read original article]() Author: Kimberly Samra (noreply@blogger.com) Published: 2025-06-13T16:03:00Z Content type: opinion Language: en Sources: [Google Online Security]() Topics: [prompt injection](), [ai security](), [Generative AI](), [Google](), [Artificial Intelligence]() Tags: [ai](), [ai-security](), [defense-in-depth](), [generative-ai](), [google](), [none](), [prompt-injection](), [red-teaming](), [security]() ### AI overview Google describes a layered defense strategy for mitigating indirect prompt injection attacks in Gemini. The approach combines model hardening, machine-learning classifiers, adversarial training, sanitization, user confirmations, and security notifications across the prompt lifecycle. ### Source excerpt Posted by Adam Gavish, Google GenAI Security Team With the rapid adoption of generative AI, a new wave of threats is emerging across the industry with the aim of manipulating the AI systems themselves. One such emerging attack vector is indirect prompt injections. Unlike direct prompt injections, where an attacker directly inputs malicious commands into a prompt, indirect prompt injections involve hidden malicious instructions within external data sources. These may include emails, documents, or calendar invites that instruct AI to exfiltrate user data or execute other rogue actions. As more governments, businesses, and individuals adopt generative AI to get more done, this subtle yet potentially potent attack becomes increasingly pertinent across the industry, demanding immediate attention and robust security measures. At Google, our teams have a longstanding precedent of investing in a defense-in-depth strategy, including robust evaluation, threat analysis, AI security best practices, AI red-teaming, adversarial training, and model hardening for generative AI tools. This approach enables safer adoption of Gemini in Google Workspace and the Gemini app (we refer to both in this blog as "Gemini" for simplicity). Below we describe our prompt injection mitigation product strategy based on extensive research, development, and deployment of improved security mitigations. A layered security approach Google has taken a layered security approach introducing security measures designed for each stage of the prompt lifecycle. From Gemini 2.5 model hardening, to purpose-built machine learning (ML) models detecting malicious instructions, to system-level safeguards, we are meaningfully elevating the difficulty, expense, and complexity faced by an attacker. This approach compels adversaries to resort to methods that are either more easily identified or demand greater resources. Our model training with adversarial data significantly enhanced our defenses against indirect prompt in