# Chaos Engineering Published articles for Chaos Engineering. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Chaos Hub & MCP Prompt Library: Harness Resilience Testing DevFeed: [Chaos Hub & MCP Prompt Library: Harness Resilience Testing]() Original publisher: [Read original article]() Author: Pritesh Kiri Published: 2026-08-05T00:00:00Z Content type: article Language: en Sources: [Harness Blog]() Topics: [Resilience](), [Chaos Engineering](), [Model Context Protocol](), [AWS Fault Injection Service (FIS)](), [MCP Server](), [Documentation](), [cursor](), [Claude]() Tags: [chaos](), [chaos-engineering](), [claude](), [cursor](), [documentation](), [mcp](), [mcp-server](), [resilience]() ### AI overview Harness Resilience Testing has added two documentation updates: Chaos Hub is now available directly in the documentation, and a Prompt Library provides ready-to-use prompts for running resilience workflows through Harness MCP using natural language. ### Source excerpt New in Harness Resilience Testing: Chaos Hub now lives in the docs, plus a Prompt Library for running chaos experiments via MCP using natural language. | Blog ## Creating an agentic feedback loop with reliability guardrails DevFeed: [Creating an agentic feedback loop with reliability guardrails]() Original publisher: [Read original article]() Author: Gavin Cahill Published: 2026-06-25T00:00:00Z Content type: article Language: en Sources: [Gremlin Blog]() Topics: [AI Development](), [AI-assisted coding](), [Chaos Engineering](), [Resilience](), [CI/CD](), [Code review]() Tags: [agentic-ai](), [ai](), [ai-for-code](), [chaos-engineering](), [ci-cd](), [code-generation](), [resilience](), [testing]() ### AI overview The article explains how reliability guardrails and resilience testing can create an agentic feedback loop for AI coding. It argues that agentic code review and QA alone may miss production failures, so fault injection and Chaos Engineering can validate system behavior under realistic failures and provide data that improves AI-generated code. It also discusses using resilience tests as an automated CI/CD gate. ### Source excerpt Reliability guardrails are essential for ensuring resilience with AI development, but they can also be used to create a feedback loop for AI context. ## Failure Modes in Distributed Systems and Patterns for Resilient Design DevFeed: [Failure Modes in Distributed Systems and Patterns for Resilient Design]() Original publisher: [Read original article]() Author: Alexandre Zajac Published: 2026-06-01T15:30:24Z Content type: article Language: en Sources: [Hungry Minds]() Topics: [distributed-systems](), [systems](), [Resilience](), [Architecture & Design](), [Chaos Engineering]() Tags: [architecture](), [chaos-engineering](), [distributed-systems](), [resilience](), [software-engineering](), [systems]() ### AI overview The article explains recurring failure modes in distributed systems, including Byzantine failures, split-brain scenarios, cascading timeouts, and partial failures. It recommends semantic health checks, defensive timeouts, circuit breakers, bulkheads, and explicit failure-mode testing to improve resilience. ### Source excerpt PLUS: SWE job market 2026 👨💻, Visual debugging for ML ⚡, S-tier demo framework 👨💻 ## Reliability Resolutions: How to build effective reliability programs that won't fade away DevFeed: [Reliability Resolutions: How to build effective reliability programs that won't fade away]() Original publisher: [Read original article]() Author: Gavin Cahill Published: 2026-01-21T00:00:00Z Content type: article Language: en Sources: [Gremlin Blog]() Topics: [Chaos Engineering](), [Availability](), [systems](), [data]() Tags: [availability](), [chaos-engineering](), [data](), [engineering](), [reliability-management](), [systems](), [testing](), [uptime]() ### AI overview This article explains how to build reliability and Chaos Engineering programs that produce lasting results. It recommends aligning reliability work with company goals, assigning ownership, creating repeatable processes, identifying data gaps, and testing specific failure modes on critical systems. Progress can be demonstrated through evidence such as validated failover and achievement of uptime targets. ### Source excerpt We're already almost through January. How are your reliability resolutions faring? Check out these key questions to help you follow-through and build an effective reliability program. ## AWS re:Invent 2025: The top sessions SREs should attend DevFeed: [AWS re:Invent 2025: The top sessions SREs should attend]() Original publisher: [Read original article]() Author: Kate Bernacchi-Sass Published: 2025-11-20T18:29:00Z Content type: article Language: en Sources: [The incident.io Blog]() Topics: [Amazon Web Services](), [site-reliability-engineering](), [Resilience](), [Chaos Engineering](), [Incident response](), [Availability](), [Serverless]() Tags: [2025](), [architecture](), [availability](), [aws](), [best-practices](), [chaos-engineering](), [cloud](), [incident](), [incident-channel](), [incident-management](), [incident-response](), [on-call](), [operational](), [outage](), [post-mortem](), [production](), [resilience](), [scale](), [slack-incident](), [sre](), [systems](), [techniques]() ### AI overview A curated guide to AWS re:Invent 2025 sessions for SREs and others focused on reliability, incident response, on-call operations, cloud resilience, and resilient systems. It highlights architecture lessons, resilience practices, and a session on testing AWS Lambda with chaos engineering and fault-injection techniques. ### Source excerpt The top sessions every SRE should see at this year's AWS re:Invent. ## How we prepare Shopify for BFCM DevFeed: [How we prepare Shopify for BFCM]() Original publisher: [Read original article]() Author: Kyle Petroski; Matthew Frail Published: 2025-11-20T14:40:48Z Content type: article Language: en Sources: [Shopify Engineering](), [Shopify Engineering - Shopify Engineering]() Topics: [Shopify](), [Chaos Engineering](), [Resilience](), [Cloud](), [Google Cloud Platform (GCP)]() Tags: [chaos-engineering](), [cloud](), [google-cloud](), [infrastructure](), [performance](), [production](), [resilience](), [shopify]() ### AI overview Shopify describes its year-round preparation for Black Friday Cyber Monday by modeling traffic, expanding capacity across multiple Google Cloud regions, reviewing infrastructure changes, and conducting risk assessments. Large-scale Game Days and fire drills simulated extreme production load, exposing issues such as Kafka bottlenecks, memory pressure, and timeouts that were fixed and revalidated. ### Source excerpt From March to October we simulated traffic tsunamis, injected chaos, and fixed every bottleneck before our merchants needed us most. ## Reliability lessons from the 2025 Cloudflare outage DevFeed: [Reliability lessons from the 2025 Cloudflare outage]() Original publisher: [Read original article]() Author: Andre Newman Published: 2025-11-20T00:00:00Z Content type: article Language: en Sources: [Gremlin Blog]() Topics: [Cloudflare](), [Bot Management](), [configuration](), [Chaos Engineering](), [Post Mortem](), [Network](), [Workers]() Tags: [2025](), [after-the-retrospective](), [bot-management](), [chaos-engineering](), [cloudflare](), [configuration](), [engineering](), [http](), [internet](), [management](), [outage](), [outages](), [post-mortem](), [workers]() ### AI overview The article examines the November 2025 Cloudflare outage, explaining how an oversized Bot Management configuration caused HTTP 5XX errors and cascading failures across dependent services. It highlights configuration propagation, service dependencies, and chaos engineering as reliability considerations. ### Source excerpt In November 2025, a misconfigured Cloudflare service led to a partial outage. Learn what happened, and what you can do to reduce the impact of similar outages. ## How to test the reliability of a Point of Sale (POS) system DevFeed: [How to test the reliability of a Point of Sale (POS) system]() Original publisher: [Read original article]() Author: Gavin Cahill Published: 2025-10-20T00:00:00Z Content type: tutorial Language: en Sources: [Gremlin Blog]() Topics: [Chaos Engineering](), [Resilience](), [autoscaling](), [Microservice](), [Complex Systems](), [systems]() Tags: [autoscaling](), [best-practices](), [chaos-engineering](), [cpu](), [engineering](), [gremlin](), [how-to](), [infrastructure](), [memory](), [microservices](), [outage](), [reliability-management](), [resilience](), [retail](), [testing]() ### AI overview This tutorial explains how to test the reliability of retail Point of Sale systems using Gremlin and Chaos Engineering. It focuses on resilience testing for microservice-based checkout systems, including autoscaling, CPU, memory, and disk I/O capacity, to identify failure conditions and reduce outages. ### Source excerpt Find out how to use Gremlin and Chaos Engineering to make sure your Point of Sale system is reliable. ## Chaos Engineering works, but it has to scale DevFeed: [Chaos Engineering works, but it has to scale]() Original publisher: [Read original article]() Author: Gavin Cahill Published: 2025-10-07T00:00:00Z Content type: article Language: en Sources: [Gremlin Blog]() Topics: [Chaos Engineering](), [Resilience](), [Testing](), [distributed-systems](), [Availability](), [site-reliability-engineering]() Tags: [availability](), [chaos-engineering](), [outages](), [resilience](), [sre](), [testing](), [tests](), [uptime]() ### AI overview Chaos Engineering can uncover failure modes and help prevent outages, but organization-wide adoption may stall when expertise is concentrated in a small number of teams. The article recommends scaling the practice through standards, validation testing, and reporting so that reliability improvements extend beyond critical services. ### Source excerpt Chaos Engineering effectively improves the reliability of systems, but it can run into snags when you try to scale. Build on Chaos Engineering with these key actions. ## How to get fast, easy insights with the Gremlin MCP Server DevFeed: [How to get fast, easy insights with the Gremlin MCP Server]() Original publisher: [Read original article]() Author: Gavin Cahill Published: 2025-08-28T00:00:00Z Content type: article Language: en Sources: [Gremlin Blog]() Topics: [MCP Server](), [Model Context Protocol](), [Chaos Engineering](), [Large Language Model](), [test-coverage](), [Authorization](), [Containers](), [dashboards](), [Security](), [API keys]() Tags: [access-control](), [ai](), [api](), [api-keys](), [applications](), [chaos-engineering](), [chatgpt](), [claude](), [components](), [core](), [data](), [gremlin](), [hardening](), [how-to](), [insights](), [installation](), [llm](), [management](), [mcp](), [mcp-server]() ### AI overview The article explains how the Gremlin MCP Server connects an LLM client to the Gremlin API so teams can explore reliability-testing data and identify insights using natural-language prompts. It covers the client-server architecture, containerized deployment, security hardening, non-destructive API operations, dashboards, reporting, evaluation, and RBAC-based control of API keys. ### Source excerpt Find out how to quickly and easily uncover new reliability insights by using the Gremlin MCP Server and your favorite LLM. ## Fix issues faster with Recommended Remediations DevFeed: [Fix issues faster with Recommended Remediations]() Original publisher: [Read original article]() Author: Gavin Cahill Published: 2025-08-22T00:00:00Z Content type: release Language: en Sources: [Gremlin Blog]() Topics: [Chaos Engineering](), [Testing]() Tags: [chaos-engineering](), [features](), [gremlin](), [recommendations](), [uptime]() ### AI overview Gremlin's Recommended Remediation analyzes fault-injection test results to identify likely failure causes and suggest ways to address reliability issues. It builds on Experiment Analysis by combining test data, metrics, health checks, and key events, then applies reliability expertise to produce tailored recommendations. ### Source excerpt Recommended Remediation speeds up teams with tailored suggestions to help you address reliability risks before they cause failures. ## How Experiment Analysis uncovers the cause behind failures DevFeed: [How Experiment Analysis uncovers the cause behind failures]() Original publisher: [Read original article]() Author: Gavin Cahill Published: 2025-08-15T00:00:00Z Content type: article Language: en Sources: [Gremlin Blog]() Topics: [experiments](), [Chaos Engineering](), [observability](), [Machine learning](), [Availability]() Tags: [ai](), [analysis](), [availability](), [chaos-engineering](), [experiment](), [failover](), [gremlin](), [observability](), [testing]() ### AI overview The article explains how Gremlin's Experiment Analysis uses machine learning and observability context to identify why Chaos Engineering tests fail, reducing the manual effort needed to investigate system behavior. It also emphasizes defining expected behavior as the basis for pass/fail decisions, illustrated by a zone failover scenario. ### Source excerpt Using knowledge of your data and your systems, Experiment Analysis connects the cause with the effect to help you pinpoint the root problem faster. ## Antifragile Systems and Teams DevFeed: [Antifragile Systems and Teams]() Original publisher: [Read original article]() Author: Tom Published: 2025-08-14T00:00:00Z Content type: opinion Language: en Sources: [Tomasz Gągor]() Topics: [systems](), [Resilience](), [DevOps](), [Chaos Engineering](), [Automation](), [Tooling](), [Netflix]() Tags: [automation](), [chaos](), [chaos-engineering](), [culture](), [devops](), [netflix](), [resilience](), [systems](), [technology]() ### AI overview A concise review of Dave Zwieback's book about antifragile systems and teams. It explains how organizations can use change, mistakes, and small failures to learn and become stronger, emphasizing DevOps practices such as culture, automation, measurement, sharing, chaos engineering, and frequent small deployments. ### Source excerpt Antifragile Systems and Teams Author: Dave Zwieback This is a short read, but it does a solid job of capturing an important idea: that the healthiest systems and teams aren't just resistant to change, they actually get stronger through it. Zwieback contrasts fragile organizations - those that try to lock things down and prevent every possible failure - with antifragile ones, which use volatility, mistakes, and small shocks as opportunities to learn and improve. ## Reliability Intelligence: your reliability expert DevFeed: [Reliability Intelligence: your reliability expert]() Original publisher: [Read original article]() Author: Gavin Cahill Published: 2025-08-11T00:00:00Z Content type: release Language: en Sources: [Gremlin Blog]() Topics: [Chaos Engineering](), [experiments](), [Artificial Intelligence](), [telemetry](), [site-reliability-engineering](), [MCP Server]() Tags: [ai](), [announcements](), [chaos-engineering](), [experiment](), [mcp-server](), [release](), [sre](), [telemetry]() ### AI overview Gremlin announces Reliability Intelligence, a release that combines Experiment Analysis, Recommended Remediation, and an MCP Server. It uses Gremlin's reliability expertise, telemetry, and trace data to help engineers run reliability tests, identify root causes, detect regressions, and remediate issues faster without slowing deployment velocity. ### Source excerpt Gremlin's Reliability Intelligence combines Experiment Analysis, Recommended Remediation, and an MCP Server to help teams increase reliability faster than ever. ## 4 Chaos Engineering recommendations from Gartner DevFeed: [4 Chaos Engineering recommendations from Gartner]() Original publisher: [Read original article]() Author: Gavin Cahill Published: 2025-07-11T00:00:00Z Content type: article Language: en Sources: [Gremlin Blog]() Topics: [Chaos Engineering](), [Resilience](), [genai](), [Testing]() Tags: [ai](), [analysts](), [chaos-engineering](), [engineering](), [generative-ai](), [resilience](), [test](), [testing]() ### AI overview The article presents four Gartner recommendations for adopting Chaos Engineering, including testing generative AI API fallback patterns and using scenario-based GameDays to evaluate outage response and identify reliability risks. ### Source excerpt Gartner recently published their 2025 Hype Cycle for Infrastructure Platforms. Here are four key recommendations from them for adopting Chaos Engineering. ## Infographic: Resilience and reliability in the cloud DevFeed: [Infographic: Resilience and reliability in the cloud]() Original publisher: [Read original article]() Author: Gavin Cahill Published: 2025-02-25T00:00:00Z Content type: article Language: en Sources: [Gremlin Blog]() Topics: [Resilience](), [Cloud](), [Chaos Engineering](), [observability](), [Amazon Web Services]() Tags: [aws](), [chaos-engineering](), [cloud](), [cost-of-downtime](), [observability](), [outages](), [resilience](), [testing](), [uptime]() ### AI overview This infographic explains why resilience and reliability matter for cloud-based software systems. It presents outage impacts and causes, discusses resilience testing and Chaos Engineering with tools such as Gremlin, and describes how organizations can evaluate the return on investment of reliability efforts. ### Source excerpt Created in partnership with AWS, this infographic shows the impact of outages, the most common causes of outages, and the results companies get from investing in resilience. ## How the Gremlin agent fails safely DevFeed: [How the Gremlin agent fails safely]() Original publisher: [Read original article]() Author: Andre Newman Published: 2025-01-30T00:00:00Z Content type: article Language: en Sources: [Gremlin Blog]() Topics: [Chaos Engineering](), [experiments](), [Integration testing](), [Linux](), [Network]() Tags: [architecture](), [chaos-engineering](), [data-center](), [gremlin](), [integration](), [linux](), [network](), [server](), [testing]() ### AI overview The article explains how the Gremlin agent makes Chaos Engineering and reliability testing safer. Agents periodically check in with the Gremlin Control Plane and automatically stop active experiments and restore systems if communication is lost. ### Source excerpt Reliability testing shouldn't feel risky. Learn how Gremlin makes testing safer with fail-safe agents and automatic rollbacks. ## How to fix the root cause of a failed reliability test DevFeed: [How to fix the root cause of a failed reliability test]() Original publisher: [Read original article]() Author: Andre Newman Published: 2025-01-21T00:00:00Z Content type: tutorial Language: en Sources: [Gremlin Blog]() Topics: [Resilience](), [Scalability](), [Availability](), [cpu](), [Latency](), [Network](), [observability](), [TLS (Transport Layer Security)](), [systems]() Tags: [availability](), [blog](), [chaos-engineering](), [cpu](), [gremlin](), [how-to](), [latency](), [metrics](), [network](), [resilience](), [scalability](), [systems](), [tls]() ### AI overview This tutorial explains how to investigate failed Gremlin reliability tests and use their results to improve system resilience. It describes the Well-Architected Cloud Test Suite, covering scalability, redundancy, dependency failures, latency, and TLS certificate expiry, along with the role of Health Checks and observability metrics and alerts. ### Source excerpt You've run your reliability tests, and unfortunately, some of them failed. No need to panic: we'll tell you how to turn that F into an A. ## What's the ROI of reliability? DevFeed: [What's the ROI of reliability?]() Original publisher: [Read original article]() Author: Gavin Cahill Published: 2025-01-13T00:00:00Z Content type: article Language: en Sources: [Gremlin Blog]() Topics: [Chaos Engineering](), [Availability](), [observability](), [Instrumentation](), [Tooling](), [systems]() Tags: [availability](), [chaos-engineering](), [cost-of-downtime](), [instrumentation](), [observability](), [outages](), [reduce](), [tooling]() ### AI overview This article explains how to calculate the return on investment of reliability and Chaos Engineering programs. It frames the Amount Gained as losses reduced by improved reliability and the Amount Spent as the salaries, tools, and resources required, with availability and downtime used to illustrate the potential business impact. ### Source excerpt Learn how to compute the ROI of a reliability or Chaos Engineering program, including how to quantify the positive impact your efforts created for the company. ## Chaos Engineering and Resilience Testing Tools: Build vs Buy DevFeed: [Chaos Engineering and Resilience Testing Tools: Build vs Buy]() Original publisher: [Read original article]() Author: Gavin Cahill Published: 2024-10-04T00:00:00Z Content type: comparison Language: en Sources: [Gremlin Blog]() Topics: [Chaos Engineering](), [Resilience](), [Testing](), [Tool]() Tags: [chaos-engineering](), [cost](), [customization](), [open-source](), [pipelines](), [private-cloud](), [resilience](), [saas](), [testing](), [vs]() ### AI overview This article compares building an in-house fault-injection tool with buying a commercial offering for Chaos Engineering and resilience testing. It discusses customization, roadmap and network control as potential benefits of building, alongside the engineering costs and maintenance responsibilities involved. ### Source excerpt Not sure whether you should build or buy a Fault Injection tool for Chaos Engineering and resilience testing? Check out the pros and cons of building vs buying. ## How role-based access control (RBAC) works in Gremlin DevFeed: [How role-based access control (RBAC) works in Gremlin]() Original publisher: [Read original article]() Author: Andre Newman Published: 2024-07-25T00:00:00Z Content type: article Language: en Sources: [Gremlin Blog]() Topics: [Authorization](), [Security](), [Chaos Engineering](), [experiments]() Tags: [access-control](), [blog-post](), [chaos-engineering](), [engineering](), [gremlin](), [least-privilege](), [security](), [testing]() ### AI overview This blog post explains how Gremlin's customizable role-based access control (RBAC) manages user privileges for reliability testing and Chaos Engineering. It describes company and team roles, additive privileges, safety restrictions, and the principle of least privilege. ### Source excerpt Gremlin recently released custom role-based access controls (RBAC) for greater control over your reliability testing. Learn how it works in this blog post. ## Intelligent Health Checks: one-click observability for reliability tests DevFeed: [Intelligent Health Checks: one-click observability for reliability tests]() Original publisher: [Read original article]() Author: Andre Newman Published: 2024-07-09T00:00:00Z Content type: article Language: en Sources: [Gremlin Blog]() Topics: [observability](), [Chaos Engineering](), [telemetry](), [Amazon Web Services]() Tags: [aws](), [blog](), [blog-post](), [chaos-engineering](), [development](), [gremlin](), [metrics](), [monitoring](), [observability](), [testing]() ### AI overview Gremlin's Intelligent Health Checks automate observability for reliability tests. With one click, Gremlin creates Health Checks that select and monitor critical metrics or HTTP endpoints, establish service baselines, and stop tests when measurements exceed acceptable thresholds. The article explains how this reduces manual metric and threshold configuration, including for AWS services. ### Source excerpt Figuring out what to monitor can be a challenge. That's why Gremlin does it for you. Learn how Gremlin automatically creates and monitors critical metrics for your AWS services. ## How to build reliable services with unreliable dependencies DevFeed: [How to build reliable services with unreliable dependencies]() Original publisher: [Read original article]() Author: Andre Newman Published: 2024-05-02T00:00:00Z Content type: article Language: en Sources: [Gremlin Blog]() Topics: [Availability](), [systems](), [Software as a service](), [LAMP](), [Network](), [Linux](), [MySQL](), [PHP]() Tags: [availability](), [blog](), [chaos-engineering](), [cloud](), [gremlin](), [how-to](), [linux](), [mysql](), [network](), [php](), [saas](), [systems]() ### AI overview This Gremlin blog explains how failed service dependencies can reduce application reliability. It discusses risks such as requests that wait indefinitely, crashes, exposed internal errors, and ineffective failover systems, then introduces proactive testing to help services withstand dependency failures and improve availability. ### Source excerpt Dependencies are everywhere, and they make reliability work difficult. How can you build reliable when you depend on services that could fail at any time, and that you have no control over? Our latest blog has the answers. ## How to make your services resilient to slow dependencies DevFeed: [How to make your services resilient to slow dependencies]() Original publisher: [Read original article]() Author: Andre Newman Published: 2024-04-24T00:00:00Z Content type: article Language: en Sources: [Gremlin Blog]() Topics: [Resilience](), [Databases](), [Software as a service](), [Network](), [Authentication](), [incident]() Tags: [authentication](), [blog-post](), [chaos-engineering](), [databases](), [dependencies](), [incident](), [network](), [resilience](), [saas](), [software]() ### AI overview This blog post explains how service dependencies can become sources of reliability problems and how applications can remain available and responsive when those dependencies are slow, unstable, or unavailable. ### Source excerpt Our applications increasingly rely on services we don't control. What happens when those services become unreliable? This blog post explains how to build software that stays available and responsive, even if your dependencies aren't. [Next page]()