# How I Learned To Stop Writing Brittle Tests and Love Expressive APIs

DevFeed: [How I Learned To Stop Writing Brittle Tests and Love Expressive APIs](<https://devfeed.tech/articles/how-i-learned-to-stop-writing-brittle-tests-and-love-expressive-apis-23852.md>)

Original publisher: [Read original article](<http://testing.googleblog.com/2024/04/how-i-learned-to-stop-writing-brittle.html>)

Author: Google Testing Bloggers (noreply@blogger.com)

Published: 2024-04-18T12:50:00Z

Content type: tutorial

Language: en

Sources: [Google Testing Blog](<https://devfeed.tech/sources/google-testing-blog.md>)

Topics: [Testing](<https://devfeed.tech/topics/testing.md>), [C++](<https://devfeed.tech/topics/c-plus-plus.md>), [Code](<https://devfeed.tech/topics/code.md>), [API](<https://devfeed.tech/topics/api.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [c-plus-plus](<https://devfeed.tech/tags/c-plus-plus.md>), [code](<https://devfeed.tech/tags/code.md>), [implementation](<https://devfeed.tech/tags/implementation.md>), [testing](<https://devfeed.tech/tags/testing.md>), [tests](<https://devfeed.tech/tags/tests.md>), [titus-winters](<https://devfeed.tech/tags/titus-winters.md>), [tott](<https://devfeed.tech/tags/tott.md>)

## AI overview

This article explains how brittle tests can fail because of irrelevant implementation details, such as error-message changes, metadata ordering, or mock call ordering. It recommends expressive test APIs that state the properties that matter, illustrated with a C++ GoogleTest example using UnorderedElementsAre instead of order-dependent ElementsAre.

## Source excerpt

This article was adapted from a Google Testing on the Toilet (TotT) episode. You can download a printer-friendly version of this TotT episode and post it in your office. By Titus Winters A valuable but challenging property for tests is "resilience," meaning a test should only fail when something important has gone wrong. However, an opposite property may be easier to see: A "brittle" test is one that fails not for real problems that would break in production, but because the test itself is fragile for innocuous reasons. Error messages, changing the order of metadata headers in a web request, or the order of calls to a heavily-mocked dependency can often cause a brittle test to fail. Expressive test APIs are a powerful tool in the fight against brittle, implementation-detail heavy tests. A test written with IsSquare(output) is more expressive (and less brittle) than a test written with details such as JsonEquals(.width = 42, .length = 42), in cases where the size of the square is irrelevant. Similar expressive designs might include unordered element matching for hash containers, metadata comparisons for photos, and activity logs in processing objects, just to name a few. As an example, consider this C++ test code: absl::flat_hash_set<int> GetValuesFromConfig(const Config&); TEST(ConfigValues, DefaultConfigsArePrime) { // Note the strange order of these values. BAD CODE, DON'T DO THIS! EXPECT_THAT(GetValuesFromConfig(Config()), ElementsAre(29, 17, 31)); } The reliance on hash ordering makes this test brittle, preventing improvements to the API being tested. A critical part of the fix to the above code was to provide better test APIs that allowed engineers to more effectively express the properties that mattered. Thus we added UnorderedElementsAre to the GoogleTest test framework and refactored brittle tests to use that: TEST(ConfigValues, DefaultConfigsArePrimeAndOrderDoesNotMatter) { EXPECT_THAT(GetValuesFromConfig(Config()), UnorderedElementsAre(17, 29, 31)); } It's