# Testing v. informal reasoning

DevFeed: [Testing v. informal reasoning](<https://devfeed.tech/articles/testing-v-informal-reasoning-77756.md>)

Original publisher: [Read original article](<https://danluu.com/tests-v-reason/>)

Published: 2014-11-03T00:00:00Z

Content type: opinion

Language: en

Sources: [Danluu](<https://devfeed.tech/sources/danluu.md>)

Topics: [Test-driven development](<https://devfeed.tech/topics/tdd.md>), [Test automation](<https://devfeed.tech/topics/test-automation.md>), [NUMA](<https://devfeed.tech/topics/numa.md>)

Tags: [high-performance](<https://devfeed.tech/tags/high-performance.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [testing](<https://devfeed.tech/tags/testing.md>)

## AI overview

The author argues that testing is necessary for complex software and hardware systems, even though it cannot prove that bugs are absent. They point to techniques such as state space reduction and symbolic execution, and to experience in CPU verification, to challenge the claim that informal reasoning is more important than testing.

## Source excerpt

This is an off-the-cuff comment for Hacker School's Paper of the Week Read Along series for Out of the Tar Pit. I find the idea itself, which is presented in sections 7-10, at the end of the paper, pretty interesting. However, I have some objections to the motivation for the idea, which makes up the first 60% of the paper. Rather than do one of those blow-by-blow rebuttals that's so common on blogs, I'll limit my comments to one widely circulated idea that I believe is not only mistaken but actively harmful. There's a claim that "informal reasoning" is more important than "testing"1, based mostly on the strength of this quote from Dijkstra: testing is hopelessly inadequate....(it) can be used very effectively to show the presence of bugs but never to show their absence. They go on to make a number of related claims, like "The key problem is that a test (of any kind) on a system or component that is in one particular state tells you nothing at all about the behavior of that system or component when it happens to be in another state.", with the conclusion that stateless simplicity is the only possible fix. Needless to say, they assume that simplicity is actually possible. I actually agree with the bit about testing -- there's no way to avoid bugs if you create a system that's too complex to formally verify. However, there are plenty of real systems with too much irreducible complexity to make simple. Drawing from my own experience, no human can possibly hope to understand a modern high-performance CPU well enough to informally reason about its correctness. That's not only true now, it's been true for decades. It becomes true the moment someone introduces any sort of speculative execution or caching. These things are inherently stateful and complicated. They're so complicated that the only way to model performance (in order to run experiments to design high performance chips) is to simulate precisely what will happen, since the exact results are too complex for humans