# Software failures, invisible complexity, and the limits of code

DevFeed: [Software failures, invisible complexity, and the limits of code](<https://devfeed.tech/articles/saving-the-world-from-code-25326.md>)

Original publisher: [Read original article](<https://kau.sh/blog/saving-the-world-from-code/>)

Author: Kaushik Gopal

Published: 2022-06-18T07:00:00Z

Content type: opinion

Language: en

Sources: [Kaushik Gopal's Site](<https://devfeed.tech/sources/kaushik-gopal-s-site.md>)

Topics: [Code](<https://devfeed.tech/topics/code.md>), [Software](<https://devfeed.tech/topics/software.md>), [Hardware](<https://devfeed.tech/topics/hardware.md>), [Visual Studio](<https://devfeed.tech/topics/visual-studio.md>), [Fortran](<https://devfeed.tech/topics/fortran.md>)

Tags: [code](<https://devfeed.tech/tags/code.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [nasa](<https://devfeed.tech/tags/nasa.md>), [software](<https://devfeed.tech/tags/software.md>), [visual-studio](<https://devfeed.tech/tags/visual-studio.md>)

## AI overview

This commentary examines how software can fail despite executing its instructions correctly, arguing that failures often arise from misunderstood requirements and the invisible complexity of computing. It discusses software-related safety investigations, Visual Studio's scale, and how programming languages and interfaces shape how people understand code.

## Source excerpt

A good friend of mine who's worked on some pretty pivotal hardware/software integrations for our time told me: "it feels like a miracle when I see the code I wrote drive physical bits. I almost can't believe it. No seriously... how does this even work?". If this sentiment resonates with you to some level, you will love reading this article: The coming software apocalypse ## This entire article is fascinating. It took a lot of restraint not to just pull-quote every paragraph from this article. The software did exactly what it was told to do. In fact it did it perfectly. The reason it failed is that it was told to do the wrong thing. Software failures are failures of understanding, and of imagination. This is the trouble with making things out of code, as opposed to something physical. "The complexity," as Leveson puts it, "is invisible to the eye." ..."Computing is fundamentally invisible," Gerard Berry said in his talk. "When your tires are flat, you look at your tires, they are flat. When your software is broken, you look at your software, you see nothing." The National Highway Traffic Safety Administration enlisted software experts from NASA to perform an intensive review of Toyota's code. After nearly 10 months, the NASA team hadn't found evidence that software was the cause -- but said they couldn't prove it wasn't. ... They showed that as little as a single bit flip -- a one in the computer's memory becoming a zero or vice versa -- could make a car run out of control." "Visual Studio is one of the single largest pieces of software in the world," he said. "It's over 55 million lines of code. And one of the things that I found out in this study is more than 98 percent of it is completely irrelevant. All this work had been put into this thing, but it missed the fundamental problems that people faced. And the biggest one that I took away from it was that basically people are playing computer inside their head. ... So the students who did well -- in fact the only ones who survi