# How to Test a Compiler Against Itself: Thirteen Generators and 7,800 Programs

DevFeed: [How to Test a Compiler Against Itself: Thirteen Generators and 7,800 Programs](<https://devfeed.tech/articles/how-to-test-a-compiler-against-itself-thirteen-generators-and-7-800-programs-93928.md>)

Original publisher: [Read original article](<https://hackernoon.com/how-to-test-a-compiler-against-itself-thirteen-generators-and-7800-programs?source=rss>)

Author: Tech\_World

Published: 2026-10-21T20:01:48Z

Content type: article

Language: en

Sources: [HackerNoon](<https://devfeed.tech/sources/hackernoon.md>)

Topics: [Software Testing](<https://devfeed.tech/topics/software-testing.md>), [Test automation](<https://devfeed.tech/topics/test-automation.md>), [Fuzzing/Fuzz testing](<https://devfeed.tech/topics/fuzzing.md>), [Compilers and Linkers](<https://devfeed.tech/topics/compilers-and-linkers.md>)

Tags: [back-end](<https://devfeed.tech/tags/back-end.md>), [backends](<https://devfeed.tech/tags/backends.md>), [bytecode](<https://devfeed.tech/tags/bytecode.md>), [candidate](<https://devfeed.tech/tags/candidate.md>), [chuks-programming-language](<https://devfeed.tech/tags/chuks-programming-language.md>), [chukslang](<https://devfeed.tech/tags/chukslang.md>), [compiler](<https://devfeed.tech/tags/compiler.md>), [errors](<https://devfeed.tech/tags/errors.md>), [free-software](<https://devfeed.tech/tags/free-software.md>), [generators](<https://devfeed.tech/tags/generators.md>), [http](<https://devfeed.tech/tags/http.md>), [native](<https://devfeed.tech/tags/native.md>), [nightly](<https://devfeed.tech/tags/nightly.md>), [programming](<https://devfeed.tech/tags/programming.md>), [release-notes](<https://devfeed.tech/tags/release-notes.md>), [self-testing](<https://devfeed.tech/tags/self-testing.md>), [software-engineering](<https://devfeed.tech/tags/software-engineering.md>), [test](<https://devfeed.tech/tags/test.md>), [tests](<https://devfeed.tech/tags/tests.md>)

## AI overview

Comparing Chuks's bytecode VM with its native compiler can miss bugs shared by both, so v0.2.0-rc.3 adds tests with answers independent of that comparison: 450 translated compiler regression programs, about 7,800 programs from 13 feature-combination generators, and native builds of Valet and CodeStudio. The tests exposed issues including cancelled tasks continuing to run, locks remaining held after errors, different `a || b` results across backends, and an HTTP parser accepting forbidden framing; each was fixed and made a permanent check. The generators pass on the full tree, and a higher-strength nightly run has caught a regression. Based on this evidence, the project says rc.3 will be its last candidate and plans to publish v0.2.0 from this build, while acknowledging that tests cannot prove bugs are absent.

## Source excerpt

Comparing two backends only catches disagreement. How Chuks tests against known answers with 13 generators and about 7,800 programs.