# Evan Miller Evan Miller's News This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Likelihood-ratio inference on differences in quantiles DevFeed: [Likelihood-ratio inference on differences in quantiles]() Original publisher: [Read original article]() Author: Miller, Evan Published: 2024-08-05T10:15:00Z Content type: article Language: en Sources: [Evan Miller]() Topics: [Statistics](), [Inference]() Tags: [confidence-interval](), [experimentation](), [statistics]() ### AI overview This paper presents a two-sample hypothesis test and confidence interval for differences in quantiles using a likelihood-ratio test statistic. A conservative version avoids density estimation, while another version uses a density estimator and produces confidence intervals close to nominal coverage. The method can be computed from four order statistics from each sample. ### Source excerpt Quantiles can represent key operational and business metrics, but the computational challenges associated with inference has hampered their adoption in online experimentation. In a new paper, I present a two-sample difference-in-quantile hypothesis test and confidence interval based on a likelihood-ratio test statistic. It can be computed using only four order statistics from each sample. arXiv link: Likelihood-ratio inference on differences in quantiles ## Attention Is Off By One DevFeed: [Attention Is Off By One]() Original publisher: [Read original article]() Author: Evan Miller Published: 2023-07-24T16:05:00Z Content type: opinion Language: en Sources: [Evan Miller]() Topics: [Transformer](), [quantization](), [Large Language Model](), [AI Models]() Tags: [central](), [cloud](), [edge](), [llms](), [pytorch](), [quantization]() ### AI overview Evan Miller argues that the standard Transformer attention formula contains an overlooked off-by-one error. He discusses how outlier weights and activations complicate quantization and deployment, and proposes investigating fixes including Softmax One and QuietAttention. ### Source excerpt Transformer has a mathematical bug that has been overlooked for 6+ years. I propose fixing its outliers with two new devices, Softmax One and QuietAttention: Attention Is Off By One ## Using the Frisch-Waugh-Lovell Theorem to Improve CUPED Variance Reduction in Online Experiments DevFeed: [Using the Frisch-Waugh-Lovell Theorem to Improve CUPED Variance Reduction in Online Experiments]() Original publisher: [Read original article]() Author: Evan Miller Published: 2022-07-15T00:35:00Z Content type: tutorial Language: en Sources: [Evan Miller]() Topics: [experiments](), [math](), [data]() Tags: [analysis](), [experiments](), [math](), [reduction](), [variance]() ### AI overview This tutorial explains the mathematical connection between CUPED and partial linear regression through the Frisch-Waugh-Lovell Theorem. It discusses how variance reduction can decrease experiment sample sizes and proposes including a vector of treatments in CUPED regressions for additional variance reduction. ### Source excerpt A/B tests run faster with CUPED. Here I explain the underlying math, and use it to invent an even better variance-reduction technique for online experiments: You Can't Spell CUPED Without Frisch-Waugh-Lovell ## The Floppy Disk of Floating Point DevFeed: [The Floppy Disk of Floating Point]() Original publisher: [Read original article]() Author: Evan Miller Published: 2020-07-13T13:35:00Z Content type: opinion Language: en Sources: [Evan Miller]() Topics: [Computing](), [floating-point](), [x86](), [Assembly](), [cpu]() Tags: [architecture](), [assembly](), [computing](), [cpu](), [floating-point](), [precision](), [standard](), [x86]() ### AI overview An essay reflecting on the x87 floating-point instruction set and its 80-bit extended-precision design. It describes the architecture's historical hardware support for mathematical functions and considers its eventual obsolescence. ### Source excerpt An essay that bids farewell to x87, a computing architecture too long for this world: The Floppy Disk of Floating Point ## Updated: Formulas for Bayesian A/B Testing DevFeed: [Updated: Formulas for Bayesian A/B Testing]() Original publisher: [Read original article]() Author: Evan Miller Published: 2020-04-08T11:15:00Z Content type: tutorial Language: en Sources: [Evan Miller]() Topics: [A/B Testing](), [math]() Tags: [a-b-testing](), [math](), [test]() ### AI overview This article presents closed-form formulas for evaluating Bayesian A/B tests, including the probability that one variant will outperform another in the long run. It covers binary outcomes, beta distributions, derivation, and implementation considerations. ### Source excerpt I've updated Formulas for Bayesian A/B Testing to include a new equation for analyzing four-pronged (A/B/C/D) tests. ## SlowerLogLog DevFeed: [SlowerLogLog]() Original publisher: [Read original article]() Author: Evan Miller Published: 2020-02-06T18:00:00Z Content type: article Language: en Sources: [Evan Miller]() Topics: [Data structures](), [Algorithm](), [math](), [hash](), [implementation]() Tags: [algorithm](), [data-structure](), [hash](), [implementation](), [math](), [probabilistic]() ### AI overview This article introduces SlowerLogLog, a variation of HyperLogLog for estimating the number of unique elements in a set. It explains how Maximum Likelihood Estimation can estimate cardinality and standard error, support small cardinalities, and avoid requiring a power-of-two register count. The method is easier to derive and implement but much more computationally expensive because each element is hashed once per register. ### Source excerpt What rolls down stairs and counts unique items imprecisely? HyperLogLog. Also SlowerLogLog ## Preface to Frankenstein DevFeed: [Preface to Frankenstein]() Original publisher: [Read original article]() Author: Evan Miller Published: 2019-03-12T13:55:00Z Content type: opinion Language: en Sources: [Evan Miller]() Topics: [Artificial Intelligence](), [Deep learning]() Tags: [ai](), [deep-learning](), [science]() ### AI overview This preface argues that Mary Shelley's Frankenstein remains relevant to computer programmers because it explores creation, obsessive making, and the dangers of pursuing science and engineering without sufficient reflection. It connects those themes to efforts to develop general AI and questions the ethical relationship between humans and self-aware computers. ### Source excerpt Frankenstein turned two hundred last year. Why is it still relevant? My very own Preface to Frankenstein ## Things That Bother Me About Swift DevFeed: [Things That Bother Me About Swift]() Original publisher: [Read original article]() Author: Evan Miller Published: 2018-12-29T18:50:00Z Content type: opinion Language: en Sources: [Evan Miller]() Topics: [Swift](), [iphone](), [Mobile](), [Objective-C]() Tags: [iphone](), [mobile](), [objective-c](), [swift]() ### AI overview Evan Miller describes criticisms of Swift 4.2 based on reading its language guide and attempting to compile a small iPhone app. He focuses on confusing type-inference failures and Swift statements that do not allow empty clauses. ### Source excerpt I'm learning Swift. Some minor grievances I have with the language: Things That Bother Me About Swift ## Things That Bother Me About macOS DevFeed: [Things That Bother Me About macOS]() Original publisher: [Read original article]() Author: Evan Miller Published: 2018-12-20T01:36:00Z Content type: opinion Language: en Sources: [Evan Miller]() Topics: [macOS](), [Programming]() Tags: [apple](), [cloud](), [mac](), [macos](), [monitor](), [spotlight]() ### AI overview An opinion article catalogs frustrations with macOS, focusing on Spotlight's search results and Time Machine's split controls for starting backups and monitoring progress. ### Source excerpt In honor of the upcoming Festivus, I am sharing some Mac-related grievances: Things That Bother Me About macOS ## A Review of Perl 6 DevFeed: [A Review of Perl 6]() Original publisher: [Read original article]() Author: Evan Miller Published: 2017-08-13T18:05:00Z Content type: opinion Language: en Sources: [Evan Miller]() Topics: [Raku](), [Programming]() Tags: [languages](), [perl](), [programming-languages](), [review]() ### AI overview A review of Perl 6, now called Raku, based on several weeks of use. It discusses the language's long development, 2015 release, uncertain market prospects, limited guidance for practical use, and support for multiple programming paradigms. ### Source excerpt I spent a few weeks with Perl 6. Here's what I think: A Review of Perl 6 ## Why I'm Learning Perl 6 DevFeed: [Why I'm Learning Perl 6]() Original publisher: [Read original article]() Author: Evan Miller Published: 2017-07-25T14:50:00Z Content type: opinion Language: en Sources: [Evan Miller]() Topics: [Perl](), [Concurrency](), [Programming](), [Programming language]() Tags: [c](), [concurrency](), [perl](), [programming-language](), [unicode](), [virtual-machine]() ### AI overview Evan Miller explains why he is exploring Perl 6, focusing on its runtime and concurrency capabilities. He highlights MoarVM's event-loop-aware scheduler, continuations, lightweight worker routines, and concurrent bytecode execution without a GIL, while noting that the article is not a full language review. ### Source excerpt The Perl 6 virtual machine is a promising piece of technology: Why I'm Learning Perl 6 ## Adventure Games and Eigenvalues DevFeed: [Adventure Games and Eigenvalues]() Original publisher: [Read original article]() Author: Evan Miller Published: 2017-04-23T10:10:00Z Content type: tutorial Language: en Sources: [Evan Miller]() Topics: [Mathematics](), [Algorithms, Complexity](), [Formal verification](), [Graphs]() Tags: [eigenvalues](), [formal-verification](), [graph-theory](), [matrix](), [state]() ### AI overview This article explains how Markov theory and matrix manipulation can prove the absence of dead ends in a small adventure game and quantify the risk of reaching invalid states. It compares this approach with formal verification and graph-based route-finding algorithms, while noting that all approaches become impractical for large state spaces. ### Source excerpt Identifying and quantifying dead ends, the bane of all adventure game players: Adventure Games and Eigenvalues ## Big Tablets Are Coming DevFeed: [Big Tablets Are Coming]() Original publisher: [Read original article]() Author: Evan Miller Published: 2016-12-05T06:10:00Z Content type: opinion Language: en Sources: [Evan Miller]() Topics: [Microsoft]() Tags: [apple](), [computers](), [desktop](), [microsoft](), [tablet](), [work]() ### AI overview An opinion piece reflecting on the Microsoft Surface Studio and Apple's anticipated giant iPad. It argues that large, desk-oriented computers may appeal to people who want to work productively in increasingly distracting open offices. ### Source excerpt Reflections on the Microsoft Surface Studio and Apple's inevitable giant iPad: Big Tablets Are Coming ## Type Punning Functions in C DevFeed: [Type Punning Functions in C]() Original publisher: [Read original article]() Author: Evan Miller Published: 2016-08-10T15:20:00Z Content type: article Language: en Sources: [Evan Miller]() Topics: [C](), [functions](), [x86](), [Assembly](), [cpu]() Tags: [assembly](), [c](), [cpu](), [functions](), [standard](), [unix](), [windows](), [x86-64]() ### AI overview This article demonstrates how C function pointers can be used with incompatible argument orders, producing type-punning behavior that depends on x86 calling conventions rather than the C standard. It explains how register-based argument passing contributes to the observed results and warns that the technique is architecture-dependent and unsafe. ### Source excerpt Making C more flexible, with the help of x86_64 calling conventions: Type Punning Functions in C ## Elixir's In-Memory Rendering of HTML Templates DevFeed: [Elixir's In-Memory Rendering of HTML Templates]() Original publisher: [Read original article]() Author: Evan Miller Published: 2016-04-26T14:20:00Z Content type: article Language: en Sources: [Evan Miller]() Topics: [Elixir](), [HTML](), [systems](), [benchmarking](), [Erlang]() Tags: [benchmarking](), [elixir](), [html](), [systems]() ### AI overview This technical article examines how Elixir renders HTML templates in memory. It compares the system calls and data structures involved with ordinary file-writing code, explaining how gathered writes and nested lists contribute to Elixir's performance characteristics and affect benchmarking. ### Source excerpt A systems-y look at HTML template rendering: Elixir RAM and the Template of Doom ## Simple Sequential A/B Testing DevFeed: [Simple Sequential A/B Testing]() Original publisher: [Read original article]() Author: Evan Miller Published: 2015-10-13T12:15:00Z Content type: article Language: en Sources: [Evan Miller]() Topics: [A/B Testing](), [Testing]() Tags: [a-b-testing](), [conversion](), [sequential-testing](), [statistical-significance](), [testing]() ### AI overview Evan Miller presents a simple sequential A/B testing procedure that allows experiments to stop early when treatment performance appears superior or to stop without declaring a winner when the combined number of successes reaches the chosen sample size. The method is described as especially effective for low-conversion-rate experiments. ### Source excerpt Cut your A/B sample sizes in half, using this one weird trick: Simple Sequential A/B Testing ## Evaluating Splatoon's Ranking System DevFeed: [Evaluating Splatoon's Ranking System]() Original publisher: [Read original article]() Author: Evan Miller Published: 2015-07-21T00:30:00Z Content type: article Language: en Sources: [Evan Miller]() Topics: [math](), [bug]() Tags: [analysis](), [bug](), [probability-theory]() ### AI overview Evan Miller analyzes Splatoon's Ranked Battle ranking system using probability theory and matrix math. He finds that if battles are decided purely by chance, nearly 75% of players eventually reach A−, A, or A+, including more than 36% at A+, revealing a subtle flaw in the system. ### Source excerpt Evaluating Splatoon's Ranking System -- Using probability theory to uncover a subtle bug in Splatoon. ## Inferring Tweet Quality From Retweets DevFeed: [Inferring Tweet Quality From Retweets]() Original publisher: [Read original article]() Author: Evan Miller Published: 2015-07-18T03:15:00Z Content type: article Language: en Sources: [Evan Miller]() Topics: [X (Twitter)](), [Network](), [Users]() Tags: [follower](), [metric](), [model](), [network](), [random](), [twitter]() ### AI overview This article develops a model for estimating a tweet's quality as the percentage of readers who retweet it. It considers the tweet's age, random user reload behavior, and retweeting beyond the author's immediate followers using publicly available Twitter information. ### Source excerpt Inferring Tweet Quality From Retweets -- Modeling tweets, retweets, and the passage of time. ## Ranking News Items With Upvotes DevFeed: [Ranking News Items With Upvotes]() Original publisher: [Read original article]() Author: Evan Miller Published: 2015-07-14T19:15:00Z Content type: tutorial Language: en Sources: [Evan Miller]() Topics: [recommendation systems](), [Reddit]() Tags: [model](), [ranking](), [reddit]() ### AI overview Evan Miller derives a ranking formula for social news items when users can upvote but not downvote. The method estimates expected utility from a story's age and upvote count, and shows that rankings can be computed without knowing the site's total user count. ### Source excerpt Ranking News Items With Upvotes -- A formula for social news websites. ## Deriving the Reddit Formula DevFeed: [Deriving the Reddit Formula]() Original publisher: [Read original article]() Author: Evan Miller Published: 2015-07-14T15:15:00Z Content type: article Language: en Sources: [Evan Miller]() Topics: [Reddit](), [math](), [Users]() Tags: [analysis](), [expression](), [lambda](), [math](), [reddit](), [theory]() ### AI overview Evan Miller derives a Reddit "hot" formula using expected-utility theory. The analysis explains the formula's constants, logarithm, absolute value, and lack of current time, estimates optimization for users who visit every 5.43 hours, and proposes a caveated amendment. ### Source excerpt Deriving the Reddit Formula -- What can expected-utility theory tell us about Reddit's "hot" formula? ## A Taste of Rust DevFeed: [A Taste of Rust]() Original publisher: [Read original article]() Author: Evan Miller Published: 2015-05-14T17:25:00Z Content type: opinion Language: en Sources: [Evan Miller]() Topics: [Rust](), [Programming language](), [Programming]() Tags: [programming](), [programming-language](), [rust](), [software]() ### AI overview Evan Miller offers a humorous first impression of Rust, discussing the language's name and comparing Rust's protective role in multithreaded software to the weather-resistant coating formed by rust on Corten steel. The article also includes a later update acknowledging that the author did not fully appreciate Rust's optimization levels. ### Source excerpt A Taste of Rust -- I tried Rust, and it was better than eating a bucket of nails. ## Four Days of Go DevFeed: [Four Days of Go]() Original publisher: [Read original article]() Author: Evan Miller Published: 2015-04-21T22:55:00Z Content type: opinion Language: en Sources: [Evan Miller]() Topics: [Go Language](), [Reverse Engineering](), [Programming](), [cross-platform](), [Open Source](), [Terminal]() Tags: [cross-platform](), [go](), [open-source](), [reverse-engineering](), [terminal]() ### AI overview Evan Miller describes building a cross-platform, open-source hex editor for reverse-engineering binary file formats and explains why he evaluates Swift, browser technologies, C++ toolkits, and terminal applications before choosing a development platform. ### Source excerpt Four Days of Go -- If you see the Go gopher, tell him I would like my pajamas back.