# outlier-detection Published articles for outlier-detection. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Outlier Handling at Scale in Experimentation DevFeed: [Outlier Handling at Scale in Experimentation]() Original publisher: [Read original article]() Author: Margarida Moreira da Silva Published: 2026-07-01T13:44:26Z Content type: article Language: en Sources: [Booking.com Data Science]() Topics: [experiments](), [data](), [Simulation](), [Ground truth / benchmark quality](), [plotting]() Tags: [data](), [experimentation](), [experiments](), [false-positive](), [outlier-detection](), [simulation](), [variance]() ### AI overview The article examines how extreme values affect experimentation at Booking.com. It describes permutation tests and simulated A/A experiments for diagnosing distorted p-value distributions, and reports that increasing outlier magnitude and frequency can cause test failures. ### Source excerpt At Booking.com, thousands of experiments run simultaneously across highly heterogeneous users, from individual travellers to large travel agencies. This means our experiment data regularly contains legitimate but extreme values. When these go unhandled, they distort the statistical conclusions we draw, leading us to scale ideas that don't create value, or to discard ones that do. So, we need outlier handling methods that are reliable, automated, and applicable across diverse metrics without manual intervention. The Problem When extreme values are present in experiment data, they can compromise the estimation of average treatment effects (ATE), leading to unreliable test results and reduced statistical power. Even a single observation can inflate variance enough to mask a real effect or produce a spurious one. In practice, this means we risk shipping changes that appear positive but are not, or killing promising features because noise masked their real effect. At Booking.com's scale, this increase in false conclusions quickly compounds into a meaningful impact on customer experience and business outcomes. A Diagnostic Tool: the Permutation Test One way to assess whether extreme values are distorting results is the permutation test. By permuting over experiment data, we generate hundreds of simulated AA experiments where we know the ground truth: there is no real effect. Plotting the resulting p-values, we expect a uniform distribution. If it instead looks skewed, the underlying data distribution is compromising the validity of results. Plot 1: P-value distributions from simulated A/A tests. Clean normally-distributed estimated effects produce a uniform distribution (left), while the presence of extreme outliers results in skewed p-values (right), indicating a distorted false positive rate.Simulation Evidence: What Drives Failure? We ran AA permutation tests across a range of simulated data distributions to understand when they fail (i.e. not show a uniform p-value di ## BIY: Preparing a Dataset and Benchmarking AI Models for Scatterplot-Related Tasks DevFeed: [BIY: Preparing a Dataset and Benchmarking AI Models for Scatterplot-Related Tasks]() Original publisher: [Read original article]() Author: João Palmeiro Published: 2026-01-19T14:52:55Z Content type: article Language: en Sources: [Feedzai]() Topics: [Accessibility](), [LLM evaluation / benchmarking](), [multimodal-ai](), [Canvas](), [AI Models](), [dataset](), [SVG]() Tags: [accessibility](), [ai-models](), [artificial-intelligence](), [benchmarking](), [canvas](), [clustering](), [data-visualization](), [dataset](), [large-language-models](), [multimodal](), [multimodal-ai](), [outlier-detection](), [svg]() ### AI overview The article introduces Benchmark It Yourself (BIY), an effort to prepare a dataset and benchmark multimodal AI models for scatterplot-related tasks. It examines generating distribution-aware descriptions from scatterplot images to improve the accessibility of canvas charts, including direct description generation and structured-data extraction for predefined templates. Initial results were mixed. ### Source excerpt Benchmark It Yourself (BIY): Preparing a Dataset and Benchmarking AI Models for Scatterplot-Related Tasks When we need to visualize and interact with millions, or even just thousands, of individual points while analyzing data, we typically resort to rendering them in the browser using a canvas. The other common approach for the web, SVG, doesn't scale when the number of individual elements increases to such quantities. However, while solving one problem, canvas charts introduce a new challenge: accessibility. Although SVG charts are not accessible by default, they can be by design. Each part of an SVG chart has a corresponding element on the web page, allowing for a programmable, accessible experience for screen reader users. We can simply think of SVG as HTML. On the other hand, a canvas chart is just like a PNG image. If a screen reader user tries to learn more about a canvas chart, unless the developer has prepared a detailed description of it, they will just hear the word "image". There's no way to get an idea of what one of these charts represents, let alone extract any insights. For static charts, the solution can be as simple as preparing a description and integrating it into the rendered chart. However, for platforms leveraging dynamic, large datasets, automatically generating these descriptions is not a simple task, especially for charts like scatterplots where data distributions can assume countless forms. At Feedzai, we started exploring ways to generate data distribution-aware descriptions for scatterplots from their respective images using recent multimodal AI models. When the raw data is not available, or the datasets are composed of several thousand or million instances, relying on chart images and these models becomes tempting. This combination has the potential to generate such descriptions and serve them alongside their respective charts, significantly improving the accessibility of canvas charts. That said, we focused on two main directions: using ## Anomaly Detection in Time Series Using Statistical Analysis DevFeed: [Anomaly Detection in Time Series Using Statistical Analysis]() Original publisher: [Read original article]() Author: Ivan Shubin Published: 2025-04-15T18:45:36Z Content type: tutorial Language: en Sources: [Booking.com Development - Medium]() Topics: [Time Series](), [Statistics](), [Monitoring](), [data](), [Website]() Tags: [analysis](), [anomaly-detection](), [article](), [behavior](), [data](), [grafana](), [metrics](), [monitoring](), [outlier-detection](), [sre](), [statistics](), [time-series](), [web]() ### AI overview This article explains how to build a statistical anomaly detection system for time series data. It describes why static thresholds and comparisons with the same point one week earlier can miss recurring or gradual problems, and introduces standard deviation as a foundational statistical measure. ### Source excerpt Setting up alerts for metrics isn't always straightforward. In some cases, a simple threshold works just fine -- for example, monitoring disk space on a device. You can just set an alert at 10% remaining, and you're covered. The same goes for tracking available memory on a server. But what if we need to monitor something like user behavior on a website? Imagine running a web store where you sell products. One approach might be to set a minimum threshold for daily sales and check it once a day. But what if something goes wrong, and you need to catch the issue much sooner -- within hours or even minutes? In that case, a static threshold won't cut it because user activity fluctuates throughout the day. This is where anomaly detection comes in. What exactly is anomaly detection? Instead of relying on simple rules, it involves analyzing historical data to spot unusual patterns. There are various ways to implement anomaly detection, including machine learning and statistical analysis. In this article, we'll focus on the statistical approach and walk through how we built our own anomaly detection system for time series data from scratch at Booking. The Naïve Approach One common mistake I've seen across different companies and teams is trying to detect anomalies by simply comparing a business metric to its value exactly one week ago. This week vs previous week At first glance, this approach isn't entirely useless -- you can catch some anomalies, as shown in the image above. But is it a reliable long-term solution? Not really. The big flaw is that today's anomaly becomes next week's baseline. That means if the same issue occurs again at the same time next week, it may go completely unnoticed because we're now comparing against a flawed reference point. Outage in previous week That doesn't look right, our simplistic approach doesn't know that last week's data was compromised. Another limitation of this method is that it only considers a single week at a time. But what if perform