# A/B Testing Published articles for A/B Testing. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Optimizing agent system prompts with Amazon Bedrock AgentCore DevFeed: [Optimizing agent system prompts with Amazon Bedrock AgentCore]() Original publisher: [Read original article]() Author: Han Ding Published: 2026-09-16T15:47:39Z Content type: article Language: en Sources: [Artificial Intelligence]() Topics: [Amazon Bedrock AgentCore](), [Prompt optimization](), [Prompt Engineering](), [Traces](), [Benchmark](), [A/B Testing](), [Automated prompt engineering]() Tags: [a-b-testing](), [amazon-bedrock](), [amazon-bedrock-agentcore](), [benchmark](), [best-practices](), [evaluation](), [intermediate-200](), [open-source](), [prompt-optimization](), [system-prompts](), [traces]() ### AI overview This technical companion explains how Amazon Bedrock AgentCore's system prompt optimizer uses production agent traces and reward signals to propose revised system prompts. It describes the reflector engine, recommendation explanations, offline batch evaluation, online A/B testing, and promotion workflow, and reports benchmark results for Single Agent Reflector and the experimental open source Sub-Agent Reflector. ### Source excerpt AgentCore optimization turns production traces into proposed configuration changes, then validates them before promotion. This technical companion to the launch post explains how the system prompt optimizer's reflector engine works and shares benchmark results for the Single Agent and Sub-Agent Reflectors. ## Agoda Replaces 72-Shard SQL Server Price Cache with DragonflyDB DevFeed: [Agoda Replaces 72-Shard SQL Server Price Cache with DragonflyDB]() Original publisher: [Read original article]() Author: Leela Kumili Published: 2026-09-14T13:48:00Z Content type: news Language: en Sources: [InfoQ]() Topics: [Cache](), [Caching](), [migration](), [sql-server](), [Latency](), [Prometheus]() Tags: [a-b-testing](), [agoda-price-cache-dragonflydb](), [architecture-design](), [cache](), [caching](), [development](), [devops](), [distributed-cache](), [distributed-data](), [distributed-systems](), [latency](), [microservices](), [migration](), [news](), [prometheus](), [redis](), [sql-server]() ### AI overview Agoda migrated its 1.5 TB hotel Price Cache from 72 Microsoft SQL Server shards to DragonflyDB. The staged migration used workload testing, dual reads, parity metrics, and gradual traffic shifting; Agoda reported substantially lower P99 read latency. ### Source excerpt Agoda migrated its 1.5 TB hotel Price Cache from 72 SQL Server shards to DragonflyDB to handle growing read and write volumes. The migration used staged dual reads, parity validation, gradual traffic shifting, and decentralized failover detection. Agoda reports an approximately eightfold reduction in P99 read latency, with two DragonflyDB clusters providing high availability. By Leela Kumili ## How we built Datadog Experiments DevFeed: [How we built Datadog Experiments]() Original publisher: [Read original article]() Author: Chas DeVeas; Aaron Silverman; Tyler Buffington; Jonathan Fulton; Taylor Overturf Published: 2026-09-10T00:00:00Z Content type: article Language: en Sources: [Datadog | The Monitor blog]() Topics: [experiments](), [real user monitoring]() Tags: [a-b-testing](), [acquisition](), [data-analytics](), [data-engineering](), [experimentation](), [experiments](), [metrics](), [observability](), [product-analytics](), [real-user-monitoring]() ### AI overview Datadog describes rebuilding its experimentation platform to speed up confident A/B-test decisions. The article explains a flexible CUPED approach that reduces metric variance and can be applied to segments. ### Source excerpt Datadog Experiments shortens the time from result to decision with CUPED on percentiles, verifiable warehouse results, and near real-time RUM metrics. ## Why Spotify Is Not Using Bayesian A/B Testing DevFeed: [Why Spotify Is Not Using Bayesian A/B Testing]() Original publisher: [Read original article]() Author: Spotify Engineering Published: 2026-09-08T13:18:44Z Content type: article Language: en Sources: [Spotify Engineering](), [Spotify Engineering Blog]() Topics: [Data Science](), [Tooling]() Tags: [a-b-testing](), [data](), [data-science](), [experimentation](), [inference](), [statistics](), [testing](), [tooling]() ### AI overview Spotify argues that Bayesian A/B testing can be oversimplified and that its claimed advantages require careful configuration. The article compares Bayesian and frequentist approaches and explains why Spotify does not currently need Bayesian inference alongside its frequentist tooling. ### Source excerpt Clearing the confusion about what Bayesian A/B testing is. The post Why Spotify Is Not Using Bayesian A/B Testing appeared first on Spotify Engineering. ## 3 ways to optimize Firebase Remote Config fetch usage DevFeed: [3 ways to optimize Firebase Remote Config fetch usage]() Original publisher: [Read original article]() Author: Sumit Chandel; Siddhant Jain Published: 2026-08-31T00:00:00Z Content type: tutorial Language: en Sources: [Firebase Blog]() Topics: [Firebase](), [App](), [configuration](), [Latency](), [Network](), [cross-platform]() Tags: [a-b-testing](), [battery](), [best-practices](), [feature-rollouts](), [firebase](), [firebase-remote-config](), [latency](), [network](), [pricing](), [remote-config]() ### AI overview This article presents three strategies for reducing Firebase Remote Config fetch traffic. It explains how smarter fetch and activation patterns can reduce network overhead, latency, battery consumption, and usage-based costs. ### Source excerpt Reduce network fetches, battery consumption and usage costs ## Implementing dynamic feature flags with AWS AppConfig on AWS Lambda DevFeed: [Implementing dynamic feature flags with AWS AppConfig on AWS Lambda]() Original publisher: [Read original article]() Author: Daniel Abib Published: 2026-08-14T16:25:51Z Content type: tutorial Language: en Sources: [AWS Compute Blog]() Topics: [AWS Lambda](), [Serverless](), [experiments](), [configuration](), [Latency]() Tags: [a-b-testing](), [advanced-300](), [aws](), [aws-lambda](), [feature](), [feature-flags](), [how-to](), [latency](), [real-time](), [serverless](), [technical-how-to]() ### AI overview A tutorial on implementing dynamic feature flags with AWS AppConfig on AWS Lambda. It explains how feature flags support safe deployments, experiments, gradual rollouts, and changes without redeploying functions, using the AWS AppConfig Lambda extension to cache configuration locally and reduce latency. ### Source excerpt Feature toggles allow you to change application behavior in real time without deploying new code. Learn how to implement dynamic feature flags with AWS AppConfig on AWS Lambda for safe deployments, gradual rollouts, and instant rollback. ## When Can LLMs Replace Humans in A/B Tests? DevFeed: [When Can LLMs Replace Humans in A/B Tests?]() Original publisher: [Read original article]() Author: Spotify Engineering Published: 2026-08-13T18:57:22Z Content type: article Language: en Sources: [Spotify Engineering](), [Spotify Engineering Blog]() Topics: [Large Language Model](), [A/B Testing](), [datasets](), [data](), [experiments]() Tags: [a-b-testing](), [analysis](), [data](), [data-science](), [experiment](), [gpt](), [llm](), [llms](), [model](), [testing]() ### AI overview The article examines whether large language model predictions can replace human outcomes in A/B tests. Using the Upworthy dataset, it finds that calibrated predictions can recover treatment effects under specific assumptions, while raw predictions recovered only 39% of the observed human effect. ### Source excerpt TL;DR: LLM predictions can stand in for human outcomes in A/B tests, but only by assumption, not by design.... The post When Can LLMs Replace Humans in A/B Tests? appeared first on Spotify Engineering. ## Measuring time savings from Figma Make DevFeed: [Measuring time savings from Figma Make]() Original publisher: [Read original article]() Author: Remy Stewart Published: 2026-08-11T19:59:00Z Content type: article Language: en Sources: [Figma Blog]() Topics: [Figma](), [Data Science](), [Artificial Intelligence](), [A/B Testing]() Tags: [a-b-testing](), [ai](), [ai-tools](), [data-science](), [figma](), [productivity](), [research](), [testing]() ### AI overview Figma's Data Science team describes a randomized controlled trial measuring how Figma Make affects users' design work. Across 100 participants, design work was 20% faster and 16% easier overall, while PMs saw tasks become 23% faster and 37% easier. The article explains why confounders make AI time savings difficult to measure and discusses the limitations of online A/B testing and causal inference using log data. ### Source excerpt The Figma Data Science team assumed that AI saves users time--but quantifying it required a new approach to research design. ## How to Build an A/B Testing Framework for Modern CI/CD Pipelines DevFeed: [How to Build an A/B Testing Framework for Modern CI/CD Pipelines]() Original publisher: [Read original article]() Author: Aaron Newcomb Published: 2026-08-10T00:00:00Z Content type: tutorial Language: en Sources: [Harness Blog]() Topics: [A/B Testing](), [CI/CD](), [CI/CD Pipeline](), [feature flags](), [progressive delivery](), [Monitoring]() Tags: [a-b-testing](), [ci-cd](), [feature-flags](), [how-to](), [monitoring](), [progressive-delivery](), [rollback](), [verification]() ### AI overview This tutorial explains how to integrate A/B testing into CI/CD pipelines using feature flags, progressive delivery, real-time metrics, automated verification, and rollback controls. It argues that deployment should be decoupled from release so teams can validate changes before full rollout. ### Source excerpt Learn how to build an A/B testing framework in your CI/CD pipeline. Accelerate software delivery, reduce risk, and boost innovation. Start now! | Blog ## The effect distribution: The missing piece in experimentation programs DevFeed: [The effect distribution: The missing piece in experimentation programs]() Original publisher: [Read original article]() Author: Tyler Buffington Published: 2026-07-02T00:00:00Z Content type: article Language: en Sources: [Datadog | The Monitor blog]() Topics: [experiments]() Tags: [a-b-testing](), [experiment](), [experimentation](), [experiments](), [testing]() ### AI overview The article explains why effect distributions are essential for interpreting results across experimentation programs. It shows how statistically significant results can all be false positives when true effects are concentrated at zero, and introduces the challenge of estimating true effects from noisy observed effects. ### Source excerpt Learn about the importance of considering the effect distribution when running experiments. ## Datadog acquires Adaptive ML DevFeed: [Datadog acquires Adaptive ML]() Original publisher: [Read original article]() Author: Alexis Lê-Quôc Published: 2026-06-30T00:00:00Z Content type: news Language: en Sources: [Datadog | The Monitor blog]() Topics: [Artificial Intelligence](), [agent observability](), [Frontier AI](), [Reinforcement learning](), [Synthetic Data Generation](), [A/B Testing](), [Deployment](), [observability](), [Security](), [real-time]() Tags: [a-b-testing](), [acquisition](), [ai](), [ai-agents](), [autonomous](), [data](), [deployment](), [frontier-ai](), [models](), [observability](), [platform](), [production](), [real-time](), [reinforcement-learning](), [security](), [synthetic-data-generation]() ### AI overview Datadog announces the acquisition of Adaptive ML, whose Adaptive Engine helps enterprises build, own, and deploy specialized AI agents and models. The platform supports fine-tuning open models with reinforcement learning and synthetic data, evaluating them with AI judges and A/B testing, and using production signals to improve subsequent training. ### Source excerpt Datadog has acquired Adaptive ML, a platform for building, owning, and deploying specialized AI agents and models. ## See your serverless: introducing the Headlamp plugin for Knative DevFeed: [See your serverless: introducing the Headlamp plugin for Knative]() Original publisher: [Read original article]() Author: Mudit Maheshwari; Kahiro Okina Craftsman Software; Inc Published: 2026-06-25T18:00:00Z Content type: article Language: en Sources: [Kubernetes Blog]() Topics: [Serverless](), [Kubernetes](), [autoscaling](), [Routing (disambiguation)](), [A/B Testing](), [Open Source]() Tags: [a-b-testing](), [autoscaling](), [cli](), [kubernetes](), [open-source](), [routing](), [serverless]() ### AI overview This article introduces a Headlamp plugin for Knative that provides a unified interface for exploring and managing serverless workloads on Kubernetes. It covers resource relationships, KService editing, traffic splitting across revisions, autoscaling configuration, logs, redeployments, pod restarts, and permission-aware actions. ### Source excerpt Headlamp is an open-source, extensible Kubernetes SIG UI project designed to let you explore, manage, and debug cluster resources. Knative brings serverless workloads to Kubernetes, handling traffic routing, autoscaling, and revision management so teams can deploy and iterate without fighting infrastructure. But operating Knative workloads day-to-day can be difficult, there's still a lot of jumping between the kn CLI, kubectl, and the Kubernetes UI to get a full picture of what's running. We built the Headlamp Knative plugin to bridge that very gap, allowing operators to inspect, understand and act on their workloads all from a single place. This plugin was built as part of the LFX mentorship. Here's a tour of what we shipped. Here is a short walkthrough of the Knative plugin for Headlamp: Integrating Knative resources with Headlamp's map view Headlamp's resource mapping works for Knative CRDs too. You can see how KServices, Revisions, and DomainMappings relate to each other in a single graph view. KService management: edit traffic splits, restart pods, and view logs A KService is the top-level resource in Knative: it manages the lifecycle of Routes, Configurations, Revisions, and everything needed to run and expose your application. The plugin gives KServices a full detail view with an Edit Mode toggle for making live changes to traffic splits, autoscaling annotations, and more. Common actions like viewing the YAML, opening logs, triggering a redeploy, or restarting backing pods are surfaced in the header, gated by your current RBAC permissions. Traffic splitting: route across revisions for gradual rollouts and testing Knative makes it possible to route traffic across multiple Revisions of the same service. This is useful for canary releases, gradual rollouts, tagged preview URLs, and A/B testing. The plugin shows the traffic assigned to each Revision, the latest ready Revision, readiness status, age, and configured tags. In edit mode, you can adjust percentages an ## Flight Booking & Airlines Quantitative UX: 3 High-Level Takeaways from 30+ Charts DevFeed: [Flight Booking & Airlines Quantitative UX: 3 High-Level Takeaways from 30+ Charts]() Original publisher: [Read original article]() Author: Richard Lam Published: 2026-06-24T08:04:00Z Content type: article Language: en Sources: [Baymard Institute]() Topics: [Flight](), [User experience (UX)](), [data](), [benchmarking](), [Mobile](), [A/B Testing]() Tags: [a-b-testing](), [airline](), [apps](), [article](), [booking](), [desktop](), [mobile](), [research](), [survey](), [travel](), [ux]() ### AI overview Baymard reports quantitative UX findings from a survey of 3,125 US online shoppers who use flight-booking and airline sites. The article highlights that shoppers use multiple sources when searching for flights, three quarters belong to at least one airline loyalty program, and business travelers are more likely to bundle travel-related bookings. ### Source excerpt (Note: Unfortunately, e-mail and RSS don't support advanced layouts and features. If the graphics in this article look strange, you may want to read the article in your web browser.) Key Stats & Takeaways 30+ new insights on Flight Booking & Airline shopper habits and preferences 3,125 US online shoppers surveyed in this quantitative UX study Flight Booking & Airline shoppers use a variety of sources when looking for flights, are highly likely to participate in loyalty programs, and commonly bundle other travel-related bookings at the same time, particularly when travelling for business. At Baymard, we've just released new Quantitative Insights into people who shop on "Flight Booking & Airlines" sites, expanding our understanding of the habits and preferences of online shoppers in this category. These insights are visualizations based on survey data that supplement and support our large-scale UX research findings and benchmarking of the Flight Booking & Airlines industry. The 30+ insights address the Flight Booking & Airlines online shopping experience, spanning a breadth of topics: sources for flight searching, types of add-ons purchased, usage of apps and external tools, importance of cancellations, sales and scarcity incentives, and motivations for bundling and loyalty programs. Some of the charts also explore sub-segments of the broader audience such as business travelers and whether they are more likely to purchase add-ons, or how device usage between desktop and mobile changes as flight price becomes more expensive. These Quantitative Insights empower you to align stakeholders through objective, survey-backed data, streamline A/B testing with high-potential hypotheses, and discover and address industry-specific UX challenges. In this article, we'll highlight 3 high-level findings that reflect how shoppers evaluate and purchase flights online: Flight shoppers use a variety of sources when looking for flights Three quarters of flight shoppers are part of at leas ## Gamification 2.0. Beyond Points and Badges: Designing for Players, Not Metrics. Conclusion DevFeed: [Gamification 2.0. Beyond Points and Badges: Designing for Players, Not Metrics. Conclusion]() Original publisher: [Read original article]() Author: Montgomery Singman Published: 2026-06-16T05:02:36Z Content type: opinion Language: en Sources: [UX Magazine]() Topics: [User experience (UX)](), [App](), [A/B Testing]() Tags: [a-b-testing](), [app](), [games](), [gaming](), [ux]() ### AI overview This conclusion to a gamification series argues that product teams should design genuinely valuable and enjoyable experiences for users rather than applying game mechanics solely to improve engagement metrics. It recommends identifying the game genre that naturally fits an app and learning from that genre's underlying psychology. ### Source excerpt Part 6 of the "Gamification Series." Design for players, not metrics Through decades of watching games evolve from 8-bit sprites to photorealistic 3D, from arcade cabinets to cloud gaming, from niche hobby to global culture, one principle has remained constant: people play games because games are worth playing. Not because games award points. Not because The post Gamification 2.0. Beyond Points and Badges: Designing for Players, Not Metrics. Conclusion appeared first on UX Magazine. ## Kotlin Multiplatform in Production: Two Real-World Use Cases from Booking.com DevFeed: [Kotlin Multiplatform in Production: Two Real-World Use Cases from Booking.com]() Original publisher: [Read original article]() Author: Diego Gómez Olvera Published: 2026-06-05T15:09:18Z Content type: article Language: en Sources: [Booking.com Development - Medium]() Topics: [Kotlin Multiplatform](), [compose-multiplatform](), [experiments](), [A/B Testing](), [Android](), [iOS](), [Design system](), [Mobile](), [Development]() Tags: [a-b-testing](), [android](), [booking](), [bookingcom](), [compose](), [compose-multiplatform](), [concepts](), [consistency](), [data](), [development](), [engineering](), [experiment](), [experimentation](), [experiments](), [ios](), [java](), [jetpack-compose](), [kotlin](), [kotlin-multiplatform](), [mobile](), [multiplatform](), [objective-c]() ### AI overview This article describes two Booking.com engineering use cases for Kotlin Multiplatform and Compose Multiplatform: a shared experimentation library for consistent experiment assignments across Android and iOS, and hosting an Android design system in a web browser. ### Source excerpt Introduction For the majority of Booking.com travelers, mobile is the primary channel for researching, planning, and booking trips. Recent data shows that over 80% of travelers rely on a mobile app during the research phase, with more than half of all bookings occurring on mobile devices. Consequently, the Android and iOS platforms are critical to the company's product strategy; engineering choices made here have significant repercussions for the entire organisation. To maintain agility at this scale, two elements must function in unison: Strict decision validation: At any time, Booking.com manages over 1,000 simultaneous experiments across its product suite, with hundreds active on mobile. Every minor adjustment undergoes A/B testing via our proprietary experimentation library before reaching the user. A unified design system ensures product consistency and makes design goals transparent to all contributors, not just maintenance engineers. This article examines two specific engineering challenges solved using Kotlin Multiplatform (KMP) and Compose Multiplatform (CMP): Developing a shared experimentation library to ensure uniform experiment assignments across Android and iOS. Using Compose Multiplatform to host our Android design system in a web browser, bridging the gap between design concepts and implementation. While both cases use the same underlying technology, each provides unique insights into multiplatform development. Use case 1: shared experimentation library on Android and iOSThe problem with two implementations Historically, our internal experimentation library, responsible for managing experiment assignments, evaluations, and tracking on mobile, was maintained as two distinct codebases: a mix of Java and Kotlin for Android and Objective-C for iOS. While intended to be identical, managing two languages with fluctuating team resources inevitably led to logic drift. Discrepancies in event-tracking and experiment-fetching behaviours emerged, though they wer ## The Causality Gap: Measuring the True Impact of Voluntary Adoption in Digital Marketplaces DevFeed: [The Causality Gap: Measuring the True Impact of Voluntary Adoption in Digital Marketplaces]() Original publisher: [Read original article]() Author: Lin Jia Published: 2026-05-22T08:05:43Z Content type: article Language: en Sources: [Booking.com Data Science]() Topics: [A/B Testing](), [doubleml](), [Data Science](), [Tool]() Tags: [a-b-testing](), [causal-inference](), [causal-machine-learning](), [causality](), [data-science](), [doubleml](), [e-commerce](), [experimentation](), [featured](), [measuring]() ### AI overview This article explains why standard A/B tests can mismeasure features that require voluntary adoption in digital marketplaces. It presents Randomized Encouragement Design combined with Double Machine Learning as a way to estimate both causal impact among adopters and overall rollout impact. ### Source excerpt by Lin Jia, Kexin Fei The Content of this post has been presented at Pydata Amsterdam 2026 and the slides can be found here TL; DR Whenever a feature requires voluntary adoption, standard A/B testing breaks: low adoption flattens the topline, and self-selection makes adopters incomparable to non-adopters. Combining Randomized Encouragement Design (RED) with Double Machine Learning (DoubleML) recovers two answers -- the causal lift for adopters and the rollout's overall impact. When they diverge, the gap turns an ambiguous topline into a sharp product decision -- build a better product, or build a better adoption funnel. 1. The Opt-In Barrier Across Demand and Supply Across the tech industry, many platform features rely on voluntary adoption. A customer chooses whether to claim a promotional discount. A traveller opts into a loyalty program. An e-commerce seller enables a smart-pricing tool. In every case, the platform cannot force adoption -- and the feature's true impact becomes hard to measure, on both demand and supply. At Booking.com this challenge spans both sides of the marketplace -- travellers choosing to log in, partners choosing to adopt new features. Unlike a search-ranking change that applies to 100% of traffic, opt-in features create a "trilemma" for Product Data Science: Voluntary Adoption (the "Opt-In" Barrier): Users must actively enable the feature. A standard A/B test cannot separate the product's effect from the motivation that drove users to adopt it. Extreme Heterogeneity: Travellers range from once-a-year holidaymakers to travel agencies booking thousands of nights; partners range from single-apartment hosts to hotel chains. This variance is noise on both sides. Finite Sample Sizes: Opt-in features target a finite sub-segment, so we cannot simply "run the test longer" to gain power. When these stack up, a flat topline can hide a strong product behind a weak adoption funnel. Genuinely strong features get killed, and resources flow into the wrong int ## What's new from Firebase at Google I/O 2026 DevFeed: [What's new from Firebase at Google I/O 2026]() Original publisher: [Read original article]() Author: Seba Gnagnarella; Arthur Thompson; Kristen Richards Published: 2026-05-19T01:00:00Z Content type: news Language: en Sources: [Firebase Blog]() Topics: [Firebase](), [Agent Skills](), [Google](), [Mobile](), [Android Studio](), [Flutter](), [Model Context Protocol]() Tags: [2026](), [a-b-testing](), [agent-skills](), [ai-logic](), [ai-studio](), [android](), [android-studio](), [antigravity](), [app-check](), [application-design-center](), [authentication](), [cloud-logging](), [cloud-run](), [crashlytics](), [firebase](), [firestore](), [flutter](), [google](), [google-i-o](), [hybrid-inference](), [ios](), [mcp](), [mobile](), [news](), [remote-config](), [security]() ### AI overview Firebase announces updates from Google I/O 2026, including integrations with Google Antigravity and Android Studio, expanded Agent Skills for mobile development, and support for Firestore, Firebase Authentication, Crashlytics, and Remote Config. ### Source excerpt News, tutorials, and updates from the Firebase team. ## Kubernetes v1.36: New Metric for Route Sync in the Cloud Controller Manager DevFeed: [Kubernetes v1.36: New Metric for Route Sync in the Cloud Controller Manager]() Original publisher: [Read original article]() Author: Lukas Metzner Published: 2026-05-15T18:35:00Z Content type: article Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [Cloud](), [A/B Testing](), [API]() Tags: [a-b-testing](), [api](), [article](), [cloud](), [github](), [kubernetes](), [slack](), [testing]() ### AI overview Kubernetes v1.36 adds the alpha route_controller_route_sync_total metric to the Cloud Controller Manager. The metric helps operators evaluate watch-based route reconciliation, which syncs routes when nodes change instead of on a fixed interval. ### Source excerpt This article was originally published with the wrong date. It was later republished, dated the 15th of May 2026. Kubernetes v1.36 introduces a new alpha counter metric route_controller_route_sync_total to the Cloud Controller Manager (CCM) route controller implementation at k8s.io/cloud-provider. This metric increments each time routes are synced with the cloud provider. A/B testing watch-based route reconciliation This metric was added to help operators validate the CloudControllerManagerWatchBasedRoutesReconciliation feature gate introduced in Kubernetes v1.35. That feature gate switches the route controller from a fixed-interval loop to a watch-based approach that only reconciles when nodes actually change. This reduces unnecessary API calls to the infrastructure provider, lowering pressure on rate-limited APIs and allowing operators to make more efficient use of their available quota. To A/B test this, compare route_controller_route_sync_total with the feature gate disabled (default) versus enabled. In clusters where node changes are infrequent, you should see a significant drop in the sync rate with the feature gate turned on. Example: expected behavior With the feature gate disabled (the default fixed-interval loop), the counter increments steadily regardless of whether any node changes occurred: # After 10 minutes with no node changes route_controller_route_sync_total 60 # After 20 minutes, still no node changes route_controller_route_sync_total 120 With the feature gate enabled (watch-based reconciliation), the counter only increments when nodes are actually added, removed, or updated: # After 10 minutes with no node changes route_controller_route_sync_total 1 # After 20 minutes, still no node changes -- counter unchanged route_controller_route_sync_total 1 # A new node joins the cluster -- counter increments route_controller_route_sync_total 2 The difference is especially visible in stable clusters where nodes rarely change. Where can I give feedback? If you ## The Illusion of Choice: How Micro-Decisions Guide Macro-Control DevFeed: [The Illusion of Choice: How Micro-Decisions Guide Macro-Control]() Original publisher: [Read original article]() Author: Tushar Deshmukh Published: 2026-04-21T06:23:51Z Content type: opinion Language: en Sources: [UX Magazine]() Topics: [User experience (UX)](), [Usability](), [User interface design](), [A/B Testing]() Tags: [a-b-testing](), [article](), [design-patterns](), [interfaces](), [usability](), [ux]() ### AI overview This article examines how small interface decisions can influence user behavior and create an illusion of choice. It argues that choice architecture, including button prominence, wording, navigation order, and default states, can steer users toward predetermined outcomes, and calls for UX practices that balance usability with responsibility. ### Source excerpt Part 8 of the "Ethical UX Series." "The greatest enemy of freedom is a happy slave." -- Friedrich von Schiller What is "ethical UX," and why this series matters In the design world, we often speak about empathy, innovation, and delight. But underneath the surface of many successful interfaces lies a subtle, often unchecked force The post The Illusion of Choice: How Micro-Decisions Guide Macro-Control appeared first on UX Magazine. ## Beyond A/B Testing: Using Surrogacy and Region-Splits to Measure Long-Term Effects in Marketplaces DevFeed: [Beyond A/B Testing: Using Surrogacy and Region-Splits to Measure Long-Term Effects in Marketplaces]() Original publisher: [Read original article]() Author: Iraklikhorguani Published: 2026-03-25T13:56:39Z Content type: article Language: en Sources: [Lyft Engineering - Medium]() Topics: [A/B Testing](), [App](), [Optimization]() Tags: [a-b-testing](), [app](), [cost](), [drivers](), [driving](), [growth](), [optimization](), [payments](), [pricing](), [resources](), [revenue](), [testing]() ### AI overview Lyft describes why standard A/B tests may not capture the long-term and market-mediated effects of pricing, payments, and incentive decisions in its multi-sided marketplace. The article introduces surrogacy and region splits as approaches for measuring those effects. ### Source excerpt Image generated with Gemini 3 Pro (Google), 2026. Written by Amber Wang and Yoonji Kim at Lyft. Background Whenever you use the Lyft app, there is a complex balancing act happening behind the scenes. Various levers are used to keep the marketplace running smoothly; Base prices and coupons for riders affect demand, while driver pay and bonuses impact the level of available supply. Since every change to prices and payments impacts Lyft's costs and revenue, they lead to key optimization problems, such as: How should we allocate budget between driver incentives and rider incentives? How do we invest resources to achieve x% rides growth, and how much does it cost in terms of short term profit? These are the questions the Foundational Models team at Lyft tries to answer in a systematic way. A key ingredient is understanding the effects of different types of investments -- for instance, what will happen if we increase the total budget for driver incentives by x%? What will happen if we increase the rider price of all rides by y%? It's worth noting that the long term effects of such decisions tend to dominate the short term effects: we may earn more short term profit from a ride if we charge riders more and pay drivers less, but lose riders and drivers in the long run. Estimating the long term effects of resource allocation decisions is challenging in a multi-sided marketplace such as Lyft. Because these decisions tend to be consequential, their effects go beyond first order effects on directly affected users. For example, if we increase driver incentive spending by x% in week 1, drivers will drive more in week 1 (short term effect), and may return to drive a bit more in the following weeks (direct long term effects). But this is not the full picture: in week 1, when there is a positive increase in driver hours as the result of more incentives, riders will enjoy better experiences (e.g. less surge pricing, shorter wait times) and may want to return to Lyft in the future. How ## Firebase A/B Testing is now available for the web DevFeed: [Firebase A/B Testing is now available for the web]() Original publisher: [Read original article]() Author: Sumit Chandel; Athira Manu Published: 2026-03-04T00:00:00Z Content type: release Language: en Sources: [Firebase Blog]() Topics: [A/B Testing](), [Firebase](), [Web](), [Website](), [Google Analytics]() Tags: [a-b-testing](), [announce](), [firebase](), [firebase-remote-config](), [google-analytics](), [react](), [release](), [remote-config](), [typescript](), [web](), [web-apps]() ### AI overview Firebase A/B Testing is now available for web apps, extending its existing mobile app experimentation capabilities. The feature uses Google Analytics and Firebase Remote Config to run and analyze experiments, track metrics such as retention, revenue, and engagement, and identify statistically significant variations. ### Source excerpt From Mobile to Web: Easily run, analyze, and scale product experiments to drive real impact using Firebase A/B Testing now available for your web apps! ## Chargebee Review 2026: Pricing, Features and Fit DevFeed: [Chargebee Review 2026: Pricing, Features and Fit]() Original publisher: [Read original article]() Author: Joshua D'Costa Published: 2025-12-16T00:00:00Z Content type: comparison Language: en Sources: [Dodo Payments Blog]() Topics: [Software as a service](), [A/B Testing]() Tags: [2026](), [a-b-testing](), [alternatives](), [automated](), [billing](), [checkout](), [customer](), [enterprise](), [experiments](), [features](), [payments](), [paypal](), [retry](), [review](), [saas](), [self-service](), [subscription](), [teams]() ### AI overview A review of Chargebee's pricing, subscription management features, payment integrations, trial workflows, and fit for SaaS and digital businesses. It highlights Chargebee's enterprise focus, broad capabilities, and steeper learning curve for smaller teams. ### Source excerpt Comprehensive Chargebee review covering pricing tiers, subscription management features, enterprise focus, and which SaaS teams benefit most in 2026. ## Normalized Entropy or Apply Rate? Evaluation Metrics for Online Modeling Experiments DevFeed: [Normalized Entropy or Apply Rate? Evaluation Metrics for Online Modeling Experiments]() Original publisher: [Read original article]() Author: Megan Chen Published: 2025-11-11T06:16:53Z Content type: opinion Language: en Sources: [Indeed]() Topics: [recommendation systems](), [Model Development](), [decision-making](), [benchmarking]() Tags: [a-b-testing](), [data-science](), [decision-making](), [evaluation](), [experiments](), [machine-learning](), [metric](), [models](), [ranking](), [unsorted]() ### AI overview Indeed examines whether model performance metrics or product metrics should guide online modeling experiments. It discusses how optimizing individual ranking models may not align with broader business goals and considers evaluation metrics for model rollouts. ### Source excerpt Introduction At Indeed, our mission is to help people get jobs. We connect job seekers with their next career opportunities and assist employers in finding the ideal candidates. This makes matching a fundamental problem in the products we develop. The Ranking Models team is responsible for building Machine Learning models that drive matching between job [...] ## Speed with Rigor: testing smarter with group sequential design DevFeed: [Speed with Rigor: testing smarter with group sequential design]() Original publisher: [Read original article]() Author: Gabriela Lewenfus Published: 2025-08-05T17:25:29Z Content type: article Language: en Sources: [OLX]() Topics: [Testing](), [A/B Testing](), [experiments](), [data](), [implementation]() Tags: [a-b-testing](), [ab-testing](), [data](), [efficiency](), [experiment](), [experimentation](), [false-positive](), [implementation](), [pitfalls](), [sequential-testing](), [statistics](), [testing]() ### AI overview The article explains how group sequential testing can replace fixed-horizon A/B tests with monitoring at predefined intervals while controlling error rates. It discusses how this approach addresses peeking, may allow ineffective tests to stop early, and involves practical trade-offs and implementation challenges. ### Source excerpt generated by chatgptIntroduction In the world of A/B testing and experimentation, it's tempting to check results frequently and stop tests earlier if they seem significant (a practice known as peeking). However, this practice dramatically inflates the false positive rate, leading to misleading conclusions. Imagine you're running an A/B test, and after just 10 days, the results look like a clear win. You might be tempted to end the experiment early and move forward with the release of the new feature. After all, why wait another two weeks as originally planned? But here's the catch: the more often you check the results, the higher the chance you'll spot a "win" that isn't real. Frequent peeking increases the risk of false positives -- in other words, drawing the wrong conclusion just by chance. To prevent peeking while ensuring statistical power, a common practice is to set a minimum sample size that must be reached before ending the experiment. However, this fixed horizon approach can extend the experiment's duration and delay decision-making. Group sequential testing (GST) provides a statistical framework to monitor results at predefined intervals while maintaining control over error rates. At OLX, we migrated from fixed horizon experiments to GST to achieve faster insights, lower costs, and greater efficiency, enabling our teams to make smarter, data-driven decisions with speed and confidence. In this post, I'll explain how GST works, how it can accelerate experiments while still preventing the pitfalls of peeking, and how we can use it even to stop ineffective tests early. Finally, I'll discuss real-world trade-offs and implementation challenges. Peeking is a trap Before we dive into GST, let's first understand what "peeking" is and why it's so concerning. Peeking occurs when the experimenter repeatedly checks an experiment's results and finishes it once a statistically significant effect appears. This inflates the false positive rate, leading to unreliable conclu [Next page]()