# Catching Anomalies Early in Mobile App Releases

DevFeed: [Catching Anomalies Early in Mobile App Releases](<https://devfeed.tech/articles/catching-anomalies-early-in-mobile-app-releases-20337.md>)

Original publisher: [Read original article](<https://engblog.nextdoor.com/catching-anomalies-early-in-mobile-app-releases-ac95adf9da81?source=rss----5e54f11cdfdf---4>)

Author: Walt Leung

Published: 2023-01-11T15:27:14Z

Content type: article

Language: en

Sources: [Nextdoor](<https://devfeed.tech/sources/nextdoor.md>)

Topics: [Mobile](<https://devfeed.tech/topics/mobile.md>), [observability](<https://devfeed.tech/topics/observability.md>), [Android](<https://devfeed.tech/topics/android.md>), [iOS](<https://devfeed.tech/topics/ios.md>)

Tags: [android](<https://devfeed.tech/tags/android.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [ios](<https://devfeed.tech/tags/ios.md>), [metrics](<https://devfeed.tech/tags/metrics.md>), [mobile](<https://devfeed.tech/tags/mobile.md>), [mobile-app-development](<https://devfeed.tech/tags/mobile-app-development.md>), [mobile-apps](<https://devfeed.tech/tags/mobile-apps.md>), [observability](<https://devfeed.tech/tags/observability.md>), [regression](<https://devfeed.tech/tags/regression.md>), [releases](<https://devfeed.tech/tags/releases.md>), [software-development](<https://devfeed.tech/tags/software-development.md>), [statistics](<https://devfeed.tech/tags/statistics.md>)

## AI overview

This engineering article explains how Nextdoor uses phased rollouts and observability practices to detect anomalies in iOS and Android mobile app releases. It describes why aggregate metrics can make regressions difficult to detect at low adoption levels.

## Source excerpt

How Nextdoor catches mobile app release anomalies at 1% adoption At Nextdoor, our mobile applications on iOS and Android serve content to tens of millions of weekly active users. At this scale, we run a weekly release process for both iOS and Android, shipping hundreds of changes across multiple teams and dozens of mobile engineers. Our team uses several observability processes and rollout strategies to keep these deployments safe and scalable. We most notably use phased rollouts to minimize the impact of a potentially bad release. Phased rollouts allow us to gradually increase the adoption of users for a new app version. For example, we can have a new app version be released to only 1% of users on the 1st day, 2% of users on the 2nd day, and so on. That way, if a new release were accidentally shipped with an uncaught regression, having it at 1% rollout means it affects fewer users, reduces its severity level, and gives us more time to react. However, for many of our critical business metrics where a failure can sometimes be silent, most out-of-the-box observability approaches don't work with phased rollouts. This is largely due to two problems: Observability typically happens at an aggregate level. For example, we look at app sessions or revenue on a daily basis, across all users for a platform. The behavior of early adopters on an app version differs from the median behavior of all users. Most importantly, early adopters are more active, almost by definition, to be in an early rollout of the new app version. At Nextdoor, Daily Users are more likely to adopt releases over Weekly Users, Weekly Users over Monthly Users, and so on. For example, consider an app session regression on a hypothetical iOS version v1.234.5 released March 4. If we had unknowingly introduced a regression where we didn't count an app session 5% of the time, at a 1% rollout, our aggregate impact would be expected to be roughly 0.05 x 0.01 = 0.05% of all iOS app sessions, which is practically im