# Stories by Headspace on Medium Stories by Headspace on Medium This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Open Sourcing Transformer Embeddings DevFeed: [Open Sourcing Transformer Embeddings]() Original publisher: [Read original article]() Author: Headspace Published: 2022-11-02T20:56:38Z Content type: release Language: en Sources: [Stories by Headspace on Medium]() Topics: [Embeddings](), [Transformer](), [Natural language processing](), [Python](), [tokenization](), [Inference](), [PyTorch]() Tags: [embeddings](), [inference](), [machine](), [machine-learning](), [naturallanguageprocessing](), [nlp](), [open](), [open-source](), [pypi](), [python](), [pytorch](), [s3](), [tokenization](), [training]() ### AI overview Headspace Health open-sources transformer-embeddings, an internal Python package, and publishes it to PyPI. The package supports pooled embeddings for phrases, sentences, paragraphs, and documents, with configurable pooling and batch sizes, CPU or GPU execution, and S3 export. ### Source excerpt Photo by Jr Korpa on UnsplashBy Setu Shah, Staff Machine Learning Engineer The Data Science team at Headspace Health is a heavy user of building, training and production-izing transformer-based NLP models. Depending on the application, our NLP models operate at varying levels of atomicity: token, word, phrase, sentence, paragraph and document. A lot of our models rely on embeddings from off-the-shelf Transformer-based models, while some are fine-tuned for the application we are building for. Similarly, as a part of our work, we often try out different models during exploration, while comparing various pooling methods. About 18 months ago, we discovered the need for a streamlined way to use embeddings across our work for our exploration and production use cases. That is when we started building transformer-embeddings, an internal Python package that made it easy to interact with transformer-based models that are built using PyTorch. Today, we are open-sourcing this package and publishing it to PyPI to make it easy for anyone in the transformer and NLP communities to be able to use aggregated and pooled embeddings for phrases, sentences and paragraphs and documents, at varying levels of aggregation, in any of their applications. You should use this if you want to... Automatically apply tokenization (with the model defaults) before your model's forward pass. Stack outputs from the model into a single, iterable array that map 1:1 with your input. Simplify interactions with any transformer model available on the HuggingFace Model Hub for exploration and inference. Easily apply and compare the impact of different pooling strategies (mean, max, min, pooler) on your downstream tasks. Use your model on CPUs or GPUs, without worrying about if you asked PyTorch to use the right device. Export the model and additional artifacts (custom scikit-learn / tree-based models, model cards, etc.) to S3. Customize batch sizes for different models as you play with them. You should not use t ## Our Android testing process DevFeed: [Our Android testing process]() Original publisher: [Read original article]() Author: Headspace Published: 2022-06-22T05:16:52Z Content type: article Language: en Sources: [Stories by Headspace on Medium]() Topics: [Testing](), [Android](), [unit test](), [Mocking]() Tags: [android](), [android-testing](), [firebase](), [junit](), [mental-health](), [mobile-app-development](), [mobile-test-automation](), [mocking](), [quality-assurance](), [test-lab](), [testing]() ### AI overview Headspace describes its Android testing strategy after rewriting its app, including unit-test coverage standards, selective end-to-end testing, mocking, dependency injection, Robolectric, and Firebase Test Lab. ### Source excerpt Our testing strategy, how we integrate it into our development workflow, and what the future holds for testing at Headspace. By John Qualls, Senior Software Engineer, Greg Rami, Principal Software Engineer, and Anton Gerasimenko, Software Engineer Introduction When we took on the daunting task of rewriting our entire app at the end of 2019, one of our key focus areas was testability. At the time, we had less than 20% of our code covered by unit tests -- no integration or end-to-end (E2E) tests -- and adding any tests to the code base was a big effort. From the beginning, we agreed on the standard of at least 80% unit test coverage on all Pull Requests (PRs), E2E tests for critical flows, and an architecture that was focused on testability. The result is a more robust codebase. Today, we're able to capture critical bugs via our tests suite coverage before they hit our users and, very importantly, it's now trivial, and very fast, to add new tests. Let's detail our testing strategy, how we integrate it into our development workflow, and what the future holds for testing at Headspace. Testing Strategy Shortly after the rewrite, we had hundreds of unit tests and around twenty E2E tests. We wanted to optimize the number of E2E tests that we do to only certain use cases where it makes sense to take on the additional time and costs in exchange for the reduced risk. Test selection pyramid for cost and speedUnit Tests Our unit tests are straightforward. They test a very small unit of code by relying on JUnit 5 and different layers of mocking: MockK to mock all the dependencies that are unrelated to the current test. This is made easier by using good architecture principles such as Single Responsibility Principle (SRP) and Dependency Injection (DI) We introduced a Java Faker library, that provides "fakes" for primitive values that do not need to have a specific value Similarly, we have our own "model Fakers" that provide domain objects. That way when the underlying classes chang ## How Headspace Scaled Identity and Minimized Security Risk with Auth0 CIAM on AWS DevFeed: [How Headspace Scaled Identity and Minimized Security Risk with Auth0 CIAM on AWS]() Original publisher: [Read original article]() Author: Headspace Published: 2021-12-21T22:41:06Z Content type: article Language: en Sources: [Stories by Headspace on Medium]() Topics: [Auth0](), [Amazon Web Services](), [Authentication](), [Authorization](), [identity and access management](), [Security]() Tags: [auth0](), [authentication](), [authorization](), [aws](), [headspace](), [identity-and-access-management](), [identity-management](), [mobile](), [security](), [security-risk-management]() ### AI overview The article promotes a webinar featuring Headspace, AWS, and Auth0 about scaling customer identity and reducing security risk. It describes Headspace's need for a secure identity solution for its large monthly user base and enterprise customers, and presents authentication, authorization, and identity and access management as services delivered with AWS and Auth0. ### Source excerpt George Torres, Director of Engineering at Headspace recently participated in an AWS and AWS Partner Auth0 webinar on scaling identity and minimizing security risk. You can use this link to register and watch the webinar. Delivering security and mindfulness at scale Headspace's mission is to improve the health and happiness of the world by providing guided meditations and mindfulness content through its mobile application. The company sought out a scalable and secure identity solution to support its millions of monthly active users and accommodate continued growth. Additionally, it needed to support its enterprise customers who were including Headspace services in their employee benefits packages. With Amazon Web Services (AWS) and AWS Partner Auth0, Headspace was able to achieve both. In this webinar, hear how to implement a frictionless digital experience with authentication and authorization as a service. Watch this webinar, and see how to: Focus on innovation instead of building and maintaining identity solutions Incorporate identity into any customer-facing application quickly, securely, and at scale Enable secure identity and access management across your environments Who should watch? CTOs, CIOs, CISOs, VPs of IT, IT Managers, IT Directors, IT Supervisors, Software Developers, Software Development Managers, System Architects, C-Level or LOB Managers, CEOs, CMOs, COOs, LOB Directors, Sr. Solution Architects, Sr. Security Architects, Sr. Security Engineers, Sr. Identity Engineers, IT Security Directors, Tech Leads, Engineering Managers, Technology Executives, Product Managers, Security Identity Managers, ITOps Webinar SpeakersClare Nelson Sr. Partner Development Manager, Amazon Web Services (AWS), Inc. Shiven Ramji Chief Product Officer, Auth0 George Torres Director of Engineering, Headspace How Headspace Scaled Identity and Minimized Security Risk with Auth0 CIAM on AWS was originally published in Headspace-engineering on Medium, where people are continuing the ## Mindful Experimentation: Evaluate Recommendation System Performance using A/B Testing at Headspace DevFeed: [Mindful Experimentation: Evaluate Recommendation System Performance using A/B Testing at Headspace]() Original publisher: [Read original article]() Author: Headspace Published: 2021-11-29T23:34:41Z Content type: article Language: en Sources: [Stories by Headspace on Medium]() Topics: [recommendation systems](), [A/B Testing](), [data](), [Machine Learning & Artificial Intelligence]() Tags: [a-b-testing](), [causal-inference](), [data-science](), [experimentation](), [headspace](), [machine-learning](), [product-experimentation](), [recommendation-system]() ### AI overview This article describes how Headspace evaluates its personalized Content Customizer recommendation system with online controlled experiments, or A/B tests. It explains why the full recommendation system should be assessed before production and outlines experiment design, including recommendation surfaces, supported platforms, and audience selection. ### Source excerpt Author: Rohan Singh Rajput. Rohan is a Senior Data Scientist at Headspace. He combines his passion for Machine Learning with Causal Inference to improve the mindfulness and meditation practices of Headspace users. "If you can't measure it, you can't improve it." -- Peter Drucker. Motivation Content Customizer is Headspace's personalized recommendation system. Content Customizer uses historical data to train its machine learning model and provide personalized recommendations to the users, helping them discover more relevant content. There are various components involved in building a recommendation system, and the ML model is only one of them. Therefore, it is essential to evaluate the effectiveness of the recommendation system as a whole before deploying it to production. Online Controlled Experiments help us to assess our system's impact with statistical evidence. Online Controlled Experiments, a.k.a A/B testing, are the gold standard for estimating causality with high probability. A data-driven decision-making culture helps estimate the measurement's uncertainty to refute the null hypothesis based on experimental data. Furthermore, a random assignment of the users into a control-treatment group allows us to safely ignore the unobserved factors and model the parameters as random variables1. Experiment Design The following components are required to design the experiment. Recommendation Surface Area: We have a total of three surface areas for this experiment: First, in the Today tab, we will use the last three slots to display ML recommendations. Figure 1: Dynamic Playlist on Today's Tab Second is the Hero module, which is the top banner area of the Meditate/Sleep/Focus/Move tabs. Lastly, we will be using the recommended sub-tabs that also have three slots each for ML-powered content. Figure 2: Hero and Recommended Module of other four tabs In total, we have 19 places available for the experiment. The platform for Recommendation: Headspace serves on multiple platform ## Explainable and Accessible AI: Using Push Notifications to Broaden the Reach of ML at Headspace DevFeed: [Explainable and Accessible AI: Using Push Notifications to Broaden the Reach of ML at Headspace]() Original publisher: [Read original article]() Author: Headspace Published: 2021-10-18T19:08:11Z Content type: article Language: en Sources: [Stories by Headspace on Medium]() Topics: [Machine learning](), [recommendation systems](), [Cloud](), [Back end](), [Android](), [iOS](), [Web]() Tags: [ai](), [android](), [backend](), [cloud](), [headspace](), [ios](), [machine-learning](), [ml](), [notifications](), [push-notification](), [push-notifications](), [recommendations](), [web]() ### AI overview Headspace describes infrastructure that combines machine-learning recommendations, cloud services, and Braze to deliver personalized content through push notifications to members who have not recently opened the app. The article explains the limitations of relying on in-app access and says the initial experiment was implemented as an evergreen production model. ### Source excerpt Author: Matt Linder / Co-Author: Koyuki Nakamori TL;DR Headspace wanted to serve personalized content to members who had not recently opened the app We built out infrastructure that allowed us to leverage our Cloud services, plus the 3rd party Braze Customer Engagement platform, to serve Machine Learning-recommended content to members via push notifications Our first experiment doing so got GREAT results We implemented that experiment as an evergreen model in production You should, too Introduction and Problem Statement Headspace's core products are iOS, Android, and web-based apps that focus on improving the health and happiness of their users through mindfulness, meditation, sleep, exercise, and focus content. Machine learning models are core to our user experiences by offering recommendations that engage our users with relevant, personalized content that builds consistent habits in their lifelong journey. Headspace also has an amazing Lifecycle marketing team, who do incredible work in providing regular communications -- through email, push notifications, in-app modals, and other surfaces -- that further our users' engagement, helping them on their journey from prospects to members to advocates. Until recently, those two aspects -- Machine Learning and the out-of-app communication channels served by Lifecycle- were totally separate. Traditionally, users consume our ML models' personalized recommendations by Navigating to one of the app's tabs / views that contains content (for example, our Today tab). Sending a request from the app client to ask our backend content services to load content to serve users. Our content services then forward the request to our Prediction Service, which supplies a content recommendation for that user (or, if none is available, generates a default fallback content to serve). The Problem This approach has three primary limitations: It isn't able to directly respond to the immediate user's context. For instance, if a user has recently sear ## The MLOps Playbook: Best Practices for Ensuring Reliability of ML Systems DevFeed: [The MLOps Playbook: Best Practices for Ensuring Reliability of ML Systems]() Original publisher: [Read original article]() Author: Headspace Published: 2021-09-28T21:04:00Z Content type: opinion Language: en Sources: [Stories by Headspace on Medium]() Topics: [MLOps](), [Machine learning](), [Model Development](), [version-control](), [Software Engineering]() Tags: [benchmarking](), [data-science](), [dataops](), [datasets](), [machine-learning](), [mlops](), [model-development](), [observability](), [quality](), [software-engineering](), [version-control]() ### AI overview This article presents an MLOps playbook for assessing the production readiness and operational reliability of machine learning systems. It recommends version-controlling model code, data, parameters, and metrics; accounting for model complexity and operating cost; evaluating quality across important data slices; and testing for inclusion and potential bias. ### Source excerpt Author: Mo Messidi, a seasoned DataOps leader at Headspace with a decade of experience in both startups and enterprises. His mission is to help organizations operationalize their data. This article presents a simple, yet comprehensive, set of MLOps best practices for organizations to assess the production readiness of machine learning systems. It has also proved beneficial for assessing off-the-shelf MLOps platforms for feature and functionality completeness. There may be a whole range of software engineering best practices towards producing trustworthy software, but similar best practices for machine learning system operations are only in their infancy. Stage A: Model Development Model code, data, parameters, and metrics are version controlled It is important to know the code, data, and artifacts that produced a model. You can do this by having good version control for the model specification, including hyper-parameters and experiment artifacts. This ensures reproducibility, enables rollbacks, and de-risks system changes. A simpler model is not better The more complex a model, the higher its cost to operate. Adding a complexity tax to model assessment equations can help reveal the true incremental value of a given model. Model quality is sufficient for all important data slices ML models quality metrics can easily get lost in the averages when benchmarking against full datasets. It is important to examine quality independently for temporal and location variations. It is common for models to exhibit large drops in quality for specific data slices e.g. users in Denmark vs. users in Europe. The model is tested for considerations of inclusion ML unfairness may occur due to the way that people's choices affect what training data is used for something like word embedding. This can then lead to biased system behavior because it is based on these bad choices done during training data set creation. Measuring what you are doing is important to make systems for everyone. For ## Real-Time Machine Learning at Headspace DevFeed: [Real-Time Machine Learning at Headspace]() Original publisher: [Read original article]() Author: Headspace Published: 2021-08-16T16:53:19Z Content type: article Language: en Sources: [Stories by Headspace on Medium]() Topics: [Machine Learning & Artificial Intelligence](), [real-time](), [Inference](), [recommendations](), [Apache Spark](), [databricks](), [Amazon Simple Queue Service (SQS)](), [Amazon Web Services](), [Amazon SageMaker AI](), [Streaming](), [Orchestration](), [Model Development]() Tags: [android](), [apache-spark](), [aws](), [data](), [databricks](), [headspace-engineering](), [inference](), [machine-learning](), [orchestration](), [real-time](), [recommendations](), [sqs](), [streaming](), [web]() ### AI overview This article describes Headspace's infrastructure for real-time machine learning inference and personalized content recommendations. It explains how the team uses modular Publishing, Receiver, Orchestration, and Serving layers with Apache Spark Structured Streaming on Databricks, AWS SQS, Lambda, and SageMaker. ### Source excerpt Author: Yu Chen, Senior Software Engineer / Co-author: Koyuki Nakamori, Senior Engineering Manager Introduction Data is often most valuable when it can be immediately leveraged to make decisions in the moment, but traditionally consumer data is ingested, transformed, persisted, and sits dormant for lengthy periods of time before being used by machine learning and analytics teams. Headspace's core products are iOS, Android, and web-based apps that focus on improving the health and happiness of its users through mindfulness, meditation, sleep, exercise, and focus content. Machine learning models are core to our user experiences by offering recommendations that engage our users with new relevant, personalized content that builds consistent habits in their lifelong journey. Finding a way to leverage user data to generate real-time insights and decisions means that consumer-facing products like the Headspace app can dramatically shorten the end-to-end user feedback loop: actions that users perform a few seconds, or minutes ago can be incorporated in generating a more relevant, personalized, and context-specific content recommendation for the user. This means our ML models could incorporate dynamic features that update throughout the course of a user's day, or even an individual session: Current session bounce rates for sleep content Semantic embeddings for recent user search terms (if a user recently searched for "preparing for big exam", the ML model can assign more weight to Focus-themed meditations) Users' biometric data (i.e., if step counts and heart rate are increasing over the last 10 minutes, we can recommend Move or Exercise content) The Headspace Machine Learning team architected a solution by decomposing the infrastructure systems into modular Publishing, Receiver, Orchestration, and Serving layers, leveraging Apache Spark Structured Streaming on Databricks, AWS SQS, Lambda, and Sagemaker to deliver real-time inference capabilities for our machine learning mod ## Human-centered Software Development DevFeed: [Human-centered Software Development]() Original publisher: [Read original article]() Author: Headspace Published: 2021-05-27T20:52:21Z Content type: opinion Language: en Sources: [Stories by Headspace on Medium]() Topics: [Development](), [User Experience](), [Software Engineering](), [interfaces](), [Code](), [App]() Tags: [developers](), [development](), [end-user-experience](), [engineering](), [experience-design](), [interfaces](), [software](), [software-development](), [software-engineer](), [user-experience](), [user-experience-design]() ### AI overview The article argues that software development should be human-centered because developers are users of the code and systems they build. It applies user-experience principles to the development process, emphasizing code readability, maintainability, and attention to engineers' human factors. ### Source excerpt Build stronger software by designing for human interaction Author: Edward Park, Senior Software Engineer Creating software is a human-centered endeavor. The software that we build is used directly by human users. Whether we're building a mobile application, service, or tool, the cumulative moments of delight (or abhorrence) that our human users experience determine the efficacy of the software we build. For this reason, software organizations invest in user experience design, in addition to the nuts-and-bolts of the technology under the hood. Product designers leverage their expertise in interaction psychology to create user flows that are useful, familiar, and discoverable. Designers understand that application interfaces are only as effective as their users find them to be useful. Therefore, the best user experiences incorporate visual components that facilitate task completion while working with the psychological heuristics that humans intuitively employ. Humans seek sensory cues and constraints to navigate a deluge of stimuli and process large amounts of information, and we do so while interacting with computing interfaces as well as the real world. In essence, software organizations are constantly trying to optimize how humans interact with computers. Today, the practice of using design to hone user experience is predominantly applied to visual interfaces. These are commonly the front ends of applications, where humans interact with the visually rendered software (in a browser or mobile app). What about the act of software development itself? OUR FELLOW DEVELOPERS ARE USERS, TOO Most observers would think that software development strives to translate human instructions into machine-understandable directives in a reductive, one-way communication flow. We are "programming" computers to do our bidding, and we do so with cold, logical artifacts composed of code modules. While the software itself is mostly evaluated against human experiences, we compartmentalize th ## Web Authentication at Headspace using Auth0 DevFeed: [Web Authentication at Headspace using Auth0]() Original publisher: [Read original article]() Author: Headspace Published: 2021-04-26T20:43:56Z Content type: tutorial Language: en Sources: [Stories by Headspace on Medium]() Topics: [Authentication](), [Auth0](), [Web](), [Cookies](), [browser]() Tags: [auth0](), [authentication](), [browser](), [cookies](), [login](), [password](), [server](), [web](), [web-authentication]() ### AI overview This article explains modern web authentication using usernames, passwords, session tokens, and browser cookies. It describes Headspace's authentication requirements across web, backend, iOS, and Android applications and explains why the company chose Auth0 as its authentication provider. ### Source excerpt Author: Jesse Bond, Senior Software Engineer, Web Introduction: How does modern authentication work? Authentication allows users to securely log into a system and verify that they are who they say they are in subsequent requests. On today's web, this usually involves creating an account using a combination of username and password. Let's start with a quick refresher on how modern web authentication works. The first step is always a user submitting their login credentials into a form. Once the form is submitted, the server will determine whether or not the username and password match that of a previously registered user. If they do match, the server will return what is known as a session token. A session token is a unique identifier that authenticates requests as coming from the same user that just logged in. It's important to note that session tokens usually are temporary and have an expiration, which is determined by the server. Now that the user has a session token, each request will need to submit the session token to ensure authentication. It might sound tedious to have to submit the session token on each request, but have no fear, the cookie is here! Cookies are small bits of information that the browser stores for a domain. Cookies are also automatically submitted when a request is sent by the browser. Does this sound like a great way to handle the session token or what? Correct! Session tokens are almost always stored in a browser cookie to make authenticating requests a breeze. Authentication requirements Headspace has a large number of Single Page Apps, backend services, websites, and iOS and Android applications. We needed a robust solution for authentication that met the following requirements: Multiple platform and language support Scalable to millions of users Reliable with high uptime Ability to separate our users into different groups with varying permissions (i.e., admins vs. standard users) Ability to restrict user access to certain applications. Fo ## Filling the Stack: Five Things That Helped Me Transition From Frontend to Backend Development DevFeed: [Filling the Stack: Five Things That Helped Me Transition From Frontend to Backend Development]() Original publisher: [Read original article]() Author: Headspace Published: 2021-03-30T18:57:45Z Content type: tutorial Language: en Sources: [Stories by Headspace on Medium]() Topics: [backend-development](), [Front end](), [API](), [Development](), [React](), [Redux]() Tags: [api](), [backend-development](), [development](), [facebook](), [frontend](), [instagram](), [linkedin](), [react](), [redux]() ### AI overview A Headspace engineer shares five tips drawn from transitioning from frontend to backend/API engineering while working at the company. The article describes communicating career intentions, exploring internal opportunities, and building new skills on the job. ### Source excerpt A few tips on how I added Backend/API engineer to my list of skills while on the job at Headspace. Author: Lee Richardson, Senior Software Engineer Intro I started my career at Headspace (1000 years ago, way back in 2019!) as a Web engineer. Our frontend stack is primarily React, Redux, Redux-Saga, Emotion and several internal libraries for shared functionality/UI components. The first big feature I worked on was a group meditation feature, which we designed to be compatible with live audio or video streams. We were excited to add a live, community-minded feature into the app experience. After several initial product iterations, we decided to focus on other community-minded features, and our squad no longer needed a Web engineer (quick side note on what Headspace calls "squads" -- these are cross-functional groups organized around a particular business focus, usually containing a product owner, tech lead, and engineers across various disciplines). I started to feel lost, like my squad was moving onto bigger and better things and I was being left behind. After a few days of low-grade sulking, I started having conversations with engineers from other squads, taking a lap around the company to see what everyone was working on, and what might be a good fit for my skills. After some research, I transitioned to be primarily a backend engineer in late 2019.In this article, I share a few tips on how my skills expanded while on the job. Make Your Intentions Known When my future role became less defined, I started to doubt myself and my future at Headspace. After some mindful reflection and meditation, I was able to catch myself in a negative thought pattern and snap out of it. I had conversations with my manager and colleagues, to get a feel for what everyone was working on. I was explicit about wanting to find a new team. Word got out, and I eventually decided that I wanted to join the Headspace Health team to explore the healthcare space. I had previously worked with several