# The MLOps Playbook: Best Practices for Ensuring Reliability of ML Systems

DevFeed: [The MLOps Playbook: Best Practices for Ensuring Reliability of ML Systems](<https://devfeed.tech/articles/the-mlops-playbook-best-practices-for-ensuring-reliability-of-ml-systems-24580.md>)

Original publisher: [Read original article](<https://medium.com/headspace-engineering/the-mlops-playbook-best-practices-for-ensuring-reliability-of-ml-systems-75203dc60763?source=rss-3da90e297190------2>)

Author: Headspace

Published: 2021-09-28T21:04:00Z

Content type: opinion

Language: en

Sources: [Stories by Headspace on Medium](<https://devfeed.tech/sources/stories-by-headspace-on-medium.md>)

Topics: [MLOps](<https://devfeed.tech/topics/mlops.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>), [version-control](<https://devfeed.tech/topics/version-control.md>), [Software Engineering](<https://devfeed.tech/topics/software-engineering.md>)

Tags: [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [dataops](<https://devfeed.tech/tags/dataops.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [mlops](<https://devfeed.tech/tags/mlops.md>), [model-development](<https://devfeed.tech/tags/model-development.md>), [observability](<https://devfeed.tech/tags/observability.md>), [quality](<https://devfeed.tech/tags/quality.md>), [software-engineering](<https://devfeed.tech/tags/software-engineering.md>), [version-control](<https://devfeed.tech/tags/version-control.md>)

## AI overview

This article presents an MLOps playbook for assessing the production readiness and operational reliability of machine learning systems. It recommends version-controlling model code, data, parameters, and metrics; accounting for model complexity and operating cost; evaluating quality across important data slices; and testing for inclusion and potential bias.

## Source excerpt

Author: Mo Messidi, a seasoned DataOps leader at Headspace with a decade of experience in both startups and enterprises. His mission is to help organizations operationalize their data. This article presents a simple, yet comprehensive, set of MLOps best practices for organizations to assess the production readiness of machine learning systems. It has also proved beneficial for assessing off-the-shelf MLOps platforms for feature and functionality completeness. There may be a whole range of software engineering best practices towards producing trustworthy software, but similar best practices for machine learning system operations are only in their infancy. Stage A: Model Development Model code, data, parameters, and metrics are version controlled It is important to know the code, data, and artifacts that produced a model. You can do this by having good version control for the model specification, including hyper-parameters and experiment artifacts. This ensures reproducibility, enables rollbacks, and de-risks system changes. A simpler model is not better The more complex a model, the higher its cost to operate. Adding a complexity tax to model assessment equations can help reveal the true incremental value of a given model. Model quality is sufficient for all important data slices ML models quality metrics can easily get lost in the averages when benchmarking against full datasets. It is important to examine quality independently for temporal and location variations. It is common for models to exhibit large drops in quality for specific data slices e.g. users in Denmark vs. users in Europe. The model is tested for considerations of inclusion ML unfairness may occur due to the way that people's choices affect what training data is used for something like word embedding. This can then lead to biased system behavior because it is based on these bad choices done during training data set creation. Measuring what you are doing is important to make systems for everyone. For