# Evaluating Binary Classification Metrics on Imbalanced Datasets

DevFeed: [Evaluating Binary Classification Metrics on Imbalanced Datasets](<https://devfeed.tech/articles/stop-misusing-roc-curve-and-gini-navigate-imbalanced-datasets-with-confidence-35654.md>)

Original publisher: [Read original article](<https://engineering.klarna.com/stop-misusing-roc-curve-and-gini-navigate-imbalanced-datasets-with-confidence-5edec4c187d7?source=rss----86090d14ab52---4>)

Author: Angel Igareta

Published: 2023-11-09T09:22:53Z

Content type: tutorial

Language: en

Sources: [Klarna Engineering](<https://devfeed.tech/sources/klarna-engineering.md>)

Topics: [Data Science](<https://devfeed.tech/topics/data-science.md>), [Machine Learning & Artificial Intelligence](<https://devfeed.tech/topics/machine-learning-artificial-intelligence.md>), [data](<https://devfeed.tech/topics/data.md>)

Tags: [classification](<https://devfeed.tech/tags/classification.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [false-positives](<https://devfeed.tech/tags/false-positives.md>), [imbalanced-data](<https://devfeed.tech/tags/imbalanced-data.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [model](<https://devfeed.tech/tags/model.md>), [predictions](<https://devfeed.tech/tags/predictions.md>)

## AI overview

This tutorial examines GINI and ROC_AUC for evaluating binary classification models, explaining why their effectiveness can be compromised by imbalanced datasets and presenting the Precision-Recall curve as a potentially more robust alternative.

## Source excerpt

Discover how the Precision-Recall curve can provide a more robust metric for binary classification in data science and machine learning. Imagine stepping into the complex world of binary classification problems. As a Senior Data Scientist at Klarna, this is my day-to-day reality. Binary classification is a cornerstone of data science, with applications touching everything from credit default predictions to medical diagnoses and spam detection. Yet, these problems come with their own unique set of challenges. Metrics such as the GINI coefficient and ROC_AUC often serve as our compass in this maze. They are widely trusted and used for evaluating models. But here's the catch: they might not always point us in the right direction. Can we rely on them blindly, or do we need to dig deeper? The path gets even more challenging when we encounter imbalanced datasets. In such cases, the effectiveness of our trusted metrics can be seriously compromised. In this post, I invite you to join me on a journey to explore these metrics in greater depth. We will question their effectiveness, understand their limitations, and reveal alternatives that could prove to be more reliable navigational tools in the world of binary classification problems. Understanding Model Predictions and Metrics To truly grasp the nuances of model evaluation, let's start by setting the stage with a real-world scenario that we often encounter at Klarna. Imagine we're tasked with predicting customer loan defaults. We have two categories to consider -- paid or default. However, in our scenario, the default rate is a mere 2%. This is a classic case of data imbalance, and it's exactly the kind of challenge we're up against. To evaluate our model's performance in this scenario, we need to understand its predictions. We break these down into four distinct outcomes, also known as the confusion matrix: True Positives (TP): These are the customers who our model correctly identifies as defaulters. False Positives (FP): T