# Scaling recommendations service at OLX

DevFeed: [Scaling recommendations service at OLX](<https://devfeed.tech/articles/scaling-recommendations-service-at-olx-20390.md>)

Original publisher: [Read original article](<https://tech.olx.com/scaling-recommendations-service-at-olx-db4548813e3a?source=rss----761b019b483f---4>)

Author: Jordi Esteve Sorribas

Published: 2025-07-08T15:03:23Z

Content type: article

Language: en

Sources: [OLX](<https://devfeed.tech/sources/olx.md>)

Topics: [FastAPI](<https://devfeed.tech/topics/fastapi.md>), [recommendation systems](<https://devfeed.tech/topics/recommendation-systems.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [Back end](<https://devfeed.tech/topics/backend.md>), [Python](<https://devfeed.tech/topics/python.md>), [distributed-systems](<https://devfeed.tech/topics/distributed-systems.md>), [async](<https://devfeed.tech/topics/async.md>)

Tags: [async](<https://devfeed.tech/tags/async.md>), [backend](<https://devfeed.tech/tags/backend.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [fastapi](<https://devfeed.tech/tags/fastapi.md>), [latency](<https://devfeed.tech/tags/latency.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [python](<https://devfeed.tech/tags/python.md>), [recommendation-systems](<https://devfeed.tech/tags/recommendation-systems.md>), [recommendations](<https://devfeed.tech/tags/recommendations.md>), [scalability](<https://devfeed.tech/tags/scalability.md>)

## AI overview

This article describes how OLX scaled a Python FastAPI recommendations service to handle tens of thousands of requests per second with p99 latency below 10 ms. It discusses the service's ownership, data sources, and the use of asynchronous non-blocking I/O.

## Source excerpt

Optimizing FastAPI at Scale: Lessons from OLX's Recommendation PlatformPhoto by Rosy KoIn distributed systems, there is a motto that says 'you are as slow as your slowest tasks'. In Python, thanks to the notorious Global Interpreter Lock (GIL), this issue is amplified: 'your slowest task will make every other task slower'. In this article, I'll walk you through the optimizations we made to scale a FastAPI service that now handles tens of thousands of requests per second, achieving a p99 latency under 10ms.Introduction OLX is a global online marketplace that enables users to buy and sell goods and services, primarily through classified ads. We have a clear vision: to create leading marketplace ecosystems enabled by tech, powered by trust, and loved by customers. Every month, we engage 45 million app users and support over 73 million active listings. To help users seamlessly navigate this vast inventory, we've integrated recommendation systems across multiple touchpoints in all our platforms. These recommendations are powered by the recommendations platform, which is responsible for delivering personalized suggestions across various contexts. Most, if not all, of these are served through a dedicated recommendations service. Over the past few months, we've built and scaled this system within the data team, successfully shifting the ownership from a shared backend service to a service fully owned by the team to gain greater autonomy and flexibility. The team decided to build it with Python, as it is the go-to language for the data and machine learning team and is the most widely used language within both the team and the broader domain. While Python allows for rapid development and prototyping, working at scale has surfaced several challenges and trade-offs. It hasn't been an easy journey, but it's one that's taught us a lot and significantly matured our infrastructure and processes. To Async or Not Async The service consumes data from various sources: ScyllaDB, DynamoD