# Achieving Near-Linear Training Scalability for Pinterest's Foundation Models

DevFeed: [Achieving Near-Linear Training Scalability for Pinterest's Foundation Models](<https://devfeed.tech/articles/achieving-near-linear-training-scalability-for-pinterest-s-foundation-models-1225.md>)

Original publisher: [Read original article](<https://medium.com/pinterest-engineering/achieving-near-linear-training-scalability-for-pinterests-foundation-models-14d4f59fe6f6?source=rss----4c5a5f6279b6---4>)

Author: Pinterest Engineering

Published: 2026-06-25T16:01:02Z

Content type: article

Language: en

Sources: [Pinterest Engineering Blog - Medium](<https://devfeed.tech/sources/pinterest-engineering-blog-medium.md>)

Topics: [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [networking](<https://devfeed.tech/topics/networking.md>)

Tags: [aws](<https://devfeed.tech/tags/aws.md>), [distributed-training](<https://devfeed.tech/tags/distributed-training.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [foundation-models](<https://devfeed.tech/tags/foundation-models.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [nccl](<https://devfeed.tech/tags/nccl.md>), [pinterest](<https://devfeed.tech/tags/pinterest.md>), [scalability](<https://devfeed.tech/tags/scalability.md>), [training](<https://devfeed.tech/tags/training.md>)

## AI overview

Pinterest describes optimizing multi-node distributed training for an embedding-heavy foundation ranking model. The reported improvements raise scaling from 1.13x to 2.0x at two nodes and from 1.21x to 3.9x at four nodes, with 7.5x scaling at eight nodes.

## Source excerpt

Sheng Huang | Software Engineer, AI Platform; Pong Eksombatchai | Machine Learning Engineer, Applied Sciences; Saurabh Vishwas Joshi | Software Engineer, AI Platform; Gaurav Arora | Software Engineer, AI Platform; Karthik Anantha Padmanabhan | Engineering Director, AI Platform At Pinterest, foundation models power recommendations for over 600 million monthly active users. Our latest Foundation Model (ACM RecSys 2025) pre-trains on two years of user activity data and is deployed into Home feed and Related Pins ranking, the platform's two most important recommendation systems. Multi-node distributed training is the key to unlocking the next level of that capacity.¹ But when we first attempted multi-node training, adding a second machine made training 5x slower, producing a scaling factor of roughly 0.2x. Enabling AWS Elastic Fabric Adapter (EFA) for OS-bypass networking fixed the networking layer and recovered a viable baseline, but scaling was still poor: 1.13x at 2 nodes and 1.21x at 4 nodes. Three extra nodes, 3x more GPUs, 3x more cost, yet only 21% more throughput. This post describes how we took 2-node scaling from 1.13x to 2.0x and 4-node scaling from 1.21x to 3.9x (97.5% of ideal), then extended to 8 nodes at 7.5x. The larger models this unlocked have driven significant engagement gains across Pinterest's recommendation surfaces. Figure 1: Training scalability before and after optimization. Left: before EFA and optimization, adding a second node degraded throughput to 0.2x of single-node. Right: after optimization, scaling is near-linear across 2, 4, and 8 nodes, with 8-node reaching 7.5x (93.75% of ideal).Background Training scalability measures whether adding more resources yields proportionally more throughput. Training efficiency measures how much throughput you extract from the same resources. This post focuses on scalability. Our Foundation Ranking Model is embedding-heavy: approximately 99% of parameters reside in embedding tables, with the dense transf