# Hacking Scale by Better Stack

A weekly newsletter about building and scaling software from engineers at Better Stack

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## How LinkedIn Reduced GPU Memory Usage by 60% for LLM Training

DevFeed: [How LinkedIn Reduced GPU Memory Usage by 60% for LLM Training](<https://devfeed.tech/articles/how-linkedin-reduced-gpu-memory-usage-by-60-for-llm-training-17974.md>)

Original publisher: [Read original article](<https://newsletter.betterstack.com/p/how-linkedin-reduced-gpu-memory-usage>)

Author: Richard Oliver Bray

Published: 2025-01-16T15:19:14Z

Content type: tutorial

Language: en

Sources: [Hacking Scale by Better Stack](<https://devfeed.tech/sources/hacking-scale-by-better-stack.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [GPU optimization](<https://devfeed.tech/topics/gpu-optimization.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [Library](<https://devfeed.tech/topics/library.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>)

Tags: [gpu](<https://devfeed.tech/tags/gpu.md>), [library](<https://devfeed.tech/tags/library.md>), [llm](<https://devfeed.tech/tags/llm.md>), [memory](<https://devfeed.tech/tags/memory.md>), [performance](<https://devfeed.tech/tags/performance.md>), [techniques](<https://devfeed.tech/tags/techniques.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

This article explains how LinkedIn improved the efficiency of training large language models. It describes the resource demands of pre-training and reports that LinkedIn reduced GPU memory usage by 60% by applying selected GPU performance techniques in a library.

### Source excerpt

LinkedIn hand-picked the best GPU performance techniques and put them in a library

## How Tinder Secures Its 500+ Microservices

DevFeed: [How Tinder Secures Its 500+ Microservices](<https://devfeed.tech/articles/how-tinder-secures-its-500-microservices-17980.md>)

Original publisher: [Read original article](<https://newsletter.betterstack.com/p/how-tinder-secures-its-500-microservices>)

Author: Richard Oliver Bray

Published: 2025-01-08T13:40:48Z

Content type: tutorial

Language: en

Sources: [Hacking Scale by Better Stack](<https://devfeed.tech/sources/hacking-scale-by-better-stack.md>)

Topics: [Microservices](<https://devfeed.tech/topics/microservices.md>), [Security](<https://devfeed.tech/topics/security.md>), [Architecture & Design](<https://devfeed.tech/topics/architecture-design.md>), [Amazon API Gateway](<https://devfeed.tech/topics/amazon-api-gateway.md>), [Authorization](<https://devfeed.tech/topics/authorization.md>), [Authentication](<https://devfeed.tech/topics/authentication.md>), [Routing (disambiguation)](<https://devfeed.tech/topics/routing.md>)

Tags: [api-gateway](<https://devfeed.tech/tags/api-gateway.md>), [authentication](<https://devfeed.tech/tags/authentication.md>), [authorization](<https://devfeed.tech/tags/authorization.md>), [mesh](<https://devfeed.tech/tags/mesh.md>), [microservices](<https://devfeed.tech/tags/microservices.md>), [routing](<https://devfeed.tech/tags/routing.md>), [security](<https://devfeed.tech/tags/security.md>), [tls](<https://devfeed.tech/tags/tls.md>)

### AI overview

This article explains why Tinder's externally accessible services created security and maintenance challenges, and why the team built a customized API gateway to provide consistent session management, scalable configuration, and integration with its Envoy service mesh.

### Source excerpt

Tinder's highly customised solution that fixed their microservice security chaos

## How DoorDash Improved Redis to Handle 10+ Million Reads per Second

DevFeed: [How DoorDash Improved Redis to Handle 10+ Million Reads per Second](<https://devfeed.tech/articles/how-doordash-improved-redis-to-handle-10-million-reads-per-second-17970.md>)

Original publisher: [Read original article](<https://newsletter.betterstack.com/p/how-doordash-improved-redis-to-handle>)

Author: Richard Oliver Bray

Published: 2024-12-18T14:03:03Z

Content type: article

Language: en

Sources: [Hacking Scale by Better Stack](<https://devfeed.tech/sources/hacking-scale-by-better-stack.md>)

Topics: [Redis](<https://devfeed.tech/topics/redis.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Databases](<https://devfeed.tech/topics/databases.md>), [personalization](<https://devfeed.tech/topics/personalization.md>)

Tags: [databases](<https://devfeed.tech/tags/databases.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [performance](<https://devfeed.tech/tags/performance.md>), [personalization](<https://devfeed.tech/tags/personalization.md>), [recommendations](<https://devfeed.tech/tags/recommendations.md>), [redis](<https://devfeed.tech/tags/redis.md>)

### AI overview

DoorDash used Redis to support machine-learning search and recommendation systems requiring tens of millions of reads per second. The article explains Redis's performance and cost tradeoffs and describes why DoorDash selected it over Cassandra, CockroachDB, and Scylla.

### Source excerpt

A handful of insanely clever things DoorDash did to make Redis blazingly fast

## Why Dropbox Built the Robinhood Load Balancer

DevFeed: [Why Dropbox Built the Robinhood Load Balancer](<https://devfeed.tech/articles/how-dropbox-saved-millions-of-dollars-by-building-a-load-balancer-17971.md>)

Original publisher: [Read original article](<https://newsletter.betterstack.com/p/how-dropbox-saved-millions-of-dollars>)

Author: Richard Oliver Bray

Published: 2024-12-11T14:01:03Z

Content type: article

Language: en

Sources: [Hacking Scale by Better Stack](<https://devfeed.tech/sources/hacking-scale-by-better-stack.md>)

Topics: [Cloud](<https://devfeed.tech/topics/cloud.md>), [Algorithms](<https://devfeed.tech/topics/algorithms.md>), [cloud-infrastructure](<https://devfeed.tech/topics/cloud-infrastructure.md>), [servers](<https://devfeed.tech/topics/servers.md>), [real-time](<https://devfeed.tech/topics/real-time.md>)

Tags: [cloud](<https://devfeed.tech/tags/cloud.md>), [dropbox](<https://devfeed.tech/tags/dropbox.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [load-balancing](<https://devfeed.tech/tags/load-balancing.md>), [network](<https://devfeed.tech/tags/network.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [round-robin](<https://devfeed.tech/tags/round-robin.md>), [server](<https://devfeed.tech/tags/server.md>), [servers](<https://devfeed.tech/tags/servers.md>)

### AI overview

The article explains how Dropbox handled overloaded virtual servers with different hardware capabilities. Its traditional fixed-weight round-robin approach could not distribute load evenly, so the team built Robinhood, a dynamic weighted round-robin load balancer with gRPC support.

### Source excerpt

Dropbox saved resources by creating a superior version of a tool everyone uses

## How Stripe Used MongoDB Sharding and Replication to Scale Payments with Minimal Downtime

DevFeed: [How Stripe Used MongoDB Sharding and Replication to Scale Payments with Minimal Downtime](<https://devfeed.tech/articles/how-stripe-processed-1-trillion-in-payments-with-zero-downtime-17979.md>)

Original publisher: [Read original article](<https://newsletter.betterstack.com/p/how-stripe-processed-1-trillion-in>)

Author: Richard Oliver Bray

Published: 2024-12-04T14:02:44Z

Content type: article

Language: en

Sources: [Hacking Scale by Better Stack](<https://devfeed.tech/sources/hacking-scale-by-better-stack.md>)

Topics: [MongoDB](<https://devfeed.tech/topics/mongodb.md>), [Databases](<https://devfeed.tech/topics/databases.md>), [Replication](<https://devfeed.tech/topics/replication.md>), [Database](<https://devfeed.tech/topics/database.md>)

Tags: [database](<https://devfeed.tech/tags/database.md>), [developer](<https://devfeed.tech/tags/developer.md>), [mongodb](<https://devfeed.tech/tags/mongodb.md>), [replication](<https://devfeed.tech/tags/replication.md>), [sharding](<https://devfeed.tech/tags/sharding.md>)

### AI overview

The article explains how Stripe used MongoDB, sharding, replication sets, and query routing to scale payment data while maintaining reliability during migrations. It describes the database architecture and the roles of shards, replicas, Mongos instances, and config servers.

### Source excerpt

Stripe's hand-crafted system that guarantees their data never gets lost

## How Shopify Reduced Metrics Resources by 75%

DevFeed: [How Shopify Reduced Metrics Resources by 75%](<https://devfeed.tech/articles/how-shopify-reduced-metrics-resources-by-75-17977.md>)

Original publisher: [Read original article](<https://newsletter.betterstack.com/p/how-shopify-reduced-metrics-resources>)

Author: Richard Oliver Bray

Published: 2024-11-27T13:02:25Z

Content type: article

Language: en

Sources: [Hacking Scale by Better Stack](<https://devfeed.tech/sources/hacking-scale-by-better-stack.md>)

Topics: [Shopify](<https://devfeed.tech/topics/shopify.md>), [Grafana](<https://devfeed.tech/topics/grafana.md>), [observability](<https://devfeed.tech/topics/observability.md>), [Monitoring](<https://devfeed.tech/topics/monitoring.md>), [cloud-infrastructure](<https://devfeed.tech/topics/cloud-infrastructure.md>), [Server](<https://devfeed.tech/topics/server.md>)

Tags: [cloud-infrastructure](<https://devfeed.tech/tags/cloud-infrastructure.md>), [container](<https://devfeed.tech/tags/container.md>), [grafana](<https://devfeed.tech/tags/grafana.md>), [metrics](<https://devfeed.tech/tags/metrics.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [observability](<https://devfeed.tech/tags/observability.md>), [servers](<https://devfeed.tech/tags/servers.md>), [shopify](<https://devfeed.tech/tags/shopify.md>)

### AI overview

The article explains how Shopify built its own observability tool, Observe, after third-party logging, metrics, and tracing services became expensive and difficult to integrate. Observe uses Grafana for its user interface and visualization, while Shopify monitors metrics from infrastructure that can scale to about one million endpoints.

### Source excerpt

Shopify saved resources by breaking big tools into tiny reusable services

## How GitHub Reduced Repo Storage Size by Over 90%

DevFeed: [How GitHub Reduced Repo Storage Size by Over 90%](<https://devfeed.tech/articles/how-github-reduced-repo-storage-size-by-over-90-17972.md>)

Original publisher: [Read original article](<https://newsletter.betterstack.com/p/how-github-reduced-repo-storage-size>)

Author: Richard Oliver Bray

Published: 2024-11-20T14:03:31Z

Content type: article

Language: en

Sources: [Hacking Scale by Better Stack](<https://devfeed.tech/sources/hacking-scale-by-better-stack.md>)

Topics: [GitHub](<https://devfeed.tech/topics/github.md>), [data](<https://devfeed.tech/topics/data.md>), [Git](<https://devfeed.tech/topics/git.md>), [Code](<https://devfeed.tech/topics/code.md>)

Tags: [data](<https://devfeed.tech/tags/data.md>), [git](<https://devfeed.tech/tags/git.md>), [github](<https://devfeed.tech/tags/github.md>), [servers](<https://devfeed.tech/tags/servers.md>), [storage](<https://devfeed.tech/tags/storage.md>)

### AI overview

This article explains how GitHub identified unreachable Git data that was consuming storage and reduced the size of specific projects by more than 90%. It introduces Git objects, branches, and the ways objects can become unreachable.

### Source excerpt

GitHub tackled its data overload by introducing a completely new type of file

## How SQLite made Notion 30% Faster

DevFeed: [How SQLite made Notion 30% Faster](<https://devfeed.tech/articles/how-sqlite-made-notion-30-faster-17978.md>)

Original publisher: [Read original article](<https://newsletter.betterstack.com/p/how-sqlite-made-notion-30-faster>)

Author: Richard Oliver Bray

Published: 2024-11-06T14:00:53Z

Content type: article

Language: en

Sources: [Hacking Scale by Better Stack](<https://devfeed.tech/sources/hacking-scale-by-better-stack.md>)

Topics: [Notion](<https://devfeed.tech/topics/notion.md>), [SQLite](<https://devfeed.tech/topics/sqlite.md>), [Caching](<https://devfeed.tech/topics/caching.md>), [browser](<https://devfeed.tech/topics/browser.md>)

Tags: [browser](<https://devfeed.tech/tags/browser.md>), [caching](<https://devfeed.tech/tags/caching.md>), [let](<https://devfeed.tech/tags/let.md>), [sqlite](<https://devfeed.tech/tags/sqlite.md>), [tech](<https://devfeed.tech/tags/tech.md>)

### AI overview

This article examines why Notion's page navigation could feel slow with complex content and describes its attempts to improve browser caching. It focuses on using SQLite to cache data in the browser after LocalStorage and IndexedDB approaches had limitations.

### Source excerpt

Notion made a tough bet on emerging tech, and it paid off in a big way

## How Canva Scaled Their Search to Handle 1M+ Searches Per Minute

DevFeed: [How Canva Scaled Their Search to Handle 1M+ Searches Per Minute](<https://devfeed.tech/articles/how-canva-scaled-their-search-to-handle-1m-searches-per-minute-17968.md>)

Original publisher: [Read original article](<https://newsletter.betterstack.com/p/how-canva-scaled-their-search-to>)

Author: Richard Oliver Bray

Published: 2024-10-23T13:01:58Z

Content type: article

Language: en

Sources: [Hacking Scale by Better Stack](<https://devfeed.tech/sources/hacking-scale-by-better-stack.md>)

Topics: [Web](<https://devfeed.tech/topics/web.md>), [Database](<https://devfeed.tech/topics/database.md>), [Data structures](<https://devfeed.tech/topics/data-structures.md>), [Library](<https://devfeed.tech/topics/library.md>), [Code](<https://devfeed.tech/topics/code.md>), [A/B Testing](<https://devfeed.tech/topics/a-b-testing.md>)

Tags: [a-b-testing](<https://devfeed.tech/tags/a-b-testing.md>), [code](<https://devfeed.tech/tags/code.md>), [data-structure](<https://devfeed.tech/tags/data-structure.md>), [database](<https://devfeed.tech/tags/database.md>), [library](<https://devfeed.tech/tags/library.md>), [search](<https://devfeed.tech/tags/search.md>), [software](<https://devfeed.tech/tags/software.md>), [web](<https://devfeed.tech/tags/web.md>)

### AI overview

The article describes Canva's search system, which supports searches across templates, media, fonts, and audio at very high request volumes. It explains that separate servers and search indexes created duplicated code and complicated updates and testing, and introduces search indexes as a way to make text retrieval faster.

### Source excerpt

Canva massively leveled-up their search by stripping away duplicated processes

## How Uber Reduced Their Log Size By 99%

DevFeed: [How Uber Reduced Their Log Size By 99%](<https://devfeed.tech/articles/how-uber-reduced-their-log-size-by-99-17981.md>)

Original publisher: [Read original article](<https://newsletter.betterstack.com/p/how-uber-reduced-their-log-size-by>)

Author: Richard Oliver Bray

Published: 2024-10-09T13:02:55Z

Content type: article

Language: en

Sources: [Hacking Scale by Better Stack](<https://devfeed.tech/sources/hacking-scale-by-better-stack.md>)

Topics: [Logging](<https://devfeed.tech/topics/logging.md>), [Hadoop](<https://devfeed.tech/topics/hadoop.md>), [Filesystems](<https://devfeed.tech/topics/filesystems.md>), [big-data](<https://devfeed.tech/topics/big-data.md>), [Apache Spark](<https://devfeed.tech/topics/spark.md>), [data](<https://devfeed.tech/topics/data.md>)

Tags: [apache-spark](<https://devfeed.tech/tags/apache-spark.md>), [big-data](<https://devfeed.tech/tags/big-data.md>), [cli](<https://devfeed.tech/tags/cli.md>), [clickhouse](<https://devfeed.tech/tags/clickhouse.md>), [data](<https://devfeed.tech/tags/data.md>), [hadoop](<https://devfeed.tech/tags/hadoop.md>), [hdfs](<https://devfeed.tech/tags/hdfs.md>), [logging](<https://devfeed.tech/tags/logging.md>)

### AI overview

The article explains how Uber addressed the storage cost of generating roughly 5 PB of INFO-level logs each month. It describes Uber's use of HDFS and related data-processing tools, while reporting that the company reduced log storage size by 99%.

### Source excerpt

Uber broke apart an open source tool to massively compress their logs

## How Discord Processes 30+ Petabytes of Data

DevFeed: [How Discord Processes 30+ Petabytes of Data](<https://devfeed.tech/articles/how-discord-processes-30-petabytes-of-data-17969.md>)

Original publisher: [Read original article](<https://newsletter.betterstack.com/p/how-discord-processes-30-petabytes>)

Author: Richard Oliver Bray

Published: 2024-09-25T13:02:10Z

Content type: article

Language: en

Sources: [Hacking Scale by Better Stack](<https://devfeed.tech/sources/hacking-scale-by-better-stack.md>)

Topics: [Discord](<https://devfeed.tech/topics/discord.md>), [data](<https://devfeed.tech/topics/data.md>), [scylladb](<https://devfeed.tech/topics/scylladb.md>), [NoSQL](<https://devfeed.tech/topics/nosql.md>), [C++](<https://devfeed.tech/topics/c-plus-plus.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [c-plus-plus](<https://devfeed.tech/tags/c-plus-plus.md>), [data](<https://devfeed.tech/tags/data.md>), [discord](<https://devfeed.tech/tags/discord.md>), [nosql](<https://devfeed.tech/tags/nosql.md>), [scylladb](<https://devfeed.tech/tags/scylladb.md>)

### AI overview

This article explains how Discord analyzes billions of daily messages and manages large-scale data processing. It describes message storage in ScyllaDB, data cleaning, machine-learning use cases, and Discord's development of Derived to automate data transformation for analysis.

### Source excerpt

Discord's genius approach to automating insights from billions of messages

## Figma's 100x Approach to Scaling Its Collaborative Experience

DevFeed: [Figma's 100x Approach to Scaling Its Collaborative Experience](<https://devfeed.tech/articles/figma-s-100x-approach-to-scaling-its-collaborative-experience-17966.md>)

Original publisher: [Read original article](<https://newsletter.betterstack.com/p/figmas-100x-approach-to-scaling-its>)

Author: Richard Oliver Bray

Published: 2024-09-11T13:03:05Z

Content type: article

Language: en

Sources: [Hacking Scale by Better Stack](<https://devfeed.tech/sources/hacking-scale-by-better-stack.md>)

Topics: [Figma](<https://devfeed.tech/topics/figma.md>), [Scalability](<https://devfeed.tech/topics/scalability.md>), [Databases](<https://devfeed.tech/topics/databases.md>), [GraphQL](<https://devfeed.tech/topics/graphql.md>), [React](<https://devfeed.tech/topics/react.md>), [Ruby](<https://devfeed.tech/topics/ruby.md>), [WebSocket](<https://devfeed.tech/topics/websocket.md>), [Redux](<https://devfeed.tech/topics/redux.md>), [.NET 11 Preview 7](<https://devfeed.tech/topics/net-11-preview-7.md>)

Tags: [database](<https://devfeed.tech/tags/database.md>), [figma](<https://devfeed.tech/tags/figma.md>), [graphql](<https://devfeed.tech/tags/graphql.md>), [react](<https://devfeed.tech/tags/react.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [redux](<https://devfeed.tech/tags/redux.md>), [ruby](<https://devfeed.tech/tags/ruby.md>), [scalability](<https://devfeed.tech/tags/scalability.md>), [sharding](<https://devfeed.tech/tags/sharding.md>), [web](<https://devfeed.tech/tags/web.md>)

### AI overview

The article explains how Figma scaled real-time collaboration by developing LiveGraph, an in-house system inspired by GraphQL that acts as a shared data store or cache and manages database queries, updates, and client subscriptions. It also describes the scaling pressures that led to the LiveGraph 100x project.

### Source excerpt

How Figma destroyed their old setup and used the pieces to build a better one

## What caused 8.5 million Windows computers to crash in the CrowdStrike Falcon incident

DevFeed: [What caused 8.5 million Windows computers to crash in the CrowdStrike Falcon incident](<https://devfeed.tech/articles/here-s-what-really-caused-8-5-million-computers-to-crash-17967.md>)

Original publisher: [Read original article](<https://newsletter.betterstack.com/p/heres-what-really-caused-85-million>)

Author: Richard Oliver Bray

Published: 2024-08-28T13:00:58Z

Content type: article

Language: en

Sources: [Hacking Scale by Better Stack](<https://devfeed.tech/sources/hacking-scale-by-better-stack.md>)

Topics: [Security](<https://devfeed.tech/topics/security.md>), [Endpoint Security & XDR](<https://devfeed.tech/topics/endpoint-security-xdr.md>), [Windows](<https://devfeed.tech/topics/windows.md>), [Kernel](<https://devfeed.tech/topics/kernel.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [endpoint-security](<https://devfeed.tech/tags/endpoint-security.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [security](<https://devfeed.tech/tags/security.md>), [windows](<https://devfeed.tech/tags/windows.md>)

### AI overview

This article explains how a CrowdStrike Falcon software issue caused 8.5 million Windows machines to crash on July 19, 2024. It describes Falcon's endpoint-security architecture, including cloud servers, machine-learning analysis, threat intelligence, and kernel-level sensors.

### Source excerpt

How one security product crippled the world because of bad programming

## How Netflix Uses Throttling to Prevent 4 Big Streaming Problems

DevFeed: [How Netflix Uses Throttling to Prevent 4 Big Streaming Problems](<https://devfeed.tech/articles/how-netflix-uses-throttling-to-prevent-4-big-streaming-problems-17975.md>)

Original publisher: [Read original article](<https://newsletter.betterstack.com/p/how-netflix-uses-throttling-to-prevent>)

Author: Richard Oliver Bray

Published: 2024-08-15T13:02:09Z

Content type: article

Language: en

Sources: [Hacking Scale by Better Stack](<https://devfeed.tech/sources/hacking-scale-by-better-stack.md>)

Topics: [Netflix](<https://devfeed.tech/topics/netflix.md>), [Streaming](<https://devfeed.tech/topics/streaming.md>), [Back end](<https://devfeed.tech/topics/backend.md>), [Amazon API Gateway](<https://devfeed.tech/topics/amazon-api-gateway.md>), [Cloud](<https://devfeed.tech/topics/cloud.md>)

Tags: [api-gateway](<https://devfeed.tech/tags/api-gateway.md>), [backend](<https://devfeed.tech/tags/backend.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [netflix](<https://devfeed.tech/tags/netflix.md>), [streaming](<https://devfeed.tech/tags/streaming.md>)

### AI overview

The article explains how Netflix uses throttling to shield content playback from backend problems, including network failures, underscaled services, retry storms, and faulty deployments. It also describes the role of an API Gateway in routing and monitoring requests.

### Source excerpt

Netflix reveals their unconventional trick to keep viewers happy

## How Instagram Reduced Video Processing Time and Improved Video Quality

DevFeed: [How Instagram Reduced Video Processing Time and Improved Video Quality](<https://devfeed.tech/articles/how-instagram-saved-90-of-computing-power-improved-video-quality-17973.md>)

Original publisher: [Read original article](<https://newsletter.betterstack.com/p/how-instagram-saved-90-of-computing>)

Author: Richard Oliver Bray

Published: 2024-08-08T13:02:43Z

Content type: article

Language: en

Sources: [Hacking Scale by Better Stack](<https://devfeed.tech/sources/hacking-scale-by-better-stack.md>)

Topics: [Instagram](<https://devfeed.tech/topics/instagram.md>), [Compression](<https://devfeed.tech/topics/compression.md>), [Streaming](<https://devfeed.tech/topics/streaming.md>), [Algorithms](<https://devfeed.tech/topics/algorithms.md>)

Tags: [compression](<https://devfeed.tech/tags/compression.md>), [encoding](<https://devfeed.tech/tags/encoding.md>), [h-264](<https://devfeed.tech/tags/h-264.md>), [instagram](<https://devfeed.tech/tags/instagram.md>), [quality](<https://devfeed.tech/tags/quality.md>), [streaming](<https://devfeed.tech/tags/streaming.md>), [video](<https://devfeed.tech/tags/video.md>)

### AI overview

The article explains Instagram's video upload and publishing pipeline, focusing on encoding and packaging. It describes how Instagram uses AV1 and H.264 encodings, and reports that reducing processing time also improved video quality.

### Source excerpt

Ditching this key process gave Instagram users a better video experience

## How OpenAI Scaled Kubernetes to 7,500 Nodes by Removing One Plugin

DevFeed: [How OpenAI Scaled Kubernetes to 7,500 Nodes by Removing One Plugin](<https://devfeed.tech/articles/how-openai-scaled-kubernetes-to-7-500-nodes-by-removing-one-plugin-17976.md>)

Original publisher: [Read original article](<https://newsletter.betterstack.com/p/how-openai-scaled-kubernetes-to-7500>)

Author: Richard Oliver Bray

Published: 2024-07-31T13:54:05Z

Content type: article

Language: en

Sources: [Hacking Scale by Better Stack](<https://devfeed.tech/sources/hacking-scale-by-better-stack.md>)

Topics: [Kubernetes](<https://devfeed.tech/topics/kubernetes.md>), [OpenAI](<https://devfeed.tech/topics/openai.md>), [Network](<https://devfeed.tech/topics/network.md>), [Containers](<https://devfeed.tech/topics/containers.md>)

Tags: [containers](<https://devfeed.tech/tags/containers.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [network](<https://devfeed.tech/tags/network.md>), [openai](<https://devfeed.tech/tags/openai.md>)

### AI overview

The article explains that OpenAI expanded its Kubernetes infrastructure from 2,500 to 7,500 nodes and identifies removing the Flannel CNI plugin as the most important change behind the scaling effort. It describes Flannel's role in pod networking and why its approach became slow at larger cluster sizes, although the supplied text ends before the replacement or complete technical explanation is presented.

### Source excerpt

The one change that improved OpenAI's network.