# High Scalability Building bigger, faster, more reliable websites. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Kafka 101 DevFeed: [Kafka 101]() Original publisher: [Read original article]() Author: ByteByteGo Published: 2024-05-09T18:55:21Z Content type: tutorial Language: en Sources: [High Scalability]() Topics: [Kafka](), [distributed-systems](), [Streaming](), [Architecture & Design](), [Data structures](), [Microservice](), [Persistence]() Tags: [apache-kafka](), [architecture](), [data-structure](), [distributed-systems](), [kafka](), [microservices](), [open-source](), [persistence](), [real-time-data-streaming]() ### AI overview This introductory article explains Apache Kafka's origins, distributed streaming architecture, and use as a central platform for coordinating data among services and downstream systems. It also describes Kafka's log-based storage model, including immutability, ordered records, concurrent reads, and optimization for high-throughput, cost-efficient HDD storage. ### Source excerpt This is a guest article by Stanislav Kozlovski, an Apache Kafka Committer. If you would like to connect with Stanislav, you can do so on Twitter and LinkedIn. Originally developed in LinkedIn during 2011, Apache Kafka is one of the most popular open-source Apache projects out ## How Hotstar Built a Scalable Real-Time Emoji System for Sports Viewing DevFeed: [How Hotstar Built a Scalable Real-Time Emoji System for Sports Viewing]() Original publisher: [Read original article]() Author: ByteByteGo Published: 2024-03-26T15:32:38Z Content type: article Language: en Sources: [High Scalability]() Topics: [Emoji](), [Architecture & Design](), [Scalability](), [real-time](), [API](), [HTTP](), [client]() Tags: [api](), [architecture](), [article](), [asynchronous](), [blog-post](), [emojis](), [http](), [message-queue](), [real-time](), [scalability]() ### AI overview This article explains how Hotstar built an in-house system to collect user-submitted emojis in real time, condense them into an emoji swarm showing audience mood, and support very high submission volumes. It covers horizontal scalability, component decomposition, asynchronous processing, load balancing, auto-scaling, and HTTP API request handling. ### Source excerpt This blog post was written by Dedeepya Bonthu. This is a repost from her Medium article, approved by the author. In stadiums, sports fans love to express themselves by cheering for their favorite teams, holding up placards and team logos. Emoji's allow fans at home to rapidly express ## Brief History of Scaling Uber DevFeed: [Brief History of Scaling Uber]() Original publisher: [Read original article]() Author: ByteByteGo Published: 2024-03-14T09:53:07Z Content type: article Language: en Sources: [High Scalability]() Topics: [Node.js](), [Back end](), [Concurrency](), [real-time](), [Python](), [LAMP]() Tags: [architecture](), [article](), [backend](), [blog-post](), [concurrency](), [lamp](), [linkedin](), [node-js](), [python](), [real-time](), [scaling]() ### AI overview This article traces Uber's technical scaling from its initial LAMP-based application to a backend using Node.js for real-time workloads and Python for another service. It describes early concurrency problems, the need to process real-time location data, and Node.js's asynchronous event loop and V8 engine as reasons for its adoption. ### Source excerpt This blog post was written by Josh Clemm, Senior Director of Engineering at Uber Eats. This is a repost from his LinkedIn article, approved by the author. On a cold evening in Paris in 2008, Travis Kalanick and Garrett Camp couldn't get a cab. That's when ## Behind AWS S3's Massive Scale DevFeed: [Behind AWS S3's Massive Scale]() Original publisher: [Read original article]() Author: ByteByteGo Published: 2024-03-06T16:15:44Z Content type: article Language: en Sources: [High Scalability]() Topics: [Amazon S3](), [Amazon Web Services](), [cloud-infrastructure](), [Data Infrastructure](), [Microservices](), [Architecture & Design]() Tags: [api](), [aws](), [aws-s3](), [cloud-native](), [machine](), [microservices](), [replication](), [s3](), [service](), [storage]() ### AI overview This guest article examines how AWS S3 grew from a storage service into a massive-scale backbone for analytics, machine learning, backups, and other cloud-native data infrastructure. It outlines S3's growth, major feature milestones, and a high-level architecture involving front-end, namespace, storage, and storage-management services. ### Source excerpt This is a guest article by Stanislav Kozlovski, an Apache Kafka Committer. If you would like to connect with Stanislav, you can do so on Twitter and LinkedIn. AWS S3 is a service every engineer is familiar with. It's the service that popularized the notion of cold-storage ## What the Swedbank outage reveals about the limits of traditional change management DevFeed: [What the Swedbank outage reveals about the limits of traditional change management]() Original publisher: [Read original article]() Author: Bruce Johnston Published: 2023-08-16T16:05:12Z Content type: opinion Language: en Sources: [High Scalability]() Topics: [incident](), [risk-management]() Tags: [banking](), [incident](), [management](), [outage](), [risk-management]() ### AI overview The article examines the Swedish FSA's judgment on Swedbank's April 2022 outage, which followed an unapproved IT change and affected customer balances and payments. It argues that traditional change-management controls, including manual approvals and change meetings, do not by themselves guarantee safe and secure changes. ### Source excerpt This week I've been reading through the recent judgment from the Swedish FSA on the Swedbank outage. If you're unfamiliar with this story, Swedbank had a major outage in April 2022 that was caused by an unapproved change to their IT systems. It temporarily left nearly ## Lessons Learned Running Presto at Meta Scale DevFeed: [Lessons Learned Running Presto at Meta Scale]() Original publisher: [Read original article]() Author: Philip Bell Published: 2023-07-16T15:42:21Z Content type: article Language: en Sources: [High Scalability]() Topics: [presto](), [Meta](), [Automation](), [Deployment](), [gateway](), [SQL](), [Availability](), [Open Source](), [datacenter]() Tags: [analytics](), [automation](), [availability](), [data-centers](), [database-scalability](), [ddatabase](), [deployment](), [gateway](), [high-availablilty](), [high-performance](), [high-scalability](), [load-balancer](), [open-source](), [presto](), [scalability](), [sql]() ### AI overview The article presents lessons from running Presto, an open source SQL query engine, at Meta scale. It discusses deploying releases across many clusters, maintaining availability for interactive queries, using the Gateway load balancer to route queries, and automating cluster updates and capacity management. ### Source excerpt Presto is a free, open source SQL query engine. We've been using it at Meta for the past ten years, and learned a lot while doing so. Running anything at scale - tools, processes, services - takes problem solving to overcome unexpected challenges. Here are four things we learned while ## Gossip Protocol Explained DevFeed: [Gossip Protocol Explained]() Original publisher: [Read original article]() Author: NK Published: 2023-07-16T15:38:02Z Content type: tutorial Language: en Sources: [High Scalability]() Topics: [distributed-systems](), [Protocol (disambiguation)](), [Availability](), [consistency](), [Scalability](), [Resilience]() Tags: [architecture](), [availability](), [consistency](), [distributed-algorithm](), [distributed-system](), [distributed-systems](), [messages](), [protocol](), [resilience](), [scalability]() ### AI overview This article explains the gossip protocol as a peer-to-peer approach for maintaining state and communicating in distributed systems. It contrasts gossip-based state management with centralized services such as Apache Zookeeper and discusses availability, consistency, scalability, resilience, and message-broadcasting techniques. ### Source excerpt You can subscribe to the system design newsletter to excel in system design interviews and software architecture. The original article was published on systemdesign.one website. What Is Gossip Protocol? The typical problems in a distributed system are the following [1], [11]: maintaining the system state (liveness of nodes) communication ## Consistent hashing algorithm DevFeed: [Consistent hashing algorithm]() Original publisher: [Read original article]() Author: NK Published: 2023-02-22T16:39:15Z Content type: tutorial Language: en Sources: [High Scalability]() Topics: [consistent hashing](), [Algorithm](), [distributed-systems](), [Cache](), [Scalability](), [Server]() Tags: [algorithm](), [architecture](), [cache](), [consistent-hashing](), [distributed-system](), [hashing](), [high-availability](), [load-balancing](), [low-latency](), [scalability](), [sharding](), [system-design]() ### AI overview The article explains how consistent hashing places nodes and data keys on a virtual hash ring, then assigns data by traversing the ring clockwise to the next node. It discusses cache-server scaling, hotspot reduction, dynamic load, replication, availability, and latency in distributed systems. ### Source excerpt You can subscribe to the system design newsletter to excel in system design interviews and software architecture. You can view the original article Consistent hashing explained on systemdesign.one website. How does consistent hashing work? At a high level, consistent hashing performs the following operations: The output of the hash ## What is Cloud Computing? According to ChatGPT. DevFeed: [What is Cloud Computing? According to ChatGPT.]() Original publisher: [Read original article]() Author: High Scalability Published: 2022-12-16T17:04:08Z Content type: opinion Language: en Sources: [High Scalability]() Topics: [cloud-computing](), [ChatGPT]() Tags: [ai](), [ai-ml](), [availability](), [chatgpt](), [cloud-computing](), [data](), [data-processing](), [networking](), [reliability](), [scalability]() ### AI overview The article presents ChatGPT's explanation of cloud computing, including internet-based access to shared computing resources, scalability, cost savings, flexibility, and availability. It also comments that ChatGPT's answers are broad but lack depth and argues that AI tools are more likely to augment people than replace them for now. ### Source excerpt DALL-E picture generated from the answer. I was curious, given all the ChatGPT love, what it would make of some of our favorite topics. I was both impressed and unimpressed. Impressed. ChatGPT instantly generated an response that might make a good answer in a Miss Universe contest. You know, ## Stuff The Internet Says On Scalability For December 2nd, 2022 DevFeed: [Stuff The Internet Says On Scalability For December 2nd, 2022]() Original publisher: [Read original article]() Author: High Scalability Published: 2022-12-02T17:08:30Z Content type: article Language: en Sources: [High Scalability]() Topics: [Scalability](), [Cloud](), [API](), [AWS Lambda](), [Caching](), [DDoS](), [Memory Safety](), [ibm]() Tags: [api](), [aws-lambda](), [caching](), [cloud](), [ddos](), [hot-links](), [memory-safety](), [scalability]() ### AI overview A December 2, 2022 HighScalability roundup collects statistics and observations about internet-scale systems, including GraphQL traffic, AWS Lambda usage, cloud migrations, caching, edge latency, DDoS mitigation, and memory safety. ### Source excerpt Never fear, HighScalability is here! 1958: An engineer wiring an early IBM computer 2021: An engineer wiring an early IBM quantum computer. @enclanglement My Stuff: I'm proud to announce a completely updated and expanded version of Explain the Cloud Like I'm 10! This version adds 2x ## The Cloud is Not a Railroad - An Argument Against the Vertical Separation of Cloud Providers DevFeed: [The Cloud is Not a Railroad - An Argument Against the Vertical Separation of Cloud Providers]() Original publisher: [Read original article]() Author: High Scalability Published: 2022-10-24T10:52:23Z Content type: opinion Language: en Sources: [High Scalability]() Topics: [Cloud](), [cloud-infrastructure](), [Software]() Tags: [cloud](), [developer](), [industry](), [infrastructure](), [providers](), [software]() ### AI overview The article argues against vertically separating cloud providers, saying the proposed regulation would not solve the relevant problems and could reduce the quality of software developers build by limiting the services cloud providers can offer. ### Source excerpt There's a move to regulate cloud providers by vertically separating the services they offer. Like railroads of yore, who were not allowed to provide freight services on top of their base services, cloud providers would not be allowed to provide services on top of their base platform services. ## Sponsored Post: G-Core Labs, Close, Wynter, Pinecone, Kinsta, Bridgecrew, IP2Location, StackHawk, InterviewCamp.io, Educative, Stream, Fauna, Triplebyte DevFeed: [Sponsored Post: G-Core Labs, Close, Wynter, Pinecone, Kinsta, Bridgecrew, IP2Location, StackHawk, InterviewCamp.io, Educative, Stream, Fauna, Triplebyte]() Original publisher: [Read original article]() Author: High Scalability Published: 2022-07-18T13:38:51Z Content type: article Language: en Sources: [High Scalability]() Topics: [site-reliability-engineering](), [Databases](), [NoSQL](), [Monitoring](), [cloud-infrastructure]() Tags: [databases](), [hiring](), [infrastructure](), [monitoring](), [nosql](), [site-reliability](), [sponsored](), [sponsored-post]() ### AI overview A sponsored "Who's Hiring?" post containing recruitment and promotional notices for Site Reliability Engineer, system administrator, DevOps Engineer, SysOps Engineer, and JavaScript Developer roles, along with training and developer-service promotions. The supplied text does not present a substantive technical finding. ### Source excerpt Who's Hiring? Close is building the sales communication platform of the future and we're looking for a Site Reliability Engineer to help us accomplish that goal. If you have a passion for databases (both SQL and NoSQL), significant experience building, managing, and monitoring infrastructure, databases, and ## Stuff The Internet Says On Scalability For July 11th, 2022 DevFeed: [Stuff The Internet Says On Scalability For July 11th, 2022]() Original publisher: [Read original article]() Author: High Scalability Published: 2022-07-11T15:11:55Z Content type: article Language: en Sources: [High Scalability]() Topics: [Scalability](), [Kafka](), [Go Language](), [Amazon Web Services]() Tags: [aws](), [go](), [hot-links](), [kafka](), [latency](), [scalability]() ### AI overview A July 2022 roundup of internet material related to scalability, including infrastructure figures, distributed data processing with Kafka and AWS, processor design, Go generics performance, concurrent connections, and security and privacy statistics. ### Source excerpt Never fear, HighScalability is here! Every cell a universe. Most detailed image of a human cell to date. @microscopicture Other images considered: one byte of RAM in 1946; visual guide on troubleshooting Kubernetes; Cloudflare using lava lamps to generate cryptographic keys; 5MB of data looked like in 1966 My Stuff: ## Sponsored Post: G-Core Labs, Close, Wynter, Pinecone, Kinsta, Bridgecrew, IP2Location, StackHawk, InterviewCamp.io, Educative, Stream, Fauna, Triplebyte DevFeed: [Sponsored Post: G-Core Labs, Close, Wynter, Pinecone, Kinsta, Bridgecrew, IP2Location, StackHawk, InterviewCamp.io, Educative, Stream, Fauna, Triplebyte]() Original publisher: [Read original article]() Author: High Scalability Published: 2022-06-10T14:39:03Z Content type: article Language: en Sources: [High Scalability]() Topics: [site-reliability-engineering](), [Databases](), [Monitoring](), [cloud-infrastructure](), [NoSQL](), [Back end](), [Linux](), [Ansible](), [hosting](), [PHP](), [WordPress]() Tags: [ansible](), [backend](), [databases](), [hosting](), [important](), [infrastructure](), [linux](), [monitoring](), [nosql](), [php](), [site-reliability](), [sponsored](), [sponsored-post]() ### AI overview A sponsored post listing hiring and recruitment opportunities from Close, Wynter, Kinsta, InterviewCamp.io, and Triplebyte. The listings seek site reliability, system administration, DevOps, SysOps, JavaScript, and related technology candidates, mentioning databases, infrastructure, Linux, PHP, Ansible, LXC, hosting, and WordPress. ### Source excerpt Who's Hiring? Close is building the sales communication platform of the future and we're looking for a Site Reliability Engineer to help us accomplish that goal. If you have a passion for databases (both SQL and NoSQL), significant experience building, managing, and monitoring infrastructure, databases, and