# Twitch

Latest stories tagged with Engineering in Twitch Blog on Medium

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## ThisEmoteDoesNotExist: Training a GAN for Twitch Emotes

DevFeed: [ThisEmoteDoesNotExist: Training a GAN for Twitch Emotes](<https://devfeed.tech/articles/thisemotedoesnotexist-training-a-gan-for-twitch-emotes-20454.md>)

Original publisher: [Read original article](<https://medium.com/twitch-news/thisemotedoesnotexist-training-a-gan-for-twitch-emotes-a742b6354b73?source=rss----3ae745429979--engineering>)

Author: Avery Gnolek

Published: 2019-07-24T22:05:46Z

Content type: tutorial

Language: en

Sources: [Twitch](<https://devfeed.tech/sources/twitch.md>)

Topics: [Twitch](<https://devfeed.tech/topics/twitch.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [Keras](<https://devfeed.tech/topics/keras.md>), [datasets](<https://devfeed.tech/topics/datasets.md>), [Python](<https://devfeed.tech/topics/python.md>), [pixel](<https://devfeed.tech/topics/pixel.md>)

Tags: [dataset](<https://devfeed.tech/tags/dataset.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [keras](<https://devfeed.tech/tags/keras.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [python](<https://devfeed.tech/tags/python.md>), [training](<https://devfeed.tech/tags/training.md>), [twitch](<https://devfeed.tech/tags/twitch.md>)

### AI overview

A developer describes building a progressive-growing GAN for Twitch emotes as a personal machine-learning project. The article covers scraping roughly 2 million Twitch emote assets, implementing a proof-of-concept GAN with Keras, and training it on a random 200,000-image sample.

### Source excerpt

The idea for this project began when a coworker and I were talking about NVIDIA's photo-realistic generated human faces using StyleGAN and they mentioned "I wish someone made one of those for Twitch emotes." I had always wanted to take some time to learn more about convolution neural networks and was in the middle of a machine learning project for work, so it seemed like trying to build this would be a quick and relevant personal project. The original plan (with my time estimates): Scrape all of the Twitch emote image assets (1 day) Write a progressive growing GAN implementation using Keras as a proof of concept (1 week) Adapt the emote dataset for use with a real research-caliber GAN implementation (1-2 days) Train the GAN on the emote dataset (1 day) 1 . Scrape all of the emote image assets This part of the project was actually quite straightforward. Twitch stores all of the emote images on a CDN using a monotonically increasing numeric emote_id. Each emote is available in three different sizes (1.0, 2.0, 3.0) corresponding to the resolutions of 28x28, 56x56, and 112x112. emoticons/778927/1.0emoticons/778927/2.0emoticons/v1/778927/3.0 I wrote a quick python scraper which would go through ~2 million emote_ids of three different sizes and download the images locally. Even at a modest 200 requests per second, this was able to finish overnight. In fact, I think the limiting factor for the speed of this step was actually writing the image files to my HDD, not the network requests themselves. So far, I was right on track with my time estimate. 2. Write a progressive growing GAN implementation using Keras as a proof of concept As I had no experience with CNNs or deep learning before this project, I wanted to at least attempt to write a progressive growing GAN myself as a way to learn before switching to an existing implementation. As Twitch emotes are not even power of two sizes, it was not possible to exactly replicate the 4x4 pixel to 128x128 pixel growing architecture

## Go memory ballast: How I learnt to stop worrying and love the heap

DevFeed: [Go memory ballast: How I learnt to stop worrying and love the heap](<https://devfeed.tech/articles/go-memory-ballast-how-i-learnt-to-stop-worrying-and-love-the-heap-20449.md>)

Original publisher: [Read original article](<https://medium.com/twitch-news/go-memory-ballast-how-i-learnt-to-stop-worrying-and-love-the-heap-26c2462549a2?source=rss----3ae745429979--engineering>)

Author: Ross Engers

Published: 2019-04-10T21:44:55Z

Content type: article

Language: en

Sources: [Twitch](<https://devfeed.tech/sources/twitch.md>)

Topics: [Go Language](<https://devfeed.tech/topics/go-language.md>), [Twitch](<https://devfeed.tech/topics/twitch.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [Optimization](<https://devfeed.tech/topics/optimization.md>), [API](<https://devfeed.tech/topics/api.md>), [Amazon EC2](<https://devfeed.tech/topics/amazon-ec2.md>), [GraphQL](<https://devfeed.tech/topics/graphql.md>), [nginx](<https://devfeed.tech/topics/nginx.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [ec2](<https://devfeed.tech/tags/ec2.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [gateway](<https://devfeed.tech/tags/gateway.md>), [go](<https://devfeed.tech/tags/go.md>), [golang](<https://devfeed.tech/tags/golang.md>), [graphql](<https://devfeed.tech/tags/graphql.md>), [latency](<https://devfeed.tech/tags/latency.md>), [load-balancer](<https://devfeed.tech/tags/load-balancer.md>), [memory](<https://devfeed.tech/tags/memory.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [programming](<https://devfeed.tech/tags/programming.md>), [server](<https://devfeed.tech/tags/server.md>), [twitch](<https://devfeed.tech/tags/twitch.md>)

### AI overview

A Twitch engineering article explains how a small Go memory-ballast change reduced CPU utilization on API frontend servers by about 30% and peak-load 99th-percentile API latency by about 45%. It describes the Visage gateway, traffic spikes, and the investigation behind the optimization.

### Source excerpt

Go memory ballast: How I learned to stop worrying and love the heap I'm a big fan of small code changes that can have large impact. This may seem like an obvious thing to state, but let me explain: These type of changes often involve diving into and understanding things one is not familiar with. Even with the most well factored code, there is a maintenance cost to each optimization you add, and it's usually (although not always) pretty linear with the amount of lines of code you end up adding/changing. We recently rolled out a small change that reduced the CPU utilization of our API frontend servers at Twitch by ~30% and reduced overall 99th percentile API latency during peak load by ~45%. This blog post is about the change, the process of finding it and explaining how it works. Setting the stage We have a service at Twitch called Visage, that functions as our API frontend. Visage is the central gateway for all externally originating API traffic. It is responsible for a bunch of things, from authorization to request routing, to (more recently) server-side GraphQL. As such, it has to scale to handle user traffic patterns that are somewhat out of our control. As an example, a common traffic pattern we see is a "refresh storm." This occurs when a popular broadcaster's stream drops due to a blip in their internet connectivity. In response, the broadcaster restarts the stream. This usually causes the viewers to repeatedly refresh their pages, and suddenly we have a lot more API traffic to deal with. Visage is a Go application (built with Go 1.11 at the time of this change) that runs on EC2 behind a load balancer. Being on EC2 it scales well horizontally, for the most part. However, even with the magic of EC2 and Auto Scaling groups, we still have the problem of dealing with very large traffic spikes. During refresh storms, we frequently have surges of millions of requests over a few seconds, on the order of 20x our normal load. On top of this, we would see API latency de

## How Twitch Addresses Scalability and Authentication

DevFeed: [How Twitch Addresses Scalability and Authentication](<https://devfeed.tech/articles/how-twitch-addresses-scalability-and-authentication-20451.md>)

Original publisher: [Read original article](<https://medium.com/twitch-news/how-twitch-addresses-scalability-and-authentication-718d6ed3c471?source=rss----3ae745429979--engineering>)

Author: Eugene Pivovarov

Published: 2019-03-15T19:13:30Z

Content type: article

Language: en

Sources: [Twitch](<https://devfeed.tech/sources/twitch.md>)

Topics: [Twitch](<https://devfeed.tech/topics/twitch.md>), [Authentication](<https://devfeed.tech/topics/authentication.md>), [OAuth 2.0](<https://devfeed.tech/topics/oauth2.md>), [Scalability](<https://devfeed.tech/topics/scalability.md>), [JSON Web Tokens](<https://devfeed.tech/topics/jwt.md>), [API](<https://devfeed.tech/topics/api.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [authentication](<https://devfeed.tech/tags/authentication.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [identity-management](<https://devfeed.tech/tags/identity-management.md>), [jwt](<https://devfeed.tech/tags/jwt.md>), [latency](<https://devfeed.tech/tags/latency.md>), [lazy-loading](<https://devfeed.tech/tags/lazy-loading.md>), [oauth-2-0](<https://devfeed.tech/tags/oauth-2-0.md>), [scalability](<https://devfeed.tech/tags/scalability.md>), [tls](<https://devfeed.tech/tags/tls.md>), [twitch](<https://devfeed.tech/tags/twitch.md>)

### AI overview

This article explains how Twitch's Identity team addressed authentication scalability and performance. It describes reducing login-form latency through prefetching and lazy loading, and reducing downstream validation workload by exchanging OAuth 2.0 access tokens for short-lived JWTs containing user information, scopes, and a digital signature.

### Source excerpt

Curious how Twitch Identity services addresses scalability and performance challenges related to authentication? Last Tuesday, the Twitch Identity team -- which helps Twitch users create and manage their digital identity and owns the services that provide authentication, authorization, and user metadata management -- hosted a Meetup.com event to explain just that. But we're recapping here just for you! First challenge: Login form load The goal of authentication is to verify who the user is. In a typical login flow, the user supplies a password along with a username and the service validates the credentials. Login user interface The Twitch Login form used to load very slowly, as long as 10 seconds in some Asian and Pacific countries. The browser had to make a DNS request for the authentication backend, initiate a TLS connection to authentication service, make a request to load the page, and then make additional requests for more assets. Altogether, there were five round trips and over 150 KB of data transfer. To improve load latency, the team considered a few options: prefetching, connection reuse, inlining, and lazy loading. Ultimately, the solution relies mostly on prefetching by making the login modal part of the main page, but it employs lazy load for fetching heavy assets. As a result, the median load latency dropped as much as 20x from 4.1 sec to 0.2 sec. Second challenge: User session validation At the end of the login flow, the browser obtains a long-lived OAuth 2.0 access token and stores it in a cookie. This token is required for subsequent requests to the Twitch API and the corresponding downstream services. If each service had to validate the OAuth token directly against the token dispensary, the latter would need to handle calls from multiple services for every API request. To reduce the workload, the Identity team implemented an approach that employs self-contained JSON Web Tokens (JWT). Login and request flow In this approach, before forwarding the reque

## Twitch for Android: From Meme to Dream

DevFeed: [Twitch for Android: From Meme to Dream](<https://devfeed.tech/articles/twitch-for-android-from-meme-to-dream-20456.md>)

Original publisher: [Read original article](<https://medium.com/twitch-news/twitch-for-android-from-meme-to-dream-141e6b7e8416?source=rss----3ae745429979--engineering>)

Author: Joaquim Verges

Published: 2019-02-26T21:45:39Z

Content type: article

Language: en

Sources: [Twitch](<https://devfeed.tech/sources/twitch.md>)

Topics: [Android](<https://devfeed.tech/topics/android.md>), [Mobile](<https://devfeed.tech/topics/mobile.md>), [test](<https://devfeed.tech/topics/test.md>), [unit test](<https://devfeed.tech/topics/unit-test.md>), [ui](<https://devfeed.tech/topics/ui.md>), [Code](<https://devfeed.tech/topics/code.md>)

Tags: [android](<https://devfeed.tech/tags/android.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [architecture-pattern](<https://devfeed.tech/tags/architecture-pattern.md>), [development](<https://devfeed.tech/tags/development.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [mobile](<https://devfeed.tech/tags/mobile.md>), [test](<https://devfeed.tech/tags/test.md>), [twitch](<https://devfeed.tech/tags/twitch.md>), [ui](<https://devfeed.tech/tags/ui.md>), [unit-test](<https://devfeed.tech/tags/unit-test.md>)

### AI overview

Twitch describes how its PogDroid Android app grew from a small app with basic streaming and chat functionality into a larger project. The team redesigned the app, rewrote much of it, introduced a common MVP architecture, and organized code into data, presentation, and view layers to make unit testing easier.

### Source excerpt

Meet PogDroid, our beloved Android app, to watch the latest memes unfold live while on the go. PogDroid started with very humble beginnings, and has seen incredible growth in the past couple of years. We quadrupled the team, redesigned the entire app, and rewrote most of it in only a few months using the latest and greatest Android has to offer. Buckle up: this is PogDroid's epic adventure. Level 1: Meme app with big dreams Back in March 2017, PogDroid was maintained by a very small team of engineers, and covered the most basic functionality of Twitch: browse and watch a stream with Chat. Old PogDroid, before the big makeover Mobile, and Android in particular, was getting more and more users, and the company decided it was time for a serious investment in PogDroid. The plan was to give the app a fresh new look, and build up some important features that were missing. The team grew only a little bit, but were tasked to take on this big project. As with most apps maintained by a very small team, the code structure was a mess. It became clear that such a big redesign compounded with adding new features could not be built cleanly on top of the current foundation. There was no common pattern to build screens, not a single unit test, and a lot of core classes were more than 3,000 lines of code long, with very complex state management, handling everything from network requests to UI rendering. We all agreed that in order to make PogDroid what we were dreaming of, we first needed to rethink its foundations. Step 1: Agree on a common architecture pattern The first thing we did is come up with a common design pattern to build features and screens. We wanted this design pattern to be: easy to understand hard to get wrong flexible enough to be applied to any feature or screen easy to unit test We quickly settled on a straightforward MVP pattern that looks like this: With these layers in mind, it became easy to categorize the type of classes we needed for every feature: Typical c

## How VP9 delivers value for Twitch's esports live streaming

DevFeed: [How VP9 delivers value for Twitch's esports live streaming](<https://devfeed.tech/articles/how-vp9-delivers-value-for-twitch-s-esports-live-streaming-20450.md>)

Original publisher: [Read original article](<https://medium.com/twitch-news/how-does-vp9-deliver-value-for-twitchs-esports-live-streaming-35db26f6322f?source=rss----3ae745429979--engineering>)

Author: Yueshi Shen

Published: 2018-12-19T21:12:17Z

Content type: article

Language: en

Sources: [Twitch](<https://devfeed.tech/sources/twitch.md>)

Topics: [Twitch](<https://devfeed.tech/topics/twitch.md>), [Compression](<https://devfeed.tech/topics/compression.md>), [fpga](<https://devfeed.tech/topics/fpga.md>), [Streaming](<https://devfeed.tech/topics/streaming.md>), [Hardware](<https://devfeed.tech/topics/hardware.md>), [browsers](<https://devfeed.tech/topics/browsers.md>), [Playback](<https://devfeed.tech/topics/playback.md>)

Tags: [browsers](<https://devfeed.tech/tags/browsers.md>), [compression](<https://devfeed.tech/tags/compression.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [encoding](<https://devfeed.tech/tags/encoding.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [fpga](<https://devfeed.tech/tags/fpga.md>), [h-264](<https://devfeed.tech/tags/h-264.md>), [live-streaming](<https://devfeed.tech/tags/live-streaming.md>), [streaming](<https://devfeed.tech/tags/streaming.md>), [twitch](<https://devfeed.tech/tags/twitch.md>), [videos](<https://devfeed.tech/tags/videos.md>), [vp9](<https://devfeed.tech/tags/vp9.md>)

### AI overview

This article explains how Twitch selected FPGA hardware for real-time VP9 encoding of premium esports and partner live streams. It discusses VP9's compression advantages and implementation challenges, and reports at least 25% bitrate savings compared with the highest-quality H.264 encoders used in Twitch's production.

### Source excerpt

By: Akrum Elkhazin, Video Algorithm Architect, NGCodec, akrum.elkhazin@ngcodec.com Avinash Ramachandran, Video Software Architect, NGCodec, avinash.ramachandran@ngcodec.com Roshan Baliga, Product Manager, Google, rdb@google.com Jai Krishnan, Product Manager, Google, jaikk@google.com Tarek Amara, Senior Video Specialist, Twitch, amatarek@twitch.tv Alex Converse, Senior Software Engineer, Twitch, alexconv@twitch.tv Yueshi Shen, Principal Research Engineer, Twitch, yshen@twitch.tv Summary: VP9 will soon help Twitch offer a better viewing experience for premium esports and partner channels Video compression is the key to successful delivery of digital video across various applications like broadcast, teleconference, surveillance, and online streaming services. Since 2003 (i.e., 15 years ago), H.264 has been the state-of-the-art video compression format and has enabled HDTV, Blu-ray Disc, Internet video websites (e.g., YouTube, Twitch), and so on. Nevertheless, according to Twitch's recent analysis, H.264 has reached its compression performance limit, particularly for real-time encoding of gaming content at the HD resolution (1080p60). On the other hand, newer-generation video standards, namely VP9, HEVC, and AV1, show significant compression gain, which can bring considerable commercial benefits to content platforms (e.g., offering viewers better video quality, reducing the video loading time and the buffering rate, increasing the customer reach, decreasing the IP transit cost). Currently, although decoding and playback of VP9 video are widely supported on devices and browsers used by Twitch's audience, encoding gaming video content with a high efficiency and real-time performance is a substantial challenge due to the high complexity of VP9. Through a rigorous feasibility study, we have eventually selected FPGA as the hardware platform for real-time VP9 encoding and are deploying it to broadcast our premium eSports and partner channels in the near future (please watch t

## Artisanal Objective-C Sum Types

DevFeed: [Artisanal Objective-C Sum Types](<https://devfeed.tech/articles/artisanal-objective-c-sum-types-20448.md>)

Original publisher: [Read original article](<https://medium.com/twitch-news/artisanal-objective-c-sum-types-8ea1ab9da342?source=rss----3ae745429979--engineering>)

Author: Heath Borders

Published: 2018-05-14T18:49:35Z

Content type: tutorial

Language: en

Sources: [Twitch](<https://devfeed.tech/sources/twitch.md>)

Topics: [Objective-C](<https://devfeed.tech/topics/objective-c.md>), [enum](<https://devfeed.tech/topics/enum.md>), [C](<https://devfeed.tech/topics/c.md>), [Swift](<https://devfeed.tech/topics/swift.md>)

Tags: [c](<https://devfeed.tech/tags/c.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [enum](<https://devfeed.tech/tags/enum.md>), [implementation](<https://devfeed.tech/tags/implementation.md>), [ios](<https://devfeed.tech/tags/ios.md>), [objective-c](<https://devfeed.tech/tags/objective-c.md>), [swift](<https://devfeed.tech/tags/swift.md>), [twitch](<https://devfeed.tech/tags/twitch.md>), [types](<https://devfeed.tech/tags/types.md>)

### AI overview

The article explains how to represent sum types in Objective-C. It contrasts Swift enums and C unions with enums, then presents a class-based, callback-driven approach for modeling distinct Objective-C object types under ARC.

### Source excerpt

A sum type combines many possible differently-typed values into a single value, expressed in Swift as an enum. https://medium.com/media/6217cfe231386e04ce2e7228080fae7a/href In C, we can achieve the power of Swift's enum with a combination of a C union and a C enum: https://medium.com/media/3f1696d38fcec2600a942ffbe03c8cd4/href First, I'm sure you notice Swift's enum is more concise, but it's also safer. In C, the compiler doesn't prevent us from either creating an example with a mismatched type or from consuming a union as a mismatched type: https://medium.com/media/82f0586deab20a07abc615a9e459028e/href Trying to mismatch our enum types in Swift won't compile. Yay! https://medium.com/media/03d37cc33364b48d851beb4ccf38a97d/href Unfortunately, in Objective-C, we can't use Objective-C objects in structs or unions in ARC, so we can't use our C sum type with Objective-C objects. However, we can build a similar construct by hand. First, we need a class to capture all of the distinct types, and expose them through a single block-based callback interface: https://medium.com/media/78216b190ad6501c26ad89d0864b1c5c/href We must declare forward references for our distinct types, which we'll define later. We mark new and init as NS_UNAVAILABLE so that consumers won't be able to instantiate our base class directly. If they use -Wobjc-designated-initializers, they won't be able to subclass our base class either (without importing ExamplePrivate.h, which we won't distribute to them). Next, we'll declare our distinct types: https://medium.com/media/329743de725ccd196c3f535306714cf9/hrefhttps://medium.com/media/1785598f614b3963a0a39ec81d8c5dac/href They're simply plain-old-objective-c-objects that extend our base class. Our base class and our distinct types all have switchFoo:bar:. The distinct types will simply call the respective callback block with self. The base class will have an empty implementation. We use NS_REQUIRES_SUPER on switchFoo:bar: because if we add another distinct

## Twirp: An RPC Framework for Go Focused on Simplicity

DevFeed: [Twirp: An RPC Framework for Go Focused on Simplicity](<https://devfeed.tech/articles/twirp-a-sweet-new-rpc-framework-for-go-20455.md>)

Original publisher: [Read original article](<https://medium.com/twitch-news/twirp-a-sweet-new-rpc-framework-for-go-5f2febbf35f?source=rss----3ae745429979--engineering>)

Author: Spencer Nelson

Published: 2018-01-16T20:19:57Z

Content type: release

Language: en

Sources: [Twitch](<https://devfeed.tech/sources/twitch.md>)

Topics: [Twirp](<https://devfeed.tech/topics/twirp.md>), [Remote Procedure Call (RPC)](<https://devfeed.tech/topics/rpc.md>), [Go Language](<https://devfeed.tech/topics/go-language.md>), [HTTP](<https://devfeed.tech/topics/http.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [Back end](<https://devfeed.tech/topics/backend.md>)

Tags: [backend](<https://devfeed.tech/tags/backend.md>), [developer](<https://devfeed.tech/tags/developer.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [go](<https://devfeed.tech/tags/go.md>), [golang](<https://devfeed.tech/tags/golang.md>), [http](<https://devfeed.tech/tags/http.md>), [new-features](<https://devfeed.tech/tags/new-features.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [rpc](<https://devfeed.tech/tags/rpc.md>), [twirp](<https://devfeed.tech/tags/twirp.md>), [twitch](<https://devfeed.tech/tags/twitch.md>)

### AI overview

Twitch releases Twirp, an open-source RPC framework for communication between backend servers written in Go. It uses structured RPCs over HTTP 1.1 and emphasizes simplicity, stability, and modularity compared with REST APIs and gRPC.

### Source excerpt

Twirp is a new RPC framework available under an Apache 2 open source license. Today Twitch is releasing an RPC framework we use for communication between backend servers written in Go. It's called Twirp, and it's available now under an Apache 2 open source license. Twirp has been tremendously successful at Twitch -- it has grown in usage exponentially, roughly tripling every three months as more and more internal teams continue to adopt it due to its advantages over "REST" APIs or gRPC, its two closest competitors. Structured RPCs are much easier to design and maintain than URL-oriented REST APIs, as they let you focus on business logic instead of routing schemes. Changing APIs to add new fields or methods is much easier and the peculiarities of serialization (like, say, JSON's lack of 64-bit numbers) can be hidden from view. gRPC implements structured RPCs but we found that its complexity and sprawl brought an unacceptable number of bugs -- and we had trouble justifying its hard http/2 requirement. Twirp is a structured RPC framework, but with an emphasis on simplicity. It works on HTTP 1.1, chooses stability and modularity over an expansive feature set, and then gets out of the way. Twirp is so simple, in fact, that you can make valid requests on the command line with cURL without much thought. It's as simple as a Content-Type header, a proper payload, and the right URL -- all of which are very well-standardized. So, for example: # This is a valid Twirp request: curl \ -header 'Content-Type:application/json' \ -data '{"user": "spencer", "email": "spencer@twitch.tv"}' \ http://localhost:9090/twirp/twitch.example.EmailBoss/UpdateEmail We think you'll want to use Twirp if you're working in a service-oriented architecture and program mostly in Go. We also think you'll be able to drop it into your system one service at a time, even if you're using several non-Go languages. The core design of Twirp is language-agnostic and we're planning to expand into new languages, but o

## Twitch invites you to take on the ICME 2018 Grand Challenge

DevFeed: [Twitch invites you to take on the ICME 2018 Grand Challenge](<https://devfeed.tech/articles/twitch-invites-you-to-take-on-the-icme-2018-grand-challenge-20457.md>)

Original publisher: [Read original article](<https://medium.com/twitch-news/twitch-invites-you-to-take-on-the-icme-2018-grand-challenge-2b3824d3537b?source=rss----3ae745429979--engineering>)

Author: Yueshi Shen

Published: 2017-12-01T17:56:37Z

Content type: article

Language: en

Sources: [Twitch](<https://devfeed.tech/sources/twitch.md>)

Topics: [Streaming](<https://devfeed.tech/topics/streaming.md>), [HTTP](<https://devfeed.tech/topics/http.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Playback](<https://devfeed.tech/topics/playback.md>), [Algorithms, Complexity](<https://devfeed.tech/topics/algorithms-complexity.md>)

Tags: [caching](<https://devfeed.tech/tags/caching.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [http](<https://devfeed.tech/tags/http.md>), [latency](<https://devfeed.tech/tags/latency.md>), [live-streaming](<https://devfeed.tech/tags/live-streaming.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [network](<https://devfeed.tech/tags/network.md>), [playback](<https://devfeed.tech/tags/playback.md>), [streaming](<https://devfeed.tech/tags/streaming.md>)

### AI overview

Twitch presents the ICME 2018 Grand Challenge, which seeks signal-processing and machine-learning algorithms to estimate download bandwidth for HTTP chunked video streaming. The article explains that chunked transfer can reduce live-streaming latency but produces noisy throughput measurements that complicate adaptive bitrate playback.

### Source excerpt

Because of its scalability and low cost, HTTP streaming is now widely used for live broadcasting on the open Internet. A number of platforms including Twitch, Twitter, and Amazon Video are now providing live video services based on HLS or DASH for both user-generated content (UGC) and professionally-generated content (PGC). However, in comparison to more traditional live streaming protocols such as RTMP, HLS and DASH incur much longer end-to-end broadcast latency. The latency is typically 10-30 seconds depending on implementations. On the other hand, low (less than 5 second) broadcast latency is highly demanded as it is critical to interactivity between a broadcaster and their audience. One of the most successful efforts to reduce the latency of HTTP streaming is to utilize the HTTP chunked transfer coding, which enables a video segment to be generated and transmitted concurrently. For more technical details, please read Twitter's recent technical blog "Introducing LHLS Media Streaming." However, compared with segment-based HTTP download, chunked transfer coding makes the bandwidth estimation a lot harder for any ABR playback algorithm. This Grand Challenge is to call for signal-processing/machine-learning algorithms that can effectively estimate download bandwidth based on the noisy samples of chunked-based download throughput. Problem Description: For segment-based HTTP streaming, the download bandwidth can simply be calculated as Bandwidth = segment size / segment download time This is, in general, quite accurate, since the size of a video segment is typically large enough (1 or more seconds of video data). On the other hand, in HTTP chunked transfer coding, a video segment is transmitted in multiple smaller pieces, i.e. the response to one HTTP request is broken into a series of chunks (see Figure 1). Figure 1: Use HTTP chunked transfer to transmit a video segment in multiple chunks from Video Source / HTTP Origin through CDN to Player As we can see from Figure

## Live Video Transmuxing/Transcoding: FFmpeg vs TwitchTranscoder, Part II

DevFeed: [Live Video Transmuxing/Transcoding: FFmpeg vs TwitchTranscoder, Part II](<https://devfeed.tech/articles/live-video-transmuxing-transcoding-ffmpeg-vs-twitchtranscoder-part-ii-20453.md>)

Original publisher: [Read original article](<https://medium.com/twitch-news/live-video-transmuxing-transcoding-ffmpeg-vs-twitchtranscoder-part-ii-4973f475f8a3?source=rss----3ae745429979--engineering>)

Author: Yueshi Shen

Published: 2017-10-23T21:08:55Z

Content type: article

Language: en

Sources: [Twitch](<https://devfeed.tech/sources/twitch.md>)

Topics: [FFmpeg (Fast Forward Moving Picture Experts Group)](<https://devfeed.tech/topics/ffmpeg.md>), [Transcodings](<https://devfeed.tech/topics/transcodings.md>), [Code](<https://devfeed.tech/topics/code.md>), [Concurrency](<https://devfeed.tech/topics/concurrency.md>), [C](<https://devfeed.tech/topics/c.md>)

Tags: [c](<https://devfeed.tech/tags/c.md>), [code](<https://devfeed.tech/tags/code.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [encoding](<https://devfeed.tech/tags/encoding.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [ffmpeg](<https://devfeed.tech/tags/ffmpeg.md>), [files](<https://devfeed.tech/tags/files.md>), [function](<https://devfeed.tech/tags/function.md>), [loops](<https://devfeed.tech/tags/loops.md>), [message-queue](<https://devfeed.tech/tags/message-queue.md>), [programming](<https://devfeed.tech/tags/programming.md>), [thread](<https://devfeed.tech/tags/thread.md>), [threading](<https://devfeed.tech/tags/threading.md>), [threads](<https://devfeed.tech/tags/threads.md>), [video](<https://devfeed.tech/tags/video.md>)

### AI overview

This second part of a two-part series examines FFmpeg Release 3.3's threading model and transcoding pipeline for a one-input, multiple-output scenario. It traces how FFmpeg orchestrates I/O, filtering, decoding, encoding, and output processing, and reports that the number of input files determines input-thread creation, so a one-input scenario uses a single thread.

### Source excerpt

By: Jeff Gong, Software Engineer, jeffgon@twitch.tv Sahil Dhanju, Software Engineer Intern Chih-Chiang Lu, Senior Software Engineer, chihchil@twitch.tv Yueshi Shen, Principal Research Engineer, yshen@twitch.tv Special thanks go to: Christopher Kennedy, Staff Video Engineer at Crunchyroll/Ellation John Nichols, Principal Software Engineer at Xilinx, jnichol@xilinx.com for their information on FFmpeg and reviewing this article. Note: This is the second part of a 2-part series. Make sure you read Part 1 first. FFmpeg's 1-In-N-Out Pipeline. Why doesn't it handle the technical issues discussed earlier? How does FFmpeg programmatically deal with instances where a single input stream is required to generate multiple transcoded and/or transmuxed outputs? We went directly into the latest FFmpeg Release 3.3. source code in order to understand its threading model and transcoding pipeline. In the top-level ffmpeg.c file, the transcode() function (line 4544) loops and repeatedly calls transcode_step() (line 4478) until its inputs are completely processed, or until the user interrupts the execution. Transcode_step() wraps the main pipeline and orchestrates file I/O, filtering, decoding and encoding amongst many other immediate steps. During the initial setup phase, init_input_threads() (line 4020) is called, and based on the number of input files, a number of new threads may be spawned to process the input. if (nb_input_files == 1) { return 0;}for (i = 0; i < nb_input_files; i++) { ... ret = av_thread_message_queue_alloc(&f->in_thread_queue, f->thread_queue_size, sizeof(AVPacket)); // line 4033} In line 4033, we see that the number of threads spawned is solely determined by the number of inputs. This means FFmpeg will process a 1-in-N-out scenario using only a single thread. In get_input_packet() (line 4055), the multithreaded companion function get_input_packet_mt() (line 4047) is only called if the number of input files is greater than one. get_input_packet_mt() can read input

## Live Video Transmuxing/Transcoding: FFmpeg vs TwitchTranscoder, Part I

DevFeed: [Live Video Transmuxing/Transcoding: FFmpeg vs TwitchTranscoder, Part I](<https://devfeed.tech/articles/live-video-transmuxing-transcoding-ffmpeg-vs-twitchtranscoder-part-i-20452.md>)

Original publisher: [Read original article](<https://medium.com/twitch-news/live-video-transmuxing-transcoding-ffmpeg-vs-twitchtranscoder-part-i-489c1c125f28?source=rss----3ae745429979--engineering>)

Author: Yueshi Shen

Published: 2017-10-10T18:40:03Z

Content type: article

Language: en

Sources: [Twitch](<https://devfeed.tech/sources/twitch.md>)

Topics: [FFmpeg (Fast Forward Moving Picture Experts Group)](<https://devfeed.tech/topics/ffmpeg.md>), [Twitch](<https://devfeed.tech/topics/twitch.md>), [Transcodings](<https://devfeed.tech/topics/transcodings.md>), [Streaming](<https://devfeed.tech/topics/streaming.md>), [HTTP](<https://devfeed.tech/topics/http.md>), [stream-processing](<https://devfeed.tech/topics/stream-processing.md>)

Tags: [architecture](<https://devfeed.tech/tags/architecture.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [ffmpeg](<https://devfeed.tech/tags/ffmpeg.md>), [http](<https://devfeed.tech/tags/http.md>), [live-streaming](<https://devfeed.tech/tags/live-streaming.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [stream-processing](<https://devfeed.tech/tags/stream-processing.md>), [streaming](<https://devfeed.tech/tags/streaming.md>), [transcoder](<https://devfeed.tech/tags/transcoder.md>), [twitch](<https://devfeed.tech/tags/twitch.md>), [video](<https://devfeed.tech/tags/video.md>)

### AI overview

This first part of a two-part series explains Twitch's live video processing pipeline and compares using FFmpeg with Twitch's in-house TwitchTranscoder. It describes how RTMP uploads are converted into multiple-bitrate HLS variants and outlines the live transcoding requirements that FFmpeg meets or does not meet.

### Source excerpt

By: Jeff Gong, Software Engineer, jeffgon@twitch.tv Sahil Dhanju, Software Engineer Intern Chih-Chiang Lu, Senior Software Engineer, chihchil@twitch.tv Yueshi Shen, Principal Research Engineer, yshen@twitch.tv Special thanks go to Christopher Kennedy, Staff Video Engineer at Crunchyroll/Ellation John Nichols, Principal Software Engineer at Xilinx, jnichol@xilinx.com for their information on FFmpeg and reviewing this article. Note: This is the first part of a 2-part series. Please read part 2 after finishing this article. Background Twitch is the world's leading live streaming platform for video games, esports, and other emerging creative content. Every month, more than 2.2 million unique content creators stream or upload video on our website. At its peak, Twitch ingests tens of thousands of concurrent live video streams and delivers them to viewers across the world. Figure 1 depicts the architecture of our live video Content Delivery Network (CDN) which delivers tens of thousands of concurrent live streams internationally. Figure 1: Architecture of Twitch's Live Video CDN Twitch, like many other live streaming services, receives live stream uploads in Real-Time Messaging Protocol (RTMP) from its broadcasters. RTMP is a protocol designed to stream video and audio on the Internet, and is mainly used for point to point communication. To then scale our live stream content to countless viewers, Twitch uses HTTP Live Streaming (HLS), an HTTP-based media streaming communications protocol that most video websites also use. Within the live stream processing pipeline, the transcoder module is in charge of converting an incoming RTMP stream into the HLS format with multiple variants (e.g., 1080p, 720p, etc.). These variants have different bitrates so that viewers with different levels of download bandwidth are able to consume live video streams at the best possible quality for their connection. Figure 2 depicts the input and output of the transcoder module in our live video CD