# Production Engineering Published articles for Production Engineering. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## MTIA 300: Meta's First Training Chip with Built-in NICs and Communication-Offloading Engines DevFeed: [MTIA 300: Meta's First Training Chip with Built-in NICs and Communication-Offloading Engines]() Original publisher: [Read original article]() Author: Rajiv Krishnamurthy; Wes Bland Published: 2026-08-24T17:45:52Z Content type: article Language: en Sources: [Engineering at Meta]() Topics: [recommendation systems](), [Language models]() Tags: [chip-design](), [communication](), [data-infrastructure](), [devinfra](), [gpu](), [hardware](), [inference](), [infrastructure](), [meta](), [networking-traffic](), [performance](), [production-engineering](), [training]() ### AI overview Meta describes MTIA 300, an in-house accelerator for training ranking and recommendation models, with built-in network chiplets and a co-designed HCCL communication library. The design targets communication-heavy distributed training by integrating RDMA NICs into the chip package and offloading communication work. ### Source excerpt MTIA 300 is the first of Meta's family of in-house training and inference accelerators optimized for training ranking and recommendation models. We're sharing how MTIA 300's built-in NIC chiplets allow it to meet the communication needs associated with training recommendation models with superior performance over general-purpose GPUs. By co-designing MTIA's communication library, HCCL, alongside the [...] Read More... The post MTIA 300: Meta's First Training Chip with Built-in NICs and Communication-Offloading Engines appeared first on Engineering at Meta. ## From User Sequences to Scaling Laws: A Multi-Stage Architecture for Meta's Ads Ranking DevFeed: [From User Sequences to Scaling Laws: A Multi-Stage Architecture for Meta's Ads Ranking]() Original publisher: [Read original article]() Author: Steven De Gryze; Parshva Doshi; Sean O'Byrne; Arnold Overwijk; Dinesh Ramasamy; Lee Xiong Published: 2026-08-05T19:20:20Z Content type: article Language: en Sources: [Engineering at Meta](), [Meta ML Applications]() Topics: [recommendation systems](), [Temporal data]() Tags: [ads](), [architecture](), [data-infrastructure](), [meta](), [ml-applications](), [production-engineering](), [recommendation-systems](), [scaling-laws](), [tokenization]() ### AI overview Meta describes a multi-stage sequence-model architecture for ads ranking that separates offline user modeling from lightweight online ranking. It also uses dense tokenization and target-aware attention to learn feature interactions, with reported conversion and ad-click lifts across Instagram and Facebook. ### Source excerpt Every day, Meta's recommendation platforms handle billions of user interactions, generating rich temporal signals that capture individual preferences and intent across products, ads, and content. In our 2024 post on sequence learning for ads recommendations, we showed how modeling the order and timing of user actions (rather than relying on static, manually engineered sparse features) [...] Read More... The post From User Sequences to Scaling Laws: A Multi-Stage Architecture for Meta's Ads Ranking appeared first on Engineering at Meta. ## 10 Years of Meta's Commitment to Python DevFeed: [10 Years of Meta's Commitment to Python]() Original publisher: [Read original article]() Author: Chris Wiltz Published: 2026-06-30T16:00:46Z Content type: opinion Language: en Sources: [Meta AI Research](), [Meta ML Applications]() Topics: [Meta](), [Python](), [Open Source](), [Programming language](), [Software](), [PyTorch]() Tags: [ai-research](), [culture](), [data-infrastructure](), [devinfra](), [engineering](), [meta](), [ml-applications](), [open-source](), [production-engineering](), [programming](), [python](), [pytorch](), [software]() ### AI overview Meta reflects on its 10th consecutive year sponsoring the Python Software Foundation and explains Python's importance across its engineering stack, including products, infrastructure, and AI research. ### Source excerpt This year marks Meta's 10th consecutive year as a sponsor of the Python Software Foundation (PSF), the charitable organization dedicated to advancing, supporting, and protecting the open-source Python programming language and the community that sustains it. Python is one of the world's most influential programming languages, and we use it across our engineering stack, from [...] Read More... The post 10 Years of Meta's Commitment to Python appeared first on Engineering at Meta. ## Cisco and OpenAI redefine enterprise engineering with Codex DevFeed: [Cisco and OpenAI redefine enterprise engineering with Codex]() Original publisher: [Read original article]() Published: 2026-05-27T11:00:00Z Content type: article Language: en Sources: [OpenAI News]() Topics: [AI-assisted coding](), [C++]() Tags: [ai-engineering](), [ai-security](), [c-plus-plus](), [cli](), [codex](), [compliance](), [developer-productivity](), [enterprise](), [open-source](), [openai](), [production-engineering]() ### AI overview Cisco describes using Codex across enterprise engineering workflows to build AI Defense, accelerate feature delivery, and increase defect-resolution throughput. ### Source excerpt Cisco and OpenAI are redefining enterprise engineering with Codex, helping Cisco scale AI-native development, accelerate AI Defense work, and automate defect remediation. ## Meta's Infrastructure Evolution and the Advent of AI DevFeed: [Meta's Infrastructure Evolution and the Advent of AI]() Original publisher: [Read original article]() Author: Yee Jiun Song; Kaushik Veeraraghavan Published: 2025-09-29T13:00:15Z Content type: article Language: en Sources: [Meta AI Research]() Topics: [Meta](), [Artificial Intelligence](), [Hardware](), [Network](), [cloud-infrastructure](), [Terraform]() Tags: [ai](), [ai-research](), [apache](), [data-center-engineering](), [data-centers](), [data-infrastructure](), [data-management](), [database](), [devinfra](), [hardware](), [infrastructure](), [lamp](), [linux](), [meta](), [ml-applications](), [mysql](), [network](), [networking-traffic](), [open-source](), [production-engineering](), [scale]() ### AI overview Meta describes how its infrastructure evolved from a small university-focused social network into a globally networked operation serving more than 3.4 billion people. The article explains that AI has changed infrastructure-scaling assumptions and requires innovation across hardware, software, networks, and data centers, while outlining earlier database, caching, social graph, ranking, and photo-service scaling work. ### Source excerpt Over the past 21 years, Meta has grown exponentially from a small social network connecting a few thousand people in a handful of universities in the U.S. into several apps and novel hardware products that serve over 3.4 billion people throughout the world. Our infrastructure has evolved significantly over the years, growing from a [...] Read More... The post Meta's Infrastructure Evolution and the Advent of AI appeared first on Engineering at Meta. ## What I Learned in One Year as an SRE Trainee DevFeed: [What I Learned in One Year as an SRE Trainee]() Original publisher: [Read original article]() Published: 2023-01-06T00:00:00Z Content type: article Language: en Sources: [SoundCloud Backstage Blog]() Topics: [SRE](), [Production Engineering](), [Infrastructure as code](), [Monitoring](), [Incident response](), [Kubernetes](), [migration](), [datacenter](), [Cloud](), [Google]() Tags: [cloud](), [data-center](), [engineering-culture](), [google-cloud](), [incident-response](), [infrastructure-as-code](), [knowledge-sharing](), [kubernetes](), [learning](), [migration](), [monitoring](), [music](), [production-engineering](), [site-reliability-engineering](), [sre]() ### AI overview A Site Reliability Engineering trainee at SoundCloud reflects on a first year spent learning how SRE applies software engineering to operations, reliability, and scalability. The article discusses the broad scope of the role, including Infrastructure as Code, Monitoring, Incident Response training, Kubernetes upgrades, infrastructure decommissioning, and a service migration from a data center to Google Cloud. ### Source excerpt I recently celebrated my one year anniversary as a Site Reliability Engineering (SRE) trainee at SoundCloud. Looking back, I had very little... ## Lessons from Leading a Remote Engineering Team DevFeed: [Lessons from Leading a Remote Engineering Team]() Original publisher: [Read original article]() Author: Lawrence Mandel Published: 2019-07-25T18:00:00Z Content type: opinion Language: en Sources: [Shopify Engineering](), [Shopify Engineering - Shopify Engineering]() Topics: [Production Engineering](), [Shopify]() Tags: [canada](), [career](), [engineering-management](), [global](), [health](), [mobility](), [production-engineering](), [remote](), [uk](), [work]() ### AI overview A Shopify engineering leader shares lessons from managing remote teams, including the benefits of a global hiring pool, a more diverse workforce, stronger employee retention, and location-based team capabilities. ### Source excerpt Shopify's Director of Production Engineering talks about the benefits of remote teams, building a remote team, and setting your people up to succeed. ## Preparing Shopify for Black Friday and Cyber Monday DevFeed: [Preparing Shopify for Black Friday and Cyber Monday]() Original publisher: [Read original article]() Author: Camilo Lopez Published: 2018-12-21T19:00:00Z Content type: article Language: en Sources: [Shopify Engineering](), [Shopify Engineering - Shopify Engineering]() Topics: [Production Engineering](), [Shopify](), [Chaos Engineering](), [Latency](), [Testing](), [systems]() Tags: [engineering](), [latency](), [load](), [performance](), [platform](), [production](), [production-engineering](), [shopify](), [systems](), [testing]() ### AI overview Shopify's Production Engineering teams prepare the platform for Black Friday and Cyber Monday by maintaining reliability and scalability for heavy traffic and sales volumes. Their preparations include game days that use controlled fault injection, artificial latency, dependency degradation, and fallback verification, along with synthetic load testing to evaluate applications and system components under expected demand. ### Source excerpt Making commerce better for everyone is a challenge we face on a daily basis. For our Production Engineering team, it means ensuring that our 600,000+ merchants have a reliable and scalable platform to support their business needs. We need to be able to support everything our merchants throw at us--including the influx of holiday traffic during Black Friday and Cyber Monday (BFCM). All of this needs to happen without an interruption in service. ## Director of Engineering, Lawrence Mandel Talks Road to Leadership, Growth, and Finding Balance. DevFeed: [Director of Engineering, Lawrence Mandel Talks Road to Leadership, Growth, and Finding Balance.]() Original publisher: [Read original article]() Author: Toni Akinwumi Published: 2018-12-03T21:12:00Z Content type: article Language: en Sources: [Shopify Engineering](), [Shopify Engineering - Shopify Engineering]() Topics: [Shopify](), [Production Engineering](), [Swing](), [MediaWiki](), [Open Source](), [Automation](), [Scripting, bash](), [web applications](), [XML](), [ibm](), [Mozilla]() Tags: [apache](), [automation](), [developer](), [documentation](), [eclipse](), [engineering](), [growth](), [ibm](), [java](), [leadership](), [mozilla](), [open-source](), [production-engineering](), [projects](), [shopify](), [software](), [software-developer](), [tools](), [web-applications](), [work]() ### AI overview This interview profiles Lawrence Mandel, a Director of Production Engineering at Shopify who leads the Developer Acceleration team. It discusses the team's work building tools, conventions, documentation, and automation for developers; Mandel's experience at IBM and Mozilla; his open-source projects and Java web application work; and his views on career growth, leadership, time management, family, health, and work. ### Source excerpt Lawrence Mandel is a Director of Production Engineering leading Shopify's Developer Acceleration team and has been at Shopify for over a year. He previously worked at IBM and Mozilla where he started as a software developer before transitioning into leadership roles. Through all his work experience, he's learned to understand the meaning of time management and to prioritize the most important things in his life, which are his family, health, and work. ## Iterating Towards a More Scalable Ingress DevFeed: [Iterating Towards a More Scalable Ingress]() Original publisher: [Read original article]() Author: Francisco Mejia Published: 2018-08-31T18:00:00Z Content type: article Language: en Sources: [Shopify Engineering](), [Shopify Engineering - Shopify Engineering]() Topics: [Shopify](), [Kubernetes](), [nginx](), [Cloud](), [Production Engineering]() Tags: [ingress-nginx](), [kubernetes](), [load-balancer](), [nginx](), [open-source](), [optimization](), [performance](), [production-engineering](), [shopify]() ### AI overview Shopify describes scaling its Kubernetes infrastructure and migrating cluster ingress from Google Cloud Load Balancer Controller to ingress-nginx. The article focuses on optimizing ingress-nginx dynamic configuration to reduce NGINX reloads and improve performance during frequent deployments. ### Source excerpt Shopify, the leading cloud-based, multi-channel commerce platform, is growing at an incredibly fast pace. Since the beginning of 2016, the number of merchants on the platform increased from 375,000 to 600,000+. As the platform scales, we face new and exciting challenges such as implementing Shopify's Pod architecture and future proofing our cloud storage usage. Shopify's infrastructure relies heavily on Kubernetes to serve millions of requests every minute. ## E-Commerce at Scale: Inside Shopify's Tech Stack - Stackshare.io DevFeed: [E-Commerce at Scale: Inside Shopify's Tech Stack - Stackshare.io]() Original publisher: [Read original article]() Author: Kir Shatrov Published: 2018-08-08T15:30:00Z Content type: article Language: en Sources: [Shopify Engineering](), [Shopify Engineering - Shopify Engineering]() Topics: [Shopify](), [Production Engineering](), [Rails](), [Scalability](), [Ruby](), [Monitoring](), [Architecture & Design](), [incident]() Tags: [incident](), [infrastructure](), [monitoring](), [production](), [production-engineering](), [ruby](), [scalability](), [shopify](), [tech-stack]() ### AI overview The article examines how Shopify operates a large-scale, multi-channel commerce platform. It describes the Production Engineering department, shared responsibility for monitoring and incident response, scalability practices for unpredictable flash-sale traffic, and Shopify's long-lived Ruby on Rails codebase. ### Source excerpt 9 minute read Before 2015, we had an Operations and Performance team. Around this time, we decided to create the Production Engineering department and merge the teams. The department is responsible for building and maintaining common infrastructure that allows the rest of product development teams to run their code. Both Production Engineering and all the product development teams share responsibility for the ongoing operation of our end user applications. This means all technical roles share monitoring and incident response, with escalation happening laterally to bring in any skill set required to restore service in case of problems. ## Shopify's Infrastructure Collaboration with Google DevFeed: [Shopify's Infrastructure Collaboration with Google]() Original publisher: [Read original article]() Author: Dale Neufeld Published: 2018-03-26T12:00:00Z Content type: article Language: en Sources: [Shopify Engineering](), [Shopify Engineering - Shopify Engineering]() Topics: [Google](), [Kubernetes](), [Database]() Tags: [cloud](), [collaboration](), [docker](), [google](), [infrastructure](), [kubernetes](), [production-engineering](), [scalability](), [shopify]() ### AI overview Shopify describes its infrastructure evolution and its collaboration with Google to build its cloud. The article connects this move to scaling a Rails application, containerizing workloads with Docker, and the arrival of Google Kubernetes Engine. ### Source excerpt We're always working to deliver the best commerce experience to our merchants and their customers. We provide a seamless merchant experience while shaping the future of retail by building a platform that can handle the traffic of a Kylie Cosmetic flash sale (they sell out in 20 seconds), ship ne ## Tell Your Stories: The Benefits of Strategic Engineering Communications DevFeed: [Tell Your Stories: The Benefits of Strategic Engineering Communications]() Original publisher: [Read original article]() Author: Jaime Woo Published: 2017-10-25T19:30:00Z Content type: article Language: en Sources: [Shopify Engineering](), [Shopify Engineering - Shopify Engineering]() Topics: [Shopify](), [Production Engineering](), [front-end-development](), [data-engineering](), [GraphQL](), [Rails]() Tags: [articles](), [data-engineering](), [developers](), [development](), [engineering](), [front-end-development](), [production-engineering](), [shopify](), [technical]() ### AI overview Shopify established a formal engineering communications function as the company grew across offices and technical teams. The initiative shared engineering knowledge internally, communicated Shopify's work externally, supported recruitment, reduced duplicated work and mistakes, and encouraged cross-team idea sharing through technical talks, newsletters, and approachable articles. ### Source excerpt In early 2016, we faced a problem at Shopify. We were growing quickly, and decisions could no longer be made across the room, so to speak. Four offices became five, accommodating that growth raised interesting questions like: how would new people know the history of the company, and how could existing Shopifolk keep up with new developments? In addition to sharing knowledge inside the company, we also wanted to let people outside Shopify know what we were working on to give back to the community and to support recruitment efforts. Engineering communications was born to solve a specific problem. A valued saying here is "do things, tell people," but, while we're very good at the first part, we weren't living up to expectations on the second. Ad hoc worked when we were smaller, but with technical stories now coming from teams as varied as production engineering, mobile, front-end development, and data engineering, we needed something more formalized. Strong communications inside the engineering team could help prevent the overlap of work by different teams or the duplication of mistakes, and it could support cross-pollination of ideas. ## Why Shopify Moved to The Production Engineering Model DevFeed: [Why Shopify Moved to The Production Engineering Model]() Original publisher: [Read original article]() Author: John Arthorne Published: 2016-10-30T08:14:00Z Content type: article Language: en Sources: [Shopify Engineering](), [Shopify Engineering - Shopify Engineering]() Topics: [web applications](), [networking]() Tags: [automation](), [devops](), [production-engineering](), [shopify](), [sre]() ### AI overview Shopify describes moving from separate development and operations teams to a Production Engineering model for frequently changing large-scale web applications. The model emphasizes automation, fewer disconnected operational teams, and reusable tools and infrastructure for developers. ### Source excerpt 6 minute read The traditional model of running large-scale computer systems divides work into Development and Operations as distinct and separate teams. This split works reasonably well for computer systems that are changed or updated very rarely, and organizations sometimes require this if they're deploying and operating software built by a different company or organization. However, this rigid divide fails for large-scale web applications that are undergoing frequent or even continuous change. DevOps is the term for a movement that's gathered steam in the past decade to bring together these disciplines. ## Replacing Google Authenticator with Yubikey on OS X DevFeed: [Replacing Google Authenticator with Yubikey on OS X]() Original publisher: [Read original article]() Author: Square Engineering Published: 2013-05-28T16:03:00Z Content type: tutorial Language: en Sources: [Square Corner Blog RSS Feed]() Topics: [Authentication](), [MFA](), [ssh](), [Homebrew](), [Script](), [Production Engineering]() Tags: [2](), [authentication](), [bastion](), [code](), [engineering](), [github](), [google](), [install](), [make](), [production-engineering](), [script](), [ssh]() ### AI overview A tutorial explains how to configure a Yubikey to generate TOTP codes compatible with Google Authenticator for SSH two-factor authentication on OS X. It also describes using AppleScript and Automator to enter the current code with a global shortcut. ### Source excerpt Two-factor authentication made even easier.