# Transformer A neural-network architecture based solely on attention mechanisms, dispensing with recurrence and convolutions. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Where does all the VRAM go during LLM inference? DevFeed: [Where does all the VRAM go during LLM inference?]() Original publisher: [Read original article]() Author: Avi Chawla Published: 2026-09-17T19:30:12Z Content type: article Language: en Sources: [Daily Dose of Data Science]() Topics: [Inference](), [Large Language Model](), [GPU](), [quantization](), [Transformer](), [Decoding]() Tags: [concurrency](), [decoding](), [gpu](), [inference](), [llm](), [model-architecture](), [quantization](), [vram]() ### AI overview This article explains how GPU memory is allocated during large language model inference. It distinguishes mostly fixed model-weight memory from dynamic memory used by the KV cache, temporary activations and workspace, and runtime overhead. Context length, batch size, concurrency and model architecture affect whether the workload fits, while quantization reduces weight memory but does not guarantee higher throughput. ### Source excerpt ...explained visually ## The guest journey, updated in real time: extending Airbnb's sequence recommender with Chronon DevFeed: [The guest journey, updated in real time: extending Airbnb's sequence recommender with Chronon]() Original publisher: [Read original article]() Author: Pengyu Hou Published: 2026-09-17T17:01:02Z Content type: article Language: en Sources: [The Airbnb Tech Blog - Medium]() Topics: [real-time](), [recommendation systems](), [data](), [Transformer](), [Inference](), [Latency](), [Open Source]() Tags: [ai](), [data](), [embeddings](), [engineering](), [inference](), [infrastructure](), [latency](), [machine-learning](), [open-source](), [ranking](), [real-time](), [repo](), [results](), [sequence](), [technology]() ### AI overview Airbnb describes extending its sequence-based recommender with Chronon's Push Mode and Near-real-time Model Transform capabilities. The changes update guest activity and model features more quickly, reducing the staleness of search-ranking inputs compared with the previous daily batch pipeline. ### Source excerpt How two new Chronon capabilities, Push Mode and NRT Model Transform, allows us to provide more relevant search results instantly as a guest explores, rather than waiting for the next batch run. By: Pengyu Hou, Yuli Han, Daochen Zha, Haozhen Ding, Xin Liu, Sophie Wang, Pallavi Adusumilli, Sherry Li, Henry Saputra, Chun How Tan, Huiji Gao, Yan Zhang, Stephanie Moyerman, Yi Li, and Sanjeev Katariya A guest's interaction with Airbnb doesn't pause to wait for a nightly batch job. Someone might browse a dozen listings on a Tuesday afternoon, run a new search that evening, and expect the next search to reflect the recent activity; it's also to Airbnb's benefit for that to be the case. In our previous post, Personalizing Airbnb search by learning from the guest journey, we described how we built a Transformer-based sequence encoder that creates better, more personalized search rankings for a guest using the booking, review, and browsing data that is most relevant to them -- their own. That system ran as a daily batch job: each night it processed the previous day's activity and refreshed embeddings for guests who had something new to show for it. That design worked well, but it left a gap. Activity from earlier the same day wouldn't show up in the embedding until the following day's run, on top of the pipeline's own processing lag -- in practice, up to nearly two days of staleness. For a guest actively planning a trip, that meant the ranking model was often working from a slightly outdated picture of what they wanted, and the recent activities are often highly relevant to current search needs. This is a limitation that our original JourneyFormer research had already flagged as needing new serving infrastructure to solve. In this post, we describe how we closed that gap by adding two new capabilities to Chronon, Airbnb's feature platform: Near-real-time Model Transform and Push Mode. Chronon is an open source project, and these capabilities have been contributed back to our pub ## HeyGen x Google Cloud: Bringing Avatar IV to TPUs DevFeed: [HeyGen x Google Cloud: Bringing Avatar IV to TPUs]() Original publisher: [Read original article]() Published: 2026-09-12T11:04:33.891311Z Content type: article Language: en Sources: [Google Developers Blog]() Topics: [Google](), [Cloud](), [Artificial Intelligence](), [Optimization](), [AI Infrastructure](), [Transformer](), [PyTorch](), [Compiler](), [Streaming](), [real-time](), [Hardware](), [API]() Tags: [ai](), [ai-infrastructure](), [api](), [cloud](), [code](), [compiler](), [engineering](), [generation](), [google](), [hardware](), [model](), [optimization](), [performance](), [pytorch](), [real-time](), [streaming](), [time](), [tpu](), [transformers](), [video]() ### AI overview HeyGen and Google Cloud describe porting the 18B+ parameter Avatar IV talking-head video generation pipeline to an eight-chip Trillium TPU host. Using torchax, JAX, XLA, FSDP sharding, Ulysses sequence parallelism, and custom Pallas kernels, the team improved performance by 1.86x for real-time chunked streaming while preserving output quality through strict quality gates. ### Source excerpt HeyGen ported their 18B+ parameter Avatar IV video generation model to Google Cloud's Trillium (v6e) TPUs via torchax and XLA, utilizing FSDP and Ulysses sequence parallelism across an eight-chip mesh. To achieve a 1.86x speedup for real-time streaming, the engineering team pipelined exposed all-to-all collectives, aligned sparse attention block sizes to eliminate mask padding, and bypassed softmax serial dependencies using a precomputed Cauchy-Schwarz upper bound. These custom Pallas kernel and compiler optimizations were deployed only after passing rigorous two-tier quality gates to guarantee byte-identical or mathematically equivalent pixel outputs. ## Amlogic A123X and C305X2 Arm Cortex-A320 SoCs target industrial and low-power AIoT applications DevFeed: [Amlogic A123X and C305X2 Arm Cortex-A320 SoCs target industrial and low-power AIoT applications]() Original publisher: [Read original article]() Author: Jean-Luc Aufranc (CNXSoft) Published: 2026-09-11T03:13:41Z Content type: news Language: en Sources: [CNX Software - Embedded Systems News]() Topics: [Arm](), [cpu](), [Embedded Systems](), [Artificial Intelligence](), [Internet of things](), [Hardware](), [object-detection](), [Neural Network](), [Transformer]() Tags: [aiot](), [amlogic](), [arm](), [armv9](), [camera](), [computer-vision](), [cortex-a320](), [edge-ai](), [embedded](), [ethernet](), [h-264](), [hardware](), [iot](), [linux](), [llm](), [low-power](), [neural](), [npu](), [object-detection](), [processor](), [security](), [soc]() ### AI overview Amlogic has announced the A123X quad-core and C305X2 dual-core Arm Cortex-A320 SoCs for industrial and battery-powered edge AI and IoT devices. The preliminary specifications include video encoding and decoding, NPUs, image signal processing, camera interfaces, networking, USB, and low-power features. The article notes that full specifications, block diagrams, and software details are not yet available. ### Source excerpt Amlogic has unveiled the A123X quad-core and C305X2 dual-core Arm Cortex-A320 SoCs for industrial and battery-powered Edge AI and IoT applications such as robots, dashcams, IP cameras, video conferencing equipment, and so on. The Arm Cortex-A320 low-power Armv9 CPU core was introduced in February 2025, and Amlogic is the first silicon vendor to announce Cortex-A320 SoCs. Details are sparse, with no full specifications or block diagrams and limited software information, but let's see what we know so far. Amlogic A123X Amlogic A123X specifications: CPU - Quad-core Arm Cortex-A320 processor (Armv9.2-A, SVE2) GPU - None or not disclosed VPU H.264/H.265 encoding at 4K @ 60fps H.264/H.265 decoding at 4K @ 30fps AI 4 TOPS ADLA2 NPU for object detection and tracking CNN models 8 TOPS ADLA3 NPU supporting hardware-accelerated Transformer operations for ViT, LLM, etc. Neural network-based hardware SED engine for low-power audio event identification ISP - Low-light HDR ISP Supports [...] The post Amlogic A123X and C305X2 Arm Cortex-A320 SoCs target industrial and low-power AIoT applications appeared first on CNX Software - Embedded Systems News. ## SimpleDesign: A Joint Model for Protein Sequence and Structure Codesign DevFeed: [SimpleDesign: A Joint Model for Protein Sequence and Structure Codesign]() Original publisher: [Read original article]() Published: 2026-09-11T00:00:00Z Content type: article Language: en Sources: [Apple Machine Learning Research]() Topics: [multimodal](), [Transformer](), [datasets]() Tags: [benchmarks](), [drug-discovery](), [generation](), [model](), [multimodal](), [training]() ### AI overview SimpleDesign is a single-stage, end-to-end multimodal generative model for jointly designing protein sequences and three-dimensional structures. It uses Transformer-based multimodal backbones, trains directly in data space on more than 2 million sequence-structure pairs, and achieves competitive results on co-design and unconditional generation benchmarks. ### Source excerpt Proteins are fundamental to biological processes, with their function determined by the complex interplay between the amino acid sequence and the three-dimensional structure. Developing generative models capable of understanding this intrinsically multi-modal relationship is crucial for fields like drug discovery and protein engineering. Existing models often rely on a multi-stage training process where autoencoders that tokenize data into latent representations are trained in a first stage. Secondly, a generative model is trained on the latent representation of the autoencoder(s), i.e... ## How we built a benchmarking framework to horizontally accelerate transaction model research DevFeed: [How we built a benchmarking framework to horizontally accelerate transaction model research]() Original publisher: [Read original article]() Author: Nubank Editorial Published: 2026-09-03T13:53:30Z Content type: article Language: en Sources: [Nubank]() Topics: [benchmarking](), [Benchmark](), [Transformer](), [scaling](), [data](), [experiments]() Tags: [batch](), [benchmarking](), [data-science-machine-learning](), [distributed](), [evaluation](), [performance](), [research](), [training](), [workflow]() ### AI overview Nubank describes a benchmarking framework for horizontally evaluating transformer-based transaction representation models across more than 20 benchmarks. The automated, reproducible workflow tests data, architecture, and training changes across multiple tasks and trials, helping identify improvements that generalize. It increased the team's experimentation capacity to roughly five times more experiments per month. ### Source excerpt The framework that transformed weeks of manual experimentation into automated pipelines for horizontal transaction model research The post How we built a benchmarking framework to horizontally accelerate transaction model research appeared first on Building Nubank. ## How we built a benchmarking framework to horizontally accelerate transaction model research DevFeed: [How we built a benchmarking framework to horizontally accelerate transaction model research]() Original publisher: [Read original article]() Author: Nubank Editorial Published: 2026-09-03T13:53:30Z Content type: article Language: en Sources: [Nubank]() Topics: [benchmarking](), [Framework](), [Transformer](), [data]() Tags: [architectures](), [automated](), [benchmarking](), [data](), [data-science-machine-learning](), [experimentation](), [framework](), [model](), [performance](), [pipelines](), [training]() ### AI overview Nubank built an automated benchmarking framework for horizontally evaluating transformer-based transaction representation models across multiple downstream tasks and trials. The framework made experimentation reproducible and statistically rigorous, helping the team identify improvements that generalize across applications. It increased the team's capacity to run experiments by roughly five times per month while reducing operational overhead. ### Source excerpt The framework that transformed weeks of manual experimentation into automated pipelines for horizontal transaction model research The post How we built a benchmarking framework to horizontally accelerate transaction model research appeared first on Building Nubank. ## Backblaze Drive Stats: How an Open Dataset Powers Academic and AI/ML Research DevFeed: [Backblaze Drive Stats: How an Open Dataset Powers Academic and AI/ML Research]() Original publisher: [Read original article]() Author: Stephanie Doyle Published: 2026-08-13T15:08:38Z Content type: article Language: en Sources: [Backblaze Blog | Cloud Storage & Cloud Backup]() Topics: [dataset](), [DRIVE](), [Machine learning](), [Transformer](), [Disk image]() Tags: [ai-ml](), [articles](), [b2cloud](), [cloud-storage](), [data](), [drive](), [featured](), [featured-cloud-storage](), [hard-drive-stats](), [ml](), [research](), [source](), [storage]() ### AI overview The article explains how Backblaze Drive Stats evolved from an internal hard-drive reliability tool into an open dataset used in academic and AI/ML research. It describes the dataset's real-world scale, quarterly publication, SMART attributes, labeled failures, broad manufacturer coverage, and use in hard-drive failure prediction research. ### Source excerpt Backblaze Drive Stats has been cited in more than 105 academic papers and AI/ML projects since 2018. Explore the research it powers and download the dataset. The post Backblaze Drive Stats: How an Open Dataset Powers Academic and AI/ML Research appeared first on Backblaze Blog | Cloud Storage & Cloud Backup ## Inside NVIDIA Rubin GPU Architecture: Powering the Era of Agentic AI DevFeed: [Inside NVIDIA Rubin GPU Architecture: Powering the Era of Agentic AI]() Original publisher: [Read original article]() Author: Eduardo Alvarez Published: 2026-07-21T18:15:00Z Content type: article Language: en Sources: [NVIDIA Developer](), [NVIDIA Technical Blog]() Topics: [GPU](), [Nvidia](), [Artificial Intelligence](), [Inference](), [Inference Performance](), [long-context](), [datacenter](), [Transformer](), [Deployment]() Tags: [accelerate](), [agentic](), [agentic-ai](), [agentic-ai-generative-ai](), [ai](), [ai-agent](), [ai-factory](), [ai-inference](), [architecture](), [blackwell](), [cache](), [compute](), [data-center](), [data-center-cloud](), [developer-tools-techniques](), [dsx](), [efficiency](), [featured](), [gpu](), [inference](), [inference-performance](), [latency](), [long-context](), [memory](), [mixture-of-experts-moe](), [nvfp4](), [nvidia](), [rubin-gpu](), [scale](), [tensor-cores](), [top-stories](), [training](), [vera-rubin]() ### AI overview This article examines the NVIDIA Rubin GPU architecture and its co-designed Vera Rubin platform for agentic AI inference. It describes how Tensor Cores, HBM4 memory, the Transformer Engine, NVFP4 performance, cache, decoding, and scale-up systems address throughput, latency, long-context execution, and rack-scale deployment. ### Source excerpt What began as discrete AI model training and human-facing chat interfaces has evolved into always-on AI factories dedicated to producing intelligence at scale.... ## Personalizing Airbnb search by learning from the guest journey DevFeed: [Personalizing Airbnb search by learning from the guest journey]() Original publisher: [Read original article]() Author: Daochen Zha Published: 2026-07-21T17:01:04Z Content type: article Language: en Sources: [The Airbnb Tech Blog - Medium]() Topics: [Transformer](), [recommendation systems](), [Temporal data](), [Data Science]() Tags: [ai](), [blog](), [conversion](), [engineering](), [machine-learning](), [model](), [recommendation-system](), [research](), [scale](), [search](), [technology](), [training]() ### AI overview Airbnb describes a Transformer-based sequence model for personalizing search by encoding years of guest behavior, including listing views, bookings, reviews, and cancellations. The system learns richer representations of guest preferences to improve listing relevance and booking conversion while addressing very long, noisy event sequences and the cost of training on hundreds of millions of search-label pairs. ### Source excerpt How we built a Transformer-based sequence model that encodes years of guest behavior to surface the right listings at the right time. By: Daochen Zha, Chun How Tan, Xin Liu, Bin Xu, Han Zhao, Xiaowei Liu, Jun Shi, Tracy Yu, Hui Gao, Huiji Gao, Liwei He, Michael Kinoti, Stephanie Moyerman, and Sanjeev Katariya Introduction Planning a trip on Airbnb rarely happens in a single session. A guest searching for a place to stay in San Francisco might browse dozens of listings over several days, leaving behind a trail of views. Typically, over a period of years, that same guest will have accumulated many previous bookings, reviews, and the occasional cancellation. Taken together, these events reveal a great deal about what that guest values in a stay. For years, Airbnb's search ranking captured this through hand-crafted features: aggregated statistics such as total past bookings or average listing price. These worked well, but as the feature count grew into the hundreds, the approach became harder to scale and increasingly limited in expressiveness. In this blog post, we describe how we built a sequence modeling system that encodes the full guest journey using a Transformer, learning richer representations of guest preferences to deliver more personalized search results. An example of a guest journey, which is typically long, exploratory, and complex.Challenges Event sequences per guest present three core challenges. First, they are dominated by listing views, which account for the vast majority of all events -- some guests accumulate hundreds of thousands of them -- making raw sequences computationally intractable to model directly. The distribution of event types, with the majority being listing views. Second, unlike social media platforms, which optimize for engagement, Airbnb optimizes for booking conversion. Bookings are rare, compared to events, and deliberate, whereas a listing view could reflect genuine intent or simply idle browsing. Building a model that generalizes ## Building a Transformer-Based Category Recommender at Thumbtack DevFeed: [Building a Transformer-Based Category Recommender at Thumbtack]() Original publisher: [Read original article]() Author: Andrew Morss Published: 2026-07-20T20:23:27Z Content type: article Language: en Sources: [Thumbtack Engineering - Medium]() Topics: [recommendation systems](), [Transformer](), [Machine learning](), [email](), [push-notification]() Tags: [catalog](), [data-science](), [email](), [engineering](), [machine-learning](), [model](), [push-notification](), [recommendations](), [recommender-systems](), [torch]() ### AI overview This engineering article describes Thumbtack's category recommender, which selects home-service categories for push and email notifications. It explains how the system addresses popularity bias through negative sampling and how Thumbtack's small category catalog and sparse user activity shape the recommendation problem. ### Source excerpt A look at compensating for popularity bias in recommender systems using negative sampling strategies By: Andrew Morss, Senior Applied Scientist Introduction A recommender system is a machine learning model that, given a user and a catalog of items, predicts which items that user is most likely to want. Recommenders set your YouTube playlist, determine what items Amazon suggests for you, push you songs on Spotify and customize your Steam store. If you're a homeowner, Thumbtack's recommender systems can suggest home projects for you such as house cleaning or lawn mowing. Thumbtack connects users with professionals spread across almost 500 different job categories. Job categories are a kind of occupational bucket that Thumbtack uses as a taxonomy to categorize local service professionals (pros). Pros have the opportunity to choose categories of jobs that they want to receive leads on; examples include things like "Electrical and Wiring Repair", "Local Moving" or "Snow Plowing". Thumbtack's Lifecycle marketing team uses a category recommender to choose which categories to surface to users in our push and email notifications. Choosing the right ones is critical. A well-chosen suggestion is how a customer starts a search for a new project, finds the right service professional, and ultimately gets work done. An example of a push notification and an in-email category recommendation. The Problem Space: Category Recommendation There are several features of category recommendations at Thumbtack that are fairly unique in the item recommender space. The number of categories that we have is relatively small (~500) compared to the number of items that are usually seen in recommender systems, which simplifies the problem considerably. Usually recommenders require a two stage approach where an initial, low compute, filtering stage reduces the number of items to some "candidate" list of items before a more complicated model ranks that smaller set of items. With a relatively small cat ## Exploring Hierarchical Interest Representation For Meta Ads Deep Funnel Optimization DevFeed: [Exploring Hierarchical Interest Representation For Meta Ads Deep Funnel Optimization]() Original publisher: [Read original article]() Author: Yuhui Ouyang; Di Wang; Sreedal Menon; Jie Tian Published: 2026-07-15T17:00:52Z Content type: article Language: en Sources: [Engineering at Meta](), [Meta AI Research]() Topics: [Optimization](), [Embeddings](), [Graphs](), [recommendation systems](), [LLMs](), [Artificial Intelligence](), [Transformer](), [data]() Tags: [ads](), [ai](), [ai-research](), [data](), [embeddings](), [engineering](), [frontier-ai](), [generative](), [learning](), [meta](), [optimization](), [personalization](), [reasoning](), [research](), [retrieval](), [scale]() ### AI overview Meta describes Hierarchical Interest Representation, an upstream system that learns unified embeddings for users, advertisers, products, and services. It combines graph learning, multimodal content processed through LLMs, engagement signals, and self-supervised distillation to improve personalization, retrieval, ranking, and deep-funnel advertising optimization. ### Source excerpt Hierarchical Interest Representation is a research area for Meta Ads. We're exploring an upstream representation layer over the universe of Ads entities - users, advertisers, products, services - learning unified embeddings that connect users' inferred interests with the breadth of what advertisers offer in their deep funnel ads. The innovations in Hierarchical Interest Representation are [...] Read More... The post Exploring Hierarchical Interest Representation For Meta Ads Deep Funnel Optimization appeared first on Engineering at Meta. ## 🗓 This Week In AI Research (1-8 July 26) DevFeed: [🗓 This Week In AI Research (1-8 July 26)]() Original publisher: [Read original article]() Author: Dr. Ashish Bamania Published: 2026-07-12T11:25:32Z Content type: article Language: en Sources: [Into AI]() Topics: [Artificial Intelligence](), [AI Research](), [Large Language Model](), [Transformer](), [qwen](), [grpo](), [Algorithms](), [releases]() Tags: [agentic](), [ai](), [algorithm](), [algorithms](), [grpo](), [llm](), [models](), [performance](), [qwen](), [ranking](), [releases](), [research](), [rl](), [training](), [update]() ### AI overview A weekly roundup of AI research papers and releases highlights findings that reinforcement-learning gains can be concentrated in a single transformer layer and presents LLM-as-a-Verifier, a framework for continuous scoring and ranking of agentic-task solutions. ### Source excerpt The top 10 research papers and AI releases this week (SpaceXAI's Grok 4.5, OpenAI's GPT-Live voice models, Cognition's SWE-1.7, Meta's Muse Spark 1.1, and many more) ## Token efficiency: getting more signal into the context window DevFeed: [Token efficiency: getting more signal into the context window]() Original publisher: [Read original article]() Author: Jim Allen Wallace Published: 2026-07-01T00:00:00Z Content type: article Language: en Sources: [Redis Blog]() Topics: [long-context](), [Transformer](), [Large Language Model](), [Retrieval Augmented Generation (RAG)]() Tags: [agentic](), [context-window](), [efficiency](), [generation](), [hallucinations](), [llms](), [rag](), [reasoning](), [tech-de](), [tokens](), [transformers]() ### AI overview This guide explains why adding more context can worsen model performance. It examines token efficiency, the "lost in the middle" effect, context rot, attention limits in transformers, and the gap between advertised and effective context windows, with implications for RAG and agentic systems. ### Source excerpt You've probably hit this counterintuitive moment: you give your model more context to work with, expecting better answers, and the answers get worse. More tokens were supposed to mean more information, more grounding, fewer hallucinations. Instead, yo... ## GenPage: Towards End-to-End Generative Homepage Construction at Netflix DevFeed: [GenPage: Towards End-to-End Generative Homepage Construction at Netflix]() Original publisher: [Read original article]() Author: Netflix Technology Blog Published: 2026-06-29T13:01:02Z Content type: article Language: en Sources: [Netflix](), [Netflix TechBlog - Medium]() Topics: [Netflix](), [recommendation systems](), [Large Language Model](), [Machine learning](), [Reinforcement learning](), [Transformer]() Tags: [ai](), [app](), [diversity](), [engineering](), [feature-engineering](), [generation](), [generative](), [large-language-models](), [machine-learning](), [model](), [netflix](), [optimization](), [ranking](), [recommendation-system](), [reinforcement-learning](), [rl]() ### AI overview This Netflix developer article introduces GenPage, a generative approach that uses a single autoregressive model to construct a personalized homepage by generating recommendation rows, entities, and layout together. It describes replacing a multi-stage recommendation pipeline with end-to-end modeling and using reinforcement learning to optimize whole-page rewards, including interactions such as diversity and the balance between rows. ### Source excerpt Authors: Lequn Wang, Jiangwei Pan, and Linas Baltrunas Figure 1. Autoregressive homepage generation. GenPage builds a Netflix homepage one row or entity at a time, each one conditioned on what's already on the page and the user's context.Introduction The Netflix homepage is the first thing users see when they open the app and the primary way they discover content to enjoy. Almost every part of it is personalized, including which rows appear, which entities show up within those rows, and how everything is arranged on the page. Constructing that homepage is a genuinely hard problem. It is not simply producing one ranked list. The homepage is a structured, two-dimensional layout, made up of recommendation rows and the entities within them. Here, an entity can be a movie, show, game, live event, or other recommendable item. Each choice can affect the value of the others. Traditionally, it is built through a complex, multi-stage pipeline, with separate components for candidate generation and ranking at both the row and entity levels. We saw an opportunity to rethink this design. Large language models have shown that a single generative model can perform diverse tasks just by generating a response to a prompt. Inspired by this prompt-response paradigm, we trained a single generative model to build the homepage by directly answering one question: Given everything we know about this user and this request, what homepage should we generate to maximize user satisfaction? We call this approach GenPage. It treats the user history and request context as the prompt, and autoregressively generates the entire homepage as the response (Figure 1). Unlike most generative recommenders, such as TIGER, HSTU, and OneRec, which generate flat ranked lists, GenPage generates the rows, entities, and layout together. This shift is motivated by several goals: End-to-end modeling. A single transformer model that constructs the page from raw input signals can replace a complex multi-stage recommen ## LLMs: The Essential Guide DevFeed: [LLMs: The Essential Guide]() Original publisher: [Read original article]() Author: Nikki Siapno Published: 2026-06-26T14:44:38Z Content type: tutorial Language: en Sources: [Level Up Coding System Design Newsletter]() Topics: [Large Language Model](), [systems](), [tokenization](), [Transformer](), [distributed-systems](), [Inference](), [Embeddings]() Tags: [embeddings](), [guide](), [inference](), [latency](), [llm](), [llms](), [tokenization](), [transformers]() ### AI overview This guide explains LLMs as distributed systems with layers including models, training pipelines, retrieval, inference infrastructure, safety layers, and product experience. It introduces how tokenization and Transformer architectures work, and connects system layers to debugging, scaling, latency, hallucinations, context loss, and cost. ### Source excerpt Most engineers only see the API layer. Here's what's happening underneath. ## From Scoring to Spelling: Rebuilding Ads Retrieval at Instacart DevFeed: [From Scoring to Spelling: Rebuilding Ads Retrieval at Instacart]() Original publisher: [Read original article]() Author: Karuna Ahuja Published: 2026-06-02T18:50:19Z Content type: article Language: en Sources: [Instacart]() Topics: [Transformer](), [recommendations](), [systems](), [context](), [real-time]() Tags: [bert](), [context](), [generative](), [inference](), [machine-learning](), [model](), [real-time](), [recommendation-system](), [recommendations](), [retrieval]() ### AI overview Instacart describes rebuilding its ads retrieval system by moving from a product-scoring encoder to a generative model that predicts product IDs token by token. The system uses contextual shopping sessions and a BERT-like transformer to retrieve sponsored and organic recommendations. ### Source excerpt Key Contributors: Karuna Ahuja, Marko Avdalovic, Soroush Sobhkhiz, Shrikar Archak, Xiyu Wang, Ji Chao Zhang, Hao Yan Introduction Every time a user opens Instacart, they see product recommendations: on the retailer home page, in search results, and alongside their cart. Many of these recommendations are sponsored products surfaced by a retrieval model that decides which products to show from a vast ads product catalog. A relevant ad helps users discover products they didn't know they needed; a less relevant one generates friction. Two years ago, we introduced Contextual Recommendations (CR), a BERT-based sequence model powering retrieval for both ads and organic recommendations across all major browse surfaces. In this post, we'll focus on our ads retrieval. We will detail how we rebuilt the system, by moving from an encoder that scores products to a generative model that spells them out, token by token. By doing so, we unlocked a new level of contextual matching -- ensuring brands appear exactly when users want them, while simultaneously opening up discovery of thousands of relevant products the previous system couldn't retrieve. Contextual Recommendations: A recap At its core, CR treats grocery shopping as a language modeling task, where atomic product IDs function as tokens and, the finite subset of the catalog it is trained on, acts as its 'vocabulary'. The model leverages the user's real-time session, which includes product views, item page visits, and cart additions, as a sequence of these product tokens. A BERT-like transformer is then trained on millions of authentic shopping sessions to predict the next token (i.e. singular product) in the sequence. This process allows the model to learn and capture complex purchasing patterns, such as the tendency for users who add pasta and olive oil to frequently add garlic next. This single retrieval layer replaced multiple ad-hoc systems and powers recommendation carousels across all major browse surfaces, serving both ## Physics AI research that's shaping the industry. DevFeed: [Physics AI research that's shaping the industry.]() Original publisher: [Read original article]() Published: 2026-05-27T12:00:05Z Content type: news Language: en Sources: [Mistral AI Blog]() Topics: [AI Research](), [Physics-guided deep learning](), [Artificial Intelligence](), [data](), [datasets](), [AI Foundation Models](), [GPU](), [Machine learning](), [Transformer]() Tags: [ai](), [ai-research](), [data](), [datasets](), [deep-learning](), [design](), [energy](), [engineering](), [gpu](), [industry](), [machine-learning](), [models](), [performance](), [physics](), [real-time](), [research]() ### AI overview Mistral describes its acquisition of Emmi AI and its focus on Physics AI for industrial engineering. The article surveys published work on CFD, neural surrogates, foundation models, datasets, plasma turbulence, and real-time industrial simulation across aerospace, automotive, semiconductors, and energy. ### Source excerpt Published breakthroughs pushing the state of the art. ## Introducing the Ettin Reranker Family DevFeed: [Introducing the Ettin Reranker Family]() Original publisher: [Read original article]() Author: Tom Aarsen Published: 2026-05-19T00:00:00Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [sentence-transformers](), [Transformer](), [long-context](), [Agent Skill]() Tags: [ai-coding](), [community](), [long-context](), [nlp](), [open-source](), [ranking](), [research](), [retrieval](), [sentence-transformers]() ### AI overview The article introduces six Sentence Transformers CrossEncoder rerankers built on Ettin ModernBERT encoders. It explains their distillation-based training, retrieval-then-rerank usage, quality and cost trade-offs, and support for up to 8K tokens of context. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## LLM 训练与推理的基本理解 DevFeed: [LLM 训练与推理的基本理解]() Original publisher: [Read original article]() Author: Joway Published: 2026-05-17T00:00:00Z Content type: tutorial Language: zh Sources: [Random Thoughts]() Topics: [Large Language Model](), [Language models](), [Transformer]() Tags: [embedding](), [encoding](), [language](), [large-language-model](), [llm](), [model](), [query](), [tech](), [token](), [value]() ### AI overview This Chinese tutorial explains foundational LLM training and inference concepts, including vector operations, linear layers, Softmax, LayerNorm, tokenization, token IDs, BPE, token and positional embeddings, and the initial steps of self-attention using Q, K, and V vectors. ### Source excerpt 学习一个技术最好的方式就是能够写一片文章把这个技术的原理解释清楚，本文记录了我在阅读 《Build a Large Language Model (From Scratch)》一书以及和 Claude Code 对话过程中的笔记，仅供参考。 术语解释 向量点积 定义：向量点积为标量 a = (a1, a2, a3) b = (b1, b2, b3) a - b = a1*b1 + a2*b2 + a3*b3 几何意义： a - b = |a| |b| cos(theta) 其中 theta 是两个向量的夹角。 ## A guide to Transformer architecture in modern language models DevFeed: [A guide to Transformer architecture in modern language models]() Original publisher: [Read original article]() Author: Ashish Pratap Singh Published: 2026-05-14T04:15:11Z Content type: tutorial Language: en Sources: [AlgoMaster Newsletter]() Topics: [Transformer architecture](), [LLMs](), [Transformer]() Tags: [architecture-pattern](), [better](), [deep-dive](), [layer](), [llms](), [model](), [parallel](), [performance](), [semantics](), [sequence](), [syntax](), [tokens](), [transformer-architecture](), [transformers]() ### AI overview This tutorial explains the Transformer architecture, including its original encoder-decoder design for translation and the decoder-only variant used for modern language generation. It describes decoder components such as masked multi-head self-attention, feed-forward networks, layer normalization, and residual connections, and introduces the Pre-LayerNorm pattern. ### Source excerpt A single 2017 research paper changed the future of AI forever and gave rise to multiple unicorn companies. ## AlphaEvolve: How our Gemini-powered coding agent is scaling impact across fields DevFeed: [AlphaEvolve: How our Gemini-powered coding agent is scaling impact across fields]() Original publisher: [Read original article]() Author: AlphaEvolve team Published: 2026-05-06T10:43:49Z Content type: article Language: en Sources: [Google DeepMind News]() Topics: [Artificial Intelligence](), [coding](), [Google](), [Optimization](), [AI Infrastructure](), [Cache](), [Hardware](), [Inference](), [Transformer](), [Compiler](), [AI Platform](), [Algorithms, Complexity]() Tags: [ai](), [ai-infrastructure](), [algorithms](), [coding](), [efficiency](), [gemini](), [generation](), [google](), [hardware](), [inference](), [optimization](), [science]() ### AI overview AlphaEvolve, a Gemini-powered coding agent, has moved beyond pilot testing into Google's core infrastructure. It is being used to optimize TPU design, cache replacement policies, Google Spanner compaction, and compiler strategies, while also improving training, inference, simulations, routing, and model optimization across commercial and scientific applications. ### Source excerpt Explore how AlphaEvolve's Gemini-powered algorithms are driving impact across business, infrastructure, and science. ## Introducing NVIDIA Nemotron 3 Nano Omni: Long-Context Multimodal Intelligence for Documents, Audio and Video Agents DevFeed: [Introducing NVIDIA Nemotron 3 Nano Omni: Long-Context Multimodal Intelligence for Documents, Audio and Video Agents]() Original publisher: [Read original article]() Author: Tuomas Rintamaki; Amala Sanjay Deshmukh; Nabin Mulepati; Collin McCarthy; Pritam Biswas; Arushi Goel; Alexandre Milesi; Danial Mohseni Taheri; Kateryna Chumachenko; Isabel Hulseman; Zhehuai Chen; Kara Published: 2026-04-28T15:58:57Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [multimodal](), [Nemotron](), [long-context](), [asr](), [computer-use](), [Mamba](), [model architecture](), [moe](), [Transformer](), [Nvidia](), [Reinforcement learning](), [NVFP4]() Tags: [alternatives](), [analysis](), [architecture](), [benchmarks](), [computer-use](), [efficiency](), [long-context](), [models](), [multimodal](), [nemotron](), [nvidia](), [optimization](), [reasoning](), [reinforcement-learning](), [speed]() ### AI overview NVIDIA introduces Nemotron 3 Nano Omni, an omni-modal model for document analysis, image reasoning, speech recognition, long audio-video understanding, computer use, and general reasoning. It combines a hybrid Mamba-Transformer Mixture-of-Experts backbone with vision and audio encoders, supports long multimodal contexts, and reports strong benchmark accuracy, throughput, reasoning speed, and system efficiency. ### Source excerpt Introducing NVIDIA Nemotron 3 Nano Omni: Long-Context Multimodal Intelligence for Documents, Audio and Video Agents - NVIDIA Nemotron 3 Nano Omni is a new omni-modal understanding model built for real-world document analysis, multiple image reasoning, automatic speech recognition, long audio-video understanding, agentic computer use, and general reasoning. - It extends the Nemotron multimodal line from a strong vision-language system to a broader text + image + video + audio model. ## Diffusion LLMs, Explained Simply DevFeed: [Diffusion LLMs, Explained Simply]() Original publisher: [Read original article]() Author: Dr. Ashish Bamania Published: 2026-04-24T14:23:01Z Content type: tutorial Language: en Sources: [Dr. Ashish Bamania]() Topics: [Large Language Model](), [text-generation](), [Transformer]() Tags: [artificial-intelligence](), [data-science](), [diffusion](), [introduction](), [llms](), [machine-learning](), [programming](), [technology](), [text-generation](), [tokens]() ### AI overview An introductory explanation of diffusion large language models, contrasting them with autoregressive Transformer-based chatbots that generate responses sequentially, one token at a time. ### Source excerpt A gentle but comprehensive introduction to Diffusion LLMs. Continue reading on Level Up Coding " [Next page]()