# TPU Published articles for TPU. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## HeyGen x Google Cloud: Bringing Avatar IV to TPUs DevFeed: [HeyGen x Google Cloud: Bringing Avatar IV to TPUs]() Original publisher: [Read original article]() Published: 2026-09-12T11:04:33.891311Z Content type: article Language: en Sources: [Google Developers Blog]() Topics: [Google](), [Cloud](), [Artificial Intelligence](), [Optimization](), [AI Infrastructure](), [Transformer](), [PyTorch](), [Compiler](), [Streaming](), [real-time](), [Hardware](), [API]() Tags: [ai](), [ai-infrastructure](), [api](), [cloud](), [code](), [compiler](), [engineering](), [generation](), [google](), [hardware](), [model](), [optimization](), [performance](), [pytorch](), [real-time](), [streaming](), [time](), [tpu](), [transformers](), [video]() ### AI overview HeyGen and Google Cloud describe porting the 18B+ parameter Avatar IV talking-head video generation pipeline to an eight-chip Trillium TPU host. Using torchax, JAX, XLA, FSDP sharding, Ulysses sequence parallelism, and custom Pallas kernels, the team improved performance by 1.86x for real-time chunked streaming while preserving output quality through strict quality gates. ### Source excerpt HeyGen ported their 18B+ parameter Avatar IV video generation model to Google Cloud's Trillium (v6e) TPUs via torchax and XLA, utilizing FSDP and Ulysses sequence parallelism across an eight-chip mesh. To achieve a 1.86x speedup for real-time streaming, the engineering team pipelined exposed all-to-all collectives, aligned sparse attention block sizes to eliminate mask padding, and bypassed softmax serial dependencies using a precomputed Cauchy-Schwarz upper bound. These custom Pallas kernel and compiler optimizations were deployed only after passing rigorous two-tier quality gates to guarantee byte-identical or mathematically equivalent pixel outputs. ## Enterprise-Grade Precision for Long-Context Multimodal Embedding Inference on Cloud TPU DevFeed: [Enterprise-Grade Precision for Long-Context Multimodal Embedding Inference on Cloud TPU]() Original publisher: [Read original article]() Author: Anthony Su; Injae Kwak Published: 2026-09-12T11:04:33.891311Z Content type: article Language: en Sources: [Google Developers Blog]() Topics: [Embeddings](), [vllm](), [Inference](), [long-context](), [autoscaling](), [Kubernetes](), [Cloud](), [Google](), [Artificial Intelligence](), [model-deployment](), [Open Source]() Tags: [autoscaling](), [embedding](), [github](), [gpu](), [inference](), [kubernetes](), [long-context](), [multimodal](), [open-source](), [retrieval](), [tpu](), [vllm]() ### AI overview This article explains how Google Cloud integrated native TPU support into vLLM to serve long-context, multimodal embedding models at production scale. It describes elastic TPU provisioning with Google Kubernetes Engine, autoscaling across accelerator types, and TPU-specific optimizations for long sequences and chunked prefill. The resulting setup is designed to preserve numerical parity with GPU reference baselines while supporting high-throughput semantic retrieval applications. ### Source excerpt Google Cloud has natively integrated TPU support into the vLLM serving engine, allowing developers to elastically scale high-demand embedding pipelines using Google Kubernetes Engine (GKE). To handle massive 15K+ token contexts for models like Qwen3-Embedding-8B, the engineering team implemented TPU-specific optimizations such as hardware-safe tensor alignment, JAX/XLA compilation pre-warming, and a hybrid StepPool architecture for chunked prefill management. These enhancements achieve near-perfect numerical parity with reference GPU baselines, and developers can immediately leverage the open-sourced setup recipes on the AI-Hypercomputer GitHub to build their own high-throughput semantic retrieval applications. ## How to use Google microbenchmarks for evaluating TPU performance DevFeed: [How to use Google microbenchmarks for evaluating TPU performance]() Original publisher: [Read original article]() Author: Junjie Qian; Chi Shuen Lee; Yu-Hsuan (Amy) Lin; Haixiong (Sean) Wang Published: 2026-09-12T11:04:33.891311Z Content type: tutorial Language: en Sources: [Google Developers Blog]() Topics: [Google AI](), [Inference Performance]() Tags: [architecture](), [benchmarks](), [compute](), [developers](), [google](), [guides](), [hardware](), [how-to](), [kernel](), [machine-learning](), [memory](), [mesh](), [metrics](), [model](), [open](), [open-source](), [performance](), [scale](), [software](), [tpu]() ### AI overview A tutorial on using Google's TPU microbenchmark suite to measure network, compute, memory, host-transfer, and attention performance. The results can establish a Roofline baseline and guide workload-specific optimization. ### Source excerpt Google's open-source TPU microbenchmark suite provides developers with granular performance metrics across Network, Compute, HBM, Host Transfer, and Attention components to validate real-world hardware capabilities. By leveraging these benchmarks to establish a Roofline model, engineers can accurately diagnose whether their machine learning workloads are compute-, memory-, or network-bound. This empirical baseline directly guides targeted software optimizations--such as kernel tuning, mesh sharding, and rematerialization--to maximize hardware utilization for large-scale model deployments. ## WeatherNext: AI model achieves breakthrough in forecasting cyclones DevFeed: [WeatherNext: AI model achieves breakthrough in forecasting cyclones]() Original publisher: [Read original article]() Author: WeatherNext team Published: 2026-08-06T15:06:15Z Content type: article Language: en Sources: [Google DeepMind News]() Topics: [Artificial Intelligence](), [Open Source Models & Datasets](), [Google](), [data](), [Code]() Tags: [accelerate](), [ai](), [code](), [community](), [data](), [google](), [infrastructure](), [model](), [models](), [open](), [performance](), [research](), [science](), [speed](), [tpu]() ### AI overview WeatherNext is an AI weather-forecasting model that uses Functional Generative Networks to produce large ensembles of predictions and capture uncertainty. It generates 15-day forecasts in under a minute on a TPU, supports cyclone forecasting at relatively coarse resolution, and is being open sourced with its code and model weights for research and operational use. ### Source excerpt WeatherNext enables accurate cyclone forecasts that can give an extra day of warning. Now we are open sourcing the model. ## Unlocking TPU performance: Deep kernel profiling with XProf DevFeed: [Unlocking TPU performance: Deep kernel profiling with XProf]() Original publisher: [Read original article]() Author: Google Open Source (noreply@blogger.com) Published: 2026-06-08T18:30:00Z Content type: release Language: en Sources: [Google Open Source Blog]() Topics: [AI Development](), [telemetry](), [debugging](), [Compiler](), [Google]() Tags: [compiler](), [debugging](), [google](), [ml-dev-tools](), [performance](), [profiling](), [telemetry](), [tpu](), [tpu-optimization](), [tpu-performance]() ### AI overview Google introduces the Kernel Profiling suite in XProf for debugging and optimizing custom Pallas kernels on Google TPUs. It combines static compilation tracking with dynamic, sub-microsecond hardware telemetry and exposes compilation details through HLO graphs and lowered MLIR text. ### Source excerpt by Yogesh SY, AI Infra Google Unlocking TPU performance: Deep kernel profiling with XProf As machine learning workloads scale to unprecedented heights, developers are increasingly writing highly specialized Tensor Processing Unit (TPU) kernels using frameworks like Pallas, Mosaic, and Triton to maximize hardware performance. However, customizing high-performance kernels has historically introduced a major engineering challenge: optimization blind spots. To legacy performance profilers, custom compilation paths appear as opaque execution paths. Developers are left with single, massive execution blocks in their trace captures, lacking granular visibility into what is actually occurring inside the chip's internal components. Did a vector processing instruction stall? Was matrix math idle due to data loading bottlenecks? Traditional profiling relies heavily on compile-time static cost models to estimate kernel efficiency. While helpful for standard operations, these models cannot capture dynamic runtime realities like instruction execution stalls, memory subsystem congestion, or hardware scheduling conflicts. To open this opaque execution path, we are excited to introduce the Kernel Profiling suite in XProf--a low-level hardware debugging suite engineered specifically for Pallas kernel authoring and optimization on Google TPUs. By combining static compilation tracking with dynamic, sub-microsecond hardware telemetry, XProf Kernel provides the deep transparency required to optimize high-scale ML workloads. Deep visibility: HLO Graphs & MLIR Inspection The first step in debugging any custom kernel is understanding how your high-level code is translated by the compiler. When compiling a JAX or PyTorch model, the compiler generates a High-Level Optimizer (HLO) graph. Previously, custom calls inside these graphs remained completely obscured. XProf's updated Graph Viewer resolves this by exposing the internal compilation logic of these custom regions directly. To unlock this d ## 10 Confusing LLM Concepts, Explained Simply DevFeed: [10 Confusing LLM Concepts, Explained Simply]() Original publisher: [Read original article]() Author: Dr. Ashish Bamania Published: 2026-06-01T15:52:18Z Content type: tutorial Language: en Sources: [Dr. Ashish Bamania]() Topics: [Large Language Model](), [Artificial Intelligence](), [AI Agent](), [grpo](), [dataset](), [coding](), [math]() Tags: [agent](), [ai](), [artificial-intelligence](), [coding](), [cpu](), [data-science](), [dataset](), [gpu](), [grpo](), [llm](), [llms](), [machine-learning](), [math](), [programming](), [quantization](), [technology](), [tpu](), [train](), [training]() ### AI overview This tutorial introduces LLM concepts including on-policy and off-policy learning. It explains how models generate, score, and learn from responses, including the use of GRPO, teacher models, and datasets. The supplied excerpt also identifies CPU, GPU, TPU, pruning, and quantization as covered topics. ### Source excerpt The role of CPU/ GPU/ TPU in LLM workflows, Pruning, Quantization, and more. Continue reading on Level Up Coding " ## Decoupled DiLoCo: A new frontier for resilient, distributed AI training DevFeed: [Decoupled DiLoCo: A new frontier for resilient, distributed AI training]() Original publisher: [Read original article]() Author: Arthur Douillard and the DiLoCo team Published: 2026-04-22T10:20:03Z Content type: article Language: en Sources: [Google DeepMind News]() Topics: [Artificial Intelligence](), [Training AI Models](), [AI Infrastructure](), [Resilience](), [data centers](), [Hardware](), [Machine learning](), [networking](), [Google]() Tags: [ai](), [ai-infrastructure](), [ai-training](), [communication](), [compute](), [data-centers](), [hardware](), [infrastructure](), [networking](), [performance](), [research](), [resilience](), [scale](), [tpu](), [train]() ### AI overview Google describes Decoupled DiLoCo, a resilient distributed training architecture that trained a 12-billion-parameter model across four U.S. regions using achievable wide-area connectivity. By overlapping communication with computation, it was reported to be more than 20 times faster than conventional synchronization and could continue operating despite failures. ### Source excerpt Google's new distributed architecture keeps AI training runs on track across distant data centers, with exceptional efficiency - even when hardware fails. ## Leveraging CPU memory for faster, cost-efficient TPU LLM training DevFeed: [Leveraging CPU memory for faster, cost-efficient TPU LLM training]() Original publisher: [Read original article]() Author: Google Open Source (noreply@blogger.com) Published: 2026-04-10T18:30:00Z Content type: tutorial Language: en Sources: [Google Open Source Blog]() Topics: [cpu](), [Large Language Model](), [intel](), [Google Cloud Platform (GCP)](), [Cloud](), [implementation]() Tags: [accelerators](), [cloud-tpu](), [cost](), [cpu](), [host-offloading](), [intel-xeon](), [jax](), [llm-training](), [memory](), [performance](), [tpu]() ### AI overview This practical guide explains how to use host activation offloading with JAX on TPU platforms. It describes moving selected activations from TPU device memory to Intel Xeon CPU memory to reduce accelerator memory pressure and support larger models or batch sizes, while discussing potential throughput and cost benefits. ### Source excerpt by Keyur Ruganathbhai Ranipa, Qinglan Xiang, Vrushabh Sanghavi, Ramesh AG & Weilin Wang, Intel and Penporn Koanantakool, Google Host offloading with JAX on Intel® Xeon® processors As Large Language Models (LLMs) continue to scale into the hundreds of billions of parameters, device memory capacity has become a big limiting factor in training, as intermediate activations from every layer in the forward pass are needed in the backward pass. To reduce device memory pressure, these activations can be rematerialized during the backward pass, trading memory for recomputation. While rematerialization enables larger models to fit within limited device memory, it significantly increases training time and cost. Intel® Xeon® processors (5th and 6th Gen) with Advanced Matrix Extensions (AMX) enable practical host offloading of selected memory- and compute-intensive components in JAX training workflows. This approach can help teams train larger models, relieve accelerator memory pressure, improve end-to-end throughput, and reduce total cost of ownership--particularly on TPU-based Google Cloud instances. By publishing these results and implementation details, Google and Intel aim to promote transparency and share practical guidance with the community. This post describes how to enable activation offloading for JAX on TPU platforms and outlines considerations for building scalable, cost-aware hybrid CPU-accelerator training workflows. Figure 1. Google Cloud TPU Pod commonly used in LLM training. Host offloading Traditional LLM training is usually done on device accelerators alone. However, modern host machines have much larger memory size than accelerators (512GB or more) and can offer extra compute power, e.g., TFLOPS in case of Intel® Xeon® Scalable Processor with AMX capability. Leveraging host resources can be a great alternative to rematerialization. Host offloading selectively moves computation or data between host and device to optimize performance and memory usage. Host memo ## WeatherNext 2: Our most advanced weather forecasting model DevFeed: [WeatherNext 2: Our most advanced weather forecasting model]() Original publisher: [Read original article]() Author: The WeatherNext team Published: 2025-11-17T15:09:23Z Content type: article Language: en Sources: [Google DeepMind News]() Topics: [Artificial Intelligence](), [Google](), [model architecture](), [Inference](), [BigQuery](), [Google Cloud Platform (GCP)](), [AI Platform]() Tags: [ai](), [bigquery](), [google](), [inference](), [model](), [model-architecture](), [none](), [tpu](), [vertex-ai]() ### AI overview Google DeepMind and Google Research introduce WeatherNext 2, an AI weather forecasting model that generates hundreds of possible scenarios faster, at higher resolution, and with improved accuracy. Its forecast data is available through Earth Engine and BigQuery, with custom model inference offered through Vertex AI, and its technology is being integrated into several Google products. ### Source excerpt The new AI model delivers more efficient, more accurate and higher-resolution global weather predictions. ## TPU vs. GPU: Differences in Performance, Applications, Cost, and Ecosystem DevFeed: [TPU vs. GPU: Differences in Performance, Applications, Cost, and Ecosystem]() Original publisher: [Read original article]() Published: 2025-03-11T23:10:26Z Content type: comparison Language: en Sources: [Learn on Tailscale]() Topics: [GPU](), [Artificial Intelligence](), [AI Infrastructure](), [Deep learning](), [Google](), [Google Cloud Platform (GCP)](), [High-Performance Computing](), [Neural Network]() Tags: [ai](), [applications](), [deep-learning](), [efficiency](), [google](), [google-cloud-platform](), [gpu](), [high-performance-computing](), [speed](), [tpu](), [vs]() ### AI overview This comparison explains how Google TPUs and GPUs differ in AI processing. TPUs are designed for high-speed, low-precision deep-learning computation on Google Cloud, while GPUs provide more flexible parallel processing and broad framework compatibility. ### Source excerpt Two key players dominate the efficiency and speed of AI applications: the Graphics Processing Unit (GPU) and the Tensor Processing Unit (TPU). Both have their strengths and weaknesses. ## How good are LLMs at fixing their mistakes? A chatbot arena experiment with Keras and TPUs DevFeed: [How good are LLMs at fixing their mistakes? A chatbot arena experiment with Keras and TPUs]() Original publisher: [Read original article]() Author: Martin Görner Published: 2024-12-05T00:00:00Z Content type: opinion Language: en Sources: [Hugging Face - Blog]() Topics: [Large Language Model](), [Chat Bot](), [gradio]() Tags: [chatbots](), [gradio](), [hallucinations](), [keras](), [llm](), [llms](), [open-source](), [spaces](), [tpu]() ### AI overview The author describes a small experiment testing whether LLMs can correct code-generation mistakes after receiving feedback in plain English. The setup uses a mobile-assistant prompt that requires single-line executable Python API calls and compares conversations with multiple chatbots in a Gradio interface. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## Google Cloud TPUs made available to Hugging Face users DevFeed: [Google Cloud TPUs made available to Hugging Face users]() Original publisher: [Read original article]() Author: Simon Pagezy; Michelle Habonneau; Philipp Schmid; Alvaro Moran Published: 2024-07-09T00:00:00Z Content type: release Language: en Sources: [Hugging Face - Blog]() Topics: [inference-endpoints](), [Sovereign AI](), [Google Cloud Platform (GCP)](), [hugging face](), [spaces](), [optimum](), [tgi](), [gemma](), [llama](), [model-deployment](), [Latency](), [Hardware]() Tags: [gcp](), [gemma](), [google-cloud](), [hardware](), [hugging-face](), [inference](), [inference-endpoints](), [latency](), [llama](), [open-source](), [optimum](), [partnerships](), [performance](), [spaces](), [tgi](), [tpu]() ### AI overview Hugging Face announces that Google Cloud TPUs are available for Inference Endpoints and Spaces. Google TPU v5e configurations can deploy supported models through managed infrastructure, while Optimum TPU and Text Generation Inference help train and serve models on TPUs. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science. ## Welcome Gemma 2 - Google's new open LLM DevFeed: [Welcome Gemma 2 - Google's new open LLM]() Original publisher: [Read original article]() Author: Philipp Schmid; Omar Sanseviero; Pedro Cuenca; Lewis Tunstall; Tom Aarsen; Vaibhav Srivastav Published: 2024-06-27T00:00:00Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [gemma](), [Large Language Model](), [Google](), [Open Source](), [Fine-tuning](), [rlhf](), [Transformer]() Tags: [community](), [fine-tuning](), [gcp](), [gemma](), [google](), [llm](), [nlp](), [open-source](), [research](), [rlhf](), [tokens](), [tpu](), [transformers](), [warp]() ### AI overview The article introduces Gemma 2, Google's open large language model family available in 9-billion- and 27-billion-parameter sizes, with base and instruction-tuned variants. It describes the models' training data, permissive licensing, architectural improvements, TPU-based training, and instruction-tuning methods including supervised fine-tuning, distillation, RLHF, and model merging. ### Source excerpt We're on a journey to advance and democratize artificial intelligence through open source and open science.