# AWS Trainium

AWS Trainium is a purpose-built AI chip and integrated computing platform for large-scale AI training and inference, with supporting software and developer tooling.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Exa boosts embedding throughput by 50% to index billions of documents on AWS AI chips

DevFeed: [Exa boosts embedding throughput by 50% to index billions of documents on AWS AI chips](<https://devfeed.tech/articles/exa-boosts-embedding-throughput-by-50-to-index-billions-of-documents-on-aws-ai-chips-141408.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/exa-boosts-embedding-throughput-by-50-to-index-billions-of-documents-on-aws-ai-chips/>)

Author: Summer Devlin

Published: 2026-11-18T23:58:22Z

Content type: news

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [AWS Inferentia](<https://devfeed.tech/topics/aws-inferentia.md>), [AWS AI chips](<https://devfeed.tech/topics/aws-ai-chips.md>), [neuron](<https://devfeed.tech/topics/neuron.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [search-engines](<https://devfeed.tech/topics/search-engines.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-chips](<https://devfeed.tech/tags/ai-chips.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-ai-chips](<https://devfeed.tech/tags/aws-ai-chips.md>), [aws-inferentia](<https://devfeed.tech/tags/aws-inferentia.md>), [aws-neuron](<https://devfeed.tech/tags/aws-neuron.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [cost](<https://devfeed.tech/tags/cost.md>), [custom](<https://devfeed.tech/tags/custom.md>), [customer-solutions](<https://devfeed.tech/tags/customer-solutions.md>), [database](<https://devfeed.tech/tags/database.md>), [economics](<https://devfeed.tech/tags/economics.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [index](<https://devfeed.tech/tags/index.md>), [inference](<https://devfeed.tech/tags/inference.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [neuron](<https://devfeed.tech/tags/neuron.md>), [optimized](<https://devfeed.tech/tags/optimized.md>), [performance](<https://devfeed.tech/tags/performance.md>), [search](<https://devfeed.tech/tags/search.md>), [throughput](<https://devfeed.tech/tags/throughput.md>), [unit-economics](<https://devfeed.tech/tags/unit-economics.md>), [vector](<https://devfeed.tech/tags/vector.md>)

### AI overview

Exa reports a 47% improvement in performance and cost per million documents embedded on AWS Neuron hardware versus its initial baseline. Starting with FP16 data parallelism on Inferentia2, it optimized weight layout and precision, used a drop-in attention kernel, and built a fused SwiGLU MLP kernel to reduce memory spill. Custom attention kernels tailored to the model's shapes then cut padding work and instruction steps. Each change passed a correctness gate of 0.999 cosine similarity to the reference; performance was measured with a 30-minute saturation benchmark. The workload uses one NeuronCore per pod, and the same approach scales through data parallelism on Trainium2. Exa says tuning across hardware and Regions helps its scheduler match shifting demand and availability. The excerpt describes these results and methods but does not provide absolute throughput or cost figures.

### Source excerpt

Exa's neural search engine is backed by an in-house vector database, so the unit economics of embedding tens of billions of web documents drive the business. This post shares how Exa optimized embedding inference on AWS Neuron hardware (Inferentia2 and Trainium2) with custom NKI kernels, reaching a 47% gain in performance and cost per million documents.

## Computed Permutation Trellis Coding: Near-Optimal Quantization from Two Multiplications and a 64-Byte Table

DevFeed: [Computed Permutation Trellis Coding: Near-Optimal Quantization from Two Multiplications and a 64-Byte Table](<https://devfeed.tech/articles/computed-permutation-trellis-coding-near-optimal-quantization-from-two-multiplications-and-a-64-byte-table-139947.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22019>)

Author: Muhammed Fatih Bal{\\i}n

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [quantization](<https://devfeed.tech/topics/quantization.md>), [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>), [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>)

Tags: [ai-accelerator](<https://devfeed.tech/tags/ai-accelerator.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [bits](<https://devfeed.tech/tags/bits.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ar](<https://devfeed.tech/tags/cs-ar.md>), [cs-it](<https://devfeed.tech/tags/cs-it.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [encoding](<https://devfeed.tech/tags/encoding.md>), [floating-point](<https://devfeed.tech/tags/floating-point.md>), [fp4](<https://devfeed.tech/tags/fp4.md>), [fp8](<https://devfeed.tech/tags/fp8.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [kl-divergence](<https://devfeed.tech/tags/kl-divergence.md>), [llama](<https://devfeed.tech/tags/llama.md>), [llama-3-1](<https://devfeed.tech/tags/llama-3-1.md>), [llm](<https://devfeed.tech/tags/llm.md>), [math-it](<https://devfeed.tech/tags/math-it.md>), [permutation](<https://devfeed.tech/tags/permutation.md>), [quantization](<https://devfeed.tech/tags/quantization.md>), [table](<https://devfeed.tech/tags/table.md>), [tensors](<https://devfeed.tech/tags/tensors.md>), [trellis](<https://devfeed.tech/tags/trellis.md>)

### AI overview

CPTC jointly encodes tensor values and decodes them using two multiplications plus a 64-byte table, whose unevenly accessed entries can represent tail values. One decoder supports bit widths 1-6 and exactly reproduces standard integer and floating-point formats. The abstract reports that at 4 bits it comes within 0.07 bits of the information-theoretic ideal, matches a random-permutation trellis code with a table thousands of times larger, and exceeds QTIP codebooks. Without fine-tuning, it reports the lowest KL divergence to unquantized Llama 3.1 8B and 70B Instruct weights among tested QTIP comparisons at every tested width, by up to 27%. The decoder is built into AWS Trainium 4; reported throughput matches native FP4/FP8, while fused kernels quantize an 8B model's linear layers in under two minutes on one GPU. The Trainium encoding time is from cycle-level simulation.

### Source excerpt

arXiv:2611.22019v1 Announce Type: cross Abstract: We propose Computed Permutation Trellis Coding (CPTC), a low-bit quantization format for tensors written once and read many times. Formats that round each value on its own, such as INT4 and FP4, waste a large part of their bits against the information-theoretic ideal. Trellis codes recover this loss by encoding values jointly, but prior decoders either need a large lookup table for their best quality or compute values with a hash that restricts what they can decode. CPTC's decoder multiplies its state by two coefficients and adds the top bits of the products to index a 64-byte table. Because that sum is not uniformly distributed, the rarely used ends of a small table can hold values far out in the tails. One decoder serves every bit width from 1 to 6, reproduces the standard integer and floating-point formats exactly, and comes within 0.07 bits of the ideal at 4 bits. It matches a random-permutation trellis code whose table is thousands of times larger, and exceeds every QTIP codebook. In LLM weight quantization without fine-tuning, CPTC has the lowest KL divergence to the unquantized model at every tested bit width on Llama 3.1 8B and 70B Instruct against QTIP's codebooks (by up to 27%). The decoder has been built into the AWS Trainium 4 AI accelerator, where it runs at the throughput of the native FP4 and FP8 formats for a negligible fraction of the systolic array's area. Fused encoding kernels quantize an 8B model's linear layers in under two minutes on one GPU and, in cycle-level simulation, on one Trainium 4 chip.

## Deploy frontier AI models on AWS Trainium with SGLang-Neuron

DevFeed: [Deploy frontier AI models on AWS Trainium with SGLang-Neuron](<https://devfeed.tech/articles/deploy-frontier-ai-models-on-aws-trainium-with-sglang-neuron-130933.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/deploy-frontier-ai-models-on-aws-trainium-with-sglang-neuron/>)

Author: Alex Nails

Published: 2026-11-12T22:03:34Z

Content type: article

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [neuron](<https://devfeed.tech/topics/neuron.md>), [AWS AI chips](<https://devfeed.tech/topics/aws-ai-chips.md>), [PyTorch](<https://devfeed.tech/topics/pytorch.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-chips](<https://devfeed.tech/tags/ai-chips.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [ai-training](<https://devfeed.tech/tags/ai-training.md>), [ai-training-and-inference](<https://devfeed.tech/tags/ai-training-and-inference.md>), [amazon-ec2](<https://devfeed.tech/tags/amazon-ec2.md>), [announcements](<https://devfeed.tech/tags/announcements.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [container](<https://devfeed.tech/tags/container.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [deploy](<https://devfeed.tech/tags/deploy.md>), [frontier](<https://devfeed.tech/tags/frontier.md>), [frontier-ai](<https://devfeed.tech/tags/frontier-ai.md>), [frontier-ai-models](<https://devfeed.tech/tags/frontier-ai-models.md>), [models](<https://devfeed.tech/tags/models.md>), [neuron](<https://devfeed.tech/tags/neuron.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [serve](<https://devfeed.tech/tags/serve.md>), [sglang](<https://devfeed.tech/tags/sglang.md>), [trainium](<https://devfeed.tech/tags/trainium.md>)

### AI overview

SGLang-Neuron brings SGLang serving and reinforcement-learning rollouts to AWS Trainium through an AWS Deep Learning Container, while retaining SGLang's server interface and generation APIs. The out-of-tree plugin selects the Neuron platform with `SGLANG_PLATFORM=neuron`; SGLang core handles the API server, scheduler, request lifecycle, RadixAttention, and prefix-cache bookkeeping, while the plugin supplies Trainium-specific execution. It uses native PyTorch operations and Neuron Kernel Interface kernels compiled with `torch.compile`. For PD disaggregation, a prefill server transfers prompt KV cache to a decode server using NIXL over EFA. The walkthrough uses a Trainium3 EC2 instance and the `pytorch-inference-sglang-neuronx` container; supported models, features, and validated configurations are documented in the project resources.

### Source excerpt

SGLang-Neuron is now publicly available through an AWS Deep Learning Container. With SGLang-Neuron, you can serve models and generate reinforcement learning rollouts with SGLang on AWS Trainium using the same SGLang interface you already know, all running on native PyTorch.

## Scale distributed AI workloads with Ray and Anyscale on Amazon EC2

DevFeed: [Scale distributed AI workloads with Ray and Anyscale on Amazon EC2](<https://devfeed.tech/articles/scale-distributed-ai-workloads-with-ray-and-anyscale-on-amazon-ec2-132423.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/architecture/scale-distributed-ai-workloads-with-ray-and-anyscale-on-amazon-ec2/>)

Author: Sindhura Palakodety

Published: 2026-11-12T20:51:20Z

Content type: article

Language: en

Sources: [AWS Architecture Blog](<https://devfeed.tech/sources/aws-architecture-blog.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [distributed-training](<https://devfeed.tech/topics/distributed-training.md>), [Amazon EC2](<https://devfeed.tech/topics/amazon-ec2.md>), [Graviton](<https://devfeed.tech/topics/graviton.md>), [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [AWS Inferentia](<https://devfeed.tech/topics/aws-inferentia.md>), [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-chips](<https://devfeed.tech/tags/ai-chips.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [ai-ml](<https://devfeed.tech/tags/ai-ml.md>), [ai-workloads](<https://devfeed.tech/tags/ai-workloads.md>), [amazon-api-gateway](<https://devfeed.tech/tags/amazon-api-gateway.md>), [amazon-cloudwatch](<https://devfeed.tech/tags/amazon-cloudwatch.md>), [amazon-ec2](<https://devfeed.tech/tags/amazon-ec2.md>), [aws](<https://devfeed.tech/tags/aws.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [intermediate-200](<https://devfeed.tech/tags/intermediate-200.md>), [ml](<https://devfeed.tech/tags/ml.md>), [models](<https://devfeed.tech/tags/models.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [pipelines](<https://devfeed.tech/tags/pipelines.md>), [production](<https://devfeed.tech/tags/production.md>), [prototype](<https://devfeed.tech/tags/prototype.md>), [ray](<https://devfeed.tech/tags/ray.md>), [scale](<https://devfeed.tech/tags/scale.md>), [scaling](<https://devfeed.tech/tags/scaling.md>), [thought-leadership](<https://devfeed.tech/tags/thought-leadership.md>)

### AI overview

Ray and Anyscale provide a managed way to scale AI/ML pipelines on AWS while keeping Ray workload code consistent from a single instance to distributed clusters. Amazon EC2 offers the most flexibility, Amazon EKS fits organizations already operating Kubernetes but adds orchestration complexity and operator configuration, and SageMaker HyperPod targets resilience-critical large training runs with health monitoring, node replacement, and checkpoint recovery. Anyscale manages cluster lifecycle and autoscaling; compute and data remain in the customer's AWS account. The text describes matching CPU, GPU, Trainium, and Inferentia resources to pipeline stages, using Spot for fault-tolerant batch work, and combining Savings Plans, Capacity Blocks, reservations, and On-Demand capacity. Its suggested progression is to validate locally, scale data processing, expand training on a chosen deployment path, then deploy inference; it says this can be done without workload code changes.

### Source excerpt

Building multimodal AI/ML pipelines on Amazon EC2 means scaling distributed compute, controlling costs across stages, and keeping up with new models and hardware. This post shows how the Ray-powered Anyscale application on AWS helps you deploy Ray across three compute options and scale from prototype to production without code changes.

## With $23M in funding, Luminal is the latest startup to put Nvidia's CUDA moat in its crosshairs

DevFeed: [With $23M in funding, Luminal is the latest startup to put Nvidia's CUDA moat in its crosshairs](<https://devfeed.tech/articles/with-23m-in-funding-luminal-is-the-latest-startup-to-put-nvidia-s-cuda-moat-in-its-crosshairs-130458.md>)

Original publisher: [Read original article](<https://siliconangle.com/2026/11/12/with-23m-in-funding-luminal-is-the-latest-startup-to-put-nvidias-cuda-moat-in-its-crosshairs/>)

Author: Mike Wheatley

Published: 2026-11-12T15:00:44Z

Content type: news

Language: en

Sources: [SiliconANGLE](<https://devfeed.tech/sources/siliconangle.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>), [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-chips](<https://devfeed.tech/tags/ai-chips.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [amd](<https://devfeed.tech/tags/amd.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [aws](<https://devfeed.tech/tags/aws.md>), [chip](<https://devfeed.tech/tags/chip.md>), [cloud-computing](<https://devfeed.tech/tags/cloud-computing.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [dalton-caldwell](<https://devfeed.tech/tags/dalton-caldwell.md>), [early-stage](<https://devfeed.tech/tags/early-stage.md>), [felicis](<https://devfeed.tech/tags/felicis.md>), [funding](<https://devfeed.tech/tags/funding.md>), [generative-ai](<https://devfeed.tech/tags/generative-ai.md>), [infra](<https://devfeed.tech/tags/infra.md>), [investors](<https://devfeed.tech/tags/investors.md>), [joe-fioti](<https://devfeed.tech/tags/joe-fioti.md>), [led](<https://devfeed.tech/tags/led.md>), [liquid](<https://devfeed.tech/tags/liquid.md>), [luminal](<https://devfeed.tech/tags/luminal.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [news](<https://devfeed.tech/tags/news.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-corp](<https://devfeed.tech/tags/nvidia-corp.md>), [search-compiler](<https://devfeed.tech/tags/search-compiler.md>), [semiconductors](<https://devfeed.tech/tags/semiconductors.md>), [series](<https://devfeed.tech/tags/series.md>), [series-a](<https://devfeed.tech/tags/series-a.md>), [siliconangle](<https://devfeed.tech/tags/siliconangle.md>), [software](<https://devfeed.tech/tags/software.md>), [standard-capital](<https://devfeed.tech/tags/standard-capital.md>), [startup](<https://devfeed.tech/tags/startup.md>), [the-latest](<https://devfeed.tech/tags/the-latest.md>), [together-ai](<https://devfeed.tech/tags/together-ai.md>), [top-story-5](<https://devfeed.tech/tags/top-story-5.md>), [trainium](<https://devfeed.tech/tags/trainium.md>), [venture-capital](<https://devfeed.tech/tags/venture-capital.md>), [y-combinator](<https://devfeed.tech/tags/y-combinator.md>)

### AI overview

Luminal's $23 million Series A will support its effort to reduce the work required to run AI inference on chips outside Nvidia's CUDA ecosystem. Its search compiler benchmarks ways to execute a neural network on a target chip, generates code for the best-performing option, and caches it for recurring workloads; a managed inference cloud provides the runtime infrastructure. The company targets scalable, continuous inference applications and says it processes more than 45 trillion tokens monthly, with significant workloads running on AWS Trainium. The excerpt describes alternatives including Google's and AWS's own frameworks, Spectral Compute's toolkit for running CUDA-based code on other chips, and the open-source UXL Foundation's OneAPI. Luminal launched its compiler and cloud four months before the article; its planned expansion includes broader chip and model support.

### Source excerpt

Luminal AI Inc. has outlined a plan to tear down Nvidia Corp.'s formidable software moat in the artificial intelligence chip industry after raising $23 million in early-stage funding. Today's Series A round was led by Standard Capital and saw participation from existing investors such as Y Combinator, SagaVC, Liquid 2 and Felicis, bringing Luminal's total [...] The post With $23M in funding, Luminal is the latest startup to put Nvidia's CUDA moat in its crosshairs appeared first on SiliconANGLE.

## The attendee's guide to the AWS re:Invent 2026 AI/ML track

DevFeed: [The attendee's guide to the AWS re:Invent 2026 AI/ML track](<https://devfeed.tech/articles/the-attendee-s-guide-to-the-aws-re-invent-2026-ai-ml-track-129626.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/the-attendees-guide-to-the-aws-reinvent-2026-ai-ml-track/>)

Author: Markus Adhiwiyogo

Published: 2026-11-11T21:05:59Z

Content type: article

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Amazon Web Services](<https://devfeed.tech/topics/aws.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Amazon Bedrock](<https://devfeed.tech/topics/amazon-bedrock.md>), [Amazon SageMaker](<https://devfeed.tech/topics/amazon-sagemaker.md>), [AWS AI chips](<https://devfeed.tech/topics/aws-ai-chips.md>), [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [trust](<https://devfeed.tech/topics/trust.md>), [ai security](<https://devfeed.tech/topics/ai-security.md>), [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>), [neuron](<https://devfeed.tech/topics/neuron.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [human in the loop agent](<https://devfeed.tech/topics/human-in-the-loop-agent.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-chips](<https://devfeed.tech/tags/ai-chips.md>), [ai-ml](<https://devfeed.tech/tags/ai-ml.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-re-invent](<https://devfeed.tech/tags/aws-re-invent.md>), [aws-re-invent-2026](<https://devfeed.tech/tags/aws-re-invent-2026.md>), [build](<https://devfeed.tech/tags/build.md>), [can](<https://devfeed.tech/tags/can.md>), [chips](<https://devfeed.tech/tags/chips.md>), [foundational-100](<https://devfeed.tech/tags/foundational-100.md>), [goals](<https://devfeed.tech/tags/goals.md>), [guide](<https://devfeed.tech/tags/guide.md>), [helps](<https://devfeed.tech/tags/helps.md>), [inference](<https://devfeed.tech/tags/inference.md>), [ml](<https://devfeed.tech/tags/ml.md>), [models](<https://devfeed.tech/tags/models.md>), [more](<https://devfeed.tech/tags/more.md>), [re-invent](<https://devfeed.tech/tags/re-invent.md>), [uncategorized](<https://devfeed.tech/tags/uncategorized.md>)

### AI overview

The guide organizes more than 140 AWS re:Invent 2026 AI/ML sessions by topic and experience level, from chips and model training or inference to agent development, trust, sustainability, and deployment locations. It distinguishes recorded 60-minute lecture-style breakouts from interactive, generally non-recorded chalk talks, builder sessions, workshops, and code talks. It maps service choices: Amazon Bedrock provides managed access to models through a unified API, while SageMaker AI supports custom model training, fine-tuning, and deployment; the guide says many customers use both. AgentCore supports agents across model and provider mixes. Session recommendations cover practical tradeoffs such as model quality, speed, cost, capacity, and data residency; governance and observability; and infrastructure factors including latency, sovereignty, compute density, and scale. It also highlights agent safeguards, evaluation, and limits of what can be proven about agent behavior. The article recommends reserving seats early and notes that some sessions may be watched on demand.

### Source excerpt

AWS re:Invent 2026 includes more than 140 AI/ML track sessions. This attendee's guide helps you find the breakouts, chalk talks, workshops, and builders' sessions that fit your role and goals, spanning AI chips, models and inference, agents, agent trust, and sustainability, so you can build a smarter re:Invent schedule.

## Anthropic's IPO roadshow video

DevFeed: [Anthropic's IPO roadshow video](<https://devfeed.tech/articles/anthropic-s-ipo-roadshow-video-129840.md>)

Original publisher: [Read original article](<https://simonwillison.net/2026/Nov/11/anthropic-ipo-roadshow/>)

Author: Simon Willison

Published: 2026-11-11T15:11:59Z

Content type: article

Language: en

Sources: [Simon Willison's Weblog](<https://devfeed.tech/sources/simon-willison-s-weblog.md>)

Topics: [Claude](<https://devfeed.tech/topics/claude.md>), [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-2-321](<https://devfeed.tech/tags/ai-2-321.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [amd](<https://devfeed.tech/tags/amd.md>), [anthropic](<https://devfeed.tech/tags/anthropic.md>), [anthropic-355](<https://devfeed.tech/tags/anthropic-355.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [business](<https://devfeed.tech/tags/business.md>), [chip](<https://devfeed.tech/tags/chip.md>), [chips](<https://devfeed.tech/tags/chips.md>), [claude](<https://devfeed.tech/tags/claude.md>), [claude-mythos-fable](<https://devfeed.tech/tags/claude-mythos-fable.md>), [claude-mythos-fable-48](<https://devfeed.tech/tags/claude-mythos-fable-48.md>), [co-founder](<https://devfeed.tech/tags/co-founder.md>), [compute](<https://devfeed.tech/tags/compute.md>), [days](<https://devfeed.tech/tags/days.md>), [discovery](<https://devfeed.tech/tags/discovery.md>), [driver](<https://devfeed.tech/tags/driver.md>), [effective](<https://devfeed.tech/tags/effective.md>), [engineers](<https://devfeed.tech/tags/engineers.md>), [generative-ai](<https://devfeed.tech/tags/generative-ai.md>), [generative-ai-2-060](<https://devfeed.tech/tags/generative-ai-2-060.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [gpus-16](<https://devfeed.tech/tags/gpus-16.md>), [ipo](<https://devfeed.tech/tags/ipo.md>), [ipo-roadshow](<https://devfeed.tech/tags/ipo-roadshow.md>), [llms](<https://devfeed.tech/tags/llms.md>), [llms-2-023](<https://devfeed.tech/tags/llms-2-023.md>), [recursive-self-improvement](<https://devfeed.tech/tags/recursive-self-improvement.md>), [recursive-self-improvement-4](<https://devfeed.tech/tags/recursive-self-improvement-4.md>), [video](<https://devfeed.tech/tags/video.md>)

### AI overview

Anthropic says Claude is trained and served across AWS Trainium, Google TPUs, NVIDIA GPUs and AMD GPUs, enabled by a long-running multi-chip strategy that requires separate, costly software and inference stacks. The roadshow excerpt says Mythos got Claude running on AMD in three days with almost no human intervention, compared with prior platform integrations that took dozens of engineers months; Claude ported or wrote nearly all the kernels. Anthropic describes effective compute as physical capacity multiplied by discovered compute multipliers, which can reduce the hardware needed for a given training run or enable a more capable model with the same capacity. It argues these multipliers build on one another and speculates Claude may discover them autonomously as self-improvement advances. These are claims from an excerpt of the roadshow transcript, not independently verified results.

### Source excerpt

Anthropic's IPO roadshow video Anthropic kicked off their IPO roadshow yesterday, and their roadshow video has some previously unreleased information about how their business works. The section on "Approach to compute" with co-founder Tom Brown is particularly interesting, describing how Anthropic train and serve Claude across AWS Trainium, Google TPUs, NVIDIA GPUs, and AMD GPUs. I'm not sure why there's no official video transcript, but here's one that I extracted using MacWhisper. Highlights below: We train and serve Claude now across all four major chip families. That's only possible because early on we made an unconventional bet on a multi-chip strategy. At the time, many people thought that running AI models on different chips wasn't worth the effort. Each chip needs its own tailored software stack and inference stack, which is expensive and difficult to build. We've now been doing it for years, and we've gotten incredibly good at it. [...] As an example, this year, we started working with AMD GPUs for the first time. In the past, bringing on a new chip platform took dozens of engineers months of work This time, we had Mythos try it out. It got Claude running on AMD in just three days, with basically no human intervention. Claude ported or wrote almost 100% of the kernels. [...] The second driver of our compute advantage is our rate of discovery of compute multipliers. We focus on total effective compute, which is the multiplicative product of physical capacity times the compute multipliers that we discover. Each one we discover makes our pool of total effective compute bigger. For example, say you need 100 units of compute to train a model of a certain intelligence. A 4x compute multiplier means you can use 25 units of compute to train the exact same model. Or you can use those hundred physical units to train an even more intelligent model that would have previously required 400 units. Compute multipliers create a competitive advantage for us that's hard to re

## Exploiting Native Hardware Heterogeneity: Custom GPSIMD Operators for Accelerating Emerging LLM Memory on AWS Trainium

DevFeed: [Exploiting Native Hardware Heterogeneity: Custom GPSIMD Operators for Accelerating Emerging LLM Memory on AWS Trainium](<https://devfeed.tech/articles/exploiting-native-hardware-heterogeneity-custom-gpsimd-operators-for-accelerating-emerging-llm-memory-on-aws-trainium-126732.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.11820>)

Author: Zifan He, Jason Cong

Published: 2026-11-10T05:00:00Z

Content type: news

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [neuron](<https://devfeed.tech/topics/neuron.md>)

Tags: [aws](<https://devfeed.tech/tags/aws.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [customizable](<https://devfeed.tech/tags/customizable.md>), [decode](<https://devfeed.tech/tags/decode.md>), [deepseek-v4-1-flash](<https://devfeed.tech/tags/deepseek-v4-1-flash.md>), [emerging-memory](<https://devfeed.tech/tags/emerging-memory.md>), [fpga](<https://devfeed.tech/tags/fpga.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [hash](<https://devfeed.tech/tags/hash.md>), [hashing](<https://devfeed.tech/tags/hashing.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-memory](<https://devfeed.tech/tags/llm-memory.md>), [lookup](<https://devfeed.tech/tags/lookup.md>), [memory](<https://devfeed.tech/tags/memory.md>), [operators](<https://devfeed.tech/tags/operators.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [software](<https://devfeed.tech/tags/software.md>), [speed](<https://devfeed.tech/tags/speed.md>), [toolchain](<https://devfeed.tech/tags/toolchain.md>), [v4-1-flash](<https://devfeed.tech/tags/v4-1-flash.md>)

### AI overview

The work builds a toolchain for compiling C++ custom operators for Trainium 3 GPSIMD cores and injecting them into an NKI-compiled NEFF, addressing the lack of support for custom GPSIMD operators through the current NKI software stack. In tests against a hand-optimized NKI baseline for DeepSeek-V4.1-Flash, the two operators speed up decode-time n-gram hashing by 1.79x and table retrieval by up to 4.95x; total Engram computation improves by up to 1.14x. The abstract reports these results but does not describe evaluation beyond that setup.

### Source excerpt

arXiv:2611.11820v1 Announce Type: new Abstract: Emerging memory modules of large language models, such as Engram in DeepSeek-V4.1-Flash, mix irregular integer work (a 64-bit n-gram hash and a sparse table lookup) with dense computation (a GEMM and a gate). When deploying on a GPU-FPGA system, we can disaggregate operations to different devices for Engram to speed up by 5.9$\times$. AWS Trainium provides a similar heterogeneity natively: each Trainium NeuronCore pairs dense engines with a customizable GPSIMD Engine, and all engines share the on-chip SBUF. However, the Neuron Kernel Interface (NKI) exposes the GPSIMD cores only through fixed built-in instructions, and the official GPSIMD custom-operator toolchain does not support Trainium 2 or 3 and cannot be used with NKI. In this work, we build a toolchain that compiles C++ custom operators for the GPSIMD cores and injects them into the compiled NEFF from the NKI kernel on Trainium 3, expanding the customizability of GPSIMD with the current software stack. Over a hand-optimized NKI baseline of DeepSeek-V4.1-Flash, two GPSIMD operators speed up the n-gram hashing by 1.79$\times$ at decode, the table retrieval by up to 4.95$\times$, and the whole Engram computation by up to 1.14$\times$.

## AWS Launches TorchNeuron to Run PyTorch Models on AWS Trainium Without Modifications

DevFeed: [AWS Launches TorchNeuron to Run PyTorch Models on AWS Trainium Without Modifications](<https://devfeed.tech/articles/aws-launches-torchneuron-to-run-pytorch-models-on-aws-trainium-without-modifications-123022.md>)

Original publisher: [Read original article](<https://www.infoq.com/news/2026/11/aws-torchneuron-trainium-pytorch/>)

Author: Sergio De Simone

Published: 2026-11-09T11:00:00Z

Content type: release

Language: en

Sources: [InfoQ](<https://devfeed.tech/sources/infoq.md>)

Topics: [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [neuron](<https://devfeed.tech/topics/neuron.md>), [PyTorch](<https://devfeed.tech/topics/pytorch.md>), [AWS AI chips](<https://devfeed.tech/topics/aws-ai-chips.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-chips](<https://devfeed.tech/tags/ai-chips.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [ai-ml-data-engineering](<https://devfeed.tech/tags/ai-ml-data-engineering.md>), [amazon](<https://devfeed.tech/tags/amazon.md>), [annapurna-labs](<https://devfeed.tech/tags/annapurna-labs.md>), [apis](<https://devfeed.tech/tags/apis.md>), [architectures](<https://devfeed.tech/tags/architectures.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-torchneuron-trainium-pytorch](<https://devfeed.tech/tags/aws-torchneuron-trainium-pytorch.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [backend](<https://devfeed.tech/tags/backend.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [conference](<https://devfeed.tech/tags/conference.md>), [developers](<https://devfeed.tech/tags/developers.md>), [development](<https://devfeed.tech/tags/development.md>), [distributed-training](<https://devfeed.tech/tags/distributed-training.md>), [drug-discovery](<https://devfeed.tech/tags/drug-discovery.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [ml-data-engineering](<https://devfeed.tech/tags/ml-data-engineering.md>), [native](<https://devfeed.tech/tags/native.md>), [news](<https://devfeed.tech/tags/news.md>), [no-code](<https://devfeed.tech/tags/no-code.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>), [pytorch-conference](<https://devfeed.tech/tags/pytorch-conference.md>), [torchneuron](<https://devfeed.tech/tags/torchneuron.md>)

### AI overview

Amazon introduced TorchNeuron, a native PyTorch backend for AWS Trainium that aims to reduce migration work: with PyTorch automatic device detection, the text says switching from CUDA requires replacing `.to('cuda')` with `.to('neuron')`. It supports distributed training APIs including DDP, FSDP2 and DTensor, Torch streams and events, and custom performance-critical operators through the Neuron Kernel Interface. Amazon says it validated 100,000 popular models and more than 800 architectures. The backend is also presented as enabling shared hardware capacity for training and inference. The excerpt does not establish production performance or cost; cited observers say workload-specific performance, cost per token, reproducibility, and total cost of ownership remain to be tested.

### Source excerpt

At the recent PyTorch Conference North America 2026, Amazon introduced TorchNeuron, a native PyTorch backend that enables developers to run existing PyTorch code on AWS Trainium AI chips with virtually no code modifications. By Sergio De Simone

## AWS Weekly Roundup: Amazon Polly Generative engine, TorchNeuron for Trainium, and more (October 26, 2026)

DevFeed: [AWS Weekly Roundup: Amazon Polly Generative engine, TorchNeuron for Trainium, and more (October 26, 2026)](<https://devfeed.tech/articles/aws-weekly-roundup-amazon-polly-generative-engine-torchneuron-for-trainium-and-more-october-26-2026-101068.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/aws/aws-weekly-roundup-amazon-polly-generative-engine-torchneuron-for-trainium-and-more-october-26-2026/>)

Author: Daniel Abib

Published: 2026-10-26T15:02:24Z

Content type: news

Language: en

Sources: [AWS News Blog](<https://devfeed.tech/sources/aws-news-blog.md>)

Topics: [aws inspector](<https://devfeed.tech/topics/aws-inspector.md>), [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [neuron](<https://devfeed.tech/topics/neuron.md>), [Amazon SageMaker](<https://devfeed.tech/topics/amazon-sagemaker.md>), [Amazon ElastiCache](<https://devfeed.tech/topics/amazon-elasticache.md>), [Amazon Bedrock](<https://devfeed.tech/topics/amazon-bedrock.md>), [AWS AI chips](<https://devfeed.tech/topics/aws-ai-chips.md>), [Amazon Web Services](<https://devfeed.tech/topics/aws.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [ai](<https://devfeed.tech/tags/ai.md>), [amazon](<https://devfeed.tech/tags/amazon.md>), [amazon-bedrock](<https://devfeed.tech/tags/amazon-bedrock.md>), [amazon-bedrock-agentcore](<https://devfeed.tech/tags/amazon-bedrock-agentcore.md>), [amazon-ec2](<https://devfeed.tech/tags/amazon-ec2.md>), [amazon-elastic-container-service](<https://devfeed.tech/tags/amazon-elastic-container-service.md>), [amazon-elastic-file-system-efs](<https://devfeed.tech/tags/amazon-elastic-file-system-efs.md>), [amazon-elasticache](<https://devfeed.tech/tags/amazon-elasticache.md>), [amazon-polly](<https://devfeed.tech/tags/amazon-polly.md>), [amazon-sagemaker-ai](<https://devfeed.tech/tags/amazon-sagemaker-ai.md>), [announcements](<https://devfeed.tech/tags/announcements.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-cloud-financial-management](<https://devfeed.tech/tags/aws-cloud-financial-management.md>), [aws-datasync](<https://devfeed.tech/tags/aws-datasync.md>), [aws-neuron](<https://devfeed.tech/tags/aws-neuron.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [aws-waf](<https://devfeed.tech/tags/aws-waf.md>), [call](<https://devfeed.tech/tags/call.md>), [engine](<https://devfeed.tech/tags/engine.md>), [generative](<https://devfeed.tech/tags/generative.md>), [generative-ai](<https://devfeed.tech/tags/generative-ai.md>), [i](<https://devfeed.tech/tags/i.md>), [markets](<https://devfeed.tech/tags/markets.md>), [news](<https://devfeed.tech/tags/news.md>), [october](<https://devfeed.tech/tags/october.md>), [office](<https://devfeed.tech/tags/office.md>), [point](<https://devfeed.tech/tags/point.md>), [rds-for-oracle](<https://devfeed.tech/tags/rds-for-oracle.md>), [returning](<https://devfeed.tech/tags/returning.md>), [security](<https://devfeed.tech/tags/security.md>), [small](<https://devfeed.tech/tags/small.md>), [torchneuron](<https://devfeed.tech/tags/torchneuron.md>), [trainium](<https://devfeed.tech/tags/trainium.md>), [week-in-review](<https://devfeed.tech/tags/week-in-review.md>), [weekly](<https://devfeed.tech/tags/weekly.md>), [weekly-roundup](<https://devfeed.tech/tags/weekly-roundup.md>)

### AI overview

AWS Continuum is generally available as a vulnerability-management service that builds a context graph across repositories and AWS environments, tests findings in an isolated sandbox, ranks attack paths, and proposes fixes as pull requests requiring human approval. AWS honeypot telemetry recorded exploitation attempts four days before disclosure of two CVEs, with proof-of-concept exploits circulating before the embargo lifted, underscoring the short response window described in the article. Other launches include Amazon ECS Gateway, which lets platform teams configure shared networking while application teams connect services (up to 25 services per ALB), and an Amazon Polly upgrade adding 14 voices, including two Brazilian Portuguese voices, at unchanged API and price. TorchNeuron adds native PyTorch support for Trainium through a configuration change; SageMaker AI training sessions manage GPU operations while charging for processed tokens; Bedrock adds schema-compliant JSON for Claude models; and ElastiCache Serverless supports zero-downtime Valkey or Redis OSS migration.

### Source excerpt

A few weeks ago I presented to a small group of commercial and technology leaders from Brazilian agribusiness at the AWS office in São Paulo. My talk ran through where generative AI fits across the different stages of an agribusiness operation, and the point I kept returning to is that the markets we call traditional [...]

## Does Your Model Run on AWS Trainium? Look It Up

DevFeed: [Does Your Model Run on AWS Trainium? Look It Up](<https://devfeed.tech/articles/does-your-model-run-on-aws-trainium-look-it-up-97540.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/torch-neuronx-model-support>)

Author: David Corvoysier

Published: 2026-10-23T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [neuron](<https://devfeed.tech/topics/neuron.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [aws](<https://devfeed.tech/tags/aws.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [model](<https://devfeed.tech/tags/model.md>), [model-support](<https://devfeed.tech/tags/model-support.md>), [neuron](<https://devfeed.tech/tags/neuron.md>), [run](<https://devfeed.tech/tags/run.md>), [torch](<https://devfeed.tech/tags/torch.md>), [trainium](<https://devfeed.tech/tags/trainium.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

The `torch-neuronx` model support Space lets you check whether popular Hugging Face Transformers models run unmodified on AWS Trainium and provides the tested `run.py`, stack versions, and optional cached compilation graphs. Its Selected, Trending, and Downloads views report results and, for tested models, core count, wall time including compilation, and peak memory; exclusions may reflect either prioritization or incompatibility. Tests use Trainium2, cover models that fit on at most 16 cores, and check only 32 generated tokens, so PASS does not establish throughput or longer-generation quality. At the time described, coverage is limited to Transformers text-generation and image-text-to-text models. The underlying versioned results are also available in the public `aws-neuron/torch-neuronx-model-support` dataset.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## The science behind TorchNeuron, Amazon's new PyTorch backend for AWS Trainium chips

DevFeed: [The science behind TorchNeuron, Amazon's new PyTorch backend for AWS Trainium chips](<https://devfeed.tech/articles/the-science-behind-torchneuron-amazon-s-new-pytorch-backend-for-aws-trainium-chips-96329.md>)

Original publisher: [Read original article](<https://www.amazon.science/blog/the-science-behind-torchneuron-amazons-new-pytorch-backend-for-aws-trainium-chips>)

Author: Robert Hundt

Published: 2026-10-22T19:36:42Z

Content type: article

Language: en

Sources: [Amazon Science homepage](<https://devfeed.tech/sources/amazon-science-homepage.md>)

Topics: [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [PyTorch](<https://devfeed.tech/topics/pytorch.md>), [neuron](<https://devfeed.tech/topics/neuron.md>), [AWS AI chips](<https://devfeed.tech/topics/aws-ai-chips.md>), [Amazon EC2](<https://devfeed.tech/topics/amazon-ec2.md>), [Persistence](<https://devfeed.tech/topics/persistence.md>), [Graviton](<https://devfeed.tech/topics/graviton.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [amazon](<https://devfeed.tech/tags/amazon.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [backend](<https://devfeed.tech/tags/backend.md>), [chips](<https://devfeed.tech/tags/chips.md>), [cloud-and-systems](<https://devfeed.tech/tags/cloud-and-systems.md>), [efficiently](<https://devfeed.tech/tags/efficiently.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [models](<https://devfeed.tech/tags/models.md>), [optimizations](<https://devfeed.tech/tags/optimizations.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>), [runtime](<https://devfeed.tech/tags/runtime.md>), [torchneuron](<https://devfeed.tech/tags/torchneuron.md>), [trainium](<https://devfeed.tech/tags/trainium.md>)

### AI overview

TorchNeuron is an open-source backend that translates PyTorch code to run on AWS Trainium, generally with minimal or no code changes. It offers opt-in adaptive eager mode, which gathers operations into small graphs, compiles and caches them, and overlaps compilation with execution to reduce overhead while preserving source-code correlation for debugging. Generated executables (NEFFs) are cached in memory and on disk; across accelerators, compilation can be shared so identical code is compiled once and reused. In a test of a large language model spread across accelerators, these optimizations cut time to the first training step by about a third. For evaluation, AI agents surveyed open-source models, selected 841 representing varied architectures, and built a registry of tensor shapes. Fast shape-based tests screen revisions before tests on the representative set and then the full model distribution, enabling earlier regression detection.

### Source excerpt

Runtime optimizations and a thorough evaluation process driven by agentic AI help ensure that models written in PyTorch will execute efficiently and reliably on Trainium systems.

## Google and AWS open new paths for PyTorch beyond Nvidia

DevFeed: [Google and AWS open new paths for PyTorch beyond Nvidia](<https://devfeed.tech/articles/google-and-aws-open-new-paths-for-pytorch-beyond-nvidia-95871.md>)

Original publisher: [Read original article](<https://thenewstack.io/pytorch-tpu-trainium-migration/>)

Author: Meredith Shubel

Published: 2026-10-22T15:23:23Z

Content type: news

Language: en

Sources: [The New Stack](<https://devfeed.tech/sources/the-new-stack.md>)

Topics: [PyTorch](<https://devfeed.tech/topics/pytorch.md>), [AWS AI chips](<https://devfeed.tech/topics/aws-ai-chips.md>), [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [neuron](<https://devfeed.tech/topics/neuron.md>), [Google](<https://devfeed.tech/topics/google.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [AWS Inferentia](<https://devfeed.tech/topics/aws-inferentia.md>), [LiteRT](<https://devfeed.tech/topics/litert.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-chips](<https://devfeed.tech/tags/ai-chips.md>), [ai-development](<https://devfeed.tech/tags/ai-development.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [ai-research](<https://devfeed.tech/tags/ai-research.md>), [attention](<https://devfeed.tech/tags/attention.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [backend](<https://devfeed.tech/tags/backend.md>), [bare-metal](<https://devfeed.tech/tags/bare-metal.md>), [beta](<https://devfeed.tech/tags/beta.md>), [conference](<https://devfeed.tech/tags/conference.md>), [developer-tools](<https://devfeed.tech/tags/developer-tools.md>), [google](<https://devfeed.tech/tags/google.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [native](<https://devfeed.tech/tags/native.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>), [pytorch-conference](<https://devfeed.tech/tags/pytorch-conference.md>), [stack](<https://devfeed.tech/tags/stack.md>), [torchtpu](<https://devfeed.tech/tags/torchtpu.md>), [tpus](<https://devfeed.tech/tags/tpus.md>)

### AI overview

Google's TorchTPU and AWS's TorchNeuron make it easier to run PyTorch workloads on their respective AI chips, but peak performance can still require hardware-specific tuning. TorchTPU, developed with Meta, runs existing workloads without JAX runtime dependencies; eager execution supports debugging, while `torch.compile` can optimize TPU use. PyTorch says distributed workloads using TorchTPU's default Skinny Collectives can incur roughly 30-40% end-to-end performance loss versus an optimized approach, and points to TPU-specific kernels and production stacks for peak performance. TorchNeuron is in public beta for Trainium2 and Trainium3. Eager workloads may need only `.to('neuron')`; compiled models also require selecting the Neuron backend. AWS says configuration-only migration works in most cases, while a Trainium2 case study required changes to an attention kernel, Muon communication, and backward-pass compilation. Both expand PyTorch hardware options, with migration ease not guaranteeing top performance.

### Source excerpt

At PyTorch Conference North America 2026 this week, Google open-sourced TorchTPU, a native PyTorch stack for Google TPUs, now available The post Google and AWS open new paths for PyTorch beyond Nvidia appeared first on The New Stack.

## Train Qwen3 on AWS Trainium with torchtitan

DevFeed: [Train Qwen3 on AWS Trainium with torchtitan](<https://devfeed.tech/articles/train-qwen3-on-aws-trainium-with-torchtitan-93901.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/train-qwen3-on-aws-trainium-with-torchtitan/>)

Author: Josh Longenecker

Published: 2026-10-21T20:00:22Z

Content type: tutorial

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [neuron](<https://devfeed.tech/topics/neuron.md>), [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [AWS AI chips](<https://devfeed.tech/topics/aws-ai-chips.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [all](<https://devfeed.tech/tags/all.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [backend](<https://devfeed.tech/tags/backend.md>), [config](<https://devfeed.tech/tags/config.md>), [experts](<https://devfeed.tech/tags/experts.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [loading](<https://devfeed.tech/tags/loading.md>), [mixture-of-experts](<https://devfeed.tech/tags/mixture-of-experts.md>), [modeling](<https://devfeed.tech/tags/modeling.md>), [native](<https://devfeed.tech/tags/native.md>), [qwen3](<https://devfeed.tech/tags/qwen3.md>), [safetensors](<https://devfeed.tech/tags/safetensors.md>), [structure](<https://devfeed.tech/tags/structure.md>), [technical-how-to](<https://devfeed.tech/tags/technical-how-to.md>), [train](<https://devfeed.tech/tags/train.md>), [trainium](<https://devfeed.tech/tags/trainium.md>), [transformers](<https://devfeed.tech/tags/transformers.md>), [v5](<https://devfeed.tech/tags/v5.md>), [with](<https://devfeed.tech/tags/with.md>)

### AI overview

On a single AWS Trainium trn2.48xlarge, torchtitan can train Qwen3 through three paths using the same torchrun launcher and config structure: Transformers v5 via AutoModelForCausalLM, torchtitan's native Qwen3 model selected with config.model_spec, or NKI kernels for mixture-of-experts layers. The Transformers path can load published Hugging Face weights and save standard safetensors; the article reports a first-step loss of 6.54 from published weights versus 12.43 from random initialization for Qwen3-4B-Instruct-2507. The NKI example uses 64 data-parallel ranks without expert parallelism, and reduces Qwen3-30B-A3B to four layers and 64 experts to fit the instance. Custom NKI operators can be attached through torchtitan's override registry, but the rotary-kernel example demonstrates registration and autograd integration, not a performance gain. Pin the walkthrough's versions because the Transformers integration and Trainium overrides can change between torchtitan releases.

### Source excerpt

Train Qwen3 on AWS Trainium with torchtitan three ways on a single trn2.48xlarge: the Hugging Face Transformers v5 modeling backend, torchtitan's native Qwen3, and NKI mixture-of-experts kernels. The same torchrun launcher and config structure work unchanged across all three paths, including loading and saving published Hugging Face safetensors.

## Profiling TorchNeuron (Part 1): System-level profiling on AWS Trainium.

DevFeed: [Profiling TorchNeuron (Part 1): System-level profiling on AWS Trainium.](<https://devfeed.tech/articles/profiling-torchneuron-part-1-system-level-profiling-on-aws-trainium-92970.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/amazon/profiling-torchneuron-part-1-system-level-profilin>)

Author: Josh Longenecker

Published: 2026-10-21T16:04:43Z

Content type: tutorial

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [neuron](<https://devfeed.tech/topics/neuron.md>), [PyTorch](<https://devfeed.tech/topics/pytorch.md>), [Learning](<https://devfeed.tech/topics/learning.md>)

Tags: [aws](<https://devfeed.tech/tags/aws.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [compile](<https://devfeed.tech/tags/compile.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-performance](<https://devfeed.tech/tags/inference-performance.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [performance](<https://devfeed.tech/tags/performance.md>), [profiling](<https://devfeed.tech/tags/profiling.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>), [torchneuron](<https://devfeed.tech/tags/torchneuron.md>), [trainium](<https://devfeed.tech/tags/trainium.md>)

### AI overview

The walkthrough shows how to locate inference time on AWS Trainium with `torch.profiler` and Neuron Explorer. Configure `NeuronConfig` through `experimental_config`, include `ProfilerActivity.PrivateUse1`, and call `torch.neuron.synchronize()` in the profiling loop so device times are captured. Sort `key_averages()` by device time to find costly operations, then use Explorer to trace an operation from PyTorch dispatch through runtime scheduling to hardware execution. In the RMSNorm examples, the naive sequence uses separate operations and takes 2.355 ms of Neuron time; `torch.rms_norm()` fuses them into one kernel at 403.3 μs, while `torch.compile(backend='neuron')` fuses the original function at 375.5 μs. Both fused runs take about one-sixth the naive time in these measurements. The post attributes the reduction to fewer dispatches and avoiding intermediate values. It presents this as a workflow to apply to larger models; device-level engine and roofline analysis is reserved for Part 2.

### Source excerpt

Many ML practitioners have some intuition about how to make models run fast, but fewer know how to analyze why a model is slow. In Part 1 of this series, you will learn how to systematically analyze inference performance and identify where the time is spent. This post starts from scratch and assumes no prior profiling experience. The walkthrough uses AWS Trainium, a purpose-built AI chip for deep learning training and inference workloads, and TorchNeuron, its open source native PyTorch backend.

## Profiling TorchNeuron (Part 1): System-level profiling on AWS Trainium.

DevFeed: [Profiling TorchNeuron (Part 1): System-level profiling on AWS Trainium.](<https://devfeed.tech/articles/profiling-torchneuron-part-1-system-level-profiling-on-aws-trainium-93066.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/amazon/profiling-torchneuron-part-1>)

Author: Josh Longenecker

Published: 2026-10-21T16:04:43Z

Content type: tutorial

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [neuron](<https://devfeed.tech/topics/neuron.md>), [PyTorch](<https://devfeed.tech/topics/pytorch.md>), [AWS AI chips](<https://devfeed.tech/topics/aws-ai-chips.md>), [Learning](<https://devfeed.tech/topics/learning.md>)

Tags: [aws](<https://devfeed.tech/tags/aws.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [inference-performance](<https://devfeed.tech/tags/inference-performance.md>), [profiling](<https://devfeed.tech/tags/profiling.md>), [system](<https://devfeed.tech/tags/system.md>), [torchneuron](<https://devfeed.tech/tags/torchneuron.md>), [trainium](<https://devfeed.tech/tags/trainium.md>)

### AI overview

The post shows how to profile PyTorch inference on AWS Trainium with `torch.profiler` and Neuron tools, then trace device time from framework dispatch through runtime scheduling to hardware execution in Neuron Explorer. It compares three RMSNorm implementations under the same inputs and profiler schedule: separate PyTorch operations take 2.355 ms of Neuron time, while `torch.rms_norm()` and `torch.compile(backend="neuron")` each fuse the work into one kernel dispatch and take about 403 μs and 375.5 μs, respectively, in these runs. The walkthrough explains configuring `NeuronConfig`, capturing a clean trace with `ProfilerActivity.PrivateUse1`, synchronizing before export, and using `key_averages()` to find device-time hotspots. These results illustrate the benefit of fusion for this operation; Part 1 does not yet cover device-level engine or roofline analysis.

### Source excerpt

Many ML practitioners have some intuition about how to make models run fast, but fewer know how to analyze why a model is slow. In Part 1 of this series, you will learn how to systematically analyze inference performance and identify where the time is spent. This post starts from scratch and assumes no prior profiling experience. The walkthrough uses AWS Trainium, a purpose-built AI chip for deep learning training and inference workloads, and TorchNeuron, its open source native PyTorch backend.

## DIFFLET: A Fast Inference Engine for Diffusion Transformer Models on AWS Trainium

DevFeed: [DIFFLET: A Fast Inference Engine for Diffusion Transformer Models on AWS Trainium](<https://devfeed.tech/articles/difflet-a-fast-inference-engine-for-diffusion-transformer-models-on-aws-trainium-93738.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.23047>)

Author: Bin Ma, Min Yu, Vijay Kodamalla, Jonny Liu, Da Li, Dong Li

Published: 2026-10-21T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [Cache](<https://devfeed.tech/topics/cache.md>), [jetstream](<https://devfeed.tech/topics/jetstream.md>)

Tags: [1-5](<https://devfeed.tech/tags/1-5.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [attention](<https://devfeed.tech/tags/attention.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [backend](<https://devfeed.tech/tags/backend.md>), [cache](<https://devfeed.tech/tags/cache.md>), [caching](<https://devfeed.tech/tags/caching.md>), [compilation](<https://devfeed.tech/tags/compilation.md>), [component](<https://devfeed.tech/tags/component.md>), [controller](<https://devfeed.tech/tags/controller.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [engine](<https://devfeed.tech/tags/engine.md>), [fast](<https://devfeed.tech/tags/fast.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-engine](<https://devfeed.tech/tags/inference-engine.md>), [models](<https://devfeed.tech/tags/models.md>), [trainium](<https://devfeed.tech/tags/trainium.md>), [transformer](<https://devfeed.tech/tags/transformer.md>)

### AI overview

Difflet serves diffusion Transformer pipelines on AWS Trainium2 by sharing one ahead-of-time compilation backend across five model pipelines, splitting joint attention across cores by head, and using a host controller to choose at each denoising step between running the denoiser and reusing its prior output. A small on-device probe supplies the caching signal, while a resident process keeps programs loaded across requests. Across five image and video model families, fused attention makes denoiser steps 1.2-4.3x faster than PyTorch/XLA attention in the same framework and is the only attention reported to compile at 32.8k visual tokens. Resident serving is 1.5-3.4x faster than standalone runs that reload weights from a warm file cache on four of the five models.

### Source excerpt

arXiv:2610.23047v2 Announce Type: replace Abstract: On AWS Trainium2, every component of a diffusion model runs as a program compiled ahead of time for fixed shapes, while pipelines differ across models, prompt lengths across requests, and caching decisions across denoising steps. We present Difflet, a serving engine in which five model pipelines share one compilation backend, joint attention is split across cores by attention head, and a host controller decides at each denoising step whether to run the denoiser or reuse its earlier output. A small probe program computes the caching signal on the device and returns one number per step, and a resident model process keeps every program loaded across requests. Across five image and video model families, fused attention accelerates denoiser steps by 1.2-4.3x over the same framework using PyTorch/XLA attention, and at 32.8k visual tokens it is the only attention that compiles. On four of the five models, resident serving is 1.5-3.4x faster than standalone runs that reload weights from a warm file cache for each generation.

## Beyond the KV cache: Training and serving i14's Laplace 3B linear attention model on AWS Trainium2 with native PyTorch

DevFeed: [Beyond the KV cache: Training and serving i14's Laplace 3B linear attention model on AWS Trainium2 with native PyTorch](<https://devfeed.tech/articles/beyond-the-kv-cache-training-and-serving-i14-s-laplace-3b-linear-attention-model-on-aws-trainium2-with-native-pytorch-91288.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/beyond-the-kv-cache-training-and-serving-i14s-laplace-3b-linear-attention-model-on-aws-trainium2-with-native-pytorch/>)

Author: Kevin Shabahang

Published: 2026-10-20T21:14:10Z

Content type: article

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [PyTorch](<https://devfeed.tech/topics/pytorch.md>), [model architecture](<https://devfeed.tech/topics/model-architecture.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [AWS AI chips](<https://devfeed.tech/topics/aws-ai-chips.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Transformer](<https://devfeed.tech/topics/transformer.md>), [High-Performance Computing](<https://devfeed.tech/topics/high-performance-computing.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [attention](<https://devfeed.tech/tags/attention.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [cache](<https://devfeed.tech/tags/cache.md>), [context](<https://devfeed.tech/tags/context.md>), [customer-solutions](<https://devfeed.tech/tags/customer-solutions.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [key](<https://devfeed.tech/tags/key.md>), [kv-cache](<https://devfeed.tech/tags/kv-cache.md>), [learn](<https://devfeed.tech/tags/learn.md>), [linear](<https://devfeed.tech/tags/linear.md>), [memory](<https://devfeed.tech/tags/memory.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [native](<https://devfeed.tech/tags/native.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>), [recurrent](<https://devfeed.tech/tags/recurrent.md>), [sequence](<https://devfeed.tech/tags/sequence.md>), [state](<https://devfeed.tech/tags/state.md>), [training](<https://devfeed.tech/tags/training.md>), [transformer](<https://devfeed.tech/tags/transformer.md>), [with](<https://devfeed.tech/tags/with.md>)

### AI overview

Laplace 3B uses Gated Linear Attention to keep inference state fixed at 29.5 MB per sequence regardless of context length; this compresses prior context, so retrieval quality still needs evaluation for each workload. i14 and AWS adapted the model to Trainium2 with native PyTorch through TorchNeuron Native, using 16 chunks of 128 tokens for GLA, statically unrolling the backward recurrence, and redesigning Muon communication with two all_to_all operations. On one trn2.48xlarge, training reached about 120,000 tokens per second; the all_to_all design cut per-rank traffic 24-fold and raised throughput 1.63-fold. Serving through an unmodified vLLM engine reached 11,544 output tokens per second across 1,280 concurrent sequences. Compiling decode improved throughput, while variable batch shapes incurred compilation costs that fixed-width padding removed. Compiled BF16 output can differ slightly from eager output, which the text recommends when exact reproducibility matters.

### Source excerpt

Learn how i14 trained and served Laplace 3B, a linear attention model, on AWS Trainium2 using native PyTorch. Its recurrent state stays fixed at 29.5 MB per sequence regardless of context length, avoiding the growing key-value cache of transformer models and keeping serving memory predictable.

## DeepSeek open-sources the inference engine behind its API. And Huawei gets equal billing with Nvidia.

DevFeed: [DeepSeek open-sources the inference engine behind its API. And Huawei gets equal billing with Nvidia.](<https://devfeed.tech/articles/deepseek-open-sources-the-inference-engine-behind-its-api-and-huawei-gets-equal-billing-with-nvidia-90665.md>)

Original publisher: [Read original article](<https://thenewstack.io/deepseek-deepinfer-huawei-nvidia/>)

Author: Paul Sawers

Published: 2026-10-20T14:58:36Z

Content type: news

Language: en

Sources: [The New Stack](<https://devfeed.tech/sources/the-new-stack.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [DeepSeek Harness](<https://devfeed.tech/topics/dsh.md>), [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-chips](<https://devfeed.tech/tags/ai-chips.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [apache-2-0](<https://devfeed.tech/tags/apache-2-0.md>), [asynchronous](<https://devfeed.tech/tags/asynchronous.md>), [batch](<https://devfeed.tech/tags/batch.md>), [chips](<https://devfeed.tech/tags/chips.md>), [code](<https://devfeed.tech/tags/code.md>), [community](<https://devfeed.tech/tags/community.md>), [compute-costs](<https://devfeed.tech/tags/compute-costs.md>), [computing](<https://devfeed.tech/tags/computing.md>), [deepseek](<https://devfeed.tech/tags/deepseek.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [huawei](<https://devfeed.tech/tags/huawei.md>), [inference](<https://devfeed.tech/tags/inference.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [stack](<https://devfeed.tech/tags/stack.md>)

### AI overview

DeepSeek has released DeepInfer, the inference engine it uses for its models, under the Apache 2.0 license. It supports offline inference and includes optimizations for distributing work across chips, speculative decoding, asynchronous scheduling, structured outputs, and request handling through the companion deepseek-recipe library. The release supports Nvidia SM100 GPUs and Huawei Ascend 950 NPUs, though some features are Nvidia-only. It does not include several components needed for a complete production service, such as KV cache management, cross-machine inference stages, and replica load balancing.

### Source excerpt

DeepSeek has open-sourced the inference engine it uses to serve its models in production, a step it said last year The post DeepSeek open-sources the inference engine behind its API. And Huawei gets equal billing with Nvidia. appeared first on The New Stack.

## Amazon introduces toolkit for running PyTorch workloads on AWS Trainium chips

DevFeed: [Amazon introduces toolkit for running PyTorch workloads on AWS Trainium chips](<https://devfeed.tech/articles/amazon-introduces-toolkit-for-running-pytorch-workloads-on-aws-trainium-chips-90481.md>)

Original publisher: [Read original article](<https://siliconangle.com/2026/10/19/amazon-introduces-toolkit-for-running-pytorch-workloads-on-aws-trainium-chips/>)

Author: Maria Deutscher

Published: 2026-10-20T00:23:31Z

Content type: news

Language: en

Sources: [SiliconANGLE](<https://devfeed.tech/sources/siliconangle.md>)

Topics: [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [PyTorch](<https://devfeed.tech/topics/pytorch.md>), [neuron](<https://devfeed.tech/topics/neuron.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-training](<https://devfeed.tech/tags/ai-training.md>), [amazon](<https://devfeed.tech/tags/amazon.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [chip](<https://devfeed.tech/tags/chip.md>), [chips](<https://devfeed.tech/tags/chips.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [llm](<https://devfeed.tech/tags/llm.md>), [news](<https://devfeed.tech/tags/news.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>), [the-latest](<https://devfeed.tech/tags/the-latest.md>), [torchneuron](<https://devfeed.tech/tags/torchneuron.md>), [trainium](<https://devfeed.tech/tags/trainium.md>)

### AI overview

Amazon introduced TorchNeuron, an open-source toolkit that lets developers run PyTorch workloads on AWS Trainium chips with minimal code changes. It supports Trainium2 and Trainium3, includes pre-packaged NKI operators for AI model development, and works with tools such as TorchTitan to optimize distributed training workflows.

### Source excerpt

Amazon.com Inc. today introduced TorchNeuron, an open-source toolkit that enables developers to run PyTorch workloads on its AWS Trainium chips. PyTorch is the industry's most popular artificial intelligence development framework. Until now, running AI projects powered by the tool on Trainium-based Amazon Web Services instances involved a significant amount of manual work. That increased development [...] The post Amazon introduces toolkit for running PyTorch workloads on AWS Trainium chips appeared first on SiliconANGLE.

## How Deepgram optimized voice AI at scale with AWS Trainium

DevFeed: [How Deepgram optimized voice AI at scale with AWS Trainium](<https://devfeed.tech/articles/how-deepgram-optimized-voice-ai-at-scale-with-aws-trainium-90433.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/how-deepgram-optimized-voice-ai-at-scale-with-aws-trainium/>)

Author: Pierre Lienhart

Published: 2026-10-19T16:23:22Z

Content type: article

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [AWS AI chips](<https://devfeed.tech/topics/aws-ai-chips.md>), [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [neuron](<https://devfeed.tech/topics/neuron.md>), [speech-to-speech](<https://devfeed.tech/topics/speech-to-speech.md>), [data-engineering](<https://devfeed.tech/topics/data-engineering.md>), [chainguard vms](<https://devfeed.tech/topics/chainguard-vms.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [ai](<https://devfeed.tech/tags/ai.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [customer-solutions](<https://devfeed.tech/tags/customer-solutions.md>), [deepgram](<https://devfeed.tech/tags/deepgram.md>), [inference](<https://devfeed.tech/tags/inference.md>), [neuron](<https://devfeed.tech/tags/neuron.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [rust](<https://devfeed.tech/tags/rust.md>), [speech-to-text](<https://devfeed.tech/tags/speech-to-text.md>), [throughput](<https://devfeed.tech/tags/throughput.md>), [trainium](<https://devfeed.tech/tags/trainium.md>), [voice](<https://devfeed.tech/tags/voice.md>), [voice-ai](<https://devfeed.tech/tags/voice-ai.md>)

### AI overview

Deepgram and AWS ported the Nova-3 speech-to-text model to Trainium2 for offline batch inference. They tuned kernels and built a Rust data-parallel runtime. On the evaluated workloads, the setup processed 11-70% more audio per dollar than Deepgram's previous compute options.

### Source excerpt

Deepgram worked with the AWS Forward Deployed Engineering team to port its Nova-3 speech-to-text model to AWS Trainium2 for offline batch inference. This post covers porting to the AWS Neuron stack, kernel-level tuning, a data-parallel Rust runtime, and the throughput and cost results on batch transcription.

## Introducing TorchNeuron: Native PyTorch backend for AWS Trainium

DevFeed: [Introducing TorchNeuron: Native PyTorch backend for AWS Trainium](<https://devfeed.tech/articles/introducing-torchneuron-native-pytorch-backend-for-aws-trainium-90224.md>)

Original publisher: [Read original article](<https://pytorch.org/blog/introducing-torchneuron-native-pytorch-backend-for-aws-trainium/>)

Author: Nafea Bshara, Robert Hundt, Ron Diamant, Colin Brace, Rahul Solanki, Maen Suleiman, Khaled ElGalaind, Temesghen Kahsai, Cyril Bortolato, Indu Thangakrishnan, Shauheen Zahirazami, Sarang Ozarde, Adarsh

Published: 2026-10-19T12:45:23Z

Content type: release

Language: en

Sources: [PyTorch](<https://devfeed.tech/sources/blog-pytorch.md>)

Topics: [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [PyTorch](<https://devfeed.tech/topics/pytorch.md>), [AWS AI chips](<https://devfeed.tech/topics/aws-ai-chips.md>), [OpenTelemetry](<https://devfeed.tech/topics/opentelemetry.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-research](<https://devfeed.tech/tags/ai-research.md>), [apache-2-0](<https://devfeed.tech/tags/apache-2-0.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [blog](<https://devfeed.tech/tags/blog.md>), [models](<https://devfeed.tech/tags/models.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>), [research](<https://devfeed.tech/tags/research.md>), [transformers](<https://devfeed.tech/tags/transformers.md>), [video](<https://devfeed.tech/tags/video.md>)

### AI overview

PyTorch announces TorchNeuron, an open source native backend that lets developers run PyTorch workloads on AWS Trainium. It supports eager execution, torch.compile, distributed APIs, custom Neuron Kernel Interface operators, and profiling. TorchNeuron is scheduled for public beta on October 20, 2026, under the Apache 2.0 license.

### Source excerpt

Frontier models span language, video and audio, on architectures from dense and mixture-of-experts transformers to diffusion, state-space, world and structure-prediction models. PyTorch is where AI research happens, where the models... The post Introducing TorchNeuron: Native PyTorch backend for AWS Trainium appeared first on PyTorch.

## TorchNeuron: running HF models on AWS Trainium is easier than ever

DevFeed: [TorchNeuron: running HF models on AWS Trainium is easier than ever](<https://devfeed.tech/articles/torchneuron-running-hf-models-on-aws-trainium-is-easier-than-ever-90237.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/hf-torchneuron>)

Author: Régis Pierrard; Simon Pagezy; Michael Benayoun; Jingya Huang; Alvaro Moran; David Corvoysier; Jeff Boudier

Published: 2026-10-19T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [AWS AI chips](<https://devfeed.tech/topics/aws-ai-chips.md>), [neuron](<https://devfeed.tech/topics/neuron.md>), [AI Development](<https://devfeed.tech/topics/ai-development.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>), [NeMo](<https://devfeed.tech/topics/nemo.md>), [optimum](<https://devfeed.tech/topics/optimum.md>), [peft](<https://devfeed.tech/topics/peft.md>), [trl](<https://devfeed.tech/topics/trl.md>), [lunar foundation model](<https://devfeed.tech/topics/lunar-foundation-model.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>), [AI Integration Strategies](<https://devfeed.tech/topics/ai-integration-strategies.md>)

Tags: [ai-chips](<https://devfeed.tech/tags/ai-chips.md>), [apis](<https://devfeed.tech/tags/apis.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [async](<https://devfeed.tech/tags/async.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [code](<https://devfeed.tech/tags/code.md>), [cost](<https://devfeed.tech/tags/cost.md>), [diffusers](<https://devfeed.tech/tags/diffusers.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [eos](<https://devfeed.tech/tags/eos.md>), [neuron](<https://devfeed.tech/tags/neuron.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [torch](<https://devfeed.tech/tags/torch.md>)

### AI overview

TorchNeuron adds a native PyTorch backend for AWS Trainium, supporting eager execution, torch.compile, and distributed APIs. Hugging Face describes how its libraries can use the backend for model training and inference, with examples covering text generation, fine-tuning, and image generation. The article says TorchNeuron is planned for public beta starting October 20, 2026.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Introducing Reflection Beam: an open-weight model for agentic coding on AWS

DevFeed: [Introducing Reflection Beam: an open-weight model for agentic coding on AWS](<https://devfeed.tech/articles/introducing-reflection-beam-an-open-weight-model-for-agentic-coding-on-aws-88891.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/introducing-reflection-beam-an-open-weight-model-for-agentic-coding-on-aws/>)

Author: Clement Perrot

Published: 2026-10-16T15:23:10Z

Content type: tutorial

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Amazon Web Services](<https://devfeed.tech/topics/aws.md>), [lunar foundation model](<https://devfeed.tech/topics/lunar-foundation-model.md>), [Amazon SageMaker](<https://devfeed.tech/topics/amazon-sagemaker.md>), [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>), [Amazon Elastic Kubernetes Service](<https://devfeed.tech/topics/amazon-elastic-kubernetes-service.md>), [flux](<https://devfeed.tech/topics/flux.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [Apache Cassandra](<https://devfeed.tech/topics/cassandra.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [agentic-coding](<https://devfeed.tech/tags/agentic-coding.md>), [ai](<https://devfeed.tech/tags/ai.md>), [amazon-sagemaker-ai](<https://devfeed.tech/tags/amazon-sagemaker-ai.md>), [amazon-sagemaker-hyperpod](<https://devfeed.tech/tags/amazon-sagemaker-hyperpod.md>), [amazon-sagemaker-jumpstart](<https://devfeed.tech/tags/amazon-sagemaker-jumpstart.md>), [api](<https://devfeed.tech/tags/api.md>), [authentication](<https://devfeed.tech/tags/authentication.md>), [autoscaling](<https://devfeed.tech/tags/autoscaling.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-cli](<https://devfeed.tech/tags/aws-cli.md>), [aws-command-line-interface-aws-cli](<https://devfeed.tech/tags/aws-command-line-interface-aws-cli.md>), [aws-identity-and-access-management-iam](<https://devfeed.tech/tags/aws-identity-and-access-management-iam.md>), [batch](<https://devfeed.tech/tags/batch.md>), [batching](<https://devfeed.tech/tags/batching.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-infrastructure](<https://devfeed.tech/tags/gpu-infrastructure.md>), [inference](<https://devfeed.tech/tags/inference.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [over](<https://devfeed.tech/tags/over.md>), [sglang](<https://devfeed.tech/tags/sglang.md>), [technical-how-to](<https://devfeed.tech/tags/technical-how-to.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

### AI overview

Reflection AI's Beam is an open-weight model for agentic coding and long-horizon software tasks. The tutorial explains deployment options on AWS, from a managed SageMaker JumpStart endpoint to Kubernetes-based SageMaker HyperPod Inference using JumpStart, SGLang, or vLLM. It covers GPU capacity planning, deployment setup, performance measurement, tuning, and cleanup.

### Source excerpt

Reflection AI's Beam is an open-weight model for agentic coding and long-horizon tasks. This post shows how to deploy Beam on AWS: start with an Amazon SageMaker JumpStart real-time endpoint, or use Amazon SageMaker HyperPod Inference with JumpStart, SGLang, or vLLM for Kubernetes-native control over distributed GPU infrastructure.

[Next page](<https://devfeed.tech/topics/aws-trainium.md?cursor=WyIyMDI2LTEwLTE2VDE1OjIzOjEwKzAwOjAwIiwgIjkzNjYyYzY3LWRiM2EtNDgxZi1hOWVlLTcxYjFlMTQ3ZTgxMyJd>)