# AWS Inferentia

AWS Inferentia is an AWS-designed chip family for high-performance, cost-efficient deep-learning and generative-AI inference in Amazon EC2, supported by the AWS Neuron SDK and frameworks such as PyTorch and TensorFlow.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Exa boosts embedding throughput by 50% to index billions of documents on AWS AI chips

DevFeed: [Exa boosts embedding throughput by 50% to index billions of documents on AWS AI chips](<https://devfeed.tech/articles/exa-boosts-embedding-throughput-by-50-to-index-billions-of-documents-on-aws-ai-chips-141408.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/exa-boosts-embedding-throughput-by-50-to-index-billions-of-documents-on-aws-ai-chips/>)

Author: Summer Devlin

Published: 2026-11-18T23:58:22Z

Content type: news

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [AWS Inferentia](<https://devfeed.tech/topics/aws-inferentia.md>), [AWS AI chips](<https://devfeed.tech/topics/aws-ai-chips.md>), [neuron](<https://devfeed.tech/topics/neuron.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [search-engines](<https://devfeed.tech/topics/search-engines.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-chips](<https://devfeed.tech/tags/ai-chips.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-ai-chips](<https://devfeed.tech/tags/aws-ai-chips.md>), [aws-inferentia](<https://devfeed.tech/tags/aws-inferentia.md>), [aws-neuron](<https://devfeed.tech/tags/aws-neuron.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [cost](<https://devfeed.tech/tags/cost.md>), [custom](<https://devfeed.tech/tags/custom.md>), [customer-solutions](<https://devfeed.tech/tags/customer-solutions.md>), [database](<https://devfeed.tech/tags/database.md>), [economics](<https://devfeed.tech/tags/economics.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [index](<https://devfeed.tech/tags/index.md>), [inference](<https://devfeed.tech/tags/inference.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [neuron](<https://devfeed.tech/tags/neuron.md>), [optimized](<https://devfeed.tech/tags/optimized.md>), [performance](<https://devfeed.tech/tags/performance.md>), [search](<https://devfeed.tech/tags/search.md>), [throughput](<https://devfeed.tech/tags/throughput.md>), [unit-economics](<https://devfeed.tech/tags/unit-economics.md>), [vector](<https://devfeed.tech/tags/vector.md>)

### AI overview

Exa reports a 47% improvement in performance and cost per million documents embedded on AWS Neuron hardware versus its initial baseline. Starting with FP16 data parallelism on Inferentia2, it optimized weight layout and precision, used a drop-in attention kernel, and built a fused SwiGLU MLP kernel to reduce memory spill. Custom attention kernels tailored to the model's shapes then cut padding work and instruction steps. Each change passed a correctness gate of 0.999 cosine similarity to the reference; performance was measured with a 30-minute saturation benchmark. The workload uses one NeuronCore per pod, and the same approach scales through data parallelism on Trainium2. Exa says tuning across hardware and Regions helps its scheduler match shifting demand and availability. The excerpt describes these results and methods but does not provide absolute throughput or cost figures.

### Source excerpt

Exa's neural search engine is backed by an in-house vector database, so the unit economics of embedding tens of billions of web documents drive the business. This post shares how Exa optimized embedding inference on AWS Neuron hardware (Inferentia2 and Trainium2) with custom NKI kernels, reaching a 47% gain in performance and cost per million documents.

## High-quality cost-effective AI image generation on Amazon ECS

DevFeed: [High-quality cost-effective AI image generation on Amazon ECS](<https://devfeed.tech/articles/high-quality-cost-effective-ai-image-generation-on-amazon-ecs-136885.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/containers/high-quality-cost-effective-ai-image-generation-on-amazon-ecs/>)

Author: Nick Biso

Published: 2026-11-17T00:19:06Z

Content type: article

Language: en

Sources: [Containers](<https://devfeed.tech/sources/containers.md>)

Topics: [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [ai-image-generator](<https://devfeed.tech/topics/ai-image-generator.md>), [AWS Inferentia](<https://devfeed.tech/topics/aws-inferentia.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>), [Amazon EC2](<https://devfeed.tech/topics/amazon-ec2.md>), [Amazon Web Services](<https://devfeed.tech/topics/aws.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [ai-image-generation](<https://devfeed.tech/tags/ai-image-generation.md>), [amazon-ec2](<https://devfeed.tech/tags/amazon-ec2.md>), [amazon-elastic-container-service](<https://devfeed.tech/tags/amazon-elastic-container-service.md>), [auto-scaling](<https://devfeed.tech/tags/auto-scaling.md>), [ci-cd](<https://devfeed.tech/tags/ci-cd.md>), [cost](<https://devfeed.tech/tags/cost.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [ecs](<https://devfeed.tech/tags/ecs.md>), [technical-how-to](<https://devfeed.tech/tags/technical-how-to.md>)

### AI overview

The described Amazon ECS service runs Z-Image-Turbo on AWS Inferentia2, with one service per image resolution and one request at a time per instance because inference saturates the accelerator. Its key deployment mechanism is compiling model artifacts once in CI/CD, caching them on Amazon EFS, and loading them into containers at startup; the post reports this cuts startup to about two minutes from over 30 minutes and avoids recompilation when changes do not affect the model graph. The excerpt estimates $0.003 per 512px image and $0.011 per 1024px image at $1.97/hour for an inf2.8xlarge in us-east-1, while noting costs vary with utilization and other AWS resources still incur charges. It also describes resolution-based routing and streaming intermediate image frames over WebSockets. Deployment requires AWS quotas for Inf2 instances, CDK bootstrap, and listed local tools; first deployment takes about 45 minutes, including compilation.

### Source excerpt

Learn how to run Z-Image-Turbo, a 6-billion-parameter diffusion transformer, on AWS Inferentia2 and Amazon ECS to generate high-quality images at a fraction of a cent each, with auto-scaling, arbitrary resolution support, and a complete CI/CD pipeline.

## Scale distributed AI workloads with Ray and Anyscale on Amazon EC2

DevFeed: [Scale distributed AI workloads with Ray and Anyscale on Amazon EC2](<https://devfeed.tech/articles/scale-distributed-ai-workloads-with-ray-and-anyscale-on-amazon-ec2-132423.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/architecture/scale-distributed-ai-workloads-with-ray-and-anyscale-on-amazon-ec2/>)

Author: Sindhura Palakodety

Published: 2026-11-12T20:51:20Z

Content type: article

Language: en

Sources: [AWS Architecture Blog](<https://devfeed.tech/sources/aws-architecture-blog.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [distributed-training](<https://devfeed.tech/topics/distributed-training.md>), [Amazon EC2](<https://devfeed.tech/topics/amazon-ec2.md>), [Graviton](<https://devfeed.tech/topics/graviton.md>), [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [AWS Inferentia](<https://devfeed.tech/topics/aws-inferentia.md>), [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-chips](<https://devfeed.tech/tags/ai-chips.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [ai-ml](<https://devfeed.tech/tags/ai-ml.md>), [ai-workloads](<https://devfeed.tech/tags/ai-workloads.md>), [amazon-api-gateway](<https://devfeed.tech/tags/amazon-api-gateway.md>), [amazon-cloudwatch](<https://devfeed.tech/tags/amazon-cloudwatch.md>), [amazon-ec2](<https://devfeed.tech/tags/amazon-ec2.md>), [aws](<https://devfeed.tech/tags/aws.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [intermediate-200](<https://devfeed.tech/tags/intermediate-200.md>), [ml](<https://devfeed.tech/tags/ml.md>), [models](<https://devfeed.tech/tags/models.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [pipelines](<https://devfeed.tech/tags/pipelines.md>), [production](<https://devfeed.tech/tags/production.md>), [prototype](<https://devfeed.tech/tags/prototype.md>), [ray](<https://devfeed.tech/tags/ray.md>), [scale](<https://devfeed.tech/tags/scale.md>), [scaling](<https://devfeed.tech/tags/scaling.md>), [thought-leadership](<https://devfeed.tech/tags/thought-leadership.md>)

### AI overview

Ray and Anyscale provide a managed way to scale AI/ML pipelines on AWS while keeping Ray workload code consistent from a single instance to distributed clusters. Amazon EC2 offers the most flexibility, Amazon EKS fits organizations already operating Kubernetes but adds orchestration complexity and operator configuration, and SageMaker HyperPod targets resilience-critical large training runs with health monitoring, node replacement, and checkpoint recovery. Anyscale manages cluster lifecycle and autoscaling; compute and data remain in the customer's AWS account. The text describes matching CPU, GPU, Trainium, and Inferentia resources to pipeline stages, using Spot for fault-tolerant batch work, and combining Savings Plans, Capacity Blocks, reservations, and On-Demand capacity. Its suggested progression is to validate locally, scale data processing, expand training on a chosen deployment path, then deploy inference; it says this can be done without workload code changes.

### Source excerpt

Building multimodal AI/ML pipelines on Amazon EC2 means scaling distributed compute, controlling costs across stages, and keeping up with new models and hardware. This post shows how the Ray-powered Anyscale application on AWS helps you deploy Ray across three compute options and scale from prototype to production without code changes.

## Fast by Default: Compile-Time Heuristics for Helion Kernels

DevFeed: [Fast by Default: Compile-Time Heuristics for Helion Kernels](<https://devfeed.tech/articles/fast-by-default-compile-time-heuristics-for-helion-kernels-131024.md>)

Original publisher: [Read original article](<https://pytorch.org/blog/fast-by-default-compile-time-heuristics-for-helion-kernels/>)

Author: Caleb Min Sup Kim, Ethan Che, Jongsok Choi, Jason Ansel

Published: 2026-11-12T16:45:26Z

Content type: article

Language: en

Sources: [PyTorch](<https://devfeed.tech/sources/blog-pytorch.md>)

Topics: [kernels](<https://devfeed.tech/topics/kernels.md>), [CUDA Tile](<https://devfeed.tech/topics/cuda-tile.md>), [AWS Inferentia](<https://devfeed.tech/topics/aws-inferentia.md>)

Tags: [analysis](<https://devfeed.tech/tags/analysis.md>), [analyze](<https://devfeed.tech/tags/analyze.md>), [aot](<https://devfeed.tech/tags/aot.md>), [arithmetic](<https://devfeed.tech/tags/arithmetic.md>), [assembler](<https://devfeed.tech/tags/assembler.md>), [attention](<https://devfeed.tech/tags/attention.md>), [axis](<https://devfeed.tech/tags/axis.md>), [backend](<https://devfeed.tech/tags/backend.md>), [backends](<https://devfeed.tech/tags/backends.md>), [batch](<https://devfeed.tech/tags/batch.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [blackwell](<https://devfeed.tech/tags/blackwell.md>), [block](<https://devfeed.tech/tags/block.md>), [blog](<https://devfeed.tech/tags/blog.md>), [dataflow](<https://devfeed.tech/tags/dataflow.md>), [helion](<https://devfeed.tech/tags/helion.md>), [ir](<https://devfeed.tech/tags/ir.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>), [reduction](<https://devfeed.tech/tags/reduction.md>)

### AI overview

Helion's compile-time heuristics analyze kernel IR and dataflow to choose settings such as block sizes, warps, and pipeline stages without benchmarking candidates. They classify kernels as matmul, reduction, or pointwise and estimate register, shared-memory, and tensor-memory use to guide choices; unknown compile-time extents still fall back to Helion's prior default. On the reported NVIDIA benchmarks, the heuristic improved results over that default: 1.18x handwritten Flash Linear Attention performance, 1.27x vLLM CUDA performance for reduction kernels, and 1.09x torch.compile max-autotune on a reduction suite. It also provided strong initial configs for autotuning, though seeded and unseeded searches reached nearly equal final results. The estimates are approximate, results include a jagged HSTU attention regression, and the heuristics are enabled for H100 and B200 on Triton; other backends remain unevaluated.

### Source excerpt

TL;DR Helion now uses compile-time heuristics that analyze a kernel's IR and dataflow to pick fast settings without slow autotuning. It classifies the kernel as matmul, reduction, or pointwise and... The post Fast by Default: Compile-Time Heuristics for Helion Kernels appeared first on PyTorch.

## Expensive accelerators, no workload-level visibility: Introducing Amazon CloudWatch accelerated compute insights

DevFeed: [Expensive accelerators, no workload-level visibility: Introducing Amazon CloudWatch accelerated compute insights](<https://devfeed.tech/articles/expensive-accelerators-no-workload-level-visibility-introducing-amazon-cloudwatch-accelerated-compute-insights-129501.md>)

Original publisher: [Read original article](<https://thenewstack.io/cloudwatch-accelerated-compute-insights/>)

Author: Sabari Sawant

Published: 2026-11-11T18:00:00Z

Content type: article

Language: en

Sources: [Kubernetes Overview, News and Trends | The New Stack](<https://devfeed.tech/sources/kubernetes-overview-news-and-trends-the-new-stack.md>), [The New Stack](<https://devfeed.tech/sources/the-new-stack.md>)

Topics: [Amazon CloudWatch](<https://devfeed.tech/topics/amazon-cloudwatch.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [AWS Inferentia](<https://devfeed.tech/topics/aws-inferentia.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>)

Tags: [accelerators](<https://devfeed.tech/tags/accelerators.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [amazon-cloudwatch](<https://devfeed.tech/tags/amazon-cloudwatch.md>), [amazon-eks](<https://devfeed.tech/tags/amazon-eks.md>), [automated-remediation](<https://devfeed.tech/tags/automated-remediation.md>), [aws-inferentia](<https://devfeed.tech/tags/aws-inferentia.md>), [aws-marketplace](<https://devfeed.tech/tags/aws-marketplace.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [observability](<https://devfeed.tech/tags/observability.md>), [post-contributed](<https://devfeed.tech/tags/post-contributed.md>), [sponsor-aws-marketplace](<https://devfeed.tech/tags/sponsor-aws-marketplace.md>), [sponsored-post-contributed](<https://devfeed.tech/tags/sponsored-post-contributed.md>)

### AI overview

Amazon CloudWatch accelerated compute insights adds device-level GPU and Inferentia visibility for workloads on Amazon EKS, connecting accelerator health and utilization to Kubernetes workload metadata, estimated waste, and remediation guidance. It supports NVIDIA GPUs, AWS Trainium, and AWS Inferentia; GPU and Inferentia metrics are normalized into a shared health view, with MIG instances and other partitioning schemes recognized. It flags oversized instances, low training tensor-core utilization against a workload's historical baseline, and inference throughput below hardware capacity. Its cost figures are operational estimates, not billing or chargeback records; AWS Cost Explorer and the Cost and Usage Report remain the authoritative sources. Setup requires CloudWatch Observability EKS add-on v7.0.1 or later for self-hosted inference, with OTel Container Insights enabled. Telemetry can be viewed in CloudWatch Omni for workload-level troubleshooting or in the CloudWatch console for fleet-wide views. AWS DevOps Agent can investigate health events and propose actions for approval; autonomous remediation is optional.

### Source excerpt

Accelerator bills have become one of the largest lines in the infrastructure budget. GPUs have become the most expensive resource The post Expensive accelerators, no workload-level visibility: Introducing Amazon CloudWatch accelerated compute insights appeared first on The New Stack.

## Amazon's TorchNeuron aims to make PyTorch apps easier to run on Trainium

DevFeed: [Amazon's TorchNeuron aims to make PyTorch apps easier to run on Trainium](<https://devfeed.tech/articles/amazon-is-taking-its-biggest-shot-at-nvidia-yet-with-new-torchneuron-dev-tool-90512.md>)

Original publisher: [Read original article](<https://techcrunch.com/2026/10/19/amazon-is-taking-its-biggest-shot-at-nvidia-yet-with-new-torchneuron-dev-tool/>)

Author: Julie Bort

Published: 2026-10-19T20:09:24Z

Content type: news

Language: en

Sources: [TechCrunch | AI News & Artificial Intelligence](<https://devfeed.tech/sources/techcrunch-ai-news-artificial-intelligence.md>)

Topics: [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [AWS Inferentia](<https://devfeed.tech/topics/aws-inferentia.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [amazon](<https://devfeed.tech/topics/amazon.md>), [groq](<https://devfeed.tech/topics/groq.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [amazon](<https://devfeed.tech/tags/amazon.md>), [amazon-web-services](<https://devfeed.tech/tags/amazon-web-services.md>), [aws](<https://devfeed.tech/tags/aws.md>), [coreweave](<https://devfeed.tech/tags/coreweave.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [deepgram](<https://devfeed.tech/tags/deepgram.md>), [elevenlabs](<https://devfeed.tech/tags/elevenlabs.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>), [startups](<https://devfeed.tech/tags/startups.md>), [torchneuron](<https://devfeed.tech/tags/torchneuron.md>), [trainium](<https://devfeed.tech/tags/trainium.md>)

### AI overview

Amazon Web Services released TorchNeuron, a tool intended to let developers run PyTorch AI applications on its Trainium chips with a one-line configuration change. The article describes Amazon's effort to reduce PyTorch's reliance on Nvidia's CUDA ecosystem, its plans to contribute hardware-agnostic changes upstream, and AWS's claims about cost and production infrastructure. Whether this gives developers more choice or adds configuration work remains uncertain.

### Source excerpt

Amazon wants to break Nvidia's hold on PyTorch, the free toolkit used to build most AI apps, so that more developers run their apps on alternative chips like its own.

## Push-Based KV Transfer to Improve TTFT and TPOT for Disaggregated Inference in vLLM

DevFeed: [Push-Based KV Transfer to Improve TTFT and TPOT for Disaggregated Inference in vLLM](<https://devfeed.tech/articles/push-based-kv-transfer-to-improve-ttft-and-tpot-for-disaggregated-inference-in-vllm-87122.md>)

Original publisher: [Read original article](<https://vllm.ai/blog/2026-10-14-nixl-push-connector>)

Author: Sunita Nadampalli (AWS), Nicolò Lucchesi (Mistral)

Published: 2026-10-14T00:00:00Z

Content type: article

Language: en

Sources: [vLLM Blog](<https://devfeed.tech/sources/vllm-blog.md>)

Topics: [vllm](<https://devfeed.tech/topics/vllm.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [gpudirect rdma](<https://devfeed.tech/topics/gpudirect-rdma.md>), [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [AWS Inferentia](<https://devfeed.tech/topics/aws-inferentia.md>), [quantization](<https://devfeed.tech/topics/quantization.md>)

Tags: [ai-engineering](<https://devfeed.tech/tags/ai-engineering.md>), [amazon-ec2](<https://devfeed.tech/tags/amazon-ec2.md>), [aws-ec2](<https://devfeed.tech/tags/aws-ec2.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [disaggregation](<https://devfeed.tech/tags/disaggregation.md>), [inference](<https://devfeed.tech/tags/inference.md>), [kv-cache](<https://devfeed.tech/tags/kv-cache.md>), [llm-inference](<https://devfeed.tech/tags/llm-inference.md>), [model-serving](<https://devfeed.tech/tags/model-serving.md>), [nixl](<https://devfeed.tech/tags/nixl.md>), [open-source-ai](<https://devfeed.tech/tags/open-source-ai.md>), [performance](<https://devfeed.tech/tags/performance.md>), [rdma](<https://devfeed.tech/tags/rdma.md>), [vllm](<https://devfeed.tech/tags/vllm.md>), [vllm-blog](<https://devfeed.tech/tags/vllm-blog.md>)

### AI overview

The article introduces a push-based NIXL connector for vLLM disaggregated inference. It overlaps KV-cache transfer setup with prefill, moving work off the decode instance's latency-critical path. In a Qwen3-32B benchmark on a one-prefill, one-decode deployment using Amazon EC2 P5en instances with EFA, push mode achieved up to 3x better time to first token and up to 30% better time per output token than pull mode; results vary by workload and configuration.

### Source excerpt

A push-based NIXL connector for vLLM disaggregated serving: the prefill instance writes KV directly into decode memory over RDMA, taking the router relay, decode-side allocation, and transfer setup off the TTFT critical path.

## Amazon In Talks To Buy AI Startup Decart For Around $7 Billion

DevFeed: [Amazon In Talks To Buy AI Startup Decart For Around $7 Billion](<https://devfeed.tech/articles/amazon-in-talks-to-buy-ai-startup-decart-for-around-7-billion-85582.md>)

Original publisher: [Read original article](<https://hardware.slashdot.org/story/26/10/12/0059238/amazon-in-talks-to-buy-ai-startup-decart-for-around-7-billion>)

Author: BeauHD

Published: 2026-10-12T16:00:00Z

Content type: news

Language: en

Sources: [Slashdot](<https://devfeed.tech/sources/slashdot.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [AWS Inferentia](<https://devfeed.tech/topics/aws-inferentia.md>), [AWS AI chips](<https://devfeed.tech/topics/aws-ai-chips.md>), [World models](<https://devfeed.tech/topics/world-models.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [ai-startup](<https://devfeed.tech/tags/ai-startup.md>), [aws](<https://devfeed.tech/tags/aws.md>), [business](<https://devfeed.tech/tags/business.md>), [decart](<https://devfeed.tech/tags/decart.md>), [trainium](<https://devfeed.tech/tags/trainium.md>)

### AI overview

Amazon is reportedly in talks to acquire AI startup Decart for up to $7 billion. Decart's DOS software helps developers optimize AI models for different processors, potentially shortening the process from months to weeks and allowing workloads to shift when requirements or chip availability change. Decart also develops the Lucy and Oasis world models for generating simulated 3D environments.

### Source excerpt

Amazon is reportedly in talks to acquire AI startup Decart in a deal that could value the company at up to $7 billion. Decart's DOS software helps developers optimize AI models for different processors in weeks rather than months, a capability that could be particularly useful to AWS as it pushes its own Trainium chips. SiliconANGLE reports: The company has developed software called DOS, which can help developers to speed up AI training and inference workloads by optimizing their neural networks to run on a specific processor. In most AI workloads, developers must optimize a neural network for the chip they plan to run it on. This process used to take months, but with DOS, the workflow necessary to do this can be compressed into just a couple of weeks. As a result, companies can use Decart's software to shift AI workloads to different processors if their requirements change, or if chip availability becomes an issue. In addition to DOS, Decart is also a model developer, responsible for creating the Lucy and Oasis world models, which are designed to generate and simulate realistic three-dimensional environments for video games, robotic training and other use cases. It's still possible another suitor could emerge and outbid Amazon. Last month, Anthropic PBC was reportedly engaged in talks to buy Decart, according to a report by Bloomberg. However, it ultimately pulled out of a reported $6 billion deal after conducting due diligence on the startup. It's not clear what ultimately dissuaded Anthropic from pursuing a deal. The Wall Street Journal first reported (paywalled) the news. Read more of this story at Slashdot.

## Track GPU spend across your team with Cost Centers

DevFeed: [Track GPU spend across your team with Cost Centers](<https://devfeed.tech/articles/track-gpu-spend-across-your-team-with-cost-centers-78156.md>)

Original publisher: [Read original article](<https://www.runpod.io/blog/track-gpu-spend-across-your-team-with-cost-centers>)

Author: Brendan McKeag

Published: 2026-09-13T15:06:30Z

Content type: tutorial

Language: en

Sources: [Runpod Blog.](<https://devfeed.tech/sources/runpod-blog.md>)

Topics: [cloud cost management](<https://devfeed.tech/topics/cloud-cost-management.md>), [AWS Inferentia](<https://devfeed.tech/topics/aws-inferentia.md>)

Tags: [ai-workloads](<https://devfeed.tech/tags/ai-workloads.md>), [cost](<https://devfeed.tech/tags/cost.md>), [data-center](<https://devfeed.tech/tags/data-center.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [let](<https://devfeed.tech/tags/let.md>), [platform](<https://devfeed.tech/tags/platform.md>), [team](<https://devfeed.tech/tags/team.md>)

### AI overview

Runpod cost centers group billable resources such as Pods, Serverless endpoints, Network Volumes, and Clusters so teams can see monthly GPU spend by group on their invoices. The article explains how to create cost centers, assign resources, review uncategorized charges, and account for billing-period timing and data delays.

### Source excerpt

Runpod's platform continues to evolve, and now, so does your ability to keep tabs on where your GPU dollars are going. We've rolled out cost centers, a.