# Sambanova

Find what you need to accelerate your AI journey

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## The Air-Cooled Token Factory: Intel GPU D-Series + SambaRack SN50

DevFeed: [The Air-Cooled Token Factory: Intel GPU D-Series + SambaRack SN50](<https://devfeed.tech/articles/the-air-cooled-token-factory-intel-gpu-d-series-sambarack-sn50-135574.md>)

Original publisher: [Read original article](<https://sambanova.ai/blog/the-air-cooled-token-factory-intel-gpu-d-series-sambarack-sn50>)

Author: SambaNova | SuperComputing

Published: 2026-11-16T04:47:57Z

Content type: article

Language: en

Sources: [Sambanova](<https://devfeed.tech/sources/sambanova.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [Language models](<https://devfeed.tech/topics/language-models.md>), [intel](<https://devfeed.tech/topics/intel.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [ai](<https://devfeed.tech/tags/ai.md>), [air](<https://devfeed.tech/tags/air.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [blog](<https://devfeed.tech/tags/blog.md>), [brownfield](<https://devfeed.tech/tags/brownfield.md>), [data-centers](<https://devfeed.tech/tags/data-centers.md>), [decode](<https://devfeed.tech/tags/decode.md>), [deploy](<https://devfeed.tech/tags/deploy.md>), [economics](<https://devfeed.tech/tags/economics.md>), [gpu-d-series](<https://devfeed.tech/tags/gpu-d-series.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-workloads](<https://devfeed.tech/tags/inference-workloads.md>), [intel](<https://devfeed.tech/tags/intel.md>), [optimize](<https://devfeed.tech/tags/optimize.md>), [prefill](<https://devfeed.tech/tags/prefill.md>), [sambanova](<https://devfeed.tech/tags/sambanova.md>), [sn50](<https://devfeed.tech/tags/sn50.md>), [speed](<https://devfeed.tech/tags/speed.md>), [supercomputing](<https://devfeed.tech/tags/supercomputing.md>), [technology](<https://devfeed.tech/tags/technology.md>), [token-factory](<https://devfeed.tech/tags/token-factory.md>)

### AI overview

The described system separates inference by phase: Intel's air-cooled 350 W D-Series GPUs handle prompt prefill and KV-cache construction, while SambaNova SN50 RDUs handle token decode. In the demonstration, four D-Series GPUs are paired with SambaRack SN50 running GLM-5.3. NVIDIA Dynamo routes requests with cache awareness; vLLM serves both paths, and NIXL transfers KV cache over UCX/RDMA on standard RoCEv2 Ethernet. The design aims to reuse cached prefixes, reduce repeated prefill and KV transfers, and let providers scale GPU prefill or SN50 decode capacity independently. The article says SN50 racks typically use 20 kW and argues the air-cooled configuration could fit existing brownfield data centers, avoiding some liquid-cooling retrofit needs. These are described capabilities and deployment aims; the excerpt provides no quantitative performance results for the demonstrated setup.

### Source excerpt

Over the last several months, many customers have been evaluating and deploying disaggregated inference workloads to help optimize the speed and economics of AI. At SuperComputing 2026, SambaNova and Intel are excited to demonstrate how we are advancing a heterogeneous inference architecture that pairs Intel GPU D-Series for prefill with SambaNova SN50 RDUs for decode. The end result is a solution that delivers an air-cooled Token Factory, so that each phase of inference runs on the hardware best suited to it. Below is a recorded demonstration of the first system running GLM-5.3 on both hardware stacks in an air-cooled facility. As we will explore in this blog, customers will be able to deploy this solution in existing brownfield data centers, which will become increasingly important over the next couple years.

## GLM-5.3 Now Running on SambaCloud

DevFeed: [GLM-5.3 Now Running on SambaCloud](<https://devfeed.tech/articles/glm-5-3-now-running-on-sambacloud-129327.md>)

Original publisher: [Read original article](<https://sambanova.ai/blog/glm-5.3-now-running-on-sambacloud>)

Author: SambaNova

Published: 2026-11-11T18:16:24Z

Content type: release

Language: en

Sources: [Sambanova](<https://devfeed.tech/sources/sambanova.md>)

Topics: [agentic-coding](<https://devfeed.tech/topics/agentic-coding.md>), [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [agentic workflows](<https://devfeed.tech/topics/agentic-workflows.md>), [AI-assisted coding](<https://devfeed.tech/topics/ai-assisted-coding.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Frontier AI](<https://devfeed.tech/topics/frontier-ai.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agent-tasks](<https://devfeed.tech/tags/agent-tasks.md>), [agentic-coding](<https://devfeed.tech/tags/agentic-coding.md>), [agentic-workloads](<https://devfeed.tech/tags/agentic-workloads.md>), [blog](<https://devfeed.tech/tags/blog.md>), [complex](<https://devfeed.tech/tags/complex.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [glm-5-3](<https://devfeed.tech/tags/glm-5-3.md>), [open-weight-models](<https://devfeed.tech/tags/open-weight-models.md>), [software-engineering](<https://devfeed.tech/tags/software-engineering.md>), [terminal-bench](<https://devfeed.tech/tags/terminal-bench.md>), [tokens](<https://devfeed.tech/tags/tokens.md>)

### AI overview

GLM-5.3 is presented as an open-weight model for long-running coding and agent tasks, with reported scores of 28.3 on Terminal-Bench 3.0 and 66.9 on DeepSWE v1.1. The article also reports 31.4% on Z.ai's internal Code Bench at about 50K output tokens per task, versus Claude Opus 4.8's 29.5% at 120K, and cites strong CyberGym and cybersecurity assessments. SambaCloud offers the model as "GLM-5.3" at over 200 tokens/s; an example agent trace has 41 calls over 17.5 minutes, with more than 91% of input served from cache. The excerpt includes Python API setup and provider configuration for ZCode, Claude Code, OpenCode, and Codex CLI; Codex requires the Responses API and supported reasoning levels in its model catalog.

### Source excerpt

GLM-5.3 is Z.ai's flagship model for complex software engineering and long-horizon agent tasks. It reaches state-of-the-art results among open-weight models on Terminal-Bench 3.0 and Agents' Last Exam, and it does so while using fewer output tokens per task than several closed frontier models. GLM-5.3 is now available on SambaCloud at over 200 tokens/s under the model string GLM-5.3. Read Z.ai's full GLM-5.3 announcement here.

## Supermicro Will Build and Sell SambaNova-Powered Inference Systems

DevFeed: [Supermicro Will Build and Sell SambaNova-Powered Inference Systems](<https://devfeed.tech/articles/supermicro-will-build-and-sell-sambanova-powered-inference-systems-114232.md>)

Original publisher: [Read original article](<https://sambanova.ai/blog/supermicro-will-build-and-sell-sambanova-powered-inference-systems>)

Author: SambaNova

Published: 2026-11-03T14:02:19Z

Content type: release

Language: en

Sources: [Sambanova](<https://devfeed.tech/sources/sambanova.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [Agentic AI](<https://devfeed.tech/topics/what-is-agentic-ai.md>), [systems](<https://devfeed.tech/topics/systems.md>)

Tags: [acceleration](<https://devfeed.tech/tags/acceleration.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [batch](<https://devfeed.tech/tags/batch.md>), [blog](<https://devfeed.tech/tags/blog.md>), [chips](<https://devfeed.tech/tags/chips.md>), [data-centers](<https://devfeed.tech/tags/data-centers.md>), [enterprises](<https://devfeed.tech/tags/enterprises.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [inference](<https://devfeed.tech/tags/inference.md>), [integrate](<https://devfeed.tech/tags/integrate.md>), [neoclouds](<https://devfeed.tech/tags/neoclouds.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-gpus](<https://devfeed.tech/tags/nvidia-gpus.md>), [running](<https://devfeed.tech/tags/running.md>), [supermicro](<https://devfeed.tech/tags/supermicro.md>), [support](<https://devfeed.tech/tags/support.md>), [systems](<https://devfeed.tech/tags/systems.md>), [world](<https://devfeed.tech/tags/world.md>)

### AI overview

Supermicro will integrate, validate, deliver, and support inference systems pairing NVIDIA GPUs with SambaNova SN50 RDUs, with SambaNova engineers helping tune models for SN50 deployments. The design assigns prompt prefill, described as compute-bound, to GPUs and token-by-token decode, described as memory-bound, to RDUs; the source says this combination can improve speed and efficiency for agentic workloads, which spend much of their time in decode. SN50 systems are presented as deployable in existing data centers without extremely high rack power densities. Rollout begins with 10,000 SN50 chips, first for NeoClouds and sovereign clouds, then for enterprises running AI in their own data centers. SambaNova expects the partnership to enable more high-performance inference capacity in 2027 than otherwise.

### Source excerpt

Supermicro will integrate, validate, deliver, and support disaggregated inference systems that pair NVIDIA GPUs with SambaNova SN50 RDUs for agentic AI inference in data centers around the world. The systems will roll out in phases, beginning with an initial batch of 10,000 SN50 chips. They will be distributed first to NeoClouds and sovereign clouds, then expand to enterprises running AI in their own data centers.

## MiniMax M3.1 Flash Now Available on SambaCloud

DevFeed: [MiniMax M3.1 Flash Now Available on SambaCloud](<https://devfeed.tech/articles/minimax-m3-1-flash-now-available-on-sambacloud-98650.md>)

Original publisher: [Read original article](<https://sambanova.ai/blog/minimax-m3.1-flash-now-available-on-sambacloud>)

Author: SambaNova

Published: 2026-10-24T03:19:34Z

Content type: release

Language: en

Sources: [Sambanova](<https://devfeed.tech/sources/sambanova.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [agentic-coding](<https://devfeed.tech/topics/agentic-coding.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [Cybersecurity](<https://devfeed.tech/topics/cybersecurity.md>), [AI-assisted coding](<https://devfeed.tech/topics/ai-assisted-coding.md>), [Loop Engineering](<https://devfeed.tech/topics/loop-engineering.md>), [Frontier AI](<https://devfeed.tech/topics/frontier-ai.md>)

Tags: [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-tasks](<https://devfeed.tech/tags/agentic-tasks.md>), [agents](<https://devfeed.tech/tags/agents.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [blog](<https://devfeed.tech/tags/blog.md>), [coding](<https://devfeed.tech/tags/coding.md>), [computer-use](<https://devfeed.tech/tags/computer-use.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cybergym](<https://devfeed.tech/tags/cybergym.md>), [cybersecurity](<https://devfeed.tech/tags/cybersecurity.md>), [deepswe-v1-1](<https://devfeed.tech/tags/deepswe-v1-1.md>), [faster](<https://devfeed.tech/tags/faster.md>), [inference](<https://devfeed.tech/tags/inference.md>), [long-running-agents](<https://devfeed.tech/tags/long-running-agents.md>), [m3-1-flash](<https://devfeed.tech/tags/m3-1-flash.md>), [minimax](<https://devfeed.tech/tags/minimax.md>), [minimax-m3-1-flash](<https://devfeed.tech/tags/minimax-m3-1-flash.md>), [model](<https://devfeed.tech/tags/model.md>), [osworld](<https://devfeed.tech/tags/osworld.md>), [osworld-v2](<https://devfeed.tech/tags/osworld-v2.md>), [performance](<https://devfeed.tech/tags/performance.md>)

### AI overview

MiniMax M3.1 Flash is available on SambaCloud at over 200 tokens/s, with the source reporting nearly 3x faster inference and 70% lower usage costs than its predecessor. It highlights results of 73.7% on DeepSWE v1.1, 61.7 on OSWorld V2, and 82% on CyberGym, alongside claims of strong performance in enterprise workflow automation, coding, computer use, and cybersecurity. The source describes DeepSWE as testing independent codebase work and validation, and OSWorld V2 as measuring computer operation across applications. MiniMax M3 requests on SambaCloud are being rerouted to M3.1 Flash; callers are asked to change the model string to `MiniMax-M3.1-Flash`. The supplied text includes a Python API example for making a chat completion request.

### Source excerpt

Frontier Agentic Performance at a Fraction of the Cost. Build Fast, Cheap, Long-Running Agents on SambaCloud MiniMax M3.1 Flash is the next-generation Flash model from MiniMax, built for agentic tasks, coding, computer use, and cybersecurity. It delivers a generational leap in capability over its predecessor while making inference nearly 3x faster and cutting usage costs by 70%. It hits 73.7% on DeepSWE v1.1, 61.7 on OSWorld V2, and 82% on CyberGym, placing it alongside top models on the benchmarks that matter for real-world agents.

## What Is an AI Factory? Definition & Architecture | SambaNova

DevFeed: [What Is an AI Factory? Definition & Architecture | SambaNova](<https://devfeed.tech/articles/what-is-an-ai-factory-definition-architecture-sambanova-96383.md>)

Original publisher: [Read original article](<https://sambanova.ai/blog/what-is-an-ai-factory>)

Author: SambaNova

Published: 2026-10-22T20:08:28Z

Content type: article

Language: en

Sources: [Sambanova](<https://devfeed.tech/sources/sambanova.md>)

Topics: [AI Factory](<https://devfeed.tech/topics/ai-factory.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>), [Sovereign AI](<https://devfeed.tech/topics/sovereign-ai.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [digital sovereignty](<https://devfeed.tech/topics/digital-sovereignty.md>), [Cloud Architecture](<https://devfeed.tech/topics/cloud-architecture.md>), [observability](<https://devfeed.tech/topics/observability.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-factory](<https://devfeed.tech/tags/ai-factory.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [ai-systems](<https://devfeed.tech/tags/ai-systems.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [blog](<https://devfeed.tech/tags/blog.md>), [compute](<https://devfeed.tech/tags/compute.md>), [data](<https://devfeed.tech/tags/data.md>), [deploy](<https://devfeed.tech/tags/deploy.md>), [develop](<https://devfeed.tech/tags/develop.md>), [experimentation](<https://devfeed.tech/tags/experimentation.md>), [governance](<https://devfeed.tech/tags/governance.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [models](<https://devfeed.tech/tags/models.md>), [optimize](<https://devfeed.tech/tags/optimize.md>), [organizations](<https://devfeed.tech/tags/organizations.md>), [production](<https://devfeed.tech/tags/production.md>), [scale](<https://devfeed.tech/tags/scale.md>), [structured](<https://devfeed.tech/tags/structured.md>), [system](<https://devfeed.tech/tags/system.md>), [systems](<https://devfeed.tech/tags/systems.md>), [technology](<https://devfeed.tech/tags/technology.md>)

### AI overview

An AI factory is an integrated production system combining data, models, compute, software automation, monitoring, and governance to train, deploy, and improve AI at scale. A token factory focuses on inference and is judged by throughput, latency, energy and cost per token, and utilization. The text describes cloud-hosted, self-hosted, and provider-operated approaches: cloud hosting avoids owning hardware, while self-hosting offers greater control but requires expertise and ongoing investment. Its inference architecture pairs GPUs for training and prompt prefill with reconfigurable dataflow units (RDUs) for token-by-token decode; a platform layer manages models, workloads, performance, and operational controls. The article also identifies power capacity, predictable latency and throughput, compliance, and payback economics as design considerations. SambaNova's product performance and savings figures are vendor claims reported in the text, including benchmark results and an ROI target based on a stated utilization assumption.

### Source excerpt

An AI factory is an end-to-end system that combines data, AI models, compute infrastructure, and governance to train, deploy, and continuously improve AI at scale. As AI moves from experimentation to production, organizations need this kind of structured way to continuously develop, deploy, and optimize AI systems at scale.

## MiniMax M3 Prompt Caching on SambaCloud: How It Works

DevFeed: [MiniMax M3 Prompt Caching on SambaCloud: How It Works](<https://devfeed.tech/articles/minimax-m3-prompt-caching-on-sambacloud-how-it-works-81507.md>)

Original publisher: [Read original article](<https://sambanova.ai/blog/introducing-prompt-caching-for-minimax-m3-on-sambacloud>)

Author: SambaNova

Published: 2026-10-07T16:56:47Z

Content type: article

Language: en

Sources: [Sambanova](<https://devfeed.tech/sources/sambanova.md>)

Topics: [prompt](<https://devfeed.tech/topics/prompt.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [context window](<https://devfeed.tech/topics/context-window.md>), [cache-control](<https://devfeed.tech/topics/cache-control.md>)

Tags: [blog](<https://devfeed.tech/tags/blog.md>), [caching](<https://devfeed.tech/tags/caching.md>), [prompt-caching](<https://devfeed.tech/tags/prompt-caching.md>), [technology](<https://devfeed.tech/tags/technology.md>), [time-to-first-token](<https://devfeed.tech/tags/time-to-first-token.md>), [tokens](<https://devfeed.tech/tags/tokens.md>)

### AI overview

SambaCloud has introduced automatic prompt caching for MiniMax M3. Stable request prefixes of at least 4,096 tokens can be served from cache, lowering cached-token prices by 90% and reducing time to first token by 35% to 88% across the tested context lengths. Developers can improve cache reuse by placing stable content first and keeping it byte-identical across requests.

### Source excerpt

Prompt caching, first introduced on SambaCloud with MiniMax M2.7, is now available for MiniMax M3. When requests share a stable prefix of at least 4,096 tokens, SambaCloud can serve that prefix from cache instead of recomputing it, with no code changes required. Cached tokens are billed at 90% below the standard input rate, and across 8k to 192k tokens of context, cache hits cut time to first token (TTFT) by 35% to 88%.

## Sovereign AI: Own Your Infrastructure, Models & Inference

DevFeed: [Sovereign AI: Own Your Infrastructure, Models & Inference](<https://devfeed.tech/articles/sovereign-ai-own-your-infrastructure-models-inference-81512.md>)

Original publisher: [Read original article](<https://sambanova.ai/blog/sovereign-ai>)

Author: SambaNova

Published: 2026-09-28T07:00:00Z

Content type: article

Language: en

Sources: [Sambanova](<https://devfeed.tech/sources/sambanova.md>)

Topics: [Sovereign AI](<https://devfeed.tech/topics/sovereign-ai.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [clean energy data centers](<https://devfeed.tech/topics/clean-energy-data-centers.md>), [ML model versioning](<https://devfeed.tech/topics/ml-model-versioning.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [blog](<https://devfeed.tech/tags/blog.md>), [data-centers](<https://devfeed.tech/tags/data-centers.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [security-and-privacy](<https://devfeed.tech/tags/security-and-privacy.md>), [sovereign-ai](<https://devfeed.tech/tags/sovereign-ai.md>), [technology](<https://devfeed.tech/tags/technology.md>)

### AI overview

The article presents sovereign AI as keeping an organization's infrastructure, models, and data under its own control. It describes SambaNova's SambaRack and SambaStack, including their on-premises deployment, air-cooled rack design, and inference hardware, as options for organizations seeking to run and own AI models within their own environments.

### Source excerpt

Artificial intelligence has become vital to nations, governments, and large corporations. Many of these large organizations are implementing Sovereign AI systems, which will provide them with artificial intelligence services, including infrastructure, data, models, and a trained workforce combined with the security and data privacy features that will enable them to achieve their agenda. A sovereign AI solution is one that is fully contained within an entity, such as a company or country, and enables the organization to meet its objectives while maintaining security and privacy. Increasingly, that comes down to a single question: who owns the model itself. The second is where it runs, and whether the data centers already inside your borders have the power and cooling to host it.

## SN50 Premium Inference: A Faster ROI for Neoclouds

DevFeed: [SN50 Premium Inference: A Faster ROI for Neoclouds](<https://devfeed.tech/articles/sn50-premium-inference-a-faster-roi-for-neoclouds-81511.md>)

Original publisher: [Read original article](<https://sambanova.ai/blog/sn50-premium-inference-a-faster-roi-for-neoclouds>)

Author: SambaNova

Published: 2026-09-17T19:30:41Z

Content type: article

Language: en

Sources: [Sambanova](<https://devfeed.tech/sources/sambanova.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [feature-store](<https://devfeed.tech/topics/feature-store.md>)

Tags: [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [blog](<https://devfeed.tech/tags/blog.md>), [business](<https://devfeed.tech/tags/business.md>), [compute](<https://devfeed.tech/tags/compute.md>), [cost](<https://devfeed.tech/tags/cost.md>), [costs](<https://devfeed.tech/tags/costs.md>), [inference](<https://devfeed.tech/tags/inference.md>), [neoclouds](<https://devfeed.tech/tags/neoclouds.md>), [roi](<https://devfeed.tech/tags/roi.md>)

### AI overview

The article presents SambaNova's SN50 as a way for neocloud providers to offer faster inference for large models and coding-agent workloads. It pairs purpose-built RDU decode with compatible GPUs for prefill, and describes a six-month return-on-investment target based on recovering infrastructure costs from revenue after operating costs, using 70% billable utilization as an illustrative planning assumption.

### Source excerpt

AI agents are creating a premium inference business for neoclouds. Coding agents read a repository, generate changes, run tests, and return to the model with the results. As code, tool outputs, and prior turns accumulate, inputs can exceed 100,000 tokens. Artificial Analysis' AA-AgentPerf benchmark replays real coding-agent trajectories with inputs reaching approximately 131,000 tokens per request. Customers need large models that stay responsive as that work grows.

## What is AI Inference? | SambaNova

DevFeed: [What is AI Inference? | SambaNova](<https://devfeed.tech/articles/what-is-ai-inference-sambanova-81515.md>)

Original publisher: [Read original article](<https://sambanova.ai/blog/what-is-ai-inference>)

Author: SambaNova

Published: 2026-09-08T19:00:00Z

Content type: article

Language: en

Sources: [Sambanova](<https://devfeed.tech/sources/sambanova.md>)

Topics: [Inference](<https://devfeed.tech/topics/inference.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>)

Tags: [accelerated](<https://devfeed.tech/tags/accelerated.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-compute](<https://devfeed.tech/tags/ai-compute.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [blog](<https://devfeed.tech/tags/blog.md>), [cache](<https://devfeed.tech/tags/cache.md>), [technology](<https://devfeed.tech/tags/technology.md>)

### AI overview

AI inference uses trained models to generate outputs from new data. The article explains how inference works, including the prefill and decode phases, and discusses deployment environments, performance and cost constraints, scaling, and specialized hardware such as reconfigurable dataflow units. It also promotes SambaNova's inference products.

### Source excerpt

The word "inference," in English, means a conclusion drawn through reasoning and evidence. Similarly, AI inference relates to an AI model's ability to infer, or extrapolate, conclusions in new situations, using information gained from training, response, and the fine tuning process. In short, AI inference is the process of using AI models to generate predictions or outputs from new data.

## Hot Chips 2026: SN50 RDU Dataflow at Scale

DevFeed: [Hot Chips 2026: SN50 RDU Dataflow at Scale](<https://devfeed.tech/articles/hot-chips-2026-sn50-rdu-dataflow-at-scale-81506.md>)

Original publisher: [Read original article](<https://sambanova.ai/blog/hot-chips-2026-dataflow-at-scale>)

Author: Raghu Prabhakar, Chief Architect, SambaNova

Published: 2026-09-02T19:33:06Z

Content type: article

Language: en

Sources: [Sambanova](<https://devfeed.tech/sources/sambanova.md>)

Topics: [Service fabric](<https://devfeed.tech/topics/service-fabric.md>), [AWS AI chips](<https://devfeed.tech/topics/aws-ai-chips.md>), [NVLink](<https://devfeed.tech/topics/nvlink.md>)

Tags: [400g](<https://devfeed.tech/tags/400g.md>), [ai](<https://devfeed.tech/tags/ai.md>), [blog](<https://devfeed.tech/tags/blog.md>), [chips](<https://devfeed.tech/tags/chips.md>), [scale](<https://devfeed.tech/tags/scale.md>), [technology](<https://devfeed.tech/tags/technology.md>)

### AI overview

SambaNova's Hot Chips 2026 presentation describes the SN50 RDU architecture for AI inference, focusing on decode bandwidth utilization, dataflow execution, and scale-up and scale-out Ethernet fabrics. SambaNova reports decode taking 75% to 97% of inference time across six analyzed configurations and models DeepSeek-R1 per-user throughput increasing from 200 to 500 tokens per second as the system grows from 64 to 256 RDUs, with modeled MBU between 44% and 51%. The throughput figures are modeled operating points, not production measurements.

### Source excerpt

At Hot Chips 2026, the industry converged on a new bottleneck: AI is moving from one-shot answers to agents that reason, call tools, and keep working. That means far more generated tokens, and far more time spent in decode.

## MiniMax M3 Running Fastest on SambaCloud

DevFeed: [MiniMax M3 Running Fastest on SambaCloud](<https://devfeed.tech/articles/minimax-m3-running-fastest-on-sambacloud-81508.md>)

Original publisher: [Read original article](<https://sambanova.ai/blog/minimax-m3-running-fastest-on-sambacloud>)

Author: justin.woo@sambanova.ai (Justin Woo)

Published: 2026-08-24T22:41:42Z

Content type: release

Language: en

Sources: [Sambanova](<https://devfeed.tech/sources/sambanova.md>)

Topics: [Frontier AI](<https://devfeed.tech/topics/frontier-ai.md>), [Loop Engineering](<https://devfeed.tech/topics/loop-engineering.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [blog](<https://devfeed.tech/tags/blog.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [for-developers](<https://devfeed.tech/tags/for-developers.md>), [minimax](<https://devfeed.tech/tags/minimax.md>)

### AI overview

MiniMax M3 is an open-weight model presented for long-context agent workflows and software engineering. The article reports a 1M-token context window using MiniMax Sparse Attention, benchmark results for coding and agent tasks, and internal tests in which the model worked autonomously on a CUDA kernel optimization task and reproduced research experiments. It also describes the model's multimodal capabilities and availability on SambaCloud.

### Source excerpt

Frontier Coding, 1M Context, and Native Multimodality. Build Long-Horizon Agents on SambaCloud TL;DR MiniMax M3 runs fastest on SambaCloud, where it is available today for developers building long-horizon, long-context agents. On coding and agent benchmarks, MiniMax M3 scores 59.0% on SWE-Bench Pro, 66.0% on Terminal-Bench 2.1, and 74.2% on MCP Atlas, making it one of the strongest open-weight models available. MiniMax M3 introduces MiniMax Sparse Attention (MSA), delivering a 1M-token context window with more than 9x faster prefill and more than 15x faster decoding than the previous generation. In MiniMax's internal testing, MiniMax M3 ran autonomously for roughly 24 hours to optimize an FP8 GEMM CUDA kernel, lifting hardware peak utilization from 7.6% to 71.3%, a 9.4x speedup, with zero human intervention. MiniMax M3 is natively multimodal, trained on text, image, and video from step 0, so it can read charts, formulas, and screenshots and operate a desktop computer. MiniMax M3 is the successor to MiniMax M2.7 and their most capable model to date. It is built for long-horizon agent workflows, complex software engineering, and multimodal tasks that were previously the exclusive territory of closed-source frontier models. It hits 59.0% on SWE-Bench Pro, 66.0% on Terminal-Bench 2.1, and 74.2% on MCP Atlas, making it one of the strongest open-weight options available for coding and agent harnesses.

## Running Bigger AI Models, More Efficiently | SambaNova

DevFeed: [Running Bigger AI Models, More Efficiently | SambaNova](<https://devfeed.tech/articles/running-bigger-ai-models-more-efficiently-sambanova-81509.md>)

Original publisher: [Read original article](<https://sambanova.ai/blog/running-bigger-ai-models-more-efficiently-sambanova>)

Author: SambaNova

Published: 2026-08-11T20:34:04Z

Content type: article

Language: en

Sources: [Sambanova](<https://devfeed.tech/sources/sambanova.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [data centers](<https://devfeed.tech/topics/data-centers.md>), [Frontier AI](<https://devfeed.tech/topics/frontier-ai.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [blog](<https://devfeed.tech/tags/blog.md>), [cost](<https://devfeed.tech/tags/cost.md>), [inference](<https://devfeed.tech/tags/inference.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [power](<https://devfeed.tech/tags/power.md>), [technology](<https://devfeed.tech/tags/technology.md>)

### AI overview

A Fortune panel featuring SambaNova CEO Rodrigo Liang and Adaption Labs CEO Sara Hooker argues that production AI is constrained by model size, cost, and power. Hooker emphasizes changing model architectures and the memory traffic created by agentic inference, while Liang describes SambaNova's hardware approach: keeping data local, using full precision, and combining different hardware for different inference stages. The article presents efficiency as key to serving large models at scale.

### Source excerpt

TL;DR AI has moved from experimentation into production, and that shift exposes three hard constraints: models keep getting bigger, cost climbs steeply at scale, and power becomes a physical ceiling. On a recent Fortune panel, SambaNova CEO Rodrigo Liang and Adaption Labs CEO Sara Hooker agreed the industry's central problem is now efficiency, though they approach it from different angles: Liang from the infrastructure side, Hooker from the model-architecture side. Large models are not going away. For the most demanding workloads, they are unavoidable, so the real question is how to run them efficiently rather than whether to run them at all. Inference in the agentic era behaves nothing like the single-model throughput problem of the past. In Hooker's words, "inference is a different beast." Constant data movement, not raw compute, is the bottleneck. SambaNova's focus is premium inference, defined on the panel by speed and size: Running the largest models much more efficiently, at full precision, rather than trading accuracy for speed. Are Large Models Here to Stay? The question hung over the whole conversation: Are today's largest foundation models where AI is heading, or will we look back on them as a detour? A recent Fortune panel titled "From the AI We Have to the AI We Need" put it directly to two people building very different answers, SambaNova co-founder and CEO Rodrigo Liang and Adaption Labs co-founder and CEO Sara Hooker, moderated by Fortune AI editor Jeremy Kahn.

## Premium Inference: Serving Fast Tokens for Agentic AI

DevFeed: [Premium Inference: Serving Fast Tokens for Agentic AI](<https://devfeed.tech/articles/premium-inference-serving-fast-tokens-for-agentic-ai-81513.md>)

Original publisher: [Read original article](<https://sambanova.ai/blog/the-premium-inference-moment-is-here>)

Author: Vasanth Mohan

Published: 2026-08-10T23:56:04Z

Content type: article

Language: en

Sources: [Sambanova](<https://devfeed.tech/sources/sambanova.md>)

Topics: [model-serving](<https://devfeed.tech/topics/model-serving.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [Agentic AI](<https://devfeed.tech/topics/what-is-agentic-ai.md>), [Deployment & Scaling](<https://devfeed.tech/topics/deployment-scaling.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>)

Tags: [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [blog](<https://devfeed.tech/tags/blog.md>), [business](<https://devfeed.tech/tags/business.md>), [compute](<https://devfeed.tech/tags/compute.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-providers](<https://devfeed.tech/tags/inference-providers.md>), [latency](<https://devfeed.tech/tags/latency.md>), [neoclouds](<https://devfeed.tech/tags/neoclouds.md>), [pricing](<https://devfeed.tech/tags/pricing.md>), [product](<https://devfeed.tech/tags/product.md>), [routing](<https://devfeed.tech/tags/routing.md>), [technology](<https://devfeed.tech/tags/technology.md>)

### AI overview

The article argues that agentic AI workloads create demand for premium inference tiers that prioritize responsive token generation. It describes a hybrid serving architecture that uses GPUs for compute-heavy prefill, SambaNova RDUs for latency-sensitive decode, and a serving layer to route workloads. It also presents SambaRack SN50 performance configurations and discusses how providers can price faster inference.

### Source excerpt

AI demand is heating up. Models are getting larger, more capable, and more expensive to serve, and every new release raises expectations for what an AI experience should feel like. TL;DR Premium inference is fast, responsive serving for large, intelligent models. Agentic AI has turned it into a distinct product tier rather than a nice-to-have. Agent loops multiply latency. OpenAI reported a long-horizon Codex agent running roughly 25 hours and consuming approximately 13 million tokens. Slow decode delays the entire product, not just one response. The market is already pricing speed. MiniMax charges $2.40 per 1 M high-speed output tokens versus $1.20 for standard output tokens. OpenAI, Anthropic, and Fireworks all ship fast tiers at a premium. Delivering premium inference requires disaggregation: GPUs for compute-heavy prefill, SambaNova RDUs for latency-sensitive decode, with a serving layer routing between them. SambaRack SN50 runs MiniMax M2.7 at roughly 820 tokens per second (TPS) for premium interactivity, or roughly 420 TPS when the goal is throughput and concurrency. For inference providers and neoclouds, premium inference is a tier they can price, not a cost they have to absorb.

## SambaRack SN50 Benchmarked on MiniMax M2.7 by SemiAnalysis

DevFeed: [SambaRack SN50 Benchmarked on MiniMax M2.7 by SemiAnalysis](<https://devfeed.tech/articles/sambarack-sn50-benchmarked-on-minimax-m2-7-by-semianalysis-81510.md>)

Original publisher: [Read original article](<https://sambanova.ai/blog/semianalysis-benchmarks-sambarack-sn50-with-fast-inference-on-minimax-m2.7>)

Author: Vasanth Mohan

Published: 2026-07-30T17:20:21Z

Content type: news

Language: en

Sources: [Sambanova](<https://devfeed.tech/sources/sambanova.md>)

Topics: [GPU](<https://devfeed.tech/topics/gpu.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>), [Snowflake streaming data architecture](<https://devfeed.tech/topics/snowflake-streaming-data-architecture.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [blog](<https://devfeed.tech/tags/blog.md>), [compute](<https://devfeed.tech/tags/compute.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [data-centers](<https://devfeed.tech/tags/data-centers.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-performance](<https://devfeed.tech/tags/inference-performance.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [kv-cache](<https://devfeed.tech/tags/kv-cache.md>), [speed](<https://devfeed.tech/tags/speed.md>), [technology](<https://devfeed.tech/tags/technology.md>)

### AI overview

SemiAnalysis benchmarked SambaRack SN50 running MiniMax M2.7 with vLLM in two configurations: tensor parallelism across 16 chips achieved about 800 tokens per second for per-user responsiveness, while a configuration combining tensor and data parallelism achieved about 400 tokens per second while serving more concurrent users. The article describes SN50 decode performance alongside GPU-based prefill and discusses scaling the setup to larger chip clusters.

### Source excerpt

MiniMax M2.7 is a model used by many of our customers around the world that helps augment their coding and agentic workflows using the fast inference speed of SambaNova's SN40 to accelerate their tasks. SambaNova's SN50 chip improves running this model significantly, which we previewed at RAISE with MiniMax M2.7 running over 800 tokens/second as validated by Artificial Analysis.

## What Are AI Data Centers? | SambaNova

DevFeed: [What Are AI Data Centers? | SambaNova](<https://devfeed.tech/articles/what-are-ai-data-centers-sambanova-81514.md>)

Original publisher: [Read original article](<https://sambanova.ai/blog/what-are-ai-data-centers>)

Author: SambaNova

Published: 2026-07-24T21:59:23Z

Content type: article

Language: en

Sources: [Sambanova](<https://devfeed.tech/sources/sambanova.md>)

Topics: [data centers](<https://devfeed.tech/topics/data-centers.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [nuclear energy for AI](<https://devfeed.tech/topics/nuclear-energy-for-ai.md>)

Tags: [ai-data-centers](<https://devfeed.tech/tags/ai-data-centers.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [blog](<https://devfeed.tech/tags/blog.md>), [business](<https://devfeed.tech/tags/business.md>), [data-centers](<https://devfeed.tech/tags/data-centers.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>)

### AI overview

AI data centers combine accelerators, CPUs, networking, storage, and suitable power and cooling to support AI training and inference. The article explains why many organizations can run inference in existing facilities, and compares cloud, on-premises, and hybrid deployment models. It also distinguishes data center infrastructure from the broader systems and processes of an AI factory.

### Source excerpt

The term "data center" indicates a physical facility that stores servers, networking, and other infrastructure necessary to run IT operations in an organization.