# B300 Published articles for B300. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Characterizing CUDA Locality Domains on Blackwell GPUs DevFeed: [Characterizing CUDA Locality Domains on Blackwell GPUs]() Original publisher: [Read original article]() Author: Rocky Wang, Joyce Zhou, Keith Jiang, Jason Zhao Published: 2026-11-06T05:00:00Z Content type: article Language: en Sources: [arXiv: Distributed Computing]() Topics: [CUDA](), [GPU optimization](), [Multi-GPU](), [NVIDIA Blackwell](), [NCCL]() Tags: [abstract](), [also](), [arxiv](), [b300](), [blackwell](), [can](), [changes](), [code](), [communication](), [costs](), [critical](), [cross](), [cs-dc](), [cuda](), [data](), [deepgemm](), [domain](), [domains](), [driver](), [gpus](), [locality]() ### AI overview On one node with eight NVIDIA B300 GPUs, the study finds that CUDA 13.4 Locality Domains can reduce latency, but benefits depend on data access patterns. Die-aware adaptations reportedly improve NCCL 2.32.3 AllGather by up to 43.5%, nvbandwidth DtoD by 111.3%, and DeepGEMM EP by 37%. The abstract also reports allocation overhead for sizes above 32 MB, linked to barriers and TLB invalidation. The authors propose a GPU driver kernel module fix, submitted to NVIDIA, that they say could reduce this overhead 7-10x without user-code changes. ### Source excerpt arXiv:2611.07535v1 Announce Type: new Abstract: Low latency is critical to user experience. As NVIDIA Blackwell and Rubin GPUs advance multi-die scaling, data placement and cross-die communication become increasingly important. However, the latency implications and underlying costs of cross-die communication and die-aware execution remain largely unexplored. In this report, we present a comprehensive evaluation of the Locality Domain (LD) feature introduced in CUDA 13.4. Conducted on a single node equipped with eight NVIDIA B300 GPUs, our study examines three aspects: memory management and access, computation, and communication, to identify the sources of latency and, more importantly, explore how LD can mitigate them. Our measurements show that the benefits of LD depend on data access patterns. Through die-aware adaptations and integration with widely used libraries and tools, we achieve speedups of up to 43.5% for NCCL 2.32.3 (AllGather), 111.3% for nvbandwidth (DtoD), and 37% for DeepGEMM (EP). We also identify non-negligible LD-related GPU memory allocation overhead when allocation sizes exceed 32 MB due to barrier and TLB invalidation, we propose a fix to the GPU driver kernel module (submitted to NVIDIA) and show that overhead could be reduced 7x ~ 10x without requiring any user-code changes. ## flash-comm: Self-Synchronizing Collective Communication for LLM Decoding DevFeed: [flash-comm: Self-Synchronizing Collective Communication for LLM Decoding]() Original publisher: [Read original article]() Author: Carrie Wang, Jack Li, Jason Zhao Published: 2026-11-05T05:00:00Z Content type: article Language: en Sources: [arXiv: Distributed Computing]() Topics: [LLM optimization](), [NCCL](), [Large Language Model]() Tags: [asynchronous](), [b300](), [collective](), [communication](), [consistency](), [cost](), [cs-dc](), [data-movement](), [decoding](), [encoding](), [gpu](), [gpus](), [latency](), [library](), [llm](), [median](), [message](), [protocol](), [reduction](), [selector](), [synchronization]() ### AI overview flash-comm fuses readiness synchronization with data movement: each data block carries readiness evidence so receivers can validate needed blocks without separate inter-GPU barriers. A cost-guided selector adapts readiness encoding to message size and GPU count, balancing synchronization overhead against extra transfer cost. It also separates transport order from reduction order to preserve result consistency with asynchronous arrivals. On eight NVIDIA B300 GPUs, all-reduce latency fell by a median 28.8% and up to 43.7% versus custom AR v2; end-to-end decoding time per output token fell 1.7-6.8%. Since all-reduce was 11.4% of baseline decoding time, optimizing it alone limits possible overall latency gains. Standalone all-gather latency fell by a median 23.8% and up to 58.9%. ### Source excerpt arXiv:2611.06634v1 Announce Type: new Abstract: We present flash-comm, a communication library that achieves efficient transport for LLM decoding by fusing readiness synchronization with data movement. flash-comm introduces self-synchronizing transfers in which each data block carries evidence of its readiness, allowing receivers to directly validate the blocks they need without separate inter-GPU barriers. Based on this principle, a cost-guided protocol selector adapts the readiness encoding to message size and the number of participating GPUs, balancing synchronization overhead against additional data-transfer cost. flash-comm further decouples transport order from reduction order to preserve result consistency across GPUs under asynchronous data arrivals. On a server with eight NVIDIA B300 GPUs, flash-comm reduces all-reduce latency by a median of 28.8% and up to 43.7% compared with custom AR v2 (the state-of-the-art all-reduce implementation). In end-to-end decoding, flash-comm reduces time per output token (TPOT) by 1.7-6.8%. Note that all-reduce accounts for 11.4% of the baseline decoding time, which bounds the latency reduction possible from optimizing this operation alone. The same mechanism also supports standalone all-gather, reducing latency by a median of 23.8% and up to 58.9% across eight B300 GPUs. ## When Strong Reasoners Join the Pool: Small Union Premiums and Costly Runaway Generations in LLM Routing DevFeed: [When Strong Reasoners Join the Pool: Small Union Premiums and Costly Runaway Generations in LLM Routing]() Original publisher: [Read original article]() Author: Teng-Ruei Chen Published: 2026-11-04T05:00:00Z Content type: news Language: en Sources: [arXiv: Machine Learning]() Topics: [Model Routing](), [LLM evaluation / benchmarking](), [Nemotron](), [math-verify]() Tags: [abstract](), [account](), [all](), [arxiv](), [b300](), [benchmarks](), [budget](), [but](), [companion](), [counts](), [coupling](), [cs-cl](), [cs-lg](), [decoding](), [fixed](), [gap](), [join](), [llm](), [pool](), [routing](), [small](), [union]() ### AI overview On MATH-500, the measured benefit of taking any correct answer from multiple models was small: the empirical product-coupling premium was 0.215-0.588 percentage points across two- and three-model pools. In a fixed protocol with 30 seeded samples per problem, Nemotron Nano 9B v2 and Nemotron Super 49B v1.5 each reached 96.1% accuracy, versus 76.2% for Llama-3.3-70B-Instruct. Selecting among models offered 1.27 points of empirical headroom over always calling Nano, while sampling Nano three times produced practically the same any-correct rate as calling all three models once. Only 31 responses hit the 65,536-token cap, but those responses contributed up to 60.2% of a model's output-length-squared proxy; long generations also prevented completion of a planned Llama-3.1-8B run. The authors caution that the premium is not directly comparable to a companion study's 3.5-point result because the models, pool size, and protocol differed, and recommend reporting the pool, protocol, samples per cell, and decoding budget with oracle gaps. ### Source excerpt arXiv:2611.04098v1 Announce Type: new Abstract: Routing benchmarks often judge a router against a hindsight oracle that counts a query as solved whenever any model's recorded answer is correct, but under stochastic decoding that oracle mixes two different objects: the clairvoyant ceiling of committing to one model per query and the premium of calling several models and keeping any correct answer. A companion study measured this premium at 3.5 percentage points (pp) for 11 small open models on MATH-500; how large it is once current reasoning models enter the pool was unknown. We measure it with a fixed-protocol, pre-committed generation on a single NVIDIA B300: all 500 MATH-500 problems, 30 seeded samples per problem, Llama-3.3-70B-Instruct and NVIDIA's Nemotron Nano 9B v2 and Nemotron Super 49B v1.5 (a NAS derivative of the same 70B) under vendor sampling, a 65,536-token cap, a pinned Math-Verify scorer, and an append-only attempt ledger, for 45,000 scored responses. The two reasoning models both reach 96.1% accuracy against 76.2% for the 70B baseline. The empirical product-coupling premium is 0.215-0.588 pp across all two- and three-model pools; the empirical plug-in selection headroom over always calling Nano is 1.27 pp; and resampling Nano three times gives practically the same empirical any-correct rate as calling all three models once. Only 31 responses reached the cap, 25 of them with exactly periodic tails, yet the capped responses account for up to 60.2% of a model's output-length-squared proxy, and long generations made a fourth planned model, Llama-3.1-8B, too slow to finish. The premium is far below the companion's 3.5 pp, but models, pool size and protocol changed together; an oracle gap should therefore be reported with the pool, the protocol, the samples per cell and the decoding budget that produced it. ## Matching Workloads to Blackwell: Power-Resolved Benchmarking of NVIDIA B200 and B300 across HPC and AI Workloads DevFeed: [Matching Workloads to Blackwell: Power-Resolved Benchmarking of NVIDIA B200 and B300 across HPC and AI Workloads]() Original publisher: [Read original article]() Author: Mohsen Zand, Trung Dac Nguyen, H. Birali Runesha Published: 2026-11-02T05:00:00Z Content type: article Language: en Sources: [arXiv: Distributed Computing]() Topics: [NVIDIA Blackwell](), [High-Performance Computing](), [vllm](), [LLM optimization](), [AI Infrastructure]() Tags: [ai-workloads](), [b300](), [benchmarking](), [blackwell](), [blackwell-ultra](), [computer-vision](), [cs-ar](), [cs-dc](), [device](), [espresso](), [gpu](), [gpus](), [guidance](), [harness](), [hpc](), [inference](), [kernel](), [llama](), [llm](), [nvidia](), [nvidia-b200](), [power]() ### AI overview Across the tested workloads, NVIDIA B300 is not a uniform upgrade over B200: it trails by up to 5.6x on convolutional inference, 8x on 3D vision transformer training, 24-27% on 70B LLM throughput, and 16x on double-precision Quantum ESPRESSO. Power measurements show B300 draws only 36-53% of its 1100 W envelope on affected convolutional workloads, while both GPUs reach 95-97% of TDP on ViT-Large, which B300 wins. Since the benchmark keeps inputs in device memory, it excludes host-side data pipelines as the cause and localizes the convolutional gap to per-kernel cost. The abstract also reports a 2.1x time-to-first-token difference between two B300 nodes with identical silicon, attributed solely to a development vLLM build. Results cover two eight-GPU clusters, computer-vision workloads, Llama-3-8B and 70B inference, and six HPC codes. ### Source excerpt arXiv:2610.41919v1 Announce Type: new Abstract: We evaluate NVIDIA B200 and B300 (Blackwell Ultra) GPUs on two independent eight-GPU clusters, spanning computer-vision inference and training, LLM inference with Llama-3-8B and 70B~\cite{llama3}, and six production HPC codes. The B300 is not a uniform successor to the B200: it trails by up to 5.6$\times$ on convolutional inference, 8$\times$ on 3D vision transformer training, 24--27\% on 70B LLM throughput, and 16$\times$ on double-precision Quantum ESPRESSO. We resolve the convolutional deficit using per-GPU board power rather than throughput alone: on the affected workloads the B300 draws only 36--53\% of its 1100\,W envelope, whereas on ViT-Large, which it wins, both parts saturate at 95--97\% of TDP. Because our harness keeps inputs resident in device memory, a host-side data pipeline is excluded by construction, localizing the cause to per-kernel cost. Two B300 nodes of identical silicon differ by 2.1$\times$ in time-to-first-token solely from a development vLLM build. We give per-workload procurement guidance. ## Batch Inference Is Now Generally Available on Runpod DevFeed: [Batch Inference Is Now Generally Available on Runpod]() Original publisher: [Read original article]() Author: The Team at Runpod Published: 2026-10-28T16:04:37Z Content type: release Language: en Sources: [Runpod Blog.]() Topics: [model-serving](), [AI Infrastructure](), [GPU](), [Web APIs]() Tags: [2](), [3](), [account](), [all](), [api](), [api-key](), [application](), [authorization](), [away](), [b300](), [backlog](), [based](), [batch](), [existing](), [inference](), [queue](), [send](), [serverless](), [up](), [workers]() ### AI overview Runpod Batch Inference lets you submit up to 3 million requests a day to an existing queue-based Serverless endpoint, using dedicated workers so batch jobs do not take capacity from live `/run` traffic. It suits work that can wait: start times are not guaranteed, requests run in no set order, and multi-hour latency is possible. Create a batch through the console or API, then finalize it to start processing; draft batches do not run. Check request counts and statuses, inspect per-request errors, and page through results. A batch can hold up to 5,000 requests, and an endpoint can run up to 10 active batches, subject to account limits. Billing is based on compute time per request, with discounts of up to 30% depending on GPU type (20% on H100, H200, B200, and B300). Since the service provides no pre-finalization cost estimate, the excerpt recommends trying 100 requests and extrapolating. ### Source excerpt Batch Inference is now generally available on Runpod Serverless. Send up to 3 million requests a day to an existing queue-based endpoint, on dedicated workers. ## Liquid AI Releases LFM3: 6 Open-Weight 2B-Class Multimodal Models DevFeed: [Liquid AI Releases LFM3: 6 Open-Weight 2B-Class Multimodal Models]() Original publisher: [Read original article]() Author: Asif Razzaq Published: 2026-10-27T21:08:02Z Content type: release Language: en Sources: [MarkTechPost]() Topics: [AI Foundation Models](), [Open Source Models & Datasets](), [sglang](), [llama.cpp](), [Ollama](), [Transformers](), [qwen](), [context window](), [gemma](), [grpo]() Tags: [agent](), [agentic](), [agentic-ai](), [agentic-tasks](), [agents](), [ai-infrastructure](), [ai-shorts](), [api](), [architecture](), [artificial-intelligence](), [attention](), [audio](), [automationbench](), [b300](), [backend](), [coding](), [context](), [context-window](), [convolution](), [editors-pick](), [language-model](), [large-language-model](), [liquid-ai](), [models](), [multimodal](), [new-releases](), [staff](), [tech-news](), [technology](), [text](), [uncategorized](), [vision]() ### AI overview Liquid AI's LFM3 release comprises six open-weight, roughly 2B-class models sharing a tri-hybrid backbone, with text, vision, audio, multimodal, and decision variants. The LFM3-1.7B text model has a 256K context window and is reported to use about 1 GB at 4-bit; the family includes GGUF, MLX, ONNX, FP8, and NVFP4 checkpoints. Liquid AI reports strong results on some agent benchmarks, including BFCL V4 and WebArena-Verified versus Qwen3.5-9B, while the smaller model trails it on GPQA Diamond, SWE-bench Verified, and 128K RULER. Other variants target video-capable vision, multilingual speech, combined vision/audio/language, and one-pass decisions. Deployment options include local runtimes and cloud services; the text notes that 128K context requires 1.6 GB of KV cache in the cited on-device setup. ### Source excerpt Liquid AI released LFM3, 6 open-weight 2B-class models for text, vision, audio, omni and decisions. LFM3-1.7B uses a tri-hybrid backbone, a 256K context window and fits in about 1 GB at 4-bit. The post Liquid AI Releases LFM3: 6 Open-Weight 2B-Class Multimodal Models appeared first on MarkTechPost. ## BiLoCo: Binary Low-Rank Corrections for Post-Training Quantization DevFeed: [BiLoCo: Binary Low-Rank Corrections for Post-Training Quantization]() Original publisher: [Read original article]() Author: David Jin, Beshr IslamBouli, Tarushii Goel, Han Guo, Yoon Kim Published: 2026-10-26T04:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [Post-training optimization](), [quantization](), [LLM Techniques](), [LLM optimization]() Tags: [abstract](), [arxiv](), [b300](), [batch](), [bits](), [blackwell](), [concurrently](), [cs-lg](), [cuda](), [diffusion](), [diffusion-models](), [errors](), [fp4](), [gpus](), [implement](), [inference](), [kernel](), [language-model](), [post-training](), [quantization](), [rank]() ### AI overview BiLoCo reduces FP4 quantization errors with a low-rank correction whose left and right factors are binary sign vectors instead of BF16. The authors report 16x less storage per rank-one component than a high-precision correction; because the correction uses additions and subtractions, it can run on CUDA cores alongside the FP4 tensor-core GEMM. On 4B-72B LLMs, BiLoCo reportedly matches or improves on memory-matched SVDQuant for PTQ. A fused B300 kernel combines the correction with the GEMM in its epilogue. At batch size 1, it is at most 11% slower than FP4-only GEMM for ranks up to 512; at rank 128, it is at most 20% slower up to batch size 8. ### Source excerpt arXiv:2610.32243v1 Announce Type: new Abstract: Post-training quantization (PTQ) to FP4 has emerged as a key technique for reducing large language model inference cost, especially on Blackwell GPUs, which offer native FP4 tensor core support. Direct PTQ to FP4, however, still leaves a noticeable accuracy gap. Recent work on SVDQuant mitigates the errors introduced through FP4 quantization for diffusion models by adding high-precision low-rank corrections obtained via SVD. However, this high-precision low-rank correction requires many bits per rank, and it runs on the same tensor cores as the main FP4 GEMM. We introduce BiLoCo, a low-rank correction method whose left and right factors are stored as binary ${\{\pm 1\}}$ sign vectors rather than as BF16 numbers. The signed format uses $16{\times}$ less storage per rank-one component, and since the correction now consists of additions/subtractions, this can be run on CUDA cores concurrently with the FP4 tensor-core GEMM. On 4B--72B LLMs, BiLoCo matches or improves memory-matched SVDQuant for PTQ. We implement BiLoCo on B300 as a fused single kernel, where the FP4 GEMM runs on tensor cores, the bit-packed correction on the CUDA cores of the same launch, and the two are combined in the GEMM epilogue. With batch size $1$, the fused kernel is at most $11\%$ slower than the FP4-only GEMM for every rank up to $512$. At rank $128$, it is at most $20\%$ slower up to batch size $8$. ## Aivres NVIDIA Vera Rubin and HGX B300 at OCP Summit DevFeed: [Aivres NVIDIA Vera Rubin and HGX B300 at OCP Summit]() Original publisher: [Read original article]() Author: Vic A Published: 2026-10-25T21:51:55Z Content type: news Language: en Sources: [ServeTheHome]() Topics: [NVIDIA Vera Rubin](), [Vera Rubin NVL72](), [NVIDIA Vera](), [Nvidia](), [NVLink](), [NVFP4](), [AI Infrastructure](), [ddr5]() Tags: [2026](), [ai](), [aivres](), [amd](), [amd-epyc](), [b300](), [bandwidth](), [blackwell](), [bluefield-dpu](), [compute](), [news](), [nvidia](), [nvidia-vera](), [nvidia-vera-rubin](), [ocp](), [ocp-summit](), [platforms](), [rubin](), [summit](), [vera](), [vera-rubin]() ### AI overview At OCP Global Summit 2026, Aivres showed NVIDIA platforms spanning a Vera Rubin NVL72 liquid-cooled rack, an HGX Rubin NVL8 compute tray, Vera CPU hardware, and an HGX B300 server. The KRS8000V4 rack is listed with 18 compute trays and nine NVLink switch trays, totaling 72 Rubin GPUs and 36 Vera CPUs; Aivres says it is fully liquid-cooled and can operate at 45°C. The KL2288 is a 2U, fanless liquid-cooled HGX Rubin NVL8 tray with eight GPUs and dual AMD EPYC 9006 processors. Its listed specifications include up to 400 PFLOPS for NVFP4 inference, 280 PFLOPS for training, 2.3TB HBM4, and 176TB/s HBM bandwidth. The excerpt also describes liquid-cooled CPUs and memory, rack quick disconnects, and busbar power. ### Source excerpt We check out the range of Aivres NVIDIA Vera, Vera Rubin, and Blackwell Ultra platforms at OCP Summit 2026 The post Aivres NVIDIA Vera Rubin and HGX B300 at OCP Summit appeared first on ServeTheHome. ## Serving DeepSeek-V4.1-Flash at Scale: vLLM Matches DeepInfer and Scales to GB300 NVL72 DevFeed: [Serving DeepSeek-V4.1-Flash at Scale: vLLM Matches DeepInfer and Scales to GB300 NVL72]() Original publisher: [Read original article]() Author: Inferact and the vLLM Team Published: 2026-10-23T00:00:00Z Content type: article Language: en Sources: [vLLM Blog]() Topics: [vllm](), [GB300 NVL72](), [LLM optimization](), [LLM Techniques](), [NVLink]() Tags: [agentic](), [ai-engineering](), [async](), [attention](), [b300](), [balancing](), [deepinfer](), [deepseek](), [deepseek-v4-1-flash](), [disaggregation](), [gb300](), [gb300-nvl72](), [gpu](), [large-scale-serving](), [llm-inference](), [model-serving](), [nvl72](), [open-source-ai](), [performance](), [rack](), [scale](), [tokens](), [user](), [v4-1-flash](), [vllm](), [vllm-blog](), [wide]() ### AI overview On one GB300 NVL72 rack, vLLM's P/D-disaggregated, Wide-EP serving of DeepSeek-V4.1-Flash reaches 954K tokens/s per GPU on SemiAnalysis AgentX at a P90 of 196 tokens/s per user, reported as 7.9x the authors' earlier single-node result. Decode throughput on GB200 is within 6% of DeepSeek's DeepInfer across batch sizes 8-256; the prefill curves track, though differing setups prevent a direct speedup comparison. The reported gains come from fitting prefill to DEP2, extending bounded replay to layer 20, using larger GPU pages for shared Engram tables and DeepSelect for prefill ranking, plus Wide-EP with EPLB for decode. At DEP32, EPLB substantially improves expert-load balance, while CPU overhead can erase GPU-side scaling gains. A specialized decode path reaches 826 P90 tokens/s for one user on four B300 GPUs, 44% above the standard path. Results depend on the stated hardware, configurations and workloads. ### Source excerpt With P/D disaggregation and Wide-EP on a GB300 NVL72 rack, vLLM serves DeepSeek-V4.1-Flash on SemiAnalysis AgentX at 954K tokens/s per GPU at a P90 of 196 tokens/s per user, 7.9x our aggregated single-node result. ## Engineered for Agentic AI: NVIDIA HGX B300 on CoreWeave Cloud DevFeed: [Engineered for Agentic AI: NVIDIA HGX B300 on CoreWeave Cloud]() Original publisher: [Read original article]() Author: Harsh Singh Banwait Published: 2026-10-22T16:27:39Z Content type: release Language: en Sources: [Coreweave Blog Posts]() Topics: [Agentic AI](), [Nvidia](), [NVFP4](), [NVLink](), [Cloud](), [Multi-GPU](), [NCCL](), [observability](), [AI Networking](), [Cloud Architecture]() Tags: [about](), [accelerate](), [accelerated](), [accelerators](), [advanced](), [agent](), [agent-workflows](), [agentic](), [agentic-ai](), [agentic-ai-development](), [agentic-ai-systems](), [agentic-systems](), [agents](), [ai](), [b300](), [cloud](), [coreweave](), [nvidia](), [performance](), [scale]() ### AI overview CoreWeave says NVIDIA HGX B300 is generally available on its cloud for agentic AI workloads, pairing higher memory and NVFP4 performance with networking, orchestration, monitoring, and liquid cooling. The excerpt reports 270 GB of GPU memory versus 180 GB on HGX B200, and 14 PFLOPs of NVFP4 performance versus 9 PFLOPs; it says high-precision performance remains at the B200 level. CoreWeave's benchmarks, compared with HGX H200, report 3.42x faster token generation on Kimi K2.5, 2.61x on DeepSeek-R1, 4.93x faster DeepSeek-R1 end-to-end latency, and 1.95x measured collective bandwidth. Compared with B200, Kimi K2.5 concurrency improved 1.41x. The tests used different model configurations across architectures, including NVFP4 on B300 versus FP8 block scaling on H200 for DeepSeek-R1. The platform also offers Mission Control performance visibility, Kubernetes-based workload orchestration, Slurm on Kubernetes, W&B Weave tracing, and object storage cited at up to 7 GB/s per GPU. ### Source excerpt Unlock the performance of NVIDIA HGX B300 at scale on CoreWeave. ## From NVIDIA GTC 2026 to Production: What CoreWeave Showed and Why It Matters DevFeed: [From NVIDIA GTC 2026 to Production: What CoreWeave Showed and Why It Matters]() Original publisher: [Read original article]() Author: CoreWeave Published: 2026-10-22T16:27:39Z Content type: article Language: en Sources: [Coreweave Blog Posts]() Topics: [Production Engineering](), [observability](), [Physical AI]() Tags: [2026](), [agent](), [agentic](), [agentic-ai](), [agents](), [ai](), [ai-cloud](), [ai-systems](), [ai-workloads](), [arena](), [availability](), [b300](), [content-type](), [coreweave](), [gtc](), [gtc-2026](), [launches](), [nvidia](), [nvidia-gtc](), [production](), [real-world]() ### AI overview At NVIDIA GTC 2026, CoreWeave presented AI production as an infrastructure and operations challenge alongside advances in models and agents. It announced general availability of NVIDIA HGX B300 on CoreWeave Cloud and highlighted Weights & Biases features for serverless reinforcement learning, agent evaluation, and mobile training-run monitoring. Recent launches included SUNK for multi-thousand-GPU training, CoreWeave ARENA for assessing real workloads before rollout, Capacity Plans for flexible consumption, and Dedicated Inference for controlled execution of custom models. The article says teams face unpredictable training and inference performance, agent failures, and demands for latency, cost, reliability, and visibility. Its demos emphasized predictable execution, continuous evaluation and improvement, and full-stack observability through Mission Control. These are the article's descriptions of CoreWeave offerings and GTC demonstrations; the excerpt provides no comparative benchmarks or independent validation. ### Source excerpt From major launches to real-world demos, CoreWeave's GTC 2026 recap explores the industry shifts shaping how AI is built, run, and improved in production. ## Comet Cloud targets small businesses underserved by GPU cloud providers DevFeed: [Comet Cloud targets small businesses underserved by GPU cloud providers]() Original publisher: [Read original article]() Author: Sloane Kali Faye Published: 2026-10-21T18:19:13Z Content type: news Language: en Sources: [SiliconANGLE]() Topics: [AI Infrastructure](), [Cloud](), [Infrastructure as a Service (IaaS)](), [Finance]() Tags: [accelerators](), [agentic-ai](), [ai](), [ai-cloud](), [ai-factory](), [ai-inference](), [ai-infrastructure](), [ai-powered-pcs](), [ai-return-on-investment](), [ai-software](), [ai-startups](), [ai-tools](), [amazon-web-services](), [andromeda](), [artificial-intelligence](), [b300](), [cloud](), [comet-cloud](), [cube-event-coverage](), [cyber-resilience](), [data-protection](), [dave-vellante](), [dedicated-gpu-clusters](), [dell](), [dell-technologies-new-york-symposium-2026](), [dellnewyorksymposium](), [demand](), [digital-transformation](), [enterprise-ai](), [enterprise-storage](), [financial-services](), [financial-technology](), [financial-transformation](), [gpu](), [gpu-cloud](), [hybrid-cloud](), [inference](), [infra](), [it-modernization](), [multicloud](), [neocloud](), [news](), [nvidia](), [providers](), [servers](), [small-and-midsize-businesses](), [small-businesses](), [thecube](), [training]() ### AI overview Comet Cloud is targeting small and midsize businesses that founder Thomas Kato says struggle to secure GPU capacity from hyperscalers. It offers dedicated bare metal clusters that isolate a company's data, currently supplying infrastructure to partners such as Andromeda. Kato cited a target price of about $4 per hour for Nvidia B300 systems, compared with an estimated $15-$20 or more at hyperscalers. The company plans to add software services and distribute AI tools through its payments merchant network as inference use grows. Expansion depends on securing power, GPU allocations and financing: Kato said one 48-rack Nvidia Vera Rubin cluster could require $1 billion-$1.2 billion in contracted revenue to obtain financing. These prices, plans and financing figures are presented as Kato's statements. ### Source excerpt The GPU cloud market has so far been shaped by its largest buyers, who can commit to multiyear contracts for scarce accelerators. As artificial intelligence workloads tilt from training toward inference, some neocloud providers are betting the next surge in demand will come from a far broader customer base. Demand for neocloud capacity is shifting as [...] The post Comet Cloud targets small businesses underserved by GPU cloud providers appeared first on SiliconANGLE. ## Nutanix Private Inference System With Supermicro Puts Eight NVIDIA B300 Blackwell Ultra GPUs per Node Behind Dedicated NX Management and Storage Clusters, Ships Early 2027 DevFeed: [Nutanix Private Inference System With Supermicro Puts Eight NVIDIA B300 Blackwell Ultra GPUs per Node Behind Dedicated NX Management and Storage Clusters, Ships Early 2027]() Original publisher: [Read original article]() Author: Harold Fritts Published: 2026-10-21T15:11:15Z Content type: news Language: en Sources: [StorageReview.com]() Topics: [AI Infrastructure](), [AI Architecture](), [nutanix](), [NVIDIA Blackwell]() Tags: [ai](), [b300](), [bare-metal](), [berlin](), [blackwell](), [blackwell-ultra](), [clusters](), [cpus](), [enterprise](), [gpus](), [gtc](), [inference](), [management](), [node](), [nutanix](), [nvidia](), [nvidia-b300](), [nx](), [private](), [storage](), [supermicro](), [system]() ### AI overview Nutanix announced three infrastructure collaborations at NVIDIA GTC Berlin. With Supermicro, it plans an early-2027 private inference system using eight NVIDIA B300 GPUs per node, separate NX management and storage clusters, and Spectrum-X networking. The design combines Nutanix Cloud Platform, NKP, and Enterprise AI, covering bare-metal provisioning through model-as-a-service; the companies say dedicated management is intended to protect GPU execution from management-plane failures. No pricing or measured cost-per-token figures are published. Separately, Nutanix plans Day Zero NVIDIA Vera CPU support, starting with NKP Bare Metal and later expanding to virtualized environments, with no dates or performance figures. Its planned Netris stack extends tenant networking from workloads and DPUs to physical switches, with hardware-enforced isolation and automated network services; a tech preview is planned for early 2027, with no general availability date set. ### Source excerpt Nutanix announced three collaborations today at NVIDIA GTC Berlin: a jointly engineered private inference system with Supermicro built on servers with eight NVIDIA B300 Blackwell Ultra GPUs per node, Day Zero support for NVIDIA Vera CPUs starting with NKP Bare Metal, and a multitenant stack with Netris that extends tenant isolation into the physical network. The post Nutanix Private Inference System With Supermicro Puts Eight NVIDIA B300 Blackwell Ultra GPUs per Node Behind Dedicated NX Management and Storage Clusters, Ships Early 2027 appeared first on StorageReview.com. ## Conjugate Compilation: Self-Improving Agentic Co-Compilation for Speculative Decoding DevFeed: [Conjugate Compilation: Self-Improving Agentic Co-Compilation for Speculative Decoding]() Original publisher: [Read original article]() Author: Annabelle Sujun Tang, Rohan Mahapatra, Christopher Priebe, Hanyang Xu, Manya Ghobadi, Christopher Rossbach, Hadi Esmaeilzadeh Published: 2026-10-21T04:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [LLM optimization](), [Model Optimizer](), [GPU optimization](), [monte carlo tree search](), [agentic-coding]() Tags: [agentic](), [b300](), [compilation](), [cs-dc](), [cs-lg](), [cs-pl](), [gpu](), [gpu-utilization](), [optimization](), [performance](), [rtx-5090](), [speculative-decoding]() ### AI overview Conjugate Compilation jointly optimizes the draft and target models in speculative decoding by evaluating each compiled plan in their co-located shared execution; isolated kernel improvements may not translate to faster joint execution. An LLM proposes cross-model optimization plans, with measured execution scores, kernel verdicts, compiler feedback, and Monte Carlo tree search guiding later proposals. Interleaved operand layouts place blocks from both models in common tiles to co-locate GEMM and non-GEMM work while preserving their mathematics. Across six single-GPU configurations on NVIDIA B300 and RTX 5090, the system reports 1.37x geometric-mean per-round speedup over conventional execution, rising to 1.45x with interleaved kernels and 1.59x in the best case. Average GPU utilization improves 1.50x on B300 and 1.46x on RTX 5090. ### Source excerpt arXiv:2610.25012v1 Announce Type: new Abstract: As LLM adoption surges, scaling model serving requires unlocking stranded GPU capacity alongside deploying more GPUs. Speculative decoding brings two models into the serving process: a small draft model proposes tokens that a larger target model verifies. Yet compiling these models individually leaves their shared execution outside the compiler's optimization boundary. We introduce CONJUGATE COMPILATION, which reframes speculative decoding as a cross-model compilation problem and jointly optimizes the draft and target models within their shared execution rather than treating each in isolation. Our key insight is that cross-model compilation should be informed by the measured performance of the co-located shared execution it produces: an optimization that improves a kernel in isolation need not improve the models when they execute together. To expand cross-model opportunities, we devise operand layouts that interleave blocks from both models within common tiles, co-locating GEMM and non-GEMM work while preserving their mathematics. We realize CONJUGATE COMPILATION through a self-improving agentic search system in which an LLM edits cross-model optimization plans using the compiler and GPU execution as its environment. Each proposed plan is compiled and evaluated through its co-located shared execution, whose measured score, together with kernel-level verdicts and compiler feedback, informs subsequent proposals, while Monte Carlo tree search accumulates rewards to guide exploration. Across six single-GPU speculative-decoding configurations on NVIDIA B300 and RTX 5090 GPUs, CONJUGATE COMPILATION achieves a geometric-mean per-round speedup of 1.37x over conventional execution. Interleaved kernels raise this speedup to 1.45x, reaching 1.59x in the best case. CONJUGATE COMPILATION also improves GPU utilization on average by 1.46x on RTX 5090 and 1.50x on B300. ## miniTE: Diagnosing and Bridging the Execution Gap in Native NVFP4 Reinforcement Learning DevFeed: [miniTE: Diagnosing and Bridging the Execution Gap in Native NVFP4 Reinforcement Learning]() Original publisher: [Read original article]() Author: Zhen Li, Mingfa Feng, Yiming Zhang, Yang Yu, Changhai Zhou, Pengxiang Li, Dongkuan Wu, Congkai Xie, Hongxia Yang Published: 2026-10-21T04:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [Reinforcement learning](), [NVFP4](), [quantization](), [Large Language Model](), [LLM evaluation / benchmarking]() Tags: [alignment](), [b300](), [benchmark](), [benchmark-performance](), [consistency](), [cs-ai](), [cs-lg](), [diagnostics](), [engine](), [evaluation](), [execution](), [experiments](), [framework](), [gpu](), [implementation](), [kernels](), [language-models](), [large-language-models-llms](), [learning](), [native](), [nvfp4](), [reinforcement-learning](), [stat-ml]() ### AI overview miniTE addresses policy-evaluation inconsistencies between separate RL training and sampling engines: matching weights and numerical formats alone did not ensure consistent execution. Its lightweight framework combines execution diagnostics, cross-engine numerical alignment, and native W4A4 GEMM computation, without changing the RL objective. In long-horizon experiments on Qwen3-30B-A3B, training was stable and benchmark performance was close to BF16; evaluation quality exceeded naive Transformer Engine NVFP4. On one NVIDIA B300 GPU node, the optimized learner reached 33,540 tokens/s for complete learner updates--1.39x BF16 and 1.47x TE NVFP4 throughput. The abstract also says the framework, kernels, diagnostics, and training configurations were released. ### Source excerpt arXiv:2610.23928v1 Announce Type: new Abstract: Native NVFP4 promises to accelerate reinforcement learning (RL) for large language models (LLMs), but achieving stable learning across separate training and sampling engines remains challenging. We show that matching model weights and numerical formats is insufficient to ensure consistent policy evaluation, highlighting the importance of cross-engine numerical execution. We introduce miniTE, a lightweight framework that combines execution diagnostics, cross-engine numerical alignment, and efficient native W4A4 (4-bit weights and activations) GEMM computation for LLM RL. It improves numerical execution consistency without modifying the RL objective, while reducing implementation overhead to realize the benefits of native low-bit computation. Long-horizon RL experiments on Qwen3-30B-A3B demonstrate stable training and benchmark performance close to BF16, outperforming naive Transformer Engine (TE) NVFP4 in evaluation quality. On single NVIDIA B300 GPU node, the optimized learner reaches 33,540 tokens/s, yielding 1.39x and 1.47x the throughput of BF16 and TE NVFP4, respectively, for complete learner updates. We released the framework, kernels, diagnostics, and training configurations to make native low-bit RL easier. ## Under the Hood: Serving Kimi K3 DevFeed: [Under the Hood: Serving Kimi K3]() Original publisher: [Read original article]() Author: Shree Murthy Published: 2026-07-30T17:10:40Z Content type: article Language: en Sources: [DigitalOcean]() Topics: [LLM Techniques](), [AI Infrastructure](), [Multi-GPU](), [NVLink](), [model architecture](), [Distributed Systems & Parallel Computing]() Tags: [ai](), [amd](), [architecture](), [attention](), [b300](), [bandwidth](), [benchmarks](), [cache](), [capacity](), [deployment](), [digitalocean](), [distributed](), [engineering](), [experts](), [fabric](), [gpu](), [gpus](), [hardware](), [k3](), [kimi](), [kimi-k3]() ### AI overview DigitalOcean says serving Kimi K3 on launch day required 8-GPU NVIDIA HGX B300 or AMD Instinct MI350X servers: its roughly 1.56 TB of weights need about 195 GiB per GPU, leaving limited memory for KV cache and activations. Neither GPU can hold the full weights alone, so high-speed NVLink or Infinity Fabric interconnects support distributed attention and expert-parallel computation. DigitalOcean used llm-d for its native support of mixed GPU types, enabling onboarding to both NVIDIA and AMD platforms. The excerpt identifies hardware selection and stack integration as parts of the effort but ends mid-sentence, so it does not provide the full account or verification details. ### Source excerpt DigitalOcean launched Kimi K3 on day 0. It's already one of the most popular models on the platform and across the market: second most likes on Hugging Face, sixth most traffic on OpenCode. Getting a model this size running well on day zero took real work across several teams. Thanks to Moonshot AI, Inferact, RadixArk, NVIDIA, and AMD for the help getting there. Standing up a new model, integrating it into DigitalOcean's Inference Engine, and showcasing its unique attributes on day 0 takes three things: the right hardware, a tuned serving stack, and rigorous verification against Moonshot's own benchmarks. Here are the lessons we learned along the way: Hardware selection and implementation We selected NVIDIA HGX™ B300 and AMD Instinct™ MI350x GPUs to run K3 because these instances provide the memory capacity, FLOPs, and interconnect horsepower necessary for a model of K3's size and architecture. We built our distributed inference stack with llm-d because it includes native support for GPU type heterogeneity. This let us quickly onboard K3 to both AMD and NVIDIA platforms. Kimi K3 has roughly 2.78 trillion total parameters, 896 routed experts, and an attention stack that interleaves 69 Kimi Delta Attention (KDA) layers with 24 Gated Multi-head Latent Attention (MLA) layers. Kimi-K3 weights are ~1.56 TB in total, which requires about 195 GiB per GPU. Given such a large memory footprint for the weights alone, and a need to keep enough headroom for KV cache and activations, the practical unit of deployment is an 8x NVIDIA HGX B300 or AMD Instinct MI350X server. Both have 288GB of VRAM capacity, and after loading the weights, there is still some amount of practical memory left for the KV cache. Entire weights cannot be loaded on a single GPU. That's where the high-speed scaled-up NVIDIA's NVLink or AMD's Infinity Fabric is critical to ensure there is enough interconnect horsepower for bandwidth intensive, latency sensitive attention and expert parallel computations. Model