# TensorRT

NVIDIA TensorRT is an SDK for optimizing deep learning inference on NVIDIA GPUs, compiling trained models into deployment engines.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## HELENA for 5G NR LEO NTN Channel Estimation: A Comparative Evaluation

DevFeed: [HELENA for 5G NR LEO NTN Channel Estimation: A Comparative Evaluation](<https://devfeed.tech/articles/helena-for-5g-nr-leo-ntn-channel-estimation-a-comparative-evaluation-137016.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.14735>)

Author: Miguel Camelo Botero, Nina Slamnik-Krije\\v{s}torac, Johann Marquez-Barja

Published: 2026-11-17T05:00:00Z

Content type: comparison

Language: en

Sources: [arXiv: Networking](<https://devfeed.tech/sources/arxiv-networking.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>), [Jetson Orin](<https://devfeed.tech/topics/jetson-orin.md>), [RTX PRO](<https://devfeed.tech/topics/rtx-pro.md>)

Tags: [5g](<https://devfeed.tech/tags/5g.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [attention](<https://devfeed.tech/tags/attention.md>), [channel](<https://devfeed.tech/tags/channel.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ni](<https://devfeed.tech/tags/cs-ni.md>), [cs-pf](<https://devfeed.tech/tags/cs-pf.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [doppler](<https://devfeed.tech/tags/doppler.md>), [eess-sp](<https://devfeed.tech/tags/eess-sp.md>), [embedded](<https://devfeed.tech/tags/embedded.md>), [energy](<https://devfeed.tech/tags/energy.md>), [estimator](<https://devfeed.tech/tags/estimator.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [fp16](<https://devfeed.tech/tags/fp16.md>), [high-performance](<https://devfeed.tech/tags/high-performance.md>), [inference](<https://devfeed.tech/tags/inference.md>), [jetson](<https://devfeed.tech/tags/jetson.md>), [latency](<https://devfeed.tech/tags/latency.md>), [leo](<https://devfeed.tech/tags/leo.md>), [low-earth-orbit](<https://devfeed.tech/tags/low-earth-orbit.md>), [non-terrestrial-networks](<https://devfeed.tech/tags/non-terrestrial-networks.md>), [ntn](<https://devfeed.tech/tags/ntn.md>)

### AI overview

The study finds HELENA's unchanged dual-attention architecture remains competitive for the evaluated LEO NTN channel-estimation task after retraining, without an NTN-specific redesign. Its SNR-averaged NMSE is within 0.29 dB of AttRNet with receiver compensation and 0.72 dB with residual impairments. HELENA beats global LMMSE in the compensated condition and all evaluated LMMSE variants under residual impairments, though profile-aware filtering is more accurate with compensation. FP16 TensorRT inference on an RTX PRO 4500 reaches 0.0573 ms P99, under the 0.5 ms budget. On a 10 W Jetson Orin NX, HELENA uses 66.0% less energy per sample than AttRNet, but its 0.9620 ms P99 misses that budget; only the less accurate LSiDNN-48 meets it. Results suggest embedded latency remains a challenge for accurate estimation.

### Source excerpt

arXiv:2609.14735v2 Announce Type: replace-cross Abstract: Deep Learning (DL)-based channel estimation has shown high accuracy and low latency in terrestrial 5G NR, but Low Earth Orbit (LEO) Non-Terrestrial Networks (NTNs) introduce Doppler and synchronization impairments that may require NTN-specific architectures. We investigate whether HELENA, a dual-attention neural channel estimator originally designed for terrestrial channels, remains effective after NTN retraining and suitable across high-performance and power-constrained inference platforms. Its unchanged architecture is trained on paired receiver-compensated and residual-impaired datasets and evaluated alongside terrestrial-origin, NTN-specific, and classical estimators. HELENA's SNR-averaged Normalized Mean Squared Error (NMSE), computed by arithmetic averaging in linear scale, remains within 0.29 and 0.72 dB of the most accurate learned estimator, AttRNet, in the two conditions, respectively. It outperforms global Linear Minimum Mean Squared Error (LMMSE) under compensation and all evaluated LMMSE variants under residual impairments, although profile-aware filtering achieves lower error under compensation. On an RTX PRO 4500, FP16 TensorRT inference reaches 0.0573 ms 99th-Percentile (P99) latency, below the adopted 0.5 ms estimator budget. On a 10 W Jetson Orin NX, HELENA consumes 66.0% less energy per sample than AttRNet, but its 0.9620 ms P99 exceeds the budget; only the less accurate LSiDNN-48 meets it. Thus, HELENA remains effective without NTN-specific redesign for the evaluated task, while accurate embedded inference remains a latency challenge.

## Run Native DLA Inference in Python on NVIDIA Jetson AGX Orin Using cuDLA

DevFeed: [Run Native DLA Inference in Python on NVIDIA Jetson AGX Orin Using cuDLA](<https://devfeed.tech/articles/run-native-dla-inference-in-python-on-nvidia-jetson-agx-orin-using-cudla-133567.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/run-native-dla-inference-in-python-on-nvidia-jetson-agx-orin-using-cudla/>)

Author: Nikshay Shrivastava

Published: 2026-11-13T18:57:10Z

Content type: tutorial

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [Jetson Orin](<https://devfeed.tech/topics/jetson-orin.md>), [Jetson](<https://devfeed.tech/topics/jetson.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-applications](<https://devfeed.tech/tags/ai-applications.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [applications](<https://devfeed.tech/tags/applications.md>), [compute](<https://devfeed.tech/tags/compute.md>), [computer-vision-video-analytics](<https://devfeed.tech/tags/computer-vision-video-analytics.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [edge](<https://devfeed.tech/tags/edge.md>), [inference](<https://devfeed.tech/tags/inference.md>), [jetson](<https://devfeed.tech/tags/jetson.md>), [networks](<https://devfeed.tech/tags/networks.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-jetson](<https://devfeed.tech/tags/nvidia-jetson.md>), [nvidia-jetson-agx-orin](<https://devfeed.tech/tags/nvidia-jetson-agx-orin.md>), [python](<https://devfeed.tech/tags/python.md>), [robotics](<https://devfeed.tech/tags/robotics.md>)

### AI overview

The excerpt describes using `cuda.bindings.cudla` 13.3.0+ to run a TensorRT-compiled DLA loadable from Python on Jetson Orin. JetPack provides `libcudla.so`; inference needs that runtime, while TensorRT is needed only to compile the model. Python 3.10 or later is required. The demonstrated workflow uses hybrid `CUDA_DLA` mode and one CUDA stream to coordinate GPU work with DLA inference: load the model bytes, query tensor descriptors, allocate and register GPU buffers, enqueue the task, synchronize, and copy results back for Python postprocessing. Descriptor fields provide sizes and layouts, but preprocessing and output interpretation remain model-specific. Not every layer is DLA-compatible, so the ONNX graph may need trimming. Standalone mode is unsupported in this release. The excerpt says the cuDLA API is identical on RTX Spark, though its detailed setup is centered on Jetson.

### Source excerpt

AI applications at the edge often need to run multiple neural networks within limited compute and power budgets. NVIDIA Deep Learning Accelerator (DLA)...

## Programmable GPU-Accelerated dApps for Real-Time Edge Inference and ISAC on NVIDIA Aerial Testbed

DevFeed: [Programmable GPU-Accelerated dApps for Real-Time Edge Inference and ISAC on NVIDIA Aerial Testbed](<https://devfeed.tech/articles/programmable-gpu-accelerated-dapps-for-real-time-edge-inference-and-isac-on-nvidia-aerial-testbed-127830.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2512.06493>)

Author: Davide Villa, Mauro Belgiovine, Nicholas Hedberg, Michele Polese, Chris Dick, Tommaso Melodia

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Networking](<https://devfeed.tech/sources/arxiv-networking.md>)

Topics: [real-time](<https://devfeed.tech/topics/real-time.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>), [network-intelligence](<https://devfeed.tech/topics/network-intelligence.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>)

Tags: [5g](<https://devfeed.tech/tags/5g.md>), [6g](<https://devfeed.tech/tags/6g.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-applications](<https://devfeed.tech/tags/ai-applications.md>), [ai-logic](<https://devfeed.tech/tags/ai-logic.md>), [ai-native](<https://devfeed.tech/tags/ai-native.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [artificial-intelligence-ai](<https://devfeed.tech/tags/artificial-intelligence-ai.md>), [backends](<https://devfeed.tech/tags/backends.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [cells](<https://devfeed.tech/tags/cells.md>), [cellular](<https://devfeed.tech/tags/cellular.md>), [cs-ni](<https://devfeed.tech/tags/cs-ni.md>), [dapps](<https://devfeed.tech/tags/dapps.md>), [edge](<https://devfeed.tech/tags/edge.md>), [edge-inference](<https://devfeed.tech/tags/edge-inference.md>), [gpu-accelerated](<https://devfeed.tech/tags/gpu-accelerated.md>), [inference](<https://devfeed.tech/tags/inference.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [testbed](<https://devfeed.tech/tags/testbed.md>)

### AI overview

The paper presents an open-source framework that feeds physical-layer signals from an NVIDIA Aerial Testbed GPU-accelerated gNB to edge AI applications using the Open RAN dApp architecture. It reports about 84 μs of framework latency per slot and about 123 μs for control loops with lightweight TensorRT inference, and evaluates two GPU platforms under different sharing modes as the number of cells and dApps increases. Its cuSense demonstration uses uplink DMRS channel estimates, removes static multipath, and applies a neural network to locate a moving person without dedicated sensing hardware or changes to 5G signals. On a 3GPP-compliant 5G NR deployment, cuSense reports 73 cm mean error on runs unseen during training, 79% of predictions within 1 m, and 241 μs live runtime. The framework and dataset are released as open source.

### Source excerpt

arXiv:2512.06493v3 Announce Type: replace Abstract: The transition of cellular networks to (i) software-based systems on commodity hardware and (ii) platforms for services beyond connectivity introduces critical system-level challenges. As sensing emerges as a key feature toward 6G standardization, supporting Integrated Sensing and Communication (ISAC) with limited bandwidth and piggybacking on communication signals, while maintaining high reliability and performance, remains a fundamental challenge. In this paper, we provide two key contributions. First, we present a programmable, open-source framework for processing PHY signals through real-time, GPU-accelerated Artificial Intelligence (AI) applications on the edge Radio Access Network (RAN) infrastructure. Building on the Open RAN dApp architecture, the framework interfaces with a GPU-accelerated gNB based on NVIDIA Aerial Testbed (ATB), feeding PHY data to custom AI logic with about 84 $\mu$s of framework latency per slot and control loops with inference in about 123 $\mu$s for lightweight TensorRT models, multiple inference engines, and support for several AI backends. We evaluate the framework on two GPU platforms, with different GPU sharing modes and an increasing number of cells and dApps. Second, we demonstrate the framework capabilities through cuSense, an indoor localization dApp that consumes uplink DMRS channel estimates, removes static multipath components, and runs a neural network to infer the position of a moving person. Evaluated on a 3GPP-compliant 5G NR deployment, cuSense achieves a mean localization error of 73 cm on runs unseen during training, with 79% of predictions falling within 1 meter, and runs live in 241 $\mu$s, without dedicated sensing hardware or modifications to the 5G signals. The framework is released as open source, together with the cuSense dataset, providing a reference design for future AI-native RANs and ISAC applications.

## How AI Helps Recyclers Turn Mixed Waste Into Reusable Material

DevFeed: [How AI Helps Recyclers Turn Mixed Waste Into Reusable Material](<https://devfeed.tech/articles/how-ai-helps-recyclers-turn-mixed-waste-into-reusable-material-129051.md>)

Original publisher: [Read original article](<https://blogs.nvidia.com/blog/ai-recycling-startups/>)

Author: Katie Michel

Published: 2026-11-11T03:00:20Z

Content type: article

Language: en

Sources: [NVIDIA](<https://devfeed.tech/sources/nvidia-blog.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-for-good](<https://devfeed.tech/tags/ai-for-good.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [conveyor-belt](<https://devfeed.tech/tags/conveyor-belt.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [data-center](<https://devfeed.tech/tags/data-center.md>), [defects](<https://devfeed.tech/tags/defects.md>), [edge](<https://devfeed.tech/tags/edge.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [healthcare-and-life-sciences](<https://devfeed.tech/tags/healthcare-and-life-sciences.md>), [inception](<https://devfeed.tech/tags/inception.md>), [jetson](<https://devfeed.tech/tags/jetson.md>), [material](<https://devfeed.tech/tags/material.md>), [materials](<https://devfeed.tech/tags/materials.md>), [robotics](<https://devfeed.tech/tags/robotics.md>), [sustainability](<https://devfeed.tech/tags/sustainability.md>), [tensorrt](<https://devfeed.tech/tags/tensorrt.md>), [vision](<https://devfeed.tech/tags/vision.md>), [vision-ai](<https://devfeed.tech/tags/vision-ai.md>), [waste](<https://devfeed.tech/tags/waste.md>)

### AI overview

AI vision systems and robots are being used to identify and sort plastics, household recyclables and textiles, addressing labor-intensive sorting and inconsistent material quality. AIRECO runs vision inference and robotic picking on NVIDIA Jetson Orin NX at a pilot plant; its test-line recognition accuracy exceeds 95%, while its capacity and cost reductions are targets for a planned production line. EverestLabs uses edge RTX GPUs for real-time robotic sorting and data center GPUs with CUDA for training; its deployed systems have identified about 40 billion objects. Greyparrot combines GPU model training with Jetson edge detection to analyze conveyor-belt contents at scale, with facilities reporting 10-30% efficiency gains and Amcor reporting a 12% increase in food-grade plastic yield. reverse.fashion uses more than a dozen high-resolution models on Jetson, optimized with TensorRT, to classify textile attributes and defects; its system is estimated to sort in an hour what previously took a week. The examples describe differing applications and reported outcomes, not a common comparative evaluation.

### Source excerpt

Vision AI and robotics pick what's worth recovering out of conveyor belts of used clothing, bottles and mixed recyclables, diverting thousands of tons of materials from landfills.

## Start and Restore GPU Workloads in Seconds with NVIDIA Snapshot

DevFeed: [Start and Restore GPU Workloads in Seconds with NVIDIA Snapshot](<https://devfeed.tech/articles/start-and-restore-gpu-workloads-in-seconds-with-nvidia-snapshot-127100.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/start-and-restore-gpu-workloads-in-seconds-with-nvidia-snapshot/>)

Author: Ron Kahn

Published: 2026-11-10T15:00:00Z

Content type: article

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [TensorRT-LLM](<https://devfeed.tech/topics/tensorrt-llm.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>), [NVIDIA DSX](<https://devfeed.tech/topics/nvidia-dsx.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-factory](<https://devfeed.tech/tags/ai-factory.md>), [ai-platforms-deployment](<https://devfeed.tech/tags/ai-platforms-deployment.md>), [checkpoint](<https://devfeed.tech/tags/checkpoint.md>), [cloud-services](<https://devfeed.tech/tags/cloud-services.md>), [component](<https://devfeed.tech/tags/component.md>), [data-center-cloud](<https://devfeed.tech/tags/data-center-cloud.md>), [dsx](<https://devfeed.tech/tags/dsx.md>), [dsx-ai-factory-platform](<https://devfeed.tech/tags/dsx-ai-factory-platform.md>), [featured](<https://devfeed.tech/tags/featured.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [inference](<https://devfeed.tech/tags/inference.md>), [kubecon](<https://devfeed.tech/tags/kubecon.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [mlops](<https://devfeed.tech/tags/mlops.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [platform](<https://devfeed.tech/tags/platform.md>), [restore](<https://devfeed.tech/tags/restore.md>), [snapshot](<https://devfeed.tech/tags/snapshot.md>)

### AI overview

NVIDIA Snapshot checkpoints an initialized GPU workload's CPU and GPU memory, then restores it as a Kubernetes Pod on a node with the same GPU model and driver. This skips weight loading, compilation, and graph capture; infrastructure provisioning and container image loading remain outside the reported restore time. Across the tested vLLM, SGLang, and TensorRT LLM configurations on B200 GPUs, restores took 3.6-42.7 seconds, 8-36 times faster than cold starts. Checkpoint creation took 15 seconds to 5.5 minutes, but checkpoint plus restore was faster than one cold start in every tested configuration. Large eight-GPU MoE restores took under 43 seconds, with GPU state transfer the largest restore stage. Workloads need to meet Snapshot's compatibility requirements and provide a small pause/resume handshake. Excerpts identify ready-made examples and multi-GPU support but do not give the full requirements.

### Source excerpt

NVIDIA Snapshot, a new open-source component of the NVIDIA DSX AI Factory Platform, makes initialized GPU workloads reusable across inference...

## A Unified Framework for License Plate Identification and Character-Recognition

DevFeed: [A Unified Framework for License Plate Identification and Character-Recognition](<https://devfeed.tech/articles/a-unified-framework-for-license-plate-identification-and-character-recognition-125354.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.11289>)

Author: Saurabh Kumar, Dewant Katare, Prakriti Jain

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [image recognition](<https://devfeed.tech/topics/image-recognition.md>), [Jetson Orin](<https://devfeed.tech/topics/jetson-orin.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>)

Tags: [ai-assisted](<https://devfeed.tech/tags/ai-assisted.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [detection](<https://devfeed.tech/tags/detection.md>), [edge](<https://devfeed.tech/tags/edge.md>), [framework](<https://devfeed.tech/tags/framework.md>), [int8](<https://devfeed.tech/tags/int8.md>), [jetson](<https://devfeed.tech/tags/jetson.md>), [jetson-orin](<https://devfeed.tech/tags/jetson-orin.md>), [jetson-orin-nano](<https://devfeed.tech/tags/jetson-orin-nano.md>), [latency](<https://devfeed.tech/tags/latency.md>), [license-plate](<https://devfeed.tech/tags/license-plate.md>), [memory](<https://devfeed.tech/tags/memory.md>), [power-consumption](<https://devfeed.tech/tags/power-consumption.md>), [recognition](<https://devfeed.tech/tags/recognition.md>), [tensorrt](<https://devfeed.tech/tags/tensorrt.md>), [throughput](<https://devfeed.tech/tags/throughput.md>)

### AI overview

The framework combines license plate detection and character recognition in one network for edge hardware. A shared CSP-Darknet/PANet encoder feeds an anchor-free localization head and a RoIAlign sequence-recognition branch that decodes characters from aligned feature maps, avoiding image-space cropping and a second recognition backbone. Trained on about 37,000 manually verified images covering all 50 U.S. states and varied capture conditions, its INT8 TensorRT deployment on an 8 GB NVIDIA Jetson Orin Nano reports 97.8% mAP50 and 90.7% full-sequence accuracy. It reaches 147 FPS at 6.8 ms end-to-end latency, using 315 MB inference memory and 9.4 W average power. Compared with the YOLOv8s+CRNN baseline, it reports 7.1x throughput and 77.8% less allocated inference memory.

### Source excerpt

arXiv:2611.11289v1 Announce Type: new Abstract: This paper presents a unified framework for license plate detection and character recognition on low-power edge hardware. The proposed network uses a shared CSP-Darknet and PANet feature encoder with an anchor-free localization head and a RoIAlign-based sequence-recognition branch. Character decoding is performed directly on aligned feature maps, avoiding image-space plate cropping and a second recognition backbone. The model is trained on an AI-assisted dataset of approximately 37,000 manually verified images covering license plate layouts from all 50 United States. The dataset includes variations in illumination, viewpoint, weather, blur, occlusion, and plate design. The framework is converted to an INT8 TensorRT engine and evaluated on an NVIDIA Jetson Orin Nano with 8 GB memory. The INT8 model shows an mAP50 of 97.8% and a full-sequence accuracy of 90.7%. It records 147 FPS at 6.8 ms end-to-end latency, with 315 MB inference memory and 9.4 W average power consumption. Relative to the YOLOv8s+CRNN baseline, the proposed deployment provides a 7.1x throughput increase and a 77.8% reduction in allocated inference memory

## HMMF-Net: A Lightweight Hierarchical Multi-Frame Multimodal Fusion Network for Accurate Fish Feeding Intensity Assessment

DevFeed: [HMMF-Net: A Lightweight Hierarchical Multi-Frame Multimodal Fusion Network for Accurate Fish Feeding Intensity Assessment](<https://devfeed.tech/articles/hmmf-net-a-lightweight-hierarchical-multi-frame-multimodal-fusion-network-for-accurate-fish-feeding-intensity-assessment-124638.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.09224>)

Author: Phi-Hung Hoang, Van-Manh Tran, Nam-Thuan Trinh, Thi-Thu-Hong Phan

Published: 2026-11-09T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>), [Jetson Orin](<https://devfeed.tech/topics/jetson-orin.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [3d](<https://devfeed.tech/tags/3d.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [adaptive](<https://devfeed.tech/tags/adaptive.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [assessment](<https://devfeed.tech/tags/assessment.md>), [attention](<https://devfeed.tech/tags/attention.md>), [automated](<https://devfeed.tech/tags/automated.md>), [av](<https://devfeed.tech/tags/av.md>), [axis](<https://devfeed.tech/tags/axis.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [best](<https://devfeed.tech/tags/best.md>), [branch](<https://devfeed.tech/tags/branch.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [fish](<https://devfeed.tech/tags/fish.md>), [frame](<https://devfeed.tech/tags/frame.md>), [fusion](<https://devfeed.tech/tags/fusion.md>), [lightweight](<https://devfeed.tech/tags/lightweight.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [network](<https://devfeed.tech/tags/network.md>)

### AI overview

HMMF-Net combines temporally separated video frames and log-mel audio features, then progressively exchanges information between them through compact messenger tokens. Its visual branch adapts 2D EfficientNetB0; its acoustic branch uses adaptive multi-axis residual units to model temporal, frequency, and local time-frequency patterns. On AV-FFIA five-fold cross-validation, it reports 96.77% accuracy and 96.88% F1 with 3.987M parameters and 1.002 GFLOPs. Against the best visual configuration's 95.49% accuracy, the authors report a 28.38% reduction in classification error. On Jetson Orin Nano with TensorRT FP32, inference took 11.54 ms per 2-second sample and the model was 18.81 MB; the abstract says accuracy remained comparable.

### Source excerpt

arXiv:2611.09224v1 Announce Type: cross Abstract: Automated fish feeding intensity assessment (FFIA) is important for precision aquaculture and feed cost optimization. However, unimodal methods are vulnerable to acoustic noise and visual disturbances, while existing multimodal frameworks often rely on computationally intensive 3D convolutions or Transformers, limiting their use on edge devices. To address these limitations, we propose HMMF-Net, a lightweight multimodal architecture that combines early multi-frame visual encoding and multi-axis acoustic modeling through a hierarchical bidirectional messenger cross-modal fusion (BMCF) mechanism. The visual branch captures temporal variations in feeding behavior by stacking temporally distant frames as a multi-channel input to an adapted 2D EfficientNetB0 backbone. The acoustic branch employs adaptive multi-axis residual units (AMARU) to extract temporal, frequency, and local time-frequency representations from log-mel spectrograms. BMCF progressively exchanges information between the two modalities at intermediate representation levels through compact messenger tokens, reducing the computational overhead of dense cross-modal attention. Evaluated on the AV-FFIA dataset using five-fold cross-validation, HMMF-Net achieves 96.77% accuracy and 96.88% F1-score on the held-out test folds, with 3.987M parameters and 1.002 GFLOPs. Compared with the best-performing visual configuration (95.49% accuracy), HMMF-Net reduces the classification error by 28.38%. Deployment on an NVIDIA Jetson Orin Nano using TensorRT with FP32 precision maintains comparable accuracy, with an inference time of 11.54 ms per 2-second sample and a model size of 18.81 MB. These results demonstrate that HMMF-Net combines effective multimodal representation learning with computational efficiency, enabling accurate and practical FFIA on resource-constrained edge platforms.

## ProCerDCL: Query-Efficient Black-Box Probabilistic Verification of Deep Learning Compiler Optimization

DevFeed: [ProCerDCL: Query-Efficient Black-Box Probabilistic Verification of Deep Learning Compiler Optimization](<https://devfeed.tech/articles/procerdcl-query-efficient-black-box-probabilistic-verification-of-deep-learning-compiler-optimization-103145.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.36499>)

Author: Zexin Li, Weibo Zhang, Linyi Li, Arvind Easwaran, Simin Chen

Published: 2026-10-28T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [Compiler](<https://devfeed.tech/topics/compiler.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [acceptance](<https://devfeed.tech/tags/acceptance.md>), [apply](<https://devfeed.tech/tags/apply.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [backend](<https://devfeed.tech/tags/backend.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [black](<https://devfeed.tech/tags/black.md>), [box](<https://devfeed.tech/tags/box.md>), [budget](<https://devfeed.tech/tags/budget.md>), [can](<https://devfeed.tech/tags/can.md>), [case-studies](<https://devfeed.tech/tags/case-studies.md>), [compiler](<https://devfeed.tech/tags/compiler.md>), [compiler-optimization](<https://devfeed.tech/tags/compiler-optimization.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-pl](<https://devfeed.tech/tags/cs-pl.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [deep](<https://devfeed.tech/tags/deep.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [learning](<https://devfeed.tech/tags/learning.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [probabilistic](<https://devfeed.tech/tags/probabilistic.md>), [query](<https://devfeed.tech/tags/query.md>), [verification](<https://devfeed.tech/tags/verification.md>)

### AI overview

ProCerDCL tests whether a binary property oracle's violation probability under a fixed, sampleable input distribution is below a threshold. Assuming independent sampling and a stateless oracle, it returns acceptance, rejection, or inconclusive, with error probabilities bounded by specified budgets; rejection requires evidence that the probability exceeds the threshold by a user-set margin. A single sequential test handles repeated checks using corrected exact binomial intervals or an anytime-valid confidence sequence. Applied to decision-level consistency between reference and compiled model executions, it covers TorchInductor and TensorRT. In paired clean-label radius searches, the baseline used 3.6-4.2 times more unique samples under matched parameters and offered a weaker decision contract. At a threshold of 10⁻³, checks across 840 input-backend instances from 21 tasks produced 817 acceptances, 19 inconclusive verdicts, and 4 rejections. The abstract also reports diagnostic case studies of two documented compiler faults.

### Source excerpt

arXiv:2610.36499v1 Announce Type: cross Abstract: We present \tool, a query-efficient framework for black-box probabilistic verification. Given a fixed, sampleable input distribution and a binary property oracle, \tool assesses the property's violation probability $p$ against a threshold $\theta$. Under independent sampling and stateless oracle behavior, acceptance asserts $p\le\theta$, rejection asserts $p>\theta+\eta$ for a user-specified separation margin, and an inconclusive verdict indicates insufficient evidence for either conclusion. The probability of each type of erroneous verdict is bounded by its specified error budget. To support this three-valued contract efficiently, \tool uses a single sequential test, eliminating the interval family required by the baseline. The test accounts for repeated sampling checks through either corrected exact binomial confidence intervals or an anytime-valid confidence sequence. We apply \tool to probabilistic local model semantic consistency, where the property oracle detects decision-level inconsistencies between reference and compiled executions under a specified perturbation distribution. Our evaluation covers 24 tasks across TorchInductor and TensorRT and examines sampling efficiency, conclusiveness, and diagnostic behavior. For sampling efficiency, we use paired one-model clean-label radius searches: the baseline requires 3.6-4.2 times as many unique samples under matched numerical parameters while providing a weaker decision contract. For model semantic consistency, checks at $\theta=10^{-3}$ yield 817 acceptances, 19 inconclusive verdicts, and 4 rejections across 840 input--backend instances from 21 tasks. Case studies of two documented compiler faults further demonstrate what these local verdicts can diagnose.

## CrowdCC: Jointly Benchmarking Corruption Robustness and Deployment Quantization in Crowd Counting

DevFeed: [CrowdCC: Jointly Benchmarking Corruption Robustness and Deployment Quantization in Crowd Counting](<https://devfeed.tech/articles/crowdcc-jointly-benchmarking-corruption-robustness-and-deployment-quantization-in-crowd-counting-103119.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.36340>)

Author: S Mahmudul Hasan, Lu Peng

Published: 2026-10-28T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [quantization](<https://devfeed.tech/topics/quantization.md>), [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>), [Jetson Orin](<https://devfeed.tech/topics/jetson-orin.md>)

Tags: [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [inference-efficiency](<https://devfeed.tech/tags/inference-efficiency.md>), [measurements](<https://devfeed.tech/tags/measurements.md>), [precision](<https://devfeed.tech/tags/precision.md>), [quantization](<https://devfeed.tech/tags/quantization.md>), [robustness](<https://devfeed.tech/tags/robustness.md>)

### AI overview

CrowdCC jointly benchmarks corruption robustness and deployment quantization for crowd counting. It evaluates 26 models across four datasets and 23 corruptions at three severity levels, using TensorRT engines at FP32, FP16, and post-training INT8, with throughput and energy measurements on Jetson Orin NX configurations. Capture-chain corruptions caused the largest degradation, and clean accuracy did not fully predict corrupted performance. INT8 changed robustness more than FP16, without a consistent direction, and did not reliably improve latency or energy; reformat overhead and limited INT8 coverage were associated with inefficiency or slowdowns. The results show that precision can change error rankings and robustness-cost tradeoffs, so model selection should account for both robustness and measured deployment cost.

### Source excerpt

arXiv:2610.36340v1 Announce Type: cross Abstract: Crowd-counting models deployed on edge devices face two shifts: inputs are degraded along the capture and transmission pipeline, while models are quantized to satisfy resource constraints. Yet corruption robustness and deployment quantization are typically evaluated separately. We introduce CrowdCC, a controlled corruption benchmark that jointly evaluates these effects using 23 deployment-motivated corruptions at three severity levels, four crowd-counting datasets, and 26 models using A100 TensorRT engines at FP32, FP16, and post-training INT8 precision, paired with throughput and energy measurements from corresponding Jetson Orin NX configurations. The corruptions span six families, with head annotations adjusted when geometry or visibility changes. Our evaluation shows that capture-chain corruptions consistently produce the largest degradation across the evaluated models, while crowd density affects absolute and relative error differently. Clean accuracy does not fully predict corrupted performance. INT8 alters corruption robustness substantially more than FP16, with no systematic direction, and does not consistently improve latency or energy. INT8 inference efficiency is strongly associated with reformat overhead in the compiled engine, while slowdowns are also observed under limited INT8 coverage. Precision changes corrupted-error rankings and robustness-cost frontiers, showing why model selection must consider corruption robustness and measured deployment cost together. The code is available at https://github.com/Tulane-Computer-Architecture-Systems/CrowdCC-Release.

## Scaling NVIDIA TensorRT Performance Optimization Beyond a Single Build Route with the Global Performance Tuner, Part 2

DevFeed: [Scaling NVIDIA TensorRT Performance Optimization Beyond a Single Build Route with the Global Performance Tuner, Part 2](<https://devfeed.tech/articles/scaling-nvidia-tensorrt-performance-optimization-beyond-a-single-build-route-with-the-global-performance-tuner-part-2-97809.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/scaling-nvidia-tensorrt-performance-optimization-beyond-a-single-build-route-with-the-global-performance-tuner-part-2/>)

Author: Tanya Lenz

Published: 2026-10-23T17:27:52Z

Content type: article

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [TensorRT](<https://devfeed.tech/topics/tensorrt.md>), [High-Performance Computing](<https://devfeed.tech/topics/high-performance-computing.md>), [performance-optimization](<https://devfeed.tech/topics/performance-optimization.md>)

Tags: [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [algorithms-numerical-techniques](<https://devfeed.tech/tags/algorithms-numerical-techniques.md>), [api](<https://devfeed.tech/tags/api.md>), [best-practices](<https://devfeed.tech/tags/best-practices.md>), [c-plus-plus](<https://devfeed.tech/tags/c-plus-plus.md>), [compilation](<https://devfeed.tech/tags/compilation.md>), [compiler](<https://devfeed.tech/tags/compiler.md>), [compiler-optimization](<https://devfeed.tech/tags/compiler-optimization.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [data-center-cloud](<https://devfeed.tech/tags/data-center-cloud.md>), [developer-tools-techniques](<https://devfeed.tech/tags/developer-tools-techniques.md>), [edge-computing](<https://devfeed.tech/tags/edge-computing.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [performance](<https://devfeed.tech/tags/performance.md>), [performance-optimization](<https://devfeed.tech/tags/performance-optimization.md>), [performance-tuning](<https://devfeed.tech/tags/performance-tuning.md>), [tensorrt](<https://devfeed.tech/tags/tensorrt.md>)

### AI overview

TensorRT 11.1's Global Performance Tuner searches model-specific build routes and selects the fastest measured route that passes the user-defined accuracy requirement. A route specifies tunable compiler knob values; TensorRT still performs the remaining engine-building work, including tactic selection. In `trtexec`, `fast` varies one knob at a time, `mixed` adds combinations informed by that scan, and `full` tests the complete combination space for selected knobs. `--dryRun` previews `fast` and `full` route lists, while a tuning cache and `--continue` support resuming interrupted sweeps; `--saveAllEngines` aids debugging at potentially substantial storage cost. Across ten evaluated workloads, the excerpt reports 1.14x-1.63x improvements over each workload's default route. Results depend on model, shapes, hardware, software stack, and tuner versions, so routes need validation in the deployment environment. The C++ APIs enable custom tuners; a Python implementation is planned for Torch-TensorRT 2.15.

### Source excerpt

The Global Performance Tuner is a feature for automatic performance tuning released in NVIDIA TensorRT 11.1. It provides automatic, accuracy-aware and...

## Scaling NVIDIA TensorRT Performance Optimization Beyond a Single Build Route with the Global Performance Tuner, Part 1

DevFeed: [Scaling NVIDIA TensorRT Performance Optimization Beyond a Single Build Route with the Global Performance Tuner, Part 1](<https://devfeed.tech/articles/scaling-nvidia-tensorrt-performance-optimization-beyond-a-single-build-route-with-the-global-performance-tuner-part-1-97808.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/scaling-nvidia-tensorrt-performance-optimization-beyond-a-single-build-route-with-the-global-performance-tuner-part-1/>)

Author: Tanya Lenz

Published: 2026-10-23T17:27:35Z

Content type: article

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [TensorRT](<https://devfeed.tech/topics/tensorrt.md>), [performance-optimization](<https://devfeed.tech/topics/performance-optimization.md>), [quantization](<https://devfeed.tech/topics/quantization.md>)

Tags: [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [algorithm](<https://devfeed.tech/tags/algorithm.md>), [algorithms-numerical-techniques](<https://devfeed.tech/tags/algorithms-numerical-techniques.md>), [compilation](<https://devfeed.tech/tags/compilation.md>), [compiler](<https://devfeed.tech/tags/compiler.md>), [data-center-cloud](<https://devfeed.tech/tags/data-center-cloud.md>), [developer-tools-techniques](<https://devfeed.tech/tags/developer-tools-techniques.md>), [edge-computing](<https://devfeed.tech/tags/edge-computing.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [neural-network](<https://devfeed.tech/tags/neural-network.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [performance](<https://devfeed.tech/tags/performance.md>), [performance-optimization](<https://devfeed.tech/tags/performance-optimization.md>), [scaling](<https://devfeed.tech/tags/scaling.md>), [tensorrt](<https://devfeed.tech/tags/tensorrt.md>)

### AI overview

TensorRT's Global Performance Tuner, released in TensorRT 11.1, addresses the limits of relying on shared scheduling heuristics as models and deployment environments diversify. The article frames compilation as two linked tasks: generating candidate engine solutions and selecting among them. A tuner can expand the available build routes, validate each candidate against an accuracy tolerance, and measure end-to-end inference time to select the fastest feasible solution. This makes accuracy a constraint on performance tuning and allows routes to adapt per model and environment. The framework's limits include build-time tradeoffs and imperfect candidate selection: heuristics may prune the best tactic, and isolated kernel profiling may not predict whole-engine runtime. The excerpt introduces the framework and mathematical model; it defers algorithm details and usage practices to Part 2 and documentation.

### Source excerpt

Deploying a trained neural network involves more than translating each operator into a GPU kernel. It is a compilation problem with many legitimate answers. For...

## NVIDIA Transforms European Retail, From Digital Discovery to Physical Operations

DevFeed: [NVIDIA Transforms European Retail, From Digital Discovery to Physical Operations](<https://devfeed.tech/articles/nvidia-transforms-european-retail-from-digital-discovery-to-physical-operations-95047.md>)

Original publisher: [Read original article](<https://blogs.nvidia.com/blog/european-retail-open-models-robotics/>)

Author: Mavi Nafria

Published: 2026-10-22T05:00:23Z

Content type: news

Language: en

Sources: [NVIDIA](<https://devfeed.tech/sources/nvidia-blog.md>)

Topics: [Physical AI](<https://devfeed.tech/topics/physical-ai.md>), [Omniverse](<https://devfeed.tech/topics/omniverse.md>), [Robotics Simulation](<https://devfeed.tech/topics/robotics-simulation.md>), [3D](<https://devfeed.tech/topics/3d.md>), [Nemotron](<https://devfeed.tech/topics/nemotron.md>), [NeMo](<https://devfeed.tech/topics/nemo.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>), [RTX PRO](<https://devfeed.tech/topics/rtx-pro.md>), [digital sovereignty](<https://devfeed.tech/topics/digital-sovereignty.md>), [Image](<https://devfeed.tech/topics/image.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>)

Tags: [accelerated](<https://devfeed.tech/tags/accelerated.md>), [ai](<https://devfeed.tech/tags/ai.md>), [collaboration](<https://devfeed.tech/tags/collaboration.md>), [commerce](<https://devfeed.tech/tags/commerce.md>), [costs](<https://devfeed.tech/tags/costs.md>), [digital](<https://devfeed.tech/tags/digital.md>), [digital-twin](<https://devfeed.tech/tags/digital-twin.md>), [discovery](<https://devfeed.tech/tags/discovery.md>), [europe](<https://devfeed.tech/tags/europe.md>), [gtc-2026](<https://devfeed.tech/tags/gtc-2026.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-costs](<https://devfeed.tech/tags/inference-costs.md>), [intelligence](<https://devfeed.tech/tags/intelligence.md>), [models](<https://devfeed.tech/tags/models.md>), [nemotron](<https://devfeed.tech/tags/nemotron.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [open](<https://devfeed.tech/tags/open.md>), [operations](<https://devfeed.tech/tags/operations.md>), [platform](<https://devfeed.tech/tags/platform.md>), [results](<https://devfeed.tech/tags/results.md>), [retail](<https://devfeed.tech/tags/retail.md>), [retail-and-consumer-packaged-goods](<https://devfeed.tech/tags/retail-and-consumer-packaged-goods.md>), [robotic](<https://devfeed.tech/tags/robotic.md>), [technology](<https://devfeed.tech/tags/technology.md>)

### AI overview

European retailers in the excerpt use open models and physical AI for commerce, product discovery, research and warehouse planning. Shopify reports that post-training Nemotron 3.5 Lightning for a Sidekick GraphQL subagent beat a leading frontier model on its internal accuracy benchmark while reducing serving costs 90% and making responses 7x faster; its vLLM optimization also improved five other use cases. Zooplus' AWS prototype reached 86% search accuracy. Photoroom standardized about 40,000 Vestiaire Collective listing images daily and reported higher sales liquidity and product views. For operations, Otto Group simulates warehouse changes before physical implementation, while ALLSIDES creates SimReady digital twins for store simulation and product tracking. L'Oréal uses accelerated molecular screening to narrow ingredient combinations before lab testing. These are company-reported results; the excerpt does not establish independent comparisons across deployments.

### Source excerpt

European retailers are using open models and physical intelligence across use cases -- from a commerce platform cutting inference costs by 90% to to a fulfillment center where collaboration of different robotic fleets is simulated inside a digital twin. Frontier Results With Open Models Retail leaders and technology companies across Europe are using NVIDIA-accelerated open [...]

## Reliability-Preserving Compression for Deployable Medical Image Segmentation

DevFeed: [Reliability-Preserving Compression for Deployable Medical Image Segmentation](<https://devfeed.tech/articles/reliability-preserving-compression-for-deployable-medical-image-segmentation-93486.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.25460>)

Author: Sagar Kumar, Shailja Sharma, Tirthajit Baruah, Punit Rathore

Published: 2026-10-21T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [quantization](<https://devfeed.tech/topics/quantization.md>), [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>), [Medical imaging](<https://devfeed.tech/topics/medical-imaging.md>), [Compression](<https://devfeed.tech/topics/compression.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>)

Tags: [3d](<https://devfeed.tech/tags/3d.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [actions](<https://devfeed.tech/tags/actions.md>), [aggregate](<https://devfeed.tech/tags/aggregate.md>), [also](<https://devfeed.tech/tags/also.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [backend](<https://devfeed.tech/tags/backend.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [benefit](<https://devfeed.tech/tags/benefit.md>), [but](<https://devfeed.tech/tags/but.md>), [can](<https://devfeed.tech/tags/can.md>), [changes](<https://devfeed.tech/tags/changes.md>), [compression](<https://devfeed.tech/tags/compression.md>), [cost](<https://devfeed.tech/tags/cost.md>), [critical](<https://devfeed.tech/tags/critical.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [distillation](<https://devfeed.tech/tags/distillation.md>), [eess-iv](<https://devfeed.tech/tags/eess-iv.md>), [eess-sp](<https://devfeed.tech/tags/eess-sp.md>), [image](<https://devfeed.tech/tags/image.md>), [medical](<https://devfeed.tech/tags/medical.md>), [reliability](<https://devfeed.tech/tags/reliability.md>)

### AI overview

SGAV-MP compresses pretrained MRI/CT segmentation models by selecting layer precisions under reliability constraints. It ranks quantization actions with a benefit-risk score and accepts them only after full-model reliability validation on a calibration set, addressing the concern that Dice can remain stable while uncertainty changes. Across five 2D backbones and three 3D benchmarks, it reportedly retains FP32-level Dice and high uncertainty-map agreement, with compression of 3.59x-7.8x in 2D and 4.73x-7.82x in 3D. TensorRT deployment measurements are reported separately from storage compression.

### Source excerpt

arXiv:2610.25460v1 Announce Type: cross Abstract: Post-training compression of pretrained medical segmentation models commonly uses quantization, pruning, or distillation to reduce model size and computational cost. Existing medical segmentation quantization studies mainly evaluate Dice, model size, memory, and latency, but this is incomplete for safety-critical use because compressed models should also preserve uncertainty signals that indicate unreliable predictions. We study this problem in compressed MRI/CT segmentation and formalize why Dice can remain stable while uncertainty changes. Motivated by this failure mode, we propose score-guided aggregate-validated mixed precision (SGAV-MP), a post-training compression framework that selects layer precisions under reliability constraints. SGAV-MP ranks quantization actions with a compact benefit-risk score and accepts them only after full-model reliability validation on a calibration set. Across five 2D backbones and three 3D benchmarks, SGAV-MP preserves FP32-level Dice and high uncertainty-map agreement, achieving $3.59\times$--$7.8\times$ compression in 2D and $4.73\times$--$7.82\times$ in 3D. We also report backend-supported TensorRT deployment measurements separately to distinguish runtime execution from reliability-preserving storage compression.

## How Shopify turns open models into task experts

DevFeed: [How Shopify turns open models into task experts](<https://devfeed.tech/articles/how-shopify-turns-open-models-into-task-experts-91330.md>)

Original publisher: [Read original article](<https://shopify.engineering/open-model-task-experts>)

Author: Andrew McNamara

Published: 2026-10-21T02:12:40Z

Content type: article

Language: en

Sources: [Shopify Engineering](<https://devfeed.tech/sources/shopify-engineering.md>), [Shopify Engineering - Shopify Engineering](<https://devfeed.tech/sources/shopify-engineering-shopify-engineering.md>)

Topics: [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Nemotron](<https://devfeed.tech/topics/nemotron.md>), [Open Source Models & Datasets](<https://devfeed.tech/topics/open-source-models-datasets.md>), [Frontier AI](<https://devfeed.tech/topics/frontier-ai.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>), [Shopify](<https://devfeed.tech/topics/shopify.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [3](<https://devfeed.tech/tags/3.md>), [ace](<https://devfeed.tech/tags/ace.md>), [added](<https://devfeed.tech/tags/added.md>), [build](<https://devfeed.tech/tags/build.md>), [experts](<https://devfeed.tech/tags/experts.md>), [faster](<https://devfeed.tech/tags/faster.md>), [frontier](<https://devfeed.tech/tags/frontier.md>), [frontier-model](<https://devfeed.tech/tags/frontier-model.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [open](<https://devfeed.tech/tags/open.md>), [post](<https://devfeed.tech/tags/post.md>), [shopify](<https://devfeed.tech/tags/shopify.md>), [software](<https://devfeed.tech/tags/software.md>), [token](<https://devfeed.tech/tags/token.md>), [trains](<https://devfeed.tech/tags/trains.md>), [work](<https://devfeed.tech/tags/work.md>)

### AI overview

Shopify's Sidekick keeps a frontier model for merchant interaction and planning, while open-model subagents handle focused, token-heavy tasks. For GraphQL, Shopify post-trained NVIDIA Nemotron 3.5 Lightning through a loop that scores quality with a judge calibrated to human labels, improves the frontier-agent baseline, benchmarks open models and serving stacks, turns failed production conversations into training examples, and retrains on NVIDIA hardware. Across six loops, the model's GraphQL score rose from 37.7 to 74.5, versus 73 for the frontier model. In production-traffic replay, it returned responses at a 0.87-second median versus 6.50 seconds for the frontier API, with the slowest 1% finishing 8x sooner and serving costing about 90% less. Self-healing repaired 69% of attempted failures across 86 nightly runs. Tool results varied by workload: TensorRT-LLM did the most work per GPU for Buyer Profile; NVIDIA measured SimGym 4.5x faster on B200 than H200 at 1.48x lower cost. An Analytics training job ran 1.86x faster on eight B300s than eight H200s, using 46% fewer GPU-hours.

### Source excerpt

Sidekick's frontier model hands its most token-heavy work to open models that Shopify post-trains and serves itself. Here's the loop we use to build them, and where NVIDIA's models, software, and GPUs make each turn faster and cheaper.

## EBU expands its sovereign media framework as Adobe and NVIDIA speed video search

DevFeed: [EBU expands its sovereign media framework as Adobe and NVIDIA speed video search](<https://devfeed.tech/articles/nvidia-advances-agentic-security-for-media-from-european-public-service-broadcasting-to-global-content-production-90255.md>)

Original publisher: [Read original article](<https://blogs.nvidia.com/blog/media-agentic-security-european-public-service-broadcasting-global-content-production/>)

Author: Richard Kerris

Published: 2026-10-19T15:00:31Z

Content type: news

Language: en

Sources: [NVIDIA](<https://devfeed.tech/sources/nvidia-blog.md>)

Topics: [Holoscan for Media](<https://devfeed.tech/topics/holoscan-for-media.md>), [Nemotron](<https://devfeed.tech/topics/nemotron.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>), [CUDA-X](<https://devfeed.tech/topics/cuda-x.md>), [FedRAMP Rev 5](<https://devfeed.tech/topics/fedramp-rev-5.md>)

Tags: [accelerated-computing](<https://devfeed.tech/tags/accelerated-computing.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [broadcasting](<https://devfeed.tech/tags/broadcasting.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [content](<https://devfeed.tech/tags/content.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [customer-stories](<https://devfeed.tech/tags/customer-stories.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [developers](<https://devfeed.tech/tags/developers.md>), [events](<https://devfeed.tech/tags/events.md>), [gtc-2026](<https://devfeed.tech/tags/gtc-2026.md>), [media-and-entertainment](<https://devfeed.tech/tags/media-and-entertainment.md>), [news](<https://devfeed.tech/tags/news.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-in-europe](<https://devfeed.tech/tags/nvidia-in-europe.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [production](<https://devfeed.tech/tags/production.md>), [public](<https://devfeed.tech/tags/public.md>), [security](<https://devfeed.tech/tags/security.md>), [sovereign-ai](<https://devfeed.tech/tags/sovereign-ai.md>)

### AI overview

The European Broadcasting Union and NVIDIA are expanding a framework for European public service media, defining requirements for sovereign cloud deployments, open models and secure, governed systems. Separately, Adobe is using NVIDIA GPUs and software to speed video search in Frame.io; the article reports faster embedding generation, higher throughput and lower search latency, and describes semantic search tools used by German producer UFA.

### Source excerpt

At a creative industry event that took place today in Berlin -- hosted in collaboration with Adobe, Bertelsmann, NVIDIA and UFA -- the latest advancements in intelligent media were on display. Key to the news: the European Broadcasting Union (EBU) is collaborating with NVIDIA to expand its sovereign framework with cloud, open models and integration [...]

## NVIDIA Releases Nemotron 3.5 Super VL, a 120B Open Multimodal Agent Model

DevFeed: [NVIDIA Releases Nemotron 3.5 Super VL, a 120B Open Multimodal Agent Model](<https://devfeed.tech/articles/nvidia-releases-nemotron-3-5-super-vl-a-120b-open-multimodal-agent-model-88271.md>)

Original publisher: [Read original article](<https://www.marktechpost.com/2026/10/15/nvidia-nemotron-3-5-super-vl-release/>)

Author: Asif Razzaq

Published: 2026-10-15T19:53:50Z

Content type: news

Language: en

Sources: [MarkTechPost](<https://devfeed.tech/sources/marktechpost.md>)

Topics: [Nemotron](<https://devfeed.tech/topics/nemotron.md>), [smolvlm](<https://devfeed.tech/topics/smolvlm.md>), [NVIDIA DGX](<https://devfeed.tech/topics/nvidia-dgx.md>), [quantization](<https://devfeed.tech/topics/quantization.md>), [sglang](<https://devfeed.tech/topics/sglang.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>), [TensorRT-LLM](<https://devfeed.tech/topics/tensorrt-llm.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agent-harness](<https://devfeed.tech/tags/agent-harness.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [ai-shorts](<https://devfeed.tech/tags/ai-shorts.md>), [applications](<https://devfeed.tech/tags/applications.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [edge](<https://devfeed.tech/tags/edge.md>), [editors-pick](<https://devfeed.tech/tags/editors-pick.md>), [enterprise-ai](<https://devfeed.tech/tags/enterprise-ai.md>), [for-devs](<https://devfeed.tech/tags/for-devs.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [moe](<https://devfeed.tech/tags/moe.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [nemotron](<https://devfeed.tech/tags/nemotron.md>), [new-releases](<https://devfeed.tech/tags/new-releases.md>), [nvfp4](<https://devfeed.tech/tags/nvfp4.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [software-engineering](<https://devfeed.tech/tags/software-engineering.md>), [staff](<https://devfeed.tech/tags/staff.md>), [tech-news](<https://devfeed.tech/tags/tech-news.md>), [technology](<https://devfeed.tech/tags/technology.md>)

### AI overview

NVIDIA released Nemotron 3.5 Super VL, an open 120-billion-parameter multimodal mixture-of-experts model with 12 billion active parameters and a context window of up to 1 million tokens. The article describes its architecture, reported benchmark results, deployment options and customization tools, and introduces the smaller Nemotron 3.5 Bolt models for local and embedded agents. NVIDIA-reported results and comparisons are identified as vendor claims in the article.

### Source excerpt

NVIDIA released Nemotron 3.5 Super VL, an open 120B-parameter multimodal MoE model with 12B active parameters, 1M-token context and NVFP4 weights that run on 1 DGX Spark, plus 2 Bolt edge models. The post NVIDIA Releases Nemotron 3.5 Super VL, a 120B Open Multimodal Agent Model appeared first on MarkTechPost.

## NVIDIA highlights U.K. AI deployments across healthcare, finance and other sectors

DevFeed: [NVIDIA highlights U.K. AI deployments across healthcare, finance and other sectors](<https://devfeed.tech/articles/from-enterprise-to-earthshot-uk-s-ecosystem-moves-from-ambition-to-application-with-nvidia-86544.md>)

Original publisher: [Read original article](<https://blogs.nvidia.com/blog/uk-ecosystem-deployment/>)

Author: Anthony Hills

Published: 2026-10-14T06:04:15Z

Content type: news

Language: en

Sources: [NVIDIA](<https://devfeed.tech/sources/nvidia-blog.md>)

Topics: [NVIDIA DGX](<https://devfeed.tech/topics/nvidia-dgx.md>), [Nemotron](<https://devfeed.tech/topics/nemotron.md>), [TensorRT-LLM](<https://devfeed.tech/topics/tensorrt-llm.md>), [BioNeMo](<https://devfeed.tech/topics/bionemo.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>), [NVIDIA NIM](<https://devfeed.tech/topics/nvidia-nim.md>)

Tags: [agentic](<https://devfeed.tech/tags/agentic.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-compute](<https://devfeed.tech/tags/ai-compute.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [build](<https://devfeed.tech/tags/build.md>), [building](<https://devfeed.tech/tags/building.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [compute](<https://devfeed.tech/tags/compute.md>), [computing](<https://devfeed.tech/tags/computing.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [developers](<https://devfeed.tech/tags/developers.md>), [developing](<https://devfeed.tech/tags/developing.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [enterprises](<https://devfeed.tech/tags/enterprises.md>), [financial-services](<https://devfeed.tech/tags/financial-services.md>), [healthcare-and-life-sciences](<https://devfeed.tech/tags/healthcare-and-life-sciences.md>), [inception](<https://devfeed.tech/tags/inception.md>), [nemotron](<https://devfeed.tech/tags/nemotron.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-blackwell](<https://devfeed.tech/tags/nvidia-blackwell.md>), [nvidia-in-europe](<https://devfeed.tech/tags/nvidia-in-europe.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [quantum-computing](<https://devfeed.tech/tags/quantum-computing.md>), [scale](<https://devfeed.tech/tags/scale.md>), [smart-spaces](<https://devfeed.tech/tags/smart-spaces.md>), [startups](<https://devfeed.tech/tags/startups.md>), [uk](<https://devfeed.tech/tags/uk.md>)

### AI overview

NVIDIA describes U.K. deployments and partnerships using its AI computing platforms across energy, healthcare, public services, finance and quantum research. Examples include AI models and infrastructure for customer support, medical triage and speech recognition, banking operations, and quantum error correction.

### Source excerpt

Across industries, the U.K. is putting the full stack to work -- and the scale of deployment is growing fast, from enterprises transforming their businesses to startups solving some of the planet's hardest problems. NVIDIA is putting that principle into practice, including through a five-year, $250 million compute commitment to strengthen the U.K.'s agentic safety [...]

## Build Local AI Apps with C++ and NVIDIA TensorRT RTX Samples

DevFeed: [Build Local AI Apps with C++ and NVIDIA TensorRT RTX Samples](<https://devfeed.tech/articles/build-local-ai-apps-with-c-and-nvidia-tensorrt-rtx-samples-63572.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/build-local-ai-apps-with-c-and-nvidia-tensorrt-rtx-samples/>)

Author: Luca Spindler

Published: 2026-10-01T17:59:27Z

Content type: tutorial

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [Local AI](<https://devfeed.tech/topics/local-ai.md>), [onnx runtime](<https://devfeed.tech/topics/onnx-runtime.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>), [llama.cpp](<https://devfeed.tech/topics/llama-cpp.md>)

Tags: [build](<https://devfeed.tech/tags/build.md>), [c-plus-plus](<https://devfeed.tech/tags/c-plus-plus.md>), [cli](<https://devfeed.tech/tags/cli.md>), [cmake](<https://devfeed.tech/tags/cmake.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [conversion](<https://devfeed.tech/tags/conversion.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [deploy](<https://devfeed.tech/tags/deploy.md>), [developer-tools-techniques](<https://devfeed.tech/tags/developer-tools-techniques.md>), [dgx-spark](<https://devfeed.tech/tags/dgx-spark.md>), [directx](<https://devfeed.tech/tags/directx.md>), [execution](<https://devfeed.tech/tags/execution.md>), [featured](<https://devfeed.tech/tags/featured.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-acceleration](<https://devfeed.tech/tags/gpu-acceleration.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [image](<https://devfeed.tech/tags/image.md>), [image-generation](<https://devfeed.tech/tags/image-generation.md>), [inference](<https://devfeed.tech/tags/inference.md>), [local-ai](<https://devfeed.tech/tags/local-ai.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [runtime](<https://devfeed.tech/tags/runtime.md>), [tensorrt](<https://devfeed.tech/tags/tensorrt.md>)

### AI overview

DIN Deploy is an open-source collection of C++ samples for integrating local AI models into native Windows and Linux applications. The article explains its Python-to-ONNX export and ONNX Runtime deployment workflow, TensorRT RTX acceleration, and examples for speech recognition, segmentation, and image generation.

### Source excerpt

Adding AI models to local applications requires a portable model format, a reliable runtime, and acceleration that works across target systems. Do Inference Now...

## AI Native by Design: Lessons Learned from Building NVIDIA TensorRT Model Connect

DevFeed: [AI Native by Design: Lessons Learned from Building NVIDIA TensorRT Model Connect](<https://devfeed.tech/articles/ai-native-by-design-lessons-learned-from-building-nvidia-tensorrt-model-connect-62112.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/ai-native-by-design-lessons-learned-from-building-nvidia-tensorrt-model-connect/>)

Author: Tanya Lenz

Published: 2026-09-29T19:10:51Z

Content type: article

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [software development triangle](<https://devfeed.tech/topics/software-development-triangle.md>), [DGX Station](<https://devfeed.tech/topics/dgx-station.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>)

Tags: [agentic-ai-generative-ai](<https://devfeed.tech/tags/agentic-ai-generative-ai.md>), [code](<https://devfeed.tech/tags/code.md>), [coding-agents](<https://devfeed.tech/tags/coding-agents.md>), [data-center-cloud](<https://devfeed.tech/tags/data-center-cloud.md>), [developer-tools-techniques](<https://devfeed.tech/tags/developer-tools-techniques.md>), [featured](<https://devfeed.tech/tags/featured.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [isolation](<https://devfeed.tech/tags/isolation.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [training-ai-models](<https://devfeed.tech/tags/training-ai-models.md>), [validation](<https://devfeed.tech/tags/validation.md>)

### AI overview

The NVIDIA team describes building TensorRT Model Connect, an open source collection of C++ model reference implementations, around coding agents. The project uses independent workstreams, model-family isolation, reversible changes, and automated validation so candidate implementations can be evaluated with tests, reference comparisons, benchmarks, and human review.

### Source excerpt

Parallel work, model-family isolation, reversible changes, and GPU-backed validation shaped an open source project designed around coding agents NVIDIA TensorRT...

## Simplifying Model Serving Across Multiple GPUs with NVIDIA TensorRT Multi-Device Integration in NVIDIA Dynamo-Triton

DevFeed: [Simplifying Model Serving Across Multiple GPUs with NVIDIA TensorRT Multi-Device Integration in NVIDIA Dynamo-Triton](<https://devfeed.tech/articles/simplifying-model-serving-across-multiple-gpus-with-nvidia-tensorrt-multi-device-integration-in-nvidia-dynamo-triton-57601.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/simplifying-model-serving-across-multiple-gpus-with-nvidia-tensorrt-multi-device-integration-in-nvidia-dynamo-triton/>)

Author: Tanya Lenz

Published: 2026-09-21T21:51:06Z

Content type: tutorial

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [TensorRT](<https://devfeed.tech/topics/tensorrt.md>), [Dynamo-Triton](<https://devfeed.tech/topics/dynamo-triton.md>), [model-serving](<https://devfeed.tech/topics/model-serving.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>), [gRPC](<https://devfeed.tech/topics/grpc.md>), [Parallelism](<https://devfeed.tech/topics/parallelism.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [Deployment](<https://devfeed.tech/topics/deployment.md>)

Tags: [agentic-ai-generative-ai](<https://devfeed.tech/tags/agentic-ai-generative-ai.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [backend](<https://devfeed.tech/tags/backend.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [developer-tools-techniques](<https://devfeed.tech/tags/developer-tools-techniques.md>), [dynamo-triton](<https://devfeed.tech/tags/dynamo-triton.md>), [featured](<https://devfeed.tech/tags/featured.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [grpc](<https://devfeed.tech/tags/grpc.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-performance](<https://devfeed.tech/tags/inference-performance.md>), [latency](<https://devfeed.tech/tags/latency.md>), [model-serving](<https://devfeed.tech/tags/model-serving.md>), [multi-gpu](<https://devfeed.tech/tags/multi-gpu.md>), [nccl](<https://devfeed.tech/tags/nccl.md>), [networking-communications](<https://devfeed.tech/tags/networking-communications.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [parallelism](<https://devfeed.tech/tags/parallelism.md>), [scaling](<https://devfeed.tech/tags/scaling.md>), [tensorrt](<https://devfeed.tech/tags/tensorrt.md>)

### AI overview

This tutorial explains how TensorRT multi-device inference runs a single network across multiple GPUs using NCCL-backed collectives, while Dynamo-Triton exposes one gRPC model endpoint. It demonstrates the integration with NVIDIA Cosmos 3 Nano video generation and describes distributing the denoising transformer across up to eight GPUs.

### Source excerpt

The compute and memory demands of generative AI increasingly exceed what a single GPU can provide. NVIDIA TensorRT multi-device inference is a new capability...

## TensorRT Edge-LLM Completes the MLPerf Edge Agentic Benchmark 6.4x Faster on Jetson AGX Thor

DevFeed: [TensorRT Edge-LLM Completes the MLPerf Edge Agentic Benchmark 6.4x Faster on Jetson AGX Thor](<https://devfeed.tech/articles/tensorrt-edge-llm-completes-the-mlperf-edge-agentic-benchmark-6-4x-faster-on-jetson-agx-thor-31485.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/tensorrt-edge-llm-completes-the-mlperf-edge-agentic-benchmark-6-4x-faster-on-jetson-agx-thor/>)

Author: Elizabeth Goodman

Published: 2026-09-16T20:37:07Z

Content type: article

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [Jetson AGX Thor](<https://devfeed.tech/topics/jetson-agx-thor.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [quantization](<https://devfeed.tech/topics/quantization.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [agentic-ai-generative-ai](<https://devfeed.tech/tags/agentic-ai-generative-ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [featured](<https://devfeed.tech/tags/featured.md>), [jetson](<https://devfeed.tech/tags/jetson.md>), [jetson-agx-thor-developer-kit](<https://devfeed.tech/tags/jetson-agx-thor-developer-kit.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-benchmarking](<https://devfeed.tech/tags/llm-benchmarking.md>), [mlperf](<https://devfeed.tech/tags/mlperf.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [performance](<https://devfeed.tech/tags/performance.md>), [quantization](<https://devfeed.tech/tags/quantization.md>), [tensorrt](<https://devfeed.tech/tags/tensorrt.md>), [tensorrt-llm](<https://devfeed.tech/tags/tensorrt-llm.md>), [thor](<https://devfeed.tech/tags/thor.md>)

### AI overview

This article reports that NVIDIA TensorRT Edge-LLM ran Qwen3.6-27B on a single NVIDIA Jetson AGX Thor Developer Kit for the MLPerf Inference v6.1 Edge Agentic benchmark. Using NVFP4 quantization, tree-based multi-token prediction, and KV cache reuse, it achieved 52.33 tokens per second and completed 1,007 turns in 24 minutes and 36 seconds, 6.4 times faster than the llama.cpp reference submission.

### Source excerpt

AI agents are moving from cloud data centers to vehicles, robots, and other edge devices. Unlike a chatbot that answers a single prompt, an agent works through...

## Edge AI Inference for Manufacturing

DevFeed: [Edge AI Inference for Manufacturing](<https://devfeed.tech/articles/edge-ai-inference-for-manufacturing-64083.md>)

Original publisher: [Read original article](<https://roboflow.com/blog/edge-ai-inference-for-manufacturing>)

Author: Timothy M

Published: 2026-09-03T17:52:00Z

Content type: tutorial

Language: en

Sources: [Roboflow](<https://devfeed.tech/sources/roboflow-blog.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [Holoscan](<https://devfeed.tech/topics/holoscan.md>), [Jetson](<https://devfeed.tech/topics/jetson.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>), [quantization](<https://devfeed.tech/topics/quantization.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [business](<https://devfeed.tech/tags/business.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [edge](<https://devfeed.tech/tags/edge.md>), [edge-ai](<https://devfeed.tech/tags/edge-ai.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [integration](<https://devfeed.tech/tags/integration.md>), [manufacturing](<https://devfeed.tech/tags/manufacturing.md>), [ot](<https://devfeed.tech/tags/ot.md>), [plc](<https://devfeed.tech/tags/plc.md>)

### AI overview

This guide explains how to deploy computer vision inference on manufacturing lines, where millisecond decision deadlines, high camera bandwidth, and WAN outages favor local processing. It covers model optimization and edge hardware, integration with PLCs and manufacturing systems, fleet management, and IT/OT controls.

### Source excerpt

A reject decision must happen in milliseconds, cameras generate data continuously, and the line still runs when the WAN does not. This guide covers edge vision deployment for manufacturing, from hardware and model optimization to PLC integration and IT/OT controls.

## ZDI-26-593: NVIDIA TensorRT ONNX File Parsing Heap-based Buffer Overflow Remote Code Execution Vulnerability

DevFeed: [ZDI-26-593: NVIDIA TensorRT ONNX File Parsing Heap-based Buffer Overflow Remote Code Execution Vulnerability](<https://devfeed.tech/articles/zdi-26-593-nvidia-tensorrt-onnx-file-parsing-heap-based-buffer-overflow-remote-code-execution-vulnerability-91636.md>)

Original publisher: [Read original article](<http://www.zerodayinitiative.com/advisories/ZDI-26-593/>)

Published: 2026-08-24T05:00:00Z

Content type: news

Language: en

Sources: [Zero Day Initiative](<https://devfeed.tech/sources/zero-day-initiative.md>)

Topics: [heap overflow](<https://devfeed.tech/topics/heap-overflow.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>), [onnx](<https://devfeed.tech/topics/onnx.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [allows](<https://devfeed.tech/tags/allows.md>), [based](<https://devfeed.tech/tags/based.md>), [buffer](<https://devfeed.tech/tags/buffer.md>), [buffer-overflow](<https://devfeed.tech/tags/buffer-overflow.md>), [code](<https://devfeed.tech/tags/code.md>), [code-execution](<https://devfeed.tech/tags/code-execution.md>), [execution](<https://devfeed.tech/tags/execution.md>), [exploit](<https://devfeed.tech/tags/exploit.md>), [file](<https://devfeed.tech/tags/file.md>), [heap](<https://devfeed.tech/tags/heap.md>), [installations](<https://devfeed.tech/tags/installations.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [onnx](<https://devfeed.tech/tags/onnx.md>), [parsing](<https://devfeed.tech/tags/parsing.md>), [remote](<https://devfeed.tech/tags/remote.md>), [remote-code-execution](<https://devfeed.tech/tags/remote-code-execution.md>), [remote-code-execution-vulnerability](<https://devfeed.tech/tags/remote-code-execution-vulnerability.md>), [tensorrt](<https://devfeed.tech/tags/tensorrt.md>), [vulnerability](<https://devfeed.tech/tags/vulnerability.md>)

### AI overview

NVIDIA TensorRT's ONNX model parser has a heap-based buffer overflow because it copies user-supplied data into a fixed-length heap buffer without properly validating its length. An attacker can exploit it to execute code in the current process, but the target must first visit a malicious page or open a malicious file. NVIDIA has issued an update to address the vulnerability. The advisory assigns CVE-2026-24268 and a CVSS rating of 7.8.

### Source excerpt

This vulnerability allows remote attackers to execute arbitrary code on affected installations of NVIDIA TensorRT. User interaction is required to exploit this vulnerability in that the target must visit a malicious page or open a malicious file. The ZDI has assigned a CVSS rating of 7.8. The following CVEs are assigned: CVE-2026-24268.

## ZDI-26-591: NVIDIA TensorRT ONNX File Parsing Heap-based Buffer Overflow Remote Code Execution Vulnerability

DevFeed: [ZDI-26-591: NVIDIA TensorRT ONNX File Parsing Heap-based Buffer Overflow Remote Code Execution Vulnerability](<https://devfeed.tech/articles/zdi-26-591-nvidia-tensorrt-onnx-file-parsing-heap-based-buffer-overflow-remote-code-execution-vulnerability-91634.md>)

Original publisher: [Read original article](<http://www.zerodayinitiative.com/advisories/ZDI-26-591/>)

Published: 2026-08-24T05:00:00Z

Content type: news

Language: en

Sources: [Zero Day Initiative](<https://devfeed.tech/sources/zero-day-initiative.md>)

Topics: [heap overflow](<https://devfeed.tech/topics/heap-overflow.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>), [onnx](<https://devfeed.tech/topics/onnx.md>), [Exploit](<https://devfeed.tech/topics/exploit.md>)

Tags: [buffer-overflow](<https://devfeed.tech/tags/buffer-overflow.md>), [code-execution](<https://devfeed.tech/tags/code-execution.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [remote-code-execution](<https://devfeed.tech/tags/remote-code-execution.md>), [remote-code-execution-vulnerability](<https://devfeed.tech/tags/remote-code-execution-vulnerability.md>), [tensorrt](<https://devfeed.tech/tags/tensorrt.md>), [vulnerability](<https://devfeed.tech/tags/vulnerability.md>)

### AI overview

ZDI-26-591 (CVE-2026-24272) is a heap-based buffer overflow in NVIDIA TensorRT's parsing of ONNX models. The flaw stems from failing to validate user-supplied data length before copying it into a fixed-length heap buffer, which can let an attacker execute code in the current process. Exploitation requires user interaction: the target must visit a malicious page or open a malicious file. NVIDIA has issued an update to address the vulnerability. The ZDI assigned a CVSS rating of 7.8.

### Source excerpt

This vulnerability allows remote attackers to execute arbitrary code on affected installations of NVIDIA TensorRT. User interaction is required to exploit this vulnerability in that the target must visit a malicious page or open a malicious file. The ZDI has assigned a CVSS rating of 7.8. The following CVEs are assigned: CVE-2026-24272.

[Next page](<https://devfeed.tech/topics/tensorrt.md?cursor=WyIyMDI2LTA4LTI0VDA1OjAwOjAwKzAwOjAwIiwgIjQzMmE2MThjLTMwOWYtNGM0YS1hOTYzLTQyZDgzNmUxMGEzMSJd>)