# Configure, Certify, and Operate Kubernetes GPU Clusters with NVIDIA DSX OS

DevFeed: [Configure, Certify, and Operate Kubernetes GPU Clusters with NVIDIA DSX OS](<https://devfeed.tech/articles/configure-certify-and-operate-kubernetes-gpu-clusters-with-nvidia-dsx-os-119476.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/configure-certify-and-operate-kubernetes-gpu-clusters-with-nvidia-dsx-os/>)

Author: Sowmyan Soman

Published: 2026-11-05T17:00:00Z

Content type: tutorial

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [gpu monitoring](<https://devfeed.tech/topics/gpu-monitoring.md>), [NVIDIA DSX](<https://devfeed.tech/topics/nvidia-dsx.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [agentic-ai-generative-ai](<https://devfeed.tech/tags/agentic-ai-generative-ai.md>), [basic](<https://devfeed.tech/tags/basic.md>), [can](<https://devfeed.tech/tags/can.md>), [cloud-services](<https://devfeed.tech/tags/cloud-services.md>), [clusters](<https://devfeed.tech/tags/clusters.md>), [configure](<https://devfeed.tech/tags/configure.md>), [data-center-cloud](<https://devfeed.tech/tags/data-center-cloud.md>), [developer-tools-techniques](<https://devfeed.tech/tags/developer-tools-techniques.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [distributed-training](<https://devfeed.tech/tags/distributed-training.md>), [dsx](<https://devfeed.tech/tags/dsx.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-clusters](<https://devfeed.tech/tags/gpu-clusters.md>), [health](<https://devfeed.tech/tags/health.md>), [health-checks](<https://devfeed.tech/tags/health-checks.md>), [job](<https://devfeed.tech/tags/job.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [leaving](<https://devfeed.tech/tags/leaving.md>), [node](<https://devfeed.tech/tags/node.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-dsx](<https://devfeed.tech/tags/nvidia-dsx.md>), [one](<https://devfeed.tech/tags/one.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [os](<https://devfeed.tech/tags/os.md>), [with](<https://devfeed.tech/tags/with.md>)

## AI overview

NVIDIA DSX OS combines three open source tools to keep Kubernetes GPU clusters configured, workload-certified, and healthy through changes and repairs. AICR builds version-locked configuration recipes, renders them for existing deployers, and can detect drift; it generates artifacts but does not apply them. NVCRE runs GPU, network, storage, and AI workload tests, reports measurements and suspected faulty nodes, and applies performance thresholds only when configured. NVSentinel continuously monitors existing health signals without using GPU time. It starts in reporting-only mode; when configured, it can quarantine or drain unhealthy nodes and coordinate NVCRE revalidation before they return to service. The tools divide responsibility: NVCRE records failures and attribution, while NVSentinel handles quarantine and repair. Lifecycle automation can validate new and remediated nodes, but driver, operator, or fabric changes need a separate pipeline-triggered validation process. The excerpt assumes the tools, cluster dependencies, and NVCRE test provider are installed and configured.

## Source excerpt

One underperforming GPU can slow an entire distributed training job. The node may pass basic health checks and show normal utilization, leaving operators to...