# Kubernetes Blog The Kubernetes blog is used by the project to communicate new features, community reports, and any news that might be relevant to the Kubernetes community. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Kubernetes v1.37: Hardening Container Storage with Bind Mount Options and EmptyDir Permissions DevFeed: [Kubernetes v1.37: Hardening Container Storage with Bind Mount Options and EmptyDir Permissions]() Original publisher: [Read original article]() Author: Nispriha Jagan; Neeraj Krishna Gopalakrishna Published: 2026-09-16T18:30:00Z Content type: article Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [Security](), [mount](), [Filesystems](), [chmod](), [Unix]() Tags: [chmod](), [container](), [containers](), [filesystem](), [hardening](), [kubernetes](), [linux](), [mount](), [permissions](), [storage](), [volume](), [volumes]() ### AI overview Kubernetes v1.37 adds bind mount options and emptyDir permission modes to strengthen storage security. The article explains how noexec, nosuid, nodev, Unix permissions, and the sticky bit can help enforce security policies on writable volumes. ### Source excerpt Kubernetes v1.37 brings important storage security features: emptyDir permission modes and bind mount options. They help application programmers and security professionals implement rigorous security policies, for example, prohibiting deletion of files across containers or execution of arbitrary binaries from writable volumes, directly in Kubernetes without any complicated circumvention. Linux storage and permission fundamentals Before diving into the new Kubernetes features, let us briefly review the low-level Linux security mechanisms that make them possible. Bind mount flags When Linux mounts or remounts a directory, Virtual File System (VFS) flags control what actions are permitted on that filesystem: noexec: Do not permit direct execution of any binaries on the mounted filesystem. nosuid: Do not allow set-user-identifier or set-group-identifier bits to take effect. nodev: Do not interpret character or block special devices on the file system. Directory permissions and the sticky bit Standard Unix permissions regulate access across three scopes: Owner, Group, and Others (e.g., 0755 or 0777). Beyond standard read, write, and execute bits, Linux supports the sticky bit (as in mode 01777). When applied to a directory, the sticky bit ensures that a file inside that directory can only be deleted or renamed by the file's owner or root. This is essential for shared writable directories like /tmp. Motivation for the improvements Why does Kubernetes need bind mount options and emptyDir permissions? The primary goal of these features is to increase the security of Kubernetes workloads by allowing security-related bind mount options on volume mounts. By default, volumes are bind-mounted into containers by the container runtime and kubelet without noexec, nosuid, or nodev flags. This default can undermine security. For example, with noexec missing, a compromised process can use any writable volume (emptyDir, PersistentVolume, etc.) to download, chmod +x, and execute arbitra ## Kubernetes v1.37: Pod-Level Resource Managers graduated to Beta DevFeed: [Kubernetes v1.37: Pod-Level Resource Managers graduated to Beta]() Original publisher: [Read original article]() Author: Kevin Torres Martinez Published: 2026-09-15T18:30:00Z Content type: release Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [cpu](), [Hardware](), [Latency](), [API](), [gRPC](), [Monitoring]() Tags: [api](), [cpu](), [hardware](), [kubernetes](), [latency](), [monitoring](), [numa](), [release](), [v1]() ### AI overview Kubernetes v1.37 graduates Pod-Level Resource Managers to Beta, disabled by default. The feature lets Kubelet resource managers use pod-level declarations for hardware placement, enabling NUMA-aligned exclusive resources for primary containers while placing sidecars in a shared pod-isolated pool. The release also adds pod-level reporting to the PodResources gRPC API. ### Source excerpt With the release of Kubernetes v1.37, the Pod-Level Resource Managers feature has graduated to Beta status (disabled by default)! First introduced as an Alpha feature in Kubernetes v1.36, this enhancement builds on Pod-Level Resources by equipping Kubelet's Topology Manager, CPU Manager, and Memory Manager to use Pod-level resource declarations (.spec.resources) directly when making hardware placement decisions. Bringing pod-level resources to node managers Before this feature, obtaining exclusive NUMA-aligned CPU cores or memory for latency-critical applications forced cluster operators into an all-or-nothing choice: assign integer resource requests to every container in the Pod, or forfeit exclusive NUMA alignment entirely. For modern workloads running lightweight sidecars (such as logging agents or telemetry exporters), allocating dedicated physical cores to auxiliary containers was wasteful. Pod-Level Resource Managers solves this challenge by enabling hybrid allocation models. The Kubelet can reserve exclusive NUMA-aligned resources for primary application containers while placing non-Guaranteed sidecars into a pod-isolated shared pool. This ensures primary workloads get unthrottled, NUMA-local performance while sidecars benefit from running in a pod-isolated shared pool, enjoying local NUMA alignment and protection from external node interference without consuming dedicated physical cores. What's new in Beta Graduating to Beta brings key operational and API enhancements: Graduation to Beta: Controlled by the PodLevelResourceManagers feature gate, available to opt in (disabled by default) in Kubernetes v1.37. PodResources API Reporting: The v1 PodResources gRPC service (PodResourcesLister) introduces top-level cpu_ids and memory fields on PodResources responses. Monitoring tools and device plugins can query pod-level exclusive assignments directly without double-counting container allocations. Getting started and providing feedback For a deep dive into the tech ## Kubernetes v1.37: Memory QoS Graduates to Beta DevFeed: [Kubernetes v1.37: Memory QoS Graduates to Beta]() Original publisher: [Read original article]() Author: Qi Wang; Sohan Kunkerkar Published: 2026-09-14T18:30:00Z Content type: article Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [releases](), [configuration](), [Linux](), [Kernel]() Tags: [clusters](), [configuration](), [container](), [kernel](), [kubernetes](), [linux](), [memory](), [qos](), [releases](), [upgrade](), [v1]() ### AI overview Kubernetes v1.37 promotes Memory QoS to Beta and enables it by default on Linux nodes using cgroup v2. The article explains the updated defaults, configuration options for memory throttling and tiered memory protection, and upgrade behavior intended to preserve existing runtime behavior. ### Source excerpt Memory QoS has graduated to Beta in Kubernetes v1.37 and is now enabled by default. On Linux nodes running cgroup v2, the feature uses the memory controller to give the kernel better guidance on how to treat container memory. It was first introduced as Alpha in v1.22, and expanded in v1.36 with tiered memory reservation. This post covers what changed in v1.37, what the Beta promotion means for cluster operators, and how to configure the feature. What changed in v1.37Memory QoS is Beta and enabled by default The MemoryQoS feature gate is now Beta in v1.37. This means every v1.37 kubelet has the feature gate turned on without any configuration change. Turning on the feature by default is safe because the default kubelet configuration does not enable memory throttling or memory reservation. No memory.high, memory.min, or memory.low values are written to cgroups unless you explicitly configure them. You can opt into specific behaviors through kubelet configuration fields: Set memoryThrottlingFactor (for example, 0.9) to enable memory.high throttling on Burstable and BestEffort containers. The default is null, which means no throttling. Set memoryReservationPolicy to TieredReservation to enable tiered memory protection via memory.min and memory.low. The default is None, which means no memory reservation. Default memoryThrottlingFactor changed to null In earlier Alpha releases, memoryThrottlingFactor defaulted to 0.9, which meant enabling the feature gate caused the kubelet to set memory.high on containers. In v1.37, the default is null, so the kubelet does not set memory.high unless you configure a value. This change was made because, with the feature gate now on by default, an automatic memory.high could throttle workloads that were previously running without throttling. Making it null ensures that upgrading to v1.37 does not change runtime behavior for existing clusters. If your kubelet configuration file already contains an explicit memoryThrottlingFactor value, that ## Kubernetes Changed Block Tracking API - Beta Differences DevFeed: [Kubernetes Changed Block Tracking API - Beta Differences]() Original publisher: [Read original article]() Author: Prasad Ghangal Published: 2026-09-14T18:30:00Z Content type: article Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [API](), [gRPC]() Tags: [2026](), [compatibility](), [container-image-registry](), [developer](), [documentation](), [kubernetes](), [registry](), [release](), [snapshots](), [storage](), [streaming](), [update]() ### AI overview This Kubernetes developer article explains the Beta changes to Changed Block Tracking (CBT) support for CSI drivers. The SnapshotMetadataService CRD moved from v1alpha1 to v1beta1, with no automatic conversion, and CBT remains limited to block volumes. It also outlines compatibility requirements and the steps for upgrading and trying the feature. ### Source excerpt Changed Block Tracking (CBT) support for CSI drivers shipped as Alpha in September 2025. With the March 2026 v1.0.0 release of the external-snapshot-metadata project, the feature moved to Beta. If you aren't yet familiar with changed block tracking for storage in Kubernetes, the Alpha announcement covers the motivation, the three primary components (the CSI SnapshotMetadata gRPC service, the SnapshotMetadataService CRD, and the external-snapshot-metadata sidecar), and a walkthrough of how to use the API. CBT currently applies to block volumes; file-volume and network file-share changed-list tracking is not covered by this feature. This post focuses on what is different in Beta. What's new in Beta The main change in that release was the promotion of the SnapshotMetadataService CRD from v1alpha1 to v1beta1. The CRD used to advertise a driver's metadata service now serves cbt.storage.k8s.io/v1beta1. The schema itself is unchanged, but this release removed v1alpha1 (rather than serving it alongside the new version). If you are upgrading from Alpha, you need to: Re-apply the CRD definition shipped with v1.0.0. Update SnapshotMetadataService manifests to use apiVersion: cbt.storage.k8s.io/v1beta1. Update any client or controller code that talks to the CRD. This is a one-time change. There is no automatic conversion between the two versions. Compatibility Minimum Kubernetes version: 1.33 CSI spec: 1.10 or newer Container image: registry.k8s.io/sig-storage/csi-snapshot-metadata:v1.0.0 Trying it out The Getting Started section in the Alpha blog still applies. In short: Make sure your CSI driver supports volume snapshots and ships the external-snapshot-metadata sidecar. Install the SnapshotMetadataService CRD (the v1beta1 definition from the v1.0.0 release). Create a SnapshotMetadataService resource for your driver. Use a client -- snapshot-metadata-lister, or your own implementation -- to call GetMetadataAllocated and GetMetadataDelta. If you want to see the full flow end-to-e ## Kubernetes v1.37: Native Histograms Graduates to Beta DevFeed: [Kubernetes v1.37: Native Histograms Graduates to Beta]() Original publisher: [Read original article]() Author: Richa Banker Published: 2026-09-11T18:30:00Z Content type: release Language: en Sources: [Kubernetes Blog]() Topics: [telemetry]() Tags: [kubernetes](), [latency](), [metrics](), [observability](), [prometheus](), [storage](), [telemetry](), [time-series]() ### AI overview Kubernetes v1.37 enables Prometheus native histograms by default at Beta. The article explains how dynamic exponential buckets improve metric resolution and quantile accuracy while reducing time-series, scraping, and storage overhead compared with classic histograms. ### Source excerpt I'm excited to announce that native histogram support for Kubernetes metrics is graduating to Beta and is enabled by default in Kubernetes v1.37! Native histograms (previously introduced as Alpha in Kubernetes v1.36 under KEP-5808) bring high-resolution, low-cardinality observability to Kubernetes metrics. By adopting Prometheus Native Histograms, Kubernetes components now expose latency and duration metrics with far greater accuracy while significantly reducing telemetry storage and scraping overhead. Why move beyond classic histograms? Since the early days of Kubernetes observability, duration and latency metrics (such as API server request latencies or scheduling durations) have relied on classic Prometheus histograms. Classic histograms require metric authors to define a static list of cumulative bucket boundaries (le labels), such as 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10. While familiar, this approach introduces three major challenges: The Bucket Guessing Game: If a workload's latency profile changes, for example, shifting into microsecond ranges or experiencing long-tail tail latencies beyond the highest bucket, the histogram loses visibility. Specifying bucket boundaries upfront requires knowing the distribution before observing it High Cardinality & Storage Cost: With classic histograms, each bucket boundary is exported as a separate time series (_bucket{le="..."}). A histogram with 10 buckets across multiple labels multiplies the number of time series by 10, increasing memory consumption in Prometheus and inflating time series database (TSDB) storage costs Interpolation Error in Quantiles: Calculating percentiles using histogram_quantile() relies on linear interpolation between static bucket boundaries. When bucket spans are coarse, quantile calculations can suffer from significant estimation error What are Prometheus native histograms? Prometheus Native Histograms replace static user-defined buckets with dynamic, exponential buckets. Inst ## Kubernetes v1.37: Scheduler Preemption for In-Place Pod Resize (Alpha) DevFeed: [Kubernetes v1.37: Scheduler Preemption for In-Place Pod Resize (Alpha)]() Original publisher: [Read original article]() Author: Natasha Sarkar Published: 2026-09-10T18:30:00Z Content type: article Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [Containers]() Tags: [containers](), [cpu](), [feature](), [kubernetes](), [memory](), [node](), [resource](), [scale]() ### AI overview Kubernetes v1.37 adds an alpha scheduler-preemption feature for in-place Pod resizes. It can preempt lower-priority workloads to make capacity available for valid, higher-priority resize requests that are deferred on full nodes. ### Source excerpt In Kubernetes, resource allocation has historically been a static decision made during a Pod's initial scheduling and placement. With the graduation of the core in-Place Pod resize feature to General Availability in v1.35, application developers and cluster operators gained the powerful ability to dynamically adjust CPU and memory allocations of running containers without incurring disruptive restarts or application downtime. However, in-place resizing introduced a unique resource scheduling gap: if a running Pod requested a resource scale-up that exceeded the host node's allocatable headroom, the Kubelet was forced to mark the request as Deferred. The Pod would remain parked in this state indefinitely, waiting for resources on the node to naturally free up. To bridge this scheduling gap, Kubernetes v1.37 introduces scheduler preemption for in-place Pod resize (Alpha), behind the InPlacePodVerticalScalingSchedulerPreemption feature gate. This feature allows the Kubernetes scheduler to actively free up capacity on a fully-utilized node by preempting lower-priority workloads, enabling the pending in-place resizes of critical, higher-priority applications to succeed. The "deferred" resize challenge To understand why this preemption mechanism is needed, it is helpful to look at how Kubernetes handles running Pod resizing. When a user or controller (such as the Vertical Pod Autoscaler) updates the resource requests of an active container, the Kubelet evaluates whether the underlying node has enough spare allocatable capacity to fulfill the increase. If the node's resources are fully utilized and cannot satisfy the new limits, the Kubelet sets the container's resizeStatus (reported in the Pod's status.containerStatuses[]) to Deferred. Unlike an Infeasible resize request (which is immediately rejected because it exceeds physical machine boundaries, namespace limit ranges, or admission quotas) a Deferred status indicates that the request is valid but is temporarily unable t ## Kubernetes v1.37: Introducing Node Lifecycle Conditions DevFeed: [Kubernetes v1.37: Introducing Node Lifecycle Conditions]() Original publisher: [Read original article]() Author: Ryan Hallisey Published: 2026-09-09T18:30:00Z Content type: release Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [releases]() Tags: [apis](), [automation](), [feature](), [kubernetes](), [node](), [release](), [releases]() ### AI overview Kubernetes v1.37 introduces well-known Node lifecycle conditions for communicating drain, maintenance, and graceful shutdown status. The alpha feature gate is disabled by default and currently does not cause core components to consume or enforce these conditions. ### Source excerpt Kubernetes has many ways to describe what is happening on a Node. Readiness, taints, Pod state, labels, annotations, and provider-specific APIs each expose part of the picture. What has been missing is a shared, Kubernetes-owned way to say that a Node is draining, undergoing maintenance, or undergoing Graceful Node Shutdown. Kubernetes v1.37 introduces five well-known Node conditions that provide that description: DrainInProgress Drained MaintenancePlanned MaintenanceInProgress GracefulNodeShutdownInProgress The new Node lifecycle conditions Condition What it reports DrainInProgress The Node is actively being drained according to the administrator's chosen drain criteria. Drained The Node has reached the drain criteria selected by the administrator. MaintenancePlanned The Node is expected to undergo a change in the future. MaintenanceInProgress The Node is actively undergoing maintenance. GracefulNodeShutdownInProgress Graceful Node Shutdown is determined to be in progress on the Node. Maintenance can include hardware or software rollout, remediation, decommissioning, or debugging. Whether maintenance requires a drain depends on its impact. A Kubernetes upgrade usually should follow a drain, while a kernel live patch might not need one. Like other Node conditions, each lifecycle condition uses status to report whether the observation is active: True: the lifecycle state is currently observed. False: the lifecycle state is not currently observed. Unknown: Kubernetes cannot determine whether the lifecycle state is active. The reason provides a stable, machine-readable cause for the current status, and message can provide additional human-readable detail. For example, an authorized maintenance controller could publish: # Node .status excerpt status: conditions: - type: MaintenancePlanned status: "True" reason: MaintenanceWindow lastTransitionTime: "2026-12-09T12:00:00Z" message: "Hardware maintenance is scheduled for this Node" What changes in Kubernetes v1.37 The v1.3 ## Kubernetes v1.37: Advancing Workload-Aware Scheduling DevFeed: [Kubernetes v1.37: Advancing Workload-Aware Scheduling]() Original publisher: [Read original article]() Published: 2026-09-08T18:30:00Z Content type: release Language: en Sources: [Kubernetes Blog]() Topics: [releases]() Tags: [api](), [batch](), [go](), [integration](), [kubernetes](), [release]() ### AI overview Kubernetes v1.37 advances workload-aware scheduling by promoting Workload and PodGroup APIs, gang scheduling, workload-aware preemption, and shared DRA ResourceClaims for PodGroups to Beta. It also introduces CompositePodGroup and controller integration APIs for complex distributed and batch workloads. ### Source excerpt AI/ML and complex batch workloads continue to push the boundaries of Kubernetes scheduling. Following the foundational workload-centric enhancements introduced in previous releases, Kubernetes v1.37 delivers the next major milestone in the Workload-Aware Scheduling (WAS) journey. In this release, the core Workload and PodGroup APIs--enabling gang scheduling--along with Workload-Aware Preemption (WAP) and shared DRA ResourceClaims for PodGroups, all graduate to Beta, solidifying their role in the Kubernetes ecosystem. To address the hierarchical scheduling requirements of modern high-performance distributed workloads, v1.37 introduces the new CompositePodGroup API. This new API allows expressing multi-level topology constraints, gang scheduling, and preemption policies for complex, heterogeneous groups of Pods. Crucially, this architectural expansion unlocks native scheduling support for advanced workload structures commonly managed by higher-order extension APIs such as JobSet and LeaderWorkerSet (LWS). Alongside these API additions, v1.37 focuses on streamlining adoption by introducing a new set of controller integration APIs and the workloadbuilder Go library. These provide standardized building blocks that significantly simplify how out-of-tree controllers can integrate with WAS capabilities. Utilizing these new tools, the native Job controller integration has been upgraded to fully consume the expanded WAS APIs--enabling advanced scheduling policies, flexible disruption modes, and topology-aware scheduling for standard batch workloads. Gang scheduling and Workload / PodGroup APIs Kubernetes v1.37 delivers a major milestone: Workload / PodGroup APIs and gang scheduling are officially graduating to Beta. This graduation signals that native, "all-or-nothing" scheduling for workloads is solidifying for wider adoption. Key updates to the API and gang scheduling algorithm in this release include: Beta graduation and API versioning changes The core Workload and PodGroup A ## Kubernetes v1.37: KubeletInUserNamespace (aka Rootless mode) Graduates to Beta DevFeed: [Kubernetes v1.37: KubeletInUserNamespace (aka Rootless mode) Graduates to Beta]() Original publisher: [Read original article]() Author: Akihiro Suda Published: 2026-09-04T18:30:00Z Content type: release Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [Vulnerabilities](), [Containers]() Tags: [containers](), [feature](), [kubernetes](), [linux](), [production](), [v1](), [vulnerabilities]() ### AI overview Kubernetes v1.37 promotes KubeletInUserNamespace, or rootless mode, to beta. It lets Kubernetes node components run as a non-root host user in a Linux user namespace, reducing the impact of container-breakout vulnerabilities while complementing conventional hardening such as seccomp. ### Source excerpt Kubernetes v1.37 promotes the KubeletInUserNamespace feature gate to beta. With this feature enabled, all of the node components (kubelet, CRI and OCI runtimes, CNI plugins, and kube-proxy) can run as a non-root user on the host, using a Linux user namespace. This technique is also known as rootless mode. The work started as an experiment in 2018, and was merged into Kubernetes v1.22 (2021) as an alpha feature (Kubernetes Enhancement Proposal KEP-2033). This feature should not be confused with user namespaces for pods (hostUsers: false with the UserNamespacesSupport feature gate, GA since v1.36), which puts pods in user namespaces but still runs the node components as root. These two features do not conflict. Moreover, they can be combined to nest Kubernetes inside Kubernetes without resorting to the full privileged: true. Why run the node components in a user namespace? Because the node components have historically had container-breakout vulnerabilities that could compromise full root privileges on the host. Examples of such vulnerabilities include: CVE-2022-0811 ("cr8escape"): CRI-O could be tricked into setting arbitrary sysctls, such as kernel.core_pattern, resulting in arbitrary code execution as root on the host CVE-2023-27561: runc could be tricked into bypassing the masked paths of a container via a volume mount race, exposing the host's procfs files (a regression of CVE-2019-19921) CVE-2024-10220: the kubelet could be made to execute arbitrary commands as root via gitRepo volumes (gitRepo volumes had a similar vulnerability, CVE-2018-11235, back in 2018 too) CVE-2025-31133: runc could be tricked into bind-mounting attacker-controlled paths and writing to the host's procfs files, such as /proc/sysrq-trigger and /proc/sys/kernel/core_pattern CVE-2026-53488: containerd could be tricked into executing arbitrary commands on the host, via crafted labels in a container image By running the node components in a user namespace, the potential damage is confined to th ## Kubernetes v1.37: DRA Updates DevFeed: [Kubernetes v1.37: DRA Updates]() Original publisher: [Read original article]() Author: Kashish Verma Published: 2026-09-03T18:30:00Z Content type: release Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [releases]() Tags: [api](), [devices](), [drivers](), [kubernetes](), [release](), [resource](), [updates]() ### AI overview Kubernetes 1.37 updates Dynamic Resource Allocation, including GA support for extended resources, device status reporting, stable device taints and tolerations, and a standardized NUMA-node device attribute. ### Source excerpt Kubernetes 1.37 is here and Dynamic Resource Allocation (DRA) keeps pushing past where it started! This release brings DRA Extended Resource support to GA, a milestone the team has been building toward for three straight releases. Several more features graduate to Beta or GA. A fresh batch of alpha features rounds out the release. I'll dive into what's new for DRA in Kubernetes 1.37! What's stable in 1.37 DRA Extended Resource support has graduated to GA. This is the mechanism that lets DRA drivers satisfy requests made through the traditional extended resource API, think example.com/gpu in a Pod spec, without requiring a separate device plugin alongside the DRA driver. An extended resource name can be set directly on a DeviceClass, and Pods requesting it get matched to a device through DRA with no ResourceClaim needed on the workload's part. It's been on a steady path since KEP acceptance in 1.34. Alpha landed in 1.35, Beta in 1.36, and now it's Stable. For cluster operators, this is what makes DRA adoption gradual. Existing workloads written against extended resources keep working unmodified while the backend allocation logic moves over to DRA. ResourceClaims status with possible standardized network interface data adds a devices field to ResourceClaim .status, letting DRA drivers report per-device status, including, for network devices, the interface name, MAC address, and IP addresses. This gives users and controllers visibility into device state that was previously invisible once a device was configured in a Pod, and makes it possible to build things like network services that rely on a device's reported IPs. DRA: device taints and tolerations is now Stable; DRA drivers can mark devices as tainted so they're skipped for new Pod scheduling, and cluster admins can apply the same taints cluster-wide via a DeviceTaintRule, without reconfiguring drivers. Pods already using a tainted device can be evicted automatically, unless their ResourceClaim explicitly tolerates ## Kubernetes v1.37: Scale Workloads to Zero with HorizontalPodAutoscaler DevFeed: [Kubernetes v1.37: Scale Workloads to Zero with HorizontalPodAutoscaler]() Original publisher: [Read original article]() Author: Johannes Würbach Published: 2026-09-02T18:30:00Z Content type: release Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [autoscaling]() Tags: [autoscaling](), [deployment](), [kubernetes](), [metrics](), [prometheus]() ### AI overview Kubernetes v1.37 adds beta, default-enabled HPA support for scaling eligible workloads to zero and back using object or external metrics. It explains queue-based scaling, cold-start trade-offs, and the metrics adapter setup required for an external Prometheus metric. ### Source excerpt Kubernetes v1.37 includes API support for horizontal autoscaling of workloads down to zero replicas. This feature is now Beta and enabled by default. A HorizontalPodAutoscaler (HPA) that uses a suitable object metric or external metric can now scale a workload to zero replicas, then bring it back when the metric changes. Before v1.37, you needed an add-on or external component, or you had to enable the Alpha feature gate, to scale from zero. It is now part of core Kubernetes. Scaling to zero removes the last idle Pod from workloads such as queue consumers and batch processors. The savings are largest when each Pod reserves expensive resources, including dedicated CPUs or GPUs. The trade-off is cold-start time: the HPA must observe the metric, schedule a Pod, and start the application. This works well when work can wait in a durable queue. Kubernetes Services do not buffer requests while no Pods are ready, so HTTP and other request-driven workloads need a separate buffering layer. Why scaling from zero needs a different metric The HPA commonly scales on CPU or memory usage. Both metrics come from running Pods. Once the replica count reaches zero, there are no Pods left to measure and no signal that can tell the HPA to scale back up. Object and external metrics do not have that limitation. A queue length, for example, exists independently of the workers that consume it. The HPA can continue reading the queue length while no workers are running. The following example scales a queue consumer to and from zero using an external metric. Configure an external metric The following example uses a Prometheus metric named queue_consumer_lag. It assumes that Prometheus already collects a series similar to this one: queue_consumer_lag{namespace="default",name="worker_tasks"} Kubernetes needs a metrics adapter to make that value available through the External Metrics API. One implementation is the Prometheus Adapter, which can expose the series using an externalRules entry: extern ## Kubernetes v1.37: etcd RangeStream Cuts Memory Use on Large List Reads DevFeed: [Kubernetes v1.37: etcd RangeStream Cuts Memory Use on Large List Reads]() Original publisher: [Read original article]() Author: Jeffrey Ying Published: 2026-09-01T18:30:00Z Content type: release Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [Remote Procedure Call (RPC)](), [Caching]() Tags: [cache](), [kubernetes](), [memory](), [releases](), [rpc](), [streaming]() ### AI overview Kubernetes v1.37 promotes etcd RangeStream to beta. With etcd v3.7, streamed collection reads reduce and stabilize memory use by processing chunks instead of assembling entire pages. ### Source excerpt I am excited to announce that etcd RangeStream is graduating to beta in Kubernetes v1.37. Paired with etcd v3.7, it reduces the memory the API server and etcd need to read a large collection, and makes peak usage more predictable. The cost of large reads The API server serves most list and watch requests from its in-memory watch cache. Populating that cache requires reading a resource's full state from etcd, at startup and on every re-initialization. For a resource with many objects, or large ones, such as Pods, that read is expensive. The API server already paginated these reads, asking etcd for a fixed number of keys at a time rather than the whole collection at once. But a page bounded by key count has no awareness of object size, so a page of large objects can still be very large. That makes memory usage hard to predict, and a bad combination of object size and concurrent reads can be enough to trigger an OOM. etcd's unary Range assembles each page in full before sending it, and the API server holds it while decoding, so the same payload sits in memory on both sides at once. Most of that cost lands on etcd, which is also where streaming helps most. Streaming reads with RangeStream etcd v3.7 adds a streaming version of that read, the RangeStream RPC. It takes the same RangeRequest as Range and returns the same result set, but instead of building the whole response up front, etcd splits it into chunks and streams them. Chunk size is tuned adaptively to the values being returned, so a collection of large objects is bounded by bytes rather than by a key count, and memory is freed as the stream progresses instead of being held until a whole page is assembled. When the feature is enabled, the API server uses RangeStream wherever it reads a whole collection out of etcd. This includes watch cache initialization, and the fallback paths where a list request cannot be served from the cache and reads etcd directly. In either case the API server decodes each chunk as it arri ## Kubernetes v1.37: Storage Version Migration Enabled by Default DevFeed: [Kubernetes v1.37: Storage Version Migration Enabled by Default]() Original publisher: [Read original article]() Author: Michael Aspinwall Published: 2026-08-31T18:30:00Z Content type: article Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [migration](), [API](), [Encryption](), [releases]() Tags: [api](), [encryption](), [kubernetes](), [manifest](), [migration](), [releases](), [support](), [v1]() ### AI overview Kubernetes v1.37 makes Storage Version Migration generally available and enables it by default. The built-in API and control-plane controller migrate existing stored resources to the current storage version, addressing stale versions, encryption-at-rest changes, and key rotation. ### Source excerpt I am excited that storage version migration (SVM) has graduated to General Availability (GA) in Kubernetes v1.37! After a number of releases of work and testing, the built-in StorageVersionMigration API (storagemigration.k8s.io/v1) and control plane controller are now fully stable and enabled by default across all v1.37 Kubernetes clusters. The problem with stale storage versions In Kubernetes, stored API resources are written using a specific storage version (schema representation). The way Kubernetes interacts with object storage fundamentally requires mutation of a resource in order to ensure that the latest storage version is used for all resources. This creates problems when you want to change the storage version of a resource. One example of a scenario where you may want to change the storage version of a resource is when you are promoting a CRD to drop an older API version (such as v1alpha1) to a newer version (leaving just v1beta1 and v1). It's a problem to drop the older API version whilst there are still resources stored with the old alpha version. To avoid problems, you designate v1 as the new storage version; but, on it's own, that's not enough. While new writes are stored as v1, any existing resource could remain stored as v1alpha1 or v1beta1 in storage. You cannot safely remove v1alpha1 from the CRD's .status.storedVersions or drop serving support until every single resource in storage has been re-written to not be serialized and stored with the alpha version. Another relevant example is encryption at rest and, related, key rotation. When you configure encryption at rest or rotate encryption keys, existing resources in storage remain unencrypted (or encrypted under old keys) until they are actively re-written through the Kubernetes API server. Historically, cluster administrators and CRD authors had to rely on manual kubectl get / kubectl replace scripts, or to deploy the out-of-tree kube-storage-version-migrator component to force re-writes. These app ## Kubernetes v1.37: Pod Certificates and Cluster Trust Bundles DevFeed: [Kubernetes v1.37: Pod Certificates and Cluster Trust Bundles]() Original publisher: [Read original article]() Author: Taahir Ahmed Published: 2026-08-28T18:30:00Z Content type: article Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [Authentication](), [JSON Web Tokens](), [TLS (Transport Layer Security)](), [Cloud]() Tags: [authentication](), [blog](), [cloud](), [cryptographic](), [json](), [kubernetes](), [post](), [tls](), [tokens]() ### AI overview Kubernetes 1.37 introduces generally available Pod Certificates and Cluster Trust Bundles, adding built-in X.509 certificate issuance for TLS and mTLS. The article explains how this production identity mechanism relates to existing service account JWTs and addresses the security limitations of bearer tokens. ### Source excerpt Pod Certificate / Cluster Trust Bundles Blog Post Kubernetes brings a wealth of features that make it easy to run your production workloads securely and reliably. While aspects like scheduling, health checks and resource limits are probably at the front of your mind, one other important feature of Kubernetes is production identity -- how your workload can authenticate to other systems in order to do its job. Up until now, the primary production identity mechanism built into Kubernetes has been service account JWTs (JSON Web Tokens). These are cryptographically-signed tokens, issued by the control plane of your cluster, that let anyone in the world understand who is calling when your workload uses them. In Kubernetes 1.37, the foundations of a new built-in production identity technology have gone GA. Pod Certificates (and the closely-associated Cluster Trust Bundles) build X.509 certificate issuance for TLS and mTLS directly into core Kubernetes. Why? Service account JWTs have a lot going for them: They are built directly into Kubelet, and work pretty magically. They are written to your workload container's filesystem before your workload starts up, and automatically kept up to date. The issuance system follows least-privilege principles; the node restriction admission plugin ensures that tokens can only be requested by the Kubelet that is actually currently running your pod. They can be federated, allowing you to use them to authenticate to other systems outside of Kubernetes. Service account JWTs underpin the pod-to-cloud authentication store for all of the largest cloud providers, and have widespread support across many additional services and software packages. If it can understand JWTs, you can authenticate to it with a service account token. However, service account JWTs have one big downside -- they are bearer tokens. With bearer tokens, if you have the token, then you are the identity asserted by the token. And since you necessarily have to hand copies of the J ## Kubernetes v1.37: Metrics API graduates to stable DevFeed: [Kubernetes v1.37: Metrics API graduates to stable]() Original publisher: [Read original article]() Author: ChengHao Yang Published: 2026-08-27T18:30:00Z Content type: article Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [API](), [autoscaling](), [cpu](), [Monitoring]() Tags: [api](), [autoscaling](), [cpu](), [developers](), [kubernetes](), [memory](), [metrics](), [monitoring](), [production](), [release](), [resource](), [v1]() ### AI overview Kubernetes v1.37 promotes the Metrics API to the stable v1 version. The API provides CPU and memory usage for nodes and Pods, supporting commands such as kubectl top and resource-metrics-based autoscaling, without changing the collected metrics, resource types, or fields. ### Source excerpt Kubernetes v1.37 promotes the metrics.k8s.io API to stable (v1). This API provides CPU and memory usage for nodes and Pods, and is the API behind commands such as kubectl top and resource-metrics-based autoscaling. For cluster operators and application developers, this graduation means that the API now has the stability guarantees associated with a Kubernetes stable API. The v1 API has the same resource types and fields as v1beta1; this is an API-version graduation, not a change to the metrics that are collected or returned. A long-lived API reaches stable The resource Metrics API was introduced as alpha in Kubernetes v1.6 and became beta in v1.8. It has remained unchanged and has been used in production for years by clients including the HorizontalPodAutoscaler (HPA) and kubectl top. Kubernetes v1.37 formally graduates that proven API to metrics.k8s.io/v1. The API exposes two resource types: NodeMetrics, for CPU and memory usage for a node. PodMetrics, for CPU and memory usage for a Pod, with a per-container breakdown in its containers field. The API remains intentionally small. It provides the resource metrics needed for autoscaling and basic inspection; it is not a replacement for a full monitoring pipeline or the custom metrics (custom.metrics.k8s.io) API. What changed with the v1.37 release? The v1 API surface is identical to v1beta1, except for the API version. There are no renamed fields, new fields, or changes to the meaning of the returned CPU and memory values. For example, a client can retrieve node metrics from the stable endpoint: kubectl get --raw /apis/metrics.k8s.io/v1/nodes Likewise, it can retrieve metrics for the pods in a namespace: kubectl get --raw /apis/metrics.k8s.io/v1/namespaces/default/pods kubectl top supports both API versions. It prefers v1 when available and automatically falls back to v1beta1 on clusters that do not yet serve v1. The HPA controller currently supports only v1beta1. Support for discovery-based selection between v1 and v ## Kubernetes v1.37: Garhwal DevFeed: [Kubernetes v1.37: Garhwal]() Original publisher: [Read original article]() Author: Kubernetes v Published: 2026-08-26T00:00:00Z Content type: article Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [Kubernetes v1.33](), [releases]() Tags: [community](), [kubernetes](), [release](), [releases](), [v1]() ### AI overview Kubernetes v1.37, named Garhwal, introduces 67 enhancements across Stable, Beta, and Alpha stages, including a deprecation or removal. The article also explains the release theme and its Himalayan-inspired logo. ### Source excerpt Editors: Arsh Sharma, Christopher Tineo, Kirti Goyal, Sophia Ugochukwu, Swathi Rao, Troy Connor Similar to previous releases, the release of Kubernetes v1.37 introduces new Stable, Beta, and Alpha features. The consistent delivery of high-quality releases underscores the strength of our development cycle and the vibrant support from our community. This release consists of 67 enhancements. Of those enhancements, 16 have graduated to Stable, 23 have graduated to Beta, 27 are entering Alpha, and 1 is a deprecation/removal. Release theme and logo The theme for Kubernetes v1.37 is Garhwal (गढ़वाल, pronounced gaṛhvāl), a Himalayan region of Uttarakhand, India. The snow-capped peaks of the Garhwal Himalaya, deodar forests, terraced fields, rivers and streams, and mountain paths shape both the region and the logo. Together, these elements reflect a community in which every layer, route, and contribution is connected. The logo is imagined as a window into Garhwal's landscape.1 Inside, terraced fields climb towards the snowy peaks, each level supported by the one below, much as every Kubernetes release depends on work carried forward. A river winds through the valley and gathers mountain streams, reflecting contributions from many SIGs and communities flowing into one project. The deodar forest represents the wider Kubernetes ecosystem, where distinct projects share common ground and grow side by side. Stonework and woodcraft shape the path and mountain house, placing people at the centre and evoking shared foundations maintained for those who follow. Above the river, colourful flags catch the wind and bring the scene to life. Encircling the scene is a patterned frame inspired by basketry woven from ringaal, a flexible dwarf Himalayan bamboo. Individual strips gain strength when interlaced, just as code, reviews, tests, documentation, and coordination come together to make a release. Within the frame, the Himalayan monal, Uttarakhand's state bird, lives at high altitudes in t ## How to Pretty-Print Your Kubernetes YAML as KYAML and Why You'd Want To DevFeed: [How to Pretty-Print Your Kubernetes YAML as KYAML and Why You'd Want To]() Original publisher: [Read original article]() Author: Kashish Verma Published: 2026-08-11T18:00:00Z Content type: tutorial Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [configuration](), [JSON](), [bug]() Tags: [bug](), [how-to](), [json](), [kubernetes](), [tutorial]() ### AI overview KYAML is presented as a stricter subset of YAML for Kubernetes manifests. The article explains how it reduces ambiguity by standardizing structure and value types while remaining compatible with existing YAML tooling, and contrasts its configuration-writing experience with JSON. ### Source excerpt YAML has been the standard way to write Kubernetes manifests for years. Every example, tutorial, and configuration file you come across is written in it. The problem isn't that YAML is a bad format. It's that YAML gives you a lot of choices, and not all of them are equally good for writing Kubernetes manifests. Some features make files harder to read, some are easy to misuse and others can lead to surprising behavior. The interesting part is that Kubernetes doesn't actually need most of those features. It only relies on a small subset of YAML. This led to a simple question: if Kubernetes only needs a small part of YAML, why not standardize on that part and avoid the rest? Instead of introducing a new configuration language, SIG CLI introduced KYAML, a stricter, more consistent way to write YAML. What is KYAML? KYAML is a strict subset (or "dialect") of standard YAML, designed to be parseable by the existing ecosystem without any changes, as proposed in KEP 5295. It does not introduce a new format or a new parser. It just narrows the scope of choices you make when writing YAML, so everyone ends up making the same ones. Think of it less like a new language and more like an agreed-upon style. Everything valid in KYAML is valid YAML. How KYAML solves it Standard YAML has a few well-known traps and JSON is not without its own. Whitespace sensitivity. Indentation defines structure in YAML, which means a wrongly indented file can remain syntactically valid while representing a different object than intended. This gets especially painful with templating tools like Helm, where you are manipulating indentation from outside the YAML context. Silent type coercion. String quoting is optional in YAML, which sounds convenient until it is not. Some values that look like strings get coerced into other types without warning. The classic example is the "Norway Bug". country: NO In standard YAML, NO is parsed as a boolean false, not the string "NO" and it has caught more than a few peo ## Gateway API v1.6: TCPRoute and UDPRoute Graduate to Standard DevFeed: [Gateway API v1.6: TCPRoute and UDPRoute Graduate to Standard]() Original publisher: [Read original article]() Author: Beka Modebadze; Ricardo Katz Published: 2026-08-03T16:00:00Z Content type: article Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [networking](), [Protocol (disambiguation)](), [API](), [releases]() Tags: [api](), [kubernetes](), [networking](), [release](), [v1]() ### AI overview Gateway API v1.6.0 expands Kubernetes service networking with TCPRoute and UDPRoute graduating to the Standard channel and v1 API, enabling portable Layer 4 routing for raw TCP and UDP traffic. The release also separates experimental API resources into a distinct group. ### Source excerpt The Kubernetes SIG Network community is thrilled to share the release of Gateway API v1.6.0, which was released on June 30th of this year! Gateway API has become the standard for modern, role-oriented, and expressive service networking in Kubernetes. In previous releases, Gateway API established a production-grade foundation for HTTP and TLS layer 7 traffic. With version 1.6.0, Gateway API takes a major step forward by expanding standard layer 4 protocol routing and introducing cleaner API boundaries for experimental innovation. Here is a quick summary of what's new in Gateway API v1.6.0: TCPRoute and UDPRoute Graduate to Standard: Raw L4 TCP and UDP traffic routing reach GA stability in the v1 API version. Experimental API Group Separation: Experimental resources transition to a distinct API group (gateway.networking.x-k8s.io) with an X prefix to make experimental vs. standard boundaries crystal clear. Let's dive into the details! TCPRoute and UDPRoute graduate to Standard Leads: Nick Young, Ricardo Katz and Zac Nixon GEP-2644 - TCPRoute GEP-2645 - UDPRoute Until now, Gateway API only offered a stable routing model for HTTP and TLS traffic. Workloads that speak a raw protocol over TCP or UDP - databases, DNS, VoIP, gaming, IoT telemetry - had no portable way to plug into a Gateway. Users either fell back to a plain Kubernetes Service, or to an implementation-specific CRD that doesn't travel between Gateway controllers. TCPRoute and UDPRoute close that gap: they route traffic to backends based on protocol and port alone, no L7 awareness required. With this release, both have graduated from the Experimental channel to Standard, and moved to the v1 API version. The v1alpha2 version of each was deprecated as of the v1.6 release, and will be removed in a future release. How it works A Gateway needs a listener that allows TCPRoute attachment: apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: example-gateway spec: gatewayClassName: example-gateway-cla ## Kubernetes v1.37 Sneak Peek DevFeed: [Kubernetes v1.37 Sneak Peek]() Original publisher: [Read original article]() Author: Arsh Sharma; Christopher Tineo; Kirti Goyal; Sophia Ugochukwu; Swathi Rao; Troy Connor Published: 2026-07-31T16:00:00Z Content type: article Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [Command-line interface](), [bug](), [Linux](), [configuration]() Tags: [blog](), [bug](), [cli](), [kernel](), [kubernetes](), [logging](), [memory](), [release](), [resource](), [secrets](), [startup]() ### AI overview This Kubernetes blog previews planned changes for the v1.37 release, including deprecations to kubectl run's filename flag and kube-proxy's ipvs mode. It also describes a fix preventing Static Pods from referencing Secrets or ConfigMaps and the phaseout of cgroup v1 support. ### Source excerpt As we get closer to the release date for Kubernetes v1.37, the project develops and matures, features may be deprecated, removed, or replaced with better ones for the project's overall health. This blog outlines some of the planned changes for the Kubernetes v1.37 release that the release team feels you should be aware of for the continued maintenance of your Kubernetes environment and keeping up to date with the latest changes. The information below reflects the current status of the v1.37 release and may change before the actual release date. Deprecations and removals for Kubernetes v1.37Kubectl: kubectl run --filename/-f to be deprecated The --filename (or -f) flag for kubectl run is being deprecated as the generated pod is always built purely from CLI arguments like NAME and --image. See kubernetes/kubernetes#138671 for the original issue and discussion. Kubelet: Static Pods can no longer reference Secrets or ConfigMaps Static Pods were never meant to read API resources directly, since they aren't created through the API server -- but a bug let them reference Secrets or ConfigMaps via fields like configMapRef or secretRef. That bug is now fixed: as of v1.37 these references are strictly prohibited, and the PreventStaticPodAPIReferences feature gate that previously let you opt out of the restriction has been removed. See kubernetes/kubernetes#140226 for the original issue and discussion. Deprecating kube-proxy's support for ipvs mode kube-proxy support for ipvs mode was introduced in v1.8 to resolve iptables performance bottlenecks. However, since the kernel ipvs API alone cannot fully implement Kubernetes Services, ipvs mode continues to use iptables underneath (KEP-3866, "The ipvs mode of kube-proxy will not save us"). Clusters running kube-proxy in ipvs mode (or mode: ipvs in KubeProxyConfiguration) would now be logging a deprecation warning on startup. The deprecation timeline looks like this: By v1.40, ipvs mode for kube-proxy is expected to be disabled by de ## How the controller-runtime Cache Actually Works, and Why Your Controller Does Not Crash the API Server DevFeed: [How the controller-runtime Cache Actually Works, and Why Your Controller Does Not Crash the API Server]() Original publisher: [Read original article]() Author: Andrei Kvapil; Timofei Larkin Published: 2026-07-29T18:00:00Z Content type: article Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [Go Language](), [Cache](), [Caching]() Tags: [article](), [cache](), [go](), [golang](), [internals](), [kubernetes](), [memory](), [network]() ### AI overview This article explains how the controller-runtime cache works in Kubernetes controllers written in Go. It describes the local in-memory cache populated by list and watch operations, contrasts cached reads with direct API-server access, and examines the practical effects on memory usage, network traffic, read consistency, and reconciler behavior. ### Source excerpt This article has been revised since it was first published, to correct several significant technical inaccuracies in the original text. Kubernetes has long been the default platform for distributed workloads, and writing your own controller for it is now a matter of a few hours. The common path -- Golang, using kubebuilder on top of controller-runtime -- gives you a project scaffold, types, and a reconciler. For typical scenarios that is more than enough. But as soon as load grows or the controller starts behaving in ways you did not expect, a whole class of edge cases shows up. Most of them trace back to the same root cause: a fuzzy mental model of how controller-runtime works inside. If you write Kubernetes controllers in Go, this article should help you build a coherent picture and avoid expensive surprises in production. This article walks through the internals of controller-runtime and, along the way, shows which architectural decisions are baked into Kubernetes itself. The starting point is how controllers actually read objects from the Kubernetes API. A common misconception goes like this: r.Get() inside Reconcile queries kube-apiserver directly; r.List() returns a fresh, live view of the world; and after r.Update() you can re-read the object and immediately see the new state. In practice the model is the opposite: controller-runtime operates against a local copy of the data populated through list + watch. Reads inside a reconciler cost almost nothing and do not load the control plane even at hundreds of calls per second -- but the price of this design is that a controller can quietly consume gigabytes of memory, perform hidden O(n) scans, and regularly trip over stale reads. This post is aimed at engineers who already write controllers in Go with controller-runtime but want to consolidate the pieces into a single mental model rather than carry around a bag of isolated observations. The focus is the practical impact on production clusters: memory, network traffi ## Building a Custom Metrics Exporter for Kubernetes DevFeed: [Building a Custom Metrics Exporter for Kubernetes]() Original publisher: [Read original article]() Author: Victor David Effiok Published: 2026-07-14T18:00:00Z Content type: tutorial Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [autoscaling](), [Grafana Cloud Metrics](), [Time Series](), [Go Language](), [WebSocket]() Tags: [autoscaling](), [building](), [code](), [go](), [http](), [kubernetes](), [metrics](), [prometheus](), [server](), [time-series]() ### AI overview A tutorial on building a custom Kubernetes metrics exporter that exposes application or external state for Prometheus, enabling queries, alerts, and HorizontalPodAutoscaler scaling. It covers exporter architecture, direct instrumentation, metric formatting, and choosing counters, gauges, or histograms. ### Source excerpt Kubernetes ships with built-in awareness of CPU and memory, but most real-world scaling decisions depend on signals that live entirely outside that narrow window: how many messages are waiting in a queue, how long the last batch job took, how many active WebSocket connections a pod is holding. When the built-in metrics are not enough, a metrics exporter bridges that gap. This post walks through writing one from scratch, packaging it as a container, and wiring it into a cluster so that Prometheus -- and ultimately the HorizontalPodAutoscaler -- can consume it. What a metrics exporter actually does An exporter is a small HTTP server with a single responsibility: expose application state as text on a /metrics endpoint. Prometheus scrapes that endpoint on a regular interval, stores the time-series data, and makes it available for queries, alerts, and autoscaling rules. In some cases you can instrument your application directly -- embedding the Prometheus client library and exposing /metrics from within the same process -- rather than running a separate exporter. A standalone exporter makes more sense when the data source is external to your application or when you do not control the application code. The format Prometheus expects is plain text -- one metric per line, with a name, optional labels, and a numeric value. Client libraries handle the serialization for you, so in practice you only need to decide what to measure and call the right function when that value changes. Choosing what to measure Before writing any code, it helps to decide what kind of signal you are dealing with. The Prometheus data model has three main types: Counters only ever increase. They are the right tool for totals: requests served, jobs processed, errors encountered. Never use a counter for a value that can go down. Gauges represent a current snapshot of a value that can rise and fall freely. Queue depth, active connections, and cache size are all gauges. Histograms record the distribution of obse ## Operating AI/ML Workloads on Kubernetes: A Headlamp Plugin for Kubeflow DevFeed: [Operating AI/ML Workloads on Kubernetes: A Headlamp Plugin for Kubeflow]() Original publisher: [Read original article]() Author: Alok Dangre Published: 2026-07-13T20:00:00Z Content type: article Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [dashboards](), [AI Development](), [API](), [Web](), [data]() Tags: [ai-ml](), [apache](), [api](), [dashboards](), [kubernetes](), [site-reliability]() ### AI overview This article introduces a Headlamp plugin for Kubeflow that surfaces Kubeflow custom resources and Kubernetes-level workload information in a general-purpose Kubernetes UI. It helps operators troubleshoot notebooks, training jobs, experiments, pipelines, Pods, and related resources through the Kubernetes API. ### Source excerpt Kubernetes has quietly become the default platform for AI and machine learning. Whether you run notebook servers for data scientists, schedule distributed training jobs, tune hyperparameters, or orchestrate multi-step ML pipelines, those workloads increasingly land on a Kubernetes cluster. Kubeflow is one of the most popular ways to assemble that stack, and it does so the Kubernetes-native way: every capability is exposed as a Custom Resource Definition (CRD). That design is a gift to cluster operators, because it means ML workloads can be observed and managed with the same primitives as everything else in the cluster. But in practice the specialized ML dashboards that ship with these platforms hide the Kubernetes layer underneath. When a notebook is stuck or a training run fails, the operator is often left dropping back to kubectl to find out what actually happened at the Pod level. This post introduces the Headlamp Kubeflow plugin, which closes that gap by surfacing Kubeflow's custom resources directly inside a general-purpose Kubernetes UI. It is a worked example of a pattern any CRD-heavy platform can follow: meet operators where they already work, and show them the cluster-level truth. Headlamp itself is an extensible Kubernetes web UI maintained under Kubernetes SIG UI and licensed under Apache 2.0. It runs as a desktop app or in-cluster, and its plugin system lets anyone add first-class views for custom resources. Why operators need a different view Purpose-built ML dashboards help data scientists submit experiments, pipelines, and notebooks. Cluster operators and site reliability engineers (SREs) troubleshoot the Kubernetes resources underneath, and they ask different questions: Why is a notebook stuck? Is it ImagePullBackOff, OOMKilled, or a Pod waiting on a PersistentVolumeClaim? Which Run resources failed recently across namespaces? Which parameter set does a Katib Experiment report as optimal? Do TrainJob resources reference the expected TrainingRuntime ## Kubernetes Dashboard to Headlamp: A Step-by-Step Guide DevFeed: [Kubernetes Dashboard to Headlamp: A Step-by-Step Guide]() Original publisher: [Read original article]() Author: Vincent T Published: 2026-07-13T18:00:00Z Content type: tutorial Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [dashboards](), [migration](), [Web app](), [client](), [API](), [YAML](), [Authorization]() Tags: [api](), [app](), [debug](), [dev](), [features](), [forms](), [guide](), [kubernetes](), [logs](), [migration](), [resources](), [scale](), [tokens](), [ui](), [web]() ### AI overview A step-by-step guide to moving from Kubernetes Dashboard to Headlamp. It compares their deployment models, authentication, permissions, resource-management workflows, and multi-cluster capabilities, then introduces a pre-migration checklist. ### Source excerpt 1. Before you start: know what is changing Kubernetes Dashboard and Headlamp both show what is running in a cluster, but they work differently. When Headlamp runs on the desktop, it uses your existing kubeconfig to connect to one or more clusters and can be extended with plugins. When Headlamp runs inside a cluster, it uses a Kubernetes ServiceAccount to access the API and follow RBAC rules. Kubernetes Dashboard, in contrast, only runs in-cluster and always relies on service account tokens. Understanding these models early helps you choose the right setup and permissions. 1.1 How Kubernetes Dashboard works Dashboard is a web app that runs inside your cluster. You install it in the cluster, often with Helm. You usually run one Dashboard per cluster. You often reach it with kubectl port-forward or an ingress. You log in with a Bearer token. That token is often from a service account. It includes forms that help you create resources. It leans on tables and lists for navigation. It feels like this: a UI that lives with the cluster. 1.2 How Headlamp works Headlamp acts more like a Kubernetes client with a UI. It can run on your desktop or in a cluster. It reads your kubeconfig, like kubectl does. It can show more than one cluster in one place. It favors YAML when you create or change resources. It includes list views and a visual map. You can add features with plugins. Headlamp is a UI that follows your identity, not your cluster. 1.3 What stays the same Many workflows will feel familiar: Browse workloads and resources Filter by namespace Inspect YAML, events, and status View logs Take actions your RBAC allows 1.4 What changes A few things will feel different: Login shifts from pasted tokens to kubeconfig (and sometimes SSO). Creation shifts from forms to "apply YAML." Multi-cluster becomes normal, not a special case. The map view helps you see how resources connect. 2. Pre-migration checklist This checklist helps you avoid surprises during the switch. It makes sure Head ## Announcing etcd v3.7.0 DevFeed: [Announcing etcd v3.7.0]() Original publisher: [Read original article]() Author: SIG Etcd Leads Published: 2026-07-08T12:00:00Z Content type: release Language: en Sources: [Kubernetes Blog]() Topics: [releases](), [Database](), [Kubernetes](), [Latency](), [Remote Procedure Call (RPC)]() Tags: [changelog](), [cpu](), [database](), [kubernetes](), [latency](), [memory](), [performance](), [release](), [releases](), [rpc]() ### AI overview etcd v3.7.0 is a minor release of the distributed key-value store used as a core Kubernetes component. It introduces RangeStream for chunked large result sets, improves performance and leases, removes the legacy v2store, and completes a protobuf overhaul. ### Source excerpt This article is a mirror of the original announcement Today, SIG etcd is releasing etcd v3.7.0, the latest minor release of the popular distributed key-value store and core Kubernetes component. v3.7 ships the long-requested RangeStream feature, delivers several other performance improvements, removes the last remnants of the legacy v2store, and completes a major protobuf overhaul. You can download etcd v3.7.0 here: Source code Binaries Official container images This release also includes new versions of the two core etcd dependencies, bbolt v1.5.0 and raft v3.7.0. For instructions on installing etcd, see the install documentation. For the full list of changes, see the etcd v3.7 changelog. A heartfelt thank you to all the contributors who made this release possible! Major features The most significant changes in v3.7.0 include: RangeStream -- stream large result sets in chunks instead of buffering the whole response. Keys-only range requests, faster and more reliable leases, and several other performance improvements. etcd now boots entirely from v3store, eliminating a long-standing dependency on the legacy v2 store A completed protobuf overhaul, replacing outdated protobuf libraries with fully supported ones. etcd v3.7 ships with bbolt v1.5.1 and raft v3.7.0. FeaturesRangeStream In etcd v3.6 and earlier, it is challenging to work with requests that return large result sets. The database would buffer the full result set before sending, leading to unpredictable latency and memory usage, both on the server and the client. The RangeStream RPC lets calling applications accept result sets in chunks, reducing latency and making buffering memory usage more predictable. Instructions on how to use RangeStream in gRPC calls and in etcdctl can be found in the etcd documentation. Users should try it out for their own applications. In coordinated releases, the RangeStream feature will become available to users running the upcoming v1.37 of Kubernetes by enabling the EtcdRangeStre [Next page]()