Kubernetes Performance Tuning & Optimization (2026)

What is Kubernetes performance tuning?

Kubernetes (K8s) performance tuning, also called performance optimization, means finding what slows your workloads down and fixing it. Most fixes are configuration changes. You size requests and limits to real use, scale on the right signal, place pods well, and adjust a few node, network and storage settings. Measure first, then change one thing at a time.

This guide goes from the most common bottleneck to the rarest. It covers CPU and memory, autoscaling, scheduling, nodes, network, storage, the control plane and GPU workloads. It reflects Kubernetes as of October 2026, when 1.37 is the current release.

Common Kubernetes performance problems

Most slowdowns show up as one of a few symptoms. The table maps each one to its usual cause and to the section that covers the fix.

Our Kubernetes consulting team starts with read-only access and a written list of what to fix first.

Kubernetes performance optimization best practices

These ten practices cover the most common bottlenecks. The sections below explain each one.

1. Measure first. Set a latency SLO for each service, and track p99 latency, CPU throttling and memory.
2. Give every container CPU and memory requests that match its real use.
3. On latency-sensitive services, consider a CPU request with no CPU limit. Keep memory limits.
4. Scale replicas on a metric that tracks load, and run a node autoscaler so new pods don't wait in Pending.
5. Spread replicas across zones and nodes, and keep GPU nodes for GPU pods.
6. Reserve CPU and memory on every node for the kubelet and the OS.
7. Set the kube-proxy mode explicitly, and speed up DNS with a lower ndots or NodeLocal DNSCache.
8. Match volume IOPS and throughput to the workload, and keep etcd on fast SSDs.
9. Share GPUs with MIG or time-slicing, and queue batch jobs with Kueue.
10. Automate routine sizing with the VPA once its recommendations look right.

How to measure Kubernetes performance

Start with what users feel. Google's SRE book names four golden signals: latency, traffic, errors and saturation. Track latency at the 99th percentile, because latency often rises before a service runs out of capacity. Set a latency SLO for each service, so every change has a clear pass or fail.

Then find the resource that saturates. Four signals cover most cases.

• CPU throttling. cAdvisor exports container_cpu_cfs_throttled_periods_total and container_cpu_cfs_periods_total. Their ratio shows how often a container hit its CPU limit.
• Memory. Compare container_memory_working_set_bytes with each container's memory limit.
• Pressure stall information (PSI). PSI metrics are stable since Kubernetes 1.36. They show how long tasks waited for CPU, memory or I/O, per container. They need Linux 4.20 or later and cgroup v2.
• Events. kubectl get events --field-selector reason=FailedScheduling lists pods that found no room.

This Prometheus query gives the share of throttled periods per container:

sum by (namespace, pod, container) (
  rate(container_cpu_cfs_throttled_periods_total[5m])
)
/
sum by (namespace, pod, container) (
  rate(container_cpu_cfs_periods_total[5m])
)

A result of 0.25 means the container was throttled in a quarter of its periods. Look first at containers that are throttled often while their service is slow.

Metrics Server feeds the HPA, the VPA and kubectl top. Its maintainers say it is meant only for autoscaling, so use Prometheus for monitoring. Our Metrics Server guide covers the setup, and our Kubernetes monitoring guide covers the full pipeline and its tools.

Test before and after every change. k6 and Locust load-test applications. ClusterLoader2, the official Kubernetes scalability test framework, and kube-burner load-test the cluster itself. If the cluster looks healthy and latency is still high, profile the application. Go's net/http/pprof package, for example, serves profiling data over HTTP.

Kubernetes requests, limits and CPU throttling

The scheduler places pods by their requests. On a busy node, the CPU request also works as a weight, so containers with larger CPU requests get more CPU time. If you set a limit without a request, Kubernetes copies the limit into the request.

How CPU limits cause throttling

A CPU limit is a hard ceiling. The kernel enforces it with a CFS quota for each period, which is 100 ms by default. Once a container spends its quota, its threads wait for the next period. For example, a container with a 1-CPU limit and four busy threads can spend its quota in 25 ms, then wait 75 ms. Users see that wait as latency, even when the node has idle cores.

For latency-sensitive services, consider a CPU request with no CPU limit. The request still guarantees the container its weighted share under contention, and spare CPU absorbs bursts. Keep CPU limits where you need a hard ceiling:

• shared nodes with many tenants;
• batch jobs that run beside services;
• Guaranteed pods that need exclusive cores.

Match Go and Java runtimes to container limits

Some language runtimes size themselves from the CPUs and memory they detect. Check that they see the container's limits.

• Go. Since Go 1.25, GOMAXPROCS defaults to the container's CPU limit when that is lower than the number of CPUs available. The new default needs go 1.25 or later in the module's go.mod. Go ignores CPU requests, so without a CPU limit, GOMAXPROCS still matches every CPU the container can see. Set the GOMAXPROCS environment variable on those containers.
• Java. The JVM detects container limits by default on Linux. Its default maximum heap is 25% of the memory it detects. Raise it with -XX:MaxRAMPercentage when the app needs more heap.

Memory limits and OOMKilled

Memory limits work differently from CPU limits. The kernel enforces memory limits with OOM kills, and only when it detects memory pressure. A container can briefly exceed its limit, then get killed. Kubernetes restarts it, and kubectl describe pod shows Reason: OOMKilled. If the kills repeat, the pod shows CrashLoopBackOff. If that happens, check the code for a leak before you raise the limit. Our troubleshooting guide shows how to confirm an OOM kill. A container that uses more than its memory request can also be evicted when the node runs short.

Kubernetes QoS classes

Requests and limits also set each pod's QoS class, which decides which pods go first when a node runs short of resources.

A pod with no requests and no limits is BestEffort. A LimitRange sets default requests and limits for a namespace, so no pod lands without them.

Resize pods without restarting them

In-place pod resize is stable since Kubernetes 1.35. You can change a running container's CPU and memory without recreating the pod. You need kubectl 1.32 or later. This example resizes a Guaranteed pod:

kubectl patch pod my-app --subresource resize --patch \
  '{"spec":{"containers":[{"name":"app","resources":{"requests":{"cpu":"800m"},"limits":{"cpu":"800m"}}}]}}'

A resize can't change the pod's QoS class. Pods managed by the static CPU or Memory Manager can't be resized in place. Many apps and runtimes can't adjust their memory use while running. For those, set the container's resizePolicy to RestartContainer for memory.

This section sizes resources for latency. Our Kubernetes cost optimization checklist sizes them for the bill.

Kubernetes autoscaling for performance

For performance, autoscaling must add capacity before users feel the load. Our Kubernetes autoscaling guide covers the HPA, VPA, KEDA and Cluster Autoscaler in depth, and Karpenter has its own guide. These settings matter most for latency.

Scale on the signal that tracks load. The HPA computes CPU utilization as a share of each pod's request, and takes no action on CPU if any container has no CPU request. For latency-sensitive services, scale on requests per second or queue depth through custom or external metrics. If the HPA never adds replicas, start with the guide's checks for <unknown> targets.

Keep room for the next burst. Pods that fit on no node wait in Pending until a node autoscaler adds one, and node start-up time depends mostly on your cloud. If they can't wait, run low-priority placeholder pods that real pods preempt.

Expect cold starts from zero. In Kubernetes 1.37, the HPA can scale a workload to zero as a beta feature, enabled by default, and KEDA can too. Services don't buffer requests while no pods are ready, so HTTP services need a buffering layer in front.

The VPA keeps requests close to real use. The section on optimizing without manual tuning shows how to roll it out.

Kubernetes scheduling for performance

Where pods land affects latency as much as how big they are.

• Spread replicas. Topology spread constraints spread pods across zones and nodes, for availability and better resource use.
• Use affinity with care. Pod affinity keeps chatty services together, and anti-affinity keeps replicas apart. The docs warn that inter-pod affinity can slow scheduling in large clusters significantly. They advise against it above several hundred nodes. For spreading replicas, topology spread constraints give finer control.
• Keep special hardware free. Taint GPU nodes so pods that don't need GPUs stay off them.
• Protect critical pods. With PriorityClasses, the scheduler evicts lower-priority pods when a higher-priority pod can't be placed.

CPU Manager and Topology Manager for latency-sensitive pods

By default, the kubelet enforces CPU limits with CFS quota, and a container's threads can move between cores. For workloads sensitive to CPU cache affinity and scheduling latency, use the CPU Manager static policy, stable since 1.26. It gives exclusive cores to containers in Guaranteed pods with whole-number CPU requests.

The Topology Manager, stable since 1.27, aligns a pod's CPUs and devices on the same NUMA node. The Memory Manager, stable since 1.32, guarantees memory and huge pages to Guaranteed pods in the same way.

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cpuManagerPolicy: static
reservedSystemCPUs: "0,1"
topologyManagerPolicy: single-numa-node

Three cautions apply:

• The static policy needs some CPUs reserved for the system.
• Changing the policy means draining the node, stopping the kubelet and deleting /var/lib/kubelet/cpu_manager_state before you restart it.
• With the restricted or single-numa-node policy, a pod that can't be aligned fails admission and is not rescheduled. Run such pods under a Deployment or another controller.

Kubernetes node and kernel tuning

Reserve resources for the system. By default, pods can use a node's whole capacity, and they compete with system daemons. Set kubeReserved and systemReserved through Node Allocatable so the kubelet and the OS keep their share.

Run cgroup v2 with a current runtime. Kubernetes deprecated cgroup v1 in 1.35, and the kubelet no longer starts on a cgroup v1 node by default. Check your OCI runtime too. Since runc 1.3.2 and crun 1.23, runtimes use a new CPU weight formula. Under it, a 1-CPU request gets a weight of 102, close to the default of 100. The old formula gave about 39, which lowered pods' priority against system processes.

Set sysctls the supported way. Kubernetes splits sysctls into safe and unsafe sets.

• Safe ones, such as net.ipv4.ip_local_port_range and the TCP keepalive settings, can be set per pod in its securityContext. Version 1.37 added net.ipv4.tcp_slow_start_after_idle and net.ipv4.tcp_notsent_lowat to the safe set.
• Unsafe ones, such as net.core.somaxconn, must be allowed per node in the kubelet's allowedUnsafeSysctls setting. A pod that asks for one that isn't allowed is scheduled but fails to start.
• Node-level sysctls belong in the node image or a privileged DaemonSet.

Keep OS tuning in profiles. TuneD applies tuning profiles that cover sysctl, sysfs and kernel boot parameters, so node settings live in one place.

Pre-allocate huge pages. Apps that use huge pages need nodes that pre-allocate them. A pod's huge page requests must equal its limits.

Shrink images. The kubelet pulls one image at a time by default (serializeImagePulls: true). A large image can delay other pods starting on the same node. Multi-stage builds leave build tools out of the final image, and distroless images contain only the app and its runtime dependencies. On nodes that start many pods at once, set serializeImagePulls: false in the kubelet configuration and cap parallel pulls with maxParallelImagePulls.

Set probes carefully. The docs warn that badly set liveness probes can cause cascading failures, restarting containers under high load. Give slow-starting apps a startup probe. Our CrashLoopBackOff guide shows how to spot a probe that keeps restarting a container. Add a readiness probe, so Services send no traffic to a pod until it has loaded its data and warmed its caches. On dense nodes, prefer HTTP or TCP probes over exec probes, which fork new processes each time.

Kubernetes network performance

Our Kubernetes networking deep dive explains the network model and the choice of CNI. These settings affect speed the most.

Choose the kube-proxy mode on purpose. iptables is still the default in 1.37. In iptables mode, first-packet latency grows as the number of Services grows. The nftables mode needs kernel 5.13 or later. It processes endpoint changes faster and handles packets more efficiently. The packet-handling gain only shows in clusters with tens of thousands of Services. IPVS mode is deprecated since 1.35. It will be off by default from 1.40 and removed in 1.43. Set the mode explicitly, so an upgrade can't change it.

Or replace kube-proxy with eBPF. Cilium can fully replace kube-proxy. Its defaults favor compatibility over speed. Its tuning guide lists a performance profile with netkit, eBPF host routing and BIG TCP. That profile needs kernel 6.8 or later, and it is easiest to roll out on new nodes.

Get the MTU right. Encapsulation shrinks the usable packet size. Calico's MTU table shows a 1,500-byte network leaving 1,450 bytes with VXLAN on IPv4 and 1,440 with WireGuard. Calico detects the MTU automatically by default. Use the highest MTU that causes no fragmentation or dropped packets.

Fix DNS. Every pod gets options ndots:5 by default. Under resolver rules, a name with fewer dots goes through each cluster search domain before the real lookup. For pods that mostly call external hosts, lower it:

dnsConfig:
  options:
  - name: ndots
    value: "2"

NodeLocal DNSCache, stable since 1.18, runs a DNS cache on every node. It skips iptables DNAT and connection tracking, which reduces conntrack races. It can also send queries to CoreDNS over TCP. That cuts tail latency from dropped UDP packets, which can cause DNS timeouts of up to 30 seconds.

Keep traffic in the zone. Set a Service's trafficDistribution to PreferSameZone to prefer endpoints in the client's zone. Our load balancing guide covers traffic policies.

Kubernetes storage performance

For databases and other data workloads, the volume type sets the performance ceiling. As of October 2026, an AWS gp3 volume includes a baseline of 3,000 IOPS and 125 MiB/s. You can provision up to 80,000 IOPS and 2,000 MiB/s at extra cost. Check what your volumes allow before tuning the app.

Use CSI drivers. The in-tree AWS EBS and GCE PD drivers were removed in 1.27 and 1.28. StorageClasses that name kubernetes.io/aws-ebs or kubernetes.io/gce-pd are out of date. The CSI drivers also enable the next feature.

Change performance on a live volume. VolumeAttributesClass is generally available since 1.34. It changes IOPS or throughput without a new volume, where the CSI driver supports it. The AWS EBS CSI driver does:

apiVersion: storage.k8s.io/v1
kind: VolumeAttributesClass
metadata:
  name: gp3-fast
driverName: ebs.csi.aws.com
parameters:
  type: gp3
  iops: "6000"
  throughput: "250"

Point a claim at it with volumeAttributesClassName: gp3-fast. EBS enforces a cooldown between modifications, so plan changes ahead.

Use local disks for the lowest latency. Local persistent volumes use disks attached to the node. They are static only and tied to one node. If that node fails, the pod can't run until the node returns. Use a StorageClass with volumeBindingMode: WaitForFirstConsumer, so binding waits for the pod's other constraints.

On bare metal, the storage system is yours to tune as well. Our Ceph on Kubernetes guide covers Ceph's hardware and network recommendations. Ceph's install docs recommend Rook for running Ceph in Kubernetes. We moved a long-term client's big-data platform onto on-prem Kubernetes. We still run those clusters, and we are moving their databases onto them.

Kubernetes control plane performance: etcd, API server and scheduler

Most of this section applies when you run the control plane yourself.

etcd. etcd is very sensitive to disk write latency. Its docs recommend SSDs and about 500 sequential IOPS for heavily loaded clusters. Cloud providers usually publish concurrent IOPS, which can be ten times the sequential figure.

Watch the disk metrics etcd_disk_wal_fsync_duration_seconds and etcd_disk_backend_commit_duration_seconds. The etcd FAQ treats a disk as reasonably fast when the WAL fsync p99 is under 10 ms and the backend commit p99 is under 25 ms. Run etcd on dedicated machines.

Adding etcd members raises availability and costs some performance, so don't autoscale etcd. In 1.37, etcd RangeStream streams large list reads in chunks and lowers memory use. It is beta, on by default, and needs etcd 3.7. Large clusters can also store Events in a separate etcd.

API server. API Priority and Fairness, stable since 1.29, classifies and queues requests, so one noisy controller can't starve the rest. When a priority level rejects excess traffic, clients get HTTP 429. Watch apiserver_flowcontrol_rejected_requests_total, and adjust FlowSchemas for the clients that matter.

Scheduler. In large clusters, the scheduler stops looking once it finds enough feasible nodes. The default threshold, percentageOfNodesToScore, goes from 50% at 100 nodes to 10% at 5,000, with a floor of 5%. The docs advise keeping the default in clusters of several hundred nodes or fewer. Above that, a lower value schedules faster, and a higher one places pods better. They advise against going below 10% unless scheduling throughput matters more than placement. The setting is a top-level field of the scheduler configuration:

apiVersion: kubescheduler.config.k8s.io/v1
kind: KubeSchedulerConfiguration
percentageOfNodesToScore: 20

For scale, Kubernetes 1.37 is designed for up to 5,000 nodes, 110 pods per node, 150,000 pods and 300,000 containers.

Kubernetes performance for machine learning and GPU workloads

For machine learning, performance means busy GPUs and jobs that start when their resources are free.

Schedule GPUs. A device plugin exposes GPUs as a resource such as nvidia.com/gpu. You request GPUs in limits, and if you also set requests, the two must be equal. The NVIDIA GPU Operator installs and manages the driver, the device plugin, the container toolkit, GPU node labels and DCGM monitoring. Taint GPU nodes and add a matching toleration to GPU pods:

spec:
  tolerations:
  - key: nvidia.com/gpu
    operator: Exists
    effect: NoSchedule
  containers:
  - name: trainer
    image: registry.example.com/trainer:1.0
    resources:
      limits:
        nvidia.com/gpu: 1

Share GPUs. A normal GPU request gives a pod a whole GPU. Smaller jobs can share one in two ways.

With time-slicing, asking for two GPU replicas doesn't guarantee twice the compute.

Dynamic Resource Allocation. The core of DRA went GA in Kubernetes 1.34, and it is on by default. Pods claim devices by attributes, such as GPU model or memory, through ResourceClaims. In 1.37, DRA extended-resource support is GA too. A pod that asks for an extended resource such as example.com/gpu can be served by a DRA driver with no device plugin. NVIDIA's GPU Operator docs list whole-GPU and existing-MIG allocation through its DRA driver as generally available. Dynamic MIG, consumable shares and MPS are still alpha. The driver's own README says its Helm chart still installs with GPU allocation off by default. Under the GPU Operator, a cluster runs either the DRA driver or the device plugin.

Queue batch jobs. Distributed training needs all its workers at once. Kueue decides when a job waits, starts or is preempted, with quotas and fair sharing between teams. It supports Kubeflow training jobs, Ray, JobSet and plain Jobs, and it can admit a job all-or-nothing. Kubernetes 1.37 also has native gang scheduling in beta. It is off by default, behind the GenericWorkload feature gate.

Train and serve. For training, Kubeflow Trainer offers one TrainJob API across PyTorch, JAX, DeepSpeed and other frameworks. The v1 TFJob and PyTorchJob operators are now legacy. Keep GPUs fed by staging data on fast storage near the nodes. Trainer's distributed data cache streams data straight to GPU nodes. For inference, scale on request rate or queue length. Expect a cold start whenever a model server scales up from zero.

Back up training state. Pods can be recreated from their manifests, so back up what they write. Have training jobs save checkpoints to a persistent volume or object storage, so a failed or preempted job resumes from the last checkpoint. PyTorch Distributed Checkpoint saves and loads a model sharded across many workers. Take CSI volume snapshots of volumes that hold datasets or models, where your driver supports them. Snapshots have been stable since Kubernetes 1.20. Velero backs up cluster resources and persistent volumes together. Keep manifests in Git, and test a restore before you need one.

How to optimize Kubernetes without manual tuning

You can automate most routine tuning, so the cluster keeps adjusting as load changes. The VPA sets requests and can apply them in place. The HPA and KEDA set replica counts. Cluster Autoscaler or Karpenter add and remove nodes. People still choose the SLOs, the scaling metrics, node-level policies and fixes inside the code.

Start in recommendation mode. Run the VPA with updateMode: "Off", and compare its targets with your current requests. Switch to InPlaceOrRecreate, generally available since VPA 1.6, once they look right. That mode resizes pods in place and evicts them only when that fails. The old Auto mode is deprecated. Goldilocks shows VPA recommendations for a whole namespace in one dashboard. Our cost optimization checklist walks through the VPA setup with a manifest.

Automation has limits:

• The VPA and the HPA must not act on the same CPU or memory metric.
• A VPA recommendation can exceed the largest node, which leaves pods unschedulable.
• In-place resize can't change a pod's QoS class or resize Guaranteed pods on nodes with the static CPU or Memory Manager. In InPlaceOrRecreate mode, the VPA evicts and recreates those pods instead.
• No autoscaler fixes a slow query, a lock or a chatty client.

Commercial platforms run the same loop across many clusters. Before you adopt one, check which changes it makes on its own and which it only recommends.

AI SRE agents take on a different job. They investigate alerts across metrics, logs and recent deploys, and can propose a fix. Keep a person approving each change. Our guide to AI SRE compares the tools and shows how to test one.

Kubernetes performance tuning tools

---

If you'd rather not tune all of this alone, our Kubernetes consulting team starts with read-only access and a written list of what to fix first. We can also run your clusters afterwards, with on-call by agreement.

Frequently Asked Questions

Should you set CPU limits in Kubernetes?

Set CPU limits where you need a hard ceiling, such as shared nodes, batch jobs or Guaranteed pods with exclusive cores. For latency-sensitive services, consider a CPU request without a limit. The request still guarantees a weighted share under contention, and the container avoids CFS throttling. Keep memory limits, because without one a container's memory use has no upper bound.

How do you optimize Kubernetes without manual tuning?

Automate the routine parts. Run the Vertical Pod Autoscaler in recommendation mode first, then let it apply requests in place. Use the HPA or KEDA for replica counts and Cluster Autoscaler or Karpenter for nodes. People still set SLOs, scaling metrics, CPU and NUMA policies, and application fixes, which no autoscaler handles.

Why is my Kubernetes cluster slow?

Common causes are CPU throttling from tight limits, memory pressure and evictions, and slow DNS lookups. Saturated volumes, an overloaded API server and slow etcd disks are next. Check throttling ratios, PSI stall metrics, events and etcd disk latency first. If all of those look healthy, profile the application itself to find the slow code path.

Can Kubernetes resize a pod without restarting it?

Yes. In-place pod resize is stable since Kubernetes 1.35. You change a running container's CPU and memory through the pod's resize subresource, and the container's resize policy decides whether it restarts. A resize can't change the pod's QoS class. The Vertical Pod Autoscaler applies recommendations this way in its InPlaceOrRecreate mode.

How do you share a GPU between pods in Kubernetes?

Use NVIDIA Multi-Instance GPU to split a supported data-center GPU, such as an A100 or H100, into up to seven isolated instances. Use time-slicing on GPUs without MIG, or when isolation matters less, since time-sliced pods get no memory or fault isolation. Dynamic Resource Allocation, generally available since Kubernetes 1.34, adds attribute-based requests and more sharing options through vendor drivers.

How do you back up Kubernetes pods that run machine learning workloads?

Pods can be recreated from their manifests, so back up what they write. Save training checkpoints to a persistent volume or object storage, so a preempted job can resume. Protect volumes that hold datasets or models with CSI snapshots, where your driver supports them. Velero backs up cluster resources and volumes together. Test a restore before you need one.

Related Reading

• Kubernetes Autoscaling: HPA, VPA, Karpenter and KEDA
• Kubernetes Cost Optimization: A Practical Checklist
• A Deep Dive into Kubernetes Networking
• Kubernetes Metrics Server: A comprehensive guide