Volumes, PersistentVolumes, and Claims

Kubernetes containers are disposable, but many applications still need files. A web server may need a mounted configuration directory, a batch job may need scratch space shared by two containers, and a database needs bytes that survive a Pod restart. Volumes, PersistentVolumes, and Claims are the Kubernetes storage model that separates those needs from the container image.

The practical outcome of this lesson is specific: you should be able to decide when a Pod volume is enough, when a PersistentVolumeClaim is required, how binding works between claims and real storage, and how to express the same choices cleanly in a Helm chart. In this Kubernetes and Helm course, storage is where chart authors move beyond stateless Deployments and start describing durable application behavior.

Purpose and Outcome

A container filesystem is recreated with the container. Kubernetes may restart the container on the same node, replace the Pod on another node, or delete the Pod during an upgrade. Any file written only inside the container layer can disappear. A volume gives containers in a Pod a mounted filesystem path whose lifetime and backing store are defined by the volume type.

Some volumes are tied to the Pod. emptyDir is created when the Pod is assigned to a node and removed when the Pod is removed from that node. Other volumes point at durable storage. Kubernetes models durable storage with a cluster-scoped PersistentVolume, a namespace-scoped PersistentVolumeClaim, and usually a StorageClass that knows how to provision disks through a CSI driver.

How the Storage Mechanism Works

The storage path has three distinct layers. First, the Pod has spec.volumes. This list names mountable storage sources. Second, each container chooses where those named volumes appear with volumeMounts. Third, for persistent storage, the Pod volume usually references a persistentVolumeClaim by name.

A PersistentVolumeClaim, or PVC, is a request for storage. It asks for capacity, access modes, and optionally a storage class. The claim does not normally describe a cloud disk ID or NFS export. Instead, it declares what the workload needs. The control plane then binds that claim to a PersistentVolume, or PV, that satisfies the request.

A PersistentVolume is the cluster object representing the actual storage resource. It can be created manually by an administrator, or dynamically by a provisioner. Modern clusters usually use Container Storage Interface, or CSI, drivers. The external provisioner watches for unbound claims, asks the storage backend to create a volume, then creates a matching PV object and binds it to the claim.

Binding is one-to-one. A PVC binds to exactly one PV, and a PV can be bound to only one claim. The binder compares requested capacity, access modes, storage class, selectors, and volume mode. Capacity matching is minimum-based: a 5Gi claim can bind to a 10Gi PV, but not to a 2Gi PV. Once bound, the claim name becomes the stable reference used by Pods.

Scheduling also matters. Some storage can attach only to specific zones or nodes. With volumeBindingMode: WaitForFirstConsumer on the StorageClass, Kubernetes waits until a Pod using the claim is scheduled before choosing or provisioning storage. That prevents a disk from being created in a zone where the Pod cannot run.

Syntax and Object Anatomy

A Pod volume has a local name and one source. The container mount uses that name and a mount path. The following example uses emptyDir for temporary files shared by one container. It is not durable, but it is simple and fast for cache, scratch, or intermediate output.

apiVersion: v1
kind: Pod
metadata:
  name: scratch-demo
spec:
  containers:
    - name: writer
      image: busybox:1.36
      command: ["sh", "-c", "echo ready > /work/status.txt && sleep 3600"]
      volumeMounts:
        - name: scratch
          mountPath: /work
  volumes:
    - name: scratch
      emptyDir: {}

Expected behavior: after the Pod starts, /work/status.txt exists inside the container. If the container restarts in the same Pod, the file usually remains. If the Pod is deleted and recreated, the emptyDir is empty again because its lifetime was the Pod, not the application release.

A PVC asks for durable storage. The important fields are accessModes, resources.requests.storage, storageClassName, and sometimes volumeMode. Filesystem mode gives the Pod a mounted filesystem. Block mode exposes a raw block device and is used by software that formats or manages the device itself.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: app-data
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: standard
  resources:
    requests:
      storage: 5Gi

Expected behavior: in a cluster with a working default or named standard StorageClass, this claim eventually becomes Bound. A dynamically created PV appears and references the claim. If no matching StorageClass or provisioner exists, the claim stays Pending.

A workload consumes the claim through a Pod volume. The Pod does not mount the PV directly; it mounts the PVC. This indirection is the core contract. Chart authors can keep the same template while each cluster chooses its own provisioner, disk type, and retention policy.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: notes
spec:
  replicas: 1
  selector:
    matchLabels:
      app: notes
  template:
    metadata:
      labels:
        app: notes
    spec:
      containers:
        - name: notes
          image: busybox:1.36
          command: ["sh", "-c", "date >> /data/visits.txt; sleep 3600"]
          volumeMounts:
            - name: data
              mountPath: /data
      volumes:
        - name: data
          persistentVolumeClaim:
            claimName: app-data

Expected behavior: the container appends a timestamp to /data/visits.txt. If the Pod is replaced and the claim remains, the new Pod sees the existing file. Because the access mode is ReadWriteOnce, this design uses one replica. Scaling this Deployment above one may fail or create attach conflicts depending on the storage backend and scheduling.

Static and Dynamic Provisioning

Dynamic provisioning is the common default because the claim drives disk creation. Static provisioning is still useful when storage already exists, when an administrator must control exact backing resources, or when using local disks, NFS exports, or pre-created volumes.

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-manual-nfs
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  storageClassName: manual-nfs
  nfs:
    server: 10.0.0.25
    path: /exports/reports
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: reports-data
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: manual-nfs
  resources:
    requests:
      storage: 5Gi

Expected behavior: the claim can bind to the 10Gi PV because the requested size is smaller, the access mode is compatible, and the storage class matches. The reclaim policy is Retain, so deleting the claim does not delete the underlying NFS data. An administrator must deliberately clean or reuse the retained volume.

Helm Chart Design

Helm does not change Kubernetes storage semantics. It gives you a way to expose storage decisions through values while rendering consistent manifests. A chart should let operators choose whether persistence is enabled, which StorageClass to use, how much storage to request, and whether to use an existing claim.

persistence:
  enabled: true
  existingClaim: ""
  storageClassName: standard
  accessModes:
    - ReadWriteOnce
  size: 5Gi

This values fragment keeps environment-specific storage outside the template. A development cluster might use a small default class, while production might use a replicated SSD-backed class. The chart should not hard-code a provider-specific class unless the chart is intentionally private to one platform.

{{- if and .Values.persistence.enabled (not .Values.persistence.existingClaim) }}
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: {{ include "notes.fullname" . }}
  labels:
    {{- include "notes.labels" . | nindent 4 }}
spec:
  accessModes:
    {{- toYaml .Values.persistence.accessModes | nindent 4 }}
  {{- with .Values.persistence.storageClassName }}
  storageClassName: {{ . | quote }}
  {{- end }}
  resources:
    requests:
      storage: {{ .Values.persistence.size | quote }}
{{- end }}

This Helm template creates a PVC only when persistence is enabled and no existing claim is supplied. The expected rendered output is a normal PVC manifest. The key design choice is ownership: if the chart creates the claim, uninstall behavior and release lifecycle affect it. If the operator supplies existingClaim, the chart consumes storage owned outside the release.

Design Choices and Trade-Offs

Use emptyDir when data is temporary and tied to the Pod. It is appropriate for scratch space, shared files between sidecars, and caches that can be rebuilt. It is not appropriate for user uploads, database files, or queue state that must survive rescheduling.

Use a PVC when data belongs to the application beyond a single Pod instance. The trade-off is operational complexity. You must understand access modes, attach limits, backup policy, reclaim policy, resize behavior, and whether the storage backend supports the workload pattern.

ReadWriteOnce means the volume can be mounted read-write by one node at a time. It does not automatically mean only one Pod can use it, because multiple Pods on the same node may be possible with some drivers. For most chart decisions, treat it as single-writer storage. ReadWriteMany supports multiple nodes writing, but requires a backend such as NFS or a distributed filesystem and often has different latency and consistency characteristics.

Stateful applications usually fit StatefulSet better than Deployment. A StatefulSet can create one PVC per replica using volumeClaimTemplates, preserving stable identity such as data-postgres-0. A Deployment with one shared PVC is acceptable for a single-instance app, but it is a poor fit for replicated databases that need separate disks.

Failure Modes and Troubleshooting

Symptom: a PVC remains Pending. Cause: the StorageClass name is wrong, no default class exists, the provisioner is unavailable, or static PV requirements do not match. Diagnose: run kubectl describe pvc app-data and read the Events section. Then list classes with kubectl get storageclass and check for provisioner pods in the storage driver’s namespace. Correct: set the right storageClassName, install or repair the provisioner, or create a PV whose class, capacity, and access modes satisfy the claim.

Symptom: a Pod is stuck in ContainerCreating with mount or attach errors. Cause: the node cannot attach the disk, the volume is already attached elsewhere, the storage backend is unreachable, or the CSI node plugin is unhealthy. Diagnose: inspect kubectl describe pod, check node events, and review CSI controller and node plugin logs. Correct: reduce replicas for single-writer claims, allow the controller to detach from the old node, fix node plugin health, or move to a storage class that supports the needed access mode.

Symptom: data disappears after uninstalling a Helm release. Cause: the chart-created PVC was deleted, and the PV reclaim policy or dynamic provisioner deleted the backing volume. Diagnose: check Helm manifests, PVC history, PV reclaim policy, and storage backend snapshots. Correct: use backups, snapshots, Retain where appropriate, an externally managed claim, or Helm resource policies only when you have a documented cleanup process.

Symptom: writes fail with permission denied even though the volume is mounted. Cause: the filesystem ownership or security context does not match the container user. Diagnose: exec into the container and run id, ls -ld /data, and a small write test. Correct: set an appropriate securityContext, initialize permissions with an init container when allowed, or choose an image that writes as the expected UID.

Security, Performance, and Reliability

Storage is a security boundary because it persists data after Pods and images change. Do not mount broad host paths into application Pods unless there is a narrow administrative reason. Avoid putting secrets into durable application directories; use Kubernetes Secrets or an external secret system with rotation. For multi-tenant clusters, StorageClasses and admission policies should prevent ordinary workloads from selecting privileged host storage.

Performance depends on the backend. Network filesystems may support multi-writer access but have higher latency. Block storage may be fast for a single writer but limited to one node. Capacity requests are not performance guarantees unless the storage class maps size to IOPS or throughput. Measure the actual workload, including fsync-heavy writes, many small files, and startup time after rescheduling.

Reliability requires backup and restore, not just persistence. A PVC can preserve corrupted data just as reliably as valid data. Stateful Helm releases should document snapshot schedules, restore commands, reclaim policy, and whether uninstalling the release is allowed to delete data.

Hands-On Lab

Prerequisites: a Kubernetes cluster you can safely test in, kubectl configured for that cluster, and either a default StorageClass or permission to create a PVC using an existing class. Use a temporary namespace so cleanup is clear.

  1. Create a namespace with kubectl create namespace storage-lab.
  2. Apply the app-data PVC from this lesson in that namespace. If your cluster does not have a standard class, change only storageClassName to a valid class from kubectl get storageclass.
  3. Verify binding with kubectl -n storage-lab get pvc app-data. The expected phase is Bound. If it remains Pending, describe the claim and resolve the event message before continuing.
  4. Apply the notes Deployment from this lesson in the same namespace.
  5. Wait for the Pod with kubectl -n storage-lab rollout status deployment/notes.
  6. Read the file with kubectl -n storage-lab exec deploy/notes -- cat /data/visits.txt. The expected output is at least one timestamp line.
  7. Delete the Pod with kubectl -n storage-lab delete pod -l app=notes, wait for the replacement, and read the file again. The previous timestamp should still be present because the PVC survived Pod replacement.
  8. Cleanup with kubectl delete namespace storage-lab. Before doing this in a real environment, confirm whether the PV reclaim policy deletes or retains the backing storage.

Assessment Exercises

  1. A chart currently uses emptyDir for uploaded user images. Explain the failure that occurs during Pod replacement and design the minimum PVC-based change needed.
  2. A team wants three replicas of a Deployment to write to one ReadWriteOnce claim. Identify two possible symptoms and propose a safer Kubernetes object or storage mode.
  3. You uninstall a Helm release and later discover the database disk is gone. Which Kubernetes and Helm ownership decisions would you inspect, and what change would prevent a repeat?
  4. A PVC is Pending even though a PV exists with enough capacity. List three other matching fields or conditions that can prevent binding.
  5. Design a values interface for a chart that supports both chart-created PVCs and operator-supplied claims. Explain which option should own data cleanup.

Summary

Volumes attach filesystems to Pods, but their lifetime depends on the volume type. emptyDir is Pod-scoped temporary storage. A PVC is a namespaced request for durable storage. A PV represents the actual backing volume, either statically prepared or dynamically created by a StorageClass and provisioner. Helm should expose storage choices without hiding Kubernetes semantics: claim size, access mode, class, ownership, and existing-claim support all affect correctness. Treat persistent storage as application state with explicit binding, troubleshooting, backup, restore, and cleanup behavior.