Dynamic Provisioning (Enterprise)

Note

Dynamic provisioning is an Enterprise-only feature. It requires the admin server (admin.flexfs) and the account token of the account that will own the volumes. Community deployments should use static provisioning.

Dynamic provisioning allows Kubernetes to automatically create a new flexFS volume when a PersistentVolumeClaim is submitted. The CSI controller calls the admin server REST API to create the volume; later, at mount time, the node driver retrieves a volume token and mounts the volume for pods.

Prerequisites

• The flexFS CSI driver is installed
• An Enterprise admin server is running and reachable from the cluster
• You have the account token

Step 1: Create a Secret

The CSI driver needs the admin server address and an account token to create volumes. Store these in a Kubernetes Secret:

apiVersion: v1
kind: Secret
metadata:
  name: flexfs-secret
  namespace: default
stringData:
  adminAddr: <admin-addr>
  token: <account-token>

Replace <admin-addr> with the address of your admin server (for example, admin.example.com:443) and <account-token> with a valid account token.

kubectl apply -f secret.yaml

Caution

The Secret contains sensitive credentials. Restrict access with RBAC and consider using a secrets management solution in production.

Step 2: Create a StorageClass

The StorageClass tells Kubernetes to use the flexFS CSI provisioner and specifies volume creation parameters:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: flexfs-dynamic
provisioner: csi.flexfs.io
allowVolumeExpansion: true
parameters:
  csi.storage.k8s.io/provisioner-secret-namespace: default
  csi.storage.k8s.io/provisioner-secret-name: flexfs-secret
  csi.storage.k8s.io/node-publish-secret-namespace: default
  csi.storage.k8s.io/node-publish-secret-name: flexfs-secret
  csi.storage.k8s.io/controller-expand-secret-namespace: default
  csi.storage.k8s.io/controller-expand-secret-name: flexfs-secret
  provider: aws
  region: <region>

provider and region are not optional decoration: the admin server needs them (or an explicit metaStore + blockStore) to place the volume, and rejects a request carrying neither with a 400. For provider: aws, blockAPI defaults to s3; for gcp, azure, and oci you must also set blockAPI (gcs, azure, or oci respectively) or the request is rejected with a 400. See StorageClass parameters for the full list.

You can add optional parameters to control volume settings at creation time. See the StorageClass parameters reference for all options.

Example with custom parameters

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: flexfs-encrypted
provisioner: csi.flexfs.io
allowVolumeExpansion: true
parameters:
  csi.storage.k8s.io/provisioner-secret-namespace: default
  csi.storage.k8s.io/provisioner-secret-name: flexfs-secret
  csi.storage.k8s.io/node-publish-secret-namespace: default
  csi.storage.k8s.io/node-publish-secret-name: flexfs-secret
  csi.storage.k8s.io/controller-expand-secret-namespace: default
  csi.storage.k8s.io/controller-expand-secret-name: flexfs-secret
  provider: aws
  region: <region>
  blockSize: 4Mi
  compression: "true"
  compressionAlgo: lz4
  encryption: "true"
  maxInodes: "1000000"

Mounting a volume created with encryption: "true" requires its encryption secret, at least 8 characters long. Add it as the secret field of the Secret named by node-publish-secret-name:

stringData:
  adminAddr: <admin-addr>
  token: <account-token>
  secret: <encryption-secret>

Step 3: Create a PersistentVolumeClaim

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: flexfs-dynamic
  namespace: default
spec:
  storageClassName: flexfs-dynamic
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 100Gi

Caution

Request the size you actually want. FlexFS volumes are not pre-allocated — they grow as data is written — but the request is applied to the new volume as an enforced quota, rounded up to a whole block, and writing past it fails with ENOSPC. Raise a quota later by editing the claim (with allowVolumeExpansion: true); it cannot be shrunk. See Storage quotas.

kubectl apply -f dynamic-volume.yaml

Once the PVC is created, the CSI provisioner will:

1. Call CreateVolume on the flexFS CSI controller
2. The controller sends a POST /v1/volumes request to the admin server to create a new volume
3. A PersistentVolume is automatically created and bound to the PVC

Step 4: Use the volume in a pod

apiVersion: v1
kind: Pod
metadata:
  name: my-app
spec:
  containers:
    - name: app
      image: ubuntu:latest
      command: ["sleep", "infinity"]
      volumeMounts:
        - name: data
          mountPath: /data
  volumes:
    - name: data
      persistentVolumeClaim:
        claimName: flexfs-dynamic

Volume lifecycle

One volume per claim. The provisioner names each volume pvc-<PVC UID>, so every claim gets its own flexFS volume. Volumes are never reused across claims: delete a PVC and create another with the same name and you get a new UID, and therefore a new, empty volume. Binding to an existing volume is what static provisioning is for.

Retries adopt rather than duplicate. If provisioning is retried after a lost response, the admin server returns the volume it already created for that name instead of making a second one. If a volume of that name exists but has a smaller quota than the claim asks for, the driver reports ALREADY_EXISTS rather than binding the claim to something too small.

Deletion retires the volume. When the claim is deleted and the reclaim policy is Delete, the driver calls DELETE /v1/volumes/for-name/<volume-name>, which retires the volume: it is marked retired, its volume tokens are removed, and the metadata server then deletes its files and blocks. The volume’s retention does not apply once it is retired, so a deleted claim’s data cannot be recovered or mounted at an earlier point in time. Deleting a claim whose volume is already gone succeeds rather than failing. See Retiring a Volume.

With reclaimPolicy: Retain, the driver is never asked to delete anything: the volume, its tokens, and its data stay until someone retires them with configure.flexfs delete volume.

Storage quotas

The claim’s resources.requests.storage is enforced on the created volume, not merely recorded (if the claim also sets resources.limits.storage, the limit is what the quota is set to, since that is the ceiling the claim says must not be exceeded):

• It is rounded up to a whole block, so a 1 GiB claim on a 4 MiB block size becomes 256 blocks.
• df inside the pod reports the quota, and writing past it fails with ENOSPC.
• The limit counts blocks, not bytes. Each stored block uses one unit of the quota whether it is full or partly filled, and compression does not change the count, so many small files can reach the quota well before their combined size does. df computes its used figure from file sizes, so it can show less usage than the quota enforces.
• maxInodes on the StorageClass caps inodes; it is unlimited unless set, since a claim expresses only bytes.

Growing a volume. Set allowVolumeExpansion: true on the StorageClass and edit the claim’s request. The csi-resizer sidecar calls the driver, which raises the volume’s quota. The new limit is recorded at once, but a mount that is already running picks it up on its next volume-settings refresh — about a minute in practice — so for up to that long an already-full volume keeps returning ENOSPC and df keeps showing the old size. No remount or pod restart is needed; it simply is not instant.

Expansion only ever raises a quota. Kubernetes refuses to shrink a claim below its recorded capacity, so a dynamically provisioned volume’s limit is whatever its claim last asked for.

Caution

Let the claim be the only thing that sets a dynamically provisioned volume’s quota. Changing it directly with configure.flexfs update volume --maxBlocks/--maxInodes leaves the claim advertising a size the volume no longer enforces, and nothing reconciles the two.

One request is refused: expanding a volume that has no quota. A volume with no limit already permits more than any request, so there is nothing to raise, and reporting an unlimited capacity — exabytes — for the claim would be worse than an error. Any volume without a storage quota is in this state, including one bound through a static PV. Set a limit deliberately with configure.flexfs update volume --maxBlocks/--maxInodes if you want one.

A pre-existing volume bound through a static PV is not affected by any of this: its PV capacity is informational, and quotas are set with configure.flexfs update volume --maxBlocks/--maxInodes.

Access modes

FlexFS supports the following CSI access modes:

Access mode	Supported	Description
ReadWriteMany	Yes	Multiple pods on any nodes can read and write simultaneously
ReadWriteOnce	Yes	Read-write from a single node. Any number of pods on that node can use the volume at the same time (see the note below).
ReadWriteOncePod	Yes	Read-write for a single pod in the cluster (see the note below)
ReadOnlyMany	Yes	Multiple pods on any nodes with read-only access

Note

ReadWriteOncePod, and sharing a ReadWriteOnce volume between pods on the same node, require the Kubernetes ReadWriteOncePod feature gate. It is enabled by default on Kubernetes 1.27 and later and is off by default on 1.25 and 1.26. With the gate off, ReadWriteOncePod claims are rejected by Kubernetes, and a ReadWriteOnce volume can be used by only one pod per node: a second pod on the same node stays in ContainerCreating with an already mounted error. Use ReadWriteMany if pods on those versions must share a volume.

Next steps

• StorageClass parameters reference
• Static provisioning for pre-existing volumes
• Troubleshooting