Performance Tuning

This guide covers the key tuning parameters for optimizing flexFS performance across different workloads.

Block size

The block size determines how file data is split into chunks for storage. In Enterprise it is set when the volume is created (configure.flexfs create volume --blockSize) and cannot be changed afterward; it must be a power of 2 between 256 KiB and 8 MiB. Community volumes use a fixed 2 MiB block size.

Block size	Best for	Trade-offs
256KiB	Small files, random access	More metadata overhead, more requests per large file
512KiB	Mixed workloads	Balanced
1MiB	General purpose	Balanced for mixed workloads
2MiB	Large sequential files	Less metadata, fewer requests
4MiB	Large files, HPC, genomics	Enterprise default. Optimal for files > 100 MB
8MiB	Very large sequential files, streaming	Highest throughput for large files, wastes space on small files

Tip

For workloads dominated by large files (genomics BAM/CRAM, AI/ML training data, video), use 4MiB or 8MiB. For workloads with many small files (source code, configuration), use 256KiB or 512KiB.

Disk cache

The on-disk cache provides a second tier of caching that can be much larger than the memory cache. It is per-process and its contents do not survive a restart: at startup the mount client removes the flexFS cache files it finds in its folder, leaving any other files alone, except blocks not yet uploaded (see Disk writeback). The default path (~/.flexfs/mount/cache/<name>/<pid>) is <pid>-scoped so each mount process has its own folder; keep the <pid> component if you override it.

Flag	Description
--diskFolder	Path to the on-disk cache folder
--diskMaxBlockSize	Maximum processed block size that will be cached to disk (e.g. 256K, 1M)
--diskQuota	Maximum disk space to use. Accepts sizes (e.g., 500M, 50G, 1T) or a percentage of the cache filesystem’s total size (e.g., 80%). Disk caching is disabled unless this is set.

See the mount.flexfs CLI reference for types and defaults.

The quota must fit in the space currently free on the cache filesystem. If it does not, the mount logs Warning: invalid disk quota: exceeds available space (disabling block cache) and runs without a disk cache, which also turns off --diskWriteback. A quota below 32 MiB also disables the disk cache. Check the mount’s startup banner for the diskQuota line to confirm the cache is active.

mount.flexfs start <volume-name> /mnt/data \
  --diskFolder '/var/cache/flexfs/<pid>' \
  --diskQuota 100G

Disk writeback

Enable disk-level writeback caching to mask write latency:

mount.flexfs start <volume-name> /mnt/data \
  --diskFolder '/var/cache/flexfs/<pid>' \
  --diskQuota 100G \
  --diskWriteback

Disk writeback requires a disk cache, so --diskQuota must be set — without it there is no disk cache and --diskWriteback has no effect. Blocks larger than --diskMaxBlockSize (default 256K) bypass the disk cache and are uploaded directly, so set it to 0 (no limit) for writeback to cover full blocks. The limit applies to the stored block size after compression and encryption, which can be slightly larger than the volume block size for incompressible data.

With --diskWriteback enabled, writes are acknowledged as soon as the block is written to the local disk cache. The block is then asynchronously uploaded to object storage (or the proxy). This significantly reduces write latency for workloads that can tolerate a short window where data exists only on local disk.

Blocks that are written but not yet uploaded stay on local disk until their upload succeeds, including when the mount process crashes or is killed. A host power loss can lose blocks the operating system had not yet written to the disk, unless --diskSync is set (see below). Each block is checked before it is uploaded: a block whose file a power loss or disk fault left empty, incomplete or otherwise damaged, or whose file is not a regular file with a single link owned by the user the mount runs as, is not uploaded; the mount logs an error naming it, and its file is renamed with a .corrupt suffix for you to delete. Any mount of the same volume on that host with the same --diskFolder setting uploads such blocks, whether or not it enables a disk cache itself: at startup and then every minute it looks for cache folders whose mount process is no longer running, uploads the blocks they hold, and removes them. A running mount locks its own cache folder, so no other mount takes it over while it runs; on a filesystem without file locking the mount logs a warning and relies on the process ID in the folder name instead. Read-only mounts do not search for these folders, and a folder is only recovered if it is owned by the user the mount runs as and is not writable by its group or other users. A folder holding blocks for another volume is kept and named in the mount log; delete it manually if that volume is no longer in use. Other mounts cannot read those blocks until they are uploaded.

A block the local disk fails to write is uploaded directly before its write is acknowledged, and the mount logs a warning at most once a minute. Repeated warnings mean writeback is not taking effect; check the disk holding --diskFolder.

By default, a block is handed to the operating system without waiting for it to reach the disk, so fsync() and close() on the mount do not guarantee that the block survives a host power loss. Add --diskSync to flush each block to the disk before its write is acknowledged:

mount.flexfs start <volume-name> /mnt/data \
  --diskFolder '/var/cache/flexfs/<pid>' \
  --diskQuota 100G \
  --diskWriteback \
  --diskSync

With --diskSync, once fsync() or close() returns, its data survives a host power loss. Every block write the disk cache takes then waits for the disk, which adds write latency and reduces write throughput, most on network-backed disks. A block the disk fails to flush is uploaded directly, as above. If the file system holding --diskFolder cannot flush the cache folder itself, the mount does not start and logs an error naming the folder. --diskSync has no effect without --diskWriteback; the mount logs a warning and ignores it.

Caution

With disk writeback, data written but not yet uploaded to object storage is lost if the local disk fails. Use a reliable local storage medium.

Proxy group placement

For Enterprise deployments using proxy groups:

• Place proxy servers in the same region as the mount clients they serve.
• Mount clients automatically select the lowest-latency proxy group via RTT probing.
• Use multiple proxy servers per group for load distribution (blocks are distributed evenly across the group).

Workload-specific recommendations

Genomics / HPC

mount.flexfs start genomics-vol /mnt/data \
  --diskFolder '/nvme/flexfs-cache/<pid>' \
  --diskQuota 500G \
  --diskMaxBlockSize 0

The prefetch budget and readahead window are both auto-tuned and are deliberately left alone here.

AI/ML training data

mount.flexfs start training-vol /mnt/data \
  --diskFolder '/nvme/flexfs-cache/<pid>' \
  --diskQuota 1T \
  --diskMaxBlockSize 0

General interactive workloads

mount.flexfs start shared-vol /mnt/data \
  --diskFolder '/var/cache/flexfs/<pid>' \
  --diskQuota 80%

Next steps

• Capacity planning
• Logging and diagnostics — server and mount client logs
• Caching architecture — understand the three-tier cache model