Scaling to 1000+ Mounts

This guide covers tuning recommendations for deployments with hundreds or thousands of concurrent mount clients accessing a single metadata server.

Metadata Server Sizing

The metadata server is the primary scaling bottleneck. Each mount client maintains a persistent RPC session.

Hardware Recommendations

Mount Count	CPU Cores	RAM	Storage
100-500	4-8	16-32 GiB	SSD
500-1000	8-16	32-64 GiB	NVMe SSD
1000+	16+	64+ GiB	NVMe SSD

Memory Tuning

The metadata server’s database memory cache defaults to 50% of system RAM. For high session counts, this is generally appropriate. Monitor the metadata server’s Prometheus metrics for cache hit rates.

Proxy Group Sizing

For large deployments, proxy groups reduce the load on object storage and improve read performance.

Sizing Guidelines

• 2-4 proxy servers per group for up to 500 mount clients.
• 4-8 proxy servers per group for 500-2000 mount clients.
• Blocks are distributed evenly across group members.
• Each proxy server should have fast local SSD storage sized to the active working set.

Mount Client Tuning

Block Cache

For compute nodes with available local disk:

mount.flexfs start <volume-name> /mnt/flexfs \
  --diskFolder '/local-ssd/cache/<pid>' \
  --diskQuota 80% \
  --diskMaxBlockSize 0

This keeps repeated reads from reaching the proxy or object storage. Set --diskMaxBlockSize 0 so full blocks are cached; the limit applies to the stored block size after compression and encryption, which can exceed the volume block size for incompressible data.

Kernel Tuning

Network Tuning

On a metadata server that many mount clients connect to at once (for example, after a restart), make sure the listen accept queue and SYN backlog are large enough. net.core.somaxconn defaults to 4096 on Linux 5.4 and later but to 128 on older kernels; net.ipv4.tcp_max_syn_backlog defaults to a value based on the host’s memory:

sudo sysctl -w net.core.somaxconn=4096
sudo sysctl -w net.ipv4.tcp_max_syn_backlog=4096

Deployment Patterns

Separate Volume Groups

For very large deployments, split workloads across multiple volumes with separate metadata servers. This eliminates the single metadata server as a bottleneck:

• Volume A (team 1): meta-server-1, block-store-1
• Volume B (team 2): meta-server-2, block-store-2

Each volume can use the same or different proxy groups.

fstab-Based Mass Deployment

For deploying mounts across many hosts, use fstab entries:

<volume-name>  /mnt/flexfs  flexfs  _netdev,nofail,nosuid,x-systemd.mount-timeout=660  0 0

Combined with configuration management (Ansible, Puppet, Chef), credential initialization and fstab entries can be rolled out to thousands of hosts.

Kubernetes

The CSI driver automatically manages mount lifecycle in Kubernetes. For large clusters, deploy the CSI node DaemonSet on all worker nodes and use PersistentVolumeClaims for pod access.

Monitoring at Scale

Monitor these key metrics across all metadata servers:

• RPC operations per second: Tracks overall load.
• RPC latency percentiles: Detects degradation.
• Active sessions: Counts connected mount clients.
• Volume size gauges: Capacity planning.

Set up Prometheus alerting for:

• RPC latency p99 exceeding thresholds.
• Session count approaching known limits.
• Metadata server process restarts.