Common Issues

Mount Failures

”mount point is already mounted”

The mount point is already in use by a working mount, so the client will not disturb it.

Solution: Check for existing processes with ps aux | grep mount.flexfs. If the mount is stale rather than working, see the stale mount section below.

”mount point is mounted but not usable”

Something is mounted here but it is not responding, and the client cannot tell whether it is safe to remove.

Solution: Unmount it and mount again:

sudo umount -l /mnt/flexfs
sudo mount /mnt/flexfs

“mount point is not empty and —nonEmpty flag has not been set”

The target directory contains files or subdirectories.

Solution: Either empty the directory or add the --nonEmpty flag (or nonEmpty in fstab options).

“invalid mount point: does not exist”

The mount point directory does not exist.

Solution: Create the directory: mkdir -p /mnt/flexfs.

”invalid admin addr: empty”

The credentials file is missing or does not contain an adminAddr field.

Solution: Run mount.flexfs init creds --adminAddr <addr> to initialize credentials, or verify the credentials file at the path shown in the error.

”invalid volume token: empty” or “invalid volume token: not a UUID”

The volume token is missing or malformed in the credentials file.

Solution: Re-run mount.flexfs init creds with a valid volume token.

ACL mount refuses to start

An --acl mount exits at startup, logging that it requires kernel POSIX ACL support and naming the running kernel.

Cause: the kernel is 4.9 or newer, so flexFS asked it to enforce ACLs, but its FUSE module was compiled without POSIX ACL support (CONFIG_FUSE_FS_POSIX_ACL). This is unusual — mainstream distribution kernels enable it.

flexFS refuses rather than continuing, because at that point the kernel has already taken over basic permission checking and would ignore any ACL entry granting access beyond the standard owner/group/other bits. The mount would look healthy while quietly denying access that getfacl says should be allowed.

Resolution: use a kernel with POSIX ACL support enabled, or mount without --acl if extended ACLs are not required. Verify with:

grep CONFIG_FUSE_FS_POSIX_ACL /boot/config-$(uname -r)

A kernel older than 4.9 is not affected: there flexFS enforces ACLs itself and the mount starts normally.

FUSE Errors

”/dev/fuse: No such file or directory”

The FUSE kernel module is not loaded.

Solution:

sudo modprobe fuse

For persistent loading, add fuse to /etc/modules-load.d/fuse.conf.

”fusermount: command not found”

The FUSE3 userspace tools are not installed.

Solution:

# Debian/Ubuntu
sudo apt install fuse3

# RHEL/CentOS
sudo yum install fuse3

Stale Mounts

”Transport endpoint is not connected”

This indicates a stale FUSE mount where the daemon process has died.

Solution: Mount again. Any of these will clean up the stale mount first and report an error if they cannot:

sudo mount /mnt/flexfs                              # fstab entry
sudo mount.flexfs start <name> /mnt/flexfs          # direct

If that reports a failure, unmount by hand and retry:

sudo umount -l /mnt/flexfs

Mount appears in /proc/mounts but is not responding

Solution: Force unmount and remount:

sudo fusermount -u /mnt/flexfs
sudo mount.flexfs start <name> /mnt/flexfs

Or use update.flexfs --force to perform a full hard reset of all mounts.

SSL/TLS Errors

”x509: certificate signed by unknown authority”

This occurs when connecting to a server using a self-signed certificate and the client does not trust it.

Tip

FlexFS components are configured to accept self-signed certificates by default for server-to-server communication. This error most commonly appears when using external tools (e.g., curl) to access flexFS endpoints.

Solution: Use curl -k or configure the system trust store. For flexFS components, SSL is handled automatically via auto-generated certificates.

”tls: failed to verify certificate”

Similar to the above. If connecting configure.flexfs to an admin server with a custom certificate, ensure the certificate is valid or use the internal --noAdminSSL flag for testing.

Credential Errors

”creds file already exists” during init

Solution: Use --force to overwrite: mount.flexfs init creds --force --adminAddr <addr>

Credentials file not found

Solution: Run the appropriate init creds subcommand for the component. The default credential paths are:

• Mount: ~/.flexfs/mount/creds/<volume-name>
• Meta: ~/.flexfs/meta/creds
• Admin: ~/.flexfs/admin/creds
• Proxy: ~/.flexfs/proxy/creds
• Configure: ~/.flexfs/configure/creds

Connectivity Issues

Admin server unreachable

Mount clients and metadata servers require connectivity to the admin server.

Solution:

1. Verify the admin server is running: manage.flexfs status admin
2. Check network connectivity: curl -k https://<admin-addr>/v1/status
3. Verify firewall rules allow traffic on port 443 (or the configured --bindAddr port)

Proxy unreachable

When proxy groups are configured but unreachable, mount clients fall back to direct object storage access. This is normal behavior, not an error.

Solution: If you expect proxy acceleration, verify:

1. The proxy server is running: manage.flexfs status proxy
2. Network connectivity between mount client and proxy
3. The proxy group is associated with the volume in the admin server

Permission Errors

”Operation not permitted” on a file you own, even as root

Ordinary permission failures report Permission denied (EACCES). Operation not permitted (EPERM) on a write, delete, rename, chmod, or chown that root should be allowed to perform usually means the inode carries the immutable or append-only flag.

Diagnosis: lsattr <path> — an i or a in the flag field is the cause. Use lsattr -d <dir> for a directory, since a bare lsattr lists its entries instead. An immutable directory also refuses new entries, so this can surface as a failure to create a file rather than to change one.

Solution: sudo chattr -i <path> or sudo chattr -a <path> to clear it.

”chattr: Operation not permitted while setting flags”

Setting these flags requires root as the mount sees it. On a --rootSquash mount every caller’s UID is remapped before the request reaches the filesystem, so chattr cannot be used at all there — including by real root. Reading flags with lsattr is unaffected. See Inode flags.

”chattr: Operation not supported while setting flags”

Only immutable (+i) and append-only (+a) are implemented. Any other flag is rejected outright rather than silently ignored, so a request never appears to succeed without taking effect.

Performance Issues

Slow sequential reads

Solution: Ensure block prefetching is not disabled. Check that the metadata server is responsive (monitor via /metrics). Consider increasing the block size for workloads with large sequential reads.

High write latency

Solution: For remote object storage, enable local disk writeback caching with --diskWriteback on the mount client. For Enterprise deployments, deploy a proxy group in the same region as the compute.

Service Management

Services not starting after reboot

Solution: Ensure systemd units are enabled:

sudo systemctl enable flexfs-admin.service
sudo systemctl enable flexfs-meta.service

Or use manage.flexfs start to manually start all services.

”this command must be run as root”

Most manage.flexfs and update.flexfs operations require root privileges.

Solution: Use sudo or run as root.