POSIX Compliance

FlexFS implements a comprehensive set of POSIX filesystem operations through Linux FUSE (Filesystem in Userspace). Applications interact with flexFS exactly as they would with a local filesystem — no code changes or special APIs are required.

Supported FUSE operations

The mount client (mount.flexfs) implements the following FUSE operations:

Namespace operations

Operation	FUSE Op	Description
Lookup	FUSE_LOOKUP	Resolve a file name within a directory to its inode and attributes
Create	FUSE_CREATE	Create and open a new regular file
MkDir	FUSE_MKDIR	Create a directory
MkNod	FUSE_MKNOD	Create a filesystem node (regular file, device special file, FIFO)
Symlink	FUSE_SYMLINK	Create a symbolic link
Link	FUSE_LINK	Create a hard link to an existing inode
Unlink	FUSE_UNLINK	Remove a file
RmDir	FUSE_RMDIR	Remove a directory
Rename	FUSE_RENAME / FUSE_RENAME2	Rename or move a file or directory

File data operations

Operation	FUSE Op	Description
Open	FUSE_OPEN	Open a file
Read	FUSE_READ	Read data from an open file
Write	FUSE_WRITE	Write data to an open file
Flush	FUSE_FLUSH	Flush file data (called on each close())
Fsync	FUSE_FSYNC	Synchronize file data to storage
Release	FUSE_RELEASE	Close a file handle
Fallocate	FUSE_FALLOCATE	Preallocate or deallocate space for a file
Lseek	FUSE_LSEEK	Seek for data or holes (SEEK_DATA, SEEK_HOLE)
Ioctl	FUSE_IOCTL	Inode-flag ioctls only — the chattr/lsattr interface (see Inode flags). Every other command returns ENOTTY

Directory operations

Operation	FUSE Op	Description
OpenDir	FUSE_OPENDIR	Open a directory for reading
ReadDir	FUSE_READDIR	Read directory entries
ReadDirPlus	FUSE_READDIRPLUS	Read directory entries with pre-fetched attributes
ReleaseDir	FUSE_RELEASEDIR	Close a directory handle
Ioctl	FUSE_IOCTL	Inode-flag ioctls on a directory — chattr/lsattr apply to directories as well as files, where they govern the directory’s entries (see Inode flags)

Attribute operations

Operation	FUSE Op	Description
GetAttr	FUSE_GETATTR	Get file attributes (stat)
SetAttr	FUSE_SETATTR	Set file attributes (chmod, chown, truncate, utimes)
Access	FUSE_ACCESS	Check file access permissions
StatFs	FUSE_STATFS	Get filesystem statistics (df)
ReadLink	FUSE_READLINK	Read the target of a symbolic link

Extended attribute operations

Operation	FUSE Op	Description
GetXattr	FUSE_GETXATTR	Get an extended attribute value
SetXattr	FUSE_SETXATTR	Set an extended attribute
ListXattr	FUSE_LISTXATTR	List all extended attribute names
RemoveXattr	FUSE_REMOVEXATTR	Remove an extended attribute

Locking operations

Operation	FUSE Op	Description
GetLk	FUSE_GETLK	Test whether a lock could be placed
SetLk	FUSE_SETLK	Acquire or release a lock (non-blocking)
SetLkW	FUSE_SETLKW	Acquire or release a lock (blocking, with retry)

Lifecycle operations

Operation	FUSE Op	Description
Init	FUSE_INIT	Initialize the FUSE session and negotiate capabilities
Destroy	FUSE_DESTROY	Tear down the FUSE session
Forget	FUSE_FORGET	Release a cached inode reference
BatchForget	FUSE_BATCH_FORGET	Release multiple cached inode references

File locking

FlexFS supports traditional POSIX record locks (fcntl), open file description locks, and BSD locks (flock). Locks are coordinated through the metadata server, making them effective across all mount clients for a volume.

POSIX locks (fcntl)

POSIX-style byte-range locks are supported via fcntl(F_SETLK), fcntl(F_SETLKW), and fcntl(F_GETLK). These support shared (read) and exclusive (write) locks on arbitrary byte ranges.

Block alignment: When coordinated concurrent I/O is not available (see below), flexFS cannot guarantee atomic operations within a single block across concurrent mounts, so POSIX byte-range locks are aligned to block boundaries. A lock on bytes 100-200 of a file with a 4 MiB block size will effectively lock the entire first block (bytes 0 through 4,194,303). This is a best-effort approach that trades strict byte-range precision for correctness in the distributed case.

Coordinated concurrent I/O (default): When both the mount client and the metadata server support it (negotiated automatically at connection time), byte-range locks are passed through with their exact requested bounds — no block-boundary expansion. This is safe because block writes use a compare-and-swap against the metadata server: a mount that modifies part of a block records the block version it started from, and the server accepts the new version only if that version is still current. If another mount changed the block in the meantime, the writer re-fetches the current block, re-applies only the byte ranges it authored, and retries. Two mounts writing disjoint byte ranges within the same block therefore both succeed, and a lost update is never silently committed. The uncontended write path is unaffected; the re-merge cost is paid only under genuine contention. Older clients or servers that do not support the feature transparently fall back to block-aligned locking.

A re-merge also refreshes the committing mount’s own view. While a mount holds unpublished writes to a block, its reads show its own changes immediately and other mounts’ changes only as of its last synchronization — the documented writer-driven propagation. When its writes are then published and merged with what other mounts wrote in the meantime, the mount also refreshes its local caches for that file, so the newly merged content is what all subsequent reads on that mount return. Readers there do not need to reopen the file or take any other action to see it.

Open file description locks

Open file description locks are taken with fcntl(F_OFD_SETLK), fcntl(F_OFD_SETLKW), and fcntl(F_OFD_GETLK). They are supported and coordinated across mounts on the same terms as traditional record locks, with the same byte-range and block-alignment behavior.

They differ from traditional record locks in what owns the lock. A traditional lock belongs to the process, so two descriptors opened separately by one process do not conflict, and closing any descriptor for the file drops every lock that process held on it. An OFD lock belongs to the open file description instead: two separate opens in one process do conflict, descriptors shared through dup() or fork() share the lock, and the lock is released only when the last descriptor referring to that open file is closed. That makes them the safer choice for multi-threaded programs.

BSD locks (flock)

BSD-style whole-file locks are supported via flock(). These locks apply to the whole file and are keyed to the open file handle.

Blocking locks

SetLkW (the blocking variant of SetLk) retries the lock acquisition until the lock is granted or an error occurs.

Cross-mount coordination

All lock state is stored on the metadata server. When mount client A holds a lock, mount client B will see the conflict via GetLk and will block (or receive EAGAIN) on SetLk.

A lock is released when it is explicitly unlocked, or on close. Which close depends on the lock: a traditional record lock goes when the holding process closes any descriptor for the file, an OFD lock when the last descriptor for that open file is closed, and a BSD lock when its file handle is closed.

If a mount client dies or is cut off while holding locks, its locks are not released immediately. The server marks the client’s lock session orphaned and reclaims it after a 15 minute grace period, checked once a minute — so a lock held by a lost mount can block other mounts for up to about 16 minutes, and a blocking SetLkW waiter elsewhere waits that long. The grace period exists so that a mount which reconnects, including one restarted by an in-place update, keeps its locks instead of losing them to a transient outage.

Cache barriers

A lock coordinates caches as well as access, so the ordinary POSIX idiom — take the lock, read, modify, write, release — is correct across mounts without an explicit fsync.

Releasing a lock publishes first. Before the metadata server is told the lock is going away, the releasing mount commits its pending block keys and file size, so the lock cannot pass to a peer that is only able to read the file as it stood at the last commit. Acquiring a lock discards second: a mount granted a lock drops what it had cached for the file — attributes, its own block-key map, and the kernel’s page cache — so the first read under the lock resolves against the metadata server. This holds for O_DIRECT readers too, and it is why lseek(fd, 0, SEEK_END) under a lock returns the file’s real end rather than the end this mount last saw.

Without these barriers a lock excludes correctly and synchronizes nothing, and the read-modify-write and seek-to-end-and-append idioms silently lose all but one in N of their updates across N mounts. The barriers are not optional; they have no supported off switch.

The cost is paid where the work is. A release with nothing pending issues no extra round trip, so read locks are unaffected; a release that does have data to publish pays for the commit, which is the same cost an explicit fsync in the same place would have. Both are far cheaper than the alternative, and neither applies to locks taken on files that are never written.

Concurrent appends

Files opened with O_APPEND are handled specially. The Linux kernel resolves an append offset from the inode size it has cached, which under close-to-open consistency can be stale on other mounts. To make appends correct across mounts, flexFS re-resolves the append offset in userspace.

When coordinated concurrent I/O is negotiated (the default; see File locking), concurrent appends to the same file are serialized: each O_APPEND write is assigned a distinct region, so concurrent appends never overwrite one another, whether the writers share a mount or not. This costs one additional metadata round trip per append write and affects only files opened O_APPEND; all other writes are unchanged. Contending appenders take turns rather than interleaving, and a waiting append blocks until it can proceed, like a blocking lock.

Readers on the same mount as an active appender are served fresh data on every read, so a follower tailing a live append stream (a log shipper, an ingest agent, tail -f) never reads NUL bytes where appended data belongs. Files opened for reading before appending began keep normal caching; for those, stale NULs are extremely rare but not strictly impossible under many aggressive readers of the same region, so a consumer that must never act on a NUL run should re-read before trusting it, or open with O_DIRECT.

Readers on other mounts are never shown appended bytes ahead of their data: the file size peer mounts observe advances only as appended data is durably committed, so a reader — tail -f included — sees the file grow with real content, never with transient zeros standing in for in-flight appends. If an appending client dies, the size other mounts observe stops advancing until its session is reclaimed (the same grace period that releases its file locks); any region it reserved but never wrote then reads as zeros, permanently.

Exclusive file locks exclude appenders on other mounts. While one mount holds an exclusive lock covering the end of a file, an O_APPEND write from a different mount waits for the unlock rather than failing or landing beneath the locked range, and lseek(SEEK_END) under that lock sees a fully backed end of file. Neither side’s bytes can be silently overwritten. Shared locks, and exclusive locks that do not cover the end of the file, do not interact with appends at all.

Within a single mount that exclusion does not apply, for the same reason it does not on a local filesystem: advisory locks of every kind — record locks, open file description locks, and flock alike — do not exclude O_APPEND writers, so a lock held by one process does not stop another process on the same mount from appending. Where one process appends with O_APPEND while another takes a lock and does lseek(SEEK_END) followed by write, the offset the second process resolved can be stale by the time its write lands, and the two can target the same region — the ordinary race between those two idioms, on any filesystem. When several processes on one mount append to a shared file, have all of them use O_APPEND, which is coordinated and never overwrites.

When coordinated concurrent I/O is unavailable, the mount falls back to resolving the offset from its local (per-mount) view of the size, which is safe for a single writer but does not coordinate appends across mounts.

Concurrent namespace changes

Namespace operations — creating, linking, removing, and renaming names — are decided by the metadata server, once, no matter how many mount clients race the same operation on the same name. Exactly one caller succeeds and every other caller receives the standard POSIX error:

Race	One winner gets	Everyone else gets
open(O_CREAT|O_EXCL)	success	EEXIST
mkdir	success	EEXIST
link	success	EEXIST
unlink	success	ENOENT
rmdir	success	ENOENT

This holds across mounts exactly as it does between two processes on one mount, so the usual coordination idioms are safe: claiming a work item by creating a lock file with O_EXCL, or by being the one process whose unlink of it succeeds.

Two details are worth knowing when several clients clean up the same directory at once:

• A file already removed by another client reports No such file or directory. That is the honest POSIX answer, and standard tools handle it — rm -f suppresses it, and rm -rf keeps going. Scripts that treat any rm failure as fatal should use -f when concurrent cleanup is possible.
• A directory listing can briefly show a name a peer has just removed (see directory-entry coherence); removing it then reports ENOENT. Treat that as “already done”, not as damage.

Renaming over an existing file

rename onto an existing target replaces it atomically, and that atomicity holds for observers on other mounts: the target name always resolves — to the old file until the change reaches that mount, then to the new one. There is no moment when the name reads as missing. This is what makes the standard write-temp-then-rename pattern (used by editors, rsync, and most safe-write libraries) dependable with concurrent readers on other mounts: a reader may briefly still open the previous version, but it never gets ENOENT for a name that exists.

Rename and stored blocks

rename is a metadata-only operation: the old and new entries are updated in a single transaction on the metadata server, and the file’s stored blocks are neither moved nor rewritten. Renaming is therefore fast and independent of file size. The blocks keep the keys they were written under, so object names in the bucket follow the inode that owns the data rather than its current path — worth knowing when inspecting a bucket directly, since it holds no path-shaped layout to browse.

Hard links

FlexFS fully supports hard links via the Link operation. Multiple directory entries can reference the same inode, and the file’s link count (nlink) is maintained by the metadata server. The file’s data blocks are shared across all links; deleting a link decrements the link count, and the data blocks are only freed when the link count reaches zero and no file handles remain open.

Hard link semantics are consistent across mount clients — creating a hard link on one mount client is immediately visible to all other clients.

Symbolic links

Symbolic links are created via the Symlink operation and read via ReadLink. The link target is stored as a metadata field on the inode. When encryption is enabled, the link target is encrypted using AES-256-GCM with a deterministic nonce (SHAKE-256 hash of the target string).

Special files

The MkNod operation supports creating:

• Regular files
• FIFO (named pipe) nodes
• Character and block device special files

The device major/minor numbers and file mode are preserved in the inode metadata.

Inode flags (chattr / lsattr)

FlexFS implements the inode-flag ioctls that chattr(1) and lsattr(1) use, so the two flags below can be set and read exactly as on a local Linux filesystem. No mount option is required — the flags are always available, including on volumes without --xattr.

Flag	chattr	Effect
Immutable	chattr +i	The file cannot be modified, deleted, renamed, hard-linked to, truncated, or have its extended attributes changed. Metadata changes are refused as Linux refuses them: chmod, a chown that names an owner or group, and a utimes that names a timestamp all fail, including a touch to the current time.
Append-only	chattr +a	The file can only be opened for writing with O_APPEND, and cannot be deleted, renamed, hard-linked to, truncated, or have its extended attributes changed. Metadata rules match immutable with one exception: a touch to the current time succeeds, because that is the operation an append-only log needs.

sudo chattr +i /mnt/flexfs/data/important.dat
lsattr /mnt/flexfs/data/important.dat
# ----i--------------- /mnt/flexfs/data/important.dat
sudo chattr -i /mnt/flexfs/data/important.dat

Both the legacy FS_IOC_GETFLAGS/FS_IOC_SETFLAGS pair and the newer FS_IOC_FSGETXATTR/FS_IOC_FSSETXATTR (fsxattr) interface are supported, so the flags work through the kernel’s fileattr path as well as through older chattr builds.

Which operations enforce the flags

Enforcement is not limited to writes. Thirteen operations consult the flags: Open, SetAttr (chmod, chown, truncate, utimes), Fallocate, SetXattr, RemoveXattr, Unlink, RmDir, Rename, Link, Create, MkNod, MkDir, and Symlink. A refused operation returns EPERM.

The flags also apply to directories, where they govern the directory’s entries: an immutable directory rejects new entries and removals alike, while an append-only directory permits new entries but refuses to remove or rename existing ones.

Append-only is not simply a stricter immutable — growing the file is the one thing it exists to permit. So an append-only file accepts fallocate that extends it but rejects a hole punch, whereas an immutable file rejects every fallocate mode.

Attribute changes are the one place where which attribute the request carries decides the answer, so setting a flag does not turn every metadata change into an EPERM:

Attribute change	chattr +i	chattr +a
chmod	EPERM	EPERM
chown naming an owner, a group, or both	EPERM	EPERM
touch -d — or anything naming a timestamp	EPERM	EPERM
truncate / ftruncate	EPERM	EPERM
chown : file — names neither owner nor group	OK	OK
touch — both timestamps to now, neither named	EPERM	OK

Two pairs of rows look similar and are not: only the argument-less form of each call is permitted. A chown that names a uid or a gid is refused on both flags — including one that names the file’s current owner — and so is any touch that supplies a timestamp. chown : file and a touch that names no timestamp are the two specific forms that pass, and neither is the no-op it looks like:

• A bare touch on an append-only file succeeds, and writes both timestamps. That is the operation the flag exists to allow: a log-rotation tool has to be able to stamp a file it is permitted to append to. Setting only one of the two timestamps is not this case — that counts as naming a timestamp, and is refused on both flags.

• chown : file is not a no-op either. It names no owner, but Linux treats it as a request to drop privilege, so on a file it also clears any setuid or setgid bit and removes security.capability. It is permitted precisely so that those bits can still be dropped from a flagged file; refusing it would strand a setuid bit with no way to clear it.

Local Linux filesystems do not all agree on every row here, so treat the table as what flexFS does rather than as a statement about Linux in general.

Limits worth knowing

Caution

Setting a flag requires UID 0 as the mount sees it, and --rootSquash makes that impossible. Linux gates these flags on CAP_LINUX_IMMUTABLE, which FUSE does not forward; flexFS approximates it as UID 0. On a --rootSquash mount the caller’s UID is remapped to --anonUID before the request reaches the filesystem, so even a genuine root caller is refused with EPERM and chattr cannot be used at all. The same holds on an --allSquash mount, which remaps every caller. Reading flags with lsattr is unrestricted and works normally.

• Enforcement is client-side. The flags are stored on the inode and checked by the mount client; the metadata server does not interpret them. They protect a file against callers going through a flexFS mount, which is every ordinary path to the data — but they are not a server-side guarantee, and a caller with direct credentialed access to the metadata server or the object store is not bound by them. Treat them as the accident-and-mistake protection they are on a local filesystem, not as a retention or compliance control.

• Only i and a are implemented. chattr requests that set any other flag are rejected with EOPNOTSUPP rather than silently ignored, so a request never appears to succeed without taking effect. lsattr reports the other positions as unset.

• An already-open file descriptor is not revoked. Setting a flag blocks new write opens; a descriptor opened for writing beforehand can still write() through it. Linux behaves the same way — the immutable check lives in the open path, not in write(). ftruncate on such a descriptor is refused for both flags, which may be stricter than the local filesystem you are comparing against.

• Cross-mount visibility follows attribute caching. A flag set on one mount becomes visible to others on the same terms as any other attribute change: normally within milliseconds via the metadata server’s invalidation notifications, and bounded by the attribute-cache TTL if a notification is missed. A peer mount that has the old attributes cached can briefly still permit an operation the flag now forbids. See Attribute and directory-entry caching.

• --ro mounts refuse to change flags, returning EROFS. Reading them still works.

• The flags are invisible in the xattr namespace. They are persisted in a private extended attribute that getxattr, listxattr, setxattr, and removexattr all filter out, so getfattr -d does not show it and it cannot be forged or cleared by writing an xattr directly — the ioctls are the only interface. On an encrypted volume, an entry that cannot be decrypted is read as both flags set, failing closed rather than silently unprotecting the file.

Extended attributes (xattrs)

Extended attributes are stored as key-value pairs on each inode in the metadata server. They are enabled with the --xattr flag (or implicitly by --acl or --rootSquash).

Operation	Behavior
getxattr	Returns the value for a named attribute
setxattr	Sets or replaces an attribute. Supports XATTR_CREATE (fail if exists) and XATTR_REPLACE (fail if absent) flags
listxattr	Lists all attribute names on an inode
removexattr	Removes a named attribute

When encryption is enabled, both attribute names and values are encrypted. Names use deterministic encryption (for server-side matching); values use random nonces.

Which namespaces are supported

An attribute may only be created in a namespace the filesystem defines, the same set a native Linux filesystem registers:

Namespace	Notes
user.*	Unprivileged. Permitted only on regular files and directories; a sticky directory restricts writes to its owner.
trusted.*	Administrative. Requires UID 0 as the mount sees it, and is invisible to everyone else — a non-privileged read is answered ENODATA rather than EPERM, and listxattr omits these names.
security.*	Stored like any other attribute. Exactly one is interpreted: security.capability, which is dropped when an unprivileged caller modifies the file.
system.posix_acl_access, system.posix_acl_default	The POSIX ACL attributes, by exact name.

setxattr with any other name returns EOPNOTSUPP. Note that system. is not a general-purpose prefix: it is reserved for the two ACL attributes above, so a name like system.foo is refused even though its prefix looks defined.

This matters because the kernel cannot apply the rule for us. A FUSE filesystem is handed every attribute name unfiltered — the FUSE module registers a single catch-all handler — so the namespace policy the VFS applies on behalf of a native filesystem has to be applied by flexFS itself. Without it, a name no native filesystem would accept is stored and then silently dropped by the first cp -a, rsync -X or tar --xattrs that copies the file onto a native filesystem, with no error to indicate the loss.

Note

Only creation is restricted. getxattr, listxattr and removexattr still serve out-of-namespace names, because earlier flexFS releases accepted them and an attribute already written to a volume has to remain reachable — in particular removable. This is deliberately laxer than Linux, which refuses all four operations; the alternative would strand existing data with no way to read or delete it through the filesystem. To audit a volume, getfattr -d lists any such names and setfattr -x removes them.

Access control lists (ACLs)

FlexFS supports POSIX extended ACLs (also known as POSIX.1e draft ACLs), which are stored as extended attributes (system.posix_acl_access and system.posix_acl_default). ACLs are enabled with the --acl flag.

The ACL implementation includes:

• Standard POSIX permission checks (user, group, other) for all operations.
• Extended ACL evaluation when ACL xattrs are present, supporting named user and group entries.
• SUID/SGID handling: SUID/SGID bits influence the effective user and group for permission checks.
• Sticky bit: The sticky bit on directories restricts deletion to the file owner, directory owner, or root.
• Root squashing: When --rootSquash is enabled, operations by uid 0 / gid 0 are remapped to a configurable uid / gid (default: 65534 / 65534, the conventional anonymous identity). Root squashing implies ACL support.
• All squashing: When --allSquash is enabled, operations by every uid / gid are remapped to that same anonymous pair, and the caller’s supplementary groups are discarded with them. All squashing implies root squashing, and with it ACL support.

Where ACLs are enforced

On Linux 4.9 and later the kernel evaluates ACLs itself, and an --acl mount performs like any other — including full attribute and directory-entry caching. On older kernels (in practice RHEL/CentOS 7), and whenever --rootSquash or --allSquash is set, the mount client evaluates every operation instead; that is equally correct but slower, because the kernel’s metadata caches have to be turned off for the checks to be reliable. mount.flexfs picks the mode at mount time and records it in the log as ACL enforcement: kernel or ACL enforcement: client.

If a kernel is new enough but its FUSE module was built without POSIX ACL support, an --acl mount fails to start with an error naming the kernel, rather than starting in a state where ACLs that grant extra access would be silently ignored. See Caching Architecture for the detail.

ACL-related mount options

Flag	Description
--acl	Enable extended ACL support (implies --xattr)
--allSquash	Squash every uid / gid to the anonymous uid / gid (implies --rootSquash)
--anonGID	GID squashed callers are mapped to
--anonUID	UID squashed callers are mapped to
--noExec	Disable execution of files
--noSUID	Disable SUID/SGID special permissions
--rootSquash	Enable root squashing (implies --acl)
--umask	Umask override (octal, with or without a leading 0 — 22 and 0022 mean the same thing; max 0777)

See the mount.flexfs CLI reference for types and defaults.

These flags can be set locally on the mount command line or centrally via volume flags and volume token flags in configure.flexfs.

File I/O behavior

Memory-mapped files

Both read-only (mmap with PROT_READ) and writable shared (MAP_SHARED with PROT_WRITE) memory mappings are supported through the kernel page cache. Pages dirtied through a shared mapping are written back to object storage on msync(), on munmap(), and by normal page-cache writeback; the resulting data is durable and becomes visible to other mounts on the usual close-to-open terms. Because flexFS does not enable the kernel’s FUSE writeback cache, mmap-dirtied pages are flushed as ordinary block writes rather than being coalesced, so a heavy random mmap-write workload is less efficient than the equivalent write() calls — but it is correct.

O_DIRECT alignment

FlexFS imposes no alignment requirement on O_DIRECT I/O. A read or write with a misaligned offset or a length that is not a multiple of the block size succeeds rather than failing with EINVAL. This is permitted — O_DIRECT is not part of POSIX, and open(2) states that its alignment restrictions vary by filesystem and may be absent entirely. The restriction block-backed filesystems enforce comes from handing the user buffer to the block layer for DMA, so the granularity they demand is a property of the underlying device rather than of the filesystem: a block-backed filesystem on a device with 512-byte logical sectors accepts 512-granular offsets and lengths and rejects anything finer with EINVAL, and that threshold moves with the device geometry and the kernel version. FlexFS has no such path — the kernel copies the data through the FUSE channel and flexFS then translates the request into block operations against object storage — so no alignment requirement exists to impose. Alignment is therefore a performance consideration rather than a correctness one: a sub-block write becomes a read-modify-write of the containing block. Aligning O_DIRECT I/O to the st_blksize reported by stat() — which is the volume block size — avoids that cost.

Seeking

Standard seek operations (SEEK_SET, SEEK_CUR, SEEK_END) are handled by the kernel’s FUSE layer. FlexFS implements SEEK_DATA and SEEK_HOLE via the Lseek operation, which queries the metadata server for sparse file information.

Mount options

FlexFS supports standard mount options that affect POSIX behavior:

Flag	Effect
--atTime <RFC3339>	Mount the filesystem at a historical point in time (read-only).
--noAtime	Do not update access time on file opens.
--nonEmpty	Allow mounting over a non-empty directory.
--ro	Read-only mount. All write operations return EROFS. Implies --noAtime.

See the mount.flexfs CLI reference for types and defaults.

Known limitations

Note

These are inherent trade-offs in a distributed FUSE filesystem over object storage, not bugs. They apply to all network FUSE filesystems to varying degrees.

• Byte-range lock granularity (legacy / fallback only): When coordinated concurrent I/O is negotiated — the default for current clients and servers — POSIX byte-range locks use their exact requested bounds and concurrent sub-block writes from different mounts are safe (see File locking). Only when talking to an older client or server that does not support the feature are byte-range locks aligned to block boundaries, so sub-block locking effectively locks the whole block.

• Close-to-open consistency: Data written by one mount client becomes visible to other mount clients after the writing client calls close() or fsync() and the reading client opens the file. Opening the file is what delivers this, so a reader that opens after the writer’s close() has returned always gets the new data — including with O_DIRECT, and no matter how soon after the write it opens. A file a reader already had open when the peer wrote is not covered: those readers catch up on their own, normally within a few milliseconds, and mmap mappings refresh on the same terms. In-progress writes that have not been flushed may not be visible to other clients at all. Checking on open costs one round trip to the metadata server per open(), which workloads that open very large numbers of files may notice. See Caching Architecture.

• Buffered reads are not a snapshot: A read() served from the kernel page cache can return a mixture of old and new content if a writer overwrites the same region at that exact moment. The stored data is never mixed (each write commits atomically, and re-reading returns it intact); the mixing can happen only in what one in-flight read returns, and only under concurrent overwrite of the same bytes. This is kernel page-cache behavior inherent to FUSE filesystems, and it is most pronounced when the reader and the writer share a mount. Readers that need each read call to be all-old-or-all-new should open with O_DIRECT (which reads a consistent copy at a small caching cost) or take a shared lock around the read, which also excludes the writer (see Cache barriers).

• Two ways to read a size, one in-flight append apart: Under continuous appends from another mount, fstat() and lseek(SEEK_END) on the same open file can momentarily disagree by a single in-flight append — both only ever move forward, but they can learn of the newest append at slightly different moments. Under a file lock they agree exactly, and the lock is what the seek-then-write idiom needs to exclude appenders on other mounts (see Concurrent appends). Unlocked observers that only watch a file grow (progress monitors, tail -f) are unaffected in practice: they see every size twice within milliseconds of each other.

• Attribute and directory-entry caching: File attributes are cached by the kernel for a bounded TTL (default: 3600 seconds / 1 hour), and directory-entry (name) lookups — including negative/not-found results — for a much shorter TTL (default: 1 second). The metadata server pushes invalidation notifications when remote clients modify files, so cached attributes are refreshed promptly (typically within milliseconds), and a disrupted notification channel triggers a full kernel attribute/data flush on reconnect; the attribute-cache TTL is only a fallback for a missed notification. Directory-entry caches are invalidated on a best-effort basis: a create, rename, or delete on one mount usually reaches a name another mount has already cached within milliseconds, but no client can know every name the kernel holds, so cross-mount name coherence is still bounded by the directory-entry TTL in the worst case — and unlike attributes, names are not covered by the reconnect flush. (The mount client additionally keeps an internal dentry cache to cut LOOKUP RPCs; that cache is also the record it uses to decide which names are worth asking the kernel to drop — see Caching Architecture.) Both kernel caches are disabled when --acl is enforced by the mount client rather than the kernel — on kernels older than 4.9, or with --rootSquash / --allSquash — because every lookup must then reach the mount client so the ACL check always runs. On current kernels --acl caches exactly as a normal mount does; see POSIX ACL mode.

• d_ino of . and .. in raw readdir: every real directory entry reports its true inode number, but the two synthetic . and .. entries carry a placeholder rather than the serial number stat would return for the same name. POSIX asks for the two to agree. In practice this rarely matters, because for the . and .. entries the raw readdir inode field is advisory: stat is authoritative, and standard tools that need a name’s identity call stat. Prefer stat over the readdir field wherever identity matters.

• statx does not report STATX_ATTR_IMMUTABLE / STATX_ATTR_APPEND: a statx(2) call returns stx_attributes = 0 with the corresponding stx_attributes_mask bits clear, meaning “this filesystem does not report these attributes” — even on a file where lsattr correctly shows +i or +a. This affects every FUSE filesystem rather than flexFS specifically: the kernel’s FUSE statx handler narrows the reply to the basic stats plus creation time and discards the attribute fields entirely, so no userspace filesystem can populate them. Use lsattr, or the FS_IOC_GETFLAGS / FS_IOC_FSGETXATTR ioctls, to read these flags — see Inode flags.