Metrics Reference

The metadata server (meta.flexfs), the proxy server (proxy.flexfs), and the mount client (mount.flexfs) expose Prometheus metrics via an HTTP /metrics endpoint. On the servers the endpoint is always enabled and shares the main bind address; on the mount client it is opt-in (enabled with --metrics) and served on its own port.

Metadata server metrics

Namespace: flexfs_meta

Metric names in the tables below omit this prefix — volume_blocks is flexfs_meta_volume_blocks in full.

RPC metrics

Metric	Type / Labels	Description
rpc_duration_seconds	Histogram · volume_id, method	RPC operation latency in seconds.
rpc_ops_total	Counter · volume_id, method, status	Total RPC operations by method and status.

Histogram buckets (seconds): 0.00001, 0.000025, 0.00005, 0.0001, 0.00025, 0.0005, 0.001, 0.0025, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10

REST metrics

Separate from the WebSocket RPC channel above, the metadata server exposes an HTTP REST API: the versioned v1/v2 admin and reporting endpoints, plus the unversioned /, /status, and /metrics routes. These metrics cover that HTTP surface.

Metric	Type / Labels	Description
rest_duration_seconds	Histogram · method	REST operation latency in seconds.
rest_ops_total	Counter · api_version, method, status	Total REST operations by API version, method and status. api_version is v1, v2, or none for the unversioned routes (/, /status, and /metrics). Use it to see whether anything still calls the deprecated v1 API on a server that has re-enabled it.

Histogram buckets (seconds): 0.0001, 0.00025, 0.0005, 0.001, 0.0025, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10

Block-key and CAS metrics

The op label carries the block-key RPC method: SetBlockKeys, SetBlockKeysAndSize, ListBlockKeys, and the compare-and-swap variants SetBlockKeysCAS / SetBlockKeysAndSizeCAS. The compare-and-swap variants install each block index only if the client’s base key still matches the stored key; a mismatch is reported as a conflict for the client to re-merge and retry. The CAS metrics are recorded only for the two *CAS methods.

Metric	Type / Labels	Description
block_keys_per_call	Histogram · volume_id, op	Number of block keys touched in a single block-key RPC.
cas_conflict_retired_blocks_total	Counter · volume_id	Block objects retired because their compare-and-swap conflicted (uploaded but not installed).
cas_conflicts_per_call	Histogram · volume_id, op	Number of block indices that conflicted in a single compare-and-swap call.
cas_conflicts_total	Counter · volume_id, op	Total block indices that lost a compare-and-swap (base key no longer matched).
cas_installed_total	Counter · volume_id, op	Total block indices installed by a compare-and-swap (base key matched).

block_keys_per_call histogram buckets: 1, 4, 16, 64, 256, 1024, 4096

cas_conflicts_per_call histogram buckets: 0, 1, 2, 4, 8, 16, 64, 256

Volume info metrics

Metric	Type / Labels	Description
volume_info	Gauge · volume_id, volume_name	Volume metadata. Always 1 per volume; use labels to map IDs to names.

Volume gauge metrics

Metric	Type / Labels	Description
volume_blocks	Gauge · volume_id	Number of active blocks.
volume_blocks_retired	Gauge · volume_id	Number of retired blocks.
volume_dentries	Gauge · volume_id	Number of directory entries.
volume_inodes	Gauge · volume_id	Number of inodes.
volume_size_bytes	Gauge · volume_id	Total volume size in bytes.

Volume I/O counters

Metric	Type / Labels	Description
volume_blocks_deleted_total	Counter · volume_id	Total block objects deleted from the block store by the retirement/GC loops.
volume_blocks_read_total	Counter · volume_id	Total blocks fetched from the block store by this volume’s mounts; cache hits are not counted.
volume_blocks_written_total	Counter · volume_id	Total blocks uploaded to the block store by this volume’s mounts.
volume_bytes_read_total	Counter · volume_id	Total bytes read at FUSE boundary.
volume_bytes_written_total	Counter · volume_id	Total bytes written at FUSE boundary.

These four aggregate what every mount of the volume reports, so they carry the same meanings as the mount’s own I/O throughput counters — blocks_* counts remote block operations only, bytes_* counts traffic at the FUSE boundary. Unlike the mount-side counters they advance in steps, on each mount’s 60-second reporting interval, so query them with rate() over a window of a few minutes. They are held in memory, so they restart from zero when the metadata server does; rate() and increase() account for that, but a raw value is only cumulative since the server last started.

Session metrics

Metric	Type / Labels	Description
sessions	Gauge · volume_id	Active client sessions per volume.

Lock metrics

Metric	Type / Labels	Description
append_grants_count	Gauge · volume_id	Append grants currently held (serialized append authority).
lock_sessions_in_memory_count	Gauge · volume_id	Total lock sessions held in memory.
locks_in_memory_count	Gauge · volume_id	Total advisory locks held in memory.

Database folder metrics

Metric	Type / Labels	Description
db_folder_disk_available_bytes	Gauge	Available space on the filesystem containing the database folder. This is the value compared against the configured low-disk floor.
db_folder_disk_capacity_bytes	Gauge	Total capacity of the filesystem containing the database folder.

Database internals

Metric	Type / Labels	Description
db_block_cache_hits_total	Counter	Block cache hits (shared across volumes).
db_block_cache_misses_total	Counter	Block cache misses (shared across volumes).
db_block_cache_size_bytes	Gauge	Block cache size in bytes (shared across volumes).
db_compaction_count	Counter · volume_id	Total compactions.
db_compaction_estimated_debt_bytes	Gauge · volume_id	Estimated compaction debt in bytes.
db_disk_usage_bytes	Gauge · volume_id	Total disk space used.
db_flush_count	Counter · volume_id	Total flushes.
db_memtable_count	Gauge · volume_id	Number of memtables.
db_memtable_size_bytes	Gauge · volume_id	Memtable size in bytes.
db_read_amplification	Gauge · volume_id	Read amplification.
db_table_iters	Gauge · volume_id	Open sstable iterators.
db_wal_bytes_in_total	Counter · volume_id	Logical bytes written to WAL.
db_wal_bytes_written_total	Counter · volume_id	Physical bytes written to WAL.
db_wal_files	Gauge · volume_id	Number of live WAL files.
db_wal_size_bytes	Gauge · volume_id	Size of live WAL data in bytes.
db_write_stall_seconds_total	Counter · volume_id	Cumulative seconds writes have spent stalled, including any stall in progress.
db_write_stalled	Gauge · volume_id	1 while a write stall is in progress, else 0.
db_write_stalls_total	Counter · volume_id, reason	Write stall episodes by reason: L0 file count limit exceeded or memtable count limit reached.

---

Proxy server metrics

Namespace: flexfs_proxy

Metric names in the tables below omit this prefix — cache_clean_blocks is flexfs_proxy_cache_clean_blocks in full.

REST metrics

Metric	Type / Labels	Description
rest_duration_seconds	Histogram · method	REST operation latency in seconds.
rest_ops_total	Counter · method, status	Total REST operations by method and status.

Histogram buckets (seconds): 0.0001, 0.00025, 0.0005, 0.001, 0.0025, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10

Cache metrics

Metric	Type / Labels	Description
cache_clean_blocks	Gauge	Number of clean cached blocks.
cache_clean_bytes	Gauge	Total size of clean cached blocks in bytes.
cache_dirty_blocks	Gauge	Number of dirty blocks pending writeback.
cache_dirty_bytes	Gauge	Total size of dirty blocks in bytes.
cache_disk_capacity_bytes	Gauge	Total capacity of the filesystem containing the cache folder.
cache_disk_quota_bytes	Gauge	Configured disk quota in bytes.

Database folder metrics

Metric	Type / Labels	Description
db_folder_disk_available_bytes	Gauge	Available space on the filesystem containing the database folder. This is the value compared against the configured low-disk floor.
db_folder_disk_capacity_bytes	Gauge	Total capacity of the filesystem containing the database folder.

Database internals

Metric	Type / Labels	Description
db_block_cache_hits_total	Counter	Block cache hits.
db_block_cache_misses_total	Counter	Block cache misses.
db_block_cache_size_bytes	Gauge	Block cache size in bytes.
db_compaction_count	Counter	Total compactions.
db_compaction_estimated_debt_bytes	Gauge	Estimated compaction debt in bytes.
db_disk_usage_bytes	Gauge	Total disk space used by index.
db_flush_count	Counter	Total flushes.
db_memtable_count	Gauge	Number of memtables.
db_memtable_size_bytes	Gauge	Memtable size in bytes.
db_read_amplification	Gauge	Read amplification.
db_table_iters	Gauge	Open sstable iterators.
db_wal_bytes_in_total	Counter	Logical bytes written to WAL.
db_wal_bytes_written_total	Counter	Physical bytes written to WAL.
db_wal_files	Gauge	Number of live WAL files.
db_wal_size_bytes	Gauge	Size of live WAL data in bytes.

---

Mount client metrics

Namespace: flexfs_mount

Metric names in the tables below omit this prefix — dirty_blocks is flexfs_mount_dirty_blocks in full.

Unlike the servers, the mount client only exposes metrics when started with --metrics, on the port set by --metricsPort (default 6074). When disabled, no endpoint is served and instrumentation is skipped. Values that are already tracked as live state (cache sizes, queue depths, byte counters) are read only at scrape time; the rest are recorded on the relevant code paths.

FUSE operation metrics

Metric	Type / Labels	Description
fuse_errors_total	Counter · op, status	Total non-OK FUSE replies to the kernel, by op and status (e.g. EIO, ENOENT, EPERM).
fuse_op_duration_seconds	Histogram · op	FUSE operation handling latency in seconds, by op.
fuse_ops_total	Counter · op	Total FUSE operations dispatched, by op.
open_file_handles	Gauge	Currently open file handles.

fuse_op_duration_seconds histogram buckets (seconds): 0.00001, 0.000025, 0.00005, 0.0001, 0.00025, 0.0005, 0.001, 0.0025, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10

The NotifyReply op

Both op-labeled series above include NotifyReply, which is not driven by the workload — it is the kernel acknowledging the mount’s internal cache-maintenance messages. A healthy idle mount shows a steady low rate of NotifyReply — up to 64/s, and never more. Exclude it when reading flexfs_mount_fuse_ops_total as a measure of workload.

Kernel notification metrics

The mount pushes invalidation/notification ops to the kernel to keep its caches coherent. The op label is one of InvalInode (drop an inode’s cached attributes and data pages), InvalEntry (drop a cached directory-entry name), Retrieve (an internal cache-maintenance probe — see caching), Delete (tell the kernel a directory entry was removed, dropping the cached name and the inode it referred to), or Store.

Retrieve normally dominates the op counter and is not a sign of trouble: it is a non-destructive housekeeping op, and its rate is capped regardless of how many inodes the mount holds.

Metric	Type / Labels	Description
append_barriers_deferred_total	Counter	Append coordination steps that did not complete before their append was answered. Expected to stay at zero. When one occurs, a reader racing that append can briefly see zeros in the just-appended range; re-reading the offset returns correct data within milliseconds.
append_barriers_total	Counter	Coordinated appends that completed their coherence step before being answered. Not an error: a steady rate under an append-and-tail workload is normal, and an appender nobody is reading shows zero.
fuse_notify_errors_total	Counter · op	Cache updates the kernel did not apply, by operation. Updates for a file or name the kernel has already dropped are not counted — that is the expected answer and not a fault. Expected to stay at zero. Any sustained rate means the mount is failing to keep the kernel’s cache in step with what other mounts have changed; check that the mount is still serving and that its connection to the metadata server is up.
fuse_notify_ops_total	Counter · op	Cache updates sent to the kernel, by operation. The denominator for fuse_notify_errors_total.
kernel_dentry_invalidations_dropped_total	Counter	Updates to cached file and directory NAMES that were discarded because the mount was backed up. Benign and needs no action: the affected name refreshes within the entry-cache validity window regardless, so the worst case is briefly seeing a stale name. These are discarded first when the mount falls behind, ahead of updates to file contents and attributes, which go stale for much longer.
kernel_inval_queue_depth	Gauge	Cross-mount cache updates waiting to be sent to the kernel, across the whole mount. Expected to be at or near zero; brief spikes while a large cache flush drains are normal. Sustained depth means the mount is producing updates faster than the kernel is accepting them. Read it with kernel_inval_queue_max_lane_depth, which is the one that shows a stall.
kernel_inval_queue_max_lane_depth	Gauge	The largest backlog on any single delivery queue, and the value to alert on (see Alerting). Work is spread across many queues by file, so one queue stuck on a single file barely moves the total above while showing clearly here. Expected to be at or near zero, with brief spikes during a cache flush. A value that stays up means updates for a share of the mount’s files are held up behind one file; check whether reads from the object store are slow.
kernel_invalidations_coalesced_total	Counter	Cache updates merged into one already waiting for the same file. Work avoided, not lost. A high count is normal and healthy on a mount whose files are changed often from elsewhere.
kernel_invalidations_deferred_total	Counter	Cross-mount cache updates briefly held back because the file was being read at that moment, then retried. Holding one back keeps a single busy file from delaying updates for every other file behind it. Each is retried a limited number of times and then sent regardless, so nothing is discarded here. A low steady rate is normal where several mounts read and write the same files. A rate rising together with flexfs_mount_kernel_inval_queue_max_lane_depth means the mount is falling behind on delivering them.
kernel_invalidations_dropped_total	Counter	Cache updates discarded because the mount was backed up. Expected to stay at zero. Each one leaves this mount serving file contents or attributes that another mount has already changed, until the file is no longer open or the attribute-cache validity window expires.
lock_data_invalidations_deferred_total	Counter	Cross-mount lock acquisitions where the mount’s cache refresh did not finish before the lock was handed to the application. The refresh still happens, so coherence is not lost; there is only a brief window in which the first read under the lock can return this mount’s stale data. File sizes stay correct — only data can be stale. Expected to stay at zero on Linux; not incremented on macOS.
open_attr_invalidations_deferred_total	Counter	Opens where the mount could not refresh the file’s size and attributes before answering, and refreshed them immediately afterwards instead. The refresh still happens, but for a moment the application can see the file as it was: if another mount had just extended it, a read taken in that instant can come back short. A low steady rate is normal and needs no action — it mostly counts files the kernel was no longer caching, where there was nothing stale to correct. A sustained rise above that means the mount is not answering requests properly; check that it is still serving and that its connection to the metadata server is up. See Caching Architecture.

Cache metrics

Metric	Type / Labels	Description
attr_cache_size	Gauge	Number of entries in the attribute cache.
blockkey_cache_size	Gauge	Number of inodes with cached block keys.
cache_events_total	Counter · cache, event	Cache lookups by cache (mem, disk, attr, blockkey, dentry) and outcome (hit, miss). The openrefresh cache is the close-to-open freshness check rather than a lookup: it counts only opens that checked with the metadata server, where hit means the file was unchanged and miss means a peer had changed it and the cached copy was dropped. Its total is the extra metadata load close-to-open consistency costs this mount; see Caching Architecture.
dentry_cache_size	Gauge	Number of entries in the internal (per-mount) dentry cache.
ledger_accounting_faults_total	Counter	Open files the mount was no longer tracking a kernel reference for, counted once per file when the fault is first detected. Expected to stay at zero. A non-zero value means cross-mount invalidation is suppressed for that file for as long as it stays open, so a peer’s changes to it can go unseen — capture the mount log and report it. Nothing repairs the count deliberately — it clears only when a fresh lookup of the name re-establishes the reference, or the file is closed — so read this as a count of affected files, not as a rate that should decay.
ledger_flushes_dropped_total	Counter	Cache flushes for a file with a known-bad lookup count that could not be attempted because the kernel invalidation queue was full, leaving that file’s cached copy unflushed. Indicates sustained invalidation backpressure. A subset of flexfs_mount_kernel_invalidations_dropped_total — do not sum the two, and do not read a zero here as proof that no flush was lost: a flush that folded into an invalidation already queued for the same file, which was then itself dropped, is counted only in that broader metric. It is the one to alert on; this one attributes.
ledger_leaks_reclaimed_total	Counter	Ledger entries reclaimed by the mount’s background housekeeping. Routine — a low rate is normal and needs no action. Useful only for explaining movement in flexfs_mount_ledger_size, which it is directly comparable with.
ledger_size	Gauge	Number of inodes the kernel currently holds a lookup reference to.
ledger_sweep_effective_confirm_ttl_seconds	Gauge	How long the mount’s background housekeeping trusts a residency check before re-checking an inode. Stretched in proportion to flexfs_mount_ledger_size so probing stays paced on large mounts. This explains reclamation latency: a leaked entry is reclaimed within roughly this long of the kernel dropping the inode, and a multi-day value on a very large ledger is by design.
ledger_sweep_entries_examined_total	Counter	Ledger entries walked by the background housekeeping. Diagnostic only. Bounded by a per-pass cap (~8k/s); a rate pinned at the cap just means the housekeeping is pacing itself — the expected steady state on a large quiet ledger.
ledger_sweep_probes_total	Counter	Residency checks attempted by the background housekeeping. Diagnostic only. A rate pinned at 64/s indicates a burst (such as a large listing) or a backlog draining, and should subside.
ledger_sweep_yields_total	Counter	Housekeeping passes skipped because the kernel invalidation queue was busy. Occasional yields are by design; a sustained rate alongside a growing flexfs_mount_ledger_size means the housekeeping is being starved.
stale_attr_installs_rejected_total	Counter · source	Attribute updates discarded for being older than what the mount had already applied for the same file — a delayed peer notification (source="notify") or a metadata reply that was overtaken in flight (source="reply"). Each one is a stale update correctly ignored, keeping file attributes (size included) moving only forward; none is lost work. A steady trickle is normal on a busy multi-mount volume and needs no action. See Caching Architecture.

I/O throughput metrics

Metric	Type / Labels	Description
block_bytes_read_total	Counter · tier	Total block bytes read, by tier (on-store buffer size).
block_bytes_written_total	Counter · tier	Total block bytes written, by tier (on-store buffer size).
blocks_read_total	Counter	Total blocks fetched from the remote block store; cache hits are not counted.
blocks_written_total	Counter	Total blocks uploaded to the remote block store.
fuse_bytes_read_total	Counter	Total bytes read at the FUSE boundary.
fuse_bytes_written_total	Counter	Total bytes written at the FUSE boundary.

All six counters update in real time, as each operation completes. The blocks_* counters count remote block operations: a read served from the mem or disk cache never reaches the block store and so is not counted, which is what makes them worth reading against the mount’s cache_events_total hit rates. They count the same operations as block_bytes_*, so dividing the bytes by the blocks gives the average on-store block size. The block_bytes_* counters record the on-store buffer size (post-compression/encryption) actually transferred, split by the tier label: proxy for traffic served through a proxy group, direct for traffic to the object store backend (including proxy-bypass fallbacks). Both tiers are always present, at zero until traffic flows. Sum over tier for the total, or keep it to attribute amplification to one tier; comparing against fuse_bytes_* quantifies read/write amplification. The tier label uses the same values as block_op_duration_seconds, so throughput and latency can be sliced or joined by tier in a single query.

Block storage metrics

Latency of remote block operations, split by tier (proxy when served through a proxy group, direct when read/written directly against the backend) and op (get, put). The tier label carries the same two values as the throughput counters, so latency and throughput can be sliced or joined by tier in a single query. Only successful operations are timed; failures are counted separately in store_errors_total.

Metric	Type / Labels	Description
block_op_duration_seconds	Histogram · tier, op	Remote block operation latency in seconds.
store_errors_total	Counter · layer, op	Block store errors, by layer (blockstore for synchronous failures, disk for asynchronous disk-cache write-back failures) and op (get, put).

block_op_duration_seconds histogram buckets (seconds): 0.0005, 0.001, 0.0025, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10, 30

Block processing metrics

In-process transform latency. The op label is one of encrypt, decrypt, compress, decompress. Encrypt/decrypt are recorded only when volume encryption is enabled.

Metric	Type / Labels	Description
process_duration_seconds	Histogram · op	In-process block transform latency in seconds.

process_duration_seconds histogram buckets (seconds): 0.00005, 0.0001, 0.00025, 0.0005, 0.001, 0.0025, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5

Write-back and CAS metrics

Metric	Type / Labels	Description
cas_conflicts_total	Counter · op	Block-key compare-and-swap conflicts, by op.
cas_retries_total	Counter · op	Block-key compare-and-swap retries, by op.
dirty_blocks	Gauge	Dirty blocks awaiting write-back.
read_backed_wait_timeouts_total	Counter	Reads that waited for claimed-but-not-yet-arrived bytes (see read_backed_waits_total below) and gave up before they arrived, serving what was there. Expected to stay at zero. Non-zero means a read was served before its bytes arrived, so it can contain zeros for data that exists, and a consumer that trusts those zeros without re-reading can observe a false hole.
read_backed_waits_total	Counter	Reads that reached a region of the file whose size was already known but whose bytes had not yet landed on this mount, and waited for them to arrive. A low steady rate is expected wherever one mount appends to a file it is also reading.
read_short_reply_refreshes_total	Counter	Reads that were answered a second time because the file grew while the first answer was being prepared — this keeps a reader of an actively growing file from being handed stale data. A steady rate while files are being appended to and read at the same time is normal; the retry is served from local memory and adds microseconds.
sync_slots_capacity	Gauge	Configured write-back sync slot capacity.
sync_slots_in_use	Gauge	Write-back sync slots currently in use.

Prefetch metrics

Metric	Type / Labels	Description
prefetch_active	Gauge	Currently active prefetch streams.
prefetch_issued_total	Counter	Read-ahead prefetches dispatched to the block store.
prefetch_skipped_total	Counter · reason	Scheduled prefetches skipped before fetching, by reason (dirty, empty, pending, key_error).
prefetch_slots	Gauge	Configured maximum concurrent prefetch operations.

Metadata connection metrics

Metric	Type / Labels	Description
meta_connected	Gauge	Whether the meta-server connection is up (1) or down (0).
meta_connects_total	Counter	Meta-server connections established, the initial one included. Compare with meta_disconnects_total: the two stay within one of each other, and a gap that persists is an outage the client never recovered from.
meta_disconnects_total	Counter	Meta-server connection-loss events.
meta_inflight_requests	Gauge	In-flight requests to the meta server.
meta_rpc_duration_seconds	Histogram · method	Meta-server RPC round-trip latency in seconds, by method.
meta_rpc_errors_total	Counter · method, status	Meta-server RPC responses with a genuine failure status, by method and status. Expected outcomes (ErrNotFound, ErrConflict, etc.) are not counted as errors.
meta_send_queue_depth	Gauge	Depth of the meta-server send queue.

meta_rpc_duration_seconds histogram buckets (seconds): 0.00005, 0.0001, 0.00025, 0.0005, 0.001, 0.0025, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10

Buffer pool metrics

The pool label identifies which pool the buffers belong to:

pool	Buffers used for
Block	File data moving between the mount and the block store. The busiest of the three, and the one sized by the mount’s block-cache settings.
Fuse	Requests and replies exchanged with the kernel. Sized automatically. Appears once the mount has completed its handshake with the kernel, so it is briefly absent at startup.
Name	File and directory names.

Metric	Type / Labels	Description
buffer_pool_borrowed	Gauge · pool	Buffers currently in use.
buffer_pool_borrowed_total	Counter · pool	Total buffers taken from the pool over the life of the mount.
buffer_pool_created_total	Counter · pool	Total buffers allocated by the pool. Stays far below borrowed_total when the pool is reusing buffers effectively.
buffer_pool_returned_total	Counter · pool	Total buffers returned to the pool.

buffer_pool_borrowed is a current count, not a running total — use buffer_pool_borrowed_total if you want a rate.

A high steady buffer_pool_borrowed{pool="Block"} is normal and is not a leak. The mount’s block caches account for most of it, so it scales with their configured sizes. Judge it by whether it grows without bound over hours, not by its absolute value.

Memory cache metrics

Always present on mount.flexfs. The in-memory (L1) block cache cannot be turned off; leaving its capacity at the default selects an automatic size rather than disabling the cache.

Both metrics are counted in blocks. Multiply by the volume’s block size for an approximate memory footprint.

Metric	Type / Labels	Description
mem_cache_capacity	Gauge	Configured maximum blocks. Constant for the life of the mount.
mem_cache_size	Gauge	Blocks currently held in the in-memory block cache.

This cache is normally the largest single consumer of the mount’s memory, so these two metrics are what let you attribute that memory to a setting you control. To judge whether the in-memory cache needs a larger capacity, read occupancy against capacity alongside the mem hit rate from cache_events_total — neither answers the question on its own:

Occupancy	mem hit rate	Reading
Below capacity	Any	Not under pressure; the working set fits.
At capacity	High	Correctly sized.
At capacity	Poor	The working set does not fit, and every miss costs a fetch from the block store. This is when raising the in-memory cache capacity pays.

# L1 utilization.
flexfs_mount_mem_cache_size / flexfs_mount_mem_cache_capacity

# Approximate resident bytes held by the L1 cache (4 MiB default block size).
flexfs_mount_mem_cache_size * 4 * 1024 * 1024

Disk cache metrics

Present only when a local disk cache is configured (--diskQuota). When none is, these series are absent rather than zero, and their absence is not a fault — it is how a mount running without an L2 cache looks. The same is true of cache_events_total for the disk cache, which reports zero hits and zero misses. Guard any expression that divides by disk_cache_quota_bytes accordingly.

Metric	Type / Labels	Description
disk_cache_clean_bytes	Gauge	Bytes of clean (uploaded) blocks held in the disk cache.
disk_cache_dirty_bytes	Gauge	Bytes of dirty (not-yet-uploaded) blocks held in the disk cache.
disk_cache_quota_bytes	Gauge	Configured disk cache capacity in bytes.

---

Standard Go and process metrics

All three components also expose the default Prometheus Go runtime and process collectors:

• go_* metrics (goroutines, GC stats, memory allocation)
• process_* metrics (CPU time, resident memory, file descriptors, start time)

Do not use process_network_* for per-mount bandwidth

process_network_receive_bytes_total and process_network_transmit_bytes_total measure the whole network namespace rather than just this process, and can count the same traffic more than once. Expect them to read well above what the mount is actually moving.

For a mount’s traffic to and from the block store, use flexfs_mount_block_bytes_read_total and flexfs_mount_block_bytes_written_total instead. Those are measured by the mount itself. Keep the process_network_* counters for checking whether a host’s network is saturated overall.

To see how much the block store is doing per byte the applications actually read or write, compare the block counters against the FUSE counters:

# Bytes fetched from the block store per byte delivered to applications.
# The block counters carry a `tier` label the FUSE counters do not, so sum it
# away first — otherwise the division matches nothing and returns no result.
sum without (tier) (rate(flexfs_mount_block_bytes_read_total[5m]))
  / rate(flexfs_mount_fuse_bytes_read_total[5m])

# The same for writes.
sum without (tier) (rate(flexfs_mount_block_bytes_written_total[5m]))
  / rate(flexfs_mount_fuse_bytes_written_total[5m])

A ratio above 1 is expected, because a cache miss fetches a whole block and a partial write rewrites one. A persistently large ratio alongside small average request sizes suggests the volume’s block size does not suit the workload.

Next steps

• Prometheus setup — scrape configuration
• Grafana dashboards — visualization
• Alerting — alert rules based on these metrics
• Logging and diagnostics — mount and server logs