Metadata REST API (v2)

Caution

This page documents an internal API. These endpoints are used by CLI utilities (analyze.flexfs, find.flexfs, dedup.flexfs). They are not intended for direct external use.

Every endpoint except /status and /metrics requires a bearer token. See Authentication for which token each one takes.

The metadata server exposes a REST API alongside its RPC protocol. The REST endpoints serve utilities and monitoring. All query parameters use kebab-case (e.g., volume-id, base-path). Each endpoint responds only to the HTTP method(s) shown below; a request to a valid path with an unsupported method receives 405 Method Not Allowed.

Versioning

This page documents version 2, which is served only at /v2/.... A bare path (no version slug) means version 1, which is deprecated and served only when the metadata server is started with the deprecated v1 routes enabled, so pin /v2/... explicitly in scripts. The /status and /metrics endpoints carry no version slug and are always served.

Authentication

Each endpoint takes one of two tokens in an Authorization: Bearer header.

Endpoints	Token
/find, /find-old, /analyze/files, /analyze/folders, /analyze/users, /duplicates	A volume token carrying the admin flag
/locks, /sessions, /size-bins, /verify, /missing-objects, and the maintenance endpoints	The metadata server token — the UUID configured with meta.flexfs init creds
/status, /metrics	None

Reporting queries

The endpoints that list a volume’s contents return file and directory names together with their owners and permissions, so they are restricted to volume tokens granted the admin flag. Grant it with configure.flexfs:

configure.flexfs create volume-token --volumeID <volume-name> --flags admin --notes "reporting"

The flag is set on a volume token, not on a volume, and it has no effect when the same token is used to mount. See Volume token flags.

find.flexfs, analyze.flexfs and dedup.flexfs read this token from their own credentials files; see find.flexfs.

The token identifies the volume, so these endpoints do not take a vid / volume-id parameter; a request that supplies one is answered 400.

Where the token is scoped to a subdirectory with a mount path, that subdirectory is the query’s root: base-path resolves beneath it, results are reported relative to it, and the query cannot reach anything above it. Such a token reports on exactly the subtree it would mount.

curl -k -H "Authorization: Bearer <volume-token>" \
  "https://<meta-addr>/v2/find?fields=path,size,perm"

Administrative endpoints

The endpoints that report on the server rather than on a volume’s contents take the metadata server’s own token, as the maintenance endpoints do. These still name a volume with vid / volume-id.

curl -k -H "Authorization: Bearer <meta-token>" \
  "https://<meta-addr>/v2/sessions"

Common Query Parameters

These parameters are accepted by most endpoints:

Parameter	Type	Description
vid / volume-id	string	Volume UUID or name, on the administrative and maintenance endpoints. The two spellings are equivalent, and either accepts a UUID or a volume name; a name is resolved to its UUID before the request runs, and either form is checked against the volumes this server has open — an identifier matching none of them, including a well-formed UUID, is rejected with 400. Required by /locks; optional on the others, where omitting it covers every open volume. Not accepted on the reporting queries, whose volume comes from the token.
base-path	string	Base32-encoded path to scope the query to, relative to the token’s mount path
mount-path	string	The client’s mount point (plain text, e.g. /mnt/flexfs). Used to interpret symbolic links whose targets are absolute paths — see below.
at-time	uint64	Point-in-time in nanoseconds since epoch (for time-travel mounts)
pretty	bool	On JSON endpoints, pretty=true indents the response (default is compact). No effect on text endpoints.

Error responses

A failed request carries its reason as a plain-text body, so a client can show it rather than just a status line:

• 400 — the caller’s request is at fault: an unparsable or out-of-range query parameter, a base-path that does not resolve, a backup dest that is or contains the db folder, or a volume identifier matching no open volume. The body names the cause, e.g. validation error: invalid limit parameter: must be between 1 and 1000 or validation error: no such volume: "archive". Query parameters an endpoint does not recognize are ignored.
• 401 — the request carried no bearer token, or one the server does not accept: an unknown or revoked volume token, or a value that is not the metadata server’s token. The body says which.
• 403 — the volume token is valid but does not carry the admin flag, or an API access rule refused the caller’s source address before any handler ran. Access rules apply only on a server whose --accessFile exists; a server with no access file, which is the default, admits every address. Rules name an endpoint by its path template with the version slug removed — /locks for /v2/locks. See API Access Control.
• 500 — the server failed, and the reason is written to the metadata server’s log. The body is empty, except from /backup, /compact, /scrub, /recount/all, /recount/blocks-retired, /missing-objects and DELETE /locks, which return the error text.
• 502 — the admin server was reached but refused this metadata server’s own request to validate the volume token — for example an access rule blocking this server’s address, or a route it does not serve — so the token’s validity was never determined. The token is not at fault; the fix is on the admin server (check its access rules and version), not the caller’s credential. The body says so.
• 503 — a server this metadata server depends on was unavailable, or the request was for a volume that has stopped accepting changes after a metadata read error: a find, find-old, analyze, duplicates, size bins or missing objects request, a request that needed to change it, or a verify request that hit a read error. Either the admin server could not be reached to validate a volume token, so whether it is valid is unknown, or a query that reports or filters on cost could not obtain the billing rates, in which case the body begins cost unavailable; both set a Retry-After: 5 header. For a volume that has stopped accepting changes, the body begins volume unavailable after a metadata read error and a Retry-After: 60 header is set; a find or find-old response already under way is cut off instead. A query, verify or missing objects request still running when the metadata server shuts down gets the body metadata server is shutting down and a Retry-After: 30 header; a response already under way is cut off instead, so the client sees an incomplete transfer rather than a complete one. A blocking backup, compaction, recount or scrub request still running at shutdown may get the same response or have its connection closed. Retry.

Symbolic links in base-path

Endpoints that accept base-path (/find, /duplicates, and the three /analyze/* endpoints) follow symbolic links found among its components, including the final one, as long as the link target stays within the volume. Links with relative targets (such as ../data) resolve on their own. Links with absolute targets (such as /mnt/flexfs/data) are written in the client’s namespace, so the server needs the mount-path parameter to translate them; without it, or when the target falls outside the mount, the request fails with 400 Bad Request and a message naming the link. The CLI utilities supply mount-path automatically when run from inside a mount.

Endpoints

Analyze Files

GET /v2/analyze/files
Authorization: Bearer <volume-token>

Returns the top files by size, size_bin, or cost within a volume or subtree.

Parameter	Type	Default	Description
base-path	string		Base32-encoded path prefix
mount-path	string		Client mount point, for resolving absolute symbolic-link targets
at-time	uint64		Point-in-time (nanoseconds since epoch)
header	bool	false	Include column headers
limit	uint32	100	Maximum results (1-1000)
order-by	string	size	Sort field: size, size_bin, or cost

Response: 200 OK with tab-separated text, one file per line.

Analyze Folders

GET /v2/analyze/folders
Authorization: Bearer <volume-token>

Returns per-directory recursive size and cost.

Parameter	Type	Default	Description
base-path	string		Base32-encoded path prefix
mount-path	string		Client mount point, for resolving absolute symbolic-link targets
at-time	uint64		Point-in-time (nanoseconds since epoch)
header	bool	false	Include column headers
max-depth	uint32	unlimited	Maximum depth; 0 returns the root row only

Response: 200 OK with tab-separated text.

Analyze Users

GET /v2/analyze/users
Authorization: Bearer <volume-token>

Returns per-user size and cost.

Parameter	Type	Default	Description
base-path	string		Base32-encoded path prefix
mount-path	string		Client mount point, for resolving absolute symbolic-link targets
at-time	uint64		Point-in-time (nanoseconds since epoch)
header	bool	false	Include column headers
limit	uint32	100	Maximum results (1-1000)
order-by	string	cost	Sort field: size or cost

Response: 200 OK with tab-separated text (UID, size, cost).

Find

GET /v2/find
Authorization: Bearer <volume-token>

Searches the filesystem with filters on inode attributes.

Parameter	Type	Description
base-path	string	Base32-encoded path prefix
mount-path	string	Client mount point, for resolving absolute symbolic-link targets
at-time	uint64	Point-in-time (nanoseconds since epoch)
fields	string	Comma-separated output fields, written in the order given: atime, blocks, btime, cost, ctime, gid, ino, mode, mtime, nlink, path, perm, size, size_bin, type, uid, xattrs. Default path.
header	bool	Include column headers
name	string	Dentry name filter (supports glob)
type	string	Inode type: b, c, d, f, l, p, s
uid	uint32	UID filter
gid	uint32	GID filter
ino	uint64	Inode number filter (must be non-zero)
limit	uint64	Maximum number of results (0 or omitted = unlimited)
empty	bool	true matches directories with no entries and regular files with no allocated blocks (including sparse files that have a size but no data); false matches the opposite. Other file types never match. With at-time, directories are judged as of that time.
sparse	bool	Sparse file filter (false matches non-sparse files). Regular files only.
perm	string	Required permission bits (octal, with or without a leading 0 — 755 and 0755 mean the same thing; max 07777). Matches when (mode & perm-mask) == perm; bits outside perm-mask are rejected.
perm-mask	string	Bits of the file mode to test (octal, with or without a leading 0; default 0777, max 07777, must be non-zero)
min-size, max-size	uint64	Byte size range (regular files only)
min-blocks, max-blocks	uint64	Block count range (regular files only)
min-cost, max-cost	float64	Monthly cost range (regular files only)
min-nlink, max-nlink	uint32	Hard link count range
min-depth, max-depth	uint32	Depth range
min-atime, max-atime	uint64	Access time range (seconds since epoch)
min-btime, max-btime	uint64	Birth time range (seconds since epoch)
min-ctime, max-ctime	uint64	Change time range (seconds since epoch)
min-mtime, max-mtime	uint64	Modification time range (seconds since epoch)
min-size-bin, max-size-bin	uint32	Size bin range (0-127, regular files only)

Response: 200 OK with tab-separated text.

Find Old

GET /v2/find-old
Authorization: Bearer <volume-token>

Lists non-directory files whose access time is older than a cutoff — useful for identifying stale data. The Find endpoint is more capable and covers the same need via its max-atime filter.

Parameter	Type	Default	Description
co	int64	2 years ago	Cutoff access time, in seconds since epoch. Files with atime older than this are returned.
lim	uint64	100000	Maximum number of results (0 = unlimited)

Response: 200 OK with one line per file in the form size:atime:btime:base32path, where size is the effective size (min(size, blksize × blocks)) and atime/btime are seconds since epoch.

Duplicates

GET /v2/duplicates
Authorization: Bearer <volume-token>

Returns groups of regular files that share the same size and block count, which are candidates for deduplication. Files with no allocated blocks are never included, and a group needs at least two distinct inodes. Every Find filter is accepted except type; the parameters below are the ones the dedup.flexfs utility uses.

Parameter	Type	Description
base-path	string	Base32-encoded path prefix
mount-path	string	Client mount point, for resolving absolute symbolic-link targets
at-time	uint64	Point-in-time (nanoseconds since epoch)
limit	uint64	Maximum number of duplicate groups (0 or omitted = unlimited). When set, the largest groups by file size are retained.
min-size, max-size	uint64	Byte size range
min-blocks, max-blocks	uint64	Block count range

Response: 200 OK with a JSON array of group objects, ordered by size descending then blocks descending. Each group has size (uint64), blocks (uint64), and files (array of {path, ino, nlink, btime} objects). Multiple paths for the same inode (hard links) are included as separate entries. When limit is set, only the largest groups are returned.

Locks

GET /v2/locks
Authorization: Bearer <meta-token>

Lists the volume’s currently held file locks.

Parameter	Type	Description
vid / volume-id	string	Volume UUID or name (required)

Response: 200 OK with a JSON array of active lock objects (empty array if the volume holds no locks).

Sessions

GET /v2/sessions
Authorization: Bearer <meta-token>

Returns the client sessions currently connected to the server.

Parameter	Type	Description
vid / volume-id	string	Volume UUID or name. Omit to report every volume.

Response: 200 OK — a JSON object keyed by volume ID; each value is that volume’s array of session objects. A specified volume yields just that one, with an empty array if it has no clients; omitting it yields every volume that has at least one client. Each session object:

Field	Type	Description
id	string	Session UUID
remote_addr	string	Client remote address
local_addr	string	The client’s own address, as the client reports it
build_version	string	Client build version
proto_version	string	Negotiated protocol version
command	string	Client command line
kernel	string	Client kernel version
cpus	uint32	Client CPU count
ram	uint64	Client RAM in bytes
uid	uint32	Client user ID
gid	uint32	Client group ID
started	string	Session start time
blocks_read	uint64	Blocks read this reporting period
blocks_written	uint64	Blocks written this reporting period
bytes_read	uint64	Bytes read this reporting period
bytes_written	uint64	Bytes written this reporting period

Fields with zero or empty values are omitted. The read/write counters reflect the current reporting period and reset periodically.

Size Bins

GET /v2/size-bins
Authorization: Bearer <meta-token>

Returns the volume’s billed storage distributed across activity-based size bins. Each bin is a 30-day age tier reflecting how recently the files’ data was used: bin 0 holds data active within the last 30 days, bin 1 data 30–60 days old, and so on up to bin 127.

Parameter	Type	Description
vid / volume-id	string	Volume UUID or name. Omit to report every open volume.

Response: 200 OK — a JSON object keyed by volume ID; each value maps size-bin index (0–127) to the total billed size, in bytes, of the files whose data falls in that tier (sparse files counted at their allocated size). A specified volume yields just that one; omitting it yields all open volumes except any that have stopped accepting changes after a metadata read error.

Missing Objects

GET /v2/missing-objects
Authorization: Bearer <meta-token>

Scans block keys and reports any whose backing object is absent from the block store. Each key is verified with a HEAD against the block store.

Parameter	Type	Default	Description
vid / volume-id	string		Volume UUID or name. Omit to scan every open volume.
live-only	bool	false	Restrict the scan to live block keys
retired-only	bool	false	Restrict the scan to retired block keys (mutually exclusive with live-only)
concurrency	int	16	Number of parallel HEAD requests. The default matches the metadata server’s block-store concurrency ceiling; higher values queue on that limit rather than adding parallelism.

Response: 200 OK — a JSON object keyed by volume ID; each value is that volume’s array of entries (empty if none). A specified volume yields just that one; omitting it yields all open volumes. Each entry has kind (missing or error), ino, idx, retired, key, and error (error entries only). In all-volumes mode, a volume whose scan fails (e.g. one already running) is reported with a single error entry (with ino and idx of 0 and no key) rather than failing the request.

Status	Meaning
200	Scan ran; results returned keyed by volume ID.
400	An invalid parameter: unresolvable volume, live-only and retired-only both set, or a malformed concurrency/boolean value.
401	Missing or invalid bearer token.
409	A single-volume scan was requested and one is already in progress for it.
500	A single-volume scan failed.
503	A single-volume scan was requested for a volume that has stopped accepting changes after a metadata read error, with a Retry-After: 60 header, or the server is shutting down, with a Retry-After: 30 header.

Verify

GET /v2/verify
Authorization: Bearer <meta-token>

Runs a read-only integrity check of a volume’s metadata (attr, block-key, and dentry scans plus counter comparisons). This endpoint never modifies data; use the offline CLI (meta.flexfs verify --fix) to repair.

Parameter	Type	Description
vid / volume-id	string	Volume UUID or name. Omit to verify all open volumes.

Response: 200 OK — a JSON object keyed by volume ID; each value is that volume’s VerifyReport (per-scan counts plus an overall ok boolean). A specified volume yields just that one; omitting it yields all open volumes. In all-volumes mode, a volume whose check fails, such as one whose metadata cannot be read, is reported with ok set to false and the reason in error rather than failing the request.

Metrics

GET /metrics

Returns Prometheus-format metrics for the metadata server.

Status

GET /status

Returns server health status. Unversioned: it is served whichever API versions this server serves, and there is no /v2/status.

Maintenance

Note

The maintenance endpoints (POST /v2/backup, POST /v2/compact, POST /v2/scrub, POST /v2/reconcile, POST /v2/recount/all, POST /v2/recount/blocks-retired, and DELETE /v2/locks) change server state and take the metadata server’s bearer token, as the administrative endpoints above do. <meta-token> is the UUID configured with meta.flexfs init creds.

Note

The long-running maintenance endpoints (POST /v2/backup, POST /v2/compact, POST /v2/scrub, POST /v2/reconcile, POST /v2/recount/all, POST /v2/recount/blocks-retired) share a blocking query flag and lock per volume: runs on different volumes proceed concurrently. A run conflicts only with another run of the same operation on the same volume — and an all-volumes run (no vid) conflicts with every run of that operation. The two recount endpoints count as one operation.

• Default (omitted) — runs in the background: 202 Accepted right away, 409 Conflict if a conflicting run is already in progress, or 503 Service Unavailable if the server is shutting down. Failures surface only in the server log.
• blocking=true — waits for any conflicting run to finish, then runs and returns 200 OK with a summary (never 409). Use it (and check the status) when a script must confirm success — e.g. an automated backup.

Backup

POST /v2/backup
Authorization: Bearer <meta-token>

Writes a consistent, point-in-time checkpoint of one or all volumes into a staging directory while the server keeps running. The staging directory is safe to rsync offsite. See Maintenance → Online backup for the recommended workflow.

<meta-token> is the metadata server’s token (the UUID configured with meta.flexfs init creds).

Parameter	Type	Default	Description
dest	string	<dbFolder>/.backup	Staging directory for the checkpoint. Must not be or contain the db folder. Place it on the same filesystem as --dbFolder so database files are hard-linked; on another filesystem they are copied, which takes longer and uses the full space.
vid / volume-id	string	""	A single volume UUID or name. Omit to checkpoint all open volumes.
blocking	bool	false	Wait for the checkpoint and return a summary. See the maintenance note above.

Response: with blocking=true, 200 OK and body checkpointed N volume(s) (the count distinguishes a real backup from a no-op, e.g. zero open volumes). Otherwise 202 Accepted, no body.

Status	Meaning
200	(blocking=true) Checkpoint completed; body reports the number of volumes written.
202	Checkpoint started in the background.
400	The vid / volume-id value could not be resolved to a volume, or dest is or contains the db folder.
401	Missing or invalid bearer token.
409	(async only) A conflicting checkpoint is already in progress.
500	Checkpoint failed.
503	The server is shutting down. A blocking backup running when the server shuts down may return this status with a Retry-After: 30 header or have its connection closed.

Example — an automated backup that must fail loudly waits for completion (-f fails on non-2xx; self-signed cert, so -k):

curl -fk -X POST \
  -H "Authorization: Bearer <meta-token>" \
  "https://<meta-addr>/v2/backup?blocking=true"

Compact

POST /v2/compact
Authorization: Bearer <meta-token>

Triggers a manual compaction of the metadata database for one or all volumes.

Parameter	Type	Default	Description
vid / volume-id	string		A single volume UUID or name. Omit to compact all open volumes.
blocking	bool	false	Wait for the compaction and return a summary. See the maintenance note above.

Response: with blocking=true, 200 OK and body compacted N volume(s). Otherwise 202 Accepted, no body.

Status	Meaning
200	(blocking=true) Compaction completed; body reports the number of volumes compacted.
202	Compaction started in the background.
400	The vid / volume-id value could not be resolved to a volume.
401	Missing or invalid bearer token.
409	(async only) A conflicting compaction is already in progress.
500	(blocking=true) Compaction failed.
503	The server is shutting down. A blocking compaction running when the server shuts down may return this status with a Retry-After: 30 header or have its connection closed.

Scrub

POST /v2/scrub
Authorization: Bearer <meta-token>

Retires leaked past-EOF block keys for one or all volumes.

Parameter	Type	Default	Description
vid / volume-id	string		A single volume UUID or name. Omit to scrub all open volumes.
blocking	bool	false	Wait for the scrub and return a summary. See the maintenance note above.

Response: with blocking=true, 200 OK and body retired N leaked block key(s): M inode(s) affected, K volume(s) skipped. Otherwise 202 Accepted, no body.

Status	Meaning
200	(blocking=true) Scrub completed; body reports the counts.
202	Scrub started in the background.
400	The vid / volume-id value could not be resolved to a volume.
401	Missing or invalid bearer token.
409	(async only) A conflicting scrub is already in progress.
500	(blocking=true) Scrub failed.
503	The server is shutting down. A blocking scrub running when the server shuts down may return this status with a Retry-After: 30 header or have its connection closed.

Reconcile

POST /v2/reconcile
Authorization: Bearer <meta-token>

Checks that every object in the block store is accounted for in the metadata store and removes any that are not. An object is removed only when it is named like a block of one of the server’s volumes, is at least 48 hours old, and no live or retired block key refers to it. If the metadata for an object cannot be read, the object is kept and the error is logged. A block store is skipped while any volume that uses it has stopped accepting changes after a metadata read error.

Parameter	Type	Default	Description
vid / volume-id	string		A single volume UUID or name. Omit to reconcile all open volumes. When set, objects belonging to other volumes sharing the same block store are left untouched.
blocking	bool	false	Wait for the reconciliation and return a summary. See the maintenance note above.

Response: with blocking=true, 200 OK and body reconciled N object(s): M deleted, K marked for deletion. Otherwise 202 Accepted, no body.

Status	Meaning
200	(blocking=true) Reconciliation completed; body reports the counts.
202	Reconciliation started in the background.
400	The vid / volume-id value could not be resolved to a volume.
401	Missing or invalid bearer token.
409	(async only) A conflicting reconciliation is already in progress.
503	(async only) The server is shutting down.

Recount All

POST /v2/recount/all
Authorization: Bearer <meta-token>

Recounts a volume’s stored totals (inodes, directory entries, blocks, retired blocks, and size) from its metadata and saves the corrected values while the server keeps running. These totals feed the volume metrics and usage reports. Use it when Verify reports countsOK: false.

Parameter	Type	Default	Description
vid / volume-id	string		A single volume UUID or name. Omit to recount all open volumes.
blocking	bool	false	Wait for the recount and return the results. See the maintenance note above.

Response: with blocking=true, 200 OK with a JSON object keyed by volume ID; each value has after and before objects holding the totals after and before the recount: inode, dentry, block, blockRetired, and size. When vid is omitted, retired volumes and volumes whose recount fails are left out, and failures are logged. Otherwise 202 Accepted, no body.

Status	Meaning
200	(blocking=true) Recount completed; body reports the totals.
202	Recount started in the background.
400	The vid / volume-id value could not be resolved to a volume, or names a retired volume.
401	Missing or invalid bearer token.
409	(async only) A conflicting recount is already in progress.
500	(blocking=true) Recount failed.
503	The server is shutting down, or (blocking=true) the volume has stopped accepting changes after a metadata read error, with a Retry-After: 60 header.

Recount Retired Blocks

POST /v2/recount/blocks-retired
Authorization: Bearer <meta-token>

Recounts only a volume’s retired block total from its metadata and saves the corrected value while the server keeps running. It takes less time than Recount All on a large volume. Use it when Verify reports that only blockRetired differs between storedCounts and actualCounts.

Parameter	Type	Default	Description
vid / volume-id	string		A single volume UUID or name. Omit to recount all open volumes.
blocking	bool	false	Wait for the recount and return the results. See the maintenance note above.

Response: with blocking=true, 200 OK with a JSON object keyed by volume ID; each value has after and before, the retired block total after and before the recount. When vid is omitted, volumes whose recount fails are left out, and failures are logged. Otherwise 202 Accepted, no body.

Status	Meaning
200	(blocking=true) Recount completed; body reports the totals.
202	Recount started in the background.
400	The vid / volume-id value could not be resolved to a volume.
401	Missing or invalid bearer token.
409	(async only) A conflicting recount is already in progress.
500	(blocking=true) Recount failed.
503	The server is shutting down, or (blocking=true) the volume has stopped accepting changes after a metadata read error, with a Retry-After: 60 header.

Delete Locks

DELETE /v2/locks
Authorization: Bearer <meta-token>

Deletes held file locks for a volume, optionally scoped to a single inode — the DELETE verb on the /v2/locks resource that GET lists.

Parameter	Type	Default	Description
vid / volume-id	string		Volume UUID or name (required)
ino	uint64	0	Inode to scope the deletion to. 0 or omitted deletes all locks in the volume.

Response: 200 OK with a plain-text body deleted N lock(s).

Status	Meaning
200	Locks deleted; body reports the count.
400	Missing/unresolvable volume, or an invalid ino.
401	Missing or invalid bearer token.
500	Deletion failed.