Skip to content

Logging and Diagnostics

FlexFS provides multiple logging and diagnostic tools for troubleshooting performance issues and understanding system behavior.

In daemon mode (the default), the mount client writes logs to a file. Specify the log file path with:

Terminal window
mount.flexfs start <volume-name> /mnt/data --logFile /var/log/flexfs/mount.log

If no --logFile is specified, logs are written to ~/.flexfs/mount/logs/<name>-<pid>.log by default, where <name> is the volume name and <pid> is the mount process ID.

In foreground mode (--foreground), logs are written to stdout.

Enable detailed logging with the --verbose flag:

Terminal window
mount.flexfs start <volume-name> /mnt/data --verbose

Verbose mode produces significantly more output and should be used for debugging only.

For servers:

Terminal window
meta.flexfs start --verbose
proxy.flexfs start --verbose

The mount client can expose a Prometheus /metrics endpoint with --metrics, served over plain HTTP on --metricsPort (default 6074). This is the recommended way to monitor a running mount. See the Metrics Reference for the full catalog and Prometheus setup for scrape configuration.

The metadata server and proxy server log to stdout by default. Use --verbose for detailed operation logging.

When running under systemd, logs are captured by the journal:

Terminal window
# Metadata server logs
journalctl -u flexfs-meta -f
# Proxy server logs
journalctl -u flexfs-proxy -f

Where an access file is in use, lines like these describe it. All of them are logged whether or not --verbose is set: a refused request is either an attack or a rule that is wrong, and both need to be visible in a default deployment.

Access rules loaded from "/root/.flexfs/meta/access": global 10.0.0.0/8, 1 endpoint rule(s)
access | DELETE /locks 10.1.2.3
access | global rule gates: /backup, /compact, /locks, /metrics, /status, /verify, ...
access | not gated by the global rule: / (add an [endpoint."..."] section to gate one)
Denied "DELETE" "/locks" from "203.0.113.9" (access rules)
Error (access): cannot reload "/root/.flexfs/meta/access", keeping the rules already in force: ...
Access rules removed ("/root/.flexfs/meta/access") - all endpoints unrestricted

The global rule gates: line is the one to read after an edit — it names every endpoint the save just put behind the whitelist, which on the admin and free-tier servers includes the endpoints mount clients and the metadata server call.

A denial is logged once per source, method and endpoint every ten seconds. Requests to paths the server does not serve share one entry per source and method, so a scanner walking URLs cannot flood the log; whatever the throttle drops is counted on the next line it writes (... 49 further denial(s) not logged). A reload failure repeats no more than once a minute for as long as the file stays broken.

Worth alerting on: any Error (access) line, and a /status response whose access.state is stale — both mean the file on disk is not what the server is enforcing. See API Access Control.

The CSI driver logs to stdout within the container. Enable verbose logging by passing --verbose to the csi.flexfs start command:

Terminal window
# View CSI driver logs
kubectl logs -n kube-system <flexfs-csi-node-pod> -c flexfs-csi-driver

Mount client logs within the CSI driver are written to /var/lib/kubelet/flexfs/<volume-name>/log-rw (or log-ro for read-only mounts).