Prometheus Setup

The flexFS metadata server, proxy server, and mount client expose Prometheus metrics on an HTTP /metrics endpoint. This page covers how to configure Prometheus to scrape these targets.

Endpoints

Server	Default endpoint	Protocol
meta.flexfs	https://<meta-addr>/metrics	HTTPS (unless --noSSL)
proxy.flexfs	https://<proxy-addr>/metrics	HTTPS (unless --noSSL)
mount.flexfs	http://<mount-host>:6073/metrics	HTTP

On the metadata and proxy servers the metrics endpoint is served on the same bind address and port as the main server. The mount client is different: it serves metrics only when started with --metrics, on a dedicated port (--metricsPort, default 6073), over plain HTTP.

Scrape configuration

Add the following jobs to your prometheus.yml:

Metadata server

scrape_configs:
  - job_name: "flexfs-meta"
    scheme: https
    tls_config:
      insecure_skip_verify: true
    static_configs:
      - targets:
          - "meta-server-1:443"
        labels:
          instance: "meta-server-1"

Proxy server

  - job_name: "flexfs-proxy"
    scheme: https
    tls_config:
      insecure_skip_verify: true
    static_configs:
      - targets:
          - "proxy-server-1:443"
          - "proxy-server-2:443"
        labels:
          region: "us-east-1"

Note

insecure_skip_verify: true is needed when using auto-generated self-signed certificates. If you have deployed custom TLS certificates signed by a trusted CA, remove this setting and provide the CA certificate instead.

Using a trusted CA

scrape_configs:
  - job_name: "flexfs-meta"
    scheme: https
    tls_config:
      ca_file: /path/to/ca.pem
    static_configs:
      - targets:
          - "meta-server-1:443"

Without TLS

If the servers are running with --noSSL:

scrape_configs:
  - job_name: "flexfs-meta"
    scheme: http
    static_configs:
      - targets:
          - "meta-server-1:8080"

Mount client

The mount client serves metrics over plain HTTP on --metricsPort (default 6073), and only when started with --metrics. Because mounts are per-host and may come and go, prefer file-based or node-level service discovery over a static target list where practical:

  - job_name: "flexfs-mount"
    scheme: http
    file_sd_configs:
      - files:
          - /etc/prometheus/flexfs-mounts/*.yml

For a fixed set of mount hosts, a static list also works:

  - job_name: "flexfs-mount"
    scheme: http
    static_configs:
      - targets:
          - "app-host-1:6073"
          - "app-host-2:6073"

Kubernetes service discovery

If flexFS servers are running inside Kubernetes, use kubernetes_sd_configs or a ServiceMonitor (with Prometheus Operator):

ServiceMonitor example

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: flexfs-meta
  labels:
    app: flexfs-meta
spec:
  selector:
    matchLabels:
      app: flexfs-meta
  endpoints:
    - port: https
      scheme: https
      tlsConfig:
        insecureSkipVerify: true
      interval: 15s

Relabeling

To add meaningful labels to your metrics, use relabel_configs:

scrape_configs:
  - job_name: "flexfs-meta"
    scheme: https
    tls_config:
      insecure_skip_verify: true
    static_configs:
      - targets: ["meta-1:443"]
        labels:
          environment: "production"
          region: "us-east-1"

Scrape interval

A 15-second scrape interval works well for most deployments. For high-throughput environments, you may reduce it to 10 seconds. Avoid intervals shorter than 5 seconds to minimize overhead.

scrape_configs:
  - job_name: "flexfs-meta"
    scrape_interval: 15s
    scrape_timeout: 10s
    # ...

Verifying metrics

Test that metrics are reachable:

# Metadata server
curl -sk https://meta-server-1:443/metrics | head -20

# Proxy server
curl -sk https://proxy-server-1:443/metrics | head -20

# Mount client (plain HTTP on the metrics port)
curl -s http://app-host-1:6073/metrics | head -20

You should see lines like:

# HELP flexfs_meta_rpc_ops_total Total RPC operations by method and status.
# TYPE flexfs_meta_rpc_ops_total counter
flexfs_meta_rpc_ops_total{volume_id="abc123",method="Lookup",status="OK"} 42

Next steps

• Metrics reference — full catalog of all metrics
• Grafana dashboards — visualize the data
• Alerting — set up alerts on key metrics
• Logging and diagnostics — logs, RTT logging, and pprof profiling