Prometheus Setup

The flexFS metadata server, proxy server, and mount client expose Prometheus metrics on an HTTP /metrics endpoint. This page covers how to configure Prometheus to scrape these targets.

Endpoints

Server	Default endpoint	Protocol
meta.flexfs	https://<meta-addr>/metrics	HTTPS (unless --noSSL)
proxy.flexfs	https://<proxy-addr>/metrics	HTTPS (unless --noSSL)
mount.flexfs	http://<mount-host>:6075/metrics	HTTP

On the metadata and proxy servers the metrics endpoint is served on the same bind address and port as the main server. The mount client is different: it serves metrics only when started with --metrics, on a dedicated port (--metricsPort, default 6075), over plain HTTP.

Note

If the metadata server has an access file (--accessFile), its /metrics endpoint is subject to the same source-address allow-list as the rest of its API. Add the Prometheus scraper’s address to that access file, or scrapes are rejected with 403. Proxy servers and mount clients have no access file. See API Access Control.

Scrape configuration

Add the following jobs to your prometheus.yml:

Metadata server

scrape_configs:
  - job_name: "flexfs-meta"
    scheme: https
    tls_config:
      insecure_skip_verify: true
    static_configs:
      - targets:
          - "meta-server-1:8443"
        labels:
          instance: "meta-server-1"

Proxy server

  - job_name: "flexfs-proxy"
    scheme: https
    tls_config:
      insecure_skip_verify: true
    static_configs:
      - targets:
          - "proxy-server-1:9443"
          - "proxy-server-2:9443"
        labels:
          region: "us-east-1"

Note

insecure_skip_verify: true is needed when using auto-generated self-signed certificates. If you have deployed custom TLS certificates signed by a trusted CA, remove this setting and provide the CA certificate instead.

Using a trusted CA

scrape_configs:
  - job_name: "flexfs-meta"
    scheme: https
    tls_config:
      ca_file: /path/to/ca.pem
    static_configs:
      - targets:
          - "meta-server-1:8443"

Without TLS

If the servers are running with --noSSL:

scrape_configs:
  - job_name: "flexfs-meta"
    scheme: http
    static_configs:
      - targets:
          - "meta-server-1:8443"

The port matches the server’s --bindAddr. The examples use the installer defaults, 8443 for the metadata server and 9443 for proxy servers; a server started without --bindAddr listens on 443. --noSSL disables TLS but does not change the port.

Mount client

The mount client serves metrics over plain HTTP on --metricsPort (default 6075) on all interfaces, and only when started with --metrics. Give each mount on a host its own --metricsPort: a mount whose port is already in use serves no metrics and keeps retrying until the port is free, logging address already in use; retrying. Because mounts are per-host and may come and go, prefer file-based or node-level service discovery over a static target list where practical:

  - job_name: "flexfs-mount"
    scheme: http
    file_sd_configs:
      - files:
          - /etc/prometheus/flexfs-mounts/*.yml

For a fixed set of mount hosts, a static list also works:

  - job_name: "flexfs-mount"
    scheme: http
    static_configs:
      - targets:
          - "app-host-1:6075"
          - "app-host-2:6075"

Kubernetes service discovery

If flexFS servers are running inside Kubernetes, use kubernetes_sd_configs or a ServiceMonitor (with Prometheus Operator):

ServiceMonitor example

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: flexfs-meta
  labels:
    app: flexfs-meta
spec:
  selector:
    matchLabels:
      app: flexfs-meta
  endpoints:
    - port: https
      scheme: https
      tlsConfig:
        insecureSkipVerify: true
      interval: 15s

Adding labels

To attach labels such as environment or region to every series scraped from a target, set them under labels in static_configs:

scrape_configs:
  - job_name: "flexfs-meta"
    scheme: https
    tls_config:
      insecure_skip_verify: true
    static_configs:
      - targets: ["meta-1:8443"]
        labels:
          environment: "production"
          region: "us-east-1"

Scrape interval

A 15-second scrape interval works well for most deployments. For high-throughput environments, you may reduce it to 10 seconds. Avoid intervals shorter than 5 seconds to minimize overhead.

scrape_configs:
  - job_name: "flexfs-meta"
    scrape_interval: 15s
    scrape_timeout: 10s
    # ...

Verifying metrics

Test that metrics are reachable:

# Metadata server
curl -sk https://meta-server-1:8443/metrics | head -20

# Proxy server
curl -sk https://proxy-server-1:9443/metrics | head -20

# Mount client (plain HTTP on the metrics port)
curl -s http://app-host-1:6075/metrics | head -20

You should see lines like:

# HELP flexfs_meta_rpc_ops_total Total RPC operations by method and status.
# TYPE flexfs_meta_rpc_ops_total counter
flexfs_meta_rpc_ops_total{method="Lookup",status="OK",volume_id="abc123"} 42

Next steps

• Metrics reference — full catalog of all metrics
• Grafana dashboards — visualize the data
• Alerting — set up alerts on key metrics
• Logging and diagnostics — server and mount client logs