Preparation

Metadata servers can either be self-hosted or hosted by Paradigm4. Customers also have the option to deploy flexFS metadata servers in a quorum-based highly available (HA) configuration. Please inquire for more details.

note

The following sections are only relevant to those who choose to self-host flexFS.

Block store

Every flexFS volume must be assigned to a block store (which is typically equivalent to an object storage bucket). Using prefixes, one or more flexFS volumes can safely be assigned to an existing block store, even when it already contains data unrelated to flexFS.

info

For best performance, block stores should generally be located in the same cloud region as any metadata servers and mount clients making use of them. There are configurations, however, for which this rule-of-thumb does not apply.

Block store credentials

The flexFS metadata service needs to be able to authenticate against the block store. This can be accomplished either by obtaining and supplying static credentials to the metadata server during initialization, or by attaching an IAM instance role to the metadata server's AWS host and instructing the metadata server to use it (when using AWS S3).

AWS IAM instance role

For convenience and added security you may choose to attach an IAM instance role with the metadata server's host in AWS. To do so, you first must create an IAM policy granting permissions on all S3 buckets assigned to flexFS volumes managed by the metadata server.

• Services > Identity and Access Management (IAM) > Policies [ Create policy ]

policy-example.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:DeleteObject",
                "s3:GetObject",
                "s3:ListBucket",
                "s3:PutObject"
            ],
            "Resource": [
                "arn:aws:s3:::bucket-name",
                "arn:aws:s3:::bucket-name/*"
            ]
        }
    ]
}

• Save the policy.

Create a role associated with the S3 bucket access policy you just created.

• Services > Identity and Access Management (IAM) > Roles [ Create role ]
• Select Custom trust policy and specify the following:

role-trust-policy-example.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "sts:AssumeRole"
            ],
            "Principal": {
                "Service": [
                    "ec2.amazonaws.com"
                ]
            }
        }
    ]
}

• Click Next and then select the policy created above.
• Save the role.
• Attach this role to the host instance created in the next step.

Metadata server host

Before installing the metadata server, a host must be provisioned for it. The required host specifications will vary dramatically based on level of usage. Factors include:

• Number of flexFS volumes assigned to the server.
• Number and size of the files in each volume.
• Total demand from all mount clients.

There are also two metadata engines to choose from:

• An in-memory snapshot engine with higher server memory requirements and higher performance. We recommend a host with at least 3 GB of RAM per 1 million inodes for this engine.
• An on-disk sqlite engine with lower server memory requirements and lower performance. We recommend a host with at least 2 GB of RAM total for this engine.

For both engines, we recommend a host with at least 3 GB hard drive capacity per 1 million flexFS inodes + 1 GB for Linux.

Metadata server token

Before running a flexFS metadata server, it is necessary to obtain a metadata server token from Paradigm4 that serves to authenticate the server with the flexFS administrative service. This is a standard step in the flexFS onboarding process.