aws.flexfs

Caution

This entire page documents an internal component. aws.flexfs is the AWS Marketplace integration server operated by Paradigm4. It is not intended for customer deployment.

aws.flexfs connects the flexFS Enterprise Edition AWS Marketplace listing to the stat server. It:

• Serves the registration page that AWS Marketplace sends subscribers to. The page creates a stat customer, shows the license key and installation steps, and emails the key. A failed email is retried every 5 minutes for 24 hours, then logged as an error.
• Reads subscription notifications from an SQS queue, and suspends or resumes the customer’s license grants in the stat server to match.
• Reports each customer’s hourly usage to AWS with BatchMeterUsage, in GiB-months for each of four activity tiers.
• Reads each subscription’s agreement prices and sets the customer’s rate bins in the stat server to match.

It keeps its own SQLite database and calls the stat server only through its marketplace API. Calls to the stat server are retried every second on connection errors and server errors, so a stat server restart of a few seconds does not interrupt registration, and longer outages only delay grant updates, rate bin updates, and metering. A customer the stat server rejects, or whose request fails, is skipped for that pass without holding up the others; a rejection is logged once until it changes, and a pass stops after 3 consecutive failed requests. A rejected statToken (401) or a disabled marketplace API (503) counts as a failed request, so it stops every pass and is logged each time.

Deployment

• Run aws.flexfs start --noSSL --bindAddr 127.0.0.1:<port> behind a TLS-terminating reverse proxy such as Nginx, with a publicly trusted certificate for the registration hostname. The proxy should set X-Real-IP or X-Forwarded-For so logs show the subscriber’s address, and should rate-limit /register and /complete.
• Set the listing’s fulfillment URL to https://<registration hostname>/register.
• Create an Amazon EventBridge rule on the seller account’s default event bus in us-east-1 with the event pattern {"source": ["aws.agreement-marketplace"]}, and target the SQS queue set in queueURL. The queue’s access policy must allow events.amazonaws.com to send messages from that rule. Other messages on the queue, such as AWS Marketplace SNS notifications, are logged and dropped.
• AWS API credentials come from the default AWS credential chain (instance role, environment, or profile) and must belong to the seller account. The credentials need aws-marketplace:ResolveCustomer, aws-marketplace:BatchMeterUsage, aws-marketplace:SearchAgreements, and aws-marketplace:GetAgreementTerms, plus sqs:ReceiveMessage and sqs:DeleteMessage on the queue.
• Set marketplaceToken in the stat server’s credentials file to the same value as statToken here.

Metering

• Each hour, a customer’s stored data is split into four activity tiers by size bin, and each tier is reported as its own usage record, in GiB-months. The listing must define these four dimensions:

  Dimension	Size bins	Last activity
  hot_gib_months	0-5	Less than 180 days ago
  warm_gib_months	6-11	180 to 360 days ago
  cool_gib_months	12-23	360 to 720 days ago
  cold_gib_months	24 and above	720 days ago or more

• Quantities are whole numbers. Each tier’s fractions are carried forward to its next hour separately.

• Usage from a stat server that does not report size bins, such as an older version, is not metered: the request counts as failed and is retried until the stat server is upgraded.

• An hour is metered 3 hours after it starts, once reports delayed by up to 2 hours have reached the stat server. Reports delayed longer are still recorded by the stat server but are not billed through AWS. When the final reporting window starts, its hour becomes the final hour: AWS accepts final usage for only about an hour, so it is metered once it is 55 minutes old, and later hours are not metered. Logs identify customers by the last 12 characters of their license key.

• Each subscription (license ARN) has its own stat customer and license key, and its usage is metered against its own license ARN. A resubscription is a new subscription with a new license key, so hours between subscriptions are never metered. Several concurrent subscriptions from one AWS account are metered independently.

• A subscription’s first usage record is timestamped when usage began (the later of the subscription and the registration), not at the start of that hour.

• License Updated - Manufacturer marks a subscription active. Purchase Agreement Ended with status CANCELLED, EXPIRED, or TERMINATED marks it unsubscribing, and metering continues until License Deprovisioned - Manufacturer starts the final reporting window: the final hour is metered, and the subscription is marked unsubscribed one hour later. If no License Deprovisioned - Manufacturer arrives within 24 hours of the agreement ending, a warning is logged and the final reporting window starts then. Agreement events are matched to a subscription through the agreement ID carried by its license events. An agreement that ends because it was replaced or renewed does not end the subscription.

• Notifications for a license that has no subscription yet are applied once the subscriber registers, and agreement notifications once a license notification links the agreement. Notifications that never match a subscription are retired after 7 days, and handled notifications are deleted after 30 days.

• Registration is refused, and no license key is issued, for a subscription that has ended.

• A subscription that AWS has not confirmed within 48 hours, with no usage accepted by AWS, has its license grants suspended and an error is logged.

• The final hour and the one-hour final reporting window are taken from the notification’s own timestamp, so a processing backlog does not extend them.

• Subscription notifications only move a subscription forward (pending, active, unsubscribing, unsubscribed), so a redelivered or out-of-order notification cannot reactivate it.

• Records AWS does not accept are retried every minute, after records that have not been sent yet. When AWS rejects a batch because of one record (for example an out-of-range timestamp or a customer who is not entitled), the batch is retried one record at a time so that record does not hold up the others, and the rejected record is retried at most once an hour. Network errors, throttling, and errors that affect every record, such as an invalid dimension, end the pass, and all records are retried on the next one. AWS accepts records for up to 24 hours, and records for the previous month until 06:00 UTC on the 1st. Records still unsent 23 hours after their hour (or at 05:30 UTC on the 1st for the previous month) are marked expired and logged as errors.

Rate bins

The stat server uses a customer’s rate bins for its cost figures and revenue projections. AWS bills from the agreement itself, so rate bins never affect what a subscriber is charged.

• aws.flexfs finds the subscription’s active agreement, reads its usage prices for the four dimensions, and sets the customer’s rate bins in the stat server to {"0": <hot>, "6": <warm>, "12": <cool>, "24": <cold>}. Prices are read right after registration, again once a day, and whenever a notification for the subscription arrives, including Purchase Agreement Amended, which changes nothing else about the subscription. Only pending and active subscriptions are checked.
• A new customer starts with a rate of 0, so statistics recorded before the prices are read show no cost.
• When no active agreement is found, a warning is logged once and the lookup is retried every 5 minutes.
• When the subscription’s newest agreement is CANCELLED, EXPIRED, or TERMINATED and none is active, a warning is logged and the subscription is handled as if Purchase Agreement Ended had arrived. This catches a missed notification within a day.
• When the agreement has no usage pricing, is not priced in USD, or lacks a valid price for any of the four dimensions, an error is logged, the customer’s rate bins are left unchanged, and the agreement is checked again the next day.
• Agreement API errors are retried every 5 minutes. A pass stops after 3 consecutive failed lookups.

Persistent Flags

These flags apply to every subcommand. Flag names on every subcommand are matched case-insensitively and ignoring hyphens.

Flag	Type	Default	Description	Visibility
--reportErrors	bool	false	Report errors and panics to Paradigm4	Public

Subcommands

Subcommand	Description	Visibility
completion	Generate a shell autocompletion script	Public
deinit creds	Remove the credentials file	Public
deinit systemd	Remove the systemd service unit	Public
init creds	Initialize the credentials file	Public
init systemd	Create and enable a systemd service unit	Public
license	Print license information	Public
start	Start the AWS Marketplace server	Public
version	Print the build version	Public

deinit creds

aws.flexfs deinit creds [flags]

Removes the credentials file. If the credentials file is referenced by a systemd unit, the command refuses unless --force is passed, in which case it removes the references too. deinit on its own does the same thing as deinit creds.

Flag	Type	Default	Description	Visibility
--credsFile	string	~/.flexfs/aws/creds	Credentials file path	Public
--force	bool	false	Remove creds even if referenced by systemd (removes references too)	Public

deinit systemd

sudo aws.flexfs deinit systemd

Removes the systemd service unit (flexfs-aws.service). Requires root.

init creds

aws.flexfs init creds [flags]

Initializes the credentials file. Every field can also be written to the file by hand, and each can be overridden at start by a hidden flag of the same name. A field left at its default, such as --region, --statAddr or --supportEmail, is not written to the file, and a field missing from the file takes its default. init on its own does the same thing as init creds.

Flag	Type	Default	Description	Visibility
--credsFile	string	~/.flexfs/aws/creds	Credentials file path	Public
--force	bool	false	Overwrite existing credentials file	Public
--productCode	string	""	Listing product code (required)	Public
--queueURL	string	""	SQS queue URL for subscription notifications (required)	Public
--region	string	us-east-1	AWS Marketplace metering region	Public
--smtpAddr	string	""	SMTP server address (required)	Public
--smtpFrom	string	""	SMTP from email address (required)	Public
--smtpPass	string	""	SMTP server password (prompts if omitted and a username is set)	Public
--smtpUser	string	""	SMTP server username	Public
--statAddr	string	stat.flexfs.io	Stat server address	Public
--statToken	string	""	Stat server marketplace token, a UUID (prompts if omitted)	Public
--supportEmail	string	support@flexfs.io	Support email address shown to subscribers	Public

init systemd

sudo aws.flexfs init systemd [flags]

Creates and enables a systemd service unit (flexfs-aws.service). Requires root. Optional: the server can also be run directly with start by any user. The service runs as root, so it uses root’s /root/.flexfs defaults, credentials included, unless --startFlags names other paths.

Flag	Type	Default	Description	Visibility
--force	bool	false	Overwrite existing systemd unit file	Public
--now	bool	false	Start the service immediately after enabling	Public
--startFlags	string	""	Additional flags to pass to the start command	Public

start

aws.flexfs start [flags]

Starts the registration server and the notification, grant, rate bin, and metering loops. Any user can run it, and ~ in the defaults is that user’s home folder. A non-root user needs a --bindAddr port above 1023.

Flag	Type	Default	Description	Visibility
--bindAddr	string	0.0.0.0:443	Address and port to bind	Public
--credsFile	string	~/.flexfs/aws/creds	Credentials file path	Public
--dbFolder	string	~/.flexfs/aws	Database folder path	Public
--noSSL	bool	false	Disable SSL for the registration server	Public
--noStatSSL	bool	false	Disable SSL for stat server connections	Internal
--pprof	bool	false	Enable pprof profiler	Internal
--pprofPort	int	6061	Pprof server port	Internal
--productCode	string	""	Listing product code (overrides credentials file)	Internal
--queueURL	string	""	SQS queue URL (overrides credentials file)	Internal
--region	string	""	AWS Marketplace metering region (overrides credentials file)	Internal
--smtpAddr	string	""	SMTP server address (overrides credentials file)	Internal
--smtpFrom	string	""	SMTP from email address (overrides credentials file)	Internal
--smtpPass	string	""	SMTP server password (overrides credentials file)	Internal
--smtpUser	string	""	SMTP server username (overrides credentials file)	Internal
--sqliteOpts	string	_journal=WAL&_cache_size=10240&_fk=true&_timeout=5000	SQLite database connection options	Internal
--sslCert	string	~/.flexfs/ssl/cert	SSL certificate file path. A self-signed certificate and key are created if neither exists.	Public
--sslKey	string	~/.flexfs/ssl/key	SSL private key file path	Public
--statAddr	string	""	Stat server address (overrides credentials file)	Internal
--statToken	string	""	Stat server marketplace token (overrides credentials file)	Internal
--supportEmail	string	""	Support email address (overrides credentials file)	Internal
--verbose, -v	bool	false	Enable verbose logging	Public