aws.flexfs

Caution

This entire page documents an internal component. aws.flexfs is the AWS Marketplace integration server operated by Paradigm4. It is not intended for customer deployment.

aws.flexfs connects the flexFS Enterprise Edition AWS Marketplace listing to the stat server. It:

• Serves the registration page that AWS Marketplace sends subscribers to. The page creates a stat customer, shows the license key and installation steps, and emails the key.
• Reads subscription notifications from an SQS queue, and suspends or resumes the customer’s license grants in the stat server to match.
• Reports each customer’s hourly usage to AWS with BatchMeterUsage, in hot-equivalent GiB-months.

It keeps its own SQLite database and calls the stat server only through its marketplace API. Calls to the stat server are retried every second on connection errors and server errors, so a stat server restart of a few seconds does not interrupt registration, and longer outages only delay grant updates and metering. A customer the stat server rejects, or whose request fails, is skipped for that pass without holding up the others; a rejection is logged once until it changes, and a pass stops after 3 consecutive failed requests. A rejected statToken (401) or a disabled marketplace API (503) counts as a failed request, so it stops every pass and is logged each time.

Deployment

• Run aws.flexfs start --noSSL --bindAddr 127.0.0.1:<port> behind a TLS-terminating reverse proxy such as Nginx, with a publicly trusted certificate for the registration hostname. The proxy should set X-Real-IP or X-Forwarded-For so logs show the subscriber’s address, and should rate-limit /register and /complete.
• Set the listing’s fulfillment URL to https://<registration hostname>/register.
• Create an Amazon EventBridge rule on the seller account’s default event bus in us-east-1 with the event pattern {"source": ["aws.agreement-marketplace"]}, and target the SQS queue set in queueURL. The queue’s access policy must allow events.amazonaws.com to send messages from that rule. A listing that publishes to an AWS Marketplace SNS subscription topic can subscribe the same queue to that topic instead; both message formats are accepted.
• AWS API credentials come from the default AWS credential chain (instance role, environment, or profile) and must belong to the seller account. The credentials need aws-marketplace:ResolveCustomer, aws-marketplace:BatchMeterUsage, sqs:ReceiveMessage, and sqs:DeleteMessage on the queue.
• Set marketplaceToken in the stat server’s credentials file to the same value as statToken here.

Metering

• Each hour’s quantity is the hot-equivalent GiB-months reported by the stat server, computed from the customer’s own rate bins in the stat server. Rate bins in rateBins are only the defaults for new customers. The AWS price for the dimension is the price of one hot GiB-month, and the ratios between a customer’s rate bins set its tier discounts.
• Quantities are whole numbers. Fractions are carried forward to the next hour.
• An hour is metered 3 hours after it starts, once reports delayed by up to 2 hours have reached the stat server. Reports delayed longer are still recorded by the stat server but are not billed through AWS. When a subscriber begins unsubscribing, the current hour becomes the final hour: AWS accepts final usage for only about an hour, so it is metered once it is 55 minutes old, and later hours are not metered. Logs identify customers by the last 12 characters of their license key.
• Each subscription (license ARN) has its own stat customer and license key, and its usage is metered against its own license ARN. A resubscription is a new subscription with a new license key, so hours between subscriptions are never metered. Several concurrent subscriptions from one AWS account are metered independently.
• A subscription’s first usage record is timestamped when usage began (the later of the subscription and the registration), not at the start of that hour.
• License Updated - Manufacturer marks a subscription active. Purchase Agreement Ended with status CANCELLED, EXPIRED, or TERMINATED, and License Deprovisioned - Manufacturer, start the final reporting window: the final hour is metered, and the subscription is marked unsubscribed one hour later. Agreement events are matched to a subscription through the agreement ID carried by its license events. An agreement that ends because it was replaced or renewed does not end the subscription.
• A notification that identifies the buyer only by account or customer identifier, with no license ARN or agreement ID, is not applied when it matches more than one open or recently closed (within 24 hours) subscription and at least one of them is open; a warning is logged so it can be applied by hand. Otherwise it is applied to the open subscription, or to the newest one when none is open. Notifications that never match a subscription are retired after 7 days, and handled notifications are deleted after 30 days.
• Registration is refused, and no license key is issued, for a subscription that has failed or ended.
• A subscription that AWS has not confirmed within 48 hours, with no usage accepted by AWS, has its license grants suspended and an error is logged.
• The final hour and the one-hour final reporting window are taken from the notification’s own timestamp when it has one, so a processing backlog does not extend them.
• Subscription notifications only move a subscription forward (pending, failed, active, unsubscribing, unsubscribed), so a redelivered or out-of-order notification cannot reactivate it.
• Records AWS does not accept are retried every minute, after records that have not been sent yet. When AWS rejects a batch because of one record (for example an out-of-range timestamp or a customer who is not entitled), the batch is retried one record at a time so that record does not hold up the others, and the rejected record is retried at most once an hour. Network errors, throttling, and errors that affect every record, such as an invalid dimension, end the pass, and all records are retried on the next one. AWS accepts records for up to 24 hours, and records for the previous month until 06:00 UTC on the 1st. Records still unsent 23 hours after their hour (or at 05:30 UTC on the 1st for the previous month) are marked expired and logged as errors.

Persistent Flags

These flags apply to every subcommand.

Flag	Type	Default	Description	Visibility
--reportErrors	bool	false	Report errors and panics to Paradigm4	Public

Subcommands

Subcommand	Description	Visibility
deinit creds	Remove the credentials file	Public
deinit systemd	Remove the systemd service unit	Public
init creds	Initialize the credentials file	Public
init systemd	Create and enable a systemd service unit	Public
license	Print license information	Public
start	Start the AWS Marketplace server	Public
version	Print the build version	Public

deinit creds

aws.flexfs deinit creds [flags]

Removes the credentials file. If the credentials file is referenced by a systemd unit, the command refuses unless --force is passed, in which case it removes the references too.

Flag	Type	Default	Description	Visibility
--credsFile	string	~/.flexfs/aws/creds	Credentials file path	Public
--force	bool	false	Remove creds even if referenced by systemd (removes references too)	Public

deinit systemd

sudo aws.flexfs deinit systemd

Removes the systemd service unit (flexfs-aws.service). Requires root.

init creds

aws.flexfs init creds [flags]

Initializes the credentials file. Every field can also be written to the file by hand, and each can be overridden at start by a hidden flag of the same name. init on its own does the same thing as init creds.

Flag	Type	Default	Description	Visibility
--credsFile	string	~/.flexfs/aws/creds	Credentials file path	Public
--dimension	string	""	Usage dimension API name from the listing (required)	Public
--force	bool	false	Overwrite existing credentials file	Public
--productCode	string	""	Listing product code (required)	Public
--queueURL	string	""	SQS queue URL for subscription notifications (required)	Public
--rateBins	string	""	Default rate bins for new customers as JSON, such as {"0":0.10,"3":0.05}; bin 0 must be positive (required)	Public
--region	string	us-east-1	AWS Marketplace metering region	Public
--smtpAddr	string	""	SMTP server address (required)	Public
--smtpFrom	string	""	SMTP from email address (required)	Public
--smtpPass	string	""	SMTP server password (prompted for on a terminal if omitted and a username is set)	Public
--smtpUser	string	""	SMTP server username	Public
--statAddr	string	stat.flexfs.io	Stat server address	Public
--statToken	string	""	Stat server marketplace token, at least 32 characters with no spaces (prompted for on a terminal if omitted)	Public
--supportEmail	string	support@flexfs.io	Support email address shown to subscribers	Public

init systemd

sudo aws.flexfs init systemd [flags]

Creates and enables a systemd service unit (flexfs-aws.service). Requires root. Optional: the server can also be run directly with start by any user. The service runs as root, so it uses root’s /root/.flexfs defaults, credentials included, unless --startFlags names other paths.

Flag	Type	Default	Description	Visibility
--force	bool	false	Overwrite existing systemd unit file	Public
--now	bool	false	Start the service immediately after enabling	Public
--startFlags	string	""	Additional flags to pass to the start command	Public

start

aws.flexfs start [flags]

Starts the registration server and the notification, grant, and metering loops. Any user can run it, and ~ in the defaults is that user’s home folder. A non-root user needs a --bindAddr port above 1023.

Flag	Type	Default	Description	Visibility
--bindAddr	string	0.0.0.0:443	Address and port to bind	Public
--credsFile	string	~/.flexfs/aws/creds	Credentials file path	Public
--dbFolder	string	~/.flexfs/aws	Database folder path	Public
--dimension	string	""	Usage dimension API name (overrides credentials file)	Internal
--noSSL	bool	false	Disable SSL for the registration server	Public
--noStatSSL	bool	false	Disable SSL for stat server connections	Public
--pprof	bool	false	Enable pprof profiler	Internal
--pprofPort	int	6061	Pprof server port	Internal
--productCode	string	""	Listing product code (overrides credentials file)	Internal
--queueURL	string	""	SQS queue URL (overrides credentials file)	Internal
--rateBins	string	""	Default rate bins as JSON (overrides credentials file)	Internal
--region	string	""	AWS Marketplace metering region (overrides credentials file)	Internal
--smtpAddr	string	""	SMTP server address (overrides credentials file)	Internal
--smtpFrom	string	""	SMTP from email address (overrides credentials file)	Internal
--smtpPass	string	""	SMTP server password (overrides credentials file)	Internal
--smtpUser	string	""	SMTP server username (overrides credentials file)	Internal
--sqliteOpts	string	_journal=WAL&_cache_size=10240&_fk=true&_timeout=5000	SQLite database connection options	Internal
--sslCert	string	~/.flexfs/ssl/cert	SSL certificate file path. A self-signed certificate and key are created if neither exists.	Public
--sslKey	string	~/.flexfs/ssl/key	SSL private key file path	Public
--statAddr	string	""	Stat server address (overrides credentials file)	Internal
--statToken	string	""	Stat server marketplace token (overrides credentials file)	Internal
--supportEmail	string	""	Support email address (overrides credentials file)	Internal
--verbose, -v	bool	false	Enable verbose logging	Public