API Reference¶

This section contains the complete API documentation auto-generated from source code docstrings.

Core Modules¶

ML Manager¶

class ml_provisioner.core.ml_manager.MlManager(config_file_name, action, dry_run=False, log_level='INFO', force=False, app_config=None)¶

Bases: object

Manages AWS ML project infrastructure provisioning and lifecycle operations using CloudFormation.

This class provides tier-based ML project scaffolding for enterprise workloads. It loads pre-built tier blueprints (starter, professional, enterprise), applies client configuration, validates security rules, generates CloudFormation templates, and manages the complete ML product lifecycle from creation to deletion.

Key Features:

Tier-based ML project generation (starter, professional, enterprise)
Generic use-case parameterization — use_case drives resource naming only
Pure data definition blueprints — no placeholder substitution
CloudFormation template generation via programmatic Python dict construction
Parameter Store integration for ML resource ID storage
IAM policy generation for least-privilege access
Conditional generation for source_control (codecommit/s3) and vpc_integration mode
Dry-run mode for safe testing
Force flag for mutating operations

Parameters:

config_file_name (str)
action (str)
dry_run (bool)
log_level (str)
force (bool)

ml_name¶

Name of the ML product being managed

Type:: str

region¶

AWS region for deployment

Type:: str

action¶

Current action being performed

Type:: str

dry_run¶

If True, simulates operations without making changes

Type:: bool

force¶

If True, forces mutating AWS operations without confirmation

Type:: bool

ml_stack_name¶

CloudFormation stack name

Type:: str

tier¶

Selected tier (starter, professional, enterprise)

Type:: str

use_case¶

ML use case name — naming parameter only, not a solution

Type:: str

source_control¶

Source control type (codecommit or s3)

Type:: str

cf_client¶: Boto3 CloudFormation client

ssm_client¶: Boto3 Systems Manager client

logger¶: Configured logger instance

Class Constants:: STACK_CREATE_WAITER_DELAY (int): Delay between stack creation checks (seconds) STACK_CREATE_WAITER_MAX_ATTEMPTS (int): Maximum stack creation wait attempts STACK_DELETE_WAITER_DELAY (int): Delay between stack deletion checks (seconds) STACK_DELETE_WAITER_MAX_ATTEMPTS (int): Maximum stack deletion wait attempts DRIFT_POLL_INTERVAL (int): Seconds between drift status checks DRIFT_POLL_TIMEOUT (int): Total timeout for drift detection (10 minutes) CHANGESET_POLL_INTERVAL (int): Seconds between changeset status checks CHANGESET_MAX_ATTEMPTS (int): Maximum attempts for changeset creation

Example

>>> from ml_provisioner.core import MlManager
>>> manager = MlManager(
...     config_file_name='globalbank-prod-c001-us-west-1-demand-forecasting-ml.yaml',
...     action='create-prov-template',
...     dry_run=False
... )
>>> exit_code = manager.execute()

Note

Requires valid AWS credentials with appropriate IAM permissions
Configuration file must pass validation schema
Use dry_run=True to preview changes before execution
Enterprise tier requires VPC to be deployed before deployment

CHANGESET_MAX_ATTEMPTS = 60¶

CHANGESET_POLL_INTERVAL = 5¶

DRIFT_POLL_INTERVAL = 10¶

DRIFT_POLL_TIMEOUT = 600¶

STACK_CREATE_WAITER_DELAY = 10¶

STACK_CREATE_WAITER_MAX_ATTEMPTS = 60¶

STACK_DELETE_WAITER_DELAY = 10¶

STACK_DELETE_WAITER_MAX_ATTEMPTS = 60¶

__init__(config_file_name, action, dry_run=False, log_level='INFO', force=False, app_config=None)¶

Initializes class members from config file and parameters.

Parameters:

config_file_name (str) – Name of the configuration file.
action (str) – Action to be performed.
dry_run (bool) – Dry run flag, if True no changes are made.
log_level (str) – Logging level for the operation.
force (bool) – If True, forces mutating AWS operations without confirmation.
app_config (AppConfig) – Application configuration object (AppConfig instance).

execute()¶

Executes the specified action on the ML product.

Returns:: 0 for success, 1 for failure.
Return type:: int

Configuration¶

Configuration Loader¶

class ml_provisioner.config.loader.AppConfig(config_dict)¶

Bases: object

Application configuration with embedded defaults.

Loads configuration from YAML files and provides convenient access to configuration values with automatic path resolution.

Parameters:: config_dict (Dict[str, Any])

__init__(config_dict)¶

Parameters:: config_dict (Dict[str, Any]) – Raw configuration dictionary.

get(key_path, default=None)¶

Gets a config value using dot notation.

Parameters:

key_path (str) – Dot-separated path (e.g., ‘directories.configs_dir’)
default (Any) – Default value if key not found

Return type:

Any

get_directory(dir_name)¶

Gets a directory path as a Path object.

Parameters:: dir_name (str) – Directory name (configs_dir, reports_dir, etc.)
Raises:: ValueError – If directory not configured
Return type:: Path

ml_provisioner.config.loader.load_app_config(custom_config_path=None)¶

Loads and returns application configuration.

Resolution order: 1. Custom config file (if provided via –app-config) 2. Embedded default config (always available)

Raises:

FileNotFoundError – If custom config specified but not found
ValueError – If config file is invalid YAML

Parameters:

custom_config_path (str | None)

Return type:

AppConfig

Generators¶

CloudFormation Generator¶

class ml_provisioner.generators.cfn_generator.CfnGenerator(blueprint, config, ml_name_override=None)¶

Bases: object

Generates CloudFormation templates from a product blueprint and client configuration.

Resolves all config values at construction time and exposes a single generate() method that returns a complete CFN template dict.

Parameters:

blueprint (ProductBlueprint)
config (Dict[str, Any])
ml_name_override (str)

__init__(blueprint, config, ml_name_override=None)¶

Parameters:

blueprint (ProductBlueprint) – Product blueprint defining resources and SSM outputs.
config (Dict[str, Any]) – Parsed client configuration dictionary.
ml_name_override (str, optional) – Override for the derived ml_name (used by test-deploy).

generate()¶

Generates a CloudFormation template based on the product blueprint and configuration.

Returns:: The generated CloudFormation template.
Return type:: Dict[str, Any]

Utilities¶

HTML Generator¶

HTML documentation generator for ML provisioning.

Provides post-deployment reporting:

generate_deployment_documentation() — CONFIDENTIAL stack report with live resource IDs

ml_provisioner.utils.html_generator.generate_deployment_documentation(config_file, stack_name, region, stack_info, output_path)¶

Generate CONFIDENTIAL HTML report for ML product deployment with live resource IDs.

Parameters:

config_file (str) – Path to configuration YAML file.
stack_name (str) – CloudFormation stack name.
region (str) – AWS region.
stack_info (Dict[str, Any]) – Stack information from describe_stacks.
output_path (str) – Path where HTML file should be saved.

Returns:

Path to generated HTML file.

Return type:

str

Review Report¶

Pre-deployment review report generator for ML provisioning.

Produces an HTML report summarizing what will be deployed:

generate_review_html() — human-friendly HTML with config summary, resources grouped by category, and SSM paths to be published.

ml_provisioner.utils.review_report.generate_review_html(blueprint, config_data, ml_name, output_path)¶

Generate an HTML pre-deployment review report for an ML product.

Parameters:

blueprint (ProductBlueprint) – Loaded tier blueprint.
config_data (Dict[str, Any]) – Raw client configuration dictionary.
ml_name (str) – Constructed ML product name.
output_path (str) – Filesystem path where the HTML file should be saved.

Returns:

Path to the generated HTML file.

Return type:

str

Products¶

Product Loader¶

class ml_provisioner.products.loader.ProductLoader¶

Bases: object

Loads tier blueprint definitions from YAML files in schemas/products/.

The loader reads pure data definition YAML files (starter.yaml, professional.yaml, enterprise.yaml) and maps them into ProductBlueprint dataclasses. No placeholder substitution or override merging is performed here — that is the responsibility of CfnGenerator.

The YAML files are self-contained — each tier file includes all resources from previous tiers plus its own additions. The loader reads the file as-is without any inheritance or merging logic.

VALID_TIERS = {'enterprise', 'professional', 'starter'}¶

list_products(products_dir)¶

Scans the products directory and returns metadata for all available tiers.

Parameters:

products_dir (Path) – Path to the schemas/products/ directory.

Returns:

List of dicts with ‘tier’, ‘version’,: ’description’, and ‘resource_count’ keys, sorted by tier order (starter → professional → enterprise).

Return type:

List[Dict[str, Any]]

Raises:

FileNotFoundError – If the products directory does not exist.

load_product(tier, products_dir)¶

Loads a tier blueprint YAML file and maps it into a ProductBlueprint dataclass.

Parameters:

tier (str) – Tier name — starter, professional, or enterprise.
products_dir (Path) – Path to the schemas/products/ directory.

Returns:

Fully constructed ProductBlueprint object.

Return type:

ProductBlueprint

Raises:

ValueError – If tier is not one of starter, professional, enterprise.
FileNotFoundError – If the tier YAML file does not exist.
ValueError – If the YAML file is empty or malformed.

Product Validator¶

class ml_provisioner.products.validator.ProductValidator¶

Bases: object

Validates a ProductBlueprint against security and structural rules before CloudFormation template generation.

Checks are scoped strictly to resources provisioned by ML Provisioner. Issues are categorized as ERROR (blocks generation) or WARNING (proceeds with warning). See APPLICATION_ARCHITECTURE.md Security Validation section for the full rationale behind each check.

Class Constants:: WARNING (str): Warning level — generation proceeds with warning. ERROR (str): Error level — generation halted unless –force is used. SENSITIVE_IAM_ACTIONS (set): High-risk IAM actions that trigger a warning. HARDCODED_KEY_PATTERN (re.Pattern): Regex to detect plaintext AWS access keys. MIN_LOG_RETENTION_DAYS (int): Minimum CloudWatch log retention in days. BLOCKED_PORTS (set): Ports blocked from 0.0.0.0/0 on endpoint SecurityGroup.

ERROR = 'ERROR'¶

HARDCODED_KEY_PATTERN = re.compile('(?:AKIA|ASIA)[0-9A-Z]{16}')¶

MIN_LOG_RETENTION_DAYS = 90¶

WARNING = 'WARNING'¶

validate(blueprint, config_data)¶

Run all validation checks on the blueprint and client config.

Parameters:

blueprint (ProductBlueprint) – Loaded tier blueprint to validate.
config_data (dict) – Raw client configuration dictionary.

Returns:

List of issues found. Empty list means clean.

Return type:

List[ValidationIssue]

Data Models¶

Product Blueprint¶

class ml_provisioner.models.product.ProductBlueprint(tier, version, description, resources=<factory>, ssm_outputs=<factory>)

Bases: object

Represents a complete tier blueprint loaded from a schemas/products YAML file.

A ProductBlueprint is a pure data definition — it declares what resources exist and their structure. It contains no CloudFormation syntax, no client-specific values, and no placeholder tokens. The CfnGenerator reads this alongside the client config to construct the final CloudFormation template.

Parameters:

tier (str)
version (str)
description (str)
resources (List[ResourceDefinition])
ssm_outputs (List[str])

tier

Tier identifier — starter, professional, or enterprise.

Type:: str

version

Blueprint version string.

Type:: str

description

Human-readable description of the tier.

Type:: str

resources

Ordered list of resource definitions.

Type:: List[ResourceDefinition]

ssm_outputs

List of SSM Parameter Store output keys to store after deployment (e.g., ‘ModelPackageGroupArn’, ‘RepositoryUrl’).

Type:: List[str]

__init__(tier, version, description, resources=<factory>, ssm_outputs=<factory>)

Parameters:

tier (str)
version (str)
description (str)
resources (List[ResourceDefinition])
ssm_outputs (List[str])

Return type:

None

get_resource_by_key(key)

Lookup a resource definition by its logical key.

Parameters:: key (str) – The logical key of the resource to find.
Returns:: The matching resource, or None if not found.
Return type:: Optional[ResourceDefinition]

get_resources_by_type(resource_type)

Return all resource definitions matching a given CloudFormation type.

Parameters:: resource_type (str) – CloudFormation resource type to filter by (e.g., ‘AWS::IAM::Role’).
Returns:: List of matching resource definitions.
Return type:: List[ResourceDefinition]

tier: str

version: str

description: str

resources: List[ResourceDefinition]

ssm_outputs: List[str]

class ml_provisioner.models.product.ResourceDefinition(key, type, properties=<factory>)

Bases: object

Represents a single resource entry in a tier blueprint.

Parameters:

key (str)
type (str)
properties (Dict[str, Any])

key

Logical key identifying the resource (e.g., ‘sagemaker_project’).

Type:: str

type

AWS CloudFormation resource type (e.g., ‘AWS::SageMaker::Project’).

Type:: str

properties

Additional properties from the blueprint (e.g., count, names, versioning, condition, services, encrypt).

Type:: Dict[str, Any]

__init__(key, type, properties=<factory>)

Parameters:

key (str)
type (str)
properties (Dict[str, Any])

Return type:

None

key: str

type: str

properties: Dict[str, Any]

License¶

License Validator¶

class ml_provisioner.license.validator.LicenseValidator¶

Bases: object

Validates product licenses against the AWS Marketplace License Manager.

This class handles the end-to-end license verification workflow, including secure communication with AWS APIs to confirm customer entitlements. To optimize performance and minimize latency, it employs an in-memory caching mechanism, reducing API calls for frequently validated tokens or customers.

product_sku¶

AWS Marketplace product sku to validate against.

Type:: str

region¶

AWS region for the Marketplace API.

Type:: str

license_client¶: Boto3 client for AWS Marketplace License Manager.

key_fingerprint¶: The fingerprint of the public key used for license validation.

entitlement_name¶: The name of the entitlement to check for in the AWS License Manager.

license_cache_file¶

Path to the local cache file.

Type:: str

cache_duration¶

Duration for which the cache is considered valid (hours).

Type:: timedelta

Integration Workflow:

Checks cache for active, unexpired validation result.
If missing/expired, calls CheckoutLicense API.
Validates product SKU and entitlement status.
Updates cache and returns validation status.

__init__()¶

Initialize the LicenseValidator with AWS Marketplace configuration.

Reads configuration from environment variables and sets up the AWS License Manager Client for license validation.

Environment Variables:: MARKETPLACE_PRODUCT_SKU: AWS Marketplace product SKU (required for validation) AWS_REGION: AWS region for Marketplace API (default: ‘us-east-1’)

verify_aws_marketplace_subscription()¶

Validate AWS Marketplace subscription via License Manager CheckoutLicense API.

Returns:: True if valid license, False otherwise
Return type:: bool

CLI Interface¶

class ml_provisioner.cli.MlProvisionerCLI¶

Bases: BaseCLI

CLI for ML Provisioner Tool.

Extends BaseCLI to implement ML-specific actions and arguments. Supports 12 actions grouped by safety level:

Local: validate-config, list-products, show-product, create-policy,
create-prov-template, validate-prov-template, create-review-report
Read-only: show-changes, check-drift
Mutating: test-deploy, deploy-product, delete-product

Includes AWS Marketplace license validation before executing any actions.

__init__()¶: Initialize the ML Provisioner CLI.

add_custom_arguments(parser)¶

Add ML Provisioner-specific CLI arguments.

Parameters:: parser (argparse.ArgumentParser) – The argparse parser to add arguments to.
Return type:: None

create_manager_instance(args)¶

Create a new MlManager instance.

Parameters:: args (argparse.Namespace) – Parsed command-line arguments.
Returns:: Instance initialized with CLI arguments and app config.
Return type:: MlManager

get_actions()¶

Return the list of valid actions for the ML Provisioner CLI.

Returns:: List of action strings
Return type:: List[str]

get_actions_help()¶

Return help text describing all available actions.

Returns:: Formatted help text for all ML Provisioner actions
Return type:: str

get_example_usage()¶

Return example usage instructions.

Detects Docker environment via /.dockerenv or /proc/1/cgroup and adjusts command examples accordingly.

Returns:: Example usage string with Docker or native CLI commands
Return type:: str

get_manager_class()¶

Return the manager class for this CLI.

Returns:: MlManager class reference
Return type:: Type[Any]

is_action_required()¶

Determine if an action argument is required for this CLI.

Returns:: Always True for this CLI
Return type:: bool

requires_force(action)¶

Determine if an action requires the --force flag.

Parameters:: action (str) – The action being performed.
Returns:: True for mutating AWS operations (deploy-product, delete-product).
Return type:: bool

ml_provisioner.cli.main()¶

Entry point for ML Provisioner CLI.

Validates AWS Marketplace license before running the CLI. Exits with code 1 if license validation fails.

Returns:: None