Integration ExamplesΒΆ
Examples showing how the ML Provisioner integrates with other provisioners as an input consumer, and how its SSM Parameter Store outputs are consumed by downstream automation and MLOps tooling.
Table of ContentsΒΆ
Discovering ML Provisioner SSM OutputsΒΆ
After a successful deploy-product, the ML Provisioner publishes resource identifiers to
SSM Parameter Store under the path /ml/{ml_name}/. The available parameters depend on
the tier deployed.
Retrieve All Outputs for a DeploymentΒΆ
ML_NAME=globalbank-prod-c001-us-west-2-demand-forecasting-ml
aws ssm get-parameters-by-path \
--path /ml/${ML_NAME}/ \
--recursive \
--region us-west-2 \
--query 'Parameters[*].{Name:Name,Value:Value}' \
--output table
SSM Output Reference by TierΒΆ
SSM Path |
Tier |
Description |
|---|---|---|
|
All |
SageMaker Model Registry ARN |
|
All (CodeCommit only) |
CodeCommit model-build repo clone URL |
|
Professional, Enterprise |
S3 artifacts bucket name |
|
Professional, Enterprise |
CloudWatch Dashboard name |
|
Enterprise |
KMS key ARN for encryption |
|
Enterprise |
CloudWatch compliance log group name |
|
Enterprise |
VPC Endpoint ID for |
|
Enterprise |
VPC Endpoint ID for |
|
Enterprise |
VPC Endpoint ID for |
|
Enterprise |
VPC Endpoint ID for |
|
Enterprise (standalone mode only) |
Endpoint security group ID |
VPC Provisioner IntegrationΒΆ
The ML Provisioner enterprise tier requires a VPC. When using vpc_source: ssm, the VPC
Provisioner publishes VPC and subnet IDs to SSM Parameter Store, and the ML Provisioner
reads them at deployment time via CFN parameters.
VPC Provisioner SSM Outputs (inputs to ML Provisioner)ΒΆ
# Verify VPC Provisioner has published the required parameters before deploying
VPC_NAME=globalbank-prod-c001-us-west-2-vpc
aws ssm get-parameter \
--name /vpc/${VPC_NAME}/VPCId \
--region us-west-2 \
--query 'Parameter.Value' --output text
aws ssm get-parameter \
--name /vpc/${VPC_NAME}/PrivateSubnetIds \
--region us-west-2 \
--query 'Parameter.Value' --output text
ML Provisioner Configuration (vpc_source: ssm)ΒΆ
vpc_integration:
mode: standalone # or sg-provisioner
vpc_source: ssm
vpc_parameter_store_path: /vpc/globalbank-prod-c001-us-west-2-vpc/VPCId
subnet_parameter_store_path: /vpc/globalbank-prod-c001-us-west-2-vpc/PrivateSubnetIds
route_table_ids: []
CloudFormation resolves the VPC ID and subnet IDs at stack creation time by reading the SSM parameters directly. No manual copying of IDs is needed.
SG Provisioner IntegrationΒΆ
When vpc_integration.mode: sg-provisioner, the ML Provisioner reads a security group ID
from SSM Parameter Store (published by the SG Provisioner) and attaches it to the VPC
endpoints instead of creating its own security group.
SG Provisioner SSM Output (input to ML Provisioner)ΒΆ
# Verify SG Provisioner has published the security group ID before deploying
SG_NAME=globalbank-prod-c001-us-west-2-sg
aws ssm get-parameter \
--name /sg/${SG_NAME}/app/SecurityGroupId \
--region us-west-2 \
--query 'Parameter.Value' --output text
ML Provisioner Configuration (mode: sg-provisioner)ΒΆ
vpc_integration:
mode: sg-provisioner
vpc_source: ssm
vpc_parameter_store_path: /vpc/globalbank-prod-c001-us-west-2-vpc/VPCId
subnet_parameter_store_path: /vpc/globalbank-prod-c001-us-west-2-vpc/PrivateSubnetIds
sg_parameter_store_path: /sg/globalbank-prod-c001-us-west-2-sg/app/SecurityGroupId
route_table_ids: []
In sg-provisioner mode, the SecurityGroupId SSM output is not published by the ML
Provisioner (the SG already exists and is managed separately).
Provisioner Deployment OrderΒΆ
VPC Provisioner and SG Provisioner must be deployed before ML Provisioner:
1. vpc-provisioner β publishes /vpc/.../VPCId and /vpc/.../PrivateSubnetIds
2. sg-provisioner β publishes /sg/.../SecurityGroupId
3. ml-provisioner β reads from VPC and SG SSM paths, deploys ML stack
Consuming SSM Outputs via AWS CLIΒΆ
Once the ML stack is deployed, MLOps automation scripts can retrieve outputs directly from SSM Parameter Store.
Retrieve Model Registry ARNΒΆ
ML_NAME=globalbank-prod-c001-us-west-2-demand-forecasting-ml
aws ssm get-parameter \
--name /ml/${ML_NAME}/ModelPackageGroupArn \
--region us-west-2 \
--query 'Parameter.Value' --output text
Retrieve S3 Artifacts Bucket Name (Professional, Enterprise)ΒΆ
aws ssm get-parameter \
--name /ml/${ML_NAME}/BucketName \
--region us-west-2 \
--query 'Parameter.Value' --output text
Retrieve CodeCommit Repository URLΒΆ
aws ssm get-parameter \
--name /ml/${ML_NAME}/RepositoryUrl \
--region us-west-2 \
--query 'Parameter.Value' --output text
Retrieve KMS Key ARN (Enterprise)ΒΆ
aws ssm get-parameter \
--name /ml/${ML_NAME}/KmsKeyArn \
--region us-west-2 \
--query 'Parameter.Value' --output text
Consuming SSM Outputs via Boto3ΒΆ
MLOps scripts and automation pipelines can retrieve ML Provisioner outputs programmatically.
Retrieve All ML Provisioner OutputsΒΆ
python3 << 'EOF'
import boto3
def get_ml_outputs(ml_name: str, region: str) -> dict:
ssm = boto3.client('ssm', region_name=region)
paginator = ssm.get_paginator('get_parameters_by_path')
outputs = {}
for page in paginator.paginate(Path=f'/ml/{ml_name}/', Recursive=True):
for param in page['Parameters']:
key = param['Name'].split('/')[-1]
outputs[key] = param['Value']
return outputs
ml_name = 'globalbank-prod-c001-us-west-2-demand-forecasting-ml'
outputs = get_ml_outputs(ml_name, 'us-west-2')
model_registry_arn = outputs.get('ModelPackageGroupArn')
bucket_name = outputs.get('BucketName')
repository_url = outputs.get('RepositoryUrl')
kms_key_arn = outputs.get('KmsKeyArn')
print(f"model_registry_arn: {model_registry_arn}")
print(f"bucket_name: {bucket_name}")
print(f"repository_url: {repository_url}")
print(f"kms_key_arn: {kms_key_arn}")
EOF
Example output:
model_registry_arn: arn:aws:sagemaker:us-west-2:123456789012:model-package-group/globalbank-prod-c001-us-west-2-demand-forecasting-ml-models
bucket_name: globalbank-prod-c001-us-west-2-demand-forecasting-ml-artifacts
repository_url: https://git-codecommit.us-west-2.amazonaws.com/v1/repos/globalbank-prod-c001-us-west-2-demand-forecasting-ml-model-build
kms_key_arn: arn:aws:kms:us-west-2:123456789012:key/95997aeb-aad2-466a-adec-852e51609002
Retrieve a Single ParameterΒΆ
python3 << 'EOF'
import boto3
def get_ml_parameter(ml_name: str, key: str, region: str) -> str:
ssm = boto3.client('ssm', region_name=region)
response = ssm.get_parameter(
Name=f'/ml/{ml_name}/{key}'
)
return response['Parameter']['Value']
ml_name = 'globalbank-prod-c001-us-west-2-demand-forecasting-ml'
model_registry_arn = get_ml_parameter(ml_name, 'ModelPackageGroupArn', 'us-west-2')
print(f"mlname: {ml_name}")
print(f"model_registry_arn: {model_registry_arn}")
EOF
Example output:
mlname: globalbank-prod-c001-us-west-2-demand-forecasting-ml
model_registry_arn: arn:aws:sagemaker:us-west-2:123456789012:model-package-group/globalbank-prod-c001-us-west-2-demand-forecasting-ml-models
Register a Model Package Using ML Provisioner OutputsΒΆ
import boto3
def register_model(ml_name: str, region: str, model_uri: str, description: str):
ssm = boto3.client('ssm', region_name=region)
sm = boto3.client('sagemaker', region_name=region)
group_arn = ssm.get_parameter(
Name=f'/ml/{ml_name}/ModelPackageGroupArn'
)['Parameter']['Value']
response = sm.create_model_package(
ModelPackageGroupName=group_arn,
ModelPackageDescription=description,
InferenceSpecification={
'Containers': [{'Image': model_uri}],
'SupportedContentTypes': ['text/csv'],
'SupportedResponseMIMETypes': ['text/csv']
},
ModelApprovalStatus='PendingManualApproval'
)
return response['ModelPackageArn']