Automatiser Amazon SageMaker à l'aide d'Amazon EventBridge - Amazon SageMaker

Automatiser Amazon SageMaker à l'aide d'Amazon EventBridge

Amazon EventBridge surveille les événements de changement d'état dans Amazon SageMaker. EventBridge vous permet d'automatiser SageMaker et de répondre automatiquement à des événements tels qu'un changement d'état de tâche d'entraînement ou un changement d'état de point de terminaison. Les événements SageMaker sont envoyés à EventBridge en quasi temps réel. Vous pouvez écrire des règles simples pour indiquer quels événements vous intéressent et les actions automatisées à effectuer quand un événement correspond à une règle. Pour obtenir un exemple de création d'une règle, veuillez consulter Planifier un pipeline avec Amazon EventBridge.

Voici des exemples d'actions pouvant être déclenchées automatiquement :

  • Appel d'une fonction AWS Lambda

  • Appel de la fonctionnalité Exécuter la commande d'Amazon EC2

  • Relais de l'événement à Amazon Kinesis Data Streams

  • Activation d'une machine d'état AWS Step Functions

  • Notification d'une rubrique Amazon SNS ou d'une file d'attente AWS SMS

Changement d'état d'une tâche d'entraînement

Indique un changement d'état d'une tâche d'entraînement SageMaker.

Si la valeur de TrainingJobStatus est Failed, l'événement contient le champ FailureReason, qui fournit une description de la raison de l'échec de la tâche d'entraînement.

{ "version": "0", "id": "844e2571-85d4-695f-b930-0153b71dcb42", "detail-type": "SageMaker Training Job State Change", "source": "aws.sagemaker", "account": "123456789012", "time": "2018-10-06T12:26:13Z", "region": "us-east-1", "resources": [ "arn:aws:sagemaker:us-east-1:123456789012:training-job/kmeans-1" ], "detail": { "TrainingJobName": "89c96cc8-dded-4739-afcc-6f1dc936701d", "TrainingJobArn": "arn:aws:sagemaker:us-east-1:123456789012:training-job/kmeans-1", "TrainingJobStatus": "Completed", "SecondaryStatus": "Completed", "HyperParameters": { "Hyper": "Parameters" }, "AlgorithmSpecification": { "TrainingImage": "TrainingImage", "TrainingInputMode": "TrainingInputMode" }, "RoleArn": "arn:aws:iam::123456789012:role/SMRole", "InputDataConfig": [ { "ChannelName": "Train", "DataSource": { "S3DataSource": { "S3DataType": "S3DataType", "S3Uri": "S3Uri", "S3DataDistributionType": "S3DataDistributionType" } }, "ContentType": "ContentType", "CompressionType": "CompressionType", "RecordWrapperType": "RecordWrapperType" } ], "OutputDataConfig": { "KmsKeyId": "KmsKeyId", "S3OutputPath": "S3OutputPath" }, "ResourceConfig": { "InstanceType": "InstanceType", "InstanceCount": 3, "VolumeSizeInGB": 20, "VolumeKmsKeyId": "VolumeKmsKeyId" }, "VpcConfig": { }, "StoppingCondition": { "MaxRuntimeInSeconds": 60 }, "CreationTime": "1583831889050", "TrainingStartTime": "1583831889050", "TrainingEndTime": "1583831889050", "LastModifiedTime": "1583831889050", "SecondaryStatusTransitions": [ ], "Tags": { } } }

Changement d'état de tâche de réglage d'hyperparamètre

Indique un changement d'état d'une tâche de réglage d'hyperparamètre SageMaker.

{ "version": "0", "id": "844e2571-85d4-695f-b930-0153b71dcb42", "detail-type": "SageMaker HyperParameter Tuning Job State Change", "source": "aws.sagemaker", "account": "123456789012", "time": "2018-10-06T12:26:13Z", "region": "us-east-1", "resources": [ "arn:aws:sagemaker:us-east-1:123456789012:tuningJob/x" ], "detail": { "HyperParameterTuningJobName": "016bffd3-6d71-4d3a-9710-0a332b2759fc", "HyperParameterTuningJobArn": "arn:aws:sagemaker:us-east-1:123456789012:tuningJob/x", "TrainingJobDefinition": { "StaticHyperParameters": {}, "AlgorithmSpecification": { "TrainingImage": "trainingImageName", "TrainingInputMode": "inputModeFile", "MetricDefinitions": [ { "Name": "metricName", "Regex": "regex" } ] }, "RoleArn": "roleArn", "InputDataConfig": [ { "ChannelName": "channelName", "DataSource": { "S3DataSource": { "S3DataType": "s3DataType", "S3Uri": "s3Uri", "S3DataDistributionType": "s3DistributionType" } }, "ContentType": "contentType", "CompressionType": "gz", "RecordWrapperType": "RecordWrapper" } ], "VpcConfig": { "SecurityGroupIds": [ "securityGroupIds" ], "Subnets": [ "subnets" ] }, "OutputDataConfig": { "KmsKeyId": "kmsKeyId", "S3OutputPath": "s3OutputPath" }, "ResourceConfig": { "InstanceType": "instanceType", "InstanceCount": 10, "VolumeSizeInGB": 500, "VolumeKmsKeyId": "volumeKeyId" }, "StoppingCondition": { "MaxRuntimeInSeconds": 3600 } }, "HyperParameterTuningJobStatus": "status", "CreationTime": "1583831889050", "LastModifiedTime": "1583831889050", "TrainingJobStatusCounters": { "Completed": 1, "InProgress": 0, "RetryableError": 0, "NonRetryableError": 0, "Stopped": 0 }, "ObjectiveStatusCounters": { "Succeeded": 1, "Pending": 0, "Failed": 0 }, "Tags": {} } }

Changement d'état de tâche de transformation

Indique un changement d'état d'une tâche de transformation par lots SageMaker.

Si la valeur de TransformJobStatus est Failed, l'événement contient le champ FailureReason, qui fournit une description de la raison de l'échec de la tâche d'entraînement.

{ "version": "0", "id": "844e2571-85d4-695f-b930-0153b71dcb42", "detail-type": "SageMaker Transform Job State Change", "source": "aws.sagemaker", "account": "123456789012", "time": "2018-10-06T12:26:13Z", "region": "us-east-1", "resources": ["arn:aws:sagemaker:us-east-1:123456789012:transform-job/myjob"], "detail": { "TransformJobName": "4b52bd8f-e034-4345-818d-884bdd7c9724", "TransformJobArn": "arn:aws:sagemaker:us-east-1:123456789012:transform-job/myjob", "TransformJobStatus": "another status... GO", "FailureReason": "failed why 1", "ModelName": "i am a beautiful model", "MaxConcurrentTransforms": 5, "MaxPayloadInMB": 10, "BatchStrategy": "Strategizing...", "Environment": { "environment1": "environment2" }, "TransformInput": { "DataSource": { "S3DataSource": { "S3DataType": "s3DataType", "S3Uri": "s3Uri" } }, "ContentType": "content type", "CompressionType": "compression type", "SplitType": "split type" }, "TransformOutput": { "S3OutputPath": "s3Uri", "Accept": "accept", "AssembleWith": "assemblyType", "KmsKeyId": "kmsKeyId" }, "TransformResources": { "InstanceType": "instanceType", "InstanceCount": 3 }, "CreationTime": "2018-10-06T12:26:13Z", "TransformStartTime": "2018-10-06T12:26:13Z", "TransformEndTime": "2018-10-06T12:26:13Z", "Tags": {} } }

Changement d'état de point de terminaison

Indique un changement d'état d'un point de terminaison d'inférence en temps réel hébergé SageMaker.

Voici un exemple d'événement avec un point de terminaison à l'état IN_SERVICE.

{ "version": "0", "id": "d2921b5a-b0ad-cace-a8e3-0f159d018e06", "detail-type": "SageMaker Endpoint State Change", "source": "aws.sagemaker", "account": "123456789012", "time": "1583831889050", "region": "us-west-2", "resources": [ "arn:aws:sagemaker:us-west-2:123456789012:endpoint/myendpoint" ], "detail": { "EndpointName": "MyEndpoint", "EndpointArn": "arn:aws:sagemaker:us-west-2:123456789012:endpoint/myendpoint", "EndpointConfigName": "MyEndpointConfig", "ProductionVariants": [ { "DesiredWeight": 1.0, "DesiredInstanceCount": 1.0 } ], "EndpointStatus": "IN_SERVICE", "CreationTime": 1592411992203.0, "LastModifiedTime": 1592411994287.0, "Tags": { } } }

Changement d'état de groupe de fonctions

Indique une modification du champ FeatureGroupStatus ou OfflineStoreStatus d'un groupe de fonctions SageMaker.

{ "version": "0", "id": "93201303-abdb-36a4-1b9b-4c1c3e3671c0", "detail-type": "SageMaker Feature Group State Change", "source": "aws.sagemaker", "account": "123456789012", "time": "2021-01-26T01:22:01Z", "region": "us-east-1", "resources": [ "arn:aws:sagemaker:us-east-1:123456789012:feature-group/sample-feature-group" ], "detail": { "FeatureGroupArn": "arn:aws:sagemaker:us-east-1:123456789012:feature-group/sample-feature-group", "FeatureGroupName": "sample-feature-group", "RecordIdentifierFeatureName": "RecordIdentifier", "EventTimeFeatureName": "EventTime", "FeatureDefinitions": [ { "FeatureName": "RecordIdentifier", "FeatureType": "Integral" }, { "FeatureName": "EventTime", "FeatureType": "Fractional" } ], "CreationTime": 1611624059000, "OnlineStoreConfig": { "EnableOnlineStore": true }, "OfflineStoreConfig": { "S3StorageConfig": { "S3Uri": "s3://offline/s3/uri" }, "DisableGlueTableCreation": false, "DataCatalogConfig": { "TableName": "sample-feature-group-1611624059", "Catalog": "AwsDataCatalog", "Database": "sagemaker_featurestore" } }, "RoleArn": "arn:aws:iam::123456789012:role/SageMakerRole", "FeatureGroupStatus": "Active", "Tags": {} } }

Changement d'état de package de modèles

Indique un changement d'état d'un package de modèles SageMaker.

{ "version": "0", "id": "844e2571-85d4-695f-b930-0153b71dcb42", "detail-type": "SageMaker Model Package State Change", "source": "aws.sagemaker", "account": "123456789012", "time": "2021-02-24T17:00:14Z", "region": "us-east-2", "resources": [ "arn:aws:sagemaker:us-east-2:123456789012:model-package/versionedmp-p-idy6c3e1fiqj/2" ], "source": [ "aws.sagemaker" ], "detail": { "ModelPackageGroupName": "versionedmp-p-idy6c3e1fiqj", "ModelPackageVersion": 2, "ModelPackageArn": "arn:aws:sagemaker:us-east-2:123456789012:model-package/versionedmp-p-idy6c3e1fiqj/2", "CreationTime": "2021-02-24T17:00:14Z", "InferenceSpecification": { "Containers": [ { "Image": "257758044811.dkr.ecr.us-east-2.amazonaws.com/sagemaker-xgboost:1.0-1-cpu-py3", "ImageDigest": "sha256:4dc8a7e4a010a19bb9e0a6b063f355393f6e623603361bd8b105f554d4f0c004", "ModelDataUrl": "s3://sagemaker-project-p-idy6c3e1fiqj/versionedmp-p-idy6c3e1fiqj/AbaloneTrain/pipelines-4r83jejmhorv-TrainAbaloneModel-xw869y8C4a/output/model.tar.gz" } ], "SupportedContentTypes": [ "text/csv" ], "SupportedResponseMIMETypes": [ "text/csv" ] }, "ModelPackageStatus": "Completed", "ModelPackageStatusDetails": { "ValidationStatuses": [], "ImageScanStatuses": [] }, "CertifyForMarketplace": false, "ModelApprovalStatus": "Rejected", "MetadataProperties": { "GeneratedBy": "arn:aws:sagemaker:us-east-2:123456789012:pipeline/versionedmp-p-idy6c3e1fiqj/execution/4r83jejmhorv" }, "ModelMetrics": { "ModelQuality": { "Statistics": { "ContentType": "application/json", "S3Uri": "s3://sagemaker-project-p-idy6c3e1fiqj/versionedmp-p-idy6c3e1fiqj/script-2021-02-24-10-55-15-413/output/evaluation/evaluation.json" } } }, "LastModifiedTime": "2021-02-24T17:00:14Z" } }

Changement d'état d'exécution de pipeline

Indique un changement de l‘état d'exécution d'un pipeline SageMaker.

{ "version": "0", "id": "315c1398-40ff-a850-213b-158f73kd93ir", "detail-type": "SageMaker Model Building Pipeline Execution Status Change", "source": "aws.sagemaker", "account": "123456789012", "time": "2021-03-15T16:10:11Z", "region": "us-east-1", "resources": ["arn:aws:sagemaker:us-east-1:123456789012:pipeline/myPipeline-123", "arn:aws:sagemaker:us-east-1:123456789012:pipeline/myPipeline-123/execution/p4jn9xou8a8s"], "detail": { "pipelineExecutionDisplayName": "SomeDisplayName", "currentPipelineExecutionStatus": "Succeeded", "previousPipelineExecutionStatus": "Executing", "executionStartTime": "2021-03-15T16:03:13Z", "executionEndTime": "2021-03-15T16:10:10Z", "pipelineExecutionDescription": "SomeDescription", "pipelineArn": "arn:aws:sagemaker:us-east-1:123456789012:pipeline/myPipeline-123", "pipelineExecutionArn": "arn:aws:sagemaker:us-east-1:123456789012:pipeline/myPipeline-123/execution/p4jn9xou8a8s" } }

Changement d'état d'étape de pipeline

Indique un changement d'état d'une étape de pipeline SageMaker.

Si le cache contient un résultat, l'événement contient le champ cacheHitResult.

Si la valeur de currentStepStatus est Failed, l'événement contient le champ failureReason qui décrit la raison de l'échec de l'étape.

{ "version": "0", "id": "ea37ccbb-5e2b-05e9-4073-1daazc940304", "detail-type": "SageMaker Model Building Pipeline Execution Step Status Change", "source": "aws.sagemaker", "account": "123456789012", "time": "2021-03-15T16:10:10Z", "region": "us-east-1", "resources": ["arn:aws:sagemaker:us-east-1:123456789012:pipeline/myPipeline-123", "arn:aws:sagemaker:us-east-1:123456789012:pipeline/myPipeline-123/execution/p4jn9xou8a8s"], "detail": { "metadata": { "processingJob": { "arn": "arn:aws:sagemaker:us-east-1:123456789012:processing-job/pipelines-p4jn9xou8a8s-myprocessingstep1-tmgxry49ug" } }, "stepStartTime": "2021-03-15T16:03:14Z", "stepEndTime": "2021-03-15T16:10:09Z", "stepName": "myprocessingstep1", "stepType": "Processing", "previousStepStatus": "Executing", "currentStepStatus": "Succeeded", "pipelineArn": "arn:aws:sagemaker:us-east-1:123456789012:pipeline/myPipeline-123", "pipelineExecutionArn": "arn:aws:sagemaker:us-east-1:123456789012:pipeline/myPipeline-123/execution/p4jn9xou8a8s" } }

Changement d'état d'une image SageMaker

Indique un changement d'état d'une image SageMaker.

{ "version": "0", "id": "cee033a3-17d8-49f8-865f-b9ebf485d9ee", "detail-type": "SageMaker Image State Change", "source": "aws.sagemaker", "account": "123456789012", "time": "2021-04-29T01:29:59Z", "region": "us-east-1", "resources": ["arn:aws:sagemaker:us-west-2:123456789012:image/cee033a3-17d8-49f8-865f-b9ebf485d9ee"], "detail": { "ImageName": "cee033a3-17d8-49f8-865f-b9ebf485d9ee", "ImageArn": "arn:aws:sagemaker:us-west-2:123456789012:image/cee033a3-17d8-49f8-865f-b9ebf485d9ee", "ImageStatus": "Creating", "Version": 1.0, "Tags": {} } }

Changement d'état de version d'image SageMaker

Indique un changement d'état d'une version d'image SageMaker.

{ "version": "0", "id": "07fc4615-ebd7-15fc-1746-243411f09f04", "detail-type": "SageMaker Image Version State Change", "source": "aws.sagemaker", "account": "123456789012", "time": "2021-04-29T01:29:59Z", "region": "us-east-1", "resources": ["arn:aws:sagemaker:us-west-2:123456789012:image-version/07800032-2d29-48b7-8f82-5129225b2a85"], "detail": { "ImageArn": "arn:aws:sagemaker:us-west-2:123456789012:image/a70ff896-c832-4fe8-add6-eba25a0f43e6", "ImageVersionArn": "arn:aws:sagemaker:us-west-2:123456789012:image-version/07800032-2d29-48b7-8f82-5129225b2a85", "ImageVersionStatus": "Creating", "Version": 1.0, "Tags": {} } }

Pour de plus amples informations sur les valeurs d'état et leur signification pour les tâches SageMaker, veuillez consulter les liens suivants :

Pour de plus amples informations, veuillez consulter le Guide de l'utilisateur Amazon EventBridge.

Changement d'état de déploiement de point de terminaison

Important

Les exemples suivants peuvent ne pas fonctionner pour tous les points de terminaison. Pour obtenir la liste des fonctions pouvant exclure votre point de terminaison, veuillez consulter la page Exclusions.

Indique un changement d'état pour un déploiement de point de terminaison. L'exemple suivant montre une mise à jour d'un point de terminaison avec un déploiement canary bleu/vert.

{ "version": "0", "id": "0bd4a141-0a02-9d8a-f977-3924c3fb259c", "detail-type": "SageMaker Endpoint Deployment State Change", "source": "aws.sagemaker", "account": "123456789012", "time": "2021-10-25T01:52:12Z", "region": "us-west-2", "resources": [ "arn:aws:sagemaker:us-west-2:651393343886:endpoint/sample-endpoint" ], "detail": { "EndpointName": "sample-endpoint", "EndpointArn": "arn:aws:sagemaker:us-west-2:651393343886:endpoint/sample-endpoint", "EndpointConfigName": "sample-endpoint-config-1", "ProductionVariants": [ { "VariantName": "AllTraffic", "CurrentWeight": 1, "DesiredWeight": 1, "CurrentInstanceCount": 3, "DesiredInstanceCount": 3 } ], "EndpointStatus": "UPDATING", "CreationTime": 1635195148181, "LastModifiedTime": 1635195148181, "Tags": {}, "PendingDeploymentSummary": { "EndpointConfigName": "sample-endpoint-config-2", "StartTime": Timestamp, "ProductionVariants": [ { "VariantName": "AllTraffic", "CurrentWeight": 1, "DesiredWeight": 1, "CurrentInstanceCount": 1, "DesiredInstanceCount": 3, "VariantStatus": [ { "Status": "Baking", "StatusMessage": "Baking for 600 seconds (TerminationWaitInSeconds) with traffic enabled on canary capacity of 1 instance(s).", "StartTime": 1635195269181, } ] } ] } } }

L'exemple suivant indique un changement d'état pour un déploiement de point de terminaison, qui est mis à jour avec une nouvelle capacité sur une configuration de point de terminaison existante.

{ "version": "0", "id": "0bd4a141-0a02-9d8a-f977-3924c3fb259c", "detail-type": "SageMaker Endpoint Deployment State Change", "source": "aws.sagemaker", "account": "123456789012", "time": "2021-10-25T01:52:12Z", "region": "us-west-2", "resources": [ "arn:aws:sagemaker:us-west-2:651393343886:endpoint/sample-endpoint" ], "detail": { "EndpointName": "sample-endpoint", "EndpointArn": "arn:aws:sagemaker:us-west-2:651393343886:endpoint/sample-endpoint", "EndpointConfigName": "sample-endpoint-config-1", "ProductionVariants": [ { "VariantName": "AllTraffic", "CurrentWeight": 1, "DesiredWeight": 1, "CurrentInstanceCount": 3, "DesiredInstanceCount": 6, "VariantStatus": [ { "Status": "Updating", "StatusMessage": "Scaling out desired instance count to 6.", "StartTime": 1635195269181, } ] } ], "EndpointStatus": "UPDATING", "CreationTime": 1635195148181, "LastModifiedTime": 1635195148181, "Tags": {}, }

Les états de déploiement secondaires suivants sont également disponibles pour les points de terminaison (trouvés dans l'objet VariantStatus).

  • Creating : création d'instances pour la variante de production.

    Exemple de message : "Launching X instance(s)."

  • Deleting : terminaison d'instances pour la variante de production.

    Exemple de message : "Terminating X instance(s)."

  • Updating : mise à jour de la capacité pour la variante de production.

    Exemples de messages : "Launching X instance(s).", "Scaling out desired instance count to X."

  • ActivatingTraffic : activation du trafic pour la variante de production.

    Exemple de message : "Activating traffic on canary capacity of X instance(s)."

  • Baking : période d'attente pour contrôler les alarmes CloudWatch dans la configuration de restauration automatique.

    Exemple de message : "Baking for X seconds (TerminationWaitInSeconds) with traffic enabled on full capacity of Y instance(s)."