API reference - Media Insights on AWS
Data plane APIWorkflow API

API reference

Data plane API

  • GET /

  • POST /create

  • POST /download

  • GET /mediapath/{asset_id}/{workflow_id}

  • GET /metadata

  • DELETE /metadata/{asset_id}

  • DELETE /metadata/{asset_id}/{operator_name}

  • POST /upload

Workflow (control plane) API

  • GET /

  • POST /system/configuration

  • POST /workflow

  • GET /workflow/configuration/{Name}

  • POST /workflow/execution

  • GET /workflow/execution/asset/{AssetId}

  • GET /workflow/execution/status/{Status}

  • DELETE /workflow/execution/{Id}

  • GET /workflow/execution/{Id}

  • POST /workflow/operation

  • DELETE /workflow/operation/{Name}

  • POST /workflow/stage

  • DELETE /workflow/stage/{Name}

  • DELETE /workflow/{Name}

Data plane API

Create an asset in the data plane from a json input composed of the input key and bucket of the object:

POST /create

Body: 

{
    "Input": {
        "S3Bucket": "{somenbucket}",
        "S3Key": "{somekey}"
    }
}

Returns:

A dictionary mapping of the asset ID and the new location of the media object.

Retrieve metadata for an asset:

GET /metadata/{asset_id}

Returns: All asset metadata. If the result provides a cursor then you can get the next page by specifying the cursor:

GET /metadata/{asset_id}?cursor={cursor}

Add operation metadata for an asset:

POST /metadata/{asset_id}

Body: 

{
    "OperatorName": "{some_operator}",
    "Results": "{json_formatted_results}"
}

Retrieve the metadata that a specific operator created from an asset:

GET /metadata/{asset_id}/{operator_name}

Get version information:

GET /version

Returns:

A dictionary containing the version of the Media Insights on AWS framework and the version of the data plane API. Since it is possible for the Media Insights on AWS framework to be released without any API changes, these two versions can be different. The Media Insights on AWS framework and its APIs are versioned according to Semantic Versioning rules. Under this scheme, version numbers and the way they change convey meaning about backwards compatibility.

For example, if the Media Insights on AWS framework was version v2.0.4 and the workflow API was version 2.0.0, then this would return the following response: 

 b'{"ApiVersion":"2.0.0","FrameworkVersion":"v2.0.4"}'

Workflow API

Add a new system configuration parameter or update an existing Media Insights on AWS system configuration parameter

POST /system/configuration

Body: 

{
    "Name": "ParameterName",
    "Value": "ParameterValue"
}

Supported parameters:

MaxConcurrentWorkflows - Sets the maximum number of workflows that are allowed to run concurrently. Any new workflows that are added after MaxConcurrentWorkflows is reached are placed on a queue until capacity is freed by completing workflows. Use this to help avoid throttling in service API calls from workflow operators. This setting is checked each time the WorkflowSchedulerLambda is run and may take up to 60 seconds to take effect.

Returns:

Nothing

Raises:

  • 200: The system configuration was set successfully

  • 400: Bad Request

  • 500: Internal server error - an input value is not valid

Get the current Media Insights on AWS system configuration:

GET /system/configuration

Returns:

A list of dictionaries containing the current Media Insights on AWS system configuration key-value pairs.

Raises:

  • 200: The system configuration was returned successfully

  • 500: Internal server error

Create a workflow from a list of existing stages

A workflow is a pipeline of stages that are started sequentially to transform and extract metadata for a set of MediaType objects. Each stage must contain either a Next key indicating the next stage to run or an End key indicating it is the last stage.

POST /workflow

Body: 

{
    "Name": string,
    "StartAt": string - name of starting stage,
    "Stages": {
        "stage-name": {
            "Next": "string - name of next stage"
        },
        ...,
        "stage-name": {
            "End": true
        }
    }
}

Returns:

A dictionary mapping keys to the corresponding workflow created including the AWS resources used to run each stage.

Raises:

  • 200: The workflow was created successfully

  • 400: Bad Request - one of the input stages was not found or was not valid

  • 500: Internal server error

List all workflow definitions

GET /workflow

Returns:

A list of workflow definitions.

Raises:

  • 200: All workflows returned successfully

  • 500: Internal server error

Get a workflow configuration object by name

GET /workflow/configuration/{Name}

Returns:

A dictionary containing the workflow configuration.

Raises:

  • 200: All workflows returned successfully

  • 404: Not found

  • 500: Internal server error

Run a workflow

The Body contains the name of the workflow to run, and at least one input media type within the media object. A dictionary of stage configuration objects can be passed in to override the default configuration of the operations within the stages.

POST /workflow/execution

Body: 

{
"Name":"Default",
"Input": media-object
"Configuration": {
    {
    "stage-name": {
        "Operations": {
            "SplitAudio": {
               "Enabled": True,
               "MediaTypes": {
                   "Video": True/False,
                   "Audio": True/False,
                   "Frame": True/False
               }
           },
       },
   }
   ...
   }
}

Returns:

A dictionary describing the workflow run properties and the WorkflowExecutionId that can be used as the Id in /workflow/execution/{Id} API requests.

Raises:

  • 200: The workflow run was created successfully

  • 400: Bad Request - the input workflow was not found or was not valid

  • 500: Internal server error

List all workflow runs

GET /workflow/execution

Returns:

A list of workflow runs.

Raises:

  • 200: List returned successfully

  • 500: Internal server error

Get workflow runs by asset ID

GET /workflow/execution/asset/{AssetId}

Returns:

A list of dictionaries containing the workflow runs matching the AssetId.

Raises:

  • 200: List returned successfully

  • 404: Not found

  • 500: Internal server error

Get all workflow runs with the specified status

GET /workflow/execution/status/{Status}

Returns:

A list of dictionaries containing the workflow runs with the requested status.

Raises:

  • 200: All workflows returned successfully

  • 404: Not found

  • 500: Internal server error

Delete a workflow run

DELETE /workflow/execution/{Id}

Returns:

Nothing.

Raises:

  • 200: Workflow run deleted successfully

  • 404: Not found

  • 500: Internal server error

Get a workflow run by ID

GET /workflow/execution/{Id}

Returns:

A dictionary containing the workflow run.

Raises:

  • 200: The workflow run details returned successfully

  • 404: Not found

  • 500: Internal server error

Create a new operation

POST /workflow/operation

Generates an operation state machine using the operation Lambda function(s) provided.

Creates a singleton operator stage that can be used to run the operator as a single-operator stage in a workflow.

Operators can be sync or async. Sync operators complete before returning control to the invoker, while async operators return control to the invoker immediately after the operation is successfully initiated. Async operators require an additional monitoring task to check the status of the operation.

For more information on how to implement Lambda functions to be used in Media Insights on AWS operators, refer to Implementing a new operator in Media Insights on AWS.

Body: 

{
    "Name":"operation-name",
    "Type": ["Async"|"Sync"],
    "Configuration" : {
            "MediaType": "Video",
            "Enabled:": True,
            "configuration1": "value1",
            "configuration2": "value2",
            ...
        }
    "StartLambdaArn":arn,
    "MonitorLambdaArn":arn,
    "SfnExecutionRole": arn
    }

Returns:

A dictionary mapping keys to the corresponding operation:

{
    "Name": string,
    "Type": ["Async"|"Sync"],
    "Configuration" : {
        "MediaType": "Video|Frame|Audio|Text|...",
        "Enabled:": boolean,
        "configuration1": "value1",
        "configuration2": "value2",
        ...
    }
    "StartLambdaArn":arn,
    "MonitorLambdaArn":arn,
    "StateMachineExecutionRoleArn": arn,
    "StateMachineAsl": ASL-string
    "StageName": string
}

Raises:

  • 200: The operation and stage was created successfully

  • 400: Bad Request

  • one of the input lambdas was not found

  • one or more of the required input keys is missing

  • an input value is not valid

  • 409: Conflict

  • 500: Internal server error

Important

Do not try to create more than 35 new operators via /workflow/operation. The IAM inline policy used in media-insights-stack.yaml to grant InvokeFunction permission to the StepFunctionRole for new operators will exceed the maximum length allowed by IAM if users create more than 35 operators (+/- 1).

For more information, refer to the comments in this commit: aws-solutions/media-insights-on-aws@451ec2e.

Sample command that shows how to create an operator from /workflow/operation on the command line: 

OPERATOR_NAME="op1"
WORKFLOW_API_ENDPOINT="https://tvplry8vn3.execute-api.us-west-2.amazonaws.com/api/"
START_ARN="arn:aws:lambda:us-west-2:773074507832:function:mie03d-OperatorFailedLambda-11W1LAY0CWCUZ"
MONITOR_ARN="arn:aws:lambda:us-west-2:773074507832:function:mie03d-OperatorFailedLambda-11W1LAY0CWCUZ"
REGION="us-west-2"
awscurl --region ${REGION} -X POST -H "Content-Type: application/json" -d '{"StartLambdaArn": "'${START_ARN}'", "Configuration": {"MediaType": "Video", "Enabled": true}, "Type": "Async", "Name": "'${OPERATOR_NAME}'", "MonitorLambdaArn": "'${MONITOR_ARN}}'"' ${WORKFLOW_API_ENDPOINT}workflow/operation;

List all defined operators

GET /workflow/operation

Returns:

A list of operation definitions.

Raises:

  • 200: All operations returned successfully

  • 500: Internal server error

Delete an operation

DELETE /workflow/operation/{Name}

Raises:

  • 200: Operation deleted successfully

  • 500: Internal server error

Get an operation definition by name

GET /workflow/operation/{Name}

Returns:

A dictionary containing the operation definition.

Raises:

  • 200: All operations returned successfully

  • 404: Not found

  • 500: Internal server error

Create a stage state machine from a list of existing operations

A stage is a set of operations that are grouped so they can be run in parallel. When the stage is run as part of a workflow, operations within a stage are run as branches in a parallel AWS Step Functions state. The generated state machines status is tracked by the workflow engine control plane during the run.

An optional configuration for each operator in the stage can be input to override the default configuration for the stage.

POST /workflow/stage 

{
"Name":"stage-name",
"Operations": ["operation-name1", "operation-name2", ...]
}
Returns:
A dict mapping keys to the corresponding stage created including the ARN of the state machine created.

{
“Name”: string, “Operations”: [

“operation-name1”, “operation-name2”, …
], “Configuration”: {

“operation-name1”: operation-configuration-object1, “operation-name2”: operation-configuration-object1, …
} “StateMachineArn”: ARN-string

“Name”: “TestStage”, “Operations”: [

“TestOperator”
], “Configuration”: {

“TestOperator”: {
“MediaType”: “Video”, “Enabled”: true
}
}, “StateMachineArn”: “arn:aws:states:us-west-2:526662735483:stateMachine:TestStage”
}

Raises:

  • 200: The stage was created successfully

  • 400: Bad Request - one of the input state machines was not found or was not valid

  • 409: Conflict

  • 500: Internal server error

List all stage definitions

GET /workflow/stage

Returns:

A list of operation definitions.

Raises:

  • 200: All operations returned successfully

  • 500: Internal server error

Delete a stage

DELETE /workflow/stage/{Name}

Returns:

Nothing.

Raises:

  • 200: Stage deleted successfully

  • 404: Not found

  • 500: Internal server error

Get a stage definition by name

GET /workflow/stage/{Name}

Returns:

A dictionary containing the stage definition.

Raises:

  • 200: Stage definition was returned successfully

  • 404: Not found

  • 500: Internal server error

Delete a workflow

DELETE /workflow/{Name}

Returns:

Nothing.

Raises:

  • 200: Workflow deleted successfully

  • 404: Not found

  • 500: Internal server error

Get a workflow definition by name

GET /workflow/{Name}

Returns:

A dictionary containing the workflow definition.

Raises:

  • 200: Workflow definition returned successfully

  • 404: Not found

  • 500: Internal server error

Get version information

GET /version

Returns:

A dictionary containing the version of the Media Insights on AWS framework and the version of the workflow API. Since it is possible for the Media Insights on AWS framework to be released without any API changes, these two versions can be different. The Media Insights on AWS framework and its APIs are versioned according to Semantic Versioning rules. Under this scheme, version numbers and the way they change convey meaning about backwards compatibility.

For example, if the Media Insights on AWS framework was version v2.0.4 and the workflow API was version 2.0.0, then this would return the following response: 

b'{"ApiVersion":"2.0.0","FrameworkVersion":"v2.0.4"}'