AWS IoT
Developer Guide

Devices and Jobs

device communication with Jobsusing the MQTT protocolusing HTTP SigV4using HTTP TLS
device communication with Jobs

Devices can communicate with Jobs through one of three methods:

  • MQTT

  • HTTP SigV4

  • HTTP TLS

using the MQTT protocol

Communication between the Jobs service and your devices can occur over the MQTT protocol. Devices subscribe to MQTT topics to be notified of new jobs and to receive responses from the Jobs service. Devices publish on MQTT topics to query or update the state of a job execution. Each device has its own general MQTT topic. For more information about publishing and subscribing to MQTT topics, see Message Broker for AWS IoT

Note

You must use the correct endpoint when communicating with Jobs via MQTT. To find it, use the DescribeEndpoint command. For example, using the CLI:

aws iot describe-endpoint --endpoint-type iot:Data

you will get a result similar to:

{ "endpointAddress": "a1b2c3d4e5f6g7.iot.us-west-2.amazonaws.com" }

With this method, your device uses its device-specific certificate and private key to authenticate with the Jobs service.

Devices can:

  • Be notified whenever a job execution is added or removed from the list of pending job executions for a device by subscribing to the $aws/things/thing-name/jobs/notify MQTT topic, where thing-name is the name of the thing associated with the device.

  • Be notified when the next pending job execution has changed by subscribing to the $aws/things/thing-name/jobs/notify-next MQTT topic, where thing-name is the name of the thing associated with the device.

  • Update the status of a job execution by calling the UpdateJobExecution API.

  • Query the status of a job execution by calling the DescribeJobExecution API.

  • Retrieve a list of pending job executions by calling the GetPendingJobExecutions API.

  • Retrieve the next pending job execution by calling the DescribeJobExecution API with jobId $next.

  • Get and start the next pending job execution by calling the StartNextPendingJobExecution API.

The Jobs service publishes success and failure messages on an MQTT topic formed by appending accepted or rejected to the topic used to make the request. For example, if a request message is published on the $aws/things/myThing/jobs/get topic, the Jobs service publishes success messages on the $aws/things/myThing/jobs/get/accepted topic and publishes rejected messages on the $aws/things/myThing/jobs/get/rejected topic.

using HTTP SigV4

Communication between the Jobs service and your devices can occur over HTTP SigV4 on port 443. This is the method used by the AWS SDKs and CLI. For more information about those tools, see AWS CLI Command Reference: iot-jobs-data or AWS SDKs and Tools and refer to the IotJobsDataPlane section for your preferred language.

Note

You must use the correct endpoint when communicating with Jobs via HTTP SigV4 or using an AWS SDK or CLI IotJobsDataPlane command. To find it, use the DescribeEndpoint command. For example, using the CLI:

aws iot describe-endpoint --endpoint-type iot:Jobs

you will get a result similar to:

{ "endpointAddress": "a1b2c3d4e5f6g7.jobs.iot.us-west-2.amazonaws.com" }

With this method of communication, your device will use IAM credentials to authenticate with the Jobs service.

The following commands are available using this method:

  • DescribeJobExecution

    aws iot-jobs-data describe-job-execution ...

  • GetPendingJobExecutions

    aws iot-jobs-data get-pending-job-executions ...

  • StartNextPendingJobExecution

    aws iot-jobs-data start-next-pending-job-execution ...

  • UpdateJobExecution

    aws iot-jobs-data update-job-execution ...

using HTTP TLS

Communication between the Jobs service and your devices can occur over HTTP TLS on port 8443 using a third-party software client that supports this protocol.

Note

You must use the correct endpoint when communicating with Jobs via HTTP TLS. To find it, use the DescribeEndpoint command. For example, using the CLI:

aws iot describe-endpoint --endpoint-type iot:Jobs

you will get a result similar to:

{ "endpointAddress": "a1b2c3d4e5f6g7.jobs.iot.us-west-2.amazonaws.com" }

With this method, your device uses X509 certificate based authentication (for example, using its device-specific certificate and private key.)

The following commands are available using this method:

  • DescribeJobExecution

  • GetPendingJobExecutions

  • StartNextPendingJobExecution

  • UpdateJobExecution

Programming Devices to Work with Jobs

The examples in this section use MQTT to illustrate how a device works with Jobs. Alternatively, you could use the corresponding API or CLI commands if you want. For these examples, we assume a device called MyThing will subscribe to the following MQTT topics:

  • $aws/things/MyThing/jobs/notify (or $aws/things/MyThing/jobs/notify-next)

  • $aws/things/MyThing/jobs/get/accepted

  • $aws/things/MyThing/jobs/get/rejected

  • $aws/things/MyThing/jobs/jobId/get/accepted

  • $aws/things/MyThing/jobs/jobId/get/rejected

Device Workflow

In general, there are two ways a device can handle the jobs it is given to execute.

Option A: Get the next jobOption B: Pick from available jobs
Option A: Get the next job
  1. When a device first comes online, it should subscribe to the device's notify-next topic.

  2. Call the DescribeJobExecution MQTT API with jobId $next to get the next job, its job document, and other details, including any state saved in statusDetails.

  3. Call the UpdateJobExecution MQTT API to update the job status. Or, to combine this and the previous step in one call, the device can call StartNextPendingJobExecution.

  4. Optionally, you can add a step timer by setting a value for stepTimeoutInMinutes when you call either UpdateJobExecution or StartNextPendingJobExecution.

  5. Perform the actions specified by the job document using the UpdateJobExecution MQTT API to report on the progress of the job.

  6. Continue to monitor the job execution by calling the DescribeJobExecution MQTT API with this jobId. If the job execution is canceled or deleted while the device is running the job, the device should be capable of recovering to a valid state.

  7. Call the UpdateJobExecution MQTT API when finished with the job to update the job status and report success or failure.

  8. Because this job's execution status has been changed to a terminal state, the next job available for execution (if any) will change. The device is notified that the next pending job execution has changed. At this point, the device should continue as described in step 2.

If the device remains online, it will continue to receive a notifications of the next pending job execution, including its JobExecutionData, whenever it completes a job or when a new pending job execution is added. When this occurs, the device continues as described in step 2.

Option B: Pick from available jobs
  1. When a device first comes online, it should subscribe to the thing's notify topic.

  2. Call the GetPendingJobExecutions MQTT API to get a list of pending job executions.

  3. If the list contains one or more job executions, pick one.

  4. Call the DescribeJobExecution MQTT API to get the job document and other details, including any state saved in statusDetails.

  5. Call the UpdateJobExecution MQTT API to update the job status. If the includeJobDocument field is set to true in this command, the device can skip the previous step and retrieve the job document at this point.

  6. Optionally, you can add a step timer by setting a value for stepTimeoutInMinutes when you call UpdateJobExecution.

  7. Perform the actions specified by the job document using the UpdateJobExecution MQTT API to report on the progress of the job.

  8. Continue to monitor the job execution by calling the DescribeJobExecution MQTT API with this jobId. If the job execution is canceled or deleted while the device is running the job, the device should be capable of recovering to a valid state.

  9. Call the UpdateJobExecution MQTT API when finished with the job to update the job status and to report success or failure.

If the device remains online, it will be notified of all pending job executions whenever a new pending job execution becomes available. When this occurs, the device can continue as described in step 2.

If the device is unable to execute the job, it should call the UpdateJobExecution MQTT API to update the job status to REJECTED.

Starting a New Job

new job notificationmore info (15)
new job notification

When a new job is created, Jobs publishes a message on the $aws/things/thing-name/jobs/notify topic for each target device.

more info (15)

The message contains the following information:

{ "timestamp":1476214217017, "jobs":{ "QUEUED":[{ "jobId":"0001", "queuedAt":1476214216981, "lastUpdatedAt":1476214216981, "versionNumber" : 1 }] } }

The device receives this message on the '$aws/things/thingName/jobs/notify' topic when the job execution is queued.

get job informationmore info (16)
get job information

To get more information about a job execution, the device calls the DescribeJobExecution MQTT API with the includeJobDocument field set to true.

more info (16)

If the request is successful, Jobs publishes a message on the $aws/things/MyThing/jobs/0023/get/accepted topic:

{ "clientToken" : "client-001", "timestamp" : 1489097434407, "execution" : { "approximateSecondsBeforeTimedOut": number, "jobId" : "023", "status" : "QUEUED", "queuedAt" : 1489097374841, "lastUpdatedAt" : 1489097374841, "versionNumber" : 1, "jobDocument" : { < contents of job document > } } }

Note

If the request fails, Jobs publishes a message on the $aws/things/MyThing/jobs/0023/get/rejected topic.

The device now has the job document, that it can interpret to perform the remote operations for the job. If the job document contains an Amazon S3 presigned URL, the device can use that URL to download any required files for the job.

Report Job Execution Status

update execution statusmore info (17)
update execution status

As the device is executing the job, it can call the UpdateJobExecution MQTT API to update the status of the job execution.

more info (17)

For example, a device can update the job execution status to IN_PROGRESS by publishing the following message on the $aws/things/MyThing/jobs/0023/update topic:

{ "status":"IN_PROGRESS", "statusDetails": { "progress":"50%" }, "expectedVersion":"1", "clientToken":"client001" }

Jobs responds by publishing a message to the $aws/things/MyThing/jobs/0023/update/accepted or $aws/things/MyThing/jobs/0023/update/rejected topic:

{ "clientToken":"client001", "timestamp":1476289222841 }

The device can combine the two previous requests by calling StartNextPendingJobExecution, that gets and starts the next pending job execution and allows the device to update the job execution status. This request also returns the job document when there is a job execution pending.

If the job contains a TimeoutConfig, the in progress timer will start running. You can also set a step timer for a specific job execution by setting a value for stepTimeoutInMinutes when you call UpdateJobExecution. The step timer applies only to the job execution that you update, and you can set a new value for this timer each time you update a job execution. You can also create a step timer when you call StartNextPendingJobExecution. If the job execution remains in the IN_PROGRESS status for longer than the step timer interval, it will fail and switch to the terminal TIMED_OUT status. The step timer has no effect on the in progress timer that you set when you create a job.

The status field can be set to IN_PROGRESS, SUCCEEDED, FAILED. You cannot update the status of a job execution that is already in a terminal state.

report execution completedmore info (18)
report execution completed

When the device is finished executing the job, it calls the UpdateJobExecution MQTT API. If the job was successful, set status to SUCCEEDED and, in statusDetails in the message payload, add other information about the job as name/value pairs. The in progress and step timers end when job execution is complete.

more info (18)

For example:

{ "status":"SUCCEEDED", "statusDetails": { "progress":"100%" }, "expectedVersion":"2", "clientToken":"client-001" }

If the job was not successful, set status to FAILED and, in statusDetails, add information about the error that occurred:

{ "status":"FAILED", "statusDetails": { "errorCode":"101", "errorMsg":"Unable to install update" }, "expectedVersion":"2", "clientToken":"client-001" }

Note

The statusDetails attribute can contain any number of name/value pairs.

When Jobs receives this update, it publishes a message on the $aws/things/MyThing/jobs/notify topic to indicate the job execution is complete:

{ "timestamp":1476290692776, "jobs":{} }

Additional Jobs

additional jobsmore info (19)
additional jobs

If there are other job executions pending for the device, they are included in the message published to $aws/things/MyThing/jobs/notify.

more info (19)

For example:

{ "timestamp":1476290692776, "jobs":{ "QUEUED":[{ "jobId":"0002", "queuedAt":1476290646230, "lastUpdatedAt":1476290646230 }], "IN_PROGRESS":[{ "jobId":"0003", "queuedAt":1476290646230, "lastUpdatedAt":1476290646230 }] } }

Jobs Notifications

AWS IoT Jobs publishes MQTT messages to reserved topics when jobs are pending or when the first job execution in the list changes. Devices can keep track of pending jobs by subscribing to these topics.

Job notifications are published to MQTT topics as JSON payloads. There are two kinds of notifications:

  1. ListNotification: A ListNotification contains a list of no more than 10 pending job executions. The job executions in this list have status values of either IN_PROGRESS or QUEUED. They are sorted by status (IN_PROGRESS job executions before QUEUED job executions) and then by the times when they were queued.

    A ListNotification is published whenever one of the following criteria is met.

    1. A new job execution is queued or changes to a non-terminal status (IN_PROGRESS or QUEUED).

    2. An old job execution changes to a terminal status (FAILED, SUCCEEDED, CANCELED, TIMED_OUT, REJECTED, or REMOVED).

  2. NextNotification: A NextNotification contains summary information about the one job execution that is next in the queue.

    A NextNotification is published whenever the first job execution in the list changes.

    1. A new job execution is added to the list as QUEUED, and it is the first one in the list.

    2. The status of an existing job execution that was not the first one in the list changes from QUEUED to IN_PROGRESS and becomes the first one in the list. (This happens when there are no other IN_PROGRESS job executions in the list or when the job execution whose status changes from QUEUED to IN_PROGRESS was queued earlier than any other IN_PROGRESS job execution in the list.)

    3. The status of the job execution that is first in the list changes to a terminal status and is removed from the list.

For more information about publishing and subscribing to MQTT topics, see Message Broker for AWS IoT

Note

Notifications are not available when you use HTTP SigV4 or HTTP TLS to commmunicate with jobs.

job pendingmore info (11)
job pending

AWS IoT Jobs publishes a message on an MQTT topic when a job is added to or removed from the list of pending job executions for a thing, or the first job execution in the list changes:

  • $aws/things/thingName/jobs/notify

  • $aws/things/thingName/jobs/notify-next

more info (11)

The messages contain the following example payloads:

$aws/things/thingName/jobs/notify:

{ "timestamp" : 10011, "jobs" : { "IN_PROGRESS" : [ { "jobId" : "other-job", "queuedAt" : 10003, "lastUpdatedAt" : 10009, "executionNumber" : 1, "versionNumber" : 1 } ], "QUEUED" : [ { "jobId" : "this-job", "queuedAt" : 10011, "lastUpdatedAt" : 10011, "executionNumber" : 1, "versionNumber" : 0 } ] } }

$aws/things/thingName/jobs/notify-next:

{ "timestamp" : 10011, "execution" : { "jobId" : "other-job", "status" : "IN_PROGRESS", "queuedAt" : 10009, "lastUpdatedAt" : 10009, "versionNumber" : 1, "executionNumber" : 1, "jobDocument" : {"c":"d"} } }

Possible job execution status values are QUEUED, IN_PROGRESS, FAILED, SUCCEEDED, CANCELED, TIMED_OUT, REJECTED, and REMOVED.

The following series of examples shows the kinds of notifications that are published to each topic as job executions are created and change from one state to another.

First, one job, called job1, is created. This notification is published to the jobs/notify topic:

{ "timestamp": 1517016948, "jobs": { "QUEUED": [ { "jobId": "job1", "queuedAt": 1517016947, "lastUpdatedAt": 1517016947, "executionNumber": 1, "versionNumber": 1 } ] } }

This notification is published to the jobs/notify-next topic:

{ "timestamp": 1517016948, "execution": { "jobId": "job1", "status": "QUEUED", "queuedAt": 1517016947, "lastUpdatedAt": 1517016947, "versionNumber": 1, "executionNumber": 1, "jobDocument": { "operation": "test" } } }

When another job is created (job2), this notification is published to the jobs/notify topic:

{ "timestamp": 1517017192, "jobs": { "QUEUED": [ { "jobId": "job1", "queuedAt": 1517016947, "lastUpdatedAt": 1517016947, "executionNumber": 1, "versionNumber": 1 }, { "jobId": "job2", "queuedAt": 1517017191, "lastUpdatedAt": 1517017191, "executionNumber": 1, "versionNumber": 1 } ] } }

A notification is not published to the jobs/notify-next topic, because the next job in the queue (job1) has not changed. When job1 starts to execute, its status changes to IN_PROGRESS. No notifications are published because the list of jobs and the next job in the queue have not changed.

When a third job (job3) is added, this notification is published to the jobs/notify topic:

{ "timestamp": 1517017906, "jobs": { "IN_PROGRESS": [ { "jobId": "job1", "queuedAt": 1517016947, "lastUpdatedAt": 1517017472, "startedAt": 1517017472, "executionNumber": 1, "versionNumber": 2 } ], "QUEUED": [ { "jobId": "job2", "queuedAt": 1517017191, "lastUpdatedAt": 1517017191, "executionNumber": 1, "versionNumber": 1 }, { "jobId": "job3", "queuedAt": 1517017905, "lastUpdatedAt": 1517017905, "executionNumber": 1, "versionNumber": 1 } ] } }

A notification is not published to the jobs/notify-next topic because the next job in the queue is still job1.

When job1 is complete, its status changes to SUCCEEDED, and this notification is published to the jobs/notify topic:

{ "timestamp": 1517186269, "jobs": { "QUEUED": [ { "jobId": "job2", "queuedAt": 1517017191, "lastUpdatedAt": 1517017191, "executionNumber": 1, "versionNumber": 1 }, { "jobId": "job3", "queuedAt": 1517017905, "lastUpdatedAt": 1517017905, "executionNumber": 1, "versionNumber": 1 } ] } }

At this point, job1 has been removed from the queue, and the next job to be executed is job2. This notification is published to the jobs/notify-next topic:

{ "timestamp": 1517186269, "execution": { "jobId": "job2", "status": "QUEUED", "queuedAt": 1517017191, "lastUpdatedAt": 1517017191, "versionNumber": 1, "executionNumber": 1, "jobDocument": { "operation": "test" } } }

If job3 needs to begin executing before job2 (which is not recommended), the status of job3 can be changed to IN_PROGRESS. If this happens, job2 is no longer next in the queue, and this notification is published to the jobs/notify-next topic:

{ "timestamp": 1517186779, "execution": { "jobId": "job3", "status": "IN_PROGRESS", "queuedAt": 1517017905, "startedAt": 1517186779, "lastUpdatedAt": 1517186779, "versionNumber": 2, "executionNumber": 1, "jobDocument": { "operation": "test" } } }

No notification is published to the jobs/notify topic because no job has been added or removed.

If the device rejects job2 and updates its status to REJECTED, this notification is published to the jobs/notify topic:

{ "timestamp": 1517189392, "jobs": { "IN_PROGRESS": [ { "jobId": "job3", "queuedAt": 1517017905, "lastUpdatedAt": 1517186779, "startedAt": 1517186779, "executionNumber": 1, "versionNumber": 2 } ] } }

If job3 (which is still in progress) is force deleted, this notification is published to the jobs/notify topic:

{ "timestamp": 1517189551, "jobs": {} }

At this point, the queue is empty. This notification is published to the jobs/notify-next topic:

{ "timestamp": 1517189551 }