Queue hopping
A job stays in a SUBMITTED state, waiting to be processed, until the queue that
you submit it to has available resources. To prevent long wait times, you can configure
your job to automatically move to another queue after a set amount of time. This is
called queue hopping.
Keep the following definitions in mind with queue hopping.
- Submission queue
-
The queue that you originally submit a job to is its submission queue.
- Destination queue
-
The queue that your job moves to when it hops queues is its destination queue.
- Wait time
-
The amount of time your job waits in its submission queue until it can hop to its destination queue.
- Hop
-
A job hops when it moves from its submission queue to its destination queue after its wait time elapses. A job that moves queues is also referred to as a hopped job.
A common use case for queue hopping is to move jobs from a reserved queue to an on-demand
queue during a spike in usage. For example, you might automatically move jobs that sit
in a SUBMITTED state for longer than 10 minutes.
Note
When you set up queue hopping from a reserved queue to an on-demand queue, MediaConvert bills you according to the queue type that your job ultimately runs in. If your job runs in a reserved queue, MediaConvert doesn't bill you separately for the job because the cost is already covered by what you pay for your reserved queue. If your job runs in an on-demand queue, MediaConvert bills you for the job at the on-demand rate.
Topics
Set up queue hopping
When you set up queue hopping, you specify the submission queue, the wait time, and the destination queue. Typically, the submission queue is a reserved queue and the destination queue is an on-demand queue. The following tabs show different options for setting up queue hopping.
View job history
When a job hops queues, the values for the settings queue and
priority remain how you set them when you created the job. You
can see the values for the job's post-hop destination and queue priority. The
following tabs provide two options for viewing a job's history and queue
priority.
Billing tags for hopped jobs
If you use billing tags on your jobs and set your billing tags source to Queue, the charges for your jobs are always listed under the tags for the submission queue. To track how much you were billed for a job that hops queues, you can set the billing tags source to Job. For more information about using tags to sort your AWS bill, see Setting up AWS Elemental MediaConvert resources for cost allocation through tagging.
Note
Cost allocation based on queue only applies to jobs that run in on-demand queues. When your submission queue is a reserved queue and your job hops to an on-demand queue, the charges for that on-demand job appear in your cost allocation report. If you don't put tags on your reserved queue, those charges appear in the report unsorted.
Listing hopped jobs
When you view your job, MediaConvert displays the queue that you submitted
your job. For example, if you submit a job to Queue1, and it hops
to Queue2, that job appears in lists that are filtered for
Queue1. It doesn't appear in lists filtered for
Queue2.
Set job priority for hopped jobs
When you set up a job for queue hopping, you can specify the priority for the job in the new queue. If you don't specify a new priority, the job keeps the priority number from its submission queue.
If you use different guidelines for choosing the values for priority
between the two queues, make sure to specify a new priority value for the job in the
destination queue.
For information about setting the job's priority within the submission queue, see Job priority.
The following tabs provide different options for setting the priority of a hopped job.
Specify accelerated transcoding for hopped jobs
To reduce the transcoding time for certain jobs, use accelerated transcoding. In most cases, you submit accelerated jobs to on-demand queues, because reserved queues cannot run accelerated jobs. However, you can submit a job with Accelerated transcoding set Preferred to a reserved queue. When you do, if the job hops to an on-demand queue, it will run with acceleration enabled. For more information about accelerated transcoding, see Accelerated transcoding in the MediaConvert User Guide.
The following tabs provide different options for setting accelerated transcoding.
Queue hopping behavior with paused queues
Jobs don't hop from a queue while it's paused, but they hop freely to paused queues.
Hopping from a paused queue
Jobs don't hop from a queue while it's paused. Queue hopping behavior depends on how long the queue is paused. Consider these two situations:
You submit a job to a queue, pause the queue for longer than the queue hopping wait time, then reactivate it.
In this situation, whether the job hops depends on where the job is in the queue. If there are any jobs ahead of it in the queue, the job hops to the destination queue. If there are no jobs ahead of it in the queue, MediaConvert processes it without hopping.
For example, imagine that you submit a job to Queue1 with a wait time of 15
minutes and a destination of Queue2. Five minutes after you submit
the job, you pause Queue1. Ten minutes later, the job remains in
Queue1. Half an hour after that, you activate
Queue1. At that time, there are no jobs ahead of it in
Queue1, so the job runs from Queue1.
You submit a job to a queue. You pause the queue and then reactivate it before the wait time passes.
In this situation, the time that the queue is paused doesn't affect queue hopping at all.
For example, imagine that you submit a job to Queue1 with a wait time of 15
minutes and a destination of Queue2. Five minutes after you submit
the job, you pause Queue1. One minute later, you reactivate
Queue1. Nine minutes later (15 minutes after you submitted the
job), there are still jobs ahead of it in the queue. Therefore, the job hops to
Queue2, as though you hadn't paused the queue.
Hopping to a paused queue
Jobs hop freely from active queues to paused queues. For example, imagine that you submit
a job to Queue1 with a wait time of 15 minutes and a destination of
Queue2. Then, five minutes after you submit the job, you pause
Queue2. Ten minutes later (15 minutes after you submit the
job), the job hops to Queue2 and remains there, waiting until you
activate the queue.
