Executions can optionally be started using high priority using the priority field.
Or when using the command-line
# Run the Step named StepName with high priority
$ vh exec run <StepName> --priority
Availability
Queue priority can be enabled on on-premises and cloud virtual machine execution environments. Kubernetes and Slurm execution environments do not currently support queue priority through Valohai. The priority tools on the web app are available if the execution environment supports prioritization.
If setting priority using the command line tool or an API call, an error response is provided if the execution environment does not support priority or enable its support.
Execution priority
Created executions are placed in the execution environment’s queue according to their priority, to the last place in its priority class. A newly created high priority execution will be run after all previously created high priority executions have been processed, first-in first-out (FIFO). In practice, high priority executions form a second FIFO queue in front of the normal priority executions’ queue. If no high priority executions are scheduled, a priority queue behaves like a standard FIFO queue.
As long as the execution is waiting in the queue, its priority can be changed in the web app. If the priority of an execution is changed while it is waiting in the queue, it will be rescheduled in the queue as if it was a new execution, placed last in its respective priority.
Executions that are already running will not be canceled or delayed if a higher priority job is added to the queue.
Usage considerations
Enabling queue priority is recommended for on-premises, dedicated hosted execution environments with a mostly constant capacity, allowing division of jobs into high priority jobs that should be processed right away and lower priority jobs that will be processed when there is spare capacity.
Scalable execution environments can quickly increase capacity to process the entire queue and decrease capacity all the way down to nothing when there is nothing more to do in the queue, making prioritisation less important as spare capacity is handled through scaling.
Priority queueing makes the queue service perform more work compared to standard first-in first-out queueing when scheduling new executions. High capacity environments with many simultaneous executions in the queue may have reduced performance or require allocating more resources for queue services.