Page MenuHome

Idea: Task scheduling using random sampling
Open, NormalPublic


Just an idea. Right now we have job & task priorities, and the task with the highest priority of the job with the highest priority is run first. As a result, jobs with higher priorities will completely starve jobs with lower priorities.

To give an example: if I want to render something on Flamenco, but with low priority to not disturb the Secret Lair renders, effectively I have to wait until the Secret Lair is completely done. Instead, I would rather see Flamenco pick up a task of a low-priority job every now and then. That way low-prio jobs are worked on, just slower than high-prio jobs.

I think this can be done using nonuniform random sampling. By using the job priority to determine the probability that a task of that job is chosen, there will still be a small probability for low-prio tasks to be executed. This of course should only be done for the job selection; tasks within a job should keep their relative execution order.

We could even take it one step further, and take job age into account. That way older jobs are more likely to be worked on than new jobs. Not sure if that's worth the effort, though.