DrawManager: Improved Task Scheduler #75120

New Issue

Jeroen Bakker · 2020-03-27T09:56:54+01:00

Jeroen Bakker commented

2020-03-27 09:56:54 +01:00

The current task scheduler when converting meshes into GPU Batches can be improved.

Current Implementation

Every object is handled in serial. When a mesh object needs to create GPUBatches the generation of these batches are done in parallel.
After all batches for an object has been created the next object is processed.

Some extractors are scheduled without threading as they are really fast, but the current implementation schedules these with high priority.

Target Implementation

A single Task Scheduler is created that handles all objects in parallel.
There are some hurdles to take into account:

lines_loose is a sub buffer of lines It can only run after the lines buffer has been created.
objects can share meshes. We should not schedule a mesh twice.
Some extractors are scheduled without threading as they are really fast. In the target situation they would be scheduled with normal priority.
mesh_render_data_create could also be run as a task.
Use a different control flow where a task can have related sub-tasks. These subtasks will be calculated after the parent has finished. There are multiple ways how to implement this
Parent task knows its children and adds them to the task scheduler when the parent task is finished Child task has a parent task id and the scheduler checks if that task is already completed. This has some draw backs as chained tasks will get lower prio.

Benefits

The algorithm of some extractors cannot be spliced into chunks and therefor don't schedule optimal. In current implementation they are scheduled with a higher priority to be sure they are completed as first. In the target implementation these tasks will be scheduled better as we schedule all these extractors in a single task scheduler.
In scenes with multiple characters the performance would increase as more work can be scheduled in parallel and less syncs are needed.

The current task scheduler when converting meshes into GPU Batches can be improved. ## Current Implementation Every object is handled in serial. When a mesh object needs to create GPUBatches the generation of these batches are done in parallel. After all batches for an object has been created the next object is processed. * Some extractors are scheduled without threading as they are really fast, but the current implementation schedules these with high priority. ![Screenshot from 2020-03-27 13-28-10.png](https://archive.blender.org/developer/F8430362/Screenshot_from_2020-03-27_13-28-10.png) ## Target Implementation A single Task Scheduler is created that handles all objects in parallel. There are some hurdles to take into account: * `lines_loose` is a sub buffer of `lines` It can only run after the lines buffer has been created. * objects can share meshes. We should not schedule a mesh twice. * Some extractors are scheduled without threading as they are really fast. In the target situation they would be scheduled with normal priority. * `mesh_render_data_create` could also be run as a task. * Use a different control flow where a task can have related sub-tasks. These subtasks will be calculated after the parent has finished. There are multiple ways how to implement this **Parent task knows its children and adds them to the task scheduler when the parent task is finished** Child task has a parent task id and the scheduler checks if that task is already completed. This has some draw backs as chained tasks will get lower prio. ### Benefits * The algorithm of some extractors cannot be spliced into chunks and therefor don't schedule optimal. In current implementation they are scheduled with a higher priority to be sure they are completed as first. In the target implementation these tasks will be scheduled better as we schedule all these extractors in a single task scheduler. * In scenes with multiple characters the performance would increase as more work can be scheduled in parallel and less syncs are needed.

Jeroen Bakker commented

2020-03-27 09:56:54 +01:00

Added subscriber: @Jeroen-Bakker