Cycles OpenCL: Proposal Select better base features to reduce recompilation #62252

New Issue

Jeroen Bakker · 2019-03-06T11:52:31+01:00

Jeroen Bakker commented

2019-03-06 11:52:31 +01:00

Currently there are many features that will trigger a recompilation. This design task will reevaluate the current setup and come with a better approach.

Current situation

NODE_GROUP_LEVEL_*: currently all the scenes of our benchmark always uses NODE_GROUP_LEVEL_MAX during foreground and background rendering. As these are always selected we could remove this, or come with a better set for eacht NODE_GROUP_LEVEL_?
NODE_FEATURE_*: There are some compile directives and node features that are linked. like NODE_FEATURE_VOLUME and use_volume. NODE_FEATURE_HAIR and use_hair.
all features are detected based on the existing of nodes or other data in the scene.

Experiments

We have done several experiments by enabling different features by default. We used the BMW, Classroom, Pavilion, Fishy cat and Barbershop scenes for benchmarking.
The reference hardware is AMD Ryzen 1700 with 16GB ram and a Vega Frontier Edition.
We tested these configurations during background rendering.

Subsurface and volumes did not increased the render time noticeable, but most of the compilation time is concentrates around those two features. Enable other features will increase the compilation times for volume and subsurface.

	Current situation	Proposal	Enable hair	Enable motion	Enable subsurface	Enable volume	Enable subsurface + volume	Proposed + subsurface
BMW
Compile times	12.25	15.61	13.19	12.53	16.92	21.84	25.73	18.71
Render times	1:35	1:37	1:46	1:46	1:37	1:39	1:40	1:40

Proposal

Enable use_transparent by default
Enable use_shadow_tricks by default
Enable use_transparent by default
Enable use_principled by default
Enable use_denoising by default

		experimental = false;
		max_nodes_group = NODE_GROUP_LEVEL_MAX;
		nodes_features = 0;
		use_hair = false;
		use_object_motion = false;
		use_camera_motion = false;
		use_baking = false;
		use_subsurface = false;
		use_volume = false;
		use_integrator_branched = false;
		use_patch_evaluation = false;
		use_transparent = true;
		use_shadow_tricks = true;
		use_principled = true;
		use_denoising = true;
		use_shader_raytrace = false;
		use_true_displacement = false;
		use_background_light = false;

	Current	Proposal
BMW
Compile times	12.25	15.61
Render times	1:35	1:37
CLASSROOM
Compile times	12.14	15.42
Render times	3:38	3:28
BARBERSHOP
Compile times	30.94	34.15
Render times	9:21	9:22
PAVILION
Compile times	12.63	15.37
Render times	7:50	7:47
FISHY_CAT
Compile times	13.5	19.48
Render times	3:14	3:16

User experience

With this new setup the kernels will be less recompiled when building a scene (or doing the lighting). Kernels will still need to be compiled when

switching between path tracing and branched path
enable hair
enable volumetric
enable subsurface
enable subdivision patch evaluation
use nodes that use raytracing techniques ( NODE_BEVEL and NODE_AMBIENT_OCCLUSION)
use camera or object motion (background rendering only)

I would only suggest to enable these defaults in foreground and background rendering. The selected features are not noticeable in compilation time and rendertime.

Currently there are many features that will trigger a recompilation. This design task will reevaluate the current setup and come with a better approach. **Current situation** * `NODE_GROUP_LEVEL_*`: currently all the scenes of our benchmark always uses `NODE_GROUP_LEVEL_MAX` during foreground and background rendering. As these are always selected we could remove this, or come with a better set for eacht `NODE_GROUP_LEVEL_?` * `NODE_FEATURE_*`: There are some compile directives and node features that are linked. like `NODE_FEATURE_VOLUME` and `use_volume`. `NODE_FEATURE_HAIR` and `use_hair`. * all features are detected based on the existing of nodes or other data in the scene. **Experiments** We have done several experiments by enabling different features by default. We used the BMW, Classroom, Pavilion, Fishy cat and Barbershop scenes for benchmarking. The reference hardware is AMD Ryzen 1700 with 16GB ram and a Vega Frontier Edition. We tested these configurations during background rendering. Subsurface and volumes did not increased the render time noticeable, but most of the compilation time is concentrates around those two features. Enable other features will increase the compilation times for volume and subsurface. | | Current situation | Proposal| Enable hair| Enable motion| Enable subsurface| Enable volume| Enable subsurface + volume| Proposed + subsurface | -- | -- | -- | -- | -- | -- | -- | -- | -- | |BMW | | | | | | | | | |Compile times |12.25| 15.61| 13.19| 12.53| 16.92| 21.84| 25.73| 18.71| |Render times| 1:35| 1:37| 1:46| 1:46| 1:37| 1:39| 1:40| 1:40 | **Proposal** * Enable `use_transparent` by default * Enable `use_shadow_tricks` by default * Enable `use_transparent` by default * Enable `use_principled` by default * Enable `use_denoising` by default ``` experimental = false; max_nodes_group = NODE_GROUP_LEVEL_MAX; nodes_features = 0; use_hair = false; use_object_motion = false; use_camera_motion = false; use_baking = false; use_subsurface = false; use_volume = false; use_integrator_branched = false; use_patch_evaluation = false; use_transparent = true; use_shadow_tricks = true; use_principled = true; use_denoising = true; use_shader_raytrace = false; use_true_displacement = false; use_background_light = false; ``` | | Current | Proposal | -- | -- | -- | |**BMW**| |Compile times |12.25 | 15.61 |Render times |1:35 | 1:37 |**CLASSROOM**| |Compile times |12.14 | 15.42 |Render times |3:38 | 3:28 |**BARBERSHOP**| |Compile times |30.94 |34.15 |Render times |9:21 |9:22 |**PAVILION** | |Compile times |12.63 |15.37 |Render times |7:50 |7:47 |**FISHY_CAT** | |Compile times |13.5 |19.48 |Render times |3:14 |3:16 **User experience** With this new setup the kernels will be less recompiled when building a scene (or doing the lighting). Kernels will still need to be compiled when * switching between path tracing and branched path * enable hair * enable volumetric * enable subsurface * enable subdivision patch evaluation * use nodes that use raytracing techniques ( `NODE_BEVEL` and `NODE_AMBIENT_OCCLUSION`) * use camera or object motion (background rendering only) I would only suggest to enable these defaults in foreground and background rendering. The selected features are not noticeable in compilation time and rendertime.

Jeroen Bakker self-assigned this 2019-03-06 11:52:31 +01:00

Jeroen Bakker commented

2019-03-06 11:52:31 +01:00

Added subscriber: @Jeroen-Bakker

Jeroen Bakker commented

2019-03-06 12:06:46 +01:00

I would propose to implement this as a last step. In the upcoming month I want to reorganize the kernels and that might influence the choices made here. I don't think that the proposed choices stated above will be effected, only we might want to add subsurface or volume when we are able to get their compile times better.

Brecht Van Lommel commented

2019-03-06 16:22:26 +01:00

Added subscriber: @brecht

Brecht Van Lommel commented

2019-03-06 16:22:26 +01:00

Interesting stats. I agree we can enable the proposed features always.

There is some possibility to optimize some of the more expensive features individually so that maybe they can become part of the default too, but we'll have to see if there is enough time.

Interesting stats. I agree we can enable the proposed features always. There is some possibility to optimize some of the more expensive features individually so that maybe they can become part of the default too, but we'll have to see if there is enough time.

noki paike commented

2019-03-06 19:36:24 +01:00

Added subscriber: @nokipaike

noki paike commented

2019-03-06 19:36:24 +01:00

This comment was removed by @nokipaike

*This comment was removed by @nokipaike*

Brecht Van Lommel commented

2019-03-06 20:15:53 +01:00

@nokipaike, please stay on topic.

noki paike commented

2019-03-06 20:43:55 +01:00

@brecht
I apologize, I thought they were related and that they could be good insights from a user perspective that could also be useful via development to do "the possibility of excluding individual objects or individual functions not strictly necessary" from the rendering.

@brecht I apologize, I thought they were related and that they could be good insights from a user perspective that could also be useful via development to do "the possibility of excluding individual objects or individual functions not strictly necessary" from the rendering.

Jeroen Bakker commented

2019-03-07 11:27:23 +01:00

Closed as duplicate of #61751

Jeroen Bakker closed this issue

2019-03-07 11:27:23 +01:00

Jeroen Bakker commented

2019-03-07 11:29:03 +01:00

Changed status from 'Duplicate' to: 'Open'

Jeroen Bakker reopened this issue

2019-03-07 11:29:03 +01:00

Jeroen Bakker commented

2019-03-15 11:31:10 +01:00