Page MenuHome

Tile stealing glitches with adaptive sampling
Closed, ResolvedPublicBUG

Description

System Information
Operating system: Windows-10-10.0.19041-SP0 64 Bits
Graphics card: GeForce GTX 1080 with Max-Q Design/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 457.09

Blender Version
Broken: version: 2.92.0 Alpha, branch: master, commit date: 2020-11-02 14:36, hash: rB7872bcafa0ff
Caused by rB517ff40b124b: Cycles: Implement tile stealing to improve CPU+GPU rendering performance

Short description of error
Adaptive sampling introduces random errors on the stolen tiles

Exact steps for others to reproduce the error

Open the following file


Enable GPU+CPU mode in the preferences

Hit F12
If it's not very visible the first time, try again

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Tagging as 2.92 so it doesn't get on the way of the 2.91 release coordination.

@Dalai Felinto (dfelinto) Seeing the amount of reports this has generated, maybe we should disable the feature automatically in case of collision. For example, if hybrid is enabled, disable adaptive for now.

@Dalai Felinto (dfelinto) Seeing the amount of reports this has generated, maybe we should disable the feature automatically in case of collision. For example, if hybrid is enabled, disable adaptive for now.

This might be a useful option because when rendering on a farm with Afanasy (in our case) we have to load Blender and disable the CPU to save the settings. It would anyway a good idea to add a simple button in the Render Settings to disable/enable CPU for hybrid rendering saved in the scene.

@Dalai Felinto (dfelinto) Seeing the amount of reports this has generated, maybe we should disable the feature automatically in case of collision. For example, if hybrid is enabled, disable adaptive for now.

This might be a useful option because when rendering on a farm with Afanasy (in our case) we have to load Blender and disable the CPU to save the settings. It would anyway a good idea to add a simple button in the Render Settings to disable/enable CPU for hybrid rendering saved in the scene.

No more options needed. Just a temporary check that disables something that is BROKEN. Until it gets fixed.

I applied Lukas' patch and it works perfectly. It just needs to get reviewed and accepted.

@Dalai Felinto (dfelinto) Seeing the amount of reports this has generated, maybe we should disable the feature automatically in case of collision. For example, if hybrid is enabled, disable adaptive for now.

This might be a useful option because when rendering on a farm with Afanasy (in our case) we have to load Blender and disable the CPU to save the settings. It would anyway a good idea to add a simple button in the Render Settings to disable/enable CPU for hybrid rendering saved in the scene.

No more options needed. Just a temporary check that disables something that is BROKEN. Until it gets fixed.

From a production pov it should be an option to decide to render on a farm with or without CPU. I would recommend to add a feature in the Render Setting and not Preferences only to disable the hybrid mode. Because setting to GPU will take the Preferences settings only locally. And this is not really useful eg. when you do distributed rendering on machines with GPU turned on but some use CPU and some not.

Summary: A button below the CPU/GPU menu will override the local System settings and set it to GPU only. Turned off will enable the local System settings as usual.

@Michael Klein (Renderbricks) Please don't post feature requests on the bug tracker. Besides, you can already configure this via Python and/or the render farm should provide a proper setup with their preferences.

@Michael Klein (Renderbricks) Please don't post feature requests on the bug tracker. Besides, you can already configure this via Python.

Understood! I just responded to Daniel. Where can I post a feature request being reviewed by the dev team?

This page lists where you can leave feedback or make feature proposals.

@Dalai Felinto (dfelinto) Seeing the amount of reports this has generated, maybe we should disable the feature automatically in case of collision. For example, if hybrid is enabled, disable adaptive for now.

This might be a useful option because when rendering on a farm with Afanasy (in our case) we have to load Blender and disable the CPU to save the settings. It would anyway a good idea to add a simple button in the Render Settings to disable/enable CPU for hybrid rendering saved in the scene.

No more options needed. Just a temporary check that disables something that is BROKEN. Until it gets fixed.

From a production pov it should be an option to decide to render on a farm with or without CPU. I would recommend to add a feature in the Render Setting and not Preferences only to disable the hybrid mode. Because setting to GPU will take the Preferences settings only locally. And this is not really useful eg. when you do distributed rendering on machines with GPU turned on but some use CPU and some not.

Summary: A button below the CPU/GPU menu will override the local System settings and set it to GPU only. Turned off will enable the local System settings as usual.

This is wrong. The renderfarm itself should configure the rendering units based on each virtual machine´s capabilities. This does not belong to the blend file since the same blend file could end up being rendered in different hardware.

@Dalai Felinto (dfelinto) Seeing the amount of reports this has generated, maybe we should disable the feature automatically in case of collision. For example, if hybrid is enabled, disable adaptive for now.

This might be a useful option because when rendering on a farm with Afanasy (in our case) we have to load Blender and disable the CPU to save the settings. It would anyway a good idea to add a simple button in the Render Settings to disable/enable CPU for hybrid rendering saved in the scene.

No more options needed. Just a temporary check that disables something that is BROKEN. Until it gets fixed.

From a production pov it should be an option to decide to render on a farm with or without CPU. I would recommend to add a feature in the Render Setting and not Preferences only to disable the hybrid mode. Because setting to GPU will take the Preferences settings only locally. And this is not really useful eg. when you do distributed rendering on machines with GPU turned on but some use CPU and some not.

Summary: A button below the CPU/GPU menu will override the local System settings and set it to GPU only. Turned off will enable the local System settings as usual.

This is wrong. The renderfarm itself should configure the rendering units based on each virtual machine´s capabilities. This does not belong to the blend file since the same blend file could end up being rendered in different hardware.

I see. Yes, makes sense. My bad. Thanks for the time.

@Dalai Felinto (dfelinto) Seeing the amount of reports this has generated, maybe we should disable the feature automatically in case of collision. For example, if hybrid is enabled, disable adaptive for now.

Please do not disable adaptive sampling, it would be much better if the tile stealing is disabled instead, given it improves render times barely in certain tile sizes assuming that the combo is the culprit.

Also this might create undesirable situations on render farms.

You're right it's better to disable tile stealing. Although there's supposedly a proper fix in the way?

@Dalai Felinto (dfelinto) Seeing the amount of reports this has generated, maybe we should disable the feature automatically in case of collision. For example, if hybrid is enabled, disable adaptive for now.

Please do not disable adaptive sampling, it would be much better if the tile stealing is disabled instead, given it improves render times barely in certain tile sizes assuming that the combo is the culprit.

Also this might create undesirable situations on render farms.

You're right it's better to disable tile stealing. Although there's supposedly a proper fix in the way?

According to user Alaska who replied to my missing tile report, this is the fix https://developer.blender.org/D9445

You're right it's better to disable tile stealing. Although there's supposedly a proper fix in the way?

According to user Alaska who replied to my missing tile report, this is the fix https://developer.blender.org/D9445

I am wondering why the fix isn't implemented in 2.92 yet? It's an important fix because actually Adaptive Sampling with hybrid rendering is some kind of useless but a very important and powerful feature.

Forgive my noobness but if they were commited two days ago should the experimental builds not have picked up the patch already ?

These glitches are still visible in today's experimental build. On linux using 2x NVIDIA cards and a threadripper.

I tested yesterday with the experimental build and also today. In my case the problem with the tile stealing is solved when using adaptive sampling with optix GPU +CPU rendering.
I´m on windows and have a Geforce RTX 3090.

Also works for me on Windows, TR1950x and 2080.

In my case it breaks using OPTIX but actually works on CUDA.

Not sure - if the remaining issue can be confirmed to be Optix-specific, @Patrick Mours (pmoursnv) might know more.

I double checked on Windows 10 (Latest) and Linux (Ubuntu 20.04) with Nvidia 460.x on both Optix and CUDA now. I also tried with multiple files. Seems to work. Someone else can confirm an Optix Issue here?

mi (homeworker) added a comment.EditedJan 15 2021, 10:11 AM

Work for me with an Asus laptop GeForce GTX 950M + Intel Core i7 integrated GPU. Both CUDA and OptiX Win 10 - driver 461.09 01/07/2021

Not sure - if the remaining issue can be confirmed to be Optix-specific, @Patrick Mours (pmoursnv) might know more.

My latest test showed up that the bug seems to be related to the type of GPU (and maybe the type of CPU?).

Laptop with RTX 2070 Super and i7-10875H

  • 2.83 LTS
  • 2.91
  • 2.92 Alpha without the fix
  • CUDA and OptiX mode

Result: No broken tiles anymore in all versions. Not driver related. I made all tests with the latest Studio and Game driver.

My second test was running on a Win 10 system with GTX 1080Ti and Ryzen Threadripper 1950X. All versions will render broken tiles. The latest 2.92 Beta has fixed the issue.

Excuse my question, but which version is needed to test the tile stealing feature?
I've a nvidia RTX 3090 here and an AMD 3970X CPU with 32 cores. In my tests the "Candidate" version of Blender 2.92 has nearly no difference in rendering speed, or just marginal better.

Is the tile stealing feature implemented in the current 2.92 Candidate version or do I expect too much of this option?

Excuse my question, but which version is needed to test the tile stealing feature?
I've a nvidia RTX 3090 here and an AMD 3970X CPU with 32 cores. In my tests the "Candidate" version of Blender 2.92 has nearly no difference in rendering speed, or just marginal better.

Is the tile stealing feature implemented in the current 2.92 Candidate version or do I expect too much of this option?

From what I watch here 2.92 RC is using tile stealing. When the GPU has finished it will definitely take over the CPU tiles. From my logic it should be like this: If you have a slow CPU but a very fast GPU there is maybe not much of a difference between GPU only and hybrid mode. Because the GPU might solve the whole render while the CPU is still working on the first tiles. But if you have a very fast CPU there should be a difference. How fast is the AMD 3970X compared to the RTX 3090? I have a 16 core Threadripper and GTX 1080Ti which are equal in speed. I will do a test.

Excuse my question, but which version is needed to test the tile stealing feature?
I've a nvidia RTX 3090 here and an AMD 3970X CPU with 32 cores. In my tests the "Candidate" version of Blender 2.92 has nearly no difference in rendering speed, or just marginal better.

Is the tile stealing feature implemented in the current 2.92 Candidate version or do I expect too much of this option?

From what I watch here 2.92 RC is using tile stealing. When the GPU has finished it will definitely take over the CPU tiles. From my logic it should be like this: If you have a slow CPU but a very fast GPU there is maybe not much of a difference between GPU only and hybrid mode. Because the GPU might solve the whole render while the CPU is still working on the first tiles. But if you have a very fast CPU there should be a difference. How fast is the AMD 3970X compared to the RTX 3090? I have a 16 core Threadripper and GTX 1080Ti which are equal in speed. I will do a test.

My AMD Ryzen Threadripper 3790X is slower then the RTX 3090. I guess there is no "consumer" CPU, that renders tiles faster than a nvidia RTX 3090 currently...
I have an old workstation with a 1080Ti and will make some tests there, soon. Currently my old machine is not ready to go.

Excuse my question, but which version is needed to test the tile stealing feature?
I've a nvidia RTX 3090 here and an AMD 3970X CPU with 32 cores. In my tests the "Candidate" version of Blender 2.92 has nearly no difference in rendering speed, or just marginal better.

Is the tile stealing feature implemented in the current 2.92 Candidate version or do I expect too much of this option?

From what I watch here 2.92 RC is using tile stealing. When the GPU has finished it will definitely take over the CPU tiles. From my logic it should be like this: If you have a slow CPU but a very fast GPU there is maybe not much of a difference between GPU only and hybrid mode. Because the GPU might solve the whole render while the CPU is still working on the first tiles. But if you have a very fast CPU there should be a difference. How fast is the AMD 3970X compared to the RTX 3090? I have a 16 core Threadripper and GTX 1080Ti which are equal in speed. I will do a test.

My AMD Ryzen Threadripper 3790X is slower then the RTX 3090. I guess there is no "consumer" CPU, that renders tiles faster than a nvidia RTX 3090 currently...
I have an old workstation with a 1080Ti and will make some tests there, soon. Currently my old machine is not ready to go.

I use a laptop here with an 8 core i7-10875H and RTX 2070 Super. My test scene renders like this: The GPU will finish their part while the 16 CPU tiles are still rendering. The speed advantage is finally the difference between the 16 CPU tiles and 16 GPU tiles. Not really much. I will update this in a few minutes with results.

Render times

GPU only: 6m 59s
Hybrid: 6m 33s

27s difference by adding the 8 core CPU. The 16 CPU tiles were finished by the GPU at the end.

Now I will do a test on my Threadripper with GTX 1080Ti.

@Christoph Werner (Taros) I made an extreme test with tiles of 512x512. This gives me the suspicion that the GPU takes over the tiles but renders them from scratch because the render time in this case with different Adaptive Sampling settings for a faster result was nearly the same.

GPU only: 1m 45s
Hybrid: 1m 49s

This means: As long as the CPU will not finish the initial tiles it will not have an effect on the overall render time.

Maybe a dev can give us a precise answer about how tile stealing works.

@Christoph Werner (Taros) The test with the AMD 1950X 16 core and GTX 1080Ti shows the following result:

GPU only: 12m 57s
Hybrid: 10m 45s

Here the 32 tiles are nearly as fast as the GPU. The speed advantage comes from the fact that the Threadripper finishes a couple of tiles before the GPU starts to take the remaining ones over.

But this will not answer why the first test is 27s faster?