Blender crashes when using Path Guiding #101529

New Issue

Andreas Resch · 2022-10-01T17:12:05+02:00

Andreas Resch commented

2022-10-01 17:12:05 +02:00

System Information
Operating system: Windows-10-10.0.19044-SP0 64 Bits
Graphics card: NVIDIA GeForce RTX 3060/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 516.94

Blender Version
Broken: version: 3.4.0 Alpha, branch: master, commit date: 2022-09-30 22:40, hash: 206dead860
Worked: (newest version of Blender that worked as expected)

Short description of error
Blender crashes when using Path Guiding in the middle of rendering. Not sure what the reason is - maybe related to volumetric shaders.

Here's a test scene that crashes after about 400 samples and the related crash log.

http://www.argfx.at/upload/PathGuidingTest_01.zip
http://www.argfx.at/upload/PathGuidingTest_01.crash.txt.zip

Exact steps for others to reproduce the error

Open .blend file
Toggle path guiding

PathGuidingTest_01.zip

PathGuidingTest_01.crash.txt.zip

**System Information** Operating system: Windows-10-10.0.19044-SP0 64 Bits Graphics card: NVIDIA GeForce RTX 3060/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 516.94 **Blender Version** Broken: version: 3.4.0 Alpha, branch: master, commit date: 2022-09-30 22:40, hash: `206dead860` Worked: (newest version of Blender that worked as expected) **Short description of error** Blender crashes when using Path Guiding in the middle of rendering. Not sure what the reason is - maybe related to volumetric shaders. Here's a test scene that crashes after about 400 samples and the related crash log. http://www.argfx.at/upload/PathGuidingTest_01.zip http://www.argfx.at/upload/PathGuidingTest_01.crash.txt.zip **Exact steps for others to reproduce the error** - Open .blend file - Toggle path guiding [PathGuidingTest_01.zip](https://archive.blender.org/developer/F13605580/PathGuidingTest_01.zip) [PathGuidingTest_01.crash.txt.zip](https://archive.blender.org/developer/F13605582/PathGuidingTest_01.crash.txt.zip)

Andreas Resch commented

2022-10-01 17:12:05 +02:00

Added subscriber: @AndreasResch

Pratik Borhade commented

2022-10-02 04:30:11 +02:00

Added subscriber: @PratikPB2123

Pratik Borhade commented

2022-10-02 04:30:11 +02:00

Changed status from 'Needs Triage' to: 'Needs User Info'

Pratik Borhade commented

2022-10-02 04:30:11 +02:00

Hi, thanks for the report. I'm unable to download the test file. Could upload the zip file directly instead of the link?
Please also provide the exact way to trigger the crash.

Hi, thanks for the report. I'm unable to download the test file. Could upload the zip file directly instead of the link? Please also provide the exact way to trigger the crash.

liang yunfei commented

2022-10-02 09:48:27 +02:00

Added subscriber: @liang-yunfei

liang yunfei commented

2022-10-02 09:48:27 +02:00

Hi, I can't reproduce the crash you describe using your project. This is the result of noise reduction in your project after sampling 800 rounds, I did not change any settings in your project. Maybe you can try re-rendering a few times?

Hi, I can't reproduce the crash you describe using your project. This is the result of noise reduction in your project after sampling 800 rounds, I did not change any settings in your project. Maybe you can try re-rendering a few times? ![renderresult.exr](https://archive.blender.org/developer/F13605570/renderresult.exr)

Andreas Resch commented

2022-10-02 09:57:12 +02:00

Here are the files.

PathGuidingTest_01.zip

PathGuidingTest_01.crash.txt.zip

Here are the files. [PathGuidingTest_01.zip](https://archive.blender.org/developer/F13605580/PathGuidingTest_01.zip) [PathGuidingTest_01.crash.txt.zip](https://archive.blender.org/developer/F13605582/PathGuidingTest_01.crash.txt.zip)

Andreas Resch commented

2022-10-02 10:00:25 +02:00

In #101529#1426416, @liang-yunfei wrote:
Hi, I can't reproduce the crash you describe using your project. This is the result of noise reduction in your project after sampling 800 rounds, I did not change any settings in your project. Maybe you can try re-rendering a few times?

It doesn't crash all the time. You can try to duplicate the sphere in the scene 2 or 3 times and render again.

> In #101529#1426416, @liang-yunfei wrote: > Hi, I can't reproduce the crash you describe using your project. This is the result of noise reduction in your project after sampling 800 rounds, I did not change any settings in your project. Maybe you can try re-rendering a few times? > ![renderresult.exr](https://archive.blender.org/developer/F13605570/renderresult.exr) It doesn't crash all the time. You can try to duplicate the sphere in the scene 2 or 3 times and render again.

Pratik Borhade commented

2022-10-02 11:31:26 +02:00

Changed status from 'Needs User Info' to: 'Needs Triage'

Pratik Borhade commented

2022-10-04 04:21:29 +02:00

Changed status from 'Needs Triage' to: 'Confirmed'

liang yunfei commented

2022-10-06 04:42:20 +02:00

Added subscriber: @brecht

liang yunfei commented

2022-10-06 04:42:20 +02:00

I have confirmed this problem. you can create a new file, make a Suzanne as glass, make a ground, and then let the path-guiding method render, depending on your computer hardware, the software will crash after a fixed number of sampling. The number of samples to crash varies with the hardware of the computer, good computers may crash after a very high number of samples, bad computers will crash after fewer samples, but they will eventually crash. Or you can use this project :caise (1).blend
My computer crashed when I sampled more than 40,000 times, and both preview and rendering would cause the software to crash, and no crash log is generated.@brecht Van Lommel (brecht) Could you please check this question, I would be very grateful.
Edit:
I found the reason for the crash: I set the training samples to 0, and as the path-guiding sampling continues, the RAM consumed by blender keeps increasing until the program crashes. That is, if your computer's RAM is large enough, it will crash very late, and if your computer's RAM is small, the crash will come earlier.
Computer RAM consumed by 16000 samples:

Computer RAM consumed by 32000 samples:

Computer RAM consumed by 48000 samples:

My blender crashes after blender consumes 80% of RAM.I also found something very, very interesting.
If I sample 12800 times at a very, very small resolution, and then render at a larger resolution, I only need to sample a very small number of times to achieve good results, it must have something to do with the RAM consumed by blender.
Sampling 128000 times with a small resolution, the resolution is small, and the sampling is very fast:

Don't change any settings, just scale the preview size, 865 samples:

Instead of rendering 12800 times with a small resolution beforehand, directly sample 865 times with a larger resolution：

You can see that directly sampling 865 times is much more noisy than the former.
What I want is, can I prevent blender from crashing because it consumes too much memory? And whether it can be rendered with a smaller resolution during path-guiding learning, and then rendered with a larger resolution after learning.
I will be very happy if these two questions can be resolved!@sebastian-4 Herholz

I have confirmed this problem. you can create a new file, make a Suzanne as glass, make a ground, and then let the path-guiding method render, depending on your computer hardware, the software will crash after a fixed number of sampling. The number of samples to crash varies with the hardware of the computer, good computers may crash after a very high number of samples, bad computers will crash after fewer samples, but they will eventually crash. Or you can use this project :[caise (1).blend](https://archive.blender.org/developer/F13620330/caise__1_.blend) My computer crashed when I sampled more than 40,000 times, and both preview and rendering would cause the software to crash, and no crash log is generated.@brecht Van Lommel (brecht) Could you please check this question, I would be very grateful. Edit: I found the reason for the crash: I set the training samples to 0, and as the path-guiding sampling continues, the RAM consumed by blender keeps increasing until the program crashes. That is, if your computer's RAM is large enough, it will crash very late, and if your computer's RAM is small, the crash will come earlier. Computer RAM consumed by 16000 samples: ![16000samples.png](https://archive.blender.org/developer/F13622462/16000samples.png) Computer RAM consumed by 32000 samples: ![32000samples.png](https://archive.blender.org/developer/F13622465/32000samples.png) Computer RAM consumed by 48000 samples: ![48000sampls.png](https://archive.blender.org/developer/F13622468/48000sampls.png) My blender crashes after blender consumes 80% of RAM.I also found something very, very interesting. If I sample 12800 times at a very, very small resolution, and then render at a larger resolution, I only need to sample a very small number of times to achieve good results, it must have something to do with the RAM consumed by blender. Sampling 128000 times with a small resolution, the resolution is small, and the sampling is very fast: ![128000.png](https://archive.blender.org/developer/F13622514/128000.png) Don't change any settings, just scale the preview size, 865 samples: ![865samples.png](https://archive.blender.org/developer/F13622521/865samples.png) Instead of rendering 12800 times with a small resolution beforehand, directly sample 865 times with a larger resolution： ![865direct.png](https://archive.blender.org/developer/F13622527/865direct.png) You can see that directly sampling 865 times is much more noisy than the former. What I want is, can I prevent blender from crashing because it consumes too much memory? And whether it can be rendered with a smaller resolution during path-guiding learning, and then rendered with a larger resolution after learning. I will be very happy if these two questions can be resolved!@sebastian-4 Herholz

liang yunfei commented

2022-10-06 12:26:04 +02:00

Added subscriber: @sebastian-4

Brecht Van Lommel commented

2022-10-06 19:20:17 +02:00

Added subscriber: @sherholz
Removed subscriber: @sebastian-4

Added subscriber: @sherholz Removed subscriber: @sebastian-4

Brecht Van Lommel commented

2022-10-06 19:20:17 +02:00

CC @sherholz.

Marking as high priority to ensure we look at it before the release.

CC @sherholz. Marking as high priority to ensure we look at it before the release.

Fernando Alcala commented

2022-10-06 21:11:50 +02:00

Added subscriber: @Fernando-Alcala

Fernando Alcala commented

2022-10-06 21:11:50 +02:00

Liang, you can move the camera around and keep training data. If anything else changes, such as moving an object or changing a setting, you lose training data.

Sebastian Herholz commented

2022-10-07 11:53:02 +02:00

Thank you @liang-yunfei, for your analysis, this helps a lot.
This pretty much helped to figure out the cause of the problem, which seems to be a wayyy large bound for the spatial guiding structure.
At these high sample counts and with unlimited training, the structure would refine continuously, which causes the memory increase.
I will post a fix soon.
As a side note, unlimited training does not bring you many performance benefits (in terms of better sampling) and only leads to a performance decrease due to the additional training time.
Unfortunately, we did not find a good automatic way (which is robust in production) to determine the optimal number of training iterations you need.
As a compromise, we decided to give the artist the option to set the number of training iterations (usually 64-128 are enough).

@AndreasResch could you share more information about your system like:

overall memory size (RAM)
used memory when loading the scene (no rendering)
used memory when rendering (without guiding)

I am curious if your bug is related to what @liang-yunfei reported since after 400spp the guiding structure should not be large.

Thank you @liang-yunfei, for your analysis, this helps a lot. This pretty much helped to figure out the cause of the problem, which seems to be a wayyy large bound for the spatial guiding structure. At these high sample counts and with unlimited training, the structure would refine continuously, which causes the memory increase. I will post a fix soon. As a side note, unlimited training does not bring you many performance benefits (in terms of better sampling) and only leads to a performance decrease due to the additional training time. Unfortunately, we did not find a good automatic way (which is robust in production) to determine the optimal number of training iterations you need. As a compromise, we decided to give the artist the option to set the number of training iterations (usually 64-128 are enough). @AndreasResch could you share more information about your system like: - overall memory size (RAM) - used memory when loading the scene (no rendering) - used memory when rendering (without guiding) I am curious if your bug is related to what @liang-yunfei reported since after 400spp the guiding structure should not be large.

Andreas Resch commented

2022-10-07 16:10:17 +02:00

RAM should not be an issue.

*) The system has 64GB of RAM
*) With path guiding the RAM jumps from 10,8GB to 11,7GB and then crashes.
*) Without the RAM it reaches a maximum of 11,3GB during rendering.

RAM should not be an issue. *) The system has 64GB of RAM *) With path guiding the RAM jumps from 10,8GB to 11,7GB and then crashes. *) Without the RAM it reaches a maximum of 11,3GB during rendering.

Sebastian Herholz commented

2022-10-07 18:07:17 +02:00

@AndreasResch OK, then your problem is probably not related to the memory problem reported by @liang-yunfei
At least on Linux I am not able to replicate your bug. I am getting a lot of warnings that some blend files are missing:

Read blend: /data/sherholz/Develop-Blender/scenes/PathGuidingTest_01 (2)/PathGuidingTest_01.blend
Info: Read library:  '/data/sherholz/Develop-Blender/scenes/Work/Verschiedenes ... P-T/Schmidt Spiele/Mille Fiori/_BLENDER/MilleFioriErw_Spielszene_01.blend', '//..\Work\Verschiedenes ... P-T\Schmidt Spiele\Mille Fiori\_BLENDER\MilleFioriErw_Spielszene_01.blend', parent '<direct>'
Info: Cannot find lib '/data/sherholz/Develop-Blender/scenes/Work/Verschiedenes ... P-T/Schmidt Spiele/Mille Fiori/_BLENDER/MilleFioriErw_Spielszene_01.blend'
Info: LIB: Image: 'Bildschirmfoto 2021-06-09 um 11.24.jpg' missing from '/data/sherholz/Develop-Blender/scenes/Work/Verschiedenes ... P-T/Schmidt Spiele/Mille Fiori/_BLENDER/MilleFioriErw_Spielszene_01.blend', parent '<direct>'
Info: Read library:  '/c/Programme/Vshade/Vshade - Color.blend', 'C:\Programme\Vshade\Vshade - Color.blend', parent '<direct>'
Info: Cannot find lib '/c/Programme/Vshade/Vshade - Color.blend'
Info: LIB: NodeTree: 'Vshade - Absorption Calc. (Glass)' missing from '/c/Programme/Vshade/Vshade - Color.blend', parent '<direct>'
Info: LIB: NodeTree: 'Vshade - Fresnel (Lite)' missing from '/c/Programme/Vshade/Vshade - Color.blend', parent '<direct>'
Info: Read library:  '/c/Program Files/Vshade/Vshade - Shader - Cycles.blend', 'C:\Program Files\Vshade\Vshade - Shader - Cycles.blend', parent '<direct>'
Info: Cannot find lib '/c/Program Files/Vshade/Vshade - Shader - Cycles.blend'
Info: LIB: NodeTree: 'Vshade - Solid Shader (Lite)' missing from '/c/Program Files/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>'
Info: Read library:  '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', 'C:\Programme\Vshade\Vshade - Shader - Cycles.blend', parent '<direct>'
Info: Cannot find lib '/c/Programme/Vshade/Vshade - Shader - Cycles.blend'
Info: LIB: NodeTree: 'Vshade - Glossy Shader (Basic)' missing from '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>'
Info: LIB: NodeTree: 'Vshade - Micro Roughness' missing from '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>'
Info: LIB: NodeTree: 'Vshade - Refraction Shader' missing from '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>'
Info: LIB: NodeTree: 'Vshade - Refraction Shader (Base)' missing from '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>'
Info: LIB: NodeTree: 'Vshade - Set Shadow' missing from '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>'
Info: Read library:  '/data/sherholz/Develop-Blender/scenes/Work/Verschiedenes ... P-T/Schmidt Spiele/Mille Fiori/_BLENDER/MilleFioriErw_Spielszene_01.blend', '//..\Work\Verschiedenes ... P-T\Schmidt Spiele\Mille Fiori\_BLENDER\MilleFioriErw_Spielszene_01.blend', parent '<direct>'
Warning: Unable to open '/data/sherholz/Develop-Blender/scenes/Work/Verschiedenes ... P-T/Schmidt Spiele/Mille Fiori/_BLENDER/MilleFioriErw_Spielszene_01.blend': No such file or directory
Info: Cannot find lib '/data/sherholz/Develop-Blender/scenes/Work/Verschiedenes ... P-T/Schmidt Spiele/Mille Fiori/_BLENDER/MilleFioriErw_Spielszene_01.blend'
Info: LIB: Image: 'Bildschirmfoto 2021-06-09 um 11.24.jpg' missing from '/data/sherholz/Develop-Blender/scenes/Work/Verschiedenes ... P-T/Schmidt Spiele/Mille Fiori/_BLENDER/MilleFioriErw_Spielszene_01.blend', parent '<direct>'
Info: Read library:  '/c/Programme/Vshade/Vshade - Color.blend', 'C:\Programme\Vshade\Vshade - Color.blend', parent '<direct>'
Warning: Unable to open '/c/Programme/Vshade/Vshade - Color.blend': No such file or directory
Info: Cannot find lib '/c/Programme/Vshade/Vshade - Color.blend'
Info: LIB: NodeTree: 'Vshade - Absorption Calc. (Glass)' missing from '/c/Programme/Vshade/Vshade - Color.blend', parent '<direct>'
Info: LIB: NodeTree: 'Vshade - Fresnel (Lite)' missing from '/c/Programme/Vshade/Vshade - Color.blend', parent '<direct>'
Info: Read library:  '/c/Program Files/Vshade/Vshade - Shader - Cycles.blend', 'C:\Program Files\Vshade\Vshade - Shader - Cycles.blend', parent '<direct>'
Warning: Unable to open '/c/Program Files/Vshade/Vshade - Shader - Cycles.blend': No such file or directory
Info: Cannot find lib '/c/Program Files/Vshade/Vshade - Shader - Cycles.blend'
Info: LIB: NodeTree: 'Vshade - Solid Shader (Lite)' missing from '/c/Program Files/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>'
Info: Read library:  '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', 'C:\Programme\Vshade\Vshade - Shader - Cycles.blend', parent '<direct>'
Warning: Unable to open '/c/Programme/Vshade/Vshade - Shader - Cycles.blend': No such file or directory
Info: Cannot find lib '/c/Programme/Vshade/Vshade - Shader - Cycles.blend'
Info: LIB: NodeTree: 'Vshade - Glossy Shader (Basic)' missing from '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>'
Info: LIB: NodeTree: 'Vshade - Micro Roughness' missing from '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>'
Info: LIB: NodeTree: 'Vshade - Refraction Shader' missing from '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>'
Info: LIB: NodeTree: 'Vshade - Refraction Shader (Base)' missing from '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>'
Info: LIB: NodeTree: 'Vshade - Set Shadow' missing from '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>'
Warning: 4 libraries and 9 linked data-blocks are missing (including 0 ObjectData and 0 Proxies), please check the Info and Outliner editors for details

Could this be the reason why me and @liang-yunfei can not replicate your observed crashing behaviour?

@AndreasResch OK, then your problem is probably not related to the memory problem reported by @liang-yunfei At least on Linux I am not able to replicate your bug. I am getting a lot of warnings that some blend files are missing: ``` Read blend: /data/sherholz/Develop-Blender/scenes/PathGuidingTest_01 (2)/PathGuidingTest_01.blend Info: Read library: '/data/sherholz/Develop-Blender/scenes/Work/Verschiedenes ... P-T/Schmidt Spiele/Mille Fiori/_BLENDER/MilleFioriErw_Spielszene_01.blend', '//..\Work\Verschiedenes ... P-T\Schmidt Spiele\Mille Fiori\_BLENDER\MilleFioriErw_Spielszene_01.blend', parent '<direct>' Info: Cannot find lib '/data/sherholz/Develop-Blender/scenes/Work/Verschiedenes ... P-T/Schmidt Spiele/Mille Fiori/_BLENDER/MilleFioriErw_Spielszene_01.blend' Info: LIB: Image: 'Bildschirmfoto 2021-06-09 um 11.24.jpg' missing from '/data/sherholz/Develop-Blender/scenes/Work/Verschiedenes ... P-T/Schmidt Spiele/Mille Fiori/_BLENDER/MilleFioriErw_Spielszene_01.blend', parent '<direct>' Info: Read library: '/c/Programme/Vshade/Vshade - Color.blend', 'C:\Programme\Vshade\Vshade - Color.blend', parent '<direct>' Info: Cannot find lib '/c/Programme/Vshade/Vshade - Color.blend' Info: LIB: NodeTree: 'Vshade - Absorption Calc. (Glass)' missing from '/c/Programme/Vshade/Vshade - Color.blend', parent '<direct>' Info: LIB: NodeTree: 'Vshade - Fresnel (Lite)' missing from '/c/Programme/Vshade/Vshade - Color.blend', parent '<direct>' Info: Read library: '/c/Program Files/Vshade/Vshade - Shader - Cycles.blend', 'C:\Program Files\Vshade\Vshade - Shader - Cycles.blend', parent '<direct>' Info: Cannot find lib '/c/Program Files/Vshade/Vshade - Shader - Cycles.blend' Info: LIB: NodeTree: 'Vshade - Solid Shader (Lite)' missing from '/c/Program Files/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>' Info: Read library: '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', 'C:\Programme\Vshade\Vshade - Shader - Cycles.blend', parent '<direct>' Info: Cannot find lib '/c/Programme/Vshade/Vshade - Shader - Cycles.blend' Info: LIB: NodeTree: 'Vshade - Glossy Shader (Basic)' missing from '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>' Info: LIB: NodeTree: 'Vshade - Micro Roughness' missing from '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>' Info: LIB: NodeTree: 'Vshade - Refraction Shader' missing from '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>' Info: LIB: NodeTree: 'Vshade - Refraction Shader (Base)' missing from '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>' Info: LIB: NodeTree: 'Vshade - Set Shadow' missing from '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>' Info: Read library: '/data/sherholz/Develop-Blender/scenes/Work/Verschiedenes ... P-T/Schmidt Spiele/Mille Fiori/_BLENDER/MilleFioriErw_Spielszene_01.blend', '//..\Work\Verschiedenes ... P-T\Schmidt Spiele\Mille Fiori\_BLENDER\MilleFioriErw_Spielszene_01.blend', parent '<direct>' Warning: Unable to open '/data/sherholz/Develop-Blender/scenes/Work/Verschiedenes ... P-T/Schmidt Spiele/Mille Fiori/_BLENDER/MilleFioriErw_Spielszene_01.blend': No such file or directory Info: Cannot find lib '/data/sherholz/Develop-Blender/scenes/Work/Verschiedenes ... P-T/Schmidt Spiele/Mille Fiori/_BLENDER/MilleFioriErw_Spielszene_01.blend' Info: LIB: Image: 'Bildschirmfoto 2021-06-09 um 11.24.jpg' missing from '/data/sherholz/Develop-Blender/scenes/Work/Verschiedenes ... P-T/Schmidt Spiele/Mille Fiori/_BLENDER/MilleFioriErw_Spielszene_01.blend', parent '<direct>' Info: Read library: '/c/Programme/Vshade/Vshade - Color.blend', 'C:\Programme\Vshade\Vshade - Color.blend', parent '<direct>' Warning: Unable to open '/c/Programme/Vshade/Vshade - Color.blend': No such file or directory Info: Cannot find lib '/c/Programme/Vshade/Vshade - Color.blend' Info: LIB: NodeTree: 'Vshade - Absorption Calc. (Glass)' missing from '/c/Programme/Vshade/Vshade - Color.blend', parent '<direct>' Info: LIB: NodeTree: 'Vshade - Fresnel (Lite)' missing from '/c/Programme/Vshade/Vshade - Color.blend', parent '<direct>' Info: Read library: '/c/Program Files/Vshade/Vshade - Shader - Cycles.blend', 'C:\Program Files\Vshade\Vshade - Shader - Cycles.blend', parent '<direct>' Warning: Unable to open '/c/Program Files/Vshade/Vshade - Shader - Cycles.blend': No such file or directory Info: Cannot find lib '/c/Program Files/Vshade/Vshade - Shader - Cycles.blend' Info: LIB: NodeTree: 'Vshade - Solid Shader (Lite)' missing from '/c/Program Files/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>' Info: Read library: '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', 'C:\Programme\Vshade\Vshade - Shader - Cycles.blend', parent '<direct>' Warning: Unable to open '/c/Programme/Vshade/Vshade - Shader - Cycles.blend': No such file or directory Info: Cannot find lib '/c/Programme/Vshade/Vshade - Shader - Cycles.blend' Info: LIB: NodeTree: 'Vshade - Glossy Shader (Basic)' missing from '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>' Info: LIB: NodeTree: 'Vshade - Micro Roughness' missing from '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>' Info: LIB: NodeTree: 'Vshade - Refraction Shader' missing from '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>' Info: LIB: NodeTree: 'Vshade - Refraction Shader (Base)' missing from '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>' Info: LIB: NodeTree: 'Vshade - Set Shadow' missing from '/c/Programme/Vshade/Vshade - Shader - Cycles.blend', parent '<direct>' Warning: 4 libraries and 9 linked data-blocks are missing (including 0 ObjectData and 0 Proxies), please check the Info and Outliner editors for details ``` Could this be the reason why me and @liang-yunfei can not replicate your observed crashing behaviour?

Sebastian Herholz commented

2022-10-07 18:50:17 +02:00

@liang-yunfei I also want to answer your two questions:
1. Memory problem:
This was a bug and will be fixed soon

2. Training at low resolution vs high resolution:
The behavior you reported is expected.
Path guiding learns an approximation of the light distribution in the scene by using information
from the paths of previous rendering iterations. Therefore higher resolutions will provide more information (samples)
than lower resolutions, since more paths are traced (one per pixel) per rendering iteration.
As a result, the guiding structure has higher quality than if you would run it at a lower resolution.
Since the learned guiding structure is directly used after the first rendering iteration, the sampling in the second iteration is better, and
therefore the quality of the updated structure becomes even better.
this is also the reason why a smaller number of high-resolution training passes usually outperform many training passes at lower resolutions.

@liang-yunfei I also want to answer your two questions: **1. Memory problem:** This was a bug and will be fixed soon **2. Training at low resolution vs high resolution:** The behavior you reported is expected. Path guiding learns an approximation of the light distribution in the scene by using information from the paths of previous rendering iterations. Therefore higher resolutions will provide more information (samples) than lower resolutions, since more paths are traced (one per pixel) per rendering iteration. As a result, the guiding structure has higher quality than if you would run it at a lower resolution. Since the learned guiding structure is directly used after the first rendering iteration, the sampling in the second iteration is better, and therefore the quality of the updated structure becomes even better. this is also the reason why a smaller number of high-resolution training passes usually outperform many training passes at lower resolutions. ```

liang yunfei commented

2022-10-07 20:01:02 +02:00

Added subscriber: @sebastian-4

liang yunfei commented

2022-10-07 20:01:02 +02:00

@sherholz Thank you very much for your patient reply! After hearing you say that 64-128 training iterations are usually sufficient, I realized that resolution is also one of the important factors in training results. I tested training with high resolution in my project, and with just 128 iterations I got the same results as training with low resolution for thousands of iterations. However, some smaller caustic rays still cannot be calculated. I know that it is not designed to calculate caustics, but in some cases, the original cycle cannot calculate caustics at all, so this method can sometimes improve the calculation speed by tens of thousands of times. Your explanation is wonderful, please be sure to write it in the blender manual after this feature is officially added to blender.
However, I still have a question, path-guiding sometimes introduces a lot of very bright color noise, and these noises do not become significantly darker as the number of samples increases.I wonder if this is inevitable or if it can be optimized in a future version.
Path-guiding creates a lot of bright colored noise, which makes denoise impossible:

After 1.6 million rounds of sampling in the traditional sampling method, there is almost no bright color noise：

@sherholz Thank you very much for your patient reply! After hearing you say that 64-128 training iterations are usually sufficient, I realized that resolution is also one of the important factors in training results. I tested training with high resolution in my project, and with just 128 iterations I got the same results as training with low resolution for thousands of iterations. However, some smaller caustic rays still cannot be calculated. I know that it is not designed to calculate caustics, but in some cases, the original cycle cannot calculate caustics at all, so this method can sometimes improve the calculation speed by tens of thousands of times. Your explanation is wonderful, please be sure to write it in the blender manual after this feature is officially added to blender. However, I still have a question, path-guiding sometimes introduces a lot of very bright color noise, and these noises do not become significantly darker as the number of samples increases.I wonder if this is inevitable or if it can be optimized in a future version. Path-guiding creates a lot of bright colored noise, which makes denoise impossible: ![16000path-guiding.png](https://archive.blender.org/developer/F13629441/16000path-guiding.png) After 1.6 million rounds of sampling in the traditional sampling method, there is almost no bright color noise： ![1600000cycle.png](https://archive.blender.org/developer/F13629469/1600000cycle.png)

Sebastian Herholz commented

2022-10-07 20:25:44 +02:00

@liang-yunfei how many samples did you use for the path guided version?
There are two things:

I saw that you played around with DEBUG parameters like guiding probabilities.
And you set them from 0.5 to 1.0. These parameters are for debugging only and should not be changed.
0.5 means that you do 50% BSDF sampling and 50 guiding, which is a stable conservative ratio so that the path tracer
trusts the guiding cache but still explores the scene to capture any contribution not observed by the cache.
If you go full 100% into guiding it can happen that you mis or hardly under explore important parts of your scene.
I assume you use less samples in the PG image.
Since guiding has to learn from over time, early samples will still have some fireflies or firefiles are generated
from sampling decisions which are not or can not be guided (e.g., reflect/refract on glass).
These fireflies still need a lot of samples to be average out like in the plain 1.6M rendering.

aA solution would be to either use more indirect radiance clamping (which would make the image darker) or
Cycles would need to add some more inteligent firefly removal approach.

@liang-yunfei how many samples did you use for the path guided version? There are two things: 1. I saw that you played around with DEBUG parameters like guiding probabilities. And you set them from 0.5 to 1.0. These parameters are for debugging only and should not be changed. 0.5 means that you do 50% BSDF sampling and 50 guiding, which is a stable conservative ratio so that the path tracer trusts the guiding cache but still explores the scene to capture any contribution not observed by the cache. If you go full 100% into guiding it can happen that you mis or hardly under explore important parts of your scene. 2. I assume you use less samples in the PG image. Since guiding has to learn from over time, early samples will still have some fireflies or firefiles are generated from sampling decisions which are not or can not be guided (e.g., reflect/refract on glass). These fireflies still need a lot of samples to be average out like in the plain 1.6M rendering. aA solution would be to either use more indirect radiance clamping (which would make the image darker) or Cycles would need to add some more inteligent firefly removal approach.

liang yunfei commented

2022-10-07 20:37:10 +02:00

@sherholz Thank you very much!The image that uses path-guiding has only 16000 samples. It is already 2:30 in the morning here in China, so I have to go to bed. I will get up tomorrow morning and test the method you mentioned,and I will give you a reply as soon as possible.

Andreas Resch commented

2022-10-07 23:18:11 +02:00

In #101529#1429262, @sherholz wrote:
@AndreasResch OK, then your problem is probably not related to the memory problem reported by @liang-yunfei
At least on Linux I am not able to replicate your bug. I am getting a lot of warnings that some blend files are missing:

But I'm not actively using any of those linked any more in the test scene. That's one of those cases where Blender isn't that great at deleting stuff that isn't used any more.

> In #101529#1429262, @sherholz wrote: > @AndreasResch OK, then your problem is probably not related to the memory problem reported by @liang-yunfei > At least on Linux I am not able to replicate your bug. I am getting a lot of warnings that some blend files are missing: > But I'm not actively using any of those linked any more in the test scene. That's one of those cases where Blender isn't that great at deleting stuff that isn't used any more.

Andreas Resch commented

2022-10-07 23:26:14 +02:00

Here's a new file that still crashes. It's basically the same scene but without the linked items (that weren't used in the scene anyway).

PathGuidingTest_02.zip

Here's a new file that still crashes. It's basically the same scene but without the linked items (that weren't used in the scene anyway). [PathGuidingTest_02.zip](https://archive.blender.org/developer/F13631184/PathGuidingTest_02.zip)

liang yunfei commented

2022-10-08 06:08:59 +02:00

@sherholz For the first thing you said, I found that setting the guiding probabilities to 1, at least in this project of mine, resulted in a less noisy and sharper caustics image than setting the guiding probabilities to 0.5. In fact I found in most of my test projects that setting the guiding probabilities to 1 resulted in less noise and sharper caustics.
In addition, I forgot to mention that the reason why my test project has colored noise is because my glass objects all use my dispersion shader. If you use blender's default shader, the color of the noise should be solid.
Guiding probabilities 1, trainning 1024 , 2000 samples:

Guiding probabilities 0.5, trainning 1024 , 2000 samples:

Guiding probabilities 1, trainning 1024 , 2000 samples, with denoise:

Guiding probabilities 0.5, trainning 1024 , 2000 samples, with denoise:

Guiding probabilities 1, trainning 1024 , 2000 samples:

Guiding probabilities 0.5, trainning 1024 , 2000 samples:

Luxcore 30s light tracing:

The blue reference line is used to compare the degree of color separation with luxcore, my dispersion shader produces exactly the same result as luxcore.
Then, something interesting happened, in the diamond test project, the path-guiding method (Changing the distribution type of this item did not improve anything) doesn't seem to be able to calculate a sharper caustics.
Guiding probabilities 1, trainning 1024 , 2000 samples:

Guiding probabilities 0.5, trainning 1024 , 2000 samples:

GPU , 2000 samples:

The Distribution type of all the above projects is Directional quad tree.
For your second point, I may have to wait until path-guiding supports gpu before I can do such a high number of samples! So I just have to be patient.

@sherholz For the first thing you said, I found that setting the guiding probabilities to 1, at least in this project of mine, resulted in a less noisy and sharper caustics image than setting the guiding probabilities to 0.5. In fact I found in most of my test projects that setting the guiding probabilities to 1 resulted in less noise and sharper caustics. In addition, I forgot to mention that the reason why my test project has colored noise is because my glass objects all use my dispersion shader. If you use blender's default shader, the color of the noise should be solid. Guiding probabilities 1, trainning 1024 , 2000 samples: ![1 2000.png](https://archive.blender.org/developer/F13635003/1_2000.png) Guiding probabilities 0.5, trainning 1024 , 2000 samples: ![0.5 2000.png](https://archive.blender.org/developer/F13635006/0.5_2000.png) Guiding probabilities 1, trainning 1024 , 2000 samples, with denoise: ![1 2000denoised.png](https://archive.blender.org/developer/F13635010/1_2000denoised.png) Guiding probabilities 0.5, trainning 1024 , 2000 samples, with denoise: ![0.5 2000denoised.png](https://archive.blender.org/developer/F13635015/0.5_2000denoised.png) Guiding probabilities 1, trainning 1024 , 2000 samples: ![sesan 1 2000.png](https://archive.blender.org/developer/F13635017/sesan_1_2000.png) Guiding probabilities 0.5, trainning 1024 , 2000 samples: ![sesan 0.5 2000.png](https://archive.blender.org/developer/F13635020/sesan_0.5_2000.png) Luxcore 30s light tracing: ![F5_BRHYZ7Q1H8DHC46MN`_1.png](https://archive.blender.org/developer/F13635069/F5_BRHYZ7Q1H8DHC46MN__1.png) The blue reference line is used to compare the degree of color separation with luxcore, my dispersion shader produces exactly the same result as luxcore. Then, something interesting happened, in the diamond test project, the path-guiding method (Changing the distribution type of this item did not improve anything) doesn't seem to be able to calculate a sharper caustics. Guiding probabilities 1, trainning 1024 , 2000 samples: ![zuanshi 1 2000.png](https://archive.blender.org/developer/F13635030/zuanshi_1_2000.png) Guiding probabilities 0.5, trainning 1024 , 2000 samples: ![zuan0.5 2000.png](https://archive.blender.org/developer/F13635032/zuan0.5_2000.png) GPU , 2000 samples: ![zuanshi gpu 2000.png](https://archive.blender.org/developer/F13635036/zuanshi_gpu_2000.png) The Distribution type of all the above projects is Directional quad tree. For your second point, I may have to wait until path-guiding supports gpu before I can do such a high number of samples! So I just have to be patient.

liang yunfei commented

2022-10-08 06:16:59 +02:00

@AndreasResch The project you've given now has allowed me to successfully reproduce the crash, and I'll be looking into the project settings that might be causing the crash.
Edit:
I have found the cause of the crash: in this project, using both glass and volume shaders caused the crash, using either the glass alone or the volume shader alone did not cause the crash.

@sherholz

@AndreasResch The project you've given now has allowed me to successfully reproduce the crash, and I'll be looking into the project settings that might be causing the crash. Edit: I have found the cause of the crash: in this project, using both glass and volume shaders caused the crash, using either the glass alone or the volume shader alone did not cause the crash. ![image.png](https://archive.blender.org/developer/F13635269/image.png) @sherholz

Sebastian Herholz commented

2022-10-08 10:08:14 +02:00

@AndreasResch Thanks for the update, I can now also reproduce the crash.
I will have a look at it after the weekend.

@AndreasResch Thanks for the update, I can now also reproduce the crash. I will have a look at it after the weekend.

liang yunfei commented

2022-10-10 10:37:50 +02:00

@sherholz Please can I ask why path guiding doesn't work in such a project?
Cycle GPU 2000 samples:

Path guiding 2000 samples:

Luxcore light tracing 20s:

Project:
pathguiding.blend

@sherholz Please can I ask why path guiding doesn't work in such a project? Cycle GPU 2000 samples: ![Path tracing2000.png](https://archive.blender.org/developer/F13647175/Path_tracing2000.png) Path guiding 2000 samples: ![Path tracing2000.png](https://archive.blender.org/developer/F13647178/Path_tracing2000.png) Luxcore light tracing 20s: ![Luxcore 20s.png](https://archive.blender.org/developer/F13647182/Luxcore_20s.png) Project: [pathguiding.blend](https://archive.blender.org/developer/F13647194/pathguiding.blend)

Sebastian Herholz commented

2022-10-10 12:56:26 +02:00

@AndreasResch I wrote some patches which should solve your problem.
Could you please check the current experimental build:
https://builder.blender.org/download/experimental/cycles_path_guiding/

@liang-yunfei, I think the scene is just too hard to render.
On a first test, I think two things could be the problem here:

Since path guiding learns from previous rendering iterations, there has to be
a high enough initial probability of finding these caustic paths so that path guiding can learn from them.
Here this initial probability might be too low.
No all decisions are guided, mainly the ones on diffuse surfaces. The "reflect or refract" decision is un-guided and therefore based on the Fresnel term.
It could be that in this scene, Fresnel things it is best to reflect (based on the material), but it would be better to refract (based on the indirect lighting).
If this is the main cause of the noise than path guiding can not help here.

@AndreasResch I wrote some patches which should solve your problem. Could you please check the current experimental build: https://builder.blender.org/download/experimental/cycles_path_guiding/ @liang-yunfei, I think the scene is just too hard to render. On a first test, I think two things could be the problem here: 1. Since path guiding learns from previous rendering iterations, there has to be a high enough initial probability of finding these caustic paths so that path guiding can learn from them. Here this initial probability might be too low. 2. No all decisions are guided, mainly the ones on diffuse surfaces. The "reflect or refract" decision is un-guided and therefore based on the Fresnel term. It could be that in this scene, Fresnel things it is best to reflect (based on the material), but it would be better to refract (based on the indirect lighting). If this is the main cause of the noise than path guiding can not help here.

liang yunfei commented

2022-10-10 14:47:16 +02:00

@sherholz Thank you very much, your answer is so detailed. I still have a question, that is, in your plan, path guiding will support refraction shader and specular shader in the future, will path guiding be able to help sample such projects at that time?

Andreas Resch commented

2022-10-10 17:21:05 +02:00

In #101529#1430066, @sherholz wrote:
@AndreasResch I wrote some patches which should solve your problem.
Could you please check the current experimental build:
https://builder.blender.org/download/experimental/cycles_path_guiding/

It works fine now. No more crashing.

> In #101529#1430066, @sherholz wrote: > @AndreasResch I wrote some patches which should solve your problem. > Could you please check the current experimental build: > https://builder.blender.org/download/experimental/cycles_path_guiding/ It works fine now. No more crashing.

Sebastian Herholz commented

2022-10-13 15:39:45 +02:00

Changed status from 'Confirmed' to: 'Resolved'

Sebastian Herholz closed this issue

2022-10-13 15:39:45 +02:00

Sebastian Herholz self-assigned this 2022-10-13 15:39:45 +02:00

Sebastian Herholz commented

2022-10-13 15:39:46 +02:00

closing this task because it is fixed with D16219

closing this task because it is fixed with [D16219](https://archive.blender.org/developer/D16219)

blender-admin commented

2022-10-18 13:59:12 +02:00

This issue was referenced by 2006c3ed10

This issue was referenced by 2006c3ed106d7d7931c6f76704765b3842c34fec

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Blender crashes when using Path Guiding #101529