Denoise render stop Os system #76790

New Issue

CHET · 2020-05-15T18:30:32+02:00

CHET commented

2020-05-15 18:30:32 +02:00

System Information
Operating system: Windows-10-10.0.18362-SP0 64 Bits
Graphics card: GeForce GTX 950M/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 441.87

Blender Version
Broken: version: 2.90 (sub 1), branch: master, commit date: 2020-05-11 21:02, hash: 542ff416e2
Worked: (newest version of Blender that worked as expected)

Short description of error
Intel Denoiser (in compositing nodes) with resolution +5000 stops the operating system. This forces me to do an emergency reboot, which negatively affects the system.
Add protection of system resources from the blender; I am not happy that the blender is very risky for the PC to work. And optimize the operation of nodes!

**System Information** Operating system: Windows-10-10.0.18362-SP0 64 Bits Graphics card: GeForce GTX 950M/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 441.87 **Blender Version** Broken: version: 2.90 (sub 1), branch: master, commit date: 2020-05-11 21:02, hash: `542ff416e2` Worked: (newest version of Blender that worked as expected) **Short description of error** Intel Denoiser (in compositing nodes) with resolution +5000 stops the operating system. This forces me to do an emergency reboot, which negatively affects the system. Add protection of system resources from the blender; I am not happy that the blender is very risky for the PC to work. And optimize the operation of nodes!

CHET commented

2020-05-15 18:30:32 +02:00

Added subscriber: @cheteron

Ankit Meel commented

2020-05-15 19:14:33 +02:00

Added subscriber: @ankitm

Ankit Meel commented

2020-05-15 19:14:33 +02:00

Since blender is supplied the same for render farms, powerful machines & low end machines, the burden of proper usage falls onto the user.
If there's a specific file, use case that you can share about And optimize the operation of nodes!, it'd be better.

Since blender is supplied the same for render farms, powerful machines & low end machines, the burden of proper usage falls onto the user. If there's a specific file, use case that you can share about *And optimize the operation of nodes!*, it'd be better.

CHET commented

2020-05-15 19:35:55 +02:00

What? Optimize blender! It shouldn't crash system! Stop excuses!

Germano Cavalcante commented

2020-05-15 19:56:54 +02:00

Added subscriber: @mano-wii

Germano Cavalcante commented

2020-05-15 19:56:54 +02:00

@cheteron, could you provide a simple file showing the problem?

Also keep in mind that this bug tracker is only for bugs and errors.
While we do continue to work on improving performance in general, potential performance improvements are not handled as bug reports.

To improve performance, consider using less complex geometry, simpler shaders and smaller textures.

@cheteron, could you provide a simple file showing the problem? Also keep in mind that this bug tracker is only for bugs and errors. While we do continue to work on improving performance in general, potential performance improvements are not handled as bug reports. To improve performance, consider using less complex geometry, simpler shaders and smaller textures.

Alaska commented

2020-05-16 01:09:05 +02:00

Added subscriber: @Alaska

Alaska commented

2020-05-16 01:09:05 +02:00

Changed status from 'Needs Triage' to: 'Archived'

Alaska closed this issue

2020-05-16 01:09:05 +02:00

Alaska self-assigned this 2020-05-16 01:09:05 +02:00

Alaska commented

2020-05-16 01:09:05 +02:00

@cheteron Blender operates like most other programs. It uses the amount of memory it needs and the operating system (Windows) handles where that information is stored. If the amount of memory required exceeds the amount of RAM on your system, then Windows will start to use what's known as a "page file". This is a place on your storage device (SSD or HDD) where tasks that use more memory than you have RAM get placed to ensure the application continues to run. Because the page file is slower to access than RAM, it can make everything, including Windows, run slowly or even appear to come to a halt. If Blender or any other application continues to use up all the space that's made available to it through the page file system, then Blender or the application will crash.

As far as I can tell, there is no way for a application to know if it is using the page file. As a result, there's no way for Blender to detect if it's doing this and to stop the compositing task. It's also unwise for Blender to do this as there are many scenarios where you would want Blender to continue running when it starts to use the page file.

As for how to fix this. The only real fix from my understand would be for the developers to optimize the code of the denoise node such that it uses less RAM. The only issue is that you could always increase the resolution of your render, use more denoise nodes, or remove RAM from your system to reproduce the issue. So although the optimization may work for you at a resolution of 5000x5000 on a 16GB RAM computer. It may not work at 10000x10000. Or it may not work at a resolution of 5000x5000 on a 8GB RAM computer. There's no way the Blender developers can make it so the denoise node works at every resolution on every RAM configuration, because to do that, they would have to make the denoise node used 0bytes of RAM. Which means there would be no denoise node.

Besides, the denoise node is developed by Intel. So most of the optimization will have to come from Intel. At the moment, there is a plan to upgrade the Intel denoiser found in Blender to version 1.2.0 (#76184). This should bring along some improvements in speed and reduce memory consumption but it may not entirely fix your issue.

I am marking the report as Invalid on the basis that your issue is caused by a "out of memory" issue and the way that Windows handles this. Not a issue with Blender.

If you believe there is a issue to investigate and that the issue isn't due to you running out of memory (as I've described above), then please create a new report. When making the new report, please explain why you believe the issue isn't a out of memory issue, and provide evidence if you can.

Also, "optimize the operation of nodes" isn't a valid report unless you have evidence that there is a significant reduction in performance between the tested version of Blender (2.90) and the previous release (2.83/2.82). Keep in mind that the developers are always working on making Blender run faster and use less memory. A report should only be made about slow performance or high memory usage if Blender started to perform significantly slower or uses significantly more memory due to a recent update. Take for example this report #75432.

@cheteron Blender operates like most other programs. It uses the amount of memory it needs and the operating system (Windows) handles where that information is stored. If the amount of memory required exceeds the amount of RAM on your system, then Windows will start to use what's known as a "page file". This is a place on your storage device (SSD or HDD) where tasks that use more memory than you have RAM get placed to ensure the application continues to run. Because the page file is slower to access than RAM, it can make everything, including Windows, run slowly or even appear to come to a halt. If Blender or any other application continues to use up all the space that's made available to it through the page file system, then Blender or the application will crash. As far as I can tell, there is no way for a application to know if it is using the page file. As a result, there's no way for Blender to detect if it's doing this and to stop the compositing task. It's also unwise for Blender to do this as there are many scenarios where you would want Blender to continue running when it starts to use the page file. As for how to fix this. The only real fix from my understand would be for the developers to optimize the code of the denoise node such that it uses less RAM. The only issue is that you could always increase the resolution of your render, use more denoise nodes, or remove RAM from your system to reproduce the issue. So although the optimization may work for you at a resolution of 5000x5000 on a 16GB RAM computer. It may not work at 10000x10000. Or it may not work at a resolution of 5000x5000 on a 8GB RAM computer. There's no way the Blender developers can make it so the denoise node works at every resolution on every RAM configuration, because to do that, they would have to make the denoise node used 0bytes of RAM. Which means there would be no denoise node. Besides, the denoise node is developed by Intel. So most of the optimization will have to come from Intel. At the moment, there is a plan to upgrade the Intel denoiser found in Blender to version 1.2.0 (#76184). This should bring along some improvements in speed and reduce memory consumption but it may not entirely fix your issue. I am marking the report as Invalid on the basis that your issue is caused by a "out of memory" issue and the way that Windows handles this. Not a issue with Blender. If you believe there is a issue to investigate and that the issue isn't due to you running out of memory (as I've described above), then please create a new report. When making the new report, please explain why you believe the issue isn't a out of memory issue, and provide evidence if you can. Also, "optimize the operation of nodes" isn't a valid report unless you have evidence that there is a significant reduction in performance between the tested version of Blender (2.90) and the previous release (2.83/2.82). Keep in mind that the developers are always working on making Blender run faster and use less memory. A report should only be made about slow performance or high memory usage if Blender started to perform significantly slower or uses significantly more memory due to a recent update. Take for example this report #75432.

CHET commented

2020-05-16 02:48:21 +02:00

I wrote above. Render settings:
Resolution 5000*5000 or more. In compositing add denoise node. When blender try denoise, the whole system stops responding.
Need manual limits for memory and cpu
testest.blend

I wrote above. Render settings: Resolution 5000*5000 or more. In compositing add denoise node. When blender try denoise, the whole system stops responding. Need manual limits for memory and cpu [testest.blend](https://archive.blender.org/developer/F8537890/testest.blend) ![image.png](https://archive.blender.org/developer/F8537799/image.png)

Alaska commented

2020-05-16 03:09:26 +02:00

The issue you're experiencing is most like memory related, not CPU related. If it was CPU related, then the entire system should freeze if you used the denoise node on any scene at any resolution. If you would like to test what happens if you limit CPU resources, you can do that by opening Blender and your scene, then open the task manager (Press Ctrl+Alt+Delete and select "Task Manger"), select the "More details" drop down, Select the tab at the top labeled "Details", find Blender, right click it and select "Select affinity" and untick a few (not all) of the boxes and confirm the change. Then render your scene. Now Blender will only run on a percentage of your CPU cores. Re-opening Blender will reset the affinity so don't worry about setting it back to normal.

As for putting a limit in place for RAM, it could be done, but I don't believe it will be on by default. And when enabled, I believe compositing would just fail rather than continue compositing.

Are you able to run some tests for me.

Run the test with limited CPU access (described above). Does it still freeze?
If your computer still freezes, what happens if you leave your computer for a while? (E.G. 10 minutes) Does it finish the compositing/denoising?

The issue you're experiencing is most like memory related, not CPU related. If it was CPU related, then the entire system should freeze if you used the denoise node on **any** scene at **any** resolution. If you would like to test what happens if you limit CPU resources, you can do that by opening Blender and your scene, then open the task manager (Press Ctrl+Alt+Delete and select "Task Manger"), select the "More details" drop down, Select the tab at the top labeled "Details", find Blender, right click it and select "Select affinity" and untick a few (not all) of the boxes and confirm the change. Then render your scene. Now Blender will only run on a percentage of your CPU cores. Re-opening Blender will reset the affinity so don't worry about setting it back to normal. As for putting a limit in place for RAM, it could be done, but I don't believe it will be on by default. And when enabled, I believe compositing would just fail rather than continue compositing. Are you able to run some tests for me. 1. Run the test with limited CPU access (described above). Does it still freeze? 2. If your computer still freezes, what happens if you leave your computer for a while? (E.G. 10 minutes) Does it finish the compositing/denoising?

Alaska commented

2020-05-16 03:11:38 +02:00

Also, in your original report you state that the issue occurs in Blender 2.90. Did you test with Blender 2.83 or 2.82a? And did they produce the same results?

Evan Wilson commented

2020-05-16 04:14:36 +02:00

Added subscriber: @EAW

Evan Wilson commented

2020-05-16 04:14:36 +02:00

No crash here, just 2:57 staring at a progress bar at 0% before it was done.

System Information
Operating system: Windows-10-10.0.18362-SP0 64 Bits
Graphics card: GeForce GTX 470/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 376.51

Blender Version
version: 2.90 (sub 2), branch: master, commit date: 2020-05-15 19:12, hash: eaf7d36d66

No crash here, just 2:57 staring at a progress bar at 0% before it was done. **System Information** Operating system: Windows-10-10.0.18362-SP0 64 Bits Graphics card: GeForce GTX 470/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 376.51 **Blender Version** version: 2.90 (sub 2), branch: master, commit date: 2020-05-15 19:12, hash: `eaf7d36d66`

CHET commented

2020-05-16 17:04:12 +02:00

In #76790#932691, @Alaska wrote:
Also, in your original report you state that the issue occurs in Blender 2.90. Did you test with Blender 2.83 or 2.82a? And did they produce the same results?

2.83, 2.0 - same results.

PC freeze, and i waiting 10-20 minutes, then restart system, and render without denoise

> In #76790#932691, @Alaska wrote: > Also, in your original report you state that the issue occurs in Blender 2.90. Did you test with Blender 2.83 or 2.82a? And did they produce the same results? 2.83, 2.0 - same results. PC freeze, and i waiting 10-20 minutes, then restart system, and render without denoise

Alaska commented

2020-05-17 00:49:09 +02:00

@cheteron with you being able to reproduce the issue with in 2.90, 2.83 and I presume 2.82a, and nobody else being able to reproduce the issue, then we must assume the issue is caused by your computer not Blender.

Yes, the Blender developers can put limits in for how much RAM and how many CPU cores the Intel denoiser can use. But if these were to be implemented, then you wouldn't be able to use the compositor on these high resolution scenes or you would be able to use it, but it'll be extraordinarily slow. That may seem like a better option than your computer freezing and you having to turn the computer off and on again to fix it. But Windows has systems in place to stop these kinds of things from happening. They just don't seem to be working in your situation. But once again, that's not a Blender issue, it's a Windows issue.

It's also possible that you're experiencing a hardware issue. The only way we could tell was if you ran tests on other operating systems (E.G. Linux) or a fresh install of Windows. But once again, that's not a issue with Blender, that's a issue with the hardware in your laptop.

I'm sorry we're unable to help you out with your issue.

@cheteron with you being able to reproduce the issue with in 2.90, 2.83 and I presume 2.82a, and nobody else being able to reproduce the issue, then we must assume the issue is caused by your computer not Blender. Yes, the Blender developers can put limits in for how much RAM and how many CPU cores the Intel denoiser can use. But if these were to be implemented, then you wouldn't be able to use the compositor on these high resolution scenes or you would be able to use it, but it'll be extraordinarily slow. That may seem like a better option than your computer freezing and you having to turn the computer off and on again to fix it. But Windows has systems in place to stop these kinds of things from happening. They just don't seem to be working in your situation. But once again, that's not a Blender issue, it's a Windows issue. It's also possible that you're experiencing a hardware issue. The only way we could tell was if you ran tests on other operating systems (E.G. Linux) or a fresh install of Windows. But once again, that's not a issue with Blender, that's a issue with the hardware in your laptop. I'm sorry we're unable to help you out with your issue.

Jesse Yurkovich commented

2020-05-17 01:24:16 +02:00

Added subscriber: @deadpin

Jesse Yurkovich commented

2020-05-17 01:24:16 +02:00

Yes, the OIDN denoiser is very memory hungry indeed. However, there's at least 1 thing blender itself can do to potentially mitigate this (if this is truly an out of memory situation): Use the OIDN API setting which limits memory.

OIDN exposes a "maxMemoryMB" setting that blender can set and by default it's set to somewhere around 6GB. Perhaps we need to hook that up and expose a system preference to see if it potentially helps the situation?

https://openimagedenoise.github.io/documentation.html

int |		maxMemoryMB |	6000 |	approximate maximum amount of scratch memory to use in megabytes (actual memory usage may be higher); limiting memory usage may cause slower denoising due to internally splitting the image into overlapping tiles, but cannot cause the denoising to fail |

Yes, the OIDN denoiser is very memory hungry indeed. However, there's at least 1 thing blender itself can do to potentially mitigate this (if this is truly an out of memory situation): Use the OIDN API setting which limits memory. OIDN exposes a "maxMemoryMB" setting that blender can set and by default it's set to somewhere around 6GB. Perhaps we need to hook that up and expose a system preference to see if it potentially helps the situation? https://openimagedenoise.github.io/documentation.html ``` int | maxMemoryMB | 6000 | approximate maximum amount of scratch memory to use in megabytes (actual memory usage may be higher); limiting memory usage may cause slower denoising due to internally splitting the image into overlapping tiles, but cannot cause the denoising to fail | ```

Alaska commented

2020-05-17 01:37:33 +02:00

@deadpin The issue is that Blender shouldn't be causing the entire system to freeze when using the Intel denoiser at any memory usage setting because the page file system exists. This makes me believe the issue is more to do with Windows or the hardware. We also have to consider that once the denoising is complete, there's still a massive multi-gigabyte image file that needs to be stored in RAM. So if 6GB of denoiser is enough to cause issues on the users computer (assuming it is a out of memory issue), then the 5000x5000 resolution image after denoising is also something to consider as the cause for a out of memory issue.

As for implementing an option to control memory usage for the denoiser in the Blender settings, I will leave that up to the Blender developers.

@deadpin The issue is that Blender shouldn't be causing the entire system to freeze when using the Intel denoiser at any memory usage setting because the page file system exists. This makes me believe the issue is more to do with Windows or the hardware. We also have to consider that once the denoising is complete, there's still a massive multi-gigabyte image file that needs to be stored in RAM. So if 6GB of denoiser is enough to cause issues on the users computer (assuming it is a out of memory issue), then the 5000x5000 resolution image after denoising is also something to consider as the cause for a out of memory issue. As for implementing an option to control memory usage for the denoiser in the Blender settings, I will leave that up to the Blender developers.

CHET commented

2020-05-17 03:12:18 +02:00

Can u add option limit? Without without third-party resources.
I have seen smart memory limits on other programs, that allow program not to take more than necessary.

Can u add option limit? Without without third-party resources. I have seen smart memory limits on other programs, that allow program not to take more than necessary.

Alaska commented

2020-05-17 03:26:17 +02:00

In #76790#933016, @cheteron wrote:
Can u add option limit? Without without third-party resources.
I have seen smart memory limits on other programs, that allow program not to take more than necessary.

Sadly, I do not know how to add a memory limit as I'm well experienced in programming. This will need to be looked over by the developers. It should be noted that reports made at https:*developer.blender.org should be made for bug reports. Feature requests should be made over at https:*blender.community/c/rightclickselect/.

Also, Blender is currently setup like "other programs". It's using only what it thinks it needs. Optimizations can and are being made to reduce memory usage in Blender.

> In #76790#933016, @cheteron wrote: > Can u add option limit? Without without third-party resources. > I have seen smart memory limits on other programs, that allow program not to take more than necessary. Sadly, I do not know how to add a memory limit as I'm well experienced in programming. This will need to be looked over by the developers. It should be noted that reports made at https:*developer.blender.org should be made for bug reports. Feature requests should be made over at https:*blender.community/c/rightclickselect/. Also, Blender is currently setup like "other programs". It's using only what it thinks it needs. Optimizations can and are being made to reduce memory usage in Blender.

YAFU commented

2020-05-17 03:42:57 +02:00

Added subscriber: @YAFU

YAFU commented

2020-05-17 03:42:57 +02:00

This comment was removed by @YAFU

*This comment was removed by @YAFU*

Alaska commented

2020-05-17 07:52:22 +02:00

@cheteron I have added you as a subscriber to a task created to add a memory limit to the Intel denoiser (D7758). Thank you @deadpin. You will most likely hear news from the developers on whether or not this should be added to Blender in that task.

Hopefully this patch can help you assuming it gets accepted.

It should be noted that I still believe your issue isn't Blender related, but something to do with Windows or your computer hardware. However, having this option available may give you the option to work around the issue in the meantime.

@cheteron I have added you as a subscriber to a task created to add a memory limit to the Intel denoiser ([D7758](https://archive.blender.org/developer/D7758)). Thank you @deadpin. You will most likely hear news from the developers on whether or not this should be added to Blender in that task. Hopefully this patch can help you assuming it gets accepted. It should be noted that I still believe your issue isn't Blender related, but something to do with Windows or your computer hardware. However, having this option available may give you the option to work around the issue in the meantime.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Denoise render stop Os system #76790