Memory leak with GPU subdivision on rendering #100969

Closed
opened 2022-09-10 15:01:20 +02:00 by Philipp Weißenbach · 24 comments

System Information
Operating system: Linux-5.15.0-47-lowlatency-x86_64-with-glibc2.35 64 Bits
Graphics card: NVIDIA GeForce GTX 980/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 470.141.03

Blender Version
Broken: version: 3.4.0 Alpha, branch: master, commit date: 2022-09-09 22:58, hash: 352d55b1c8 and older versions after 3.02
Worked: (3.01 and older)

Short description of error
While rendering an animation on a blender version newer than 3.0.1, GPU Subdivision causes
blender to leak the memory and crashes while rendering significantly slower.

I found this out after stripping down my project to this test file and removed almost everything unrelevant.

SHRINKWRAP_CRASH.blend

The shrinkwrap-modifier of the "Dress"-object, targeting the animated "Body"-object seems to cause the error.

The shrinkwrap-type doesnt seem to make any difference. It seems to happen only on animated targets -like this*"Body"*-object. However, the other shrinkwrap-modifier on the*"Belt"*-object doesnt cause any problem......

Here are some Compairison, rendering up to frame 100 between:
blender 3.01...
{F13476736, size=full}

... and 3.30 LTS
(Rendertimes on top of the images)
{F13476738, size=full}

On my computer, it crashed after overflowing both virtual (swap) and physical memory (around frame200).

Exact steps for others to reproduce the error

*1 .Open the file. It shoud look like this..
Screenshot.jpg

*2. Select Menu Render-Render Animation (Ctrl-F12)

*3. Open you taskmanager and watch the memory usage. Let it render for some time. The memory shoud go linear up, until it crashes.

*4. (after crash..) Re-open blender and the file.

*5. Select the "Dress"-named object in the outline tab.
Outliner.jpg

*6. Disable shrinkwrap-modifier on realtime and rendering.
{F13476692, size=full}

*7.Repeat steps 2 and 3
Blender330peakFixed.jpg

Now it shoud work.
Note. if you don`t do step.4 and re-open Blender, there is a change that the memory is not cleared up. However, it shoud not go lineary up like the first time.

*8. Try the same with the "Belt"-named Object. It shoud not make any differences..

Thats It. I hope, this was helpfull.

Happy Bugfixing;)

**System Information** Operating system: Linux-5.15.0-47-lowlatency-x86_64-with-glibc2.35 64 Bits Graphics card: NVIDIA GeForce GTX 980/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 470.141.03 **Blender Version** Broken: version: 3.4.0 Alpha, branch: master, commit date: 2022-09-09 22:58, hash: `352d55b1c8` and older versions after 3.02 Worked: (3.01 and older) **Short description of error** While rendering an animation on a blender version newer than 3.0.1, GPU Subdivision causes blender to leak the memory and crashes while rendering significantly slower. I found this out after stripping down my project to this test file and removed almost everything unrelevant. [SHRINKWRAP_CRASH.blend](https://archive.blender.org/developer/F13476763/SHRINKWRAP_CRASH.blend) The shrinkwrap-modifier of the *"Dress"*-object, targeting the animated *"Body"*-object seems to cause the error. The shrinkwrap-type doesn`t seem to make any difference. It seems to happen only on animated targets -like this*"Body"*-object. However, the other shrinkwrap-modifier on the*"Belt"*-object doesn`t cause any problem...... Here are some Compairison, rendering up to frame 100 between: blender 3.01... {[F13476736](https://archive.blender.org/developer/F13476736/Blender302peak.jpg), size=full} ... and 3.30 LTS (Rendertimes on top of the images) {[F13476738](https://archive.blender.org/developer/F13476738/Blender330peak.jpg), size=full} On my computer, it crashed after overflowing both virtual (swap) and physical memory (around frame200). **Exact steps for others to reproduce the error** *1 .Open the file. It shoud look like this.. ![Screenshot.jpg](https://archive.blender.org/developer/F13476642/Screenshot.jpg) *2. Select Menu Render-Render Animation (Ctrl-F12) *3. Open you taskmanager and watch the memory usage. Let it render for some time. The memory shoud go linear up, until it crashes. *4. (after crash..) Re-open blender and the file. *5. Select the "Dress"-named object in the outline tab. ![Outliner.jpg](https://archive.blender.org/developer/F13476752/Outliner.jpg) *6. Disable shrinkwrap-modifier on realtime and rendering. {[F13476692](https://archive.blender.org/developer/F13476692/disable.jpg), size=full} *7.Repeat steps 2 and 3 ![Blender330peakFixed.jpg](https://archive.blender.org/developer/F13476702/Blender330peakFixed.jpg) Now it shoud work. Note. if you don`t do step.4 and re-open Blender, there is a change that the memory is not cleared up. However, it shoud not go lineary up like the first time. *8. Try the same with the "Belt"-named Object. It shoud not make any differences.. Thats It. I hope, this was helpfull. Happy Bugfixing;)

Added subscriber: @cpt_weissnerd

Added subscriber: @cpt_weissnerd
Member

Added subscriber: @OmarEmaraDev

Added subscriber: @OmarEmaraDev
Member

Changed status from 'Needs Triage' to: 'Needs User Info'

Changed status from 'Needs Triage' to: 'Needs User Info'
Member

I can't reproduce the memory leak myself after rendering 100 frames, also can't detect the leak through memory tracing.
If I understand correctly, this only happens when rendering, not playing the animation or doing anything else.
Does this happen for EEVEE only or also Cycles?

I can't reproduce the memory leak myself after rendering 100 frames, also can't detect the leak through memory tracing. If I understand correctly, this only happens when rendering, not playing the animation or doing anything else. Does this happen for EEVEE only or also Cycles?

Yes. It only happens during the rendering process. Not during the preview.
It primary happens on EEVEE, and as I found out on WORKBENCH to. (the Rendering-Resolution doesnt make a difference) On Cycles, however, it doesnt seem to happen as far as i coud test (The Memory doesn`t seem to go up).

Again. It happens on every Blenderversion that I`ve tested, newer than 3.01 - Including the 3.4.0-alpha that i have downloaded a few minutes ago;)

Note: I only have 16GB RAM. Depending on you Memory-Size, you may not notice the Memory-increase during the first 100 frames.

During my tests. The memory increases linear about 7.6GB after rendering 100 frames on both EEVEE and WORKBENCH.

Hope this helps.

Yes. It only happens during the rendering process. Not during the preview. It primary happens on EEVEE, and as I found out on WORKBENCH to. (the Rendering-Resolution doesn`t make a difference) On Cycles, however, it doesn`t seem to happen as far as i coud test (The Memory doesn`t seem to go up). Again. It happens on every Blenderversion that I`ve tested, newer than 3.01 - Including the 3.4.0-alpha that i have downloaded a few minutes ago;) Note: I only have 16GB RAM. Depending on you Memory-Size, you may not notice the Memory-increase during the first 100 frames. During my tests. The memory increases linear about 7.6GB after rendering 100 frames on both EEVEE and WORKBENCH. Hope this helps.
Member

I can't replicate on either EEVEE or Workbench.
Could it be that your /tmp directory is mounted on RAM? Does changing the output directory resolve the issue?

I can't replicate on either EEVEE or Workbench. Could it be that your `/tmp` directory is mounted on RAM? Does changing the output directory resolve the issue?

No. It doesn`t.
I changed everyting in the "File Pahts" to a path on my harddisk. It changes nothing.

Screenshot at frame 201 right before the crash :
Filepaths_bevore crash.jpg

I also tested everything on a different Machine with Windows10 -also with 16GB RAM- .
I downloaded the latest build of Blender 3.4.0-alpha from there and loaded the file.
The results where the same as on my ubuntu (Memory goes linear up during the rendering):
windowsTest.jpg

During this evening, i managed to create a more simple testfile just using the default objects after testing and trying a lot.
It produces exactly the same issue, but even more efficient.

NewVersion.blend

Just open the file and select "Render Animation"

On blender 3.01, i can render allday.
Newversion_F21_301.jpg

On blender 3.41Alpha and 3.30LTS. It crashes around frame 15
And if you brake up the rendering -like I did here on frame 13- the memory stays high and doesn`t go back immediately.

Newversion_F13_341.jpg

at least on my 16GB RAM- Machine...

If you don`t see anithing on you one, try to increase carefully the first Subsurf-modifier on the Suzanne-Object.
{F13489233 size=full}
Again, Good luck.
I hope this helps.

No. It doesn`t. I changed everyting in the "File Pahts" to a path on my harddisk. It changes nothing. Screenshot at frame 201 right before the crash : ![Filepaths_bevore crash.jpg](https://archive.blender.org/developer/F13488761/Filepaths_bevore_crash.jpg) I also tested everything on a different Machine with Windows10 -also with 16GB RAM- . I downloaded the latest build of Blender 3.4.0-alpha from there and loaded the file. The results where the same as on my ubuntu (Memory goes linear up during the rendering): ![windowsTest.jpg](https://archive.blender.org/developer/F13488774/windowsTest.jpg) During this evening, i managed to create a more simple testfile just using the default objects after testing and trying a lot. It produces exactly the same issue, but even more efficient. [NewVersion.blend](https://archive.blender.org/developer/F13489178/NewVersion.blend) Just open the file and select "Render Animation" On blender 3.01, i can render allday. ![Newversion_F21_301.jpg](https://archive.blender.org/developer/F13489192/Newversion_F21_301.jpg) On blender 3.41Alpha and 3.30LTS. It crashes around frame 15 And if you brake up the rendering -like I did here on frame 13- the memory stays high and doesn`t go back immediately. ![Newversion_F13_341.jpg](https://archive.blender.org/developer/F13489205/Newversion_F13_341.jpg) at least on my 16GB RAM- Machine... If you don`t see anithing on you one, try to increase carefully the first Subsurf-modifier on the Suzanne-Object. {[F13489233](https://archive.blender.org/developer/F13489233/subsurf.jpg) size=full} Again, Good luck. I hope this helps.
Member

I feel like this might be due to GPU Subdivision, which I don't have enabled. Can you try if the issue still happens with Preferences > Viewport > Subdivision > GPU Subdivision disabled?

I feel like this might be due to GPU Subdivision, which I don't have enabled. Can you try if the issue still happens with `Preferences > Viewport > Subdivision > GPU Subdivision` disabled?

!!!JACKPOT!!!!!!

If i turn GPU Subdivision off (it was on on default), blender 3.4.0 renders just normal and doesn`t overflow the memory.
gpusub.jpg

Thanks so far:)

!!!JACKPOT!!!!!! If i turn GPU Subdivision off (it was on on default), blender 3.4.0 renders just normal and doesn`t overflow the memory. ![gpusub.jpg](https://archive.blender.org/developer/F13492648/gpusub.jpg) Thanks so far:)
Member

Changed status from 'Needs User Info' to: 'Needs Triage'

Changed status from 'Needs User Info' to: 'Needs Triage'
Member

Good to hear. My GPU doesn't support GPU Subdivision, that's why I couldn't reproduce, so I will leave this to someone with the needed hardware.

Good to hear. My GPU doesn't support GPU Subdivision, that's why I couldn't reproduce, so I will leave this to someone with the needed hardware.
Omar Emara changed title from shrinkwrap overflows memory while rendering and crashes. to Memory leak with GPU subdivision on rendering 2022-09-15 08:26:22 +02:00

Added subscribers: @kevindietrich, @mano-wii

Added subscribers: @kevindietrich, @mano-wii

Changed status from 'Needs Triage' to: 'Confirmed'

Changed status from 'Needs Triage' to: 'Confirmed'

I can confirm the constant increase in memory (resembling a leak) and consequent crash.

In fact it seems to be related to the GPU subdivision (although without this option there is also a very slight progressive consumption of memory).

Here is a table with a sample of the allocations between two moments:
image.png

Cc @kevindietrich

I can confirm the constant increase in memory (resembling a leak) and consequent crash. In fact it seems to be related to the GPU subdivision (although without this option there is also a very slight progressive consumption of memory). Here is a table with a sample of the allocations between two moments: ![image.png](https://archive.blender.org/developer/F13499720/image.png) Cc @kevindietrich

Added subscriber: @Jacob9

Added subscriber: @Jacob9

shrinkwrap_memory_leak.blend

I just encountered this bug as well. Here is a minimal blend file which reproduces the leak (with GPU subdivision enabled).

However I have found a strange workaround, the bug only occurs when the target object has the subdivision modifier in the last slot. If there is another active modifier after the subdivision, the bug does not occur. In the attached blend file I have put an array modifier in the last slot of the object that is the target of the shrinkwrap and I have left it disabled, but if you enable it and render animation, the bug disappears.

[shrinkwrap_memory_leak.blend](https://archive.blender.org/developer/F13510437/shrinkwrap_memory_leak.blend) I just encountered this bug as well. Here is a minimal blend file which reproduces the leak (with GPU subdivision enabled). However I have found a strange workaround, the bug only occurs when the target object has the subdivision modifier in the last slot. If there is another active modifier after the subdivision, the bug does not occur. In the attached blend file I have put an array modifier in the last slot of the object that is the target of the shrinkwrap and I have left it disabled, but if you enable it and render animation, the bug disappears.
Member

Added subscriber: @Jeroen-Bakker

Added subscriber: @Jeroen-Bakker
Member

@Jacob9 that is expeceted. GPU subdivision only kicks in when the subdivision modifier is the last modifier.

I will check with ASAN if I can find anything.

@Jacob9 that is expeceted. GPU subdivision only kicks in when the subdivision modifier is the last modifier. I will check with ASAN if I can find anything.
Jeroen Bakker self-assigned this 2022-11-07 15:32:24 +01:00
Jeroen Bakker removed their assignment 2022-11-08 09:53:34 +01:00
Member

Unassigning for now as my linux system needs to be reinstalled.

Unassigning for now as my linux system needs to be reinstalled.

@Jeroen-Bakker, I quickly investigated, the leak is because we garbage collect the subdivision caches, but the caches are only freed at the end of the animation render (via DRW_cache_free_old_subdiv()). We would need to free the caches at the end of each frame, but I am not really sure where would the best place for that be.

@Jeroen-Bakker, I quickly investigated, the leak is because we garbage collect the subdivision caches, but the caches are only freed at the end of the animation render (via `DRW_cache_free_old_subdiv()`). We would need to free the caches at the end of each frame, but I am not really sure where would the best place for that be.
Member

@kevindietrich thanks for looking into this. In the viewport this is done just after drawing a single view.
For viewport animation and final render we could do it at the time the results are downloaded.

@kevindietrich thanks for looking into this. In the viewport this is done just after drawing a single view. For viewport animation and final render we could do it at the time the results are downloaded.
Jeroen Bakker self-assigned this 2022-11-11 08:14:40 +01:00

This issue was referenced by 2eabe0a320

This issue was referenced by 2eabe0a320e820081b4dc34a8409ce02c2edec66

This issue was referenced by 88c956c13b

This issue was referenced by 88c956c13be40aeaaf3828369ab63801413fef82
Member

Changed status from 'Confirmed' to: 'Resolved'

Changed status from 'Confirmed' to: 'Resolved'
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
7 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#100969
No description provided.