Possible Asset Browser Memory Leak #89038

New Issue

Amelia Wheeler · 2021-06-10T20:58:34+02:00

Amelia Wheeler commented

2021-06-10 20:58:34 +02:00

System Information
Operating system: Windows 10
Graphics card: 4095MB NVIDIA GeForce GTX 1070
Blender Version
Broken: 3.00 Alpha, 8c3f4f7edf, master, 2020-06-10
Worked: Unknown. This bug is relating to the WIP Asset Browser project, which didn't exist in earlier versions. (Unless I'm mistaken and it was introduced in 2.93?)

Short description of error
Assets (At least Images and World Shader setups using HDRI's) are not being unloaded from memory (staying around in the image buffer (imb_addrectfloatImBuf) specifically), causing Blender's memory usage to go up to as high as 20~GB at least on my machine, if not crash outright. As far as I can tell this only happens (at least to this extent) when assets are using unpacked image files. This appears to happen regardless of image file size, with maybe 800 MB of HD images still taking up 5~GB of memory. For reference I'm using 4 HDRI's taken from https://hdrihaven.com/ ranging from 80MB to 300MB in size. Though the one that causes the most memory usage seems to be the image with the highest resolution but not file size. My original thought was that it had to do with how Blender was generating thumbnail images for the assets (possibly similar to what's being talked about in #83336?), but I'm not familiar enough with Blenders internal processes to actually say either way.

Exact steps for others to reproduce the error

Create a new file. Add a couple HDRI images of any size but don't pack them into the .Blend file. Turn those images into World shaders setups. Mark the images and World shaders as assets. If the memory usage hasn't already spiked, switching whatever World shader is being used in the scene or renaming any of the image/node seems to always cause it to at this point.

**System Information** Operating system: Windows 10 Graphics card: 4095MB NVIDIA GeForce GTX 1070 **Blender Version** Broken: 3.00 Alpha, 8c3f4f7edf9d, master, 2020-06-10 Worked: Unknown. This bug is relating to the WIP Asset Browser project, which didn't exist in earlier versions. (Unless I'm mistaken and it was introduced in 2.93?) **Short description of error** Assets (At least Images and World Shader setups using HDRI's) are not being unloaded from memory (staying around in the image buffer (imb_addrectfloatImBuf) specifically), causing Blender's memory usage to go up to as high as 20~GB at least on my machine, if not crash outright. As far as I can tell this only happens (at least to this extent) when assets are using unpacked image files. This appears to happen regardless of image file size, with maybe 800 MB of HD images still taking up 5~GB of memory. For reference I'm using 4 HDRI's taken from https://hdrihaven.com/ ranging from 80MB to 300MB in size. Though the one that causes the most memory usage seems to be the image with the highest resolution but not file size. My original thought was that it had to do with how Blender was generating thumbnail images for the assets (possibly similar to what's being talked about in #83336?), but I'm not familiar enough with Blenders internal processes to actually say either way. **Exact steps for others to reproduce the error** Create a new file. Add a couple HDRI images of any size but don't pack them into the .Blend file. Turn those images into World shaders setups. Mark the images and World shaders as assets. If the memory usage hasn't already spiked, switching whatever World shader is being used in the scene or renaming any of the image/node seems to always cause it to at this point.

Amelia Wheeler commented

2021-06-10 20:58:34 +02:00

Added subscriber: @Ames

Germano Cavalcante commented

2021-07-09 22:06:32 +02:00

Added subscriber: @mano-wii

Germano Cavalcante commented

2021-07-09 22:06:32 +02:00

If I'm not mistaken, #83336 refers more to icons saved in HD. This report seems to refer to RAM memory.
But 20GB seems unrealistic.
Is it just an icon or is it the entire file that becomes an asset that can be accessed and added to the .blend file?
Adding HDRI images, even unpackaged, requires a fast-access copy of the texture in memory. I don't know how this can be avoided.
Blender has a garbage collector that clears the cache of some unused textures. You only need to wait 1 minute or less for the memory to be saved.

But I'm not familiar with the assets code, this might actually be a bug in the end.

If I'm not mistaken, #83336 refers more to icons saved in HD. This report seems to refer to RAM memory. But 20GB seems unrealistic. Is it just an icon or is it the entire file that becomes an asset that can be accessed and added to the .blend file? Adding HDRI images, even unpackaged, requires a fast-access copy of the texture in memory. I don't know how this can be avoided. Blender has a garbage collector that clears the cache of some unused textures. You only need to wait 1 minute or less for the memory to be saved. But I'm not familiar with the assets code, this might actually be a bug in the end.

lllllllllll commented

2021-07-15 19:16:58 +02:00

Added subscriber: @LJ3D

lllllllllll commented

2021-07-20 19:38:41 +02:00

While I am unsure of the details of why, I too experience RAM usage which seems unreasonably high when generating thumbnails.
I run out of memory completely (I have 16gb of RAM, with maybe 5gb or so being used up by windows) if I try and put more than about 30 4K materials into the asset browser (With a size on disk thats only around 1.25GB).

While 30 4K materials is a reasonable amount of materials, I still think 10gb+ of RAM usage for 1.25gb of textures is a bit strange and it gets in the way of creating large libraries of materials

I am using windows, if you want any more info then I'm happy to help.

While I am unsure of the details of why, I too experience RAM usage which seems unreasonably high when generating thumbnails. I run out of memory completely (I have 16gb of RAM, with maybe 5gb or so being used up by windows) if I try and put more than about 30 4K materials into the asset browser (With a size on disk thats only around 1.25GB). While 30 4K materials is a reasonable amount of materials, I still think 10gb+ of RAM usage for 1.25gb of textures is a bit strange and it gets in the way of creating large libraries of materials I am using windows, if you want any more info then I'm happy to help.

Amelia Wheeler commented

2021-07-20 20:00:11 +02:00

In #89038#1193852, @LJ3D wrote:

I am using windows, if you want any more info then I'm happy to help.

That's actually a good point, I'll check later whether or not the behavior is the same on Linux.

> In #89038#1193852, @LJ3D wrote: > > I am using windows, if you want any more info then I'm happy to help. That's actually a good point, I'll check later whether or not the behavior is the same on Linux.

lllllllllll commented

2021-07-23 15:12:43 +02:00

In #89038#1193855, @Ames wrote:
That's actually a good point, I'll check later whether or not the behavior is the same on Linux.

Just checked myself, and the behaviour is in fact the same on Linux.

> In #89038#1193855, @Ames wrote: > That's actually a good point, I'll check later whether or not the behavior is the same on Linux. Just checked myself, and the behaviour is in fact the same on Linux.

Philipp Oeser commented

2022-02-23 15:05:54 +01:00

Added subscribers: @JulianEisel, @lichtwerk

Philipp Oeser commented

2022-02-23 15:05:54 +01:00

Changed status from 'Needs Triage' to: 'Needs Developer To Reproduce'

Philipp Oeser commented

2022-02-23 15:05:54 +01:00

{F12884211 size=full}
@JulianEisel: does this mean you could/can confirm?
(personally I cant using the steps provided, checked in master)

@LJ3D : could you also re-check in a fresh 3.1 / 3.2 build from https://builder.blender.org/download/daily/?

{[F12884211](https://archive.blender.org/developer/F12884211/image.png) size=full} @JulianEisel: does this mean you could/can confirm? (personally I cant using the steps provided, checked in master) @LJ3D : could you also re-check in a fresh 3.1 / 3.2 build from https://builder.blender.org/download/daily/?

Philipp Oeser removed the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Possible Asset Browser Memory Leak #89038