Page MenuHome

VideoTexture module repeated loading of images causes memory leak
Closed, ResolvedPublic

Description

System Information
Mac OS X 10.10.1
MacBook Pro 15in Early 2011 2.2 GHz Core i7 16GB Ram, Intel HD3000 / Radeon 6750M

Blender Version
Broken: 2.27b hash: 9e963ae 2014-10-21 11:38
Worked: 2.69.0 r60991 Mac 64bit

Short description of error
Repeatedly loading images from disk using the VideoTexture module will cause the ram usage of Blender to constantly increase while running the scene under the BGE. This leads eventually to the BGE hanging while running requiring a force quit.

Exact steps for others to reproduce the error

  1. Open Blender 2.72b
  2. Load the attached .blend file.
  3. Run the BGE by pressing P (The scene will automatically start cycling through textures.)
  4. Observe the blender process memory usage in a system monitoring application. (Activity Monitor on OS X)

Included in the folder is a screen cap showing the memory use while running. I did not observe this in 2.69.0 with over 1,000 repeated loads of images.

Event Timeline

Joshua Dolan (jdolan) raised the priority of this task from to Needs Triage by Developer.
Joshua Dolan (jdolan) updated the task description. (Show Details)
Joshua Dolan (jdolan) set Type to Bug.

I can confirm the bug.

Nothing seems to help. Tried Texture.close().

Tried also this:

if not hasattr(logic, "texture"):
    mat_id = texture.materialID(obj, "IMDetails000.png")
    logic.texture = texture.Texture(obj, mat_id)
    logic.texture.source = texture.ImageFFmpeg(img_name)
else:
    logic.texture.source.reload(img_name)
logic.texture.refresh(True)

So even just reloading will cause the memory leak.

Still present in 2.74rc and release.

Benoit,

As you are touching again VideoTexture module, could you investigate this bug?

Update: While investigating this a bit more it seems that it is the .png images that are causing the problem. Exporting the same images as .jpg allows the textures to be swapped continuously without any issues. (I stopped the loop at 3,500 where as with the original .png files it will crash between 250 and 254.) I will continue to test with other image types that FFMpeg supports.

Edit:
Tested the following formats with these results yesterday -
Image Type, Number of loads, Result, Number of tests
jpeg, 3,500, no crash, 5
jpeg 2k, ~318, crash, 3
png, ~250, crash, 20+
tiff lzw, ~300, crash, 2
tiff, ~258, crash, 2
tga rle, 1,000+, no crash, 2

Jorge Bernal (lordloki) lowered the priority of this task from Needs Triage by Developer to Normal.May 15 2015, 3:10 PM
Aaron Carlisle (Blendify) triaged this task as Confirmed, Medium priority.Jun 7 2015, 2:25 AM

I think that T43033 can be merged seems like the same issue

I haven't check it but perhaps:
thread_count = 1 --> stopCache() + m_isThreaded = false;

Hi! I tested your solution but it doesn't work. The problem seems related only to PNG decoding. You can delete this lines:

foreach ($list as $item) {
if (m_isImage)
            {
                // If we're an image, we're probably not going to be here often,
                // so we don't want to deal with delayed frames from threading.
                // There might be a better way to handle this, but I'll leave that
                // for people more knowledgeable with ffmpeg than myself. We don't
                // need threading for a single image anyways.
                m_codecCtx->thread_count = 1;
            }
  work_miracles($item);
}

and if you replace png images with jpg, it will work without problem. The memory leak is due to threading but I had a look at AVCodecContext (m_codecCtx type = AVCodecContext) API and found no simple way to end the threads. I read somewhere that new ffmpeg libraries have a better support of PNG but I think it's complex to update libraries (how many files to modify)? Another (maybe stupid) idea would be to convert png images in another format before decoding (I guess it will take some time). But threading with an external API is too complex to manage in my opinion. So I'll try the idea to convert png before decoding (but I don't know if it's possible and efficient). I'm not experimented so you may have better ideas :) Good day/night! A link I found on the net: https://trac.ffmpeg.org/ticket/4404

Hi! I propose this fix: http://www.pasteall.org/59457/diff

The problem is that the new while loop can turn in an infinite while loop if tha data can't definitely be read. But we can make a counter m_counter = 0; m_counter+=1 (at the end of the while loop), change while ((input->data[0]==0 && input->data[1]==0 && input->data[2]==0 && input->data[3]==0) || m_counter < 10). And after the while loop, m_counter = 0; (or something like that)

Sorry about my poor english. Good day/night!

http://www.pasteall.org/59472/diff (That's what I thought about...). From what I have read about png decoding with certain ffmpeg versions, to properly decode png you can use the codecContext->thread_count = 1 workaround but in Blender it can cause memory leak, Or you can try to decode png several time until this is finally decoded properly. That's the solution I used. It's not elegant but it seems that works. What do you think about it? I'll ask IRC to have other devs opinion about that.