Page MenuHome

Python/BMesh: Add/removing textures between renders causes memory leak/ram crash
Open, Confirmed, MediumPublic

Description

System Information
Operating system and graphics card
Ubuntu 18.04, Nvidia GTX 960M

Blender Version
Broken: blender-2.80-17bc0567268-linux-glibc219-x86_64

Working Blender Version
Blender 2.79b stable (ubuntu repository) with cycles

Short description of error
If you run the attached code and target it a directory with a large number of images, it will repeatedly:

  1. Load an image from file
  2. Set the image to be the content of a "Image Texture" node
  3. Delete/remove the image that use to be the image texture node
  4. Render

As it processes images, blender takes more and more memory. Eventually it will crash. If you leave out step 4, then it will not.

Exact steps for others to reproduce the error
Based on a (as simple as possible) attached .blend file with minimum amount of steps

Import python code, run

LoadAndRemoveTextures("/folderWithManyImages", 2000)

in the console window.


Code:

import datetime
import sys

import bpy
import bmesh
from mathutils import Vector
from mathutils import Euler
from bpy_extras.object_utils import world_to_camera_view
import random
import math
import json

import time
import os
import gc

def RemoveImageFromMemory (passedName):
    # Extra test because this can crash Blender.
    img = bpy.data.images[passedName]
    try:
        img.user_clear()
        can_continue = True
    except:
        can_continue = False
    
    if can_continue == True:
        try:
            bpy.data.images.remove(img, True)
            result = True
        except:
            result = False
    else:
        result = False
    return result

def CollectBackgroundImagePaths(sourceDirectoryPath, imageExtension):
    file_paths = []

    for root, dirs, files in os.walk(sourceDirectoryPath):
        for file_name in files:
            if file_name.endswith(imageExtension):
                file_paths.append(root + "/" + file_name)
    return file_paths

def SetBackgroundImage(backgroundImagePath):
    #Store reference to old image so it can be removed
    old_image_name = bpy.data.materials[0].node_tree.nodes.get("Image Texture").image.name

    #Load new image to be used
    new_image = bpy.data.images.load(filepath = backgroundImagePath)

    #Set the new image to be used
    bpy.data.materials[0].node_tree.nodes.get("Image Texture").image = new_image

    #Delete the old one from the file
    remove_succeeded = RemoveImageFromMemory(old_image_name)
    gc.collect()

    if not remove_succeeded:
        print("Error removing " + old_image_name)
    else:
        print("Removing " + old_image_name + " worked")

def LoadAndRemoveTextures(texturesFolderPath, numberToLoad):
    background_image_paths = CollectBackgroundImagePaths(texturesFolderPath, ".png")

    for example_index in range(0, numberToLoad):
        SetBackgroundImage(random.choice(background_image_paths))

        #If you alternate adding/removing images and rendering, it will eat all the ram on your system and crash
        bpy.ops.render.render( write_still=True )

Details

Type
Bug

Event Timeline

Further investigation reveals that switching the renderer from Eevee to cycles also makes it stop eating all the ram, so it seems to be Eevee specific.

Bastien Montagne (mont29) lowered the priority of this task from Needs Triage by Developer to Confirmed, Medium.Jul 25 2018, 4:00 PM

Hrmmpfs… this is crashing here, in the scene_update call from render operator… depsgraph tries to access freed image ID, which is weird since relations are tagged to be updated in ID relink code.

No time to investigate further, but there are definitively issues here!

If I may ask, is there anything I can do to bump up priority on this? I'm doing some deep learning research that requires 3 million+ examples to be generated and switching from Eevee to Cycles with low samples has made rendering time go from 2.5 days to something like 10 days while reducing quality.

Can I post a bounty or something?

I have found that this causes the Syncing memory to keep going up unbounded. On first render, memory peak is around 4000Mb, on the second it goes up to 8000Mb. This has completely killed my ability to render my scene.

Philipp Oeser (lichtwerk) renamed this task from Add/removing textures between renders causes memory leak/ram crash to Python/BMesh: Add/removing textures between renders causes memory leak/ram crash.Tue, Nov 5, 12:32 PM