Page MenuHome

OBJ importer run out of memory fix
Closed, ArchivedPublic

Description

The current OBJ importer script is not very memory friendly. On a medium large scene (100 mb, 1M faces, 1.5K groups) it runs out of memory on my 16 gig machine.

After a discussion with Campbell Barton itired to refactor the split_mesh method to be a generator, this worked and was fast but it did lead to a possibility of unwanted splitting of objects.

I profiled the prblem some more and found the real root cause: vert_remap = [-1] * len(verts_loc)
This initis a lists with in my case over a million values. take a few of those lists and there is my problem.

I changed the list to a dict under the assumptions:
a) preformance is not much worse
b) when splitting the actual usage of the space if sparse. (vertices remapped are about (tot_vers / number_of_objects)
c) key being an integer should be pretty good in terms of preformance (not sure if this is case for python)

For the files I have to test with this holds. It is almost as fast as fast as the naive generator and faster then it was.

Added a patch that changes 2 lines and does not change code semanticly.


Details

Type
Patch

Event Timeline

applied r3084 with some edits for faster access.