Delete Hierarchy - To delete a hierarchy takes very long #60419

New Issue

Marcus Papathoma · 2019-01-11T13:05:16+01:00

Marcus Papathoma commented

2019-01-11 13:05:16 +01:00

System Information
Operating system: Windows 10 Pro x64
Graphics card: Geforce GTX 1080Ti

Blender Version
Broken: all
Worked: never

Short description of error
Delete Hierarchy - To delete a hierarchy takes very long.

Exact steps for others to reproduce the error
I now had the time to prepare a testscene. At the Blender Conferece I think Sergey said, we should send a file to reproduce the problem. He or another developer will have a look!
The scene reflects the object and polycount, we have in our production files. We often have to delete a hierarchy with lots of objects.
In other programs we use, with the same poly and object count, delete hierarchy works in a second. So the object or polycount couldn´t be an excuse.
delete_hierarchy_test.blend
Open the attached scene
Right click select Row7 in the outliner.
Choose delete hierarchy.
Count the time that it takes do delete. On my system it was about 20seconds. Thats a lot for only deleting!
Please make delete hierarchy faster!

I tested the attached scene on two different computers.
One at work. AMD Threadripper 1950X 16Cores@3,4 GHz - here it takes 20sec to delete the Row7 hierarchy.
One at home. Intel Xeon E5-2697v3 2x14Cores@2,6GHz - here it takes only 12sec to delete the Row7 hierarchy.
But that is only one hierarchy. Deleting multiple hierarchies takes longer. I thing it depends on the count of objects you delete.

I hope it is worth to investigate it further?!

**System Information** Operating system: Windows 10 Pro x64 Graphics card: Geforce GTX 1080Ti **Blender Version** Broken: all Worked: never **Short description of error** Delete Hierarchy - To delete a hierarchy takes very long. **Exact steps for others to reproduce the error** I now had the time to prepare a testscene. At the Blender Conferece I think Sergey said, we should send a file to reproduce the problem. He or another developer will have a look! The scene reflects the object and polycount, we have in our production files. We often have to delete a hierarchy with lots of objects. In other programs we use, with the same poly and object count, delete hierarchy works in a second. So the object or polycount couldn´t be an excuse. [delete_hierarchy_test.blend](https://archive.blender.org/developer/F6253159/delete_hierarchy_test.blend) Open the attached scene Right click select Row7 in the outliner. Choose delete hierarchy. Count the time that it takes do delete. On my system it was about 20seconds. Thats a lot for only deleting! Please make delete hierarchy faster! I tested the attached scene on two different computers. One at work. AMD Threadripper 1950X 16Cores@3,4 GHz - here it takes 20sec to delete the Row7 hierarchy. One at home. Intel Xeon E5-2697v3 2x14Cores@2,6GHz - here it takes only 12sec to delete the Row7 hierarchy. But that is only one hierarchy. Deleting multiple hierarchies takes longer. I thing it depends on the count of objects you delete. I hope it is worth to investigate it further?!

Marcus Papathoma commented

2019-01-11 13:05:16 +01:00

Added subscriber: @machieb

John commented

2019-01-11 18:34:41 +01:00

Added subscriber: @jfraser

John commented

2019-01-11 18:34:41 +01:00

@machieb What are your hardware specs, on my desktop machine ( i7@3.4GHz, 16GB RAM ) using git commit: blender-2.80.0-git.80ffa73b3f87 it takes around 10 seconds?

Marcus Papathoma commented

2019-01-12 17:38:26 +01:00

@jfraser: I testest it at work. My computer there is a AMD Threadripper 1950X 16Cores@3,4GHz 64GB Ram using the build from yesterday.
Now I tested it at home. At my desktop Intel Xeon E5-2697v3 (2x14Cores@2,6Ghz 128GB RAM) it takes about 12seconds deleting the hierarchy using the build from today and the scene from above.

I opend this ticket because deleting something takes too much time, especially in larger scenes. Maybe the task is only single threaded?
I also started another ticket with the same scene. There you can see that undo takes to much time. The undo of a cube rotation takes at a lot of time. I think it should be investigated.
At work we use other 3D packages where an undo with the same poly and objectcount takes only a few seconds.

@jfraser: I testest it at work. My computer there is a AMD Threadripper 1950X 16Cores@3,4GHz 64GB Ram using the build from yesterday. Now I tested it at home. At my desktop Intel Xeon E5-2697v3 (2x14Cores@2,6Ghz 128GB RAM) it takes about 12seconds deleting the hierarchy using the build from today and the scene from above. I opend this ticket because deleting something takes too much time, especially in larger scenes. Maybe the task is only single threaded? I also started another ticket with the same scene. There you can see that undo takes to much time. The undo of a cube rotation takes at a lot of time. I think it should be investigated. At work we use other 3D packages where an undo with the same poly and objectcount takes only a few seconds.

Marcus Papathoma changed title from ~~Delete Hierarchy - To delete a hierarchy takes more than 30 seconds~~ to Delete Hierarchy - To delete a hierarchy takes very long

2019-01-14 10:28:46 +01:00

Bastien Montagne commented

2019-01-15 11:42:10 +01:00

Added subscriber: @mont29

Bastien Montagne commented

2019-01-15 11:42:10 +01:00

Changed status from 'Open' to: 'Archived'

Bastien Montagne closed this issue

2019-01-15 11:42:10 +01:00

Bastien Montagne self-assigned this 2019-01-15 11:42:10 +01:00

Bastien Montagne commented

2019-01-15 11:42:10 +01:00

Thanks for the report, but this is not a bug at all. You cannot expect proper deletion of 1k objects to happen that quick, first of all because it's not actually deleting the object, it's unlinking it from current collection, and then, if object is not used anymore, it is deleted. You also have to handle (i.e. check) for possible links to/from all other IDs (among which, another 6k of objects).

Note that there is absolutely no reason to use real objects for that kind of things, modifiers and/or instantiation should be used instead. Blender objects are very high-level, advanced and complex data, typically not designed to be used in huge amounts of identical copies.

Thanks for the report, but this is not a bug at all. You cannot expect proper deletion of 1k objects to happen that quick, first of all because it's not actually deleting the object, it's unlinking it from current collection, and then, if object is not used anymore, it is deleted. You also have to handle (i.e. check) for possible links to/from all other IDs (among which, another 6k of objects). Note that there is absolutely no reason to use real objects for that kind of things, modifiers and/or instantiation should be used instead. Blender objects are very high-level, advanced and complex data, typically not designed to be used in huge amounts of identical copies.

Brecht Van Lommel commented

2019-01-15 11:44:30 +01:00

Added subscriber: @brecht

Brecht Van Lommel commented

2019-01-15 11:44:30 +01:00

I would say that we can do much better here, and this is something that is worth improving. But the current behavior is expected given the current code architecture, and so outside the scope of a bug report.

Marcus Papathoma commented

2019-01-15 12:54:12 +01:00

@mont29 : Thanks for your answer. As I wrote in the description, I tried to create a simple scene that mimics the scenes we use every day in our company. I can´t send you a 2GB blend file with client data. Therefore a used the default cube subdivided it and used the array modifier to create an amount of objects, to get an object count like in our scenes. The other rows of cubes were only linked copies, to hold the filesize low.
As brecht wrote it is apparently a limitation of the blender code architecture, that deleting takes a lot of time, if you have a high object and polycount in a scene.

@brecht : Thanks for your answer. I hope in the future you can improve that part of blender!

@mont29 : Thanks for your answer. As I wrote in the description, I tried to create a simple scene that mimics the scenes we use every day in our company. I can´t send you a 2GB blend file with client data. Therefore a used the default cube subdivided it and used the array modifier to create an amount of objects, to get an object count like in our scenes. The other rows of cubes were only linked copies, to hold the filesize low. As brecht wrote it is apparently a limitation of the blender code architecture, that deleting takes a lot of time, if you have a high object and polycount in a scene. @brecht : Thanks for your answer. I hope in the future you can improve that part of blender!

Bastien Montagne commented

2019-01-17 12:42:35 +01:00

32dc085289 adds a hidden experimental speed-up for that kind of massive deletion (about 20% in cased described here, this will vary a lot depending on amount of datablocks in the .blend file, and amount of collections etc.). To test it, you'll have to set debug value to 666 (F3, search for Debug Menu operator, and type 666 in numfield of the pop-up that appears).

Again, this is experimental, some quick tests seem to show it's working, but hard to tell with all the complicated cases of real life files. So use (test) it with precautions, assuming it might corrupted your .blend or crash blender, for now. ;)

32dc085289 adds a hidden experimental speed-up for that kind of massive deletion (about 20% in cased described here, this will vary a lot depending on amount of datablocks in the .blend file, and amount of collections etc.). To test it, you'll have to set debug value to 666 (F3, search for `Debug Menu` operator, and type `666` in numfield of the pop-up that appears). Again, this is *experimental*, some quick tests seem to show it's working, but hard to tell with all the complicated cases of real life files. So use (test) it with precautions, assuming it might corrupted your .blend or crash blender, for now. ;)

Marcus Papathoma commented

2019-01-17 13:21:48 +01:00

@mont29 : Thank you very much! I will test it!

Marcus Papathoma commented

2019-01-21 10:51:49 +01:00

@mont29: Today I´ve tested your hidden experimental speed-up for delete hierarchy with two of our production files.
Scene 1:
Size: 2,12GB
18.457,133 Faces
18.457,134 Tris
6590 Objekte
4043 Objekte deleted

normal delete hierarchy: 28seconds
experimental speed-up: 20seconds
speedup: 28,57%

Scene 2:
Size: 2,13GB
19.601,078 Faces
19.601,788 Tris
8546 Objekte
4033 Objekte deleted

normal delete hierarchy: 48seconds
experimental speed-up: 34seconds
speedup: 29,16%

There were no links in these files only geometry in empties and collections.
I saved an opened the files no crash.
Tell me if I can help testing further and if you need more information.

It is a very good speedup. Thanks for your effort!

@mont29: Today I´ve tested your hidden experimental speed-up for delete hierarchy with two of our production files. Scene 1: Size: 2,12GB 18.457,133 Faces 18.457,134 Tris 6590 Objekte 4043 Objekte deleted normal delete hierarchy: 28seconds experimental speed-up: **20seconds** **speedup: 28,57%** Scene 2: Size: 2,13GB 19.601,078 Faces 19.601,788 Tris 8546 Objekte 4033 Objekte deleted normal delete hierarchy: 48seconds experimental speed-up: **34seconds** **speedup: 29,16%** There were no links in these files only geometry in empties and collections. I saved an opened the files no crash. Tell me if I can help testing further and if you need more information. It is a very good speedup. Thanks for your effort!

Bastien Montagne commented

2019-01-21 19:58:14 +01:00

@machieb Thanks for testing! Ideally, doing some tests with more complex production files (involving armatures and/or linked data) would also be nice… Did some here, but I cannot cover all cases. :)

Otherwise, I think that within a few days I will just invert the debug value effect (i.e. enable new code by default), and we will see. Better do that soon enough so that it get well tested while we are still in beta phase. Will also investigate adding this to other delete operations (when they involve a significant amount of IDs).

Note that number of faces should not have any effect here, what is slow is going over all IDs (objects, but also collections, etc.) to check and remove references to deleted ones.
Also related, speed-up will become more and more important when you delete a bigger amount of IDs (i.e. if you delete 95% of your objects at once, it could be 2-3 times quicker, while if you only delete 5% of the objects, speed-up will be marginal at best).

@machieb Thanks for testing! Ideally, doing some tests with more complex production files (involving armatures and/or linked data) would also be nice… Did some here, but I cannot cover all cases. :) Otherwise, I think that within a few days I will just invert the debug value effect (i.e. enable new code by default), and we will see. Better do that soon enough so that it get well tested while we are still in beta phase. Will also investigate adding this to other delete operations (when they involve a significant amount of IDs). Note that number of faces should not have any effect here, what is slow is going over all IDs (objects, but also collections, etc.) to check and remove references to deleted ones. Also related, speed-up will become more and more important when you delete a bigger amount of IDs (i.e. if you delete 95% of your objects at once, it could be 2-3 times quicker, while if you only delete 5% of the objects, speed-up will be marginal at best).

Marcus Papathoma commented

2019-01-24 11:32:23 +01:00

@mont29 : I tested another production file with linked data. Sorry we don´t use amatures in our big production files, so I couldn´t test that usecase.
In the tested scene there are 8893 objects and I deleted more than half of them (4850objects). Without the optimization it took 1min:58sec and with your new code 1min:26sec.
I saved and reopened the file and couldn´d find any problems or issues.

@mont29 : I tested another production file with linked data. Sorry we don´t use amatures in our big production files, so I couldn´t test that usecase. In the tested scene there are 8893 objects and I deleted more than half of them (4850objects). Without the optimization it took 1min:58sec and with your new code 1min:26sec. I saved and reopened the file and couldn´d find any problems or issues.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Delete Hierarchy - To delete a hierarchy takes very long #60419