Depsgraph problem #38041

New Issue

Sam Vila · 2014-01-03T02:28:43+01:00

Sam Vila commented

2014-01-03 02:28:43 +01:00

Hi, I´m testing the depsgraph because this was a problem I had with Blender before and I hit the limite sudently when I wa trying to handle complex scenes with many many objects so I was interested in see what depsgraph could do. So I did a test that you can find in here:

degraphtest.blend

Basically just open the blend file and hit run script and it will do everything for you, calculate the time that it takes on a previous version of Blender and compare with a depsgraph version. If you open the taks manager and see the performance of the CPUs you can see the big difference in the flow of data, in previous versions you see only 1 cpu really working while in the new version is using all my cores BUT! the performance is basically the same! It took me 47 seconds to finish the operation in an old version of Blender and it´s taking me exactly the same amount even if I´m using now all the cores! Shouldn´t be far faster doing this operations? The graphs shows that the data is flowing much more but maybe is not doing the right calculations or is getting into somekind of bucle?!

Please confirm or clarify this. Thanks

System Information
W7, 64 bit. Nvidia GTX 460

Blender Version

Short description of error

Exact steps for others to reproduce the error

Run the script and analize the CPU results.

Hi, I´m testing the depsgraph because this was a problem I had with Blender before and I hit the limite sudently when I wa trying to handle complex scenes with many many objects so I was interested in see what depsgraph could do. So I did a test that you can find in here: [degraphtest.blend](https://archive.blender.org/developer/F60802/degraphtest.blend) ![depsgraph_CPU.jpg](https://archive.blender.org/developer/F60803/depsgraph_CPU.jpg) Basically just open the blend file and hit run script and it will do everything for you, calculate the time that it takes on a previous version of Blender and compare with a depsgraph version. If you open the taks manager and see the performance of the CPUs you can see the big difference in the flow of data, in previous versions you see only 1 cpu really working while in the new version is using all my cores BUT! the performance is basically the same! It took me 47 seconds to finish the operation in an old version of Blender and it´s taking me exactly the same amount even if I´m using now all the cores! Shouldn´t be far faster doing this operations? The graphs shows that the data is flowing much more but maybe is not doing the right calculations or is getting into somekind of bucle?! Please confirm or clarify this. Thanks **System Information** W7, 64 bit. Nvidia GTX 460 **Blender Version** **Short description of error** **Exact steps for others to reproduce the error** Run the script and analize the CPU results.

Sam Vila commented

2014-01-03 02:28:43 +01:00

Changed status to: 'Open'

Sam Vila commented

2014-01-03 02:28:43 +01:00

Added subscriber: @samvila

Sergey Sharybin was assigned by Thomas Dinges

2014-01-03 02:32:38 +01:00

Thomas Dinges commented

2014-01-03 02:32:38 +01:00

Added subscriber: @ThomasDinges

Joshua Leung commented

2014-01-03 03:00:35 +01:00

Added subscriber: @JoshuaLeung

Joshua Leung commented

2014-01-03 03:00:35 +01:00

From the looks of things, the multithreading we perform for object updating is not really the bottleneck in your file.

Rather, the bottlenecks are still in the tools/operators that your script runs. Those are still practically single-threaded, and will still need to be executed sequentially (i.e. in the order that you've got them in the file). Furthermore, the specific operations being performed here are mainly about geometry creation and splitting, as well as object creation (which has a notable O(n^2) bottleneck in the step where it tries to sort the newly created object somewhere in the list of objects - see tamito's thread on the bf-committers mailing list about Freestyle scene creation slowness for more details).

The multiple core use you're seeing happens between each operator call, when the scene and its data is recalculated.

From the looks of things, the multithreading we perform for object updating is not really the bottleneck in your file. Rather, the bottlenecks are still in the tools/operators that your script runs. Those are still practically single-threaded, and will still need to be executed sequentially (i.e. in the order that you've got them in the file). Furthermore, the specific operations being performed here are mainly about geometry creation and splitting, as well as object creation (which has a notable O(n^2) bottleneck in the step where it tries to sort the newly created object somewhere in the list of objects - see tamito's thread on the bf-committers mailing list about Freestyle scene creation slowness for more details). The multiple core use you're seeing happens between each operator call, when the scene and its data is recalculated.

Sam Vila commented

2014-01-03 03:12:02 +01:00

Hi aligorith, I get your point but if that's the case then it should be much resource efficient. Because as it's working right now and base on your reply looks like even if it's mainly a single-threaded operation is also still trying to get as much profit as possible on the CPU cores but the result is exactly the same. My point is... if it's going to be the same result it should also be using the same resources as before. I don't know why all my cores are at 100% now but I don't get any significant variation in the time. It would be better to stress less the CPU if at the end is going to be exactly the same.

Ideally I would love to see operations as this one to be multi-threaded and taking full advantage of all the cores.

Hi aligorith, I get your point but if that's the case then it should be much resource efficient. Because as it's working right now and base on your reply looks like even if it's mainly a single-threaded operation is also still trying to get as much profit as possible on the CPU cores but the result is exactly the same. My point is... if it's going to be the same result it should also be using the same resources as before. I don't know why all my cores are at 100% now but I don't get any significant variation in the time. It would be better to stress less the CPU if at the end is going to be exactly the same. Ideally I would love to see operations as this one to be multi-threaded and taking full advantage of all the cores.

Sergey Sharybin commented

2014-01-03 10:39:45 +01:00

Changed status from 'Open' to: 'Archived'

Sergey Sharybin closed this issue

2014-01-03 10:39:45 +01:00

Sergey Sharybin commented

2014-01-03 10:39:45 +01:00

This is nothing to do with the dependency graph, as was mentioned you're benchmarking a single-threaded operations and they'll show nothing to you.

And you're using rather creepy comparison method -- comparing 2.69 which was built with msvc2008 and current master which was built with msvc2013. There might be something specific to OpenMP implementation or so which would lead to higher CPU load. You would need to use master branch compiled with msvc2008.

In my case i don't see any meaningful difference between official 2.69 release and current master compiled with msvc2008.

Release:

Current master:

Average CPU load was around 40% in both cases btw.

Since msvc2013 is still not officially supported does not consider this is a bug. Just a nice thing to keep eye on for the windows maintainers i'd say.

Thanks for the report, but unless you'll see the same issue with msvc2008 builds wouldn't consider this is a bug at this point.

This is nothing to do with the dependency graph, as was mentioned you're benchmarking a single-threaded operations and they'll show nothing to you. And you're using rather creepy comparison method -- comparing 2.69 which was built with msvc2008 and current master which was built with msvc2013. There might be something specific to OpenMP implementation or so which would lead to higher CPU load. You would need to use master branch compiled with msvc2008. In my case i don't see any meaningful difference between official 2.69 release and current master compiled with msvc2008. Release: ![CPU-2.69-release.png](https://archive.blender.org/developer/F60959/CPU-2.69-release.png) Current master: ![CPU-master.png](https://archive.blender.org/developer/F60961/CPU-master.png) Average CPU load was around 40% in both cases btw. Since msvc2013 is still not officially supported does not consider this is a bug. Just a nice thing to keep eye on for the windows maintainers i'd say. Thanks for the report, but unless you'll see the same issue with msvc2008 builds wouldn't consider this is a bug at this point.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Depsgraph problem #38041