2.8 Builds are limited to 16 threads of processing #60585

New Issue

Chris Clawson · 2019-01-17T04:30:48+01:00

Chris Clawson commented

2019-01-17 04:30:48 +01:00

Operating system: Windows 10 and Ubuntu 18.04 - 32 Core Threadripper 2990WX 64GB ECC memory
Graphics card: NVIDIA Quadro P2000
Last build tested: Jan 17 e57ee5934a , Jan 17 ad707115d5
Multi-thread editing features in both of the 64 bit Windows and Linux 64 builder.blender.org builds appear to be limited to 16 threads of computing. For example, Sculpt with smoothing brush runs the first 16 threads at 100% utilization, while leaving the other 75% of the CPU idle. Other features, such as mesh editing, seem to have the same restriction.
This issue was observed using htop in Ubuntu/Linux 18.04 and CPUID in Windows. Both provide realtime display of CPU thread utilization.
Update
Recreate the problem using Windows and the bmw27_cpu benchmark file.
Sculpting bmw27_cpu, with the smooth brush, works with any build, but only using the first 16 threads as described earlier. Other mesh editing exhibits similar limitations.

Operating system: Windows 10 and Ubuntu 18.04 - 32 Core Threadripper 2990WX 64GB ECC memory Graphics card: NVIDIA Quadro [P2000](https://archive.blender.org/developer/P2000.txt) Last build tested: Jan 17 e57ee5934a30 , Jan 17 ad707115d5bc Multi-thread editing features in both of the 64 bit Windows and Linux 64 builder.blender.org builds appear to be limited to 16 threads of computing. For example, Sculpt with smoothing brush runs the first 16 threads at 100% utilization, while leaving the other 75% of the CPU idle. Other features, such as mesh editing, seem to have the same restriction. This issue was observed using htop in Ubuntu/Linux 18.04 and CPUID in Windows. Both provide realtime display of CPU thread utilization. Update Recreate the problem using Windows and the bmw27_cpu benchmark file. Sculpting bmw27_cpu, with the smooth brush, works with any build, but only using the first 16 threads as described earlier. Other mesh editing exhibits similar limitations.

Chris Clawson commented

2019-01-17 04:30:48 +01:00

Added subscriber: @ChrisClawson

Brecht Van Lommel commented

2019-01-17 19:23:32 +01:00

Added subscriber: @brecht

Sergey Sharybin was assigned by Brecht Van Lommel

2019-01-17 19:23:32 +01:00

Brecht Van Lommel commented

2019-01-17 19:23:32 +01:00

I can confirm this.

blender-admin commented

2019-01-18 19:15:38 +01:00

This issue was referenced by blender/cycles@a0825869a1

This issue was referenced by blender/cycles@a0825869a1cfb6f685ebd7fb6a2eca792c879e4b

blender-admin commented

2019-01-18 19:15:38 +01:00

This issue was referenced by c5eb10b110

This issue was referenced by c5eb10b1104bb0f15695be5d4394bbb8303ad092

blender-admin commented

2019-01-18 19:15:38 +01:00

This issue was referenced by be8202d90b

This issue was referenced by be8202d90bd9bd8e87d5111c45fb412925b2d472

Brecht Van Lommel commented

2019-01-18 19:19:38 +01:00

Changed status from 'Open' to: 'Resolved'

Brecht Van Lommel closed this issue

2019-01-18 19:19:38 +01:00

Chris Clawson commented

2019-02-12 03:00:48 +01:00

This report has led to correcting an important bug in Cycles rendering, but my intention was to report that my Threadripper never attempts to utilize more than 16 threads for any multi-threaded tasks.

Cycles, as of build 036ec5cae4 (Feb. 12) correctly uses all 64 threads in the AMD 32 core chip – but features such as Eevee and Workbench rendering, sculpt, compositor, and fluid simulation all make heavy use of the first sequential 16 threads of the CPU, but ignore the rest of the chip. This is also the case for any of the older 2.79 builds.

Please observe this limitation by running the included file in a AMD 32 Core Threadripper system, while monitoring CPU utilization, and do the following:

Perform a Sculpt operation on the object.
Render an Animation, using Eevee.
Render an Animation, using Workbench.
Render animation with Cycles CPU, and note that this is the only function which uses all 64 threads.
bug28.blend

This report has led to correcting an important bug in Cycles rendering, but my intention was to report that my Threadripper never attempts to utilize more than 16 threads for any multi-threaded tasks. Cycles, as of build 036ec5cae4f7 (Feb. 12) correctly uses all 64 threads in the AMD 32 core chip – but features such as Eevee and Workbench rendering, sculpt, compositor, and fluid simulation all make heavy use of the first sequential 16 threads of the CPU, but ignore the rest of the chip. This is also the case for any of the older 2.79 builds. Please observe this limitation by running the included file in a AMD 32 Core Threadripper system, while monitoring CPU utilization, and do the following: 1. Perform a Sculpt operation on the object. 2. Render an Animation, using Eevee. 3. Render an Animation, using Workbench. 4. Render animation with Cycles CPU, and note that this is the only function which uses all 64 threads. [bug28.blend](https://archive.blender.org/developer/F6584613/bug28.blend)

Chris Clawson commented

2019-02-12 03:03:32 +01:00

Changed status from 'Resolved' to: 'Open'

Chris Clawson reopened this issue

2019-02-12 03:03:32 +01:00

Chris Clawson commented

2019-02-12 03:03:32 +01:00

Sorry - I should have flagged this as "Open" in my previous comment.

Brecht Van Lommel commented

2019-02-12 19:35:46 +01:00

@Sergey, I can confirm this on Ubuntu. Commenting out numaAPI_RunProcessOnNode in threads.c avoids the problem.

To me it seems as if this is somehow inherited by threads. If that's the case I'm not sure what the right solution is, seems rather tricky to figure out all the places that might start a thread, some of them outside our control.

@Sergey, I can confirm this on Ubuntu. Commenting out `numaAPI_RunProcessOnNode` in `threads.c` avoids the problem. To me it seems as if this is somehow inherited by threads. If that's the case I'm not sure what the right solution is, seems rather tricky to figure out all the places that might start a thread, some of them outside our control.

Sergey Sharybin commented

2019-02-12 23:10:49 +01:00

@brecht, for our thread pool it's fairly simple fix. But is a good point about threads summoned by someone else.
Guess half-decent solution would be to not set affinity for main thread, but set it in thread pools (both Blender and Cycles) to:

Make sure data is localized per node/CPU group
Allow use of more than 64 threads on Windows

Wouldn't work if Cycles is indirectly initializes a threads (maybe in OIIO/OSL). Not sure if that's happening, and if so i don't think we can have any decent solution.

@brecht, for our thread pool it's fairly simple fix. But is a good point about threads summoned by someone else. Guess half-decent solution would be to not set affinity for main thread, but set it in thread pools (both Blender and Cycles) to: - Make sure data is localized per node/CPU group - Allow use of more than 64 threads on Windows Wouldn't work if Cycles is indirectly initializes a threads (maybe in OIIO/OSL). Not sure if that's happening, and if so i don't think we can have any decent solution.

Chris Clawson commented

2019-02-13 22:27:25 +01:00

@Sergey, Please, let me repeat that Cycles currently works fine. All 64 threads are very active with a Cycles CPU render on the Threadripper. It is all the other functionality within Blender that has the 16 thread limitation.

@Sergey, Please, let me repeat that **Cycles currently works fine**. All 64 threads are very active with a Cycles CPU render on the Threadripper. It is all the ***other functionality*** within Blender that has the 16 thread limitation.

Sergey Sharybin commented

2019-02-14 09:28:45 +01:00

@ChrisClawson, I understand that, Those are all related issues, rooting to the fact that we've tried to give fastest core to the main thread. Cycles does special trickery to distribute its threads on all cores. Similar thing can be done for threads on Blender side. But we have no control over threads created by external libraries, like OpenJpeg and OpenEXR. Also, Cycles uses OIIO/OSL which might be creating threads as well, and those we also have no control over.

blender-admin commented

2019-03-05 12:43:40 +01:00

This issue was referenced by 76608f5ec5

This issue was referenced by 76608f5ec5e9737c7ef680a2234b8e3347b61c7b

Brecht Van Lommel commented

2019-03-05 12:48:58 +01:00

Changed status from 'Open' to: 'Resolved'

Brecht Van Lommel closed this issue

2019-03-05 12:48:58 +01:00

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

2.8 Builds are limited to 16 threads of processing #60585