Page MenuHome

Fix for multiprocessing on windows
AcceptedPublic

Authored by Campbell Barton (campbellbarton) on Fri, May 22, 7:16 AM.

Details

Summary

This patch is meant to fix multiprocessing on windows, avoiding the workaround described here.

Diff Detail

Repository
rB Blender
Branch
TEMP-MULTIPROCESSING-MODULE-FIX (branched from master)
Build Status
Buildable 8159
Build 8159: arc lint + arc unit

Event Timeline

Campbell Barton (campbellbarton) requested review of this revision.Fri, May 22, 7:16 AM
Campbell Barton (campbellbarton) created this revision.

Somewhat of a good news bad news situation , it's no longer uncontrollably spawning new blenders so that's a huge improvement.

With the following test snippet (from their manual)

from multiprocessing import Pool

def f(x):
    return x*x

if __name__ == '__main__':
    p = Pool(5)
    print(p.map(f, [1, 2, 3]))

running blender -b --python mptest.py

Blender 2.90 (sub 2) (hash efcc6d7dba6c built 2020-05-22 13:52:45)
Read prefs: C:\Users\ray\AppData\Roaming\Blender Foundation\Blender\2.90\config\userpref.blend
AL lib: (EE) UpdateDeviceParams: Failed to set 48000hz, got 44100hz instead
found bundled python: k:\BlenderGit\2019_ninja_release\bin\2.90\python
[1, 4, 9]

Blender quit

so that's great , and i have not tested it, but i assume it will work from an addon as well.

however running it from the text editor in blender , locks blender up in an endless loop of

Read prefs: C:\Users\ray\AppData\Roaming\Blender Foundation\Blender\2.90\config\userpref.blend
AL lib: (EE) UpdateDeviceParams: Failed to set 48000hz, got 44100hz instead
found bundled python: k:\BlenderGit\2019_ninja_release\bin\2.90\python
Read blend: C:\Users\ray\Downloads\mp_test.blend
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "k:\BlenderGit\2019_ninja_release\bin\2.90\python\lib\multiprocessing\spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "k:\BlenderGit\2019_ninja_release\bin\2.90\python\lib\multiprocessing\spawn.py", line 114, in _main
    prepare(preparation_data)
  File "k:\BlenderGit\2019_ninja_release\bin\2.90\python\lib\multiprocessing\spawn.py", line 225, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "k:\BlenderGit\2019_ninja_release\bin\2.90\python\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
    run_name="__mp_main__")
  File "k:\BlenderGit\2019_ninja_release\bin\2.90\python\lib\runpy.py", line 261, in run_path
    code, fname = _get_code_from_file(run_name, path_name)
  File "k:\BlenderGit\2019_ninja_release\bin\2.90\python\lib\runpy.py", line 231, in _get_code_from_file
    with open(fname, "rb") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\ray\\Downloads\\mp_test.blend\\mptest.py'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "k:\BlenderGit\2019_ninja_release\bin\2.90\python\lib\multiprocessing\spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "k:\BlenderGit\2019_ninja_release\bin\2.90\python\lib\multiprocessing\spawn.py", line 114, in _main
  File "k:\BlenderGit\2019_ninja_release\bin\2.90\python\lib\multiprocessing\spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "k:\BlenderGit\2019_ninja_release\bin\2.90\python\lib\multiprocessing\spawn.py", line 114, in _main
    prepare(preparation_data)
  File "k:\BlenderGit\2019_ninja_release\bin\2.90\python\lib\multiprocessing\spawn.py", line 225, in prepare
    prepare(preparation_data)
    _fixup_main_from_path(data['init_main_from_path'])
  File "k:\BlenderGit\2019_ninja_release\bin\2.90\python\lib\multiprocessing\spawn.py", line 225, in prepare
  File "k:\BlenderGit\2019_ninja_release\bin\2.90\python\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
    _fixup_main_from_path(data['init_main_from_path'])
    run_name="__mp_main__")
  File "k:\BlenderGit\2019_ninja_release\bin\2.90\python\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
  File "k:\BlenderGit\2019_ninja_release\bin\2.90\python\lib\runpy.py", line 261, in run_path
    run_name="__mp_main__")
  File "k:\BlenderGit\2019_ninja_release\bin\2.90\python\lib\runpy.py", line 261, in run_path
    code, fname = _get_code_from_file(run_name, path_name)
    code, fname = _get_code_from_file(run_name, path_name)
  File "k:\BlenderGit\2019_ninja_release\bin\2.90\python\lib\runpy.py", line 231, in _get_code_from_file
  File "k:\BlenderGit\2019_ninja_release\bin\2.90\python\lib\runpy.py", line 231, in _get_code_from_file
Traceback (most recent call last):
    with open(fname, "rb") as f:
    with open(fname, "rb") as f:

That seems to be a new usecase though (which by the looks of it should also plague the other platforms) which shouldn't hold up this diff.

This revision is now accepted and ready to land.Fri, May 22, 4:46 PM