Python multiprocessing error - NotADirectoryError #93980

New Issue

Josh · 2021-12-12T02:55:23+01:00

Josh commented

2021-12-12 02:55:23 +01:00

System Information
Operating system: Mac OS Monterey 12.0.1
Graphics card: None

Blender Version
Broken: 3.0.0
Worked: I haven't tried other versions

Short description of error

When scripting with python and using multiprocessing, the Child process crashes with a strange error.

Exact steps for others to reproduce the error

Currently getting an error with multiprocessing, this is the code:

from multiprocessing import Process
import os

def info(title):

print(title)
print('module name:', name)
if hasattr(os, 'getppid'): # only available on Unix
print('parent process:', os.getppid())
print('process id:', os.getpid())


def f(name):

info('function f')
print('hello', name)


def main():

info('main line')
p = Process(target=f, args=('bob',))
p.start()
p.join()
print("done")



if __name__ == '__main__':

main()

This is the output when using Blenders Python:

main line
module name: __main__
parent process: 21172
process id: 27529
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Applications/Blender.app/Contents/Resources/3.0/python/lib/python3.9/multiprocessing/spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/Applications/Blender.app/Contents/Resources/3.0/python/lib/python3.9/multiprocessing/spawn.py", line 125, in _main
    prepare(preparation_data)
  File "/Applications/Blender.app/Contents/Resources/3.0/python/lib/python3.9/multiprocessing/spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "/Applications/Blender.app/Contents/Resources/3.0/python/lib/python3.9/multiprocessing/spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "/Applications/Blender.app/Contents/Resources/3.0/python/lib/python3.9/runpy.py", line 267, in run_path
    code, fname = _get_code_from_file(run_name, path_name)
  File "/Applications/Blender.app/Contents/Resources/3.0/python/lib/python3.9/runpy.py", line 237, in _get_code_from_file
    with io.open_code(decoded_path) as f:
NotADirectoryError: [Errno 20] Not a directory: '~/Documents/Blender/webcam.blend/test.py'
done

Expected output (This is what I get when I don't use Blenders Python [Python 3.9.0]):

main line
module name: __main__
parent process: 26548
process id: 27144
function f
module name: __mp_main__
parent process: 27144
process id: 27161
hello bob
done

It seems like the child that gets created from multiprocessing just crashes. Anyone have any ideas?

**System Information** Operating system: Mac OS Monterey 12.0.1 Graphics card: None **Blender Version** Broken: 3.0.0 Worked: I haven't tried other versions **Short description of error** When scripting with python and using multiprocessing, the Child process crashes with a strange error. **Exact steps for others to reproduce the error** Currently getting an error with multiprocessing, this is the code: ```lang=python from multiprocessing import Process import os def info(title): ``` print(title) print('module name:', __name__) if hasattr(os, 'getppid'): # only available on Unix print('parent process:', os.getppid()) print('process id:', os.getpid()) ``` def f(name): ``` info('function f') print('hello', name) ``` def main(): ``` info('main line') p = Process(target=f, args=('bob',)) p.start() p.join() print("done") ``` if __name__ == '__main__': ``` main() ``` ``` This is the output when using Blenders Python: ``` main line module name: __main__ parent process: 21172 process id: 27529 Traceback (most recent call last): File "<string>", line 1, in <module> File "/Applications/Blender.app/Contents/Resources/3.0/python/lib/python3.9/multiprocessing/spawn.py", line 116, in spawn_main exitcode = _main(fd, parent_sentinel) File "/Applications/Blender.app/Contents/Resources/3.0/python/lib/python3.9/multiprocessing/spawn.py", line 125, in _main prepare(preparation_data) File "/Applications/Blender.app/Contents/Resources/3.0/python/lib/python3.9/multiprocessing/spawn.py", line 236, in prepare _fixup_main_from_path(data['init_main_from_path']) File "/Applications/Blender.app/Contents/Resources/3.0/python/lib/python3.9/multiprocessing/spawn.py", line 287, in _fixup_main_from_path main_content = runpy.run_path(main_path, File "/Applications/Blender.app/Contents/Resources/3.0/python/lib/python3.9/runpy.py", line 267, in run_path code, fname = _get_code_from_file(run_name, path_name) File "/Applications/Blender.app/Contents/Resources/3.0/python/lib/python3.9/runpy.py", line 237, in _get_code_from_file with io.open_code(decoded_path) as f: NotADirectoryError: [Errno 20] Not a directory: '~/Documents/Blender/webcam.blend/test.py' done ``` Expected output (This is what I get when I don't use Blenders Python [Python 3.9.0]): ``` main line module name: __main__ parent process: 26548 process id: 27144 function f module name: __mp_main__ parent process: 27144 process id: 27161 hello bob done ``` It seems like the child that gets created from multiprocessing just crashes. Anyone have any ideas?

Josh commented

2021-12-12 02:55:23 +01:00

Added subscriber: @struny

Fen commented

2021-12-12 19:45:26 +01:00

Added subscriber: @chemicalcrux

Fen commented

2021-12-12 19:45:26 +01:00

The problem is that it's trying to run a script with the name of the Text object in blender. So, if the Text object is called "Text", it tries to run a script called "Text" (which does not exist).

In my case, I'm running a copy of Blender I built, and it appears to be trying to find the script in the D:/ directory.

FileNotFoundError: [Errno 2] No such file or directory: 'D:\\Text'

If I save the text file as "test.py", then rename the Text object to "test.py", the script works correctly...but that's pretty much just a coincidence.

The problem is that it's trying to run a script with the name of the Text object *in blender*. So, if the Text object is called "Text", it tries to run a script called "Text" (which does not exist). In my case, I'm running a copy of Blender I built, and it appears to be trying to find the script in the `D:/` directory. `FileNotFoundError: [Errno 2] No such file or directory: 'D:\\Text'` If I save the text file as "test.py", then rename the Text object to "test.py", the script works correctly...but that's pretty much just a coincidence.

Josh commented

2021-12-12 20:16:45 +01:00

@chemicalcrux
For me it seems like by default it is looking in my .blend file (NotADirectoryError: [Errno 20] Not a directory: '~/Documents/Blender/webcam.blend/test.py') to find the python script, and I am not sure why. I already have my text block named test.py.

I was able to find a work around:

If you modify the __file__ variable and point it directly to where the python script is located, it seems to work.

if __name__ == '__main__':

file = '../../~/Documents/Blender/testing/test.py'
...

@chemicalcrux For me it seems like by default it is looking in my `.blend` file (`NotADirectoryError: [Errno 20] Not a directory: '~/Documents/Blender/webcam.blend/test.py'`) to find the python script, and I am not sure why. I already have my text block named `test.py`. I was able to find a work around: If you modify the `__file__` variable and point it directly to where the python script is located, it seems to work. ```lang=python if __name__ == '__main__': ``` __file__ = '../../~/Documents/Blender/testing/test.py' ... ```

Philipp Oeser commented

2022-01-13 17:00:50 +01:00

Added subscriber: @lichtwerk

Philipp Oeser commented

2022-01-13 17:00:50 +01:00

Changed status from 'Needs Triage' to: 'Needs Developer To Reproduce'

Philipp Oeser commented

2022-01-13 17:00:50 +01:00

Not getting this error on Linux (tried 3.0 and master).

Looks like this has been discussed before [on Windows at that time] https://blender.stackexchange.com/questions/8530/how-to-get-python-multiprocessing-module-working-on-windows

So the __file__ trick is probably the recommended workaround, will doubt this is considered a bug, will let #python_api decide though.

Not getting this error on Linux (tried 3.0 and master). Looks like this has been discussed before [on Windows at that time] https://blender.stackexchange.com/questions/8530/how-to-get-python-multiprocessing-module-working-on-windows So the `__file__` trick is probably the recommended workaround, will doubt this is considered a bug, will let #python_api decide though.

Jindřich Čech commented

2022-02-15 12:54:17 +01:00

Added subscriber: @Jindrich-Cech

Jindřich Čech commented

2022-02-15 12:54:17 +01:00

This bug is persistent up to Blender 3.2.0 (Alpha). MacOS Blender Text editor provides bad script file path to modules (in my case /Users/jindra/Projects/Blender/gps.blend/gps.py instead of /Users/jindra/Documents/gps.py). If I run the same commands in Blender Python console - works OK. Everything is clear from added pictures...

This bug is persistent up to Blender 3.2.0 (Alpha). MacOS Blender Text editor provides bad script file path to modules (in my case /Users/jindra/Projects/Blender/gps.blend/gps.py instead of /Users/jindra/Documents/gps.py). If I run the same commands in Blender Python console - works OK. Everything is clear from added pictures...![DirectoryError-Python-TextEditor.png](https://archive.blender.org/developer/F12868233/DirectoryError-Python-TextEditor.png) ![Text-Editor_Run-Script_output.png](https://archive.blender.org/developer/F12868235/Text-Editor_Run-Script_output.png)

Philipp Oeser removed the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Python multiprocessing error - NotADirectoryError #93980