Import order matters: segmentation fault if PyTorch is imported before bpy
Closed, DuplicatePublic

Description

System Information
Ubuntu 14.04 and 16.04

Blender Version
2.78, 2.79, 2.79a, 2.79b

Short description of error
Segmentation fault occurs if bpy module is imported first. It does not matter which version of PyTorch I am using; this always occurs.

Exact steps for others to reproduce the error
If I do import torch first and them import bpy I get a segmentation fault. I'm not picky about importing the other module first and then do import torch but for some reason, I need to do import torch first in my work and this has made things a bit complicated for me as I have had to invent ways to get around the segmentation fault issue. My guess is this is that bpy is trying to access some pre-allocated memory locations and this is causing the issue. I wonder if Blender developers can dig more into this issue. Below you can find the backtrace obtained via dgb bt:

import torch
[New Thread 0x2aaaaf6aa700 (LWP 9233)]
[New Thread 0x2aaaaf8ab700 (LWP 9234)]
[New Thread 0x2aaab3aac700 (LWP 9235)]
[New Thread 0x2aaab5cad700 (LWP 9236)]
[New Thread 0x2aaab7eae700 (LWP 9237)]
[New Thread 0x2aaaba0af700 (LWP 9238)]
[New Thread 0x2aaabc2b0700 (LWP 9239)]
[New Thread 0x2aaabe4b1700 (LWP 9240)]
[New Thread 0x2aaac06b2700 (LWP 9241)]
[New Thread 0x2aaac28b3700 (LWP 9242)]
[New Thread 0x2aaac4ab4700 (LWP 9243)]
[New Thread 0x2aaac6cb5700 (LWP 9244)]
[New Thread 0x2aaac6eb6700 (LWP 9245)]
[New Thread 0x2aaacb0b7700 (LWP 9246)]
[New Thread 0x2aaacd2b8700 (LWP 9247)]
[New Thread 0x2aaacf4b9700 (LWP 9248)]
[New Thread 0x2aaad16ba700 (LWP 9249)]
[New Thread 0x2aaad38bb700 (LWP 9250)]
[New Thread 0x2aaad5abc700 (LWP 9251)]
[New Thread 0x2aaad7cbd700 (LWP 9252)]
[New Thread 0x2aaad9ebe700 (LWP 9253)]
[New Thread 0x2aaadc0bf700 (LWP 9254)]
[New Thread 0x2aaade2c0700 (LWP 9255)]
[New Thread 0x2aaae04c1700 (LWP 9257)]
[New Thread 0x2aaae26c2700 (LWP 9258)]
[New Thread 0x2aaae48c3700 (LWP 9259)]
[New Thread 0x2aaae6ac4700 (LWP 9260)]
[New Thread 0x2aaae6cc5700 (LWP 9261)]
[New Thread 0x2aaaeaec6700 (LWP 9262)]
[New Thread 0x2aaaed0c7700 (LWP 9263)]
[New Thread 0x2aaaef2c8700 (LWP 9264)]
[New Thread 0x2aaaf14c9700 (LWP 9265)]
[New Thread 0x2aaaf36ca700 (LWP 9266)]
[New Thread 0x2aaaf38cb700 (LWP 9267)]
[New Thread 0x2aaaf7acc700 (LWP 9268)]
[New Thread 0x2aaaf9ccd700 (LWP 9269)]
[New Thread 0x2aaafbece700 (LWP 9270)]
[New Thread 0x2aaafe0cf700 (LWP 9272)]
[New Thread 0x2aab002d0700 (LWP 9273)]
[New Thread 0x2aab024d1700 (LWP 9274)]
[New Thread 0x2aab046d2700 (LWP 9275)]
[New Thread 0x2aab068d3700 (LWP 9277)]
[New Thread 0x2aab08ad4700 (LWP 9278)]
[New Thread 0x2aab0acd5700 (LWP 9281)]
[New Thread 0x2aab0ced6700 (LWP 9282)]
[New Thread 0x2aab0f0d7700 (LWP 9283)]
[New Thread 0x2aab112d8700 (LWP 9284)]
[Thread 0x2aaaf38cb700 (LWP 9267) exited]
[Thread 0x2aab0f0d7700 (LWP 9283) exited]
[Thread 0x2aab0ced6700 (LWP 9282) exited]
[Thread 0x2aab112d8700 (LWP 9284) exited]
[Thread 0x2aab0acd5700 (LWP 9281) exited]
[Thread 0x2aab08ad4700 (LWP 9278) exited]
[Thread 0x2aab068d3700 (LWP 9277) exited]
[Thread 0x2aab046d2700 (LWP 9275) exited]
[Thread 0x2aab024d1700 (LWP 9274) exited]
[Thread 0x2aab002d0700 (LWP 9273) exited]
[Thread 0x2aaafe0cf700 (LWP 9272) exited]
[Thread 0x2aaafbece700 (LWP 9270) exited]
[Thread 0x2aaaf9ccd700 (LWP 9269) exited]
[Thread 0x2aaaf7acc700 (LWP 9268) exited]
[Thread 0x2aaaf36ca700 (LWP 9266) exited]
[Thread 0x2aaaf14c9700 (LWP 9265) exited]
[Thread 0x2aaaef2c8700 (LWP 9264) exited]
[Thread 0x2aaaed0c7700 (LWP 9263) exited]
[Thread 0x2aaaeaec6700 (LWP 9262) exited]
[Thread 0x2aaae6cc5700 (LWP 9261) exited]
[Thread 0x2aaae6ac4700 (LWP 9260) exited]
[Thread 0x2aaae48c3700 (LWP 9259) exited]
[Thread 0x2aaae26c2700 (LWP 9258) exited]
[Thread 0x2aaae04c1700 (LWP 9257) exited]
[Thread 0x2aaade2c0700 (LWP 9255) exited]
[Thread 0x2aaadc0bf700 (LWP 9254) exited]
[Thread 0x2aaad9ebe700 (LWP 9253) exited]
[Thread 0x2aaad7cbd700 (LWP 9252) exited]
[Thread 0x2aaad5abc700 (LWP 9251) exited]
[Thread 0x2aaad38bb700 (LWP 9250) exited]
[Thread 0x2aaad16ba700 (LWP 9249) exited]
[Thread 0x2aaacf4b9700 (LWP 9248) exited]
[Thread 0x2aaacd2b8700 (LWP 9247) exited]
[Thread 0x2aaacb0b7700 (LWP 9246) exited]
[Thread 0x2aaac6eb6700 (LWP 9245) exited]
[Thread 0x2aaac6cb5700 (LWP 9244) exited]
[Thread 0x2aaac4ab4700 (LWP 9243) exited]
[Thread 0x2aaac28b3700 (LWP 9242) exited]
[Thread 0x2aaac06b2700 (LWP 9241) exited]
[Thread 0x2aaabe4b1700 (LWP 9240) exited]
[Thread 0x2aaabc2b0700 (LWP 9239) exited]
[Thread 0x2aaaba0af700 (LWP 9238) exited]
[Thread 0x2aaab7eae700 (LWP 9237) exited]
[Thread 0x2aaab5cad700 (LWP 9236) exited]
[Thread 0x2aaab3aac700 (LWP 9235) exited]
[Thread 0x2aaaaf8ab700 (LWP 9234) exited]
[Thread 0x2aaaaf6aa700 (LWP 9233) exited]
>>> import bpy
[New Thread 0x2aab112d8700 (LWP 9300)]
[New Thread 0x2aab0f0d7700 (LWP 9301)]
[New Thread 0x2aab0ced6700 (LWP 9302)]
[New Thread 0x2aab0acd5700 (LWP 9303)]
[New Thread 0x2aab08ad4700 (LWP 9304)]
[New Thread 0x2aab068d3700 (LWP 9305)]
[New Thread 0x2aab046d2700 (LWP 9306)]
[New Thread 0x2aab024d1700 (LWP 9307)]
[New Thread 0x2aab002d0700 (LWP 9308)]
[New Thread 0x2aaafe0cf700 (LWP 9309)]
[New Thread 0x2aaafbece700 (LWP 9310)]
[New Thread 0x2aaaf9ccd700 (LWP 9311)]
[New Thread 0x2aaaf7acc700 (LWP 9312)]
[New Thread 0x2aaaf38cb700 (LWP 9313)]
[New Thread 0x2aaaf36ca700 (LWP 9314)]
[New Thread 0x2aaaf14c9700 (LWP 9315)]
[New Thread 0x2aaaef2c8700 (LWP 9316)]
[New Thread 0x2aaaed0c7700 (LWP 9317)]
[New Thread 0x2aaaeaec6700 (LWP 9318)]
[New Thread 0x2aaab7eae700 (LWP 9319)]
[New Thread 0x2aaaba0af700 (LWP 9320)]
[New Thread 0x2aaabc2b0700 (LWP 9321)]
[New Thread 0x2aaabe4b1700 (LWP 9322)]
[New Thread 0x2aaac06b2700 (LWP 9323)]
[New Thread 0x2aaac28b3700 (LWP 9324)]
[New Thread 0x2aaac4ab4700 (LWP 9325)]
[New Thread 0x2aaacb0b7700 (LWP 9326)]
[New Thread 0x2aaacd2b8700 (LWP 9327)]
[New Thread 0x2aaacf4b9700 (LWP 9328)]
[New Thread 0x2aaad16ba700 (LWP 9329)]
[New Thread 0x2aaad38bb700 (LWP 9330)]
[New Thread 0x2aaad5abc700 (LWP 9331)]
[New Thread 0x2aaad7cbd700 (LWP 9332)]
[New Thread 0x2aaad9ebe700 (LWP 9333)]
[New Thread 0x2aaadc0bf700 (LWP 9334)]
[New Thread 0x2aaade2c0700 (LWP 9335)]
[New Thread 0x2aaae04c1700 (LWP 9336)]
[New Thread 0x2aaae26c2700 (LWP 9337)]
[New Thread 0x2aaae48c3700 (LWP 9338)]
[New Thread 0x2aabbf200700 (LWP 9339)]
[New Thread 0x2aabbf401700 (LWP 9340)]
[New Thread 0x2aabbf602700 (LWP 9341)]
[New Thread 0x2aabbf803700 (LWP 9342)]
[New Thread 0x2aabbfa04700 (LWP 9343)]
[New Thread 0x2aabbfc05700 (LWP 9344)]
[New Thread 0x2aabbfe06700 (LWP 9345)]
[New Thread 0x2aabc0007700 (LWP 9346)]
[New Thread 0x2aabc0208700 (LWP 9347)]

Thread 1 "python3" received signal SIGSEGV, Segmentation fault.
0x00002aaba30a100e in cuewInit () from /usr/lib/python3/dist-packages/bpy.so
(gdb) bt
#0  0x00002aaba30a100e in cuewInit () from /usr/lib/python3/dist-packages/bpy.so
#1  0x00002aaba22f8cd4 in ccl::device_cuda_init() () from /usr/lib/python3/dist-packages/bpy.so
#2  0x00002aaba22eb34c in ccl::Device::available_devices() () from /usr/lib/python3/dist-packages/bpy.so
#3  0x00002aaba21e0bc9 in ccl::get_device_types_func(_object*, _object*) () from /usr/lib/python3/dist-packages/bpy.so
#4  0x00000000004e9b7f in PyCFunction_Call ()
#5  0x00000000005372f4 in PyEval_EvalFrameEx ()
#6  0x0000000000540f9b in PyEval_EvalCodeEx ()
#7  0x00000000004ebd23 in ?? ()
#8  0x00000000005c1797 in PyObject_Call ()
#9  0x0000000000534d90 in PyEval_CallObjectWithKeywords ()
#10 0x00002aaba19865cf in bpy_prop_enum_itemf_cb () from /usr/lib/python3/dist-packages/bpy.so
#11 0x00002aaba1daf54a in RNA_property_enum_items_ex () from /usr/lib/python3/dist-packages/bpy.so
#12 0x00002aaba1daf5f3 in RNA_property_enum_items () from /usr/lib/python3/dist-packages/bpy.so
#13 0x00002aaba1daf992 in RNA_property_enum_value () from /usr/lib/python3/dist-packages/bpy.so
#14 0x00002aaba1976b1f in pyrna_py_to_prop () from /usr/lib/python3/dist-packages/bpy.so
#15 0x00002aaba1977739 in pyrna_struct_setattro () from /usr/lib/python3/dist-packages/bpy.so
#16 0x0000000000593e39 in PyObject_SetAttr ()
#17 0x0000000000537b71 in PyEval_EvalFrameEx ()
#18 0x0000000000540f9b in PyEval_EvalCodeEx ()
#19 0x00000000004ebd23 in ?? ()
#20 0x00000000005c1797 in PyObject_Call ()
#21 0x00002aaba1981e8c in bpy_app_generic_callback () from /usr/lib/python3/dist-packages/bpy.so
#22 0x00002aaba1f0130d in BLI_callback_exec () from /usr/lib/python3/dist-packages/bpy.so
#23 0x00002aaba157c77e in WM_init () from /usr/lib/python3/dist-packages/bpy.so
#24 0x00002aaba156ef15 in main_python_enter () from /usr/lib/python3/dist-packages/bpy.so
#25 0x00002aaba197028d in bpy_module_delay_init () from /usr/lib/python3/dist-packages/bpy.so
#26 0x00002aaba19702ed in dealloc_obj_dealloc () from /usr/lib/python3/dist-packages/bpy.so
#27 0x00000000005a0189 in PyDict_SetItem ()
#28 0x00000000005a114c in PyDict_SetItemString ()
#29 0x000000000051ffb6 in PyModule_AddObject ()
#30 0x00000000006100db in _PyImport_LoadDynamicModuleWithSpec ()
#31 0x0000000000610538 in ?? ()
#32 0x00000000004e9c36 in PyCFunction_Call ()
#33 0x000000000053dbbb in PyEval_EvalFrameEx ()
#34 0x0000000000540199 in ?? ()
#35 0x000000000053c1d0 in PyEval_EvalFrameEx ()
#36 0x000000000053b7e4 in PyEval_EvalFrameEx ()
#37 0x000000000053b7e4 in PyEval_EvalFrameEx ()
#38 0x000000000053b7e4 in PyEval_EvalFrameEx ()
#39 0x000000000053b7e4 in PyEval_EvalFrameEx ()
#40 0x0000000000540f9b in PyEval_EvalCodeEx ()
#41 0x00000000004ebd23 in ?? ()
#42 0x00000000005c1797 in PyObject_Call ()
#43 0x00000000005c257a in _PyObject_CallMethodIdObjArgs ()
#44 0x00000000005260c8 in PyImport_ImportModuleLevelObject ()
#45 0x0000000000549e78 in ?? ()
#46 0x00000000004e9ba7 in PyCFunction_Call ()
#47 0x00000000005c1797 in PyObject_Call ()
#48 0x0000000000534d90 in PyEval_CallObjectWithKeywords ()
#49 0x000000000053a1c7 in PyEval_EvalFrameEx ()
#50 0x0000000000540199 in ?? ()
#51 0x0000000000540e4f in PyEval_EvalCode ()
#52 0x000000000060c272 in ?? ()
#53 0x000000000046b89f in PyRun_InteractiveOneObject ()
#54 0x000000000046ba48 in PyRun_InteractiveLoopFlags ()
#55 0x000000000046cfa0 in ?? ()
#56 0x00000000004cf2bd in ?? ()
#57 0x00000000004cfeb1 in main ()

However, if I do import bpy first and then import torch backtrace would be empty and things work fine.

Also, this is how I compile Blender and PyTorch after installing their dependencies:

git clone https://git.blender.org/blender.git
cd blender
git checkout 8ef39d5c882896bd75e0d4d17fb3e3d4710fc768 # Blender 2.79
git submodule update --init --recursive
git submodule foreach git checkout master
git submodule foreach git pull --rebase origin master

blender/build_files/build_environment/install_deps.sh --source=/blender-git/

PPATH="$(which python3)"
cmake blender \
    -DCMAKE_INSTALL_PREFIX=/usr/lib/python3/dist-packages \
    -DWITH_INSTALL_PORTABLE=OFF \
    -DWITH_PYTHON_INSTALL=OFF \
    -DWITH_PYTHON_MODULE=ON \
    -DPYTHON_SITE_PACKAGES=/usr/lib/python3/dist-packages \
    -DPYTHON_VERSION=3.5 \
    -DWITH_OPENAL=ON \
    -DWITH_CODEC_AVI=ON \
    -DWITH_MOD_OCEANSIM=ON \
    -DWITH_CODEC_FFMPEG=ON \
    -DWITH_SYSTEM_GLEW=ON \
    -DWITH_FFTW3=ON \
    -DWITH_OPENCOLORIO=ON \
    -DWITH_GAMEENGINE=OFF \
    -DWITH_PLAYER=OFF
make -j 7
make install

git clone --recursive https://github.com/pytorch/pytorch
cd pytorch
python setup.py install

Details

Type
Bug
Amir (Warrior) updated the task description. (Show Details)
This comment was removed by Amir (Warrior).

This issue on PyTorch GitHub page might be helpful for the developers: https://github.com/pytorch/pytorch/issues/6194

Based on one of the comments from one of PyTorch developers, it seems that bpy has some issues when initializing cuda if PyTorch is imported first.

This issue still persists ...