Page MenuHome

Crash accessing python gpu module in background mode.
Closed, ResolvedPublic

Description

Just run e.g. path/to/blender -b --python-expr "import bpy; bpy.ops.wm.addon_enable(module='space_view3d_math_vis')" (or even enable that add-on in userpref and try to start Blender 2.8 in background mode, I guess).

There are several level to that issue (and several ways to fix it, too, but that’s not my area of code so will live it to the experts ;) ):

  • math_vis should absolutely not do what it does in first lines of its draw.py, calling gpu.shader.from_builtin() at module level, i.e. during init… tsst. This can be solved in many ways (not even sure it's worth storing in py those shaders? afaik this is cached internally anyway?), as long as it does not do any call outside of functions. Lazy init ftw!
  • Think gpu module code (gpu_py_shader.c and friends) should ensure it has a valid OpenGL context blabla too? Unless add-ons are expected to do those checks themselves before using it?

Event Timeline

Bastien Montagne (mont29) triaged this task as Confirmed, Medium priority.

@Campbell Barton (campbellbarton) assigned to you because math_vis is your add-on, and think you were involved in gpu api work? maybe @Clément Foucault (fclem) or other OpenGL / Gfx guys are interested too…

math_vis thingy is currently blocking i18n update process, will have to work around that unless it’s fixed soonish. ;)

I think it would be best if the addons chek out these errors themselves.
But for this, instead of crash it would be better to raise an error message:

1diff --git a/source/blender/gpu/GPU_init_exit.h b/source/blender/gpu/GPU_init_exit.h
2index e89c970b7d9..8f1a42c8795 100644
3--- a/source/blender/gpu/GPU_init_exit.h
4+++ b/source/blender/gpu/GPU_init_exit.h
5@@ -38,6 +38,7 @@ extern "C" {
6
7 void GPU_init(void);
8 void GPU_exit(void);
9+bool GPU_is_initialized(void);
10
11 #ifdef __cplusplus
12 }
13diff --git a/source/blender/gpu/intern/gpu_init_exit.c b/source/blender/gpu/intern/gpu_init_exit.c
14index 55d0466c929..d21acb188d9 100644
15--- a/source/blender/gpu/intern/gpu_init_exit.c
16+++ b/source/blender/gpu/intern/gpu_init_exit.c
17@@ -73,7 +73,6 @@ void GPU_init(void)
18 }
19
20
21-
22 void GPU_exit(void)
23 {
24 if (!G.background) {
25@@ -92,3 +91,9 @@ void GPU_exit(void)
26
27 initialized = false;
28 }
29+
30+
31+bool GPU_is_initialized(void)
32+{
33+ return initialized;
34+}
35diff --git a/source/blender/python/gpu/gpu_py_shader.c b/source/blender/python/gpu/gpu_py_shader.c
36index 5f1ea7a33ce..25ffd768a07 100644
37--- a/source/blender/python/gpu/gpu_py_shader.c
38+++ b/source/blender/python/gpu/gpu_py_shader.c
39@@ -29,6 +29,7 @@
40
41 #include "BLI_utildefines.h"
42
43+#include "GPU_init_exit.h"
44 #include "GPU_shader.h"
45 #include "GPU_shader_interface.h"
46
47@@ -721,6 +722,11 @@ static PyObject *bpygpu_shader_from_builtin(PyObject *UNUSED(self), PyObject *ar
48 return NULL;
49 }
50
51+ if (!GPU_is_initialized()) {
52+ PyErr_SetString(PyExc_SystemError, "This function can not be called without a GPU context");
53+ return NULL;
54+ }
55+
56 GPUShader *shader = GPU_shader_get_builtin_shader(shader_id);
57
58 return BPyGPUShader_CreatePyObject(shader, true);
59diff --git a/source/blender/python/gpu/gpu_py_types.c b/source/blender/python/gpu/gpu_py_types.c
60index d9ef0736f8e..b767ee1b56c 100644
61--- a/source/blender/python/gpu/gpu_py_types.c
62+++ b/source/blender/python/gpu/gpu_py_types.c
63@@ -27,6 +27,10 @@
64
65 #include <Python.h>
66
67+#include "BLI_utildefines.h"
68+
69+#include "GPU_init_exit.h"
70+
71 #include "../generic/py_capi_utils.h"
72 #include "../generic/python_utildefines.h"
73
74@@ -43,12 +47,31 @@ static struct PyModuleDef BPyGPU_types_module_def = {
75 .m_name = "gpu.types",
76 };
77
78+static PyObject *bpygpu_type_new_error(
79+ PyTypeObject *UNUSED(type),
80+ PyObject *UNUSED(args),
81+ PyObject *UNUSED(kwds))
82+{
83+ PyErr_SetString(PyExc_SystemError, "This object can not be created without a GPU context");
84+ return NULL;
85+}
86+
87 PyObject *BPyInit_gpu_types(void)
88 {
89 PyObject *submodule;
90
91 submodule = PyModule_Create(&BPyGPU_types_module_def);
92
93+ if (!GPU_is_initialized()) {
94+ /* Warning: Only restarting Blender for it to work again. */
95+ BPyGPUVertFormat_Type.tp_new = bpygpu_type_new_error;
96+ BPyGPUVertBuf_Type.tp_new = bpygpu_type_new_error;
97+ BPyGPUIndexBuf_Type.tp_new = bpygpu_type_new_error;
98+ BPyGPUBatch_Type.tp_new = bpygpu_type_new_error;
99+ BPyGPUOffScreen_Type.tp_new = bpygpu_type_new_error;
100+ BPyGPUShader_Type.tp_new = bpygpu_type_new_error;
101+ }
102+
103 if (PyType_Ready(&BPyGPUVertFormat_Type) < 0)
104 return NULL;
105 if (PyType_Ready(&BPyGPUVertBuf_Type) < 0)

animation nodes addon also triggers this crash
Also tested the patch and it works.

IMHO, first thing first, addons should not call those gpu functions at module level, that means they get called during first eval of the code (first import). This is bad practice in general, unless absolutely mandatory we avoid that kind of extra processing since it makes Blender loading slower.

In fact, not even sure why those global storage of gpu.shader.from_builtin()results are needed at all, afaict those are already cached internally, so call itself should be cheap after first time? But if they are needed, then a lazy init solution should be used instead (like using a dummy class with accessors, or so).

I can fix the Object Scatter and Math Vis addon in that regard (maybe also the UV Export addon).
However, it might take me a couple of days because I'm on vacation and have to work on my Bachelor thesis..
Feel free to assign the addon updates to me.

Will those addons never work in background mode, or is it just the wrong time to initialize? (I'm especially talking about the png export of the Export UV addon, which uses opengl to render the image)

IMHO, first thing first, addons should not call those gpu functions at module level (...)

Do you recommend adding an assert or raise error if the gpu module is imported at module level?

@Jacques Lucke (JacquesLucke) Think most of the time they won't work in background mode, no, which would solve most of the crashes already. For cases where OGL context is used in background mode, indeed that patch sounds OK to me, and it's add-on responsibility to handle the issue properly. And no rush here, it can wait for you to get back from holidays! ;)

@Germano Cavalcante (mano-wii) *importing* gpu module is fine, what is not fine is calling some of its functions at global level.