Toolbar icons do not honour BLENDER_SYSTEM_DATAFILES #63775

New Issue

Shane Ambler · 2019-04-21T07:50:11+02:00

Shane Ambler commented

2019-04-21 07:50:11 +02:00

System Information
Operating system: FreeBSD-12.0-STABLE-amd64-64bit-ELF 64 Bits
Graphics card: GeForce GT 730/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 390.87

Blender Version
Broken: version: 2.80 (sub 58), branch: master, commit date: 2019-04-20 17:01, hash: d11d5403f0

Short description of error
The environment variable BLENDER_SYSTEM_DATAFILES can be set to allow placing datafiles in different locations. The toolbar icons fail to follow this path and can only be found using the 2.80/datafiles path construct.

Exact steps for others to reproduce the error
Move the 2.80/datafiles folder to a different location than blender and set BLENDER_SYSTEM_DATAFILES to match. Only the toolbar icons fail to be found.

Also setting BLENDER_USER_DATAFILES does not help. Could/should this also be included in the icon search path?

**System Information** Operating system: FreeBSD-12.0-STABLE-amd64-64bit-ELF 64 Bits Graphics card: GeForce GT 730/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 390.87 **Blender Version** Broken: version: 2.80 (sub 58), branch: master, commit date: 2019-04-20 17:01, hash: `d11d5403f0` **Short description of error** The environment variable BLENDER_SYSTEM_DATAFILES can be set to allow placing datafiles in different locations. The toolbar icons fail to follow this path and can only be found using the `2.80/datafiles` path construct. **Exact steps for others to reproduce the error** Move the `2.80/datafiles` folder to a different location than blender and set BLENDER_SYSTEM_DATAFILES to match. Only the toolbar icons fail to be found. Also setting BLENDER_USER_DATAFILES does not help. Could/should this also be included in the icon search path?

Shane Ambler commented

2019-04-21 07:50:11 +02:00

Added subscriber: @sambler

Sebastian Parborg commented

2019-04-25 12:57:36 +02:00

Added subscribers: @brecht, @ZedDB

Brecht Van Lommel was assigned by Sebastian Parborg

2019-04-25 12:57:36 +02:00

Sebastian Parborg commented

2019-04-25 12:57:36 +02:00

@brecht who to assign this to?

Brecht Van Lommel removed their assignment 2019-04-25 12:59:29 +02:00

Philipp Oeser was assigned by Brecht Van Lommel

2019-04-25 12:59:29 +02:00

Brecht Van Lommel commented

2019-04-25 12:59:29 +02:00

Added subscriber: @lichtwerk

Brecht Van Lommel commented

2019-04-25 12:59:29 +02:00

I think @lichtwerk did a fix in this area recently?

Philipp Oeser commented

2019-04-29 18:55:38 +02:00

first note(s):

space_toolsystem_common.py looks for icon folder in datafiles via bpy.utils.resource_path('LOCAL'), then bpy.utils.resource_path('SYSTEM')
under the hood resource_path uses BKE_appdir_folder_id_version (not good -- doesnt check environment variables at all)
there is also bpy.utils.user_resource('DATAFILES')
under the hood user_resource uses BKE_appdir_folder_id_user_notest (this checks environment variables but could fail -- because it doesnt check for subfolder existance -- see fe7c7d2820)
also above would only work with BLENDER_USER_DATAFILES, not BLENDER_SYSTEM_DATAFILES

to reliably go over all possible directories [including checking if environment variables are set] it would be good to have the equivalent of what init_iconfile_list() does:
icondir = BKE_appdir_folder_id(BLENDER_DATAFILES, "icons")
[In a way, this was done prior to fe7c7d2820]

Needs a bit more thinking to get it right from bpy, stay tuned...

first note(s): - `space_toolsystem_common.py` looks for icon folder in datafiles via `bpy.utils.resource_path('LOCAL')`, then `bpy.utils.resource_path('SYSTEM')` - under the hood `resource_path` uses `BKE_appdir_folder_id_version` (not good -- doesnt check environment variables at all) - there is also bpy.utils.user_resource('DATAFILES') - under the hood `user_resource` uses `BKE_appdir_folder_id_user_notest` (this checks environment variables but could fail -- because it doesnt check for subfolder existance -- see fe7c7d2820) - also above would only work with BLENDER_USER_DATAFILES, not BLENDER_SYSTEM_DATAFILES to reliably go over all possible directories [including checking if environment variables are set] it would be good to have the equivalent of what `init_iconfile_list()` does: `icondir = BKE_appdir_folder_id(BLENDER_DATAFILES, "icons")` [In a way, this was done prior to fe7c7d2820] Needs a bit more thinking to get it right from bpy, stay tuned...

Abid Maqbool commented

2019-05-31 05:49:37 +02:00

Added subscriber: @AbidMaqbool

Shane Ambler commented

2019-07-19 10:00:03 +02:00

Any ideas that could fix this for the 2.80 release?

Philipp Oeser commented

2019-07-19 11:26:31 +02:00

Looking at this again today

Philipp Oeser commented

2019-07-19 12:55:02 +02:00

Added subscriber: @ideasman42

Philipp Oeser commented

2019-07-19 12:55:02 +02:00

To recap: from the bpy side we have:

bpy.utils.resource_path('LOCAL'/'SYSTEM'/'USER')
- this doesnt take environment variables into account, because this returns the toplevel resource folder, environment variables tweak the "sublevel" [config, datafiles, scripts]
bpy.utils.user_resource('CONFIG'/'DATAFILES'/'SCRIPTS')
- checks _USER_ environment variables (BLENDER_USER_DATAFILES, BLENDER_USER_CONFIG, ...)
- this works, but we cant use it in space_toolsystem_common.py because we need a general path, not just the user path

So as it seems we have no way to query a resource path from python that takes environment variables into account other than the _USER_ ones.
We would need such functionality in bpy for space_toolsystem_common.py to be able to reliably check where to get icons from.

How about adding:

bpy.utils.resource('CONFIG'/'DATAFILES'/'SCRIPTS')?
- this would then use BKE_appdir_folder_id (which covers the environment variables)
  - in detail: we would use the BLENDER_DATAFILES folder_id in BKE_appdir_folder_id [this goes through the whole hierarchy of user first (envvar precedence), then local, then system (envvar precedence)]
  - for 'CONFIG' and 'SCRIPTS': we'd need to add BLENDER_CONFIG and BLENDER_SCRIPTS [to go through the whole hierarchy of user first (envvar precedence), then local, then system (envvar precedence)]
alternatively: bpy.utils.resource_datafiles() - takes no arguments
- same as above, but does it only for DATAFILES (which is the most common usecase I guess)
- can just use BKE_appdir_folder_id with BLENDER_DATAFILES folder_id (so we are fully covered already)

Sorry for the long blathering (I hope I am not overcomlicating things), but before implementing this, would like to get feedback from @brecht and @ideasman42 here...

To recap: from the bpy side we have: - `bpy.utils.resource_path`('LOCAL'/'SYSTEM'/'USER') - this doesnt take environment variables into account, because this returns the toplevel resource folder, environment variables tweak the "sublevel" [config, datafiles, scripts] - `bpy.utils.user_resource`('CONFIG'/'DATAFILES'/'SCRIPTS') - checks `_USER_` environment variables (BLENDER_USER_DATAFILES, BLENDER_USER_CONFIG, ...) - this works, but we cant use it in `space_toolsystem_common.py` because we need a general path, not just the user path So as it seems we have no way to query a resource path from python that takes environment variables into account other than the `_USER_` ones. We would need such functionality in bpy for `space_toolsystem_common.py` to be able to reliably check where to get icons from. How about adding: - `bpy.utils.resource`('CONFIG'/'DATAFILES'/'SCRIPTS')? - this would then use `BKE_appdir_folder_id` (which covers the environment variables) - in detail: we would use the `BLENDER_DATAFILES` `folder_id` in `BKE_appdir_folder_id` [this goes through the whole hierarchy of user first (envvar precedence), then local, then system (envvar precedence)] - for 'CONFIG' and 'SCRIPTS': we'd need to add `BLENDER_CONFIG` and `BLENDER_SCRIPTS` [to go through the whole hierarchy of user first (envvar precedence), then local, then system (envvar precedence)] - alternatively: `bpy.utils.resource_datafiles()` - takes no arguments - same as above, but does it only for DATAFILES (which is the most common usecase I guess) - can just use `BKE_appdir_folder_id` with `BLENDER_DATAFILES` `folder_id` (so we are fully covered already) Sorry for the long blathering (I hope I am not overcomlicating things), but before implementing this, would like to get feedback from @brecht and @ideasman42 here...

Philipp Oeser commented

2019-07-19 15:38:24 +02:00

Note though that I'll be away next week, so from my side, this wont be ready for 2.80 (unless someone else jumps in)...

Philipp Oeser commented

2019-07-19 17:01:23 +02:00

See the following for implementing (and using) bpy.utils.resource_datafiles()

P1045: T63775_resource_datafiles



diff --git a/release/scripts/modules/bpy/utils/__init__.py b/release/scripts/modules/bpy/utils/__init__.py
index e6424de0742..8d913f54a70 100644
--- a/release/scripts/modules/bpy/utils/__init__.py
+++ b/release/scripts/modules/bpy/utils/__init__.py
@@ -44,6 +44,7 @@ __all__ = (
     "manual_map",
     "previews",
     "resource_path",
+    "resource_datafiles",
     "script_path_user",
     "script_path_pref",
     "script_paths",
@@ -62,6 +63,7 @@ from _bpy import (
     escape_identifier,
     register_class,
     resource_path,
+    resource_datafiles,
     script_paths as _bpy_script_paths,
     unregister_class,
     user_resource as _user_resource,
diff --git a/release/scripts/startup/bl_ui/space_toolsystem_common.py b/release/scripts/startup/bl_ui/space_toolsystem_common.py
index e7e95c26b55..565ee121744 100644
--- a/release/scripts/startup/bl_ui/space_toolsystem_common.py
+++ b/release/scripts/startup/bl_ui/space_toolsystem_common.py
@@ -195,11 +195,11 @@ class ToolSelectPanelHelper:
             assert(type(icon_name) is str)
             icon_value = _icon_cache.get(icon_name)
             if icon_value is None:
-                dirname = bpy.utils.resource_path('LOCAL')
+                dirname = bpy.utils.resource_datafiles()
                 if not os.path.exists(dirname):
-                    # TODO(campbell): use a better way of finding datafiles.
-                    dirname = bpy.utils.resource_path('SYSTEM')
-                filename = os.path.join(dirname, "datafiles", "icons", icon_name + ".dat")
+                    # TODO(philipp): what to do then?
+                    pass
+                filename = os.path.join(dirname, "icons", icon_name + ".dat")
                 try:
                     icon_value = bpy.app.icons.new_triangles_from_file(filename)
                 except Exception as ex:
diff --git a/source/blender/python/intern/bpy.c b/source/blender/python/intern/bpy.c
index 92ba7704b19..b436336844d 100644
--- a/source/blender/python/intern/bpy.c
+++ b/source/blender/python/intern/bpy.c
@@ -231,6 +231,22 @@ static PyObject *bpy_resource_path(PyObject *UNUSED(self), PyObject *args, PyObj
   return PyC_UnicodeFromByte(path ? path : "");
 }
 
+PyDoc_STRVAR(bpy_resource_datafiles_doc,
+             ".. function:: script_paths()\n"
+             "\n"
+             "   Return the resource datafiles path.\n"
+             "\n"
+             "   :return: the resource datafiles path.\n"
+             "   :rtype: string\n");
+static PyObject *bpy_resource_datafiles(PyObject *UNUSED(self))
+{
+  const char *path;
+
+  path = BKE_appdir_folder_id(BLENDER_DATAFILES, NULL);
+
+  return PyC_UnicodeFromByte(path ? path : "");
+}
+
 PyDoc_STRVAR(bpy_escape_identifier_doc,
              ".. function:: escape_identifier(string)\n"
              "\n"
@@ -298,6 +314,12 @@ static PyMethodDef meth_bpy_resource_path = {
     METH_VARARGS | METH_KEYWORDS,
     bpy_resource_path_doc,
 };
+static PyMethodDef meth_bpy_resource_datafiles = {
+    "resource_datafiles",
+    (PyCFunction)bpy_resource_datafiles,
+    METH_NOARGS,
+    bpy_resource_datafiles_doc,
+};
 static PyMethodDef meth_bpy_escape_identifier = {
     "escape_identifier",
     (PyCFunction)bpy_escape_identifier,
@@ -400,6 +422,9 @@ void BPy_init_modules(void)
   PyModule_AddObject(mod,
                      meth_bpy_resource_path.ml_name,
                      (PyObject *)PyCFunction_New(&meth_bpy_resource_path, NULL));
+  PyModule_AddObject(mod,
+                     meth_bpy_resource_datafiles.ml_name,
+                     (PyObject *)PyCFunction_New(&meth_bpy_resource_datafiles, NULL));
   PyModule_AddObject(mod,
                      meth_bpy_escape_identifier.ml_name,
                      (PyObject *)PyCFunction_New(&meth_bpy_escape_identifier, NULL));

See the following for implementing (and using) `bpy.utils.resource_datafiles()` [P1045: T63775_resource_datafiles](https://archive.blender.org/developer/P1045.txt) ``` diff --git a/release/scripts/modules/bpy/utils/__init__.py b/release/scripts/modules/bpy/utils/__init__.py index e6424de0742..8d913f54a70 100644 --- a/release/scripts/modules/bpy/utils/__init__.py +++ b/release/scripts/modules/bpy/utils/__init__.py @@ -44,6 +44,7 @@ __all__ = ( "manual_map", "previews", "resource_path", + "resource_datafiles", "script_path_user", "script_path_pref", "script_paths", @@ -62,6 +63,7 @@ from _bpy import ( escape_identifier, register_class, resource_path, + resource_datafiles, script_paths as _bpy_script_paths, unregister_class, user_resource as _user_resource, diff --git a/release/scripts/startup/bl_ui/space_toolsystem_common.py b/release/scripts/startup/bl_ui/space_toolsystem_common.py index e7e95c26b55..565ee121744 100644 --- a/release/scripts/startup/bl_ui/space_toolsystem_common.py +++ b/release/scripts/startup/bl_ui/space_toolsystem_common.py @@ -195,11 +195,11 @@ class ToolSelectPanelHelper: assert(type(icon_name) is str) icon_value = _icon_cache.get(icon_name) if icon_value is None: - dirname = bpy.utils.resource_path('LOCAL') + dirname = bpy.utils.resource_datafiles() if not os.path.exists(dirname): - # TODO(campbell): use a better way of finding datafiles. - dirname = bpy.utils.resource_path('SYSTEM') - filename = os.path.join(dirname, "datafiles", "icons", icon_name + ".dat") + # TODO(philipp): what to do then? + pass + filename = os.path.join(dirname, "icons", icon_name + ".dat") try: icon_value = bpy.app.icons.new_triangles_from_file(filename) except Exception as ex: diff --git a/source/blender/python/intern/bpy.c b/source/blender/python/intern/bpy.c index 92ba7704b19..b436336844d 100644 --- a/source/blender/python/intern/bpy.c +++ b/source/blender/python/intern/bpy.c @@ -231,6 +231,22 @@ static PyObject *bpy_resource_path(PyObject *UNUSED(self), PyObject *args, PyObj return PyC_UnicodeFromByte(path ? path : ""); } +PyDoc_STRVAR(bpy_resource_datafiles_doc, + ".. function:: script_paths()\n" + "\n" + " Return the resource datafiles path.\n" + "\n" + " :return: the resource datafiles path.\n" + " :rtype: string\n"); +static PyObject *bpy_resource_datafiles(PyObject *UNUSED(self)) +{ + const char *path; + + path = BKE_appdir_folder_id(BLENDER_DATAFILES, NULL); + + return PyC_UnicodeFromByte(path ? path : ""); +} + PyDoc_STRVAR(bpy_escape_identifier_doc, ".. function:: escape_identifier(string)\n" "\n" @@ -298,6 +314,12 @@ static PyMethodDef meth_bpy_resource_path = { METH_VARARGS | METH_KEYWORDS, bpy_resource_path_doc, }; +static PyMethodDef meth_bpy_resource_datafiles = { + "resource_datafiles", + (PyCFunction)bpy_resource_datafiles, + METH_NOARGS, + bpy_resource_datafiles_doc, +}; static PyMethodDef meth_bpy_escape_identifier = { "escape_identifier", (PyCFunction)bpy_escape_identifier, @@ -400,6 +422,9 @@ void BPy_init_modules(void) PyModule_AddObject(mod, meth_bpy_resource_path.ml_name, (PyObject *)PyCFunction_New(&meth_bpy_resource_path, NULL)); + PyModule_AddObject(mod, + meth_bpy_resource_datafiles.ml_name, + (PyObject *)PyCFunction_New(&meth_bpy_resource_datafiles, NULL)); PyModule_AddObject(mod, meth_bpy_escape_identifier.ml_name, (PyObject *)PyCFunction_New(&meth_bpy_escape_identifier, NULL)); ```

Philipp Oeser commented

2019-07-19 18:29:20 +02:00

note we are also in API freeze, so above solution would be for 2.81 I guess...

Shane Ambler commented

2019-07-22 04:24:56 +02:00

When setting BLENDER_SYSTEM_DATAFILES to relocate datafiles, P1045 works for me.

When setting BLENDER_SYSTEM_DATAFILES to relocate datafiles, [P1045](https://archive.blender.org/developer/P1045.txt) works for me.

blender-admin commented

2019-07-23 08:23:56 +02:00

This issue was referenced by 07f3ad06fc

This issue was referenced by 07f3ad06fc0900f719294c7b59598ac58e410dea

Campbell Barton commented

2019-07-23 08:31:50 +02:00

Changed status from 'Open' to: 'Resolved'

Campbell Barton closed this issue

2019-07-23 08:31:50 +02:00

Campbell Barton commented

2019-07-23 08:33:02 +02:00

Looked into this, BLENDER_DATAFILES isn't correct since it may use BLENDER_USER_DATAFILES which we never want in this case.

Committed alternate fix.

Looked into this, `BLENDER_DATAFILES` isn't correct since it may use `BLENDER_USER_DATAFILES` which we never want in this case. Committed alternate fix.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Toolbar icons do not honour BLENDER_SYSTEM_DATAFILES #63775