Page MenuHome

bpy.data custom properties support
Needs ReviewPublic

Authored by Alexander Romanov (a.romanov) on Mar 14 2016, 3:32 PM.

Details

Summary

The main issue which is solved by this patch is that the Blender data structure doesn't allow to store custom properties per file.
There are some workarounds described here:
http://blender.stackexchange.com/questions/8290/how-to-register-global-i-e-unique-per-file-properties
But all of theese workarounds are hacks. So this patch makes it possible to store user properties in bpy.data.
Note that you can't link them from another file, they are more like current file metadata.

Diff Detail

Event Timeline

Alexander Romanov (a.romanov) retitled this revision from to bpy.data custom properties support.Mar 14 2016, 3:32 PM
Alexander Romanov (a.romanov) updated this object.
Alexander Romanov (a.romanov) updated this object.
Alexander Romanov (a.romanov) updated this revision to Diff 6255.

First of all. think in most cases this shouldn't be needed, however - as with Blender there are some limited cases where you may want to store data in the file
(auto-pack options for eg).

Even so, could you give some detailed use case as to why this is needed?

Other notes with this patch

Am not really keen on having scripts add new members to bpy.data via bpy.props, as we do with Scene/Object/Mesh... etc.

This is nice to keep for accessing the blend file as data-base, adding arbitrary attributes is a bit confusing.

Also, with this patch there's no way to access properties of linked in files, which would be good to support.

Possible alternative

WindowManager Data-Block:
We could handle this by writing WindowManager properties to disk (which is a singleton) properties to disk (currently they aren't saved and are used for run-time data).

To allow both runtime and saved vars we could support the SKIP_SAVE property, to optionally not save run-time data to disk - this can be generally useful anyway.

Library Data-Block:

Similar to using window-manager, a library data block could store properties, and advantage with this is you could read it from linked Blend files.
however having a local library data-block currently doesn't exist.

Access via attribute:

We could add bpy.data.properties, not really keen on this, but it at least means we dont add data directly into bpy.data.

First of all. think in most cases this shouldn't be needed, however - as with Blender there are some limited cases where you may want to store data in the file
(auto-pack options for eg).
Even so, could you give some detailed use case as to why this is needed?

  1. In blend4web we use mapping to exported file: Now we use scene property with duplicated value for each scene. But would like to use just one.
  2. Such property could be a name of the author or other meta information.
  3. Also we have some tool configuration which is more better to store per "exchange" file, not in local preferences.

Other notes with this patch

Am not really keen on having scripts add new members to bpy.data via bpy.props, as we do with Scene/Object/Mesh... etc.
This is nice to keep for accessing the blend file as data-base, adding arbitrary attributes is a bit confusing.
Also, with this patch there's no way to access properties of linked in files, which would be good to support.

I assumed that this properties should have no ability to link in file, but it makes sense for possible future using.

Possible alternative

WindowManager Data-Block:
We could handle this by writing WindowManager properties to disk (which is a singleton) properties to disk (currently they aren't saved and are used for run-time data).
To allow both runtime and saved vars we could support the SKIP_SAVE property, to optionally not save run-time data to disk - this can be generally useful anyway.

It could be a simplest solution, but now there is no ability to link WindowManager and the purpose of linking is not clear for me. But in any case, it's suitable for our needs.

Library Data-Block:
Similar to using window-manager, a library data block could store properties, and advantage with this is you could read it from linked Blend files.
however having a local library data-block currently doesn't exist.

This seems to me the best solution. And possibly we need no local library data-block. bpy.data.filepath is already correnspond to lib.filepath. So why can't we make id properties translation? And for implementation we can use this patch.

This is nice to keep for accessing the blend file as data-base, adding arbitrary attributes is a bit confusing.

So bpy.data - database; bpy.data[propname] - some root database metadata. To get metadata of linked database we could use bpy.data.libraries[libname][propname]

Access via attribute:
We could add bpy.data.properties, not really keen on this, but it at least means we dont add data directly into bpy.data.

If properties is a collection, then this is the same case as with any other data-block, but it makes sense if adding/removing of elements through UI is limited (and this is the main idea to hide such entity). And collection is more preferable then just the only one datablock, because addon developers would be able to separate their own global properties using property data-block as a property group.

This feature has been requested numerous times, see also e.g. http://blender.stackexchange.com/questions/15729/metadata-notes-comments-in-blend-files We'll also likely need this at some point in asset project (advanced asset engines are likely to want to store some data in their libraries somehow).

I’d rather like to see this added to Library datablock, yes. I think we should split between runtime representation of those props, and how they are actually saved in .blend, too.

  1. Runtime representation: added to Library IDs. Then there is problem of how to give access to 'current file' properties - we already have filepath in bpy.data, so I wonder if we should not add a Library(-like) struct to main, with filepath, status (saved/dirty/etc.), custom properties… Would be cleaner than putting everything directly in Main (bpy.data) as we do now.
  1. How/where do we store those custom props in .blend? This has to be carefully thought, given the fact we use 'last' Lib datablock as marker of 'we are reading place-holder data for datablocks linked from that library', this is part of .blend format definition. So think we should avoid storing custom props from/in Library datablocks (also, would not make sense to store those from linked libraries). Guess proposed solution in this patch is fine, but maybe we could also investigate using a real meta-data solution (similar to how we handle file preview, e.g. adding a single text blob just after preview, and then agreeing on some format [XMP? mere JSON?] for it). Would make its extraction from .blend much simpler and lighter than handling a collection of custom props…
  1. Runtime representation: added to Library IDs. Then there is problem of how to give access to 'current file' properties - we already have filepath in bpy.data, so I wonder if we should not add a Library(-like) struct to main, with filepath, status (saved/dirty/etc.), custom properties… Would be cleaner than putting everything directly in Main (bpy.data) as we do now.

I can put all this stuff into substructure.

  1. How/where do we store those custom props in .blend? This has to be carefully thought, given the fact we use 'last' Lib datablock as marker of 'we are reading place-holder data for datablocks linked from that library', this is part of .blend format definition. So think we should avoid storing custom props from/in Library datablocks (also, would not make sense to store those from linked libraries). Guess proposed solution in this patch is fine, but maybe we could also investigate using a real meta-data solution (similar to how we handle file preview, e.g. adding a single text blob just after preview, and then agreeing on some format [XMP? mere JSON?] for it). Would make its extraction from .blend much simpler and lighter than handling a collection of custom props…

Actually, we want to use these properties in UI, thus JSON or other format as replacement is not what we need.

  1. How/where do we store those custom props in .blend? This has to be carefully thought, given the fact we use 'last' Lib datablock as marker of 'we are reading place-holder data for datablocks linked from that library', this is part of .blend format definition. So think we should avoid storing custom props from/in Library datablocks (also, would not make sense to store those from linked libraries). Guess proposed solution in this patch is fine, but maybe we could also investigate using a real meta-data solution (similar to how we handle file preview, e.g. adding a single text blob just after preview, and then agreeing on some format [XMP? mere JSON?] for it). Would make its extraction from .blend much simpler and lighter than handling a collection of custom props…

Actually, we want to use these properties in UI, thus JSON or other format as replacement is not what we need.

You misunderstood that point. It is strictly about how to save it in .blend file. UI & co are run-time topic, which goes to point 1 (and hence uses custom props). What I’m saying here is that, to ease usage outside of Blender context, we may want to save those custom props (meta .blend file data) in an easier/more universal format than binary dump of customprops.

You misunderstood that point. It is strictly about how to save it in .blend file. UI & co are run-time topic, which goes to point 1 (and hence uses custom props). What I’m saying here is that, to ease usage outside of Blender context, we may want to save those custom props (meta .blend file data) in an easier/more universal format than binary dump of customprops.

Ah, got it now. But what if I don't want my property to be in the meta information block (just want it to be "per file")? Assume that we have made substructure called "properties" for the above-mentioned stuff. So, for meta information we could use addressing like bpy.data.properties.metainfo["blabla"] or bpy.data.metainfo["blabla"]. And for usual properties just bpy.data.properties["blabla"]? Thus, it seems that there should be two separate IDProperty *prop and two separate issues. One of these properties would be serialized for external use.

I’d rather not have two systems here, and I do not really see why you would not want your prop in metadata area? using some json-like stuff is just to ease external access, everybody can read a .blend file content anyway, it’s just a bit harder when it’s a binary dump…

I’d rather not have two systems here, and I do not really see why you would not want your prop in metadata area? using some json-like stuff is just to ease external access, everybody can read a .blend file content anyway, it’s just a bit harder when it’s a binary dump…

I just imagined that people will use this metadata storage for things that is not true metadata (for example some tool state). And for example, some file manager will do blend file metadata representation as is, without concrete filed selection. Thus many blend files will show the garbage that would be better to hide.