Add UUID to Blender data-blocks #83019

New Issue

Bastien Montagne · 2020-11-25T20:43:50+01:00

Bastien Montagne commented

2020-11-25 20:43:50 +01:00

Having UUIDs to reference data-blocks of Blender would help in some cases with interaction with third party tools, especially when relations between different types of data storage is needed, and handling renaming of data-blocks across those is not possible.

This design task is here to define what would be expected from such feature, and how various issues should be addressed.

See also this ML discussion.*Disclaimer: this is a draft design proposal currently.//

Core Ideas

UUIDs should be 'real' ones, 128bits long.
UUIDs should be generated on demand, not systematically. This process is not free, and internal handling of data-blocks by Blender itself should not use them at all.
** Think we should still store them as a systematic fixed chunk of data in the ID block in the .blend file though (being zeroed for non-generated ones), to ease their retrieval from third-party code 'parsing' .blend files?

Issues to Investigate

Collision and Uniqueness

UUIDs should be absolutely unique withing a .blend file. This would be easy to ensure.Linkeddata-block would not be concerned by this, since they belong to another .blend file. UUIDs should be reasonably unique across .blend files.
Creating new IDs from existing ones (internal copy, appending from a library, creating proxies or overrides from linked data) should nullify the UUID of the new data-block. 'Save file' operations explicitly creating a duplicate file (Save as, Save copy) should maybe enforce re-generating UUIDs?
*** .blend file duplication would be the main source of collision. Not sure how to deal with that case, besides offering the possibility of a manual re-generation of UUIDs by user?
*** If the 'file copy' case is solved, collision between different .blend files remains possible, though extremely unlikely.

Runtime IDs

Some system may generate a random number of data-blocks in a procedural way (e.g. from future geometry nodes evaluation.* Persistence of UUID in such IDs is probably not possible? We should then accept that it would need to be re-generated when nodetree re-create data-blocks?
*** This only concerns IDs that would be generated in Main database (and hence saved on disk), not runtime IDs.

Implementation Hints

Search for existing implementations of UUIDs (e.g. in python).
Using OS-provided tools should be preferred solution as much as possible.
How should RNA (Py API) expose those 128bits UUIDs, as an array of 4 integers? An hexadecimal-encoded string?

Having UUIDs to reference data-blocks of Blender would help in some cases with interaction with third party tools, especially when relations between different types of data storage is needed, and handling renaming of data-blocks across those is not possible. This design task is here to define what would be expected from such feature, and how various issues should be addressed. See also this [ML discussion](https:*lists.blender.org/pipermail/bf-committers/2020-November/050751.html).*Disclaimer: this is a **draft** design proposal currently.// Core Ideas ---------- * UUIDs should be 'real' ones, 128bits long. * UUIDs should be generated on demand, not systematically. *This process is not free, and internal handling of data-blocks by Blender itself should not use them at all.* ** Think we should still store them as a systematic fixed chunk of data in the ID block in the .blend file though (being zeroed for non-generated ones), to ease their retrieval from third-party code 'parsing' .blend files? Issues to Investigate ------- # Collision and Uniqueness **UUIDs should be absolutely unique withing a .blend file. *This would be easy to ensure.**Linked**data-block would not be concerned by this, since they belong to another .blend file.*** UUIDs should be reasonably unique across .blend files. ***Creating new IDs from existing ones (internal copy, appending from a library, creating proxies or overrides from linked data) should nullify the UUID of the new data-block.*** 'Save file' operations explicitly creating a duplicate file (Save as, Save copy) should maybe enforce re-generating UUIDs? *** .blend file duplication would be the main source of collision. Not sure how to deal with that case, besides offering the possibility of a manual re-generation of UUIDs by user? *** If the 'file copy' case is solved, collision between different .blend files remains possible, though extremely unlikely. # Runtime IDs **Some system may generate a random number of data-blocks in a procedural way (e.g. from future geometry nodes evaluation.*** Persistence of UUID in such IDs is probably not possible? We should then accept that it would need to be re-generated when nodetree re-create data-blocks? *** This only concerns IDs that would be generated in Main database (and hence saved on disk), not runtime IDs. Implementation Hints --------- * Search for existing implementations of UUIDs (e.g. in python). * Using OS-provided tools should be preferred solution as much as possible. * How should RNA (Py API) expose those 128bits UUIDs, as an array of 4 integers? An hexadecimal-encoded string?

Bastien Montagne commented

2020-11-25 20:43:50 +01:00

Changed status from 'Needs Triage' to: 'Confirmed'

Bastien Montagne commented

2020-11-25 20:43:50 +01:00

Added subscribers: @mont29, @brecht

Brecht Van Lommel commented

2020-12-01 17:32:21 +01:00

to ease their retrieval from third-party code 'parsing' .blend files?

I don't think this is really important to take this into account, in general I don't think third-party code should be parsing .blend files. If they do I imagine they should be able to read arbitrary structs anyway.

UUIDs should be absolutely unique withing a .blend file. This would be easy to ensure.

It is easy, but if we want to make them unique across .blend files, then we probably use an algorithm where it's not so easy to ensure this. We could check for collisions, but the memory/performance cost is probably not worth it.

Runtime IDs: Persistence of UUID in such IDs is probably not possible?

I think we can probably generate persistent IDs, by hashing the UUID of the original datablock with the persistent ID for instances somehow:
https://docs.blender.org/api/current/bpy.types.DepsgraphObjectInstance.html#bpy.types.DepsgraphObjectInstance.persistent_id

> to ease their retrieval from third-party code 'parsing' .blend files? I don't think this is really important to take this into account, in general I don't think third-party code should be parsing .blend files. If they do I imagine they should be able to read arbitrary structs anyway. > UUIDs should be absolutely unique withing a .blend file. This would be easy to ensure. It is easy, but if we want to make them unique across .blend files, then we probably use an algorithm where it's not so easy to ensure this. We could check for collisions, but the memory/performance cost is probably not worth it. > Runtime IDs: Persistence of UUID in such IDs is probably not possible? I think we can probably generate persistent IDs, by hashing the UUID of the original datablock with the persistent ID for instances somehow: https://docs.blender.org/api/current/bpy.types.DepsgraphObjectInstance.html#bpy.types.DepsgraphObjectInstance.persistent_id

Toni Alatalo commented

2020-12-02 11:10:28 +01:00

Added subscriber: @antont

Toni Alatalo commented

2020-12-02 11:10:28 +01:00

Hi, just a simple comment: maybe the UUIDs are in a way not Blender core data, but more like saved information about a previous export. So consecutive exports to update the model in game engine or BIM tool could manage the update in the import respectively. Blender already has internal IDs which are unique within the session. The idea to have optional UUIDs is in line with this thinking. I mean, an export can create the UUIDs when necessary, but Blender would then store them. Maybe there are no other consequences from this point of view.

The UUIDs could also be in just some exporter specific history data but it seems nice that Blender would support them internally so they'd show in the GUI the same way and it would be consistent for all exports (given they support UUIDs, what else besides BIM does?).

Hi, just a simple comment: maybe the UUIDs are in a way not Blender core data, but more like saved information about a previous export. So consecutive exports to update the model in game engine or BIM tool could manage the update in the import respectively. Blender already has internal IDs which are unique within the session. The idea to have optional UUIDs is in line with this thinking. I mean, an export can create the UUIDs when necessary, but Blender would then store them. Maybe there are no other consequences from this point of view. The UUIDs could also be in just some exporter specific history data but it seems nice that Blender would support them internally so they'd show in the GUI the same way and it would be consistent for all exports (given they support UUIDs, what else besides BIM does?).

Swann Martinez commented

2020-12-02 13:12:56 +01:00

Added subscriber: @slumber

Himanshi Kalra commented

2020-12-02 14:33:55 +01:00

Added subscriber: @Calra

Robert Guetzkow commented

2020-12-02 15:17:59 +01:00

Added subscriber: @rjg

Philippe Crassous commented

2020-12-03 09:33:52 +01:00

Added subscriber: @PhilippeCrassous

Aurelien Vaillant commented

2021-08-12 11:22:11 +02:00

Added subscriber: @stilobique

Aurelien Vaillant commented

2021-08-12 11:25:58 +02:00

Having UUIDs to reference data-blocks of Blender would help in some cases with interaction with third party tools, especially when relations between different types of data storage is needed, and handling renaming of data-blocks across those is not possible.

That excalty my use case. It's a must have to use an implementation way. Nothing is design about a python use, expose this value with the BPY ?

> Having UUIDs to reference data-blocks of Blender would help in some cases with interaction with third party tools, especially when relations between different types of data storage is needed, and handling renaming of data-blocks across those is not possible. That excalty my use case. It's a must have to use an implementation way. Nothing is design about a python use, expose this value with the BPY ?

Bastien Montagne commented

2021-08-12 15:17:49 +02:00

@stilobique this would obviously be exposed to Python yes, through RNA system. only question here is how to do that (array of ints, hex string, ...), as stated in the task.

AquaticNightmare commented

2021-08-13 12:03:22 +02:00

Added subscriber: @AquaticNightmare

Daniel Salazar commented

2022-02-12 23:12:16 +01:00

Added subscriber: @zanqdo

Daniel Salazar commented

2022-02-12 23:12:16 +01:00

Hi, this is a proposal for a UUID based linking system which would be essentially unbreakable, unlike the flimsy absolute or relative paths

https://youtu.be/8dOJLy6yp3U

Please take a look! I think it's a natural followup

Hi, this is a proposal for a UUID based linking system which would be essentially unbreakable, unlike the flimsy absolute or relative paths https://youtu.be/8dOJLy6yp3U Please take a look! I think it's a natural followup

Mattias Fredriksson commented

2022-11-21 14:30:12 +01:00

Added subscriber: @Osares

Aeron Miles commented

2023-01-08 16:34:46 +01:00

Added subscriber: @Aeron-Miles

Philipp Oeser removed the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Add UUID to Blender data-blocks #83019

Core Ideas

Issues to Investigate

Collision and Uniqueness

Runtime IDs

Implementation Hints