ID Deduplication on Append - UX #90546

New Issue

Bastien Montagne · 2021-08-09T15:25:24+02:00

Bastien Montagne commented

2021-08-09 15:25:24 +02:00

This task is about which behavior should be default when linking/appending IDs, and assets, and how much/how users should be able to customize those behaviors.

NOTE: Currently this is mostly listing topics to be discussed. This is also a first pass, there are likely many more use cases/topics to be discussed.

Problem

Currently, when appending a data-block, Blender appends (i.e. link and make local) the ID and all of its dependencies.

This means that if a single object in appended ten times, you will get ten copies of the object, but also of its meshes, materials, animations, etc.

This behavior may be fine in some cases, but not in others.

Default Behaviors

Generic Considerations

To avoid having many copies of the same ID, the first appended copy needs to be re-used for the subsequent appends.

Do we always re-use that ID, even if it has already been modified by the user locally?

Think first appended ID should always be re-used, reliably detecting modified IDs compared to linked one is possible, but tricky and not very efficient (RNA diffing)... There could be a mechanism though to break that reference such that next append creates a new copy of it. No diffing. User has ways to make an existing local ID not re-usable anymore, and to force not re-using an ID when appending again.

Could we use the UserPreferences settings for data copying, to decide which ID types we systematically create new copies for, and which ID types we try to re-use previously appended ID?

Specific Cases

Here is a list of different use cases, possible behaviors being:

Link.
Append (Link + Make Local, aka new copy).
Re-use of previously appended local ID.

Append Material

Data type	DEFAULT	Alternative Behavior 1
Material itself:	re-use	duplicate
Image:	NA	re-use
Node groups:	NA	duplicate
Actions:	NA	duplicate

Append Object

Data type	DEFAULT	Alternative Behavior 1
Object itself:	duplicate	duplicate
Obdata:	re-use	duplicate
Actions:	re-use	duplicate
Materials:	re-use	Apply Materials Rules ?

Append Collection

Collection itself:* In case of Empty-based instantiation:
*In case of ViewLayer-based instantiation: Object: same as above for Object case?

User Customization

Change behavior in UserSettings

Should user be able to change default behavior in user settings, or only on a case-by-case basis when actually linking/appending IDs or assets?

Change Behavior in Asset/File browsers

Should user be able to toggle options like "Link everything", or "Never deduplicate" ? etc.

This task is about which behavior should be default when linking/appending IDs, and assets, and how much/how users should be able to customize those behaviors. NOTE: Currently this is mostly listing topics to be discussed. This is also a first pass, there are likely many more use cases/topics to be discussed. # Problem Currently, when appending a data-block, Blender appends (i.e. link and make local) the ID and all of its dependencies. This means that if a single object in appended ten times, you will get ten copies of the object, but also of its meshes, materials, animations, etc. This behavior may be fine in some cases, but not in others. # Default Behaviors ## Generic Considerations To avoid having many copies of the same ID, the first appended copy needs to be re-used for the subsequent appends. # Do we always re-use that ID, even if it has already been modified by the user locally? *Think first appended ID should always be re-used, reliably detecting modified IDs compared to linked one is possible, but tricky and not very efficient (RNA diffing)... There could be a mechanism though to break that reference such that next append creates a new copy of it.* **No diffing.** User has ways to make an existing local ID not re-usable anymore, and to force not re-using an ID when appending again. # ~~Could we use the UserPreferences settings for data copying, to decide which ID types we systematically create new copies for, and which ID types we try to re-use previously appended ID?~~ ## Specific Cases Here is a list of different use cases, possible behaviors being: * Link. * Append (Link + Make Local, aka new copy). * Re-use of previously appended local ID. ### Append Material | Data type | DEFAULT | Alternative Behavior 1 | | --- | --- | --- | | Material itself: | re-use | *duplicate* | | Image: | NA | re-use | | Node groups: | NA | *duplicate* | | Actions: | NA | *duplicate* | ### Append Object | Data type | DEFAULT | Alternative Behavior 1 | | --- | --- | --- | | Object itself: | *duplicate* | *duplicate* | | Obdata: | re-use | *duplicate* | | Actions: | re-use | *duplicate* | | Materials: | re-use | Apply Materials Rules ? | ### Append Collection **Collection itself:*** In case of Empty-based instantiation: ***In case of ViewLayer-based instantiation:** Object: *same as above for Object case?* # User Customization ## Change behavior in UserSettings Should user be able to change default behavior in user settings, or only on a case-by-case basis when actually linking/appending IDs or assets? ## Change Behavior in Asset/File browsers Should user be able to toggle options like "Link everything", or "Never deduplicate" ? etc.

Bastien Montagne commented

2021-08-09 15:25:24 +02:00

Changed status from 'Needs Triage' to: 'Confirmed'

Bastien Montagne commented

2021-08-09 15:25:24 +02:00

Added subscriber: @mont29

Jeroen Bakker commented

2021-08-10 10:05:01 +02:00

Added subscriber: @Jeroen-Bakker

Jeroen Bakker commented

2021-08-10 10:05:01 +02:00

Efficiency depends also on how many times the code will be run. If the diffing process only runs once is a different store than running it multiple times. Updating caches also takes time and should be added into the consideration.

The mechanism could also be configured per operator. In D12116 I added a mechanism that could do this per operator in a generic way (not pushing this patch, just mentioning this for consideration).

Efficiency depends also on how many times the code will be run. If the diffing process only runs once is a different store than running it multiple times. Updating caches also takes time and should be added into the consideration. The mechanism could also be configured per operator. In [D12116](https://archive.blender.org/developer/D12116) I added a mechanism that could do this per operator in a generic way (not pushing this patch, just mentioning this for consideration).

Bastien Montagne added this to the Core project 2023-02-09 15:42:54 +01:00

Bastien Montagne removed this from the Core project 2023-02-09 18:20:37 +01:00

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

ID Deduplication on Append - UX #90546

Problem

Default Behaviors

Generic Considerations

Do we always re-use that ID, even if it has already been modified by the user locally?

Could we use the UserPreferences settings for data copying, to decide which ID types we systematically create new copies for, and which ID types we try to re-use previously appended ID?

Specific Cases

Append Material

Append Object

Append Collection

User Customization

Change behavior in UserSettings

Change Behavior in Asset/File browsers