Page MenuHome

Library Override: Investigate Resync Improvements.
Confirmed, NormalPublicDESIGN

Description

Having to manually deal with resync of overrides is a growing problem in production context. Further more, not all possible cases are yet covered by current resync code.

We have to find a way to detect all cases where a resync is needed. Then either warn user about it, or do it as a new, automatic step after .blend file has been read?

IMPORTANT: Current idea is to automatically and silently resync when user did not create any explicit overrides in replaced IDs. otherwise, we should rather tag out-of-sync IDs, and inform user about the issue.

Here is a list of cases (changes in library data) that affect ID pointers:

Note: Status reflects current code in override-refactor-tmp branch, not yet in master.

Step I

1) New ID

A new ID is added in the hierarchy/dependency chain of linked data that is already overridden, and that new ID should also be overridden (because e.g. it has dependencies over already overridden IDs). Or an existing linked ID starts using another one that is overridden, and therefor would also need to become overridden.

Detection
  • This can be easily detected using same code we currently use to re-sync/rebuild the overrides.
Handling
  • Current re-sync code is already working fine in that case.

2) ID re-assignment - Already Overridden

An ID usage in the hierarchy/dependency chain of linked data that is already overridden, gets re-assigned to a different ID that is also already overridden.

Detection
  • This needs some minimal extra/new work in override properties, to tag operations over ID pointer properties as 'matching' original linked data, or not, when generating override operations. We can then detect a mismatch when applying those rules (on file loading e.g.).
Handling
  • Current re-sync code is already working fine in that case.

3) ID re-assignment - Not Yet Overridden

An ID usage in the hierarchy/dependency chain of linked data that is already overridden, gets re-assigned to a different ID that would now need to be overridden.

Detection
  • Same case as 2) above.
Handling
  • Will require a full resync, not always properly handled by current resync code (when the previous assigned ID remains a valid one with an existing override).

4) ID unassignment

An ID usage in the hierarchy/dependency chain of linked data that is already overridden, gets unassigned.

Detection
  • Same case as 2) above.
Handling
  • Current re-sync code is already working fine in that case.

5) Removed ID

A currently overridden ID in the hierarchy/dependency chain of linked data that is already overridden, gets deleted (or not used anymore) in the linked data.

Note that this is close to 4) unassignment case above, but we may also need to clear the overrides of the removed ID? Although regular 'not used anymore' cleanup process on save & reload should also do the trick here.

Detection
  • Same case as 2) above.
Handling
  • Current re-sync code is already working fine in that case.

Step II

Report detected 'need resync' cases in UI, and/or perform needed resync automatically.

The later solution would be ideal, however it is not yet clear how heavy that process will be (and hence how much time it would add to loading production .blend files).

  • Try to only re-override needed IDs (in theory partial re-sync of only affected IDs should be doable?).
  • Add post-loading step to re-generate needed overrides.

Event Timeline

Bastien Montagne (mont29) changed the task status from Needs Triage to Confirmed.Dec 15 2020, 6:43 PM
Bastien Montagne (mont29) created this task.
Bastien Montagne (mont29) changed the subtype of this task from "Report" to "Design".Dec 15 2020, 8:18 PM

I've been testing overrides a bit, automatic re-sync at file loading seems an important feature. On some cases you just want to change something and re-render a bunch (100's) of shots without having to manually re-sync them. This can append several times during a production.
One solution (not the best) could be to have python operators that allow someone to implement automatic resync if needed . The resync can take times on some heavy assets (eg sets) which can make opening shots longer, but to me it's better than having to fix manually everything.
I'm really looking forward using overrides seems really cool and a great improvements toward proxies.