Page MenuHome

Solving the 'missing libraries' issue
Closed, ResolvedPublic

Tokens
"Yellow Medal" token, awarded by karja."Like" token, awarded by postolka."Like" token, awarded by Lapineige."Like" token, awarded by zlsa."Like" token, awarded by gandalf3."Love" token, awarded by GiantCowFIlms."Mountain of Wealth" token, awarded by Davd."Love" token, awarded by sdilmob."Love" token, awarded by tuqueque."Love" token, awarded by bliblubli.
Authored By

Description

This task aims at solving the issue caused by 'missing libraries'.

Note: This is a sub-task of T46049.

Currently, when a library file is missing for some reason (removed, rename, network filesystem outline, etc.), when you open a .blend file using some of its data, you get a warning about missing data and… must absolutely not save your .blend, otherwise you'll definitively lose track of those missing datablocks.

In relation with asset project, I’ve been working on this issue, and came with this proposal, with two main steps:

  1. Do not lose track of missing IDs anymore.
  2. Then, we should be able (in outliner probably) to edit missing libs' paths and reload the missing datablocks at any time.

I. Do not lose track of missing IDs

Now in Master

The idea here is to add a “place holder” ID when we cannot load a real one from a library. That place holder has the same type as missing one, and is a 'lib' ID, but contains no data.

Thus, you can use it as a regular library datablock in Blender - will just contain default data (no geometry, 'Empty' type for objects, default settings, etc. etc.).

And you can save and reload your file as you want - once library becomes available again, its data will be read at load time again.

D1394 implements that (only partially, as a proof of concept), it seems to work pretty well, but would need agreement this direction is valid before going further.

II. Reload the missing datablocks

First experimental builds available, see this post

Main part here is to be able to replace some ID by another without having to save/reload… WIP code is in id-remap branch.

Event Timeline

This can of course be really useful and its great you work on this.

Note that I think reloading missing datablocks (even on a very basic level - though Python API for example),
is important to have working - in order that this feature be very useful.

Said different - if the feature is added and reload ends up being an unsolvable can of worms, Then probably its better to rethink entire patch.


Some notes...

  • What if someone wants to leave the data missing and edit it? (make local and edit-in-place)? Would this be supported?
  • What if someone wants to replace a missing data-block with another one which exists in the file? - already linked (seems reasonable and probably not too hard to support)
  • A way to remove missing datablocks is probably needed too.

A detail from reading the patch... you split init from add, that makes sense, but introduce possible problems where existing data-block has all sorts of data (brush mtex... scene keying sets... all sorts of inter-linking pointers).

Suggest to be strict here and add an assert if _any_ data is non-zero after the ID struct. Otherwise we could end up in quite crappy situations with initialize data overwriting.

This can of course be really useful and its great you work on this.

Note that I think reloading missing datablocks (even on a very basic level - though Python API for example),
is important to have working - in order that this feature be very useful.

Said different - if the feature is added and reload ends up being an unsolvable can of worms, Then probably its better to rethink entire patch.

100% agree. Just wanted first advice on the idea of placeholder IDs. ;)


Some notes...

  • What if someone wants to leave the data missing and edit it? (make local and edit-in-place)? Would this be supported?

Missing data survives multiple .blend save & load. Further more, if:

  • you make a local copy of (aka localize) missing linked data, you just get 'empty' data, which is then completely detached from lib, as with non-missing data.
  • you make a proxy and edit that proxy, then proxy editing is saved and also works fine when missing lib becomes available again (note that this was checked with proxy for objects only, not sure about armature - but probably just nothing happens in this case, since there are no bones in placeholder data…).
  • What if someone wants to replace a missing data-block with another one which exists in the file? - already linked (seems reasonable and probably not too hard to support)

Think this is a bit an orthogonal topic (as in, could be useful even outside the 'missing lib' case), but definitively worth it, yes.

  • A way to remove missing datablocks is probably needed too.

Sure, that would be part of Outliner's handling of those imho.

A detail from reading the patch... you split init from add, that makes sense, but introduce possible problems where existing data-block has all sorts of data (brush mtex... scene keying sets... all sorts of inter-linking pointers).

Suggest to be strict here and add an assert if _any_ data is non-zero after the ID struct. Otherwise we could end up in quite crappy situations with initialize data overwriting.

Good point, will do. :)

Thanks for starting to work on this. This will be a major improvement and time saver when dealing with file shuffling and refactoring.

Do you plan to introduce a "Replace/Relocate" functionality to point an existing (and currently linked) library to a different endpoint? It would be extremely useful for live changes while the blendfile is open and also for a programmatic approach, where we remap some paths on startup or via a script. I guess given your design proposal this will be possible, but I just wanted to mention this.

Again, thanks for looking into this area, it will have great impact.

Yes, that kind of feature is definitively part of step II. :)

Just a note that work is now in a public branch, missing-libs.

hewi added a subscriber: hewi.Jul 17 2015, 1:37 AM

Ok, so pushed publicly another branch, tackling the other main part of this project - being able to replace ID datablocks by another 'on the fly'.

id-remap branch contains code allowing to replace all usages of a given ID by another ID (of same type of course).

Only contains quite rough, unfinished and most likely broken code, but it does survive some very very basic tests.

Together with ID placeholder, this should give us solid foundations for a much more powerful and flexible datablocks (and libraries) handling.

Bastien Montagne (mont29) updated the task description. (Show Details)

First experimental builds available, see this post

Bastien Montagne (mont29) closed this task as Resolved.Jun 23 2016, 8:28 PM
Bastien Montagne (mont29) claimed this task.

Recent commits (rB897e97f078e6 and rB7547c6a250cd6f) merged last part of this work into master, time to close this task. :)