Page MenuHome

Handle Data Dependencies
Confirmed, NormalPublicTO DO

Description

  • Indirect datablocks get appended (no auto dedup to start)
  • External files can use pack, append, then unpack (into e.g. textures/ folder next to .blend)

Event Timeline

Julian Eisel (Severin) renamed this task from Handle Data Dependencies (Pack Data) to Handle Data Dependencies.Nov 12 2020, 3:11 PM
Julian Eisel (Severin) changed the task status from Needs Triage to Confirmed.
Julian Eisel (Severin) created this task.
Julian Eisel (Severin) changed the subtype of this task from "Report" to "To Do".

It's a question on how to move a file with it's dependencies, at least if these are in relative subfolders. In the Python API, things work strangely. E.g. to change the path where the image will be unpacked, changing image.filepath doesn't work. You need to change image.packed_file.filepath to have the unpack working where you want it. It would be great to be able to pack/unpack without opening the file, and possibly also read unpacked file paths to be able to move them with the .blend.

I am happy to hear you are thinking of the essential "deduplication" function. I have seen this problem with duplicated materials on various forums reviewing your new Asset Browser.

Imagine a high quality city library, with about a 100 buildings and city detail meshes, sharing about a 100 textures. Upon dragging one building asset into the user project, almost all the 100 textures of about 4GB get uploaded to the GPU. If you drag the same (or a similar) building again, the textures get uploaded again, totaling to 8GB, and waiting the same time (5-10 seconds) again. By the time you are on the 4th asset, you have used up more GPU memory (and waited more) than the entire source library file.

Link importing by default would solve this problem of course. But if you were to put an option into the Asset Browser UI to change the dragging behavior to Append/Copy instead, the duplication of the materials and textures would still make a mess of the scene. Imagine changing the glass reflectivity on all the buildings, now you have to go through them all manually. There is no basic dedup function in Blender either. The current Asset Browser viewport option after dragging to change the import to a Link is currently broken with an "Object Not Found" error. It is also inefficient changing this after the drag and all that uploading to the GPU already.

As idiolistic as it is to embed everything (assets, textures, videos) into a single blender file for sharing with peers, it is also not practical in the industry for serious use. I think the new Asset, Linking and Library Override functions getting built into Blender are paving the way for more complex and higher quality projects. While the Library mentality is very useful for kickstarting new scenes, we must also consider these functions as basic as the "#include" in C++, to break down a scene into manageable modular pieces. Just like textures, it would make sense to include and search for asset blender files within the same directory (say the original link location was not found), not necessarily used as Libraries, but just dependencies. In that case, peer sharing linked data would be just as convenient as including those files in the same directory (or subdirectory), just like textures, without embedding everything into a single grand file.

Currently, the manual Link function of Blender works well in our pipeline, and your new filter by Asset feature is very useful. I am sure you are putting more thought into a robust and flexible API backend than the current UI. We are looking forward trying the Asset Browser with thumbnails when the Link importing becomes functional.