Asset System: Data Storage, Reading & UI Access #88184

New Issue

Julian Eisel · 2021-05-10T18:56:15+02:00

Julian Eisel commented

2021-05-10 18:56:15 +02:00

Asset System: Data Storage, Reading & UI Access

Internal architecture for storing and accessing assets (actually asset-handles, i.e. an efficient representation of the asset), as well as preparing them for display in the UI.

Current Design

The current Asset Browser design is based on the File Browser. This already has a system for data storage, reading and UI access: FileList

About FileList

Each File Browser editor stores a FileList. It does a fair bit of heavy lifting. Its main responsibilities are:

Concurrent, possibly recursive loading of files from disk (including .blend libraries).
Reading of different kinds of "files" (general files, .blend libray data, current Main).
Storing all necessary data on the read files (name, path, preview image, file type, ...)
Lazy-cached, concurrent loading of preview images. (Previews around the current view rectangle are loaded into cache.)
Consistently storing selection state.
Filtering and sorting.

There is also a fair amount of code for the old asset engine design.

All of these features are good to have in general, we should not remove them.

Its Problems

FileList does too many things; there is no separation of concerns.

On its own that already makes maintainance hard. For the asset system we need something well designed, or we'll quickly run into problems when designing further functionality, like the Python API.

Code is very hard to follow and reason about.
The asset system requires a storage that is independent of the File Browser.

We need global access for Python and the asset views (UI template to show a mini asset browser in regular layouts, e.g. panels, menus, regions, etc).

FileList does UI specific things that are already done elsewhere in the UI.

E.g. the filtering and sorting it does are also done for UI-Lists, Outliners, animation channel lists, etc. This redundency is unnecessary overhead.
The asset view uses template uses UI-Lists plus `FileList` via an asset specific abstraction. So the redundancy is quite apparent and ugly here.

Proposal

We do not need to start entirely from scratch. We should be able to pick apart FileList and re-build it into a new design. Here is how that could look.

Separation of Concerns

The different responsibilities from above can be split into a few basic concerns:

Reading: Advanced reading of different kinds of data.
Storing: Storing the data as some abstraction (e.g. file or asset).
Viewing: Provide a view into the data for the UI (includes preview image loading/caching, filtering, sorting, etc.)

These are the main concernes that should be separated clearly, with well defined interfaces in-between.

Reading

AbstractFileReader? TBD.
(A bit awkward to call assets "files", besides that it makes sense. Better naming idea welcome.)

TODO. Basically: Recursive, concurrent loading of files from disk (including .blend libraries). Different kinds of files, so OOP style inheritance makes sense.

In future we might want a file-reading implementation for asset caches too. These would be simple .json files with meta-data stored inside an asset library.

Could be exposed to Python somehow, so Add-ons can implement custom asset reading, e.g. via the web.

Storing

Base class AbstractFileList.

The asset specific implementation would look like this:

class AssetList : public AbstractFileList {
  *...* Iterators, queries to get meta-data like names, previews and `AssetData`,

// etc.

};

There could be a separate asset list for each library. That way we can cache that data as we like. Note that we can also lazy-load parts of an asset library into the list.

We'll need some way to identify an asset library (also needed in DNA, so uses C here):

struct AssetLibraryReference {
  /* ... */
  /* Type information and custom library identifier(s). */
};

Sybren had the interesting idea of using URIs for this.

There would be a global storage for assets:

class AssetListStorage {

/* Global map as storage of asset-lists, per library. */
static AssetListMap global_storage_;

public:
  *...* Functions to fetch libraries into storage. All functions `static`.
};

Note 1:
In principle, asset storage for external asset libraries could be kept alive over file reads.

Note 2:
The File Browser could use some global storage system as well. So that loading the same directory in multiple File Browsers will share data.

This essentially is a "model" in the model-view-controller pattern.

Viewing

If the storage part represents the model, this part represents the view. Idea is a general abstraction: uiModelView. It gives a view into a (potentially big) data-set, i.e. a model. This could be useful for the rest of Blender too, but we can keep it very simple for the start.

For assets, there would be the following model-view:

class AssetView : public uiModelView {
  *...* Methods for lazy-caching, filtering, and querying information about the 
  // model (e.g. the name and preview of an item).
}

UIs like the UIList or the File Browser can get their view onto data via the uiModelView interface. Once the data is fetched into such a view, they do not need to know about the exact implementation (e.g. if this is an asset list, a file list, an RNA collection, ...).

Putting it all Together

Who is responsible for putting this all together?

Few things:

An Add-on can register custom asset libraries, using an AssetLibraryReference (could be a URI).
Other libraries are registered by Blender, as is already done.
If the Asset Library is activated in the UI, the UI or editor asks the AssetListStorage to fetch the needed assets. For example the asset-view UI template does this before creating the list.
The AssetList or AssetListStorage creates the appropriate reader and invokes it via the AbstractFileReader interface. The reading can be threaded for non-local data.
The UI/editor requests an AssetView from the read AssetList. This way it gets a view into the data that it can work with for drawing, interaction, filtering, etc.
Special attention needs to be put on data-updates. Especially local data needs to be represented in a consistent state, or things can go bad. E.g. deletion of local asset data, undo/redo, file reading (with and without UI), threaded loading of preview images, ... (See 3d706bb0f3dd: Do proper updates when local asset data changes).

The File Browser

The File Browser should be ported to using this new design as well. I.e. FileList should be entirely replaced by the new design.

Workload

All of this sounds like a lot of work. Things can be kept very simple for the start though. We don't need rich APIs yet. This is just about thinking carefully about the general direction, which will have long-term impact. Once we know which direction to go, we can add just the bits necessary for what we need for the current project goals.

# Asset System: Data Storage, Reading & UI Access Internal architecture for storing and accessing assets (actually asset-handles, i.e. an efficient representation of the asset), as well as preparing them for display in the UI. ## Current Design The current Asset Browser design is based on the File Browser. This already has a system for data storage, reading and UI access: `FileList` **About `FileList`** Each File Browser editor stores a `FileList`. It does a fair bit of heavy lifting. Its main responsibilities are: * Concurrent, possibly recursive loading of files from disk (including .blend libraries). * Reading of different kinds of "files" (general files, .blend libray data, current `Main`). * Storing all necessary data on the read files (name, path, preview image, file type, ...) * Lazy-cached, concurrent loading of preview images. (Previews around the current view rectangle are loaded into cache.) * Consistently storing selection state. * Filtering and sorting. There is also a fair amount of code for the old asset engine design. All of these features are good to have in general, we should not remove them. **Its Problems** * `FileList` does too many things; there is no separation of concerns. ``` On its own that already makes maintainance hard. For the asset system we need something well designed, or we'll quickly run into problems when designing further functionality, like the Python API. ``` * Code is very hard to follow and reason about. * The asset system requires a storage that is independent of the File Browser. ``` We need global access for Python and the asset views (UI template to show a mini asset browser in regular layouts, e.g. panels, menus, regions, etc). ``` * `FileList` does UI specific things that are already done elsewhere in the UI. ``` E.g. the filtering and sorting it does are also done for UI-Lists, Outliners, animation channel lists, etc. This redundency is unnecessary overhead. The asset view uses template uses UI-Lists plus `FileList` via an asset specific abstraction. So the redundancy is quite apparent and ugly here. ``` ---- ## Proposal We do not need to start entirely from scratch. We should be able to pick apart `FileList` and re-build it into a new design. Here is how that could look. **Separation of Concerns** The different responsibilities from above can be split into a few basic concerns: * **Reading:** Advanced reading of different kinds of data. * **Storing:** Storing the data as some abstraction (e.g. file or asset). * **Viewing:** Provide a view into the data for the UI (includes preview image loading/caching, filtering, sorting, etc.) These are the main concernes that should be separated clearly, with well defined interfaces in-between. **Reading** `AbstractFileReader`? TBD. *(A bit awkward to call assets "files", besides that it makes sense. Better naming idea welcome.)* *TODO. Basically: Recursive, concurrent loading of files from disk (including .blend libraries). Different kinds of files, so OOP style inheritance makes sense.* In future we might want a file-reading implementation for asset caches too. These would be simple .json files with meta-data stored inside an asset library. Could be exposed to Python somehow, so Add-ons can implement custom asset reading, e.g. via the web. **Storing** Base class `AbstractFileList`. The asset specific implementation would look like this: ```lang=cpp class AssetList : public AbstractFileList { *...* Iterators, queries to get meta-data like names, previews and `AssetData`, ``` // etc. ``` }; ``` There could be a separate asset list for each library. That way we can cache that data as we like. Note that we can also lazy-load parts of an asset library into the list. We'll need some way to identify an asset library (also needed in DNA, so uses C here): ```lang=cpp struct AssetLibraryReference { /* ... */ /* Type information and custom library identifier(s). */ }; ``` Sybren had the interesting idea of using URIs for this. There would be a global storage for assets: ```lang=cpp class AssetListStorage { ``` /* Global map as storage of asset-lists, per library. */ static AssetListMap global_storage_; ``` public: *...* Functions to fetch libraries into storage. All functions `static`. }; ``` > **Note 1:** > In principle, asset storage for external asset libraries could be kept alive over file reads. > **Note 2:** > The File Browser could use some global storage system as well. So that loading the same directory in multiple File Browsers will share data. This essentially is a "model" in the model-view-controller pattern. #### Viewing If the storage part represents the *model*, this part represents the *view*. Idea is a general abstraction: `uiModelView`. It gives a view into a (potentially big) data-set, i.e. a model. This could be useful for the rest of Blender too, but we can keep it very simple for the start. For assets, there would be the following model-view: ```lang=cpp class AssetView : public uiModelView { *...* Methods for lazy-caching, filtering, and querying information about the // model (e.g. the name and preview of an item). } ``` UIs like the `UIList` or the File Browser can get their view onto data via the `uiModelView` interface. Once the data is fetched into such a view, they do not need to know about the exact implementation (e.g. if this is an asset list, a file list, an RNA collection, ...). **Putting it all Together** Who is responsible for putting this all together? Few things: * An Add-on can register custom asset libraries, using an `AssetLibraryReference` (could be a URI). * Other libraries are registered by Blender, as is already done. * If the Asset Library is activated in the UI, the UI or editor asks the `AssetListStorage` to fetch the needed assets. For example the asset-view UI template does this before creating the list. * The `AssetList` or `AssetListStorage` creates the appropriate reader and invokes it via the `AbstractFileReader` interface. The reading can be threaded for non-local data. * The UI/editor requests an `AssetView` from the read `AssetList`. This way it gets a view into the data that it can work with for drawing, interaction, filtering, etc. * Special attention needs to be put on data-updates. Especially local data needs to be represented in a consistent state, or things can go bad. E.g. deletion of local asset data, undo/redo, file reading (with and without UI), threaded loading of preview images, ... (See [3d706bb0f3dd: Do proper updates when local asset data changes](https://developer.blender.org/rB3d706bb0f3dd04f7f1d01dbe008463648a6f66f2)). **The File Browser** The File Browser should be ported to using this new design as well. I.e. `FileList` should be entirely replaced by the new design. ## Workload All of this sounds like a lot of work. Things can be kept very simple for the start though. We don't need rich APIs yet. This is just about thinking carefully about the general direction, which will have long-term impact. Once we know which direction to go, we can add just the bits necessary for what we need for the current project goals.

Julian Eisel commented

2021-05-10 18:56:15 +02:00

Added subscriber: @JulianEisel

Julian Eisel commented

2021-06-14 17:42:08 +02:00

Quick update on how the asset system could work together with the file system/list:

Quick update on how the asset system could work together with the file system/list: ![54D6E717-11AB-4339-9B5E-88DA429EA7DB_1_105_c.jpeg](https://archive.blender.org/developer/F10171110/54D6E717-11AB-4339-9B5E-88DA429EA7DB_1_105_c.jpeg)

Mike Drake commented

2022-01-13 19:50:35 +01:00

Added subscriber: @Michael-Drake

Alexey Adamitsky commented

2022-11-15 08:36:05 +01:00

Added subscriber: @AlexeyAdamitsky

Alaska commented

2024-04-07 06:16:25 +02:00

I am removing the Needs Triage label. This is under the general rule that Design and TODO tasks should not have a status.

If you believe this task is no longer relevant, feel free to close it.

I am removing the `Needs Triage` label. This is under the general rule that Design and TODO tasks should not have a status. If you believe this task is no longer relevant, feel free to close it.

Alaska removed the

Status

Needs Triage

label 2024-04-07 06:16:34 +02:00

Julian Eisel commented

2024-04-08 20:08:15 +02:00

I don't really want to go this direction anymore, it was from early design exploration, so closing this.

Julian Eisel closed this issue

2024-04-08 20:08:20 +02:00

Blender Bot added the

Status

Archived