Library overrides #73318

Open
opened 2020-01-22 16:40:30 +01:00 by Dalai Felinto · 53 comments

Status: Achieved 2021 targets have been 'archived' at the end of this task, which is now updated for 2022 targets.


Team

Commissioner: @eyecandy @Hjalti @Rikstopher
Project leader: @mont29
Project members: @JulianEisel @Jeroen-Bakker

2022 Targets

Milestone 1 - Resync Performances

  • #94059 (Library Override Resync Performance).
    • #95283 (LibOverride Stronger Hierarchy)
    • #94185 (Outliner ID remapping performance), #95279 (Remap multiple items in referenced data)
    • #95682 (Library Override partial Resync)

Milestone 2 - Improve Usability

  • #95708 (Library Override - Usability).
    • #95707 (Library Override - System override IDs)
    • #95802 (Library Override - Outliner UI/UX)
    • #95816 (Library Override - 'Cherry-Picked' Properties)

Milestone 3 - Dynamic Overrides

  • #54792 (Dynamic Overrides - View layer overrides with override sets) (old, to be revised/updated and fleshed out)

Not A Milestone - Library Overrides for Material

  • Solve issues related to the weird way materials are linked to object or obdata.
  • Support properly nodegroups (i.e. embedded data-blocks, some preliminary work has already been done but needs to be finalized and validated). #82404 (Library Overrides: Shader drivers stop working after save and reload)

Not A Milestone - Future Ideas
Those are ideas existing from the beginning of the project, and which have a partial implementation in core code, but need more UI/UX design and development.

  • #72629 (Library Override: Support editing of animation data in overriden data-blocks.).
  • #82503 (Add Support of PointCaches to Library Overrides).
  • Allow addition of CustomData layers to geometry IDs (new UV maps, shape keys, vertex groups...). This will require a lot of preliminary work.
  • Advanced types of override operations (multiplicative factor for 'factor' porperties like scale, 'bit wise' operations on enum flags, ...).

Relevant links:


2021 Targets

Description
Big picture: Replace animation proxy system, and support local changes from linked data.

Use cases:

  • Animators using proxies for animation.
  • Multiple instances of the same character with different animations.
  • Multiple instances of the same character with variations (e.g., material tweaks, ...).
  • Set dressing with local variations of instanced assets.
  • Override linked in character with cached animation [cache modifier]

Design:

  • Asses feasibility of auto-override - direct change of linked data
  • Need storage optimization document #78944 (LibOverride: How to handle heavy data-blocks (meshes, shape-keys, actions...))
  • Need final signed off design for outliner interaction
  • Need final design for the ID template #78012 (UI: Search Menu Design for Linked Data-blocks)

Engineer plan:
None - underlying code is already implemented.

Work plan

Milestone 1 - Usability Pass

  • #76555 (Library Overrides: Usability issues/paper-cuts)
  • #78012 (UI: Search Menu Design for Linked Data-blocks), #79959 (Data-block Selector Design Changes)
  • D7631: Library Overrides UI: Show library overrides in Outliner
  • #73154 (Library Overrides - UI: Add access to the 'auto override' flag of IDs) (Might become unnecessary, postponed until we know.)
  • Enhance ways to add more than one overrides of same data (this is currently possible, but not obvious nor user-friendly at all).
  • #77083 (Library Overrides Constraint and Modifier Access)
  • #79562 (LibOverride: Support edition of lists (colletions) of ID properties/py-defined RNA collections)

Milestone 2 - Proxy Replacement

  • #81049 (Add operator to convert proxies to library overrides )
  • #72629 (Library Override: Support editing of animation data in overriden data-blocks.)
  • #82503 (Add Support of PointCaches to Library Overrides)
  • Implement real override of shapekeys data-blocks (not editing existing shapekeys, but being able to add new ones, and change influence of existing ones). on hold, we probably want to get rid of shapekeys as data-blocks first.

Milestone 3 - Storage Optimization

  • Do not store whole Mesh data-block in overrides, this is useless and uses a lot of disk space (other obdata IDs are much less critical on that regard, being much lighter, #78944 (LibOverride: How to handle heavy data-blocks (meshes, shape-keys, actions...))).
  • ShapeKeys may need investigation here as well.

Milestone 5 - Remove Legacy

Not A Milestone - Address technical Dept

  • #82160 (LibOverrides - Refactor how diffing of RNA collections is handled, and extend diffing possibilities.). Refactor/technical dept is fixed, potential future improvements are for later.
  • #83811 (Library Override: Investigate Resync Improvements.).

Relevant links:

**Status:** Achieved 2021 targets have been 'archived' at the end of this task, which is now updated for [2022 targets](https://code.blender.org/2022/02/overrides-workshop/). --- **Team** **Commissioner:** @eyecandy @Hjalti @Rikstopher **Project leader:** @mont29 **Project members:** @JulianEisel @Jeroen-Bakker ### 2022 Targets **Milestone 1 - Resync Performances** - [x] #94059 (Library Override Resync Performance). - [x] #95283 (LibOverride Stronger Hierarchy) - [x] #94185 (Outliner ID remapping performance), #95279 (Remap multiple items in referenced data) - [x] #95682 (Library Override partial Resync) **Milestone 2 - Improve Usability** - [ ] #95708 (Library Override - Usability). - [ ] #95707 (Library Override - System override IDs) - [ ] #95802 (Library Override - Outliner UI/UX) - [ ] #95816 (Library Override - 'Cherry-Picked' Properties) **Milestone 3 - Dynamic Overrides** - [ ] #54792 (Dynamic Overrides - View layer overrides with override sets) *(old, to be revised/updated and fleshed out)* **Not A Milestone - Library Overrides for Material** - [ ] Solve issues related to the weird way materials are linked to object or obdata. - [ ] Support properly nodegroups (i.e. embedded data-blocks, some preliminary work has already been done but needs to be finalized and validated). #82404 (Library Overrides: Shader drivers stop working after save and reload) **Not A Milestone - Future Ideas** *Those are ideas existing from the beginning of the project, and which have a partial implementation in core code, but need more UI/UX design and development.* - [ ] #72629 (Library Override: Support editing of animation data in overriden data-blocks.). - [ ] #82503 (Add Support of PointCaches to Library Overrides). - [ ] Allow addition of CustomData layers to geometry IDs (new UV maps, shape keys, vertex groups...). *This will require a lot of preliminary work.* - [ ] Advanced types of override operations (multiplicative factor for 'factor' porperties like scale, 'bit wise' operations on enum flags, ...). --- **Relevant links**: * [Blender Manual ](https://docs.blender.org/manual/en/latest/files/linked_libraries/library_overrides.html) * [the workshop report about overrides from January 2022 ](https://code.blender.org/2022/02/overrides-workshop/) ----------------------- ### 2021 Targets **Description** **Big picture:** Replace animation proxy system, and support local changes from linked data. **Use cases**: * Animators using proxies for animation. * Multiple instances of the same character with different animations. * Multiple instances of the same character with variations (e.g., material tweaks, ...). * Set dressing with local variations of instanced assets. * Override linked in character with cached animation [cache modifier] **Design:** - [x] `Asses feasibility of auto-override - direct change of linked data` - [x] `Need storage optimization document` #78944 (LibOverride: How to handle heavy data-blocks (meshes, shape-keys, actions...)) - [x] `Need final signed off design for outliner interaction` - [x] `Need final design for the ID template` #78012 (UI: Search Menu Design for Linked Data-blocks) **Engineer plan:** None - underlying code is already implemented. **Work plan** **Milestone 1 - Usability Pass** - [x] #76555 (Library Overrides: Usability issues/paper-cuts) - [x] #78012 (UI: Search Menu Design for Linked Data-blocks), #79959 (Data-block Selector Design Changes) - [x] [D7631: Library Overrides UI: Show library overrides in Outliner](https://archive.blender.org/developer/D7631) - [x] *#73154 (Library Overrides - UI: Add access to the 'auto override' flag of IDs) (Might become unnecessary, postponed until we know.)* - [x] Enhance ways to add more than one overrides of same data (this is currently possible, but not obvious nor user-friendly at all). - [x] #77083 (Library Overrides Constraint and Modifier Access) - [x] #79562 (LibOverride: Support edition of lists (colletions) of ID properties/py-defined RNA collections) **Milestone 2 - Proxy Replacement** - [x] #81049 (Add operator to convert proxies to library overrides ) - [x] #72629 (Library Override: Support editing of animation data in overriden data-blocks.) - [x] #82503 (Add Support of PointCaches to Library Overrides) - [x] ~~Implement real override of shapekeys data-blocks (not editing existing shapekeys, but being able to add new ones, and change influence of existing ones).~~ *on hold, we probably want to get rid of shapekeys as data-blocks first.* **Milestone 3 - Storage Optimization** - [x] Do not store whole Mesh data-block in overrides, this is useless and uses a lot of disk space (other obdata IDs are much less critical on that regard, being much lighter, #78944 (LibOverride: How to handle heavy data-blocks (meshes, shape-keys, actions...))). - [x] ShapeKeys may need investigation here as well. **Milestone 5 - Remove Legacy** - [x] #91671 (Remove Proxies). **Not A Milestone - Address technical Dept** - [x] #82160 (LibOverrides - Refactor how diffing of RNA collections is handled, and extend diffing possibilities.). *Refactor/technical dept is fixed, potential future improvements are for later.* - [x] #83811 (Library Override: Investigate Resync Improvements.). --- **Relevant links**: * #53500 (Library Override (parent task)) * [Blender Manual ](https://docs.blender.org/manual/en/latest/files/linked_libraries/library_overrides.html) * @eyecandy did a video to show shortcomes of the current implementation. [state_of_overrides_in_production.mp4](https://archive.blender.org/developer/F8624582/state_of_overrides_in_production.mp4)
Bastien Montagne was assigned by Dalai Felinto 2020-01-22 16:40:30 +01:00
Author
Owner
Added subscribers: @JulianEisel, @Hjalti, @mont29, @eyecandy, @dfelinto

#53500 was marked as duplicate of this issue

#53500 was marked as duplicate of this issue

Added subscriber: @AditiaA.Pratama

Added subscriber: @AditiaA.Pratama

Added subscriber: @Scaredyfish

Added subscriber: @Scaredyfish

Added subscriber: @Pipeliner

Added subscriber: @Pipeliner

Added subscriber: @Josephbburg

Added subscriber: @Josephbburg

Added subscriber: @DuarteRamos

Added subscriber: @DuarteRamos

Changed status from 'Needs Triage' to: 'Confirmed'

Changed status from 'Needs Triage' to: 'Confirmed'

This issue was referenced by 1743326889

This issue was referenced by 174332688936911d777ae39d540637edeed1561b

This issue was referenced by bd3ab27410

This issue was referenced by bd3ab27410efc1b3f41eaeda0e9a0ea92749297e

This issue was referenced by 417ebc3845

This issue was referenced by 417ebc3845fde909fb1cd96b31581bbfa285d869

Added subscribers: @DanielGrauer, @filibis, @Mantissa, @karlisstigis, @jesusmabas, @breakingspell, @eoyilmaz, @davidandrade, @LucianoMunoz, @slumber, @lemenicier_julien, @lrevardel, @anonym, @icappiello, @angavrilov, @Mets, @jiku, @MarcelLegindi, @satishgoda1, @reed2000, @carlosmu, @item412, @rollin, @SamGreen, @MauricioMarinho, @spockTheGray-4

Added subscribers: @DanielGrauer, @filibis, @Mantissa, @karlisstigis, @jesusmabas, @breakingspell, @eoyilmaz, @davidandrade, @LucianoMunoz, @slumber, @lemenicier_julien, @lrevardel, @anonym, @icappiello, @angavrilov, @Mets, @jiku, @MarcelLegindi, @satishgoda1, @reed2000, @carlosmu, @item412, @rollin, @SamGreen, @MauricioMarinho, @spockTheGray-4
Contributor

Added subscriber: @RedMser

Added subscriber: @RedMser

Added subscriber: @DragonicSculptor

Added subscriber: @DragonicSculptor

Removed subscriber: @carlosmu

Removed subscriber: @carlosmu

Added subscriber: @BintangPratama

Added subscriber: @BintangPratama

Added subscriber: @Wesley-Rossi

Added subscriber: @Wesley-Rossi

Added subscriber: @juang3d

Added subscriber: @juang3d

Added subscriber: @mcolinp

Added subscriber: @mcolinp

Added subscriber: @TakeshiFunahashi

Added subscriber: @TakeshiFunahashi

Added subscriber: @mpocztowski

Added subscriber: @mpocztowski

Added subscriber: @Alex-9

Added subscriber: @Alex-9

Added subscriber: @MattCurtis

Added subscriber: @MattCurtis
Member

Added subscriber: @zanqdo

Added subscriber: @zanqdo

Added subscriber: @MaciejMorgas

Added subscriber: @MaciejMorgas

This issue was referenced by aa64fd69e7

This issue was referenced by aa64fd69e733e49317101ea60299f2179830ae95

Added subscriber: @ErickNyanduKabongo

Added subscriber: @ErickNyanduKabongo

Added subscriber: @Tasch

Added subscriber: @Tasch
Member

Added subscriber: @Jeroen-Bakker

Added subscriber: @Jeroen-Bakker

Added subscriber: @stilobique

Added subscriber: @stilobique

Added subscriber: @sebbas

Added subscriber: @sebbas

Added subscriber: @danterazor

Added subscriber: @danterazor

Added subscriber: @Sparazza

Added subscriber: @Sparazza

Added subscriber: @RobertS

Added subscriber: @RobertS

Here's another movie demonstrating the beginner user experience:

annotated-experience.mp4

I feel that Blender developers usually don't care about beginners, so go ahead and ignore/delete this if you want to.

Here's another movie demonstrating the beginner user experience: [annotated-experience.mp4](https://archive.blender.org/developer/F10045980/annotated-experience.mp4) I feel that Blender developers usually don't care about beginners, so go ahead and ignore/delete this if you want to.

Added subscriber: @JohanTriHandoyo

Added subscriber: @JohanTriHandoyo

Added subscriber: @AndyCuccaro

Added subscriber: @AndyCuccaro

Added subscriber: @BEHE

Added subscriber: @BEHE

Added subscriber: @Garek

Added subscriber: @Garek

Added subscriber: @Wovchick

Added subscriber: @Wovchick

Removed subscriber: @item412

Removed subscriber: @item412

Removed subscriber: @mpocztowski

Removed subscriber: @mpocztowski

Added subscriber: @Rikstopher

Added subscriber: @Rikstopher

Added subscriber: @SteffenD

Added subscriber: @SteffenD

Removed subscriber: @davidandrade

Removed subscriber: @davidandrade

Added subscriber: @Emi_Martinez

Added subscriber: @Emi_Martinez

Added subscriber: @frameshift

Added subscriber: @frameshift

Removed subscriber: @MattCurtis

Removed subscriber: @MattCurtis

Added subscriber: @Max-49

Added subscriber: @Max-49

Added subscriber: @Asger

Added subscriber: @Asger

Added subscriber: @CHARLES-MURRAY

Added subscriber: @CHARLES-MURRAY

Added subscriber: @pauanyu_blender

Added subscriber: @pauanyu_blender
Member

Just wanted to slip in a comment here to summarize a discussion we had IRL, regarding this bullet point:

Allow addition of CustomData layers to geometry IDs (new UV maps, shape keys, vertex groups...). This will require a lot of preliminary work.

This would be huge for us, and allow us to phase out a lot of hacky pipeline tools and remove a lot of maintenance overhead. Our use cases (very much in a nutshell):

  1. Multiple artists working on an asset: Each artist needs control over a lot of overlapping data. For example, Vertex Groups on a character may need to be added for modeling, rigging, and shading, which could all be a series of .blend files linked in a chain.
  2. Animators want to tweak extreme deformations by adding a shape key on an overridden mesh and sculpting into it.

@mont29 Could you re-iterate here some of the major challenges that will need to be tackled in the coming years, before these ideas have any chance of becoming a reality?

Just wanted to slip in a comment here to summarize a discussion we had IRL, regarding this bullet point: > Allow addition of CustomData layers to geometry IDs (new UV maps, shape keys, vertex groups...). This will require a lot of preliminary work. This would be huge for us, and allow us to phase out a lot of hacky pipeline tools and remove a lot of maintenance overhead. Our use cases (very much in a nutshell): 1. Multiple artists working on an asset: Each artist needs control over a lot of overlapping data. For example, Vertex Groups on a character may need to be added for modeling, rigging, and shading, which could all be a series of .blend files linked in a chain. 2. Animators want to tweak extreme deformations by adding a shape key on an overridden mesh and sculpting into it. @mont29 Could you re-iterate here some of the major challenges that will need to be tackled in the coming years, before these ideas have any chance of becoming a reality?
Philipp Oeser removed the
Interest
Core
label 2023-02-09 14:43:15 +01:00
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
45 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#73318
No description provided.