Page MenuHome

Port operators to use COW (Parent Task)
Closed, InvalidPublicTO DO

Description

This is the parent task for the copy-on-write operator porting taskforce.

With copy-on-write in place we have clear separation between original data and evaluated state: dependency graph gets original data, goes through all the animation systems, drivers, constraints and modifiers stack and gets evaluated state of that data within the current context.
This means that now operators should read from an evaluated state and write to original data, and to inform dependency graph about changes.
Practical example: if an operator needs to get an object matrix, it should read it from the evaluated version. But if the operator needs to update the object’s location, it should write it to the original object and tag it as updated.

For further information read the dependency graph design.

How this works:

  1. Find an operator you want to port in the linked sub-tasks below.
  2. Submit patches for review using arcanist - tagging @Sergey Sharybin (sergey) and @Joshua Leung (aligorith) as reviewers.
  3. Once the patch is reviewed and committed, the operator in the task will be marked as "done".

Note: Not all operators will actually need fixing (many should be fine already, without needing further changes). In that case, instead of submitting a patch, mark the operator as done directly.

What to do:

The code work is straightforward. But there are things which will help migrating operators:

  • To go from original object to evaluated, use DEG_get_evaluated_object(). To go from original ID to evaluated use DEG_get_evaluated_id(). Keep in mind, these functions do hash lookup based on ID, so if multiple lookups of same object is needed try to store result of this lookup.
  • To go from evaluated to original object use DEG_get_original_object(). To go from evaluated to original ID use DEG_get_original_id(). These functions are cheap.
  • There is also a DEG_get_evaluated_rna_pointer() method that can be used in exceptional cases. Unlike the other functions, it is slow - other methods should be used where possible.
  • If some operator changes interface-only data (example: scene’s 3d cursor location) apart form notifier also use DEG_id_tag_update(id, DEG_TAG_COPY_ON_WRITE); (e.g., rB6a75a1a669d2).

Other examples:

  • rB40199c1d10a9 - Manipulator: Use evaluated object's matrix to get manipulator position.
  • rBeb521b22b2b1 - Make View Selected to be aware of copy-on-write.
  • rB3e26b84397fc - Camera manipulator: Make it aware of evaluated version of object.

Testing:

Run Blender normally.

  • If the operator works as well as it did in 2.7, it works!
  • If there are problems (e.g. things won't update, invalid results, etc.) the operator (still) needs some more fixes applied.

We also found the following tips useful:

  • Add the --debug-depsgraph-tag to check if the correct tagging updates are happening
  • Use the --disable-copy-on-write flag to check if something else in 2.8 (e.g. Message Bus system) may be causing refresh issues
  • Try checking what happens when using the operators in an animated setup. Some tools only break during animation
  • Sometimes other object data properties may be broken/requiring updates (e.g. armature settings, constraint influence, or shape keys).

Design Notes:

  1. Context (bContext) contains original non-evaluated data. This means CTX_data_scene will give original scene, not evaluated. This is also valid for context in Python.
  2. Main structure is always holding original datablocks.
  3. Naming is very important to keep things clear. Agreed convention is: for evaluated versions of data we add _eval suffix to variables.
  4. Where possible, mark evaluated data variables as const

Sub-tasks for each module's operators:

Stuff you can do now:

Major Changes Required (High Priority - for Blender Studio Migration to 2.8):

Major Changes Required (Lower Priority):

Likely Only Minor Changes Needed:

Postponed (Blocking/Pending Issues need resolving first)

Waiting for Sergey to test/check first
  • Curve/Surface (CURVE_OT_*)
  • Font (FONT_OT_*)
  • Metaballs (META_OT_*)
  • Lattices (LATTICE_OT_*)
Waiting for Grease Pencil branch merge

Grease Pencil module (editors/gpencil)

Should be fine without needing fixes
  • Console
  • View2D
  • Text Editor
May need fixes at some point (but aren't operators/for taskforce work)
  • editors/space_view3d/view3d_manipulator_*.c

Related Objects

StatusSubtypeAssignedTask
InvalidTO DOJoshua Leung (aligorith)
ResolvedTO DOSergey Sharybin (sergey)
ResolvedTO DOJoshua Leung (aligorith)
InvalidTO DOBrecht Van Lommel (brecht)
InvalidTO DOBrecht Van Lommel (brecht)
InvalidTO DOBrecht Van Lommel (brecht)
InvalidTO DOBrecht Van Lommel (brecht)
ResolvedBUGDalai Felinto (dfelinto)
InvalidTO DOBrecht Van Lommel (brecht)
InvalidTO DOBrecht Van Lommel (brecht)
ResolvedTO DOBastien Montagne (mont29)
InvalidTO DOBrecht Van Lommel (brecht)
InvalidTO DOBrecht Van Lommel (brecht)
InvalidTO DOBrecht Van Lommel (brecht)
InvalidTO DOBrecht Van Lommel (brecht)
InvalidTO DOBrecht Van Lommel (brecht)
InvalidTO DOBrecht Van Lommel (brecht)
InvalidTO DOBrecht Van Lommel (brecht)
InvalidTO DOBrecht Van Lommel (brecht)
InvalidTO DOBrecht Van Lommel (brecht)
InvalidTO DOBrecht Van Lommel (brecht)
InvalidTO DOBrecht Van Lommel (brecht)
InvalidTO DOSergey Sharybin (sergey)
ResolvedTO DOAntonio Vazquez (antoniov)

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Dalai Felinto (dfelinto) changed the status of subtask T54811: COW Operators: Edit Armature from Unknown Status to Resolved.May 14 2018, 10:21 AM

I made this quick patch to show if Blender is running with or without COW enabled as I kept getting confused with multiple Blenders while testing.
It has an include for depsgraph, so double check your includes if editing that file.

P691

Dalai Felinto (dfelinto) changed the status of subtask T54812: COW Operators: Pose Armature from Unknown Status to Resolved.May 28 2018, 6:30 PM
Bastien Montagne (mont29) changed the status of subtask T54820: COW Operators: Paint from Unknown Status to Resolved.Jun 19 2018, 4:16 PM

fixed the wiki links, @Sergey Sharybin (sergey) can you transfer dependency graph design to the new wiki? (had to place a en.blender.org link for now)

@nBurn (nBurn) transferred the page, link has been updated.

Brecht Van Lommel (brecht) changed the status of subtask T54833: COW Operators: Other Geometry Types from Unknown Status to Unknown Status.Nov 19 2018, 12:40 PM
Brecht Van Lommel (brecht) changed the status of subtask T54830: COW Operators: UV Edit from Unknown Status to Unknown Status.
Brecht Van Lommel (brecht) changed the status of subtask T54828: COW Operators: Sequencer from Unknown Status to Unknown Status.
Brecht Van Lommel (brecht) changed the status of subtask T54827: COW Operators: Outliner from Unknown Status to Unknown Status.
Brecht Van Lommel (brecht) changed the status of subtask T54825: COW Operators: Node Editor from Unknown Status to Unknown Status.
Brecht Van Lommel (brecht) changed the status of subtask T54824: COW Operators: Image Editor from Unknown Status to Unknown Status.
Brecht Van Lommel (brecht) changed the status of subtask T54823: COW Operators: Movie Clip Editor from Unknown Status to Unknown Status.
Brecht Van Lommel (brecht) changed the status of subtask T54822: COW Operators: Sound from Unknown Status to Unknown Status.
Brecht Van Lommel (brecht) changed the status of subtask T54819: COW Operators: Render from Unknown Status to Unknown Status.
Brecht Van Lommel (brecht) changed the status of subtask T54831: COW Operators: Transform from Unknown Status to Unknown Status.
Brecht Van Lommel (brecht) changed the status of subtask T54829: COW Operators: 3D View from Unknown Status to Unknown Status.Nov 19 2018, 12:42 PM
Brecht Van Lommel (brecht) changed the status of subtask T54832: COW Operators: User Interface Stuff from Unknown Status to Unknown Status.
Brecht Van Lommel (brecht) changed the status of subtask T54821: COW Operators: Sculpt from Unknown Status to Unknown Status.
Brecht Van Lommel (brecht) changed the status of subtask T54818: COW Operators: Physics from Unknown Status to Unknown Status.
Brecht Van Lommel (brecht) changed the status of subtask T54817: COW Operators: Object from Unknown Status to Unknown Status.
Brecht Van Lommel (brecht) changed the status of subtask T54815: COW Operators: Mesh from Unknown Status to Unknown Status.
Brecht Van Lommel (brecht) changed the status of subtask T54814: COW Operators: Mask from Unknown Status to Unknown Status.
Brecht Van Lommel (brecht) changed the status of subtask T54813: COW Operators: Import/Export from Unknown Status to Unknown Status.
Brecht Van Lommel (brecht) changed the task status from Unknown Status to Unknown Status.Nov 19 2018, 12:45 PM

Most operators appear to be working, and we will use bug tracker to handle remaining issues.