Port operators to use COW (Parent Task)
Open, NormalPublic

Description

This is the parent task for the copy-on-write operator porting taskforce.

With copy-on-write in place we have clear separation between original data and evaluated state: dependency graph gets original data, goes through all the animation systems, drivers, constraints and modifiers stack and gets evaluated state of that data within the current context.
This means that now operators should read from an evaluated state and write to original data, and to inform dependency graph about changes.
Practical example: if an operator needs to get an object matrix, it should read it from the evaluated version. But if the operator needs to update the object’s location, it should write it to the original object and tag it as updated.

For further information read the dependency graph design.

How this works:

  1. Find an operator you want to port in the linked sub-tasks below.
  2. Submit patches for review using arcanist - tagging @Sergey Sharybin (sergey) and @Joshua Leung (aligorith) as reviewers.
  3. Once the patch is reviewed and committed, the operator in the task will be marked as "done".

Note: Not all operators will actually need fixing (many should be fine already, without needing further changes). In that case, instead of submitting a patch, mark the operator as done directly.

What to do:

The code work is straightforward. But there are things which will help migrating operators:

  • To go from original object to evaluated, use DEG_get_evaluated_object(). To go from original ID to evaluated use DEG_get_evaluated_id(). Keep in mind, these functions do hash lookup based on ID, so if multiple lookups of same object is needed try to store result of this lookup.
  • To go from evaluated to original object use DEG_get_original_object(). To go from evaluated to original ID use DEG_get_original_id(). These functions are cheap.
  • There is also a DEG_get_evaluated_rna_pointer() method that can be used in exceptional cases. Unlike the other functions, it is slow - other methods should be used where possible.
  • If some operator changes interface-only data (example: scene’s 3d cursor location) apart form notifier also use DEG_id_tag_update(id, DEG_TAG_COPY_ON_WRITE); (e.g., rB6a75a1a669d2).

Other examples:

  • rB40199c1d10a9 - Manipulator: Use evaluated object's matrix to get manipulator position.
  • rBeb521b22b2b1 - Make View Selected to be aware of copy-on-write.
  • rB3e26b84397fc - Camera manipulator: Make it aware of evaluated version of object.

Testing:

Run Blender normally.

  • If the operator works as well as it did in 2.7, it works!
  • If there are problems (e.g. things won't update, invalid results, etc.) the operator (still) needs some more fixes applied.

We also found the following tips useful:

  • Add the --debug-depsgraph-tag to check if the correct tagging updates are happening
  • Use the --disable-copy-on-write flag to check if something else in 2.8 (e.g. Message Bus system) may be causing refresh issues
  • Try checking what happens when using the operators in an animated setup. Some tools only break during animation
  • Sometimes other object data properties may be broken/requiring updates (e.g. armature settings, constraint influence, or shape keys).

Design Notes:

  1. Context (bContext) contains original non-evaluated data. This means CTX_data_scene will give original scene, not evaluated. This is also valid for context in Python.
  2. Main structure is always holding original datablocks.
  3. Naming is very important to keep things clear. Agreed convention is: for evaluated versions of data we add _eval suffix to variables.
  4. Where possible, mark evaluated data variables as const

Sub-tasks for each module's operators:

Stuff you can do now:

Major Changes Required (High Priority - for Blender Studio Migration to 2.8):

Major Changes Required (Lower Priority):

Likely Only Minor Changes Needed:

Postponed (Blocking/Pending Issues need resolving first)

Waiting for Sergey to test/check first
  • Curve/Surface (CURVE_OT_*)
  • Font (FONT_OT_*)
  • Metaballs (META_OT_*)
  • Lattices (LATTICE_OT_*)
Waiting for Grease Pencil branch merge

Grease Pencil module (editors/gpencil)

Should be fine without needing fixes
  • Console
  • View2D
  • Text Editor
May need fixes at some point (but aren't operators/for taskforce work)
  • editors/space_view3d/view3d_manipulator_*.c

Details

Type
To Do

Related Objects

Joshua Leung (aligorith) triaged this task as Normal priority.
Joshua Leung (aligorith) moved this task from Tasks to Doing on the Code Quest board.
Joshua Leung (aligorith) renamed this task from Port operators to use COW to Port operators to use COW (Parent Task).

I made this quick patch to show if Blender is running with or without COW enabled as I kept getting confused with multiple Blenders while testing.
It has an include for depsgraph, so double check your includes if editing that file.

P691

fixed the wiki links, @Sergey Sharybin (sergey) can you transfer dependency graph design to the new wiki? (had to place a en.blender.org link for now)

@nBurn (nBurn) transferred the page, link has been updated.