This is interesting. I can indeed reproduce that behavior with your file. However, if you go to a different workspace and create a compositor there, the backdrop reacts to color management just fine. Your compositing workspace seems to be broken somehow, but I have no idea how. Maybe @Jeroen Bakker (jbakker) has an idea?

I cannot confirm that, it works here, both in 2.93 and in 2.92. Can you post a blendfile where it happens?

I have tried with the Fabrik movie and Tears of Steel from the DVD. I have tracked some markers and then scrubbed and clicked like crazy, also reloaded the footage, but couldn't make it go off sync. So it seems to be fine.

Without the footage and without any markers in the blendfile it is impossible to figure out why the file might be crashing. Could it be that your memory is full? How much RAM do you have? Does it also crash if you only track one single marker?

Oh, that is a misunderstanding indeed. I thought "drag selection" refers to move the selection, not "drag a selection box". :)
Yeah, that is a todo then :)

Uhm, no, this is supposed to work. And it also does work here. (b4147aeeab7d, Linux)
I create a mask, click on the white dot in the center and can move the mask with it.

You are right, this is different than before. But this is not really bug, but rather a change in behavior. Lock to Selection now also works in Masking, where it locks the view to the selected markers, but not with the completely broken behavior as in 2.91 and before, but rather in a way that actually works. Part of this change is that the view follows the active selection, but without centering the view. That way it is less jumpy and less distracting.

In T81662#1108879, @Robert Nederhorst (rnederhorst) wrote:

In T81662#1108669, @Robert Nederhorst (rnederhorst) wrote:

I have used multiple images for projection. Example : Frame 1 and Frame 200 have different angles from the tracked camera. Faces visible on frame 200 are different than on frame 1. So I want to use frame 200 image to paint from the frame 200 camera. Same with frame 1 camera/image. This allows me to blend images together on to the geometry.

Sure, I do that as well, but this is not the issue here. In terms of datablocks, technically it is still the same Background Image on frame 1 and 200, e.g. a movie or image sequence, just a different frame.

In T84769#1100664, @Sean Kennedy (hype) wrote:

If you leave the default cube in the scene, it does not move it to the foreground collection. If you delete the default cube, it will create a cube in that foreground collection.

Thanks for the hint, this should be fixed now. Instead of always creating a new collection, it will rename the default collection to "foreground", if there really is only 1 collection in your scene.

Should the Backdrop in the node view be automatically turned on when using the Nodes checkbox? I don't ever use the Backdrop anymore, but if most people do, I guess it would make sense to leave on.

I still use it. I didn't touch that setting, would leave it for now.

Checking both Nodes and Shadow Catcher with in both cycles and eevee, there weren't any nodes added for shadows. I haven't used Eevee enough to know if that's necessary, but for Cycles, should there be some nodes? Also, should Indirect be turned on for the "shadow catcher" collection?

No, the cycles setup uses the Shadow Catcher option. You don't need any nodes for that, it works via Alpha Channel. Not really fancy and not that much controls, but it's an easy setup.

I tested this for a bit and find it quite useful, though indeed rather confusing.
I would get rid of the Corner Rounding checkbox. Instead just use the slider. If it is zero, no rounding is happening, but if rounding is required, the user only has to increase the slider without having to click on a checkbox.
The Value input is indeed doing something, as I found out while trying the last file in the comment from @Michael P. (forest-house) from January 6th 2017. So I guess it's needed.
The Edge output I guess is mostly for debugging? But does it have to be colored? The keying node has an edge output too, but that one is just a value output. Maybe we could use that here as well?
More problematic are the three edge detection methods. I can only comment from a user perspective, not sure if the code is doing anything in the background that I am not aware of.
I tested with colored and with bw images.

1. Value

When edgedetection is set to Value, the threshold parameter doesn't seem to do anything. If it is anything else than zero, no AA seems to be happening whatsoever. So when using Value ED I suggest to drop the threshold slider. Unless I am missing something here.

2. Luma and Color seem to be very similar, and using the 2 sliders does indeed do *something*, but from a user perspective it is really not clear if there is a useful difference and what exactly the sliders do. Maybe the two an can be unified somehow?

I suppose most users will probably use it to smooth masks and ID passes, so I suggest to make Value be the default setting, and the user would only need 1 slider for corner rounding.

@Sergey Sharybin (sergey) I'd vote against it, since I would rather put the solve panel in the header.
Solving is something that sometimes is being done over and over again, with varying settings, so it would be cumbersome to have it in the sidebar, because you ALSO have to have track and tracking settings accessible all the time. That's why I would rather put it into the header, as shown in this image https://dev-files.blender.org/file/data/myed7vsbwffz3zipu42b/PHID-FILE-pr7w637qcto5oxwcnlgc/mce_tracking.png from here: https://developer.blender.org/D10198 It's a similar button placement as the "run script" operator in the text editor.

we shall!

In D10265#257919, @Sergey Sharybin (sergey) wrote:

I tried to find a solution to the handles_as_control_point thing, but it doesn't really work. I mean, behavior itself with lock to selection seems fine, but now I cannot pan when nothing is selected. I guess this is something for a real developer! :)

• New Tool: add_marker_and_tweak. same as add_marker_slide, but that one is now only adding markers on click
• Merge branch 'master' into arcpatch-D10198

• Implement some masking tools, polish tracking UI
• Merge branch 'master' into arcpatch-D10198

• The extras popover panel was still missing

• Add marker tool works now! (kind of)
• Add properties for add_marker_slide tool
• Merge branch 'master' into arcpatch-D10198

@Sergey Sharybin (sergey) Hey man! I am making a bit of progress in the toolification of the clip editor. But I am stuck at getting the add_marker tool to actually do something. The tool shows up, but when I click nothing happens. Could you give me a hint on how to get this to work?

• Merge branch 'master' into arcpatch-D10198
• more attempts to make add marker tool work, no success
• Merge branch 'master' into arcpatch-D10198

• Add data source options for MCE annotation tools
• Also tried to implement the add_marker tool, but to no surprise this does not work yet.

• Merge branch 'master' into arcpatch-D10198
• Fix missing tool options in header.

ED_KEYMAP_TOOL was missing, and keymap name was wrong.
Now annotation and selection tools are working.
The tool settings are still missing though.

• Add clip tools to keymap
• The tools still do not work yet, but it seemed as if those steps would be necessary.
• Also, I never used arc diff before, so apologies if this messes up somehow.
• Merge branch 'master' into arcpatch-D10198

I don't think this is a good change. These things are separate because they are separate settings that do separate things.
The tracking settings operate during tracking. Only when you actually use the tool (as in, track forward or backwards), they have an effect.
The Track settings are properties of the track that affect solving and/or display. Let's keep it that way.
Unless it might be an idea to have the tool settings (former default tracking settings) also act as tracking settings, so to unify those. But I think that might violate the whole active tool design in Blender...

In D2771#255560, @Sean Kennedy (hype) wrote:

So for it to be "Sean-interpolation", you would leave "o" turned off and simply set keyframes around the area you want to adjust, then adjust and let the falloff to nearest keyframes happen linearly.

No, both are some form of proportional editing. So without turning on "o", moving a marker is just moving it on that frame.
When you turn on "o", it will adjust the track between keyframes (if there are any). Once you start scrolling, you will expand/shrink the falloff from current frame to surrounding frames beyond/below next available keyframe.

In D10198#255460, @Aaron Carlisle (Blendify) wrote:

Solving that error is going to be the key to getting this working, without that these functions aren't getting called:

WM_toolsystem_refresh_screen_area
WM_toolsystem_update_from_context

I might have some time this evening to try to debug farther

@Aaron Carlisle (Blendify) Well I am not sure about that. I tried that before, and it only resulted in a bunch of errors. To make sure I tried again with the current version of the patch and the error still occurs: https://pasteall.org/O8Nz
It is the same error I got when I tried D10011.

I tried this revision and always got errors and no UI at all. In order to try to understand what's happening I started all over myself, which probably means I did the same things you already tried. But anyway, I got to a point where at least the UI shows up and the tool header can be populated. I am now stuck at the point where I try to make tools actually work.
https://developer.blender.org/D10198

@Sergey Sharybin (sergey) I managed to get the tool header to work in MCE! Now I will see if I manage to get a tool to work as well. I kind of doubt it though... :D

@Sean Kennedy (hype) The setup of Eevee shadow catcher should work now as well. So depending on which render engine you want to use, a different kind of shadow catcher will be setup.

In D2771#254910, @Sean Kennedy (hype) wrote:

So as long as you don't touch the scroll wheel, proportional editing never comes into play.

Well, it is still proportional editing, kind of. So you would still have to enable proportional editing with "o" or the button.
But yeah, as long as you do not scroll (or probably also Ctrl+plus/minus), it would default to the "Sean-Interpolation", and as soon as you start to scroll, the falloff are shrinks/grows.

@Sergey Sharybin (sergey) it is purrrrfect now! :)

Hm, there are still a couple of problems.

1. When tweaking the curve directly the view jumps, probably because that triggers the selection of the CVs of that curve segment. And that jumping leads to bad deformations of the curve then.
2. slide_marker (also sliding the whole mask with center point) now always moves the view. Only when Lock to Selection is enabled view should move.
3. When Lock to Selection is disabled, view jumps to center when starting slide_marker. When LtS is on it works fine though.

Handle tips are fine now, that's great.

In D2771#254695, @Sean Kennedy (hype) wrote:

I have no idea how it would be possible to implement both, or be able to choose which method you use

Hey @Sean Kennedy (hype), the check for undistortion is working now. I replaced need_stabilization with need_undistortion, the check for stabilization never worked anyway, it was always set to False.
Setup Camera is not the same as Setup Tracking Scene without nodes.
Setup Tracking Scene also makes the Film transparent, creates a foreground collection with a Cube and can setup a shadow catcher, if needed.

Okay, seems you did download the HTML file from github, not the raw file.
Try this one: https://raw.githubusercontent.com/blendfx/blender/master/setup_tracking_scene_new.py

In D10146#254230, @Sergey Sharybin (sergey) wrote:

Update against latest master.

Brings fix for playback.

Awesome! will test later !

Here is a first draft of it:
https://github.com/blendfx/blender/blob/master/setup_tracking_scene_new.py
I couldn't get it to work with operators and UI separated in clip.py and space_clip.py like it is supposed to be, so arranged them in one file in the style of an addon.
@Sean Kennedy (hype) if you want to try it out, open that file in the text editor and run it.
It also has a new operator, which could be useful for object tracking. It will create an Empty in the 3d Viewport at the position of the selected track, if it is part of a solved object track. The empty will have the objcet solver contraint already setup.
The setup for the Eevee shadow catcher still has be configured, I will do that in the next couple of days.
Also, the UI will probably have to be adjusted once the new tool system is in place.

In D10146#254100, @Sergey Sharybin (sergey) wrote:

Sure! :)

In D10146#253941, @Sergey Sharybin (sergey) wrote:

@Sebastian Koenig (sebastian_k), actually, can you test something? Because seems that your setup is broken ever since 2.80. Can you verify this (lock-to-view for mask editor in MCE does not work in 2.80-2.91)?

Press play in this file. Behaves the same for me as if Locked was off. Only difference is noticable when I transform the curve points, but not during playback.

In D10146#253886, @Sergey Sharybin (sergey) wrote:

Locking will no longer lock zoom.

Wait, but now it also won't follow the selection anymore during playback. Now it only pans the clip during transform, but not during playback.

In D10146#253868, @Sergey Sharybin (sergey) wrote:

So how about we will disallow locking code to change zoom (which will allow you to have any zoom level manually)?

Let's try! :)

Wow, that was fast! :)
It is already much better, but there are still a couple of problems:

1. When nothing is selected, panning an zooming doesn't work at all, not even in Track mode.
2. Zoom is limited. I can only zoom in so far, after a while it feels like hitting an invisible wall, and instead of zooming it starts to pan. The more points there are, the less I can zoom in.
3. Even though slide_marker works fine, also on the little vector handles, translate or tweak with right mouse does not. When selecting a handle point of a vector handle and hitting "G", the view follows that. That behavior seems quite logic programmatically I guess, but from a users point of view it doesn't make sense, since we want to focus on the curvature of the mask, and not on the thing that controls it.
4. When a handle point was selected and you switch the handle type e.g. from Aligned to Vector, the view jumps to the lower left corner of the clip, probably because there is no handle point to focus on anymore.

I guess problems 3 and 4 (and maybe even 2?) can be avoided when only curve point can be locked onto, not the handle points.

In T84850#1096264, @Sergey Sharybin (sergey) wrote:

Something's weird though. Every thing works quite nice. Except it seems not to average the anchor point but the center of the pattern area. Sorry that I only stumbled across this after it already was committed, but want to mention it anyway.
Example:
You track a marker. For some reason you offset the pattern area from the marker's anchor point with "GG"and continue to track for a few frames.
Then the feature is completely blocked for a few frames. You continue tracking after the gap by positioning the anchor point on the feature and repositioning the pattern area directly on the anchor point again.
When you then execute Average Tracks the resulting marker will use not the anchor point but the pattern area for averaging, resulting in a wrong track.
Same with averaging two or more markers. It averages not the anchor points but the pattern area, see attached image.
Is that intended, @Sergey Sharybin (sergey)? And if so, is that what we want, @Sean Kennedy (hype)?

In T83612#1095302, @Sergey Sharybin (sergey) wrote:

In T83612#1094264, @Aaron Carlisle (Blendify) wrote:

One thing that would have to be discussed with the UI team about your proposal is currently, tool settings only affect the current tool and not operators, so this would have to be discussed.

Ok, I am open for discussion. I would like to avoid though to have an extra Tracking Settings panel just for that reason in the sidebar. Personally I would not see a problem with having the settings from Add Marker tool be equivalent to the default tracking settings.

I like what you did with the solve pop-over but I think it belongs in the regular header and not the tool header because it is not tool-specific.

Ok, fair enough. I wanted to bring up the Dyntopo toggle button as an example against it, but yeah, that kind of affects the active tool as well, sooo, you are probably right :)

I have made a really crappy mockup of how I think in which direction the new MCE interface could go.

As long as we don't actually have a proper tool system, we should stick to the current solution. Sure, it deviates from the rest of Blender, but simply putting everything in the sidebar is no solution either.
Yes, the Clip Editor UI needs to be updated to be consistent with the reset of Blender, but then in a proper way with an actual tool system. That might also then benefit the Mask Editor.

In D9781#252017, @Aaron Carlisle (Blendify) wrote:

@Sebastian Koenig (sebastian_k) does that mean you approve of the changes?

Yeah, I guess that is reasonable. Thanks!

In D2771#250763, @Sean Kennedy (hype) wrote:

tracking_proportional_edit_fail_01.zip102 KBDownload

Here's a file that I created in that proportional editing branch from https://builder.blender.org/download/branches/
No footage, just tracked a single point and saved the file.

I do not understand why shader nodes were changed from green to red. Looking at the mockup of @Pablo Vazquez (pablovazquez) here https://ui.pablovazquez.art/ the colors of geometry nodes and shaders could just be swapped. I find this change quite distracting, since shader node input have been green all the time. Just think of documentation. Why not make the new node type use a new colortheme, instead of changing the existing nodes to a new theme?
Or is it to make the node sockets reflect the icons in the properties editor? Not sure if that is a strong enough reason...

Look closer: The one window shows the UV Editor, the other the Image Editor. The 2D cursor is still hidden in the Image Editor in 2.92.

I'm am sorry for being so late with commenting on this.
I think it's good having that enum in that node!
I am a bit unsure of this terminology. "Replace Alpha" makes sense to me. "Apply Mask" however seems weird to me. Why not simply call it "Multiply Alpha"? Isn't that what the node is doing anyway? "Apply mask" implies that a mask is used, which in compositing is quite common indeed, and we also do have actual masks in the compositor. But more often the Set Alpha node is not used to only apply the mask from a mask node to the image, but the result of some operations with the Alpha channel of a render or an image, sometimes involving masks, sometimes not.
Also, there seem to be just 2 options in the node, which are opposites of eachother, so if the terms are not made up of the same logic and use different words for the same thing ("Alpha", "Mask"), I think it is confusing toi the user.
That's why I would suggest to use "Multiply Alpha" instead. The tooltip then makes it clear what's happening.

In D2771#249229, @Sergey Sharybin (sergey) wrote:

Is not impossible,
but will likely require some percussive maintenance (with a big
hammer) over the transform system API. Is this something you find
essential to have, or nice to have in the observable future?

Wow, this is really cool. I tried it, and having it work like proportional editing makes it really intuitive.
And if all the planned ToDos are implemented it's going to be awesome.
Would be interesting to have this in graph view as well, there it could be even easier to see what range the proportional editing extends.
Or how about having a gradient as overlay in the cache line? That way you could see the frames and the amount of influence in the clip view as well.

@Sergey Sharybin (sergey) This is really cool, and it seems to work quite well.
There is one thing I'd suggest to change though: When averaging a single track to fill in gaps, a new marker with the filled in gaps will be created. The old marker is still there, so to the user it looks like actually a disabled marker is being created in the background. Sure, when you delete the new marker you see that the old marker is still there with all tracked frames, not just the disabled ones.
I would suggest that if you average a single track to fill in gaps, the original marker will be deleted.
When averaging multiple tracks into a new average one it makes sense to keep the orignal ones. But when averaging only a single marker, delete the old one.

In practise I think it is rather rare to have multiple background images in the context I described. At least I never did. And even if I would load a second background image, maybe as a reference or so, I'd probably only show one.
But either way, I am not sure if we should actually use the Background Image as paint input. As a way preview the paint input I don't see a practical problem with Background Images, even if more than one are present.
About the technical implementation, I have no idea, that would something more for @Sergey Sharybin (sergey) .
Maybe @Sean Kennedy (hype) has an idea as well?

First of all, thanks for very detailed and thourough description.
I am also very happy that you want to improve the projection painting workflow. It is very much needed!