This tool generates masks based on the sculpt vertex colors by clicking
on the model, similar to automatic selection tools in image editing
I'm not quite sure where M_SQRT3 and why you divide by threshold rather than comparing to it. This isn't how threshold work.
Either its logical error in the code, or the code needs explanation and better naming choice.
Don't use else after return.
Same as above.
@Sergey Sharybin (sergey) I reworked the threshold function in a way that is more similar to image editing programs with lasso selections. To me it would make more sense to have a function that produces the larger falloff possible across the color instead of a fixed falloff defined by the slope while keeping the same behavior (threshold of 0 selects only the same color, 1 selects the entire image). I was trying something like this, which produces a much bigger falloff when the threshold is 0.5, but I'm not sure how it will work in practice. I can take a look at this again in a later patch.