Fix T102923: replace zero check with epsilons with uv constrain to bounds

Small roundoff errors during UV editing can sometimes occur, most likely
due to so-called "catastrophic cancellation".

Here we set a tolerance around zero when using Constrain-To-Bounds and UV Scaling.

The tolerance is set at one quarter of a texel, on a 65536 x 65536 texture.

TODO: If this fix holds, we should formalize the tolerance into the UV editing
subsystem, perhaps as a helper function, and investigate where else it needs
to be applied.

Differential Revision: https://developer.blender.org/D16702

source/blender/editors/transform/transform_mode_resize.c

@@ -95,9 +95,14 @@ static void constrain_scale_to_boundary(const float numerator,
                                         const float denominator,
                                         float *scale)
 {
-  if (denominator == 0.0f) {
-    /* The origin of the scale is on the edge of the boundary. */
-    if (numerator < 0.0f) {
+  /* It's possible the numerator or denominator can be very close to zero due to so-called
+   * "catastrophic cancellation". See T102923 for an example. We use epsilon tests here to
+   * distinguish between genuine negative coordinates versus coordinates that should be rounded off
+   * to zero. */
+  const float epsilon = 0.25f / 65536.0f; /* i.e. Quarter of a texel on a 65536 x 65536 texture. */
+  if (fabsf(denominator) < epsilon) {
+    /* The origin of the scale is very near the edge of the boundary. */
+    if (numerator < -epsilon) {
       /* Negative scale will wrap around and put us outside the boundary. */
       *scale = 0.0f; /* Hold at the boundary instead. */
     }