Fix [T51595]: Snap to edge does not work with high zoom level

That problem occurs because of the imprecision of `short int` (16 bits).
The 3d coordinates are converted to 2d, and when they are off the screen, their values can exceed 32767! (max short int value)

One quick solution is to use float instead of short

The snap code is actually a little tricky. I want to make some arithmetic simplifications in it
This commit is contained in:
Germano Cavalcante 2017-07-11 17:03:49 -03:00
parent b3f62b68a9
commit 3f39719b5d
1 changed files with 5 additions and 8 deletions

View File

@ -1170,15 +1170,12 @@ static float dist_squared_to_projected_aabb(
vb2d[0] *= data->win_half[0];
vb2d[1] *= data->win_half[1];
//float dvec[2], edge[2], rdist;
//sub_v2_v2v2(dvec, data->mval, va2d);
//sub_v2_v2v2(edge, vb2d, va2d);
float rdist;
short dvec[2] = {data->mval[0] - va2d[0], data->mval[1] - va2d[1]};
short edge[2] = {vb2d[0] - va2d[0], vb2d[1] - va2d[1]};
float lambda = dvec[0] * edge[0] + dvec[1] * edge[1];
float dvec[2], edge[2], lambda, rdist;
sub_v2_v2v2(dvec, data->mval, va2d);
sub_v2_v2v2(edge, vb2d, va2d);
lambda = dot_v2v2(dvec, edge);
if (lambda != 0.0f) {
lambda /= edge[0] * edge[0] + edge[1] * edge[1];
lambda /= len_squared_v2(edge);
if (lambda <= 0.0f) {
rdist = len_squared_v2v2(data->mval, va2d);
r_axis_closest[main_axis] = true;