Can't select vertices in a negative scale object, with ALT-B mask active and "limit selection to visible" disabled #42264

New Issue

Marcus Richter · 2014-10-16T22:58:20+02:00

Marcus Richter commented

2014-10-16 22:58:20 +02:00

System Information
Windows 8.1
CPU: FX 8350
GPU: GTX 660Ti
RAM: 24Gb DDR3

Blender Version
Broken: Any version
Worked: None

Short description of error
Users can't select vertices in an object that has been flipped on the x/y/z axis in object mode, if ALT-B mask is active and "limit selection to visible" is disabled.
It must be a bug since selecting without an ALT-B mask is possible.
The problem can be solved by applying the scale of the object and flipping the normals

Exact steps for others to reproduce the error
bug_report.blend
To reproduce, start a new project, add any object, scale to negative on any axis, ALT-B mask part of the viewport, disable "limit selection to visible" and you won't be able to select vertices in the object.

**System Information** Windows 8.1 CPU: FX 8350 GPU: GTX 660Ti RAM: 24Gb DDR3 **Blender Version** Broken: Any version Worked: None **Short description of error** Users can't select vertices in an object that has been flipped on the x/y/z axis in object mode, if ALT-B mask is active and "limit selection to visible" is disabled. It must be a bug since selecting without an ALT-B mask is possible. The problem can be solved by applying the scale of the object and flipping the normals **Exact steps for others to reproduce the error** [bug_report.blend](https://archive.blender.org/developer/F117845/bug_report.blend) To reproduce, start a new project, add any object, scale to negative on any axis, ALT-B mask part of the viewport, disable "limit selection to visible" and you won't be able to select vertices in the object.

Marcus Richter commented

2014-10-16 22:58:20 +02:00

Changed status to: 'Open'

Marcus Richter commented

2014-10-16 22:58:20 +02:00

Added subscriber: @mstuff

Marcus Richter commented

2014-10-16 23:35:18 +02:00

Feel free to add this to low priority, it is not important.
Also I accidentally added a wireframe modifier to the object in the .blend, just ignore it.

Feel free to add this to low priority, it is not important. Also I accidentally added a wireframe modifier to the object in the .blend, just ignore it.

Bastien Montagne commented

2014-10-18 14:47:42 +02:00

Added subscriber: @mont29

Bastien Montagne commented

2014-10-18 23:03:18 +02:00

Added subscriber: @ideasman42

Bastien Montagne self-assigned this 2014-10-18 23:03:27 +02:00

Bastien Montagne commented

2014-10-18 23:09:41 +02:00

Wow… this one was a nice hidden one, took me some time to figure it out.

Dev note: Issue comes from the fact that we transform a bbox with the object matrix… which can lead to bad-ordered bbox, yielding invalid clipping planes (ED_view3d_clipping_local, in view3d_edit.c). Here is a proposed patch, which merely ensure all eight bbox vertices are in the right order (as defined in DNA_object_types.h).

P156: #42264

diff --git a/source/blender/editors/space_view3d/view3d_edit.c b/source/blender/editors/space_view3d/view3d_edit.c
index 6c0b9f3..aba6e24 100644
--- a/source/blender/editors/space_view3d/view3d_edit.c
+++ b/source/blender/editors/space_view3d/view3d_edit.c
@@ -4390,6 +4390,41 @@ static void calc_clipping_plane(float clip- [x]- [x], const BoundBox *clipbb)
 	}
 }
 
+/* BoundBox is expected to be sorted in a specific order (see its declaration), some operations may break this. */
+static void sort_boundbox(BoundBox *bb)
+{
+	int i, j, k;
+
+	/* X-sorting. */
+	for (i = 0; i < 4; i++) {
+		if (bb->vec- [x]- [x] > bb->vec[i + 4][0]) {
+			swap_v3_v3(bb->vec- [x], bb->vec[i + 4]);
+		}
+	}
+
+	/* Y-sorting. */
+	for (i = 0, j = 3; i < 2; i++, j -= 2) {
+		k = i + 4;
+		if (bb->vec- [x]- [x] > bb->vec[i + j][1]) {
+			swap_v3_v3(bb->vec- [x], bb->vec[i + j]);
+		}
+		if (bb->vec- [x]- [x] > bb->vec[k + j][1]) {
+			swap_v3_v3(bb->vec- [x], bb->vec[k + j]);
+		}
+	}
+
+	/* Z-sorting. */
+	for (i = 0, j = 1; i < 4; i += 3, j -= 2) {
+		k = i + 4;
+		if (bb->vec- [x]- [x] > bb->vec[i + j][2]) {
+			swap_v3_v3(bb->vec- [x], bb->vec[i + j]);
+		}
+		if (bb->vec- [x]- [x] > bb->vec[k + j][2]) {
+			swap_v3_v3(bb->vec- [x], bb->vec[k + j]);
+		}
+	}
+}
+
 static void calc_local_clipping(float clip_local- [x]- [x], BoundBox *clipbb, float mat- [x][4])
 {
 	BoundBox clipbb_local;
@@ -4402,6 +4437,9 @@ static void calc_local_clipping(float clip_local- [x]- [x], BoundBox *clipbb, float
 		mul_v3_m4v3(clipbb_local.vec- [x], imat, clipbb->vec[i]);
 	}
 
+	/* Above obmat multiplication may have broken 'good' sorting of our local bbox! */
+	sort_boundbox(&clipbb_local);
+
 	calc_clipping_plane(clip_local, &clipbb_local);
 }

However, I have a feeling such bbox manipulations should be done in BLI (math_geom perhaps? or even its own file?), that kind of issue may very well exist in other places too… That’d be a bigger task though, Campbell, what do you think?

Wow… this one was a nice hidden one, took me some time to figure it out. Dev note: Issue comes from the fact that we transform a bbox with the object matrix… which can lead to bad-ordered bbox, yielding invalid clipping planes (`ED_view3d_clipping_local`, in `view3d_edit.c`). Here is a proposed patch, which merely ensure all eight bbox vertices are in the right order (as defined in `DNA_object_types.h`). [P156: #42264](https://archive.blender.org/developer/P156.txt) ``` diff --git a/source/blender/editors/space_view3d/view3d_edit.c b/source/blender/editors/space_view3d/view3d_edit.c index 6c0b9f3..aba6e24 100644 --- a/source/blender/editors/space_view3d/view3d_edit.c +++ b/source/blender/editors/space_view3d/view3d_edit.c @@ -4390,6 +4390,41 @@ static void calc_clipping_plane(float clip- [x]- [x], const BoundBox *clipbb) } } +/* BoundBox is expected to be sorted in a specific order (see its declaration), some operations may break this. */ +static void sort_boundbox(BoundBox *bb) +{ + int i, j, k; + + /* X-sorting. */ + for (i = 0; i < 4; i++) { + if (bb->vec- [x]- [x] > bb->vec[i + 4][0]) { + swap_v3_v3(bb->vec- [x], bb->vec[i + 4]); + } + } + + /* Y-sorting. */ + for (i = 0, j = 3; i < 2; i++, j -= 2) { + k = i + 4; + if (bb->vec- [x]- [x] > bb->vec[i + j][1]) { + swap_v3_v3(bb->vec- [x], bb->vec[i + j]); + } + if (bb->vec- [x]- [x] > bb->vec[k + j][1]) { + swap_v3_v3(bb->vec- [x], bb->vec[k + j]); + } + } + + /* Z-sorting. */ + for (i = 0, j = 1; i < 4; i += 3, j -= 2) { + k = i + 4; + if (bb->vec- [x]- [x] > bb->vec[i + j][2]) { + swap_v3_v3(bb->vec- [x], bb->vec[i + j]); + } + if (bb->vec- [x]- [x] > bb->vec[k + j][2]) { + swap_v3_v3(bb->vec- [x], bb->vec[k + j]); + } + } +} + static void calc_local_clipping(float clip_local- [x]- [x], BoundBox *clipbb, float mat- [x][4]) { BoundBox clipbb_local; @@ -4402,6 +4437,9 @@ static void calc_local_clipping(float clip_local- [x]- [x], BoundBox *clipbb, float mul_v3_m4v3(clipbb_local.vec- [x], imat, clipbb->vec[i]); } + /* Above obmat multiplication may have broken 'good' sorting of our local bbox! */ + sort_boundbox(&clipbb_local); + calc_clipping_plane(clip_local, &clipbb_local); } ``` However, I have a feeling such bbox manipulations should be done in BLI (math_geom perhaps? or even its own file?), that kind of issue may very well exist in other places too… That’d be a bigger task though, Campbell, what do you think?

blender-admin commented

2014-10-20 11:17:36 +02:00

This issue was referenced by 3138d39047

This issue was referenced by 3138d390478524bd6c9d25454933ab8cf5ecb349

Campbell Barton commented

2014-10-20 11:21:47 +02:00

Changed status from 'Open' to: 'Resolved'

Campbell Barton closed this issue

2014-10-20 11:21:47 +02:00

Campbell Barton commented

2014-10-20 11:21:47 +02:00

Closed by commit 3138d39047.