Grease pencil + offscreen drawing error: draw is only correct after stroke is confirmed #46579

New Issue

Dalai Felinto · 2015-10-23T18:22:45+02:00

Dalai Felinto commented

2015-10-23 18:22:45 +02:00

Blender Version
Broken: master (after 2.66, 23848a70 ), or GPencil_Editing_Stage3 branch.
Worked: never

Short description of error
If I'm using offscreen viewport drawing, the current drawing lines are drawn in the wrong place (in the offscreen view3d) until the stroke is confirmed.

Exact steps for others to reproduce the error

Run this operator: www.blender.org/api/blender_python_api_2_76_1/gpu.offscreen.html
(you should then see a small viewport on top of the view3d).
Start drawing in the main scene (in my case I'm trying to draw to a surface, but all the modes should be wrong)

You should be able to see the error right away, the drawings don't match between the original viewport and the one drawn.

**Blender Version** Broken: master (after 2.66, 23848a70 ), or `GPencil_Editing_Stage3` branch. Worked: never **Short description of error** If I'm using offscreen viewport drawing, the current drawing lines are drawn in the wrong place (in the offscreen view3d) until the stroke is confirmed. **Exact steps for others to reproduce the error** * Run this operator: www.blender.org/api/blender_python_api_2_76_1/gpu.offscreen.html (you should then see a small viewport on top of the view3d). * Start drawing in the main scene (in my case I'm trying to draw to a surface, but all the modes should be wrong) You should be able to see the error right away, the drawings don't match between the original viewport and the one drawn.

Dalai Felinto commented

2015-10-23 18:22:45 +02:00

Changed status to: 'Open'

Dalai Felinto commented

2015-10-23 18:22:45 +02:00

Added subscriber: @dfelinto

Joshua Leung was assigned by Dalai Felinto

2015-10-24 13:03:38 +02:00

Dalai Felinto commented

2015-10-24 23:00:54 +02:00

I thought that the bug was present only in the offscreen buffer, but in fact, it happens to the main viewport as well.

Dalai Felinto commented

2015-10-24 23:52:17 +02:00

I have a simple fix for the main viewport (P275) but the problem is still there for the offscreen buffer

I have a simple fix for the main viewport ([P275](https://archive.blender.org/developer/P275.txt)) but the problem is still there for the offscreen buffer

Dalai Felinto commented

2015-10-24 23:59:02 +02:00

Added subscriber: @ideasman42

Dalai Felinto commented

2015-10-24 23:59:02 +02:00

I found a way to reproduce the issue without offscreen drawing after Load Factory Settings:

Duplicate the viewport
Change one of the viewports (have one switch to camera view, or simply orbit, pan or zoom one of them).
Alt + A
Draw in one of the viewports

If this is fixed, I'm confident the offscreen buffer would get fixed as well.

I found a way to reproduce the issue without offscreen drawing after Load Factory Settings: * Duplicate the viewport * Change one of the viewports (have one switch to camera view, or simply orbit, pan or zoom one of them). * Alt + A * Draw in one of the viewports If this is fixed, I'm confident the offscreen buffer would get fixed as well.

Sergey Sharybin commented

2015-10-27 12:11:24 +01:00

Added subscriber: @Sergey

Sergey Sharybin commented

2015-10-27 12:11:41 +01:00

@JoshuaLeung, mind having a look here?

Dalai Felinto commented

2015-10-27 14:46:06 +01:00

I committed a related fix 3f85eed but the original issue is still there (same for the dupli-window way of reproducing this bug)

I committed a related fix 3f85eed but the original issue is still there (same for the dupli-window way of reproducing this bug)

Dalai Felinto commented

2015-11-20 21:32:14 +01:00

@JoshuaLeung any news?

Joshua Leung commented

2015-11-21 12:17:46 +01:00

Ahh... I think I finally starting to understand what's going on here!

What's happening with Grease Pencil:

Strokes are only converted to 3D coordinates after the user has finished drawing. This was done to keep the drawing tool nice and interactive, as it's not getting bogged down in all the conversion math after each and every point drawn. A really big goal for me when initially building Grease Pencil was to make sure that it was as lag-free as I could get it, as the lagginess seen in many other drawing apps at the time was something that really bugged me (and still does).
These "temporary" stroke points are held in a separate buffer (from the stroke data stored in layer->frame). They are not drawn as part of the data-space strokes, but rather they are drawn after the screenspace ones (see next point)
There are basically 3 drawing passes:

     === Data Space ===
     (scene objects)
     GP Strokes in "Cursor" space
     === Screen Space ===
     (overlays - names, text, axis indicator, etc.)
     GP Strokes in "Screen" aligned space
     Temporary GP Strokes Drawing Buffer

Previously the temporary strokes being drawn in screenspace would not be an issue if there were multiple viewports, as only the active viewport would get redrawn.

Now, here's where things start getting quite murky, as there seem to be multiple things happening at once!

Case 1) Main viewport != Camera View.
The offscreen buffer is being rendered from the perspective of the scene camera (or actually slightly back from the camera - from the actual location of the camera). However, the temporary screenspace strokes also get drawn here, so they will end up in the wrong places

Case 2) Main viewport = Camera View (but zoomed out)
When the viewport and buffer show the same view (i.e. the view from the camera), but they are not the same aspect ratio/zoom level, so some weirdness still seems to happen. I guess this is the main source of trouble I'm guessing.

I'm not sure of the specifics here yet though. Maybe it's a real problem, maybe it isn't.

Part of the problem in the offscreen buffer script example is that both of these cases end up interacting and combining together, leading to the confusing mess that you see.

As for the Alt-A case above, what's probably happening is this:

On frame X, you start drawing the a stroke.
By the time you've finished drawing, it's already frame X+Y. Other viewports will get redrawn in the meantime (see note above about these strokes being 2d in screenspace, and not yet the 3d versions yet).
If you draw another stroke (say by the time frame Z is active), you're adding a new stroke on frame Z, etc.

Any errors you're seeing are probably the result of that.

What can we do about all this?

Well, if the Occulus users are going to be drawing using GP while wearing the headset, my intuitive response would be that we could try just showing the same 2D screenspace overlays in both eyes. The problem I guess is that the perspective may be off, and there could be some weirdness arising from having the same thing aligned like that when presented in 2 eyes.

The other option will be to move towards directly converting the stroke points as users are drawing them. A useful side effect of this is that users with a NDOF device could rotate the view while drawing the strokes, making it possible to have the strokes be fully 3d when drawing them. Hopefully though, the performance impacts of doing this aren't too bad.

Ahh... I think I finally starting to understand what's going on here! --- What's happening with Grease Pencil: * Strokes are only converted to 3D coordinates *after* the user has finished drawing. This was done to keep the drawing tool nice and interactive, as it's not getting bogged down in all the conversion math after each and every point drawn. A really big goal for me when initially building Grease Pencil was to make sure that it was as lag-free as I could get it, as the lagginess seen in many other drawing apps at the time was something that really bugged me (and still does). * These "temporary" stroke points are held in a separate buffer (from the stroke data stored in layer->frame). They are not drawn as part of the data-space strokes, but rather they are drawn after the screenspace ones (see next point) * There are basically 3 drawing passes: ``` === Data Space === (scene objects) GP Strokes in "Cursor" space === Screen Space === (overlays - names, text, axis indicator, etc.) GP Strokes in "Screen" aligned space Temporary GP Strokes Drawing Buffer ``` * Previously the temporary strokes being drawn in screenspace would not be an issue if there were multiple viewports, as only the active viewport would get redrawn. --- Now, here's where things start getting quite murky, as there seem to be multiple things happening at once! Case 1) Main viewport != Camera View. The offscreen buffer is being rendered from the perspective of the scene camera (or actually slightly back from the camera - from the actual location of the camera). However, the temporary screenspace strokes also get drawn here, so they will end up in the wrong places Case 2) Main viewport = Camera View (but zoomed out) When the viewport and buffer show the same view (i.e. the view from the camera), but they are not the same aspect ratio/zoom level, so some weirdness still seems to happen. I guess this is the main source of trouble I'm guessing. I'm not sure of the specifics here yet though. Maybe it's a real problem, maybe it isn't. Part of the problem in the offscreen buffer script example is that both of these cases end up interacting and combining together, leading to the confusing mess that you see. --- As for the Alt-A case above, what's probably happening is this: * On frame X, you start drawing the a stroke. * By the time you've finished drawing, it's already frame X+Y. Other viewports will get redrawn in the meantime (see note above about these strokes being 2d in screenspace, and not yet the 3d versions yet). * If you draw another stroke (say by the time frame Z is active), you're adding a new stroke on frame Z, etc. Any errors you're seeing are probably the result of that. --- What can we do about all this? Well, if the Occulus users are going to be drawing using GP while wearing the headset, my intuitive response would be that we could try just showing the same 2D screenspace overlays in both eyes. The problem I guess is that the perspective may be off, and there could be some weirdness arising from having the same thing aligned like that when presented in 2 eyes. The other option will be to move towards directly converting the stroke points as users are drawing them. A useful side effect of this is that users with a NDOF device could rotate the view while drawing the strokes, making it possible to have the strokes be fully 3d when drawing them. Hopefully though, the performance impacts of doing this aren't too bad.

Mike Erwin commented

2016-10-06 23:18:55 +02:00

Added subscriber: @MikeErwin

Joshua Leung was unassigned by Dalai Felinto

2019-12-23 16:37:22 +01:00

Dalai Felinto commented

2019-12-23 16:37:22 +01:00

Added subscriber: @JoshuaLeung