Page MenuHome

Windows pen input design
Needs Triage, NormalPublicDESIGN

Description

When using Wintab, it is reported that pen input misses in-between mouse events. This causes jagged lines with fast strokes when using the painting or grease pencil modes.
For some Windows users, using Windows Ink is not an option as it conflicts with the configuration of other software.

Event Timeline

Think at one point in time @Nicholas Rishel (nicholas_rishel) had a patch for this, that got reverted for "reasons" (that escape me completely at this moment) and we were planning to bring back for 3.0 ?

Am i super far off base here @Nicholas Rishel (nicholas_rishel)?

With grease pencil I had to add a lot of algorithms to "guess" what is missing and figure out what points are missing. This algorithm reduces the problems, but it kills some of the small movements done by artists. The goal is to remove all these algorithms and get the real information of the pen movements.

For example, when the artist draws a fast arc, the algorithm makes a guess of the points of the arc because we receive only a few points of the pen events. This is a very common problem because 2D artists tend to draw arcs very fast to get organic strokes.

Also, we have seen problems with the pressure, especially at the end of the strokes when the pen is separated from the drawing surface.

In case of sculpt mode, it uses the same input code as the rest of the painting modes (vertex, weights...). This code just interpolates the positions between two regular mouse events to fill the space with a fixed spacing. For brushes that are usually used with a small size (all cutting and clay brushes), the artifacts are really noticeable. These jagged line artifices is probably what some users identify with "performance issues", while it is just the input code missing the events.

Quick outline of the mouse/Wintab/WinInk situation.

WinInk "just works" 99% of the time. Several people have written it off due to poor experience years ago; today the situation is much better. However, it is sometimes disabled because it breaks Wintab for some devices and some applications don't support WinInk (e.g. GIMP), so Wintab is still necessary. Also mouse-like tablet movement looses pressure in all drivers I've seen so that's a bummer.

Mouse inbetween/high frequency (same thing) can not be made to work for absolute input (tablets, VM, remote desktop). Not an issue except...

Wintab in Blender relies on mouse input position. The last attempt to decouple the two revealed many issues in Wintab drivers, but the ones the led to reverting changes were:

  1. Getting inconsistent behavior between Wintab and Windows reported cursor position for some devices.
  2. Decoupling the two breaks mouse-like-movement for Wintab.

I believe both could be fixed with a heuristic to determine when Wintab and Windows cursor position are inconsistent, and fall back to mouse input if so. Doing heuristics on input gives me pause, but given the fall through condition I think the only concern is if toggling mouse fallback happens frequently.

@Brecht Van Lommel (brecht) @Dalai Felinto (dfelinto) @Ray molenkamp (LazyDodo) where last we left off was considering removing Wintab in 3.0. The issues outlined under WinInk includes new information not presented at the time which makes removing Wintab a non-option.

Yes, the problem is really noticeable with brushes with a small size.
From the Grease Point side this issue is one of the most important problem for medium-big 2D animation studios to adopt
GP for the final art in production. Is a recurring feedback report.

From what i got from @Antonio Vazquez (antoniov) is that "switching to windows ink for some studio's just is not an option" so I guess we have to support wintab in one form or another

there's mentions of tests by @Daniel Martinez Lara (pepeland) what is being tested and what metrics are collected there?

If the no hacks solution for wintab implies further issues with a minority of devices and drivers, I would go with that. Better having the most commonly used devices working as expected that all the devices half working. Also, with a list of supported devices/drivers at least we can point people at hardware setups that we know they properly work and are supported.
This will also allow us to include other pen based features (like tilt support) in a more predictable way across all platforms.

The no hacks solution is what was rolled back; the prevalence of devices with issues is unclear.

Generating a supported devices list for Wintab would be a lot of work for little payoff. It's not an issue that can be isolated to an API level, it's not consistent across devices by the same manufacturer (even Wacom), and you're going to hear a lot more from the people whose devices have broken than those that still work.

Breaking device support will burn good will, and if it must be done Blender should be in a better place for it.


I would recommend one of two paths forward.

Some breakage, best for average user.

Remove Automatic API, promote Windows Ink to default, modify Wintab for high frequency without heuristics.

Positives

  • Default behavior improves for most people.
  • Wintab breakage less likely to impact average user.
  • More maintainable, simpler code.

Negatives

  • Removes support for mouse-like tablet input with pressure.
    • Issue for tablet input on multi-screen setups where relative movement is common.
  • Requires user intervention when Windows Ink has been disabled.
    • Blender appears to have broken where it previously worked.
    • Potential blowback, people have strong opinions on Windows Ink.
No breakage, quietly suboptimal for some users.

Modify Wintab for high frequency with heuristics for Win32 mouse fallback when the two don't match up.

Positives

  • Ideally no user sees regression from current behavior.

Negatives

  • Adds complexity to tablet and mouse logic.
  • Wintab mouse fallback worse than using WinInk.

From what i got from @Antonio Vazquez (antoniov) is that "switching to windows ink for some studio's just is not an option" so I guess we have to support wintab in one form or another

Actually, it was @Pablo Dobarro (pablodp606) who said that. ;-)

Probably @Daniel Bystedt (dbystedt) can give you more details about windows ink issues on windows, but I know that now using windows ink for compatibility is a common situation.

Some breakage, best for average user.

Remove Automatic API, promote Windows Ink to default, modify Wintab for high frequency without heuristics.

This still seems like the best option if the downsides are small in practice.

However, it is sometimes disabled because it breaks Wintab for some devices and some applications don't support WinInk (e.g. GIMP), so Wintab is still necessary.

"switching to windows ink for some studio's just is not an option"

Regarding these two points, I'm not fully clear on what this means. I thought there was a setting to disable pen input for entering text, or some other pen user interface features. But is there also a setting to make it so reading pressure from the Windows API does not work?

If that is indeed the case, can we detect the value of that setting, or detect that there are no Windows Ink devices with pressure support? And then use that to decide to use Wintab instead?

An additional problem with Windows Ink in Blender is that it displays pressed modifier keys under the mouse cursor for some reason.

EDIT
Ah, ok... sorry I just discovered this is a problem with all software if you enable Windows Ink... same problem with Krita if I enable Ink.
It seems there is no way of disabling it in driver or windows settings. This might be something that will scare away users from Ink, plus other "extra features"
like overlays that some users might not know how to disable.

@Brecht Van Lommel (brecht) The two points relate to the question "Is Wintab still necessary" with the answer being "yes". The issue is some people disable all Windows Ink interaction via the tablet's preferences. They do this because it fixes lag issues(1) with Wintab, and some applications still require Wintab (GIMP for instance).

I'm assuming this relates to "switching to windows ink for some studio's just is not an option" but that's for @Pablo Dobarro (pablodp606) and @Daniel Bystedt (dbystedt) to confirm.

But is there also a setting to make it so reading pressure from the Windows API does not work?

The issue is that Windows Ink (WM_POINTER pen) events don't occur at all, not just that pressure doesn't work. The setting is in individual manufacturer's tablet preferences utilities and is not exposed in any API.

If that is indeed the case, can we detect the value of that setting, or detect that there are no Windows Ink devices with pressure support? And then use that to decide to use Wintab instead?

From my investigation there is was no way to determine if a connected Windows Ink device was disabled by tablet preferences(1). The APIs I've investigated (notably GetPointerDevices) don't recognize when a device that supports Windows Ink is not available due to being disabled by the tablet's preferences utility.

We can not fall back to heuristics such as checking if Wintab has input while Windows Ink does not, because some tablet drivers do not send Windows Ink messages when Wintab has been initialized for the process.

(1) There are other reasons some users disable Windows Ink, but the only one I've found that is workflow breaking is tablets where Windows Ink causes Wintab lag.
(2) I suspect this is because detection of available Windows Ink devices is handled by inspecting HID reports, whereas disabling Windows Ink in tablet preferences happens via the driver just not generating Windows Ink events. This is just an educated guess.

If it breaks for users that disabled Windows Ink I guess it's not as bad, they may be able to guess that they need to change the Blender or system configuration to get things working. I don't Windows Ink will work well for 99% of users out of the box, give the amount of issues that can be found online in various popular apps, and often with advice to disable Windows Ink to get things working.

But I also don't see an elegant solution if we can't detect if Windows Ink is disabled automatically. Maybe there is some registry setting, maybe even a Wacom specific configuration that we can detect, to get it a closer to working out of the box for everyone. But if not, I guess defaulting to Windows Ink is still the least bad option.