conflict with how Blender is sampling OSX input/stroke events #62565
Labels
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
14 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: blender/blender#62565
Loading…
Reference in New Issue
No description provided.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
System Information
Operating system: Mac OSX 10.14.3 (18D109)
Graphics card: Radeon Pro 570 4096 MB
Blender Version
Broken: 2.80beta
Worked: -
Short description of error
Blender registers wrong pressure values on stroke realease when input is given by OSX stroke events from iPad Pro ([Astropad ]]) or Force Touch Trackpad ([ https:*tenonedesign.com/inklet.php | inklet )
Exact steps for others to reproduce the error
We were recently testing a setup that replaces the Cintiq with iPad Pros. As you may know new macBook trackpads are pressure sensitive. This means that you can enable pressure recording in your apps using apple' integrated input/stroke events. This also means that there are applications out there that are enabling this feature for any osx software without having to code it.
One of the easiest app to use is called inklet and lets you use the trackpad as a graphic tablet .
The same feature is brought to iPad Pros with Astropad turning your iPad in a Cintiq for your Mac.
Unfortunately both apps (working fine in other software like photoshop or krita) are experiencing the same issue with blender and are easy to reproduce trying to draw with Grease Pencil: whatever you do the final point of the strokes records 100% pressure on release, drawing a bubble at the end.
I've been in touch with Astropad team and the say:
"With the latest, we were able to get pressure sensitivity and reproduce the high pressure end of stroke. But, we were able to get a similar result with just using a trackpad as well,
which makes us believe there is a possible conflict with how Blender is sampling input/stroke events in general"
is there something you can do to help us Mac people to have a working workflow with our hardware?
Added subscriber: @icappiello
#63769 was marked as duplicate of this issue
Added subscriber: @fsiddi
Added subscriber: @pepe-school-land
Added subscriber: @WilliamReynish
I can confirm the issue here, using an iPad Pro as input:
Blender w/ Grease Pencil:
Blender w/ Texture Paint:
In Blender, each line ends with full pressure.
Photoshop:
Photoshop and other apps act normal
Tested with Duet Display instead of Astropad. Same issue. So it seems not to be an issue with Astropad itself, but probably an issue in Blender itself.
2.79 also has the same behavior.
Added subscriber: @neilish
I'm new to blender from 2D land and have the same issue. I think I've added myself for updates?
Added subscriber: @antoniov
Added subscriber: @brecht
Not sure why you assigned this, @antoniov doesn't even have a mac for testing as far as I know.
@brecht who can this be assigned to?
I don't know of any developer with this kind of hardware, so no one currently.
If it becomes high enough priority we can try to get a setup like this, or deeply analyze the code to try to find the bug, but we have more important bugs to fix right now.
@brecht actually any macbook pro from 2016 to now has a forcetouch trackpad you can use to test it with inklet. This shares the same osx input and stroke event as astropad on ipad.
It’s very hard to believe that there isn’t any developer out there using a recent macbook pro.
Added subscriber: @nBurn
Ah you're correct. I thought he fixed bugs with Apple before. but I must have mixed him up with someone else. Looking at an earlier post of his it seems he only has access to a Windows dev environment:
Added subscribers: @JoshyB, @ZedDB
Added subscriber: @Sergey
I can reproduce the issue with Inklet software on a trackpad.
Doesn't happen with Wacom Intuos btw, so it's something specific to that software and hardware.
@Sergey it works with almost any pressure enabled software out of the box through inklet and Astropad (you can easily test krita). Believe anyway it should work just like pinching, zooming and rotating with two fingers on the trackpad are working using the regular APIs.
please have a look here:
ForceTouchCatalog: Using the Force Touch Trackpad API
there are also js using it for web development here:
JavaScript library for handling Force Touch
@brecht searching on the bug tracker it seems this issue is also happening with other setups for non-wacom users.
Blender 2.78 Grease Pencil - pen pressure problem (Yiynova Tablet)
The video at the end of that discussion shows where the issue lies and a possible fix
https://youtu.be/xkWECtX-ORs
Apparently it’s to do with a variable in gpencil_paint.c ‘pressure = 1.0f’ rather than getting changed to a default of ‘0.0f’ which solves the tablet issue but then drawing strokes with the mouse won’t work because the default is now 0.0
@JoshyB, yeah thanks. That's why I was linking the topic. Hope some developer cares enough to have a look at it.
Feel your pain, I’ve been wanting this fixed for aaaaaaaaaaages.
@Sergey Wondering if this issue could just become a setting somewhere that allows us to switch between a mouse/tablet drawing mode which just changes the pressure variable between 1.0f and 0.0f? That would seem the most straight forward solution
Added subscribers: @KonstantinBukow, @JoshuaLeung
@JoshyB @Sergey
can't this be a possible solution?
is there a reason why is not possible to set 0f on every release event?
@JoshyB Thanks for the tip, compiled blender with that option and works like a charm with both Astropad on iPad Pro/iPhone and Inklet with Trackpad.
At least works for grease pencil. For other modes I think some more files should be edited.
Hello people! I'm super-happy that my hack could help others to make their pens work correctly with blender! I don't know how to implement a clean solution which would let the mouse keep working. I still think that setting the pressure level to 0 at every release might work, but unfortunately I couldn't find out how to do that. I'd be even more happy if the real devs could find out a solution that helps everybody with not-wacom tablets to be using the GP, as a lot of other blender users might be working off finacial constraints. Let's hope everybody will be happily drawing very soon. Best regards everybody!
@KonstantinBukow I've just followed the docs to compile my own version of 2.8 with the change you suggested and it works perfectly!! THANK YOU!! I'm super excited because it means i can continue development of my Grease Pencil based character rig!
BTW it's even more important to get this fixed with Apple announcing Sidecar to allow tablet drawing for Macbooks using the iPad, if Blender only recognises certain kinds of hardware tablets then this will turn into less of a niche issue.
@antoniov Can you look at this issue? It seems like there is a solution for pen input - probably we just need to detect if you are using mouse or pen, which we already do anyway.
@WilliamReynish I would do if I could reproduce it. I have seen the problem is the line if (event->tablet_data) { (gpencil_paint.c) is not getting data when you use that type of tablets, but if we move the value of 0.0 in the else, then the mouse will not work.
Could you try to debug in you system what is the value of the event when use the tablets? maybe there are some way to detect that you have a tablet but it's not sending data... something like add in the else a code to verify if(mouse) p=1 else p=0
Probably painting operators should never use the release event mouse coordinates or pressure to draw anything? In principle there should be a mouse move event preceding it, and then the release event should merely stop the painting, not define the last part of the stroke.
It may be that GHOST does not work like this on all platforms, it would need to be tested.
Setting pressure to 0.0 for release events probably achieves the same effect in practice.
@brecht The events are filtered with the code below:
As it's the last part of the stroke, the First run is not used here, so we have an event of type MOUSEMOVE here.
EDIT: The release or press (any status change) of the mouse buttons is considered as end of stroke, but the wrong pressure in last point is generated in the code above, not in the finalization.
As I understand it, @JoshyB says this change in
gpencil_draw_apply_eventworked:So that would mean either some mouse move events are missing tablet that, and the release event is not the issue. Or the release event pressure is somehow used elsewhere and affecting the stroke. If someone can run blender with
--debug-eventsfrom the command line and paints a single problematic stroke, then we can see in the output which it is.@brecht,
i have drawn 3 strokes. All affected by the pressure issue.
here is the console output for the events:
let me know if this helps
and these are the registered events for a single stroke in console
one thing to be considered is this "fix" works only in grease pencil, all blender areas that can receive tablet pressure inputs (Texture paint, Weight paint, Vertex Paint, etc...) are affected though.
Removed subscriber: @fsiddi
@brecht did the event debug data give any clue on how to handle this?
Added subscriber: @timcarroll
The same issue happens in the release version of Catalina with Sidecar and an iPad Pro/Apple Pencil.
I've got a partial answer for what's going on -- will try to pick this up again tomorrow night, but thought I'd pass on some notes.
I instrumented the macOS side of the code to verify when the various tablet routines were being called during a basic test using Sidecar and an iPad on Catalina. The short version is that the mouse up event is getting clobbered by a tablet proximity event.
In my sidecar test case, when the mouse up event arrives, it is marked with a subtype of NSEventTypeTabletPoint, and the code calls GHOST_SystemCocoa::handleTabletEvent(void *eventPtr, short eventType):
NSEventTypeTabletPoint found
Handle Tablet Event Enter
Pressure and Fill Set
Handled a Mouse Up Tablet Event
However, before that event is processed by Blender's internal event system, the code also processes a tablet proximity event, so the log continues with:
received a tablet proximity event in WindowViewCocoa
handle tablet event enter
NSEventTypeTabletProximity
pointer is leaving
And finally, the event is processed internally:
wm_event_do_handlers: Handling event
wmEvent type:1 / LEFTMOUSE, val:2 / RELEASE,
If you look at the code near the "pointer is leaving" printf in the Cocoa code, handleTabletEvent sets the active mode to none.
ct.Active = GHOST_kTabletModeNone;
I think the second event is clobbering the mouse up event and clearing the tablet data. I don't know the rest of the codebase at all, but it looks like other code in Blender tests against GHOST_kTabletModeNone to determine whether any tablet data should be kept around. This code appears to propagate that into a different tablet format:
static void update_tablet_data(wmWindow *win, wmEvent *event)
{
}
And that data structure is the one being passed into the earlier checks on this thread that found "no tablet data", setting the strength to 1.0.
I can confirm that if I comment out tabletPoint and tabletProximity in WindowCocoaView, the problem goes away. The mouse up messages have tablet information so pressure get released as appropriate. This was just a test; those events are important for tablets with proximity sensors. Also, I found that later events (like window deactivation) were showing tablet events, which would clearly not be the right thing.
It seems like the right behavior is for there to be two separate events. First the mouse up with tablet information, and then the tablet proximity event to clear all tablet information. This is where my knowledge of Blender's architecture mostly ends. The second event would mostly be synthesized just to clear the tablet event data, but I couldn't find a similar good example in the source. Perhaps a mouse moved message could be synthesized without actually moving the mouse position? Or would an "unknown event" be acceptable?
It would be a more radical change to this code, but setting up the tablet parameters before pushing events on the queue (rather than after) feels like a more natural model. It means that the tabletProximity event on the macOS side can clear the tablet information and then it just gets picked up by the next event.
Anyway, those are some thoughts for future discussion.
Added subscriber: @jinschoi
This may or may not be an acceptable fix, but adding dispatchEvents() when leaving proximity in handleTabletEvent() flushes out the event queue and fixes the issue for me in both grease pencil and texture paint modes:
Good catch. I was looking for something like dispatchEvents() yesterday but couldn't find it.
I would put the dispatch events calls in GHOST_WindowViewCocoa.h instead.
(void)tabletPoint:(NSEvent *)event
{
systemCocoa->dispatchEvents();
systemCocoa->handleTabletEvent(event, [event type]);
}
(void)tabletProximity:(NSEvent *)event
{
}
My logic:
handleMouseEvent consistently spawns a new mouse event and then calls handleTabletEvent to add additional tablet information for that event. You can go from handleMouseEvent to handleTabletEvent(event) to handleTabletEvent(event,subevent) as a single chain of calls, and if proximity was an issue, it seems likely you'd want that to be a single event.
In contrast, the two handlers in GHOST_WindowViewCocoa.h are smashing their changes onto whatever happens to be on the queue - they don't generate new events on their own. So it might make sense for them to flush the changes out first.
I made the changes to my local copy and the problem doesn't trigger any more.
wm_event_do_handlers: Handling event
wmEvent type:1 / LEFTMOUSE, val:1 / PRESS,
tablet: active: 1, pressure 0.0784, tilt: (0.4111 -0.1333)
wm_event_do_handlers: Handling event
wmEvent type:1 / LEFTMOUSE, val:2 / RELEASE,
tablet: active: 1, pressure 0.0000, tilt: (0.3333 -0.1667)
And the next event after this one correctly does not show any tablet info.
wm_event_do_handlers: Handling event
wmEvent type:260 / WINDOW_DEACTIVATE, val:2 / RELEASE,
So, I think that's the correct solution, unless there's an edge case somewhere that you really want the tablet events to override the mouse events. It probably requires someone with a tablet that regularly receives proximity events to test it though.
This comment was removed by @icappiello
Added subscriber: @tim-50
can conform this fix works also on astropad and duet.
thanks @tim-50 Carroll (codrus) for this!
@brecht Van Lommel (brecht) is there any reason why this fix can't go in master?
We have several cintiq here, the patch seems not affect cintiqs behaviour.
Which Mac tablet is supposed to use proximity events differently?
I'm not sure about flushing the queue where you suggest. handleTabletEvent() does not seem to modify the event in the queue, it just sets some tablet parameters (pressure, tilt) in some window context object, and then presumably some other code uses that info when it comes to processing the event. For drawing, then, you want the latest tablet info available when the event gets processed or else you're always using old pressure data. The only instance where you don't want tablet data smashing is on the leaving proximity event because it leaves tablet mode, and no drawing is going to happen anyway.
Also, flushing the queue on every tablet event may have performance implications. I haven't gone through to figure out how/when the queue normally gets handled.
Added subscriber: @Mwaib1
Hi, don’t know every much about code and I'm fairly new to blender, can anyone help me out with an explanation on how to fix this issue or which blender scripts to change, I'd very much appreciate it. Thank you!
I dug slightly deeper. Events get dispatched from within a loop in WM_main() by calling wm_window_process_events(), and yes, handleTabletEvent() does not modify the event in the queue but changes values in the member variable m_tablet of GHOST_WindowCocoa. So given my reasoning above, I really don't think you want to dispatch events preemptively except in the sole case when leaving proximity.
Here's my change as a diff:
I haven’t had time to look into this further, other than to determine that sidecar consistently pairs the events. When you touch the screen you get a tablet prox event saying you moved to the screen followed by a mouse down. Then, later, a mouse up followed by the tablet prox event showing you moved away.
It still seems weird to me that Blender doesn’t generate distinct events for the tablet events but without a tablet that makes use of the proximity sensors I can’t really test how Blender behaves when you move the stylus around near the screen.
I think your change will fix the symptoms here, which is to make sure that the mouse up/ moving away message guarantees that at least one message gets sent to Blender’s event loop with the tablet data still attached during a mouse up/ pencil up sequence.
This issue was referenced by
2e9d5ba211Added subscriber: @PabloDobarro
I've now committed a fix for this. It's only for 2.83 since I consider it too risky a change to put in 2.82, as it entirely changes the way tablet data is passed from GHOST to Blender.
@antoniov @PabloDobarro if you find new issues with tablets in master, it's probably related to this change.
Changed status from 'Confirmed' to: 'Resolved'