XR Actions Discussion/Patch Review #88839

New Issue

Peter Kim · 2021-06-04T18:04:00+02:00

Peter Kim commented

2021-06-04 18:04:00 +02:00

The XR Actions (controller support) patches are currently under review #88367 and the first step, the internal abstraction layer to read controller inputs D10942 has been merged.

Next, the goal is to enable executing an operator via these controller inputs, which is addressed in the following patches: D10943, D10944, D10945.

However, these next steps raise some important design questions, namely:

How will XR actions and their associated operators/properties be saved, loaded, etc.? D10943 adds a keymap-like system to manage this although there may be better options.
How will XR input events be dispatched and handled? Will they use the regular window queue or a separate XR surface queue as proposed in D10944?
How to extend operators designed for mouse inputs so that they can be used with VR inputs? In particular, the selection operators (which use GPU select) would be useful for VR since they are not limited like raycast-based selection. D10945 offers one solution, with invoke_3d/modal_3d functions that are intended to wrap the existing operator logic by first projecting 3D controller coordinates to 2D mouse coordinates and then calling the regular invoke/modal functions.

The XR Actions (controller support) patches are currently under review #88367 and the first step, the internal abstraction layer to read controller inputs [D10942](https://archive.blender.org/developer/D10942) has been merged. Next, the goal is to enable executing an operator via these controller inputs, which is addressed in the following patches: [D10943](https://archive.blender.org/developer/D10943), [D10944](https://archive.blender.org/developer/D10944), [D10945](https://archive.blender.org/developer/D10945). However, these next steps raise some important design questions, namely: - How will XR actions and their associated operators/properties be saved, loaded, etc.? [D10943](https://archive.blender.org/developer/D10943) adds a keymap-like system to manage this although there may be better options. - How will XR input events be dispatched and handled? Will they use the regular window queue or a separate XR surface queue as proposed in [D10944](https://archive.blender.org/developer/D10944)? - How to extend operators designed for mouse inputs so that they can be used with VR inputs? In particular, the selection operators (which use GPU select) would be useful for VR since they are not limited like raycast-based selection. [D10945](https://archive.blender.org/developer/D10945) offers one solution, with invoke_3d/modal_3d functions that are intended to wrap the existing operator logic by first projecting 3D controller coordinates to 2D mouse coordinates and then calling the regular invoke/modal functions.

Peter Kim commented

2021-06-04 18:04:00 +02:00

Added subscribers: @muxed-reality, @JulianEisel, @brecht, @ideasman42

Brecht Van Lommel commented

2021-07-05 16:40:06 +02:00

It's not clear to me why this is a system separate from keymaps and regular event dispatch. I'd expect this is just another input device, and not a completely different category than keyboard, mouse, tablet or NDOF device. I could not find the motivation in the linked tasks and code reviews, but maybe I missed it.

Jacob Merrill commented

2021-07-05 16:51:40 +02:00

Added subscriber: @JacobMerrill-1

Jacob Merrill commented

2021-07-05 16:51:40 +02:00

(I am not the patch developer etc but I have used the patches extensively*)
even made games etc*)

the way it's currently exposed I was able to use it out of the box in upbge etc by adding my own gestures to poll

later we may be using gestures to invoke commands too*

"Double Gun Fingers", "Point finger" etc just seems out of place with old actions.

(I am not the patch developer etc but I have used the patches extensively*) even made games etc*) the way it's currently exposed I was able to use it out of the box in upbge etc by adding my own gestures to poll later we may be using gestures to invoke commands too* "Double Gun Fingers", "Point finger" etc just seems out of place with old actions.

Peter Kim commented

2021-07-05 16:57:41 +02:00

In #88839#1187160, @brecht wrote:
It's not clear to me why this is a system separate from keymaps and regular event dispatch. I'd expect this is just another input device, and not a completely different category than keyboard, mouse, tablet or NDOF device. I could not find the motivation in the linked tasks and code reviews, but maybe I missed it.

Thanks for your feedback. Much of the original discussion (including the switch from keymaps to XR-specific action maps) took place in D9124.

> In #88839#1187160, @brecht wrote: > It's not clear to me why this is a system separate from keymaps and regular event dispatch. I'd expect this is just another input device, and not a completely different category than keyboard, mouse, tablet or NDOF device. I could not find the motivation in the linked tasks and code reviews, but maybe I missed it. Thanks for your feedback. Much of the original discussion (including the switch from keymaps to XR-specific action maps) took place in [D9124](https://archive.blender.org/developer/D9124).

Jacob Merrill commented

2021-07-05 23:07:13 +02:00

In #88839#1187160, @brecht wrote:
It's not clear to me why this is a system separate from keymaps and regular event dispatch. I'd expect this is just another input device, and not a completely different category than keyboard, mouse, tablet or NDOF device. I could not find the motivation in the linked tasks and code reviews, but maybe I missed it.

how does one select a headset / have the actions bound to the keymap?

what if there is more than 1 headset?

> In #88839#1187160, @brecht wrote: > It's not clear to me why this is a system separate from keymaps and regular event dispatch. I'd expect this is just another input device, and not a completely different category than keyboard, mouse, tablet or NDOF device. I could not find the motivation in the linked tasks and code reviews, but maybe I missed it. how does one select a headset / have the actions bound to the keymap? what if there is more than 1 headset?

Bibin Abraham Jacob commented

2021-07-10 15:10:50 +02:00

Added subscriber: @JacAbraham

Bibin Abraham Jacob commented

2021-07-10 15:10:50 +02:00

Given the current state of the project, one fix that will make a huge difference with the usability of Blender in VR is - OpenXR Depth Buffer Support, so that blender can be used with a desktop panel inside the scene. (https://developer.oculus.com/documentation/native/pc/dg-dash/ - check for Depth Buffer Support). Consider this as a short-term fix while complete controller support is getting implemented. I'm no developer and this is just a suggestion. Thank you

Peter Kim commented

2021-09-23 09:30:41 +02:00

Changed status from 'Needs Triage' to: 'Resolved'

Peter Kim closed this issue

2021-09-23 09:30:41 +02:00

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

XR Actions Discussion/Patch Review #88839