Page MenuHome

VR: Controller Drawing
Confirmed, NormalPublicDESIGN


VR: Controller Drawing

When in a VR session, it's very important to have visual feedback on the controllers. At least to tell where they are and which direction they point at.
For a start, just having controllers visualized would be sufficient. Visualizing button presses or touchpad interactions should be done too, but is not as important.

Here is what we could do:

  • Mixed Reality provides the XR_MSFT_controller_model_preview OpenXR extension for its controllers. It uses GLTF to provide the geometry. If the GLTF add-on is available in Blender, we can use that to obtain the geometry. Other vendors will probably provide geometry this way too, in the future.
  • For other controllers, we may have to bundle geometry ourselves somehow.
  • There should be a simple fallback solution for when we don't have any geometry data. That may be a simple tube-like shape, or even a simple coordinate axis.

Open Question: With the geometry available, how should controllers be drawn in the viewport?

Controllers have to be drawn to the regular projection-layer of the OpenXR compositor. This means they must be drawn during or after regular viewport drawing. But how exactly to do this?

Few options:

  • Draw controllers as gizmos. Maybe gizmos were not made for such a case (drawing custom geometry in the viewport), but this could work fairly well. Currently shading/lighting would have to be done in a custom shader. Maybe gizmos could somehow use draw-manager shaders though. [Seems there are more and more reasons to integrate gizmos properly with the draw-manager...]
  • Have some way to hand the draw-manager custom geometry to draw with its shading.
  • Entirely custom OpenGL drawing using draw-handlers.

Related Objects

ConfirmedTO DONone

Event Timeline

Julian Eisel (Severin) updated the task description. (Show Details)
Julian Eisel (Severin) changed the subtype of this task from "Report" to "Design".
Julian Eisel (Severin) changed the task status from Needs Triage to Confirmed.May 28 2020, 4:04 PM
Julian Eisel (Severin) added a project: Restricted Project.Jun 19 2020, 11:46 PM