GSoC 2019: Core Support of Virtual Reality Headsets through OpenXR
This is a patch with all changes from the [[https://developer.blender.org/diffusion/B/history/soc-2019-openxr/ | soc-2019-openxr]] branch.
Changes from the [[https://developer.blender.org/diffusion/B/history/temp-vr-draw-thread/ | temp-vr-draw-thread]] branch are not included.
The project aimed at bringing stable, well performing VR rendering support to Blender, based on the new OpenXR specification. Further, debugging utilities should be added. A fully fledged VR experience, e.g. with support for editing 3D content with controllers, was not in scope of the project.
How to Test
Testing this patch isn't as simple as applying it and compiling with it. Information on how to test can be found here.
In short, the following is needed:
- Install an OpenXR runtime.
- Checkout the soc-2019-openxr Add-ons branch.
- Install/build the OpenXR-SDK (already bundled with precompiled Windows libs, use install_deps.sh on Linux).
- After a successful build, enable Basic VR Viewer Add-on.
- Launch the session through Window → Toggle VR Session.
- OpenXR loader from the OpenXR SDK to connect to the System’s active OpenXR runtime.
- OpenXR extension (and API-layer) management.
- Basic OpenXR event management.
- VR session management.
- Well performing VR rendering - more performance improvements are possible, but we have a quite decent baseline.
- Carefully designed error handling strategy, cancelling the VR session with a useful user error message (e.g. “Failed to get device information. Is a device plugged in?") and no side-effects to the rest of Blender.
- Compatibility with DirectX-only runtimes.
- --debug-xr command line option enabling our own debug/information prints, OpenXR debug prints and the OpenXR core validation layer.
- --debug-xr-time command line option to print frame render times and FPS information.
- wmSurface API to manage offscreen drawables without a wmWindow.
- Abstraction (currently a GHOST_Xr-API) for all OpenXR specific code. Makes higher level usage easier, but most importantly, improves maintenance (esp. when updating OpenXR versions).
- Add-on to hide VR features by default from the UI.
If needed I can explain features in more detail, for now keeping it short.
Visibility for Users
Showing a "Toggle VR Session" button in the UI by default may fool users into thinking there was full fledged VR support in Blender. To not disappoint users with false promises, I wrapped this button into an Add-on which is disabled by default. The Add-on description clearly warns that support is limited and an early preview.
Error Handling Strategy
The error handling strategy I chose uses C++ exceptions, a controversial feature. Let me explain why I think this is reasonable here.
The strategy requirements were:
- If an error occurs, cleanly exit the VR session (or destroy the entire context), causing no resource leaks or side effects to the rest of Blender.
- Show a *useful* error message to the user.
- Don't impair readability of code too much with error handling.
Here's why I chose an exception based strategy:
- Most alternatives require early exiting functions. This early exiting has to be 'bubbled up' the call stack to the point that performs error handling. For safe code, early exit checks have to be performed everywhere and code gets really impaired by error checking. Tried this first and wasn't happy at all. Even if error handling is wrapped into macros.
- All GHOST_Xr resources are managed via RAII. So stack unwinding will cause them to be released cleanly whenever an exception is thrown.
- GHOST_Xr has a clear boundary (the Ghost C-API) with only a handful of public functions. That is the only place we need to have try-catch blocks at. (Generally, try-catch blocks at kinda random places are a bad code smell IMHO. Module boundaries are a valid place to put them.)
- Exceptions allow us to pass multiple bits of error information through mulitple layers of the call stack. This information can also be made specific with a useful error message. As of now, they conain a user error message, the OpenXR error code (if any), as well as the exact source code location the error was caught at.
So the strategy I went with works as follows:
- If a VR related error occurs within GHOST_Xr, throw an exception (GHOST_XrException currently).
- OpenXR calls are wrapped into a macro throwing an exception if the return value indicates an error.
- Useful debugging information and user messages are stored in the exceptions.
- All resources must be managed through RAII, so throwing an exception will release 'dangling' ones cleanly.
- In the GHOST C-API wrappers, the exceptions are caught and contained error information is forwarded to a custom error handling callback.
- The error handling callback is set in wm_xr.c, prior to creating the XR-Context, and implements clean destruction of the context.
Notes on the GHOST_Xr-API
Early on, I decided to do the OpenXR level access through GHOST. Main reasons:
- OpenXR requires access to low level, OS dependent graphics lib data (e.g. see XrGraphicsBindingOpenGLXlibKHR)
- Some C++ features appeared handy (std::vector, RAII + exception handling, cleaner code through object methods, etc.)
- General low level nature of the OpenXR API
After all I think the functionality is too high level to live in GHOST however.
My proposal would be to add a new module to intern/ instead, named VAMR (virtual, augmented and mixed reality).
Getting the low level graphics lib data out of GHOST is tricky though. Maybe the best option is to add something like a GHOST_OpenXRGraphicsBinding class called from VAMR.
I'm not sure if this is something that should be done prior to merging, or if it's fine to plan this for further work.