USD Layers referencing, MaterialX, and USD Hydra rendering Addon for Blender #100569
Labels
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
32 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: blender/blender#100569
Loading…
Reference in New Issue
No description provided.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Intro
AMD published a "USD Hydra and MaterialX" Addon for Blender https://github.com/GPUOpen-LibrariesAndSDKs/BlenderUSDHydraAddon. Based on that and conversations at SIGGRAPH 2022, the need for Hydra rendering and USD import by referencing into Blender is clear. AMD has been working on a branch of Blender to integrate this work directly in. The goal of this is to serve such needs:
Thus this task can be divided into 3 "subtasks" (not sure how to create or track those here)
MaterialX addon for manipulating MaterialX networks
Much of this we can contribute but some parts may need help from the community and Blender developers.
USD Layers / Collections
To support bringing USD data into Blender and having it function as a linked library, we propose a new type of collection which is a “USD Layer”.
USD Layers would be created similar to Collections in the outliner and be a sub type of collections, but have different collection properties:
A USD Layer will be a reference to a USD file. USD “Prims” (objects) in that file are brought into the Blender scene when the USD Layer path changes. They create objects as subtypes of “library data” type. The data blocks (meshes) are only imported if the import toggle on the USD Collection is checked. Thus, the imported collection will have two ways of operating: Import all the USD data into Blender’s memory, or just import the USD skeleton and defer importing data until render time, referencing basically.
Objects created from the USD Collection will have to maintain a link back to the collection and their original USD Prim path in the USD file. USD attributes will be displayed in the spreadsheet editor for the object context.
An example of USD Layers in the outliner. USD Layers would be similar to collections. Note that USD composites data from layers so the order is important.
USD Layer settings
Objects linked to USD layers will have to have a way to set their path in USD
USD Hydra rendering engine addon
The USD Hydra rendering addon is a built-in rendering addon that handles rendering through USD Hydra. It will come with Hydra's GL Storm renderer by default. Users can install Hydra Render Delegates for viewport and final rendering via the addon properties. Many renderers already have Hydra delegates available which users can install. Furthermore, it de-duplicates render export code that each rendering addon does.
Rendering with the Hydra delegate handles Blender data and USD data. Blender data is exported via a USD Hydra Scene Delegate, exporting Blender Data to hydra. When exporting objects that are linked to a USD collection (and are left as a reference, not imported), those objects can be taken directly from the USD. Any overrides of transform, visibility, shader etc to the referenced object done in the blender data can be applied on top of the USD data.
For shaders, the USD Hydra addon will translate cycles node graphs automatically to MaterialX node graphs and attach them to the USD.
MaterialX Nodetree addon
This will be a separate addon that allows users to compose materialX in its own node graph. When enabled, it will give a new node graph type with the MaterialX standard nodes. It will also give an optional setting to override Blender materials with the MaterialX node graph. This option will only be enabled when using along with the USD Hydra renderer.
USD lib requirements
We can expose the USD library as a “monolithic shared library” as well exposing the python bindings. TBD what hooks we should give power users for manipulating USD data from a collection.
We think we can deliver most of this for Blender 3.5. The "translation of cycles to MaterialX" will probably be the hardest part to get right at first, but we can start with simple node graphs.
Added subscribers: @BrianSavery, @dr.sybren, @brecht
Added subscriber: @LazyDodo
The change to a shared USD lib and python bindings we'll likely to ship for 3.4, see #99618 for details
Great let me confirm that's what we'd need.
I can send you a preliminary copy of the 3.4 windows libs if you want
This overlaps with the ideas from #68933 and #100367, so we'll need to figure out how this all fits together. It may be best to have that discussion in #100367.
To be clear, the Blender Foundation has not taken a position on this as far as I know, it depends on the interpretation of the GPL.
Regardless I think native Hydra support is important.
Supporting MaterialX is important, but this also opens up a lot of questions about how to bridge MaterialX and Blender's existing texture/shader nodes, short term and long term. I think we need a two way conversion between Blender shader nodes and MaterialX at least, not just one way.
Right, sorry I wasn't asking for an opinion from BF here, just pointing out that if the developers of Renderers are complaining they don'e want to adapt their software to Blender because of GPL this is a way to get their support into Blender, since most renderers are supporting Hydra
I would question if a true "two-way" conversion is possible or necessary. We're sort of proposing a 1.5 way. Users can convert Blender node graphs to MaterialX, or import MaterialX into Blender for use via Hydra. Is it nesscesary to convert MaterialX back to Cycles node graphs? Most users I see importing in MaterialX would probably be staying rendering in Hydra. There is also the option I suppose to export OSL from MaterialX and then bring that in to Cycles.
We don't have OSL support in Eevee or Cycles GPU rendering currently, so that would only be a partial solution. Also texture baking tools.
For Blender I think it's important for its own renderers to be able to render MaterialX graphs created in other software, one way or another. For external renderers it's not an issue of course, just from the point of view of Blender users in general I think this gap in functionality will be important to tackle.
Let's discuss in the rendering meeting tomorrow. I don't disagree, but definitely needs some input from EEVEE or Cycles to help out here. The main issue I see is that the MaterialX nodes are lower level (for example Voronoi) than Blender's nodes in many cases.
Added subscriber: @makowalski
Added subscriber: @Alaska
Added subscriber: @Roggii-4
Added subscriber: @Yuro
Added subscriber: @platerytter
Added subscriber: @lichtwerk
Added subscriber: @JamesBarrie
Added subscriber: @Scaredyfish
Added subscriber: @paulgolter
Added subscriber: @Zhen-Dai
Added subscriber: @JulianEisel
The GPL FAQ from the FSF seems to disagree with this (at least in a different case, if comparable or not is a different question):
https://www.gnu.org/licenses/gpl-faq.html#NonfreeDriverKernelLinux/
Typically the BF seems to follow such FSF guidance.
Hydra support would definitely be great. Just noting that the licensing topic needs further checking with the BF.
@JulianEisel I'm not going to further argue the point. It was meant as an aside which I can remove from this design if objectionable.
Added subscriber: @ahmed.hindy96
Added subscriber: @ianhsieh
This looks great, Brian! Just a couple of questions from a render delegate point of view:
In the case like this (RenderMan, Arnold etc). They would just have to install some UI addons to Blender, to customize the UI settings for their renderer.... i.e
And we would have to provide some function (in a class type that can be overriden) to inject the render settings into the USD stream as well. Because the USD Python api is being exposed this shouldn't be too hard.
I mean the RenderMan render delegate is loading all it's own plugins right? So like if you pass the "ri:integrator = PxrPathTracer" setting through USD to the RenderMan hydra delegate that should just work? I'm saying this could all be handled in the render settings passed through USD.
My understanding is that AOVs are a handled better in Hydra2, which frankly I haven't looked too deeply at yet. And I'm not sure how ready it is yet?
Added subscriber: @satishgoda1
Added subscriber: @EstebanCovo
Added subscriber: @Plyro
Added subscriber: @Peine_Perdue
Added subscriber: @DagerD
Added subscriber: @AlexeyAdamitsky
Hi.
I'm trying to add a new structure (USD Collection), that should be almost similar to regular Blender's Collection. Now Blender builds without errors and I can see that my structure is successfully registered and added to struct Main. But if I run Blender, ED_undo_stack_get() function returns NULL and Blender crashes. I'm stuck with this.
Can anyone give me some advice on how to fix it? @JulianEisel @dr.sybren
@DagerD we should keep this task about the high level design discussion. For help with development I recommend blender.chat or devtalk.blender.org.
Also be aware that the specific design of this is being discussed in #100367, and that the decision may be to extend Blender collections rather than add entirely separate USD collections.
Added subscriber: @SonnyCampbell_Unity
Hey @DagerD yeah there is already some work underway to integrate USD with collections. There are some specific design goals in place for getting it integrated. If you'd like to join the discussion there is a meeting for the pipelines module tomorrow.
Couple comments here:
The USD Hydra rendering add-on is fairly complete, the MaterialX side needs library support to move it over, but the USD Collections part is the one that is being worked on.
With regards to a "general solution for import collections" that satisfies Blender, I have some concerns.
I think too that separating USD Layers is not a bad idea at all, even Houdini manages USD as a completely different context to not try to bend the universal standard to it's way of working, studios just take the time to learn it and adapt to the standard, flowing along the standard just makes it easier to invest on building a pipeline around it, since you know it wll play well with the larger industry ecosystem.
I also agree that the need or desire for a better USD pipeline outweights by far something like having a general solution for importing the likes of OBJ, most formats are transitory and you can just make a tool to process them in python, I've never heard of a production that wants to stay in STL or OBJ, hopefully the emphasis on general solutions doesn't hinder a well standarized USD implementation
Before deciding if USD collections should be a separate datablock, we really need a design direction for user cases beyond importing external USD files and rendering them in a Hydra. There's creating new USD layers, object selection and editing tools, collection visibility and editing tools, modifiers and geometry nodes, Blender overrides, material assignment, rendering with Cycles & Eevee, etc There's design questions about how data flows between different areas, how best to visualize and edit things pre and post composition in the outliner and 3D viewport, what the user workflow is for editing imported data, etc.
One of the major strengths of Blender is that you can mostly just create in one application without having to deal with all kinds of interop issues. If you end up with two parallel system for collections (and shader graphs), you risk a confusing interface and arbitrary limitations, and difficult maintenance long term. Improving and extending existing functionality to be compatible with USD should be a serious consideration even if adding something new is easier in the short term.
Added subscriber: @ray-1
The platform module is a supporting module, if the render module decides it's needed, we'll make it happen, materialX seems easy enough doubt adding this dep would be much trouble
Removed subscriber: @ray-1
Added subscriber: @Alan-Fregtman
Changed status from 'Needs Triage' to: 'Needs Developer To Reproduce'
Will let devs decide on which way to go, what to keep from this task, #68933 and #100367 (but change status to get it out of the triaging queue)
Changed status from 'Needs Developer To Reproduce' to: 'Confirmed'
Added subscriber: @zNight
Added subscriber: @MikePan
Added subscriber: @tlinde
Added subscriber: @Benjamin-GO
Added subscriber: @astroblitz
Added subscriber: @matthewlow-dwa
Sorry about this question - what's the current situation/progress with this ticket/branch? Is there not-private branch to test? (I assume not yet since there's none linked)
We are actively investigating the ways to improve integrations of our rendering engines (V-Ray, Vantage and others) into Blender, and Hydra delegate support is one of the options.
It's being worked on by @BogdanNagirniak in #104712, so there is a public pull request that you could test.
Hi @BrianSavery and others, I've been working on something similar to what you describe in the 'USD Layers / Collections' section because I wanted a tool that let me edit and add to a USD scene without doing a full import and export. What I've written so far is very WIP, but it does let me move prims around in scenes as well as create new prims copied off other ones.
universal-scene-description
branch: https://projects.blender.org/blender/blender/compare/universal-scene-description...expenses/blender:set_usd_path_custom_prop (latest commit at time of posting). These set up a few custom properties - most importantly the path of the prim being imported but also the layer+prim it references if any, and variant setsI'd love to have suggestions as to how I can improve my changes and the script! I think on the Blender IO side it'd be nice to A) use a better system than custom properties and B) store the initial matrices somewhere so that I don't have to click 'Store Current Transforms' initially.
I'd be very happy for the whole thing to be a core part of Blender and not just some components + an addon, scripting it is just a bit faster for now.
I'm a bit swamped right now with work for the 4.0 release, but a quick reply.
We have some design for this kind of integration in #68933. The big question for me is to which extent your the changes are something that could serve as a step towards full integration, and to which extent they are not integrated closely enough that we would consider this for inclusion in an official Blender release.
Adding some properties to imported USD data seems reasonable to me. The Python add-on doesn't sound like something we would bundle with Blender, I don't think an add-on can really integrate well enough with Blender that we would consider it part of our USD support and something we can promise to keep working longer term.
The design in #68933 has the idea of treating the USD imported datablocks as a kind of linked datablocks, and then using the existing override system to make edits to them. I still think that is a good general approach that would not just handle e.g. object transforms, but all kinds of datablocks and properties.