Page MenuHome

New volume object type
Confirmed, NormalPublicTO DO

Tokens
"Love" token, awarded by dylanneill."Love" token, awarded by Way."Love" token, awarded by arc4g."Love" token, awarded by corpse."Love" token, awarded by Beckersc."Love" token, awarded by 1D_Inc."Love" token, awarded by juang3d."Like" token, awarded by Fracture128."Burninate" token, awarded by amonpaike."Love" token, awarded by RC12."100" token, awarded by Peine_Perdue."Burninate" token, awarded by Kubo_Wu."Love" token, awarded by bintang."Like" token, awarded by TheRedWaxPolice."Like" token, awarded by sebbas."Like" token, awarded by evilvoland."Love" token, awarded by Shimoon."Love" token, awarded by Bit."Love" token, awarded by ofuscado."Love" token, awarded by MetinSeven."Love" token, awarded by jfmatheu."Love" token, awarded by BlackRainbow."Love" token, awarded by garyo123."Love" token, awarded by CobraA."Party Time" token, awarded by franMarz."Love" token, awarded by Scaredyfish."Love" token, awarded by Ztreem."Like" token, awarded by sng84."Love" token, awarded by wilBr."Love" token, awarded by charlie."Burninate" token, awarded by zinar."Love" token, awarded by xrg."Love" token, awarded by billreynish."Like" token, awarded by EAW."Like" token, awarded by Schamph.
Assigned To
None
Authored By

Description

Status: Implementation started, it includes other data types (hair, points) to be hidden from UI.


Team

Commissioner: ?
Project leader: @Brecht Van Lommel (brecht)
Project members: @Sebastián Barschkis (sebbas) @Jacques Lucke (JacquesLucke)

Description

Big picture: Volumes need their own native datablock type, so Blender can support other use cases like rendering OpenVDB files or procedurally generating volumes.

Use cases:

  • Existing use cases (smoke modifier on mesh).
  • OpenVDB file rendering.
  • Procedurally generating volumes (part of everything nodes).

Design:

  • New Volume datablock (similar to Mesh) with an OpenVDB volume data structure with an arbitrary number of attributes.
  • Rendered by Cycles and Eevee, first as dense volume as we have now, and later with native sparse volume rendering.
  • Volume objects would have modifiers (add resolution detail, fluid domain modifier, ...)

Engineer plan: ?

Work plan

Milestone 1
Time estimate: 10 workdays to complete implementation, 5 for review/docs/bugs.

  • New volume object data type, based on OpenVDB grids
    • Support reading OpenVDB files from disk
    • Add New material adds principled volume material
    • Solve problem with not enough FILTER_ID bits
  • Rendering of dense grids
    • Cycles
      • Basic integration
      • Use principle volume as default material if no material exists
      • Support different transform per grid
    • Eevee
      • Basic integration
      • Use principle volume as default material if no material exists
      • Support rotation/shear in transform
      • Shader code generation for arbitrary number of grids
      • Support different transform per grid
    • Workbench
      • Basic integration
      • Control over which grids to display (active grid only by default?)
      • Use workbench object color settings
      • Density scale setting
    • Check why example VDB grid files have such low density
  • Viewport drawing and selection of volume bounds
    • Wireframe bounds display of tree nodes (at intermediate or leaf levels)
    • Selection support (based on wireframe bounds?)
  • User interface
    • Open file browser immediately from Add menu
    • VDB / volume file type in the file browser
    • Drag & drop VDB files into 3D viewport
    • Finalize properties editor UI layout
  • Python API
    • Grids
    • Transforms
  • File loading
    • Error reporting
    • On-demand loading
    • Global cache
  • Dependency graph copy-on-write integration
    • Refactor mesh runtime storage to be reusable by other objects types
    • Automatic instancing for objects without modifiers, like meshes
    • Fix runtime backup memory leak
  • Volume frame sequences
    • Settings to use single volume or sequence, start, length, offset
    • Dependency graph integration
    • Auto detection of sequences when loading file
  • Mantaflow: output OpenVDB files with multiple grids and standard grid names
  • Optimizations
    • Share GPU textures in global volume cache
    • Preserve GPU batch cache through display setting changes
    • Cycles: optimize mesh bounds creation by using OpenVDB tree
    • Cycles: optimize voxel loading by using OpenVDB tree
    • Cycles: share 3D images like global volume cache

Milestone 2
Time estimate: ?

  • Volume modifier stack
  • Fluid domain modifier, deprecate modifier for meshes
  • Cycles sparse grid rendering
  • File packing
  • VDB tree/voxels API using pyopenvdb?
  • Workbench: additional viewport display options
  • Grid metadata: show in properties editor

Later

  • Volume modifier nodes
  • Eevee sparse grid rendering
  • Volume edit mode and file saving
  • OpenVDB level sets, clusters, perspective transform

Branch: new-object-types


Relevant links:


Details

  • The initial implementation will only read OpenVDB grids from disk, the fluid simulator or procedurally generate volumes. As a result, there is no need for file saving of volumes (assuming we don't allow operations like apply modifier). When the need for it arises, it seems most practical to treat it similar to images. That is, .vdb files are typically separate files on disk to keep the .blend file size manageable, but packing can be supported.
  • For meshes, hair and pointclouds we can the same custom data layer system for attributes. However since volume are stored in OpenVDB data structures, it's not possible. We could add custom data layer data structures that mirror OpenVDB grids, but keeping these in sync seems error prone. So, the plan is to provide RNA APIs and user interface that is consistent, but with different underlying implementation based on OpenVDB grids.
  • Volume files are typically very heavy, and need to be loaded on demand similar to images, with proper thread safety and garbage collection. We should also load just the part of the OpenVDB file that is needed for specific purposes.
    • Read list of attributes (for display in the UI)
    • Read tree structure (for fast preview in the viewport)
    • Read voxels (for full drawing and rendering)
  • For the copy-on-write dependency graph, we need to clearly design how data is referenced between copies when on-demand loading happens. We can't afford to have the same volume grids loaded multiple times. For images this is handled by not having image datablocks part of copy-on-write at all, but for volume modifiers we need it.

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Development is happening in the new-object-types branch.

This was high on my to-do list, great to see that you've started work on this! Let me know if there's anything I can help you with.

This is an important feature for me as well; I'm happy to help if there's anything I can do.

@Stefan Werner (swerner), I don't have any own code for the Cycles rendering side, which I guess you guys have (and there is also the summer of code project). So integrating that with this new volume object would help.

Beyond that I'm still iterating on the basics, it will be easier to make a concrete list of remaining tasks and distribute work in a week or two from now.

There's now basic Cycles, Eevee and workbench rendering. Still many limitations, see TODO comments in the branch for the details.
rB8b647659852c: Objects: initial viewport draw of hair, point cloud and volumes
rB7c144ac1f9cd: Volumes: cycles rendering support of volume object as dense grids

Next week for me will probably be volume sequence support, and tightening up the code, in particular dependency graph integration.

For anyone who wants to help out, the tasks listed under User Interface, Workbench, Viewporting drawing, Eevee and Cycles are a good starting point. They are pretty self-contained and not likely to conflict with other changes I plan to work on.

Added this week:

  • Frame sequences for importing animations.
  • Global volume cache to let multiple datablocks automatically share the same grid memory.
  • Simpler UX for importing OpenVDB files, though Add menu and drag & drop.

Next week I plan to make the rendering and viewport drawing more complete.

Hello, I am not a programmer, I am just a user (just trying out new things), and I discovered that the search function in the Shader Editor's Shift A menu does not work, any reason for that?

@Zijun Zhou (Eary), are you saying the search function is broken specifically in a build of the new-object-types branch? I can see it greyed out when there is no material, but once adding one it works for me, same as other object types.

@Zijun Zhou (Eary), are you saying the search function is broken specifically in a build of the new-object-types branch? I can see it greyed out when there is no material, but once adding one it works for me, same as other object types.

Wait... It's fixed in the latest daily build, so never mind.

AR (arc4g) added a subscriber: AR (arc4g).
AR (arc4g) removed a subscriber: AR (arc4g).
AR (arc4g) added a subscriber: AR (arc4g).

Downloaded the build from GraphicAll and imported a VDB and everything works in the viewport with Eevee, but when I try to render a frame with Eevee I don't get the expected result. There is no volume.

iss.vdb is too heavy to load on my GPU as a dense texture, but we can preview it in the viewport with wireframes now. Each box in the Fine display is 8x8x8 voxels.

CoarseFine

Velocity grid display:

Downloaded the latest build -- d58e80b4c6cb -- still no render result in Eevee.

Actually now no render result in Cycles as well.

***Actually looks like I am getting a render result now from Cycles and Eevee. Not sure what changed.

Way awarded a token.Wed, Feb 19, 3:20 PM