Page MenuHome
Create Task
Maniphest T73201

New volume object type
Confirmed, NormalPublicTO DO
Actions

Tags
Subscribers
Aditia A. Pratama (aditiapratama)
Andrew Charlton (Scaredyfish)
AR (arc4g)
Bjoern Siegert (nicelife)
Brecht Van Lommel (brecht)
Christian Beckers (Beckersc)
Dustin Bowser (bowserlm)
View All 31 Subscribers
Tokens
"Love" token, awarded by kivig."Love" token, awarded by pierogo."Love" token, awarded by shafannazim."100" token, awarded by Dir-Surya."Love" token, awarded by 3di."Love" token, awarded by shader."Love" token, awarded by mistajuliax."Love" token, awarded by eversimo."Love" token, awarded by leonumerique."Love" token, awarded by dylanneill."Love" token, awarded by Way."Love" token, awarded by arc4g."Love" token, awarded by corpse."Love" token, awarded by Beckersc."Love" token, awarded by 1D_Inc."Love" token, awarded by juang3d."Like" token, awarded by Fracture128."Burninate" token, awarded by amonpaike."Love" token, awarded by RC12."100" token, awarded by Peine_Perdue."Burninate" token, awarded by Kubo_Wu."Love" token, awarded by bintang."Like" token, awarded by TheRedWaxPolice."Like" token, awarded by sebbas."Like" token, awarded by evilvoland."Love" token, awarded by Shimoon."Love" token, awarded by Bit."Love" token, awarded by ofuscado."Love" token, awarded by MetinSeven."Love" token, awarded by jfmatheu."Love" token, awarded by BlackRainbow."Love" token, awarded by garyo123."Love" token, awarded by CobraA."Party Time" token, awarded by franMarz."Love" token, awarded by Scaredyfish."Love" token, awarded by Ztreem."Like" token, awarded by sng84."Love" token, awarded by wilBr."Love" token, awarded by charlie."Burninate" token, awarded by zinar."Love" token, awarded by xrg."Love" token, awarded by billreynish."Like" token, awarded by EAW."Like" token, awarded by Schamph.
Assigned To
None
Authored By
Brecht Van Lommel (brecht)
Jan 17 2020, 5:07 PM

Description

Status: Initial part of milestone 1 merged into master.

Team

Commissioner: ?
Project leader: @Brecht Van Lommel (brecht)
Project members: @Sebastián Barschkis (sebbas) @Jacques Lucke (JacquesLucke)

Description

Big picture: Volumes need their own native datablock type, so Blender can support other use cases like rendering OpenVDB files or procedurally generating volumes.

Use cases:

  • Existing use cases (smoke modifier on mesh).
  • OpenVDB file rendering.
  • Procedurally generating volumes (part of everything nodes).

Design:

  • New Volume datablock (similar to Mesh) with an OpenVDB volume data structure with an arbitrary number of attributes.
  • Rendered by Cycles and Eevee, first as dense volume as we have now, and later with native sparse volume rendering.
  • Volume objects would have modifiers (add resolution detail, fluid domain modifier, ...)

Engineer plan: ?

Work plan

Milestone 1
Time estimate: 10 workdays to complete implementation, 5 for review/docs/bugs.

Completed for merge into Master.

  • New volume object data type, based on OpenVDB grids
    • Support reading OpenVDB files from disk
    • Add New material adds principled volume material
    • Solve problem with not enough FILTER_ID bits
    • Show warning when loading VDB points file
    • Auto rotate object from Y-up to Z-up when file was written by Houdini?
  • Rendering of dense grids
    • Cycles
      • Basic integration
      • Use principle volume as default material if no material exists
      • Support different transform per grid
    • Eevee
      • Basic integration
      • Use principle volume as default material if no material exists
      • Support rotation/shear in transform
      • Shader code generation for arbitrary number of grids
      • Support different transform per grid
    • Workbench
      • Basic integration
      • Control over which grids to display (active grid only by default?)
      • Use workbench object color settings
      • Density scale setting
      • Figure out why density changes with object scale
    • Add option to specify volume density in object space
    • D1777: Cycles: implement per shader volume step size.
  • Viewport drawing and selection of volume bounds
    • Wireframe bounds display of tree nodes (at intermediate or leaf levels)
    • Points display type, that shows one point per coarse/fine node
    • Selection support (based on wireframe bounds?)
  • User interface
    • Open file browser immediately from Add menu
    • VDB / volume file type in the file browser
    • Drag & drop VDB files into 3D viewport
    • Finalize properties editor UI layout
  • Python API
    • Grids
    • Transforms
  • File loading
    • Error reporting
    • On-demand loading
    • Global cache
  • Dependency graph copy-on-write integration
    • Refactor mesh runtime storage to be reusable by other objects types
    • Automatic instancing for objects without modifiers, like meshes
    • Fix runtime backup memory leak
  • Volume frame sequences
    • Settings to use single volume or sequence, start, length, offset
    • Dependency graph integration
    • Auto detection of sequences when loading file

Remaining Tasks that could still happen for 2.83 (but likely not all).

  • Viewport selection by rendering fine boxes when drawing for selection
  • Option to pick byte/half/float precision for 3D texture in Cycles and Eevee

Milestone 2
Time estimate: ?

  • Mantaflow: output OpenVDB files with multiple grids and standard grid names
  • Auto complete grid names in shader nodes
  • Option to use lower resolution volume for viewport display
  • Cycles
    • Optimize mesh bounds creation by using OpenVDB tree
    • Optimize voxel loading by using OpenVDB tree
    • Share 3D images like global volume cache
  • Eevee/workbench
    • Share textures in global volume cache
    • Preserve batch cache through display setting changes
  • Support other applications overwriting OpenVDB files open in Blender (T75901)
  • Volume modifier stack
  • Fluid domain modifier, deprecate modifier for meshes
  • Cycles sparse grid rendering
  • File packing
  • VDB tree/voxels API using pyopenvdb?
  • Workbench: additional viewport display options
  • Grid metadata: show in properties editor

Later

  • Volume modifier nodes
  • Eevee sparse grid rendering
  • Volume edit mode and file saving
  • OpenVDB level sets, clusters, perspective transform

Relevant links:

Revisions and Commits

rB Blender
rB7537cad5761e Volumes: add render settings for volume datablock
rB006025ead0b8 Cycles: support for different 3D transform per volume grid
rB994eb1ec17ef Cycles: support rendering new Volume object type
D6955rBfd53b72871e0 Objects: Eevee and workbench rendering of new Volume, Hair, PointCloud
D6945rBb0a1cf2c9ae6 Objects: add Volume object type, and prototypes for Hair and PointCloud

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Brecht Van Lommel (brecht) added a comment.Feb 7 2020, 12:43 AM

Added this week:

  • Frame sequences for importing animations.
  • Global volume cache to let multiple datablocks automatically share the same grid memory.
  • Simpler UX for importing OpenVDB files, though Add menu and drag & drop.

Next week I plan to make the rendering and viewport drawing more complete.

Zijun Zhou (Eary) added a subscriber: Zijun Zhou (Eary).Feb 7 2020, 5:11 PM

Hello, I am not a programmer, I am just a user (just trying out new things), and I discovered that the search function in the Shader Editor's Shift A menu does not work, any reason for that?

Brecht Van Lommel (brecht) added a comment.Feb 7 2020, 6:13 PM

@Zijun Zhou (Eary), are you saying the search function is broken specifically in a build of the new-object-types branch? I can see it greyed out when there is no material, but once adding one it works for me, same as other object types.

Zijun Zhou (Eary) added a comment.Feb 7 2020, 6:24 PM
In T73201#867641, @Brecht Van Lommel (brecht) wrote:

@Zijun Zhou (Eary), are you saying the search function is broken specifically in a build of the new-object-types branch? I can see it greyed out when there is no material, but once adding one it works for me, same as other object types.

Wait... It's fixed in the latest daily build, so never mind.

AR (arc4g) awarded a token.Feb 10 2020, 5:56 AM
AR (arc4g) added a subscriber: AR (arc4g).
AR (arc4g) removed a subscriber: AR (arc4g).
AR (arc4g) added a subscriber: AR (arc4g).
Dustin Bowser (bowserlm) added a subscriber: Dustin Bowser (bowserlm).Feb 11 2020, 7:20 PM

Downloaded the build from GraphicAll and imported a VDB and everything works in the viewport with Eevee, but when I try to render a frame with Eevee I don't get the expected result. There is no volume.

Brecht Van Lommel (brecht) added a comment.Feb 16 2020, 7:28 PM

iss.vdb is too heavy to load on my GPU as a dense texture, but we can preview it in the viewport with wireframes now. Each box in the Fine display is 8x8x8 voxels.

CoarseFine

Velocity grid display:

Dustin Bowser (bowserlm) added a comment.EditedFeb 18 2020, 6:57 PM

Downloaded the latest build -- rBd58e80b4c6cb -- still no render result in Eevee.

Actually now no render result in Cycles as well.

***Actually looks like I am getting a render result now from Cycles and Eevee. Not sure what changed.

Way awarded a token.Feb 19 2020, 3:20 PM
Ulrich Büch (kologe) added a subscriber: Ulrich Büch (kologe).Feb 22 2020, 10:48 AM

While I assume such things to be sth. like the most low-priority of things to consider with this project, I'd still like to know in how far the new volume-object type could serve as groundwork for future support of pointclouds via OSL as per chapter 7.9 (page 72) in https://github.com/imageworks/OpenShadingLanguage/blob/master/src/doc/osl-languagespec.pdf?

Oh and sorry in advance in case this is the wrong place to ask this. :-)

Brecht Van Lommel (brecht) added a comment.EditedFeb 24 2020, 10:19 PM

Point-cloud sampling in OSL would share no code with the volume object, so basically unrelated. The current pointcloud texture converts point clouds to a volume, but really that's too inefficient.

Silas Opel (Schamph) added a subscriber: Silas Opel (Schamph).Mar 1 2020, 10:10 PM

Importing and rendering or shading vdb work fine but not with sequence. That only works in the viewport but the render is empty.

Still such a great addition to blender. Thanks Brecht for the work on it.

Stefan Werner (swerner) added a comment.Mar 5 2020, 10:22 PM

We're reviewing an in-house patch right now that's per-object (not per-shader) step size as a user parameter, not determined automatically at the moment. Rationale for per-object is that we tend to reuse the same shader for multiple VDB objects, which may have different resolutions and densities.

We might also have a patch for eulerian motion blur soon.

We have some in-house code for CPU-only sparse grids using OpenVDB directly. It's not necessarily pretty and it comes with performance penalties, not sure how useful this will be for the general audience.

Brecht Van Lommel (brecht) added a comment.Mar 5 2020, 10:44 PM

I agree per-object step size makes more sense, I was thinking about changing it to that. Probably also change it from step rate to step size, but still with a way to fill it in automatically.

We wouldn't use OpenVDB directly, since we want it to work on the GPU too, and the OpenVDB implementation in Interpolation.h has poor performance. I'm hoping we can do sparse volume rendering in Cycles for 2.83 since it would be really nice to render complex volumes, but probably I won't have time for it.