Expand Voronoi node functionality in Cycles
Needs ReviewPublic

Authored by Michel Anders (varkenvarken) on Aug 14 2015, 8:01 PM.
Tags
Tokens
"Love" token, awarded by Eranekao."Love" token, awarded by lordodin."Like" token, awarded by bubblebobble."Love" token, awarded by PrimeNine."Love" token, awarded by SynaGl0w."Love" token, awarded by monio."Love" token, awarded by bliblubli.

Diff Detail

Michel Anders (varkenvarken) retitled this revision from to Expand Voronoi node functionality in Cycles.Aug 14 2015, 8:01 PM
Michel Anders (varkenvarken) updated this object.

I see two problems here:

  • In general, I am a bit hesitant to extend our procedural textures this at this point, I rather see the CUDA kernel split happen first or at least have some CUDA runtime compilation, to compile more optimal code.
  • Cycles Texture Nodes are pretty understandable atm, and we only have Color and Fac outputs. Adding a "F2" output seems confusing?

I don't really want to duplicate Blender Internals options mess.

intern/cycles/render/nodes.cpp
936

Capital S in "Squared".

source/blender/makesrna/intern/rna_nodetree.c
3798

Capital S in "Squared", general rule in Blenders UI.

@Thomas Dinges (dingto): I have no opinion on the kernel split issue because it is difficult for me to have any idea of the impact on individual node code, but I agree that on the second point that we wouldn't want to have the profusion of options that BI has.
However, for most people (at least in my neighbourhood :-) F2 is perfectly logical when dealing with Voronoi textures and it would in fact be more logical to rename the Fac output to F1 but I think that would break existing noodles that use the current voronoi node, right?
Another approach might be to create a new 'Crackle' node, which delivers just F2 - F1. After all the goal of this exercise is to provide artists with exactly that texture, not to complicate existing nodes. I would be happy to provide a patch for such a node if that would be considered cleaner (but it would add an extra entry to the the list of textures in the node menu, which is a bit of clutter in itself)

Because quite a few people are convinced it would be a useful addition I went along to give it a second try :-)

To reduce the clutter I created a new patch that just offers a new VoronoiCrackle node, with the same output socket configuration as the other patterns (i.e. just Color and Fac). I included a corresponding .osl node as well, so this one compiles/works on GPU, CPU and CPU/OSL. I removed the color/intensity option as it is rather useless but I did replace it

I don't know how to add another patch to an existing patch task, but if someone is interested I have made it available on dropbox.
https://www.dropbox.com/s/guwqkdhsdbf684p/voronoi_crackle.diff?dl=0

Redo the submission, but instead of creating a new differential, pick this one from the dropdown. :)

I agree, I like the crackle option
Instead of creating a new node or new output for a node is it worth considering a new input/option for the node?
That is having an input/option that selects between f1, f2 or f2-f1 ?
Would that be more acceptable?

Michel Anders (varkenvarken) removed rB Blender as the repository for this revision.

I think I managed to update the diff now :-)

It now adds a Voronoi Crackle nore with output sockets that match the other Cycles pattern nodes (i.e. just Color and Fac).

As I mentioned before this patch contains the GPU, CPU and CPU+OSL version of the node. The .osl code is rather lengthy because the loops for each metric are written out to prevent doing string compares 27x per sample.

Could you use a function pointer in node_voronoi_crackle_texture.osl to avoid repeating the loops?
That way the string comparison sets the function pointer to whatever is used for that type
And then there is only 1 set of loops with the function pointer being evaluated
Or are function pointers frowned on?

I think the suggestion by SynaGl0w ( http://blenderartists.org/forum/showthread.php?378351-Extending-the-Voronoi-texture-node-in-Cycles&p=2922699&viewfull=1#post2922699 ) makes sense.

If extending the current Voronoi node along these lines would be acceptable I would be happy to spend the time. Just let me know.

Just to make this clear, this enhancement would mean:

  • an extension/patch to the current Voronoi node in Cycles,
  • that would maintain the output socket config of all current Cycles pattern nodes (Color + Fac)
  • that would be fully backward compatible (i.e. would keep the Cells/Intensity dropdown, and have Distance Square as the default metric)
  • that would be extended with a 'Output' dropdown (that would provide a choice of F1 (default) F2, F2 - F1 and may be some extra if that makes sense)
  • that would be extended with a 'Metric' dropdown (with Distance Squared (default), Distance, Manhattan, Chebychev and maybe the Minkowskis)

The node would work for GPU, CPU and CPU + OSL.

@Martin Norris (MartinNorris): I have to check but AFAIK function pointers are not possible in OSL (if code bloat is a big issue we can use #defines)

PS> I'll throw in some decent user documentation on the blender.org wiki for free :-)

We can't really extent built-in textures infinitely, this is something what we've got OSL for. If new voronoi texture is indeed giving better artistic control for users we should consider replacing an existing node.

Oh my...

Sergey, please understand that this patch is not niche-extra-feature but bringing back BASIC feature present in most packages/renderers.

I have been intensively learning procedural texturing since about 3 years, this include scripting and non-shader based approaches like Substance Designer. Those voronoi options are biggest shortcoming in Cycles procedurals and I can assure you this is not only my whim but opinion common for many artists. Just ask them!
Thats odd we have things like "Magic" texture, which is practically useless for texture building, but we can't use Voronoi F2 which is one of most common and useful procedural.
Nobody told to modelers that Inset feature won't be implemented as standard tool because we have Python API for that. There is no exaggeration, voronoi patterns are really that popular.

https://www.youtube.com/watch?v=gSB-v4upJzI
This tutorial shows there is big demand for texture like that (one of most watched video from this tutor). It also shows that achieving simple effect like this is ridiculously hard in Cycles. I bet after committing this patch to Master there will be flood of tutorials how to create:

  • Cracks
  • Scratches
  • Dirt
  • Plaster Walls
  • Leather
  • Skin
  • Skin with veins
  • Leaves
  • Alien/Aquatic materials
  • Sci-fi patterns
  • Probably 1000 more because of universality of Voronoi patterns in real world.

I understand that You may not see how those options can be useful but We - material creators, quite large group of Blender users - do.
Some of us waited for this for years. Please, don't kill this opportunity.

If the texture is implemented in a way which isn't introducing issues into all GPUs then i'm fine having the addition, but even then please refer to the second part of my original comment (which ok, isn't so much verbosely explained).

We already have Voronoi texture node which support F1 color cells and intensity outputs. To me it makes more sense to extend that node with F2F1 intensity/coloring (with some nice friendly name).

Hi,

I think the BA post that @Michel Anders (varkenvarken) links to is exactly the way to go
Adding options on the node does not change anything that already uses the node
And neatly encapsulates the new functionality so it is just an elaboration on an existing node
And compared to, for example, the brick texture or Musgrave texture, Voronoi is sadly restricted

Best regards

Martin

Misunderstood your post Sergey. Phew... ;)
I also agree that SynaGl0w proposition that Michel linked will work best for this features.

ok guys, i see some agreement starting here ;-) I'll start on a new patch that will extend the current voronoi node without breaking existing stuff . I'll do my best to make it run on cpu, cpu+osl and gpu (nvidia + amd) and then submit it for a thorough review.

Michel Anders (varkenvarken) set the repository for this revision to rB Blender.

Ok, I extended the node as discussed previously. A few remarks:

  • compatibility: I compared the output for the defaults with the output of the previous version and the results were pixel-by-pixel identical (both on GPU, CPU and OSL; tested by taking the output and subtracting the images). Also, loading older scenes retains the correct settings for the node,
  • hardware compatibility: I tested the code on linux-64 (ubuntu), with Intel I7/ Nvidia GTX970 and verified that GPU, CPU and OSL code works. I do not have access to an AMD card though, so I could not test that
  • performance: The code is slightly more complex, mainly because it has to differentiate between different metrics and had to shuffle more neighbor values around. The initial penalty for the existing functionality was about 20% or so for GPU and CPU (less for OSL). I did some trivial optimizations, reducing the penalty to about 10% or so (on my system, YMMV). Additional optimizations might be possible but would complicate code (see also next point) so additional efforts before a review would be wasteful I think, although especially the OSL version might benefit from some additional love.
  • code bloat: Because we have (in the svm code) already an SSE and a non-SSE version there is a fair amount of code duplication already, which is one of the reason not to spend to much time on optimizations. A large part of the difference between the SSE and non-SSE code could be hidden by wrapping large parts of the code in #define'd macros but I not a fan of that as it makes code harder to read.
  • style: I did my best to duplicate the style used in the rest of the code, but as I am new to this part of Blender there might be some style glitches

Anyway, I am open to C&C and also willing to spend some more time to see if further optimization is possible,

Only did some initial glance over the patch.

Am i right you only tested you particular GPU? If so, that's not the way to test GPU support, you at least have to compile all the kernels. And still, 32bit CUDA kernels are already broken. Those are to be repaired first and then re-verified with the patch applied.

Also, seems you didn't check OpenCL kernels?

intern/cycles/kernel/shaders/node_voronoi_texture.osl
20 ↗(On Diff #4976)

This shouldn't be needed, all the string-based comparisons are being unfolded at compile time.

intern/cycles/kernel/svm/svm_voronoi.h
153

It is guaranteed to be CPU here, meaning distance can become a functor, avoiding switch statement for each of the iterations.

source/blender/nodes/shader/nodes/node_shader_tex_voronoi.c
35

What exactly this output is and can it be named in more consistent to other nodes way?

@Sergey Sharybin (sergey): I reconfigured CMake to generate the .cubin files (instead of generating them the first time the GPU was selected) and verified that a number of kernels was generated : sm_20, sm_21, sm_30, sm_35 and sm_50, both the plain and the one marked as experimental. (a GTX970 supports sm_52 as well but I didn't upgrade the cuda driver on my Ubuntu system so I had to exclude sm_52 from the make files). The make succeeded cleanly.

I now also reran the compiled Blender with 'export CYCLES_OPENCL_TEST=ALL'. It compiled without problem and the output (when I selected the OPENCL device in the user preferences) looks fine.

I don't understand your remark about 32bit cuda kernels. Is that something I can help with? Or do you mean they have to be fixed and then later my patch should be verified against them?

re: node_voronoi_texture.osl, string comparison unrolling, : you might be right, but i did measure a (very small) performance difference. I'll double check and like I said, especially the OSL version might benefit from some additional attention (for example, unlike the GPU/CPU variants it does optimize for less than 4 neighbor situations).

re: svm_voronoi.h, functor : no problem I think, although the performance gain might be small

re: node_shader_tex_voronoi.c, name of output : I think you mean the "E" socket, which is an input socket. It's the Minkovski exponent for the generalized Minkovski metric. I could rename it to 'Minkovski Exponent' (like in the BI texture node).

The issue with 32bit CUDA is that current kernel can not be compiled for all shading compute models with 32bit bitness. Don't think there's much help needed here, just need to find some workarounds to reduce registers pressure in there.

As for OSL -- i'd rather see this solved in the OSL itself. It's somewhat a whole point of OSL about optimizing huge shader networks without manually obscuring the code.

As for exponent input and exposing all possible formulas -- i'm inconvinced we need all of them. For example:

  • Squared distance is different from actual distance by just intensity/contrast.
  • Is Manhattan really so commonly used by artists?
  • Minkowsi and Minkowski 4 has barely noticeable difference
  • Minkowski with high exponent becomes Chebyshev

I'd really suggest finding a minimal basis common for majority of artists and don't expose all weird and wonderful settings to a built-in nodes. And for those who really need exact formulas to work we can eventually provide more comprehensive OSL nodes as a part of default template library.

Manhattan F2-F1 is very popular in conjunction with mapping shifting by various noises to produce organic structures. Plain Manhattan can be used for nice technical patterns but f2-f1 is used most.

Oh my...

Sergey, please understand that this patch is not niche-extra-feature but bringing back BASIC feature present in most packages/renderers.

I have been intensively learning procedural texturing since about 3 years, this include scripting and non-shader based approaches like Substance Designer. Those voronoi options are biggest shortcoming in Cycles procedurals and I can assure you this is not only my whim but opinion common for many artists. Just ask them!
Thats odd we have things like "Magic" texture, which is practically useless for texture building, but we can't use Voronoi F2 which is one of most common and useful procedural.
Nobody told to modelers that Inset feature won't be implemented as standard tool because we have Python API for that. There is no exaggeration, voronoi patterns are really that popular.

https://www.youtube.com/watch?v=gSB-v4upJzI
This tutorial shows there is big demand for texture like that (one of most watched video from this tutor). It also shows that achieving simple effect like this is ridiculously hard in Cycles. I bet after committing this patch to Master there will be flood of tutorials how to create:

  • Cracks
  • Scratches
  • Dirt
  • Plaster Walls
  • Leather
  • Skin
  • Skin with veins
  • Leaves
  • Alien/Aquatic materials
  • Sci-fi patterns
  • Probably 1000 more because of universality of Voronoi patterns in real world.

    I understand that You may not see how those options can be useful but We - material creators, quite large group of Blender users - do. Some of us waited for this for years. Please, don't kill this opportunity.

I happen to be the author of the tutorial mentioned here, and to backup what has been said, it is certainly very exciting to have this voronoi cracks style at an artists fingertips. I also presented another tutorial at the conference https://www.youtube.com/watch?v=3vlyT3UNmFs which shows that introducing 'tiling' options for the noise and cracks would be awesome too. To produce this in cycles nodes quickly hits the SVM stack space limit and then you lose the complexity really needed for many believable texture surfaces.

I realise OSL can get around some of these limitations but then you lose the speed of the GPU (plus a bit confusing for artists to create).

Anyways, I was very excited to see this and wanted to share my support and will certainly be making some tutorials about this when it gets out there! :)

Aidy.