Add initial UDIM support to BlenKernel, Image Editor and Cycles
Needs RevisionPublic

Authored by Lukas Stockner (lukasstockner97) on Nov 17 2017, 4:13 PM.

Details

Summary

UDIM is essentially a naming convention for having multiple image tiles in one texture.
The traditional (0,0)-(1,1) UV space is assigned ID 1001, and from there on additional unit
squares are numbered in ascending order with ten tiles per row.
For example, tile 1005 is (4,0)-(5,1) and 1013 is (2,1)-(3,2).

This commit adds initial support for that convention in Blender. Instead of storing all tiles
in a single Image datablock, which would require extensive changes all over Blender, a Image
datablock now can store references to other Image datablocks - that way, the 1001 "master tile"
links to the other tiles.

This allows to handle issues like user counting and linking with the regular methods while
providing an efficient way for detecting whether a Image is a UDIM master tile (checking whether
the tile array pointer is non-NULL) and iterating over the tiles.

In addition to support in the kernel, there also are extensions for other parts that allow them
to handle UDIMs:

  • In the image editor, when the master tile is selected, all other tiles are shown in their locations. Also, a grid with UDIM IDs is shown in the background, which makes it easy to lay out the UVs. This grid can be enabled even if no image is selected currently.
  • The Cycles Image Texture Node automatically handles UDIMs - if you select a master tile, it will show a note in the node to show that the other tiles were detected and load and use them during rendering.
  • In Material mode in the viewport, the Cycles GLSL shaders also support UDIMs.
  • When opening an image with 1001 in its name, Blender will not do the usual frame sequence detection. Instead, it will parse the selected files to check which ones form UDIM textures and assign them to the master tile(s). Additionally, it will check the folder for other matching tiles, which means that just selecting the 1001 tile is enough.

I'm not really sure who to add as a reviewer here, so feel free to add anyone/yourself :)

Diff Detail

Repository
rB Blender
Branch
master
Build Status
Buildable 978
Build 978: arc lint + arc unit
Brecht Van Lommel (brecht) requested changes to this revision.Nov 17 2017, 5:43 PM

For the Cycles implementation, I'd prefer to make this part of the regular image node in the kernel, SVM and OSL. In Blender there are separate code paths for multiview, multilayer, movieclip, .. and it just became very hard to understand due to all the code duplication. It's better to treat regular images more like a 1 tile UDIM.

On the Blender side, my main concern is using an image datablock for each tile. That's going to be potentially messy in the UI datablock lists, asset browsers, etc. Maybe they could be hidden, but then you also have no control over the image settings of each tile. I think that you also want to have the same color space settings for each image for example, and keeping that in sync between all the datablocks is tricky.

I'm not sure about all the consequences, but what I would suggest is to add a udim_tile member in ImageUser, which you can set when acquiring the image buffer. This can then be treated like an image sequence framenr or multiview index. As far as I know the image/movie cache should be able to handle it.

intern/cycles/render/nodes.cpp
232

I suggest to hardcode this to 10. As far as I know this is the standard, so unless you know of applications that create UDIMs with different rules, it's better to stick to that I think.

source/blender/makesrna/intern/rna_image.c
882–887

Same comment here about hardcoding to 10.

source/blender/makesrna/intern/rna_space.c
3061–3065

What is the purpose of this option, why not always draw all the available UDIM rows?

I can imagine that for performance, it is sometimes be useful to draw only some images. Ideally this would be automatically fast with mipmaps and clipping, but that's not so simple. If we need some manually way to speed up drawing, it's not clear to me that specifying the number of rows is the right option, since you may only want to show some tiles at the end instead of the start.

source/blender/nodes/shader/nodes/node_shader_tex_image.c
77–84

This scales poorly with the number of tiles and generates a lot of code. Thinking of a possible better implementation of this it doesn't seem so simple.

OpenGL texture/sampler arrays have limitations on different texture sizes and dynamically indexing the array. Only with bindless textures it seems this can be implemented cleanly, and that's not part of any OpenGL version, it's still an extension, so we need a fallback anyway. Texture atlases would help but that's complicated too.

What could be a little bit more efficient is to compute the tile index and UV coordinate in the first node, and then pass that on. Pseudo code:

node_tex_image_udim_map:
    tile_u = u / 10.0
    tile_v = v / 10.0
    co = vec3(u - tile_u, v - tile_v, tile_u + tile_v * 10.0)
    color = missing_tile_color
node_tex_image_udim:
    if tile_index == co.z:
          color = texImage2D(tex, vec2(co.x, co.y))

An maybe add a comment in the code about the inefficiency of this.

This revision now requires changes to proceed.Nov 17 2017, 5:43 PM

Auto-detection/loading of the sequence sounds like a nice way to go, however I did not see any mention of how multiple images that have the same UDIM-ID will be treated so I thought I'd bring this up to be sure it's on the table with everything else.

Below is a link to a Substance Painter doc on this topic, it looks like Painter and a couple other packages mentioned in a video on that page all use the following naming convention.

(Normally I wouldn't mention other software packages on this site but this is about a standardized exchange format and it's not about copying features so I think it's alright, please let me know if it's not cool :)

meshName_BaseColor.1001
meshName_Normal.1001
meshName_Roughness.1001
meshName_Metallic.1001
meshName_Height.1001
meshName_Emissive.1001
meshName_AmbientOcclusio.1001

I can test the results of this patch with Substance Painter once things around here are ready to go.

Here's the Painter doc page, there is an 8 minute video towards the bottom which shows the transfer between apps in action.

https://support.allegorithmic.com/documentation/display/SPDOC/UDIM

Auto-detection/loading of the sequence sounds like a nice way to go, however I did not see any mention of how multiple images that have the same UDIM-ID will be treated so I thought I'd bring this up to be sure it's on the table with everything else.

I think you might be misunderstanding how UDIM's work? Files like meshName_BaseColor.1001 and meshName_Normal.1001 would continue to be handled as separate images. UDIM is intended to handle files like meshName_BaseColor.1001, meshName_BaseColor.1002, meshName_BaseColor.1003 as one, and those necessarily have a different "UDIM-ID".

Does it require a complete sequence?
Not all channels always start at 1001 and sometimes not all tiles are exported.
What would be the default color for those missing tiles? A user definable color would be welcome.

@Brecht Van Lommel (brecht)

When opening an image with 1001 in its name, Blender will not do the usual frame sequence detection.
Instead, it will parse the selected files to check which ones form UDIM textures and assign them to the master tile(s).
Additionally, it will check the folder for other matching tiles, which means that just selecting the 1001 tile is enough.

Since it wasn't mentioned, I just wanted to be sure pass-types for each tile were taken into account early in case it was overlooked. Things like that cause head-aches.

@Brecht Van Lommel (brecht) You may be correct in saying that I don't understand UDIM's but I think it's likely a miscommunication. The following is a hand-built UDIM setup that I pieced together just now.

I subdivided a cube in Blender then separated each of the 8 corners into UV-Islands. I then moved them one at a time along the U-axis in the UV-Image Editor.

The 1st one I left alone so that it remained in the 0-1 UV-Space.
The 2nd Island was moved to the right using the key sequence (g -> x -> 256) which is default for moving a tile when there is no image being displayed in the UV-Image Editor.
If there was an image being displayed in the UV-Image Editor then moving a tile whold require that the trailing number would be the value of the image dimension for that axis. Moving a 512x512 image one tile to the right uses the keys (g -> x -> 512).

I then exported the model as .fbx using only the "mesh" option and I then loaded the cube into Substance Painter using the UDIM option on the model importer panel.

Now Substance Painter recognized that the model is ready to be painted using multiple image passes across UDIM tiles. Select a tile in the list and start painting or dropping on materials and image layers, whatever you want. From the artist's point-of-view, it's similar to how Blender's Texture Paint Mode works with objects. In Blender the selected object is the only thing you can paint on and in Painter, the selected UDIM-tile is the only tile you can paint on.

Here is the back-and-forth from Blender to Substance Painter and back to Blender to again. cube_BaseColor.1001 is added to cube_BaseColor.1002 and UDIM-ID-1002 has a U-Axis offset of one along the x-Axis. Three is added into those first two exactly the same way except the offset for that tile is two. The fourth image has an offset of three, etc...

Here are some screen-shots of the setup. The resulting UDIM layers in Blender all appear to be aligned just as they are in Substance Painter. The only difference is that cube_BaseColor.1001 through to cube_BaseColor.1008 are the only images loaded in... eight images in total.

This final texture shows the directory that contains all the images passes for the eight UDIM-Tiles. There are 40 in total but normally there would be 48 by deafult. Normally Substance Painter would also print an Emission map to make 48 image maps in total.

That is why I brought up the importance of designing everything around loading 48 images into one material and recognizing the multiple passes for each tile so the Blender Importer can automatically plug them all into the Principled Shader node which is what I assume the Eevee will be using.

The node structure that I showed here is only 1/8th the complexity of the porper UDIM image map setup so far as I understand it. It's a monster and so far as I can tell, that is the only point to you fine programmers so far as this is all concerned. I believe your job here is to make artists happy so they don't have mental breakdowns trying to load in 48 image maps and then wire together 256 nodes every time they want to use the UDIM setup.

Please correct me if I'm wrong about any of this. It would be embarrassing for me if that's the case.( I am doing some guessing here( (I didn't read the spec :) )

Thanks for your time and have fun coding that puzzle!

Here's the .blend file with packed images, they are all 128x128

I think the main confusion is what one set of images consists of - in Substance, afaik all 48 images are considered one texture. However, this patch doesn't care about BaseColor etc. - its only purpose is having the 1001, 1002 etc. images in one node.

Therefore, to use a Substance export in Cycles, you'd need five Image Nodes (one for BaseColor, one for Normal etc.) and a Principled BSDF - in other words, exactly how you'd setup a PBR material if you weren't using UDIMs.

@Lukas Stockner (lukasstockner97) What you said sounds good and user-friendly for artists if I understand correctly. Something like the following image is how I'd imagine a nice layout to work, does this match with what you are saying?

The idea being that a folder with a dump-truck full of images is selected by the user and then the underlying code produces something like the following result in the node editor...

@marc dion (marcclintdion), it will continue to work with image texture nodes, the same as before. There will be no UDIM specific nodes.

The node wrangler addon can already do something similar to what you are thinking of, and will likely be updated to work with UDIM as well.
https://gfycat.com/CarefulWelldocumentedEkaltadeta

@Lukas Stockner (lukasstockner97) What you said sounds good and user-friendly for artists if I understand correctly. Something like the following image is how I'd imagine a nice layout to work, does this match with what you are saying?

The idea being that a folder with a dump-truck full of images is selected by the user and then the underlying code produces something like the following result in the node editor...

No, you have to manually load each channel (basecolor, normal etc) into their own imagetexture node.
You only have to select UDIM 1001 and it automatically reads in all the other tiles from 1001-1008 .
You can certainly script what you want but since every program names their channels differently (sometimes it´s basecolor, sometimes diffuse, sometimes albedo etc.), it is not as straightforward as it sounds.