UDIM: Support virtual filenames for tile sets #92696

New Issue

Jesse Yurkovich · 2021-10-31T23:24:21+01:00

Jesse Yurkovich commented

2021-10-31 23:24:21 +01:00

Context
Texture files belonging to a UDIM tile set all share a particular naming convention which indicates which file belongs to which tile and vice versa. For example, a 3 texture UDIM tile set might look like the following: monster-albedo.1001.png, monster-albedo.1002.png, monster-albedo.1003.png

There are several conventions besides the 4-digit example above. Additionally, the information can technically be placed anywhere within the filename, and it might be ambiguous with other fragments of the filename which might contain things like version numbers, image sequence numbers, or frame numbers.

In order to correctly and reliably handle such conventions, as well as display and reference an entire tile set, most software allows the use of substitution markers/tokens indicating exactly where in the filename to look for and place the UDIM information. Other open initiatives like MaterialX use these tokens and libraries like OIIO support substitutions natively today.

Filenames with these markers/tokens present will be known as "virtual filenames" for the rest of this document.

Problem
Blender does not support "virtual filenames" and instead relies on the existing Image Sequence detection code to find digits somewhere in the filename. Additionally, this processing only occurs during file load/save workflows, leaving importers/exporters to find their own solution to the problem.

This leads to the following issues:

Prevents artists from easily using texture assets produced elsewhere using a different convention
Ambiguity with other numbers in the filename in rare cases
Confusion when trying to save UDIM tile sets due to the disparity between the file name you type vs. what file names are actually used
Places undue burden on importers/exporters like USD which currently need to handle the tokens, by iterating the physical files, themselves
Bugs and complexity as Blender attempts to use concrete filenames with digits when referencing the entire UDIM set. The digits can become out of sync and are generally nonsensical for use as an identifier meant to address the entire UDIM set.

Prior bugs discussing these issues for reference: https:*developer.blender.org/T77989, https:*developer.blender.org/T90535

Example Tokens

Token	Meaning	Support	Example for u-tile of 3 and v-tile of 1
	1001 + u-tile + v-tile * 10	MaterialX / USD / OIIO	`filename.<UDIM>_ver0023.png --> filename.1014_ver0023.png`
and	(0-based) u-tile, v-tile	OIIO	`filename.<u><v>_ver0023.png --> filename.u3v1_ver0023.png`
and	(1-based) u-tile + 1, v-tile + 1	OIIO	`filename.<U><V>_ver0023.png --> filename.u4v2_ver0023.png`
	Equivalent to _	MaterialX	`filename.<UVTILE>_ver0023.png --> filename.u4_v2_ver0023.png`

Note: Blender uses the format but does not support the substitution token. Blender attempts to search and use the first sequence of consecutive numbers it finds within the filename.
Note: Notice that it's generally impossible to guess the correct format if the 0-based or 1-based formats are used within filenames.

MaterialX docs: http://www.materialx.org/assets/MaterialX.v1.38.Spec.pdf (page 18)
OIIO docs: https://openimageio.readthedocs.io/en/master/texturesys.html#udim-texture-atlases
USD docs: https://graphics.pixar.com/usd/docs/api/usd_mtlx_page_front.html

High-Level Goals

Do not regress the existing image load/save workflow
Display the "virtual filename" in the UI whenever a UDIM set is used
Use the "virtual filename" internally inside Blender
Support MaterialX's and tokens at minimum; the other tokens are trivially added after those (if necessary)
Provide importers/exporters an appropriate API for easily handling "virtual filenames"; support USD at a minimum

Approaches and Constraints
The user's natural workflow imposes a variety of opposing constraints here:

Load: As a user I want to be able to select just 1 file in a UDIM set and have Blender load all files from the set (already happens)
Load: As a user I want to select multiple files, from 1 or more UDIM sets, and load all files from each set (already happens)
Load: As a user I need the ability to load UDIM sets which have a different naming convention than what Blender uses today
Save: As a user I want to use 1 filename when saving the entire UDIM set

The crux of this is how and where to allow the user to perform scenarios 3 and 4 while still allowing scenarios 1 and 2 to function correctly.

Scenario 1 and 3 could be solved by providing a secondary UI property (in the n-panel or elsewhere) to specify the filename template. This property, if filled in, would be used instead of guessing. Or we could allow the "virtual filename" to be used directly in the File-Selector to begin with. However, both of these options prevent Scenario 2 since each set will have a different template.
Scenario 2 would need to rely on guessing the format and location of the UDIM information. This is the state today and this can be wrong for the reasons stated prior. So the question then becomes how/where to provide a location and mechanism to "fix" the guess after loading has occurred?
Scenario 3 and 4 require a change in the File-Selector to allow the "<" and ">" characters to be typed at all. These are invalid characters for filenames so we'd have to be sure any errors are properly surfaced to the user at some other point in time.

Proposal
Therefore, the proposal is to follow the below approach:

Because having a secondary UI property in the File-Selector during load/save time does not solve all scenarios, don't do this at all
Improve Blender's ability to guess more cases by default
- Try harder to guess the typical conventions in more cases than what's covered today
Provide a mechanism in the Image Editor space to "fix" image load issues
- Make use of the "Reload" button already available
- If guessing fails, the user can go to the Image Editor, type the proper substitution, and click "Reload"
For loading/saving, allow the "<" and ">" characters to be typed in the filename property of the File-Selector
- Allows cut and paste to just work and it allows them to bypass the "fix" if they are loading 1 UDIM set directly
- As long as we surface proper errors back to the user these characters are probably ok to allow
- Explore only allowing these characters to be typed if we know we're going to be loading/saving Images (disable for other file types)

Example Patch to support and token formats during load/save as well as the "fixup" approach outlined above
Note: This patch is not meant to be final code. It disables the existence check on image load and does not implement the "nicer" apis necessary for importers/exporters. It merely demonstrates basic functionality and marks the major code paths that would have to change.
D13057

[Master -- Out of sync + Non-sensical]

[Patch -- Showing what's possible so far]

**Context** Texture files belonging to a UDIM tile set all share a particular naming convention which indicates which file belongs to which tile and vice versa. For example, a 3 texture UDIM tile set might look like the following: `monster-albedo.1001.png, monster-albedo.1002.png, monster-albedo.1003.png` There are several conventions besides the 4-digit example above. Additionally, the information can technically be placed anywhere within the filename, and it might be ambiguous with other fragments of the filename which might contain things like version numbers, image sequence numbers, or frame numbers. In order to correctly and reliably handle such conventions, as well as display and reference an entire tile set, most software allows the use of substitution markers/tokens indicating exactly where in the filename to look for and place the UDIM information. Other open initiatives like MaterialX use these tokens and libraries like OIIO support substitutions natively today. Filenames with these markers/tokens present will be known as "virtual filenames" for the rest of this document. **Problem** Blender does not support "virtual filenames" and instead relies on the existing Image Sequence detection code to find digits _somewhere_ in the filename. Additionally, this processing only occurs during file load/save workflows, leaving importers/exporters to find their own solution to the problem. This leads to the following issues: - Prevents artists from easily using texture assets produced elsewhere using a different convention - Ambiguity with other numbers in the filename in rare cases - Confusion when trying to save UDIM tile sets due to the disparity between the file name you type vs. what file names are actually used - Places undue burden on importers/exporters like USD which currently need to handle the tokens, by iterating the physical files, themselves - Bugs and complexity as Blender attempts to use concrete filenames with digits when referencing the entire UDIM set. The digits can become out of sync and are generally nonsensical for use as an identifier meant to address the entire UDIM set. Prior bugs discussing these issues for reference: https:*developer.blender.org/T77989, https:*developer.blender.org/T90535 **Example Tokens** | Token | Meaning | Support | Example for u-tile of 3 and v-tile of 1 | | ----- | ---- | ---- | ---- | | <UDIM> | 1001 + u-tile + v-tile * 10 | MaterialX / USD / OIIO | `filename.<UDIM>_ver0023.png --> filename.1014_ver0023.png` | | and <v> | (0-based) u-tile, v-tile | OIIO | `filename.<v>_ver0023.png --> filename.u3v1_ver0023.png` | | and <V> | (1-based) u-tile + 1, v-tile + 1 | OIIO | `filename.<V>_ver0023.png --> filename.u4v2_ver0023.png` | | <UVTILE> | Equivalent to _<V> | MaterialX | `filename.<UVTILE>_ver0023.png --> filename.u4_v2_ver0023.png` | Note: Blender uses the <UDIM> format but does not support the substitution token. Blender attempts to search and use the first sequence of consecutive numbers it finds within the filename. Note: Notice that it's generally impossible to guess the correct format if the 0-based or 1-based formats are used within filenames. MaterialX docs: http://www.materialx.org/assets/MaterialX.v1.38.Spec.pdf (page 18) OIIO docs: https://openimageio.readthedocs.io/en/master/texturesys.html#udim-texture-atlases USD docs: https://graphics.pixar.com/usd/docs/api/usd_mtlx_page_front.html **High-Level Goals** - Do not regress the existing image load/save workflow - Display the "virtual filename" in the UI whenever a UDIM set is used - Use the "virtual filename" internally inside Blender - Support MaterialX's <UDIM> and <UVTILE> tokens at minimum; the other tokens are trivially added after those (if necessary) - Provide importers/exporters an appropriate API for easily handling "virtual filenames"; support USD at a minimum **Approaches and Constraints** The user's natural workflow imposes a variety of opposing constraints here: 1) Load: As a user I want to be able to select just 1 file in a UDIM set and have Blender load all files from the set (already happens) 2) Load: As a user I want to select multiple files, from 1 or more UDIM sets, and load all files from each set (already happens) 3) Load: As a user I need the ability to load UDIM sets which have a different naming convention than what Blender uses today 4) Save: As a user I want to use 1 filename when saving the entire UDIM set The crux of this is **how and where** to allow the user to perform scenarios 3 and 4 while still allowing scenarios 1 and 2 to function correctly. - Scenario 1 and 3 could be solved by providing a secondary UI property (in the n-panel or elsewhere) to specify the filename template. This property, if filled in, would be used instead of guessing. Or we could allow the "virtual filename" to be used directly in the File-Selector to begin with. However, both of these options prevent Scenario 2 since each set will have a different template. - Scenario 2 would need to rely on guessing the format and location of the UDIM information. This is the state today and this can be wrong for the reasons stated prior. So the question then becomes how/where to provide a location and mechanism to "fix" the guess after loading has occurred? - Scenario 3 and 4 require a change in the File-Selector to allow the "<" and ">" characters to be typed at all. These are invalid characters for filenames so we'd have to be sure any errors are properly surfaced to the user at some other point in time. **Proposal** Therefore, the proposal is to follow the below approach: - Because having a secondary UI property in the File-Selector during load/save time does not solve all scenarios, don't do this at all - Improve Blender's ability to guess more cases by default - Try harder to guess the typical conventions in more cases than what's covered today - Provide a mechanism in the Image Editor space to "fix" image load issues - Make use of the "Reload" button already available - If guessing fails, the user can go to the Image Editor, type the proper substitution, and click "Reload" - For loading/saving, allow the "<" and ">" characters to be typed in the filename property of the File-Selector - Allows cut and paste to just work and it allows them to bypass the "fix" if they are loading 1 UDIM set directly - As long as we surface proper errors back to the user these characters are probably ok to allow - Explore only allowing these characters to be typed if we know we're going to be loading/saving Images (disable for other file types) **Example Patch** to support <UDIM> and <UVTILE> token formats during load/save as well as the "fixup" approach outlined above Note: This patch is not meant to be final code. It disables the existence check on image load and does not implement the "nicer" apis necessary for importers/exporters. It merely demonstrates basic functionality and marks the major code paths that would have to change. [D13057](https://archive.blender.org/developer/D13057) [Master -- Out of sync + Non-sensical] ![master-baseline.png](https://archive.blender.org/developer/F11646716/master-baseline.png) [Patch -- Showing what's possible so far] ![patch-load-multiple.png](https://archive.blender.org/developer/F11646728/patch-load-multiple.png) ![patch-image-viewer.png](https://archive.blender.org/developer/F11646731/patch-image-viewer.png) ![patch-save.png](https://archive.blender.org/developer/F11646736/patch-save.png)

Jesse Yurkovich commented

2021-10-31 23:24:21 +01:00

Added subscriber: @deadpin

Paul Larson commented

2021-10-31 23:51:18 +01:00

Added subscriber: @GeorgiaPacific

Jesse Yurkovich self-assigned this 2021-11-09 09:41:22 +01:00

Jesse Yurkovich commented

2021-11-09 09:41:22 +01:00

Added subscribers: @LukasStockner, @ideasman42, @brecht

Jesse Yurkovich commented

2021-11-09 09:41:22 +01:00

@LukasStockner, @brecht, @ideasman42 : I'd like your input on this task before I go further. Feedback on the high-level goals, the approach I outlined, as well as any ideas you might have for this feature would be appreciated.

Jesse Yurkovich commented

2021-11-30 08:40:27 +01:00

Gentle nudge for @brecht and @campbellbarton. I'd like to make progress on this for 3.1 but need some feedback on the general approach before solidifying the patch above any further. I'm available here and in chat for any clarifying questions or feedback.

Brecht Van Lommel commented

2021-11-30 18:53:57 +01:00

Added subscriber: @mont29

Brecht Van Lommel commented

2021-11-30 18:53:57 +01:00

Sorry for the late reply.

Improve Blender's ability to guess more cases by default

Try harder to guess the typical conventions in more cases than what's covered today

Provide a mechanism in the Image Editor space to "fix" image load issues

Make use of the "Reload" button already available

If guessing fails, the user can go to the Image Editor, type the proper substitution, and click "Reload"

All this sounds good.

For loading/saving, allow the "<" and ">" characters to be typed in the filename property of the File-Selector

Allows cut and paste to just work and it allows them to bypass the "fix" if they are loading 1 UDIM set directly

As long as we surface proper errors back to the user these characters are probably ok to allow

Explore only allowing these characters to be typed if we know we're going to be loading/saving Images (disable for other file types)

This sounds reasonable to support by itself, but in the screenshot I see the image file path has tokens in it. That goes beyond the file browser, and affects add-ons, file packing, find missing files and other types of generic file path handling in Blender.

Can we put UDIM tokens in image.filepath, and what are the implications of that for the rest of Blender and add-ons? Or do we perhaps store a separate filename with tokens? It could be argued that in order for add-ons and built-in operators to fully support UDIM, they need to be aware of it regardless and storing the filename with tokens separately would still require them to be updated in some other way. So then we might as well bite the bullet? And then we need to look through the various places that use this filepath to asses what needs to be done.

@mont29 might also have an opinion on this. It's rather annoying to have to deal with file paths that are not really one filepath. Though we already have this with e.g. image or alembic sequences, just that the filepath normally actually exists on disk as the first frame in the sequence.

Or are you proposing to make a more limited change that only affects the file browser? If so I don't really understand what the plan is for e.g. importers/exporters.

Sorry for the late reply. >* Improve Blender's ability to guess more cases by default > * Try harder to guess the typical conventions in more cases than what's covered today >* Provide a mechanism in the Image Editor space to "fix" image load issues > * Make use of the "Reload" button already available > * If guessing fails, the user can go to the Image Editor, type the proper substitution, and click "Reload" All this sounds good. > * For loading/saving, allow the "<" and ">" characters to be typed in the filename property of the File-Selector > * Allows cut and paste to just work and it allows them to bypass the "fix" if they are loading 1 UDIM set directly > * As long as we surface proper errors back to the user these characters are probably ok to allow > * Explore only allowing these characters to be typed if we know we're going to be loading/saving Images (disable for other file types) This sounds reasonable to support by itself, but in the screenshot I see the image file path has tokens in it. That goes beyond the file browser, and affects add-ons, file packing, find missing files and other types of generic file path handling in Blender. Can we put UDIM tokens in `image.filepath`, and what are the implications of that for the rest of Blender and add-ons? Or do we perhaps store a separate filename with tokens? It could be argued that in order for add-ons and built-in operators to fully support UDIM, they need to be aware of it regardless and storing the filename with tokens separately would still require them to be updated in some other way. So then we might as well bite the bullet? And then we need to look through the various places that use this filepath to asses what needs to be done. @mont29 might also have an opinion on this. It's rather annoying to have to deal with file paths that are not really one filepath. Though we already have this with e.g. image or alembic sequences, just that the filepath normally actually exists on disk as the first frame in the sequence. Or are you proposing to make a more limited change that only affects the file browser? If so I don't really understand what the plan is for e.g. importers/exporters.

Jesse Yurkovich commented

2021-12-01 02:23:49 +01:00

@brecht Good point. You're correct that the Image.filepath field would be virtual for IMA_SRC_TILED images now. Any internal/external code that handles tiled images, AND makes use of the filepath, would change in some way. If we add another field, and keep the original filepath referencing a concrete path, downstream code would still have to change as you mention to handle things correctly[1].

This is the reason I had to start the patch, to see what actually breaks. If I remember right, the only thing that immediately broke, and still needs addressing, is the file existence check you allude to on Image load (BKE_image_load). It's skipped wholesale in the prototype but we'd have to do something different for real to not regress normal image sources. The general workflow for UV editing/painting/rendering seems fine but that is the extant of what I've tested to prove out the prototype.

Perhaps we're also lucky in that, today, UDIMs do not support packing...

For importers/exporters, the plan is to provide them an API that would get them out of the business of using the filepath directly as much as possible.
For reference you can see Michael's USD code that's in review which has to do things the hard way and is duplicative in some form: https://developer.blender.org/D13297 (usd_reader_material.cc)
Ideally, there would be a somewhat general way for USD to exchange with blender the virtual paths directly. So yes the proposal does go beyond the File-Selector / filepath issues and extends to the I/O area but I do not have an API example to provide just yet.

The filepath can get out of sync today in all blender versions (my first screenshot) so I'm honestly unsure what edge cases are subtly broken already if the "filepath actually exists on disk" invariant is broken.

@brecht Good point. You're correct that the `Image.filepath` field would be virtual for IMA_SRC_TILED images now. Any internal/external code that handles tiled images, AND makes use of the filepath, would change in some way. If we add another field, and keep the original filepath referencing a concrete path, downstream code would still have to change as you mention to handle things correctly[1]. This is the reason I had to start the patch, to see what actually breaks. If I remember right, the only thing that immediately broke, and still needs addressing, is the file existence check you allude to on Image load (BKE_image_load). It's skipped wholesale in the prototype but we'd have to do something different for real to not regress normal image sources. The general workflow for UV editing/painting/rendering seems fine but that is the extant of what I've tested to prove out the prototype. Perhaps we're also lucky in that, today, UDIMs do not support packing... For importers/exporters, the plan is to provide them an API that would get them out of the business of using the filepath directly as much as possible. For reference you can see Michael's USD code that's in review which has to do things the hard way and is duplicative in some form: https://developer.blender.org/D13297 (usd_reader_material.cc) Ideally, there would be a somewhat general way for USD to exchange with blender the virtual paths directly. So yes the proposal does go beyond the File-Selector / filepath issues and extends to the I/O area but I do not have an API example to provide just yet. - [x] The filepath can get out of sync today in all blender versions (my first screenshot) so I'm honestly unsure what edge cases are subtly broken already if the "filepath actually exists on disk" invariant is broken.

Brecht Van Lommel commented

2021-12-01 15:29:33 +01:00

Putting UDIM tokens in Image.filepath I think is reasonable then, the practical impact does not seem that big.

For built-in operators, I can only think of image open and save operators, and find missing files. For find missing files we could perhaps add an option for the path iterator to return one real filename from the first tile.

Putting UDIM tokens in `Image.filepath` I think is reasonable then, the practical impact does not seem that big. For built-in operators, I can only think of image open and save operators, and find missing files. For find missing files we could perhaps add an option for the path iterator to return one real filename from the first tile.

Bastien Montagne commented

2021-12-02 09:46:11 +01:00

I would rather really, really prefer to not allow forbidden path characters in our file paths. For any reasons. Doing otherwise means you have to check the entire filepath handling code to ensure it is resilient to those invalid characters, it does not 'fix' them in some places (see BLI_filename_make_safe() in BLI_path_util, that change there in D13057 is an absolute red flag for me), while still not allowing them in all-but-special-allowed-cases...

This would be opening a can of worms, and adding a significant amount of complexity to our path handling system.

So I would advocate for using a specific dedicated field for those tokens, with a dedicated proper API for it (probably at BKE_image level), and not make generic filepath handling have to deal with such special corner case all over the place.

I would rather really, really prefer to not allow forbidden path characters in our file paths. For any reasons. Doing otherwise means you have to check the entire filepath handling code to ensure it is resilient to those invalid characters, it does not 'fix' them in some places (see `BLI_filename_make_safe()` in `BLI_path_util`, that change there in [D13057](https://archive.blender.org/developer/D13057) is an absolute red flag for me), while still not allowing them in all-but-special-allowed-cases... This would be opening a can of worms, and adding a significant amount of complexity to our path handling system. So I would advocate for using a specific dedicated field for those tokens, with a dedicated proper API for it (probably at `BKE_image` level), and not make generic filepath handling have to deal with such special corner case all over the place.

Brecht Van Lommel commented

2021-12-02 18:06:27 +01:00

I think there's two levels to this. I agree that we should not be making changes at the BLI_path level. The only thing that's really relevant to this in BLI_path seems to be BLI_filename_make_safe and BLI_path_make_safe. We could keep those unchanged. < and > even work fine in filenames on Linux and macOS, and we can load and save e.g. .blend files with such characters. The reason these functions were modified in D13057 seems to be for the file browser path validation, but we can instead handle the UDIM case at the file browser level.

The other thing is if we allow these characters in Image.filepath, and I think when it comes to that it is cleaner to do that rather than store them separately. We have to deal with them one way or another, but in practice the amount of affected code seem small. I'm guessing the idea with UDIM was also that such tokens generally pass through file path handling in various apps and libraries ok.

I think there's two levels to this. I agree that we should not be making changes at the `BLI_path` level. The only thing that's really relevant to this in `BLI_path` seems to be `BLI_filename_make_safe` and `BLI_path_make_safe`. We could keep those unchanged. `<` and `>` even work fine in filenames on Linux and macOS, and we can load and save e.g. .blend files with such characters. The reason these functions were modified in [D13057](https://archive.blender.org/developer/D13057) seems to be for the file browser path validation, but we can instead handle the UDIM case at the file browser level. The other thing is if we allow these characters in `Image.filepath`, and I think when it comes to that it is cleaner to do that rather than store them separately. We have to deal with them one way or another, but in practice the amount of affected code seem small. I'm guessing the idea with UDIM was also that such tokens generally pass through file path handling in various apps and libraries ok.

Jesse Yurkovich commented

2021-12-05 08:09:16 +01:00

I updated the patch to address the < and > issue for the File-Selector, indeed the only reason BLI_filename_make_safe had to change was because of this particular code path. All other existing users of it should now be unaffected at least.

All tests now pass as well.

I updated the patch to address the `<` and `>` issue for the File-Selector, indeed the only reason `BLI_filename_make_safe` had to change was because of this particular code path. All other existing users of it should now be unaffected at least. All tests now pass as well.

Pratik Borhade commented

2022-01-19 09:50:36 +01:00

Changed status from 'Needs Triage' to: 'Confirmed'

Bastien Montagne commented

2022-01-19 09:55:30 +01:00

Changed status from 'Confirmed' to: 'Resolved'

Bastien Montagne closed this issue

2022-01-19 09:55:30 +01:00

Bastien Montagne commented

2022-01-19 09:55:30 +01:00

This design task has been resolved by D13057/rB180b66ae8a1f.

This design task has been resolved by [D13057](https://archive.blender.org/developer/D13057)/rB180b66ae8a1f.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

UDIM: Support virtual filenames for tile sets #92696