Issues with HDRIs and render result pixel values that exceed half float limits #98575

New Issue

Christoph Lohr · 2022-06-03T14:06:53+02:00

Christoph Lohr commented

2022-06-03 14:06:53 +02:00

System Information
Operating system: Windows-10-10.0.22000-SP0 64 Bits
Graphics card: NVIDIA GeForce RTX 3070/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 512.15

Blender Version
Broken: version: 3.1.2, branch: master, commit date: 2022-03-31 17:40, hash: cc66d1020c
(Also tested in latest 3.3 Alpha)

Short description of error
Some lookdev HDRI's give the scene a colored tint, that also does not disappear after lowering the strength

Exact steps for others to reproduce the error
I imported this HDRI to preview lighting in my Eevee scene: https://polyhaven.com/a/lenong_2

At first, this looks okay in blender:

But after lowering the strength, you will notice that there is a yellow light that should not be there, as well as a white spot from where the sun used to be, even if strength=0

I found a workaround by adjusting the exposure of the HDRi in advance by -1 in affinity photo. This makes the image only a tiny bit darker, but the end result is as you would expect it.
No yellow light, no sun.

I tried this with another HDRi from HDRIhaven (from another creator, this one specifically https://polyhaven.com/a/ehingen_hillside) and the issue also existed there.

Here is the blend file for testing:
HDRI_Test.blend
And the HDRI fixed by me
lenong_2_1k_fixed.hdr

**System Information** Operating system: Windows-10-10.0.22000-SP0 64 Bits Graphics card: NVIDIA GeForce RTX 3070/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 512.15 **Blender Version** Broken: version: 3.1.2, branch: master, commit date: 2022-03-31 17:40, hash: `cc66d1020c` (Also tested in latest 3.3 Alpha) **Short description of error** Some lookdev HDRI's give the scene a colored tint, that also does not disappear after lowering the strength **Exact steps for others to reproduce the error** I imported this HDRI to preview lighting in my Eevee scene: https://polyhaven.com/a/lenong_2 At first, this looks okay in blender: ![Normal_bright.png](https://archive.blender.org/developer/F13130576/Normal_bright.png) But after lowering the strength, you will notice that there is a yellow light that should not be there, as well as a white spot from where the sun used to be, even if strength=0 ![Normal_dark.png](https://archive.blender.org/developer/F13130583/Normal_dark.png) I found a workaround by adjusting the exposure of the HDRi in advance by -1 in affinity photo. This makes the image only a tiny bit darker, but the end result is as you would expect it. No yellow light, no sun. ![Fixed_dark.png](https://archive.blender.org/developer/F13130593/Fixed_dark.png) I tried this with another HDRi from HDRIhaven (from another creator, this one specifically https://polyhaven.com/a/ehingen_hillside) and the issue also existed there. Here is the blend file for testing: [HDRI_Test.blend](https://archive.blender.org/developer/F13130598/HDRI_Test.blend) And the HDRI fixed by me [lenong_2_1k_fixed.hdr](https://archive.blender.org/developer/F13130600/lenong_2_1k_fixed.hdr)

Christoph Lohr commented

2022-06-03 14:06:53 +02:00

Added subscriber: @Thane5

blender-admin commented

2022-06-03 14:06:53 +02:00

#101652 was marked as duplicate of this issue

blender-admin commented

2022-06-03 14:06:53 +02:00

#101186 was marked as duplicate of this issue

blender-admin commented

2022-06-03 14:06:53 +02:00

#98959 was marked as duplicate of this issue

blender-admin commented

2022-06-03 14:06:53 +02:00

#98780 was marked as duplicate of this issue

Jesse Yurkovich commented

2022-06-06 03:56:50 +02:00

Added subscriber: @deadpin

Jesse Yurkovich commented

2022-06-06 03:56:50 +02:00

I can confirm that "something" is wrong for both Eevee and Cycles, but not for version 3.1.2, and not exactly matching the problem described.

Currently using:
Operating system: Windows-10-10.0.19044-SP0 64 Bits
Graphics card: NVIDIA GeForce GTX 1070 with Max-Q Design/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 511.09

I was not able to reproduce the "yellow" problem in any version using Strengths of 0.1, 0.01, or 0.
However, I am observing black pixels for the overexposed portions of the sun in 3.2 and 3.3 that wasn't present in 3.1.2.

.	3.1.2	3.2 beta	3.3 alpha
Eevee
Cycles

I can confirm that "something" is wrong for both Eevee and Cycles, but not for version 3.1.2, and not exactly matching the problem described. Currently using: Operating system: Windows-10-10.0.19044-SP0 64 Bits Graphics card: NVIDIA GeForce GTX 1070 with Max-Q Design/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 511.09 - I was not able to reproduce the "yellow" problem in any version using Strengths of 0.1, 0.01, or 0. - However, I am observing black pixels for the overexposed portions of the sun in 3.2 and 3.3 that wasn't present in 3.1.2. | . | 3.1.2 | 3.2 beta | 3.3 alpha | | --- | --- | --- | --- | | Eevee | ![issue_3.1.2-eevee.png](https://archive.blender.org/developer/F13133023/issue_3.1.2-eevee.png) | ![issue_3.2-eevee.png](https://archive.blender.org/developer/F13133025/issue_3.2-eevee.png) | ![issue_3.3-eevee.png](https://archive.blender.org/developer/F13133027/issue_3.3-eevee.png) | | Cycles | ![issue_3.1.2-cycles.png](https://archive.blender.org/developer/F13133030/issue_3.1.2-cycles.png) | ![issue_3.2-cycles.png](https://archive.blender.org/developer/F13133032/issue_3.2-cycles.png) | ![issue_3.3-cycles.png](https://archive.blender.org/developer/F13133034/issue_3.3-cycles.png) |

Omar Emara commented

2022-06-06 11:12:33 +02:00

Added subscriber: @OmarEmaraDev

Omar Emara commented

2022-06-06 11:12:33 +02:00

I can't reproduce either issues in 3.3.

Christoph Lohr commented

2022-06-06 13:02:12 +02:00

Interesting that you found these black spots. I tested it again with both 3.1 and 3.3 (alpha) on my laptop, and there none of the issues were present.

For my own sanity i also tested it again on my main PC, this time with clean Blender installations of 3.1/3.3. Sadly the yellow glow issue was unchanged.

Interesting that you found these black spots. I tested it again with both 3.1 and 3.3 (alpha) on my laptop, and there none of the issues were present. For my own sanity i also tested it again on my main PC, this time with clean Blender installations of 3.1/3.3. Sadly the yellow glow issue was unchanged.

Germano Cavalcante commented

2022-06-06 16:38:19 +02:00

Added subscriber: @mano-wii

Germano Cavalcante commented

2022-06-06 16:38:19 +02:00

I can't reproduce the problem either.
It must be something specific on NVidia GPUs.

@Thane5, from the web page shown (https://polyhaven.com/a/lenong_2`), what specs of the download HDRI (EXR, 4k...)? And does that make any difference?

I can't reproduce the problem either. It must be something specific on NVidia GPUs. @Thane5, from the web page shown (https://polyhaven.com/a/lenong_2`), what specs of the download HDRI (EXR, 4k...)? And does that make any difference?

Christoph Lohr commented

2022-06-06 17:19:38 +02:00

HDR/EXR makes no difference. Higher resolutions make the yellow glow slightly weaker, but at 4k it is still very clearly visible (So far i my tests were at 1k).

Brecht Van Lommel commented

2022-06-06 17:56:26 +02:00

Added subscriber: @brecht

Brecht Van Lommel commented

2022-06-06 17:56:26 +02:00

The black pixels have values exceeding 65504, which is the maximum that can be stored in half float. Render results are stored as half float textures for display, and the way such high values are treated may depend on the driver. We may need to manually clamp it, though it would be unfortunate to pay the performance cost for that.

For image textures, an option to use full float in Eevee was added in #73086 (Eevee: strange discrepancy in handling of 32 bit floating point textures). We should check if that still works, but it's intentional that this defaults to half float as that works and reduces memory for most textures.

The black pixels have values exceeding 65504, which is the maximum that can be stored in half float. Render results are stored as half float textures for display, and the way such high values are treated may depend on the driver. We may need to manually clamp it, though it would be unfortunate to pay the performance cost for that. For image textures, an option to use full float in Eevee was added in #73086 (Eevee: strange discrepancy in handling of 32 bit floating point textures). We should check if that still works, but it's intentional that this defaults to half float as that works and reduces memory for most textures.

Philipp Oeser commented

2022-06-07 13:01:47 +02:00

Added subscriber: @lichtwerk

Philipp Oeser commented

2022-06-07 13:01:47 +02:00

Changed status from 'Needs Triage' to: 'Confirmed'

Philipp Oeser commented

2022-06-07 13:01:47 +02:00

Can confirm.

@brecht: would you consider this a TODO then?

Can confirm. @brecht: would you consider this a TODO then?

Brecht Van Lommel commented

2022-06-07 13:18:47 +02:00

I think it could be considered a bug.

Brecht Van Lommel changed title from ~~Brighness issue with some HDRIs in Eevee~~ to Issues with HDRIs and render result pixel values that exceed half float limits

2022-06-07 13:20:15 +02:00

Evan Wilson commented

2022-06-12 04:38:42 +02:00

Added subscribers: @Leftunderatree, @EAW, @Harvester, @PratikPB2123, @Fernando-Alcala

Riccardo Giovanetti commented

2022-06-12 09:52:47 +02:00

Here on:

Operating system: Windows-10-10.0.19043-SP0 64 Bits
Graphics card: NVIDIA GeForce GTX 1070/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 512.96

Blender version: 3.2.0, branch: master, commit date: 2022-06-08 10:22, hash: e05e1e3691

Using the provided sample scene and HDRi, I cannot reproduce those two issues, the black spot on the Sun, and the yellow tint nor in Eevee neither in Cycles. Yes, there is some "yellowness" at lower exposure values but nothing comparable to the level experienced by those user.

Here on: Operating system: Windows-10-10.0.19043-SP0 64 Bits Graphics card: NVIDIA GeForce GTX 1070/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 512.96 Blender version: 3.2.0, branch: master, commit date: 2022-06-08 10:22, hash: `e05e1e3691` Using the provided sample scene and HDRi, I cannot reproduce those two issues, the black spot on the Sun, and the yellow tint nor in Eevee neither in Cycles. Yes, there is some "yellowness" at lower exposure values but nothing comparable to the level experienced by those user.

Brecht Van Lommel commented

2022-06-13 12:50:22 +02:00

Yes, there appear to be some differences in how the drivers or GPUs deal with half float clamping.

Brecht Van Lommel commented

2022-06-20 13:35:57 +02:00

Added subscriber: @SebBlender

Brecht Van Lommel commented

2022-06-20 13:49:19 +02:00

Added subscriber: @pmoursnv

Brecht Van Lommel commented

2022-06-20 13:49:19 +02:00

@pmoursnv, the NVIDIA OpenGL driver does not appear to clamp to the half float max value when uploading a float buffer to be stored as half float on the GPU, instead setting the value to zero. Other drivers do seem to clamp it. Do you know if that would be considered a driver bug?

We can do the clamping ourselves, but would prefer to avoid the performance impact of that. Or perhaps we can do the conversion to half float ourselves with SIMD optimizations, which maybe has similar performance to what the driver is doing, not sure if that conversion happens CPU or GPU side.

@pmoursnv, the NVIDIA OpenGL driver does not appear to clamp to the half float max value when uploading a float buffer to be stored as half float on the GPU, instead setting the value to zero. Other drivers do seem to clamp it. Do you know if that would be considered a driver bug? We can do the clamping ourselves, but would prefer to avoid the performance impact of that. Or perhaps we can do the conversion to half float ourselves with SIMD optimizations, which maybe has similar performance to what the driver is doing, not sure if that conversion happens CPU or GPU side.

Patrick Mours commented

2022-06-20 22:02:56 +02:00

The driver does perform the usual float2half conversion, which for values exceeding the half maximum results in half infinity, and that is what is stored in the texture. During the gpu_shader_display_transform_frag shader that is translated to zero however (specifically inside "OCIO_to_display" function, where a 3D LUT is applied, which doesn't work for infinity since that doesn't translate into meaningful LUT coordinates) and so it ends up showing as zero on screen.
I wouldn't consider this a driver bug, since the OpenGL spec does not actually specify clamping behavior here (section 8.5 in the OpenGL 4.6 core spec specifies that clamping occurs when the data format is floating-point and the texture internalformat is an integer format or normalized fixed-point format, but nothing for when it is a floating-point format). So in my opinion the driver behaves correctly here.
To fix it is probably sufficient to simply filter out infinity values inside the gpu_shader_display_transform_frag shader.

The driver does perform the usual float2half conversion, which for values exceeding the half maximum results in half infinity, and that is what is stored in the texture. During the gpu_shader_display_transform_frag shader that is translated to zero however (specifically inside "OCIO_to_display" function, where a 3D LUT is applied, which doesn't work for infinity since that doesn't translate into meaningful LUT coordinates) and so it ends up showing as zero on screen. I wouldn't consider this a driver bug, since the OpenGL spec does not actually specify clamping behavior here (section 8.5 in the OpenGL 4.6 core spec specifies that clamping occurs when the data format is floating-point and the texture internalformat is an integer format or normalized fixed-point format, but nothing for when it is a floating-point format). So in my opinion the driver behaves correctly here. To fix it is probably sufficient to simply filter out infinity values inside the gpu_shader_display_transform_frag shader.

Brecht Van Lommel commented

2022-06-21 16:27:21 +02:00

Added subscribers: @fclem, @Jeroen-Bakker

Brecht Van Lommel commented

2022-06-21 16:27:21 +02:00

Thanks, I should have figured out it ends up as infinity.

For displaying images with color management we can clamp in the shader quite easily. For Eevee/Workbench it may not be ideal to add clamping for every texture read, and since interpolation/mipmapping happens before we can clamp the results may not be good.

Probably we should clamp before when uploading the texture to the GPU, but maybe @fclem or @Jeroen-Bakker has a different idea.

Thanks, I should have figured out it ends up as infinity. For displaying images with color management we can clamp in the shader quite easily. For Eevee/Workbench it may not be ideal to add clamping for every texture read, and since interpolation/mipmapping happens before we can clamp the results may not be good. Probably we should clamp before when uploading the texture to the GPU, but maybe @fclem or @Jeroen-Bakker has a different idea.

Alaska commented

2022-06-23 03:12:32 +02:00

Added subscriber: @Alaska

Raimund Klink commented

2022-06-24 11:13:42 +02:00

Added subscriber: @Raimund58

Zach Eastin commented

2022-07-08 22:44:02 +02:00

Added subscriber: @Zeastin

Pratik Borhade commented

2022-09-19 15:38:09 +02:00

Added subscriber: @Eary

Jeroen Bakker commented

2022-09-19 20:21:23 +02:00

I cannot speak from cycles PoV, but from shaders perspective I would add clamping in the shaders as inf and nan triggers undefined behavior on OpenGL. Data pre-processing on the CPU might be less performant as it adds another data translation pass.

Brecht Van Lommel commented

2022-09-19 20:36:16 +02:00

I don't think clamping in the shader will work well since it would happen after interopolation/mipmapping, so pixels that don't exceed the limit would be affected or there may be NaNs.

Jeroen Bakker commented

2022-09-19 23:56:17 +02:00

You’re right, think we could add clamping to the gpu texture creation image_gpu unit. Would also need a helper function in imbuf to clamp a part of rect_float.

In some cases additional buffer should be created to not modify the original data. Fe opener files that do not need any conversion to scene ref. Also for raw and data spaces….

You’re right, think we could add clamping to the gpu texture creation image_gpu unit. Would also need a helper function in imbuf to clamp a part of rect_float. In some cases additional buffer should be created to not modify the original data. Fe opener files that do not need any conversion to scene ref. Also for raw and data spaces….

Alaska commented

2022-10-06 23:19:20 +02:00

Added subscriber: @LABS