Shadow Catcher bug with transparency and Branched Path Tracing #52021

New Issue

Gustavo Niemeyer · 2017-07-11T04:44:03+02:00

Gustavo Niemeyer commented

2017-07-11 04:44:03 +02:00

System Information

Ubuntu Linux 16.04, CPU rendering

Blender Version
Broken: (example: 2.78.5 ba256b32ee)

Short description of error

When a transparent or semi-transparent material is used over a Shadow Catcher plane, the plane gets rendered in front of the transparent object when using Branched Path Tracing sampling.

Exact steps for others to reproduce the error

Just add a transparent object in front of a plane with Shadow Catcher enabled and use Branched Path Tracing sampling.

For example, this is the expected render obtained with Path Tracing sampling:

This is the broken one obtained with Branched Path Tracing:

This is the material and sampling settings used with the broken plane:

**System Information** Ubuntu Linux 16.04, CPU rendering **Blender Version** Broken: (example: 2.78.5 ba256b32ee5) **Short description of error** When a transparent or semi-transparent material is used over a Shadow Catcher plane, the plane gets rendered in front of the transparent object when using Branched Path Tracing sampling. **Exact steps for others to reproduce the error** Just add a transparent object in front of a plane with Shadow Catcher enabled and use Branched Path Tracing sampling. For example, this is the expected render obtained with Path Tracing sampling: ![image.png](https://archive.blender.org/developer/F657910/image.png) This is the broken one obtained with Branched Path Tracing: ![image.png](https://archive.blender.org/developer/F657912/image.png) This is the material and sampling settings used with the broken plane: ![image.png](https://archive.blender.org/developer/F657914/image.png) ![image.png](https://archive.blender.org/developer/F657916/image.png)

Gustavo Niemeyer commented

2017-07-11 04:44:03 +02:00

Changed status to: 'Open'

Gustavo Niemeyer commented

2017-07-11 04:44:03 +02:00

Added subscriber: @niemeyer

Gustavo Niemeyer commented

2017-07-11 04:46:30 +02:00

Setup for the example above:

shadow-catcher-bug.blend

Setup for the example above: [shadow-catcher-bug.blend](https://archive.blender.org/developer/F657918/shadow-catcher-bug.blend)

Sergey Sharybin self-assigned this 2017-07-11 09:36:29 +02:00

Steffen Dünner commented

2017-07-11 10:13:12 +02:00

Added subscriber: @SteffenD

Gustavo Niemeyer commented

2017-07-18 09:11:49 +02:00

Thanks for confirming the issue.

Is the fix for this likely to be in 2.79? The branched tracing greatly improves the rendering time in cases where a significant portion of the image is empty, which turns out to be a common case for the Shadow Catcher since it's integrating into existing footage.

Thanks again for looking into this.

Thanks for confirming the issue. Is the fix for this likely to be in 2.79? The branched tracing greatly improves the rendering time in cases where a significant portion of the image is empty, which turns out to be a common case for the Shadow Catcher since it's integrating into existing footage. Thanks again for looking into this.

Sergey Sharybin commented

2017-07-18 09:25:46 +02:00

The fix will be committed soon. Currently working on some unit tests to help testing shadow catcher in the future.

For the record, PT result was not correct either: if you render on a transparent film and alpha-over on an actual background you'll have artifacts. So both PT and BPT will be changed in render results.

The fix will be committed soon. Currently working on some unit tests to help testing shadow catcher in the future. For the record, PT result was not correct either: if you render on a transparent film and alpha-over on an actual background you'll have artifacts. So both PT and BPT will be changed in render results.

blender-admin commented

2017-07-18 09:25:58 +02:00

This issue was referenced by blender/cycles@a3f6911e07

This issue was referenced by blender/cycles@a3f6911e07c119da16c0f763721f6f334ede159b

blender-admin commented

2017-07-18 09:25:58 +02:00

This issue was referenced by 5f35682f3a

This issue was referenced by 5f35682f3abf8752f0cc155e7cc209bca4f3f852

Sergey Sharybin commented

2017-07-18 09:53:36 +02:00

Changed status from 'Open' to: 'Resolved'

Sergey Sharybin closed this issue

2017-07-18 09:53:36 +02:00

Gustavo Niemeyer commented

2017-07-18 16:18:04 +02:00

Thank you!

Gustavo Niemeyer commented

2017-07-28 01:46:05 +02:00

Is this supposed to be fixed in the latest builds? I'm still seeing artifacts in the nightly of 2.79 (more specifically, 2.78.5 1bd9531fda).

I've reproduced by just putting a fully transparent object in front of a shadow catcher plane, and traces of the shadow remain on it.

I can try a new reproducer if that's useful.

Is this supposed to be fixed in the latest builds? I'm still seeing artifacts in the nightly of 2.79 (more specifically, 2.78.5 1bd9531fda7). I've reproduced by just putting a fully transparent object in front of a shadow catcher plane, and traces of the shadow remain on it. I can try a new reproducer if that's useful.

Brecht Van Lommel commented

2017-07-28 02:08:47 +02:00

Added subscriber: @brecht

Brecht Van Lommel commented

2017-07-28 02:08:47 +02:00

The .blend file attached to this report seems to render correctly for me. If there's a different case that renders wrong, please do attach a .blend for that.

Gustavo Niemeyer commented

2017-07-28 02:35:49 +02:00

With that same example .blend file, just turn the factor all the way down to zero. The shadow (or AO, more likely) remain:

With that same example .blend file, just turn the factor all the way down to zero. The shadow (or AO, more likely) remain: ![image.png](https://archive.blender.org/developer/F681518/image.png) ![image.png](https://archive.blender.org/developer/F681521/image.png)

Gustavo Niemeyer commented

2017-07-30 18:44:21 +02:00

Here is a more clear example showing the issue and including the more noisy artifacts, in white background so it's easier to spot.

Rendered with 0.3 opacity it looks plausible (2.78.5 1bd9531fda):

Going all the way down to zero shows the issue:

And this composites the phantom shadow and the extra noise on top of the prior scene:

Here is the blend file for the scene:

shadow-catcher-bug-2.blend

Here is a more clear example showing the issue and including the more noisy artifacts, in white background so it's easier to spot. Rendered with 0.3 opacity it looks plausible (2.78.5 1bd9531fda7): ![image.png](https://archive.blender.org/developer/F686793/image.png) Going all the way down to zero shows the issue: ![image.png](https://archive.blender.org/developer/F686795/image.png) And this composites the phantom shadow and the extra noise on top of the prior scene: ![image.png](https://archive.blender.org/developer/F686808/image.png) Here is the blend file for the scene: [shadow-catcher-bug-2.blend](https://archive.blender.org/developer/F686811/shadow-catcher-bug-2.blend)

Sergey Sharybin commented

2017-07-31 00:53:31 +02:00

@niemeyer, thanks for the detailed files. I am currently at siggraph, so really limited on time for bugfixing. But definitely looking into figuring this out.

What you might want to do is to copy the details to a new report dedicated to this particular problem. This way it'll make it easier to reference the issue from code and track changes back in the future when needed.

@niemeyer, thanks for the detailed files. I am currently at siggraph, so really limited on time for bugfixing. But definitely looking into figuring this out. What you might want to do is to copy the details to a new report dedicated to this particular problem. This way it'll make it easier to reference the issue from code and track changes back in the future when needed.

Sergey Sharybin commented

2017-07-31 02:02:19 +02:00

Here is a potential fix:

P518: Potential fix for transparency and shadow catcher

diff --git a/intern/cycles/kernel/kernel_accumulate.h b/intern/cycles/kernel/kernel_accumulate.h
index 9ed16aceb55..acfe1a5a63b 100644
--- a/intern/cycles/kernel/kernel_accumulate.h
+++ b/intern/cycles/kernel/kernel_accumulate.h
@@ -223,6 +223,7 @@ ccl_device_inline void path_radiance_init(PathRadiance *L, int use_light_pass)
 	L->shadow_background_color = make_float3(0.0f, 0.0f, 0.0f);
 	L->shadow_radiance_sum = make_float3(0.0f, 0.0f, 0.0f);
 	L->shadow_throughput = 0.0f;
+	L->shadow_transparency = 1.0f;
 #endif
 
 #ifdef __DENOISING_FEATURES__
@@ -398,10 +399,11 @@ ccl_device_inline void path_radiance_accum_total_light(
 #endif
 }
 
-ccl_device_inline void path_radiance_accum_background(PathRadiance *L,
-                                                      ccl_addr_space PathState *state,
-                                                      float3 throughput,
-                                                      float3 value)
+ccl_device_inline void path_radiance_accum_background(
+        PathRadiance *L,
+        ccl_addr_space PathState *state,
+        float3 throughput,
+        float3 value)
 {
 #ifdef __PASSES__
 	if(L->use_light_pass) {
@@ -421,9 +423,7 @@ ccl_device_inline void path_radiance_accum_background(PathRadiance *L,
 #ifdef __SHADOW_TRICKS__
 	if(state->flag & PATH_RAY_STORE_SHADOW_INFO) {
 		L->path_total += throughput * value;
-		if(state->flag & PATH_RAY_SHADOW_CATCHER_ONLY) {
-			L->path_total_shaded += throughput * value;
-		}
+		L->path_total_shaded += throughput * value * L->shadow_transparency;
 	}
 #endif
 
@@ -671,7 +671,7 @@ ccl_device_inline float path_radiance_sum_shadow(const PathRadiance *L)
 	if(path_total != 0.0f) {
 		return path_total_shaded / path_total;
 	}
-	return 1.0f;
+	return L->shadow_transparency;
 }
 
 /* Calculate final light sum and transparency for shadow catcher object. */
diff --git a/intern/cycles/kernel/kernel_path.h b/intern/cycles/kernel/kernel_path.h
index c340b3bc968..f2f59034bbb 100644
--- a/intern/cycles/kernel/kernel_path.h
+++ b/intern/cycles/kernel/kernel_path.h
@@ -319,7 +319,8 @@ ccl_device void kernel_path_indirect(KernelGlobals *kg,
 
 #ifdef __SHADOW_TRICKS__
 		if(!(sd->object_flag & SD_OBJECT_SHADOW_CATCHER)) {
-			state->flag &= ~PATH_RAY_SHADOW_CATCHER_ONLY;
+			L->shadow_transparency *=
+			        average(shader_bsdf_transparency(kg, sd));
 		}
 #endif  /* __SHADOW_TRICKS__ */
 
@@ -644,7 +645,6 @@ ccl_device_inline float kernel_path_integrate(KernelGlobals *kg,
 		if((sd.object_flag & SD_OBJECT_SHADOW_CATCHER)) {
 			if(state.flag & PATH_RAY_CAMERA) {
 				state.flag |= (PATH_RAY_SHADOW_CATCHER |
-				               PATH_RAY_SHADOW_CATCHER_ONLY |
 				               PATH_RAY_STORE_SHADOW_INFO);
 				state.catcher_object = sd.object;
 				if(!kernel_data.background.transparent) {
@@ -656,7 +656,8 @@ ccl_device_inline float kernel_path_integrate(KernelGlobals *kg,
 			}
 		}
 		else {
-			state.flag &= ~PATH_RAY_SHADOW_CATCHER_ONLY;
+			L->shadow_transparency *=
+			        average(shader_bsdf_transparency(kg, &sd));
 		}
 #endif  /* __SHADOW_TRICKS__ */
 
diff --git a/intern/cycles/kernel/kernel_path_branched.h b/intern/cycles/kernel/kernel_path_branched.h
index 77d4f1df447..48124b36095 100644
--- a/intern/cycles/kernel/kernel_path_branched.h
+++ b/intern/cycles/kernel/kernel_path_branched.h
@@ -500,7 +500,6 @@ ccl_device float kernel_branched_path_integrate(KernelGlobals *kg,
 #ifdef __SHADOW_TRICKS__
 		if((sd.object_flag & SD_OBJECT_SHADOW_CATCHER)) {
 			state.flag |= (PATH_RAY_SHADOW_CATCHER |
-			               PATH_RAY_SHADOW_CATCHER_ONLY |
 			               PATH_RAY_STORE_SHADOW_INFO);
 			state.catcher_object = sd.object;
 			if(!kernel_data.background.transparent) {
@@ -511,7 +510,8 @@ ccl_device float kernel_branched_path_integrate(KernelGlobals *kg,
 			L->shadow_throughput = average(throughput);
 		}
 		else {
-			state.flag &= ~PATH_RAY_SHADOW_CATCHER_ONLY;
+			L->shadow_transparency *=
+			        average(shader_bsdf_transparency(kg, &sd));
 		}
 #endif  /* __SHADOW_TRICKS__ */
 
diff --git a/intern/cycles/kernel/kernel_types.h b/intern/cycles/kernel/kernel_types.h
index e6a62c42a38..e12557fc5b2 100644
--- a/intern/cycles/kernel/kernel_types.h
+++ b/intern/cycles/kernel/kernel_types.h
@@ -346,8 +346,7 @@ enum PathRayFlag {
 	PATH_RAY_DIFFUSE_ANCESTOR    = (1 << 13),
 	PATH_RAY_SINGLE_PASS_DONE    = (1 << 14),
 	PATH_RAY_SHADOW_CATCHER      = (1 << 15),
-	PATH_RAY_SHADOW_CATCHER_ONLY = (1 << 16),
-	PATH_RAY_STORE_SHADOW_INFO   = (1 << 17),
+	PATH_RAY_STORE_SHADOW_INFO   = (1 << 16),
 };
 
 /* Closure Label */
@@ -524,6 +523,7 @@ typedef ccl_addr_space struct PathRadiance {
 	 */
 	float3 shadow_radiance_sum;
 	float shadow_throughput;
+	float shadow_transparency;
 #endif
 
 #ifdef __DENOISING_FEATURES__
diff --git a/intern/cycles/kernel/split/kernel_holdout_emission_blurring_pathtermination_ao.h b/intern/cycles/kernel/split/kernel_holdout_emission_blurring_pathtermination_ao.h
index fec671be016..ad7e5d1cbee 100644
--- a/intern/cycles/kernel/split/kernel_holdout_emission_blurring_pathtermination_ao.h
+++ b/intern/cycles/kernel/split/kernel_holdout_emission_blurring_pathtermination_ao.h
@@ -127,7 +127,6 @@ ccl_device void kernel_holdout_emission_blurring_pathtermination_ao(
 			if(state->flag & PATH_RAY_CAMERA) {
 				PathRadiance *L = &kernel_split_state.path_radiance[ray_index];
 				state->flag |= (PATH_RAY_SHADOW_CATCHER |
-				                PATH_RAY_SHADOW_CATCHER_ONLY |
 				                PATH_RAY_STORE_SHADOW_INFO);
 				state->catcher_object = sd->object;
 				if(!kernel_data.background.transparent) {
@@ -143,7 +142,8 @@ ccl_device void kernel_holdout_emission_blurring_pathtermination_ao(
 			}
 		}
 		else {
-			state->flag &= ~PATH_RAY_SHADOW_CATCHER_ONLY;
+			PathRadiance *L = &kernel_split_state.path_radiance[ray_index];
+			L->shadow_transparency *= average(shader_bsdf_transparency(kg, sd));
 		}
 #endif  /* __SHADOW_TRICKS__ */

There are still some artifacts with less trivial topology than a box and AO-only lit scenes. Will need to have a closer look to see if that's something easily to solve.

P.S. If you don't mind, it's still handy to have separate report for this particular issue.

Here is a potential fix: [P518: Potential fix for transparency and shadow catcher](https://archive.blender.org/developer/P518.txt) ```diff diff --git a/intern/cycles/kernel/kernel_accumulate.h b/intern/cycles/kernel/kernel_accumulate.h index 9ed16aceb55..acfe1a5a63b 100644 --- a/intern/cycles/kernel/kernel_accumulate.h +++ b/intern/cycles/kernel/kernel_accumulate.h @@ -223,6 +223,7 @@ ccl_device_inline void path_radiance_init(PathRadiance *L, int use_light_pass) L->shadow_background_color = make_float3(0.0f, 0.0f, 0.0f); L->shadow_radiance_sum = make_float3(0.0f, 0.0f, 0.0f); L->shadow_throughput = 0.0f; + L->shadow_transparency = 1.0f; #endif #ifdef __DENOISING_FEATURES__ @@ -398,10 +399,11 @@ ccl_device_inline void path_radiance_accum_total_light( #endif } -ccl_device_inline void path_radiance_accum_background(PathRadiance *L, - ccl_addr_space PathState *state, - float3 throughput, - float3 value) +ccl_device_inline void path_radiance_accum_background( + PathRadiance *L, + ccl_addr_space PathState *state, + float3 throughput, + float3 value) { #ifdef __PASSES__ if(L->use_light_pass) { @@ -421,9 +423,7 @@ ccl_device_inline void path_radiance_accum_background(PathRadiance *L, #ifdef __SHADOW_TRICKS__ if(state->flag & PATH_RAY_STORE_SHADOW_INFO) { L->path_total += throughput * value; - if(state->flag & PATH_RAY_SHADOW_CATCHER_ONLY) { - L->path_total_shaded += throughput * value; - } + L->path_total_shaded += throughput * value * L->shadow_transparency; } #endif @@ -671,7 +671,7 @@ ccl_device_inline float path_radiance_sum_shadow(const PathRadiance *L) if(path_total != 0.0f) { return path_total_shaded / path_total; } - return 1.0f; + return L->shadow_transparency; } /* Calculate final light sum and transparency for shadow catcher object. */ diff --git a/intern/cycles/kernel/kernel_path.h b/intern/cycles/kernel/kernel_path.h index c340b3bc968..f2f59034bbb 100644 --- a/intern/cycles/kernel/kernel_path.h +++ b/intern/cycles/kernel/kernel_path.h @@ -319,7 +319,8 @@ ccl_device void kernel_path_indirect(KernelGlobals *kg, #ifdef __SHADOW_TRICKS__ if(!(sd->object_flag & SD_OBJECT_SHADOW_CATCHER)) { - state->flag &= ~PATH_RAY_SHADOW_CATCHER_ONLY; + L->shadow_transparency *= + average(shader_bsdf_transparency(kg, sd)); } #endif /* __SHADOW_TRICKS__ */ @@ -644,7 +645,6 @@ ccl_device_inline float kernel_path_integrate(KernelGlobals *kg, if((sd.object_flag & SD_OBJECT_SHADOW_CATCHER)) { if(state.flag & PATH_RAY_CAMERA) { state.flag |= (PATH_RAY_SHADOW_CATCHER | - PATH_RAY_SHADOW_CATCHER_ONLY | PATH_RAY_STORE_SHADOW_INFO); state.catcher_object = sd.object; if(!kernel_data.background.transparent) { @@ -656,7 +656,8 @@ ccl_device_inline float kernel_path_integrate(KernelGlobals *kg, } } else { - state.flag &= ~PATH_RAY_SHADOW_CATCHER_ONLY; + L->shadow_transparency *= + average(shader_bsdf_transparency(kg, &sd)); } #endif /* __SHADOW_TRICKS__ */ diff --git a/intern/cycles/kernel/kernel_path_branched.h b/intern/cycles/kernel/kernel_path_branched.h index 77d4f1df447..48124b36095 100644 --- a/intern/cycles/kernel/kernel_path_branched.h +++ b/intern/cycles/kernel/kernel_path_branched.h @@ -500,7 +500,6 @@ ccl_device float kernel_branched_path_integrate(KernelGlobals *kg, #ifdef __SHADOW_TRICKS__ if((sd.object_flag & SD_OBJECT_SHADOW_CATCHER)) { state.flag |= (PATH_RAY_SHADOW_CATCHER | - PATH_RAY_SHADOW_CATCHER_ONLY | PATH_RAY_STORE_SHADOW_INFO); state.catcher_object = sd.object; if(!kernel_data.background.transparent) { @@ -511,7 +510,8 @@ ccl_device float kernel_branched_path_integrate(KernelGlobals *kg, L->shadow_throughput = average(throughput); } else { - state.flag &= ~PATH_RAY_SHADOW_CATCHER_ONLY; + L->shadow_transparency *= + average(shader_bsdf_transparency(kg, &sd)); } #endif /* __SHADOW_TRICKS__ */ diff --git a/intern/cycles/kernel/kernel_types.h b/intern/cycles/kernel/kernel_types.h index e6a62c42a38..e12557fc5b2 100644 --- a/intern/cycles/kernel/kernel_types.h +++ b/intern/cycles/kernel/kernel_types.h @@ -346,8 +346,7 @@ enum PathRayFlag { PATH_RAY_DIFFUSE_ANCESTOR = (1 << 13), PATH_RAY_SINGLE_PASS_DONE = (1 << 14), PATH_RAY_SHADOW_CATCHER = (1 << 15), - PATH_RAY_SHADOW_CATCHER_ONLY = (1 << 16), - PATH_RAY_STORE_SHADOW_INFO = (1 << 17), + PATH_RAY_STORE_SHADOW_INFO = (1 << 16), }; /* Closure Label */ @@ -524,6 +523,7 @@ typedef ccl_addr_space struct PathRadiance { */ float3 shadow_radiance_sum; float shadow_throughput; + float shadow_transparency; #endif #ifdef __DENOISING_FEATURES__ diff --git a/intern/cycles/kernel/split/kernel_holdout_emission_blurring_pathtermination_ao.h b/intern/cycles/kernel/split/kernel_holdout_emission_blurring_pathtermination_ao.h index fec671be016..ad7e5d1cbee 100644 --- a/intern/cycles/kernel/split/kernel_holdout_emission_blurring_pathtermination_ao.h +++ b/intern/cycles/kernel/split/kernel_holdout_emission_blurring_pathtermination_ao.h @@ -127,7 +127,6 @@ ccl_device void kernel_holdout_emission_blurring_pathtermination_ao( if(state->flag & PATH_RAY_CAMERA) { PathRadiance *L = &kernel_split_state.path_radiance[ray_index]; state->flag |= (PATH_RAY_SHADOW_CATCHER | - PATH_RAY_SHADOW_CATCHER_ONLY | PATH_RAY_STORE_SHADOW_INFO); state->catcher_object = sd->object; if(!kernel_data.background.transparent) { @@ -143,7 +142,8 @@ ccl_device void kernel_holdout_emission_blurring_pathtermination_ao( } } else { - state->flag &= ~PATH_RAY_SHADOW_CATCHER_ONLY; + PathRadiance *L = &kernel_split_state.path_radiance[ray_index]; + L->shadow_transparency *= average(shader_bsdf_transparency(kg, sd)); } #endif /* __SHADOW_TRICKS__ */ ``` There are still some artifacts with less trivial topology than a box and AO-only lit scenes. Will need to have a closer look to see if that's something easily to solve. P.S. If you don't mind, it's still handy to have separate report for this particular issue.

Gustavo Niemeyer commented

2017-07-31 05:30:51 +02:00

@Sergey Thanks again for your attention on this.

I've create a more appropriate task as requested: #52229

@Sergey Thanks again for your attention on this. I've create a more appropriate task as requested: #52229

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Shadow Catcher bug with transparency and Branched Path Tracing #52021