Glossy shader bakes different than render
Closed, ResolvedPublic

Description

System Information

openSUSE 12.3 64bits
NVIDIA 480GTX SC
AMD FX-8350

Blender Version

Broken: 0efc0d5
This broke after recent fixes for this bug T40322

Short description of error

See difference in glossy results. Baked vs Rendered

Exact steps for others to reproduce the error

Bake this file


Compare with rendered output

Both Anisotropic Glossy and Regular glossy present this problem

Daniel Salazar (zanqdo) updated the task description. (Show Details)
Daniel Salazar (zanqdo) raised the priority of this task from to Needs Triage.
Daniel Salazar (zanqdo) set Type to Bug.
Daniel Salazar (zanqdo) updated the task description. (Show Details)
Daniel Salazar (zanqdo) updated the task description. (Show Details)
Dalai Felinto (dfelinto) triaged this task as Confirmed priority.May 27 2014, 12:05 AM

The issue happens with glossy direct and indirect too.

Simplified the report file to a plane

. Basically it seems like baking always consider the camera to be with rotation 0,0,0 and at the origin.

Even before that bugfix glossy was not the same as in final renders as far as I know, that bugfix can't have caused this problem. Glossy depends on the direction from which you're viewing the surface, and it's not obvious what that should be for baking.

Currently the viewing direction for each pixel is set to the normal, so at every pixel glossy is evaluated as if you're looking straight at it. Blender Internal works the same.

If you want to bake glossy so that it matches exactly what you see from the active camera, instead it could do: sd.I = normalize(camP - sd.P) for perspective and sd.I = -camD for orthographic which camP and camD as in camera_distance(..). Not sure about the math for panorama cameras, guess it can just do the same as perspective since the implementation is missing in camera_distance(..) as well still.

I think it's fine to bake as viewed by the camera. Eventually we may need an option to disable view-dependent effects, but for now I think D555 could go in.

It's nice to have both ways imo, I've been playing around with baking a bit, and found the view-independent glossy baking quite handy..
+1 to having an option to toggle this behavior as @Dalai Felinto (dfelinto) mentioned.

Just wanted to put in a vote for keeping the old behavior around too.

Excellent! Yes being able to bake it from the active camera is huge :D

Now it's all messed up in my original test file. Testing on planes is not always that useful ;)

@Daniel Salazar (zanqdo) are you sure? from the camera point of view the results are matching

(top bake, bottom render). And if you look from outside the camera point of view then it shouldn't not match since glossy is view dependent.

I do test with the reported files ;) planes are essential for fast debugging though.

Here's what I'm getting :o

@Brecht Van Lommel (brecht) any clues? I think we need to invert the direction based on the orientation to the camera. I tried some variations of dot checks to invert the ray direction (like below) but I'm not sure what the test should be. All I managed to get was to get some funky results.

if(dot(sd.I, sd.N) < 0.0f)
	sd.I = -sd.I;

The black parts are the problem on the texture? It seems those are the parts hidden from the camera POV. If the rays reaching these parts, maybe they are reflected into the inside of the object so they cannot reach the light source. @dfelino: what happens if you invert both the ray and the normal vector?

In the screenshot shown by @Daniel Salazar (zanqdo), as far as I can tell the baked result does actually match watch you see from the camera, it's just that the viewport render is from a different point of view. The problem is that it's black on the backside which doesn't look good.

I'd say that the fact that the backside is black is in a way correct. There's no way the glossy reflection will reach the camera, so there's no light on the backside. It's a situation that would never come up in regular rendering. The fact that glossy BSDFs do no fade to black smoothly at grazing angles makes it worse because you get this discontinuity. But that BSDF behaviour is by design.

But ok, we want something that looks better on the backside and that's tricky. I wonder if there is any baking software that attempts to handle this well?

The best I could come up with is to fade I to N when you get towards grazing angles, but that's a pretty big hack, though at least it looks similar then from the front and with no discontinuity on the side, just somewhat non-sensical glossy highlights on the side and back.

sd.I = camera_direction_from_point(kg, sd.P);

if(dot(sd.I, sd.N) < 0.0f)
    sd.I = sd.N;
else
    sd.I = interp(sd.N, sd.I, sqrtf(dot(sd.N, sd.I)));

@Fazekas Laszlo (totoro): if you invert the normal you're inside the mesh, where there is no light.

@Brecht Van Lommel (brecht) : I thought the black is there because the rays are already go inside, that's why I recommended the flip for the normal. By the way, what is the color exactly at the border of the black parts? Is it an environment color? I don't understand why it is not on the black parts too.

I just realized that even the viewport rendering doesn't change if I move the camera when not looking through the camera. So it's not like rendering is doing a different calculation.

@Brecht Van Lommel (brecht) this actually makes the result to be different even from the camera point of view. And the overall result (in the caterpillar) is awful :/

What about:
For combined we use sd.I = sd.N;
For glossy direct we use sd.I = camera_direction_from_point(kg, sd.P);

@Dalai Felinto (dfelinto) are you sure? If I understand you right, that doesn't make much sense :p Glossy is always dependent on viewpoint, even in the live preview :)

I just realized that even the viewport rendering doesn't change if I move the camera when not looking through the camera. So it's not like rendering is doing a different calculation.

In other words, if I'm not looking through the camera and move the "Camera" object I see no difference in the render preview. @Daniel Salazar (zanqdo) ^^

@Fazekas Laszlo (totoro): for the glossy BSDF to do anything the normal N and view direction I need to be on the same side, if there is no light reflect and it gives black.

For a face that is facing away from the camera, you can either flip I to the front of the face, or N to the back of the face. In the former case you do what @Dalai Felinto (dfelinto) proposed. In the latter case you end up on the inside of the mesh, and any path to the light will be blocked.

Not sure what you mean by the color at the border.

Perhaps something like this?

@Daniel Salazar (zanqdo) translating what I proposed (as requested on irc ;):

For combined we could use sd.I = sd.N, which would produce the result you had when you opened this report. Basically it means the "view direction" that will be used is the faces' normals.

For glossy direct we could use sd.I = camera_direction_from_point(kg, sd.P), which produce the current result in master. Basically it means the "view direction" is correct (uses the active camera as reference), but produces dark results for faces facing away from the camera.

@Dalai Felinto (dfelinto): I don't think it would be a good idea to give a different result for glossy and combined passes, I don't see how that really helps, why you would want it to be one way for combined and another way for glossy direct/indirect. Better make it an option then that affects both.

You can get closer to the camera view by tweaking the blending but then the discontinuity becomes more and more obvious, e.g.:

sd.I = interp(sd.N, sd.I, powf(dot(sd.N, sd.I), 0.2f));

In the end it's not clear to me what the purpose of baking this glossy component is. If you want something that looks similar-ish from all sides then you can use sd.I = sd.N. If you need to bake it to look at the result from one side with maybe some minor camera movement then I guess you could use a fairly low blending factor.

@Fazekas Laszlo (totoro), I'm not sure what that drawing represents exactly, but it should probably include both a camera and light to see the vectors to evaluate the BSDF and understand why some areas are black or have discontinuities.

@Brecht Van Lommel (brecht) : any camera movement ruins the illusion, it's like if you replace a mirror with a picture on the wall. Human eye is very sensitive to this, I think. So maybe you can simply leave out the hidden parts rendering. But if you render there something, it must be seamless.

@Brecht Van Lommel (brecht): my drawing was about continuing the rays on the hidden side: not reflect them but just continue in the original camera-point direction because it is exactly what happening at the border of the black parts. If you need a normal vector for this, let it be perpendicular (or near perpendicular towards the camera) to the original s.I vector then.

@Fazekas Laszlo (totoro): ok, I think what you propose is the same as @Dalai Felinto (dfelinto) then. There I is flipped to "continue" in the same direction. That mostly avoids the black, but you still get a discontinuity at grazing angles where the view direction gets suddenly flipped.

Tomorrow I will revert the fix, which means we will get sd.I = sd.N again. I will elaborate a bit more in the log. Thanks for all the tests though.

You asked on a BA thread about the view dependent baking for Glossy.

case 1: Obviously, for close up objects, view dependent is trouble since the view changes and the Glossy will look wrong.

case 2: For objects off in the distance, like ice covered mountains for example, the view angle relative to the mountains will not change noticeably when the view changes as a character moves around the scene. For side scrolling games there is no rotation of the view at all and this would be another good situation for view dependent Glossy, especially if there are dynamic GLSL lights to hid the fact that the baked highlights are not moving. They would add detail but the wrongness of them would be lost in the detail of the moving lights.

It would be nice to have the option. Especially if the option allows us to interpolate between the 2 methods so this factor can be manually adjusted for objects at various distances.

This way we could make far away objects have view dependent Glossy, and also objects that are near can have non-view dependent Glossy and objects in between could have a mix of the two.

Agreed, I wish we had this somehow :) It's just useful

IMHO, some very possible usages of POV-based glossy baking that I can think of are:

  1. Skybox, Envrionment map;
  2. 360d games, where camera position is fixed, but camera rotation is allowed. (like the classic game Beachhead);
  3. Interactive street view experiment (like Google Maps), and panoramic view for VR usage;
  4. Camera mapping animation with 3D composition aided.

etc.

I think part of the issue is feature bloat and the UI clutter that comes with this. Maybe this could be solved by not exposing this in the UI.

If there is no camera in the scene then the glossy highlights would bake semi-non directional as it is now. With a camera present, then it could bake with the camera as the view position.

@marc dion (marcclintdion) I must be missing something.. This should only require one checkbox, right? How would that be bloated?

One or two check-box's for 40 other small different features that people want would cause bloat, the UI already has problems with this. That's what tabs were meant to address.

When you add up all the options for all the various things that could potentially be added to Blender, you start to have a mess. It's not just this tool people want extra features for.

The problem is something I've seen mentioned by one of the main devs a few times over the past couple years, I thought I'd offer up an idea that would help avoid the problem for this case.

Well, if that is not a option, the only way I can think of is to first hit F12, then use that rendered image as a paint brush texture, to paint (or re-project) it from camera view. But I really don't think this is good, because, first, there would be so much loss on details; second, it leaves all back sides black (even some side sides); last, since objects have to be painted one by one, not good for scenes with quite a bit of objects.

Is there some way to add it back? Or a script we can use or anything? This is extremely important for baking stuff for games like Leon explained.
I don't agree with the "bloat" reasoning (a 3d package is complex and need to be able to do a lot), but assuming it's valid, artists should at least be able to get a script somewhere to enable it back.

Please add the feature back. There are many cases where the results are far superior to a normal-based camera vector, for example when you have a flat reflective surface and an environment map.

Note that reversing the incidence vector for backfaces will produce wonky results, because such rays won't emerge from a single origin. You need a virtual camera that is diametrically opposite the object, and get the incidence vector from that. Even better would be a drop-down where you could choose between using normals or one, two, six or 36 cameras.