@Philipp Oeser (lichtwerk) @Jeroen Bakker (jbakker) Thanks for investigating this more. I also remember at some point when I was accessing the pixels directly through "Viewer Node" image block and stored them on disk. What I realized however was the pixels were sort of rotated. I remember the depth map rendering that I got was shown 90 degrees rotated. So I have a request from you: I would appreciate if you can provide a nice Python API that allows people to easily access the rendering results without having to store the renderings on disk. Maybe bpy.ops.render.render() could take an argument like keepInMemory=True. Then people can access the rendering results through something like renderings = np.array(bpy.ops.render.results). The should also account for having a couple of output nodes. For instance, if in the node editor I have 5 Viewer nodes, renderings should be a 5 x 4 x resolution x resolution tensor.

Interesting problem.

I believe this is caused by an extreme value for bump (causing the normals to point in all kinds of directions). Or I'll quote @Brecht Van Lommel (brecht) from T54498:

When rendering with OSL it compiles the shaders to native code and executes that, the SVM shader stack isn't used

Could you explain please, why does Open Shading Language helps in this situation? Enabling this thing was kind of desperate move, but helpfull in this case, so I am intersted in a bit more info

Increasing SVM stack size will marginally increase GPU memory usage.

That's correct, you can change the volume sampling method to Distance in the material if you want the same result on the CPU. Equiangular and multiple importance sampling reduce noise a lot when there are lights inside or near the volume, but increase it when that's not the case.

Pretty sure this is the difference between decoupled volume sampling on the CPU vs regular volume sampling on GPU (decoupled isn't supported on GPU). Doesn't look like a bug to me.

2.79 5bd8ac9 is quite old, please try 2.79b and/or a build from http://builder.blender.org and see if the problem persists.

@Brecht Van Lommel (brecht) Just an update: I can get somehow around the forking issue using subprocess and doing one rendering at the time. However, the problem with using subprocess is I have to literally import everything I need from scratch. This makes the rendering much more time-consuming than doing the rendering normally. Sorry if it's too obvious that I'm just trying to convince you to prioritize this :) I hope that happens :)

Hm, it's running against the stack size limit, see here

Gathering all volume intersections would be good also to improve performance.

A possible remedy would be to gather all volume intersections in one go (similar to intersect_shadow_all) and do the ray marching in a single step. There are hints that Arnold is using such a method in " Importance Sampling Techniques for Path Tracing in Participating Media", section 6.

Ok, thanks! Then I will just build it by myself.

@Brecht Van Lommel (brecht) I hope you will prioritize this in the future. Many people like me need to use Blender as a module for their research work. Anyways, I used gdb to get you the back trace after Python freezes. Here's what I get (I had to CTRL+C at the end):

This is a very low priority issue for me, since it should be quite easy to work around in the ways I suggested and the python module is not something we officially release. Given that there are many more important bugs to solve I'm not going to spend time on this now.

@Brecht Van Lommel (brecht) I'm not sure if this could be very relevant but I used OpenEXR Python bindings package to do some tests. Here's what I found out: if I have a function that simply loads an exr file and separates its channels into (R, G,B) as shown below and I execute this function via multiprocessing.Process everything works fine. However, when I just do import bpy and execute the same function via multiprocessing.Process things do not work and Python freezes. I would say this is a bug in Blender. Could you please look into this? I would really appreciate it.

Okay, so it turns out that this is caused by one of the pointers in the NLM kernels not being aligned, but it's actually already fixed in the current master.

No, they're different issues.

@Brecht Van Lommel (brecht) yep, so we agree, it's this same bug that makes the wood floor look strange and adds the noise in the other scene.

That's probably the main change, but note that commit changes both the RR continuation probability and removed RR min bounces.

the difference in noise level in https://developer.blender.org/T54486#491645 are really due to the windows which should be completely ignored due to their visibility set to only camera rays. 2.79a ignores them, latest master not. When I render without the windows, both stable and master render the same image.

@Brecht Van Lommel (brecht) I changed line 235 of kernel_path_state.h from

Russian roulette changes can cause increased noise. We hope that these are offset by reduced render time, so for a proper comparison at least an equal time render must be done. Even then there will be some scenes that are noisier, since it's a trade-off.

I tried to reproduce it in a simple scene (a box with principled on multiscatter GGX, one opening, 1 portal and one window with only camera visibility), but somehow, the bug doesn't show up in master with this file. Making it a bit more complexe with some textures and objects to occlude and have more bounces show a clear difference in noise pattern, but not really noise levels. Complex and real use case scene on the other hand show clearly that 2.79a has a much better noise level, but I couldn't find yet, what the reason is. Keeping only the walls, windows and lights in the living room/kitchen scene above rendered significantly darker in master compared to 2.79a.

@Brecht Van Lommel (brecht) so if I understand you right, this also explains the much higher noise levels in these scene (see pictures). This scene is lighten by portals with windows set to be only visible by camera. But indeed, although those windows are only hit by not camera rays, removing the windows completely greatly improves lightning and noise in latest master. So it showcases what you said about ray visibility. Here is a comparison of scene 10 from archinteriors 43 for Blender in master and stable:
2.79a = low level noise:

I would not make a bug report to the OpenEXR project, it's not clear at all there is a bug in their library and I wouldn't expect them to investigate a vague report like that. It's a possibility but too early to tell.

@Brecht Van Lommel (brecht) I also posted an issue on OpenEXR's GitHub repo (link here). Hopefully they can also make some clarification from their side.

@Brecht Van Lommel (brecht) You said it wouldn't surprise you if some parts of Blender of OpenEXr does not work well with forking. So I wonder, which one do you think is more likely to be the case given that I can store my renderings in other formats? Would you think it's more likely that OpenEXR is the one to blame here?

@Brecht Van Lommel (brecht) I tried with spawn but I keep getting weird errors and things do not work. It seems that I have to somehow make the process spawnable and I don't know how to do that precisely. I hope you can find a solution for this. It's so annoying for me since I have to millions of renderings and I didn't expect to do it sequentially :(

If there's another bug with OpenEXR saving unrelated to multiprocessing, another bug report can be created for that,

Maybe try using the spawn start method?
https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods

@Brecht Van Lommel (brecht) Sorry for the missing information. I just realized that the problem is being caused if I call the rendering function using multiprocessing.Process and updated the script in my bug report that shows how exactly I do rendering in my project, with all other irrelevant details removed. The strange thing is I can easily render my meshes if I do not want the rendering result to be stored in EXR files. So, if I replace OPEN_EXR with PNG and replace 32 with 8/16 I will definitely get the rendering results stored on disk. This does not happen when trying to store the result in OpenEXR format and I did not know that the multiprocessing package might be causing it. What should I do now? Why I cannot store EXR files when using multiprocessing.Process?

@Brecht Van Lommel (brecht) Sorry I should have been more clear. The problem is I am compiling Blender as Python module in Ubuntu 16.04. I don't think it is possible to import the Blender module given those arguments you mentioned. Could you please double-check this as well?

Ooh sorry, thank you for the response. This is first time i used this bug report.

As this pack https://evermotion.org/shop/show_product/archinteriors-vol-43-for-blender/14563 has just been released this week, it would be great if user had an option so that it's not already broken in buildbot/next release. Blender just starts to appear on one of the most renown visualization website.

This seems to be an old (known) issue with ray visibility and multiple importance sampling, where only the BSDF sampled rays take ray visibility into account. Due to the new quad light sampling the light sampled rays have a higher weight which shows that old bug more clearly. It happens with diffuse as well, has nothing to do with specular specifically.

@Brecht Van Lommel (brecht), it's much more subtle in this case, but should be the same thing happening. The master render has a much darker specularity than stable, like in the wood renders in first post.

ok, I hope I nailed it.
stable:

Do you need a texture at all? Just remove as much as possible from the scene and shader nodes and probably you can reproduce it with some fixed values?

@Brecht Van Lommel (brecht) which bugfix commits touched the specular part of principled shader ?

OS is Linux. If the difference seen in the attached blend is not enough, I'll try to find a free wood texture alternative. The materials in the renders above are copyrighted by evermotion.

I see some difference in the intensity of the specular in the attached .blend, but nothing that looks like the attached renders. The file has some extreme color management settings as well, after setting that to the default I don't see anything obviously wrong, there have been some bugfixes that are known to affect the look.

@Lukas Stockner (lukasstockner97), if it's a SIMD issue we can disable it for 32 bit for the filter kernels, we already do that for GCC since T36316.

Here is the system info file:

@Brecht Van Lommel (brecht) I can reproduce the crash using the 32-bit Windows build of 2.79b with Wine, I'll reboot to Windows and take a look.

I mean inside Blender, open the Help menu and save the System Info from there.

You mean msinfo32.exe, right? (At least on my 32 bit system)

I guess this happens on 32 bit only, we don't get a lot of testing there. Can you attach the output of Help > System Info?

I can't reproduce this issue. I tried putting the above code in a test.py script with bpy.ops.render.render(write_still=True) at the end, and then running ./blender -b -P test.py.

We indeed do not support this GPU, because of driver issues like this.

It's unclear from this report what gpu you have, however.

Hi, I had the same issue for months. It made me crazy and I could not do anything. You might try this solution. Its easy to do and undo.

This bug excists for more than 8 months... what about a solution?

I struggled with this problem for months. Used different computers, re-installed Blender, re-installed windows (I am using 8.1 & 10), installed new and old drivers for my Nvidia card, changed the TDR and everything I could think of...but no luck.

@Brecht Van Lommel (brecht), by removing first statement you are effectively removing optimization done in rB37afa965a4a. If there is an issue there, you would also need to check on Remove Doubles operator. This code was originally matching code in the operator.

Thanx a million Brecht
Cheers!
Have a Merry and Happy Easter!!

Thanks for the log, the fix will soon be in the various builds.

Hiya Brecht,
Sorry, I didn't know that CPU+GPU wasn't yet available in 2.79b.
Didn't know either that Blender-Edge wasn't yours.
Thus my surprise to find that CPU+GPU is available in Blender edge and not in 2.79b...
Got that.