another thing is, when I render on 2 GPUs a scene that takes more then dedicated memory, the system memory will be used. If those generated textures are kept around, we have 3 times the textures. The cache Blender silently creates and the 2 versions created by the OpenCL driver. As the time needed to initialize rendering is not much faster the second time compared to using the cache, but many GB memory are away, making it more certain that swapping will be triggered, the cache may greatly slow down render for a benefit which is pretty low, at least in my tests.

ok, then it's not a bug, but wouldn't it make sens to only have this behavior when "persistent data" is checked and free it otherwise?

Really helpfull addition and it seems very robust. It shows that the latest AMD drivers also allows to use swap for buffers. I managed to render a scene that took 36GB on a 16GB (system) and 8GB (RX480) system. Of course, it was slow, but still interesting.
Only thing that it showed is that cycles seem to have like a memory leak or hidden cache. It doesn't come from the patch nor driver as the same scene rendered on master with CPU also takes 7GB after rendering VS 1.5GB before on windows 7 x64. It looks like if "persistent data" option was ticked and textures were cached. I made a simple scene to showcase:

please attach a file to reproduce the problem.

As suggested today on IRC, making robust naming conventions for bmesh operators would help new coders to jump in. It will be very helpful for making the nodal everything work faster. At the moment, the use of for example underscores is pretty random. Having easy to search names will give more time for real coding.

edited task description to reflect last fixes to the new depsgraph.

opened new task instead https://developer.blender.org/T51997

After further investigation, not only the hidden particle system is triggering problems. In old depsgraph, it the subdivision modifier is also hidden, the particle system is evaluated, but at the wrong place. Only solution is to manually make it visible in viewport and change something that triggers a new evaluation many times, until it updates (like setting the particle system on and off. In some case, it has to be done 3 or 4 time to really update the particles positions

I can confirm the bug is still there on windows. Two cases: user pref is there and only "load factory settings" will mess up. Or there is no userpref and then file->new and "load factory settings" will messup the UI.

I simplified the file as requested:

Having BW textures stored as single channel would also help to save 3/4 of the memory.

ok, I can confirm it crashes although the scene is very simple. Only planes and some textures, but nothing fancy as far as I can tell. Only 64 x 8bit textures at 4096*4096, which is 64MB per texture with RGBA, which is 4288MB of textures at max.
It looks like all textures try to be allocated at once, instead of 4GB chuncks, which is crazy and would prevent using today's GPU memory with low poly scene with a lot of textures.
@Stefan Werner (swerner) you wrote the code for infinite textures, maybe you can have a look here on how to allocate chunks of 4GB max at once.
Limiting rendering to less than 70 x 8 bit textures in 4K in 2017 is way too low. Many displacement maps today are 16 or 32bit, HDRs are also very big.
As an element of comparison, Luxcore 1.6, which is now more than a year old takes 1100MB (1400MB used of which 300MB are Blender's UI) of memory to render it.

1. Did you update your driver to latest version
2. OpenCL can use your system memory, so there should be no memory limit other than your GPU+ available motherboard memory

Anyway, please share a file before adding more text.

In my tests, noise is even, which is good and overhead of analysis seem to be low. But setting a norm of 0.005 and steps of 4 gives the same result as norm of 0.01 and steps of 16. A norm of 0.01 and a step of 4 halves the time compared to same norm but step of 16, although in theory, the max difference should be of 12 samples between those 2 settings. As I rendered with max 1000 samples and a norm of 0.01 compares in quality with 200 samples, a step of 4 could stop in the strongest discrepancy case at 200 samples, while the 16 step case would stop at 212samples, which wouldn't make a 2x longer render time. Plus the higher step value should reduce the overhead of image comparison.
norm 0.01 step 16
render time: 4minutes

thanks for the update. For some reason, when used with denoising, it is partly transparent.
Here, the sky in Barcelona:

after some time, I managed to make a file that has the bug in release build and open in debug build. But it renders correctly there and won't crash after rendering is finished. Some things more are printed to console however. Dunno if it can give any hint. Here it is:

Error: Could not acquire buffer from image
Error: Could not acquire buffer from image
Error: Could not acquire buffer from image
Error: Could not acquire buffer from image
Error: Could not acquire buffer from image
Error: Could not acquire buffer from image
Error: Could not acquire buffer from image
Error: Not freed memory blocks: 278, total unfreed memory 0.084785 MB

At least on windows, there is no crash. I tested on latest buildbot, VS2015, on victor scene, reverted during synchronization with new depsgraph ('using new depsgraph' was printed in console, so there is no typing error on my side). Maybe it's Linux-specific or random due to some threading issues?

The file is not available anymore. Please attach a file that really trigger the problem. I render files that need 16GB of memory on the RX480 without problem, so it must come from something else.
Also update your drivers before testing further.

I can confirm the bug. All tiles hitting something render normally, until the edge of the view is reached (the tile at bottom right in my case). When there is nothing to render, neither geometry nor background, it hangs.

In D2662#64177, @Brecht Van Lommel (brecht) wrote:

Ideally the adaptive sampling should take color management into account indeed, though in general it's unknown what kind of image transforms might be applied in compositing. A simple color_scene_linear_to_srgb() or sqrt() might already see a noticeable improvement.

looks good :)

Do you have a recent version of the patch that applies on latest master? I couldn't apply it, even with arcanist.

In D2662#64173, @Milan Jaros (jar091) wrote:

In D2662#64168, @john peterson (bliblubli) wrote:

If the user could paint areas needing more sample and continue render, it would avoid the limitations of computed versions for still images and remove the overhead.

Hi,

you can check the headlight in the table:
http://blender.it4i.cz/research/adaptive-rendering-based-on-a-norm-of-rendered-pixels/bmw27/

I think the algorithm detects the noise in the headlight very well.

Milan

If the user could paint areas needing more sample and continue render, it would avoid the limitations of computed versions for still images and remove the overhead.

@Bastien Montagne (mont29) If the problem solution is as simple as deleting the group when deleting the last instance, why not do it directly for the user? At least for the MB garbage it makes when making linked librairies local, wouldn't it be possible to solve the problem when the groups are made local?

I can also confirm the bug.

with 256x256 tiles (recommended size in the manual), using latest master and latest drivers from AMD on win7 x64, your file render in 1min46 with one RX480. So it would be 53s on 2 cards, which is faster than the time you report with 2.78c using 2 cards.
Second thing is that BPT wasn't available for the split kernel in 2.78c, so the file couldn't render.
Setting as incomplete until more informations are provided.

@Renato Meneses (renatomeneses) it is indeed very unlikely to happen. In very intensive rendering scene, cycles uses max 85W on my RX480, while the card draws 160W (above the 150W of PCIE specs) while rendering real time benchmark scenes like furmark. Cycles heavily underuse the GPUs (on both CUDA and OpenCL) compared to real time 3D apps, especially benchmarks. So the only thing that could burn it while rendering would be if your fans are defects, but even then, the hardware should just shut your PC down to protect it.
Whatever happened, it's not the software job to monitor temperature. Also note that overclocking, changing voltage and all other modification leading to warranty void are allowed by drivers from all vendors, but are risky and thus not supported. So I would recommand you to use latest build, which is much faster for your configuration anyway, but leave all driver options at defaults.
2.78c is not supported anymore. But I'll let @Mai Lavelle (maiself) and @Sergey Sharybin (sergey) decide here.

I also have some Blendermarket materials, so I had a look as it may have come from cycles. It comes from the cgc_oak material. But what is impressive is although I spend a lot of time to bisect, only retaining a glossy shader and the material output is enough to reproduce the crash. Of course, neither the full nor the simplified material crash on normal rendering with cycles. So I think the problem doesn't come from cycles.
Here it is, a cube with 1 shader creating a seg fault:

Hi Lukas,
this patch saves huge amount of preprocessing times, but seem to only work in simple scenes. The BMW benchmark uses it perfectly, but rendering the Viktor scene twice does the whole preprocessing a second time. As it's a really useful patch, an update would be really awesome :)

Hi,
Thanks for the report, but this addon is not part of official blender. You can report this to the original addon author.

it seems to work properly on another computer, will investigate more. Closing in the meantime.

Kernels recompiles now, but on every change, even those that don't change anything to the used kernel. For example, max_closure is set to 64 for viewport rendering to prevent recompiling. Since this patch however, adding a shader to the nodetree triggers a recompile, although having 1 or 2 shader is not important as it will be set to 64 anyway.

Hi Lukas, great to see you back on other task than Denoiser :)
When trying to build on MSVC 2015, it prints some errors:

For support please use http://www.blenderartists.org or https://blender.stackexchange.com . This is only for bug tracking. Thanks.

click on testing (top left in your picture)

I can confirm the bug.

I tried on a not-saved file and I see no bug. I don't see any reason in this case how not saving the file could influence snapping.
The problem is more that snapping doesn't respect the 'limit to visible' toggle of selection mode.
I think @Germano Cavalcante (mano-wii) wrote a patch for this, not sure if it's ready for release? Will let you decide on this one.

I also agree normals should be the same whatever the type is.

I can confirm this behavior exist and is really annoying. To be consistent with all other transform operator, it should take the screenspace and not worldspace movement.

osm importer is an addon you must activate, but here is a bug report platform. You can search on google regarding osm import.
Regarding the crash, you must provide a file and open a new bug report. Please don't mix 2 different reports.

hi @Lukas Treyer (cnd)
just tested your rotation fix and I can confirm it works on my dxf files. Great :) Will test further this WE to see if some other files from other CAD software are now wrongly rotated, but it looks good.

Couldn't reproduce here. Please use latest buildbot before reporting a bug. It may have already been fixed.

I think it's more a design problem than a bug. It's one of the area in Blender that doesn't follow the philosophy of "don't think for the user". The depsgraph will decide in which order the modifiers should be applied and sometime fails at finding the right order, because in some case their are several possibilities, all valid and it's more a question of artistic decision.
The solution to it would be to have a way for the user to order modifier execution. The everything node system will solve it on the long term. In the meantime, it could be solved with a scene stack of modifiers instead of being limited to object stack.

It is just how blender works, not a bug. I also find it not really understandable that a display option has an impact on geometry without any warning nor infos in the tooltip. But it's a design task more than a bug. Zero could also completely disable that behavior like in many other cases in Blender (letting objects use the unit system as one would expect)?

@Brecht Van Lommel (brecht) and @Sergey Sharybin (sergey) wouldn't it be better and possible to take last valid texel value instead of 0,0,0 for replacing NaNs while cycling through all the texels of the texture?

ok, tested a little bit more. Some errors only show up when other addons are activated. An easy way to see it is to activate all addons and press F8, then you get the errors form castle, creaprime and animated text plus this:

@Brendon Murphy (meta-androcto) creaprime, castle and animated text stil have reloading problems, other are solved :)

I tested the other ones again to see if the new build solve the other problems:
measure panel support F8 now as long is it was not activated, otherwise, creating a new file or reloading give this error:

Hi @Okavango (Okavango) , can't reproduce either in latest build, I must have had an old build somewhere. Updated task.

I see no errors in latest buildbots, please try with one from https://builder.blender.org/download/ and post an image showing where it's wrong if the bug is still there.

I'm speaking about addons that don't support to be reloaded. The new template system allows to have different addons activated depending on the startup file chosen. It sometime lead to reloading addons (like F8) and crashes when those don't correctly reload.

managed to get a reproducible case. But I get the feeling it's only one of many bugs. Will see, here is the file:

thanks for the quick fix :)

Sorry for the duplicate then :/ I can't merge it with a task I didn't create.

In D2655#62599, @Sergey Sharybin (sergey) wrote:

ok, sorry was my bad :/ resolved.

note that buildbot from last week was still working, so it's pretty new.

I would exclude release kernel from this. Many users work on stable but try buildbots regularly. As the buildbots use the same folder as the last and not the coming release, it will make the stable release to recompile all kernels every 2 weeks.

In T51408#432279, @Pascal Schön (VanCantus) wrote:

I can also add the emission shading if this is a liked feature for the principled shader.

You can also use a mix. As Jesterking said, you can cherry-pick all commits related to one tool only. Then you only have a few commits related to several tools that can be copy-pasted by hand.

@João Araújo (genio84) Please split the diff per tool. It will be hard to get a quality review for all of them before 2.79. It will allow to commit some of them independently.

@João Araújo (genio84) Can you update the diff with your latest fixes for review? The version here is from january.