The boolean in question was applied many month ago already in the project files. Trying to recreate the same object with a modifier, the bug doesn't show up.

After testing many versions, Bug is already present in 2.74.

@Campbell Barton (campbellbarton) I'm working on a file for the boolean part. If collada and FBX exporter manage to export the geometry correctly, even with 0 area face, why not make the obj export also work in such cases like the 2 others?

it's the result of boolean modifier on a cube. Bug happen with boolean modifier too (so a simple cube with cubes cutting it). I applied everything and removed some parts for simplification purpose.

just tested, collada export doesn't has this bug

oups sorry

Thanks for the quick fix, but maybe only one fix would be enough? Don't know if the 2 fixes together may decrease performance?

The thing is that it looks right in viewport before applying. If the bad geometry is the right one, then it should also look bad before applying. A modifier should look the same applied or not.

@rm beer (rmbeer) couldn't reproduce either with the 2 files. Maybe get a fresh buildbot and delete your config folder?

it seems to only happen with AO approx activated. I'll have a more in depth look tomorrow.

only use one function, performance stay the same.

It looked way too much, so I tested again. Time differences were due to throttling issue. When throttling is removed, both version render at the same speed. I'll update the diff with the new one.

With one function, the scandinavian scene from Chocofur renders in 471sec against 426s with the current version. So the current version renders 10.6% more frames in the same time.
Here is the diff to test:

I can confirm 17.9.1 still crashes on Vega and RX 480 with another machine and a clean install. It seems somehow the OpenCL driver wasn't actualized with normal install. So in the meantime, staying on 17.7.1 is recommended.

@Lennard Haaks (lennardh) both your file and my file rendered on latest buildbot using 17.9.1 on RX480. I did a normal install from 17.7.1, but OpenCL driver should have been actualized. I'll try today on Vega to see with clean install.

driver 17.9.1 resolves the bug.

RC1 doesn't work, build from 21 July does with 17.8.2

Driver 17.7.1 works, 17.8.2 doesn't.

I think Aaron means to make them only visible in top view or camera view for example. That's indeed very usefull. And if it works for empties, it would be great to have it work for all objects. I would add a kind of sprite rendering like in old games, where the image always faces the view, but can also be constrained to only rotate around Z for example. The empty would then have a list of views ([always show, camera, front, back, top, bottom]) were they are displayed or not, and the constraints. Blender current constraint don't work if you have multiple 3D Viewports with different views, but with OpenGL it's ok.
The X-Ray option to draw behind or in front should be enough to have then all functionalities with a much better usability.

Updated the diff with required changes. As Nvidia now also support OCL 2.0, I suppose it will make sens to switch to it for 2.8x. So I made 2 functions to workaround the address space problem of OCL 1.2. When we switch to 2.0, removing the second function will be easy and until then, changing the code is bundled in one place. If you prefer another workaround, I'm open to it.

I agree to deduplicate, although doing it as part of https://developer.blender.org/D2644 would maybe make more sens?

I remember you already had a patch to make the optimization before the kernel compile options are set/gathered. Could you share it in the tracker?

on simple scenes where the global size stays at or above 1024 on x, the speed impact is negligable. The default cube is even faster with 4spp bevel than without ... 14.05sec with and 15.36sec without. If you fill the camera FOV with cubes at 4spp, render time goes from 33sec without to 40sec with = about 21% slower. With 16spp, it goes up to 56sec = 70% slower

Welcome back Brecht :)
Regarding performance on split kernel, using a RX VEGA 64 on power save mode (to prevent throttling and get stable results):

• Applying latest revision on latest master and adding an invisible cube with bevel at 4 samples to just get the code compiled in the Barcelona scene, global size fall from 1024 to 576 on x.
• Render time increase from 430seconds to 478seconds (without synchronization time)

note that the edgeloop pairs are made correctly, so that the same loops are connected, bug happen on vertex level somehow.

Very helpful for review. It's a pleasure to see a student with so long commitment.

@Sergey Sharybin (sergey) T51975 shows variants of bug that pops up when this carve object is in scene. Sometime, some part of the mesh (the box containing the vegetables) was missing, sometime the textures were pink, sometime the shader got white. Most of the time hitting F12 would segfault. Sometime it succeeded, but the segfault appeared just after render finished (couldn't even save the file).
@Fable Fox (fablefox) debug builds indeed are working. Of course, replacing carve with bmesh also makes the file stable again.
To more easily the bugs and crash, you have to:

• open with VS2013 buildbots
• append everything in the file attached into a very complex scene (there must be something to corrupt in memory, so many shaders, lot of meshes, etc. make crash more often)
• duplicate in place ( location may be important to trigger the bug) what you appended to make the probability of crash higher (the overflows are more likely to produce a crash if many part of memory are overwritten/corrupted, so duplicate it a hundred time if it doesn't crash)
• then try to render, undo, etc...

This part of the scene was modeledin november last year and wasn't touched since then. The bugs and crash happened more often as scene complexity increased.

Note that the crash on file open is only on VS2015 release, but instability is in VS2013 = official release too. It just happens randomly during a material change or an undo or a final render start or whatever part of memory was corrupted.

ok, this is related to https://developer.blender.org/T52163. When I remove all object with carve boolean modifier, it renders good again. So it seems there is an overflow due to carve, creating random bugs like crashes, missing textures or geometry, depending on which part of memory is being overwritten.

In T47811#445375, @Campbell Barton (campbellbarton) wrote:

@mathieu menuet (bliblubli), this isn't enough info, are you interested to create a task that includes proposed changes? (something like T51219).

Builds with this patch and ASAN enabled on Linux can't render the default cube scene (crash) with cycles and OpenCL split kernel.
VS2013 builds start rendering but stay forever with an idle GPU on the first tile on complex scenes.
On Linux without ASAN it renders correctly and fast.

another thing is, when I render on 2 GPUs a scene that takes more than dedicated memory, the system memory will be used. If those generated textures are kept around, we have 3 times the textures. The cache Blender silently creates and the 2 versions created by the OpenCL driver. As the time needed to initialize rendering is not much faster the second time compared to using the cache, but many GB memory are away, making it more certain that swapping will be triggered, the cache may greatly slow down render for a benefit which is pretty low, at least in my tests.

ok, then it's not a bug, but wouldn't it make sens to only have this behavior when "persistent data" is checked and free it otherwise?

Really helpfull addition and it seems very robust. It shows that the latest AMD drivers also allows to use swap for buffers. I managed to render a scene that took 36GB on a 16GB (system) and 8GB (RX480) system. Of course, it was slow, but still interesting.
Only thing that it showed is that cycles seem to have like a memory leak or hidden cache. See report https://developer.blender.org/T52135

please attach a file to reproduce the problem.

As suggested today on IRC, making robust naming conventions for bmesh operators would help new coders to jump in. It will be very helpful for making the nodal everything work faster. At the moment, the use of for example underscores is pretty random. Having easy to search names will give more time for real coding.

edited task description to reflect last fixes to the new depsgraph.

opened new task instead https://developer.blender.org/T51997

After further investigation, not only the hidden particle system is triggering problems. In old depsgraph, it the subdivision modifier is also hidden, the particle system is evaluated, but at the wrong place. Only solution is to manually make it visible in viewport and change something that triggers a new evaluation many times, until it updates (like setting the particle system on and off. In some case, it has to be done 3 or 4 time to really update the particles positions)

I can confirm the bug is still there on windows. Two cases: user pref is there and only "load factory settings" will mess up. Or there is no userpref and then file->new and "load factory settings" will messup the UI.

I simplified the file as requested:

Having BW textures stored as single channel would also help to save 3/4 of the memory.

ok, I can confirm it crashes although the scene is very simple. Only planes and some textures, but nothing fancy as far as I can tell. Only 64 x 8bit textures at 4096*4096, which is 64MB per texture with RGBA, which is 4288MB of textures at max.
It looks like all textures try to be allocated at once, instead of 4GB chuncks, which is crazy and would prevent using today's GPU memory with low poly scene with a lot of textures.
@Stefan Werner (swerner) you wrote the code for infinite textures, maybe you can have a look here on how to allocate chunks of 4GB max at once.
Limiting rendering to less than 70 x 8 bit textures in 4K in 2017 is way too low. Many displacement maps today are 16 or 32bit, HDRs are also very big.
As an element of comparison, Luxcore 1.6, which is now more than a year old takes 1100MB (1400MB used of which 300MB are Blender's UI) of memory to render it.

1. Did you update your driver to latest version
2. OpenCL can use your system memory, so there should be no memory limit other than your GPU+ available motherboard memory

Anyway, please share a file before adding more text.

In my tests, noise is even, which is good and overhead of analysis seem to be low. But setting a norm of 0.005 and steps of 4 gives the same result as norm of 0.01 and steps of 16. A norm of 0.01 and a step of 4 halves the time compared to same norm but step of 16, although in theory, the max difference should be of 12 samples between those 2 settings. As I rendered with max 1000 samples and a norm of 0.01 compares in quality with 200 samples, a step of 4 could stop in the strongest discrepancy case at 200 samples, while the 16 step case would stop at 212samples, which wouldn't make a 2x longer render time. Plus the higher step value should reduce the overhead of image comparison.
norm 0.01 step 16
render time: 4minutes

thanks for the update. For some reason, when used with denoising, it is partly transparent.
Here, the sky in Barcelona:

after some time, I managed to make a file that has the bug in release build and open in debug build. But it renders correctly there and won't crash after rendering is finished. Some things more are printed to console however. Dunno if it can give any hint. Here it is:

Error: Could not acquire buffer from image
Error: Could not acquire buffer from image
Error: Could not acquire buffer from image
Error: Could not acquire buffer from image
Error: Could not acquire buffer from image
Error: Could not acquire buffer from image
Error: Could not acquire buffer from image
Error: Not freed memory blocks: 278, total unfreed memory 0.084785 MB

At least on windows, there is no crash. I tested on latest buildbot, VS2015, on victor scene, reverted during synchronization with new depsgraph ('using new depsgraph' was printed in console, so there is no typing error on my side). Maybe it's Linux-specific or random due to some threading issues?

The file is not available anymore. Please attach a file that really trigger the problem on latest master (we don't support 2.78 anymore). I render files that need 16GB of memory on the RX480 without problem, so it must come from something else.
Also update your drivers before testing further.

I can confirm the bug. All tiles hitting something render normally, until the edge of the view is reached (the tile at bottom right in my case). When there is nothing to render, neither geometry nor background, it hangs.

In D2662#64177, @Brecht Van Lommel (brecht) wrote:

Ideally the adaptive sampling should take color management into account indeed, though in general it's unknown what kind of image transforms might be applied in compositing. A simple color_scene_linear_to_srgb() or sqrt() might already see a noticeable improvement.

looks good :)

Do you have a recent version of the patch that applies on latest master? I couldn't apply it, even with arcanist.

In D2662#64173, @Milan Jaros (jar091) wrote:

In D2662#64168, @mathieu menuet (bliblubli) wrote:

If the user could paint areas needing more sample and continue render, it would avoid the limitations of computed versions for still images and remove the overhead.

Hi,

you can check the headlight in the table:
http://blender.it4i.cz/research/adaptive-rendering-based-on-a-norm-of-rendered-pixels/bmw27/

I think the algorithm detects the noise in the headlight very well.

Milan

If the user could paint areas needing more sample and continue render, it would avoid the limitations of computed versions for still images and remove the overhead.

@Bastien Montagne (mont29) If the problem solution is as simple as deleting the group when deleting the last instance, why not do it directly for the user? At least for the MB garbage it makes when making linked librairies local, wouldn't it be possible to solve the problem when the groups are made local?

I can also confirm the bug.

with 256x256 tiles (recommended size in the manual), using latest master and latest drivers from AMD on win7 x64, your file render in 1min46 with one RX480. So it would be 53s on 2 cards, which is faster than the time you report with 2.78c using 2 cards.
Second thing is that BPT wasn't available for the split kernel in 2.78c, so the file couldn't render.
Setting as incomplete until more informations are provided.

@Renato Meneses (renatomeneses) it is indeed very unlikely to happen. In very intensive rendering scene, cycles uses max 85W on my RX480, while the card draws 160W (above the 150W of PCIE specs) while rendering real time benchmark scenes like furmark. Cycles heavily underuse the GPUs (on both CUDA and OpenCL) compared to real time 3D apps, especially benchmarks. So the only thing that could burn it while rendering would be if your fans are defects, but even then, the hardware should just shut your PC down to protect it.
Whatever happened, it's not the software job to monitor temperature. Also note that overclocking, changing voltage and all other modification leading to warranty void are allowed by drivers from all vendors, but are risky and thus not supported. So I would recommand you to use latest build, which is much faster for your configuration anyway, but leave all driver options at defaults.
2.78c is not supported anymore. But I'll let @Mai Lavelle (maiself) and @Sergey Sharybin (sergey) decide here.

I also have some Blendermarket materials, so I had a look as it may have come from cycles. It comes from the cgc_oak material. But what is impressive is although I spend a lot of time to bisect, only retaining a glossy shader and the material output is enough to reproduce the crash. Of course, neither the full nor the simplified material crash on normal rendering with cycles. So I think the problem doesn't come from cycles.
Here it is, a cube with 1 shader creating a seg fault:

Hi Lukas,
this patch saves huge amount of preprocessing times, but seem to only work in simple scenes. The BMW benchmark uses it perfectly, but rendering the Viktor scene twice does the whole preprocessing a second time. As it's a really useful patch, an update would be really awesome :)

Hi,
Thanks for the report, but this addon is not part of official blender. You can report this to the original addon author.

it seems to work properly on another computer, will investigate more. Closing in the meantime.

Kernels recompiles now, but on every change, even those that don't change anything to the used kernel. For example, max_closure is set to 64 for viewport rendering to prevent recompiling. Since this patch however, adding a shader to the nodetree triggers a recompile, although having 1 or 2 shader is not important as it will be set to 64 anyway.

Hi Lukas, great to see you back on other task than Denoiser :)
When trying to build on MSVC 2015, it prints some errors: