Metropolis Sampling #38401

Closed
opened 2014-01-29 21:18:15 +01:00 by Lukas Stockner · 405 comments
Member

I recently decided to try adding a Metropolis sampler (no BPT, just the sampler) to Cycles to get some experience with the Cycles code layout, but it turned out working so well that I decided to post it here. Due to some problems, it's not releasable yet, but these problems (more below) should be not too hard to fix.
The code is based on the SmallLuxGPU metropolis sampler, after I first tried the PBRT code, but that one didn't seem to work nearly as well.
Basically, it works by bypassing the RNG functions: When they are called and metropolis sampling is selected, they just return a value from a sample array that is stored in the RNG pointer. This allows the sampler to make only minimal changes to the kernel, it even uses the standard kernel_path_integrate. The sampler itself is in the CPUDevice thread.
My current test scene is a simple pool with a modifier-displaced water surface, an absorption volume in the water (works great, by the way), a glass pane on one sine of the water and a Sun-HDR-combination lighting the scene.
Pool_Path.png Pool_Metro.png
The upper image is the 2.69 release, while the lower one is with the metropolis patch. Both were rendered in equal time, but the patched version is a debug build.
From the Metropolis image, the biggest problem is obvious: Tiles. The current system requires one sampler per tile, so the seams are easily visible. Also, if one tile has large bright surfaces, the noise outside of those is worse than in other tiles. The solution for this would be one sampler per thread, all working on the whole image (which would require atomics for the buffer writes).
Another problem are passes, since they're currently written directly from the kernel. However, in Metropolis sampling, they need to be weighted, but this weight is only available after the kernel is done with the ray. A solution for this would be to return the values to be written to the buffer from the kernel, so that the Device is responsible for storing them. The depth, normal, ObjectID and Alpha passes could be done in a single-pass regular pathtrace.
Also, there currently is a bug that causes standard pathtracing to crash, I still have to find the source of this one.
However, this seems like a promising feature that might be worth the work fixing the problems above.

PS: A one-hour-render of the pool looks like this: Pool_Metro_full.png

The patch is here: metropolis.diff

I recently decided to try adding a Metropolis sampler (no BPT, just the sampler) to Cycles to get some experience with the Cycles code layout, but it turned out working so well that I decided to post it here. Due to some problems, it's not releasable yet, but these problems (more below) should be not too hard to fix. The code is based on the SmallLuxGPU metropolis sampler, after I first tried the PBRT code, but that one didn't seem to work nearly as well. Basically, it works by bypassing the RNG functions: When they are called and metropolis sampling is selected, they just return a value from a sample array that is stored in the RNG pointer. This allows the sampler to make only minimal changes to the kernel, it even uses the standard kernel_path_integrate. The sampler itself is in the CPUDevice thread. My current test scene is a simple pool with a modifier-displaced water surface, an absorption volume in the water (works great, by the way), a glass pane on one sine of the water and a Sun-HDR-combination lighting the scene. ![Pool_Path.png](https://archive.blender.org/developer/F75616/Pool_Path.png) ![Pool_Metro.png](https://archive.blender.org/developer/F75619/Pool_Metro.png) The upper image is the 2.69 release, while the lower one is with the metropolis patch. Both were rendered in equal time, but the patched version is a debug build. From the Metropolis image, the biggest problem is obvious: Tiles. The current system requires one sampler per tile, so the seams are easily visible. Also, if one tile has large bright surfaces, the noise outside of those is worse than in other tiles. The solution for this would be one sampler per thread, all working on the whole image (which would require atomics for the buffer writes). Another problem are passes, since they're currently written directly from the kernel. However, in Metropolis sampling, they need to be weighted, but this weight is only available after the kernel is done with the ray. A solution for this would be to return the values to be written to the buffer from the kernel, so that the Device is responsible for storing them. The depth, normal, ObjectID and Alpha passes could be done in a single-pass regular pathtrace. Also, there currently is a bug that causes standard pathtracing to crash, I still have to find the source of this one. However, this seems like a promising feature that might be worth the work fixing the problems above. PS: A one-hour-render of the pool looks like this: ![Pool_Metro_full.png](https://archive.blender.org/developer/F75621/Pool_Metro_full.png) The patch is here: [metropolis.diff](https://archive.blender.org/developer/F75623/metropolis.diff)
Author
Member

Changed status to: 'Open'

Changed status to: 'Open'
Author
Member

Added subscriber: @LukasStockner

Added subscriber: @LukasStockner

Added subscriber: @Ace_Dragon

Added subscriber: @Ace_Dragon

Cool stuff, I really didn't expect this and it would definitely make for much easier rendering of caustics in Cycles (especially in conjunction with 'filter glossy' to help with those more difficult lightpaths).

By any chance (and this might be some crazy idea), but would it be possible to only apply the metropolis sampler for certain lightpath types (like the ones that create caustic effects), because you can already get away easily with plain pathtracing with plain diffuse bounces or other non-caustic situations (except for cases with tiny lights of course)?

Just an idea to toss around since Cycles has the functionality needed to obtain information from paths (hence thelight path node).

I think storm_st should also look at this, perhaps he can move his bidirectional sampling efforts to work off of your code.

Cool stuff, I really didn't expect this and it would definitely make for much easier rendering of caustics in Cycles (especially in conjunction with 'filter glossy' to help with those more difficult lightpaths). By any chance (and this might be some crazy idea), but would it be possible to only apply the metropolis sampler for certain lightpath types (like the ones that create caustic effects), because you can already get away easily with plain pathtracing with plain diffuse bounces or other non-caustic situations (except for cases with tiny lights of course)? Just an idea to toss around since Cycles has the functionality needed to obtain information from paths (hence the*light path* node). I think storm_st should also look at this, perhaps he can move his bidirectional sampling efforts to work off of your code.

Added subscriber: @MatthewHeimlich

Added subscriber: @MatthewHeimlich

Added subscriber: @mib2berlin

Added subscriber: @mib2berlin

Hi, testing the patch now.
I got only black for Glass BSDF, could you add your test file, please?
Looks promising. :)

Cheers, mib.

Hi, testing the patch now. I got only black for Glass BSDF, could you add your test file, please? Looks promising. :) Cheers, mib.

After patching, preview seems to be broken. Looks like only a couple of samples are updated into the viewport. F12 render works just fine.

Testing on Linux 64-bit. Very excited to play around with this! Thanks for the contribution.

After patching, preview seems to be broken. Looks like only a couple of samples are updated into the viewport. F12 render works just fine. Testing on Linux 64-bit. Very excited to play around with this! Thanks for the contribution.

Added subscriber: @MaciejJutrzenka

Added subscriber: @MaciejJutrzenka

Would be awesome to have Metropolis in cycles. I even can donate some coder to make it happen :)

Would be awesome to have Metropolis in cycles. I even can donate some coder to make it happen :)

Added subscriber: @tuqueque

Added subscriber: @tuqueque
Author
Member

The broken preview render is probably because every time the CPUDevice thread is called, a new sampler is created with the global integrator seed. This could be fixed by storing the sampler data in the tile, I'll try that once I'm at home.

The broken preview render is probably because every time the CPUDevice thread is called, a new sampler is created with the global integrator seed. This could be fixed by storing the sampler data in the tile, I'll try that once I'm at home.

Added subscriber: @tychota

Added subscriber: @tychota

Removed subscriber: @tychota

Removed subscriber: @tychota

Added subscriber: @tychota

Added subscriber: @tychota
Author
Member

Edit: Sorry for the doublepost.

Edit: Sorry for the doublepost.

Added subscriber: @ThomasDinges

Added subscriber: @ThomasDinges

I don't want to spoil the fun here (results look great), but isn't SmallLuxGPU GPL code?
I know that they re licensed the LuxRays code recently to Apache 2.0, so if your work is based on that, it's fine.

Some clarification here would be appreciated (link to the sources this patch is based on).

I don't want to spoil the fun here (results look great), but isn't SmallLuxGPU GPL code? I know that they re licensed the LuxRays code recently to Apache 2.0, so if your work is based on that, it's fine. Some clarification here would be appreciated (link to the sources this patch is based on).

Added subscriber: @lsscpp

Added subscriber: @lsscpp

Tested on vc2008 (Windows) now, and when I just switch to the Metropolis sampler, I get a mostly black/wrong image. Both Preview and F12.

Tested on vc2008 (Windows) now, and when I just switch to the Metropolis sampler, I get a mostly black/wrong image. Both Preview and F12.

Added subscriber: @sanne

Added subscriber: @sanne
Author
Member

The code is based on 48e44c150f/src/slg/sampler/sampler.cpp , which, according to the header, is Apache 2.0 licensed. I'll upload my test file as soon as I'm on my PC. I'm working on Linux x64, but I could also test it on Windows, although there shouldn't be any platform-dependent code in there.

The code is based on https://bitbucket.org/luxrender/luxrays/src/48e44c150fd53c00a8dd8722efdcaf870a561da0/src/slg/sampler/sampler.cpp , which, according to the header, is Apache 2.0 licensed. I'll upload my test file as soon as I'm on my PC. I'm working on Linux x64, but I could also test it on Windows, although there shouldn't be any platform-dependent code in there.

Thanks for the clarification, good to hear. :)

Thanks for the clarification, good to hear. :)

Tested the patch on Mac OS now, with clang compiler. Same result.
I just open one of our test files: https://svn.blender.org/svnroot/bf-blender/trunk/lib/tests/cycles/ and switch to the MLT sampler.

Render is different, lot of errors. color_ramp.png (color_ramp.blend in this example).

Tested the patch on Mac OS now, with clang compiler. Same result. I just open one of our test files: https://svn.blender.org/svnroot/bf-blender/trunk/lib/tests/cycles/ and switch to the MLT sampler. Render is different, lot of errors. ![color_ramp.png](https://archive.blender.org/developer/F75662/color_ramp.png) (color_ramp.blend in this example).

Added subscriber: @marcog

Added subscriber: @marcog

@LukasStockner: I found the issue.
Your Metropolis code in device_cpu.cpp comes after the optimized kernels (AVX, SSE41....).

I added a quick #if 0 around those, so on runtime those are skipped and the non optimized kernel (with your MLT code) gets used. :)

Edit: Run some more tests now. Caustics are better with MLT, but other things (Diffuse surfaces, background) are much more noisy. Good start though.

CC'ing @brecht, I guess he will find this interesting. :)

@LukasStockner: I found the issue. Your Metropolis code in device_cpu.cpp comes after the optimized kernels (AVX, SSE41....). I added a quick #if 0 around those, so on runtime those are skipped and the non optimized kernel (with your MLT code) gets used. :) Edit: Run some more tests now. Caustics are better with MLT, but other things (Diffuse surfaces, background) are much more noisy. Good start though. CC'ing @brecht, I guess he will find this interesting. :)
Author
Member

Ah, ok. I disabled them in my scons config, forgot about that -.-. Glad that it works now.

The overall noise in the image is pretty much expected, but a few tricks (clamping importance, user-provided importance map, noise-aware sampling) might help out with that. Also, Metropolis sampling is naturally better suited for long rendering times instead of quick previews.

Ah, ok. I disabled them in my scons config, forgot about that -.-. Glad that it works now. The overall noise in the image is pretty much expected, but a few tricks (clamping importance, user-provided importance map, noise-aware sampling) might help out with that. Also, Metropolis sampling is naturally better suited for long rendering times instead of quick previews.

Added subscriber: @FlorianRichter

Added subscriber: @FlorianRichter
Author
Member

Ok, so here's my test scene. I replaced the HDR with a sky node because it's too big to upload here.

Testscene_Pool.blend

Ok, so here's my test scene. I replaced the HDR with a sky node because it's too big to upload here. [Testscene_Pool.blend](https://archive.blender.org/developer/F75670/Testscene_Pool.blend)

Another heads up: Using Filter Glossy seems to break rendering, not sure what's going on. I'm going to try to build on my Windows desktop now, been limited to a single core on my laptop so far.

Another heads up: Using Filter Glossy seems to break rendering, not sure what's going on. I'm going to try to build on my Windows desktop now, been limited to a single core on my laptop so far.
Author
Member

New patch version, the SSE/AVX builds work now too, they just aren't executed if Metropolis is selected (Of cource, I will later add code for Metropolis to also work with those). Preview is still not fixed, the RenderTiles seem to be replaced every iteration, I still have to find a way around this (Storing in the CPUDevice doesn't work as well).

The first patch is incremental from the old one, the second one is from trunk.

metropolis_1_to_2.diff
metropolis_2.diff

New patch version, the SSE/AVX builds work now too, they just aren't executed if Metropolis is selected (Of cource, I will later add code for Metropolis to also work with those). Preview is still not fixed, the RenderTiles seem to be replaced every iteration, I still have to find a way around this (Storing in the CPUDevice doesn't work as well). The first patch is incremental from the old one, the second one is from trunk. [metropolis_1_to_2.diff](https://archive.blender.org/developer/F75674/metropolis_1_to_2.diff) [metropolis_2.diff](https://archive.blender.org/developer/F75675/metropolis_2.diff)
Member

Added subscriber: @Lockal

Added subscriber: @Lockal

Added subscriber: @JasonClarke

Added subscriber: @JasonClarke
  • include <stdint.h> in kernel_types.h, breaks compilation on msvc2008. I just removed it and cannot see issues. I don't think we need that anyway.
- include <stdint.h> in kernel_types.h, breaks compilation on msvc2008. I just removed it and cannot see issues. I don't think we need that anyway.
Author
Member

New patch version, render passes and SSE/AVX works now.
The UV, Normal, ID and Alpha passes are done in a one-sample path tracing prepass, while all other passes are done with the metropolis sampler.
SSE and AVX is now used for metropolis rendering as well.
The pure pathtracing doesn't crash anymore, apparently I fixed the bug by the way.
@MatthewHeimlich: Could you please upload a test file where Filter Glossy crashes, for me it works fine...

The next steps are now Preview rendering and trying out the Quasi-random extension in the Metropolis sampler of the regular LuxRender (no code copying this time, since this code is GPL)

Patch 2 to 3: metropolis_2_to_3.diff

Trunk to Patch 3: metropolis_3.diff

New patch version, render passes and SSE/AVX works now. The UV, Normal, ID and Alpha passes are done in a one-sample path tracing prepass, while all other passes are done with the metropolis sampler. SSE and AVX is now used for metropolis rendering as well. The pure pathtracing doesn't crash anymore, apparently I fixed the bug by the way. @MatthewHeimlich: Could you please upload a test file where Filter Glossy crashes, for me it works fine... The next steps are now Preview rendering and trying out the Quasi-random extension in the Metropolis sampler of the regular LuxRender (no code copying this time, since this code is GPL) Patch 2 to 3: [metropolis_2_to_3.diff](https://archive.blender.org/developer/F75812/metropolis_2_to_3.diff) Trunk to Patch 3: [metropolis_3.diff](https://archive.blender.org/developer/F75813/metropolis_3.diff)

Hi Lucas, get crash with metropolis_3.diff and fed1b8b.

<

Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffc7fff700 (LWP 18135)]

0x00007ffff2850849 in raise () from /lib64/libc.so.6

(gdb) bt
  • 0 0x00007ffff2850849 in raise () from /lib64/libc.so.6
  • 1 0x00007ffff2851cd8 in abort () from /lib64/libc.so.6
  • 2 0x00007ffff288f114 in __libc_message () from /lib64/libc.so.6
  • 3 0x00007ffff289496e in malloc_printerr () from /lib64/libc.so.6
  • 4 0x00007ffff2895647 in _int_free () from /lib64/libc.so.6
  • 5 0x000000000125d4f3 in ccl::RenderBuffers::device_free() ()
  • 6 0x000000000125d53c in ccl::RenderBuffers::~RenderBuffers() ()
  • 7 0x0000000001290c92 in ccl::Session::release_tile(ccl::RenderTile&) ()
  • 8 0x00000000012ab199 in boost::function1<void, ccl::RenderTile&>::operator()(ccl::RenderTile&) const ()
  • 9 0x00000000012ab9e2 in ccl::CPUDevice::thread_path_trace(ccl::DeviceTask&) ()
  • 10 0x000000000146aec8 in ccl::TaskScheduler::thread_run(int) ()
  • 11 0x0000000001294829 in ccl:🧵:run(void*) ()
  • 12 0x00007ffff63c70db in start_thread () from /lib64/libpthread.so.0
  • 13 0x00007ffff290290d in clone () from /lib64/libc.so.6
(gdb) 

BlankComparisonScene_cycles.blend

Cheers, mib.


Hi Lucas, get crash with metropolis_3.diff and fed1b8b. < ``` Program received signal SIGABRT, Aborted. [Switching to Thread 0x7fffc7fff700 (LWP 18135)] ``` 0x00007ffff2850849 in raise () from /lib64/libc.so.6 ``` (gdb) bt ``` - 0 0x00007ffff2850849 in raise () from /lib64/libc.so.6 - 1 0x00007ffff2851cd8 in abort () from /lib64/libc.so.6 - 2 0x00007ffff288f114 in __libc_message () from /lib64/libc.so.6 - 3 0x00007ffff289496e in malloc_printerr () from /lib64/libc.so.6 - 4 0x00007ffff2895647 in _int_free () from /lib64/libc.so.6 - 5 0x000000000125d4f3 in ccl::RenderBuffers::device_free() () - 6 0x000000000125d53c in ccl::RenderBuffers::~RenderBuffers() () - 7 0x0000000001290c92 in ccl::Session::release_tile(ccl::RenderTile&) () - 8 0x00000000012ab199 in boost::function1<void, ccl::RenderTile&>::operator()(ccl::RenderTile&) const () - 9 0x00000000012ab9e2 in ccl::CPUDevice::thread_path_trace(ccl::DeviceTask&) () - 10 0x000000000146aec8 in ccl::TaskScheduler::thread_run(int) () - 11 0x0000000001294829 in ccl::thread::run(void*) () - 12 0x00007ffff63c70db in start_thread () from /lib64/libpthread.so.0 - 13 0x00007ffff290290d in clone () from /lib64/libc.so.6 ``` (gdb) ``` > [BlankComparisonScene_cycles.blend](https://archive.blender.org/developer/F75817/BlankComparisonScene_cycles.blend) Cheers, mib. ``` ```

Open file, change to Metropolis, F12.

Mib

Open file, change to Metropolis, F12. Mib
Author
Member

@mib2berlin Thanks for the report, however, I can't reproduce it, even with fed1b8b and your file (by the way, two image textures were missing, but I can't imagine that they caused the bug...). Could you please build your Blender with debug info and post a backtrack again?

Either way, metropolis_4 should be ready soon, now stopping the render works again and the MCQMC extension works quite well, too.

@mib2berlin Thanks for the report, however, I can't reproduce it, even with fed1b8b and your file (by the way, two image textures were missing, but I can't imagine that they caused the bug...). Could you please build your Blender with debug info and post a backtrack again? Either way, metropolis_4 should be ready soon, now stopping the render works again and the MCQMC extension works quite well, too.

Thanks, does not crash with debug build, get crash again with release build but different BT.

<
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffbffff700 (LWP 32593)]
0x00007ffff2894a82 in malloc_consolidate () from /lib64/libc.so.6
(gdb) bt

  • 0 0x00007ffff2894a82 in malloc_consolidate () from /lib64/libc.so.6
  • 1 0x00007ffff2895cd8 in _int_malloc () from /lib64/libc.so.6
  • 2 0x00007ffff2898993 in calloc () from /lib64/libc.so.6
  • 3 0x0000000001013a8f in MEM_lockfree_callocN ()
  • 4 0x0000000000baf1a5 in render_result_new ()
  • 5 0x0000000000b89f3a in RE_engine_begin_result ()
  • 6 0x000000000122f034 in ccl::BlenderSession::do_write_update_render_tile(ccl::RenderTile&, bool) ()
  • 7 0x0000000001290e81 in ccl::Session::update_tile_sample(ccl::RenderTile&) ()
  • 8 0x00000000012bf88d in ccl::DeviceTask::update_progress(ccl::RenderTile&) ()
  • 9 0x00000000012ab952 in ccl::CPUDevice::thread_path_trace(ccl::DeviceTask&) ()
  • 10 0x000000000146afa8 in ccl::TaskScheduler::thread_run(int) ()
  • 11 0x00000000012947c9 in ccl:🧵:run(void*) ()
  • 12 0x00007ffff63c70db in start_thread () from /lib64/libpthread.so.0
  • 13 0x00007ffff290290d in clone () from /lib64/libc.so.6
    (gdb)

Opensuse Linux 13.1/64
Intel i5 3770K
GTX 760
GTX 560Ti 448 Cores
Driver 331.20

Build 482823a

Maybe it is Linux only.

Cheers, mib.

Thanks, does not crash with debug build, get crash again with release build but different BT. < Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fffbffff700 (LWP 32593)] 0x00007ffff2894a82 in malloc_consolidate () from /lib64/libc.so.6 (gdb) bt - 0 0x00007ffff2894a82 in malloc_consolidate () from /lib64/libc.so.6 - 1 0x00007ffff2895cd8 in _int_malloc () from /lib64/libc.so.6 - 2 0x00007ffff2898993 in calloc () from /lib64/libc.so.6 - 3 0x0000000001013a8f in MEM_lockfree_callocN () - 4 0x0000000000baf1a5 in render_result_new () - 5 0x0000000000b89f3a in RE_engine_begin_result () - 6 0x000000000122f034 in ccl::BlenderSession::do_write_update_render_tile(ccl::RenderTile&, bool) () - 7 0x0000000001290e81 in ccl::Session::update_tile_sample(ccl::RenderTile&) () - 8 0x00000000012bf88d in ccl::DeviceTask::update_progress(ccl::RenderTile&) () - 9 0x00000000012ab952 in ccl::CPUDevice::thread_path_trace(ccl::DeviceTask&) () - 10 0x000000000146afa8 in ccl::TaskScheduler::thread_run(int) () - 11 0x00000000012947c9 in ccl::thread::run(void*) () - 12 0x00007ffff63c70db in start_thread () from /lib64/libpthread.so.0 - 13 0x00007ffff290290d in clone () from /lib64/libc.so.6 (gdb) > Opensuse Linux 13.1/64 Intel i5 3770K GTX 760 GTX 560Ti 448 Cores Driver 331.20 Build 482823a Maybe it is Linux only. Cheers, mib.
Author
Member

Okay, this definitely looks like a out-of-bounds memory access, I'll look at it in Valgrind.
I'm on Linux too (Mint 16 64bit), so this shouldn't be a problem.

Okay, this definitely looks like a out-of-bounds memory access, I'll look at it in Valgrind. I'm on Linux too (Mint 16 64bit), so this shouldn't be a problem.
Author
Member

New patch version, MCQMC (Quasi-random numbers in the Metropolis sampler) works now, due to lack of reference images I'm not exacty sure whether it's an improvement or not, but it's definitely not worse. Also, the render is stoppable now again.

Regarding the memory bug: I fixed an out-of-bounds error (it didn't generate enough random numbers), however, I'm not sure whether this was the bug that caused the crash. Debugging this in Valgrind is notoriously difficult since it only appears sometimes and Blender in Valgrind is sloooow (30min and upwards for one iteration!).

I tried to bypass the Tile system when using metropolis, but it's such a central part of the Cycles code that this didn't work out at all. Instead, preview rendering now uses standard Path Tracing and F12 rendering still uses Tiles, this won't be as straightforward as I have thought...

metropolis_4.diff

New patch version, MCQMC (Quasi-random numbers in the Metropolis sampler) works now, due to lack of reference images I'm not exacty sure whether it's an improvement or not, but it's definitely not worse. Also, the render is stoppable now again. Regarding the memory bug: I fixed an out-of-bounds error (it didn't generate enough random numbers), however, I'm not sure whether this was the bug that caused the crash. Debugging this in Valgrind is notoriously difficult since it only appears sometimes and Blender in Valgrind is sloooow (30min and upwards for one iteration!). I tried to bypass the Tile system when using metropolis, but it's such a central part of the Cycles code that this didn't work out at all. Instead, preview rendering now uses standard Path Tracing and F12 rendering still uses Tiles, this won't be as straightforward as I have thought... [metropolis_4.diff](https://archive.blender.org/developer/F75883/metropolis_4.diff)

Hi Lucas, 4.diff give the the same error posted in my last post.
It crash on every scene I test.
If I switch to Progressive Refine it work.

With your patch it is not possible to render with GPU, gave cuda error.

Compiling CUDA kernel ...
"/usr/local/cuda-5.0/bin/nvcc" -arch=sm_20 -m64 --cubin "/daten/blender-git/build/bin/2.69/scripts/addons/cycles/kernel/kernel.cu" -o "/home/pepo/.config/blender/2.69/cache/cycles_kernel_sm20_8022DE0DC7069375EE41B6065DB5DB0F.cubin" --ptxas-options="-v" --maxrregcount=32 --use_fast_math -I"/daten/blender-git/build/bin/2.69/scripts/addons/cycles/kernel" -DNVCC -D__KERNEL_CUDA_VERSION__=50
/daten/blender-git/build/bin/2.69/scripts/addons/cycles/kernel/kernel_passes.h(50): error: too few arguments in function call

/daten/blender-git/build/bin/2.69/scripts/addons/cycles/kernel/kernel_passes.h(52): error: too few arguments in function call

/daten/blender-git/build/bin/2.69/scripts/addons/cycles/kernel/kernel_passes.h(56): error: too few arguments in function call

.
.
.

But is not important.

Thank you for your work, mib.

Hi Lucas, 4.diff give the the same error posted in my last post. It crash on every scene I test. If I switch to Progressive Refine it work. With your patch it is not possible to render with GPU, gave cuda error. ``` Compiling CUDA kernel ... "/usr/local/cuda-5.0/bin/nvcc" -arch=sm_20 -m64 --cubin "/daten/blender-git/build/bin/2.69/scripts/addons/cycles/kernel/kernel.cu" -o "/home/pepo/.config/blender/2.69/cache/cycles_kernel_sm20_8022DE0DC7069375EE41B6065DB5DB0F.cubin" --ptxas-options="-v" --maxrregcount=32 --use_fast_math -I"/daten/blender-git/build/bin/2.69/scripts/addons/cycles/kernel" -DNVCC -D__KERNEL_CUDA_VERSION__=50 /daten/blender-git/build/bin/2.69/scripts/addons/cycles/kernel/kernel_passes.h(50): error: too few arguments in function call /daten/blender-git/build/bin/2.69/scripts/addons/cycles/kernel/kernel_passes.h(52): error: too few arguments in function call /daten/blender-git/build/bin/2.69/scripts/addons/cycles/kernel/kernel_passes.h(56): error: too few arguments in function call . . . ``` But is not important. Thank you for your work, mib.
Member

@LukasStockner, you should try GCC Address Sanitizer , it is very fast, one can use it even with RelWithDebInfo or inside gdb. Here is the crashlog:
{P11, lines=15}
So it calls float4 operator+(const float4& a, const float4& b) inside kernel_write_pass_float4 with a null reference.

@LukasStockner, you should try [GCC Address Sanitizer ](http://wiki.blender.org/index.php/Dev:Doc/Tools/Debugging/GCC_Address_Sanitizer), it is very fast, one can use it even with RelWithDebInfo or inside gdb. Here is the crashlog: {[P11](https://archive.blender.org/developer/P11.txt), lines=15} So it calls `float4 operator+(const float4& a, const float4& b)` inside `kernel_write_pass_float4` with a null reference.

Someone maybe have link to the build for windows 7 64? I would like to make some test.

Someone maybe have link to the build for windows 7 64? I would like to make some test.
Author
Member

New patch version, no new features this time, instead I fixed another bug and added a check for a possible error source.

The GCC Address Sanitizer showed only the bug I fixed, now all scenes I tested (including BlankComparisonScene_cycles.blend) run error- and crash-free, even with the Address Sanitizer. If Blender still crashes for someone, please try compiling with the Address Sanitizer (if possible, with a debug build) and post the output here.
The GPU building bug was fixed, but there may be other issues remaining. Sadly, I can't test it since my GPU is still Compute Capability 1.3. However, the metropolis sampler is currently CPU-only, and this will probably not change so soon, since Metropolis sampling, in contrast to Sobol sampling, is inherently a sequential algorithm. The only way to work around this is using many sampling chains in parallel (basically, this is ERPT), but this isn't nearly as efficient as using only a small number of chains on the CPU.

Apart from bugfixes, the next steps will be the Tile system and better importance functions (some Noise-detecting algorithm could be useful, like the one described in http://graphics.cs.illinois.edu/papers/importance). Also, user-defined importance maps seem quite useful, but these would need a good UI. By the way, this should also work for non-metropolis sampling.

metropolis_5.diff

New patch version, no new features this time, instead I fixed another bug and added a check for a possible error source. The GCC Address Sanitizer showed only the bug I fixed, now all scenes I tested (including BlankComparisonScene_cycles.blend) run error- and crash-free, even with the Address Sanitizer. If Blender still crashes for someone, please try compiling with the Address Sanitizer (if possible, with a debug build) and post the output here. The GPU building bug was fixed, but there may be other issues remaining. Sadly, I can't test it since my GPU is still Compute Capability 1.3. However, the metropolis sampler is currently CPU-only, and this will probably not change so soon, since Metropolis sampling, in contrast to Sobol sampling, is inherently a sequential algorithm. The only way to work around this is using many sampling chains in parallel (basically, this is ERPT), but this isn't nearly as efficient as using only a small number of chains on the CPU. Apart from bugfixes, the next steps will be the Tile system and better importance functions (some Noise-detecting algorithm could be useful, like the one described in http://graphics.cs.illinois.edu/papers/importance). Also, user-defined importance maps seem quite useful, but these would need a good UI. By the way, this should also work for non-metropolis sampling. [metropolis_5.diff](https://archive.blender.org/developer/F76009/metropolis_5.diff)

Hi Lucas, no more crashes with the last patch withal my testfiles.
Cuda is working with my CC 2.0 and 3.0 cards.
Nice process, thanks.

I got message in terminal during render:

Pixel sampling error, expect crashing!

But render fine.

Cheers, mib.

Hi Lucas, no more crashes with the last patch withal my testfiles. Cuda is working with my CC 2.0 and 3.0 cards. Nice process, thanks. I got message in terminal during render: ``` Pixel sampling error, expect crashing! ``` But render fine. Cheers, mib.

Hi, this error message wrote 19GB in my.xsession-errors-:0 ! :)
Harddisk full (SSD) after some test render.

Cheers, mib.

Hi, this error message wrote 19GB in my.xsession-errors-:0 ! :) Harddisk full (SSD) after some test render. Cheers, mib.
Author
Member

Whoa, sorry about that :/ But this at least shows where the error lies: Somehow, the sampler goes outside of the image, I'll look at it. By the way: Is it possible that you use an image mutation range > 1 ?

Whoa, sorry about that :/ But this at least shows where the error lies: Somehow, the sampler goes outside of the image, I'll look at it. By the way: Is it possible that you use an image mutation range > 1 ?

Hi Lucas, I use Metropolis default settings in all tests.

Got crash now in debug build:

  • #0 0x00007ffff2894aab in malloc_consolidate () from /lib64/libc.so.6
  • #1 0x00007ffff2895cd8 in _int_malloc () from /lib64/libc.so.6
  • #2 0x00007ffff2898993 in calloc () from /lib64/libc.so.6
  • #3 0x00000000023d910b in MEM_lockfree_callocN (len=15360, str=0x336cf2f "Combined rgba")
  • at /daten/blender-git/blender/intern/guardedalloc/intern/mallocn_lockfree_impl.c:220
  • #4 0x00000000023d933c in MEM_lockfree_mapallocN (len=15360, str=0x336cf2f "Combined rgba")
  • at /daten/blender-git/blender/intern/guardedalloc/intern/mallocn_lockfree_impl.c:272
  • #5 0x0000000001c16ae6 in render_result_new (re=0x56edbf8, partrct=0x7fffc9807a40, crop=0, savebuffers=0, layername=0x7fffc0248788 "RenderLayer")
  • at /daten/blender-git/blender/source/blender/render/intern/source/render_result.c:515
  • #6 0x0000000001be1309 in RE_engine_begin_result (engine=0x7fffc01ffc98, x=64, y=150, w=32, h=30, layername=0x7fffc0248788 "RenderLayer")
  • at /daten/blender-git/blender/source/blender/render/intern/source/external_engine.c:208
  • #7 0x00000000025d1358 in RenderEngine_begin_result (_self=0x7fffc01ffc98, x=64, y=150, w=32, h=30, layer=0x7fffc0248788 "RenderLayer")
  • at /daten/blender-git/build/source/blender/makesrna/intern/rna_render_gen.c:2001
  • #8 0x00000000026f0795 in BL::RenderEngine::begin_result (this=0x7fffc9807b60, x=64, y=150, w=32, h=30, layer=0x7fffc0248788 "RenderLayer")
  • at /daten/blender-git/build/source/blender/makesrna/intern/RNA_blender_cpp.h:44412
  • #9 0x00000000026ebd83 in ccl::begin_render_result (b_engine=..., x=64, y=150, w=32, h=30, layername=0x7fffc0248788 "RenderLayer")
  • at /daten/blender-git/blender/intern/cycles/blender/blender_session.cpp:261
  • #10 0x00000000026ebec0 in ccl::BlenderSession::do_write_update_render_tile (this=0x7fffc0217700, rtile=..., do_update_only=true)
  • at /daten/blender-git/blender/intern/cycles/blender/blender_session.cpp:278
  • #11 0x00000000026ec2c9 in ccl::BlenderSession::update_render_tile (this=0x7fffc0217700, rtile=...)
  • at /daten/blender-git/blender/intern/cycles/blender/blender_session.cpp:326
  • #12 0x00000000026f6b9c in boost::_mfi::mf1<void, ccl::BlenderSession, ccl::RenderTile&>::operator() (this=0x7fffc023ea38, p=0x7fffc0217700, a1=...)
  • at /usr/include/boost/bind/mem_fn_template.hpp:165
  • #13 0x00000000026f6096 in boost::_bi::list2<boost::_bi::valueccl::BlenderSession*, boost::arg<1> >::operator()<boost::_mfi::mf1<void, ccl::BlenderSession, ccl::RenderTile&>, boost::_bi::list1ccl::RenderTile& > (this=0x7fffc023ea48, f=..., a=...) at /usr/include/boost/bind/bind.hpp:313
  • #14 0x00000000026f57fe in boost::_bi::bind_t<void, boost::_mfi::mf1<void, ccl::BlenderSession, ccl::RenderTile&>, boost::_bi::list2<boost::_bi::valueccl::BlenderSession*, boost::arg<1> > >::operator()ccl::RenderTile (this=0x7fffc023ea38, a1=...) at /usr/include/boost/bind/bind_template.hpp:32
  • #15 0x00000000026f5184 in boost::detail::function::void_function_obj_invoker1<boost::_bi::bind_t<void, boost::_mfi::mf1<void, ccl::BlenderSession, ccl::RenderTile&>, boost::_bi::list2<boost::_bi::valueccl::BlenderSession*, boost::arg<1> > >, void, ccl::RenderTile&>::invoke (function_obj_ptr=...,
  • a0=...) at /usr/include/boost/function/function_template.hpp:153
  • #16 0x00000000027e8103 in boost::function1<void, ccl::RenderTile&>::operator() (this=0x7fffc023ea30, a0=...)
  • at /usr/include/boost/function/function_template.hpp:767
  • #17 0x00000000027e3f1f in ccl::Session::update_tile_sample (this=0x7fffc023e7d0, rtile=...)
  • at /daten/blender-git/blender/intern/cycles/render/session.cpp:437
  • #18 0x00000000027eb3b2 in boost::_mfi::mf1<void, ccl::Session, ccl::RenderTile&>::operator() (this=0x7fffa801ea60, p=0x7fffc023e7d0, a1=...)
  • at /usr/include/boost/bind/mem_fn_template.hpp:165
  • #19 0x00000000027eadac in boost::_bi::list2<boost::_bi::valueccl::Session*, boost::arg<1> >::operator()<boost::_mfi::mf1<void, ccl::Session, ccl::RenderTile&>, boost::_bi::list1ccl::RenderTile& > (this=0x7fffa801ea70, f=..., a=...) at /usr/include/boost/bind/bind.hpp:313
  • #20 0x00000000027ea784 in boost::_bi::bind_t<void, boost::_mfi::mf1<void, ccl::Session, ccl::RenderTile&>, boost::_bi::list2<boost::_bi::valueccl::Session*, boost::arg<1> > >::operator()ccl::RenderTile (this=0x7fffa801ea60, a1=...) at /usr/include/boost/bind/bind_template.hpp:32
  • #21 0x00000000027ea2e2 in boost::detail::function::void_function_obj_invoker1<boost::_bi::bind_t<void, boost::_mfi::mf1<void, ccl::Session, ccl::RenderTile&>, boost::_bi::list2<boost::_bi::valueccl::Session*, boost::arg<1> > >, void, ccl::RenderTile&>::invoke (function_obj_ptr=..., a0=...)
  • at /usr/include/boost/function/function_template.hpp:153
  • #22 0x00000000027e8103 in boost::function1<void, ccl::RenderTile&>::operator() (this=0x7fffa801ea58, a0=...)
  • at /usr/include/boost/function/function_template.hpp:767
  • #23 0x000000000285678b in ccl::DeviceTask::update_progress (this=0x7fffa801e990, rtile=...)
  • at /daten/blender-git/blender/intern/cycles/device/device_task.cpp:103
  • #24 0x0000000002831019 in ccl::CPUDevice::thread_path_trace (this=0x7fffc023f400, task=...)
  • at /daten/blender-git/blender/intern/cycles/device/device_cpu.cpp:404
  • #25 0x000000000282f620 in ccl::CPUDevice::thread_run (this=0x7fffc023f400, task=0x7fffa801e990)
  • at /daten/blender-git/blender/intern/cycles/device/device_cpu.cpp:174
  • #26 0x000000000283370c in boost::_mfi::mf1<void, ccl::CPUDevice, ccl::DeviceTask*>::operator() (this=0x7fffa8011200, p=0x7fffc023f400,
  • a1=0x7fffa801e990) at /usr/include/boost/bind/mem_fn_template.hpp:165
  • #27 0x00000000028335bf in boost::_bi::list2<boost::_bi::valueccl::CPUDevice*, boost::_bi::valueccl::CPUDevice::CPUDeviceTask* >::operator()<boost::_mfi::mf1<void, ccl::CPUDevice, ccl::DeviceTask*>, boost::_bi::list0> (this=0x7fffa8011210, f=..., a=...) at /usr/include/boost/bind/bind.hpp:313
  • #28 0x000000000283331b in boost::_bi::bind_t<void, boost::_mfi::mf1<void, ccl::CPUDevice, ccl::DeviceTask*>, boost::_bi::list2<boost::_bi::valueccl::CPUDevice*, boost::_bi::valueccl::CPUDevice::CPUDeviceTask* > >::operator() (this=0x7fffa8011200) at /usr/include/boost/bind/bind_template.hpp:20
  • #29 0x0000000002833205 in boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf1<void, ccl::CPUDevice, ccl::DeviceTask*>, boost::_bi::list2<boost::_bi::valueccl::CPUDevice*, boost::_bi::valueccl::CPUDevice::CPUDeviceTask* > >, void>::invoke (
  • function_obj_ptr=...) at /usr/include/boost/function/function_template.hpp:153
  • #30 0x00000000026f112c in boost::function0::operator() (this=0x7fffa801e998) at /usr/include/boost/function/function_template.hpp:767
  • #31 0x0000000002d173b5 in ccl::TaskScheduler::thread_run (thread_id=3) at /daten/blender-git/blender/intern/cycles/util/util_task.cpp:249
  • #32 0x0000000002d1a922 in boost::_bi::list1<boost::_bi::value >::operator()<void (*)(int), boost::_bi::list0> (this=0x7fffc023f260,
  • f=@0x7fffc023f258: 0x2d17398 ccl::TaskScheduler::thread_run(int), a=...) at /usr/include/boost/bind/bind.hpp:253
  • #33 0x0000000002d1a5af in boost::_bi::bind_t<void, void (*)(int), boost::_bi::list1<boost::_bi::value > >::operator() (
  • this=0x7fffc023f258) at /usr/include/boost/bind/bind_template.hpp:20
  • #34 0x0000000002d1a137 in boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, void (*)(int), boost::_bi::list1<boost::_bi::value > >, void>::invoke (function_obj_ptr=...) at /usr/include/boost/function/function_template.hpp:153
  • #35 0x00000000026f112c in boost::function0::operator() (this=0x7fffc023f250) at /usr/include/boost/function/function_template.hpp:767
  • #36 0x00000000027e6f7b in ccl:🧵:run (arg=0x7fffc023f250) at /daten/blender-git/blender/intern/cycles/render/../util/util_thread.h:55
  • #37 0x00007ffff63c70db in start_thread () from /lib64/libpthread.so.0
  • #38 0x00007ffff290290d in clone () from /lib64/libc.so.6

I try with Address Sanitizer but got scrambled output because of error message.
Hope this help to catch the bug.

Cheers, mib.

Hi Lucas, I use Metropolis default settings in all tests. Got crash now in debug build: - #0 0x00007ffff2894aab in malloc_consolidate () from /lib64/libc.so.6 - #1 0x00007ffff2895cd8 in _int_malloc () from /lib64/libc.so.6 - #2 0x00007ffff2898993 in calloc () from /lib64/libc.so.6 - #3 0x00000000023d910b in MEM_lockfree_callocN (len=15360, str=0x336cf2f "Combined rgba") - at /daten/blender-git/blender/intern/guardedalloc/intern/mallocn_lockfree_impl.c:220 - #4 0x00000000023d933c in MEM_lockfree_mapallocN (len=15360, str=0x336cf2f "Combined rgba") - at /daten/blender-git/blender/intern/guardedalloc/intern/mallocn_lockfree_impl.c:272 - #5 0x0000000001c16ae6 in render_result_new (re=0x56edbf8, partrct=0x7fffc9807a40, crop=0, savebuffers=0, layername=0x7fffc0248788 "RenderLayer") - at /daten/blender-git/blender/source/blender/render/intern/source/render_result.c:515 - #6 0x0000000001be1309 in RE_engine_begin_result (engine=0x7fffc01ffc98, x=64, y=150, w=32, h=30, layername=0x7fffc0248788 "RenderLayer") - at /daten/blender-git/blender/source/blender/render/intern/source/external_engine.c:208 - #7 0x00000000025d1358 in RenderEngine_begin_result (_self=0x7fffc01ffc98, x=64, y=150, w=32, h=30, layer=0x7fffc0248788 "RenderLayer") - at /daten/blender-git/build/source/blender/makesrna/intern/rna_render_gen.c:2001 - #8 0x00000000026f0795 in BL::RenderEngine::begin_result (this=0x7fffc9807b60, x=64, y=150, w=32, h=30, layer=0x7fffc0248788 "RenderLayer") - at /daten/blender-git/build/source/blender/makesrna/intern/RNA_blender_cpp.h:44412 - #9 0x00000000026ebd83 in ccl::begin_render_result (b_engine=..., x=64, y=150, w=32, h=30, layername=0x7fffc0248788 "RenderLayer") - at /daten/blender-git/blender/intern/cycles/blender/blender_session.cpp:261 - #10 0x00000000026ebec0 in ccl::BlenderSession::do_write_update_render_tile (this=0x7fffc0217700, rtile=..., do_update_only=true) - at /daten/blender-git/blender/intern/cycles/blender/blender_session.cpp:278 - #11 0x00000000026ec2c9 in ccl::BlenderSession::update_render_tile (this=0x7fffc0217700, rtile=...) - at /daten/blender-git/blender/intern/cycles/blender/blender_session.cpp:326 - #12 0x00000000026f6b9c in boost::_mfi::mf1<void, ccl::BlenderSession, ccl::RenderTile&>::operator() (this=0x7fffc023ea38, p=0x7fffc0217700, a1=...) - at /usr/include/boost/bind/mem_fn_template.hpp:165 - #13 0x00000000026f6096 in boost::_bi::list2<boost::_bi::value<ccl::BlenderSession*>, boost::arg<1> >::operator()<boost::_mfi::mf1<void, ccl::BlenderSession, ccl::RenderTile&>, boost::_bi::list1<ccl::RenderTile&> > (this=0x7fffc023ea48, f=..., a=...) at /usr/include/boost/bind/bind.hpp:313 - #14 0x00000000026f57fe in boost::_bi::bind_t<void, boost::_mfi::mf1<void, ccl::BlenderSession, ccl::RenderTile&>, boost::_bi::list2<boost::_bi::value<ccl::BlenderSession*>, boost::arg<1> > >::operator()<ccl::RenderTile> (this=0x7fffc023ea38, a1=...) at /usr/include/boost/bind/bind_template.hpp:32 - #15 0x00000000026f5184 in boost::detail::function::void_function_obj_invoker1<boost::_bi::bind_t<void, boost::_mfi::mf1<void, ccl::BlenderSession, ccl::RenderTile&>, boost::_bi::list2<boost::_bi::value<ccl::BlenderSession*>, boost::arg<1> > >, void, ccl::RenderTile&>::invoke (function_obj_ptr=..., - a0=...) at /usr/include/boost/function/function_template.hpp:153 - #16 0x00000000027e8103 in boost::function1<void, ccl::RenderTile&>::operator() (this=0x7fffc023ea30, a0=...) - at /usr/include/boost/function/function_template.hpp:767 - #17 0x00000000027e3f1f in ccl::Session::update_tile_sample (this=0x7fffc023e7d0, rtile=...) - at /daten/blender-git/blender/intern/cycles/render/session.cpp:437 - #18 0x00000000027eb3b2 in boost::_mfi::mf1<void, ccl::Session, ccl::RenderTile&>::operator() (this=0x7fffa801ea60, p=0x7fffc023e7d0, a1=...) - at /usr/include/boost/bind/mem_fn_template.hpp:165 - #19 0x00000000027eadac in boost::_bi::list2<boost::_bi::value<ccl::Session*>, boost::arg<1> >::operator()<boost::_mfi::mf1<void, ccl::Session, ccl::RenderTile&>, boost::_bi::list1<ccl::RenderTile&> > (this=0x7fffa801ea70, f=..., a=...) at /usr/include/boost/bind/bind.hpp:313 - #20 0x00000000027ea784 in boost::_bi::bind_t<void, boost::_mfi::mf1<void, ccl::Session, ccl::RenderTile&>, boost::_bi::list2<boost::_bi::value<ccl::Session*>, boost::arg<1> > >::operator()<ccl::RenderTile> (this=0x7fffa801ea60, a1=...) at /usr/include/boost/bind/bind_template.hpp:32 - #21 0x00000000027ea2e2 in boost::detail::function::void_function_obj_invoker1<boost::_bi::bind_t<void, boost::_mfi::mf1<void, ccl::Session, ccl::RenderTile&>, boost::_bi::list2<boost::_bi::value<ccl::Session*>, boost::arg<1> > >, void, ccl::RenderTile&>::invoke (function_obj_ptr=..., a0=...) - at /usr/include/boost/function/function_template.hpp:153 - #22 0x00000000027e8103 in boost::function1<void, ccl::RenderTile&>::operator() (this=0x7fffa801ea58, a0=...) - at /usr/include/boost/function/function_template.hpp:767 - #23 0x000000000285678b in ccl::DeviceTask::update_progress (this=0x7fffa801e990, rtile=...) - at /daten/blender-git/blender/intern/cycles/device/device_task.cpp:103 - #24 0x0000000002831019 in ccl::CPUDevice::thread_path_trace (this=0x7fffc023f400, task=...) - at /daten/blender-git/blender/intern/cycles/device/device_cpu.cpp:404 - #25 0x000000000282f620 in ccl::CPUDevice::thread_run (this=0x7fffc023f400, task=0x7fffa801e990) - at /daten/blender-git/blender/intern/cycles/device/device_cpu.cpp:174 - #26 0x000000000283370c in boost::_mfi::mf1<void, ccl::CPUDevice, ccl::DeviceTask*>::operator() (this=0x7fffa8011200, p=0x7fffc023f400, - a1=0x7fffa801e990) at /usr/include/boost/bind/mem_fn_template.hpp:165 - #27 0x00000000028335bf in boost::_bi::list2<boost::_bi::value<ccl::CPUDevice*>, boost::_bi::value<ccl::CPUDevice::CPUDeviceTask*> >::operator()<boost::_mfi::mf1<void, ccl::CPUDevice, ccl::DeviceTask*>, boost::_bi::list0> (this=0x7fffa8011210, f=..., a=...) at /usr/include/boost/bind/bind.hpp:313 - #28 0x000000000283331b in boost::_bi::bind_t<void, boost::_mfi::mf1<void, ccl::CPUDevice, ccl::DeviceTask*>, boost::_bi::list2<boost::_bi::value<ccl::CPUDevice*>, boost::_bi::value<ccl::CPUDevice::CPUDeviceTask*> > >::operator() (this=0x7fffa8011200) at /usr/include/boost/bind/bind_template.hpp:20 - #29 0x0000000002833205 in boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf1<void, ccl::CPUDevice, ccl::DeviceTask*>, boost::_bi::list2<boost::_bi::value<ccl::CPUDevice*>, boost::_bi::value<ccl::CPUDevice::CPUDeviceTask*> > >, void>::invoke ( - function_obj_ptr=...) at /usr/include/boost/function/function_template.hpp:153 - #30 0x00000000026f112c in boost::function0<void>::operator() (this=0x7fffa801e998) at /usr/include/boost/function/function_template.hpp:767 - #31 0x0000000002d173b5 in ccl::TaskScheduler::thread_run (thread_id=3) at /daten/blender-git/blender/intern/cycles/util/util_task.cpp:249 - #32 0x0000000002d1a922 in boost::_bi::list1<boost::_bi::value<unsigned long> >::operator()<void (*)(int), boost::_bi::list0> (this=0x7fffc023f260, - f=@0x7fffc023f258: 0x2d17398 <ccl::TaskScheduler::thread_run(int)>, a=...) at /usr/include/boost/bind/bind.hpp:253 - #33 0x0000000002d1a5af in boost::_bi::bind_t<void, void (*)(int), boost::_bi::list1<boost::_bi::value<unsigned long> > >::operator() ( - this=0x7fffc023f258) at /usr/include/boost/bind/bind_template.hpp:20 - #34 0x0000000002d1a137 in boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, void (*)(int), boost::_bi::list1<boost::_bi::value<unsigned long> > >, void>::invoke (function_obj_ptr=...) at /usr/include/boost/function/function_template.hpp:153 - #35 0x00000000026f112c in boost::function0<void>::operator() (this=0x7fffc023f250) at /usr/include/boost/function/function_template.hpp:767 - #36 0x00000000027e6f7b in ccl::thread::run (arg=0x7fffc023f250) at /daten/blender-git/blender/intern/cycles/render/../util/util_thread.h:55 - #37 0x00007ffff63c70db in start_thread () from /lib64/libpthread.so.0 - #38 0x00007ffff290290d in clone () from /lib64/libc.so.6 I try with Address Sanitizer but got scrambled output because of error message. Hope this help to catch the bug. Cheers, mib.
Author
Member

Ok, so I found another bug, this time it was related to floating-point math. In the Mutate function, there is code like

if (x < 0.0f) x += 1.0f;
if (x >= 1.0f) x -= 1.0f;

, so I thought it would always be 0 <= x < 1. However, apparantly sometimes x is so slightly under 0 that the if is executed, but x + 1 gets rounded up to 1, which causes code like (int) (x * width) to sometimes round up to width, which then causes a out-of-bounds access.

To fix it, just add

if (s >= 1.f) s = 1.f-FLT_EPSILON;
if (s < 0.f) s = 0.f;

after line 301 in intern/cycles/device/device_cpu.cpp (This is just too small to make a metropolis_6.diff for it)

Regarding the error message: I have failed massively, I forgot the Negation in there, so the line got printed out when everything worked fine -.-
To remove it, just delete

if (m->samples[0] >= 0 && m->samples[1] >= 0 && m->samples[0] < 1 && m->samples[1] < 1)
	printf("Pixel sampling error, expect crashing!\n");

, also in device_cpu.cpp

Ok, so I found another bug, this time it was related to floating-point math. In the Mutate function, there is code like ``` if (x < 0.0f) x += 1.0f; if (x >= 1.0f) x -= 1.0f; ``` , so I thought it would always be 0 <= x < 1. However, apparantly sometimes x is so slightly under 0 that the if is executed, but x + 1 gets rounded up to 1, which causes code like (int) (x * width) to sometimes round up to width, which then causes a out-of-bounds access. To fix it, just add ``` if (s >= 1.f) s = 1.f-FLT_EPSILON; if (s < 0.f) s = 0.f; ``` after line 301 in intern/cycles/device/device_cpu.cpp (This is just too small to make a metropolis_6.diff for it) Regarding the error message: I have failed massively, I forgot the Negation in there, so the line got printed out when everything worked fine -.- To remove it, just delete ``` if (m->samples[0] >= 0 && m->samples[1] >= 0 && m->samples[0] < 1 && m->samples[1] < 1) printf("Pixel sampling error, expect crashing!\n"); ``` , also in device_cpu.cpp
Author
Member

New patch version, this time I included the bugfixes described above and fixed another quite serious bug in the sample contribution, now the rendered images are way smoother and diffuse areas converge faster.
Also, as a test for noise-based Importance Sampling, I added a perceptual noise pass as described in the paper I posted recently (In fact, currently I replaced the Mist pass as my new pass didn't work, but that's just a temporary solution), both to the Metropolis sampler and to the standard path tracer. Especially for the PT, the results are really great. As an example, I rendered the Lego Bulldozer from http://www.blendswap.com/blends/view/72124 and this are the results:

Lego_color.png
Lego_Variance.png

To test it, just activate the Mist render pass and multiply it by some small value in the compositor since the output is usually > 1.

The patch is here, but as there seem to be conflicts with the current Trunk, I recommend to apply to 3d8c106, since my local repo uses that one as origin/master.

metropolis_6.diff

New patch version, this time I included the bugfixes described above and fixed another quite serious bug in the sample contribution, now the rendered images are way smoother and diffuse areas converge faster. Also, as a test for noise-based Importance Sampling, I added a perceptual noise pass as described in the paper I posted recently (In fact, currently I replaced the Mist pass as my new pass didn't work, but that's just a temporary solution), both to the Metropolis sampler and to the standard path tracer. Especially for the PT, the results are really great. As an example, I rendered the Lego Bulldozer from http://www.blendswap.com/blends/view/72124 and this are the results: ![Lego_color.png](https://archive.blender.org/developer/F76502/Lego_color.png) ![Lego_Variance.png](https://archive.blender.org/developer/F76504/Lego_Variance.png) To test it, just activate the Mist render pass and multiply it by some small value in the compositor since the output is usually > 1. The patch is here, but as there seem to be conflicts with the current Trunk, I recommend to apply to 3d8c106, since my local repo uses that one as origin/master. [metropolis_6.diff](https://archive.blender.org/developer/F76506/metropolis_6.diff)

Here's a test of the noise map generator with some pokemon models I have laying around:

color_gengar.jpg

noise_map_gengar.jpg

Seems to be working pretty well. Highlighting shadows, edges, furry spots, etc.

Here's a test of the noise map generator with some pokemon models I have laying around: ![color_gengar.jpg](https://archive.blender.org/developer/F76522/color_gengar.jpg) ![noise_map_gengar.jpg](https://archive.blender.org/developer/F76524/noise_map_gengar.jpg) Seems to be working pretty well. Highlighting shadows, edges, furry spots, etc.
Member

Don't know what I did wrong, but with metropolis_6.diff and 3d8c106 all shadows and reflections are wrong for me. But previous versions of this patch gave plausible results (except for dark rectangles).
Выделение_010.png
Выделение_011.png

Don't know what I did wrong, but with metropolis_6.diff and 3d8c106 all shadows and reflections are wrong for me. But previous versions of this patch gave plausible results (except for dark rectangles). ![Выделение_010.png](https://archive.blender.org/developer/F76582/Выделение_010.png) ![Выделение_011.png](https://archive.blender.org/developer/F76584/Выделение_011.png)
Author
Member

I now rebased to the current trunk, everything seems to work fine. @Lockal, please try this one out as well, for me reflection/refraction works just fine (I really need a second test system, somehow bugs never show up on my system...)
Also, this version includes D301 as a first step in noise-adaptive sampling.
metropolis_7.diff

I now rebased to the current trunk, everything seems to work fine. @Lockal, please try this one out as well, for me reflection/refraction works just fine (I really need a second test system, somehow bugs never show up on my system...) Also, this version includes [D301](https://archive.blender.org/developer/D301) as a first step in noise-adaptive sampling. [metropolis_7.diff](https://archive.blender.org/developer/F76596/metropolis_7.diff)

reflection/refraction working fine here on both patches. Something I should've pointed out before: In intern/cycles/util/util_color.h line 240, Clang/OS X doesn't like

return exp10(log_i);

Jens Verwiebe pointed out to me in IRC it works with:

return pow(10, log_i);

So I had it built with that.

Also, on adaptive sampling: While I won't claim to know how they work behind the scenes, the adaptive samplers in Vray and Mental Ray seem to work fine running on each tile individually. Might it be possible to try this with Cycles? (maybe only updating the noise map every n number of samples?)

reflection/refraction working fine here on both patches. Something I should've pointed out before: In intern/cycles/util/util_color.h line 240, Clang/OS X doesn't like ``` return exp10(log_i); ``` Jens Verwiebe pointed out to me in IRC it works with: ``` return pow(10, log_i); ``` So I had it built with that. Also, on adaptive sampling: While I won't claim to know how they work behind the scenes, the adaptive samplers in Vray and Mental Ray seem to work fine running on each tile individually. Might it be possible to try this with Cycles? (maybe only updating the noise map every *n* number of samples?)
Author
Member

Thanks for the exp10 thing, of course you're right (I think exp(log_i * log(10)), as described in the GNU libc documentation, should be even faster since log(10) is constant).

Regarding adaptive sampling: Basically, adaptive sampling in the tile works as well. However, consider this: The left half of the image is nearly noise-free, while the right half is very noisy. Now, with adaptive sampling, you want the right side to receive more samples. However, when the right side is rendered, the left side is possibly not rendered yet, so there is no way to do that. So, if you implement it that way, the samples can only be adaptively distributed inside the tile, but not between tiles.

Indeed, your last sentence is quite what I also intend to do, this is the reason for D301: Rendering, for example, 10 samples with a tiled approach on the whole image and then creating a noise map that is used for inter- and intra-tile sample distribution which is then used for the next 10 samples.

Thanks for the exp10 thing, of course you're right (I think exp(log_i * log(10)), as described in the GNU libc documentation, should be even faster since log(10) is constant). Regarding adaptive sampling: Basically, adaptive sampling in the tile works as well. However, consider this: The left half of the image is nearly noise-free, while the right half is very noisy. Now, with adaptive sampling, you want the right side to receive more samples. However, when the right side is rendered, the left side is possibly not rendered yet, so there is no way to do that. So, if you implement it that way, the samples can only be adaptively distributed *inside* the tile, but not *between* tiles. Indeed, your last sentence is quite what I also intend to do, this is the reason for [D301](https://archive.blender.org/developer/D301): Rendering, for example, 10 samples with a tiled approach on the whole image and then creating a noise map that is used for inter- and intra-tile sample distribution which is then used for the next 10 samples.

Guys u are are doing soo good joob! if u need some test machines i have i5 laptop. and 2 Macbook pro laptops... And They are totaly free so i can set some long time rendering just give the link to download and say what to do :>

Guys u are are doing soo good joob! if u need some test machines i have i5 laptop. and 2 Macbook pro laptops... And They are totaly free so i can set some long time rendering just give the link to download and say what to do :>

How did you plan the adaptive sampling to work? Adaptively distributing a fixed number of samples across the image? Or setting a range of possible AA samples and letting each tile cut off where needed in that range? Because Vray/MR do the latter, and that avoids the problem of some tiles needing more samples than others. For example:

You set a min and max number of AA samples, and some target noise threshold. Once a tile reaches the min value, it checks noise level every n samples, and stops upon it either falling below the threshold, or hitting the max AA samples value. This way you can keep the coherency of individual tiles, but still let some tiles have far more samples than others.

How did you plan the adaptive sampling to work? Adaptively distributing a fixed number of samples across the image? Or setting a range of possible AA samples and letting each tile cut off where needed in that range? Because Vray/MR do the latter, and that avoids the problem of some tiles needing more samples than others. For example: You set a min and max number of AA samples, and some target noise threshold. Once a tile reaches the min value, it checks noise level every *n* samples, and stops upon it either falling below the threshold, or hitting the max AA samples value. This way you can keep the coherency of individual tiles, but still let some tiles have far more samples than others.
Member

Oops, with F12 render everything is ok (except for tiles, of course). But "rendered" viewport mode has very obvious problems. I did a full recompilation with metropolis_7 patch (gcc 4.8, linux x86-64), but the problem is still there.

Oops, with F12 render everything is ok (except for tiles, of course). But "rendered" viewport mode has very obvious problems. I did a full recompilation with metropolis_7 patch (gcc 4.8, linux x86-64), but the problem is still there.
Author
Member

@MaciejJutrzenka Currently, you have to build it from source (see http://wiki.blender.org/index.php/Dev:Doc/Building_Blender), but between downloading the source and building you have to apply the metropolis_7 patch from above.

@JasonClarke The latter one is actually a quite great idea, this would even add a stopping criterion that could be useful for renderfarms, animation rendering etc. The only question is whether the average noise of the tile or the maximum noise in the tile is considered. Alternatively, we could go the LuxRender way and add a user-provided percentage of pixels that has to pass the test.
My original idea was to use code like in the EnvMap importance sampling to map the uniform sample values to noise-accordingly distributed ones. This might be added as an option.

@MaciejJutrzenka Currently, you have to build it from source (see http://wiki.blender.org/index.php/Dev:Doc/Building_Blender), but between downloading the source and building you have to apply the metropolis_7 patch from above. @JasonClarke The latter one is actually a quite great idea, this would even add a stopping criterion that could be useful for renderfarms, animation rendering etc. The only question is whether the average noise of the tile or the maximum noise in the tile is considered. Alternatively, we could go the LuxRender way and add a user-provided percentage of pixels that has to pass the test. My original idea was to use code like in the EnvMap importance sampling to map the uniform sample values to noise-accordingly distributed ones. This might be added as an option.

Do both! Use the importance sampling within the tile to hit the threshold faster. As far as a avg noise vs max noise vs changed pixels vs pixels below threshold, I don't really know. Might be best to just give several options so people can test. After running it through some scenes, you could hide/disable modes that prove unreliable.

Do both! Use the importance sampling within the tile to hit the threshold faster. As far as a avg noise vs max noise vs changed pixels vs pixels below threshold, I don't really know. Might be best to just give several options so people can test. After running it through some scenes, you could hide/disable modes that prove unreliable.

@LukasStockner seems bit complicated however there is no some ready build with this for windows 64bit?

@LukasStockner seems bit complicated however there is no some ready build with this for windows 64bit?

Added subscriber: @ClaasKuhnen

Added subscriber: @ClaasKuhnen

lukasstockner97,

Do I read this right that there might be soon a "noise-based Importance Sampling" so Cycles spends more time on where actual noise is not not where there result is already clean?

That would be terrific!

lukasstockner97, Do I read this right that there might be soon a "noise-based Importance Sampling" so Cycles spends more time on where actual noise is not not where there result is already clean? That would be terrific!
Author
Member

@ClaasKuhnen Yes, that's currently the plan...
@MaciejJutrzenka Sadly, not yet.
@JasonClarke The only thing missing for this is an actual noise estimate since the current output is in fact a variance estimate (more precisely, a visually weighted RMS estimate), not a noise estimate. Basically, its value is proportional to the difficulty, not to the remaining noise. On the one hand, this is great, since we can sample directly from it, but on the other hand to determine the noise left in a tile we'd need a separate estimate, although for this easier methods are availible.

I'm currently working on adaptive sampling, I'll post a new patch once it works good enough.

@ClaasKuhnen Yes, that's currently the plan... @MaciejJutrzenka Sadly, not yet. @JasonClarke The only thing missing for this is an actual noise estimate since the current output is in fact a variance estimate (more precisely, a visually weighted RMS estimate), not a noise estimate. Basically, its value is proportional to the difficulty, not to the remaining noise. On the one hand, this is great, since we can sample directly from it, but on the other hand to determine the noise left in a tile we'd need a separate estimate, although for this easier methods are availible. I'm currently working on adaptive sampling, I'll post a new patch once it works good enough.

Added subscriber: @FilipPolbratt

Added subscriber: @FilipPolbratt

Added subscriber: @lopataasdf

Added subscriber: @lopataasdf

@ lukasstockner97
Regarding difficulty vs remaining noise, what about the idea shown here? (http://blenderartists.org/forum/showthread.php?236453-Measuring-Noise-in-Cycles-Renders&p=1985872&viewfull=1#post1985872)

@ lukasstockner97 Regarding difficulty vs remaining noise, what about the idea shown here? (http://blenderartists.org/forum/showthread.php?236453-Measuring-Noise-in-Cycles-Renders&p=1985872&viewfull=1#post1985872)

Added subscriber: @bned

Added subscriber: @bned
Member

Added subscriber: @gandalf3

Added subscriber: @gandalf3
Author
Member

New patch version, adaptive sampling works now in PT mode, the Noise-Aware Metropolis isn't too hard now as well.
I haven't added a stopping criterion yet, this will require a redesign of the Tile Manager since currently whole Cycles is based on the assumption that the samples per pixel is the same for every pixel (at least in a tile).
The current patch works around this by using a Sample number pass that is used in render/buffers.cpp (you can see the values of the pass in the shadow pass, which I currently use until I figure out how to output a new pass).
Also, at the moment adaptive sampling is only inside of individual tiles, every tile gets the same amunt of total work (this will change in the future, I plan to distribute samples to the tiles according to their mean importance). So, if you have one tile with high variance in the whole tile and one with basically zero variance, they will still be sampled equally.
Under the performance options, you can check adaptive sampling. There are two options: Adaptive Warmup, which sets the number of uniform samples taken per pixel to estimate importance before the adaptive sampling starts. Don't set this to low (~under 10) or it might miss difficult regions. The second one is the importance map interval, which is the number of samples taken until a new importance map is calculated. Too low values might give a significant performance hit.
Speaking about performance: Toe code is not optimized yet, in particular, the 4-pixel gaussian blur on the importance map is probably a big performance hit (maybe a simple box filter would suffice?). Also, when using progressive mode, a new map is computed every sample, so it's probably quite slow.
GPU is not included yet, but, in contrast to Metropolis, adaptive sampling should be possible there too.
@lsscpp This might work, using a more complex visual difference predictor would probably be even better, but they tend to be much slower than the approach you posted. An aternative is just a fixed samples/unit of importance value, I think I'll add an option to choose betwenn stopping criteria.

metropolis_8.diff

New patch version, adaptive sampling works now in PT mode, the Noise-Aware Metropolis isn't too hard now as well. I haven't added a stopping criterion yet, this will require a redesign of the Tile Manager since currently whole Cycles is based on the assumption that the samples per pixel is the same for every pixel (at least in a tile). The current patch works around this by using a Sample number pass that is used in render/buffers.cpp (you can see the values of the pass in the shadow pass, which I currently use until I figure out how to output a new pass). Also, at the moment adaptive sampling is only inside of individual tiles, every tile gets the same amunt of total work (this will change in the future, I plan to distribute samples to the tiles according to their mean importance). So, if you have one tile with high variance in the whole tile and one with basically zero variance, they will still be sampled equally. Under the performance options, you can check adaptive sampling. There are two options: Adaptive Warmup, which sets the number of uniform samples taken per pixel to estimate importance before the adaptive sampling starts. Don't set this to low (~under 10) or it might miss difficult regions. The second one is the importance map interval, which is the number of samples taken until a new importance map is calculated. Too low values might give a significant performance hit. Speaking about performance: Toe code is *not* optimized yet, in particular, the 4-pixel gaussian blur on the importance map is probably a big performance hit (maybe a simple box filter would suffice?). Also, when using progressive mode, a new map is computed every sample, so it's probably quite slow. GPU is not included yet, but, in contrast to Metropolis, adaptive sampling should be possible there too. @lsscpp This might work, using a more complex visual difference predictor would probably be even better, but they tend to be much slower than the approach you posted. An aternative is just a fixed samples/unit of importance value, I think I'll add an option to choose betwenn stopping criteria. [metropolis_8.diff](https://archive.blender.org/developer/F78116/metropolis_8.diff)

Wow! Can't wait to test it (whenever a build apparso on graphicall). About more advanced difference predicatore, i remember i posted somewhere a link to a paper named something like "Entropy variance". That could be something worth to look at

Wow! Can't wait to test it (whenever a build apparso on graphicall). About more advanced difference predicatore, i remember i posted somewhere a link to a paper named something like "Entropy variance". That could be something worth to look at

I actually did a search... Google this: "entropy based adaptive sampling paper"

I actually did a search... Google this: "entropy based adaptive sampling paper"

Quick test on the Mike Pan BMW scene:

No adaptive sampling (2:39): bmw_normal_2_39.png

Adaptive sampling on (warmpup=25, map update=25. 3:03): bmw_adaptive_3_02.png

Quick test on the Mike Pan BMW scene: No adaptive sampling (2:39): ![bmw_normal_2_39.png](https://archive.blender.org/developer/F78167/bmw_normal_2_39.png) Adaptive sampling on (warmpup=25, map update=25. 3:03): ![bmw_adaptive_3_02.png](https://archive.blender.org/developer/F78169/bmw_adaptive_3_02.png)
Author
Member

@JasonClarke OK, this isn't much improvement. What rendering settings did you use (sample number, tile size)?
@lsscpp Once the adaptive sampling works in Metropolis mode, I'll post Windows and Linux (both x64) builds on graphicall, for other platforms (x86 and Mac) somebody using them would have to post a build. Maybe once these are online we should put a link into blenderartists to get some beta testers. Concering the entropy paper: It certainly looks interesing, do you have any information regarding realtime performance?

@JasonClarke OK, this isn't much improvement. What rendering settings did you use (sample number, tile size)? @lsscpp Once the adaptive sampling works in Metropolis mode, I'll post Windows and Linux (both x64) builds on graphicall, for other platforms (x86 and Mac) somebody using them would have to post a build. Maybe once these are online we should put a link into blenderartists to get some beta testers. Concering the entropy paper: It certainly looks interesing, do you have any information regarding realtime performance?

The rest of the settings on that test were the defaults for the BMW scene, so 128x64 tiles, 200 progressive samples. I'm not sure it's realistic to expect much better when we still have the same number of AA samples on all tiles. Large tile sizes have a performance hit of their own in CPU mode, so just making them bigger isn't realistic either. I think we really need to wait until there's an option to stop some tiles before max samples (that way you can just pad out the max AA value and only use it on the tricky tiles).

The rest of the settings on that test were the defaults for the BMW scene, so 128x64 tiles, 200 progressive samples. I'm not sure it's realistic to expect much better when we still have the same number of AA samples on all tiles. Large tile sizes have a performance hit of their own in CPU mode, so just making them bigger isn't realistic either. I think we really need to wait until there's an option to stop some tiles before max samples (that way you can just pad out the max AA value and only use it on the tricky tiles).

@LukasStockner no, unfortunately i have no clue about performance
@JasonClarke can you please make another BMW test with no adaptive, letting cycles run for 3:03 as well, so we can see how better the algorithm distributed the samples in the same amount of time?

@LukasStockner no, unfortunately i have no clue about performance @JasonClarke can you please make another BMW test with no adaptive, letting cycles run for 3:03 as well, so we can see how better the algorithm distributed the samples in the same amount of time?
Author
Member

New patch version, this time with the focus again on Metropolis. The last patch broke it due to the samples pass, now I fixed the bug. Also, there was another pretty serious one where samples were written to the wrong pixels, this is fixed now too. Preview rendering should work now as well.

Noise-adaptive sampling in Metropolis doesn't work yet, but if you check adaptive sampling, it uses another trick from the Importance-Sampling paper I posted a while ago: The importance function is divided by the current brightness of the pixel, to that the samples are distributed more evenly (by default, the Metropolis sampler samples according to brightness). By doing so, it focuses more on good lightpaths instead of just bright regions, you can see this in the Shadow (amount of samples) and Emission (Importance) channels (by the way, sorry for overriding all the default channels...). I haven't run extensive tests with this one yet, but is seems as if it gives a nice enhancement.

Regarding tests: I rendered the Sintel hair scene from http://www.blenderguru.com/videos/how-to-render-hair-with-cycles/ in two instances of blender, one with Metropolis and one without. Both had 2 threads and two tiles, one at the top half and one on the bottom half. After 2 hours, this were the results:

Metropolis:
Sintel_MLT_2h.png

Pathtracing:
Sintel_PT_2h.png

As you can see, the Metropolis version has remarkably less noise, especially around the neck and the lower part of the hair. Also, this shows that Metropolis converges to the correct solution.
Another test is the pool scene, also 2 threads and 12 hours (probably less would have been enough):
Pool_MLT.png

The next patch might take 1-3 weeks since I have to work on another project for school (it's also rendering/CGI, so I won't get out of practise), but then the Tiling issues should be gone.

metropolis_9.diff

New patch version, this time with the focus again on Metropolis. The last patch broke it due to the samples pass, now I fixed the bug. Also, there was another pretty serious one where samples were written to the wrong pixels, this is fixed now too. Preview rendering should work now as well. Noise-adaptive sampling in Metropolis doesn't work yet, but if you check adaptive sampling, it uses another trick from the Importance-Sampling paper I posted a while ago: The importance function is divided by the current brightness of the pixel, to that the samples are distributed more evenly (by default, the Metropolis sampler samples according to brightness). By doing so, it focuses more on good lightpaths instead of just bright regions, you can see this in the Shadow (amount of samples) and Emission (Importance) channels (by the way, sorry for overriding all the default channels...). I haven't run extensive tests with this one yet, but is seems as if it gives a nice enhancement. Regarding tests: I rendered the Sintel hair scene from http://www.blenderguru.com/videos/how-to-render-hair-with-cycles/ in two instances of blender, one with Metropolis and one without. Both had 2 threads and two tiles, one at the top half and one on the bottom half. After 2 hours, this were the results: Metropolis: ![Sintel_MLT_2h.png](https://archive.blender.org/developer/F78593/Sintel_MLT_2h.png) Pathtracing: ![Sintel_PT_2h.png](https://archive.blender.org/developer/F78595/Sintel_PT_2h.png) As you can see, the Metropolis version has remarkably less noise, especially around the neck and the lower part of the hair. Also, this shows that Metropolis converges to the correct solution. Another test is the pool scene, also 2 threads and 12 hours (probably less would have been enough): ![Pool_MLT.png](https://archive.blender.org/developer/F78597/Pool_MLT.png) The next patch might take 1-3 weeks since I have to work on another project for school (it's also rendering/CGI, so I won't get out of practise), but then the Tiling issues should be gone. [metropolis_9.diff](https://archive.blender.org/developer/F78599/metropolis_9.diff)
Author
Member

It turned out I forgot to save aome code before creating the diff, so here is the correct version:

metropolis_10.diff

It turned out I forgot to save aome code before creating the diff, so here is the correct version: [metropolis_10.diff](https://archive.blender.org/developer/F78621/metropolis_10.diff)
Author
Member

Originally I wanted to release a new patch once all the tiling stuff is done, but I randomly found that a small change in the sample number pass causes the noise on diffuse areas/background to be completely gone. For example, a plane under a sky background is now nearly noise-free in a single pass.
Also, Metropolis sampling is now tile-free, every thread works on the whole image. This means that they can now also share the mean importance calculation which caused the brightness differences between tiles.
Somehow, screen-wide sampling seems to break automatic EXR writing, for example, when rendering the Sintel hair scene, after finishing/cancelling the render, Blender freezes with an OpenEXR error on the console, I'll look into this further.
Adaptive sampling is still only per tile, also the Division-By-Mean-Brightness was disabled for now. Both will be fixed in a later patch version, of course.
Just to give a impression of what the Patch is capable of now, consider this scene:

Villa_HD.png

That's the Villa scene from PBRT, imported into Blender with a quick and hacky pbrt-to-obj converter and rendered over the night. The materials are re-done with Cycles. Nearly every surface in this scene has a glossy component, even the wood and the walls/ceilings. The light comes from small emitters completely behing glass, two polit light sources inside of the spherical things on the ceiling and, although only a very small amount, from the TV. For pure Pathtracing, this scene is basically worst-case, while with Metropolis even the area to the left, illuminated through three glass panes, is nearly noise-free. The noise remaining in the back section of the room should get better once the Division-By-Mean-Brightness (this name is horrible...) works again. The dark artifacts around the light sources come from tonemapping.

Pool_Metro_8min.png

This is the classic pool scene, rendered in just 8min.

metropolis_11.diff

Originally I wanted to release a new patch once all the tiling stuff is done, but I randomly found that a small change in the sample number pass causes the noise on diffuse areas/background to be completely **gone**. For example, a plane under a sky background is now nearly noise-free in a *single pass*. Also, Metropolis sampling is now tile-free, every thread works on the whole image. This means that they can now also share the mean importance calculation which caused the brightness differences between tiles. Somehow, screen-wide sampling seems to break automatic EXR writing, for example, when rendering the Sintel hair scene, after finishing/cancelling the render, Blender freezes with an OpenEXR error on the console, I'll look into this further. Adaptive sampling is still only per tile, also the Division-By-Mean-Brightness was disabled for now. Both will be fixed in a later patch version, of course. Just to give a impression of what the Patch is capable of now, consider this scene: ![Villa_HD.png](https://archive.blender.org/developer/F79585/Villa_HD.png) That's the Villa scene from PBRT, imported into Blender with a quick and hacky pbrt-to-obj converter and rendered over the night. The materials are re-done with Cycles. Nearly every surface in this scene has a glossy component, even the wood and the walls/ceilings. The light comes from small emitters completely behing glass, two polit light sources inside of the spherical things on the ceiling and, although only a very small amount, from the TV. For pure Pathtracing, this scene is basically worst-case, while with Metropolis even the area to the left, illuminated through three glass panes, is nearly noise-free. The noise remaining in the back section of the room should get better once the Division-By-Mean-Brightness (this name is horrible...) works again. The dark artifacts around the light sources come from tonemapping. ![Pool_Metro_8min.png](https://archive.blender.org/developer/F79596/Pool_Metro_8min.png) This is the classic pool scene, rendered in just 8min. [metropolis_11.diff](https://archive.blender.org/developer/F79601/metropolis_11.diff)

Hi, I would like to test the new development but got build error.

Linking CXX static library ../../../lib/libbf_intern_cycles.a
[ 84%] Built target bf_intern_cycles
[ 84%] Built target cycles_bvh
[ 84%] Building CXX object intern/cycles/device/CMakeFiles/cycles_device.dir/device_cpu.cpp.o
In file included from /daten/blender-git/blender/intern/cycles/device/device_cpu.cpp:45:0:
/daten/blender-git/blender/intern/cycles/device/../util/util_metropolis.h:21:33: fatal error: kernel/kernel_types.h: Datei oder Verzeichnis nicht gefunden
#include <kernel/kernel_types.h>

                               ^

compilation terminated.
make- [x]: *** [intern/cycles/device/CMakeFiles/cycles_device.dir/device_cpu.cpp.o] Fehler 1
make- [x]: *** [intern/cycles/device/CMakeFiles/cycles_device.dir/all] Fehler 2
make: *** [all] Fehler 2

Opensuse 13.1/64
Intel i5 3770K
GTX 760
GTX 560Ti 448 Cores
Driver 331.49

Blender 3c3c2cd

Thank you, mib.

Hi, I would like to test the new development but got build error. Linking CXX static library ../../../lib/libbf_intern_cycles.a [ 84%] Built target bf_intern_cycles [ 84%] Built target cycles_bvh [ 84%] Building CXX object intern/cycles/device/CMakeFiles/cycles_device.dir/device_cpu.cpp.o In file included from /daten/blender-git/blender/intern/cycles/device/device_cpu.cpp:45:0: /daten/blender-git/blender/intern/cycles/device/../util/util_metropolis.h:21:33: fatal error: kernel/kernel_types.h: Datei oder Verzeichnis nicht gefunden #include <kernel/kernel_types.h> ``` ^ ``` compilation terminated. make- [x]: *** [intern/cycles/device/CMakeFiles/cycles_device.dir/device_cpu.cpp.o] Fehler 1 make- [x]: *** [intern/cycles/device/CMakeFiles/cycles_device.dir/all] Fehler 2 make: *** [all] Fehler 2 Opensuse 13.1/64 Intel i5 3770K GTX 760 GTX 560Ti 448 Cores Driver 331.49 Blender 3c3c2cd Thank you, mib.

@LukasStockner

8 minutes on CPU / GPU? That is amazing! How long it will take to made it to the let's say daily builds ?

@LukasStockner 8 minutes on CPU / GPU? That is amazing! How long it will take to made it to the let's say daily builds ?
Author
Member

@mib2berlin
Sorry, my fault, it seems like you have a remarkably strict compiler^^
metropolis_12.diff, that should fix it.
@MaciejJutrzenka
8 Minutes on a FX8350, I'm glad you guys like it. Regarding the builds, for this it has to be accepted into trunk, but IMO it's not ready for code review yet, the features aren't complete yet and the codestyle is rather messy. But once I got the Windows buildsystem running, I'll post Windows/Linux builds on GraphicsAll. Sadly, I can't build for Mac.

@mib2berlin Sorry, my fault, it seems like you have a remarkably strict compiler^^ [metropolis_12.diff](https://archive.blender.org/developer/F79651/metropolis_12.diff), that should fix it. @MaciejJutrzenka 8 Minutes on a FX8350, I'm glad you guys like it. Regarding the builds, for this it has to be accepted into trunk, but IMO it's not ready for code review yet, the features aren't complete yet and the codestyle is rather messy. But once I got the Windows buildsystem running, I'll post Windows/Linux builds on GraphicsAll. Sadly, I can't build for Mac.

Nope, it is really strict. ^^
gcc (SUSE Linux) 4.8.1

Linking CXX static library ../../../lib/libbf_intern_cycles.a
[ 84%] Built target bf_intern_cycles
[ 84%] Built target cycles_bvh
[ 84%] Building CXX object intern/cycles/device/CMakeFiles/cycles_device.dir/device_cpu.cpp.o
In file included from /daten/blender-git/blender/intern/cycles/device/device_cpu.cpp:45:0:
/daten/blender-git/blender/intern/cycles/device/../util/util_metropolis.h:21:33: fatal error: kernel/kernel_types.h: Datei oder Verzeichnis nicht gefunden
#include "kernel/kernel_types.h"

                               ^

compilation terminated.

Thanks for fast reply, mib.

Nope, it is really strict. ^^ gcc (SUSE Linux) 4.8.1 Linking CXX static library ../../../lib/libbf_intern_cycles.a [ 84%] Built target bf_intern_cycles [ 84%] Built target cycles_bvh [ 84%] Building CXX object intern/cycles/device/CMakeFiles/cycles_device.dir/device_cpu.cpp.o In file included from /daten/blender-git/blender/intern/cycles/device/device_cpu.cpp:45:0: /daten/blender-git/blender/intern/cycles/device/../util/util_metropolis.h:21:33: fatal error: kernel/kernel_types.h: Datei oder Verzeichnis nicht gefunden #include "kernel/kernel_types.h" ``` ^ ``` compilation terminated. Thanks for fast reply, mib.

OS X/Clang is complaining about something else:

Compiling ==> 'device_cpu.cpp'
In file included from intern/cycles/device/device_cpu.cpp:45:
intern/cycles/util/util_metropolis.h:80:60: error: use of undeclared identifier 'ulong'; did you mean

    'long'?
              rng = lcg_init(hash_int_2d(kg->__data.integrator.seed, ((ulong) this & 0xffffffff) ...
                                                                       ^~~~~
                                                                       long

intern/cycles/util/util_metropolis.h:80:90: error: use of undeclared identifier 'ulong'; did you mean

    'long'?
...= lcg_init(hash_int_2d(kg->__data.integrator.seed, ((ulong) this & 0xffffffff) ^ ((ulong) this & (0...
                                                                                      ^~~~~
                                                                                      long

2 errors generated.
scons: *** [/Volumes/Home/Jason/Developer/Blender/build/darwin/intern/cycles/device/device_cpu.o] Error 1
scons: building terminated because of errors.

OS X/Clang is complaining about something else: Compiling ==> 'device_cpu.cpp' In file included from intern/cycles/device/device_cpu.cpp:45: intern/cycles/util/util_metropolis.h:80:60: error: use of undeclared identifier 'ulong'; did you mean ``` 'long'? rng = lcg_init(hash_int_2d(kg->__data.integrator.seed, ((ulong) this & 0xffffffff) ... ^~~~~ long ``` intern/cycles/util/util_metropolis.h:80:90: error: use of undeclared identifier 'ulong'; did you mean ``` 'long'? ...= lcg_init(hash_int_2d(kg->__data.integrator.seed, ((ulong) this & 0xffffffff) ^ ((ulong) this & (0... ^~~~~ long ``` 2 errors generated. scons: *** [/Volumes/Home/Jason/Developer/Blender/build/darwin/intern/cycles/device/device_cpu.o] Error 1 scons: building terminated because of errors.
Author
Member

OK, I really need to install a second compiler :D

@mib2berlin
The include thing is really strange, try if this one works better. If not, I'm out of ideas.
@JasonClarke
This part was probably overkill, now it should only give a warning about precision loss, which is no problem.

metropolis_13.diff

OK, I really need to install a second compiler :D @mib2berlin The include thing is really strange, try if this one works better. If not, I'm out of ideas. @JasonClarke This part was probably overkill, now it should only give a warning about precision loss, which is no problem. [metropolis_13.diff](https://archive.blender.org/developer/F79652/metropolis_13.diff)

Lucas, may with new error you know more:

Linking CXX static library ../../../lib/libbf_intern_cycles.a
[ 84%] Built target bf_intern_cycles
[ 84%] Built target cycles_bvh
[ 84%] Building CXX object intern/cycles/device/CMakeFiles/cycles_device.dir/device_cpu.cpp.o
In file included from /daten/blender-git/blender/intern/cycles/device/device_cpu.cpp:44:0:
/daten/blender-git/blender/intern/cycles/device/../util/util_importance.h: In function ‘float* ccl::variance_to_importance(float*, ccl::KernelFilm*, int, int, int, int, int, int, int)’:
/daten/blender-git/blender/intern/cycles/device/../util/util_importance.h:28:8: warning: no previous declaration for ‘float* ccl::variance_to_importance(float*, ccl::KernelFilm*, int, int, int, int, int, int, int)’ [-Wmissing-declarations]
float* variance_to_importance(float buffer, KernelFilm film, int stride, int pass_stride, int offset, int x_ofs, int y_ofs, int width, int height) {

      ^

In file included from /daten/blender-git/blender/intern/cycles/device/device_cpu.cpp:45:0:
/daten/blender-git/blender/intern/cycles/device/../util/util_metropolis.h: In constructor ‘ccl::Metropolis::Metropolis(ccl::KernelGlobals*, double*, double*)’:
/daten/blender-git/blender/intern/cycles/device/../util/util_metropolis.h:81:65: error: cast from ‘ccl::Metropolis*’ to ‘ccl::uint {aka unsigned int}’ loses precision [-fpermissive]

 rng = lcg_init(hash_int_2d(kg->__data.integrator.seed, (uint) this));
                                                               ^

make- [x]: *** [intern/cycles/device/CMakeFiles/cycles_device.dir/device_cpu.cpp.o] Fehler 1
make- [x]: *** [intern/cycles/device/CMakeFiles/cycles_device.dir/all] Fehler 2
make: *** [all] Fehler 2

Sorry for so much trouble, mib.

Lucas, may with new error you know more: Linking CXX static library ../../../lib/libbf_intern_cycles.a [ 84%] Built target bf_intern_cycles [ 84%] Built target cycles_bvh [ 84%] Building CXX object intern/cycles/device/CMakeFiles/cycles_device.dir/device_cpu.cpp.o In file included from /daten/blender-git/blender/intern/cycles/device/device_cpu.cpp:44:0: /daten/blender-git/blender/intern/cycles/device/../util/util_importance.h: In function ‘float* ccl::variance_to_importance(float*, ccl::KernelFilm*, int, int, int, int, int, int, int)’: /daten/blender-git/blender/intern/cycles/device/../util/util_importance.h:28:8: warning: no previous declaration for ‘float* ccl::variance_to_importance(float*, ccl::KernelFilm*, int, int, int, int, int, int, int)’ [-Wmissing-declarations] float* variance_to_importance(float *buffer, KernelFilm* film, int stride, int pass_stride, int offset, int x_ofs, int y_ofs, int width, int height) { ``` ^ ``` In file included from /daten/blender-git/blender/intern/cycles/device/device_cpu.cpp:45:0: /daten/blender-git/blender/intern/cycles/device/../util/util_metropolis.h: In constructor ‘ccl::Metropolis::Metropolis(ccl::KernelGlobals*, double*, double*)’: /daten/blender-git/blender/intern/cycles/device/../util/util_metropolis.h:81:65: error: cast from ‘ccl::Metropolis*’ to ‘ccl::uint {aka unsigned int}’ loses precision [-fpermissive] ``` rng = lcg_init(hash_int_2d(kg->__data.integrator.seed, (uint) this)); ^ ``` make- [x]: *** [intern/cycles/device/CMakeFiles/cycles_device.dir/device_cpu.cpp.o] Fehler 1 make- [x]: *** [intern/cycles/device/CMakeFiles/cycles_device.dir/all] Fehler 2 make: *** [all] Fehler 2 Sorry for so much trouble, mib.

Clang barfs there too:

In file included from intern/cycles/device/device_cpu.cpp:45:
intern/cycles/util/util_metropolis.h:81:58: error: cast from pointer to smaller type 'uint' (aka 'unsigned int') loses
      information
                rng = lcg_init(hash_int_2d(kg->__data.integrator.seed, (uint) this));
                                                                       ^~~~~~~~~~~

1 error generated.

scons: *** [/Volumes/Home/Jason/Developer/Blender/build/darwin/intern/cycles/device/device_cpu.o] Error 1
scons: building terminated because of errors.
Clang barfs there too: ``` In file included from intern/cycles/device/device_cpu.cpp:45: intern/cycles/util/util_metropolis.h:81:58: error: cast from pointer to smaller type 'uint' (aka 'unsigned int') loses information rng = lcg_init(hash_int_2d(kg->__data.integrator.seed, (uint) this)); ^~~~~~~~~~~ ``` 1 error generated. ``` scons: *** [/Volumes/Home/Jason/Developer/Blender/build/darwin/intern/cycles/device/device_cpu.o] Error 1 scons: building terminated because of errors. ```

In TortoiseGit I'm getting "Path Format Detection: Fail" and it refuses to patch. Any ideas?

In TortoiseGit I'm getting "Path Format Detection: Fail" and it refuses to patch. Any ideas?
Author
Member

@mib2berlin, @JasonClarke Sorry for all these errors, I hope this patch finally works.
@MatthewHeimlich Not really since I never used TortoiseGit, but just try this one.

metropolis_14.diff

@mib2berlin, @JasonClarke Sorry for all these errors, I hope this patch finally works. @MatthewHeimlich Not really since I never used TortoiseGit, but just try this one. [metropolis_14.diff](https://archive.blender.org/developer/F79662/metropolis_14.diff)

Anyone using TortioiseGit will need to use the git Apply command. The GUI apply only works for diff's made with TG.

Anyone using TortioiseGit will need to use the git Apply command. The GUI apply only works for diff's made with TG.

Lucas, thanks, working fine now.

Cheers, mib.

Lucas, thanks, working fine now. Cheers, mib.

Ok, patch 14 did the trick! Here's two test renders. The thing that hit me immediately was how slow MLT mode is now. It took 90 seconds to do 10 samples. Progressive can do a 120-sample render in the same amount of time. (here's the outputs of those two)

Progressive:
prog_120_samples.png

MLT:
mlt_10_samples.png

(also, something seems funny with the volume sphere in MLT)

Here's a 10 sample render with the BMW, took about 8mins.bmw_10_samples.jpg

Progressive on that same scene is 2:40 for 200 samples.

Ok, patch 14 did the trick! Here's two test renders. The thing that hit me immediately was how slow MLT mode is now. It took 90 seconds to do 10 samples. Progressive can do a 120-sample render in the same amount of time. (here's the outputs of those two) Progressive: ![prog_120_samples.png](https://archive.blender.org/developer/F79664/prog_120_samples.png) MLT: ![mlt_10_samples.png](https://archive.blender.org/developer/F79665/mlt_10_samples.png) (also, something seems funny with the volume sphere in MLT) Here's a 10 sample render with the BMW, took about 8mins.![bmw_10_samples.jpg](https://archive.blender.org/developer/F79668/bmw_10_samples.jpg) Progressive on that same scene is 2:40 for 200 samples.

@LukasStockner: I had to change line 239 in util_color.h to the following, msvc2008 complained otherwise:

 return exp(log_i * log(10.0f));
@LukasStockner: I had to change line 239 in util_color.h to the following, msvc2008 complained otherwise: ``` return exp(log_i * log(10.0f));

But yeah i can observe the same, 10 Samples to render the default cube takes over a minute here, with regular Path Trace just about ~1s.

But yeah i can observe the same, 10 Samples to render the default cube takes over a minute here, with regular Path Trace just about ~1s.
Author
Member

@JasonClarke Yes, the speed is indeed extremely low. At the moment, I can think of three reasons for this:
First, sampling overhead. Especially a high max bounce value can slow it down significantly, lazy sample generation could help there. However, with increasing scene complexity, the impact of this should reduce.
Second, caching. In the classical PT, most of the geometry should be still in cache from the previous Ray, while in Metropolis, at least for large mutations, this is usually not the case. This issue probably gets bigger with scene complaxity.
Third, BVH traversal. I haven't looked at the Cycles BVH yet, but considering it is originally targeted at GPUs, there is quite a chance that it is designed for high ray coherence, which, as said above, is not given in Metropolis.
Probably, running a full Valgrind profiling session might give some hints wich one of these is responsible.

@ThomasDinges: Thanks, I'll include it in the next patch.

@JasonClarke Yes, the speed is indeed extremely low. At the moment, I can think of three reasons for this: First, sampling overhead. Especially a high max bounce value can slow it down significantly, lazy sample generation could help there. However, with increasing scene complexity, the impact of this should reduce. Second, caching. In the classical PT, most of the geometry should be still in cache from the previous Ray, while in Metropolis, at least for large mutations, this is usually not the case. This issue probably gets bigger with scene complaxity. Third, BVH traversal. I haven't looked at the Cycles BVH yet, but considering it is originally targeted at GPUs, there is quite a chance that it is designed for high ray coherence, which, as said above, is not given in Metropolis. Probably, running a full Valgrind profiling session might give some hints wich one of these is responsible. @ThomasDinges: Thanks, I'll include it in the next patch.

Hm, maybe a windows problem.
Render testfile here, it is slow but impossible with path in any time. :)

metropolis.png

Need 16 minutes on i5.

Btw., default cube need 20 seconds here.

Cheers, mib.

Hm, maybe a windows problem. Render testfile here, it is slow but impossible with path in any time. :) ![metropolis.png](https://archive.blender.org/developer/F79675/metropolis.png) Need 16 minutes on i5. Btw., default cube need 20 seconds here. Cheers, mib.

Added subscriber: @DavidSisco

Added subscriber: @DavidSisco

Lucas, I start a thread on BA about your work.
May you jump in if you finished the win/lin builds for user.

http://www.blenderartists.org/forum/showthread.php?329089-Cycles-MLT-patch

Cheers, mib.

Lucas, I start a thread on BA about your work. May you jump in if you finished the win/lin builds for user. http://www.blenderartists.org/forum/showthread.php?329089-Cycles-MLT-patch Cheers, mib.

Windows x64 Build (MSVC 2008): http://blender.dingto.org/blender_win64_mlt.zip
Based on Patch Nr. 14.

Windows x64 Build (MSVC 2008): http://blender.dingto.org/blender_win64_mlt.zip Based on Patch Nr. 14.
Author
Member

@ThomasDinges Awesome, my building system on Windows somehow refuses to work, so thanks!
@mib2berlin Thanks, certainly a good idea.
Unfortunately, my BA account was blocked by the Spamfilter, so I currently have to await manual activation...
Also, I only have access to a slow laptop and an even slower WiFi until Friday, so, while I will continue to work on the patch, I probably won't be making big progress.
Over SSH, through, I will try to run a profiler session on my PC at home, maybe this will give some more information on where the bottleneck is.
Once my BA account works, I'll post a somewhat more detailed description there. The next steps IMO are working screen-wide sample distibution (Probably the best approach is to let every tile run ~25 samples, then calculate a sample distribution. If any tile has received more samples than its allocated sample budget, it is stopped) and speed improvement (The main question here is whether BVH and cache or the sampler itself are the bottleneck).

@ThomasDinges Awesome, my building system on Windows somehow refuses to work, so thanks! @mib2berlin Thanks, certainly a good idea. Unfortunately, my BA account was blocked by the Spamfilter, so I currently have to await manual activation... Also, I only have access to a slow laptop and an even slower WiFi until Friday, so, while I will continue to work on the patch, I probably won't be making big progress. Over SSH, through, I will try to run a profiler session on my PC at home, maybe this will give some more information on where the bottleneck is. Once my BA account works, I'll post a somewhat more detailed description there. The next steps IMO are working screen-wide sample distibution (Probably the best approach is to let every tile run ~25 samples, then calculate a sample distribution. If any tile has received more samples than its allocated sample budget, it is stopped) and speed improvement (The main question here is whether BVH and cache or the sampler itself are the bottleneck).

in my tests mlt very slow and give many noise in dark areas

in my tests mlt very slow and give many noise in dark areas

if u don't have opurtiny to code... write documentation and help about settings.. and what options are doing etc manual..

if u don't have opurtiny to code... write documentation and help about settings.. and what options are doing etc manual..

Added subscriber: @holyenigma

Added subscriber: @holyenigma

m9105826, can you explain how to use the Git add command for windows.
give an example path etc.. Git is such a PITA
thanks

m9105826, can you explain how to use the Git add command for windows. give an example path etc.. Git is such a PITA thanks

Added subscriber: @SebastianRothlisberger

Added subscriber: @SebastianRothlisberger
Author
Member

New patch version, this time optimization! It turned out to be the sampling, the fix was quite easy...
Basically, if you have 8 max. bounces and 8 max. transparent bounces, it generated 16*12 = 192 sample values, while often only 1-2 bounces were needed. Now, the samples are generated on demand.
For testing I used 2 scenes: One, the default cube at 3 samples, and two, a level-2-subdivided glass Suzanne lying on a plane at 40 samples. Both used one thread.

Results Metro 14:
Glass PT: 3.4sec Metro: 12sec
Cube PT: 0.7sec Metro: 9.6sec

Metro 15:
Glass PT: 3.5sec Metro: 4sec
Cube PT: 0.7sec Metro: 1.7sec

By the way and highly off-topic: I just found Embree, a optimized BVH library for CPUs, written by Intel. Considering we use NVIDIA code currently on GPU and CPU, maybe Intel code would be optimized better on CPUs. The API looks quite nice, I'm currently trying it out in a private VCM implementation of mine.

metropolis_15.diff

New patch version, this time optimization! It turned out to be the sampling, the fix was quite easy... Basically, if you have 8 max. bounces and 8 max. transparent bounces, it generated 16*12 = 192 sample values, while often only 1-2 bounces were needed. Now, the samples are generated on demand. For testing I used 2 scenes: One, the default cube at 3 samples, and two, a level-2-subdivided glass Suzanne lying on a plane at 40 samples. Both used one thread. Results Metro 14: Glass PT: 3.4sec Metro: 12sec Cube PT: 0.7sec Metro: 9.6sec Metro 15: Glass PT: 3.5sec Metro: 4sec Cube PT: 0.7sec Metro: 1.7sec By the way and highly off-topic: I just found Embree, a optimized BVH library for CPUs, written by Intel. Considering we use NVIDIA code currently on GPU and CPU, maybe Intel code would be optimized better on CPUs. The API looks quite nice, I'm currently trying it out in a private VCM implementation of mine. [metropolis_15.diff](https://archive.blender.org/developer/F79930/metropolis_15.diff)

Really great developement, thank you Lucas.

I'll try to compile it on osx. (Is there already a "public" one? I can post mine if not)

Really great developement, thank you Lucas. I'll try to compile it on osx. (Is there already a "public" one? I can post mine if not)

Nice speedups!

Doesn't link on Windows unfortunately: (also the log(10) issue in util_color is still present)

 bf_intern_cycles.lib(device_cpu.obj) : error LNK2019: Verweis auf nicht aufgelös
 tes externes Symbol ""public: __cdecl ccl::Metropolis::Metropolis(struct ccl::Ke
 rnelGlobals *,double *,double *,class ccl::RenderTile *)" (??0Metropolis@ccl@@QE
 AA@PEAUKernelGlobals@1@PEAN1PEAVRenderTile@1@@Z)" in Funktion ""public: void __c
 decl ccl::CPUDevice::thread_path_trace(class ccl::DeviceTask &)" (?thread_path_t
 race@CPUDevice@ccl@@QEAAXAEAVDeviceTask@2@@Z)".
Nice speedups! Doesn't link on Windows unfortunately: (also the log(10) issue in util_color is still present) ``` bf_intern_cycles.lib(device_cpu.obj) : error LNK2019: Verweis auf nicht aufgelös tes externes Symbol ""public: __cdecl ccl::Metropolis::Metropolis(struct ccl::Ke rnelGlobals *,double *,double *,class ccl::RenderTile *)" (??0Metropolis@ccl@@QE AA@PEAUKernelGlobals@1@PEAN1PEAVRenderTile@1@@Z)" in Funktion ""public: void __c decl ccl::CPUDevice::thread_path_trace(class ccl::DeviceTask &)" (?thread_path_t race@CPUDevice@ccl@@QEAAXAEAVDeviceTask@2@@Z)".

lucas i think blender already uses embree code for bvh.. ?

lucas i think blender already uses embree code for bvh.. ?

We use some code from Embree 1.x, in BVH Traversal and BVH build, yes.

We use some code from Embree 1.x, in BVH Traversal and BVH build, yes.
Member

Patch 15 fails to link for me (on linux):

../../lib/libcycles_device.a(device_cpu.cpp.o): In function `ccl::CPUDevice::thread_path_trace(ccl::DeviceTask&)':
device_cpu.cpp:(.text._ZN3ccl9CPUDevice17thread_path_traceERNS_10DeviceTaskE[_ZN3ccl9CPUDevice17thread_path_traceERNS_10DeviceTaskE]+0x114): undefined reference to `ccl::Metropolis::Metropolis(ccl::KernelGlobals*, double*, double*, ccl::RenderTile*)'
device_cpu.cpp:(.text._ZN3ccl9CPUDevice17thread_path_traceERNS_10DeviceTaskE[_ZN3ccl9CPUDevice17thread_path_traceERNS_10DeviceTaskE]+0x2ab): undefined reference to `ccl::Metropolis::consider_sample(float, ccl::float4, float*, float*, ccl::PassData&)'
device_cpu.cpp:(.text._ZN3ccl9CPUDevice17thread_path_traceERNS_10DeviceTaskE[_ZN3ccl9CPUDevice17thread_path_traceERNS_10DeviceTaskE]+0x2c2): undefined reference to `ccl::Metropolis::end_sample(int)'
device_cpu.cpp:(.text._ZN3ccl9CPUDevice17thread_path_traceERNS_10DeviceTaskE[_ZN3ccl9CPUDevice17thread_path_traceERNS_10DeviceTaskE]+0x2e5): undefined reference to `ccl::Metropolis::next_sample()'
../../lib/libcycles_kernel.a(kernel.cpp.o): In function `ccl::kernel_volume_integrate_heterogeneous(ccl::KernelGlobals*, ccl::PathState*, ccl::Ray*, ccl::ShaderData*, ccl::PathRadiance*, ccl::float3*, unsigned int*)':
kernel.cpp:(.text+0x39942): undefined reference to `ccl::metro_get_sample(ccl::Metropolis*, int)'
kernel.cpp:(.text+0x39a49): undefined reference to `ccl::metro_get_sample(ccl::Metropolis*, int)'
../../lib/libcycles_kernel.a(kernel.cpp.o): In function `ccl::kernel_volume_integrate_homogeneous(ccl::KernelGlobals*, ccl::PathState*, ccl::Ray*, ccl::ShaderData*, ccl::PathRadiance*, ccl::float3*, unsigned int*)':
kernel.cpp:(.text+0x3e47c): undefined reference to `ccl::metro_get_sample(ccl::Metropolis*, int)'
kernel.cpp:(.text+0x3e4dd): undefined reference to `ccl::metro_get_sample(ccl::Metropolis*, int)'
kernel.cpp:(.text+0x4062d): undefined reference to `ccl::metro_get_sample(ccl::Metropolis*, int)'
../../lib/libcycles_kernel.a(kernel.cpp.o):kernel.cpp:(.text+0x40713): more undefined references to `ccl::metro_get_sample(ccl::Metropolis*, int)' follow
collect2: error: ld returned 1 exit status
source/creator/CMakeFiles/blender.dir/build.make:281: recipe for target 'bin/blender' failed
Patch 15 fails to link for me (on linux): ``` ../../lib/libcycles_device.a(device_cpu.cpp.o): In function `ccl::CPUDevice::thread_path_trace(ccl::DeviceTask&)': device_cpu.cpp:(.text._ZN3ccl9CPUDevice17thread_path_traceERNS_10DeviceTaskE[_ZN3ccl9CPUDevice17thread_path_traceERNS_10DeviceTaskE]+0x114): undefined reference to `ccl::Metropolis::Metropolis(ccl::KernelGlobals*, double*, double*, ccl::RenderTile*)' device_cpu.cpp:(.text._ZN3ccl9CPUDevice17thread_path_traceERNS_10DeviceTaskE[_ZN3ccl9CPUDevice17thread_path_traceERNS_10DeviceTaskE]+0x2ab): undefined reference to `ccl::Metropolis::consider_sample(float, ccl::float4, float*, float*, ccl::PassData&)' device_cpu.cpp:(.text._ZN3ccl9CPUDevice17thread_path_traceERNS_10DeviceTaskE[_ZN3ccl9CPUDevice17thread_path_traceERNS_10DeviceTaskE]+0x2c2): undefined reference to `ccl::Metropolis::end_sample(int)' device_cpu.cpp:(.text._ZN3ccl9CPUDevice17thread_path_traceERNS_10DeviceTaskE[_ZN3ccl9CPUDevice17thread_path_traceERNS_10DeviceTaskE]+0x2e5): undefined reference to `ccl::Metropolis::next_sample()' ../../lib/libcycles_kernel.a(kernel.cpp.o): In function `ccl::kernel_volume_integrate_heterogeneous(ccl::KernelGlobals*, ccl::PathState*, ccl::Ray*, ccl::ShaderData*, ccl::PathRadiance*, ccl::float3*, unsigned int*)': kernel.cpp:(.text+0x39942): undefined reference to `ccl::metro_get_sample(ccl::Metropolis*, int)' kernel.cpp:(.text+0x39a49): undefined reference to `ccl::metro_get_sample(ccl::Metropolis*, int)' ../../lib/libcycles_kernel.a(kernel.cpp.o): In function `ccl::kernel_volume_integrate_homogeneous(ccl::KernelGlobals*, ccl::PathState*, ccl::Ray*, ccl::ShaderData*, ccl::PathRadiance*, ccl::float3*, unsigned int*)': kernel.cpp:(.text+0x3e47c): undefined reference to `ccl::metro_get_sample(ccl::Metropolis*, int)' kernel.cpp:(.text+0x3e4dd): undefined reference to `ccl::metro_get_sample(ccl::Metropolis*, int)' kernel.cpp:(.text+0x4062d): undefined reference to `ccl::metro_get_sample(ccl::Metropolis*, int)' ../../lib/libcycles_kernel.a(kernel.cpp.o):kernel.cpp:(.text+0x40713): more undefined references to `ccl::metro_get_sample(ccl::Metropolis*, int)' follow collect2: error: ld returned 1 exit status source/creator/CMakeFiles/blender.dir/build.make:281: recipe for target 'bin/blender' failed ```

We use some code from Embree 1.x, in BVH Traversal and BVH build, yes

Hmm. since embree is at 2.2 now,
i wonder if the embree bvh has been updated much since 1.x

*We use some code from Embree 1.x, in BVH Traversal and BVH build, yes* Hmm. since embree is at 2.2 now, i wonder if the embree bvh has been updated much since 1.x

Added subscriber: @candreacchio

Added subscriber: @candreacchio

i wonder if the embree bvh has been updated much since 1.x

According to this PDF here (their siggraph presentation for embree 2.0) There are significant speed ups... not to mention that this is only for the code to 2.0, not to mention more optimizations in 2.1 and 2.2

*i wonder if the embree bvh has been updated much since 1.x* According to this PDF [here ](http://embree.github.io/data/embree-siggraph-2013-final.pdf) (their siggraph presentation for embree 2.0) There are significant speed ups... not to mention that this is only for the code to 2.0, not to mention more optimizations in 2.1 and 2.2
, const btVector3&, btScalar, int)' defined but not used [-Wunused-function]
[ 95%] C:\Blendersvn\blender\extern\bullet2\src\BulletSoftBody\btSoftBodyInterna
ls.h:507:17: warning: 'void EvaluateMedium(const btSoftBodyWorldInfo*, const btV
ector3&, btSoftBody::sMedium&)' defined but not used [-Wunused-function]
Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/rna_tex
t.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/rna_tim
eline.c.obj
[ 95%] [ 95%] [ 95%] Building C object source/blender/makesrna/intern/CMakeFiles
/makesrna.dir/rna_tracking.c.obj
[ 95%] [ 95%] Building CXX object extern/bullet2/CMakeFiles/extern_bullet.dir/sr
c/BulletSoftBody/btSoftSoftCollisionAlgorithm.cpp.obj
[ 95%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/
rna_ui.c.obj
Building CXX object extern/bullet2/CMakeFiles/extern_bullet.dir/src/BulletSoftBo
dy/btDefaultSoftBodySolver.cpp.obj
Building CXX object extern/bullet2/CMakeFiles/extern_bullet.dir/src/LinearMath/b
tAlignedAllocator.cpp.obj
Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/rna_use
rdef.c.obj
[ 95%] [ 95%] Building CXX object extern/bullet2/CMakeFiles/extern_bullet.dir/sr
c/LinearMath/btConvexHull.cpp.obj
Building CXX object extern/bullet2/CMakeFiles/extern_bullet.dir/src/LinearMath/b
tConvexHullComputer.cpp.obj
[ 95%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/
rna_vfont.c.obj
[ 95%] [ 95%] Building C object source/blender/makesrna/intern/CMakeFiles/makesr
na.dir/rna_wm.c.obj
[ 96%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/
rna_world.c.obj
Building CXX object extern/bullet2/CMakeFiles/extern_bullet.dir/src/LinearMath/b
tGeometryUtil.cpp.obj
[ 96%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/
rna_action_api.c.obj
[ 96%] [ 96%] [ 96%] Building C object source/blender/makesrna/intern/CMakeFiles
/makesrna.dir/rna_actuator_api.c.obj
[ 96%] Building CXX object extern/bullet2/CMakeFiles/extern_bullet.dir/src/Linea
rMath/btSerializer.cpp.obj
[ 96%] Building CXX object extern/bullet2/CMakeFiles/extern_bullet.dir/src/Linea
rMath/btVector3.cpp.obj
Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/rna_ani
mation_api.c.obj
[ 97%] Building CXX object extern/bullet2/CMakeFiles/extern_bullet.dir/src/Linea
rMath/btQuickprof.cpp.obj
Building CXX object extern/bullet2/CMakeFiles/extern_bullet.dir/src/LinearMath/b
tPolarDecomposition.cpp.obj
[ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/
rna_armature_api.c.obj
[ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/
rna_camera_api.c.obj
[ 97%] [ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesr
na.dir/rna_curve_api.c.obj
[ 97%] [ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesr
na.dir/rna_controller_api.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/rna_fcu
rve_api.c.obj
[ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/
rna_lattice_api.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/rna_ima
ge_api.c.obj
Linking CXX static library ....\lib\libextern_bullet.a
[ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/
rna_main_api.c.obj
[ 97%] [ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesr
na.dir/rna_material_api.c.obj
[ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/
rna_mesh_api.c.obj
[ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/
rna_meta_api.c.obj
[ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/
rna_object_api.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/rna_tex
ture_api.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/rna_pos
e_api.c.obj
[ 97%] [ 97%] [ 97%] Building C object source/blender/makesrna/intern/CMakeFiles
/makesrna.dir/rna_scene_api.c.obj
[ 97%] [ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesr
na.dir/rna_sensor_api.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/rna_seq
uencer_api.c.obj
[ 97%] Built target extern_bullet
[ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/
rna_space_api.c.obj
[ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/
rna_text_api.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/rna_ui_
api.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/rna_wm_
api.c.obj
[ 97%] [ 97%] [ 97%] [ 97%] Building C object source/blender/makesrna/intern/CMa
keFiles/makesrna.dir/////intern/guardedalloc/intern/mallocn.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir///_
_/__/intern/guardedalloc/intern/mallocn_guarded_impl.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir///_
_/__/intern/guardedalloc/intern/mallocn_lockfree_impl.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir///_
_/__/intern/guardedalloc/intern/mmap_win.c.obj
Linking C executable ........\bin\makesrna.exe
[ 97%] Built target makesrna
[ 97%] Generating rna_ID_gen.c, rna_action_gen.c, rna_actuator_gen.c, rna_animat
ion_gen.c, rna_animviz_gen.c, rna_armature_gen.c, rna_boid_gen.c, rna_brush_gen.
c, rna_camera_gen.c, rna_cloth_gen.c, rna_color_gen.c, rna_constraint_gen.c, rna
_context_gen.c, rna_controller_gen.c, rna_curve_gen.c, rna_dynamicpaint_gen.c, r
na_fcurve_gen.c, rna_fluidsim_gen.c, rna_gpencil_gen.c, rna_group_gen.c, rna_ima
ge_gen.c, rna_key_gen.c, rna_lamp_gen.c, rna_lattice_gen.c, rna_linestyle_gen.c,
rna_main_gen.c, rna_mask_gen.c, rna_material_gen.c, rna_mesh_gen.c, rna_meta_ge
n.c, rna_modifier_gen.c, rna_movieclip_gen.c, rna_nla_gen.c, rna_nodetree_gen.c,
rna_object_gen.c, rna_object_force_gen.c, rna_packedfile_gen.c, rna_particle_ge
n.c, rna_pose_gen.c, rna_property_gen.c, rna_render_gen.c, rna_rigidbody_gen.c,
rna_rna_gen.c, rna_scene_gen.c, rna_screen_gen.c, rna_sculpt_paint_gen.c, rna_se
nsor_gen.c, rna_sequencer_gen.c, rna_smoke_gen.c, rna_sound_gen.c, rna_space_gen
.c, rna_speaker_gen.c, rna_test_gen.c, rna_text_gen.c, rna_texture_gen.c, rna_ti
meline_gen.c, rna_tracking_gen.c, rna_ui_gen.c, rna_userdef_gen.c, rna_vfont_gen
.c, rna_wm_gen.c, rna_world_gen.c
Running makesrna
Scanning dependencies of target bf_rna
[ 97%] [ 98%] [ 98%] [ 98%] [ 98%] [ 98%] [ 98%] Building C object source/blende
r/makesrna/intern/CMakeFiles/bf_rna.dir/rna_access.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_ID_ge
n.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_actua
tor_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_actio
n_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_anima
tion_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_animv
iz_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_armat
ure_gen.c.obj
[ 98%] [ 98%] [ 98%] Building C object source/blender/makesrna/intern/CMakeFiles
/bf_rna.dir/rna_boid_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_camer
a_gen.c.obj
[ 98%] C:\Blendersvn\blender\source\blender\makesrna\intern\rna_access.c: In fun
ction 'RNA_enum_is_equal':
C:\Blendersvn\blender\source\blender\makesrna\intern\rna_access.c:4757:8: warnin
g: 'cmp' may be used uninitialized in this function [-Wmaybe-uninitialized]
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_brush
_gen.c.obj
[ 98%] [ 98%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna
.dir/rna_cloth_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_color
_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_const
raint_gen.c.obj
[ 98%] [ 98%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna
.dir/rna_context_gen.c.obj
[ 98%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rn
a_controller_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_curve
_gen.c.obj
[ 98%] [ 98%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna
.dir/rna_dynamicpaint_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_fcurv
e_gen.c.obj
[ 98%] [ 98%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna
.dir/rna_fluidsim_gen.c.obj
[ 98%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rn
a_gpencil_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_group
_gen.c.obj
[ 98%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rn
a_image_gen.c.obj
[ 98%] [ 98%] [ 98%] Building C object source/blender/makesrna/intern/CMakeFiles
/bf_rna.dir/rna_lamp_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_key_g
en.c.obj
[ 98%] [ 99%] [ 99%] Building C object source/blender/makesrna/intern/CMakeFiles
/bf_rna.dir/rna_lattice_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_lines
tyle_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_mask_
gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_main_
gen.c.obj
[ 99%] [ 99%] [ 99%] Building C object source/blender/makesrna/intern/CMakeFiles
/bf_rna.dir/rna_material_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_mesh_
gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_meta_
gen.c.obj
[ 99%] [ 99%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna
.dir/rna_modifier_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_movie
clip_gen.c.obj
[ 99%] [ 99%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna
.dir/rna_nla_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_nodet
ree_gen.c.obj
[ 99%] [ 99%] [ 99%] Building C object source/blender/makesrna/intern/CMakeFiles
/bf_rna.dir/rna_object_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_objec
t_force_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_packe
dfile_gen.c.obj
[ 99%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rn
a_particle_gen.c.obj
[ 99%] [ 99%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna
.dir/rna_pose_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_prope
rty_gen.c.obj
[ 99%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rn
a_render_gen.c.obj
[ 99%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rn
a_rigidbody_gen.c.obj
[ 99%] [ 99%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna
.dir/rna_rna_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_scene
_gen.c.obj
[ 99%] [ 99%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna
.dir/rna_screen_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_sculp
t_paint_gen.c.obj
[ 99%] [ 99%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna
.dir/rna_sensor_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_seque
ncer_gen.c.obj
[ 99%] [ 99%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna
.dir/rna_smoke_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_sound
_gen.c.obj
[100%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rn
a_space_gen.c.obj
[100%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rn
a_speaker_gen.c.obj
[100%] [100%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna
.dir/rna_test_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_text_
gen.c.obj
[100%] [100%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna
.dir/rna_texture_gen.c.obj
[100%] [100%] [100%] Building C object source/blender/makesrna/intern/CMakeFiles
/bf_rna.dir/rna_timeline_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_track
ing_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_ui_ge
n.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_userd
ef_gen.c.obj
[100%] [100%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna
.dir/rna_vfont_gen.c.obj
[100%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rn
a_wm_gen.c.obj
Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_world
_gen.c.obj
Linking C static library ........\lib\libbf_rna.a
[100%] Built target bf_rna
Scanning dependencies of target bf_intern_cycles
[100%] [100%] [100%] [100%] [100%] [100%] [100%] Building CXX object intern/cycl
es/blender/CMakeFiles/bf_intern_cycles.dir/blender_camera.cpp.obj
Building CXX object intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/blende
r_mesh.cpp.obj
Building CXX object intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/blende
r_object.cpp.obj
Building CXX object intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/blende
r_curves.cpp.obj
Building CXX object intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/blende
r_particles.cpp.obj
Building CXX object intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/blende
r_python.cpp.obj
Building CXX object intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/blende
r_session.cpp.obj
[100%] [100%] Building CXX object intern/cycles/blender/CMakeFiles/bf_intern_cyc
les.dir/blender_shader.cpp.obj
Building CXX object intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/blende
r_sync.cpp.obj
Linking CXX static library ......\lib\libbf_intern_cycles.a
[100%] Built target bf_intern_cycles
Scanning dependencies of target blender
[100%] [100%] [100%] Building C object source/creator/CMakeFiles/blender.dir/cre
ator.c.obj
Building RC object source/creator/CMakeFiles/blender.dir/__/icons/winblender.rc.
obj
Building C object source/creator/CMakeFiles/blender.dir/buildinfo.c.obj
Linking CXX executable ....\bin\blender.exe
....\lib\libcycles_device.a(device_cpu.cpp.obj):device_cpu.cpp:(.text$_ZN3ccl9C
PUDevice17thread_path_traceERNS_10DeviceTaskE[_ZN3ccl9CPUDevice17thread_path_tra
ceERNS_10DeviceTaskE]+0x1c4): undefined reference to `ccl::Metropolis::Metropoli
s(ccl::KernelGlobals*, double*, double*, ccl::RenderTile*)'
....\lib\libcycles_device.a(device_cpu.cpp.obj):device_cpu.cpp:(.text$_ZN3ccl9C
PUDevice17thread_path_traceERNS_10DeviceTaskE[_ZN3ccl9CPUDevice17thread_path_tra
ceERNS_10DeviceTaskE]+0x3b3): undefined reference to `ccl::Metropolis::consider_
sample(float, ccl::float4, float*, float*, ccl::PassData&)'
c:/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.7.3/../../../../x86_64-w64-mingw32/
bin/ld.exe: ....\lib\libcycles_device.a(device_cpu.cpp.obj): bad reloc address
0x3b3 in section `.text$ZN3ccl9CPUDevice17thread_path_traceERNS_10DeviceTaskE[
ZN3ccl9CPUDevice17thread_path_traceERNS_10DeviceTaskE]'
collect2.exe: error: ld returned 1 exit status
source\creator\CMakeFiles\blender.dir\build.make:247: recipe for target `bin/ble
nder.exe' failed
mingw32-make- [x]: *** [bin/blender.exe] Error 1
CMakeFiles\Makefile2:6670: recipe for target `source/creator/CMakeFiles/blender.
dir/all' failed
mingw32-make- [x]: *** [source/creator/CMakeFiles/blender.dir/all] Error 2
makefile:135: recipe for target `all' failed
mingw32-make: *** [all] Error 2

c:\Blendersvn\build> |

Linking error Patch 15

| , const btVector3&, btScalar, int)' defined but not used [-Wunused-function] | -- | [ 95%] C:\Blendersvn\blender\extern\bullet2\src\BulletSoftBody\btSoftBodyInterna ls.h:507:17: warning: 'void EvaluateMedium(const btSoftBodyWorldInfo*, const btV ector3&, btSoftBody::sMedium&)' defined but not used [-Wunused-function] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/rna_tex t.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/rna_tim eline.c.obj [ 95%] [ 95%] [ 95%] Building C object source/blender/makesrna/intern/CMakeFiles /makesrna.dir/rna_tracking.c.obj [ 95%] [ 95%] Building CXX object extern/bullet2/CMakeFiles/extern_bullet.dir/sr c/BulletSoftBody/btSoftSoftCollisionAlgorithm.cpp.obj [ 95%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/ rna_ui.c.obj Building CXX object extern/bullet2/CMakeFiles/extern_bullet.dir/src/BulletSoftBo dy/btDefaultSoftBodySolver.cpp.obj Building CXX object extern/bullet2/CMakeFiles/extern_bullet.dir/src/LinearMath/b tAlignedAllocator.cpp.obj Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/rna_use rdef.c.obj [ 95%] [ 95%] Building CXX object extern/bullet2/CMakeFiles/extern_bullet.dir/sr c/LinearMath/btConvexHull.cpp.obj Building CXX object extern/bullet2/CMakeFiles/extern_bullet.dir/src/LinearMath/b tConvexHullComputer.cpp.obj [ 95%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/ rna_vfont.c.obj [ 95%] [ 95%] Building C object source/blender/makesrna/intern/CMakeFiles/makesr na.dir/rna_wm.c.obj [ 96%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/ rna_world.c.obj Building CXX object extern/bullet2/CMakeFiles/extern_bullet.dir/src/LinearMath/b tGeometryUtil.cpp.obj [ 96%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/ rna_action_api.c.obj [ 96%] [ 96%] [ 96%] Building C object source/blender/makesrna/intern/CMakeFiles /makesrna.dir/rna_actuator_api.c.obj [ 96%] Building CXX object extern/bullet2/CMakeFiles/extern_bullet.dir/src/Linea rMath/btSerializer.cpp.obj [ 96%] Building CXX object extern/bullet2/CMakeFiles/extern_bullet.dir/src/Linea rMath/btVector3.cpp.obj Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/rna_ani mation_api.c.obj [ 97%] Building CXX object extern/bullet2/CMakeFiles/extern_bullet.dir/src/Linea rMath/btQuickprof.cpp.obj Building CXX object extern/bullet2/CMakeFiles/extern_bullet.dir/src/LinearMath/b tPolarDecomposition.cpp.obj [ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/ rna_armature_api.c.obj [ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/ rna_camera_api.c.obj [ 97%] [ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesr na.dir/rna_curve_api.c.obj [ 97%] [ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesr na.dir/rna_controller_api.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/rna_fcu rve_api.c.obj [ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/ rna_lattice_api.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/rna_ima ge_api.c.obj Linking CXX static library ..\..\lib\libextern_bullet.a [ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/ rna_main_api.c.obj [ 97%] [ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesr na.dir/rna_material_api.c.obj [ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/ rna_mesh_api.c.obj [ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/ rna_meta_api.c.obj [ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/ rna_object_api.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/rna_tex ture_api.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/rna_pos e_api.c.obj [ 97%] [ 97%] [ 97%] Building C object source/blender/makesrna/intern/CMakeFiles /makesrna.dir/rna_scene_api.c.obj [ 97%] [ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesr na.dir/rna_sensor_api.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/rna_seq uencer_api.c.obj [ 97%] Built target extern_bullet [ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/ rna_space_api.c.obj [ 97%] Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/ rna_text_api.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/rna_ui_ api.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/rna_wm_ api.c.obj [ 97%] [ 97%] [ 97%] [ 97%] Building C object source/blender/makesrna/intern/CMa keFiles/makesrna.dir/__/__/__/__/intern/guardedalloc/intern/mallocn.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/__/__/_ _/__/intern/guardedalloc/intern/mallocn_guarded_impl.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/__/__/_ _/__/intern/guardedalloc/intern/mallocn_lockfree_impl.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/makesrna.dir/__/__/_ _/__/intern/guardedalloc/intern/mmap_win.c.obj Linking C executable ..\..\..\..\bin\makesrna.exe [ 97%] Built target makesrna [ 97%] Generating rna_ID_gen.c, rna_action_gen.c, rna_actuator_gen.c, rna_animat ion_gen.c, rna_animviz_gen.c, rna_armature_gen.c, rna_boid_gen.c, rna_brush_gen. c, rna_camera_gen.c, rna_cloth_gen.c, rna_color_gen.c, rna_constraint_gen.c, rna _context_gen.c, rna_controller_gen.c, rna_curve_gen.c, rna_dynamicpaint_gen.c, r na_fcurve_gen.c, rna_fluidsim_gen.c, rna_gpencil_gen.c, rna_group_gen.c, rna_ima ge_gen.c, rna_key_gen.c, rna_lamp_gen.c, rna_lattice_gen.c, rna_linestyle_gen.c, rna_main_gen.c, rna_mask_gen.c, rna_material_gen.c, rna_mesh_gen.c, rna_meta_ge n.c, rna_modifier_gen.c, rna_movieclip_gen.c, rna_nla_gen.c, rna_nodetree_gen.c, rna_object_gen.c, rna_object_force_gen.c, rna_packedfile_gen.c, rna_particle_ge n.c, rna_pose_gen.c, rna_property_gen.c, rna_render_gen.c, rna_rigidbody_gen.c, rna_rna_gen.c, rna_scene_gen.c, rna_screen_gen.c, rna_sculpt_paint_gen.c, rna_se nsor_gen.c, rna_sequencer_gen.c, rna_smoke_gen.c, rna_sound_gen.c, rna_space_gen .c, rna_speaker_gen.c, rna_test_gen.c, rna_text_gen.c, rna_texture_gen.c, rna_ti meline_gen.c, rna_tracking_gen.c, rna_ui_gen.c, rna_userdef_gen.c, rna_vfont_gen .c, rna_wm_gen.c, rna_world_gen.c Running makesrna Scanning dependencies of target bf_rna [ 97%] [ 98%] [ 98%] [ 98%] [ 98%] [ 98%] [ 98%] Building C object source/blende r/makesrna/intern/CMakeFiles/bf_rna.dir/rna_access.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_ID_ge n.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_actua tor_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_actio n_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_anima tion_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_animv iz_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_armat ure_gen.c.obj [ 98%] [ 98%] [ 98%] Building C object source/blender/makesrna/intern/CMakeFiles /bf_rna.dir/rna_boid_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_camer a_gen.c.obj [ 98%] C:\Blendersvn\blender\source\blender\makesrna\intern\rna_access.c: In fun ction 'RNA_enum_is_equal': C:\Blendersvn\blender\source\blender\makesrna\intern\rna_access.c:4757:8: warnin g: 'cmp' may be used uninitialized in this function [-Wmaybe-uninitialized] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_brush _gen.c.obj [ 98%] [ 98%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna .dir/rna_cloth_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_color _gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_const raint_gen.c.obj [ 98%] [ 98%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna .dir/rna_context_gen.c.obj [ 98%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rn a_controller_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_curve _gen.c.obj [ 98%] [ 98%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna .dir/rna_dynamicpaint_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_fcurv e_gen.c.obj [ 98%] [ 98%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna .dir/rna_fluidsim_gen.c.obj [ 98%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rn a_gpencil_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_group _gen.c.obj [ 98%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rn a_image_gen.c.obj [ 98%] [ 98%] [ 98%] Building C object source/blender/makesrna/intern/CMakeFiles /bf_rna.dir/rna_lamp_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_key_g en.c.obj [ 98%] [ 99%] [ 99%] Building C object source/blender/makesrna/intern/CMakeFiles /bf_rna.dir/rna_lattice_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_lines tyle_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_mask_ gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_main_ gen.c.obj [ 99%] [ 99%] [ 99%] Building C object source/blender/makesrna/intern/CMakeFiles /bf_rna.dir/rna_material_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_mesh_ gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_meta_ gen.c.obj [ 99%] [ 99%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna .dir/rna_modifier_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_movie clip_gen.c.obj [ 99%] [ 99%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna .dir/rna_nla_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_nodet ree_gen.c.obj [ 99%] [ 99%] [ 99%] Building C object source/blender/makesrna/intern/CMakeFiles /bf_rna.dir/rna_object_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_objec t_force_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_packe dfile_gen.c.obj [ 99%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rn a_particle_gen.c.obj [ 99%] [ 99%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna .dir/rna_pose_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_prope rty_gen.c.obj [ 99%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rn a_render_gen.c.obj [ 99%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rn a_rigidbody_gen.c.obj [ 99%] [ 99%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna .dir/rna_rna_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_scene _gen.c.obj [ 99%] [ 99%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna .dir/rna_screen_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_sculp t_paint_gen.c.obj [ 99%] [ 99%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna .dir/rna_sensor_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_seque ncer_gen.c.obj [ 99%] [ 99%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna .dir/rna_smoke_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_sound _gen.c.obj [100%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rn a_space_gen.c.obj [100%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rn a_speaker_gen.c.obj [100%] [100%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna .dir/rna_test_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_text_ gen.c.obj [100%] [100%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna .dir/rna_texture_gen.c.obj [100%] [100%] [100%] Building C object source/blender/makesrna/intern/CMakeFiles /bf_rna.dir/rna_timeline_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_track ing_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_ui_ge n.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_userd ef_gen.c.obj [100%] [100%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna .dir/rna_vfont_gen.c.obj [100%] Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rn a_wm_gen.c.obj Building C object source/blender/makesrna/intern/CMakeFiles/bf_rna.dir/rna_world _gen.c.obj Linking C static library ..\..\..\..\lib\libbf_rna.a [100%] Built target bf_rna Scanning dependencies of target bf_intern_cycles [100%] [100%] [100%] [100%] [100%] [100%] [100%] Building CXX object intern/cycl es/blender/CMakeFiles/bf_intern_cycles.dir/blender_camera.cpp.obj Building CXX object intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/blende r_mesh.cpp.obj Building CXX object intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/blende r_object.cpp.obj Building CXX object intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/blende r_curves.cpp.obj Building CXX object intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/blende r_particles.cpp.obj Building CXX object intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/blende r_python.cpp.obj Building CXX object intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/blende r_session.cpp.obj [100%] [100%] Building CXX object intern/cycles/blender/CMakeFiles/bf_intern_cyc les.dir/blender_shader.cpp.obj Building CXX object intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/blende r_sync.cpp.obj Linking CXX static library ..\..\..\lib\libbf_intern_cycles.a [100%] Built target bf_intern_cycles Scanning dependencies of target blender [100%] [100%] [100%] Building C object source/creator/CMakeFiles/blender.dir/cre ator.c.obj Building RC object source/creator/CMakeFiles/blender.dir/__/icons/winblender.rc. obj Building C object source/creator/CMakeFiles/blender.dir/buildinfo.c.obj Linking CXX executable ..\..\bin\blender.exe ..\..\lib\libcycles_device.a(device_cpu.cpp.obj):device_cpu.cpp:(.text$_ZN3ccl9C PUDevice17thread_path_traceERNS_10DeviceTaskE[_ZN3ccl9CPUDevice17thread_path_tra ceERNS_10DeviceTaskE]+0x1c4): undefined reference to `ccl::Metropolis::Metropoli s(ccl::KernelGlobals*, double*, double*, ccl::RenderTile*)' ..\..\lib\libcycles_device.a(device_cpu.cpp.obj):device_cpu.cpp:(.text$_ZN3ccl9C PUDevice17thread_path_traceERNS_10DeviceTaskE[_ZN3ccl9CPUDevice17thread_path_tra ceERNS_10DeviceTaskE]+0x3b3): undefined reference to `ccl::Metropolis::consider_ sample(float, ccl::float4, float*, float*, ccl::PassData&)' c:/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.7.3/../../../../x86_64-w64-mingw32/ bin/ld.exe: ..\..\lib\libcycles_device.a(device_cpu.cpp.obj): bad reloc address 0x3b3 in section `.text$_ZN3ccl9CPUDevice17thread_path_traceERNS_10DeviceTaskE[_ ZN3ccl9CPUDevice17thread_path_traceERNS_10DeviceTaskE]' collect2.exe: error: ld returned 1 exit status source\creator\CMakeFiles\blender.dir\build.make:247: recipe for target `bin/ble nder.exe' failed mingw32-make- [x]: *** [bin/blender.exe] Error 1 CMakeFiles\Makefile2:6670: recipe for target `source/creator/CMakeFiles/blender. dir/all' failed mingw32-make- [x]: *** [source/creator/CMakeFiles/blender.dir/all] Error 2 makefile:135: recipe for target `all' failed mingw32-make: *** [all] Error 2 c:\Blendersvn\build> | Linking error Patch 15

omg sorry.. can someone shorted that?
i cant edit it on here.

omg sorry.. can someone shorted that? i cant edit it on here.
Author
Member

Okay, my bad. I forgot to add util_importance.h/cpp to CMakeLists in the utils tolder, so it doesn't eork for CMake users. Just add them and it should work.

Okay, my bad. I forgot to add util_importance.h/cpp to CMakeLists in the utils tolder, so it doesn't eork for CMake users. Just add them and it should work.

hmmm.. i added it but now i get an error when cmake generating

Compiling for 64 bit with MinGW-w64.
Found unordered_map/set in std::tr1 namespace.
Blender Skipping: (bf_collada;bf_intern_ctr;bf_intern_ghostndof3dconnexion;extern_redcode)
  • Found Git: C:/Program Files (x86)/Git/cmd/git.exe
    Configuring done
    CMake Error at intern/cycles/util/CMakeLists.txt:74 (add_library):
Cannot find source file:
  util_importance.ccp
Tried extensions .c .C .c++ .cc .cpp .cxx .m .M .mm .h .hh .h++ .hm .hpp
.hxx .in .txx |
hmmm.. i added it but now i get an error when cmake generating | Compiling for 64 bit with MinGW-w64. | -- | Found unordered_map/set in std::tr1 namespace. Blender Skipping: (bf_collada;bf_intern_ctr;bf_intern_ghostndof3dconnexion;extern_redcode) - Found Git: C:/Program Files (x86)/Git/cmd/git.exe Configuring done CMake Error at intern/cycles/util/CMakeLists.txt:74 (add_library): ``` Cannot find source file: ``` ``` util_importance.ccp ``` ``` Tried extensions .c .C .c++ .cc .cpp .cxx .m .M .mm .h .hh .h++ .hm .hpp .hxx .in .txx | ```
Author
Member

You wrote .ccp instead of .cpp as the file extension.

You wrote .ccp instead of .cpp as the file extension.

umm you're correct i spelled it wrong..

however i dont have a util_importance.cpp file in that folder.. ?

can i compile with it(leave the .cpp out an just use the .h?

umm you're correct i spelled it wrong.. however i dont have a util_importance.cpp file in that folder.. ? can i compile with it(leave the .cpp out an just use the .h?
Author
Member

OK, I failed -.- I meant util_metropolis, not util_importance.

OK, I failed -.- I meant util_metropolis, not util_importance.

now i get error about compositor?

Linking C static library ....\lib\libbf_intern_opencl.a
[ 85%] [ 85%] [ 85%] [ 85%] Building CXX object intern/cycles/render/CMakeFiles/
cycles_render.dir/tile.cpp.obj
Building CXX object intern/cycles/render/CMakeFiles/cycles_render.dir/tables.cpp
.obj
[ 85%] [ 85%] [ 85%] Building CXX object source/blender/compositor/CMakeFiles/bf
_compositor.dir/operations/COM_EllipseMaskOperation.cpp.obj
Building CXX object source/blender/compositor/CMakeFiles/bf_compositor.dir/opera
tions/COM_ConvertOperation.cpp.obj
Building CXX object source/blender/compositor/CMakeFiles/bf_compositor.dir/opera
tions/COM_BoxMaskOperation.cpp.obj
Building CXX object source/blender/compositor/CMakeFiles/bf_compositor.dir/opera
tions/COM_MovieClipOperation.cpp.obj
Building CXX object source/blender/compositor/CMakeFiles/bf_compositor.dir/opera
tions/COM_DotproductOperation.cpp.obj
[ 85%] Built target bf_intern_opencl
[ 85%] [ 85%] [ 85%] Building CXX object source/blender/compositor/CMakeFiles/bf
_compositor.dir/operations/COM_AntiAliasOperation.cpp.obj
Building CXX object source/blender/compositor/CMakeFiles/bf_compositor.dir/opera
tions/COM_ConvertColorProfileOperation.cpp.obj
Building CXX object source/blender/compositor/CMakeFiles/bf_compositor.dir/opera
tions/COM_MaskOperation.cpp.obj
Linking CXX static library ......\lib\libbf_compositor.a
Linking CXX static library ......\lib\libcycles_render.a
[ 85%] [ 85%] Built target bf_compositor
Built target cycles_render
makefile:145: recipe for target `all' failed
mingw32-make: *** [all] Error 2

c:\Blendersvn\build> |

now i get error about compositor? | Linking C static library ..\..\lib\libbf_intern_opencl.a | -- | [ 85%] [ 85%] [ 85%] [ 85%] Building CXX object intern/cycles/render/CMakeFiles/ cycles_render.dir/tile.cpp.obj Building CXX object intern/cycles/render/CMakeFiles/cycles_render.dir/tables.cpp .obj [ 85%] [ 85%] [ 85%] Building CXX object source/blender/compositor/CMakeFiles/bf _compositor.dir/operations/COM_EllipseMaskOperation.cpp.obj Building CXX object source/blender/compositor/CMakeFiles/bf_compositor.dir/opera tions/COM_ConvertOperation.cpp.obj Building CXX object source/blender/compositor/CMakeFiles/bf_compositor.dir/opera tions/COM_BoxMaskOperation.cpp.obj Building CXX object source/blender/compositor/CMakeFiles/bf_compositor.dir/opera tions/COM_MovieClipOperation.cpp.obj Building CXX object source/blender/compositor/CMakeFiles/bf_compositor.dir/opera tions/COM_DotproductOperation.cpp.obj [ 85%] Built target bf_intern_opencl [ 85%] [ 85%] [ 85%] Building CXX object source/blender/compositor/CMakeFiles/bf _compositor.dir/operations/COM_AntiAliasOperation.cpp.obj Building CXX object source/blender/compositor/CMakeFiles/bf_compositor.dir/opera tions/COM_ConvertColorProfileOperation.cpp.obj Building CXX object source/blender/compositor/CMakeFiles/bf_compositor.dir/opera tions/COM_MaskOperation.cpp.obj Linking CXX static library ..\..\..\lib\libbf_compositor.a Linking CXX static library ..\..\..\lib\libcycles_render.a [ 85%] [ 85%] Built target bf_compositor Built target cycles_render makefile:145: recipe for target `all' failed mingw32-make: *** [all] Error 2 c:\Blendersvn\build> |
Author
Member

Ehm, this looks like an error in your Makefile. Try a full clean and recompiling, but since I use SCons, I don't really know what to do in CMake.

Ehm, this looks like an error in your Makefile. Try a full clean and recompiling, but since I use SCons, I don't really know what to do in CMake.
Member

CMake compiles after adding this extra patch: cmake_fix_for_metropolis_15.patch

CMake compiles after adding this extra patch: [cmake_fix_for_metropolis_15.patch](https://archive.blender.org/developer/F80030/cmake_fix_for_metropolis_15.patch)

I'm getting an error from cmake telling me there's no util_metropolis.cpp file. Indeed, upon searching, there isn't. Diff applied against master on Win7 64.

I'm getting an error from cmake telling me there's no util_metropolis.cpp file. Indeed, upon searching, there isn't. Diff applied against master on Win7 64.

lockal, thank you that did it!

compiled successfully mingw64 cmake..
metro_15 is considerably faster than metro_14

m9105826
make sure you applied the metrolis15.diff first then apply the cmake_fix for 15.patch
try it on a fresh clean blender folder

thanks lucasstockner97 keep up the goo work. :)

lockal, thank you that did it! compiled successfully mingw64 cmake.. metro_15 is considerably faster than metro_14 -------------- m9105826 make sure you applied the metrolis15.diff first then apply the cmake_fix for 15.patch try it on a fresh clean blender folder thanks lucasstockner97 keep up the goo work. :)

Indeed, no more cmake issues, but I get the same exact linker error as Thomas with VS2013, 64-bit.

Indeed, no more cmake issues, but I get the same exact linker error as Thomas with VS2013, 64-bit.

here is the metro_15 mingw64 cmake build of blender 2.70 with the new splash. :)

http://www.mediafire.com/download/o1azyqyqo3noq4u/Blender_mingw64_metropolis_patch_15.7z

here is the metro_15 mingw64 cmake build of blender 2.70 with the new splash. :) http://www.mediafire.com/download/o1azyqyqo3noq4u/Blender_mingw64_metropolis_patch_15.7z

Tried Holyenigma's build, and I have a bug report for you.

I noticed that Metropolis sampling will crash Blender whenever there's a volume material in the scene, even if the only object is the default cube.

This doesn't affect the adaptive sampling though, though it seems MingW builds will become unstable if you zoom and pan a bit while the scene is rendering.

Tried Holyenigma's build, and I have a bug report for you. I noticed that Metropolis sampling will crash Blender whenever there's a volume material in the scene, even if the only object is the default cube. This doesn't affect the adaptive sampling though, though it seems MingW builds will become unstable if you zoom and pan a bit while the scene is rendering.

more samples give more and more singledots noise around object

in pathtracing this noise is uniform and not cotrast, in mlt not

2.jpg

3.jpg

more samples give more and more singledots noise around object in pathtracing this noise is uniform and not cotrast, in mlt not ![2.jpg](https://archive.blender.org/developer/F80050/2.jpg) ![3.jpg](https://archive.blender.org/developer/F80052/3.jpg)

Lopataasdf; Having tried Luxrender before, this is an expected side effect due to how Metropolis sampling works.

You see, Metropolis is finding lightpaths that would otherwise not be found with normal pathtracing, so the dots represent those paths and they should converge to a result that contains lighting effects that are hard to get in Cycles without the patch.

Lopataasdf; Having tried Luxrender before, this is an expected side effect due to how Metropolis sampling works. You see, Metropolis is finding lightpaths that would otherwise not be found with normal pathtracing, so the dots represent those paths and they should converge to a result that contains lighting effects that are hard to get in Cycles without the patch.

There appears to be something weird going on in the MinGW build in the viewport. Looks to be some kind of issue with the Alpha value? Using just path tracing, transparency is turned off. Sorry if this is a known issue.

alpha.PNG

F12 rendering doesn't produce the same issue.

There appears to be something weird going on in the MinGW build in the viewport. Looks to be some kind of issue with the Alpha value? Using just path tracing, transparency is turned off. Sorry if this is a known issue. ![alpha.PNG](https://archive.blender.org/developer/F80057/alpha.PNG) F12 rendering doesn't produce the same issue.

ace_dragon, can you post a .blend of the volume material that crashes?

ace_dragon, can you post a .blend of the volume material that crashes?

m9105826, the dingto build does the same thing.
so it must be something else.

m9105826, the dingto build does the same thing. so it must be something else.
Member

Building fails for me (cmake, archlinux) with metro_15 and the cmake fix:

In file included from /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/kernel_random.h:18:0,
                 from /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/kernel_path.h:29,
                 from /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/kernel_avx.cpp:39:
/home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/../util/util_metropolis.h: At global scope:
/home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/../util/util_metropolis.h:31:14: warning: 'float ccl::Mutate(float, float)' declared 'static' but never defined [-Wunused-function]
 static float Mutate(const float x, const float randomValue);
              ^
/home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/../util/util_metropolis.h:32:14: warning: 'float ccl::MutateScaled(float, float, float)' declared 'static' but never defined [-Wunused-function]
 static float MutateScaled(const float x, const float range, const float randomValue);
              ^
Linking CXX static library ../../../lib/libcycles_kernel.a
[ 74%] Built target cycles_kernel
Linking CXX static library ../../../lib/libbf_freestyle.a
[ 74%] Built target bf_freestyle
Makefile:146: recipe for target 'all' failed
make[1]: *** [all] Error 2
Building fails for me (cmake, archlinux) with metro_15 and the cmake fix: ``` In file included from /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/kernel_random.h:18:0, from /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/kernel_path.h:29, from /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/kernel_avx.cpp:39: /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/../util/util_metropolis.h: At global scope: /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/../util/util_metropolis.h:31:14: warning: 'float ccl::Mutate(float, float)' declared 'static' but never defined [-Wunused-function] static float Mutate(const float x, const float randomValue); ^ /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/../util/util_metropolis.h:32:14: warning: 'float ccl::MutateScaled(float, float, float)' declared 'static' but never defined [-Wunused-function] static float MutateScaled(const float x, const float range, const float randomValue); ^ Linking CXX static library ../../../lib/libcycles_kernel.a [ 74%] Built target cycles_kernel Linking CXX static library ../../../lib/libbf_freestyle.a [ 74%] Built target bf_freestyle Makefile:146: recipe for target 'all' failed make[1]: *** [all] Error 2 ```

Holyenigma; do I really need to post a .blend, here are a few steps to see for yourself (it's real easy).

1). Open Blender
2). Give the default cube a transparent material with a scattering volume
3). Switch to Metropolis sampling and leave every other setting alone
4). Watch Blender crash when Cycles gets ready to update the render window.

And that's the thing, the crash happens not right when I hit F12, but when Cycles is about to show the initial results on the screen.

Holyenigma; do I really need to post a .blend, here are a few steps to see for yourself (it's real easy). 1). Open Blender 2). Give the default cube a transparent material with a scattering volume 3). Switch to Metropolis sampling and leave every other setting alone 4). Watch Blender crash when Cycles gets ready to update the render window. And that's the thing, the crash happens not right when I hit F12, but when Cycles is about to show the initial results on the screen.

ace_dragon, thanks i see it now.
i also noticed mingw build wont open some of my old blend files(dingto build opens then fine)
i opened the old blend with dingto build and saved it, then it loads into mingw build

ace_dragon, thanks i see it now. i also noticed mingw build wont open some of my old blend files(dingto build opens then fine) i opened the old blend with dingto build and saved it, then it loads into mingw build

i figured out what was causing the .blends on mingw to crash on open..
if the .blend is saved with transparent checked under Film, uncheck it and save the .blend

i figured out what was causing the .blends on mingw to crash on open.. if the .blend is saved with transparent checked under Film, uncheck it and save the .blend
Author
Member

Okay, thanks for the reports.

  • Transparent viewport rendering: Thanks, I see it. By the way: Metropolis preview seems to work, but not nearly as well as it should.
  • The Volume bug seems to get worse every patch, I guess it's because another commit in trunk conflicts with the patch, since before I pulled in the latest changes, it worked fine. However, if it crashes, it's maybe even better since crashes give a location where the problem is.
  • The single-pixel noise is quite expected. Also consider that from whar I cam see on the screenshots, the PT one has 1000 samples while the Metro one only has ~100. If you want to get rid of the noise, try lowering the max. consecutive rejects setting, but this will introduce bias.
  • I can load files with transparent film activated just fine, could you maybe post one that doesn't open?

Anyways, I'm going to work on this. Once I have something new, I'll post it here.

Okay, thanks for the reports. - Transparent viewport rendering: Thanks, I see it. By the way: Metropolis preview seems to work, but not nearly as well as it should. - The Volume bug seems to get worse every patch, I guess it's because another commit in trunk conflicts with the patch, since before I pulled in the latest changes, it worked fine. However, if it crashes, it's maybe even better since crashes give a location where the problem is. - The single-pixel noise is quite expected. Also consider that from whar I cam see on the screenshots, the PT one has 1000 samples while the Metro one only has ~100. If you want to get rid of the noise, try lowering the max. consecutive rejects setting, but this will introduce bias. - I can load files with transparent film activated just fine, could you maybe post one that doesn't open? Anyways, I'm going to work on this. Once I have something new, I'll post it here.

lucas, try opening this with mingw build,
works fine with vc build.

http://www.mediafire.com/download/iiboskfrnrrz6ea/fire_extingusher.7z

lucas, try opening this with mingw build, works fine with vc build. http://www.mediafire.com/download/iiboskfrnrrz6ea/fire_extingusher.7z

if the background is pink just point it to your own hdr,
or use sky background in world setting.

if the background is pink just point it to your own hdr, or use sky background in world setting.

possible new build with the newest changes.?
Windows 7 64

ahh sorry there is one. i missed

possible new build with the newest changes.? Windows 7 64 ahh sorry there is one. i missed

I had another really strange issue that I'm having trouble recreating at the moment. It was in the same session where I did the transparent screen grab. After working for a few minutes and doing a couple of f12 renders, I turned viewport rendering back on and got extremely blown out lights and strange colors not previously present in the scene. Switching back to path tracing caused different, but similar results.

Additionally, path tracing and metropolis have been converging to different results. I'll post an example .blend when I'm back on my PC.

I had another really strange issue that I'm having trouble recreating at the moment. It was in the same session where I did the transparent screen grab. After working for a few minutes and doing a couple of f12 renders, I turned viewport rendering back on and got extremely blown out lights and strange colors not previously present in the scene. Switching back to path tracing caused different, but similar results. Additionally, path tracing and metropolis have been converging to different results. I'll post an example .blend when I'm back on my PC.

Added subscriber: @xs

Added subscriber: @xs

Hi,
I've been looking through this thread to find a compiled version for Windows 7/32.
I am aware patches are released as code, not compiled, so I may miss the latest. I am not ready to experiment with compiling..
So compiled version?
Thanks,
Christos

Hi, I've been looking through this thread to find a compiled version for Windows 7/32. I am aware patches are released as code, not compiled, so I may miss the latest. I am not ready to experiment with compiling.. So compiled version? Thanks, Christos
Author
Member

New patch version, cmake should now work ouf-of-the-box and the volume bug is fixed.
It turned out that the fix for #38710 caused it, now when using Metro, the bugfix is simply not applied. This doesn't mean that #38710 now appears in Metro however, because it is related to the sampler and Metro overrides that one anyways.

This is my current issue-list in random order:

  • Builds for Windows 32bit (OK, this isn't such a big priority).
  • Viewport rendering is still broken, I still haven't found out why. Clearly needs to be fixed.
  • I noticed that in the BMW scene, sometimes it just renders the beckground with Metro on. When disabling DOF, the issue seems to disappear, but this needs further investigation.
  • @MatthewHeimlich Regarding the different results, I haven't been able to reproduce this with any scene tried, a example file would indeed be useful.
  • @holyenigma Crashing with transparent enabled: For me, the fire extinguisher works just fine. I don't have access to a x64 Windows, so I can't try the build, but I'm currently building a x86 one to test. It'd be really useful if you could upload a crashlog.
  • CMake building should be fixed now, at least it works on my Linux and (at least until now) on XP.
  • Importance equalization (my new name for Division-By-Mean-Brightness): A feature, not a bug, but definitely useful.

Have I forgotten anything? If yes, please tell me.
metropolis_16.diff

New patch version, cmake should now work ouf-of-the-box and the volume bug is fixed. It turned out that the fix for #38710 caused it, now when using Metro, the bugfix is simply not applied. This doesn't mean that #38710 now appears in Metro however, because it is related to the sampler and Metro overrides that one anyways. This is my current issue-list in random order: - Builds for Windows 32bit (OK, this isn't such a big priority). - Viewport rendering is still broken, I still haven't found out why. Clearly needs to be fixed. - I noticed that in the BMW scene, sometimes it just renders the beckground with Metro on. When disabling DOF, the issue seems to disappear, but this needs further investigation. - @MatthewHeimlich Regarding the different results, I haven't been able to reproduce this with any scene tried, a example file would indeed be useful. - @holyenigma Crashing with transparent enabled: For me, the fire extinguisher works just fine. I don't have access to a x64 Windows, so I can't try the build, but I'm currently building a x86 one to test. It'd be really useful if you could upload a crashlog. - CMake building should be fixed now, at least it works on my Linux and (at least until now) on XP. - Importance equalization (my new name for Division-By-Mean-Brightness): A feature, not a bug, but definitely useful. Have I forgotten anything? If yes, please tell me. [metropolis_16.diff](https://archive.blender.org/developer/F80461/metropolis_16.diff)

Added subscriber: @GoldenCrescent

Added subscriber: @GoldenCrescent

(Bump) Hi,
I've been looking through this thread to find a compiled version for Windows 7/32.
I am aware patches are released as code, not compiled, so I may miss the latest. I am not ready to experiment with compiling..
So compiled version?
Thanks,
Christos

(Bump) Hi, I've been looking through this thread to find a compiled version for Windows 7/32. I am aware patches are released as code, not compiled, so I may miss the latest. I am not ready to experiment with compiling.. So compiled version? Thanks, Christos
Author
Member

Just to let you know, I'm still working on the patch and making progress, the transparent viewport bug is fixed, Importance Equalization works and most of the image-wide adaptive sampling is done as well. Once everything is stable enough, I'll post it here.

Just to let you know, I'm still working on the patch and making progress, the transparent viewport bug is fixed, Importance Equalization works and most of the image-wide adaptive sampling is done as well. Once everything is stable enough, I'll post it here.
Perfect :D @LukasStockner any comment about this? http://www.luxrender.net/forum/viewtopic.php?f=8&t=10947
Author
Member

@MaciejJutrzenka OK, now that's a remarkable coincidence o.O
In fact, I've been working with VCM for over a year now for another project of mine (lightpath reuse in animated scenes, saves up to 50% of the calculations) and only one month ago I had basically the same idea (merging with vertices in the volume my adjusting the geometric couplign term). It's really awesome to see that this indeed works and has already been implemented :D
Regarding Cycles: Adding it would probably be possible, but it's not comparable to the Metro patch. You would first have to add Bidir, then add Photon-Tracing support like in PPM, then add the VCM merging code (which is particularily hard to implement correctly due to the various weighting terms) and finally add volume photon storage and beam queries. This is a HUGE amount of work and, considering that the whloe code structure of Cycles is made for Path Tracing, it would be probably hacky and messy.

By the way, the current patch takes so long because of the VCM project described above, because in 1 week there is a competition for which I still want to code a GPU implementation of my animation-VCM code, so currently it requires most of my time. Sorry for that, development will be faster afterwards.

@MaciejJutrzenka OK, now that's a remarkable coincidence o.O In fact, I've been working with VCM for over a year now for another project of mine (lightpath reuse in animated scenes, saves up to 50% of the calculations) and only one month ago I had basically the same idea (merging with vertices in the volume my adjusting the geometric couplign term). It's really awesome to see that this indeed works and has already been implemented :D Regarding Cycles: Adding it would probably be possible, but it's not comparable to the Metro patch. You would first have to add Bidir, then add Photon-Tracing support like in PPM, then add the VCM merging code (which is particularily hard to implement correctly due to the various weighting terms) and finally add volume photon storage and beam queries. This is a HUGE amount of work and, considering that the whloe code structure of Cycles is made for Path Tracing, it would be probably hacky and messy. By the way, the current patch takes so long because of the VCM project described above, because in 1 week there is a competition for which I still want to code a GPU implementation of my animation-VCM code, so currently it requires most of my time. Sorry for that, development will be faster afterwards.

Interesting piece about adaptive sampling that popped up on BA today. Dade claims it's incredibly easy to implement over an existing tiled renderer.

http://www.luxrender.net/forum/viewtopic.php?f=8&t=10955&sid=8774701c5a8832e330716af5a8f74863

Interesting piece about adaptive sampling that popped up on BA today. Dade claims it's incredibly easy to implement over an existing tiled renderer. http://www.luxrender.net/forum/viewtopic.php?f=8&t=10955&sid=8774701c5a8832e330716af5a8f74863
Author
Member

@MatthewHeimlich Wow, that looks really useful. I haven't read the paper yet, but it definitely sounds useful and usable for the patch. Really incredible work by the LuxRender guys there, always impementing the latest stuff.

@MatthewHeimlich Wow, that looks really useful. I haven't read the paper yet, but it definitely sounds useful and usable for the patch. Really incredible work by the LuxRender guys there, always impementing the latest stuff.

Added subscriber: @Lapineige

Added subscriber: @Lapineige

A video of the new adaptive sampling at work. I'd need to see some more complex scenes to judge definitively, but so far this is the first adaptive sampling method I've ever seen that appears to "just work".

https://www.youtube.com/watch?feature=player_embedded&v=P_QmdpnKTW4

A video of the new adaptive sampling at work. I'd need to see some more complex scenes to judge definitively, but so far this is the first adaptive sampling method I've ever seen that appears to "just work". https://www.youtube.com/watch?feature=player_embedded&v=P_QmdpnKTW4

@MatthewHeimlich
Excellent video: now I get it... how it works.
The threshold based on noise rather than uniform for all tiles.

Please i ask again: Does this exist in compiled "ready-to-use" form or only as a patch requiring me to compile - something I don't know how to do. I use Blender 2.69 and 2.70 under Windows 7/32

@MatthewHeimlich Excellent video: now I get it... how it works. The threshold based on noise rather than uniform for all tiles. Please i ask again: Does this exist in compiled "ready-to-use" form or only as a patch requiring me to compile - something I don't know how to do. I use Blender 2.69 and 2.70 under Windows 7/32

@xs If you do not know how to patch properly, i probably would suggest just sticking to the main blender... this is a work in progress and is not ready for mainstream use.

@xs If you do not know how to patch properly, i probably would suggest just sticking to the main blender... this is a work in progress and is not ready for mainstream use.

To be clear, what's shown in that video isn't available in Blender at all. It's a demonstration of the new method available in LuxRender that I linked to above. Should be relatively painless to integrate into Cycles, though.

To be clear, what's shown in that video isn't available in Blender at all. It's a demonstration of the new method available in LuxRender that I linked to above. Should be relatively painless to integrate into Cycles, though.
And it's what I've been waiting for a looong time... :) http://blenderartists.org/forum/showthread.php?236453-Measuring-Noise-in-Cycles-Renders&p=2001913&viewfull=1#post2001913 http://blenderartists.org/forum/showthread.php?255683-Cycles-status-%28as-of-May-14th%29&highlight=cycles+status http://blenderartists.org/forum/showthread.php?236453-Measuring-Noise-in-Cycles-Renders&p=2370199&viewfull=1#post2370199 http://blenderartists.org/forum/showthread.php?216113-Brecht-s-easter-egg-surprise-Modernizing-shading-and-rendering&p=2395532&viewfull=1#post2395532 I'm still convinced this will be the BIGGEST performance optimization

BTW, Yafaray has something similar since ages. Wondering what is the noise-aware implementation there...

BTW, Yafaray has something similar since ages. Wondering what is the noise-aware implementation there...

Isn't Yafaray's just a simple contrast-based sampler? Just hunts edges, not high-noise areas?

Isn't Yafaray's just a simple contrast-based sampler? Just hunts edges, not high-noise areas?
Author
Member

Developer-question here: How do you think this should be implemented? My current plan is to treat the Samples setting as a maximum value when using the stopping criterion and add a maximum tolerated error setting. As soon as one of them is reached, the tile stops. This, however, is incompatible with progressive rendering and Metropolis, but I think we can live with that.
In fact, most of the code required for this is already implemented for the current adaptive sampling (most importantly, a per-pixel sample-number buffer), so the stopping criterion should be running quite soon. In-Tile adaptive sampling can be kept the way it is now, only using the new variance estimate (contrast to even-sample buffer).
I'm really looking forward to see how good it works in Cycles, for SLG the speedup seems quite impressive.
BTW: The viewport bug is fixed and another one that broke DoF and Motion Blur is fixed, too. Importance equalisation works as well, once the stopping criterion works, I'll publish a new patch version.

Developer-question here: How do you think this should be implemented? My current plan is to treat the Samples setting as a maximum value when using the stopping criterion and add a maximum tolerated error setting. As soon as one of them is reached, the tile stops. This, however, is incompatible with progressive rendering and Metropolis, but I think we can live with that. In fact, most of the code required for this is already implemented for the current adaptive sampling (most importantly, a per-pixel sample-number buffer), so the stopping criterion should be running quite soon. In-Tile adaptive sampling can be kept the way it is now, only using the new variance estimate (contrast to even-sample buffer). I'm really looking forward to see how good it works in Cycles, for SLG the speedup seems quite impressive. BTW: The viewport bug is fixed and another one that broke DoF and Motion Blur is fixed, too. Importance equalisation works as well, once the stopping criterion works, I'll publish a new patch version.

Added subscriber: @Lukas-132

Added subscriber: @Lukas-132

@JasonClarke
I actually don't know ;)

@Lukas-132
cool, can't wait...
what do you mean when you say that the "max_sample_limit + maximum_tolerated_error" are incompatible with progressive rendering?

@JasonClarke I actually don't know ;) @Lukas-132 cool, can't wait... what do you mean when you say that the "max_sample_limit + maximum_tolerated_error" are incompatible with progressive rendering?

Lukas: Sounds like the best way to do things. Halt render if either criteria is reached. For certain scenes (I have one with a half lit face where almost a full half of the samples are wasted on the side that is almost completely black that comes to mind) this would be an ENORMOUS time saver.

Lukas: Sounds like the best way to do things. Halt render if either criteria is reached. For certain scenes (I have one with a half lit face where almost a full half of the samples are wasted on the side that is almost completely black that comes to mind) this would be an ENORMOUS time saver.

I dare.
There is also one more opportunity: time halt condition. To make this possible the engine should know how long each tile takes to render, so a first pass (with a minimum sample number to be decided) should be done for all the tiles. This would give two benefits: a proper render-time estimation, and a first rough look at the whole render output.

I dare. There is also one more opportunity: time halt condition. To make this possible the engine should know how long each tile takes to render, so a first pass (with a minimum sample number to be decided) should be done for all the tiles. This would give two benefits: a proper render-time estimation, and a first rough look at the whole render output.

http://ompf2.com/viewtopic.php?f=3&t=1933

Here is some more discussion from Dade and others on halt conditions/adaptive sampling. Lots of nice papers linked, lot's of good discussion of pros and cons from people who have implemented some of them.

lsscpp, I completely agree. It would be very nice to have a rendering algorithm that ran a few samples at lower sampling rates across the entire image to show a very rough preview of what's coming. Others do things this way and it makes a huge difference in catching issues early on in the render.

http://ompf2.com/viewtopic.php?f=3&t=1933 Here is some more discussion from Dade and others on halt conditions/adaptive sampling. Lots of nice papers linked, lot's of good discussion of pros and cons from people who have implemented some of them. lsscpp, I completely agree. It would be very nice to have a rendering algorithm that ran a few samples at lower sampling rates across the entire image to show a very rough preview of what's coming. Others do things this way and it makes a huge difference in catching issues early on in the render.
Author
Member

The basic implementation works now, but there is a problem: correlation. Taking only even samples of a 1000-sample render gives a slightly different picture than using all samples of a 500-sample render, at least for the BMW scene. With CMJ-samples, the problem is gone. Apart from that, it seems to work quite OK, in 1-3 days the patch should be ready for a new upload.
By the way, another open question: Should the stopping use average error or maximum error inside of a tile? Currently, the code works like "average below threshold and maximum below 2*threshold".

The basic implementation works now, but there is a problem: correlation. Taking only even samples of a 1000-sample render gives a slightly different picture than using all samples of a 500-sample render, at least for the BMW scene. With CMJ-samples, the problem is gone. Apart from that, it seems to work quite OK, in 1-3 days the patch should be ready for a new upload. By the way, another open question: Should the stopping use average error or maximum error inside of a tile? Currently, the code works like "average below threshold and maximum below 2*threshold".

IIRC LuxCore implementation uses max error inside each tile, and it sounds reasonable too. This way you always know how much the error might be.
Can't wait to test this out. Can you (ora anyone) please upload the build once it is ready?

IIRC LuxCore implementation uses max error inside each tile, and it sounds reasonable too. This way you always know how much the error *might* be. Can't wait to test this out. Can you (ora anyone) please upload the build once it is ready?

I'm not sure i'm following entirely, @LukasStockner are you keep refining Metropolis or trying the Lux new adaptive one?

I'm not sure i'm following entirely, @LukasStockner are you keep refining Metropolis or trying the Lux new adaptive one?
Author
Member

Basically both, but I think Metro works quite fine already. Once the Importance Equalisation is stable and the adaptive stuff is done, in my opinion the patch is ready for extensive testing and code cleanup.

Basically both, but I think Metro works quite fine already. Once the Importance Equalisation is stable and the adaptive stuff is done, in my opinion the patch is ready for extensive testing and code cleanup.
Author
Member

First of all, sorry for the permanent delays. I underestimated the amount of work this would take and a ton of other stuff needed attention.
However, by now I solved the Sobol problem by adding a RNG dimension that decides whether the sample goes to the "even"-buffer instead of really taking only even samples, and adaptive sampling inside the tile now also is done according to the error in the pixel so it's consistent. All in all, it starts to work really good now, after I removed some debugging code and rebased to the current trunk, metropolis_17 should be ready quite soon.

Some images here to show you how good it works, all of them are pure PT and rendered for 2:30 min:

No adaptive stuff at all:
BMW.png

Adaptive stopping, no adaptive sampling:
BMW_Stop.png

Both adaptive stopping and sampling:
BMW_Stop_Ada.png

As you can see, the noise is distributed more uniformly in the images with adaptive stopping and especially the headlights and their reflection look better. The doors, on the other hand, are a bit more noisy. This is because the adaptive stopping doesn't work any faster, it just distributes samples differently. If the headlights are to receive more samples, some areas have to receive less.

This is the same scene (obviously), this time 8:30 min:

No adaptive stuff:
BMW_HQ.png

Both adaptive stopping and sampling:
BMW_HQ_Stop_Ada.png

First of all, sorry for the permanent delays. I underestimated the amount of work this would take and a ton of other stuff needed attention. However, by now I solved the Sobol problem by adding a RNG dimension that decides whether the sample goes to the "even"-buffer instead of really taking only even samples, and adaptive sampling inside the tile now also is done according to the error in the pixel so it's consistent. All in all, it starts to work really good now, after I removed some debugging code and rebased to the current trunk, metropolis_17 should be ready quite soon. Some images here to show you how good it works, all of them are pure PT and rendered for 2:30 min: No adaptive stuff at all: ![BMW.png](https://archive.blender.org/developer/F85006/BMW.png) Adaptive stopping, no adaptive sampling: ![BMW_Stop.png](https://archive.blender.org/developer/F85008/BMW_Stop.png) Both adaptive stopping and sampling: ![BMW_Stop_Ada.png](https://archive.blender.org/developer/F85010/BMW_Stop_Ada.png) As you can see, the noise is distributed more uniformly in the images with adaptive stopping and especially the headlights and their reflection look better. The doors, on the other hand, are a bit more noisy. This is because the adaptive stopping doesn't work any faster, it just distributes samples differently. If the headlights are to receive more samples, some areas have to receive less. This is the same scene (obviously), this time 8:30 min: No adaptive stuff: ![BMW_HQ.png](https://archive.blender.org/developer/F85012/BMW_HQ.png) Both adaptive stopping and sampling: ![BMW_HQ_Stop_Ada.png](https://archive.blender.org/developer/F85014/BMW_HQ_Stop_Ada.png)

Added subscriber: @brecht

Added subscriber: @brecht

Great work!

For adaptive sampling, does it measure the noise in linear or in display space? If the former, I wonder if the result could be improved by doing it in display space, as linear space would underestimate the noise in dark areas.

Great work! For adaptive sampling, does it measure the noise in linear or in display space? If the former, I wonder if the result could be improved by doing it in display space, as linear space would underestimate the noise in dark areas.
Author
Member

Thanks!
Currently it works in linear space (I'm assuming the RenderBuffers are in linear space), but the difference between regular and even pass is weighted by a Threshold-Versus-Intensity function.
Still, additionally weighting by the tonemapping function is probably better (as far as I remember, the paper that used the even-buffer trick even did this). If I remember correctly, the linear-to-sRGB is mainly a pow() operation, the speed impact shouldn't be that high (also, the noise estimation is currently only done every 25 samples).

Thanks! Currently it works in linear space (I'm assuming the RenderBuffers are in linear space), but the difference between regular and even pass is weighted by a Threshold-Versus-Intensity function. Still, additionally weighting by the tonemapping function is probably better (as far as I remember, the paper that used the even-buffer trick even did this). If I remember correctly, the linear-to-sRGB is mainly a pow() operation, the speed impact shouldn't be that high (also, the noise estimation is currently only done every 25 samples).

I'd be interested to see this on a scene without lots of glossy noise, something where shadows or indirect noise take up a good chunk of the render time with master Cycles. Something like a half lit face should benefit quite a bit from adaptive sampling like this.

I'd be interested to see this on a scene without lots of glossy noise, something where shadows or indirect noise take up a good chunk of the render time with master Cycles. Something like a half lit face should benefit quite a bit from adaptive sampling like this.

Juicy results!
Lukas, what is actually the difference between "adaptive stopping" and "adaptive stopping+sampling" ?

Juicy results! Lukas, what is actually the difference between "adaptive stopping" and "adaptive stopping+sampling" ?

as m9105826 says. test could be done on a more difficult scene as this: http://www.blenderartists.org/forum/showthread.php?331149-The-new-Cyles-GPU-2-70-Benchmark

as m9105826 says. test could be done on a more difficult scene as this: http://www.blenderartists.org/forum/showthread.php?331149-The-new-Cyles-GPU-2-70-Benchmark

Added subscriber: @nudelZ

Added subscriber: @nudelZ
Author
Member

New patch version, the adaptive stuff got a massive update, importance equalisation now works again and CUDA/OpenCL should build again (without Metro support). Adaptive stopping works on CPU and GPU, although it might slow down GPU a bit since every test has to fetch the buffers from device memory. I didn't test it, but in ~1 week I'll get access to a CUDA 2.0-capable system so I can fix the CUDA support. Adaptive sampling will get GPU support, but it's not finished yet.
For adaptive stopping, set the Stopping Threshold value under the Performance tab to a value > 0. The adaptive stopping works parallel to the classic samples-value stopping, so if a tile hits the specified sample number, it's stopped regardless of whether its error is below threshold or not. In my experience, something around 1 gives a rough-preview-quality result while values <= 0.25 appear noise-free. The test whether the tile is done starts after some warmup samples have passed and is performed in a specifix interval, the settings for these are under the threshold. The checkbox will activate adaptive sampling inside the tiles. By the way, adaptive stopping also works for Metro.
In Metro mode, the adaptive sampling checkbox activates Importance equalisation. This setting basically causes the Metro sampler to distribute the samples more uniformly instead of sampling according to brightness (technically, it samples acccording to path brightness divided by average pixel brightness, therefor still favoring high-energy paths). Sometimes this helps really a lot, in other scenes it's pretty useless.

@lsscpp
Adaptive stopping looks at the tile and decides whether it's done or not, while adaptive sampling distributes the samples inside the tile to areas of high error.
This is useful, for example, if the tile is at the boundary of Background/Object, since without adaptive sampling, all of the tile will be sampled until the error is low enough. With adaptive sampling, however, the Object will receive more samples than the background.
Basically, adaptive sampling is what the patch already did, while adaptive stopping is the new feature.
By the way, in the 2.70 benchmark, there is nearly no improvement since the noise is already distributed quite evenly.

metropolis_17.diff

New patch version, the adaptive stuff got a massive update, importance equalisation now works again and CUDA/OpenCL should build again (without Metro support). Adaptive stopping works on CPU and GPU, although it might slow down GPU a bit since every test has to fetch the buffers from device memory. I didn't test it, but in ~1 week I'll get access to a CUDA 2.0-capable system so I can fix the CUDA support. Adaptive sampling will get GPU support, but it's not finished yet. For adaptive stopping, set the Stopping Threshold value under the Performance tab to a value > 0. The adaptive stopping works parallel to the classic samples-value stopping, so if a tile hits the specified sample number, it's stopped regardless of whether its error is below threshold or not. In my experience, something around 1 gives a rough-preview-quality result while values <= 0.25 appear noise-free. The test whether the tile is done starts after some warmup samples have passed and is performed in a specifix interval, the settings for these are under the threshold. The checkbox will activate adaptive sampling inside the tiles. By the way, adaptive stopping also works for Metro. In Metro mode, the adaptive sampling checkbox activates Importance equalisation. This setting basically causes the Metro sampler to distribute the samples more uniformly instead of sampling according to brightness (technically, it samples acccording to path brightness divided by average pixel brightness, therefor still favoring high-energy paths). Sometimes this helps really a lot, in other scenes it's pretty useless. @lsscpp Adaptive stopping looks at the tile and decides whether it's done or not, while adaptive sampling distributes the samples inside the tile to areas of high error. This is useful, for example, if the tile is at the boundary of Background/Object, since without adaptive sampling, all of the tile will be sampled until the error is low enough. With adaptive sampling, however, the Object will receive more samples than the background. Basically, adaptive sampling is what the patch already did, while adaptive stopping is the new feature. By the way, in the 2.70 benchmark, there is nearly no improvement since the noise is already distributed quite evenly. [metropolis_17.diff](https://archive.blender.org/developer/F85547/metropolis_17.diff)
Member

Added subscriber: @plasmasolutions

Added subscriber: @plasmasolutions
Member

The patch doesn't seem to apply for me with latest master..

The patch doesn't seem to apply for me with latest master..

Hi Lucas, patch not apply on 1e6fa59 from today and get build error:

Hunk #1 FAILED at 64.
1 out of 6 hunks FAILED -- saving rejects to file intern/cycles/kernel/kernel_types.h.rej

With patch -p1 < metropolis_17.diff

[ 83%] Built target cycles_device
[ 83%] Building CXX object intern/cycles/kernel/CMakeFiles/cycles_kernel.dir/kernel.cpp.o
In file included from /daten/blender-git/blender/intern/cycles/kernel/geom/geom.h:40:0,
                 from /daten/blender-git/blender/intern/cycles/kernel/kernel_path.h:27,
                 from /daten/blender-git/blender/intern/cycles/kernel/kernel.cpp:25:
/daten/blender-git/blender/intern/cycles/kernel/geom/geom_curve.h: In function ‘bool ccl::bvh_curve_intersect(ccl::KernelGlobals*, ccl::Intersection*, ccl::float3, ccl::float3, ccl::uint, int, int, float, int, ccl::uint*, float, float)’:
/daten/blender-git/blender/intern/cycles/kernel/geom/geom_curve.h:785:8: warning: variable ‘backface’ set but not used [-Wunused-but-set-variable]
   bool backface = false;
        ^
/daten/blender-git/blender/intern/cycles/kernel/kernel.cpp: In function ‘float ccl::kernel_cpu_metropolis_first_pass(ccl::KernelGlobals*, ccl::uint*, int, int, ccl::PassData*, int, int, int)’:
/daten/blender-git/blender/intern/cycles/kernel/kernel.cpp:114:85: error: ‘kernel_metropolis_first_pass’ was not declared in this scope
  return kernel_metropolis_first_pass(kg, rng_state, x, y, pd, offset, stride, sample);
                                                                                     ^
/daten/blender-git/blender/intern/cycles/kernel/kernel.cpp: In function ‘ccl::float4 ccl::kernel_cpu_metropolis_path_trace(ccl::KernelGlobals*, float*, float, float, ccl::PassData*)’:
/daten/blender-git/blender/intern/cycles/kernel/kernel.cpp:119:65: error: ‘kernel_metropolis_path_trace’ was not declared in this scope
  return kernel_metropolis_path_trace(kg, randomSamples, x, y, pd);
                                                                 ^
/daten/blender-git/blender/intern/cycles/kernel/kernel.cpp: In function ‘float ccl::kernel_cpu_metropolis_first_pass(ccl::KernelGlobals*, ccl::uint*, int, int, ccl::PassData*, int, int, int)’:
/daten/blender-git/blender/intern/cycles/kernel/kernel.cpp:115:1: warning: control reaches end of non-void function [-Wreturn-type]
 }
 ^
/daten/blender-git/blender/intern/cycles/kernel/kernel.cpp: In function ‘ccl::float4 ccl::kernel_cpu_metropolis_path_trace(ccl::KernelGlobals*, float*, float, float, ccl::PassData*)’:
/daten/blender-git/blender/intern/cycles/kernel/kernel.cpp:120:1: warning: control reaches end of non-void function [-Wreturn-type]
 }
 ^
make[2]: *** [intern/cycles/kernel/CMakeFiles/cycles_kernel.dir/kernel.cpp.o] Fehler 1
make[1]: *** [intern/cycles/kernel/CMakeFiles/cycles_kernel.dir/all] Fehler 2
make: *** [all] Fehler 2

Thanks, mib

Hi Lucas, patch not apply on 1e6fa59 from today and get build error: Hunk #1 FAILED at 64. 1 out of 6 hunks FAILED -- saving rejects to file intern/cycles/kernel/kernel_types.h.rej With patch -p1 < metropolis_17.diff ``` [ 83%] Built target cycles_device [ 83%] Building CXX object intern/cycles/kernel/CMakeFiles/cycles_kernel.dir/kernel.cpp.o In file included from /daten/blender-git/blender/intern/cycles/kernel/geom/geom.h:40:0, from /daten/blender-git/blender/intern/cycles/kernel/kernel_path.h:27, from /daten/blender-git/blender/intern/cycles/kernel/kernel.cpp:25: /daten/blender-git/blender/intern/cycles/kernel/geom/geom_curve.h: In function ‘bool ccl::bvh_curve_intersect(ccl::KernelGlobals*, ccl::Intersection*, ccl::float3, ccl::float3, ccl::uint, int, int, float, int, ccl::uint*, float, float)’: /daten/blender-git/blender/intern/cycles/kernel/geom/geom_curve.h:785:8: warning: variable ‘backface’ set but not used [-Wunused-but-set-variable] bool backface = false; ^ /daten/blender-git/blender/intern/cycles/kernel/kernel.cpp: In function ‘float ccl::kernel_cpu_metropolis_first_pass(ccl::KernelGlobals*, ccl::uint*, int, int, ccl::PassData*, int, int, int)’: /daten/blender-git/blender/intern/cycles/kernel/kernel.cpp:114:85: error: ‘kernel_metropolis_first_pass’ was not declared in this scope return kernel_metropolis_first_pass(kg, rng_state, x, y, pd, offset, stride, sample); ^ /daten/blender-git/blender/intern/cycles/kernel/kernel.cpp: In function ‘ccl::float4 ccl::kernel_cpu_metropolis_path_trace(ccl::KernelGlobals*, float*, float, float, ccl::PassData*)’: /daten/blender-git/blender/intern/cycles/kernel/kernel.cpp:119:65: error: ‘kernel_metropolis_path_trace’ was not declared in this scope return kernel_metropolis_path_trace(kg, randomSamples, x, y, pd); ^ /daten/blender-git/blender/intern/cycles/kernel/kernel.cpp: In function ‘float ccl::kernel_cpu_metropolis_first_pass(ccl::KernelGlobals*, ccl::uint*, int, int, ccl::PassData*, int, int, int)’: /daten/blender-git/blender/intern/cycles/kernel/kernel.cpp:115:1: warning: control reaches end of non-void function [-Wreturn-type] } ^ /daten/blender-git/blender/intern/cycles/kernel/kernel.cpp: In function ‘ccl::float4 ccl::kernel_cpu_metropolis_path_trace(ccl::KernelGlobals*, float*, float, float, ccl::PassData*)’: /daten/blender-git/blender/intern/cycles/kernel/kernel.cpp:120:1: warning: control reaches end of non-void function [-Wreturn-type] } ^ make[2]: *** [intern/cycles/kernel/CMakeFiles/cycles_kernel.dir/kernel.cpp.o] Fehler 1 make[1]: *** [intern/cycles/kernel/CMakeFiles/cycles_kernel.dir/all] Fehler 2 make: *** [all] Fehler 2 ``` Thanks, mib

Added subscriber: @00Ghz

Added subscriber: @00Ghz

For Metro Octane Render Team made some custom implementation. Most likely it's patented but maybe we can find a similar hybrid solution.

What do you think?

For Metro Octane Render Team made some custom implementation. Most likely it's patented but maybe we can find a similar hybrid solution. What do you think?

They use ERPT with Population Monte Carlo I think. It's not clear to me that this would be better than MLT combined with the adaptive sampling.

I suggest to look at the "Multiplexed Metropolis Light Transport" paper when it comes out, I saw some impressive results from that (sorry, I have no link) and it should fit well here.

They use ERPT with Population Monte Carlo I think. It's not clear to me that this would be better than MLT combined with the adaptive sampling. I suggest to look at the "Multiplexed Metropolis Light Transport" paper when it comes out, I saw some impressive results from that (sorry, I have no link) and it should fit well here.

Well I and many people out there want GPU support, so whatever works for that will be good.

Also Octane is getting out of core textures(no more GPU memory limit). Any chance to get something like that for cycles

Well I and many people out there want GPU support, so whatever works for that will be good. Also Octane is getting out of core textures(no more GPU memory limit). Any chance to get something like that for cycles

Added subscriber: @NahuelBelich

Added subscriber: @NahuelBelich
Author
Member

OK, here is a corrected version. One line in kernel_types.h was different between trunk and patch.
Regarding the Octane method: Running a large number of concurrent samplers on GPUs would be possible, it's quite similar to ERPT and LuxRender already uses it. However, I'm not too sure whether this is any better than PT since fewer, longer MCMC chains should explore the sample space more evenly.
The Multiplexed Metropolis paper sounds definitely interesting, I'll look at it once it's released.

metropolis_17_fixed.diff

OK, here is a corrected version. One line in kernel_types.h was different between trunk and patch. Regarding the Octane method: Running a large number of concurrent samplers on GPUs would be possible, it's quite similar to ERPT and LuxRender already uses it. However, I'm not too sure whether this is any better than PT since fewer, longer MCMC chains should explore the sample space more evenly. The Multiplexed Metropolis paper sounds definitely interesting, I'll look at it once it's released. [metropolis_17_fixed.diff](https://archive.blender.org/developer/F85824/metropolis_17_fixed.diff)

http://cs.au.dk/~toshiya/

here is the guy making it.

And apparently there is a paper on some SSS method as well.

http://cs.au.dk/~toshiya/ here is the guy making it. And apparently there is a paper on some SSS method as well.

It's a dipole based method, useless for us. This isn't the place to discuss that anyway.

It's a dipole based method, useless for us. This isn't the place to discuss that anyway.

No luck compiling against master here. Lots of errors thrown from util_color.h about ambiguous calls to overloaded functions. VC2012 on Win7 64-bit.

No luck compiling against master here. Lots of errors thrown from util_color.h about ambiguous calls to overloaded functions. VC2012 on Win7 64-bit.

Added subscriber: @akishin

Added subscriber: @akishin
Author
Member

@MatthewHeimlich I've been trying to setup a VC2012 environment to reproduce your errors (also W7 64), but can't get it to work properly. The instructions in the Wiki still use the SVN trunk, it works fine with this one. However, once I use the Git trunk, CMake fails to find the libs. My folder layout is "E:/Blender/blender" for the trunk, "E:/Blender/build" for CMake and "E:/Blender/libs/win64_vc11" for the precompiled libs (still from SVN). CMake won't find Boost, OpenJPEG and Python 3.4 (the precompiled libs still have Python 3.3) and stops afterwards. Could you tell me where you got your libs from or did you compile them yourself?
Regarding the error: I suppose it complains about linear_rgb_to_gray? If yes, a workaround would be to rename the new float4 version to linear_rgb_to_gray4 and changing the occurences in device/device_cpu.cpp (after the Metropolis kernel call) and in render/buffers.cpp (in build_importance_map()). Of course, this still needs to be fixed properly, but GCC doesn't seem to care about it at all...

@MatthewHeimlich I've been trying to setup a VC2012 environment to reproduce your errors (also W7 64), but can't get it to work properly. The instructions in the Wiki still use the SVN trunk, it works fine with this one. However, once I use the Git trunk, CMake fails to find the libs. My folder layout is "E:/Blender/blender" for the trunk, "E:/Blender/build" for CMake and "E:/Blender/libs/win64_vc11" for the precompiled libs (still from SVN). CMake won't find Boost, OpenJPEG and Python 3.4 (the precompiled libs still have Python 3.3) and stops afterwards. Could you tell me where you got your libs from or did you compile them yourself? Regarding the error: I suppose it complains about linear_rgb_to_gray? If yes, a workaround would be to rename the new float4 version to linear_rgb_to_gray4 and changing the occurences in device/device_cpu.cpp (after the Metropolis kernel call) and in render/buffers.cpp (in build_importance_map()). Of course, this still needs to be fixed properly, but GCC doesn't seem to care about it at all...

VC2012 is not supported anymore, we should remove the leftovers. Use VC2008 or VC2013.

VC2012 is not supported anymore, we should remove the leftovers. Use VC2008 or VC2013.

Such confusing naming for VS/VC. I am indeed using 2013.

Such confusing naming for VS/VC. I am indeed using 2013.
Author
Member

@ThomasDinges Thanks, works great (apart from an error complaining that lib\win64_vc12\release\python34_numpy_1.8.tar.gz is missing, but it doesn't seem to matter)
@MatthewHeimlich OK, I fixed the error. It's not the linear_rgb_to_gray(), but the logarithm in "return exp(log_i * log(10))", in the linear_gray_to_tvi() function. Changing "10" to "10.0f" fixes it.

@ThomasDinges Thanks, works great (apart from an error complaining that lib\win64_vc12\release\python34_numpy_1.8.tar.gz is missing, but it doesn't seem to matter) @MatthewHeimlich OK, I fixed the error. It's not the linear_rgb_to_gray(), but the logarithm in "return exp(log_i * log(10))", in the linear_gray_to_tvi() function. Changing "10" to "10.0f" fixes it.
Author
Member

Oh, and another thing, "struct RenderTile;" in util_metropolis.h needs to be changed to "class RenderTile;"

Another, probably unrelated thing: In a test scene, when I enable Cycles and go to the Material tab, I get an "Assertion failed" in the python34_d.dll ( ../Python/ceval.c, different lines, but always "!PyErr_Occured()" ), even on trunk. Is this normal?

Oh, and another thing, "struct RenderTile;" in util_metropolis.h needs to be changed to "class RenderTile;" Another, probably unrelated thing: In a test scene, when I enable Cycles and go to the Material tab, I get an "Assertion failed" in the python34_d.dll ( ../Python/ceval.c, different lines, but always "!PyErr_Occured()" ), even on trunk. Is this normal?
Author
Member

Since I just setup my Windows toolchain, I thought I'll just post my build here. It's based on today's trunk, built with VC2013 for Windows x64 (x32 will follow soon). It lacks some features (mainly GPU and OSL), but the standard stuff should work.

I hope I included every needed file, if something is missing, please tell me and I'll post it. By the way, the patch doesn't affect performance in classical PT without the new options in any way, in fact, on my machine this build renders faster than the 2.70a release.

http://www.mediafire.com/download/1lkr2mqdd521atu/Blender_270_Metro_VC2013_x64.zip

Since I just setup my Windows toolchain, I thought I'll just post my build here. It's based on today's trunk, built with VC2013 for Windows x64 (x32 will follow soon). It lacks some features (mainly GPU and OSL), but the standard stuff should work. I hope I included every needed file, if something is missing, please tell me and I'll post it. By the way, the patch doesn't affect performance in classical PT without the new options in any way, in fact, on my machine this build renders faster than the 2.70a release. http://www.mediafire.com/download/1lkr2mqdd521atu/Blender_270_Metro_VC2013_x64.zip

That's because 2.70(a) has been built with vc2008, vc2013 makes Cycles 15-20% faster for free.

That's because 2.70(a) has been built with vc2008, vc2013 makes Cycles 15-20% faster for free.

WOW. I tested the scene I had spoken about previously. What a savings! Image quality difference is negligible, but a bit over 75% faster render times. Excellent work!

No.png

Yes.png

WOW. I tested the scene I had spoken about previously. What a savings! Image quality difference is negligible, but a bit over 75% faster render times. Excellent work! ![No.png](https://archive.blender.org/developer/F86752/No.png) ![Yes.png](https://archive.blender.org/developer/F86754/Yes.png)

That was with only 6 of my 8 cores, btw. Tolerated error set to 3, update rate at 5, warmup at 10. Let me know if you think there are even better settings.

That was with only 6 of my 8 cores, btw. Tolerated error set to 3, update rate at 5, warmup at 10. Let me know if you think there are even better settings.

rendering freezed, when i use small render samples and corelatted multijitter

Settings.jpg

BMW1M_adptiv_sampl.blend

rendering freezed, when i use small render samples and corelatted multijitter ![Settings.jpg](https://archive.blender.org/developer/F86779/Settings.jpg) [BMW1M_adptiv_sampl.blend](https://archive.blender.org/developer/F86781/BMW1M_adptiv_sampl.blend)
Author
Member

@MatthewHeimlich Well, that's quite an improvement. Scenes like this are of course best-case for adaptive sampling, but especially big background areas are quite common.
Regarding the settings: The tolerated error sounds quite high to me, but indeed the rendered image looks just fine, so I guess it's okay.
The other two settings seem reasonable, basically they depend on how expansive tracing a single sample is (when the tile is rendered at 100 samples/sec, checking for convergence every 5 sample might give a significant slowdown, while for something like 5 samples/sec it's just fine).
This gives me another idea: Maybe a time-based update rate would be better than a sample-based one?

@lopataasdf Yes, this scene freezes for me too. Actually (or at least it seems like this to me), it's not freezing completely, but somehow blocking the GUI while rendering, since after ~1min it shortly got responsive for me again. I'll try to reproduce this on Linux since I have no idea of MSVC debugging...
Could someone try to reproduce this with another Windows build to see whether it's related to this specific build, the compiler, the OS or the whole patch?

@MatthewHeimlich Well, that's quite an improvement. Scenes like this are of course best-case for adaptive sampling, but especially big background areas are quite common. Regarding the settings: The tolerated error sounds quite high to me, but indeed the rendered image looks just fine, so I guess it's okay. The other two settings seem reasonable, basically they depend on how expansive tracing a single sample is (when the tile is rendered at 100 samples/sec, checking for convergence every 5 sample might give a significant slowdown, while for something like 5 samples/sec it's just fine). This gives me another idea: Maybe a time-based update rate would be better than a sample-based one? @lopataasdf Yes, this scene freezes for me too. Actually (or at least it seems like this to me), it's not freezing completely, but somehow blocking the GUI while rendering, since after ~1min it shortly got responsive for me again. I'll try to reproduce this on Linux since I have no idea of MSVC debugging... Could someone try to reproduce this with another Windows build to see whether it's related to this specific build, the compiler, the OS or the whole patch?

Time based could be a problem when the scene is rendered on another computer with much different performance (say, an old workstation that retired to the render farm).

What about setting warmup/update as a percentage of the total number of AA samples? Like, every 10% of completion, do the update?

Time based could be a problem when the scene is rendered on another computer with much different performance (say, an old workstation that retired to the render farm). What about setting warmup/update as a percentage of the total number of AA samples? Like, every 10% of completion, do the update?
craigar commented 2014-05-04 00:49:31 +02:00 (Migrated from localhost:3001)

Added subscriber: @craigar

Added subscriber: @craigar
craigar commented 2014-05-04 00:49:31 +02:00 (Migrated from localhost:3001)

Your build
http://www.mediafire.com/download/1lkr2mqdd521atu/Blender_270_Metro_VC2013_x64.zip
won't load on my Win 7 PC, it crashes before ever loading.

I know you talked about "vc2013 makes Cycles 15-20% faster for free. Could it be because of the "MSVCR120.dll" included should be "MSVCR130.dll" - or is this unrelated to your usage of "vc2013"?

Problem signature:

Problem Event Name:	APPCRASH
Application Name:	blender.exe
Application Version:	2.7.0.0
Application Timestamp:	5363f045
Fault Module Name:	MSVCR120.dll
Fault Module Version:	12.0.21005.1
Fault Module Timestamp:	524f83ff

My system only has a RADEON 6450 GPU, must this build run on a system with a CUDA GPU?
My system is a stock Dell XPS8300 with Windows 7 Home premium 64bit, i7 CPU, 12 GB ram

Thank You

Your build http://www.mediafire.com/download/1lkr2mqdd521atu/Blender_270_Metro_VC2013_x64.zip won't load on my Win 7 PC, it crashes before ever loading. I know you talked about "vc2013 makes Cycles 15-20% faster for free. Could it be because of the "MSVCR120.dll" included should be "MSVCR130.dll" - or is this unrelated to your usage of "vc2013"? Problem signature: ``` Problem Event Name: APPCRASH Application Name: blender.exe Application Version: 2.7.0.0 Application Timestamp: 5363f045 Fault Module Name: MSVCR120.dll Fault Module Version: 12.0.21005.1 Fault Module Timestamp: 524f83ff ``` My system only has a RADEON 6450 GPU, must this build run on a system with a CUDA GPU? My system is a stock Dell XPS8300 with Windows 7 Home premium 64bit, i7 CPU, 12 GB ram Thank You
craigar commented 2014-05-04 02:18:12 +02:00 (Migrated from localhost:3001)

Just replaced the RADEON 6450 with a nVidea 550ti, and this build still won't load in my particular system.

Also the Windows 7 Home premium 64bit is service pack 1 - I am leery of MS service packs.. :)

Thank You

Just replaced the RADEON 6450 with a nVidea 550ti, and this build still won't load in my particular system. Also the Windows 7 Home premium 64bit is service pack 1 - I am leery of MS service packs.. :) Thank You

but vc 12.0 == vc 2013

but vc 12.0 == vc 2013
craigar commented 2014-05-04 10:13:59 +02:00 (Migrated from localhost:3001)

found out why it wouldn't load - there needs to be a folder named " 2.70" , with only the folders you already included ( datafiles, python, scripts) inside of it , then it opened!

Looking forward to trying it out
Thank You

found out why it wouldn't load - there needs to be a folder named " 2.70" , with only the folders you already included ( datafiles, python, scripts) inside of it , then it opened! Looking forward to trying it out Thank You
craigar commented 2014-05-04 11:03:54 +02:00 (Migrated from localhost:3001)

rendering of "BMW1M_adptiv_sampl.blend" freezes after about 27/150 tiles (64x64), and continues to consume 98% of CPU power - have to "kill" (end task) blender using task manager, takes about 5 seconds to "kill" blender after clicking button, and minute to get to task manager, cuase the PC is responding so slow. After stopping blender the PC runs fine & can try another instance again without any noticeable problems

thanks
craig

rendering of "BMW1M_adptiv_sampl.blend" freezes after about 27/150 tiles (64x64), and continues to consume 98% of CPU power - have to "kill" (end task) blender using task manager, takes about 5 seconds to "kill" blender after clicking button, and minute to get to task manager, cuase the PC is responding so slow. After stopping blender the PC runs fine & can try another instance again without any noticeable problems thanks craig
craigar commented 2014-05-04 11:13:23 +02:00 (Migrated from localhost:3001)

tried it with 96x96 tiles, still crashes on render (13/60) and hangs the CPU drawing almost full CPU power until I force task manager to stop blender.
Thanks again

MetrpolisCrash-96tiles.JPG

tried it with 96x96 tiles, still crashes on render (13/60) and hangs the CPU drawing almost full CPU power until I force task manager to stop blender. Thanks again ![MetrpolisCrash-96tiles.JPG](https://archive.blender.org/developer/F86881/MetrpolisCrash-96tiles.JPG)

Maybe make adaptive sampling as separate patch and include to blender ? It is very useful feature.

Maybe make adaptive sampling as separate patch and include to blender ? It is very useful feature.
different noise level ![adap_noise.jpg](https://archive.blender.org/developer/F86886/adap_noise.jpg) [BlenderGuru_InteriorRenderingStarter.blend](https://archive.blender.org/developer/F86888/BlenderGuru_InteriorRenderingStarter.blend)

@loptataasdf: cranking tolerated error to 0.25 seems to get rid of it. I only see that error at preview-quality settings.

@loptataasdf: cranking tolerated error to 0.25 seems to get rid of it. I only see that error at preview-quality settings.

Hi lucas, cant apply the patch anymore with latest master

patch -p1 < metropolis_17_fixed.diff
patching file intern/cycles/app/cycles_xml.cpp
patching file intern/cycles/blender/addon/presets.py
patching file intern/cycles/blender/addon/properties.py
Hunk #3 succeeded at 623 with fuzz 1 (offset 132 lines).
patching file intern/cycles/blender/addon/ui.py
patching file intern/cycles/blender/blender_session.cpp
Hunk #1 succeeded at 420 (offset 56 lines).
patching file intern/cycles/blender/blender_sync.cpp
patching file intern/cycles/device/device.h
patching file intern/cycles/device/device_cpu.cpp
patching file intern/cycles/device/device_cuda.cpp
Hunk #2 succeeded at 1041 with fuzz 1 (offset 17 lines).
patching file intern/cycles/device/device_opencl.cpp
patching file intern/cycles/device/device_task.cpp
patching file intern/cycles/device/device_task.h
patching file intern/cycles/kernel/kernel.cpp
patching file intern/cycles/kernel/kernel.h
patching file intern/cycles/kernel/kernel_avx.cpp
patching file intern/cycles/kernel/kernel_camera.h
patching file intern/cycles/kernel/kernel_passes.h
patching file intern/cycles/kernel/kernel_path.h
Hunk #2 FAILED at 212.
Hunk #3 succeeded at 601 with fuzz 2 (offset 29 lines).
Hunk #4 succeeded at 726 (offset 28 lines).

1 out of 19 hunks FAILED -- saving rejects to file intern/cycles/kernel/kernel_path.h.rej

patching file intern/cycles/kernel/kernel_random.h
patching file intern/cycles/kernel/kernel_sse2.cpp
patching file intern/cycles/kernel/kernel_sse3.cpp
patching file intern/cycles/kernel/kernel_sse41.cpp
patching file intern/cycles/kernel/kernel_types.h
Hunk #2 succeeded at 200 (offset 28 lines).
Hunk #3 FAILED at 283.
Hunk #4 succeeded at 354 (offset 29 lines).
Hunk #5 succeeded at 851 (offset 29 lines).
Hunk #6 succeeded at 916 (offset 29 lines).

1 out of 6 hunks FAILED -- saving rejects to file intern/cycles/kernel/kernel_types.h.rej

patching file intern/cycles/kernel/kernel_volume.h
patching file intern/cycles/render/buffers.cpp
patching file intern/cycles/render/buffers.h
patching file intern/cycles/render/film.cpp
Hunk #1 succeeded at 158 with fuzz 2 (offset 3 lines).
Hunk #2 succeeded at 406 with fuzz 1 (offset 7 lines).
patching file intern/cycles/render/integrator.cpp
patching file intern/cycles/render/integrator.h
patching file intern/cycles/render/session.cpp
Hunk #1 succeeded at 31 (offset 1 line).
Hunk #2 succeeded at 45 (offset 1 line).
Hunk #3 succeeded at 54 (offset 1 line).
Hunk #4 succeeded at 378 (offset 1 line).
Hunk #5 succeeded at 410 (offset 1 line).
Hunk #6 succeeded at 419 (offset 1 line).
Hunk #7 succeeded at 461 (offset 1 line).
Hunk #8 succeeded at 484 (offset 1 line).
Hunk #9 succeeded at 549 (offset 1 line).
Hunk #10 succeeded at 855 (offset 5 lines).
Hunk #11 succeeded at 916 (offset 5 lines).
patching file intern/cycles/render/session.h
patching file intern/cycles/render/tile.cpp
patching file intern/cycles/render/tile.h
patching file intern/cycles/util/CMakeLists.txt
patching file intern/cycles/util/util_color.h
patching file intern/cycles/util/util_importance.cpp
patching file intern/cycles/util/util_importance.h
patching file intern/cycles/util/util_metropolis.cpp
patching file intern/cycles/util/util_metropolis.h

Thank you, mib.

Hi lucas, cant apply the patch anymore with latest master ``` patch -p1 < metropolis_17_fixed.diff patching file intern/cycles/app/cycles_xml.cpp patching file intern/cycles/blender/addon/presets.py patching file intern/cycles/blender/addon/properties.py Hunk #3 succeeded at 623 with fuzz 1 (offset 132 lines). patching file intern/cycles/blender/addon/ui.py patching file intern/cycles/blender/blender_session.cpp Hunk #1 succeeded at 420 (offset 56 lines). patching file intern/cycles/blender/blender_sync.cpp patching file intern/cycles/device/device.h patching file intern/cycles/device/device_cpu.cpp patching file intern/cycles/device/device_cuda.cpp Hunk #2 succeeded at 1041 with fuzz 1 (offset 17 lines). patching file intern/cycles/device/device_opencl.cpp patching file intern/cycles/device/device_task.cpp patching file intern/cycles/device/device_task.h patching file intern/cycles/kernel/kernel.cpp patching file intern/cycles/kernel/kernel.h patching file intern/cycles/kernel/kernel_avx.cpp patching file intern/cycles/kernel/kernel_camera.h patching file intern/cycles/kernel/kernel_passes.h patching file intern/cycles/kernel/kernel_path.h Hunk #2 FAILED at 212. Hunk #3 succeeded at 601 with fuzz 2 (offset 29 lines). Hunk #4 succeeded at 726 (offset 28 lines). ``` 1 out of 19 hunks FAILED -- saving rejects to file intern/cycles/kernel/kernel_path.h.rej ``` patching file intern/cycles/kernel/kernel_random.h patching file intern/cycles/kernel/kernel_sse2.cpp patching file intern/cycles/kernel/kernel_sse3.cpp patching file intern/cycles/kernel/kernel_sse41.cpp patching file intern/cycles/kernel/kernel_types.h Hunk #2 succeeded at 200 (offset 28 lines). Hunk #3 FAILED at 283. Hunk #4 succeeded at 354 (offset 29 lines). Hunk #5 succeeded at 851 (offset 29 lines). Hunk #6 succeeded at 916 (offset 29 lines). ``` 1 out of 6 hunks FAILED -- saving rejects to file intern/cycles/kernel/kernel_types.h.rej ``` patching file intern/cycles/kernel/kernel_volume.h patching file intern/cycles/render/buffers.cpp patching file intern/cycles/render/buffers.h patching file intern/cycles/render/film.cpp Hunk #1 succeeded at 158 with fuzz 2 (offset 3 lines). Hunk #2 succeeded at 406 with fuzz 1 (offset 7 lines). patching file intern/cycles/render/integrator.cpp patching file intern/cycles/render/integrator.h patching file intern/cycles/render/session.cpp Hunk #1 succeeded at 31 (offset 1 line). Hunk #2 succeeded at 45 (offset 1 line). Hunk #3 succeeded at 54 (offset 1 line). Hunk #4 succeeded at 378 (offset 1 line). Hunk #5 succeeded at 410 (offset 1 line). Hunk #6 succeeded at 419 (offset 1 line). Hunk #7 succeeded at 461 (offset 1 line). Hunk #8 succeeded at 484 (offset 1 line). Hunk #9 succeeded at 549 (offset 1 line). Hunk #10 succeeded at 855 (offset 5 lines). Hunk #11 succeeded at 916 (offset 5 lines). patching file intern/cycles/render/session.h patching file intern/cycles/render/tile.cpp patching file intern/cycles/render/tile.h patching file intern/cycles/util/CMakeLists.txt patching file intern/cycles/util/util_color.h patching file intern/cycles/util/util_importance.cpp patching file intern/cycles/util/util_importance.h patching file intern/cycles/util/util_metropolis.cpp patching file intern/cycles/util/util_metropolis.h ``` Thank you, mib.
Author
Member

@craigar I'll look into the freezing, it also happens in my Linux build.
@lopataasdf The different noise level is quite expected, since the error of the tile is currently based on an average over the tile and the max error in the tile. In your example, the right-side part of the low-noise tile probably needed more samples, so the left part also got sampled mode. In the high-noise tile, the error is more uniform so the rendering stops earlier. This is the reason why I also added the in-tile adaptive sampling (it looks to me as if you didn't use it, if you did, please tell me), which would give the right side of the low a noise tile more priority, making the tile pass the convergence test in less samples.
By the way: The smaller tiles are, the more fine-grained the adaptive stopping works. I usually use 8x8 or 16x16 in my tests, which usually (at least on CPU) is faster than large tiles.
@mib2berlin Yes, the bake merge broke the metro_17, I'm currently rebasing the patch. Once I'm done, I'll publish metro_18 (also with the MSVC fixes).

Regarding splitting the patch: There are some features used by both Metro and Adaptive (most importantly the Samples pass), so a clean split is quite difficult. My suggestion would be to setup a "advanced sampling" branch that could be merged to trunk feature by feature (Basic Adaptive Stopping, Ada-Sampling, Metro) once it's tested enough and the codestyle is fine.

@craigar I'll look into the freezing, it also happens in my Linux build. @lopataasdf The different noise level is quite expected, since the error of the tile is currently based on an average over the tile and the max error in the tile. In your example, the right-side part of the low-noise tile probably needed more samples, so the left part also got sampled mode. In the high-noise tile, the error is more uniform so the rendering stops earlier. This is the reason why I also added the in-tile adaptive sampling (it looks to me as if you didn't use it, if you did, please tell me), which would give the right side of the low a noise tile more priority, making the tile pass the convergence test in less samples. By the way: The smaller tiles are, the more fine-grained the adaptive stopping works. I usually use 8x8 or 16x16 in my tests, which usually (at least on CPU) is faster than large tiles. @mib2berlin Yes, the bake merge broke the metro_17, I'm currently rebasing the patch. Once I'm done, I'll publish metro_18 (also with the MSVC fixes). Regarding splitting the patch: There are some features used by both Metro and Adaptive (most importantly the Samples pass), so a clean split is quite difficult. My suggestion would be to setup a "advanced sampling" branch that could be merged to trunk feature by feature (Basic Adaptive Stopping, Ada-Sampling, Metro) once it's tested enough and the codestyle is fine.

Added subscriber: @ideasman42

Added subscriber: @ideasman42

Had a look at updating the patch (since some changes I made might have caused conflicts),

However it looks like ScenePassType now has no bits left, (so perhaps it has to be extended to int64_t)

Had a look at updating the patch (since some changes I made might have caused conflicts), However it looks like `ScenePassType` now has no bits left, (so perhaps it has to be extended to `int64_t`)

This patch may be almost ready to simply commit to trunk as an experimental feature, I'm currently testing adaptive metropolis sampling with adaptive stopping on a very difficult scene and several areas have already shown to converge an exponential amount faster than with generic pathtracing (more than 20x faster at the least).

If it's that intertwined, simply committing it with the current feature set might actually be less work.

This patch may be almost ready to simply commit to trunk as an experimental feature, I'm currently testing adaptive metropolis sampling with adaptive stopping on a very difficult scene and several areas have already shown to converge an exponential amount faster than with generic pathtracing (more than 20x faster at the least). If it's that intertwined, simply committing it with the current feature set might actually be less work.

The code needs to go through extensive code review first via our Differential system here: https://developer.blender.org/differential/query/open/
For 2.71 it's also too late.

The code needs to go through extensive code review first via our Differential system here: https://developer.blender.org/differential/query/open/ For 2.71 it's also too late.
Author
Member

Before submitting for code review, there are still some issues I'd like to fix:

  • Adaptive Sampling for GPU: Currently, Adaptive stopping should already work on GPU (could someone test this?), but the sampling doesn't. The sampling of the pixels could be done on the CPU and be transferred to the Kernel or also done on the GPU, but this would require a copy of the noise map.
  • Progressive + Adaptive Stopping: Currently, these don't work together. My solution would be that when both are activated, the tolerated error is lowered every iteration.
  • The Importance Equalisation requires Progressive rendering to work, this should work automatically.
  • Metro still ignores Noise, this should also be added.
  • Error estimation for Metro seems to be broken.
  • The code style and source comments need to be improved.
Before submitting for code review, there are still some issues I'd like to fix: - Adaptive Sampling for GPU: Currently, Adaptive stopping should already work on GPU (could someone test this?), but the sampling doesn't. The sampling of the pixels could be done on the CPU and be transferred to the Kernel or also done on the GPU, but this would require a copy of the noise map. - Progressive + Adaptive Stopping: Currently, these don't work together. My solution would be that when both are activated, the tolerated error is lowered every iteration. - The Importance Equalisation requires Progressive rendering to work, this should work automatically. - Metro still ignores Noise, this should also be added. - Error estimation for Metro seems to be broken. - The code style and source comments need to be improved.

So that means?

  1. Will be in 2.72?

  2. Going to get Metro on GPU as well?

So that means? 1. Will be in 2.72? 2. Going to get Metro on GPU as well?

How hard - if ever possible - is it to implement per-pixel stop condition?

How hard - if ever possible - is it to implement per-pixel stop condition?
Author
Member

@00Ghz Whether it will be in 2.72 depends on how fast it passes code review. There's a chance, but it's not sure. Regarding GPU Metro: As I said before, it would be possible, but I think it would only be slightly better than PT and not worth the effort. Maybe I'm wrong with this, but definitely, GPU-Metro will never reach CPU-Metro (since it requires more and shorter Markov-Chains) in Terms of Quality per Sample. So: Maybe later, if it actually gives a benefit, but it's currently no priority for me.
@lsscpp It's not hard at all, actually. This would require a per-pixel rendering as in the old Blender-Internal times, which might even improve Cache Coherency (and therefore speed). The Pixels could be walked with a Hilbert curve for even more Cache performance, with one thread rendering one pixel.
The main problem would be pixels falsely being considered converged. For 64 pixels this is more unlikely since the errors in the estimation cancel each other out quite well. This might be solved by rendering each pixel ~10 samples more after it converged and then testing convergence again.
So, to conclude, it would be definitely possible and is an interesting option for future development.

@00Ghz Whether it will be in 2.72 depends on how fast it passes code review. There's a chance, but it's not sure. Regarding GPU Metro: As I said before, it would be possible, but I think it would only be slightly better than PT and not worth the effort. Maybe I'm wrong with this, but definitely, GPU-Metro will never reach CPU-Metro (since it requires more and shorter Markov-Chains) in Terms of Quality per Sample. So: Maybe later, if it actually gives a benefit, but it's currently no priority for me. @lsscpp It's not hard at all, actually. This would require a per-pixel rendering as in the old Blender-Internal times, which might even improve Cache Coherency (and therefore speed). The Pixels could be walked with a Hilbert curve for even more Cache performance, with one thread rendering one pixel. The main problem would be pixels falsely being considered converged. For 64 pixels this is more unlikely since the errors in the estimation cancel each other out quite well. This might be solved by rendering each pixel ~10 samples more after it converged and then testing convergence again. So, to conclude, it would be definitely possible and is an interesting option for future development.

Well in Octane Metro(well whatever hybrid they did) gives much better results for caustics. There might be some other things but don't know them.

Well in Octane Metro(well whatever hybrid they did) gives much better results for caustics. There might be some other things but don't know them.

Another couple of test renders, this time at 720p resolution. Still only using 6 of 8 cores.

No1.png

Yes1.png

Another couple of test renders, this time at 720p resolution. Still only using 6 of 8 cores. ![No1.png](https://archive.blender.org/developer/F86997/No1.png) ![Yes1.png](https://archive.blender.org/developer/F86999/Yes1.png)

Lukas; Okay, thanks for the full rundown on the remaining issues and points that need to be resolved before it gets submitted to review, I assume that you know a lot better of what needs to be done than what testers are finding.

I can only imagine how much better it will be by then once those things are resolved :)

Lukas; Okay, thanks for the full rundown on the remaining issues and points that need to be resolved before it gets submitted to review, I assume that you know a lot better of what needs to be done than what testers are finding. I can only imagine how much better it will be by then once those things are resolved :)

Also, I think I have found that Metro actually seems to work with adaptive stopping when you're not using the 'progressive refine' mode, at least I had a very quick test render stop on its own after a minute with a high error tolerance.

Also, I think I have found that Metro actually seems to work with adaptive stopping when you're not using the 'progressive refine' mode, at least I had a very quick test render stop on its own after a minute with a high error tolerance.

Ok... I've been following closely and testing every build that has came out here...

And I have to say that this last build has given my CPU ~4 times faster renders than my GPU, when in the past, my GPU setup gives me 8x faster renders... My CPU is a rather old Intel Q9550 O.C. to 3Ghz... and my GPUs are two Nvidia GeForce 550ti.

This whole development is incredible, not only the Metro part, but the whole Adaptive Sampling and Importance Equalization give real-live-production renders an incredible boost in convergence efficiency!

Huge kudos to you, Lukas! Please, keep up the great work!

Ok... I've been following closely and testing every build that has came out here... And I have to say that this last build has given my CPU ~4 times faster renders than my GPU, when in the past, my GPU setup gives me 8x faster renders... My CPU is a rather old Intel Q9550 O.C. to 3Ghz... and my GPUs are **two** Nvidia GeForce 550ti. This whole development is incredible, not only the Metro part, but the whole Adaptive Sampling and Importance Equalization give real-live-production renders an incredible boost in convergence efficiency! Huge kudos to you, Lukas! Please, keep up the great work!

Do I read well: now your CPU rendering is 4 times faster than GPU... instead of 8 times slower??
This means 32 times faster in CPU/CPU comparison!! Or in other words, 3200% speed boost, or also rendering in ~3% of the time!
Ok, let's say that I misunderstand something here...

Do I read well: now your CPU rendering is 4 times faster than GPU... instead of 8 times slower?? This means 32 times faster in CPU/CPU comparison!! Or in other words, 3200% speed boost, or also rendering in ~3% of the time! Ok, let's say that I misunderstand something here...

It does seem fishy I guess. It does seem way to fast.

It does seem fishy I guess. It does seem way to fast.

Yes indeed, my tiny test showed way lower speed gains. Indeed almost no speed gain for bit-noisy renders. But this is probably my fault.

BTW I read this - http://lists.blender.org/pipermail/bf-cycles/2014-May/001921.html - on the mailing list And wondered if it could be used for some further balancing in the (adaptive) sampling process. Reading Brecht answer sounds like I totally misunderstood the paper though...

Yes indeed, my tiny test showed way lower speed gains. Indeed almost no speed gain for bit-noisy renders. But this is probably my fault. BTW I read this - http://lists.blender.org/pipermail/bf-cycles/2014-May/001921.html - on the mailing list And wondered if it could be used for some further balancing in the (adaptive) sampling process. Reading Brecht answer sounds like I totally misunderstood the paper though...

Yes, I'm saying that now my " CPU rendering is 4 times faster than GPU... instead of 8 times slower"...

I'm very aware that the scene I rendered was something of a "best case" scenario and not in all cases the speed improvement was that good, but it was better by 2x at the very least!

Here's a couple of renders that took around 8h (each) in CPU while they would take around 30-36h (each) in GPU. (original 1920x1080).

NocheInt02.jpg
RestExtNight02.jpg

Yes, I'm saying that now my " CPU rendering is 4 times faster than GPU... instead of 8 times slower"... I'm very aware that the scene I rendered was something of a "best case" scenario and not in all cases the speed improvement was that good, but it was better by 2x at the very least! Here's a couple of renders that took around 8h (each) in CPU while they would take around 30-36h (each) in GPU. (original 1920x1080). ![NocheInt02.jpg](https://archive.blender.org/developer/F87889/NocheInt02.jpg) ![RestExtNight02.jpg](https://archive.blender.org/developer/F87892/RestExtNight02.jpg)
craigar commented 2014-05-10 12:37:35 +02:00 (Migrated from localhost:3001)

using the build lukasstockner97 posted Fri, May 2, 10:11 PM

http://www.mediafire.com/download/1lkr2mqdd521atu/Blender_270_Metro_VC2013_x64.zip

it is only about 30% quicker on my i7 than just 2.7 cycles in a simple scene, that has a complex "diamond shader" from

http://www.blendswap.com/blends/view/39307

Where can I get newer builds of this?

Thank you
crystal_sphere_frame48.jpg

using the build lukasstockner97 posted Fri, May 2, 10:11 PM http://www.mediafire.com/download/1lkr2mqdd521atu/Blender_270_Metro_VC2013_x64.zip it is only about 30% quicker on my i7 than just 2.7 cycles in a simple scene, that has a complex "diamond shader" from http://www.blendswap.com/blends/view/39307 Where can I get newer builds of this? Thank you ![crystal_sphere_frame48.jpg](https://archive.blender.org/developer/F88133/crystal_sphere_frame48.jpg)
Author
Member

@craigar Currently nowhere since metro_17 was (up to now) the newest patch, I'll upload a 18-build shortly (most likely tomorrow).

OK, metropolis_18 is finished now. I included the MSVC fixes, so it should work for it as well (unless the new code broke it). CUDA works now, I tested it and everything compiles and runs (No, still no Metro on GPU). Adaptive sampling code now works on CUDA as well, but it gives results that are inferior to CPU and I currently have no idea why (it's still better than no adaptive, through). However, the performance of GPU rendering drops when using adaptive sampling, so that in some cases running without it might even give better results in the same time.

All in all, I changed quite a lot in both adaptive features, for example, now the adaptive stopping uses a power mean with p=4 instead of a regular quadratic mean, this gives more influence to the higher errors in the tile. Basically, the higher p is, the more the extremes are pronounced (1 gives an average, 2 the "regular" quadratic mean, and in theory infinity would give a simple max operation). Some other things were also changed, so you probably have to re-tune your parameters.

The progressive + adaptive stopping combination doesn't work yet, but the changed code layout makes it way easier now.
Importance equalisation now automatically switches on progressive rendering (for it to work well, set a low adaptive interval like 5).
I've worked a bit on codestyle, but its still not quite good yet.

Another thing: I removed the adaptive warmup, it now just uses one map interval as the warmup, but I'll probably re-add it.

Metro_19 might take a while since I currently have quite a lot of other things to do, but I'll still follow this thread. Real development will continue around the beginning of June.

metropolis_18.diff

@craigar Currently nowhere since metro_17 was (up to now) the newest patch, I'll upload a 18-build shortly (most likely tomorrow). OK, metropolis_18 is finished now. I included the MSVC fixes, so it should work for it as well (unless the new code broke it). CUDA works now, I tested it and everything compiles and runs (No, still no Metro on GPU). Adaptive sampling code now works on CUDA as well, but it gives results that are inferior to CPU and I currently have no idea why (it's still better than no adaptive, through). However, the performance of GPU rendering drops when using adaptive sampling, so that in some cases running without it might even give better results in the same time. All in all, I changed quite a lot in both adaptive features, for example, now the adaptive stopping uses a power mean with p=4 instead of a regular quadratic mean, this gives more influence to the higher errors in the tile. Basically, the higher p is, the more the extremes are pronounced (1 gives an average, 2 the "regular" quadratic mean, and in theory infinity would give a simple max operation). Some other things were also changed, so you probably have to re-tune your parameters. The progressive + adaptive stopping combination doesn't work yet, but the changed code layout makes it way easier now. Importance equalisation now automatically switches on progressive rendering (for it to work well, set a low adaptive interval like 5). I've worked a bit on codestyle, but its still not quite good yet. Another thing: I removed the adaptive warmup, it now just uses one map interval as the warmup, but I'll probably re-add it. Metro_19 might take a while since I currently have quite a lot of other things to do, but I'll still follow this thread. Real development will continue around the beginning of June. [metropolis_18.diff](https://archive.blender.org/developer/F88507/metropolis_18.diff)
Author
Member

I just read the Multiplexed Metropolis paper (from http://cs.au.dk/~toshiya/) and sadly, it's no use for Cycles at all (currently) since it focuses on choosing the right (s, t) pair for bidirectional path tracing. That's what the authors mean with "Combination of MIS and MCMC", since MIS was developed for choosing this pair at first.

I just read the Multiplexed Metropolis paper (from http://cs.au.dk/~toshiya/) and sadly, it's no use for Cycles at all (currently) since it focuses on choosing the right (s, t) pair for bidirectional path tracing. That's what the authors mean with "Combination of MIS and MCMC", since MIS was developed for choosing this pair at first.

I wouldn't worry about that right now Lukas, I and a lot of others would just be happy to see the adaptive metropolis sampling feature developed as far as you can get it and committed to Master. If you're thinking about adding a bidirectional sampler, then it would be advisable to hold that off until after this patch is completed, one thing at a time :).

I wouldn't worry about that right now Lukas, I and a lot of others would just be happy to see the adaptive metropolis sampling feature developed as far as you can get it and committed to Master. If you're thinking about adding a bidirectional sampler, then it would be advisable to hold that off until after this patch is completed, one thing at a time :).

Hi Lucas, I am sorry but _18.diff fail on Linux during patching.
I don´t try to compile.

patching file intern/cycles/kernel/kernel_types.h
Hunk #2 FAILED at 200.
Hunk #3 succeeded at 312 (offset 2 lines).
Hunk #4 succeeded at 355 (offset 2 lines).
Hunk #5 succeeded at 852 (offset 2 lines).
Hunk #6 succeeded at 917 (offset 2 lines).

1 out of 6 hunks FAILED -- saving rejects to file intern/cycles/kernel/kernel_types.h.rej


Thank you, mib

Hi Lucas, I am sorry but _18.diff fail on Linux during patching. I don´t try to compile. ``` patching file intern/cycles/kernel/kernel_types.h Hunk #2 FAILED at 200. Hunk #3 succeeded at 312 (offset 2 lines). Hunk #4 succeeded at 355 (offset 2 lines). Hunk #5 succeeded at 852 (offset 2 lines). Hunk #6 succeeded at 917 (offset 2 lines). ``` 1 out of 6 hunks FAILED -- saving rejects to file intern/cycles/kernel/kernel_types.h.rej ``` ``` Thank you, mib
Author
Member

@Ace_Dragon Well, I do have some ideas what do do after this patch, but BPT is definitely not on it (see http://lists.blender.org/pipermail/bf-cycles/2014-May/001929.html for my reasons) and of course I'll finish this patch first :D

@mib2berlin Are you sure you use the current master? I just (5min ago) pulled the newest changes and metropolis_18.diff applies for me (also on Linux). The error in kernel_types sounds as if you haven't pulled in the baking commit yet, since then one Pass would be missing. If not, could you post the intern/cycles/kernel/kernel_types.h.rej?
I'll also include a new patch, this time in a better format (true patch instead of diff), maybe this one works better (although apart from 2 functions now being inlined, there is nothing changed).

By the way: I ran some benchmarks a few days ago, and it really seems that most speed issues are fixed. First of all, when compared with a unpatched master (built with the same compiler etc.), there was no speed difference in classic PT (with all new options off) apart from the usual +- 1-2% due to background processes. The Metro sampler that was so slow in the beginning also catches up quite good, the same scene (an indoor Archviz) took 8:31 with PT (no Adaptive) and 8:57 with Metro at the same sample number (200). The problem with 10-second-tests is that Metro has to do an "first pass" for the UV and ID channels that of course influences results when rendering with 5 regular samples (the first pass doesn't count, so if you set 5 samples, it will do first pass + 5 samples). Maybe a check for only doing this when UV or ID passes are activated would be a good idea.

By the way, something I forgot to say until now: Due to the way the multi-threading for Metro currently works, if you specify 10 samples and 4 threads, every thread will run 10 samples on the whole image, resulting in 40 samples/pixel. Therefore, for a fair comparison, you'd have to set 40 samples when rendering PT. I'll change this, but currently that's how it works.

metropolis_18_n.patch

@Ace_Dragon Well, I do have some ideas what do do after this patch, but BPT is definitely not on it (see http://lists.blender.org/pipermail/bf-cycles/2014-May/001929.html for my reasons) and of course I'll finish this patch first :D @mib2berlin Are you sure you use the current master? I just (5min ago) pulled the newest changes and metropolis_18.diff applies for me (also on Linux). The error in kernel_types sounds as if you haven't pulled in the baking commit yet, since then one Pass would be missing. If not, could you post the intern/cycles/kernel/kernel_types.h.rej? I'll also include a new patch, this time in a better format (true patch instead of diff), maybe this one works better (although apart from 2 functions now being inlined, there is nothing changed). By the way: I ran some benchmarks a few days ago, and it really seems that most speed issues are fixed. First of all, when compared with a unpatched master (built with the same compiler etc.), there was no speed difference in classic PT (with all new options off) apart from the usual +- 1-2% due to background processes. The Metro sampler that was so slow in the beginning also catches up quite good, the same scene (an indoor Archviz) took 8:31 with PT (no Adaptive) and 8:57 with Metro at the same sample number (200). The problem with 10-second-tests is that Metro has to do an "first pass" for the UV and ID channels that of course influences results when rendering with 5 regular samples (the first pass doesn't count, so if you set 5 samples, it will do first pass + 5 samples). Maybe a check for only doing this when UV or ID passes are activated would be a good idea. By the way, something I forgot to say until now: Due to the way the multi-threading for Metro currently works, if you specify 10 samples and 4 threads, *every* thread will run 10 samples on the whole image, resulting in 40 samples/pixel. Therefore, for a fair comparison, you'd have to set 40 samples when rendering PT. I'll change this, but currently that's how it works. [metropolis_18_n.patch](https://archive.blender.org/developer/F88861/metropolis_18_n.patch)
Member

Build fails for me with metropolis_18_n.patch and latest master:


In file included from /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/kernel_path.h:29:0,
                 from /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/kernel_sse3.cpp:38:
/home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/kernel_accumulate.h:79:2: warning: 'bsdf_eval.ccl::BsdfEval::use_light_pass' may be used uninitialized in this function [-Wmaybe-uninitialized]
  if(eval->use_light_pass) {
  ^
In file included from /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/kernel_sse3.cpp:38:0:
/home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/kernel_path.h:864:13: note: 'bsdf_eval.ccl::BsdfEval::use_light_pass' was declared here
    BsdfEval bsdf_eval;
             ^
cc1plus: some warnings being treated as errors
intern/cycles/kernel/CMakeFiles/cycles_kernel.dir/build.make:126: recipe for target 'intern/cycles/kernel/CMakeFiles/cycles_kernel.dir/kernel_sse41.cpp.o' failed
make[2]: *** [intern/cycles/kernel/CMakeFiles/cycles_kernel.dir/kernel_sse41.cpp.o] Error 1
cc1plus: some warnings being treated as errors
intern/cycles/kernel/CMakeFiles/cycles_kernel.dir/build.make:103: recipe for target 'intern/cycles/kernel/CMakeFiles/cycles_kernel.dir/kernel_sse3.cpp.o' failed
make[2]: *** [intern/cycles/kernel/CMakeFiles/cycles_kernel.dir/kernel_sse3.cpp.o] Error 1
CMakeFiles/Makefile2:5498: recipe for target 'intern/cycles/kernel/CMakeFiles/cycles_kernel.dir/all' failed
make[1]: *** [intern/cycles/kernel/CMakeFiles/cycles_kernel.dir/all] Error 2
Makefile:146: recipe for target 'all' failed
make: *** [all] Error 2
Build fails for me with metropolis_18_n.patch and latest master: ``` In file included from /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/kernel_path.h:29:0, from /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/kernel_sse3.cpp:38: /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/kernel_accumulate.h:79:2: warning: 'bsdf_eval.ccl::BsdfEval::use_light_pass' may be used uninitialized in this function [-Wmaybe-uninitialized] if(eval->use_light_pass) { ^ In file included from /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/kernel_sse3.cpp:38:0: /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/kernel_path.h:864:13: note: 'bsdf_eval.ccl::BsdfEval::use_light_pass' was declared here BsdfEval bsdf_eval; ^ cc1plus: some warnings being treated as errors intern/cycles/kernel/CMakeFiles/cycles_kernel.dir/build.make:126: recipe for target 'intern/cycles/kernel/CMakeFiles/cycles_kernel.dir/kernel_sse41.cpp.o' failed make[2]: *** [intern/cycles/kernel/CMakeFiles/cycles_kernel.dir/kernel_sse41.cpp.o] Error 1 cc1plus: some warnings being treated as errors intern/cycles/kernel/CMakeFiles/cycles_kernel.dir/build.make:103: recipe for target 'intern/cycles/kernel/CMakeFiles/cycles_kernel.dir/kernel_sse3.cpp.o' failed make[2]: *** [intern/cycles/kernel/CMakeFiles/cycles_kernel.dir/kernel_sse3.cpp.o] Error 1 CMakeFiles/Makefile2:5498: recipe for target 'intern/cycles/kernel/CMakeFiles/cycles_kernel.dir/all' failed make[1]: *** [intern/cycles/kernel/CMakeFiles/cycles_kernel.dir/all] Error 2 Makefile:146: recipe for target 'all' failed make: *** [all] Error 2 ```
Author
Member

@gandalf3 Well, this is a warning that also appears for me when building, however, I haven't modified kernel_accumulate, so it is most likely from master. The line "cc1plus: some warnings being treated as errors" sounds like you have enabled -Werror or something similar, or are there any errors reported further up (when you build multithreaded, the actual error might be further up).?

@gandalf3 Well, this is a warning that also appears for me when building, however, I haven't modified kernel_accumulate, so it is most likely from master. The line "cc1plus: some warnings being treated as errors" sounds like you have enabled -Werror or something similar, or are there any errors reported further up (when you build multithreaded, the actual error might be further up).?
Member

Unfortunately I don't really know what I'm doing here..

I tried configuring with -Wno-error and building single threaded, but there was a different error (or maybe the same one, and I just didn't see it the first time if it was further up):


In file included from /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/kernel_math.h:20:0,
                 from /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/kernel_types.h:20,
                 from /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/render/camera.h:20,
                 from /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/blender/blender_camera.cpp:17:
/home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/util/util_color.h: In function 'float ccl::linear_gray_to_tvi(float)':
/home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/util/util_color.h:227:23: error: conversion to 'float' from 'double' may alter its value [-Werror=float-conversion]
  float log_v = log10(v);
                       ^
/home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/util/util_color.h:232:9: error: conversion to 'float' from 'double' may alter its value [-Werror=float-conversion]
   log_i = pow(0.405f*log_v + 1.6f, 2.18f) - 2.86f;
         ^
/home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/util/util_color.h:236:9: error: conversion to 'float' from 'double' may alter its value [-Werror=float-conversion]
   log_i = pow(0.249f*log_v + 0.65f, 2.7f) - 0.72f;
         ^
/home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/util/util_color.h:239:31: error: conversion to 'float' from 'double' may alter its value [-Werror=float-conversion]
  return exp(log_i * log(10.0f));
                               ^
cc1plus: some warnings being treated as errors
intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/build.make:57: recipe for target 'intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/blender_camera.cpp.o' failed
make[2]: *** [intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/blender_camera.cpp.o] Error 1
CMakeFiles/Makefile2:5318: recipe for target 'intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/all' failed
make[1]: *** [intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/all] Error 2
Makefile:146: recipe for target 'all' failed
make: *** [all] Error 2

It seems you may be right about -Werror, but I didn't set it..

Unfortunately I don't really know what I'm doing here.. I tried configuring with `-Wno-error` and building single threaded, but there was a different error (or maybe the same one, and I just didn't see it the first time if it was further up): ``` In file included from /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/kernel_math.h:20:0, from /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/kernel/kernel_types.h:20, from /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/render/camera.h:20, from /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/blender/blender_camera.cpp:17: /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/util/util_color.h: In function 'float ccl::linear_gray_to_tvi(float)': /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/util/util_color.h:227:23: error: conversion to 'float' from 'double' may alter its value [-Werror=float-conversion] float log_v = log10(v); ^ /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/util/util_color.h:232:9: error: conversion to 'float' from 'double' may alter its value [-Werror=float-conversion] log_i = pow(0.405f*log_v + 1.6f, 2.18f) - 2.86f; ^ /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/util/util_color.h:236:9: error: conversion to 'float' from 'double' may alter its value [-Werror=float-conversion] log_i = pow(0.249f*log_v + 0.65f, 2.7f) - 0.72f; ^ /home/gandalf3/.Blenderversions/gitbuild/blender/intern/cycles/util/util_color.h:239:31: error: conversion to 'float' from 'double' may alter its value [-Werror=float-conversion] return exp(log_i * log(10.0f)); ^ cc1plus: some warnings being treated as errors intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/build.make:57: recipe for target 'intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/blender_camera.cpp.o' failed make[2]: *** [intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/blender_camera.cpp.o] Error 1 CMakeFiles/Makefile2:5318: recipe for target 'intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/all' failed make[1]: *** [intern/cycles/blender/CMakeFiles/bf_intern_cycles.dir/all] Error 2 Makefile:146: recipe for target 'all' failed make: *** [all] Error 2 ``` It seems you may be right about `-Werror`, but I didn't set it..

You are right, was not on master.
Patch work and compile on my system.

Opensuse 13.1/64
i5 3770K

Thanks and cheers, mib

You are right, was not on master. Patch work and compile on my system. Opensuse 13.1/64 i5 3770K Thanks and cheers, mib
craigar commented 2014-05-14 13:41:34 +02:00 (Migrated from localhost:3001)

@LukasStockner

I see you upoladed

metropolis_18_n.patch

I don't have a compiler (nor do I know how to use a C++ compiler, or any modern code compiler) so I need a build for Windows 7 64-bit, or newer files that I can copy into the folders I made for the May 2 build

http://www.mediafire.com/download/1lkr2mqdd521atu/Blender_270_Metro_VC2013_x64.zip

Thank You

@LukasStockner I see you upoladed metropolis_18_n.patch I don't have a compiler (nor do I know how to use a C++ compiler, or any modern code compiler) so I need a build for Windows 7 64-bit, or newer files that I can copy into the folders I made for the May 2 build http://www.mediafire.com/download/1lkr2mqdd521atu/Blender_270_Metro_VC2013_x64.zip Thank You

Hi again.

I'm also still waiting for the Win64 build with the latest patch version and I'm starting to wonder if plans have switched to wait until iteration 19 is done before another build is made.

Thanks.

Hi again. I'm also still waiting for the Win64 build with the latest patch version and I'm starting to wonder if plans have switched to wait until iteration 19 is done before another build is made. Thanks.

Do we really need to be filling up task/tracker pages with requests for builds? There's plenty of that at BA as is. There's no pages here, so it makes it much harder to follow progress when there's tons of "compile for me plz". BA's forum setup makes it a lot easier to work with general discussion, plus there's a lot more people there who can help with building anyway.

Not to mention, compiling Blender REALLY isn't all that hard. There's good instructions in the wiki. If you're good enough with computers to make sense of this thread, and have permissions on your machine to install test builds, you really shouldn't have any issue rolling your own anyhow.

Do we really need to be filling up task/tracker pages with requests for builds? There's plenty of that at BA as is. There's no pages here, so it makes it much harder to follow progress when there's tons of "compile for me plz". BA's forum setup makes it a lot easier to work with general discussion, plus there's a lot more people there who can help with building anyway. Not to mention, compiling Blender REALLY isn't all that hard. There's good instructions in the wiki. If you're good enough with computers to make sense of this thread, and have permissions on your machine to install test builds, you really shouldn't have any issue rolling your own anyhow.

Added subscriber: @PeterBoos

Added subscriber: @PeterBoos

May i suggest "as an exception", this gets pushed to the Blender build bot ?.
Bypassing code reviews. This doesn't break the other cycles render methods or other stuff.
And we all know those builds are experimental anyway.
As a result we wouldn't disturb the programmer any more with build questions, or failing compilations.
Sure it might not have been peer viewed, but is it that so important ??..
Its experimental, and its the biggest blender change since well ehmm... cycles introduction.
Anyone loves a faster renderer, even if still under development.

Also then real problems / suggestions / ideas could be forum discussed BA, and perhaps more people understand this math on other forums as well.

May i suggest "as an exception", this gets pushed to the Blender build bot ?. Bypassing code reviews. This doesn't break the other cycles render methods or other stuff. And we all know those builds are experimental anyway. As a result we wouldn't disturb the programmer any more with build questions, or failing compilations. Sure it might not have been peer viewed, but is it that so important ??.. Its experimental, and its the biggest blender change since well ehmm... cycles introduction. Anyone loves a faster renderer, even if still under development. Also then real problems / suggestions / ideas could be forum discussed BA, and perhaps more people understand this math on other forums as well.

This is not how our development process works. If you want to use it now, you can do it by building Blender on your own.

Also, can we please stop with this "Please I want this" comments? They don't really help here and won't make things happen faster. Thanks!

This is not how our development process works. If you want to use it now, you can do it by building Blender on your own. Also, can we please stop with this "Please I want this" comments? They don't really help here and won't make things happen faster. Thanks!

It would be easier if we all had patience. :)

It would be easier if we all had patience. :)

http://gpupathtracer.blogspot.dk/ anyone checked this one?

they got the code on github and using MLT on GPU

http://gpupathtracer.blogspot.dk/ anyone checked this one? they got the code on github and using MLT on GPU

i compiled new master with metropolis_18_n.patch using mingw64 and cmake
(i had to disable a few things to get smake to work correctly, this isnt exclusive to this patch
i think its something wired with cmake, anyhow i disabled these items they are not available in this build

libmv
bullet
freestyle
osl
CUDA

if you would like try it
http://www.mediafire.com/download/4wjp48eo7le802j/Blender_2.7.5_Mingw64_cmake_mertopolis18.7z

i compiled new master with metropolis_18_n.patch using mingw64 and cmake (i had to disable a few things to get smake to work correctly, this isnt exclusive to this patch i think its something wired with cmake, anyhow i disabled these items they are not available in this build libmv bullet freestyle osl CUDA if you would like try it http://www.mediafire.com/download/4wjp48eo7le802j/Blender_2.7.5_Mingw64_cmake_mertopolis18.7z

Added subscriber: @SterlingRoth

Added subscriber: @SterlingRoth

@00Ghz, That is not an implementation of MLT, it's just standard path tracing.

@00Ghz, That is not an implementation of MLT, it's just standard path tracing.

I read about MLT somewhere. Guess I was mistaken

I read about MLT somewhere. Guess I was mistaken

Added subscriber: @LevonHudson

Added subscriber: @LevonHudson

I think this conversation should be moved over to blender artists forums. this thread is really for development purposes. and while it is great to see some of the images being produced, the "can someone send me a build", "i need help compiling" and "omg wowz" posts are clogging up the page.

Please continue to post on this thread http://blenderartists.org/forum/showthread.php?329089-Cycles-MLT-patch
and leave this thread to developers.

im not a moderator or anything on here, i just think developer.blender should be just that... for devs

I think this conversation should be moved over to blender artists forums. this thread is really for development purposes. and while it is great to see some of the images being produced, the "can someone send me a build", "i need help compiling" and "omg wowz" posts are clogging up the page. Please continue to post on this thread http://blenderartists.org/forum/showthread.php?329089-Cycles-MLT-patch and leave this thread to developers. *im not a moderator or anything on here, i just think developer.blender should be just that... for devs*

Added subscriber: @kwk-2

Added subscriber: @kwk-2

Added subscriber: @VictorMukayev

Added subscriber: @VictorMukayev

i thought i'd share it here
sorry if it's not related, but this Progressive Importance sampling sounds very promissing
https://corona-renderer.com/blog/research-corner-progressive-importance-sampling/

i thought i'd share it here sorry if it's not related, but this Progressive Importance sampling sounds very promissing https://corona-renderer.com/blog/research-corner-progressive-importance-sampling/

Now that's awesome indeed. Only it it would work on the GPU imagine the possibilities;)

Now that's awesome indeed. Only it it would work on the GPU imagine the possibilities;)

@LukasStockner,
I just got some probleme merging the diff ? Could you diff from master if you get some time ? Or provide a githash that merge without difficulties, so i can checkout to it ? Many thanks !

@LukasStockner, I just got some probleme merging the diff ? Could you diff from master if you get some time ? Or provide a githash that merge without difficulties, so i can checkout to it ? Many thanks !

Added subscriber: @derekbarker

Added subscriber: @derekbarker

@LukasStockner what are your plans after the full implementation of mlt

@LukasStockner what are your plans after the full implementation of mlt
Author
Member

Well, time to get back to the Blender development :D
First of all, sorry for not uploading Windows builds and not answering on the thread anymore, my CUDA toolchain just won't work on Windows and I just had so much other stuff to do...
But, I think I have news that can make up for this: Since, for said other project, I got a GTX780 from NVIDIA, I'm currently working on GPU Metropolis! There basically are two reasons for this: First of all, while reading "Physically based Rendering" and Veach's thesis, I noticed that they both recommend using more, shorter chains, so it seems that I was wrong in assuming that more chains (on GPU) mean less quality. Second, due to my current re-structuring of the patch (moving stuff from util/ into the kernel), it's not much work to add it anymore, so why not just do it?
So, once it works reasonably, I'll post a patch, hopefully it won't be long...

Regarding the Progressive Importance Sampling: Wow, that's really awesome! I read the paper and it seems quite reasonable and impementable. It's basically a Photon mapping prepass that is not used to generate the image itself, but rather to "train" the path tracer so it knows from where the light comes from and is able to find more important paths. The implementation details are a mix of PT, Photon mapping, irradiance caching and GMM machine learning.
Since it still uses PT for the image generation, it would definitely be useful for Cycles. In fact, since it already uses Lightpath-Generation etc., but no explicit path connection, it's quite a step on the way to BPT without being as hard to incorporate into Cycles. By the way, PIS should work just as well on the GPU as it does on the CPU.
So, to answer @derekbarker, until the PIS paper was posted here, the next feature on my plan have been Lightgroups. Probably I'll still do them next, but PIS now definitely also is on the plan. Lightgroups won't be as involved as Metro (I imagined representing them like scene layers in the GUI, with every activated layer adding one output to the compositor), but the PIS would be quite more work to do. Still, challenges are fun :D

@tychota Yes, the current patch won't merge anymore. 83cdd5 should work, since it's two days older than metro_18_n. Metro_19 will of course be rebased again.

Well, time to get back to the Blender development :D First of all, sorry for not uploading Windows builds and not answering on the thread anymore, my CUDA toolchain just won't work on Windows and I just had so much other stuff to do... But, I think I have news that can make up for this: Since, for said other project, I got a GTX780 from NVIDIA, I'm currently working on GPU Metropolis! There basically are two reasons for this: First of all, while reading "Physically based Rendering" and Veach's thesis, I noticed that they both recommend using more, shorter chains, so it seems that I was wrong in assuming that more chains (on GPU) mean less quality. Second, due to my current re-structuring of the patch (moving stuff from util/ into the kernel), it's not much work to add it anymore, so why not just do it? So, once it works reasonably, I'll post a patch, hopefully it won't be long... Regarding the Progressive Importance Sampling: Wow, that's really awesome! I read the paper and it seems quite reasonable and impementable. It's basically a Photon mapping prepass that is not used to generate the image itself, but rather to "train" the path tracer so it knows from where the light comes from and is able to find more important paths. The implementation details are a mix of PT, Photon mapping, irradiance caching and GMM machine learning. Since it still uses PT for the image generation, it would definitely be useful for Cycles. In fact, since it already uses Lightpath-Generation etc., but no explicit path connection, it's quite a step on the way to BPT without being as hard to incorporate into Cycles. By the way, PIS should work just as well on the GPU as it does on the CPU. So, to answer @derekbarker, until the PIS paper was posted here, the next feature on my plan have been Lightgroups. Probably I'll still do them next, but PIS now definitely also is on the plan. Lightgroups won't be as involved as Metro (I imagined representing them like scene layers in the GUI, with every activated layer adding one output to the compositor), but the PIS would be quite more work to do. Still, challenges are fun :D @tychota Yes, the current patch won't merge anymore. 83cdd5 should work, since it's two days older than metro_18_n. Metro_19 will of course be rebased again.

@LukasStockner Dude.......... I love you! <3

@LukasStockner Dude.......... I love you! <3
craigar commented 2014-06-02 12:08:37 +02:00 (Migrated from localhost:3001)

Drum rollll !!!! xxxoooxxxxooo :) :) :) * * * Go Lucas! Go * * * Lookin forward -Thanks in advance ;)

Drum rollll !!!! xxxoooxxxxooo :) :) :) * * * Go Lucas! Go * * * Lookin forward -Thanks in advance ;)

This is not a Blenderartists thread here, what is the sense of all these off-topic "Drum roll" posts? I get an e-mail about every new post.

This is not a Blenderartists thread here, what is the sense of all these off-topic "Drum roll" posts? I get an e-mail about every new post.

Removed subscriber: @ThomasDinges

Removed subscriber: @ThomasDinges

Maybe a silly question but i think it would be perfect to get a branched path version of this. Sometimes (most of the time) either translucent or glossy shader or SSS are producing the noise but diffuse isn't. So is it possible that the noise aware sample adjust branched sample so it won't spend processor or tracing diffuse rays or anything that has currently converted ?

Maybe a silly question but i think it would be perfect to get a branched path version of this. Sometimes (most of the time) either translucent or glossy shader or SSS are producing the noise but diffuse isn't. So is it possible that the noise aware sample adjust branched sample so it won't spend processor or tracing diffuse rays or anything that has currently converted ?

@tychota Yes, the current patch won't merge anymore. 83cdd5 should work, since it's two days older than metro_18_n. Metro_19 will of course be rebased again.

no it don't.
I did

git checkout -b 83cdd5
patch -p1 < pathToMetropolis1_n.patch

but stilll get plenty of rejections

.../device/device_cpu.cpp
.../device/device_cuda.cpp
.../kernel/kernel_path.h
etc

Would it be different if i use arc instead of patch -p1 ?

> @tychota Yes, the current patch won't merge anymore. 83cdd5 should work, since it's two days older than metro_18_n. Metro_19 will of course be rebased again. no it don't. I did ``` git checkout -b 83cdd5 patch -p1 < pathToMetropolis1_n.patch ``` but stilll get plenty of rejections ``` .../device/device_cpu.cpp .../device/device_cuda.cpp .../kernel/kernel_path.h etc ``` Would it be different if i use arc instead of patch -p1 ?
craigar commented 2014-06-03 10:34:28 +02:00 (Migrated from localhost:3001)

Added subscriber: @ThomasDinges

Added subscriber: @ThomasDinges
craigar commented 2014-06-03 10:34:28 +02:00 (Migrated from localhost:3001)

@ThomasDinges Please accept my sincere apologies, from now on I will keep my input here only relative to "beta testing/suggestions" of the code in development - if thats appropriate? I won't waste your time via your "automatic email" from this (or any developer) thread.
I am very grateful for all you do for the 3D community, and don't want to be disrespectful or wasteful.
I became a bit "over excited" when I read about Lucas' enthusiasm towards moving forward on implementing GPU, MLT and PIS and should have thought first to send my "cheering" to him from an appropriate forum/channel.

Thank You

@ThomasDinges Please accept my sincere apologies, from now on I will keep my input here only relative to "beta testing/suggestions" of the code in development - if thats appropriate? I won't waste your time via your "automatic email" from this (or any developer) thread. I am very grateful for all you do for the 3D community, and don't want to be disrespectful or wasteful. I became a bit "over excited" when I read about Lucas' enthusiasm towards moving forward on implementing GPU, MLT and PIS and should have thought first to send my "cheering" to him from an appropriate forum/channel. Thank You

@LukasStockner would it be possible to turn 1 gpu in to multiple small tiles so it can be more efficient with tolerated error ?

@LukasStockner would it be possible to turn 1 gpu in to multiple small tiles so it can be more efficient with tolerated error ?

About me asking earlier if it was possible to have per-pixel stop-condition, I thought of a workflow like this:

  • Each pixel gets evaluated as usual, but now every x-samples* a variable gets stored (yeah for each pixel in the tile!). Let's call it Y
  • Y is the absolute difference from the previous pixel computed and the new one. (Being that the sampling iteration brings every pixel to be averaged, Y value will be decreasing more and more)
  • Here we could set the threshold: if Y < threshold then stop sampling this pixel.

(* this could be every single pass or user-defined number of passes. Not sure what would be better.)

Not sure if i was clear. English is not my language of course.
And not sure if this is a totally absurd coding concept, or, I suspect a total memory-killer approach.

About me asking earlier if it was possible to have per-pixel stop-condition, I thought of a workflow like this: - Each pixel gets evaluated as usual, but now every x-samples* a variable gets stored (yeah for each pixel in the tile!). Let's call it *Y* - *Y* is the absolute difference from the previous pixel computed and the new one. (Being that the sampling iteration brings every pixel to be averaged, *Y* value will be decreasing more and more) - Here we could set the threshold: if *Y* < threshold then stop sampling this pixel. (* this could be every single pass or user-defined number of passes. Not sure what would be better.) Not sure if i was clear. English is not my language of course. And not sure if this is a totally absurd coding concept, or, I suspect a total memory-killer approach.

Max Consecutive Rejections - 256
Mutation distance - 0.40
Large Mutation chance - 0.40
Num. samples - 1024 using Correlated multi-jitter
Sampling - Equi-Importance
Patch revision - 18
0539b.jpg

The build I'm using is almost a month old (A MingW build from HolyEnigma), but the results you get from this patch are absolutely stunning with the right settings.

It goes to show that even the worst case scenarios can be rendered now in around 3 days max (with simpler cases surely rendering quite a lot faster than that).

Good work once again, I stand by my previous token award :)

Max Consecutive Rejections - 256 Mutation distance - 0.40 Large Mutation chance - 0.40 Num. samples - 1024 using Correlated multi-jitter Sampling - Equi-Importance Patch revision - 18 ![0539b.jpg](https://archive.blender.org/developer/F94276/0539b.jpg) The build I'm using is almost a month old (A MingW build from HolyEnigma), but the results you get from this patch are absolutely stunning with the right settings. It goes to show that even the worst case scenarios can be rendered now in around 3 days max (with simpler cases surely rendering quite a lot faster than that). Good work once again, I stand by my previous token award :)

lukas, waiting for 19 so i make a new build. :D

lukas, waiting for 19 so i make a new build. :D

Metropolis Sampling
Max Consecutive Rejections - 256
Mutation distance - 0.25
Large Mutation chance - 0.20
Num. samples - 1532(total until stopping) using Sobol
Sampling - Importance Equalization
Time - 29 hours
Patch revision - 18
0566.jpg

Only very minor touch up was needed, I would really like to see Lukas thinking of this getting this into master at least as an experimental feature as soon as he can, the potential improvements here are just too great to just see waste away.

I have also noted, the Multi-Jitter was crushed by Sobol in this case, lowering the max rejects doesn't have any rapid rise in bias introduction using that method and 256 then gives the best that the higher and lower values will give you.

I can't wait to see the next update and the next build containing it, keep up the good work. :)

Metropolis Sampling Max Consecutive Rejections - 256 Mutation distance - 0.25 Large Mutation chance - 0.20 Num. samples - 1532(total until stopping) using Sobol Sampling - Importance Equalization Time - 29 hours Patch revision - 18 ![0566.jpg](https://archive.blender.org/developer/F94603/0566.jpg) Only very minor touch up was needed, I would really like to see Lukas thinking of this getting this into master at least as an experimental feature as soon as he can, the potential improvements here are just too great to just see waste away. I have also noted, the Multi-Jitter was crushed by Sobol in this case, lowering the max rejects doesn't have any rapid rise in bias introduction using that method and 256 then gives the best that the higher and lower values will give you. I can't wait to see the next update and the next build containing it, keep up the good work. :)

Metropolis Sampling
Max Consecutive Rejections - 256
Mutation distance - 0.20
Large Mutation chance - 0.25
Num. samples - approx. 710(total until stopping) using Sobol
Sampling - Importance Equalization
Time - 70 hours
Patch revision - 18
0539.jpg

I know, same image as the first one I posted, but it looks to me like the very nature of Sobol eliminates pretty much every issue with bias and incomplete convergence that the Multi-Jitter version had. The time is the same, but the convergence is quite a bit better.

Now I guess it's back to vanilla builds for now since the MingW one is getting old, which in turns results in me hoping we get a nice big update soon like better adaptive sampling for the Metro integrator and the like. This pretty much solves the issue of trying to render scenes with complex indirect lighting in Cycles.

Metropolis Sampling Max Consecutive Rejections - 256 Mutation distance - 0.20 Large Mutation chance - 0.25 Num. samples - approx. 710(total until stopping) using Sobol Sampling - Importance Equalization Time - 70 hours Patch revision - 18 ![0539.jpg](https://archive.blender.org/developer/F95061/0539.jpg) I know, same image as the first one I posted, but it looks to me like the very nature of Sobol eliminates pretty much every issue with bias and incomplete convergence that the Multi-Jitter version had. The time is the same, but the convergence is quite a bit better. Now I guess it's back to vanilla builds for now since the MingW one is getting old, which in turns results in me hoping we get a nice big update soon like better adaptive sampling for the Metro integrator and the like. This pretty much solves the issue of trying to render scenes with complex indirect lighting in Cycles.

Added subscriber: @PRosendahl

Added subscriber: @PRosendahl
nAssembly commented 2014-06-24 00:10:28 +02:00 (Migrated from localhost:3001)

Added subscriber: @nAssembly

Added subscriber: @nAssembly

@Ace_Dragon, as for reference what is your PC hardware (processor-speed&cores / gpu /memory ),
It seams a complex scene, how many faces its made of ?, how much lights ?

And how do these render times compare to the other cycles render methods

@Ace_Dragon, as for reference what is your PC hardware (processor-speed&cores / gpu /memory ), It seams a complex scene, how many faces its made of ?, how much lights ? And how do these render times compare to the other cycles render methods
Member

Added subscriber: @VilemDuha

Added subscriber: @VilemDuha
Member

Regarding noise levels between stopped tiles in adaptive:

Could the sample-count be interpolated inside the tile? This means rendering a different sample count on each side of the tile - when one of the neighbouring tiles is stopped, the surrounding tiles don't render more samples on the connecting side, and the sample count increases towards the other non-stopped tiles.

It could also be done the other way - also tiles that are below the stopping condition render some extra samples towards the borders of non-stopped tiles.

Another idea is to render these extra samples after all tiles have stopped.

this would enable consistent grain/noise levels even at low samples...

Regarding noise levels between stopped tiles in adaptive: Could the sample-count be interpolated inside the tile? This means rendering a different sample count on each side of the tile - when one of the neighbouring tiles is stopped, the surrounding tiles don't render more samples on the connecting side, and the sample count increases towards the other non-stopped tiles. It could also be done the other way - also tiles that are below the stopping condition render some extra samples towards the borders of non-stopped tiles. Another idea is to render these extra samples after all tiles have stopped. this would enable consistent grain/noise levels even at low samples...
Author
Member

Just to show a sign of life here - I'm still working on the patch, but the full Metro rewrite necessary for the GPU is quite some work. It's working by now, but now regular PT crashes :/
Also, from what I can currently see, GPU Metro doesn't significantly outperform the CPU version. Considering I use a GTX780, that's quite disappointing, but I'll try to improve the performance.
On the positive side, the new code is already quite cleaner than the previous version, since now the whole Metro code is in the kernel instead of the device.
Another thing I'll try is to add "regular" pixel filtering for Metro mode. With PT, the current approach (shifting the camera point) works just as well, but I can imagine that for Metro this might be different. Also, this would filter functions with negative lobes (Sinc-Lanczos, Mitchell-Netravali etc.).

Just to show a sign of life here - I'm still working on the patch, but the full Metro rewrite necessary for the GPU is quite some work. It's working by now, but now regular PT crashes :/ Also, from what I can currently see, GPU Metro doesn't significantly outperform the CPU version. Considering I use a GTX780, that's quite disappointing, but I'll try to improve the performance. On the positive side, the new code is already quite cleaner than the previous version, since now the whole Metro code is in the kernel instead of the device. Another thing I'll try is to add "regular" pixel filtering for Metro mode. With PT, the current approach (shifting the camera point) works just as well, but I can imagine that for Metro this might be different. Also, this would filter functions with negative lobes (Sinc-Lanczos, Mitchell-Netravali etc.).

Well the custom MLT octane has it's slower as well by around 40% then normal PT. Not sure what kind of slowdown you get so can't say for sure.

Well the custom MLT octane has it's slower as well by around 40% then normal PT. Not sure what kind of slowdown you get so can't say for sure.

Lukas; if you're still following up on messages here...

Do you have any updates on this patch, if the GPU portion is looking to be too difficult right now, perhaps then you can at least get this done as a CPU-only feature for the initial release and work on the GPU stuff later?

If you can give an answer, that would be great.

Lukas; if you're still following up on messages here... Do you have any updates on this patch, if the GPU portion is looking to be too difficult right now, perhaps then you can at least get this done as a CPU-only feature for the initial release and work on the GPU stuff later? If you can give an answer, that would be great.
Member

Removed subscriber: @VilemDuha

Removed subscriber: @VilemDuha

Removed subscriber: @ThomasDinges

Removed subscriber: @ThomasDinges
Author
Member

Well, yes, I'm still working on it and Metro19 is nearly finished, the only problem remaining is a CUDA alignment problem, it seems that nvcc doesn't manage to align the strict itself. Once this is done (I already tracked it down to a variable, so most likely today), I'll upload it here (rebase to master etc. is already done).

Well, yes, I'm still working on it and Metro19 is nearly finished, the only problem remaining is a CUDA alignment problem, it seems that nvcc doesn't manage to align the strict itself. Once this is done (I already tracked it down to a variable, so most likely today), I'll upload it here (rebase to master etc. is already done).

So how is the CUDA performance?

So how is the CUDA performance?
Author
Member

Finally, Metro19 is finished!
So, what has changed?

  • GPU Metropolis! After some changes, it's considerably faster than CPU Metro (at least on my system) and has all features that CPU also has.

  • To do this, I rewrote (or rather re-organized) the whole sampler, moving it from the device code to the kernel. This is IMO a lot cleaner.

  • Importance Equalisation is currently broken, but of course I'll re-add it.

  • There now is an option the choose the Chain Number (number of independent samplers). On GPUs, this should be really high (I currently use 16384 or 32768, however, this results in 75MB / 150MB memory usage on the GPU for the samplers), on CPUs it should be a multiple of the thread number (1-2 chains per thread are just fine). Basically, for GPU performance, this is the performance equivalent of tile size.

  • I fixed a pretty massive bug in the cooldown phase, but I might have only added it while rewriting, so it probably wasn't present in metro_18

  • Error estimation was also rewritten, now again using variance instead of the even-samples pass. Reasons for this are: It's got a solid theoretical basis, there are no correlation issues with Sobol, it works just as well with Metro and it seems more solid. If anyone is interested in the derivation of the actual formulas used, I put in some comments, I hope it's clear enough.

  • Basically, it uses perceptually weighted standard deviation (I call it PWSD), multiplied by sqrt(N) with N being the number of samples. The reason for this is that the standard deviation convergence of MC methods like PT is O(1/sqrt(N)), so by increasing the sample count by N, the error lowers by sqrt(N). This correction is necessary so that the error goes to 0 as N goes to infinity.

  • You can see the estimated error in the Diffuse direct pass

  • There is a "Power mean exponent" setting which can be used to balance between using the average error of a tile and the maximum error. Basically, the higher this value is, the more important the maximum values get. 2 gives a regular average, while in theory infinity gives a pure maximum. However, due to numerical precision issues, going higher than 10 is probably a bad idea. I always use 4, it seems quite balanced.

  • The new system seems pretty stable to varying tile sizes, so it works just as well for GPU.

  • To see the system at work, just render an image without any option, one just with stopping and one with both options. You'll see that the error distribution gets flatter from image to image.

Some things to note:

  • Since the pass system now uses atomic writes for Metro and atomic float writes are compiler-specific, I can't guarantee that building will work on all compilers. If building fails on one, just tell me and I'll include a #define for it. Currently, I have GCC, ICC and MSVC code, the MSVC however is untested.
  • If you have an Intel CPU, consider using ICC since it currently optimizes the Metro code way better (I'll optimize the code itself, but I wanted to get this patch version out). On my Laptop (i7-4500M) Cycles runs 2,5 times as fast with it (compared to GCC 4.8, same options). On AMD CPUs, however, this is a bad idea: Due to biased dispatching , it's (for me) actually 60%*slower// than GCC.
  • Metro seems to sample less dense in the top 10% of the image. Currently, I have no idea whiy this is the case, I'll look into it.
  • The image filtering didn't actually help anything for quality, so I removed it.
  • Sorry for the tons of whitespace error in the diff, I'll clean it up.

Things still to do:

  • Adaptive Metro: It's probably ~20 lines of code, but it's linked to importance equalisation, which as said above is currently broken.
  • Progressive stopping mode: My idea for this is to render every tile for one mapping interval, then move them to a priority queue sorted by error and always render the one with the highest error for one mapping interval. Probably also not too much work.
  • Considering that the Metro code still was seriously buggy when I decided that tile-wise Metro wouldn't work, it might be interesting to give it another chance.
  • For CPU Metro, the status line is freezing.
  • For some .blends, Metro rendering freezes when you stop it, with an OpenEXR error on the console.
  • Optimization. I did some profiling and there is a lot of speed to gain.

The patch is pased on commit 49c73f.
metropolis_19.patch

Finally, Metro19 is finished! So, what has changed? - GPU Metropolis! After some changes, it's considerably faster than CPU Metro (at least on my system) and has all features that CPU also has. - To do this, I rewrote (or rather re-organized) the whole sampler, moving it from the device code to the kernel. This is IMO a lot cleaner. - Importance Equalisation is currently broken, but of course I'll re-add it. - There now is an option the choose the Chain Number (number of independent samplers). On GPUs, this should be really high (I currently use 16384 or 32768, however, this results in 75MB / 150MB memory usage on the GPU for the samplers), on CPUs it should be a multiple of the thread number (1-2 chains per thread are just fine). Basically, for GPU performance, this is the performance equivalent of tile size. - I fixed a pretty massive bug in the cooldown phase, but I might have only added it while rewriting, so it probably wasn't present in metro_18 - Error estimation was also rewritten, now again using variance instead of the even-samples pass. Reasons for this are: It's got a solid theoretical basis, there are no correlation issues with Sobol, it works just as well with Metro and it seems more solid. If anyone is interested in the derivation of the actual formulas used, I put in some comments, I hope it's clear enough. - Basically, it uses perceptually weighted standard deviation (I call it PWSD), multiplied by sqrt(N) with N being the number of samples. The reason for this is that the standard deviation convergence of MC methods like PT is O(1/sqrt(N)), so by increasing the sample count by N, the error lowers by sqrt(N). This correction is necessary so that the error goes to 0 as N goes to infinity. - You can see the estimated error in the Diffuse direct pass - There is a "Power mean exponent" setting which can be used to balance between using the average error of a tile and the maximum error. Basically, the higher this value is, the more important the maximum values get. 2 gives a regular average, while in theory infinity gives a pure maximum. However, due to numerical precision issues, going higher than 10 is probably a bad idea. I always use 4, it seems quite balanced. - The new system seems pretty stable to varying tile sizes, so it works just as well for GPU. - To see the system at work, just render an image without any option, one just with stopping and one with both options. You'll see that the error distribution gets flatter from image to image. Some things to note: - Since the pass system now uses atomic writes for Metro and atomic float writes are compiler-specific, I can't guarantee that building will work on all compilers. If building fails on one, just tell me and I'll include a #define for it. Currently, I have GCC, ICC and MSVC code, the MSVC however is untested. - If you have an Intel CPU, consider using ICC since it currently optimizes the Metro code way better (I'll optimize the code itself, but I wanted to get this patch version out). On my Laptop (i7-4500M) Cycles runs 2,5 times as fast with it (compared to GCC 4.8, same options). On AMD CPUs, however, this is a bad idea: Due to [biased dispatching ](http:*www.agner.org/optimize/blog/read.php?i=121#49), it's (for me) actually 60%*slower// than GCC. - Metro seems to sample less dense in the top 10% of the image. Currently, I have no idea whiy this is the case, I'll look into it. - The image filtering didn't actually help anything for quality, so I removed it. - Sorry for the tons of whitespace error in the diff, I'll clean it up. Things still to do: - Adaptive Metro: It's probably ~20 lines of code, but it's linked to importance equalisation, which as said above is currently broken. - Progressive stopping mode: My idea for this is to render every tile for one mapping interval, then move them to a priority queue sorted by error and always render the one with the highest error for one mapping interval. Probably also not too much work. - Considering that the Metro code still was seriously buggy when I decided that tile-wise Metro wouldn't work, it might be interesting to give it another chance. - For CPU Metro, the status line is freezing. - For some .blends, Metro rendering freezes when you stop it, with an OpenEXR error on the console. - Optimization. I did some profiling and there is a lot of speed to gain. The patch is pased on commit 49c73f. [metropolis_19.patch](https://archive.blender.org/developer/F97398/metropolis_19.patch)

Hi Lucas, _19 compiles fine on my system, CPU is working but get error during GPU kernel compilation.

/daten/blender-git/build/bin/2.71/scripts/addons/cycles/kernel/kernel_metropolis.h:20:23: fatal error: util_hash.h: Datei oder Verzeichnis nicht gefunden

#include "util_hash.h"

Opensuse 13.1/64
Intel i5 3770K
GTX 760 4 GB (Display)
GTX 560Ti 1.28 GB 448 Cores
Driver 331.67

Thank you, mib

Hi Lucas, _19 compiles fine on my system, CPU is working but get error during GPU kernel compilation. ``` /daten/blender-git/build/bin/2.71/scripts/addons/cycles/kernel/kernel_metropolis.h:20:23: fatal error: util_hash.h: Datei oder Verzeichnis nicht gefunden ``` #include "util_hash.h" Opensuse 13.1/64 Intel i5 3770K GTX 760 4 GB (Display) GTX 560Ti 1.28 GB 448 Cores Driver 331.67 Thank you, mib
Author
Member

@mib2berlin That's really weird since ../util is in both CMakeLists and SConscript. Are you building with CMake or SCons? Could you try a new build from scratch?

@mib2berlin That's really weird since ../util is in both CMakeLists and SConscript. Are you building with CMake or SCons? Could you try a new build from scratch?

Added subscriber: @hdunderscore

Added subscriber: @hdunderscore

Testing this out on msvc 2013, I made a few changes to get it to compile:

intern/cycles/device/device_cuda.cpp:
  • offset = align_up(offset, __alignof(rtile.w));
+		offset = align_up(offset, __alignof(int));

intern/cycles/kernel/kernel_compat_cpu.h:

#elif defined(_WIN32)

  • #define AtomicCASD(x, y, z) InterlockedCompareExchange64(x, z, y)
  • #define AtomicCASF(x, y, z) InterlockedCompareExchange(x, z, y)
+#define AtomicCASD(x, y, z) _InterlockedCompareExchange64(x, z, y)
+#define AtomicCASF(x, y, z) _InterlockedCompareExchange(x, z, y)
  • old = AtomicCASD(address_as_ull, assumed, newV);
+		old = AtomicCASD((volatile long long int *)address_as_ull, assumed, newV);

  • old = AtomicCASF(address_as_ull, assumed, newV);
+        old = AtomicCASF((volatile long *)address_as_ull, assumed, newV);

I also had to disable OSL, there was a difficult to trace undefined reference bug.

Very nice work !

Testing this out on msvc 2013, I made a few changes to get it to compile: ``` intern/cycles/device/device_cuda.cpp: ``` - offset = align_up(offset, __alignof(rtile.w)); ``` + offset = align_up(offset, __alignof(int)); intern/cycles/kernel/kernel_compat_cpu.h: ``` #elif defined(_WIN32) - #define AtomicCASD(x, y, z) InterlockedCompareExchange64(x, z, y) - #define AtomicCASF(x, y, z) InterlockedCompareExchange(x, z, y) ``` +#define AtomicCASD(x, y, z) _InterlockedCompareExchange64(x, z, y) +#define AtomicCASF(x, y, z) _InterlockedCompareExchange(x, z, y) ``` - old = AtomicCASD(address_as_ull, assumed, newV); ``` + old = AtomicCASD((volatile long long int *)address_as_ull, assumed, newV); ``` - old = AtomicCASF(address_as_ull, assumed, newV); ``` + old = AtomicCASF((volatile long *)address_as_ull, assumed, newV); ``` I also had to disable OSL, there was a difficult to trace undefined reference bug. Very nice work !

Hi lukasstockner97, try with clean build directory but same error.
After add complete path to util_hash.h Cuda kernel start to build but stop with:

Compiling CUDA kernel ...
"/usr/local/cuda/bin/nvcc" -arch=sm_20 -m64 --cubin "/daten/blender-git/build/bin/2.71/scripts/addons/cycles/kernel/kernel.cu" -o "/home/pepo/.config/blender/2.71/cache/cycles_kernel_sm20_D233D0AAE2C08F54D31BCF819203CB63.cubin" --ptxas-options="-v" -I"/daten/blender-git/build/bin/2.71/scripts/addons/cycles/kernel" -DNVCC -D__KERNEL_CUDA_VERSION__=60
ptxas fatal   : More than 128 textures used in entry function 'kernel_cuda_metropolis_first_pass'
CUDA kernel compilation failed, see console for details.

Refer to the Cycles GPU rendering documentation for possible solutions:
http://wiki.blender.org/index.php/Doc:2.6/Manual/Render/Cycles/GPU_Rendering

Thank you for the fast help, mib

EDIT: The kernel for my GTX 760 sm_30 is compiling and working.
Error is for GTX 560Ti 448 sm_20.
The memory consumptionon GPU is huge, 1.1 GB for BMW!
THX

Hi lukasstockner97, try with clean build directory but same error. After add complete path to util_hash.h Cuda kernel start to build but stop with: ``` Compiling CUDA kernel ... "/usr/local/cuda/bin/nvcc" -arch=sm_20 -m64 --cubin "/daten/blender-git/build/bin/2.71/scripts/addons/cycles/kernel/kernel.cu" -o "/home/pepo/.config/blender/2.71/cache/cycles_kernel_sm20_D233D0AAE2C08F54D31BCF819203CB63.cubin" --ptxas-options="-v" -I"/daten/blender-git/build/bin/2.71/scripts/addons/cycles/kernel" -DNVCC -D__KERNEL_CUDA_VERSION__=60 ptxas fatal : More than 128 textures used in entry function 'kernel_cuda_metropolis_first_pass' CUDA kernel compilation failed, see console for details. Refer to the Cycles GPU rendering documentation for possible solutions: http://wiki.blender.org/index.php/Doc:2.6/Manual/Render/Cycles/GPU_Rendering ``` Thank you for the fast help, mib EDIT: The kernel for my GTX 760 sm_30 is compiling and working. Error is for GTX 560Ti 448 sm_20. The memory consumptionon GPU is huge, 1.1 GB for BMW! THX
Author
Member

@hdunderscore Thanks, I'll add it in the next version!
@mib2berlin This error comes from the 128-textures limit on devices with sm20 or lower, since MCQMC uses another texture, one of the image textures has to be moved to the sm30+ code. I forgot to do so since I only use sm30 and sm35, sorry for that. For a quick fix, move the "__tex_image_098"-line in intern/cycles/kernel/svm/svm_image.h 2 lines down into the "#if defined ..."-block.
Regarding memory usage: Well, the Metro currently requires lots of memory, I am aware of this and will try to reduce it (this falls under optimization). One solution would be to disable lazy sample generation, another one to always re-mutate from the last large-step. This would give performance problems, however, when there is a large chain of rejected large-steps, and especially on GPUs, where every warp has to wait for all threads to finish the mutations, this would probably be a quite bad trade-off.
A quick fix for this is to reduce the bounce count, it nothing else helps and you're getting OOMs, reduce the chain count.

@hdunderscore Thanks, I'll add it in the next version! @mib2berlin This error comes from the 128-textures limit on devices with sm20 or lower, since MCQMC uses another texture, one of the image textures has to be moved to the sm30+ code. I forgot to do so since I only use sm30 and sm35, sorry for that. For a quick fix, move the "__tex_image_098"-line in intern/cycles/kernel/svm/svm_image.h 2 lines down into the "#if defined ..."-block. Regarding memory usage: Well, the Metro currently requires lots of memory, I am aware of this and will try to reduce it (this falls under optimization). One solution would be to disable lazy sample generation, another one to always re-mutate from the last large-step. This would give performance problems, however, when there is a large chain of rejected large-steps, and especially on GPUs, where every warp has to wait for all threads to finish the mutations, this would probably be a quite bad trade-off. A quick fix for this is to reduce the bounce count, it nothing else helps and you're getting OOMs, reduce the chain count.

Thank you, with changes in svm_image.h GTX 560Ti 448 start rendering.
I cant render with both cards and it seams sometimes the 560 render and sometimes the 760.
Over 6000 M. Chains GTX 560 stop working without error.
The render results also very different on GTX 560.

http://www.pasteall.org/pic/73837

http://www.pasteall.org/pic/73838

http://www.pasteall.org/pic/73839

http://www.pasteall.org/pic/73840

Very interessting, cheers, mib

Thank you, with changes in svm_image.h GTX 560Ti 448 start rendering. I cant render with both cards and it seams sometimes the 560 render and sometimes the 760. Over 6000 M. Chains GTX 560 stop working without error. The render results also very different on GTX 560. http://www.pasteall.org/pic/73837 http://www.pasteall.org/pic/73838 http://www.pasteall.org/pic/73839 http://www.pasteall.org/pic/73840 Very interessting, cheers, mib

someone requested a build..
here mingw64-cmake 7-12-14 metropolis19

with player, no CUDA or OSL

http://www.mediafire.com/download/84vzfy99a5h6n41/Blender_2.71_mingw64_cmake_7-12-14_metropolis19.7z

someone requested a build.. here mingw64-cmake 7-12-14 metropolis19 with player, no CUDA or OSL http://www.mediafire.com/download/84vzfy99a5h6n41/Blender_2.71_mingw64_cmake_7-12-14_metropolis19.7z

Added subscriber: @jemonn

Added subscriber: @jemonn

Initial thoughts....

  • -Complex scenes seem to crash the Metropolis sampler now (those that need around a gig or RAM or more), can't really determine why it crashes. Sometimes it will work with very small image sizes if the image buffer has an image being overwritten instead of empty, but avoiding it is about impossible with larger sizes (tested with the MingW and VC2013 builds). I will also note that all crashes occur when the samples are about to be drawn to the image buffer after the first pass is completed.
  • -Saving a scene with the Metropolis sampler selected, then exiting and restarting Blender, seems to permanently turn on the progressive refine option for that specific scene, at least for me it's stuck on the 'on' position for that scene even when the box is unchecked.
  • -I don't exactly see why someone would really need the sampler count option, running more than one sampler just increases noise and can even lead to progress quickly stopping altogether (check the default cube scene).

So the Metropolis functionality seems to have seen a few regressions as of rev. 19,, but the adaptive sampling seems intact after all of the changes (EDIT; Mostly, just found that a high min-bounce number will also crash Blender on the same scenes that crash Metro)..

Hope you can fix these.

EDIT: Updated information after more testing

Initial thoughts.... - -Complex scenes seem to crash the Metropolis sampler now (those that need around a gig or RAM or more), can't really determine why it crashes. Sometimes it will work with very small image sizes if the image buffer has an image being overwritten instead of empty, but avoiding it is about impossible with larger sizes (tested with the MingW and VC2013 builds). I will also note that all crashes occur when the samples are about to be drawn to the image buffer after the first pass is completed. - -Saving a scene with the Metropolis sampler selected, then exiting and restarting Blender, seems to permanently turn on the *progressive refine* option for that specific scene, at least for me it's stuck on the 'on' position for that scene even when the box is unchecked. - -I don't exactly see why someone would really need the *sampler count* option, running more than one sampler just increases noise and can even lead to progress quickly stopping altogether (check the default cube scene). So the Metropolis functionality seems to have seen a few regressions as of rev. 19,, but the adaptive sampling seems intact after all of the changes (EDIT; Mostly, just found that a high min-bounce number will also crash Blender on the same scenes that crash Metro).. Hope you can fix these. EDIT: Updated information after more testing

(hopefully) built a version of R19 with CUDA (and GPU SSS and Volumetrics) compiled with VS2013/cmake. Doesn't have OSL or Player.

It works on my computer but as it is my first build, not sure whether I've zipped everything up that is needed.

http://www.mediafire.com/download/bq551h24xxdym9e/Release.zip

(hopefully) built a version of R19 with CUDA (and GPU SSS and Volumetrics) compiled with VS2013/cmake. Doesn't have OSL or Player. It works on my computer but as it is my first build, not sure whether I've zipped everything up that is needed. http://www.mediafire.com/download/bq551h24xxdym9e/Release.zip

While playing around with settings i noticed something strange using R19 Jemonn's build (thanks), on my i7 octocore (no GPU).

I had a simple blend with 3 lights and some musical notes as objects > https:*dl.dropboxusercontent.com/u/54767531/music.blend
It does render a first preview, but then later it doesn't update (although the rendering system stays busy).
Well the setting i used are probably not good for a nice render, but its strange that the system kept busy while no improvement was made.

With other settings it worked just fine, so i just post this as maybe there is something going wrong her with metropolis or the adaptive sampling.

While playing around with settings i noticed something strange using R19 Jemonn's build (thanks), on my i7 octocore (no GPU). I had a simple blend with 3 lights and some musical notes as objects > [https:*dl.dropboxusercontent.com/u/54767531/music.blend ](https:*dl.dropboxusercontent.com/u/54767531/music.blend) It does render a first preview, but then later it doesn't update (although the rendering system stays busy). Well the setting i used are probably not good for a nice render, but its strange that the system kept busy while no improvement was made. With other settings it worked just fine, so i just post this as maybe there is something going wrong her with metropolis or the adaptive sampling.

Hey Lukas; I don't know how much it would help, but I found a potentially useful resource that has free .blend file scenes (to verify that the adaptive and metropolis sampling code works with as many cases as possible).
http://www.emirage.org/category/free-stuff/

For example, if any one of these scenes crash with the metropolis sampler being used, you know that there's a regression or two since Patch 18.

Hey Lukas; I don't know how much it would help, but I found a potentially useful resource that has free .blend file scenes (to verify that the adaptive and metropolis sampling code works with as many cases as possible). http://www.emirage.org/category/free-stuff/ For example, if any one of these scenes crash with the metropolis sampler being used, you know that there's a regression or two since Patch 18.

Hi Luckas.
First of all, thanks for all that job, that's cool !

More constructive remark: i tested the @holyenigma 's build, it crash when i render a scene with metro and render border enabeled.

@jemonn: your build crash on my computer, missing a file: "VCOMP120.DLL"

Hi Luckas. First of all, thanks for all that job, that's cool ! More constructive remark: i tested the @holyenigma 's build, it crash when i render a scene with metro and render border enabeled. @jemonn: your build crash on my computer, missing a file: "VCOMP120.DLL"

@Lapineige for some reason, it didn't copy it to my build folder, but it still works for me? anyway, I've uploaded it again, copying the contents of my build to a copy of the 2.71 folder, which should include all the files but as it works for me, I've got no idea whether it's fixed everything.

Using my build, it doesn't crash when using metro and a render border is used.

(Hopefully) fixed v19 link: http://www.mediafire.com/download/0jkmzgnk2qim00q/Blender.7z

I've been leaving all the settings at their defaults, are there any suggestions to improve performance? Is leaving sampler count at 0 (auto) okay, or should I manually change it?

@Lapineige for some reason, it didn't copy it to my build folder, but it still works for me? anyway, I've uploaded it again, copying the contents of my build to a copy of the 2.71 folder, which should include all the files but as it works for me, I've got no idea whether it's fixed everything. Using my build, it doesn't crash when using metro and a render border is used. (Hopefully) fixed v19 link: http://www.mediafire.com/download/0jkmzgnk2qim00q/Blender.7z I've been leaving all the settings at their defaults, are there any suggestions to improve performance? Is leaving sampler count at 0 (auto) okay, or should I manually change it?

Some unfortunate news about the adaptive sampling in rev. 19.

Never mind about it remaining intact from rev. 18, just found that adaptive sampling on a complex scene will also crash Blender when the adaptive stopping kicks in for the first time (providing it takes a while to get there). I can also get Blender to crash early on if I use a high number for the min bounces setting (10 or more).

The crashing issues mean the patch is unusable once a scene gets to a certain degree of complexity, back to rendering it in a vanilla Master build I guess for now. Don't forget that those eMirage scenes can used for testing of your patch and seeing if things crash for you as well.

Some unfortunate news about the adaptive sampling in rev. 19. Never mind about it remaining intact from rev. 18, just found that adaptive sampling on a complex scene will also crash Blender when the adaptive stopping kicks in for the first time (providing it takes a while to get there). I can also get Blender to crash early on if I use a high number for the *min bounces* setting (10 or more). The crashing issues mean the patch is unusable once a scene gets to a certain degree of complexity, back to rendering it in a vanilla Master build I guess for now. Don't forget that those eMirage scenes can used for testing of your patch and seeing if things crash for you as well.
Author
Member

Sorry about all the crashes...
Metro20 is making progress, in 2-3 days it should be done. The Metro crashes should be gone with it, the adaptive crashes are quite strange. Could you post a crashlog (the ones from the Temp folder), and, if possible, use a debug build so the place where it crashes can be seen better.
For me, Metro19 didn't crash once, even with scenes like the Lego bulldozer or the Pavilion from eMirage. I'll run some Valgrind tests today to search for wrong memory accesses. By the way, is it expected that cuda-memcheck crashes with Blender?
Regarding the GPU Metro problems: With Metro20, Multi-GPU will work, but the error on the GTX 560 is strange. Well, we'll see if it works with Metro20.
The min. bounces is strange, I'll have a look.

Sorry about all the crashes... Metro20 is making progress, in 2-3 days it should be done. The Metro crashes should be gone with it, the adaptive crashes are quite strange. Could you post a crashlog (the ones from the Temp folder), and, if possible, use a debug build so the place where it crashes can be seen better. For me, Metro19 didn't crash once, even with scenes like the Lego bulldozer or the Pavilion from eMirage. I'll run some Valgrind tests today to search for wrong memory accesses. By the way, is it expected that cuda-memcheck crashes with Blender? Regarding the GPU Metro problems: With Metro20, Multi-GPU will work, but the error on the GTX 560 is strange. Well, we'll see if it works with Metro20. The min. bounces is strange, I'll have a look.

@jemonn: ok the new build works ! Thanks !
But even this time rendering with MLT and render border enabeled do not crash Blender, but still fails.

@jemonn: ok the new build works ! Thanks ! But even this time rendering with MLT and render border enabeled do not crash Blender, but still fails.

Lukas, testing the VC2013 build I have with your patch, there's a chance it may just be MingW being unstable compared to the official platforms (MingW being the only platform Holy Enigma will build for).

I couldn't get the crashes I got early on with the other build in this case, so it may be MingW's fault and not the fault of your code.

If I do indeed find it a false alarm, then I'm sorry for that.


EDIT: Okay, even the use of 64 threads doesn't crash adaptive sampling with the VC2013 build, so it really is just MingW doing its thing as the least stable platform you can possibly build with. Sorry for that.

Lukas, testing the VC2013 build I have with your patch, there's a chance it may just be MingW being unstable compared to the official platforms (MingW being the only platform Holy Enigma will build for). I couldn't get the crashes I got early on with the other build in this case, so it may be MingW's fault and not the fault of your code. If I do indeed find it a false alarm, then I'm sorry for that. ------- EDIT: Okay, even the use of 64 threads doesn't crash adaptive sampling with the VC2013 build, so it really is just MingW doing its thing as the least stable platform you can possibly build with. Sorry for that.

Another question.

Do you by any chance know what the unit is for the adaptive map update value? I ask because I couldn't really obtain a difference say, between a value of 5 and one of 50, but then I found that much larger values seem to be necessary if I wanted to see a difference when using a lower error tolerance along with the adaptive map.

Perhaps if one could make sure that the value was easy for the user to understand it would be different, say, update the adaptive map every N number of passes instead if it's not being done like that already.

Another question. Do you by any chance know what the unit is for the adaptive map update value? I ask because I couldn't really obtain a difference say, between a value of 5 and one of 50, but then I found that much larger values seem to be necessary if I wanted to see a difference when using a lower error tolerance along with the adaptive map. Perhaps if one could make sure that the value was easy for the user to understand it would be different, say, update the adaptive map every N number of passes instead if it's not being done like that already.
craigar commented 2014-07-16 03:25:24 +02:00 (Migrated from localhost:3001)

MLT 19 (holyenigma MingW build) really good fast CPU renders (once I messed with it a little, and followed Lucas' suggestions) on a "physically" simple scene, with very complex "real" spectral/caustic lighting, using MLT 19 holyenigma MingW
A useable "draft" in Around 10 minutes 1920x1080 50 passes, adaptive sampling at 2.5.

Before using this MLT 19 I could NOT GET a good reasonably, noise free render any other way on this scene, I tried many combinations on both PT and BPT, at least 200 times. Tried MLT 18 and it crashed a lot on other scenes, maybe I'll try it on this scene just to see ....

Image and .blend file here
BA Thread: Cycles MLT patch
i7 2600 3.4ghz, Windows 7 64-bit

MLT 19 (holyenigma MingW build) really good fast CPU renders (once I messed with it a little, and followed Lucas' suggestions) on a "physically" simple scene, with very complex "real" spectral/caustic lighting, using MLT 19 holyenigma MingW A useable "draft" in Around 10 minutes 1920x1080 50 passes, adaptive sampling at 2.5. Before using this MLT 19 I could NOT GET a good reasonably, noise free render any other way on this scene, I tried many combinations on both PT and BPT, at least 200 times. Tried MLT 18 and it crashed a lot on other scenes, maybe I'll try it on this scene just to see .... Image and .blend file here [BA Thread: Cycles MLT patch](http://www.blenderartists.org/forum/showthread.php?329089-Cycles-MLT-patch/page12) i7 2600 3.4ghz, Windows 7 64-bit
craigar commented 2014-07-16 04:23:45 +02:00 (Migrated from localhost:3001)

rendered scene above at 4 times the samples, was at 50 but once it gets to about 70 passes it starts showing "black fireflies" - even tried it on "Suzanne" with the same spectral node AND lots of ambient light - same problem?

also tried @jemonn build of 19 w GPU (GTX580 3 GB), and the GPU mode is about 30% slower on this file, than using CPU = i7 2600 3.4ghz, Windows 7 64-bit 12 GB

rendered scene above at 4 times the samples, was at 50 but once it gets to about 70 passes it starts showing "black fireflies" - even tried it on "Suzanne" with the same spectral node AND lots of ambient light - same problem? also tried @jemonn build of 19 w GPU (GTX580 3 GB), and the GPU mode is about 30% slower on this file, than using CPU = i7 2600 3.4ghz, Windows 7 64-bit 12 GB

Added subscriber: @blakenator

Added subscriber: @blakenator

Just an FYI: out of the box patch from master to metropolis_19 yields "patch does not apply" and "trailing white space" errors and when compiling:

blender\intern\cycles\kernel\kernel_compat_cpu.h(260): error C3861: 'InterlockedCompareExchange64': identifier not found
blender\intern\cycles\kernel\kernel_compat_cpu.h(272): error C3861: 'InterlockedCompareExchange': identifier not found
blender\intern\cycles\device\..\kernel\kernel_compat_cpu.h(260): error C3861: 'InterlockedCompareExchange64': identifier not found (blender\intern\cycles\device\device_cpu.cpp)
blender\intern\cycles\device\..\kernel\kernel_compat_cpu.h(272): error C3861: 'InterlockedCompareExchange': identifier not found (blender\intern\cycles\device\device_cpu.cpp)
blender\intern\cycles\device\device_cuda.cpp(716): error C2143: syntax error : missing ')' before '.'
blender\intern\cycles\device\device_cuda.cpp(716): error C2228: left of '.w' must have class/struct/union type is 'int'
blender\intern\cycles\device\device_cuda.cpp(716): error C2059: syntax error : ')'

My machine (using the term lightly):
Windows 7
VS 2013 Win32 (builds master just fine)
CMake
portablegit (Github) with tortoisesvn (idk if it makes a difference)

Hope this helps

Just an FYI: out of the box patch from master to metropolis_19 yields "patch does not apply" and "trailing white space" errors and when compiling: ``` blender\intern\cycles\kernel\kernel_compat_cpu.h(260): error C3861: 'InterlockedCompareExchange64': identifier not found blender\intern\cycles\kernel\kernel_compat_cpu.h(272): error C3861: 'InterlockedCompareExchange': identifier not found blender\intern\cycles\device\..\kernel\kernel_compat_cpu.h(260): error C3861: 'InterlockedCompareExchange64': identifier not found (blender\intern\cycles\device\device_cpu.cpp) blender\intern\cycles\device\..\kernel\kernel_compat_cpu.h(272): error C3861: 'InterlockedCompareExchange': identifier not found (blender\intern\cycles\device\device_cpu.cpp) blender\intern\cycles\device\device_cuda.cpp(716): error C2143: syntax error : missing ')' before '.' blender\intern\cycles\device\device_cuda.cpp(716): error C2228: left of '.w' must have class/struct/union type is 'int' blender\intern\cycles\device\device_cuda.cpp(716): error C2059: syntax error : ')' ``` My machine (using the term lightly): Windows 7 VS 2013 Win32 (builds master just fine) CMake portablegit (Github) with tortoisesvn (idk if it makes a difference) Hope this helps

I've just tried the test file @craigar linked to above and something is really wrong with importance equalisation (but we all knew that anyway).

However, I have slightly different results with my CPU and GPU?


cpu.jpg
i7 4500U - 400 samples, importance equalisation off, 4 chains. (04:53:63)


gpu.jpg
Geforce GT 750M - 400 samples, importance equalisation off, 16000 chains. (08:53:28)


Also GPU rendering is really slow for me, might just be my build as @craigar said it was slower for GPU, but there usually isn't this much difference in the speeds.

I've just tried the test file @craigar linked to above and something is really wrong with importance equalisation (but we all knew that anyway). However, I have slightly different results with my CPU and GPU? --- ![cpu.jpg](https://archive.blender.org/developer/F98157/cpu.jpg) i7 4500U - 400 samples, importance equalisation off, 4 chains. (04:53:63) --- ![gpu.jpg](https://archive.blender.org/developer/F98159/gpu.jpg) Geforce GT 750M - 400 samples, importance equalisation off, 16000 chains. (08:53:28) --- Also GPU rendering is really slow for me, might just be my build as @craigar said it was slower for GPU, but there usually isn't this much difference in the speeds.
craigar commented 2014-07-17 02:16:44 +02:00 (Migrated from localhost:3001)

With the same exact test file I posted here, it takes 2 1/2 times as long to render with MLT 18 -but it is much higher quality - and no black fireflies ? Also MLT 18 shows way more of the Translucent BSDF that I slightly mixed into the prism, to give it some whitish glow, so I brought down the " fac " from 0.05 to 0.03 in the mix shader - and left it there in all of these renders so you can see the difference .

BA - "Prism test" MLT build 18 slower - but better than Build 19

With the same exact test file I posted here, it takes 2 1/2 times as long to render with MLT 18 -but it is much higher quality - and no black fireflies ? Also MLT 18 shows way more of the Translucent BSDF that I slightly mixed into the prism, to give it some whitish glow, so I brought down the " fac " from 0.05 to 0.03 in the mix shader - and left it there in all of these renders so you can see the difference . [BA - "Prism test" MLT build 18 slower - but better than Build 19 ](http://www.blenderartists.org/forum/showthread.php?329089-Cycles-MLT-patch&p=2687926#post2687926)
Author
Member

New patch version 20 is done, the two big changes are:

  • Progressive adaptive mode: If you enable both adaptive stopping and progressive rendering, it will render every tile for one mapping interval in a prepass. Afterwards, the tiles are sorted by error, and the tiles with highest errors are rendered and then reinserted into the list. Once the worst tile is below the error threshold, it stops. In the status line, the currently worst error is displayed. This works both on CPU and GPU, in theory it should also work with Metro, but Metro error estimation is still quite unstable (Variance estimation in weighted means is really annoying). The performance seems fine for me: Rendering without Progressive took 1:31 min, with Progressive it was 1:33 min (BMW scene). The only thing missing are the orange markers around the current tiles, but I still have no clue how the currently active tiles are passed to the Render API, so now, when I just enable the highlighting, every tile gets highlighted :/
  • Metro is now working with tiles again! The basic problem, different mean intensities and therefore visible tile borders has become slower, but is still there. So, how to solve it? The solusion I chose is quite simple: Run a warmup prepass to calculate it in advance instead of calculating it on the fly and therefore use wrong values in the beginning. This results in a new parameter: The number of samples to run for warmup. For quite simple scenes (BMW, Pavillon etc.), the default of 10 is just fine. If you get visible borders (in the first few seconds, these are expected since the chains might not have found the high-contribution paths yet, so give them a few seconds), increase the warmup samples: For the Villa scene (from PBRT), I ended up with 3000 samples at 32x32 tiles, but I modified this scene to be a worst-case example: High occlusion, lamps in glass, three glass panes in the windows (!)). Also, the bigger the tile is, the fewer warmup samples you need. Note that this also eliminates the black fireflies and allows Multi-GPU (one tile per GPU).

Smaller changes are:

  • Instead of a separate filter dimension, the Metro sampler now uses the fractional part of the pixel coordinate. IMO it gives cleaner filtering.
  • I added the algorithm from the paper "Automatic Parameter Control for Metropolis Light Transport", which automatically chooses a large step probability. This is especially helpful now since every tile might have a different optimum value. In the (distant) future, this might also be a good criterion for automatically choosing between Metro and PT: A value > 0.9 suggests that there is no real advantage in Metro (for example, this happens in the BMW scene for every tile except for the glossy reflection in the lower left corner and the headlights).
  • The unit for image mutation range was changed from "percentage of the image" to pixels. This means that the old default value of 0.1 will give extremely noisy results, I recommend something around 20-30. Remember to always change this from the old default!
  • You can't really compare render times to older patches since I changes the "work per sample"
  • The MSVC missing underscore was fixed.
  • The util_hash.h problem was fixed, it occurred when Cycles tried to build the CUDA kernel during runtime. I found this one by accident when my compile-time kernel was still at sm_20 from fixing the texture problem.
  • Speaking of it: The texture limit problem is also fixed.
  • Viewport rendering is broken (again). I'll try to fix this asap.
  • Importance equalisation is still broken.

So, now for the bad news: Since I have a lot to do in the next time, I won't be able to work on new features until around mid september. If there is some bug, of course I'll try to fix it (like the viewport bug). I'm really sorry for this, but I just won't have the time to work much on the patch. So, it would be great if you could extensively test the new version to find any remaining bugs, and also if any developers could probably just look at the code to find remaining problems (apart from bad coding style and whitespace errors :D).

Attached are some images I rendered while testing.
BMW_Metro_50min_20_10_.png
Sintel_10h_20.png
Villa_14h.png
Pool_4h_20.png (Note the visible tiles, I didn't use enough warmup)

So, here's the patch, created against 508e0a:
metropolis_20.patch

New patch version 20 is done, the two big changes are: - Progressive adaptive mode: If you enable both adaptive stopping and progressive rendering, it will render every tile for one mapping interval in a prepass. Afterwards, the tiles are sorted by error, and the tiles with highest errors are rendered and then reinserted into the list. Once the worst tile is below the error threshold, it stops. In the status line, the currently worst error is displayed. This works both on CPU and GPU, in theory it should also work with Metro, but Metro error estimation is still quite unstable (Variance estimation in weighted means is really annoying). The performance seems fine for me: Rendering without Progressive took 1:31 min, with Progressive it was 1:33 min (BMW scene). The only thing missing are the orange markers around the current tiles, but I still have no clue how the currently active tiles are passed to the Render API, so now, when I just enable the highlighting, *every* tile gets highlighted :/ - Metro is now working with tiles again! The basic problem, different mean intensities and therefore visible tile borders has become slower, but is still there. So, how to solve it? The solusion I chose is quite simple: Run a warmup prepass to calculate it in advance instead of calculating it on the fly and therefore use wrong values in the beginning. This results in a new parameter: The number of samples to run for warmup. For quite simple scenes (BMW, Pavillon etc.), the default of 10 is just fine. If you get visible borders (in the first few seconds, these are expected since the chains might not have found the high-contribution paths yet, so give them a few seconds), increase the warmup samples: For the Villa scene (from PBRT), I ended up with 3000 samples at 32x32 tiles, but I modified this scene to be a worst-case example: High occlusion, lamps in glass, three glass panes in the windows (!)). Also, the bigger the tile is, the fewer warmup samples you need. Note that this also eliminates the black fireflies and allows Multi-GPU (one tile per GPU). Smaller changes are: - Instead of a separate filter dimension, the Metro sampler now uses the fractional part of the pixel coordinate. IMO it gives cleaner filtering. - I added the algorithm from the paper "Automatic Parameter Control for Metropolis Light Transport", which automatically chooses a large step probability. This is especially helpful now since every tile might have a different optimum value. In the (distant) future, this might also be a good criterion for automatically choosing between Metro and PT: A value > 0.9 suggests that there is no real advantage in Metro (for example, this happens in the BMW scene for every tile except for the glossy reflection in the lower left corner and the headlights). - The unit for image mutation range was changed from "percentage of the image" to pixels. **This means that the old default value of 0.1 will give extremely noisy results, I recommend something around 20-30. Remember to always change this from the old default!** - You can't really compare render times to older patches since I changes the "work per sample" - The MSVC missing underscore was fixed. - The util_hash.h problem was fixed, it occurred when Cycles tried to build the CUDA kernel during runtime. I found this one by accident when my compile-time kernel was still at sm_20 from fixing the texture problem. - Speaking of it: The texture limit problem is also fixed. - Viewport rendering is broken (again). I'll try to fix this asap. - Importance equalisation is still broken. So, now for the bad news: Since I have a lot to do in the next time, I won't be able to work on new features until around mid september. If there is some bug, of course I'll try to fix it (like the viewport bug). I'm really sorry for this, but I just won't have the time to work much on the patch. So, it would be great if you could extensively test the new version to find any remaining bugs, and also if any developers could probably just look at the code to find remaining problems (apart from bad coding style and whitespace errors :D). Attached are some images I rendered while testing. ![BMW_Metro_50min_20_10_.png](https://archive.blender.org/developer/F98437/BMW_Metro_50min_20_10_.png) ![Sintel_10h_20.png](https://archive.blender.org/developer/F98439/Sintel_10h_20.png) ![Villa_14h.png](https://archive.blender.org/developer/F98441/Villa_14h.png) ![Pool_4h_20.png](https://archive.blender.org/developer/F98443/Pool_4h_20.png) (Note the visible tiles, I didn't use enough warmup) So, here's the patch, created against 508e0a: [metropolis_20.patch](https://archive.blender.org/developer/F98445/metropolis_20.patch)

It won't build for me:


  Error	1	error C2664: 'long _InterlockedCompareExchange(volatile long *,long,long)' : cannot convert argument 1 from 'volatile __int64 *' to 'volatile long *'	c:\dev\blender\blender\intern\cycles\kernel\kernel_compat_cpu.h	274	1	cycles_kernel
  Error	2	error C2065: '__INT_MAX__' : undeclared identifier	c:\dev\blender\blender\intern\cycles\kernel\kernel_random.h	108	1	cycles_kernel
  Error	3	error C2065: '__INT_MAX__' : undeclared identifier	c:\dev\blender\blender\intern\cycles\kernel\kernel_random.h	146	1	cycles_kernel
  Error	4	error C2065: '__INT_MAX__' : undeclared identifier	c:\dev\blender\blender\intern\cycles\kernel\kernel_path.h	1207	1	cycles_kernel
  Error	5	error C2664: 'long _InterlockedCompareExchange(volatile long *,long,long)' : cannot convert argument 1 from 'volatile __int64 *' to 'volatile long *'	C:\Dev\Blender\blender\intern\cycles\kernel\kernel_compat_cpu.h	274	1	cycles_device
  Error	6	error : identifier "__INT_MAX__" is undefined	c:\dev\blender\blender\intern\cycles\kernel\kernel_random.h	108	1	cycles_kernel_cuda
  Error	7	error : identifier "__INT_MAX__" is undefined	c:\dev\blender\blender\intern\cycles\kernel\kernel_random.h	146	1	cycles_kernel_cuda
  Error	9	error : identifier "__INT_MAX__" is undefined	c:\dev\blender\blender\intern\cycles\kernel\kernel_path.h	1207	1	cycles_kernel_cuda

Looked in the .patch and __ INT_MAX__ isn't defined anywhere.

It won't build for me: ``` Error 1 error C2664: 'long _InterlockedCompareExchange(volatile long *,long,long)' : cannot convert argument 1 from 'volatile __int64 *' to 'volatile long *' c:\dev\blender\blender\intern\cycles\kernel\kernel_compat_cpu.h 274 1 cycles_kernel Error 2 error C2065: '__INT_MAX__' : undeclared identifier c:\dev\blender\blender\intern\cycles\kernel\kernel_random.h 108 1 cycles_kernel Error 3 error C2065: '__INT_MAX__' : undeclared identifier c:\dev\blender\blender\intern\cycles\kernel\kernel_random.h 146 1 cycles_kernel Error 4 error C2065: '__INT_MAX__' : undeclared identifier c:\dev\blender\blender\intern\cycles\kernel\kernel_path.h 1207 1 cycles_kernel Error 5 error C2664: 'long _InterlockedCompareExchange(volatile long *,long,long)' : cannot convert argument 1 from 'volatile __int64 *' to 'volatile long *' C:\Dev\Blender\blender\intern\cycles\kernel\kernel_compat_cpu.h 274 1 cycles_device Error 6 error : identifier "__INT_MAX__" is undefined c:\dev\blender\blender\intern\cycles\kernel\kernel_random.h 108 1 cycles_kernel_cuda Error 7 error : identifier "__INT_MAX__" is undefined c:\dev\blender\blender\intern\cycles\kernel\kernel_random.h 146 1 cycles_kernel_cuda Error 9 error : identifier "__INT_MAX__" is undefined c:\dev\blender\blender\intern\cycles\kernel\kernel_path.h 1207 1 cycles_kernel_cuda ``` Looked in the .patch and __ INT_MAX__ isn't defined anywhere.
Author
Member

Sorry, my fault. I forgot that INT_MAX is GCC specific, just replace it with 0xffffffff. Regarding the InterlockedCompareExchange: In atomicAddF, "volatile long long int" needs to be replaced by "volatile long int".

Sorry, my fault. I forgot that __INT_MAX__ is GCC specific, just replace it with 0xffffffff. Regarding the InterlockedCompareExchange: In atomicAddF, "volatile long long int" needs to be replaced by "volatile long int".

In progressive render mode, the render images just get more and more transparent, even when saved out as pngs. This also affects the viewport render, and isn't limited to Metro, but to all renders.


untitled.png
Progressive render - mostly transparent (10 samples)


untitled2.png
Tiled render - how it should look (10 samples)


Here's a link to v.20 VC2013 with CUDA and GPU SSS/Volumes. Everything should be included in the zip this time!

https://www.mediafire.com/?j8ga5c3zb29urob

In progressive render mode, the render images just get more and more transparent, even when saved out as pngs. This also affects the viewport render, and isn't limited to Metro, but to all renders. --- ![untitled.png](https://archive.blender.org/developer/F98508/untitled.png) Progressive render - mostly transparent (10 samples) --- ![untitled2.png](https://archive.blender.org/developer/F98510/untitled2.png) Tiled render - how it should look (10 samples) --- Here's a link to v.20 VC2013 with CUDA and GPU SSS/Volumes. Everything __should__ be included in the zip this time! https://www.mediafire.com/?j8ga5c3zb29urob

Tiles.jpg

Test_metro.blend

Any recommendations to settings this scene?

![Tiles.jpg](https://archive.blender.org/developer/F98525/Tiles.jpg) [Test_metro.blend](https://archive.blender.org/developer/F98527/Test_metro.blend) Any recommendations to settings this scene?

Removed subscriber: @00Ghz

Removed subscriber: @00Ghz

To add more detail to Jemonn's bug report, the increasing transparency bug happens when using progressive Metropolis sampling, Tiled Metropolis sampling and adaptive sampling is unaffected.

Also, having the Importance Equalisation sampling option checked now crashes Blender in all cases.

To add more detail to Jemonn's bug report, the increasing transparency bug happens when using progressive Metropolis sampling, Tiled Metropolis sampling and adaptive sampling is unaffected. Also, having the *Importance Equalisation* sampling option checked now crashes Blender in all cases.
Author
Member

OK, I found the viewport bug, it was quite an easy fix. Also included are the MSVC fixes from my last post.
@lopataasdf From what I can see, the problem isn't actually the warmup, but the brightness contrast. If you look at the Villa rendering I uploaded, the tiles containing the lamps also have quite a high error since the samples are directed to where the image is brighter. The fix for this would be the importance equalisation, but as I said, it's currently broken. Just for safety, in this bugfix version, I also removed the UI element, since it crashes anyways.
metropolis_20_1.patch

OK, I found the viewport bug, it was quite an easy fix. Also included are the MSVC fixes from my last post. @lopataasdf From what I can see, the problem isn't actually the warmup, but the brightness contrast. If you look at the Villa rendering I uploaded, the tiles containing the lamps also have quite a high error since the samples are directed to where the image is brighter. The fix for this would be the importance equalisation, but as I said, it's currently broken. Just for safety, in this bugfix version, I also removed the UI element, since it crashes anyways. [metropolis_20_1.patch](https://archive.blender.org/developer/F98653/metropolis_20_1.patch)

On the 20.1 patch, got an issue with GPU Progressive mode. CPU Prog works fine, as does GPU and CPU tiled.

GPU Prog:
gpu-prog.jpg

GPU Tiled:
gpu-non-prog.jpg

GPU Prog (smaller render dimensions):
gpu-prog-small.jpg

All settings were at their defaults. I also tried changing the number of metro chains and progressive samples and render samples with no change.

On the 20.1 patch, got an issue with GPU Progressive mode. CPU Prog works fine, as does GPU and CPU tiled. GPU Prog: ![gpu-prog.jpg](https://archive.blender.org/developer/F98666/gpu-prog.jpg) GPU Tiled: ![gpu-non-prog.jpg](https://archive.blender.org/developer/F98668/gpu-non-prog.jpg) GPU Prog (smaller render dimensions): ![gpu-prog-small.jpg](https://archive.blender.org/developer/F98670/gpu-prog-small.jpg) All settings were at their defaults. I also tried changing the number of metro chains and progressive samples and render samples with no change.

Added subscriber: @capsicumpie

Added subscriber: @capsicumpie

Well, I've been testing the adaptive progressive sampling mode on a scene of mine that has a region covered in a volume and a bit of indirect lighting.

In theory, the adaptive sampling mode should lead to a massive decrease in rendertimes, in practice though...

Now as I said, it starts out promising as it immediately starts to work on areas that are more difficult to sample, this means that these areas start to converge a lot faster than in vanilla Cycles builds. However, hours in, I notice the advantage of the adaptive system start to disappear and then become a disadvantage.

Now you might wonder how it becomes a disadvantage then, well, after an hour or so, the convergence rate in some notable areas slow down so dramatically that it actually becomes less converged after so many hours than if you just continually threw samples at it like in the vanilla Master builds. It's almost like the sampler gets stuck on tiles in more indirectly lit areas and rarely get around to adding more samples to areas that you know needs more sampling the most from a visual perspective.

This is the image of the scene that I'm talking about (rendered in a vanilla Cycles build).
0568.jpg

The biggest issue with what the adaptive sampler does is in in the blue region, it's the first area to get additional samples after the initial pass, but later on the convergence rate in that area comes to a virtual standstill. I've tried to increase and decrease firefly emphasis using the exponent setting, I've tried adaptive steps from 10 to 50 samples, I've tried changing samplers, I just couldn't get convergence on the order of a notable increase in performance compared to what it otherwise would be.

So in all, I don't think the adaptive sampler might be ready for prime time yet, but I do anticipate that it will continue to be improved (eventually since you said yourself that you don't have a lot of time now).

Well, I've been testing the adaptive progressive sampling mode on a scene of mine that has a region covered in a volume and a bit of indirect lighting. In theory, the adaptive sampling mode should lead to a massive decrease in rendertimes, in practice though... Now as I said, it starts out promising as it immediately starts to work on areas that are more difficult to sample, this means that these areas start to converge a lot faster than in vanilla Cycles builds. However, hours in, I notice the advantage of the adaptive system start to disappear and then become a disadvantage. Now you might wonder how it becomes a disadvantage then, well, after an hour or so, the convergence rate in some notable areas slow down so dramatically that it actually becomes less converged after so many hours than if you just continually threw samples at it like in the vanilla Master builds. It's almost like the sampler gets stuck on tiles in more indirectly lit areas and rarely get around to adding more samples to areas that you know needs more sampling the most from a visual perspective. This is the image of the scene that I'm talking about (rendered in a vanilla Cycles build). ![0568.jpg](https://archive.blender.org/developer/F99287/0568.jpg) The biggest issue with what the adaptive sampler does is in in the blue region, it's the first area to get additional samples after the initial pass, but later on the convergence rate in that area comes to a virtual standstill. I've tried to increase and decrease firefly emphasis using the exponent setting, I've tried adaptive steps from 10 to 50 samples, I've tried changing samplers, I just couldn't get convergence on the order of a notable increase in performance compared to what it otherwise would be. So in all, I don't think the adaptive sampler might be ready for prime time yet, but I do anticipate that it will continue to be improved (eventually since you said yourself that you don't have a lot of time now).

Well, after a while, I found out how I could get much better convergence speed in the blue area of that image while maintaining the advantages of this patch, and that is to set the exponent as high as 50 while disabling the adaptive part of the sampling (as in, the actual shuffling of samples within the tile).

I noticed on another scene what appeared to be a major weakness of the adaptive part of the sampler, and that is in various cases, you get a good amount of clumpiness as to where you have spots with lots of samples interspersed with spots that have little. It seems to me in this case that the adaptive sampling code is a bit too tight in terms of focusing the samples as the clumping can also mess with the perceived error and cause s bit of inaccuracy compared to what is seen visually along with very slow convergence. I've also noticed then that true adaptivity in various cases cause inferior quality to if you just had the adaptive stopping and the prioritization of tiles based on error.

You might think what not having the in-tile adaptivity might do to convergence where the focusing of samples work well, it actually doesn't give that much of a disadvantage in terms of the big picture providing that the tiles are somewhat on the small side (I use a size that's 40 by 40 pixels).

A final note, I will tell you that Metropolis sampling still seems to crash on large scenes when Sobol distribution is used (doesn't crash with Multi-jitter), and that Sobol adaptive sampling may crash on large scenes after many hours of rendering (Multi-jitter just seems more stable overall).

Also, it would be nice if you could find the time to do periodic re-basing of the patch (if it doesn't build with new revisions) so we can at least have up to date builds (it shouldn't take too long unless there's massive changes to the Cycles code).

Well, after a while, I found out how I could get much better convergence speed in the blue area of that image while maintaining the advantages of this patch, and that is to set the exponent as high as 50 while disabling the adaptive part of the sampling (as in, the actual shuffling of samples within the tile). I noticed on another scene what appeared to be a major weakness of the adaptive part of the sampler, and that is in various cases, you get a good amount of clumpiness as to where you have spots with lots of samples interspersed with spots that have little. It seems to me in this case that the adaptive sampling code is a bit too tight in terms of focusing the samples as the clumping can also mess with the perceived error and cause s bit of inaccuracy compared to what is seen visually along with very slow convergence. I've also noticed then that true adaptivity in various cases cause inferior quality to if you just had the adaptive stopping and the prioritization of tiles based on error. You might think what not having the in-tile adaptivity might do to convergence where the focusing of samples work well, it actually doesn't give that much of a disadvantage in terms of the big picture providing that the tiles are somewhat on the small side (I use a size that's 40 by 40 pixels). A final note, I will tell you that Metropolis sampling still seems to crash on large scenes when Sobol distribution is used (doesn't crash with Multi-jitter), and that Sobol adaptive sampling may crash on large scenes after many hours of rendering (Multi-jitter just seems more stable overall). Also, it would be nice if you could find the time to do periodic re-basing of the patch (if it doesn't build with new revisions) so we can at least have up to date builds (it shouldn't take too long unless there's massive changes to the Cycles code).
Author
Member

OK, I rebased it to 7b83e3d.
@Ace_Dragon OK, that's strange. In all my test scenes, my "subjective" by-eye error estimation agreed quite well with the computed error. Could you attach/send the scene in the example (or any other scene showing this behaviour)?
The clumping with the in-tile adaptive mode is quite to be expected, unfortunately. The reason for it appearing is that sometimes a firefly appears in a single pixel, so now the error of this pixel is way higher than its neighbours. For this reason I added a smoothing step to the sampling, but if there is only one high-error pixel in the tile, you can't smooth enough to get rid of the error. This is a statistical problem: You can only calculate the variance (the basis of the current error estimation) within some confidence interval, but not exact: The variance itself is again an approximation.
Regarding the crashes: "It crashes" is pretty hard to debug, especially since for me it never crashes, even on scenes like the Lego bulldozer which I consider quite complex. COuld you please post your system data (mainly OS), build info (compiler, master version on which you applied the patch...) and (most importantly) the crashlog (if possible, generated by a debug build)? Also, does it crash randomly or always at the same tile? Could you post an example scene?
metropolis_20_2.patch

OK, I rebased it to 7b83e3d. @Ace_Dragon OK, that's strange. In all my test scenes, my "subjective" by-eye error estimation agreed quite well with the computed error. Could you attach/send the scene in the example (or any other scene showing this behaviour)? The clumping with the in-tile adaptive mode is quite to be expected, unfortunately. The reason for it appearing is that sometimes a firefly appears in a single pixel, so now the error of this pixel is way higher than its neighbours. For this reason I added a smoothing step to the sampling, but if there is only one high-error pixel in the tile, you can't smooth enough to get rid of the error. This is a statistical problem: You can only calculate the variance (the basis of the current error estimation) within some confidence interval, but not exact: The variance itself is again an approximation. Regarding the crashes: "It crashes" is pretty hard to debug, especially since for me it never crashes, even on scenes like the Lego bulldozer which I consider quite complex. COuld you please post your system data (mainly OS), build info (compiler, master version on which you applied the patch...) and (most importantly) the crashlog (if possible, generated by a debug build)? Also, does it crash randomly or always at the same tile? Could you post an example scene? [metropolis_20_2.patch](https://archive.blender.org/developer/F100938/metropolis_20_2.patch)

About the issue of fireflies messing with the error estimation, couldn't this be something that one could resolve by running a filter similar to what the compositor's despeckle node does before applying the smoothing?

Also, maybe any crashing is just me setting the exponent too high, because I do notice for one thing that the initial pass can start to get bogged down a bit if you do (left to wonder if it has a negahaven't tested though). A test render I'm currently doing also seems to do a much better job maintaining a constant noise level as well as accurate prioritization with the value lowered as well (as I know you said that you don't recommend values over 10).

About the issue of fireflies messing with the error estimation, couldn't this be something that one could resolve by running a filter similar to what the compositor's *despeckle* node does before applying the smoothing? Also, maybe any crashing is just me setting the exponent too high, because I do notice for one thing that the initial pass can start to get bogged down a bit if you do (left to wonder if it has a negahaven't tested though). A test render I'm currently doing also seems to do a much better job maintaining a constant noise level as well as accurate prioritization with the value lowered as well (as I know you said that you don't recommend values over 10).

Removed subscriber: @LevonHudson

Removed subscriber: @LevonHudson
Author
Member

That filtering idea is actually quite good! I think the despeckle node is basically a median filter, which usually excels in removing single-pixel noise. I'll look into that!
Regarding the high exponents: The problem here is numerical, since what happens is that it takes the p'th power of all the errors, sums them up and then takes the p'th root. However, even with a p of around 4, the sum is often around 0.00001, which already gives precision errors with floats. Considering that the loop is no bottleneck and only stores a single value, I'll change it to use doubles in the next version.

That filtering idea is actually quite good! I think the despeckle node is basically a median filter, which usually excels in removing single-pixel noise. I'll look into that! Regarding the high exponents: The problem here is numerical, since what happens is that it takes the p'th power of all the errors, sums them up and then takes the p'th root. However, even with a p of around 4, the sum is often around 0.00001, which already gives precision errors with floats. Considering that the loop is no bottleneck and only stores a single value, I'll change it to use doubles in the next version.

Added subscriber: @sdilmob-3

Added subscriber: @sdilmob-3

Added subscriber: @Samir-Osman

Added subscriber: @Samir-Osman

Added subscriber: @Luarvik

Added subscriber: @Luarvik

While looking at the render proces i got an idea that's maybe nice, as it might reduce render time, so i share the idea.
With adaptive rendering (i read from the threat this can be done on pixel scale as well), but that might not even be required for this idea.
Well i used camera with a focus and an empty as focus object / (zmap might work as well for this idea).

Now to the point a little noise doesn't have to be bad, real digital or analog camera's have some noise too .
However noise is more clear when its in the focus of your subject, as compared to something in the background.
So maybe its an idea to combine depth/camera focus into the error calculation of adaptive render-er.
What would be out of focus then would be a bit more noise, just as with real camera's (where that is often a bit blurred, that might be done in after processing in blender bokeh).

While looking at the render proces i got an idea that's maybe nice, as it might reduce render time, so i share the idea. With adaptive rendering (i read from the threat this can be done on pixel scale as well), but that might not even be required for this idea. Well i used camera with a focus and an empty as focus object / (zmap might work as well for this idea). Now to the point a little noise doesn't have to be bad, real digital or analog camera's have some noise too . However noise is more clear when its in the focus of your subject, as compared to something in the background. So maybe its an idea to combine depth/camera focus into the error calculation of adaptive render-er. What would be out of focus then would be a bit more noise, just as with real camera's (where that is often a bit blurred, that might be done in after processing in blender bokeh).

Added subscriber: @SergeyBarabanov

Added subscriber: @SergeyBarabanov
Member

Added subscriber: @ChengduLittleA

Added subscriber: @ChengduLittleA
Member

Added subscriber: @VilemDuha

Added subscriber: @VilemDuha
Member

PGTART -
actually noise in cycles tends to be more visible in out of focus areas, but especially when something is really close to the camera. Also, post processing filter will never be able to achieve a good quality defocus for complex scenes. Believe me, I tried to squeeze the maximum of it in some projects, and while it is a great feature, it will never ever look close to what cycles does. Remember too, that actually in cycles, you don't get a consistent zbuffer because of the sampling, and you need this for the postprocessing. This exact feature you describe is in blender internal, which enables you lower sample numbers in out-of focus and fast-motion areas, exactly for the reason you write, but that counts with the postpro. Sorry if getting off-topic here.

PGTART - actually noise in cycles tends to be more visible in out of focus areas, but especially when something is really close to the camera. Also, post processing filter will never be able to achieve a good quality defocus for complex scenes. Believe me, I tried to squeeze the maximum of it in some projects, and while it is a great feature, it will never ever look close to what cycles does. Remember too, that actually in cycles, you don't get a consistent zbuffer because of the sampling, and you need this for the postprocessing. This exact feature you describe is in blender internal, which enables you lower sample numbers in out-of focus and fast-motion areas, exactly for the reason you write, but that counts with the postpro. Sorry if getting off-topic here.
Member

This comment was removed by @gandalf3

*This comment was removed by @gandalf3*
Member

Added subscriber: @xuekepei

Added subscriber: @xuekepei

Added subscriber: @Januz

Added subscriber: @Januz

Added subscriber: @YanivGershoni

Added subscriber: @YanivGershoni
Author
Member

Okay, I'm back again, and with news: Since the Metropolis redesign helped separating the adaptive and Metro parts, I decided to split the patches and have now, for the last 3 weeks, polished the adaptive sampling to be ready for code review, for which I just submitted it at D808.
This means that for now, my main focus will be on getting adaptive sampling approved, but I'll also work on Metro in the meantime. I still don't like some parts of it, especially the sometimes ridiculously high required warmup samples and how inefficient GPUs are at it (this might be improved by sampling the first dimensions and then sorting based on image coordinates to improve coherency, for example).
So, this patch is far from dead, and I'll continue working on it. Sorry, no up-to-date Metro patch yet, that will be the next step.

Okay, I'm back again, and with news: Since the Metropolis redesign helped separating the adaptive and Metro parts, I decided to split the patches and have now, for the last 3 weeks, polished the adaptive sampling to be ready for code review, for which I just submitted it at [D808](https://archive.blender.org/developer/D808). This means that for now, my main focus will be on getting adaptive sampling approved, but I'll also work on Metro in the meantime. I still don't like some parts of it, especially the sometimes ridiculously high required warmup samples and how inefficient GPUs are at it (this might be improved by sampling the first dimensions and then sorting based on image coordinates to improve coherency, for example). So, this patch is far from dead, and I'll continue working on it. Sorry, no up-to-date Metro patch yet, that will be the next step.

This comment was removed by @PeterBoos

*This comment was removed by @PeterBoos*

Added subscriber: @MirceaKitsune

Added subscriber: @MirceaKitsune

Added subscriber: @davidKatara

Added subscriber: @davidKatara

Added subscriber: @ArtemBataev

Added subscriber: @ArtemBataev

Added subscriber: @juancarlosgzrz

Added subscriber: @juancarlosgzrz

What happened to this patch?

What happened to this patch?

Added subscriber: @BrianRacer

Added subscriber: @BrianRacer

Added subscriber: @BoShi

Added subscriber: @BoShi

This patch is very interesting - as i see they can transform cycles in to something like maxwell render and this is on cycles with gpu power - very much hope that the developers are paying due attention. Maybe there are some news about it?

This patch is very interesting - as i see they can transform cycles in to something like **maxwell render** and this is on cycles **with** gpu power - very much hope that the developers are paying due attention. Maybe there are some news about it?

Added subscriber: @00Ghz

Added subscriber: @00Ghz

Yes any progress?

Yes any progress?

Well, this is so sad. This could change the entire Blender project. Hope this come back to life.

Well, this is so sad. This could change the entire Blender project. Hope this come back to life.

Added subscriber: @bugaevc

Added subscriber: @bugaevc

Well this now is dead, no hope.

Well this now is dead, no hope.

Removed subscriber: @Samir-Osman

Removed subscriber: @Samir-Osman

Added subscriber: @rookie1024

Added subscriber: @rookie1024

Added subscriber: @Boris85

Added subscriber: @Boris85

For all people who interested in this, here are some quotes:

Lukas Stockner wrote:
"I decided to split the patches and have now, for the last 3 weeks, polished the adaptive sampling to be ready for code review, for which I just submitted it at D808. This means that for now, my main focus will be on getting adaptive sampling approved"

And quote from D808, most recent message (Aug 18 2015) by Lukas Stockner:

"//There was quite a long silence around this patch, so here's an update and the way I see it:
This approach kind of works, but not nearly as good as it should. It's not robust at all,
sometimes it just gets stuck on single pixels, sometimes the result is quite blotchy, sometimes
tile borders are extremely visible. I'm looking into many different approaches currently and it looks like there is a lot of protential. Cycles' design makes some of them impossible, but still, a big improvement should be possible. Parts of this patch will be reusable, but the approach of just taking the variance is too unstable.//"

Since that message just about two months have passed which is not too long considering how hard and complicated the task at hand is, so it is way too soon to say things like "there is no hope", "it is dead", etc. This is not helpful. In case some of you do not know: Lukas already did some great work which was finished and accepted to Blender - light portals, not to mention he added other new features and fixed some issues (date of his most recent patch committed to master is 2015-10-08).

I believe there is still a chance that metropolis sampling may be added to Blender eventually.

For all people who interested in this, here are some quotes: Lukas Stockner wrote: "*I decided to split the patches and have now, for the last 3 weeks, polished the adaptive sampling to be ready for code review, for which I just submitted it at [D808](https://archive.blender.org/developer/D808). This means that for now, my main focus will be on getting adaptive sampling approved*" And quote from [D808](https://archive.blender.org/developer/D808), most recent message (Aug 18 2015) by Lukas Stockner: "//There was quite a long silence around this patch, so here's an update and the way I see it: This approach kind of works, but not nearly as good as it should. It's not robust at all, sometimes it just gets stuck on single pixels, sometimes the result is quite blotchy, sometimes tile borders are extremely visible. I'm looking into many different approaches currently and it looks like there is a lot of protential. Cycles' design makes some of them impossible, but still, a big improvement should be possible. Parts of this patch will be reusable, but the approach of just taking the variance is too unstable.//" Since that message just about two months have passed which is not too long considering how hard and complicated the task at hand is, so it is way too soon to say things like "there is no hope", "it is dead", etc. This is not helpful. In case some of you do not know: Lukas already did some great work which was finished and accepted to Blender - light portals, not to mention he added other new features and fixed some issues (date of his most recent patch committed to master is 2015-10-08). I believe there is still a chance that metropolis sampling may be added to Blender eventually.

Removed subscriber: @FilipPolbratt

Removed subscriber: @FilipPolbratt

There is hope. It is alive.

Well, light portals are kind of useless due to the poor skylight code. It's not physically plausible nor realistic.

There is hope. It is alive. Well, light portals are kind of useless due to the poor skylight code. It's not physically plausible nor realistic.
Member

Removed subscriber: @gandalf3

Removed subscriber: @gandalf3

Removed subscriber: @00Ghz

Removed subscriber: @00Ghz

Portals are very useful , and in many cases they make it easier to get plausible and realistic result with Cycles by reducing render time without loss of quality. I was impressed by great speed up for some of my own scenes. But this is off-topic here. My original point was that the developer is still active and already made many useful contribution to Blender. Saying that developer's work is "useless" just because some other code is not yet good enough for you is unlikely to motivate anyone to work faster. Before that you said that "this now is dead, no hope" just three weeks after the developer said "I'm looking into many different approaches currently and it looks like there is a lot of potential" in another thread. You have to understand that if you want something to be done fast(er), you either have to hire a developer or do it yourself, otherwise you have to wait and hope. Complaining many times in a row and spreading your negativity is not going to help anyone.

Portals are [very useful ](http://www.blenderguru.com/tutorials/using-portals-accelerate-render-times/), and in many cases they make it easier to get plausible and realistic result with Cycles by reducing render time without loss of quality. I was impressed by great speed up for some of my own scenes. But this is off-topic here. My original point was that the developer is still active and already made many useful contribution to Blender. Saying that developer's work is "useless" just because some other code is not yet good enough for you is unlikely to motivate anyone to work faster. Before that you said that "this now is dead, no hope" just three weeks after the developer said "I'm looking into many different approaches currently and it looks like there is a lot of potential" in another thread. You have to understand that if you want something to be done fast(er), you either have to hire a developer or do it yourself, otherwise you have to wait and hope. Complaining many times in a row and spreading your negativity is not going to help anyone.

Inverse psychology.

You shouldn't pay attention to these kind of comments. But people cannot avoid these so, this was my way to get an answer. Thank you Boris.

Inverse psychology. You shouldn't pay attention to these kind of comments. But people cannot avoid these so, this was my way to get an answer. Thank you Boris.

Added subscriber: @Saiman

Added subscriber: @Saiman

I very much hope that this method will appear in the next versions. It is a good method if you do things with light and kauystikoy. Portals are very helpful. Thank you for making improvements. (Google translator)

I very much hope that this method will appear in the next versions. It is a good method if you do things with light and kauystikoy. Portals are very helpful. Thank you for making improvements. (Google translator)

Removed subscriber: @tuqueque

Removed subscriber: @tuqueque

Added subscriber: @AssAsIN10556

Added subscriber: @AssAsIN10556

ok, i see everyone using this but how do i add it to cycles myself?

ok, i see everyone using this but how do i add it to cycles myself?

Removed subscriber: @JasonClarke

Removed subscriber: @JasonClarke
Member

Added subscriber: @Blendify

Added subscriber: @Blendify
Member

@AssAsIN10556 Using this requires to apply D808 and build blender yourself.

@LukasStockner It seems that the patch fails. Either building or applying.


I can make Windows 64 bit build if the patch works.

@AssAsIN10556 Using this requires to apply [D808](https://archive.blender.org/developer/D808) and build blender yourself. @LukasStockner It seems that the patch fails. Either building or applying. ------------- I can make Windows 64 bit build if the patch works.

In #38401#352424, @Blendify wrote:
@AssAsIN10556 Using this requires to apply D808 and build blender yourself.

@LukasStockner It seems that the patch fails. Either building or applying.


I can make Windows 64 bit build if the patch works.

D808? i have no clue what that should be... could you explain it? or send a link with an explenation

> In #38401#352424, @Blendify wrote: > @AssAsIN10556 Using this requires to apply [D808](https://archive.blender.org/developer/D808) and build blender yourself. > > @LukasStockner It seems that the patch fails. Either building or applying. > > ------------- > I can make Windows 64 bit build if the patch works. [D808](https://archive.blender.org/developer/D808)? i have no clue what that should be... could you explain it? or send a link with an explenation

Added subscriber: @erel96-2

Added subscriber: @erel96-2
Member

D808 is a link to the patch. Just click it. Or just go https://developer.blender.org/D808 To grab the code. However like I said the patch no longer works and needs to be updated.

[D808](https://archive.blender.org/developer/D808) is a link to the patch. Just click it. Or just go https://developer.blender.org/D808 To grab the code. However like I said the patch no longer works and needs to be updated.

In #38401#352457, @Blendify wrote:
D808 is a link to the patch. Just click it. Or just go https://developer.blender.org/D808 To grab the code. However like I said the patch no longer works and needs to be updated

hmm... are there any older patches? that would work? (with version 2.76 maybe? lets hope so)

> In #38401#352457, @Blendify wrote: > [D808](https://archive.blender.org/developer/D808) is a link to the patch. Just click it. Or just go https://developer.blender.org/D808 To grab the code. However like I said the patch no longer works and needs to be updated hmm... are there any older patches? that would work? (with version 2.76 maybe? lets hope so)

Added subscriber: @nubeees

Added subscriber: @nubeees

Removed subscriber: @nubeees

Removed subscriber: @nubeees

Added subscriber: @nubeees

Added subscriber: @nubeees

Added subscriber: @dfelinto

Added subscriber: @dfelinto

@LukasStockner can we close this? Since this will be a patch at some point I can't see why to have this as a maniphest task meanwhile.

@LukasStockner can we close this? Since this will be a patch at some point I can't see why to have this as a maniphest task meanwhile.
Author
Member

Changed status from 'Open' to: 'Archived'

Changed status from 'Open' to: 'Archived'
Lukas Stockner self-assigned this 2017-01-04 14:06:45 +01:00
Author
Member

Yes, that's definitely a good idea - even if this was revived at some point, it would be a completely rewritten patch anyways.

Yes, that's definitely a good idea - even if this was revived at some point, it would be a completely rewritten patch anyways.

Added subscriber: @Squiddy

Added subscriber: @Squiddy

Removed subscriber: @Squiddy

Removed subscriber: @Squiddy

Added subscriber: @gabe2252

Added subscriber: @gabe2252

Added subscriber: @morph3us

Added subscriber: @morph3us

Added subscriber: @AugustoCezar

Added subscriber: @AugustoCezar

Is this patch alive?
Would be great if this patch becomes a feature someday.

Is this patch alive? Would be great if this patch becomes a feature someday.

Removed subscriber: @ArtemBataev

Removed subscriber: @ArtemBataev
Contributor

Added subscriber: @AlexanderWilms

Added subscriber: @AlexanderWilms

Added subscriber: @Tutul

Added subscriber: @Tutul

It look amazing and I hope it become a part of Cycle one day.
Is it possible to get a link to the archived work ? Maybe the git branch where it belong ?

I wish I could help but not sure I know enough about that kind of algorithm for now

It look amazing and I hope it become a part of Cycle one day. Is it possible to get a link to the archived work ? Maybe the git branch where it belong ? I wish I could help but not sure I know enough about that kind of algorithm for now

Added subscriber: @SteffenD

Added subscriber: @SteffenD

Added subscriber: @silex

Added subscriber: @silex
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
77 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#38401
No description provided.