Bidir fireflies (planned exphotonmap test)

[SATtva]

Today I was planning to conduct a test with exphotonmap on the latest hg snapshot (btw, my first try of the exphotonmap integrator). I started two renders of the same real-life interior scene (heavy geometry, occluded and indirect lights, many openings, etc). The first one was with exphotonmap integrator. I didn't optimized the settings in any way, but after 3 hours I already got a fairly decent and clean image. It's darker than it should, but this is due to low photons depth I suppose:

exphotonmap integrator test, rendered for 3 hours, 2881 spp

Here's my settings. I'm very interested to hear any recommendations:

Code: Select all

Sampler "metropolis"
   "float largemutationprob" [0.200000]
   "integer maxconsecrejects" [128]
   "bool usevariance" ["false"]

SurfaceIntegrator "exphotonmap"
   "string renderingmode" ["path"]
   "string strategy" ["auto"]
   "integer maxphotondepth" [16]
   "integer maxdepth" [5]
   "float maxdist" [0.500000]
   "integer nused" [100]
   "integer indirectphotons" [2000000]
   "integer maxdirectphotons" [2000000]
   "integer causticphotons" [20000]
   "bool finalgather" ["true"]
   "string gatherrrstrategy" ["efficiency"]
   "integer finalgathersamples" [64]
   "float gatherangle" [10.000000]


So far so good. After that I've started a rendering with the same settings on bidir, just to compare the convergence speed. And it was very good. This is how the image looked at 2h21m of rendering:

bidir integrator, rendered for 2 hours 21 minutes

The rendering went good... until something bad happened. My plan was to render this second image for the same 3 hours to have a fair comparison, but when I looked at it at 2h42m point I seen this:

bidir integrator, rendered for 2 hours 42 minutes, 679 spp

That's bad. Because it reminded me of a similar problem with spontaneous fireflies I encountered some time ago with RC6 ICC build on the cubo scene: viewtopic.php?f=16&t=2886 Any thoughts?

EDIT: Accelerator is kdtree. I can send the scene files to the devs (sorry, can't post it here: it's about 15 Mb even without textures, and it's a commercial work).

[Lord Crc]

I've noticed some fireflies with some scenes I did recently, haven't had time to track it down yet. I only got a handful tho, not as massive as yours.

[SATtva]

By the way, with these tests I can't confirm the problem psychotron reported here viewtopic.php?p=25706#p25706 concerning exphotonmap image overexposure. All images posted above have the same linear tonemapping settings, and bidir is the one looking more bright. Yes, to say that these scenes are very different is to say nothing, :) but nevertheless...

[psychotron]

interesting
I probably fire up longtime render too to see if it appear
(anyway nothing better to do beside all that bloody lowpoly work)

[Dade]

Here's my settings. I'm very interested to hear any recommendations:

Luxblend is a bit outdated when it comes to photonmapping, for instance, "maxdirectphotons" parameter doesn't exist anymore and is now called "directphotons". This means your rendering has been done with a large indirect photonmap but with a quite small direct photonmap. The parameters about photonmaps size are:

- "integer directphotons" (default: 200000);
- "integer causticphotons" (default: 20000);
- "integer indirectphotons" (default: 200000);
- "integer radiancephotons" (default: 200000);

I'm afraid you have to set them to a proper size for your scene by hand editing the .lxs file. 2.000.000 are really a lot of photons, your scene is complex so it could be the right value but I would try to do a test with less photons too (i.e. 1.000.000) and to keep "nused" at 50, it should speedup your rendering a bit.

Talking about testing, there is some "secret" parameter that is damn useful when working with photonmap:

- "bool dbg_enabledirect" (default: false);
- "bool dbg_enableindircaustic" (default: false);
- "bool dbg_enableindirdiffuse" (default: false);
- "bool dbg_enableindirspecular" (default: false);

I will show you how they work. This is a sample photonmapping rendering with only 16 samples per pixel ("bool finalgather" ["true"] "bool dbg_enabledirect" ["true"] "bool dbg_enableradiancemap" ["false"] "bool dbg_enableindircaustic" ["true"] "bool dbg_enableindirdiffuse" ["true"] "bool dbg_enableindirspecular" ["true"]):

With ("bool finalgather" ["false"] "bool dbg_enabledirect" ["true"] "bool dbg_enableradiancemap" ["false"] "bool dbg_enableindircaustic" ["false"] "bool dbg_enableindirdiffuse" ["false"] "bool dbg_enableindirspecular" ["false"]), only direct lighting is visible:

With ("bool finalgather" ["false"] "bool dbg_enabledirect" ["false"] "bool dbg_enableradiancemap" ["true"] "bool dbg_enableindircaustic" ["false"] "bool dbg_enableindirdiffuse" ["false"] "bool dbg_enableindirspecular" ["false"]), only radiance map is visible:

With ("bool finalgather" ["false"] "bool dbg_enabledirect" ["false"] "bool dbg_enableradiancemap" ["false"] "bool dbg_enableindircaustic" ["false"] "bool dbg_enableindirdiffuse" ["true"] "bool dbg_enableindirspecular" ["false"]), only indirect map is visible:

This kind of debug rendering is useful to understand if you have enough photons in the maps, for instance, this is a rendering with only 5000 photons ("bool finalgather" ["false"] "bool dbg_enabledirect" ["false"] "bool dbg_enableradiancemap" ["true"] "bool dbg_enableindircaustic" ["false"] "bool dbg_enableindirdiffuse" ["false"] "bool dbg_enableindirspecular" ["false"]):

There is some area not covered by any photon, the density is too low.

That's bad. Because it reminded me of a similar problem with spontaneous fireflies I encountered some time ago with RC6 ICC build on the cubo scene: viewtopic.php?f=16&t=2886 Any thoughts?

Lux hg snapshot compiled with GCC 4.4 ?

[psor]

I think the new bidir code is borked somehow.

Ouch!

All renderings made with lux are using the same linear tone mapping settings and
the same white balance. I haven't tried to render this scene with meshlights yet
cause of lack of resources.

The new bidir code was going crazy from the beginning of the rendering. Another
thing I'm wondering about is. Why is the white balance always different between
different integrators? I'm unable to match them with the GUI.




take care
psor

[SATtva]

Dade, thanks for the detailed explanation. I just skimmed through it, will read more thoroughly later.

Lux hg snapshot compiled with GCC 4.4 ?

That's right.

psor, what accelerator did you used and on what system? I ask because I have similar artifacts on 64-bit system with QBVH.

[SATtva]

Dade, thanks again for the disclosed art of black magic, now playing with those debugging options. :)

I see Lux currently uses only one thread for the photonmap generation stage. Will this be improved in the future?

EDIT: I think, ideally, slaves shouldn't need to regenerate the photonmap, the master should generate it once and distribute to the whole farm.

[psor]

@SATtva

Thanks for the reminder, cause I did just copy the scene over to the server and forgot about the issue.
Indeed I was using QBVH on the Windows version of lux as it's still v0.6 final.




take care
psor

[Dade]

I see Lux currently uses only one thread for the photonmap generation stage. Will this be improved in the future?

Yup, every single photon can be traced in paralell so it is something definitively to have in the future.

EDIT: I think, ideally, slaves shouldn't need to regenerate the photonmap, the master should generate it once and distribute to the whole farm.

It is a trade off between bandwidth usage and computation time. Photonmaps can be large and on a slow connection it would be faster to regenerate the maps. I guess it is something to have as an option.

BTW, there is another secret option "string photonmapsfile" to save or reload the photon maps from a file. For instance, in a animation where you move only the camera, you can build the photonmaps only once and reuse the same for every frame.

P.S. if you check the method named ExPhotonIntegrator::CreateSurfaceIntegrator() (line 582) at end of file http://src.luxrender.net/lux/file/8a996 ... tonmap.cpp you can have a (very) raw idea of all parameters available for the photonmap integrator.