SPPM renderer (CPU-only)

[SATtva]

Well, technically FLM resuming and network rendering are more or less working (although i can see the problem with radius reduction messed up)

Maybe the latest used radius could be saved in the FLM file?

There is many issues in fact.

a) The radius should be stored in the FLM, but we can solve this issue pretty easilly (in fact, using PPM proba version, we only needs to store one float in the FLM, but this is not currently possible)
b) the pass number should be stored in the FLM (same issue as a)
c) We must ensure that the render is not stoped during a pass, or only store the result of the previous passes in the FLM

Considering network rendering in heterogeneous environments, the radius reduction part can be dealt with by periodically (e.g. each 10 passes on master) sending slaves the current radius value -- at the end of the current pass each slave should resync to the new radius. As to keeping the proper number of photons per pass across all machines, i have no idea at the moment.

[SATtva]

Check the reflection of the sun from the glossy door to the glossy floor. No matter how long you render, this does not clean up.
It seems that code tries to do something right but fail

Similar situation here after 6 hours, see the top of the microwave oven (seems to be a very indirect caustic from the cooker hood lights) and the metal grill behind the rough glass. A few outlier photons also spoil the look a bit -- they've appeared right in the final hour. And an already known problem with dark corners.

Nevertheless, i'm extremely impressed how well SPPM works in this scene, it converges about twice faster than Bidir+MLT. guibou, thank you for your great work!

[Chawl]

I tried to decipher the technical discussions few pages earlier and although I'm an engineer, I'm rather stuck after some PDFs and wiki pages All I can comprehend is, handling of the glossy surfaces is quite tricky.
guibou puts an excellent effort indeed.

[guibou]

Considering network rendering in heterogeneous environments, the radius reduction part can be dealt with by periodically (e.g. each 10 passes on master) sending slaves the current radius value -- at the end of the current pass each slave should resync to the new radius. As to keeping the proper number of photons per pass across all machines, i have no idea at the moment.

In fact, using the knaus PPM approach (know as PPM proba is luxrender), the radius per pass in known in advance (in fact, the radius is only a function of the pass number, ie: Radius(i) = Radius(i - 1) * (i - alpha) / (i - 1)), so the master only have to ask each slave to render a specific pass and it works.

The issue is how to handle slaves which are taking too much time to finish the pass they where asked for. Should we ask a new slave to compute the pass ? If twos slaves are working on the same pass and gives a result, can we stupidly merge both passes and normalize the number of photons ?

Same think, if we want to implement flm merging, each flm should stores the id of each pass it contains. But how to handle duplicates ?

I tried to decipher the technical discussions few pages earlier and although I'm an engineer, I'm rather stuck after some PDFs and wiki pages All I can comprehend is, handling of the glossy surfaces is quite tricky.

Yes, glossy surfaces are tricky. An important part of the research done in rendering algorithm nowadays focus on better handling of glossy reflections.

guibou puts an excellent effort indeed.

Thank you.

[SATtva]

The issue is how to handle slaves which are taking too much time to finish the pass they where asked for. Should we ask a new slave to compute the pass ? If twos slaves are working on the same pass and gives a result, can we stupidly merge both passes and normalize the number of photons ?

Maybe a bit over-engineered idea from the renderfarm PoV, but otherwise it's quite simple i guess (no need to split a frame or normalize photons): Master stores locally a list of passes already done and pending rendering. When it sends an unique pass number to a slave, it marks this pass as sent to a particular slave, and periodically pings the slave whether it finished, and if the slave has done the work, the master downloads it, merges and marks the pass as done. Master can be configured to send batches of pass numbers to minimize network traffic -- a slave will render all the requested passes, merge them locally and send just a single film later on.

I think this approach handles all the issues and distributes work evenly regardless of slaves processing power.

EDIT: There are some corner cases of course. What if a slave has crashed or was removed from the network pool by a user? I guess in the former case (a slave aren't responding to master's pings or not recognizes a session ID) all passes sent to that slave should be marked again as "not sent" and processed by other slaves or by the master locally. The same goes for the latter case of a manually removed slave.

Same think, if we want to implement flm merging, each flm should stores the id of each pass it contains. But how to handle duplicates ?

If we could expose a couple of options in Lux, this can be dealt with: a starting pass and another halt condition -- pass number reached. Then it's up to user to render non-overlapping passes on different machines (like animation frames) which can be merged later. Similarly, with Path/Bidir a user still has to ensure not to merge several films or films with identical samples twice.

[jeanphi]

In fact, using the knaus PPM approach (know as PPM proba is luxrender), the radius per pass in known in advance (in fact, the radius is only a function of the pass number, ie: Radius(i) = Radius(i - 1) * (i - alpha) / (i - 1)), so the master only have to ask each slave to render a specific pass and it works.

The issue is how to handle slaves which are taking too much time to finish the pass they where asked for. Should we ask a new slave to compute the pass ? If twos slaves are working on the same pass and gives a result, can we stupidly merge both passes and normalize the number of photons ?

Same think, if we want to implement flm merging, each flm should stores the id of each pass it contains. But how to handle duplicates ?

In an ideal world to get the most out of SPPM, yes we need this. However if we accept a global performance loss, we can do without much of this information. We can have 2 slaves computing the same pass number at the same time, that's not a problem, what is important is that the photon density is correctly computed and preserved during merging. Not correctly accounting the pass number or the radius might lead to a slower radius reduction, but the variance will still be reduced.

Yes, glossy surfaces are tricky. An important part of the research done in rendering algorithm nowadays focus on better handling of glossy reflections.

Yes, just to put some light on the subject:
- matte (lambertian) surfaces reaction is the same in every direction so the colour mostly depends on the surrounding lights (thus the direct lighting algorithms)
- specular surfaces reflects light in a single direction, so it is quite easy to bounce light and follow a path
- glossy surfaces are in between, they can have a very strong reflection around a single direction, but if there is no light in that direction but a strong light outside, most of the perceived illumination will come from that outlying light.
Glossy surfaces reaction is really dependent on the product of the surface BSDF and the surrounding illumination, thus the great number of research papers on integrating products of functions.

Jeanphi

[CoolColJ]

Don't know if you guys have read this paper? SPPM with Metropolis sampling

http://cgcad.thss.tsinghua.edu.cn/chenj ... eprint.pdf

This paper presents an improvement to the stochastic progressive photon mapping (SPPM), a method for robustly
simulating complex global illumination with distributed ray tracing effects. Normally, similar to photon mapping
and other particle tracing algorithms, SPPM would become inefficient when the photons are poorly distributed.
An inordinate amount of photons are required to reduce the error caused by noise and bias to acceptable levels.
In order to optimize the distribution of photons, we propose an extension of SPPM with a Metropolis-Hastings
algorithm, effectively exploiting local coherence among the light paths that contribute to the rendered image. A
well-designed scalar contribution function is introduced as our Metropolis sampling strategy, targeting at specific
parts of image areas with large error to improve the efficiency of the radiance estimator. Experimental results
demonstrate that the new Metropolis sampling based approach maintains the robustness of the standard SPPM
method, while significantly improving the rendering efficiency for a wide range of scenes with complex lighting.

[jeanphi]

Hi,

I had seen this paper, Toshiya has also made available a new paper with another Markov chain approach to be presented at next SIGGRAPH. It looks quite promising and easy to implement.

Jeanphi

[CoolColJ]

Frankly SSPM's ability to find and render difficult refracted caustics is unmatched in my quick tests, even by MLT style renderers, but it takes so damn long to converge to a perfectly smooth finish

I wonder if there is a way to combine SSPM's caustic ability with plain bidrectional path traicng for all the other regular conditions etc
With a small radius size it just about looks like path tracing, noise wise....

[SATtva]

You're free to reduce photon radius to as low value as you wish.