Reworking renderer statistics

[J the Ninja]

Hi guys, not sure if here is the right place to add this

I was using maxwell for a commercial project the other day and i noticed that instead of render passes they use a Sampling level statistic. Here is some basic info from the documentation of maxwell

Sampling Level

As the render calculates you will see the image output get less and less noisy and the Sampling Level (SL) increase continuously. The SL is a basic measure of quality, the higher it is the less noise the render will have. A few key points regarding SL:

It is important to understand that there is no fixed SL number to get an acceptable quality level, because it depends entirely on the scene. Some scenes can be completely noise-free at SL 8 or even earlier, while others may need to get to SL 16 or higher. As a general rule an exterior scene will render much quicker than an interior scene because most of the lighting in an interior will be indirect and there are many more light bounces to calculate.

Each new SL takes approximately twice as long to reach as the previous one. If it took 15 minutes to go from SL 4 to SL 5, it will take about 30 minutes to go from SL 5 to SL 6.

It is uncommon needing to render beyond SL 19 - 20.

I guess it has the same rule like ive often read here , that if your render isnt clear at x samples you wont see a huge change until 2x.
Thought i should added it here since you rework the statistics, in case you wanted to check it out too

This might actually be a good idea. I'm guessing Maxwell's SL scales by powers of 2 (ex, sample level 8 means 2^8 samples per pixel)...that's actually kind of a useful thing. Regardless of how the Next Limit guys actually coded theirs, I'm wondering if we should consider switching samples to some kind of exponential measurement. Powers of two could be nice since that means low discrepancy always finishes a run at a "round number" of quality. The main issue here is that maybe a "quality measure" which has a direct/linear relationship with the number of samples on film isn't such a great idea. After all, adding 100 samples to a film with only 10 samples is going to have a huge effect. Adding 100 samples to a film that already has 10,000 isn't going to do anything noticeable at all.

So perhaps have a quality measure that is some kind of exponent? Powers of 2 perhaps? That would give us a wide range of values that doesn't leave silly-small numbers at s/px used for things like previews or animations (like powers of 10 would). Wondering if we should just refer to these as "passes" to give some consistency with SPPM. (and Cycles, which uses 1 pass = 1 sample/pixel)

Also, the carbonflux unit works fairly well with powers of 2, as 2^13.37 happens to be about 10000




.....and while we are on this train of though, which do we list efficiency as a percentage? Lux usually operates with path-tracing algorithms, which pretty much always turn in eff percentages way over 100. Why not just use a decimal? So efficiency is, say, 5.32 instead of 532%?

[Pilchard123]

Also, the carbonflux unit works fairly well with powers of 2, as 2^13.37 happens to be about 10000

Really? I mean, really? It took me about a minute to see that.

[rendermagic]

So, if "Contributions per pixel" are a function of efficiency and samples per pixel, wouldn't it make more sense to measure the quality of the render on Contributions instead of Samples? This could easily take into consideration the varying efficiency values we get when we use different algorithms.

[Abel]

I'm wondering if we should consider switching samples to some kind of exponential measurement. Powers of two could be nice since that means low discrepancy always finishes a run at a "round number" of quality. The main issue here is that maybe a "quality measure" which has a direct/linear relationship with the number of samples on film isn't such a great idea.

Maxwell's system is nice in the way that the numbers may be easier to grasp with most final renderings ending up in the 10-30 range, but after having traded Maxwell for LuxRender I've personally come to prefer our straight linear system. As you say, the quality still needs to be judged on the time that has elapsed and the quality of the rendering so far; the sampling level is just some kind of a reference.

In practice, I often render an image for a small amount of time, see how many samples per pixel it has reached and look if the visual result is in line with what I'd expect for the number of samples; based on that information, I estimate how good the image will look after the target time period. With LuxRender's system, it's easy to see that if I got 50 s/px in 30 minutes, then I'll need 5 hours to get to 500 s/px. But in Maxwell, if after 10 minutes the sampling level is 12.0 and I expect to need 18.0, how am I supposed to go about estimating? I remember there is some statistic in Maxwell that indicates when the next sampling level will be reached, but that just illustrates the problem that it is hard to do that based on the sampling level number provided.

[cwichura]

Maybe then a useful thing for the new statistics engine to calculate and display would be the estimated "X per hour" for the user, where X would be whatever makes the most sense (samples, sample level, etc) based on the current algo?

[Abel]

Maybe then a useful thing for the new statistics engine to calculate and display would be the estimated "X per hour" for the user, where X would be whatever makes the most sense (samples, sample level, etc) based on the current algo?

That's the trouble, with the proposed system there isn't a meaningful X per hour anymore. Unless we would have both the current samples per pixel AND some new unit of measurement, but I think that would just confuse things. So the more I think about it, the more I think we should just stick to what we have.

[Pilchard123]

But in Maxwell, if after 10 minutes the sampling level is 12.0 and I expect to need 18.0, how am I supposed to go about estimating? I remember there is some statistic in Maxwell that indicates when the next sampling level will be reached, but that just illustrates the problem that it is hard to do that based on the sampling level number provided.

Assuming that moving from SL(X) to SL(X+1) takes twice the time of SL(X-1) to SL(X), then the time taken to go from SL(X) to SL(Y) where Y>X is [n * (2^Y)] - [n * (2^X)] = n[(2^Y)-(2^X)], where n is a constant. Given a time T to go from SL(0) with no samples, to SL(1), then this equation becomes T[(2^Y)-(2^X)], since T[(2^1)-(2^0)] = T[2-1] = T1 = T

This can be extended to be from SL(0) to SL(X). The time for this to happen, t = T[(2^X)-(2^0)] = T[(2^X)-1].
T therefore = t/[(2^X)-1]. It can also be used for SL(A) to SL(X) where X>A. t = T[(2^X) - (2^A)]

Does that make sense, or did I make an ugly assumption somewhere?

Using this method, t = 600 seconds (minutes would work too, but I like seconds), X = 12.0 and Y = 18.0, 600 = T[(2^12)-1]

T = 600/[4096-1] = 0.14652014652014652014652014652015...

SL(12) to SL(18) = 0.14652014652014652014652014652015...[262144-4096] = 37809.230769230769230769230769231...

~ 630 minutes ~ 10.5 hours. SL(12) to SL(18) roughly takes 63 times longer than SL(0) to SL(12).

[SATtva]

Omniflux, another place where we're still missing values averaging is photons per second count in SPPM -- it jumps like crazy.

[jensverwiebe]

Hi
Sorry to kill the fun, but imho we don´t need new statistic but some features fixed, solid and fast.

Whats wrong with spp ? Thats a thrue milestone for renderquality and represents a finegrained measurement same time.
Breaking this up into quality levels would bring no benefit over visual control.

Personally iám complete against introducing such new statistic-system in Lux , expecially in the release phase.

My 2 cent .... Jens

[Omniflux]

I have found the problem with the fluctuating (or is that fluxuating, since I broke it?) S/s statistic. It was blindingly obvious, but I kept mixing up part of the networking S/s code in my head.

The previous code did not use a windowed statistic, it was just samples / elapsed time. The currently displayed value is (samples - previous samples) / (elapsed time - previous elapsed time) with a first order filter then applied over the last 600 values (10 minutes worth).

Btw, averaging all the data since the beginning of rendering isn't the best approach either. Better is a rolling average, i.e. accumulate data only for the last, say, 10-minute window and average them. This way induced speed slowdowns (due to other concurrent processes, or changes in the number of rendering threads) will be dropped off after some time, and not affect the stats values forever.

This is the method currently implemented, so I am not sure what the preferred solution is.

Should I switch it back to the original non windowed value? Maybe someone can recommend a better filter?