Premultiplied alpha

[Lord Crc]

Hi,

I'm trying to tackle the premultiplied alpha issue described here: http://www.luxrender.net/mantis/view.php?id=1015

Digging a bit deeper it seems that we're not doing this right or ideal, or I'm properly confused

First, it seems OpenEXR assumes the data is premultiplied by alpha, always. Secondly, PNG assumes the data is never premultipled. Currently both depends on the premultiplied flag.

As such I'm questioning the point of having a premultiplied alpha flag at all. For output which doesn't store alpha, we IMHO shouldn't premultiply alpha (no way to recover original), and for the outputs we have the handling is given in the file format specification anyway.

Thus I suggest we remove the premultiplied alpha flag, and add the necessary code in the EXR writer to perform the premultiplication.

edit: References
http://www.w3.org/TR/PNG-Rationale.html ... lied-alpha
http://www.openexr.com/TechnicalIntroduction.pdf

[jeanphi]

Hi,

The premultiplication must be handled before the splatting on the film to be most usefull so the option should stay for the splatting but shouldn't be used afterwards.

Jeanphi

[Lord Crc]

The premultiplication must be handled before the splatting on the film to be most usefull so the option should stay for the splatting but shouldn't be used afterwards.

Well, to correctly handle EXR writing we then need to selectively multiply by alpha in case premult wasn't selected by the user, and in the case of PNG writing we need to selectively divide by alpha in case the premul was selected.

Only the TGA and framebuffer outputs would be affect by the premult option. For the framebuffer it doesn't make a big difference once we incorporate alpha handling in the GUI, and we have a case in mantis where you suggested we remove the TGA output (which I agree on).

That is why I don't quite see the point of having the option.

[jeanphi]

Hi,

The option makes no sense if applied after the splatting, however if applied before the splatting it's really useful: say you have an architectural glass window pane: samples reflected on the glass won't be multiplied while samples going through the glass will become black. That way you get perfect background integration. With such a behaviour, you can't change the option during the render, you have to restart, but it really has a use. It's actually quite similar to the path integrator includeenvironment option.

Jeanphi

[Lord Crc]

Ok, I'll add the needed adjustments to the EXR and PNG writers then.

edit2: lemme try the adjustments once more.

[Lord Crc]

Ok, pushed a fix (I hope ).

We will need to document this in the wiki etc though. Something along these lines?

Note regarding the premultiplied alpha option:
OpenEXR requires that the colors are premultiplied, while PNG requires that the colors are NOT premultipled (straight). TGA output is "raw" and outputs either straight or premultiplied depending on the premultiplied alpha flag.

LuxRender tries to adhere to this, however due to technical reasons one might experience that PNG output is suboptimal when using premultiplied alpha. Similarly one might experience that OpenEXR ouput is suboptimal when NOT using premultiplied alpha.

As such it is recommended that one uses the OpenEXR or TGA output when the premultiplied alpha option is enabled, and PNG otherwise. When using premultiplied alpha it's required to use alpha channels for OpenEXR and TGA.

[jeanphi]

Hi,

Is that a requirement from the file format standard or just the way it works in LuxRender? Is it absolutely necessary to keep this behaviour? I think it'd be much easier and less surprising to have the same output for all formats (I guess most users won't understand why they see the background during the render and not in their EXR file). For PNG writing you can't unmultiply alpha, that's absolutely impossible because in the same pixel you might have stored values with alpha=1 and values with alpha=0 (glass transmission and glass reflection), so trying to adhere to such a spec is impossible. Better do what the user asks for and let the standard aside in this particular case.
I think we should get back to the previous and simpler version but making sure we have consistent behaviour between all file formats.

I think there's a typo in your last commit:
film/fleximage.cpp:950: I find the ! before premultiplyalpha bogus
film/fleximage.cpp:970: ditto

Jeanphi

[Lord Crc]

Is that a requirement from the file format standard or just the way it works in LuxRender?

OpenEXR standard is that if you write an alpha channel, the colors should be premultiplied. The PNG standard states quite explicitly that the colors should never be premultiplied ("straight colors").

Is it absolutely necessary to keep this behaviour? I think it'd be much easier and less surprising to have the same output for all formats (I guess most users won't understand why they see the background during the render and not in their EXR file). For PNG writing you can't unmultiply alpha, that's absolutely impossible because in the same pixel you might have stored values with alpha=1 and values with alpha=0 (glass transmission and glass reflection), so trying to adhere to such a spec is impossible. Better do what the user asks for and let the standard aside in this particular case.
I think we should get back to the previous and simpler version but making sure we have consistent behaviour between all file formats.

Ok, in that case we need to document properly that the output is does not adhere to the standards if you select OpenEXR without premultiply alpha, or premultiply alpha and PNG (when alpha channels are enabled). TGA v2 standard has a field to inform if the colors are premultiplied or not, however this is in an "additional info" block which we don't write. So for TGA we need to inform that the output is "raw".

I think there's a typo in your last commit:
film/fleximage.cpp:950: I find the ! before premultiplyalpha bogus
film/fleximage.cpp:970: ditto

No typo I think, if the colors are not premultiplied already we need to do it before writing the EXR (so that they're always premultiplied one way or another). If no alpha channel is specified we don't do anything.

I'm fine with whatever. If we want to stay within the standards I think my current implementation, along with the docu-text I wrote, is a good effort to do so. It would be nice with some feedback from users who use alpha channels in their workflow, as this is not an area I'm very familiar with.

[jeanphi]

Hi,

The EXR standard is really strange, but I can understand why it is like this. However the premultiplyalpha option has really a special semantic, much like the includeenvironment option in path integrator: it is there to allow proper compositing without aliasing when you're using a colored background, so it has to be applied before splatting and cannot be undone later. If you look at it to be like the includeenvironment option, then you don't have an issue with PNG because what you want is really a black background, the plain color is black.
So we could claim standard conformance by keeping your tweak to the EXR output (maybe add an option to be non conformant, or keep the previous default and add an option to enforce standard conformance), and keeping the old PNG output.

Jeanphi

[Lord Crc]

Well if you're doing composition using a premultiplied pipeline, then you get the nice effects as you mention. So I suppose this is why OpenEXR does it. However IMHO it wouldn't have killed them to add a flag signaling if the alpha channel was indeed premultiplied or not....

Anyway, that sounds like a good compromise, I'll make the changes after dinner