Add AlphaMode and transparency

[madsmtm]

Add:

impl<D, W> Surface<D, W> {
    pub fn alpha_mode(&self) -> AlphaMode { ... }
    pub fn supports_alpha_mode(&self, alpha_mode: AlphaMode) -> bool { ... }

    // `resize` now calls `configure`:
    pub fn resize(&mut self, width: NonZeroU32, height: NonZeroU32) -> Result<(), SoftBufferError> {
        self.configure(width, height, self.alpha_mode())
    }

    pub fn configure(
        &mut self,
        width: NonZeroU32,
        height: NonZeroU32,
        alpha_mode: AlphaMode,
    ) -> Result<(), SoftBufferError> { ... }
}

#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Debug, Default)]
pub enum AlphaMode {
    #[default]
    Opaque,
    Ignored,
    Premultiplied,
    Postmultiplied,
}

This fixes #17 and prepares for #98 / #317.

As noted in the transparency issue, it is important to make a distinction between straight and premultiplied alpha. The former can be easier to work with, but the latter is often what's actually supported by compositors.

One mode that is a bit odd here is Opaque, but it's necessary for the Web backend, because that platform doesn't support zero-copy RGBX, the alpha channel is always read (at least from what I could figure out). Adding this mode (and thus requiring that alpha channel to be 255) fixes #207.

The implementation of these modes for each platform is as follows (I have tested all these):

• Android: Transparency doesn't seem to be supported by Winit, so I haven't enabled transparency in Softbuffer yet either (since I couldn't test it).
• CoreGraphics: All right now, will be Opaque and Premultiplied with IOSurface: Use IOSurface on macOS/iOS #329.
• Wayland: Opaque, Ignored, Premultiplied.
• Web: Opaque and Postmultiplied. Premultiplied can be supported in the future with ImageBitmap IIUC.
• Win32: I think it could support Premultiplied, but I couldn't get it to work properly, so only did part of it, I'm not too familiar with Windows. I suspect it might also need something like fixed window transparency winit#2503.
• X11: Transparency not yet implemented, but pretty sure that Premultiplied could be supported.

We could fairly easily implement a conversion step to support postmultiplied alpha when premultiplied is supported by the backend, but I'd like to migrate towards a more efficient design where each backend always do zero-copying, so I haven't done that.

Expected behaviour

I've created an example transparency.rs, which renders a few different shades of orange and yellow. The expected result are as follows:

Opaque / Ignored

Premultiplied

Postmultiplied

[madsmtm]

It looks like this currently:

I suspect that this may be an intentional limitation of transparency in the "root" / that goes through the top-level layer, perhaps as a form of security measure to avoid users reading contents of contents below the window in a shader or smth? But I'll need to investigate this further, might be possible to fix with IOSurface.

[madsmtm]

I tested with #329, that fixes it. It's probably because CGImage/CGContext or whatever is postmultiplied internally (which prevents certain kinds of alpha compositing).

[madsmtm]

What should we call this? Wikipedia uses "straight", wgpu uses PostMultiplied.

[ids1024]

PostMultiplied seems a bit clearer, and if in doubt I'm inclined to make softbuffer APIs match wgpu.

[madsmtm]

One problem with going with the wgpu choice here is that PostMultiplied is not in the WebGPU spec anywhere (and the spec actually uses the term "unpremultiplied alpha").

I traced the value back to gfx-rs/gfx#2509, which seems based on https://docs.vulkan.org/refpages/latest/refpages/source/VkCompositeAlphaFlagBitsKHR.html, so I guess we'd be basing it on the Vulkan spec.

[madsmtm]

Actually, that link has an AlphaMode::Inherit, I guess we could use that approach instead of the AlphaMode::Opaque hack?

So instead of:

• AlphaMode::Opaque = "require alpha channel 1.0"
• AlphaMode::Ignored = "alpha channel ignored".

We could do:

• AlphaMode::Opaque = "alpha channel ignored" (to match WGPU's CompositeAlphaMode::Opaque)
• AlphaMode::Inherit = "whatever the platform wants to default to, which usually means you should set the alpha channel to 1.0 to be cross-platform".

The difference between the first AlphaMode::Opaque and the second AlphaMode::Inherit would be that we probably wouldn't need to debug-assert that the alpha mode is as we expect in fn present if we use AlphaMode::Inherit.

[ids1024]

Inherit doesn't seem like the right term for that. That Vulkan spec says "the application is responsible for setting the composite alpha blending mode using native window system commands". But that wouldn't be true in softbuffer, at least not on all platforms. So there's nothing to inherit the alpha mode from.

The difference between the first AlphaMode::Opaque and the second AlphaMode::Inherit would be that we probably wouldn't need to debug-assert that the alpha mode is as we expect in fn present if we use AlphaMode::Inherit.

On the other hand, the caller also doesn't benefit from that debug assert, which helps ensure they're doing things in a correct cross-platform way.

AlphaMode::Opaque is a bit of a hack, but it seems handy enough for someone wanting to put some pixels on the screen and be sure it will work across platforms.

[madsmtm]

I elected to call this Premultiplied instead of PreMultiplied (not camel casing), because most of the references I could find on the interwebs seemed to prefer to write it in one word instead of hyphenating as "pre-multiplied".

[ids1024]

If we're keeping both methods, it may be good to document the fact that resize() is just equivalent to configure() with self.alpha_mode(), or the relationship between the functions may be a bit unclear to someone just reading the API docs.

[madsmtm]

I'm not sure what we should do with configure actually. It's gonna need to take PixelFormat as well, and probably also PresentMode (since that influences the amount of buffers).

I'm a bit tempted to instead do something like:

impl Surface<'_> {
    pub fn set_size(width: NonZero<u32>, height: NonZero<u32>) /* does not return an error */ {
         self.width = width;
         self.height = height;
    }

    pub fn set_alpha_mode(alpha_mode: AlphaMode) {
        self.alpha_mode = alpha_mode;
    }

    pub fn set_pixel_format(pixel_format: PixelFormat) {
        self.pixel_format = pixel_format;
    }

    pub fn set_present_mode(present_mode: PresentMode) {
        self.present_mode = present_mode;
    }

    pub fn buffer_mut(&mut self) -> Result<Buffer<'_>, SoftbufferError> {
        self.inner.configure(self.width, self.height, self.alpha_mode, self.pixel_format, self.present_mode)?;
        self.inner.buffer_mut()
    }
}

That is, configure/resize the buffers internally before each buffer_mut call. The benefit here would mostly be for error handling, but also that 5 is kinda too many parameters for a function, especially if given that you often only want to adjust a few of Softbuffer's parameters, but leave the rest as default.

Idk., on the other hand, there's probably value in separating the "configure" errors from the "get buffer" errors.

Maybe a helpers struct like wgpu's SurfaceConfiguration is the way to go instead? That would make a resize when using default options just be:

surface.configure(SurfaceConfiguration {
    width: ...,
    height: ...,
    .. // rest is default
})?;

[ids1024]

We have .width() and .height() methods on Buffer already. May as well also have an AlphaMode method as well?

[madsmtm]

The reason I didn't is that I think the alpha mode feels conceptually more of a configuration parameter to the compositor, and thus a thing that "belongs" more on the surface.

E.g. if we had the ability to create buffers independently from surfaces, I'd want to configure the alpha mode in the surface and not on the buffer (in practice, we'd need the alpha mode to select the right pixel format, but still, conceptually that's how it feels like is should work).

On the other hand, the width and height are perhaps also conceptually surface properties, so maybe it's fine to expose them both places.

[ids1024]

That would be https://docs.rs/winit/0.30.12/winit/window/struct.WindowAttributes.html#method.with_transparent, right?

So something that's left to winit.

[ids1024]

Or looking at this differently: if Vulkan, OpenGL, or wgpu won't set this property, then it makes sense that softbuffer won't either.

[madsmtm]

I think wgpu calls setOpaque on the layer they're working with, which might be the root layer if it was created from CAMetalLayer, and otherwise it is a sublayer.

In any case, Winit doesn't currently set this property on its content view/layer, only on the window itself.

[ids1024]

Other than some very minor details, this seems pretty good. Though I have less ability to comment on backends other than Wayland. It seems like a good way to handle alpha and a step towards more complete support for handling different formats.

[ids1024]

Maybe make this "This is the default, and supported on all platforms." to be a little more explicit (though that's implied by the lack of a "Platform Dependent Behavior" section.)