Hi Alex,

On Fri, 4 Jun 2021 at 11:32, Alex Bennée alex.bennee@linaro.org wrote:

Hi All,

I was having a discussion the other day with an external team looking at using virtualised graphics for a production platform. They had some questions about features I couldn't answer due to my imperfect understanding of all the moving parts. Does anyone know of any work being done on the following:

Alternate pixel formats

I believe this is related to pixel data being expressed in different colour spaces. I see that VIRTIO_GPU_CMD_RESOURCE_CREATE_2D has the notion of different RGB layouts for pixel data. For 3D pixel data is this something that is ever visible to the virtio layer or is this all handled from within the OpenGL interface? I see there is a VIRTIO_GPU_CMD_SET_SCANOUT_BLOB for defining resources that can get referenced but that is still restricted to the same virtio_gpu_formats.

Overlay planes

As I understand it these are used for compositing multiple graphics sources. We currently working on a virtualised virtio-video backend and at some point we'll want to consider how you might overlay a 3D view from one VM with a video source from a second VM and either do the compositing in the host system or potentially a 3rd VM. Is anyone aware of any work being done in this direction yet?

I'm fairly new to the virtio-gpu and virtio_wl stuff, but whether to use overlay planes from the SoC display controller is usually decided by the compositor e.g. Weston

It will decide given the various clients and buffers it has to composite, and it also has the knowledge of the underlying display controller hardware on the SoC (how many overlay planes, what pixel formats they accept etc).

If it can be scanned out on an overlay plane that is preferable from a power efficiency PoV. Also on some SoCs with secure video pipeline we've worked on, the GPU didn't have the ability to handle secure buffers which meant they always had to be scanned out via a overlay plane (with associated limitations on what could be achieved with the composition).

So my guess is the VM would pass buffers using something like virtio-wl, and the compositor wherever that is running (basically the entity talking to the SoCs DRM/KMS kernel driver) would continue to make the decision on whether to scan out the buffer directly using an overlay plane or not.

vsync

The existing virtio-gpu spec doesn't mention vsync at all. My last experience with active vsync synchronisation was back in the days of the Atari ST so I'm a little behind on how it works with modern 3D. Is this something that ever needs to be exposed to the guest or is it preserve of the host system to ensure newly drawn frames are swapped out at the right time to avoid tearing? The Khronos pages refer to Swap Interval being a platform specific extension. Does this mean it needs to be something specifically handled by VirGL?

DRM

Probably the least interesting problem from an open source point of view but I guess for media consumers their needs to be a solution for DRM protected streams to leave a VM and be handled "somewhere" else. Are there any proposals for exposing DRM via VirtIO that anyone is aware of?

Most of the work we did before on DRM at the lowest level relies on being able to pass handles (aka file descriptors) representing the secure buffers around the system. e.g. register secure buffer fd with OP-TEE, take a fd of the decrypted buffer from OP-TEE and import it into the video decoder device. Take a fd of the decompressed image from the decoder and import it into the display controller.

If you can do those operations, without the CPU having to access the data in the buffers (e.g. to access some meta data to configure the decoder) then you should be able to do a secure playback.

I'm not sure you need to particularly "expose" any of the upper layers of DRM e.g. Widevine to virtio (although I'm new to virtio world so maybe there are usecases that I'm not aware of).

Thanks,

Peter.