On Thu, 5 Nov 2020, Alex Bennée wrote:
Julien Grall julien@xen.org writes:
(+CC Arnd, JPB)
On 05/11/2020 10:46, Alex Bennée wrote:
Stefano Stabellini stefano.stabellini@xilinx.com writes:
Hi Alex,
Thank you for doing this. I am looking forward to STR-20, I have been wanting a clean-up in that area for years. Also STR-19 is well worded and looks mostly fine. I only have a comment on the last statement:
QEMU should also be able to act as the stub setup to pass IOREQ events to a separate vhost-user daemon as a stepping stone to a future Xen aware vhost-user stub program.
Xen can actually have multiple IOREQ servers running for the same domain. Given that the code for receiving IOREQs is minimal and easily portable, it would be better to make any other daemon a proper IOREQ server talking to Xen directly, rather than having to go via QEMU.
The intention is to enable testing of the existing virtio daemons (for example virtio-gpu) where was can translate an IOREQ message and forward a vhost-user event to the daemon. I'm not expecting it to be the final performance orientated solution.
The architecture would look cleaner and would lead to far better performance, removing a step in the hot path. In other words I think the vhost-user daemon should be run as its own IOREQ server.
Well this is where we may need to consider supporting another message format from Xen. Remember the individual daemons are attempting to be hypervisor agnostic with a minimal shim dealing with the specifics of setting up on each hypervisor. That said maybe terminating IOREQ and vhost-user messages isn't that tricky or maybe another message stream makes more sense (vfio user messages?).
Receiving IOREQs directly in the virtio deamon would be trivial. It can be done in about 100 lines of code. I wouldn't worry about it.
We could discuss more standard message formats. I am certainly open to the idea. However, keep in mind that vfio is not a standard, it is a Linux interface. And also that Xen ioreqs go back to 15 years, they predate VFIO, and it would be good to align to what is already used in production on x86-land (they have been running multiple IOREQ servers for years now.)
Overall, given that it would take a couple of hours to add IOREQ handling to something like virtio-gpu, I don't think it is worth the effort.
To me, the interesting questions are the ones Julien raised about the memory mapping hypercalls.
I haven't dug into the differences yet but I suspect there is a fair degree of commonality given the problem domain.
The main difference you are going to face is accessing the guest memory. In the case of Xen, the process has to issue hypercalls in order to map/unmap guest memory. This will be mapped in its memory address space.
Currently QEMU does this for vhost-user backends and passes over a FD to the daemon.
It looks oriented to an architecture where QEMU is central to the VM management, i.e. the VMM. QEMU is not a VMM when run on Xen. In a Xen based architecture, QEMU is like one of the virtio daemons, and we can have several.