On Tue, 14 Sep 2021, Andrew Cooper wrote:
On 14/09/2021 15:44, Alex Bennée wrote:
Andrew Cooper andrew.cooper3@citrix.com writes:
On 13/09/2021 13:44, Alex Bennée wrote:
Hi,
As we consider the next cycle for Project Stratos I would like to make some more progress on hypervisor agnosticism for our virtio backends. While we have implemented a number of virtio vhost-user backends using C we've rapidly switched to using rust-vmm based ones for virtio-i2c, virtio-rng and virtio-gpio. Given the interest in Rust for implementing backends does it make sense to do some enabling work in rust-vmm to support Xen?
There are two chunks of work I can think of:
- Enough of libxl/hypervisor interface to implement an IOREQ end point.
No libxl here at all.
As of Xen 4.15, there are enough stable interfaces to implement simple IOERQ servers.
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com... was the commit where I removed the final unstable interface from varstored (terrible name) which is a dom0 backend for UEFI secure variable handling. As such, it also serves as a (not totally simple) reference of an IOERQ server.
There are a few bits and pieces of rust going on within Xen, and a whole load of plans. Also, there is a lot of interest from other downstreams in being able to write Rust backends.
We've got a placeholder xen and xen-sys crates, and placeholder work for supporting cross-compile as x86 PV and PVH stubdomains.
Are these in the rust-vmm project is elsewhere?
https://crates.io/crates/xen-sys
When I say placeholder, I really do mean placeholder.
To start this work meaningfully, we'd want to make a repo (or several) in the xen-project organisation on github or gitlab (we have both, for reasons), and set these as the upstream of the xen and xen-sys crates.
The want to have a simple IOREQ server compiled either as a dom0 backend, or as a PV or PVH stubdomains influences some of the design decisions early on, but they're all no-brainers for the longevity of the work.
Just to clarify nomenclature is a PVH stubdomain what I'm referring to as a bare metal backend, i.e: a unikernel or RTOS image that implements the backend without having to transition between some sort of userspace and it's supporting kernel?
I think so, yes, although calling it "bare metal" seems misleading for something which is a VM targetted at a specific hypervisor...
I started work on trying to reimplement varstored entirely in Rust as a hackathon project, although ran out of time trying to make hypercall buffers work (there is a bug with Box and non-global allocators causing rustc to hit an assert(). In the short term, we'll have to implement hypercall buffers in a slightly more irritating way).
Furthermore, stick to the stable hypercalls only. Xen's C libraries are disaster for cross-version compatibility, and you absolutely do not want to recompile your rust program just to run it against a different version of the hypervisor. The plan is to start with simple IOREQ servers, which are on fully stable interfaces, then stabilise further hypercalls as necessary to expand functionality.
Are the hypercalls mediated by a kernel layer or are you making direct HVC calls (on ARM) to the hypervisor from userspace?
For a dom0 backends irrespective of architecture, you need to issue ioctl()'s on the appropriate kernel device.
For a PV/PVH stubdom, you should make a call into the hypercall_page https://xenbits.xen.org/docs/latest/guest-guide/x86/hypercall-abi.html because Intel and AMD used different instructions for their equivalent of HVC.
ARM doesn't have the hypercall page ABI, so I'd expect the hypercall implementation to expand to HVC directly.
See for example arch/arm64/xen/hypercall.S in Linux