From: Jean-Philippe Brucker [mailto:jean-philippe@linaro.org] Sent: Thursday, May 20, 2021 2:07 PM To: linaro-open-discussions@op-lists.linaro.org Cc: Salil Mehta salil.mehta@huawei.com; Jonathan Cameron jonathan.cameron@huawei.com; james.morse@arm.com; lorenzo.pieralisi@arm.com; Jean-Philippe Brucker jean-philippe@linaro.org Subject: [RFC linux 0/5] KVM: arm64: Let userspace handle PSCI
As planned during the vCPU hot-add discussions from previous LOD meetings, this prototype lets userspace handle PSCI calls from a guest.
The vCPU hot-add model preferred by Arm presents all possible resources through ACPI at boot time, only marking unavailable vCPUs as hidden. The VMM prevents bringing up those vCPUs by rejecting PSCI CPU_ON calls. This allows to keep things simple for vCPU scaling enablement, while leaving the door open for hardware CPU hot-add.
This series focuses on moving PSCI support into userspace. Patches 1-3 allow userspace to request WFI to be executed by KVM. That way the VMM can easily implement the CPU_SUSPEND function, which is mandatory from PSCI v0.2 onwards (even if it doesn't have a more useful implementation than WFI, natively available to the guest). An alternative would be to poll the vGIC implemented in KVM for interrupts, but I haven't explored that solution. Patches 4 and 5 let the VMM request PSCI calls.
The guest needs additional support to deal with hidden CPUs and to gracefully handle the "NOT_PRESENT" return value from PSCI CPU_ON. The full prototype can be found here:
https://jpbrucker.net/git/linux/log/?h=cpuhp/devel https://jpbrucker.net/git/qemu/log/?h=cpuhp/devel
Hi Jean/James, Thanks for this very useful contribution and sharing patches.
I have quickly scratched all of the patches including Linux host/kvm changes and the guest changes done by James and looks good logically. I will spent time for detailed review over this weekend and get back to you with some comments.
BTW, I have tested the hot-{add,remove} and I have found it working with at least straight forward cases. Will do more testing over next couple of days.
Again...many thanks to all of you guys for taking this pain and getting us some acceptable solution. This sets up good ground for useful discussion on 25th May 2021 LOD meeting.
Thanks Salil.
Jean-Philippe Brucker (5): KVM: arm64: Replace power_off with mp_state in struct kvm_vcpu_arch KVM: arm64: Move WFI execution to check_vcpu_requests() KVM: arm64: Allow userspace to request WFI KVM: arm64: Pass hypercalls to userspace KVM: arm64: Pass PSCI calls to userspace
Documentation/virt/kvm/api.rst | 46 +++++++++++++++---- Documentation/virt/kvm/arm/psci.rst | 1 + arch/arm64/include/asm/kvm_host.h | 10 ++++- include/kvm/arm_hypercalls.h | 1 + include/kvm/arm_psci.h | 4 ++ include/uapi/linux/kvm.h | 3 ++ arch/arm64/kvm/arm.c | 66 +++++++++++++++++++-------- arch/arm64/kvm/handle_exit.c | 3 +- arch/arm64/kvm/hypercalls.c | 28 +++++++++++- arch/arm64/kvm/psci.c | 69 ++++++++++++++--------------- 10 files changed, 165 insertions(+), 66 deletions(-)
-- 2.31.1