From: James Morse [mailto:james.morse@arm.com] Sent: Thursday, June 3, 2021 4:39 PM To: Jean-Philippe Brucker jean-philippe@linaro.org; linaro-open-discussions@op-lists.linaro.org Cc: Salil Mehta salil.mehta@huawei.com; Jonathan Cameron jonathan.cameron@huawei.com; lorenzo.pieralisi@arm.com Subject: Re: [RFC linux 0/5] KVM: arm64: Let userspace handle PSCI
'lo
On 20/05/2021 14:07, Jean-Philippe Brucker wrote:
As planned during the vCPU hot-add discussions from previous LOD meetings, this prototype lets userspace handle PSCI calls from a guest.
The vCPU hot-add model preferred by Arm presents all possible resources through ACPI at boot time, only marking unavailable vCPUs as hidden. The VMM prevents bringing up those vCPUs by rejecting PSCI CPU_ON calls. This allows to keep things simple for vCPU scaling enablement, while leaving the door open for hardware CPU hot-add.
This series focuses on moving PSCI support into userspace. Patches 1-3 allow userspace to request WFI to be executed by KVM. That way the VMM can easily implement the CPU_SUSPEND function, which is mandatory from PSCI v0.2 onwards (even if it doesn't have a more useful implementation than WFI, natively available to the guest). An alternative would be to poll the vGIC implemented in KVM for interrupts, but I haven't explored that solution. Patches 4 and 5 let the VMM request PSCI calls.
As mentioned on the call, I've tested the udev output on x86 and arm64, as expected its the same: | root@vm:~# udevadm monitor | monitor will print the received events for: | UDEV - the event which udev sends out after rule processing | KERNEL - the kernel uevent | | KERNEL[33.935817] add /devices/system/cpu/cpu1 (cpu) | KERNEL[33.946333] bind /devices/system/cpu/cpu1 (cpu) | UDEV [33.953251] add /devices/system/cpu/cpu1 (cpu) | UDEV [33.958676] bind /devices/system/cpu/cpu1 (cpu)
(I've not played with the KVM changes yet)
The guest needs additional support to deal with hidden CPUs and to gracefully handle the "NOT_PRESENT" return value from PSCI CPU_ON. The full prototype can be found here:
Hopefully its possible to make those silent!
Regarding PSCI, I believe you are referring to the return value in the below code hunk(s) and the PSCI spec changes mentioned in the patch[1]?
@@ -209,6 +209,8 @@ static int __psci_cpu_on(u32 fn, unsigned long cpuid, unsigned long entry_point) int err;
err = invoke_psci_fn(fn, cpuid, entry_point, 0); + if (err == PSCI_RET_NOT_PRESENT) + return -EPROBE_DEFER; return psci_to_linux_errno(err); }
[1] https://jpbrucker.net/git/linux/commit/?h=cpuhp/devel&id=afa4089cb122637...
Perhaps, by "additional" support with the hidden cpus you mean its effect on the sizing of various data-structures and other features dependent on the explicit awareness of the present and possible cpus?
Salil.