Hi Russel,
From: Russell King [mailto:linux@armlinux.org.uk] Sent: Friday, October 21, 2022 2:17 PM To: James Morse james.morse@arm.com Cc: Salil Mehta salil.mehta@huawei.com; Joyce Qi joyce.qi@linaro.org; Jonathan Cameron jonathan.cameron@huawei.com; Lorenzo Pieralisi lorenzo.pieralisi@linaro.org; Jean-Philippe Brucker jean-philippe@linaro.org; Jean-Philippe Brucker jean-philippe.brucker@arm.com; linaro-open-discussions@op-lists.linaro.org; salil.mehta@opnsrc.net; mehta.salil.lnk@gmail.com Subject: Re: Linaro-open-discussions Digest, Vol 25, Issue 3
On Fri, Oct 21, 2022 at 02:01:20PM +0100, James Morse wrote:
Hi Russell,
On 21/10/2022 12:37, Russell King (Oracle) wrote:
On Fri, Oct 21, 2022 at 10:45:48AM +0100, James Morse wrote:
Hi Russell, Salil,
On 21/10/2022 08:38, Russell King (Oracle) wrote:
This is fine if the guest hasn't already brought the vCPU online, but if it has, blocking CPU_ON doesn't prevent the guest continuing to use the vCPU. How would that aspect be addressed with your approach?
CPU_ON shouldn't block, but can (now) return DENIED if the firmware policy
means the CPU
is "pretend not-present".
That is what I meant. Not "block" as in "wait until something".
Sorry - I read that wrong! (I've been in lockdep hell all morning).
ACPI already has mechanisms for this. The _STA value/policy can only change
when the CPU
is offline. If firmware wants request a CPU is taken offline, it can issue
a eject-request
notification. If the OS will then take the CPU offline, and call _EJ0 to say
its done, as
well as updating _OST with its progress.
As far as I'm aware, this isn't yet implemented in mainline kernels for aarch64, but the mechanism you describe is used for x86.
ACPI Handshake code is common to all the architectures and yes there are some arch specific hooks (__weak functions like acpi_{un}map_cpu) which must be overridden if "Physical CPU Hotplug" has to be supported like in x86 but this is not our intention. ARM64 does not have physical Hotplug support and we are only targeting virtual CPU Hotplug here. We are just leveraging the ACPI handshake code, meant for the physical vCPU Hotplug hot-{add, remove}, to make vCPUs as {not-}present (well kind of, in actual they are always present) in the Linux kernel while being hot-{added, removed} through those ACPI events from the firmware/VMM.
James has done most of the hard bits and suggested kernel changes to support this in August 2022:
[1] https://gitlab.arm.com/linux-arm/linux-jm/-/commits/virtual_cpu_hotplug/rfc/...
[Meaning of making {not-}present in above patches means {not-}making the CPU device available by {un-}registering_cpu() according to the ACPI events]
You might want to check, a variant of James approach which also conditionally {un-}sets the present cpu mask.
[2] https://github.com/salil-mehta/linux.git virt-cpuhp-arm64/rfc-v2/jmorse-variant-with-cond-present-cpu
[Meaning of making {not-}present in above patches means {not-}making the CPU device available by {un}register_cpu() AND also {un-}set'ing the CPU bit in the present_cpu_mask according to the ACPI events]
Above can be tested using these QEMU changes: [3] https://github.com/salil-mehta/qemu.git virt-cpuhp-armv8/rfc-v1-port29092022
Thanks Salil