As planned during the vCPU hot-add discussions from previous LOD
meetings, this prototype lets userspace handle PSCI calls from a guest.
The vCPU hot-add model preferred by Arm presents all possible resources
through ACPI at boot time, only marking unavailable vCPUs as hidden.
The VMM prevents bringing up those vCPUs by rejecting PSCI CPU_ON calls.
This allows to keep things simple for vCPU scaling enablement, while
leaving the door open for hardware CPU hot-add.
This series focuses on moving PSCI support into userspace. Patches 1-3
allow userspace to request WFI to be executed by KVM. That way the VMM
can easily implement the CPU_SUSPEND function, which is mandatory from
PSCI v0.2 onwards (even if it doesn't have a more useful implementation
than WFI, natively available to the guest). An alternative would be to
poll the vGIC implemented in KVM for interrupts, but I haven't explored
that solution. Patches 4 and 5 let the VMM request PSCI calls.
The guest needs additional support to deal with hidden CPUs and to
gracefully handle the "NOT_PRESENT" return value from PSCI CPU_ON.
The full prototype can be found here:
https://jpbrucker.net/git/linux/log/?h=cpuhp/develhttps://jpbrucker.net/git/qemu/log/?h=cpuhp/devel
Jean-Philippe Brucker (5):
KVM: arm64: Replace power_off with mp_state in struct kvm_vcpu_arch
KVM: arm64: Move WFI execution to check_vcpu_requests()
KVM: arm64: Allow userspace to request WFI
KVM: arm64: Pass hypercalls to userspace
KVM: arm64: Pass PSCI calls to userspace
Documentation/virt/kvm/api.rst | 46 +++++++++++++++----
Documentation/virt/kvm/arm/psci.rst | 1 +
arch/arm64/include/asm/kvm_host.h | 10 ++++-
include/kvm/arm_hypercalls.h | 1 +
include/kvm/arm_psci.h | 4 ++
include/uapi/linux/kvm.h | 3 ++
arch/arm64/kvm/arm.c | 66 +++++++++++++++++++--------
arch/arm64/kvm/handle_exit.c | 3 +-
arch/arm64/kvm/hypercalls.c | 28 +++++++++++-
arch/arm64/kvm/psci.c | 69 ++++++++++++++---------------
10 files changed, 165 insertions(+), 66 deletions(-)
--
2.31.1
On Thu, 29 Apr 2021 15:17:36 +0000
Jonathan Cameron via Linaro-open-discussions <linaro-open-discussions(a)op-lists.linaro.org> wrote:
> On Thu, 29 Apr 2021 15:25:38 +0100
> Lorenzo Pieralisi <lorenzo.pieralisi(a)arm.com> wrote:
>
> > On Thu, Apr 29, 2021 at 09:50:06AM +0100, Jonathan Cameron wrote:
> > > > is not really working for me. If you have a command script to share
> > > > it is welcome - looking forward to testing and reviewing the DOE
> > > > patches.
> > >
> > > I'm mostly running aarch64 emulated on top of x86 - I should sanity check
> > > it on KVM at somepoint.
> > >
> > > I'm copy typing this across machines, so whilst I hope there are no typos
> > > there might be. I've stripped back my normal case (which has a big
> > > complex topology to try and hit corner cases...)
> > > First I'd suggest checking that have right EDK2 etc to bring up pxb with normal pci
> > > rp and a device.
> > >
> > > qemu-system-aarch64 -M virt,nvdimm=on -m 4g,maxmem=8G,slots=2 -cpu max -smp 4 \
> > > -kernel Image \
> > > -drive if=non,file=full.qcow2,format=qcow2,id=hd \
> > > -nographic -no-reboot -append 'earlycon root=/dev/vda2 fsck.mode=skip' \
> > > -bios QEMU_EFI.fd \ #note this needs the pxb enablement patches -maybe upstream by now.
> > > -object memory-backend-ram,size=4G,id=mem0 \
> > > -numa node,nodeid=0,cpus=0-3,memdev=mem0 \
> > > -object memory-backend-file,id=cxl-meme1,share,mem-path=/tmp/cxltest.raw,size=2G,align=2G \
> > > -device pxb-cxl,bus_nr=128,id=cxl1.1,uid=0,len-window-base=1,window-base[0]=0x4c0000000,memdev[0]=cxl-mem1 \
> > > # above range just needs to not trample on anything.
> > > -device cxl-rp,bus=cxl1.1,id=root_port13,chassis=0,slot=1 \
> > > -device cxl-type3,bus=root_port13,memdev=cxl-mem1,lsa=cxl-mem1,id=cxl-pmem0,size=2G
> >
> > Done, thank you very much. Is there a commit base on top of which I
> > can apply Dan's CXL port enumeration patches + your DOE series ?
>
> Nope. Looks like I need to send out a rebase as some of Dan's other patches hit
> mainline. Just did the merge to see how bad it is and other than a goto label
> having disappeared (and a bunch of fuzz) seems to go in fairly cleanly.
>
> I'm not sure which order the various Intel patches currently on list would apply in.
> I tried a few possible orders and got issues. Looks like they need to rebase as well.
>
> Best of all mainline is booting on this setup mid merge window which is always
> a pleasant surprise :)
>
> I'll wait to send the rebase until near the end or after the merge window.
> We'll probably still have some issues with merging until we have an order in which
> various series will merge.
>
I sent the updated DOE patches out today.
https://lore.kernel.org/linux-pci/20210524133938.2815206-1-Jonathan.Cameron…
There is still a bit of merge mess if you also pick up Dan's Port series.
I was waiting for that to merge, but as it seems to be going slowly I went
ahead with the DOE update. There is a lot of churn in the CXL code at the moment.
Also, a highly dubious blog / setup guide can be found at:
https://people.kernel.org/jic23/howto-test-cxl-enablement-on-arm64-using-qe…
May well eat babies.
Jonathan
> J
>
> >
> > Thanks a lot Jonathan,
> > Lorenzo
> >
> > > Hmm. Perhaps I should just write a blog post and include all the random corners needed
> > > to get this up. Problem then is I'd actually have to figure out what some of the
> > > parts are doing having long forgotten the answer so might take a day or two.
> > >
> > > In meantime we can carry on here.
> > >
> > > J
> > >
> > >
> > >
> > >
> > > >
> > > > Thanks,
> > > > Lorenzo
> > > >
> > > > > >
> > > > > > I remember reading there is an IRC channel cxl related (#cxl @ OFTC ?) -
> > > > > > if there is happy to switch to it rather than bothering you with these
> > > > > > queries.
> > > > >
> > > > > I tend to avoid IRC because of potential auditing issues (no logs) so
> > > > > email is the way to go.
> > > > >
> > > > > >
> > > > > > Thanks a lot !
> > > > > > Lorenzo
> > > > > >
> > > > > > > As there are some US folks who are interested in this topic (but super busy),
> > > > > > > can we do a straw poll of whether any of them can make a call on Monday?
> > > > > > >
> > > > > > > If not go for a more China/Europe friendly time?
> > > > > > >
> > > > > > > We had a few more topics brewing, but I'm not sure they will be in a state
> > > > > > > to discuss next week (or to give anyone else time to think about them
> > > > > > > in advance).
> > > > > > >
> > > > > > > Obviously good to touch on any updates to older topics as well if anyone
> > > > > > > has any!
> > > > > > >
> > > > > > > Jonathan
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > Lorenzo
> > > > > > >
> > > > >
> > >
>
Hi Lorenzo,
Thanks for your heads up. I'm okay with this, let's check others'
availability.
Regards,
Jammy
On Mon, 10 May 2021 at 19:21, Lorenzo Pieralisi via Linaro-open-discussions
<linaro-open-discussions(a)op-lists.linaro.org> wrote:
> Hi all,
>
> it would be great if the next meeting can be scheduled for May 25th or
> May 26th instead of May 24th to avoid public holidays in UK and Europe.
>
> Please let me know if that's feasible.
>
> Thank you very much.
>
> Lorenzo
> --
> Linaro-open-discussions mailing list
> https://collaborate.linaro.org/display/LOD/Linaro+Open+Discussions+Home
> https://op-lists.linaro.org/mailman/listinfo/linaro-open-discussions
>
> From: Linaro-open-discussions
> [mailto:linaro-open-discussions-bounces@op-lists.linaro.org] On Behalf Of
> Salil Mehta via Linaro-open-discussions
> Sent: Monday, April 19, 2021 11:26 AM
> To: Lorenzo Pieralisi <lorenzo.pieralisi(a)arm.com>
> Cc: linaro-open-discussions(a)op-lists.linaro.org
> Subject: Re: [Linaro-open-discussions] LOD meeting agenda for March 22nd
>
> > From: Lorenzo Pieralisi [mailto:lorenzo.pieralisi@arm.com]
> > Sent: Monday, April 19, 2021 11:11 AM
> > To: Salil Mehta <salil.mehta(a)huawei.com>
> > Cc: Jammy Zhou <jammy.zhou(a)linaro.org>;
> > linaro-open-discussions(a)op-lists.linaro.org; Jonathan Cameron
> > <jonathan.cameron(a)huawei.com>
> > Subject: Re: [Linaro-open-discussions] LOD meeting agenda for March 22nd
> >
> > On Mon, Mar 22, 2021 at 03:09:31PM +0000, Salil Mehta wrote:
> > > Hello,
> > > Please find the requested slides containing *rough* QEMU boot time figures
> > attached.
> > > I will further refine/update the slides with Firmware/Linux Guest related
> > figures
> > > and will share later.
> >
> > Hi Salil, Jonathan,
> >
> > there is work ongoing on a PSCI based virt CPU hotplug prototype
> > but it is not material for next week discussion, we will share it
> > when it can be used (and we will need help to review it and upstream it).
>
> Hi Lorenzo,
> Thanks for the update. Surely, would be more than eager to contribute in above
> activities. I guess, to realize PSCI approach, we might need changes both in
> kernel and the QEMU, may I know if you require any help in changing the QEMU
> part?
Hi Lorenzo,
Any updates of the above work you would like to share in the upcoming meeting?
Thanks
>
> Thanks
>
> >
> > Having said that, any other pending topic for the coming Monday
> > discussion ?
> >
> > I am still catching up on CXL (mostly focusing on the SP EFI flag
> > and DAX/kmem) - not much to share there on my side.
> >
> > If there is anything that you wish to put forward please let me know.
> >
> > Thanks,
> > Lorenzo
> >
> > > Many thanks
> > > Salil
> > >
> > > > From: Linaro-open-discussions
> > > > [mailto:linaro-open-discussions-bounces@op-lists.linaro.org] On Behalf
> Of
> > > > Jammy Zhou via Linaro-open-discussions
> > > > Sent: Monday, March 22, 2021 12:38 PM
> > > > To: Vincent Guittot <vincent.guittot(a)linaro.org>
> > > > Cc: Lorenzo Pieralisi via Linaro-open-discussions
> > > > <linaro-open-discussions(a)op-lists.linaro.org>; Jon Masters
> > > > <jcm(a)jonmasters.org>
> > > > Subject: Re: [Linaro-open-discussions] LOD meeting agenda for March 22nd
> > > >
> > > > Hi All,
> > > >
> > > > The meeting notes are published in the collaborate page below (with
> > > > Jonathan's CXL slides and the recording).
> > > >
> > > >
> >
> https://collaborate.linaro.org/display/LOD/2021-3-22+Meeting+Meeting+notes
> > > >
> > > > Regards,
> > > > Jammy
> > > >
> > > > On Thu, 18 Mar 2021 at 22:48, Vincent Guittot <vincent.guittot(a)linaro.org>
> > > > wrote:
> > > >
> > > > > On Thu, 18 Mar 2021 at 15:39, Jammy Zhou via Linaro-open-discussions
> > > > > <linaro-open-discussions(a)op-lists.linaro.org> wrote:
> > > > > >
> > > > > > Hmm, it looks like I forgot to add one event in the calendar. It has
> > been
> > > > > > added by Vincent just a moment ago. Thanks!
> > > > >
> > > > > Yeah, I noticed that the meeting didn't appear in the agenda so I
> > > > > figured out to add it
> > > > >
> > > > > >
> > > > > > On Thu, 18 Mar 2021 at 22:11, Jonathan Cameron <
> > > > > jonathan.cameron(a)huawei.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Ah, I'd missed that as not in the calendar at the bottom of the page
> > > > > yet.
> > > > > > >
> > > > > > > That works fine for me.
> > > > > > >
> > > > > > > Jonathan
> > > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Lorenzo Pieralisi [mailto:lorenzo.pieralisi@arm.com]
> > > > > > > Sent: 18 March 2021 12:28
> > > > > > > To: Jonathan Cameron <jonathan.cameron(a)huawei.com>
> > > > > > > Cc: Lorenzo Pieralisi via Linaro-open-discussions <
> > > > > > > linaro-open-discussions(a)op-lists.linaro.org>; Jammy Zhou <
> > > > > > > jammy.zhou(a)linaro.org>; Jon Masters <jcm(a)jonmasters.org>
> > > > > > > Subject: Re: [Linaro-open-discussions] LOD meeting agenda for March
> > > > > 22nd
> > > > > > >
> > > > > > > On Wed, Mar 17, 2021 at 05:17:05PM +0000, Jonathan Cameron wrote:
> > > > > > >
> > > > > > > [...]
> > > > > > >
> > > > > > > > > I would also take this opportunity to get your (and Salil's)
> > > > > > > > > feedback on the virtual CPU hotplug implementation with PSCI
> if
> > you
> > > > > > > > > had time to think about what we discussed last time.
> > > > > > > >
> > > > > > > > That should be fine to talk about / keep up momentum on.
> > > > > > > >
> > > > > > > > Have we actually picked a time slot?
> > > > > > > >
> > > > > > > > I have a strangely empty day on Monday, so feel free to suggest
> one.
> > > > > > > > As Jon hasn't replied, lets go for a time that isn't too unpleasant
> > > > > > > > for those based in China.
> > > > > > >
> > > > > > > I am happy to keep schedule as advertised in:
> > > > > > >
> > > > > > >
> > > > >
> > https://collaborate.linaro.org/display/LOD/Linaro+Open+Discussions+Home
> > > > > > >
> > > > > > > namely 10AM GMT. Is it fine for everyone ?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Lorenzo
> > > > > > >
> > > > > > --
> > > > > > Linaro-open-discussions mailing list
> > > > > >
> > https://collaborate.linaro.org/display/LOD/Linaro+Open+Discussions+Home
> > > > > > https://op-lists.linaro.org/mailman/listinfo/linaro-open-discussions
> > > > >
> > > > --
> > > > Linaro-open-discussions mailing list
> > > >
> https://collaborate.linaro.org/display/LOD/Linaro+Open+Discussions+Home
> > > > https://op-lists.linaro.org/mailman/listinfo/linaro-open-discussions
> >
>
> --
> Linaro-open-discussions mailing list
> https://collaborate.linaro.org/display/LOD/Linaro+Open+Discussions+Home
> https://op-lists.linaro.org/mailman/listinfo/linaro-open-discussions
Hi Lorenzo
Any comment about this issue? The patch has been waiting for
James Morse to review. It seems he is not in this group.
Could you help to invite him to join this discussion, or ask
for his opinion in private? This issue is urgent for us.
Thank you very much.
Thanks
Xiaofei Tan
>
> Today's Topics:
>
> 1. Regression of synchronous external aborts occur in user-mode
> (tanxiaofei)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 27 Apr 2021 12:08:57 +0000
> From: tanxiaofei <tanxiaofei(a)huawei.com>
> To: "linaro-open-discussions(a)op-lists.linaro.org"
> <linaro-open-discussions(a)op-lists.linaro.org>
> Cc: "lorenzo.pieralisi(a)arm.com" <lorenzo.pieralisi(a)arm.com>, Shiju
> Jose <shiju.jose(a)huawei.com>, Jonathan Cameron
> <jonathan.cameron(a)huawei.com>, "Guohanjun (Hanjun Guo)"
> <guohanjun(a)huawei.com>
> Subject: [Linaro-open-discussions] Regression of synchronous external
> aborts occur in user-mode
> Message-ID: <a7c49b5d41384d9b877edb2c2909de41(a)huawei.com>
> Content-Type: text/plain; charset="us-ascii"
>
> Hi All,
>
> We are facing regression on our hardware platform, Kunpeng9xx series, after the
> commit 8fcc4ae6faf8 ("arm64: acpi: Make apei_claim_sea() synchronize with APEI's irq work")
> was applied and this commit was merged into the mainline v5.8-rc1.
>
> The regression occur with the user-mode SEA, synchronous external abort occur in user space,
> and with our customer delivered firmware which reports the ARM processor error record for the SEA,
> to let kernel record the error.
>
> After the analysis it was identified that the reason of this issue is that do_sea() return
> directly for the user-mode SEA if the apei_claim_sea() handled the ARM processor error record,
> but the APEI GHES driver does not effectively process the ARM processor error record
> for the cache errors, by doing the memory failure handling. Currently the
> ghes_handle_memory_failure() is called only for the memory error record.
>
> The following patch fix the regression by doing the memory failure handling for
> the ARM processor error reported for the user-mode SEA.
> https://lore.kernel.org/linux-acpi/94a38a33-a949-3cce-d617-e1476912596e@hua…
>
> V5 patch was send-out after reviews by over 4 months.
>
> Can ARM folks please acknowledge the patch and perhaps the need for a meeting
> to explain the issue and resolution +quickly resolve any open questions?
>
> Thanks
> Xiaofei
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> Linaro-open-discussions mailing list
> Linaro-open-discussions(a)op-lists.linaro.org
> https://op-lists.linaro.org/mailman/listinfo/linaro-open-discussions
>
>
> ------------------------------
>
> End of Linaro-open-discussions Digest, Vol 7, Issue 9
> *****************************************************
>
Hi all,
it would be great if the next meeting can be scheduled for May 25th or
May 26th instead of May 24th to avoid public holidays in UK and Europe.
Please let me know if that's feasible.
Thank you very much.
Lorenzo