Hi James,
I am doing changes(both ACPI interface and QoM - some prototyping required) in the Qemu to reflect
the _STA.present correctly after _EJx method has been evaluated. Will get back to you later this week.
I hope this is not blocking you in any way?
Thanks
Salil
Hi,
For time and connection details see the calendar at
https://www.trustedfirmware.org/meetings/
I have one topic for the LOC:
Can we deprecate paging of OP-TEE on Armv8-A platforms using FF-A?
Any other topic?
Thanks,
Jens
Hi James,
Question:
We are using HVC conduit for PSCI, do we need a check if hypercall being trapped to userspace is SMC or HVC?
File: linux/arch/arm64/kvm/hypercalls.c (your code change)
int kvm_hvc_user(struct kvm_vcpu *vcpu)
{
[...]
run->exit_reason = KVM_EXIT_HYPERCALL;
run->hypercall.nr = kvm_vcpu_hvc_get_imm(vcpu); -> this would be referred by userspace code?
/* Add the first parameters for fast access. */
[...]
return 0;
}
# Userspace Code Excerpt (JPB's June 2021 code reference)
In the userspace, a check similar to below might be required I guess:
+ /* Under KVM, the PSCI conduit is HVC */
+ if (imm & KVM_ARM_EXIT_HYPERCALL_SMC) {
+ XXXX_log_mask("%s: unexpected SMC exit\n", __func__);
+ return 0;
+ }
+
+/*
+ * In kvm_run::hypercall::nr, this bit indicates that the call is a SMC. When
+ * the bit is not set, the call is an HVC.
+ */
+#define KVM_ARM_EXIT_HYPERCALL_SMC (1ULL << 63)
+
I am being lazy but Could you please provide more clarification on the use
of 'hypercall.nr' for this check?
[1] https://developer.arm.com/documentation/ddi0596/2021-12/Base-Instructions/H… [HVC #0]
Thanks
Salil
Hi Salil(s),
You mentioned 'cold plug behaves differently'.
From the last call you described 'coldplug' as starting Qemu with '-S' so it doesn't
actually run the guest, then adding vCPUs before releasing Qemu to run the guest. You said
CPUs added this way can't be disabled.
(am I right so far?)
This turns out to be a bit murkier than that. You can disable these vCPUs, but the first
call will fail. The reason is very simple: Qemu is sending a device-check for the first
call, not an eject-request.
Linux prints a warning for the spurious device-check because the CPU already exists and is
even online.
An example flow, with the below debug[0], is:
# qemu -S smp cpus=1,maxcpus=3,cores=3,threads=1,sockets=1 ${REST_OF_OPTIONS}
On the Qemu monitor:
| device_add driver=host-arm-cpu,core-id=1,id=cpu1
| cont
[Qemu boots the guest]
acpi_processor_add() is called twice during boot, once for vCPU0 and once for the vCPU
that was 'coldplugged' vCPU1.
On the Qemu monitor:
| device_del cpu1
[ 56.427089] ACPI: XYZZY:ACPI_NOTIFY_DEVICE_CHECK on ACPI0007:1
[ 56.428239] ACPI: XYZZY: acpi_scan_device_check() 1 | 1
[ 56.429335] CPU: 1 PID: 105 Comm: kworker/u6:2 Not tainted
6.1.0-rc2-00028-g6eaecb5ffd26-dirty #14644
[ 56.431043] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015
[ 56.432431] Workqueue: kacpi_hotplug acpi_hotplug_work_fn
[ 56.433520] Call trace:
[ 56.434015] dump_backtrace.part.0+0xe0/0xf0
[ 56.434875] show_stack+0x18/0x40
[ 56.435546] dump_stack_lvl+0x68/0x84
[ 56.436308] dump_stack+0x18/0x34
[ 56.436983] acpi_device_hotplug+0x234/0x4e0
[ 56.437847] acpi_hotplug_work_fn+0x24/0x40
[ 56.438695] process_one_work+0x1d0/0x320
[ 56.439515] worker_thread+0x14c/0x444
[ 56.440283] kthread+0x10c/0x110
[ 56.440935] ret_from_fork+0x10/0x20
[ 56.441680] acpi ACPI0007:01: Already enumerated
[ This is because Qemu is adding a CPU that already exists ]
A definition of madness is doing the same thing and expecting a different result.
On the Qemu monitor:
| device_del cpu1
[ 67.723708] ACPI: XYZZY:ACPI_NOTIFY_EJECT_REQUEST on ACPI0007:1
[ 67.771014] psci: CPU1 killed (polled 0 ms)
[ 67.773437] XYZZY: acpi_processor_post_eject()
It looks like Qemu creates the device-check when you cold-plug the vCPU, but doesn't
deliver it, instead it delivers it _instead_ of the next notification.
Qemu v7.1.0 for doesn't do this for x86, nor does it deliver the spurious
ACPI_NOTIFY_DEVICE_CHECK early.
I'd suggest the arm64 changes are generating a ACPI_NOTIFY_DEVICE_CHECK when it shouldn't.
Thanks,
James
[0] Debug
diff --git a/drivers/acpi/acpi_processor.c b/drivers/acpi/acpi_processor.c
index a4d58e0d1452..ec72f2d2a5fb 100644
--- a/drivers/acpi/acpi_processor.c
+++ b/drivers/acpi/acpi_processor.c
@@ -492,6 +496,8 @@ static void acpi_processor_post_eject(struct acpi_device *device)
unsigned long long sta;
acpi_status status;
+ pr_err("XYZZY: acpi_processor_post_eject()\n");
+
if (!device)
return;
diff --git a/drivers/acpi/bus.c b/drivers/acpi/bus.c
index d466c8195314..f9f4c7707886 100644
--- a/drivers/acpi/bus.c
+++ b/drivers/acpi/bus.c
@@ -467,43 +467,55 @@ static void acpi_bus_notify(acpi_handle handle, u32 type, void *data)
u32 ost_code = ACPI_OST_SC_NON_SPECIFIC_FAILURE;
bool hotplug_event = false;
+ adev = acpi_get_acpi_dev(handle);
+ if (!adev)
+ goto err;
+
switch (type) {
case ACPI_NOTIFY_BUS_CHECK:
acpi_handle_debug(handle, "ACPI_NOTIFY_BUS_CHECK event\n");
+ pr_err("XYZZY:ACPI_NOTIFY_BUS_CHECK on %s:%s\n", acpi_device_hid(adev),
acpi_device_uid(adev));
hotplug_event = true;
break;
case ACPI_NOTIFY_DEVICE_CHECK:
acpi_handle_debug(handle, "ACPI_NOTIFY_DEVICE_CHECK event\n");
+ pr_err("XYZZY:ACPI_NOTIFY_DEVICE_CHECK on %s:%s\n", acpi_device_hid(adev),
acpi_device_uid(adev));
hotplug_event = true;
break;
case ACPI_NOTIFY_DEVICE_WAKE:
acpi_handle_debug(handle, "ACPI_NOTIFY_DEVICE_WAKE event\n");
+ pr_err("XYZZY:ACPI_NOTIFY_DEVICE_WAKE on %s:%s\n", acpi_device_hid(adev),
acpi_device_uid(adev));
break;
case ACPI_NOTIFY_EJECT_REQUEST:
acpi_handle_debug(handle, "ACPI_NOTIFY_EJECT_REQUEST event\n");
+ pr_err("XYZZY:ACPI_NOTIFY_EJECT_REQUEST on %s:%s\n",
acpi_device_hid(adev), acpi_device_uid(adev));
hotplug_event = true;
break;
case ACPI_NOTIFY_DEVICE_CHECK_LIGHT:
acpi_handle_debug(handle, "ACPI_NOTIFY_DEVICE_CHECK_LIGHT event\n");
+ pr_err("XYZZY:ACPI_NOTIFY_DEVICE_CHECK_LIGHT on %s:%s\n",
acpi_device_hid(adev), acpi_device_uid(adev));
/* TBD: Exactly what does 'light' mean? */
break;
case ACPI_NOTIFY_FREQUENCY_MISMATCH:
acpi_handle_err(handle, "Device cannot be configured due "
"to a frequency mismatch\n");
+ pr_err("XYZZY:ACPI_NOTIFY_FREQUENCY_MISMATCH on %s:%s\n",
acpi_device_hid(adev), acpi_device_uid(adev));
break;
case ACPI_NOTIFY_BUS_MODE_MISMATCH:
acpi_handle_err(handle, "Device cannot be configured due "
"to a bus mode mismatch\n");
+ pr_err("XYZZY:ACPI_NOTIFY_BUS_MODE_MISMATCH on %s:%s\n",
acpi_device_hid(adev), acpi_device_uid(adev));
break;
case ACPI_NOTIFY_POWER_FAULT:
acpi_handle_err(handle, "Device has suffered a power fault\n");
+ pr_err("XYZZY:ACPI_NOTIFY_POWER_FAULT on %s:%s\n", acpi_device_hid(adev),
acpi_device_uid(adev));
break;
default:
@@ -511,10 +523,6 @@ static void acpi_bus_notify(acpi_handle handle, u32 type, void *data)
break;
}
- adev = acpi_get_acpi_dev(handle);
- if (!adev)
- goto err;
-
if (adev->dev.driver) {
struct acpi_driver *driver = to_acpi_driver(adev->dev.driver);
diff --git a/drivers/acpi/scan.c b/drivers/acpi/scan.c
index 2d1a82aa1607..b2bc1c611a83 100644
--- a/drivers/acpi/scan.c
+++ b/drivers/acpi/scan.c
@@ -380,6 +380,8 @@ static int acpi_scan_device_check(struct acpi_device *adev)
{
int error;
+ pr_err("XYZZY: acpi_scan_device_check() %u | %u\n", adev->status.present,
adev->status.functional);
+
acpi_bus_get_status(adev);
if (adev->status.present || adev->status.functional) {
/*
@@ -391,6 +393,7 @@ static int acpi_scan_device_check(struct acpi_device *adev)
* again).
*/
if (adev->handler) {
+ dump_stack();
dev_warn(&adev->dev, "Already enumerated\n");
return -EALREADY;
}
Linaro Open Discussions monthly meeting
Wednesday 5 Oct 2022 ⋅ 19:00 – 20:00
Hong Kong Standard Time
Location
https://linaro-org.zoom.us/j/95682500341https://www.google.com/url?q=https%3A%2F%2Flinaro-org.zoom.us%2Fj%2F9568250…
Joyce QI 邀请您参加预先安排的 Zoom 会议。
加入 Zoom 会议
https://linaro-org.zoom.us/j/95682500341
会议号:956 8250 0341
手机一键拨号
+16699009128,,95682500341# 美国 (San Jose)
+13462487799,,95682500341# 美国 (Houston)
根据您的位置拨号
+1 669 900 9128 美国 (San Jose)
+1 346 248 7799 美国 (Houston)
+1 253 215 8782 美国 (Tacoma)
+1 646 558 8656 美国 (New York)
+1 301 715 8592 美国 (Washington DC)
+1 312 626 6799 美国 (Chicago)
888 788 0099 美国 免费
877 853 5247 美国 免费
会议号:956 8250 0341
查找本地号码:https://linaro-org.zoom.us/u/ady2J9Zn7t
Guests
linaro-open-discussions(a)op-lists.linaro.org
james.morse(a)arm.com
jonathan.cameron(a)huawei.com
lorenzo.pieralisi(a)linaro.org
ilkka(a)os.amperecomputing.com
View all guest info
https://calendar.google.com/calendar/event?action=VIEW&eid=Mjg1MjNrNWtiMWxv…
Reply for linaro-open-discussions(a)op-lists.linaro.org and view more details
https://calendar.google.com/calendar/event?action=VIEW&eid=Mjg1MjNrNWtiMWxv…
Your attendance is optional.
~~//~~
Invitation from Google Calendar: https://calendar.google.com/calendar/
You are receiving this email because you are an attendee of the event. To
stop receiving future updates for this event, decline this event.
Forwarding this invitation could allow any recipient to send a response to
the organiser, be added to the guest list, invite others regardless of
their own invitation status or modify your RSVP.
Learn more https://support.google.com/calendar/answer/37135#forwarding
Hi Lorenzo,
I have noticed you are in holiday on Nov,1st. I can switch the meeting
to Nov 2nd if other people are free.
@Lorenzo,James,
Can we arrange Zhangfei’s questions( TLB missing impact the
performance in vsva/ sva) as below in next meeting.
Or can you help involve any other experts focus on this? Appreciated so
much for your help here.
Thanks:)
Joyce
在
2022年10月28日,上午8:00,linaro-open-discussions-request@op-lists.linaro.o
rg 写道:
Send Linaro-open-discussions mailing list submissions to
linaro-open-discussions(a)op-lists.linaro.org
To subscribe or unsubscribe via email, send a message with subject
or
body 'help' to
linaro-open-discussions-request(a)op-lists.linaro.org
You can reach the person managing the list at
linaro-open-discussions-owner(a)op-lists.linaro.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Linaro-open-discussions digest..."
Today's Topics:
1. Re: Linaro-open-discussions Digest, Vol 25, Issue 3 (Zhangfei
Gao)
--------------------------------------------------------------------
--
Message: 1
Date: Thu, 27 Oct 2022 12:06:59 +0800
From: Zhangfei Gao <zhangfei.gao(a)linaro.org>
Subject: [Linaro-open-discussions] Re: Linaro-open-discussions
Digest,
Vol 25, Issue 3
To: Joyce Qi <joyce.qi(a)linaro.org>, Lorenzo Pieralisi
<lorenzo.pieralisi(a)linaro.org>, jean-philippe
<jean-philippe(a)linaro.org>
Cc: linaro-open-discussions(a)op-lists.linaro.org,
linux(a)armlinux.org.uk
Message-ID: <2fba7536-1475-4755-9084-a920addecc42(a)linaro.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Hi, Lorenzo
When we debug vsva, sva on guest with 2-stage translation,
we found tlb miss impact performance a lot, with Jean's help.
Currently we are using huge page feature in glibc to overcome this
issue.
With huge page, vsva in guest can achieve comparable performance as
sva
in host.
More details: https://docs.qq.com/doc/DRXlpQmpTSlBZTGZZ
Will do some
Basically we are using two optimization methods/
host sva: using memset fist to overcome io page fault, since cpu
alloc
physical memory is much faster than smmu.
guest vsva: using huge page to overcome tlbmiss.
The customer are asking whether (next version) silicon (smmu) can do
some help to concur tlb miss issue or io page fault issue.
If so, software can be simpler.
Any suggestions?
Thanks
On 2022/10/20 下午8:47, Joyce Qi via Linaro-open-discussions wrote:
Hi Jonathan,Lorenzo,all,
Do we have any topic to sync on our LOD meeting next week?
Thanks:)
Joyce
------------------------------
Subject: Digest Footer
Linaro-open-discussions mailing list --
linaro-open-discussions(a)op-lists.linaro.org
To unsubscribe send an email to
linaro-open-discussions-leave(a)op-lists.linaro.org
------------------------------
End of Linaro-open-discussions Digest, Vol 25, Issue 9
******************************************************
Hi Lorenzo,
I have noticed you are in holiday on Nov,1st. I can switch the meeting to Nov 2nd if other people are free.
@Lorenzo,James,
Can we arrange Zhangfei’s questions( TLB missing impact the performance in vsva/ sva) as below in next meeting.
Or can you help involve any other experts focus on this? Appreciated so much for your help here.
Thanks:)
Joyce
> 在 2022年10月28日,上午8:00,linaro-open-discussions-request@op-lists.linaro.org 写道:
>
> Send Linaro-open-discussions mailing list submissions to
> linaro-open-discussions(a)op-lists.linaro.org
>
> To subscribe or unsubscribe via email, send a message with subject or
> body 'help' to
> linaro-open-discussions-request(a)op-lists.linaro.org
>
> You can reach the person managing the list at
> linaro-open-discussions-owner(a)op-lists.linaro.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Linaro-open-discussions digest..."
>
> Today's Topics:
>
> 1. Re: Linaro-open-discussions Digest, Vol 25, Issue 3 (Zhangfei Gao)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 27 Oct 2022 12:06:59 +0800
> From: Zhangfei Gao <zhangfei.gao(a)linaro.org>
> Subject: [Linaro-open-discussions] Re: Linaro-open-discussions Digest,
> Vol 25, Issue 3
> To: Joyce Qi <joyce.qi(a)linaro.org>, Lorenzo Pieralisi
> <lorenzo.pieralisi(a)linaro.org>, jean-philippe
> <jean-philippe(a)linaro.org>
> Cc: linaro-open-discussions(a)op-lists.linaro.org, linux(a)armlinux.org.uk
> Message-ID: <2fba7536-1475-4755-9084-a920addecc42(a)linaro.org>
> Content-Type: text/plain; charset=utf-8; format=flowed
>
> Hi, Lorenzo
>
> When we debug vsva, sva on guest with 2-stage translation,
> we found tlb miss impact performance a lot, with Jean's help.
>
> Currently we are using huge page feature in glibc to overcome this issue.
> With huge page, vsva in guest can achieve comparable performance as sva
> in host.
>
> More details: https://docs.qq.com/doc/DRXlpQmpTSlBZTGZZ
> Will do some
>
> Basically we are using two optimization methods/
> host sva: using memset fist to overcome io page fault, since cpu alloc
> physical memory is much faster than smmu.
> guest vsva: using huge page to overcome tlbmiss.
>
> The customer are asking whether (next version) silicon (smmu) can do
> some help to concur tlb miss issue or io page fault issue.
> If so, software can be simpler.
> Any suggestions?
>
> Thanks
>
>
> On 2022/10/20 下午8:47, Joyce Qi via Linaro-open-discussions wrote:
>> Hi Jonathan,Lorenzo,all,
>>
>> Do we have any topic to sync on our LOD meeting next week?
>>
>> Thanks:)
>> Joyce
>>
>>
>>
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> Linaro-open-discussions mailing list -- linaro-open-discussions(a)op-lists.linaro.org
> To unsubscribe send an email to linaro-open-discussions-leave(a)op-lists.linaro.org
>
>
> ------------------------------
>
> End of Linaro-open-discussions Digest, Vol 25, Issue 9
> ******************************************************
Linaro Open Discussions monthly meeting
Tuesday 1 Nov 2022 ⋅ 22:00 – 23:00
Hong Kong Standard Time
Location
https://linaro-org.zoom.us/j/95682500341https://www.google.com/url?q=https%3A%2F%2Flinaro-org.zoom.us%2Fj%2F9568250…
Joyce QI 邀请您参加预先安排的 Zoom 会议。
加入 Zoom 会议
https://linaro-org.zoom.us/j/95682500341
会议号:956 8250 0341
手机一键拨号
+16699009128,,95682500341# 美国 (San Jose)
+13462487799,,95682500341# 美国 (Houston)
根据您的位置拨号
+1 669 900 9128 美国 (San Jose)
+1 346 248 7799 美国 (Houston)
+1 253 215 8782 美国 (Tacoma)
+1 646 558 8656 美国 (New York)
+1 301 715 8592 美国 (Washington DC)
+1 312 626 6799 美国 (Chicago)
888 788 0099 美国 免费
877 853 5247 美国 免费
会议号:956 8250 0341
查找本地号码:https://linaro-org.zoom.us/u/ady2J9Zn7t
Guests
linaro-open-discussions(a)op-lists.linaro.org
james.morse(a)arm.com
salil.mehta(a)huawei.com
Zhangfei Gao
linux(a)armlinux.org.uk
ilkka(a)os.amperecomputing.com
lorenzo.pieralisi(a)linaro.org
jonathan.cameron(a)huawei.com
View all guest info
https://calendar.google.com/calendar/event?action=VIEW&eid=Mjg1MjNrNWtiMWxv…
Reply for linaro-open-discussions(a)op-lists.linaro.org and view more details
https://calendar.google.com/calendar/event?action=VIEW&eid=Mjg1MjNrNWtiMWxv…
Your attendance is optional.
~~//~~
Invitation from Google Calendar: https://calendar.google.com/calendar/
You are receiving this email because you are an attendee on the event. To
stop receiving future updates for this event, decline this event.
Forwarding this invitation could allow any recipient to send a response to
the organizer, be added to the guest list, invite others regardless of
their own invitation status, or modify your RSVP.
Learn more https://support.google.com/calendar/answer/37135#forwarding