On Fri, Apr 23, 2021 at 08:20:59AM +0000, Vincent Guittot via Linaro-open-discussions wrote:
> Hi Song,
>
> On Fri, 23 Apr 2021 at 10:17, Song Bao Hua (Barry Song)
> <song.bao.hua(a)hisilicon.com> wrote:
> >
> > Hi Tammy,
> >
> > The time is perfect to me. Thanks!
> >
> >
> >
> > As usual, I have prepared some slides as attached. Could you please put the slides and the below link
> >
> > https://lore.kernel.org/lkml/20210420001844.9116-1-song.bao.hua@hisilicon.c…
> >
> > to
> >
> > https://collaborate.linaro.org/display/LOD/Linaro+Open+Discussions+Home
> >
> >
> >
> > I will appreciate very much if Vincent, Dietmar, Valentine and other scheduler guys can join the meeting.
>
> I have planned to attend the call
Thank you Vincent - Dietmar and Valentin will join too, talk to you
all on Monday.
Thanks,
Lorenzo
> Regards,
> Vincent
>
> >
> >
> >
> > Best Regards,
> >
> > Barry
> >
> >
> >
> > From: Jammy Zhou [mailto:jammy.zhou@linaro.org]
> > Sent: Tuesday, April 20, 2021 12:18 AM
> > To: Song Bao Hua (Barry Song) <song.bao.hua(a)hisilicon.com>
> > Cc: linaro-open-discussions(a)op-lists.linaro.org; Tim Chen <tim.c.chen(a)linux.intel.com>
> > Subject: Re: [Linaro-open-discussions] Meeting Agenda for April 26th
> >
> >
> >
> > Hi Barry,
> >
> >
> >
> > On Mon, 19 Apr 2021 at 19:11, Song Bao Hua (Barry Song) <song.bao.hua(a)hisilicon.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Linaro-open-discussions
> > > [mailto:linaro-open-discussions-bounces@op-lists.linaro.org] On Behalf Of
> > > Jammy Zhou via Linaro-open-discussions
> > > Sent: Monday, April 19, 2021 10:44 PM
> > > To: Lorenzo Pieralisi via Linaro-open-discussions
> > > <linaro-open-discussions(a)op-lists.linaro.org>
> > > Subject: [Linaro-open-discussions] Meeting Agenda for April 26th
> > >
> > > Hi All,
> > >
> > > We're going to have the next meeting on April 26th (next Monday). Please
> > > let me know if you have some topics for discussion.
> >
> > I am going to send cluster-scheduler RFC v6 tomorrow or the day after
> > tomorrow. There will be some major changes in wake_affine path. On the
> > other hand, I believe the patchset has also some other differences with
> > the one we discussed in the last meeting.
> >
> > Could you please consider adding this topic to the list?
> >
> >
> >
> > Sure, I just added it to the list
> >
> >
> >
> >
> > On the other hand, it seems we get people from UK, NZ, US, Portland,
> > China? So I am not sure what's the best time to get it started. What
> > about UTC 10:00AM?
> >
> >
> >
> > It is the planned time. Please let me know if we need to adjust it.
> >
> >
> >
> > Regards,
> >
> > Jammy
> >
> >
> >
> >
> > Thanks
> > Barry
> >
> > >
> > > Thanks,
> > > Jammy
> > > --
> > > Linaro-open-discussions mailing list
> > > https://collaborate.linaro.org/display/LOD/Linaro+Open+Discussions+Home
> > > https://op-lists.linaro.org/mailman/listinfo/linaro-open-discussions
> --
> Linaro-open-discussions mailing list
> https://collaborate.linaro.org/display/LOD/Linaro+Open+Discussions+Home
> https://op-lists.linaro.org/mailman/listinfo/linaro-open-discussions
> -----Original Message-----
> From: Linaro-open-discussions
> [mailto:linaro-open-discussions-bounces@op-lists.linaro.org] On Behalf Of
> Jammy Zhou via Linaro-open-discussions
> Sent: Monday, April 19, 2021 10:44 PM
> To: Lorenzo Pieralisi via Linaro-open-discussions
> <linaro-open-discussions(a)op-lists.linaro.org>
> Subject: [Linaro-open-discussions] Meeting Agenda for April 26th
>
> Hi All,
>
> We're going to have the next meeting on April 26th (next Monday). Please
> let me know if you have some topics for discussion.
I am going to send cluster-scheduler RFC v6 tomorrow or the day after
tomorrow. There will be some major changes in wake_affine path. On the
other hand, I believe the patchset has also some other differences with
the one we discussed in the last meeting.
Could you please consider adding this topic to the list?
On the other hand, it seems we get people from UK, NZ, US, Portland,
China? So I am not sure what's the best time to get it started. What
about UTC 10:00AM?
Thanks
Barry
>
> Thanks,
> Jammy
> --
> Linaro-open-discussions mailing list
> https://collaborate.linaro.org/display/LOD/Linaro+Open+Discussions+Home
> https://op-lists.linaro.org/mailman/listinfo/linaro-open-discussions
Hello,
Please find the requested slides containing *rough* QEMU boot time figures attached.
I will further refine/update the slides with Firmware/Linux Guest related figures
and will share later.
Many thanks
Salil
> From: Linaro-open-discussions
> [mailto:linaro-open-discussions-bounces@op-lists.linaro.org] On Behalf Of
> Jammy Zhou via Linaro-open-discussions
> Sent: Monday, March 22, 2021 12:38 PM
> To: Vincent Guittot <vincent.guittot(a)linaro.org>
> Cc: Lorenzo Pieralisi via Linaro-open-discussions
> <linaro-open-discussions(a)op-lists.linaro.org>; Jon Masters
> <jcm(a)jonmasters.org>
> Subject: Re: [Linaro-open-discussions] LOD meeting agenda for March 22nd
>
> Hi All,
>
> The meeting notes are published in the collaborate page below (with
> Jonathan's CXL slides and the recording).
>
> https://collaborate.linaro.org/display/LOD/2021-3-22+Meeting+Meeting+notes
>
> Regards,
> Jammy
>
> On Thu, 18 Mar 2021 at 22:48, Vincent Guittot <vincent.guittot(a)linaro.org>
> wrote:
>
> > On Thu, 18 Mar 2021 at 15:39, Jammy Zhou via Linaro-open-discussions
> > <linaro-open-discussions(a)op-lists.linaro.org> wrote:
> > >
> > > Hmm, it looks like I forgot to add one event in the calendar. It has been
> > > added by Vincent just a moment ago. Thanks!
> >
> > Yeah, I noticed that the meeting didn't appear in the agenda so I
> > figured out to add it
> >
> > >
> > > On Thu, 18 Mar 2021 at 22:11, Jonathan Cameron <
> > jonathan.cameron(a)huawei.com>
> > > wrote:
> > >
> > > > Ah, I'd missed that as not in the calendar at the bottom of the page
> > yet.
> > > >
> > > > That works fine for me.
> > > >
> > > > Jonathan
> > > >
> > > > -----Original Message-----
> > > > From: Lorenzo Pieralisi [mailto:lorenzo.pieralisi@arm.com]
> > > > Sent: 18 March 2021 12:28
> > > > To: Jonathan Cameron <jonathan.cameron(a)huawei.com>
> > > > Cc: Lorenzo Pieralisi via Linaro-open-discussions <
> > > > linaro-open-discussions(a)op-lists.linaro.org>; Jammy Zhou <
> > > > jammy.zhou(a)linaro.org>; Jon Masters <jcm(a)jonmasters.org>
> > > > Subject: Re: [Linaro-open-discussions] LOD meeting agenda for March
> > 22nd
> > > >
> > > > On Wed, Mar 17, 2021 at 05:17:05PM +0000, Jonathan Cameron wrote:
> > > >
> > > > [...]
> > > >
> > > > > > I would also take this opportunity to get your (and Salil's)
> > > > > > feedback on the virtual CPU hotplug implementation with PSCI if you
> > > > > > had time to think about what we discussed last time.
> > > > >
> > > > > That should be fine to talk about / keep up momentum on.
> > > > >
> > > > > Have we actually picked a time slot?
> > > > >
> > > > > I have a strangely empty day on Monday, so feel free to suggest one.
> > > > > As Jon hasn't replied, lets go for a time that isn't too unpleasant
> > > > > for those based in China.
> > > >
> > > > I am happy to keep schedule as advertised in:
> > > >
> > > >
> > https://collaborate.linaro.org/display/LOD/Linaro+Open+Discussions+Home
> > > >
> > > > namely 10AM GMT. Is it fine for everyone ?
> > > >
> > > > Thanks,
> > > > Lorenzo
> > > >
> > > --
> > > Linaro-open-discussions mailing list
> > > https://collaborate.linaro.org/display/LOD/Linaro+Open+Discussions+Home
> > > https://op-lists.linaro.org/mailman/listinfo/linaro-open-discussions
> >
> --
> Linaro-open-discussions mailing list
> https://collaborate.linaro.org/display/LOD/Linaro+Open+Discussions+Home
> https://op-lists.linaro.org/mailman/listinfo/linaro-open-discussions
Hi Lorenzo,
> > -----Original Message-----
> > From: Linaro-open-discussions
> > [mailto:linaro-open-discussions-bounces@op-lists.linaro.org] On Behalf Of
> > Lorenzo Pieralisi via Linaro-open-discussions
> > Sent: 10 December 2020 09:13
> > To: Jonathan Cameron <jonathan.cameron(a)huawei.com>
> > Cc: linaro-open-discussions(a)op-lists.linaro.org
> > Subject: Re: [Linaro-open-discussions] Suggested Agenda / timings for
> Monday
> > 7 Dec call
> >
> > On Thu, Dec 03, 2020 at 01:40:55PM +0000, Jonathan Cameron wrote:
> >
> > [...]
> >
> > > RMR related topics - follow up on last call: Lorenzo Pieralisi (ARM) - 5 mins
Just a quick one on updated RMR spec publication. I see a revision E.a here,
https://developer.arm.com/documentation/den0049/latest/
This seems to have addressed all the things we discussed. Is this a final one officially?
If yes, I couldn't understand why the RMR node flags are at an offset 8, compared to
other nodes where the node specific data normally starts at offset 16.
Could you please check and let me know.
Thanks,
Shameer
Hi Zhangfei,
On Fri, Mar 26, 2021 at 12:35:49PM +0000, Zhangfei Gao via Linaro-open-discussions wrote:
> Hi,
>
> I am looking for some suggestions about dvm [1].
>
> we are testing sva with openssl, and dvm is enabled by default to broadcast
> TLB maintenance.
> And thp is enabled in system by default, a daemon khugepaged
> (mm/khugepaged.c) is running
> to collapse memory to huge page in a period of time.
>
> And we found thp: khugepaged may cause issue to sva test case.
> https://github.com/Linaro/uadk/issues/215
>
> Two cases:
> 1. Heavy weight test case, async mode, 36+ jobs.
> openssl speed -elapsed -engine uadk -async_jobs 36 rsa2048
>
> Once collapse_huge_page happens.
> hardware may hung in io page fault, there maybe huge numbers of page fault
> keeps happening,
> while usually only several io page fault reported.
>
> 2. With high thp scan frequence, low weight test case, sync mode, 1 job.
> data may not correct.
> sudo openssl speed -engine uadk -seconds 1 rsa2048
> Doing 2048 bits public rsa's for 1s: RSA verify failure
>
> Two workarounds:
> 1. disable thp
> echo never > /sys/kernel/mm/transparent_hugepage/enabled
>
> 2. enable thp but add tlbi and ignore dvm.
> Adding arm_smmu_tlb_inv_range in arm-smmu-v3-sva.c:
> arm_smmu_mm_invalidate_range.
> It is called by khugepaged: collapse_huge_page->
> mmu_notifier_invalidate_range_end
>
>
> Looks dvm is not taking effect in some corner cases.
>
> Questions
> 1. if khugepaged collapse the memory used by device, and then change tlb,
> can dvm sync this tlb change to smmu.
>
> 2. Any possible dma is just using the memory, and collapsed by khugepaged,
> can dvm handle this case?
> Or khugepaged should not touch memory using by device, looks khugepaged
> can not distinguish.
I think hugepage is juste one symptom, and may not be the only way to
trigger the issue. On khugepaged collapse, a large invalidation is issued
which arch/arm64/include/asm/tlbflush.h transforms into a TLBI-by-ASID:
if ((!system_supports_tlb_range() &&
(end - start) >= (MAX_TLBI_OPS * stride)) ||
pages >= MAX_TLBI_RANGE_PAGES) {
flush_tlb_mm(vma->vm_mm);
return;
}
With MAX_TLBI_OPS = 512 and stride = 0x1000 we hit the size limit of 2M,
which is the size of a hugepage. My guess is that on khugepage collapse we
issue a TLBI ASIDE1IS (rather than a TLBI VALE1IS), somehow that isn't
taken into account by the SMMU, and we end up with stale TLB entries
leading to memory corruption. If that's the case, I'd suggest keeping DVM
disabled on this platform (workaround 2), to force all SVA invalidation to
go through the command queue.
Thanks,
Jean
Hi,
I am looking for some suggestions about dvm [1].
we are testing sva with openssl, and dvm is enabled by default to
broadcast TLB maintenance.
And thp is enabled in system by default, a daemon khugepaged
(mm/khugepaged.c) is running
to collapse memory to huge page in a period of time.
And we found thp: khugepaged may cause issue to sva test case.
https://github.com/Linaro/uadk/issues/215
Two cases:
1. Heavy weight test case, async mode, 36+ jobs.
openssl speed -elapsed -engine uadk -async_jobs 36 rsa2048
Once collapse_huge_page happens.
hardware may hung in io page fault, there maybe huge numbers of page
fault keeps happening,
while usually only several io page fault reported.
2. With high thp scan frequence, low weight test case, sync mode, 1 job.
data may not correct.
sudo openssl speed -engine uadk -seconds 1 rsa2048
Doing 2048 bits public rsa's for 1s: RSA verify failure
Two workarounds:
1. disable thp
echo never > /sys/kernel/mm/transparent_hugepage/enabled
2. enable thp but add tlbi and ignore dvm.
Adding arm_smmu_tlb_inv_range in arm-smmu-v3-sva.c:
arm_smmu_mm_invalidate_range.
It is called by khugepaged: collapse_huge_page->
mmu_notifier_invalidate_range_end
Looks dvm is not taking effect in some corner cases.
Questions
1. if khugepaged collapse the memory used by device, and then change
tlb, can dvm sync this tlb change to smmu.
2. Any possible dma is just using the memory, and collapsed by
khugepaged, can dvm handle this case?
Or khugepaged should not touch memory using by device, looks
khugepaged can not distinguish.
[1] DVM
Distributed Virtual Memory, a protocol for interconnect messages to provide
broadcast TLB maintenance operations (among other things).
Thanks
On Thu, 18 Mar 2021 at 15:39, Jammy Zhou via Linaro-open-discussions
<linaro-open-discussions(a)op-lists.linaro.org> wrote:
>
> Hmm, it looks like I forgot to add one event in the calendar. It has been
> added by Vincent just a moment ago. Thanks!
Yeah, I noticed that the meeting didn't appear in the agenda so I
figured out to add it
>
> On Thu, 18 Mar 2021 at 22:11, Jonathan Cameron <jonathan.cameron(a)huawei.com>
> wrote:
>
> > Ah, I'd missed that as not in the calendar at the bottom of the page yet.
> >
> > That works fine for me.
> >
> > Jonathan
> >
> > -----Original Message-----
> > From: Lorenzo Pieralisi [mailto:lorenzo.pieralisi@arm.com]
> > Sent: 18 March 2021 12:28
> > To: Jonathan Cameron <jonathan.cameron(a)huawei.com>
> > Cc: Lorenzo Pieralisi via Linaro-open-discussions <
> > linaro-open-discussions(a)op-lists.linaro.org>; Jammy Zhou <
> > jammy.zhou(a)linaro.org>; Jon Masters <jcm(a)jonmasters.org>
> > Subject: Re: [Linaro-open-discussions] LOD meeting agenda for March 22nd
> >
> > On Wed, Mar 17, 2021 at 05:17:05PM +0000, Jonathan Cameron wrote:
> >
> > [...]
> >
> > > > I would also take this opportunity to get your (and Salil's)
> > > > feedback on the virtual CPU hotplug implementation with PSCI if you
> > > > had time to think about what we discussed last time.
> > >
> > > That should be fine to talk about / keep up momentum on.
> > >
> > > Have we actually picked a time slot?
> > >
> > > I have a strangely empty day on Monday, so feel free to suggest one.
> > > As Jon hasn't replied, lets go for a time that isn't too unpleasant
> > > for those based in China.
> >
> > I am happy to keep schedule as advertised in:
> >
> > https://collaborate.linaro.org/display/LOD/Linaro+Open+Discussions+Home
> >
> > namely 10AM GMT. Is it fine for everyone ?
> >
> > Thanks,
> > Lorenzo
> >
> --
> Linaro-open-discussions mailing list
> https://collaborate.linaro.org/display/LOD/Linaro+Open+Discussions+Home
> https://op-lists.linaro.org/mailman/listinfo/linaro-open-discussions
On Mon, Mar 15, 2021 at 01:00:19AM +0000, Jammy Zhou via Linaro-open-discussions wrote:
> Hi all,
>
> We're going to have the LOD meeting in one week, currently we don't
> have any topic proposed yet. Please let me know if you have something
> to discuss.
I would be glad to have a follow-up with Jonathan on CXL expansion memory
on arm64 if possible, hopefully I can read up in the meanwhile to get up
to speed on the matter.
Lorenzo
Hi all,
We're going to have the LOD meeting in one week, currently we don't have
any topic proposed yet. Please let me know if you have something to discuss.
Regards,
Jammy