-----Original Message----- From: Tim Chen [mailto:tim.c.chen@linux.intel.com] Sent: Thursday, June 17, 2021 11:42 AM To: Song Bao Hua (Barry Song) song.bao.hua@hisilicon.com; yangyicong yangyicong@huawei.com; linaro-open-discussions@op-lists.linaro.org Cc: guodong.xu@linaro.org; tangchengchang tangchengchang@huawei.com; Zengtao (B) prime.zeng@hisilicon.com; tiantao (H) tiantao6@hisilicon.com; Jonathan Cameron jonathan.cameron@huawei.com; Linuxarm linuxarm@huawei.com Subject: Re: [PATCH 0/3] cluster-scheduler upstream plan
Barry,
I did a quick test, not on SPECrate but on some simple tests. This patchset will work okay for cpu bound tasks. But for tasks with sleep and wake up, we want to find the cpu in the same cluster first. Otherewise the wakeup will not respect the cluster.
Tim, thanks for your testing. I think this is as expected. My plan is upstreaming the whole cluster scheduler in two stages.
- patchset to support the basic cluster topology and support spreading tasks only.
- patchset to support wake_up path to support packing related tasks after 1 is done.
So in 1, cluster_sched is default disabled. And in the commit log, we will
mention
2 is not supported yet. Since 2 is much more tricky than 1, if 1 and 2 are
put
together, I am worried it is difficult for maintainers to review it.
So in the 1st patchset, we give some workload examples which like spreading and also say packing workload won't get benefit and it will be supported in a separate patchset afterwards. Does it make sense to you?
I agree.
In my 2/3, I tested stream, lkp-compress(lkp-pbzip2), kernbench, they were improved.
I think my previous mail was not quite clear. Actually the issue I observed was for spreading and not packing. In my test there is no relationship between the tasks running. Just that it sleep for a short while and wake up. So I am just observing to see if the spreading is done properly.
The simple test I did was the following:
#include <stdio.h> #include <time.h> #include <unistd.h>
int main(void) { int i,j;
while (1) { i++; j += i; if (i % 100000000 == 0) usleep(5000); }}
This will keep a CPU on my test system close to 99% busy. I ran multiple copies of this program to see if it gets spread out.
I observe that with this small sleep, I have some clumping in cluster that's not spread out into idle cluster. Wonder if you see the same.
I tried this case on kunpeng920 with 4numa*24cores=96cores by running 6 a.out on numa0 as below:
root@ubuntu:~# cat 1.c #include <stdio.h> #include <time.h> #include <unistd.h>
int main(void) { int i,j;
while (1) { i++; j += i; if (i % 100000000 == 0) usleep(5000); } } root@ubuntu:~# gcc 1.c root@ubuntu:~# numactl -N 0 ./a.out & [1] 27321 root@ubuntu:~# numactl -N 0 ./a.out & [2] 27322 root@ubuntu:~# numactl -N 0 ./a.out & [3] 27323 root@ubuntu:~# numactl -N 0 ./a.out & [4] 27324 root@ubuntu:~# numactl -N 0 ./a.out & [5] 27325 root@ubuntu:~# numactl -N 0 ./a.out & [6] 27326
I found they are spreading quite well, you can get pictures from: http://www.linuxep.com/patches/6-a.out.png
each 4 cores get 1 a.out.
Tim
Thanks Barry