Hi Yinsi,
The trouble with this is we would have to change the test definitions for the Kungpen server - which would make life complicated for the CI front end because when a job is submitted LAVA scheduler just picks then first available device. You never know which worker it’s going to end up on.
Dave
On 30 Apr 2021, at 09:14, liuyinsi@163.com wrote:
Hi Dave,
Fix git clone failed: Please try to use http://gitee.com/liu-yinsi/test-definitions http://gitee.com/liu-yinsi/test-definitions, it's a mirror of http://github.com/Linaro/test-definitions, http://github.com/Linaro/test-definitions, and it will update automatically once a day.
Thanks Yinsi
liuyinsi@163.com mailto:liuyinsi@163.com From: Dave Pigott mailto:dave.pigott@linaro.org Date: 2021-04-13 17:39 To: Jammy Zhou mailto:jammy.zhou@linaro.org CC: liuyinsi mailto:liuyinsi@163.com; Luca Di Stefano mailto:luca.distefano@linaro.org; Anmar Oueja mailto:anmar.oueja@linaro.org; Shameerali Kolothum Thodi mailto:shameerali.kolothum.thodi@huawei.com; wufengguang mailto:wufengguang@huawei.com; Guohanjun (Hanjun Guo) mailto:guohanjun@huawei.com; Jonathan Cameron mailto:jonathan.cameron@huawei.com; lkq-dev mailto:lkq-dev@op-lists.linaro.org; Chase Qi mailto:chase.qi@linaro.org Subject: Re: Issues with docker on OpenEulerOS Hi Jammy,
This is all good - not sure what, if anything, the Lab can do to fix these problems. This would appear to be a CI/Front end issue.
Thanks
Dave
On 13 Apr 2021, at 03:07, Jammy Zhou <jammy.zhou@linaro.org mailto:jammy.zhou@linaro.org> wrote:
I checked with Chase who has good experience of LAVA in China. It looks like a cache/mirror server is helpful in this context.
The download speed from S3 often is a problem for China. We may need to setup a mirror/cache server (e.g. jfrog artifactory) in the lab to mirror or cache the files that LAVA test jobs need. Once we have a cache/mirror server set, the test plan should be updated to download file from mirror/cache server instead of the original S3 server. A local mirror/cache server will improve both reliability and speed of CI.
Regards, Jammy
On Mon, 12 Apr 2021 at 17:34, Dave Pigott <dave.pigott@linaro.org mailto:dave.pigott@linaro.org> wrote: Hi Yinsi,
I had to offline the worker over the weekend because the download speed was making jobs fail due to timeouts cloning and wgetting
Not much we as the lab team can do about that.
Dave
On 9 Apr 2021, at 15:19, liuyinsi@163.com mailto:liuyinsi@163.com wrote:
Hi All,
It's great to collaborate. :)
Thank you all.
Yinsi
liuyinsi 邮箱：liuyinsi@163.com https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1&name=liuyinsi&uid=liuyinsi%40163.com&iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fqiyelogo%2FdefaultAvatar.png&items=%5B%22%E9%82%AE%E7%AE%B1%EF%BC%9Aliuyinsi%40163.com%22%5D签名由 网易邮箱大师 https://mail.163.com/dashi/dlpro.html?from=mail88 定制 On 04/09/2021 19:52, Luca Di Stefano mailto:luca.distefano@linaro.org wrote: Hi All, Thank you Yinsi for all the support. Just as a note, to have a working setup for LAVA dispatcher through docker-compose both the backported kernel fix and the Qemu version from debian backports repo were needed in order to achieve this. Thank you all. Luca
On 09/04/2021 12:46, Dave Pigott wrote:
Okay - great news. The Kungpen server is now online and the qemu devices are passing their health checks and now processing jobs.
Thank you all!
Dave
On 9 Apr 2021, at 04:57, liuyinsi@163.com mailto:liuyinsi@163.com wrote:
OK，done. Please try to login.
Thanks Yinsi
liuyinsi 邮箱：liuyinsi@163.com https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1&name=liuyinsi&uid=liuyinsi%40163.com&iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fqiyelogo%2FdefaultAvatar.png&items=%5B%22%E9%82%AE%E7%AE%B1%EF%BC%9Aliuyinsi%40163.com%22%5D签名由 网易邮箱大师 https://mail.163.com/dashi/dlpro.html?from=mail88 定制 On 04/08/2021 20:06, Luca Di Stefano mailto:luca.distefano@linaro.org wrote: Hi Yinsi, Other IP addresses that are relevant to the LAVA lab in addition to what Dave already said are:
- 51.148.40.18
- 51.148.40.7
- 51.148.40.11
Thank you. Luca
On 08/04/2021 13:02, Dave Pigott wrote:
Hi Yinsi,
The Lab IP is 51.148.40.1
Thanks
Dave
> On 8 Apr 2021, at 12:30, liuyinsi@163.com mailto:liuyinsi@163.com wrote: > > Hi Dave/Anmar, > > The firewall is enabled for some reasons. Please send your public IP address to me, i need to add your public IP address to the trustlist. > > Thanks > Yinsi > > > > > liuyinsi > 邮箱：liuyinsi@163.com > https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1&name=liuyinsi&uid=liuyinsi%40163.com&iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fqiyelogo%2FdefaultAvatar.png&items=%5B%22%E9%82%AE%E7%AE%B1%EF%BC%9Aliuyinsi%40163.com%22%5D签名由 网易邮箱大师 https://mail.163.com/dashi/dlpro.html?from=mail88 定制 > On 04/07/2021 18:21, Dave Pigott mailto:dave.pigott@linaro.org wrote: > Hi Yinsi, > > We can no longer ssh to the server. Can you resolve this please? > > Thanks > > Dave > >> On 7 Apr 2021, at 10:17, liuyinsi@163.com mailto:liuyinsi@163.com wrote: >> >> >> Ok，done. Please try to test QEMU again. >> >> Thanks >> Yinsi >> >> >> >> >> On 04/01/2021 20:57, Anmar Oueja mailto:anmar.oueja@linaro.org wrote: >> On Fri, Mar 19, 2021 at 12:26 PM Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com mailto:shameerali.kolothum.thodi@huawei.com> wrote: >> Ok, I have backported the fix to 5.10. >> >> Please find the patches here, >> https://github.com/hisilicon/kernel-dev/commits/private-v5.10-GICv2-on-v3 https://github.com/hisilicon/kernel-dev/commits/private-v5.10-GICv2-on-v3 >> >> Sanity testing at my end shows it working. Please try and let me know. >> >> Thanks Shmeer. >> >> On our side, we tried the newest stable version of QEMU and it didn't work unfortunately. The only way to fix this is either deploying the kernel above or a firmware fix. >> >> Yinsi: Are you able to build and install the kernel in the above tree? Hanjun of Fengguang might be able to help. >> >> anmar >> >> From: Jonathan Cameron >> Sent: 19 March 2021 09:13 >> To: liuyinsi@163.com mailto:liuyinsi@163.com; jammy.zhou@linaro.org mailto:jammy.zhou@linaro.org; Guohanjun (Hanjun Guo) <guohanjun@huawei.com mailto:guohanjun@huawei.com>; Pigott <dave.pigott@linaro.org mailto:dave.pigott@linaro.org> >> Cc: Anmar Oueja <anmar.oueja@linaro.org mailto:anmar.oueja@linaro.org>; Luca Di Stefano <luca.distefano@linaro.org mailto:luca.distefano@linaro.org>; wufengguang <wufengguang@huawei.com mailto:wufengguang@huawei.com>; lkq-dev <lkq-dev@op-lists.linaro.org mailto:lkq-dev@op-lists.linaro.org>; Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com mailto:shameerali.kolothum.thodi@huawei.com> >> Subject: RE: Re: Issues with docker on OpenEulerOS >> >> We’ll still need the backport as I think the fix only went in during 5.11 but hopefully not to major. >> @Shameer, let us know if you hit any problems. >> >> Thanks, >> >> Jonathan >> >> >> From: liuyinsi@163.com mailto:liuyinsi@163.com [mailto:liuyinsi@163.com mailto:liuyinsi@163.com] >> Sent: 19 March 2021 08:29 >> To: jammy.zhou@linaro.org mailto:jammy.zhou@linaro.org; Jonathan Cameron <jonathan.cameron@huawei.com mailto:jonathan.cameron@huawei.com>; Guohanjun (Hanjun Guo) <guohanjun@huawei.com mailto:guohanjun@huawei.com>; Pigott <dave.pigott@linaro.org mailto:dave.pigott@linaro.org> >> Cc: Anmar Oueja <anmar.oueja@linaro.org mailto:anmar.oueja@linaro.org>; Luca Di Stefano <luca.distefano@linaro.org mailto:luca.distefano@linaro.org>; wufengguang <wufengguang@huawei.com mailto:wufengguang@huawei.com>; lkq-dev <lkq-dev@op-lists.linaro.org mailto:lkq-dev@op-lists.linaro.org>; Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com mailto:shameerali.kolothum.thodi@huawei.com> >> Subject: Re: Re: Issues with docker on OpenEulerOS >> >> @Jammy, >> >> Our dailybuild already make v5.10 for openEuler 21.03. >> >> @Dave >> The openeuler OS host kernel has been updated to v5.10, because I have restarted the host, >> the previously running container has stopped. >> >> Thanks >> Yinsi >> liuyinsi@163.com mailto:liuyinsi@163.com >> >> From: Jammy Zhou mailto:jammy.zhou@linaro.org >> Date: 2021-03-19 08:29 >> To: Jonathan Cameron mailto:Jonathan.Cameron@huawei.com; Guohanjun (Hanjun Guo) mailto:guohanjun@huawei.com >> CC: Dave Pigott mailto:dave.pigott@linaro.org; liuyinsi mailto:liuyinsi@163.com; Anmar Oueja mailto:anmar.oueja@linaro.org; Luca Di Stefano mailto:luca.distefano@linaro.org; wufengguang mailto:wufengguang@huawei.com; lkq-dev mailto:lkq-dev@op-lists.linaro.org; Shameerali Kolothum Thodi mailto:shameerali.kolothum.thodi@huawei.com >> Subject: Re: Issues with docker on OpenEulerOS >> +Hanjun >> >> As I know, the kernel will be based on v5.10 for openEuler 21.09, but for 21.03 and 20.03 LTS, it is still based on v4.19. >> >> On Fri, 19 Mar 2021 at 00:33, Jonathan Cameron <Jonathan.Cameron@huawei.com mailto:Jonathan.Cameron@huawei.com> wrote: >> @yinsi >> >> What are chances of upgrading the host os to a more recent openEuler kernel. >> 4.19 is a bit of a stretch for a backport of the fix. >> I guess it might be easy, but would feel more comfortable with 5.10 or 5.5 >> >> Or can we ask openEuler to carry a backport of this set directly? >> >> Another alternative is to see if there is an available BIOS with the fix >> in place and upgrade. >> >> Jonathan >> >> On Thu, 18 Mar 2021 13:38:28 +0000 >> Dave Pigott <dave.pigott@linaro.org mailto:dave.pigott@linaro.org> wrote: >> >> > Hi Yinsi, >> > >> > Okay - that fixed it. We’ve now got an excellent download speed. We’re back to the problems with qemu compatibility. Working on this. >> > >> > Thanks for your help! >> > >> > Dave >> > >> > > On 18 Mar 2021, at 12:11, liuyinsi@163.com mailto:liuyinsi@163.com wrote: >> > > >> > > Hi Dave, >> > > >> > > I pull the images in openEulerOS, though you got the message about docker pull limit, you still can pull successfully, ignore the message if not block our work. >> > > >> > > Thanks >> > > Yinsi >> > > >> > > >> > > >> > > liuyinsi >> > > 邮箱：liuyinsi@163.com mailto:liuyinsi@163.com >> > > <https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1&am... https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1&name=liuyinsi&uid=liuyinsi%40163.com&iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fqiyelogo%2FdefaultAvatar.png&items=%5B%22%E9%82%AE%E7%AE%B1%EF%BC%9Aliuyinsi%40163.com%22%5D>签名由 网易邮箱大师 <https://mail.163.com/dashi/dlpro.html?from=mail88 https://mail.163.com/dashi/dlpro.html?from=mail88> 定制 >> > > >> > > On 03/18/2021 19:47, Dave Pigott <mailto:dave.pigott@linaro.org mailto:dave.pigott@linaro.org> wrote: >> > > Hi Yinsi, >> > > >> > > It’s lavasoftware/lava-dispatcher:2021.03 >> > > >> > > Dave >> > > >> > >> On 18 Mar 2021, at 11:44, liuyinsi@163.com mailto:liuyinsi@163.com <mailto:liuyinsi@163.com mailto:liuyinsi@163.com> wrote: >> > >> >> > >> Hi Dave, >> > >> >> > >> What is the docker images name you want to pull from dockerhub？ >> > >> >> > >> Thanks >> > >> Yinsi >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> liuyinsi >> > >> 邮箱：liuyinsi@163.com mailto:liuyinsi@163.com >> > >> <https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1&am... https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1&name=liuyinsi&uid=liuyinsi%40163.com&iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fqiyelogo%2FdefaultAvatar.png&items=%5B%22%E9%82%AE%E7%AE%B1%EF%BC%9Aliuyinsi%40163.com%22%5D>签名由 网易邮箱大师 <https://mail.163.com/dashi/dlpro.html?from=mail88 https://mail.163.com/dashi/dlpro.html?from=mail88> 定制 >> > >> >> > >> On 03/18/2021 19:28, Dave Pigott <mailto:dave.pigott@linaro.org mailto:dave.pigott@linaro.org> wrote: >> > >> Hi Yinsi, >> > >> >> > >> We’re trying to upgrade the docker worker to the latest release and we got the message that we’ve hit our docker pull limit. We’ve only done two pulls. Any ideas? >> > >> >> > >> Thanks >> > >> >> > >> Dave >> > >> >> > >> >> > >>> On 18 Mar 2021, at 10:47, liuyinsi@163.com mailto:liuyinsi@163.com <mailto:liuyinsi@163.com mailto:liuyinsi@163.com> wrote: >> > >>> >> > >>> Hi Dave, >> > >>> >> > >>> I notice the job 2313644 <https://lkft.validation.linaro.org/scheduler/job/2313644 https://lkft.validation.linaro.org/scheduler/job/2313644> is created 2 weeks, 6 days ago, can you test a new job, test wget http://images.validation.linaro.org/snapshots.linaro.org/components/lava/sta... http://images.validation.linaro.org/snapshots.linaro.org/components/lava/standard/debian/sid/arm64/2/vmlinuz-4.6.0-1-arm64<http://images.validation.linaro.org/snapshots.linaro.org/components/lava/sta... http://images.validation.linaro.org/snapshots.linaro.org/components/lava/standard/debian/sid/arm64/2/vmlinuz-4.6.0-1-arm64> on openEulerOS machine is ok, first download speed about 2.63MB/s. >> > >>> >> > >>> It's better to improve your download code, to face unstable and slow network issues, for example, increase timeout and add retries. >> > >>> >> > >>> Thanks >> > >>> Yinsi >> > >>> >> > >>> liuyinsi@163.com mailto:liuyinsi@163.com <mailto:liuyinsi@163.com mailto:liuyinsi@163.com> >> > >>> >> > >>> From: Dave Pigott <mailto:dave.pigott@linaro.org mailto:dave.pigott@linaro.org> >> > >>> Date: 2021-03-18 17:04 >> > >>> To: liuyinsi@163.com mailto:liuyinsi@163.com <mailto:liuyinsi@163.com mailto:liuyinsi@163.com> >> > >>> CC: Anmar Oueja <mailto:anmar.oueja@linaro.org mailto:anmar.oueja@linaro.org>; Luca Di Stefano <mailto:luca.distefano@linaro.org mailto:luca.distefano@linaro.org>; wufengguang <mailto:wufengguang@huawei.com mailto:wufengguang@huawei.com>; Jonathan Cameron <mailto:jonathan.cameron@huawei.com mailto:jonathan.cameron@huawei.com>; lkq-dev <mailto:lkq-dev@op-lists.linaro.org mailto:lkq-dev@op-lists.linaro.org>; jammy.zhou@linaro.org mailto:jammy.zhou@linaro.org <mailto:jammy.zhou@linaro.org mailto:jammy.zhou@linaro.org> >> > >>> Subject: Re: Issues with docker on OpenEulerOS >> > >>> Hi Yinsi, >> > >>> >> > >>> If you look at a failed example - e.g. https://lkft.validation.linaro.org/scheduler/job/2313644 https://lkft.validation.linaro.org/scheduler/job/2313644<https://lkft.validation.linaro.org/scheduler/job/2313644 https://lkft.validation.linaro.org/scheduler/job/2313644> - you’ll see it’s not failing on a git clone, it’s failing on a wget. >> > >>> >> > >>> Dave >> > >>> >> > >>> >> > >>>> On 18 Mar 2021, at 08:19, liuyinsi@163.com mailto:liuyinsi@163.com <mailto:liuyinsi@163.com mailto:liuyinsi@163.com> wrote: >> > >>>> >> > >>>> Hi Dave, >> > >>>> >> > >>>> "http download timeout", does it happen often? >> > >>>> "https download timeout", has this problem been solved by edit ~/.gitconfig? >> > >>>> >> > >>>> Thanks >> > >>>> Yinsi >> > >>>> >> > >>>> liuyinsi@163.com mailto:liuyinsi@163.com <mailto:liuyinsi@163.com mailto:liuyinsi@163.com> >> > >>>> >> > >>>> From: liuyinsi@163.com mailto:liuyinsi@163.com <mailto:liuyinsi@163.com mailto:liuyinsi@163.com> >> > >>>> Date: 2021-03-17 10:19 >> > >>>> To: Dave Pigott <mailto:dave.pigott@linaro.org mailto:dave.pigott@linaro.org> >> > >>>> CC: Oueja <mailto:anmar.oueja@linaro.org mailto:anmar.oueja@linaro.org>; luca <mailto:luca.distefano@linaro.org mailto:luca.distefano@linaro.org>; wufengguang <mailto:wufengguang@huawei.com mailto:wufengguang@huawei.com>; Cameron <mailto:jonathan.cameron@huawei.com mailto:jonathan.cameron@huawei.com>; lkq-dev <mailto:lkq-dev@op-lists.linaro.org mailto:lkq-dev@op-lists.linaro.org>; jammy.zhou@linaro.org mailto:jammy.zhou@linaro.org <mailto:jammy.zhou@linaro.org mailto:jammy.zhou@linaro.org> >> > >>>> Subject: Re: Re: Issues with docker on OpenEulerOS >> > >>>> >> > >>>> >> > >>>>>> >> > >>>>>> Locally, in the lab, we use KissCache for https cacheing. We would have to go through every test definition submitted by every bot and developer to change from https to git. >> > >>>>>> >> > >>>>> vi ~/.gitconfig >> > >>>>> >> > >>>>> [url "git://github.com http://github.com/ <git://github.com http://github.com/>"] >> > >>>>> insteadOf = https://github.com https://github.com/ <https://github.com/ https://github.com/> >> > >>>>> >> > >>>>> This will change from https to git. >> > >>>>> >> > >>>> >> > >>>> Hi Yinsi, >> > >>>> >> > >>>> If you look at the job definitions, e.g. https://lkft.validation.linaro.org/scheduler/job/2409008/definition#defline4... https://lkft.validation.linaro.org/scheduler/job/2409008/definition#defline40<https://lkft.validation.linaro.org/scheduler/job/2409008/definition#defline4... https://lkft.validation.linaro.org/scheduler/job/2409008/definition#defline40>, you’ll see the URL is defined there. I’m not sure where you are suggesting this could be universally changed to use git. >> > >>>> >> > >>>> Hi Dave, >> > >>>> >> > >>>> You can edit the file in any environment where you execute the clone command, for example, i get the URL and will git clone in a qemu, if i edit ~/.gitconfig in qemu before clone, >> > >>>> then it will automatically change https to git. >> > >>>> >> > >>>> Thanks >> > >>>> Yinsi >> > >>>> >> > >>>> Dave >> > >>>>>> >> > >>>>>> >> > >>>>>> >> > >>>>>> Thanks >> > >>>>>> >> > >>>>>> Dave >> > >>>>>> >> > >>>>>> >> > >>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>>> anmar >> > >> >> > > >> > >> > >> >> liuyinsi >> 邮箱：liuyinsi@163.com >> https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1&name=liuyinsi&uid=liuyinsi%40163.com&iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fqiyelogo%2FdefaultAvatar.png&items=%5B%22%E9%82%AE%E7%AE%B1%EF%BC%9Aliuyinsi%40163.com%22%5D签名由 网易邮箱大师 https://mail.163.com/dashi/dlpro.html?from=mail88 定制