Hi Benjamin,
It's been long time after last discussion. In order to make some progress, how about start working on some essential parts first? That is "Auto converting LKP job to LAVA job for execution".
Here is an example LKP job: http://124.160.11.58:20007/result/host-info/2021-08-13/vm-2p8g/debian-sid-aa...
It has job.yaml whose most relevant content is
kernel_uri: http://172.168.131.2:8000/os/debian/aarch64/sid-snapshots/2021-01-13-14-45-4... kernel_params: - user=lkp - job=/lkp/scheduled/job.yaml - RESULT_ROOT=/result/job - ip=dhcp - rootovl - ro - rdinit=/sbin/init - prompt_ramdisk=0 initrds_uri: - http://172.168.131.2:8800/initrd/osimage/debian/aarch64/sid/20201019.0.cgz - http://172.168.131.2:8800/initrd/deps/nfs/debian/aarch64/sid/run-ipconfig_20... - http://172.168.131.2:8000/os/debian/aarch64/sid-snapshots/2021-01-13-14-45-4... - http://172.168.131.2:3018/job_initrd_tmpfs/z9.8246050/job.cgz - http://172.168.131.2:8800/upload-files/lkp-tests/aarch64/v1.0.cgz - http://172.168.131.2:8800/upload-files/lkp-tests/15/1504212313f2206dffa9e0a3...
The kernel deployment is not a must-have at first. As long as you can convert job.yaml to LAVA job, then let lava-run deploy/run those initrd files in the device, the LKP test script inside those initrd files will auto startup. That basically enables the bare minimal flow to run LKP jobs in LAVA.
Thanks, Fengguang -----Original Message----- From: Benjamin Copeland [mailto:ben.copeland@linaro.org] Sent: Wednesday, May 26, 2021 2:02 AM To: wufengguang wufengguang@huawei.com Cc: Anmar Oueja anmar.oueja@linaro.org; Jammy Zhou jammy.zhou@linaro.org; Anders Roxell anders.roxell@linaro.org; lkq-dev lkq-dev@op-lists.linaro.org Subject: Re: new proposal for the OpenEuler project
Hello Fengguang,
Further our meeting regarding CompassCI and LAVA integration. As you are aware, we recommended the use of the LAVA Server. The reason for this is so you do not need to reinvent the wheel. You can, of course use LAVA dispatcher directly, but LAVA server deals with:
- Scheduling jobs This is done per user/tags/priority and device health. CompassCI can schedule jobs into lava's scheduler, and LAVA server can be left dealing with all the embedded queuing. This means you can focus on the job conversion over integrating compass-ci to deal with what LAVA server currently does. - Health checks Regular health checks to devices will happen regularly, and once failed LAVA server will deal with taking the device offline accordingly. An external scheduler would have to keep track of this otherwise. - Easy to control and supported hardware Ability to easily use PDU's. reset boards. - 10 years of development and going on.
You mentioned Compass-CI is built around a pull mode. A layer can be added next to the job conversion, which can be done per target that will allow you to submit jobs to LAVA. The LAVA results can then be returned to that layer, and be pushed to Compass-CI as normal.
Regards
Ben
On Thu, 6 May 2021 at 12:21, wufengguang wufengguang@huawei.com wrote:
Hi Ben,
OK. How about having a meeting in the following 1-2 days? We are just back from holidays.
Thanks, Fengguang
-----Original Message----- From: Benjamin Copeland [mailto:ben.copeland@linaro.org] Sent: Wednesday, April 28, 2021 4:33 PM To: wufengguang wufengguang@huawei.com Cc: Anmar Oueja anmar.oueja@linaro.org; Jammy Zhou jammy.zhou@linaro.org; Anders Roxell anders.roxell@linaro.org; lkq-dev lkq-dev@op-lists.linaro.org Subject: Re: new proposal for the OpenEuler project
Hello Fengguang,
Thank you very much for your reply.
<snip> > > > lkp-tests is designed to be reusable by other projects. It should > > relatively easy to reuse its code.
We would like to setup a call so we are able to explain our use case. We can go over how we do things from our side, and from this it would be good to figure out who is the best person will be to achieve this.
compass-ci services all run in containers, which makes their build
dependency clear.
Though there may still be a ton of boring/messy works in trying to
integrating different parts of lkp-tests/compass-ci to LKFT in order to
create a complete solution.
Perhaps, we would like to work together to triage this and work out the best way to tackle it.
Another tough question is whether LKFT duplicate the code and evolve
standalone, or reuse code in place while trying to modify
various places of the code to work in a different situation.
We are committed to collaborating with other frameworks, CompassCI included of course, and as a result, we will be pushing out changes back to you or LKP proper, where ever it makes sense.
compass-ci grows on top of lkp-tests in the very beginning, which makes
the development natural and simple.
With your team's help in achieving the goals outlined in this email, we hope that will be the case.
Regards
Ben
Thanks,
Fengguang