Hello Fengguang,
I hope this email finds you well. Since our call last week, we’ve been working internally trying to best identify issues/questions we need to draft up the scope of work for our collaboration with Compass CI and we have the following points we wish to discuss with you further.
After some internal discussions, we think and support your proposal of having Compass CI dispatch jobs to LAVA via the dispatcher. However, we did have a few questions we need your help in answering:
Q1. Do you expect to use LKFT’s lab?
Q2. Do you want to add your embedded boards to your dispatcher in your own lab?
Q3. We assume you want to test the OpenEuler kernel. Do you want to use our OpenEmbedded or OpenEuler rootFS?
Q4. Do you want to run the same test we run in LKFT?
Q5. Do you want to run LKP tests
Q6. IF you wish to run Linaro test definitions, the data will be stored in SQUAD. For LKP, we expect the tests themselves to send the test results to Compass CI directly.
We spent a lot of time discussing how to best collaborate with Compass CI to the benefit of Linaro's members and we believe bolstering benchmark testing is a key value your team can help us deliver. The focus is on benchmark test extraction, processing and comparison. The functionality we seek is in LKP so your team’s help in extracting and genericising such tooling so we can leverage it in LKFT and other test frameworks.
We can discuss the above points over email or setup a call if you like.
Thanks, Anmar
Hi Anmar,
Hello Fengguang,
I hope this email finds you well. Since our call last week, we’ve been working internally trying to best identify issues/questions we need to draft up the scope of work for our collaboration with Compass CI and we have the following points we wish to discuss with you further.
After some internal discussions, we think and support your proposal of having Compass CI dispatch jobs to LAVA via the dispatcher.
Glad to hear that, thanks!
However, we did have a few questions we need your help in answering:
OK. Let me try answering these questions in the context of "having Compass CI dispatch jobs to LAVA via the dispatcher".
Q1. Do you expect to use LKFT’s lab?
Yes in the sense of reusing LKFT lab's current facilities and functionalities to test openEuler kernel.
As for "Compass CI + LAVA dispatcher", we mainly care about implementing the functionality. You may freely choose the most convenient lab (or create an experimental instance) to use in the development and debug process.
Q2. Do you want to add your embedded boards to your dispatcher in your own lab?
I suppose so, after implemented the functionality.
During the development phase, the developer may try things out on his local side if that would be more efficient.
Q3. We assume you want to test the OpenEuler kernel. Do you want to use our OpenEmbedded or OpenEuler rootFS?
Yes we'll test openEuler kernel.
For rootFS, I guess whatever readily available (eg. OpenEmbedded) should be fine in the beginning. When the job conversion+execution flow works, we may go on to check how to deploy openEuler rootFS via LAVA for limited SUT types.
Q4. Do you want to run the same test we run in LKFT?
LKP jobs are our priority. I tend to not list LKFT jobs in the goal of this CompassCI+LAVA integration project, to reduce unnecessary complexity.
Q5. Do you want to run LKP tests
Yes. Submitting LKP jobs to LAVA controlled devices. Auto converting LKP job to LAVA job for execution.
Q6. IF you wish to run Linaro test definitions, the data will be stored in SQUAD. For LKP, we expect the tests themselves to send the test results to Compass CI directly.
OK, got it.
We spent a lot of time discussing how to best collaborate with Compass CI to the benefit of Linaro's members and we believe bolstering benchmark testing is a key value your team can help us deliver. The focus is on benchmark test extraction, processing and comparison. The functionality we seek is in LKP so your team’s help in extracting and genericising such tooling so we can leverage it in LKFT and other test frameworks.
lkp-tests is designed to be reusable by other projects. It should relatively easy to reuse its code.
compass-ci services all run in containers, which makes their build dependency clear.
Though there may still be a ton of boring/messy works in trying to integrating different parts of lkp-tests/compass-ci to LKFT in order to create a complete solution.
Another tough question is whether LKFT duplicate the code and evolve standalone, or reuse code in place while trying to modify various places of the code to work in a different situation.
compass-ci grows on top of lkp-tests in the very beginning, which makes the development natural and simple.
Thanks, Fengguang
Hello Fengguang,
Thank you very much for your reply.
<snip>
lkp-tests is designed to be reusable by other projects. It should
relatively easy to reuse its code.
We would like to setup a call so we are able to explain our use case. We can go over how we do things from our side, and from this it would be good to figure out who is the best person will be to achieve this.
compass-ci services all run in containers, which makes their build
dependency clear.
Though there may still be a ton of boring/messy works in trying to
integrating different parts of lkp-tests/compass-ci to LKFT in order to
create a complete solution.
Perhaps, we would like to work together to triage this and work out the best way to tackle it.
Another tough question is whether LKFT duplicate the code and evolve
standalone, or reuse code in place while trying to modify
various places of the code to work in a different situation.
We are committed to collaborating with other frameworks, CompassCI included of course, and as a result, we will be pushing out changes back to you or LKP proper, where ever it makes sense.
compass-ci grows on top of lkp-tests in the very beginning, which makes
the development natural and simple.
With your team's help in achieving the goals outlined in this email, we hope that will be the case.
Regards
Ben
Thanks,
Fengguang
Hi Ben,
OK. How about having a meeting in the following 1-2 days? We are just back from holidays.
Thanks, Fengguang
-----Original Message----- From: Benjamin Copeland [mailto:ben.copeland@linaro.org] Sent: Wednesday, April 28, 2021 4:33 PM To: wufengguang wufengguang@huawei.com Cc: Anmar Oueja anmar.oueja@linaro.org; Jammy Zhou jammy.zhou@linaro.org; Anders Roxell anders.roxell@linaro.org; lkq-dev lkq-dev@op-lists.linaro.org Subject: Re: new proposal for the OpenEuler project
Hello Fengguang,
Thank you very much for your reply.
<snip>
lkp-tests is designed to be reusable by other projects. It should
relatively easy to reuse its code.
We would like to setup a call so we are able to explain our use case. We can go over how we do things from our side, and from this it would be good to figure out who is the best person will be to achieve this.
compass-ci services all run in containers, which makes their build
dependency clear.
Though there may still be a ton of boring/messy works in trying to
integrating different parts of lkp-tests/compass-ci to LKFT in order to
create a complete solution.
Perhaps, we would like to work together to triage this and work out the best way to tackle it.
Another tough question is whether LKFT duplicate the code and evolve
standalone, or reuse code in place while trying to modify
various places of the code to work in a different situation.
We are committed to collaborating with other frameworks, CompassCI included of course, and as a result, we will be pushing out changes back to you or LKP proper, where ever it makes sense.
compass-ci grows on top of lkp-tests in the very beginning, which makes
the development natural and simple.
With your team's help in achieving the goals outlined in this email, we hope that will be the case.
Regards
Ben
Thanks,
Fengguang
Hello Fengguang,
Further our meeting regarding CompassCI and LAVA integration. As you are aware, we recommended the use of the LAVA Server. The reason for this is so you do not need to reinvent the wheel. You can, of course use LAVA dispatcher directly, but LAVA server deals with:
- Scheduling jobs This is done per user/tags/priority and device health. CompassCI can schedule jobs into lava's scheduler, and LAVA server can be left dealing with all the embedded queuing. This means you can focus on the job conversion over integrating compass-ci to deal with what LAVA server currently does. - Health checks Regular health checks to devices will happen regularly, and once failed LAVA server will deal with taking the device offline accordingly. An external scheduler would have to keep track of this otherwise. - Easy to control and supported hardware Ability to easily use PDU's. reset boards. - 10 years of development and going on.
You mentioned Compass-CI is built around a pull mode. A layer can be added next to the job conversion, which can be done per target that will allow you to submit jobs to LAVA. The LAVA results can then be returned to that layer, and be pushed to Compass-CI as normal.
Regards
Ben
On Thu, 6 May 2021 at 12:21, wufengguang wufengguang@huawei.com wrote:
Hi Ben,
OK. How about having a meeting in the following 1-2 days? We are just back from holidays.
Thanks, Fengguang
-----Original Message----- From: Benjamin Copeland [mailto:ben.copeland@linaro.org] Sent: Wednesday, April 28, 2021 4:33 PM To: wufengguang wufengguang@huawei.com Cc: Anmar Oueja anmar.oueja@linaro.org; Jammy Zhou jammy.zhou@linaro.org; Anders Roxell anders.roxell@linaro.org; lkq-dev lkq-dev@op-lists.linaro.org Subject: Re: new proposal for the OpenEuler project
Hello Fengguang,
Thank you very much for your reply.
<snip> > > > lkp-tests is designed to be reusable by other projects. It should > > relatively easy to reuse its code.
We would like to setup a call so we are able to explain our use case. We can go over how we do things from our side, and from this it would be good to figure out who is the best person will be to achieve this.
compass-ci services all run in containers, which makes their build
dependency clear.
Though there may still be a ton of boring/messy works in trying to
integrating different parts of lkp-tests/compass-ci to LKFT in order to
create a complete solution.
Perhaps, we would like to work together to triage this and work out the best way to tackle it.
Another tough question is whether LKFT duplicate the code and evolve
standalone, or reuse code in place while trying to modify
various places of the code to work in a different situation.
We are committed to collaborating with other frameworks, CompassCI included of course, and as a result, we will be pushing out changes back to you or LKP proper, where ever it makes sense.
compass-ci grows on top of lkp-tests in the very beginning, which makes
the development natural and simple.
With your team's help in achieving the goals outlined in this email, we hope that will be the case.
Regards
Ben
Thanks,
Fengguang
Hi Ben,
So there are 2 options
1) write a provider script, which does loop: - contact compass-ci scheduler to get the job - convert it to LAVA job - lava-run the job to DUT
2) add LAVA server to the loop, too
I'm not sure how exactly (2) will work out. It looks like will add some unnecessary indirection and complexity comparing to (1).
Further our meeting regarding CompassCI and LAVA integration. As you are aware, we recommended the use of the LAVA Server. The reason for this is so you do not need to reinvent the wheel. You can, of course use LAVA dispatcher directly, but LAVA server deals with:
- Scheduling jobs This is done per user/tags/priority and device health. CompassCI can schedule jobs into lava's scheduler, and LAVA server can be left dealing with all the embedded queuing. This means you can focus on the job conversion over integrating compass-ci to deal with what LAVA server currently does.
IMHO 2 schedulers brings more complexity than convenience.
The compass-ci scheduler will also have those scheduling capabilities. User/priority is already working. As for tags, some generized HW select/matching mechanism is in our TODO list.
- Health checks Regular health checks to devices will happen regularly, and once failed LAVA server will deal with taking the device offline accordingly. An external scheduler would have to keep track of this otherwise.
Since compass-ci works in PULL mode, its scheduler does not rely on health checks. It just waits for connection from live testboxes (or providers on behalf of them).
It's trivial to routinely send health check jobs in compass-ci, if some high level logic needs it.
- Easy to control and supported hardware Ability to easily use PDU's. reset boards.
It seems the LAVA dispatcher handle power reset by itself.
We may also write a general agent to power reset a LAVA DUT based on command/messages sent by other compass-ci entities for more complete integration.
Thanks, Fengguang
Hi Benjamin,
It's been long time after last discussion. In order to make some progress, how about start working on some essential parts first? That is "Auto converting LKP job to LAVA job for execution".
Here is an example LKP job: http://124.160.11.58:20007/result/host-info/2021-08-13/vm-2p8g/debian-sid-aa...
It has job.yaml whose most relevant content is
kernel_uri: http://172.168.131.2:8000/os/debian/aarch64/sid-snapshots/2021-01-13-14-45-4... kernel_params: - user=lkp - job=/lkp/scheduled/job.yaml - RESULT_ROOT=/result/job - ip=dhcp - rootovl - ro - rdinit=/sbin/init - prompt_ramdisk=0 initrds_uri: - http://172.168.131.2:8800/initrd/osimage/debian/aarch64/sid/20201019.0.cgz - http://172.168.131.2:8800/initrd/deps/nfs/debian/aarch64/sid/run-ipconfig_20... - http://172.168.131.2:8000/os/debian/aarch64/sid-snapshots/2021-01-13-14-45-4... - http://172.168.131.2:3018/job_initrd_tmpfs/z9.8246050/job.cgz - http://172.168.131.2:8800/upload-files/lkp-tests/aarch64/v1.0.cgz - http://172.168.131.2:8800/upload-files/lkp-tests/15/1504212313f2206dffa9e0a3...
The kernel deployment is not a must-have at first. As long as you can convert job.yaml to LAVA job, then let lava-run deploy/run those initrd files in the device, the LKP test script inside those initrd files will auto startup. That basically enables the bare minimal flow to run LKP jobs in LAVA.
Thanks, Fengguang -----Original Message----- From: Benjamin Copeland [mailto:ben.copeland@linaro.org] Sent: Wednesday, May 26, 2021 2:02 AM To: wufengguang wufengguang@huawei.com Cc: Anmar Oueja anmar.oueja@linaro.org; Jammy Zhou jammy.zhou@linaro.org; Anders Roxell anders.roxell@linaro.org; lkq-dev lkq-dev@op-lists.linaro.org Subject: Re: new proposal for the OpenEuler project
Hello Fengguang,
Further our meeting regarding CompassCI and LAVA integration. As you are aware, we recommended the use of the LAVA Server. The reason for this is so you do not need to reinvent the wheel. You can, of course use LAVA dispatcher directly, but LAVA server deals with:
- Scheduling jobs This is done per user/tags/priority and device health. CompassCI can schedule jobs into lava's scheduler, and LAVA server can be left dealing with all the embedded queuing. This means you can focus on the job conversion over integrating compass-ci to deal with what LAVA server currently does. - Health checks Regular health checks to devices will happen regularly, and once failed LAVA server will deal with taking the device offline accordingly. An external scheduler would have to keep track of this otherwise. - Easy to control and supported hardware Ability to easily use PDU's. reset boards. - 10 years of development and going on.
You mentioned Compass-CI is built around a pull mode. A layer can be added next to the job conversion, which can be done per target that will allow you to submit jobs to LAVA. The LAVA results can then be returned to that layer, and be pushed to Compass-CI as normal.
Regards
Ben
On Thu, 6 May 2021 at 12:21, wufengguang wufengguang@huawei.com wrote:
Hi Ben,
OK. How about having a meeting in the following 1-2 days? We are just back from holidays.
Thanks, Fengguang
-----Original Message----- From: Benjamin Copeland [mailto:ben.copeland@linaro.org] Sent: Wednesday, April 28, 2021 4:33 PM To: wufengguang wufengguang@huawei.com Cc: Anmar Oueja anmar.oueja@linaro.org; Jammy Zhou jammy.zhou@linaro.org; Anders Roxell anders.roxell@linaro.org; lkq-dev lkq-dev@op-lists.linaro.org Subject: Re: new proposal for the OpenEuler project
Hello Fengguang,
Thank you very much for your reply.
<snip> > > > lkp-tests is designed to be reusable by other projects. It should > > relatively easy to reuse its code.
We would like to setup a call so we are able to explain our use case. We can go over how we do things from our side, and from this it would be good to figure out who is the best person will be to achieve this.
compass-ci services all run in containers, which makes their build
dependency clear.
Though there may still be a ton of boring/messy works in trying to
integrating different parts of lkp-tests/compass-ci to LKFT in order to
create a complete solution.
Perhaps, we would like to work together to triage this and work out the best way to tackle it.
Another tough question is whether LKFT duplicate the code and evolve
standalone, or reuse code in place while trying to modify
various places of the code to work in a different situation.
We are committed to collaborating with other frameworks, CompassCI included of course, and as a result, we will be pushing out changes back to you or LKP proper, where ever it makes sense.
compass-ci grows on top of lkp-tests in the very beginning, which makes
the development natural and simple.
With your team's help in achieving the goals outlined in this email, we hope that will be the case.
Regards
Ben
Thanks,
Fengguang
Hello Wufengguang,
Apologies for the late response I have been on vacation.
I will reply once I've properly caught up, which should be sometime next week.
Regards
Ben
On Tue, 17 Aug 2021 at 13:19, wufengguang wufengguang@huawei.com wrote:
Hi Benjamin,
It's been long time after last discussion. In order to make some progress, how about start working on some essential parts first? That is "Auto converting LKP job to LAVA job for execution".
Here is an example LKP job: http://124.160.11.58:20007/result/host-info/2021-08-13/vm-2p8g/debian-sid-aa...
It has job.yaml whose most relevant content is
kernel_uri: http://172.168.131.2:8000/os/debian/aarch64/sid-snapshots/2021-01-13-14-45-4... kernel_params:
- user=lkp
- job=/lkp/scheduled/job.yaml
- RESULT_ROOT=/result/job
- ip=dhcp
- rootovl
- ro
- rdinit=/sbin/init
- prompt_ramdisk=0
initrds_uri:
- http://172.168.131.2:8800/initrd/osimage/debian/aarch64/sid/20201019.0.cgz
- http://172.168.131.2:8800/initrd/deps/nfs/debian/aarch64/sid/run-ipconfig_20...
- http://172.168.131.2:8000/os/debian/aarch64/sid-snapshots/2021-01-13-14-45-4...
- http://172.168.131.2:3018/job_initrd_tmpfs/z9.8246050/job.cgz
- http://172.168.131.2:8800/upload-files/lkp-tests/aarch64/v1.0.cgz
- http://172.168.131.2:8800/upload-files/lkp-tests/15/1504212313f2206dffa9e0a3...
The kernel deployment is not a must-have at first. As long as you can convert job.yaml to LAVA job, then let lava-run deploy/run those initrd files in the device, the LKP test script inside those initrd files will auto startup. That basically enables the bare minimal flow to run LKP jobs in LAVA.
Thanks, Fengguang -----Original Message----- From: Benjamin Copeland [mailto:ben.copeland@linaro.org] Sent: Wednesday, May 26, 2021 2:02 AM To: wufengguang wufengguang@huawei.com Cc: Anmar Oueja anmar.oueja@linaro.org; Jammy Zhou jammy.zhou@linaro.org; Anders Roxell anders.roxell@linaro.org; lkq-dev lkq-dev@op-lists.linaro.org Subject: Re: new proposal for the OpenEuler project
Hello Fengguang,
Further our meeting regarding CompassCI and LAVA integration. As you are aware, we recommended the use of the LAVA Server. The reason for this is so you do not need to reinvent the wheel. You can, of course use LAVA dispatcher directly, but LAVA server deals with:
- Scheduling jobs
This is done per user/tags/priority and device health. CompassCI can schedule jobs into lava's scheduler, and LAVA server can be left dealing with all the embedded queuing. This means you can focus on the job conversion over integrating compass-ci to deal with what LAVA server currently does.
- Health checks
Regular health checks to devices will happen regularly, and once failed LAVA server will deal with taking the device offline accordingly. An external scheduler would have to keep track of this otherwise.
- Easy to control and supported hardware Ability to easily use PDU's. reset boards.
- 10 years of development and going on.
You mentioned Compass-CI is built around a pull mode. A layer can be added next to the job conversion, which can be done per target that will allow you to submit jobs to LAVA. The LAVA results can then be returned to that layer, and be pushed to Compass-CI as normal.
Regards
Ben
On Thu, 6 May 2021 at 12:21, wufengguang wufengguang@huawei.com wrote:
Hi Ben,
OK. How about having a meeting in the following 1-2 days? We are just back from holidays.
Thanks, Fengguang
-----Original Message----- From: Benjamin Copeland [mailto:ben.copeland@linaro.org] Sent: Wednesday, April 28, 2021 4:33 PM To: wufengguang wufengguang@huawei.com Cc: Anmar Oueja anmar.oueja@linaro.org; Jammy Zhou jammy.zhou@linaro.org; Anders Roxell anders.roxell@linaro.org; lkq-dev lkq-dev@op-lists.linaro.org Subject: Re: new proposal for the OpenEuler project
Hello Fengguang,
Thank you very much for your reply.
<snip> > > > lkp-tests is designed to be reusable by other projects. It should > > relatively easy to reuse its code.
We would like to setup a call so we are able to explain our use case. We can go over how we do things from our side, and from this it would be good to figure out who is the best person will be to achieve this.
compass-ci services all run in containers, which makes their build
dependency clear.
Though there may still be a ton of boring/messy works in trying to
integrating different parts of lkp-tests/compass-ci to LKFT in order to
create a complete solution.
Perhaps, we would like to work together to triage this and work out the best way to tackle it.
Another tough question is whether LKFT duplicate the code and evolve
standalone, or reuse code in place while trying to modify
various places of the code to work in a different situation.
We are committed to collaborating with other frameworks, CompassCI included of course, and as a result, we will be pushing out changes back to you or LKP proper, where ever it makes sense.
compass-ci grows on top of lkp-tests in the very beginning, which makes
the development natural and simple.
With your team's help in achieving the goals outlined in this email, we hope that will be the case.
Regards
Ben
Thanks,
Fengguang
Hello Fengguang,
Apologies for the late reply. Vacation schedule made it difficult to have everyone in one room.
Auto-translating the LKP yaml files into a LAVA jobs, as you describe, can be done using the documentation referenced. You will likely find the schema validation part of the LAVA documentation [1] useful, and the code [2]. If you run into problems, please don’t hesitate to contact us on #lavasoftware on IRC (libera.chat) or use the Mailing list [3].
Hope this helps, Regards
Benjamin Copeland
[1] https://docs.lavasoftware.org/lava/pipeline-schema.html#job-submission-schem... [2] https://git.lavasoftware.org/lava/lava/-/tree/master/lava_common/schemas [3] https://lists.lavasoftware.org/mailman/listinfo/lava-devel
On Tue, 17 Aug 2021 at 13:19, wufengguang wufengguang@huawei.com wrote:
Hi Benjamin,
It's been long time after last discussion. In order to make some progress, how about start working on some essential parts first? That is "Auto converting LKP job to LAVA job for execution".
Here is an example LKP job: http://124.160.11.58:20007/result/host-info/2021-08-13/vm-2p8g/debian-sid-aa...
It has job.yaml whose most relevant content is
kernel_uri: http://172.168.131.2:8000/os/debian/aarch64/sid-snapshots/2021-01-13-14-45-4... kernel_params:
- user=lkp
- job=/lkp/scheduled/job.yaml
- RESULT_ROOT=/result/job
- ip=dhcp
- rootovl
- ro
- rdinit=/sbin/init
- prompt_ramdisk=0
initrds_uri:
- http://172.168.131.2:8800/initrd/osimage/debian/aarch64/sid/20201019.0.cgz
- http://172.168.131.2:8800/initrd/deps/nfs/debian/aarch64/sid/run-ipconfig_20...
- http://172.168.131.2:8000/os/debian/aarch64/sid-snapshots/2021-01-13-14-45-4...
- http://172.168.131.2:3018/job_initrd_tmpfs/z9.8246050/job.cgz
- http://172.168.131.2:8800/upload-files/lkp-tests/aarch64/v1.0.cgz
- http://172.168.131.2:8800/upload-files/lkp-tests/15/1504212313f2206dffa9e0a3...
The kernel deployment is not a must-have at first. As long as you can convert job.yaml to LAVA job, then let lava-run deploy/run those initrd files in the device, the LKP test script inside those initrd files will auto startup. That basically enables the bare minimal flow to run LKP jobs in LAVA.
Thanks, Fengguang -----Original Message----- From: Benjamin Copeland [mailto:ben.copeland@linaro.org] Sent: Wednesday, May 26, 2021 2:02 AM To: wufengguang wufengguang@huawei.com Cc: Anmar Oueja anmar.oueja@linaro.org; Jammy Zhou jammy.zhou@linaro.org; Anders Roxell anders.roxell@linaro.org; lkq-dev lkq-dev@op-lists.linaro.org Subject: Re: new proposal for the OpenEuler project
Hello Fengguang,
Further our meeting regarding CompassCI and LAVA integration. As you are aware, we recommended the use of the LAVA Server. The reason for this is so you do not need to reinvent the wheel. You can, of course use LAVA dispatcher directly, but LAVA server deals with:
- Scheduling jobs
This is done per user/tags/priority and device health. CompassCI can schedule jobs into lava's scheduler, and LAVA server can be left dealing with all the embedded queuing. This means you can focus on the job conversion over integrating compass-ci to deal with what LAVA server currently does.
- Health checks
Regular health checks to devices will happen regularly, and once failed LAVA server will deal with taking the device offline accordingly. An external scheduler would have to keep track of this otherwise.
- Easy to control and supported hardware Ability to easily use PDU's. reset boards.
- 10 years of development and going on.
You mentioned Compass-CI is built around a pull mode. A layer can be added next to the job conversion, which can be done per target that will allow you to submit jobs to LAVA. The LAVA results can then be returned to that layer, and be pushed to Compass-CI as normal.
Regards
Ben
On Thu, 6 May 2021 at 12:21, wufengguang wufengguang@huawei.com wrote:
Hi Ben,
OK. How about having a meeting in the following 1-2 days? We are just back from holidays.
Thanks, Fengguang
-----Original Message----- From: Benjamin Copeland [mailto:ben.copeland@linaro.org] Sent: Wednesday, April 28, 2021 4:33 PM To: wufengguang wufengguang@huawei.com Cc: Anmar Oueja anmar.oueja@linaro.org; Jammy Zhou jammy.zhou@linaro.org; Anders Roxell anders.roxell@linaro.org; lkq-dev lkq-dev@op-lists.linaro.org Subject: Re: new proposal for the OpenEuler project
Hello Fengguang,
Thank you very much for your reply.
<snip> > > > lkp-tests is designed to be reusable by other projects. It should > > relatively easy to reuse its code.
We would like to setup a call so we are able to explain our use case. We can go over how we do things from our side, and from this it would be good to figure out who is the best person will be to achieve this.
compass-ci services all run in containers, which makes their build
dependency clear.
Though there may still be a ton of boring/messy works in trying to
integrating different parts of lkp-tests/compass-ci to LKFT in order to
create a complete solution.
Perhaps, we would like to work together to triage this and work out the best way to tackle it.
Another tough question is whether LKFT duplicate the code and evolve
standalone, or reuse code in place while trying to modify
various places of the code to work in a different situation.
We are committed to collaborating with other frameworks, CompassCI included of course, and as a result, we will be pushing out changes back to you or LKP proper, where ever it makes sense.
compass-ci grows on top of lkp-tests in the very beginning, which makes
the development natural and simple.
With your team's help in achieving the goals outlined in this email, we hope that will be the case.
Regards
Ben
Thanks,
Fengguang
Got it, thanks for the documentation. I'll check when get the time.
Thanks, Fengguang
-----Original Message----- From: Benjamin Copeland [mailto:ben.copeland@linaro.org] Sent: Tuesday, September 14, 2021 11:33 PM To: wufengguang wufengguang@huawei.com Cc: Anmar Oueja anmar.oueja@linaro.org; Jammy Zhou jammy.zhou@linaro.org; Anders Roxell anders.roxell@linaro.org; lkq-dev lkq-dev@op-lists.linaro.org Subject: Re: new proposal for the OpenEuler project
Hello Fengguang,
Apologies for the late reply. Vacation schedule made it difficult to have everyone in one room.
Auto-translating the LKP yaml files into a LAVA jobs, as you describe, can be done using the documentation referenced. You will likely find the schema validation part of the LAVA documentation [1] useful, and the code [2]. If you run into problems, please don’t hesitate to contact us on #lavasoftware on IRC (libera.chat) or use the Mailing list [3].
Hope this helps, Regards
Benjamin Copeland
[1] https://docs.lavasoftware.org/lava/pipeline-schema.html#job-submission-schem... [2] https://git.lavasoftware.org/lava/lava/-/tree/master/lava_common/schemas [3] https://lists.lavasoftware.org/mailman/listinfo/lava-devel
On Tue, 17 Aug 2021 at 13:19, wufengguang wufengguang@huawei.com wrote:
Hi Benjamin,
It's been long time after last discussion. In order to make some progress, how about start working on some essential parts first? That is "Auto converting LKP job to LAVA job for execution".
Here is an example LKP job: http://124.160.11.58:20007/result/host-info/2021-08-13/vm-2p8g/debian- sid-aarch64/z9.8246050/
It has job.yaml whose most relevant content is
kernel_uri: http://172.168.131.2:8000/os/debian/aarch64/sid-snapshots/2021-01-13-1 4-45-46/boot/vmlinuz-5.4.0-4-arm64 kernel_params:
- user=lkp
- job=/lkp/scheduled/job.yaml
- RESULT_ROOT=/result/job
- ip=dhcp
- rootovl
- ro
- rdinit=/sbin/init
- prompt_ramdisk=0
initrds_uri:
http://172.168.131.2:8800/initrd/osimage/debian/aarch64/sid/20201019.0 .cgz
http://172.168.131.2:8800/initrd/deps/nfs/debian/aarch64/sid/run-ipcon fig_20200904.cgz
http://172.168.131.2:8000/os/debian/aarch64/sid-snapshots/2021-01-13-1 4-45-46/boot/modules-5.4.0-4-arm64.cgz
- http://172.168.131.2:3018/job_initrd_tmpfs/z9.8246050/job.cgz
- http://172.168.131.2:8800/upload-files/lkp-tests/aarch64/v1.0.cgz
http://172.168.131.2:8800/upload-files/lkp-tests/15/1504212313f2206dff a9e0a33a0ba5e5.cgz
The kernel deployment is not a must-have at first. As long as you can convert job.yaml to LAVA job, then let lava-run deploy/run those initrd files in the device, the LKP test script inside those initrd files will auto startup. That basically enables the bare minimal flow to run LKP jobs in LAVA.
Thanks, Fengguang -----Original Message----- From: Benjamin Copeland [mailto:ben.copeland@linaro.org] Sent: Wednesday, May 26, 2021 2:02 AM To: wufengguang wufengguang@huawei.com Cc: Anmar Oueja anmar.oueja@linaro.org; Jammy Zhou jammy.zhou@linaro.org; Anders Roxell anders.roxell@linaro.org; lkq-dev lkq-dev@op-lists.linaro.org Subject: Re: new proposal for the OpenEuler project
Hello Fengguang,
Further our meeting regarding CompassCI and LAVA integration. As you are aware, we recommended the use of the LAVA Server. The reason for this is so you do not need to reinvent the wheel. You can, of course use LAVA dispatcher directly, but LAVA server deals with:
- Scheduling jobs
This is done per user/tags/priority and device health. CompassCI can schedule jobs into lava's scheduler, and LAVA server can be left dealing with all the embedded queuing. This means you can focus on the job conversion over integrating compass-ci to deal with what LAVA server currently does.
- Health checks
Regular health checks to devices will happen regularly, and once failed LAVA server will deal with taking the device offline accordingly. An external scheduler would have to keep track of this otherwise.
- Easy to control and supported hardware Ability to easily use PDU's. reset boards.
- 10 years of development and going on.
You mentioned Compass-CI is built around a pull mode. A layer can be added next to the job conversion, which can be done per target that will allow you to submit jobs to LAVA. The LAVA results can then be returned to that layer, and be pushed to Compass-CI as normal.
Regards
Ben
On Thu, 6 May 2021 at 12:21, wufengguang wufengguang@huawei.com wrote:
Hi Ben,
OK. How about having a meeting in the following 1-2 days? We are just back from holidays.
Thanks, Fengguang
-----Original Message----- From: Benjamin Copeland [mailto:ben.copeland@linaro.org] Sent: Wednesday, April 28, 2021 4:33 PM To: wufengguang wufengguang@huawei.com Cc: Anmar Oueja anmar.oueja@linaro.org; Jammy Zhou jammy.zhou@linaro.org; Anders Roxell anders.roxell@linaro.org; lkq-dev lkq-dev@op-lists.linaro.org Subject: Re: new proposal for the OpenEuler project
Hello Fengguang,
Thank you very much for your reply.
<snip> > > > lkp-tests is designed to be reusable by other projects. It should > > relatively easy to reuse its code.
We would like to setup a call so we are able to explain our use case. We can go over how we do things from our side, and from this it would be good to figure out who is the best person will be to achieve this.
compass-ci services all run in containers, which makes their build
dependency clear.
Though there may still be a ton of boring/messy works in trying to
integrating different parts of lkp-tests/compass-ci to LKFT in order to
create a complete solution.
Perhaps, we would like to work together to triage this and work out the best way to tackle it.
Another tough question is whether LKFT duplicate the code and evolve
standalone, or reuse code in place while trying to modify
various places of the code to work in a different situation.
We are committed to collaborating with other frameworks, CompassCI included of course, and as a result, we will be pushing out changes back to you or LKP proper, where ever it makes sense.
compass-ci grows on top of lkp-tests in the very beginning, which makes
the development natural and simple.
With your team's help in achieving the goals outlined in this email, we hope that will be the case.
Regards
Ben
Thanks,
Fengguang
-
Anmar Oueja -
Benjamin Copeland -
wufengguang