- lava-users - op-lists.linaro.org

Passing runtime parameters to user_commands in device configuration

by מור נגלי

Hello LAVA community, I'm working with LAVA for automated testing of embedded systems with BMC (Baseboard Management Controller) and I've encountered a limitation that I'd like to ask about. My Use Case I'm testing devices that support Redfish API for power management. I need to execute various Redfish commands (forceoff, poweron, forcerestart, etc.) from the dispatcher during test jobs. Currently, I define these as user_commands in my device dictionary: jinja2 {% set user_commands = { 'redfish_forceoff': { 'do': '/opt/lava-lab/shared/lab-scripts/dispatcher_command.sh 192.168.17.111 forceoff' }, 'redfish_poweron': { 'do': '/opt/lava-lab/shared/lab-scripts/dispatcher_command.sh 192.168.17.111 poweron' }, # ... more commands} %} Then in my job file: yaml - command: name: redfish_forceoff The Problem Every time I need to add a new Redfish action, I must modify the device dictionary (Jinja file) to add a new user_command entry. I'd like users to be able to specify the action dynamically from the job file without requiring device configuration changes. *What I want to achieve:* yaml # In job file - specify action dynamicallycontext: redfish_action: forceoff actions: - command: name: redfish_command # Generic command that uses context variable What I've Investigated 1. *Deploy actions* support variable substitution (e.g., {IMAGE}), but this is hardcoded for deploy actions only 2. *Job context variables* can override device dictionary values, but only at device configuration rendering time (job parsing), not at action execution time 3. *user_commands* are static strings with no runtime parameter substitution mechanism 4. *Test definitions* support parameters, but they run on the DUT, not the dispatcher I understand from the documentation that dispatcher commands are intentionally restricted for security reasons, which makes sense. My Current Workaround I'm using Jinja2 loops to auto-generate commands from a list: jinja2 {% set redfish_actions = ['auth_test', 'forceoff', 'poweron', 'forcerestart'] %}{% set user_commands = {} %}{% for action in redfish_actions %} {% set _ = user_commands.update({'rf_' + action: {'do': rf_script + ' ' + action}}) %}{% endfor %} This reduces the pain (just add one word to a list), but still requires device configuration edits. My Questions 1. *Is there any mechanism I've missed* that allows passing runtime parameters from job files to user_commands? 2. *Is this a known limitation* that others have encountered? 3. *Would it be feasible* to add support for parameter substitution in user_commands (similar to how deploy actions work with {IMAGE}), while maintaining security constraints? 4. *Are there alternative approaches* within LAVA's architecture that I should consider? Any guidance or suggestions would be greatly appreciated. If this is a feature that doesn't exist but would be useful, I'd be happy to discuss potential implementation approaches or contribute to a feature request. Thank you for your time and for maintaining LAVA! Best regards, Mor

3 days, 17 hours

2
3
0 0

Configure SSL certifcate

by Gregory Machin

Hi, can you point me to a document for the configuration of SSL certificate on the Lava Docker image ? Thank you. Greg

1 month

1
0
0 0

Call for LAVA maintainers

by Remi Duraffort

Hello LAVA users and developers, LAVA has been around and useful for a lot of people for a long time. For some years now, the list of LAVA maintainers as been shrinking down to 2 or even 1 developer (payed by Linaro). In order to review all the patches that the LAVA developers are pushing, we are looking for developers that want to become maintainer. The idea would be to delegate some part of the maintenance of LAVA sub-systems to some developers. If you are interested, please reach out to me in the lava-users ML, privately or in the new LAVA discord server at https://discord.gg/4dVatqyw Thanks for your attention. -- Rémi Duraffort Principal Tech Lead LAVA Tech Lead Automation Software Team Linaro

1 month

2
2
0 0

Connect a device through serial without PDU

by Ali Tariq

Hello, I am trying to connect a RISC-V VisionFive 2 device with the LAVA web server. I do not have a PDU at the moment and I do not know which one will be appropriate for this device setup. The LAVA master and worker are running on the same machine, so I have localhost set up. I added the following device dictionary for VisionFive 2. {% extends 'jh7100-visionfive.jinja2' %} {% set console_backend = 'serial' %} {% set serial_port = '/dev/ttyUSB0' %} {% set serial_baud_rate = 115200 %} The dictionary template, which is extended for VisionFive 2, is from the VisionFive 1 device type. As I won't be needing initramfs and other uboot env variables, I am using visionfive 1's dictionary template just for getting the setup to work. Following is the VisionFive 1 dictionary template. {% extends 'base-uboot.jinja2' %} {% set uboot_mkimage_arch = 'riscv' %} {% set console_device = console_device|default('ttyS0') %} {% set baud_rate = baud_rate|default(115200) %} {% set booti_kernel_addr = '0x84000000' %} {% set booti_dtb_addr = '0x88000000' %} {% set booti_ramdisk_addr = '0x88300000' %} {% set uboot_initrd_high = '0xffffffffffffffff' %} {% set uboot_fdt_high = '0xffffffffffffffff' %} {% set bootloader_prompt = 'VisionFive#' %} {% set uboot_tftp_commands = [ "tftpboot {KERNEL_ADDR} {KERNEL}", "tftpboot {RAMDISK_ADDR} {RAMDISK}", "tftpboot {DTB_ADDR} {DTB}"] -%} I was able to run a job on QEMU. But the VisionFive 2 is not detected by the LAVA web server. The physical connection is complete because I can use picocom on my computer to attach to the VisionFive 2 using the command sudo picocom -b 115200 /dev/ttyUSB0. I created several job definitions just for testing the setup but they all are just loading. The device health check is bad. Following is the simplest job that I am trying to run and see if it completes. job_name: simple-uboot-test device_type: visionfive2 priority: medium visibility: public timeouts: job: minutes: 5 actions: - deploy: timeout: minutes: 1 to: u-boot - boot: method: bootloader bootloader: u-boot connection: serial commands: - printenv But all the jobs are on infinite loading. This is my first time connecting a physical device with the LAVA web server, so I would appreciate it if someone could tell me what the issue is here. Regards, Ali

1 month

2
1
0 0

Maintainership help request

by Igor Ponomarev

Hello, I recently saw the "Call for LAVA maintainers" lava-announcement email and I am willing to help. I am pretty active in LAVA's gitlab. By the way, your discord link is expired. Nobody will be able to join until a new link is created. (you can make it without expiration timer) -- Igor Ponomarev Consultant Software Engineer Collabora Ltd. Unit 106 St Johns Innovation Centre, Cowley Road, Cambridge CB4 0WS, UK Registered in England & Wales, no. 5513718

1 month

2
1
0 0

email notifications not using correct url. issue with absolute_url?

by Michael Peddie

As the subject says, email notifications in lava aren't using the correct url when linking to a job. They use the example.com url, and even after changing the value under Sites in the admin settings, it still does. Looking at the notification template for emails, it uses the job.absolute_url, but I can't quite figure out how it constructs example.com, there shouldn't be anything anywhere that it pulls that from. Any help on this would be appreciated, even if it's just insight and not a solution.

1 month, 1 week

2
1
0 0

Is there any job using the ssh.jinja2 template as a learning example?

by 93831920＠qq.com

Is there any job using the ssh.jinja2 template as a learning example? Thank you

2 months, 1 week

2
1
0 0

Is there such a job test?

by 93831920＠qq.com

The device already has a Windows system and provides an SSH server, so there is no need for deployment. After LAVA establishes an SSH connection with the device, it can directly send test instructions, and exit the SSH connection after the test is completed?

3 months, 1 week

1
0
0 0

Not able to update default to qemu.jinja2 with updated configuration

by sweta.ghosh＠nagarro.com

Hello, I would like to replace -net legacy values with -netdev as netdevice in /etc/lava-server/dispatcher-config/device-types/qemu.jinja2 file but even after creating new jinja file my device is not able to identify it. I have tried restarting all the LAVA services but still not able to . Could you please share the how can I change or update existing jinja file. Thanks, Sweta

3 months, 1 week

2
1
0 0

how to discrib a ssh device?

by 93831920＠qq.com

Device is RAM, directly connected to the FT4232H through JTAG, FT4232T connected to the windows host through USB-MicroUsb, the windows host provides the telnet test command interface to the workers of the lava automated test environment through the telnet server, in this scenario, how to create the device type, device and job in lava, Can you use lava to automate testing of RAM?

3 months, 2 weeks

1
1
0 0

Running Lava in docker on Ubuntu.

by Gregory Machin

Hi, I have been asked by our hardware dev team to stand up a server that will run Lava. I was advised that Debian was the only supported OS. But as they will be running the official docker images is there any reason I couldn't deploy use Ubuntu 24.04 LTS ? Thank you Greg

4 months, 1 week

2
1
0 0

Using LAVA to transfer file via Ymodem from LAVA dispatcher(HOST machine) to DUT

by hina.hafeez＠10xengineers.ai

Hi LAVA users, I have set-up a LAVA server, dispatcher and connected my device (banana-pi) board serially via UART to host machine on which dispatcher is running. I have defined a job which uses deploy "tmpfs" to download .itb file from local file server. After that, under "boot" action, i am using u-boot method to boot the device. Output: 1. deploy images section runs successfully 2. uboot action starts 3. After connecting to device using telnet, device is reset(pdu-reboot). 4. Device enters in Y modem state waiting for image to get transferred. 5. After some time, "CCC CCCCCCC[ 99.943] spl: ymodem err - Timed out" Issue/Question: 6. At this stage, i want LAVA dispatcher(HOST machine) to send .itb file over UART using Ymodem to device. 7. But LAVA is unable to execute this command "sz -Y --ymodem /uboot-opensbi.itb < /dev/ttyUSB0 > /dev/ttyUSB0" to transfer opensbi.itb file to device. 8. I need to know what am i missing in writing job definition for this case. 9. Which device template should i use and how it should be extended. 10. is there any job definition syntax to fulfill my requirement? Regards,

4 months, 2 weeks

3
5
0 0

Open ID Connect (OIDC) authentication providers with LAVA

by sweta.ghosh＠nagarro.com

Hello Team, I am trying to use Azure AD for user authentication in LAVA. I followed the document https://validation.linaro.org/static/docs/v2/authentication.html#using-open… and created a yaml file with all details under /etc/lava-server/settings.d and restarted lava-server-gunicorn.service. Additionally , added ssl_header configuration to trust X-Forwarded-Proto header for detecting HTTPS. cat secure_proxy_ssl_header.yaml SECURE_PROXY_SSL_HEADER: - HTTP_X_FORWARDED_PROTO - https During sign in I am able get Microsoft login page but after successfully login I get "Bad Request (400)" . Could you please suggest if Lava can work with mozilla-django-oidc or if i am missing any configuration.

5 months

1
1
0 0

PXE boot qemu vm in LAVA

by sweta.ghosh＠nagarro.com

Hello Team, I have a custom scripts which spins up a pxe server in qemu and on the same network when we try to spin up VM it boot's from network and takes OS image from pxe to boot up. I want to do the same in LAVA and execute all my automated OS test cases over newly booted VM . I have a pipeline which setup's the bridge n/w (pxe network for server and host ) and i want to trigger LAVA job to spin up VM using same network. But to that to work I require image_arg in deploy action and which require mandatory download of image . I even tried to remove the image from img_arg but it wont boot from network . Also I am not able to find option to pass qemu arguments in case I use tftp or nfs . Could you please suggest how pxe boot qemu vm in LAVA

5 months, 1 week

1
0
0 0

timeouts are broken

by Milosz Wasilewski

Hi, After the 2025.02 release download timeouts are terribly broken. This should have never happened. LAVA is silently dividing the action timeout by number of repetitions: https://gitlab.com/lava/lava/-/wikis/releases/2025.02#timeout-and-retries https://gitlab.com/lava/lava/-/commit/07b13b5e4a2bb335dcfff155974ade81ef17b… This causes perfectly good jobs to fail. Example: https://lava.infra.foundries.io/scheduler/job/73929 If such change is required it should have been done with explicit parameter set in the job definition. In the current form I consider it a bug. The only way to work around this bug is to set the timeout to some ridiculous number (3 times as long as it is normally required). IMHO this should be reverted before the next release. Best Regards, Milosz

5 months, 3 weeks

2
8
0 0

Environment variables in interactive tests

by Michael Peddie

Hi, I am trying to make use of environment variables defined on my lava-server inside interactive tests. I have had some success with this, but I am not sure why one method works over the other. For example, I do something like the following: echo 'wget --user=some-name --password=${SECRET_PASSWORD}' > somefile.sh And that works just fine. But when I already have a script (say it exists on the DUT already) that just takes it as a parameter, it doesn't work. Example: some_script.sh -p ${SECRET_PASSWORD} -u some-name I have also tried accessing it directly in the script, similar to what somefile.sh would look like on the inside, but had no success with that. I have tried exporting the environment variable as another one to see if that would work, but to no avail. Is there something I'm missing? What I can I do to achieve this without hardcoding it anywhere in my scripts? Regards, Michael

5 months, 3 weeks

2
12
0 0

deploy to flasher with download URLs

by Milosz Wasilewski

Hi, I upgraded my LAVA instance to 2025.04 yesterday I skipped one release and the upgrade was from 2024.09. Most of the jobs I run in use deploy-to-flasher with custom script handling the board flashing. Up until 2024.09 (I don't know about 2025.02) "downloads" URLs resulted in all files stored in a flat directory. My flashing scripts made use of this "feature". In 2025.04 this has changed. This commit: https://gitlab.com/lava/lava/-/commit/4d9f0ebdae9ca53baf6633f4a35e716183bd2… makes the files stored in separate directories. It feels like a step in a right direction but it broke my flashing scripts. To make a quick fix I did this in the beginning of the script: DOWNLOAD_DIRS=$(find . -maxdepth 1 -mindepth 1 -type d -printf '%f ') for d in $DOWNLOAD_DIRS; do FILE_TO_COPY=$(find ./$d -maxdepth 1 -mindepth 1 -type f -printf '%f') ln "$PWD/$d/$FILE_TO_COPY" "$PWD/$FILE_TO_COPY" done This isn't great but it solves the issue for now. After a short discussion with Chase we came to conclusion that solution that looks "more correct" is to implement support for "uniquify" for "download" URLs. Chase sent a patch here: https://gitlab.com/lava/lava/-/merge_requests/2795 This should be available in the next release. I hope this message helps someone with similar issues :) Best Regards, Milosz

6 months

1
0
0 0

Download to worker mid-test

by Michael Peddie

Hi, I am trying to write a test to conditionally extract files from a zip file for use in some scripts, e.g. unzip the file, copy/move the files I need out, remove the zip file and directory created from unzipping it. However, the zip file size exceeds the disk space of the DUT. I have considered doing something like a MultiNode job, where the DUT is booted as well as a QEMU instance, and the file is downloaded and extracted on the QEMU and moved onto the DUT, but I would prefer not to do that if possible. Is there any way to download the zip file mid-test to the worker instead, extract the files I need, and move them onto the DUT? Ideally no modifications to the code of LAVA, but if that is necessary then I can figure out an interim solution and come back to it later. Regards, Michael

7 months, 1 week

2
1
0 0

LAVA LDAP mirror groups

by Michael Peddie

Hi, I am trying to get LDAP working in my LAVA instance, and have managed to get logging into it working. The issue comes when I try to use Django's MIRROR_GROUPS setting. I am already aware that it is not built into LAVA to be configurable, and I have already taken steps to add the relevant lines in lava_server/settings/common.py to make it work (initialize to None, values.get() in the update method, then run eval on the value in the LDAP if section), but it still won't work. All other required settings are clearly working just fine, and I can even set USER_FLAGS_BY_GROUP just fine, but I would prefer to mirror certain groups that users are members of and assign permissions to the groups. Do I need to pre-create the groups before logging in to LAVA or am I missing something else/doing something wrong? Original common.py source: https://gitlab.com/lava/lava/-/blob/master/lava_server/settings/common.py

8 months

2
8
0 0

Can't do Linux auto login from LAVA job

by Ramasamy, Rajeshkumar

Hi All, I am trying to setup LAVA and execute test cases. I am using Intel x86-64 machine. LAVA able to boot the Linux without any issues, but it is stuck just before login prompt. I am able to boot the machine with manual procedure. Pls find the log below. [[0;32m OK [0m] Started [0;1;39mBerkeley Internet Name Domain (DNS)[0m. [[0;32m OK [0m] Started [0;1;39mPostfix Mail Transport Agent[0m. [[0;32m OK [0m] Reached target [0;1;39mMulti-User System[0m. [[0;32m OK [0m] Reached target [0;1;39mHost and Network Name Lookups[0m. Starting [0;1;39mRecord Runlevel Change in UTMP[0m... [[0;32m OK [0m] Finished [0;1;39mRecord Runlevel Change in UTMP[0m. Linux Update 6 intel-x86-64 ttyUSB0 <Login prompt print expected here ("intel-x86-64 login:"), but not displayed on the log> bootloader-interrupt timed out after 280 seconds end: 2.3.2 bootloader-interrupt (duration 00:04:40) [common] case: bootloader-interrupt case_id: 185 definition: lava duration: 280.00 extra: ... level: 2.3.2 namespace: common result: fail health check yaml file snippet: - boot: timeout: minutes: 5 method: ipxe commands: nfs failure_retry: 3 prompts: - '[a-zA-Z0-9\-\_]+@[a-zA-Z0-9\-\_]+:.*?#' auto_login: login_prompt: "intel-x86-64 login:" username: root password_prompt: "password:" password: root login_commands: - cat /proc/cmdline - cd / - ls - test: failure_retry: 3 timeout: minutes: 5 definitions: - repository: metadata: format: Lava-Test Test Definition 1.0 name: smoke-tests-basic description: "Basic system test command for Linaro Ubuntu images" run: steps: - printenv Thanks Rajesh

9 months

1
0
0 0

Can't do Linux login from LAVA job

by Ramasamy, Rajeshkumar

Hi All, I am trying to setup LAVA and execute test cases. I am using Intel x86-64 machine. LAVA able to boot the Linux without any issues, but it is stuck just before login prompt. I am able to boot the machine with manual procedure. Pls find the log below. [[0;32m OK [0m] Started [0;1;39mBerkeley Internet Name Domain (DNS)[0m. [[0;32m OK [0m] Started [0;1;39mPostfix Mail Transport Agent[0m. [[0;32m OK [0m] Reached target [0;1;39mMulti-User System[0m. [[0;32m OK [0m] Reached target [0;1;39mHost and Network Name Lookups[0m. Starting [0;1;39mRecord Runlevel Change in UTMP[0m... [[0;32m OK [0m] Finished [0;1;39mRecord Runlevel Change in UTMP[0m. Linux Update 6 intel-x86-64 ttyUSB0 <Login prompt print expected here ("intel-x86-64 login:"), but not displayed on the log> bootloader-interrupt timed out after 280 seconds end: 2.3.2 bootloader-interrupt (duration 00:04:40) [common] case: bootloader-interrupt case_id: 185 definition: lava duration: 280.00 extra: ... level: 2.3.2 namespace: common result: fail health check yaml file snippet: - boot: timeout: minutes: 5 method: ipxe commands: nfs failure_retry: 3 prompts: - '[a-zA-Z0-9\-\_]+@[a-zA-Z0-9\-\_]+:.*?#' auto_login: login_prompt: "intel-x86-64 login:" username: root password_prompt: "password:" password: root login_commands: - cat /proc/cmdline - cd / - ls - test: failure_retry: 3 timeout: minutes: 5 definitions: - repository: metadata: format: Lava-Test Test Definition 1.0 name: smoke-tests-basic description: "Basic system test command for Linaro Ubuntu images" run: steps: - printenv Thanks Rajesh

9 months

1
0
0 0

How to start docker container with privileged mode which used for postprocessing artifacts

by Sai Ashrith Sathujoda

We have a requirement for some test cases that the artifacts need to be modified (negative scenario) in order to verify some scenarios. we are currently using postprocess under deploy action for this purpose. As of part postprocess, we need to mount partitions of disk image and make some changes based on the requirement. However, we are unable to mount the partitions using losetup command. docker: image: debian:bookworm steps: - apt-get update - loop_dev=$(losetup -P -f --show image.wic) - echo $loop_dev The job failed with below error, losetup -P -f --show image.wic losetup: cannot find an unused loop device It looks like the docker container needs root privileges. It works when we run locally with "privileged:true". Is there any way we can pass "privileged: true" option to docker image through LAVA job definitions? Any inputs regarding this matter are highly appreciated.

9 months, 2 weeks

1
0
0 0

How to verify secure boot negative test cases

by shivanand.kunijadar＠toshiba-tsip.com

Hello, I'm trying to verify secure boot on LAVA lab, I've verified the positive case of secure boot and confirmed the secure boot is enabled from the boot logs (kernel-start-message = "EFI stub: UEFI Secure Boot is enabled."). Next, I would like to verify the negative test case as well by tampering the secure boot image where I expect the image boot shall fail. Is there way in LAVA to make the job complete(e.g. waiting for specific message in boot logs like "Can't find image information. ") even though the image fails to boot on the DUT? Ref. https://lava.ciplatform.org/scheduler/job/1236667#L81

9 months, 3 weeks

2
1
0 0

concurrent job execution

by sweta.ghosh＠nagarro.com

Hi, I am not able to submit multiple jobs on my single node lava server, currently even 1 job is running the second job waits till the 1st job is completed. I have tried to increase the job limit of my worker to 3 via lava Admin UI but still I cannot run 2 jobs in parallel. Could you please let me know how can I increase the no. jobs.

10 months, 2 weeks

1
0
0 0

Securing LAVA Setup

by Michael Peddie

Hi, I have been looking into how to secure LAVA, as so far it's just been learning how it works. There's the obvious http/https stuff, and in the documentation mentions firewalls for remote workers (which we'll be using anyways). We already have things in place for physical security and plans on how to improve, so no worries there. Is there anything not covered in the documentation that should be considered and how could we go about putting that into place? Best regards, Michael

11 months

1
0
0 0

What does the gdb boot method do?

by 叶轩杨

I saw a file 'lava-dispatcher/actions/boot/gdb.py' in the LAVA source code, which seems to define a gdb method in action->boot. But I didn't find a description of it in the LAVA documentation. The document I checked is `https://docs.lavasoftware.org/lava/actions-boot.html?highlight=boot#boot-action-reference`. Reading the source code may lead to some misunderstandings and is time-consuming. I hope someone who knows its function can explain to me what this new method does. Thank you very much!

11 months, 3 weeks

1
0
0 0

How to add custom data in the metadata field of notify callback json?

by 叶轩杨

I want to add some key:value pair in metadata of json that POST by notification of job, but I can not figure it out. I had tried add metadata in job defination and test defination, but the back json did not have my data in metadata field, it was just a []. Then I read the lava source code, I find the metadata data are produced by method `get_metadata_dict()` in class `TestJob`. This method is called at method `create_job_data` in class `TestJob`, it creates the json data which will post to callback url. `get_metadata_dict()` check whether there is an attribute `testdata`, but I can not find the place where this attribute‘s assignment. Could someone give me a way to do this? Thanks you very much! 🙏

12 months

2
4
2 0

Auto login fails after adding banner message before login prompt

by sweta.ghosh＠nagarro.com

Hello, We are testing our custom debian image and have added banner messages before login and after login and after the implementation LAVA job is failing on auto-login. Please find example banner messages: |_ _/ | | | | _| | |_____\ Info: Example.....: ABC Example......: ABC IP: etho.......:123 Info: 1. some info Some Info: 1. ### Some more info: #### login : It looks like due to a huge banner message before the login prompt it is getting confused and due to this, it gets stuck at auto login and timeouts. Please suggest if we can ignore this in LAVA to login and execute the test case Please find the job details: device_type: qemu job_name: banner timeouts: job: minutes: 30 action: minutes: 25 connection: minutes: 5 priority: medium visibility: public context: arch: amd64 lava_test_results_dir: "/home/lava-%s" actions: - deploy: timeout: minutes: 15 to: tmpfs images: rootfs: url: file:///home/image image_arg: XXXX - boot: timeout: minutes: 15 method: qemu media: tmpfs prompts: - 'login:' auto_login: login_prompt: "login:" username: abc password_prompt: "Password:" password: "abc" - test: timeout: minutes: 15 definitions: - repository: git(a)test.git from: git path: test.yaml name: basic Thanks, Sweta

1 year

4
23
0 0

Error Msg : "Connection aborted." when test monitor is used

by andy.sabathier＠nxp.com

Hello, I'm currently using version 2024.09, and when running the test monitor, I encounter the following error at the top of my job: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')). I don't notice anything unusual in the test logs. Could this be a bug with the test monitor? Best Regards, Andy

1 year

3
4
0 0

job timeouts

by Milosz Wasilewski

Hi, I think there is something terribly wrong with job timeouts in 2024.05 release. Total job timeout seems to be ignored. This was working fine on 2024.01 release. I'll try to debug, but IMHO this is a pretty critical bug that should be addressed immediately. I'll report it in gitlab as well. Best Regards, Milosz

1 year, 1 month

2
2
0 0

Copy local package for testing

by sweta.ghosh＠nagarro.com

Hello, I am spinning up debian image using Qemu and want to copy a custom package from my lava server to debian vm for testing. I could see that support for tar is planned https://docs.lavasoftware.org/lava/actions-test.html#inline-test-definition… . But is there any way I can include my custom package in tarbar created at Deploy Action to ship to my VM easily?

1 year, 1 month

4
4
0 0

Execute script while running ssh device type

by sweta.ghosh＠nagarro.com

Hello, I have multiple machines over which I test using LAVA job using primary ssh connection. As my devices OS gets installed every now and then I have to manually update the ssh public key before executing the job so that LAVA can connect to DUT over passwordless SSH. Is there a way I can include a script in device dictionary to copy the keys automatically before requesting a ssh connection? As I understand password option is not there. Kindly suggest.

1 year, 1 month

4
3
0 0

How to execute preboot command in qemu

by sweta.ghosh＠nagarro.com

Hello, I want to spin up an image with tpm enabled using qemu, and I need to execute swtpm socket command before image boot-ups via qemu. Please suggest if I could execute this command or script in qemu.jinja2 file or any other file. Thanks, Sweta

1 year, 1 month

2
1
0 0

Not able to set variable

by sweta.ghosh＠nagarro.com

Hi, I want clone a gitlab repo with id and token during job execution. To hide the token I am passing the value in my job. How can I export the value of GIT_TOKEN in environment file of LAVA? So that whenever job is executed it will read the value from its environment file. I tried to use export and tried to add "GIT_TOKEN=abcd" in env.yaml file. But I guess its not correct format. I couldn't find what kind of values we can add in env.yaml in documents. I also added GIT_TOKEN value in /etc/profile of the LAVA server still the job didn't pick it. Could you please suggest how can we clone the repo in LAVA job without exposing its password. Also I understand that ssh key of root will work but I want to avoid using ssh key of root. Kindly suggest - test: timeout: minutes: 15 definitions: - repository: https://gitid:$GIT_TOKEN@gitlab.com/lava-tests.git from: git path: tests/cisscan/cis.yaml branch: pipeline name: cis-benchmark

1 year, 2 months

3
6
0 0

Confusion about job IDs reported by lavacli for multi node tests

by Florian Bezdeka

Hi all, I run into some unexpected behavior when it comes to LAVA job IDs reported by the "lavacli jobs sumit" command for multi node tests. What I get is a list of newline separated job IDs like: lavacli jobs submit <my-job-description> 132.0 132.1 To my understanding this tries to express that 132.1 is a sub job of 132. OK, but this is a problem if I now try to download the junit report for the second job (=132.1). It seems 132.1 is not a valid job ID (API wise), it has to be translated to 133 to work properly. This is still doable, but I'm unsure if I can relay on 133 really being the 132.1 job in case there are several job submissions at the same time. Is that a bug of lavacli? What is the reasoning for returning a sub job ID which seems to be invisible to the rest of the LAVA infrastructure / API? I'm currently running lavacli 1.2-1, as shipped by Debian 12. Any input welcome... Best regards, Florian

1 year, 2 months

2
2
0 0

bootup a server and deploy debian bookworm over serial connection

by sweta.ghosh＠nagarro.com

Hello, I am new to LAVA, I want to boot up my hardware(NET-I System)which is a x86_64 and install debian bookworm image over it and execute test cases. My h/w is connected to my LAVA server via a serial device.Below is the high level job that I am planning to use. But where should I specify the serial device information and other information?Could please share any example job and device .jija2 that I can refer regarding what all information needs to be provided for LAVA server to properly connect to H/W and also is u-boot correct for my usecase? or if there is any other suggestions : device_type: your_device_type job_name: debian_boot_test timeouts: job: minutes: 15 action: minutes: 5 connection: minutes: 2 actions: - deploy: timeout: minutes: 10 to: tftp images: rootfs: url: <Debian Image> - boot: method: u-boot commands: your_device_boot_commands prompts: - 'login:' parameters: boot_options: root=/dev/ram0 - test: timeout: minutes: 5 definitions: - repository: metadata: format: Lava-Test Test Definition 1.0 name: debian-check-test description: "Test to verify Debian OS" os: - debian run: steps: - echo 'Checking if OS is Debian' - 'if [ "$(lsb_release -is)" = "Debian" ]; then echo "OS is Debian"; else echo "OS is not Debian"; fi' from: inline name: debian-check-test path: inline/debian-check-test.yaml

1 year, 3 months

3
2
0 0

Change path of lava_test_results_dir and lava_test_dir

by sweta.ghosh＠nagarro.com

Hello, I would like to change path to /mnt from below: "lava_test_dir": "/lava-%s", "lava_test_results_dir": "/lava-%s", I am using an immutable image and due to which don't have permission to write, so cannot create directory eg /lava-34 due to which cannot execute test. We have partition /mnt which we can mount and run the test cases over our os. But for that to work I need to change path. I tried changing it in /usr/lib/python3/dist-packages/lava_dispatcher/deployment_data.py , but jobs fails once I change it. I have only changed the debian section as the image we are using is a debian image. Could you please help me if there is any option to make these changes while executing job or any other file for devices. Thanks, Sweta

1 year, 4 months

2
6
0 0

RecursionError: maximum recursion depth exceeded on lava-scheduler service

by sweta.ghosh＠nagarro.com

Hi , I am trying to setup LAVA on Single Master Worker node. When i am trying to submit job it is running but no device is getting selected nor I am getting any output on UI. When I checked the lava-scheduler it is failing with below error: root@debian:/etc/apache2/sites-available# systemctl status lava-scheduler × lava-scheduler.service - LAVA scheduler Loaded: loaded (/lib/systemd/system/lava-scheduler.service; enabled; preset: enabled) Active: failed (Result: start-limit-hit) since Tue 2024-07-02 18:52:36 IST; 7min ago Duration: 1.472s Process: 10345 ExecStart=/usr/bin/lava-server manage lava-scheduler --level $LOGLEVEL --log-file $LOGFILE $EVENT_URL $IPV6 (code=exited, status=0/SUCCES> Main PID: 10345 (code=exited, status=0/SUCCESS) CPU: 1.431s Jul 02 18:52:35 debian systemd[1]: lava-scheduler.service: Deactivated successfully. Jul 02 18:52:35 debian systemd[1]: lava-scheduler.service: Consumed 1.431s CPU time. Jul 02 18:52:36 debian systemd[1]: lava-scheduler.service: Scheduled restart job, restart counter is at 5. Jul 02 18:52:36 debian systemd[1]: Stopped lava-scheduler.service - LAVA scheduler. Jul 02 18:52:36 debian systemd[1]: lava-scheduler.service: Consumed 1.431s CPU time. Jul 02 18:52:36 debian systemd[1]: lava-scheduler.service: Start request repeated too quickly. Jul 02 18:52:36 debian systemd[1]: lava-scheduler.service: Failed with result 'start-limit-hit'. Jul 02 18:52:36 debian systemd[1]: Failed to start lava-scheduler.service - LAVA scheduler. root@debian:/etc/apache2/sites-available# tail -f /var/log/lava-server/lava-scheduler.log {% extends 'qemu.jinja2' %} ^^^^^^^^^^^^^^^^^^^^^^^^^ [Previous line repeated 977 more times] File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1494, in is_up_to_date return self._uptodate() ^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/jinja2/loaders.py", line 212, in uptodate return os.path.getmtime(filename) == mtime ^^^^^^^^^^^^^^^^^^^^^^^^^^ RecursionError: maximum recursion depth exceeded Also when I am accessing /admin/lava_scheduler_app/device/ I am getting " 500 Internal Server Error maximum recursion depth exceeded" I have registered my worker node as debian,hostname as debian and device type is qemu. Could you please what I am missing.

1 year, 4 months

4
4
1 0

lava test doesn't wait for command to finish running

by Michael Peddie

Just as the subject says. I am using lava-test-case to confirm whether this particular command is done running and is successful or not, because the test has previously always ended prematurely and not done what I needed it to do. When I run the command manually outside of a lava test job it works just fine, no errors and doesn't end early. It does take a while to complete, but I don't think that's the issue as I use wget to download some large files and those have taken several minutes longer than this one command is supposed to. How do I get lava to wait for this command to end? Or to change how it checks for command failure? I get "Received signal: <ENDTC>" almost immediately after running the command, and subsequently the result=fail signal. I really don't know why it won't work, so if there is a change I can make to the files or a config somewhere I would like to know. Best regards, Michael

1 year, 4 months

2
15
0 0

MultiNode to run script->reboot->test

by Michael Peddie

Hi, I was wondering if it is possible to write a multinode test to boot a device, run a script to do some stuff then have it reboot one way or another, then run tests. There is intention for pipeline integration down the line, so if it isn't possible to do that with multinode I have considered the idea of setting it up so one regular test runs and runs the script, turns DUT off, then another test runs to boot and actually test the device. This (obviously) has its drawbacks, but it's something of a backup. If I understand correctly, where certain actions are placed in a multinode test actually matters, so it (in my mind) is possible to have a boot->test to boot a DUT then use test commands to run a script, and then after that another boot->test to boot the device or just login again and then actually run tests. So the final sort of action set up would be boot->test->boot->test, where the first two are for running the script and rebooting, and the latter two are actually testing. If there are other ideas that are better or equally viable, then Im open to those too, but multinode was the first that came to mind cos of secondary connections and the like. Best regards, Michael

1 year, 5 months

1
1
0 0

lavasoftware.org is unaccessible from gandi.net Paris SD6 datacenter

by Jacopo Mondi

Hello I am experiencing troubles connecting to apt.lavasoftware.org from a VPS located in the "Paris France SD6" datacenter hosted by gandi.net, The problematic VPS IP address is 46.226.105.174. Testing from another VPS, still hosted at gandi.net but located in a different datacenter (Luxembourg) presents no issues. Both VPS runs Debian 12 Bookworm. The symptoms I'm experiencing seems to suggest the TLS handshake gets truncated as suggested by the 'gnutls-cli' output below reported ------------------------------------------------------------------------------- # gnutls-cli -d9999 -p 443 apt.lavasoftware.org ... |<5>| REC[0x56498f5d7740]: Sent Packet[1] Handshake(22) in epoch 0 and length: 402 |<11>| HWRITE: wrote 1 bytes, 0 bytes left. |<11>| WRITE FLUSH: 402 bytes in buffer. |<11>| WRITE: wrote 402 bytes, 0 bytes left. |<3>| ASSERT: ../../lib/buffers.c[get_last_packet]:1185 |<10>| READ: Got 0 bytes from 0x3 |<10>| READ: read 0 bytes from 0x3 |<3>| ASSERT: ../../lib/buffers.c[_gnutls_io_read_buffered]:593 |<3>| ASSERT: ../../lib/record.c[recv_headers]:1195 |<3>| ASSERT: ../../lib/record.c[_gnutls_recv_in_buffers]:1321 |<3>| ASSERT: ../../lib/buffers.c[_gnutls_handshake_io_recv_int]:1467 |<3>| ASSERT: ../../lib/handshake.c[_gnutls_recv_handshake]:1600 |<3>| ASSERT: ../../lib/handshake.c[handshake_client]:3075 |<13>| BUF[HSK]: Emptied buffer *** Fatal error: The TLS connection was non-properly terminated. ------------------------------------------------------------------------------- Could you kindly check if my VPS is maybe in a range of IP addresses blacklisted by the service provider that hosts lavasoftware.org ? Thanks j

1 year, 5 months

1
0
0 0

Boot with serial test with ssh

by Michael Peddie

Hi all, I have been having issues doing solely boot and test with serial (specific issue(s) not diagnosed, but presumably something to do with telnet), but that's not the discussion here. I want to be try booting the device with the serial method then deploying/running tests using ssh, since it seems like ssh can't power on and power off the DUT. If that is wrong then please let me know so I can try that. I have two proposed ideas: 1) boot with serial -> deploy test (however works) -> run test with serial -> power off 2) boot with serial -> deploy test (however works) -> run test with ssh -> power off I started with a multinode job (I still don't quite understand multinode yet) and this is my current job definition, if you could please help point out issues with it as it doesn't work: #multinode job for controller deployment job_name: controller deploy test protocols: lava-multinode: roles: host: context: lava_test_results_dir: /tmp/lava-%s device_type: controller timeout: minutes: 10 count: 1 guest: context: lava_test_results_dir: /tmp/lava-%s request: lava-start count: 3 expect_role: host timeout: minutes: 10 connection: ssh host_role: host timeouts: job: minutes: 15 action: minutes: 5 connection: minutes: 2 priority: medium visibility: public actions: - deploy: role: - host timeout: minutes: 10 to: tftp authorize: ssh kernel: url: file:///kernel.img type: uimage ramdisk: url: file:///ramdisk.gz compression: gz dtb: url: file:///u-boot.dtb - deploy: role: - guest timeout: minutes: 10 to: ssh protocols: lava-multinode: - action: prepare-scp-overlay request: lava-wait messageID: ipv4 message: ipaddr: $ipaddr - boot: role: - host timeout: minutes: 5 method: minimal prompts: ["# "] auto_login: login_prompt: "login: " username: root - boot: role: - guest timeout: minutes: 5 prompts: ["# $"] parameters: hostID: ipv4 # messageID host_key: ipaddr # message key method: ssh connection: ssh - test: role: - host timeout: minutes: 15 definitions: - repository: metadata: format: Lava-Test Test Definition 1.0 name: install-ssh description: "install step" scope: - functional run: steps: # messageID matches, message_key as the key. - lava-send ipv4 ipaddr=$(lava-echo-ipv4 eth0) - lava-send lava_start - lava-sync clients from: inline name: ssh-inline path: inline/ssh-install.yaml - test: role: - guest timeout: minutes: 5 definitions: - repository: https://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@github.com/MichaelPed/lava… from: git path: smoke-tests/smoke.yaml name: smoke-tests # run the inline last as the host is waiting for this final sync. - repository: metadata: format: Lava-Test Test Definition 1.0 name: client-ssh description: "client complete" scope: - functional run: steps: - df -h - free - lava-sync clients from: inline name: ssh-client path: inline/ssh-client.yaml And this is my device dictionary, if you could please let me know what needs changing: {% extends 'controller.jinja2' %} {% set power_off_command = 'python /power.py 12 off' %} {% set soft_reboot_command = 'reboot'%} {% set hard_reset_command = 'python /power.py 12 reboot' %} {% set power_on_command = 'python /power.py 12 on' %} {% set connection_list = ['uart0'] %} {% set connection_commands = {'uart0': 'telnet 10.60.2.209 7001'} %} {% set connection_tags = {'uart0': ['primary', 'telnet']} %} {% set bootm_kernel_addr = '/kernel.img' %} If you guys have any questions please ask away! Also, if there is somehow something other than a multinode job that will do what I want then please let me know! Best regards, Michael

1 year, 6 months

1
0
0 0

How to run QEMU in DUT and gdb the QEMU in another console

by 叶轩杨

Hello! I want to run QEMU in DUT and gdb the QEMU in another console to simulate something like a bit flip in memory. What should I do? I just need the new console to run some gdb scripts, like `rust-gdb vmlinux -ex 'target remote /tmp/gdb_socket' -ex 'set *0x40001000 ^= (1<<5)' -ex `c` `, so the new console can not be interactive, live to run some command is enough. Thanks!

1 year, 6 months

1
0
0 0

Run pytest suite on DUT using LAVA

by Singamsetty Nagendra

Hello everyone, I would like to integrate & run a bare metal custom tests suite(pytest) on lava. Currently I am executing manually from Host Machine(where pytest is configured and pytest suite will execute) and these commands are sent to DUT and results are collected on Host Machine. I am familiar with lava-interactive method but it looks like it will not suit this requirement(Run pytest on Host and collect logs from DUT). Does lava suits for this requirement? if someone could give me advice on this that would be great support. Regards Nagendra S

1 year, 7 months

2
4
0 0

How to loop test reboot

by irreallich＠126.com

Hi, I hope to be able to test whether the system can boot normally after multiple restarts (10,000 times) Because every boot requires the auto_login of the boot action, it seems that a loop needs to be implemented in yaml, and this loop needs to include the boot action. Are there any relevant examples that you can refer to? There are roughly two types of test logic 1: while (a<10000) { -boot auto_login - test basic io test software reboot in target board a=a+1 } 2: while (a<10000) { -boot auto_login - test basic io test a=a+1 hardware reboot in worker } Thanks

1 year, 7 months

3
8
0 0

transfer_overlay wget wont work on DUT

by Michael Peddie

Hi, I am trying to use transfer_overlay as the fs on my DUT is read-only, but the /data/ and /tmp/ directories are writeable to some extent. I have a working method to use wget, and it works when I use it manually to download the overlay tarball over http, both on my DUT and on the worker device. However, the LAVA test itself always returns the error 'Network Unreachable' What are the possible reasons for this? Best regards, Michael

1 year, 7 months

3
8
0 0

Request to change status of failure retry option:

by pawanKumar

1 year, 7 months

1
0
0 0

Lava Test/Dev Environment Setup Best Practices

by Swapnil Tilaye

Hi Team Currently for the developing and running of the lava tests, we always run on the Lava master machine through lavacli. But we would like to skip the part of the running the tests from the lava master machine while developing, testing and triaging the tests written in the lava. We only want to run tests which are tested correctly to be run on through ci/cd on the lava master and for development, triaging we want to only use the lava-worker and board. We only want to use the lava master only through CI/CD. [cid:86e0132a-a975-4d04-9f31-9a89237339d8] Is there any way to run tests directly from the lava worker instead of going through lava master. I tried "lava-test-shell" and "lava-dispatcher" but it didnt work for me. Can I use the lava api for developing and triaging the test scripts? Can you please recommend the best practice or setup for developing, testing and triaging the tests written in the lava? so that anyone can run, develop or modify tests quickly and make sure its successful and then only it can run through the lava master machine. Regards, Swapnil Tilaye

1 year, 7 months

3
2
0 0

Possible regression: LAVA breaks initrds/ramsisks with attached microcode

by Florian Bezdeka

Hi all, over the last days I was trying to bring up a setup where we simply try to boot up a kernel and initrd combination that is provided by Debian "as is". Sounds trivial at first but I ran into a problem with the initrd: The initrd contains some microcode, shipped as prepended uncompressed archive. The microcode is added because the intel-microcode package was installed during image generation. When such an initrd enters the LAVA machinery, where the initrd is first unpacked and later re-packed the file gets corrupted in a way that the kernel is unable to unpack/use it. The end result was that NFS boot did not work as the mounting of the rootfs did not take place. Interestingly there is no error message, not within the unpack/repack step in LAVA and the kernel does not complain later during the boot sequence either. To disable the LAVA unpack/repack sequence I had to modify my deploy action. Setting install_overlay and install_modules [2] to false seems to qualify as workaround: - deploy: timeout: minutes: 15 to: tftp kernel: url: <kernel-url> ramdisk: url: <initrd-url> install_overlay: false install_modules: false Looking at the initramfs-tools implementation [1] I think that something similar is missing in LAVA. Does that make sense? Ideas / comments? Best regards, Florian [1] https://gitlab.com/lava/lava/-/blob/master/lava_dispatcher/actions/deploy/a… [2] https://salsa.debian.org/kernel-team/initramfs-tools/-/blob/master/unmkinit… -- Siemens AG, Technology Linux Expert Center

1 year, 7 months

2
1
0 0

"Proxy Error" message while loading more size log from LAVA jobs

by koti

Hi, We are seeing below the "Proxy Error" message while loading log size logs from the LAVA job . Is there any solution to load a long size log file from a LAVA job ? [image: image.png]

1 year, 7 months

2
2
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

lava-users