lava-users

lava-users@lists.lavasoftware.org

2 participants
918 discussions

by Zoran S

Hello to LAVA admins, You have fatal error in Lava V2. I submitted job, which hangs in submitted state indefinitely. I also see the non-existent job 92 (which is deleted) running there?! http://localhost:8080/scheduler/device/bbb01 254 2018-02-21 12:05 Reserved → Running — Job 92 running 253 2018-02-21 12:04 Idle → Reserved — Reserved for job 92 252 2018-02-21 12:03 Running → Idle — Job 91 cancelled 251 2018-02-21 12:03 Reserved → Running — Job 91 running 250 2018-02-21 12:03 Idle → Reserved — Reserved for job 91 249 2018-02-21 11:33 Running → Idle — Job 89 has ended. Setting job status Incomplete But when I look to this local pointer: http://localhost:8080/admin/lava_scheduler_app/testjob/ 102 Submittednobodylessbeaglebone-black -Feb. 21, 2018, 1:07 p.m.-- 76 Incompletenobodylessbeaglebone-black bbb01 (Running, health Unknown)Feb. 20, 2018, 9:50 a.m.Feb. 20, 2018, 9:50 a.m.Feb. 20, 2018, 9:55 a.m. 49 Completenobodylessqemu qemu01 (Idle, health Unknown)Feb. 16, 2018, 10:44 a.m.Feb. 16, 2018, 10:44 a.m.Feb. 16, 2018, 10:50 a.m. 43 Completenobodylessqemu qemu01 (Idle, health Unknown)Feb. 16, 2018, 8:42 a.m.Feb. 16, 2018, 8:43 a.m.Feb. 16, 2018, 8:49 a.m. How to force submitted job to be active job (from both CLI and GUI)? How to delete nonexistent jobs (from both CLI and GUI)? Thank you, Zoran

7 years

Lava dispatcher long test runs executions in V.L.O.

by Anibal Limon

Hi, I created a Piglit test definition [1] before i try to test it out into the V.L.O. i used test-runner to make a run locally (db410c) without lava-dispatcher, the run took ~2 hours 5 minutes [2] (wo download/flash) after that i tried to execute into the V.L.O. and it consumes 8 hours of execution and then finalize with a timeout, the test didn't reach the end and was able to execute only 14486 tests of 39249. So i download the definition and executed the lava-run command in my desktop and the full test execution was in ~2 hours 38 minutes that makes sense because needs to download and flash the boot and rootfs images. [4] I introduced a time call for measure the piglit time execution in test-runner and lava-dispatcher and the exec times are the same ~96 minutes in both executions, search for ' Thank you for running Piglit' in the logs [2][5]. I noticed when i ran lava-run into my desktop the CPU usage increases to 100% when reaches the step of actually running the test [6], i made some debugging and discover that the manner to get the results and notifications from serial console is using pexpect and certain regexes, i attached the gdb to the python process and found that the regexes executes over the whole serial buffer in very short intervals [7] That could be the reason of the delay in the test execution since the Piglit command takes the same time to run with test-runner and lava-dispatcher. Best regards, Anibal [1] https://git.linaro.org/qa/test-definitions.git/tree/automated/linux/piglit/… [2] http://qcom-hackbox.linaro.org:8090/stdout_testrunner.log.gz [3] https://lkft.validation.linaro.org/scheduler/job/101274 [4] http://qcom-hackbox.linaro.org:8090/stdout_dispatcher.log.gz [5] http://qcom-hackbox.linaro.org:8090/stdout_dispatcher_time.log.gz [6] http://qcom-hackbox.linaro.org:8090/lava_run_top.log [7] http://qcom-hackbox.linaro.org:8090/lava_run_gdb.log [8] https://git.linaro.org/lava/lava-dispatcher.git/tree/lava_dispatcher/action…

7 years

Problems with jinja2 scripts in /etc/lava-server/dispatcher-config/devices

by Zoran S

Hello, I am bringing now the target Beaglebone Black, as DUT in Lava. Here is my BBB01.jinja2 device definition: {% extends 'beaglebone-black.jinja2' %} {% set connection_command = '/vagrant/scripts/connectBBB.sh localhost 8020 root ""' %} {% set power_on_command = '/usr/bin/egctl egenie on left left left' %} {% set power_off_command = '/usr/bin/egctl egenie off left left left' %} {% set hard_reset_command = '/usr/bin/egctl egenie on left left left' %} Where the line: "vagrant/scripts/connectBBB.sh localhost 8020 root ""' represents as $@ the following: 127.0.0.1 8020 root and "" ("" represents empty password, actually <cr>). It seems that this part works correctly: end: 2.4.1 reset-device (duration 00:00:00) start: 2.4.2 u-boot-interrupt (timeout 00:03:00) Changing prompt to 'Press SPACE to abort autoboot' u-boot-interrupt: Wait for prompt Press SPACE to abort autoboot (timeout 00:03:00) spawn telnet localhost 8020 Trying ::1... Connected to localhost. Escape character is '^]'. ser2net port 8020 device /dev/ttyUSB0 [115200 N81] (Debian GNU/Linux) end: 2.4.2 u-boot-interrupt (duration 00:00:02) start: 2.4.3 bootloader-commands (timeout 00:02:58) Changing prompt to start interaction: U-Boot bootloader-commands: Wait for prompt U-Boot (timeout 00:03:00) switch to partitions #0, OK mmc0 is current device SD/MMC found on device 0 ** Unable to read file boot.scr ** ** Unable to read file uEnv.txt ** switch to partitions #0, OK mmc0 is current device Scanning mmc 0:1... switch to partitions #0, OK mmc0 is current device SD/MMC found on device 0 switch to partitions #0, OK mmc0 is current device SD/MMC found on device 0 ** Unable to read file boot.scr ** ** Unable to read file uEnv.txt ** 4161168 bytes read in 293 ms (13.5 MiB/s) 35016 bytes read in 23 ms (1.5 MiB/s) ## Flattened Device Tree blob at 88000000 Booting using the fdt blob at 0x88000000 Loading Device Tree to 8fff4000, end 8ffff8c7 ... OK Starting kernel ... [ 0.000000] Booting Linux on physical CPU 0x0 [ 0.000000] Linux version 4.13.8-jumpnow (oe-user@oe-host) (gcc version 7.2.0 (GCC)) #1 Fri Jan 12 13:27:03 CET 2018 [ 0.000000] CPU: ARMv7 Processor [413fc082] revision 2 (ARMv7), cr=10c5387d [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache [ 0.000000] OF: fdt: Machine model: TI AM335x BeagleBone Black [ 0.000000] Memory policy: Data cache writeback [ 0.000000] cma: Reserved 16 MiB at 0x9f000000 [ 0.000000] CPU: All CPU(s) started in SVC mode. But when it comes to login to kernel login, it fails! Starting OpenBSD Secure Shell server: sshd key_load_public: invalid format Could not load host key: /etc/ssh/ssh_host_rsa_key key_load_public: invalid format Could not load host key: /etc/ssh/ssh_host_dsa_key key_load_public: invalid format Could not load host key: /etc/ssh/ssh_host_ecdsa_key key_load_public: invalid format Could not load host key: /etc/ssh/ssh_host_ed25519_key sshd: no hostkeys available -- exiting. Starting rpcbind daemon...done. starting statd: done exportfs: can't open /etc/exports for reading modprobe: can't change directory to '/lib/modules': No such file or directory NFS daemon support not enabled in kernel Starting syslogd/klogd: done Poky (Yocto Project Reference Distro) 2.3.2 beaglebone /dev/ttyO0 beaglebone login: [ 170.488770] random: crng init done bootloader-commands timed out after 178 seconds end: 2.4.3 bootloader-commands (duration 00:02:58) uboot-retry failed: 1 of 1 attempts. 'bootloader-commands timed out after 178 seconds' bootloader-commands timed out after 178 seconds end: 2.4 uboot-retry (duration 00:03:00) uboot-action failed: 1 of 1 attempts. 'bootloader-commands timed out after 178 seconds' bootloader-commands timed out after 178 seconds end: 2 uboot-action (duration 00:03:00) Cleaning after the job Cleaning up download directory: /var/lib/lava/dispatcher/tmp/84/tftp-deploy-a0ZzPD/ramdisk Cleaning up download directory: /var/lib/lava/dispatcher/tmp/84/tftp-deploy-a0ZzPD/kernel Cleaning up download directory: /var/lib/lava/dispatcher/tmp/84/tftp-deploy-a0ZzPD/dtb Cleaning up download directory: /var/lib/lava/dispatcher/tmp/84/tftp-deploy-a0ZzPD/modules start: 4.1 power-off (timeout 00:00:05) nice /usr/bin/egctl egenie off left left left command output socket 1 - off socket 2 - off socket 3 - off socket 4 - off It seems that empty character as "" in jinja2 is noit well represented (or maybe I need to have somehow <CR>)?! Any advise too this problem? Thank you, Zoran zo

7 years

Re: [Lava-users] Integrating a custom bootloader

by Tim Jaacks

Hi Neil, thanks for your reply. >Why do you need an in-house bootloader? (Hint: Just because you can / have >is the worst possible reason.) Honestly, just find some other way to do it >- it will likely be much quicker. We have developed our own bootloader for our hardware, for various reasons, and want our OS to be tested in combination with it. Our devices do not support any other bootloader except our own anyway, so there is no other possibility. This is what we have, and we need to automate testing somehow. So I am trying to figure out whether LAVA is suitable for this task. >Supporting a new bootloader is a very complex and time-consuming process >for any test framework. It typically takes several months. Once done, there >is an ongoing maintenance burden as the code for the supported bootloaders >continues to advance. This is not a problem generally. We are planning long-term and want a future-proof solution. >You need a new Strategy Class, as per the documentation. You must not >overload the "method: u-boot" support for anything except U-Boot. You must >create a new boot method. Why? What is wrong with using the u-boot method for a bootloader which works similarly? As far as I can see right now, I would only remove stuff from the u-boot method and set a bunch of already existing variables to new values. I do not see the need for a completely new boot method. Is there any hard reason for this, except that it would be neater? Mit freundlichen Grüßen / Best regards Tim Jaacks DEVELOPMENT ENGINEER Garz & Fricke GmbH Tempowerkring 2 21079 Hamburg Direct: +49 40 791 899 - 55 Fax: +49 40 791899 - 39 tim.jaacks(a)garz-fricke.com www.garz-fricke.com SOLUTIONS THAT COMPLETE! Sitz der Gesellschaft: D-21079 Hamburg Registergericht: Amtsgericht Hamburg, HRB 60514 Geschäftsführer: Matthias Fricke, Manfred Garz

7 years

Integrating a custom bootloader

by Tim Jaacks

Hello, I am evaluating LAVA as a candidate for software release testing on our devices. I have already set up a LAVA instance with a Beaglebone Black board running regular health checks. Our own devices do not use a standard bootloader, but a custom one developed by our team. It has the following characteristics: * Always boots to a shell (no interruption needed) * OS is installed on internal eMMC by calling an installation script * Installation script and OS images are deployed to the device via TFTP by calling a bootloader command sequence * Installed OS is booted via a bootloader command sequence So LAVA has already implemented TFTP as a deployment method, which I can re-use. As far as I understand, this deployment method copies the files to the machine's TFTP server directory, so that they are available to the device for download. However, the actual download (from TFTP server to device) is implemented as a command sequence in the boot method. Is that correct? At least, this is what I see in the base-uboot.jinja2 device type template. I thought that it should be possible to re-use the "u-boot" bootloader class for our own bootloader, since the basic concepts (calling bootloader commands for TFTP download, calling bootloader commands for OS execution) are similar. The only difference might be that we don't need interruption, but that seems to be handled by the "needs_interrupt" parameter, which I could set to False. Are these assumptions correct from an experienced LAVA point of view? Would this be a reasonable approach for getting LAVA to work with our own bootloader? Or is it necessary to implement our own bootloader python class in lava_dispatcher/actions/boot/? Mit freundlichen Grüßen / Best regards Tim Jaacks DEVELOPMENT ENGINEER Garz & Fricke GmbH Tempowerkring 2 21079 Hamburg Direct: +49 40 791 899 - 55 Fax: +49 40 791899 - 39 tim.jaacks(a)garz-fricke.com www.garz-fricke.com SOLUTIONS THAT COMPLETE! Sitz der Gesellschaft: D-21079 Hamburg Registergericht: Amtsgericht Hamburg, HRB 60514 Geschäftsführer: Matthias Fricke, Manfred Garz

7 years

Re: [Lava-users] Unable to get job summary for running test job in master-slave configuration

by Neil Williams

Please *always* reply to the list, not to the individual. On 19 February 2018 at 13:35, mayuri mohite <mayumohite18393(a)gmail.com> wrote: > Hello Neil, > > Thanks for your valuable reply i have uncomment encryption option file in > /etc/lava-server/lava-logs still i am not getting any log on GUI. > > Here i have attached my screen shots and my master and slave log > You've used tail on the slave.log but that has excluded all of the relevant content for this particular job - the messages when the job started. Restart lava-slave, resubmit the test job and then attach the full lava-slave.log to a reply to the mailing list. You've sent screenshots of quite an old UI as well - what version of LAVA are you running? (Include the output of dpkg -l lava-server lava-dispatcher ) > > I have submitted same job which is mentioned in > https://validation.linaro.org/static/docs/v2/first-job.html > > ####Below steps i am running on server side if any wrong please correct > me:#### > 1)$ sudo apt install postgresql > 2)$ sudo apt install lava-server > 3)$ sudo a2dissite 000-default > 4)$ sudo a2enmod proxy > 5)$ sudo a2enmod proxy_http > 6)$ sudo a2ensite lava-server.conf > 7)$ sudo service apache2 restart > 8)sudo lava-server manage createsuperuser --username <username> > --email=mail_ID > 9)ADD DEVICE TYPE AND DEVICES(whose worker host is hostnamefqdn of local > machine) using GUI AND THEN RUN BELOW COMMAND to add device dictionary > $ sudo lava-server manage device-dictionary --hostname qemu01 --import > qemu01.dict > $ cat qemu01.dict > {% extends 'qemu.jinja2' %} > {% set mac_addr = '52:54:00:12:34:59' %} > {% set memory = '1024' %} > 10)sudo /usr/share/lava-dispatcher/create_certificate.py master > 11)edit /etc/lava-server/lava-master file > $ cat /etc/lava-server/lava-master > # Configuration for lava-master daemon > > # Socket addresses of the master and logger > # MASTER_SOCKET="tcp://*:5556" > # LOGGER_SOCKET="tcp://*:5555" > > # Logging level should be uppercase (DEBUG, INFO, WARNING, ERROR) > # LOGLEVEL="DEBUG" > > # Encryption > # If set, will activate encryption using the master public and the slave > # private keys > ENCRYPT="--encrypt" > MASTER_CERT="--master-cert /etc/lava-dispatcher/certificates.d/master.key_ > secret" > SLAVES_CERTS="--slaves-certs /etc/lava-dispatcher/certificates.d/" > > 11)edit /etc/lava-server/lava-logs > $ cat /etc/lava-server/lava-logs > # Encryption > # If set, will activate encryption using the master public and the slave > # private keys > ENCRYPT="--encrypt" > > MASTER_CERT="--master-cert /etc/lava-dispatcher/certificates.d/master.key_ > secret" > SLAVES_CERTS="--slaves-certs /etc/lava-dispatcher/certificates.d" > > > ####Below steps i am running on Dispatcher side if any wrong please > correct me:#### > > 1)$ sudo apt install lava-dispatcher > 2)Create certificates > $ sudo /usr/share/lava-dispatcher/create_certificate.py foo_slave_1 > 3)copy certificates to server side > 4)changed /etc/lava-dispatcher/lava-slave configuration file > > $ cat /etc/lava-dispatcher/lava-slave > # Configuration for lava-slave daemon > > # URL to the master and the logger > MASTER_URL="tcp://134.86.62.197:5556" > # LOGGER_URL="tcp://<lava-master-dns>:5555" > > # Logging level should be uppercase (DEBUG, INFO, WARNING, ERROR) > # LOGLEVEL="DEBUG" > > # Enable IPv6 to connect to the master and logger > # # IPV6="--ipv6" > # > # # Slave hostname > # # Should be set for host that have random hostname (containers, ...) > # # The hostname can be any unique string, except "lava-logs" which is > reserved > # # for the lava-logs daemon. > HOSTNAME="--hostname mayuridb.mayuri" > > # Encryption > # If set, will activate encryption using the master public and the slave > # private keys > ENCRYPT="--encrypt" > MASTER_CERT="--master-cert /etc/lava-dispatcher/certificates.d/master.key" > SLAVE_CERT="--slave-cert /etc/lava-dispatcher/certificates.d/foo_slave_1. > key_secret" > 5)Now restart lava-salve > > > ######## GUI setup ############ > 1)Add worker host as newly added worker i.e dispatcher hostname > 2)Then submit job > > On Mon, Feb 19, 2018 at 12:35 PM, Neil Williams <neil.williams(a)linaro.org> > wrote: > >> On 16 February 2018 at 12:24, mayuri mohite <mayumohite18393(a)gmail.com> >> wrote: >> >>> Hello all, >>> >>> I am following steps mentioned in https://validation.linaro.org/ >>> static/docs/v2/pipeline-server.html#zmq-curve for installing lava >>> dispatcher on local machine (slave which is my worker machine) and >>> https://validation.linaro.org/static/docs/v2/installing_on_debian.html >>> for installing lava server on another machine (master) >>> >> >> It's always useful to mention which version of LAVA you are running. >> >> Check: https://staging.validation.linaro.org/static/docs/v2/ >> pipeline-server.html#enable-master-encryption >> >> Also edit /etc/lava-server/lava-logs to enable encryption: >> >> # Encryption >> # If set, will activate encryption using the master public and the slave >> # private keys >> ENCRYPT="--encrypt" >> >> Then restart the lava-master and lava-logs services. >> >> >>> >>> I have added device through GUI using https://validation.linaro.org/ >>> static/docs/v2/first-devices.html#create-device-database documentation >>> >>> I have successfully added worker to serevr side and able to get message >>> showing that encryption has been enabled on the both master and slave side. >>> >>> I am able to run test job on added target which has worker as slave >>> machine on which my dispatcher is running. >>> My job is started it is showing runnig state but I am not getting any >>> console log in job summary. >>> >>> please help me to get debug log in job summary. >>> >>> Regards, >>> >>> Mayuri. >>> >>> _______________________________________________ >>> Lava-users mailing list >>> Lava-users(a)lists.linaro.org >>> https://lists.linaro.org/mailman/listinfo/lava-users >>> >>> >> >> >> -- >> >> Neil Williams >> ============= >> neil.williams(a)linaro.org >> http://www.linux.codehelp.co.uk/ >> > > -- Neil Williams ============= neil.williams(a)linaro.org http://www.linux.codehelp.co.uk/

7 years

deploying to secondary media

by Hemanth K V

Hello Lava Users, Is there any sample example describing the "Enabling Secondary Media". I want to boot x86 based board with primary OS booted from USB and want to deploy the image to SATA used as secondary media. Thanks, Hemanth.

7 years

Error while submitting a Test Definition | Standalone LAVA Master installation

by Conrad Djedjebi

Good morning everyone, I am getting in touch with you in order to show you an issue i am encountering while submitting a basic Test Definition through my LAVA web page (localhost). I am a new LAVA user. Both the two attached scripts are returning the following error in the interface : I checked the syntax of my yaml files using an online tool. Here is a description of the steps I followed while installing the Standalone Master instance of LAVA on my computer : - Installing lava-server - Setting up Apache2 server - Setting ser2net - Adding a superuser to lava system - Creating a device type (.jinja2) file and a device - Writing a Test Definition (.yaml file) - Submitting that Test Definition through the GUI of lava thank you for your help,

7 years

Unable to get job summary for running test job in master-slave configuration

by mayuri mohite

Hello all, I am following steps mentioned in https://validation.linaro.org/static/docs/v2/pipeline-server.html#zmq-curve for installing lava dispatcher on local machine (slave which is my worker machine) and https://validation.linaro.org/static/docs/v2/installing_on_debian.html for installing lava server on another machine (master) I have added device through GUI using https://validation.linaro.org/static/docs/v2/first-devices.html#create-devi… documentation I have successfully added worker to serevr side and able to get message showing that encryption has been enabled on the both master and slave side. I am able to run test job on added target which has worker as slave machine on which my dispatcher is running. My job is started it is showing runnig state but I am not getting any console log in job summary. please help me to get debug log in job summary. Regards, Mayuri.

7 years

[IMPORTANT - test issue #99] Issues with installing when running behind a web proxy

by Zoran S

More about proxies: Here is update on the Lava proxy business. Please, read this very carefully, since this is the solution to the problems with Lava proxies, as well as Lava DUT ones. YES, after applying what Remi advised to me, the qemu01 test: https://www.validation.linaro.org/static/docs/v2/first-job.html worked like a charm. I simply copied /etc/lava-server/env.yaml to /etc/lava-server/env.dut.yaml (and created the new file). Remi, I would like to thank you for the advise! Best Regards, Zoran Stojsavljevic ---------- Forwarded message ---------- From: Remi Duraffort <remi.duraffort(a)linaro.org> Date: Thu, Feb 15, 2018 at 4:30 PM Subject: Re: [Lava-users] [HW target questions] Pointers from Lava test suite to the HW target To: Zoran S <zoran.stojsavljevic.de(a)gmail.com> Cc: Lava Users Mailman list <lava-users(a)lists.linaro.org> Not exactly. 1/ The process that is deploying, booting and testing a DUT (Device Under test) is called lava-run. lava-run environment variables are controlled by /etc/lava-server/env.yaml By default, all environment variables are removed and only a small set of variables are added. This helps to make executions reproducible between dispatchers and instances. So if you need an environment variable to be set, then you have to add it to /etc/lava-server/env.yaml 2/ On the DUT itself, by default, we don't add or change any environment variable. Because that's the user responsibility. However, if /etc/lava-server/env.dut.yaml does exists, it will be used to add environment variables to the DUT shell. To create the list of environment variables to add to the DUT, we take the full environment from lava-run (as defined by /etc/lava-server/env.yaml) and we apply the rules from env.dut.yaml. I hope that does help you to understand how environment is setup in lava. Rgds On Wed, Feb 14, 2018 at 2:04 PM, Zoran S <zoran.stojsavljevic.de(a)gmail.com> wrote: > https://gitlab.com/cip-project/cip-testing/testing/issues/99 > > Also from: > https://lists.cip-project.org/pipermail/cip-dev/2017-July/000338.html > > At the end of test issue #99, I added the solution to this problem. > > Lava does NOT read VM's ENV variables. It ignores them. Lava reads > /etc/lava-server/env.yaml as setup proxy file! > > Included there also /etc/lava-server/env.yaml file example. > > It could be done also DIRECTLY (every time bringing up the VM) using > python, tapping into the urllib3 python code (example given): > https://stackoverflow.com/questions/31151615/how-to-handle-proxies-in-urlli… > > I've tried this, using in-line python interpreter, it worked. > > Zoran

7 years, 1 month

← Newer
1
...
69
70
71
72
73
74
75
...
92
Older →

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

lava-users