Hi, guys,
We find an issue related to job submit:
1) One team use "lavacli" to submit request, and sometimes it will report next:
07-Sep-2020 16:37:35 Unable to connect: HTTPConnectionPool(host='lava-master.sw.nxp.com', port=80): Read timed out. (read timeout=20.0)
Looks this error happens at next, what do you think about this issue?
try:
# Create the Transport object
parsed_uri = urlparse(uri)
transport = RequestsTransport(
parsed_uri.scheme,
config.get("proxy"),
config.get("timeout", 20.0),
config.get("verify_ssl_cert", True),
)
# allow_none is True because the server does support it
proxy = xmlrpc.client.ServerProxy(uri, allow_none=True, transport=transport)
version = proxy.system.version()
except (OSError, xmlrpc.client.Error) as exc:
print("Unable to connect: %s" % exc2str(exc))
return 1
2) Another team write their own python code using XMLRPC to submit job, did something like next, it reports next:
ERROR in XMLRPC.py:submitJob:63 msg: Failed to submit job, reason: <ProtocolError for chuan.su:chuan.su@lava-master.sw.nxp.com/RPC2: 502 Bad Gateway>!
try:
job_id = self.connection.scheduler.submit_job(job)
self.logger.debug("Successed to submit job , job_id: %d, platform; %s!",job_id,platform)
return job_id
except Exception as e:
self.logger.error("Failed to submit job, reason: %s!",str(e))
return None
We are currently using lava server version 2020.08, guys told me in the past days, we also encountered similar, but with very low probability. But recently it becomes very high probability.
I'd like to know if possible this will related to your changes to gunicorn eventlet? Or other possible reasons?
Thanks,
Larry
Hi, guys,
We found a blocking issue for android test, the story is next:
1. job #1 with device #1 is running for about 12 hours, during its run, it will restart the boards many times, then the usb path will e.g. start from /dev/bus/usb/003/001 to /dev/bus/usb/002, then /dev/bus/usb/003...... finally /dev/bus/usb/127.
You know, the max number here will be 127, so, if device reset again, the number will back to 001.
Adb devices in container 1:
$ adb devices
List of devices attached
040c41d4d72d7393 device
2. job #2 with device #2 starts run during the job #1 still running, then E.g. it will mknod /dev/bus/usb/003/016 to another docker-test-shell container, also cgroup privilege added.
But as the /dev/bus/usb/003/016 was once used by job #1, and this node won't be deleted from docker-test-shell container.
So, we find high probability the device #2 was seen in job #1's docker-test-shell container (Checked with adb devices).
Now, adb devices in container 2:
$adb devices
List of devices attached
After above, adb devices in container 1:
$ adb devices
List of devices attached
040c41d4d72d7393 device
23305a0a5c85d936 device
This becomes a big issue for our parallel android test.
In fact, in the old LXC days, we also find similar issues, so we made a workaround in our local:
https://github.com/atline/lava-docker-slave/blob/66f15d9da88912fc929fef5213…
In this patch, we also monitor "remove", ENV{ID_SERIAL_SHORT}, that is "if a usb leaved, let it delete the node".
But, I don't know for which reasons, in current version(2020.08), now I can just monitor "remove" in udev, can't match "remove + ENV{ID_SERIAL_SHORT}" correctly.
So, to make our local android run could work in a short time, we did a patch as next:
# diff __init__.py.bak __init__.py
157,158c157,158
< "mkdir -p %s && mknod %s c %d %d || true"
< % (os.path.dirname(node), node, major, minor),
---
> "mkdir -p %s && rm -f %s/* && mknod %s c %d %d || true"
> % (os.path.dirname(node), os.path.dirname(node), node, major, minor),
Now, no issue happens in our side, but it looks this is somewhat not universal?
So, I'm here to ask the question, have you ever found this issue? And what's your thought on this?
Hi Lava users,
I am facing some issues while using the interactive test method, for testing u-boot reset function. In v2019.11 this worked perfectly but currently while using v2020.07 and this features seems to be broken. Or am I missing something here.
Here detailed description of the issue:
We are testing the u-boot reset command below is the snippet of the job definition
...
...
...
- name: mmcinfo
command: mmcinfo
successes:
- message: "Erase Group Size:"
failures:
- message: "Unknown command "
error: "Unknown command "
- name: reset
command: reset
successes:
- message: "Hit any key to stop autoboot"
failures:
- message: "Unknown command "
error: "Unknown command "
So in v2019.11 once the "Hit any key to stop autoboot" string was matched the target switched off (check screenshot "reset-snippet-v2019-11.PNG").
But this is not working in v2020.07, after the success message is matched the target is not switched off (check screenshot "reset-snippet-v2020-07.PNG").
Regards,
Bhargav
Hi, all,
I want to do a job search with owner, E.g. in https://validation.linaro.org/scheduler/alljobs 's search box, I input next:
owner_query=qa-reports-bot
But, it return nothing to me, all jobs filtered out.
I wonder what's the correct input in this search box for "Custom query text"?
Regards,
Larry
Hello Lava-Users,
When we try to add new device we are facing the error
500 Internal Server Error
maximum recursion depth exceeded in comparison
Oops, something has gone wrong!
Recently we had updated the LAVA instance server to Debian10. The LAVA server version presently running on the instance is
lava_admin@svr-ies-ce-lava:~$ dpkg -l | grep lava
ii lava 2020.07.0002.g44e83009c+10+buster all Linaro Automated Validation Architecture metapackage
ii lava-common 2020.07.0002.g44e83009c+10+buster all Linaro Automated Validation Architecture common
ii lava-coordinator 2020.07.0002.g44e83009c+10+buster all LAVA coordinator daemon
ii lava-dev 2020.07.0002.g44e83009c+10+buster all Linaro Automated Validation Architecture developer support
ii lava-dispatcher 2020.07.0002.g44e83009c+10+buster all Linaro Automated Validation Architecture dispatcher
ii lava-dispatcher-host 2020.07.0002.g44e83009c+10+buster all LAVA dispatcher host tools
ii lava-server 2020.07.0002.g44e83009c+10+buster all Linaro Automated Validation Architecture server
ii lava-server-doc 2020.07.0002.g44e83009c+10+buster all Linaro Automated Validation Architecture documentation
ii lavacli 0.9.5-1 all LAVA XML-RPC command line interface
As specified in https://stackoverflow.com/questions/3323001/what-is-the-maximum-recursion-d… should we try to increase python recursion limit to overcome this?
Thanks,
Hemanth.
Hi Lava users,
I am trying to use the u-boot-ums deploy method.
I have taken reference from this job : https://staging.validation.linaro.org/scheduler/job/277518/definition
While submitting the job I am getting the following error:
Job error: None of the deployment strategies accepted your deployment parameters, reasons given: overlay: "to" parameter is not "overlay" docker: 'docker' not in the device configuration deploy methods download: "to" parameter is not "download" downloads: "to" parameter is not "downloads" qemu-nfs: "nfs" is not in the device configuration deploy methods images: "to" parameter is not "tmpfs" iso: "to" was not in parameters, or "to" was not "iso-installer" fastboot: "to" parameter is not "fastboot" flasher: 'flasher' not in the device configuration deploy methods fvp: 'to' was not fvp lxc: "to" parameter is not "lxc" nbd: "to" parameter is not "nbd" nfs: "to" parameter is not "nfs" removable: "media" was not "sata", "sd", or "usb" vemsd: "to" parameter is not "vemsd" mps: "mps" was not in the device configuration deploy methods musca: "musca" was not in the device configuration deploy methods ssh: "to" parameter is not "ssh" tftp: "to" parameter is not "tftp" uboot-ums: "u-boot-ums" was not in the device configuration deploy methods" recovery-mode: 'recovery' not in the device configuration deploy methods uuu: "to" parameter is not "uuu"
Please check the attached yaml file (u-boot-ums-def.yaml).
?
Regards,
Bhargav
Hi,
I saw something like "LAVA_HARD_RESET_COMMAND" was added to environment.
Yes, the test action could use this command to reboot the device now, it's useful.
But, after the command sent, the code in container still need to handle something like next:
"Expect press any key to continue"
"Press any key"
"Expect login prompt"
"Login"
Etc......
But, this exactly the same code what lava have do in a lots of python files located in "lava_dispatcher/actions/boot".
My question is: do you think it's possible for user code in "docker-test-shell" action directly send any command or use any protocol to tell lava code(maybe a listener or something else) to help to do these things, then after done, the user code in docker test shell continue to execute its own stuff?
You know, this could reuse existed lava code as much as possible. I just want to know if you have plan for this? Thanks.
