I do wonder which hardware passes SBSA ACS test suite and is available
at Linaro lab.
Qualcomm machine I use in mlab is not one of them for sure. Testsuite
crashes badly before even starting any tests.
If someone has compliant hardware then please run SBSA ACS and provide
logs to compare.
On Fri, Oct 30, 2020 at 12:07 PM Marcin Juszkiewicz via Asa-dev <
asa-dev(a)op-lists.linaro.org> wrote:
> W dniu 30.10.2020 o 13:03, Marcin Juszkiewicz via Asa-dev pisze:
> > I added new CI job: ldcg-sbsa-acs.
> >
> > It builds HEAD of qemu and edk2 and then use it to run Arm SBSA ACS
> > testsuite for SBSA levels 3-6.
> >
> > https://snapshots.linaro.org/ldcg/sbsa-acs/3/ has logs from each run.
> >
> > This allows us to check which components are still missing for each
> level.
> >
> > I created some Jira cards yesterday to list missing parts:
> >
> > ASA-65 - add PMU counters
> > ASA-66 - add GIC ITS support
> > ASA-67 - add Virtual Host Extensions (required for level 4)
> > ASA-68 - add SMMU (it is not detected by SBSA ACS now)
> >
>
ASA-68 effectively requirers ASA-66 as far as I can tell. I did try an
instantiate a SMMUv3 from ACPI but
the kernel throws errors if there is not an ITS there. (I could however
have made a mistake in my tables).
The qemu patches for SMMUv3 are upstream now so at least its correct from
QEMU side.
Graeme
> > I plan to look at those logs and create additional cards so we can see
> > what is still missing (or not enabled).
> >
> > Next run will have larger timeout as 30s looks a bit too short.
>
> Current situation summary:
>
> SBSA level | tests run | passed | failed
> ----------------------------------------
> 3 | 45 | 32 | 6
> 4 | 87 | 44 | 13
> 5 | 92 | 44 | 20
> 6 | 102 | 44 | 29
>
>
> Lot of tests were skipped.
> --
> Asa-dev mailing list
> Asa-dev(a)op-lists.linaro.org
> https://op-lists.linaro.org/mailman/listinfo/asa-dev
>
W dniu 30.10.2020 o 13:07, Marcin Juszkiewicz via Asa-dev pisze:
> Current situation summary:
>
> SBSA level | tests run | passed | failed
> ----------------------------------------
> 3 | 45 | 32 | 6
> 4 | 87 | 44 | 13
> 5 | 92 | 44 | 20
> 6 | 102 | 44 | 29
As you see SBSA level 3 is near with just 6 failed tests:
12 : Check number of PMU counters
Failed on PE - 0 for Level= 3 : Result: --FAIL-- 1
102 : If PCIe, then GIC implements ITS
Failed on PE - 0 for Level= 3 : Result: --FAIL-- 1
104 : GIC Maintenance Interrupt
Failed on PE - 0 for Level= 3 : Result: --FAIL-- 1
206 : SYS Timer if PE Timer not ON
PE Timers are not always-on.
Failed on PE - 0 for Level= 3 : Result: --FAIL-- 1
301 : Check NS Watchdog Accessibility
No Watchdogs reported 0
Failed on PE - 0 for Level= 3 : Result: --FAIL-- 1
302 : Check Watchdog WS0 interrupt
No Watchdogs reported 0
Failed on PE - 0 for Level= 3 : Result: --FAIL-- 1
Cards for failed tests:
test 12 is ASA-65 (add PMU counters)
test 102 is ASA-66 (add ITS support)
test 104 is ASA-70 (add GIC Maintenance Interrupt)
test 206 is ASA-71 (fix always-on timers)
test 301/302 is ASA-57 (add Watchdog support)
From them ASA-57 is done in QEMU, approved for EDK2.
If you have comments/ideas etc then please comment in Jira cards or in
replies.
W dniu 30.10.2020 o 13:03, Marcin Juszkiewicz via Asa-dev pisze:
> I added new CI job: ldcg-sbsa-acs.
>
> It builds HEAD of qemu and edk2 and then use it to run Arm SBSA ACS
> testsuite for SBSA levels 3-6.
>
> https://snapshots.linaro.org/ldcg/sbsa-acs/3/ has logs from each run.
>
> This allows us to check which components are still missing for each level.
>
> I created some Jira cards yesterday to list missing parts:
>
> ASA-65 - add PMU counters
> ASA-66 - add GIC ITS support
> ASA-67 - add Virtual Host Extensions (required for level 4)
> ASA-68 - add SMMU (it is not detected by SBSA ACS now)
>
> I plan to look at those logs and create additional cards so we can see
> what is still missing (or not enabled).
>
> Next run will have larger timeout as 30s looks a bit too short.
Current situation summary:
SBSA level | tests run | passed | failed
----------------------------------------
3 | 45 | 32 | 6
4 | 87 | 44 | 13
5 | 92 | 44 | 20
6 | 102 | 44 | 29
Lot of tests were skipped.
I added new CI job: ldcg-sbsa-acs.
It builds HEAD of qemu and edk2 and then use it to run Arm SBSA ACS
testsuite for SBSA levels 3-6.
https://snapshots.linaro.org/ldcg/sbsa-acs/3/ has logs from each run.
This allows us to check which components are still missing for each level.
I created some Jira cards yesterday to list missing parts:
ASA-65 - add PMU counters
ASA-66 - add GIC ITS support
ASA-67 - add Virtual Host Extensions (required for level 4)
ASA-68 - add SMMU (it is not detected by SBSA ACS now)
I plan to look at those logs and create additional cards so we can see
what is still missing (or not enabled).
Next run will have larger timeout as 30s looks a bit too short.
See <https://ci.linaro.org/jenkins/job/ldcg-sbsa-acs/1/display/redirect>
Changes:
------------------------------------------
Started by user Marcin Juszkiewicz (marcin.juszkiewicz(a)linaro.org)
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on leg-aarch64-01-f1b55a93cb1d (docker-buster-arm64-leg) in workspace <https://ci.linaro.org/jenkins/job/ldcg-sbsa-acs/ws/>
[ldcg-sbsa-acs] $ /bin/bash /tmp/jenkins16867014569957094092.sh
+ source common-code.sh
/tmp/jenkins16867014569957094092.sh: line 7: common-code.sh: No such file or directory
Build step 'Execute shell' marked build as failure
See <https://ci.linaro.org/jenkins/job/ldcg-sbsa-firmware/17/display/redirect>
Changes:
------------------------------------------
[...truncated 2.95 MB...]
Protocol test: QueryMode_Func
Instances: 2/2
Iterations: 1/1
Remained test cases: 59
Protocol test: SetMode_Conf
Instances: 1/2
Iterations: 1/1
Protocol test: SetMode_Conf
Instances: 2/2
Iterations: 1/1
Remained test cases: 58
Protocol test: SetMode_Func
Instances: 1/2
Iterations: 1/1
Protocol test: SetMode_Func
Instances: 2/2
Iterations: 1/1
Remained test cases: 57
Protocol test: AllocateBuffer_Conf
Instances: 1/1
Iterations: 1/1
Remained test cases: 56
Protocol test: AllocateBuffer_Func
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Remained test cases: 55
Protocol test: Configuration_Func
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Remained test cases: 54
Protocol test: CopyMem_Conf
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Remained test cases: 53
Protocol test: CopyMem_Func
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Test Root Bridge Io Protocol SctCopyMem () interface.
Remained test cases: 52
Protocol test: Flush_Func
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Remained test cases: 51
Protocol test: FreeBuffer_Func
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Remained test cases: 50
Protocol test: GetAttributes_Conf
Instances: 1/1
Iterations: 1/1
Remained test cases: 49
Protocol test: GetAttributes_Func
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Remained test cases: 48
Protocol test: IoRead_Conf
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Remained test cases: 47
Protocol test: IoRead_Func
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Remained test cases: 46
Protocol test: IoWrite_Conf
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Remained test cases: 45
Protocol test: IoWrite_Func
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Remained test cases: 44
Protocol test: Map_Conf
Instances: 1/1
Iterations: 1/1
Remained test cases: 43
Protocol test: MemRead_Conf
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Remained test cases: 42
Protocol test: MemRead_Func
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Remained test cases: 41
Protocol test: MemWrite_Conf
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Remained test cases: 40
Protocol test: MemWrite_Func
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Remained test cases: 39
Protocol test: PciRead_Conf
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Remained test cases: 38
Protocol test: PciRead_Func
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Remained test cases: 37
Protocol test: PciWrite_Conf
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Remained test cases: 36
Protocol test: PciWrite_Func
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Remained test cases: 35
Protocol test: PollIo_Conf
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Remained test cases: 34
Protocol test: PollIo_Func
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Remained test cases: 33
Protocol test: PollMem_Conf
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Remained test cases: 32
Protocol test: PollMem_Func
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Remained test cases: 31
Protocol test: SetAttributes_Conf
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
Remained test cases: 30
Protocol test: SetAttributes_Func
Instances: 1/1
Iterations: 1/1
This Root Bridge Device Path:
Acpi(PNP0A03,0)
NOTICE: Booting Trusted Firmware
NOTICE: BL1: v2.3(release):v2.3-673-g332d194fd
NOTICE: BL1: Built : 13:32:55, Sep 15 2020
NOTICE: BL1: Booting BL2
NOTICE: BL2: v2.3(release):v2.3-673-g332d194fd
NOTICE: BL2: Built : 13:32:57, Sep 15 2020
NOTICE: BL1: Booting BL31
NOTICE: BL31: v2.3(release):v2.3-673-g332d194fd
NOTICE: BL31: Built : 13:33:00, Sep 15 2020
[2J[04D[=3h[2J[09D[0m[35m[40m[0m[30m[47mWelcome to GRUB!
[0m[37m[40merror: no such device: ((hd0,gpt2)/EFI/BOOT)/EFI/BOOT/grub.cfg.
[0m[30m[40m[2J[01;01H[0m[37m[40m[02;30HGNU GRUB version 2.02
[01C/----------------------------------------------------------------------------\[05;02H|[76C|[06;02H|[76C|[07;02H|[76C|[08;02H|[76C|[09;02H|[76C|[10;02H|[76C|[11;02H|[76C|[12;02H|[76C|[13;02H|[76C|[14;02H|[76C|[15;02H|[76C|[16;02H|[76C|[17;02H|[76C|[18;02H\----------------------------------------------------------------------------/[19;02H[20;02H Use the ^ and v keys to select which entry is highlighted.
Press enter to boot the selected OS, `e' to edit the commands
before booting or `c' for a command-line. [05;80H
[05;03H luv [01D[0m[30m[47m[06;03H*sbbr/sbsa [0m[37m[40m[01D[07;03H [01D[08;03H [01D[09;03H [01D[10;03H [01D[11;03H [01D[12;03H [01D[13;03H [01D[14;03H [01D[15;03H [01D[16;03H [01D[17;03H [01D[02C
[06;78H[23;01H The highlighted entry will be executed automatically in 10s. [06;78H[23;01H The highlighted entry will be executed automatically in 9s. [06;78H[23;01H The highlighted entry will be executed automatically in 8s. [06;78H[23;01H The highlighted entry will be executed automatically in 7s. [06;78H[23;01H The highlighted entry will be executed automatically in 6s. [06;78H[23;01H The highlighted entry will be executed automatically in 5s. [06;78H[23;01H The highlighted entry will be executed automatically in 4s. [06;78H[23;01H The highlighted entry will be executed automatically in 3s. [06;78H[23;01H The highlighted entry will be executed automatically in 2s. [06;78H[23;01H The highlighted entry will be executed automatically in 1s. [06;78H[23;01H The highlighted entry will be executed automatically in 0s. [06;78H[0m[30m[40m[2J[01;01H[0m[37m[40m[0m[30m[40m[2J[04D[0m[37m[40m Booting `sbbr/sbsa'
/EndEntire
file path: /HardwareVendor(0d51905b-b77e-452a-a2c0-eca0cc8d514a)[9: 00 00 10 60
00 00 00 00 00 ]/Sata(0,ffff,0)/HD(2,40800,47000,026c6fdf3eb40948,2,2)
/File(\EFI\BOOT)/File(Shell.efi)/EndEntire
[2J[01;01HUEFI Interactive Shell v2.2
EDK II
UEFI v2.70 (EDK II, 0x00010000)
[1m[33m[40mMapping table[0m[37m[40m
[1m[33m[40m FS0:[0m[37m[40m [1m[37m[40mAlias(s):[0m[37m[40mHD0a65535a1:;BLK1:
VenHw(0D51905B-B77E-452A-A2C0-ECA0CC8D514A,000010600000000000)/Sata(0x
0,0xFFFF,0x0)/HD(1,GPT,DCC994A8-3772-4BF4-9936-D237B349809B,0x800,0x40000)
[1m[33m[40m FS1:[0m[37m[40m [1m[37m[40mAlias(s):[0m[37m[40mHD0a65535a2:;BLK2:
VenHw(0D51905B-B77E-452A-A2C0-ECA0CC8D514A,000010600000000000)/Sata(0x
0,0xFFFF,0x0)/HD(2,GPT,DF6F6C02-B43E-4809-8DB0-36633A94248A,0x40800,0x47000)
[1m[33m[40m BLK3:[0m[37m[40m [1m[37m[40mAlias(s):[0m[37m[40m
VenHw(93E34C7E-B50E-11DF-9223-2443DFD72085,00)
[1m[33m[40m BLK0:[0m[37m[40m [1m[37m[40mAlias(s):[0m[37m[40m
VenHw(0D51905B-B77E-452A-A2C0-ECA0CC8D514A,000010600000000000)/Sata(0x
0,0xFFFF,0x0)
Press [1m[37m[40mESC[0m[37m[40m in 5 seconds to skip [1m[33m[40mstartup.nsh[0m[37m[40m or any other key to continue.[72DPress [1m[37m[40mESC[0m[37m[40m in 4 seconds to skip [1m[33m[40mstartup.nsh[0m[37m[40m or any other key to continue.[72DPress [1m[37m[40mESC[0m[37m[40m in 3 seconds to skip [1m[33m[40mstartup.nsh[0m[37m[40m or any other key to continue.[72DPress [1m[37m[40mESC[0m[37m[40m in 2 seconds to skip [1m[33m[40mstartup.nsh[0m[37m[40m or any other key to continue.[72DPress [1m[37m[40mESC[0m[37m[40m in 1 seconds to skip [1m[33m[40mstartup.nsh[0m[37m[40m or any other key to continue.
[1m[33m[40mShell> [0m[37m[40mecho -off
SCT tests continue...
Press any key to stop the EFI SCT running
Press any key within 5 seconds Press any key within 4 seconds Press any key within 3 seconds Press any key within 2 seconds Press any key within 1 seconds Load support files ...
Load proxy files ...
Load test files ...
Continue test preparing...
Remained test cases: 30
Remained test cases: 29
Protocol test: AllocateBuffer_Conf
Instances: 1/4
Iterations: 1/1
Protocol test: AllocateBuffer_Conf
Instances: 2/4
Iterations: 1/1
Protocol test: AllocateBuffer_Conf
Instances: 3/4
Iterations: 1/1
Protocol test: AllocateBuffer_Conf
Instances: 4/4
Iterations: 1/1
Synchronous Exception at 0x000001003F434548
Synchronous Exception at 0x000001003F434548
qemu-system-aarch64: terminating on signal 15 from pid 28271 (timeout)
Build step 'Execute shell' marked build as failure