Hi Marcin,

sbsa-ref platform with external pcie devices connected to root-port has been simulated and tested in https://linaro.atlassian.net/browse/SRCPC-80 with BSA and similar observations have been made in log file "BSA_with_pcieTests_log.txt". The topology for pcie devices behind root port or PCIe-PCI bridge could be generated by adding "-device" options to our platform as in https://github.com/qemu/qemu/blob/master/docs/pcie.txt The question is do we need to explicitly add root ports etc in QEMU when we have support for the same via QEMU pcie command options.

-Shashi

On Mon, 23 Jan 2023 at 09:52, Marcin Juszkiewicz via Asa-dev < asa-dev@op-lists.linaro.org> wrote:

I looked through BSA/SBSA issues today and found one with PCI related discussion [1]. BSA ignores cards which are RCiEP (Root Complex Integrated Endpoint). And for SBSA Reference Platform it means all of them.

1. https://github.com/ARM-software/bsa-acs/issues/77

By default QEMU starts SBSA Reference Platform with two cards:

• e1000e network
• VGA graphics

Setup is done as simple as possible with both cards plugged directly into PCI Express bus:

~ # lspci -nn 00:00.0 Host bridge [0600]: Red Hat, Inc. QEMU PCIe Host bridge [1b36:0008] 00:01.0 Ethernet controller [0200]: Intel Corporation 82574L Gigabit Network Connection [8086:10d3] 00:02.0 VGA compatible controller [0300]: Device [1234:1111] (rev 02)

~ # lspci -t -[0000:00]-+-00.0 +-01.0 -02.0

VGA is simple PCI card, e1000e becames Root Complex Integrated Endpoint. EDK2 sees them, Linux sees them, both are able to use it.

But that's far from real world PCI Express scenario where PCI Express cards are connected to the bus via root ports.

Let take a look at system without cards added by default. I edited QEMU code, removed adding e1000e and VGA card. Then added several cards with root ports and bridges. Here I added NVME, USB controller, e1000e network card, Bochs display and virtio RNG PCI card:

~ # lspci -nn 00:00.0 Host bridge [0600]: Red Hat, Inc. QEMU PCIe Host bridge [1b36:0008] 00:01.0 PCI bridge [0604]: Red Hat, Inc. QEMU PCIe Root port [1b36:000c] 00:02.0 PCI bridge [0604]: Red Hat, Inc. QEMU PCIe Root port [1b36:000c] 00:03.0 PCI bridge [0604]: Red Hat, Inc. QEMU PCIe Root port [1b36:000c] 00:04.0 PCI bridge [0604]: Red Hat, Inc. QEMU PCIe Root port [1b36:000c] 00:05.0 PCI bridge [0604]: Red Hat, Inc. QEMU PCIe Root port [1b36:000c] 01:00.0 Non-Volatile memory controller [0108]: Red Hat, Inc. QEMU NVM Express Controller [1b36:0010] (rev 02) 02:00.0 USB controller [0c03]: Red Hat, Inc. QEMU XHCI Host Controller [1b36:000d] (rev 01) 03:00.0 Display controller [0380]: Device [1234:1111] (rev 02) 04:00.0 Ethernet controller [0200]: Intel Corporation 82574L Gigabit Network Connection [8086:10d3] 05:00.0 PCI bridge [0604]: Red Hat, Inc. Device [1b36:000e] 06:08.0 Unclassified device [00ff]: Red Hat, Inc. Virtio RNG [1af4:1005]

~ # lspci -t -[0000:00]-+-00.0 +-01.0-[01]----00.0 +-02.0-[02]----00.0 +-03.0-[03]----00.0 +-04.0-[04]----00.0 -05.0-[05-06]----00.0-[06]----08.0 ~ #

Each PCI Express card (01:00.0 - 04:00.0) are behind PCI Express root port, virtio RNG is behind PCIe-PCI bridge (which is behind PCIe root port). All PCIe cards are reported with "Express (v2) Endpoint" capability. See attached file for "lspci -vvvnn" output.

I have run BSA ACS against both default QEMU setup (bsa-default.txt) and against my complex one (bsa-complex.txt). As you can see with new PCI setup BSA ACS runs far more tests than before.

What next?

I think that QEMU needs patching to add root ports etc. Switching from VGA card to Bochs display would be good as well because former is PCI while former is PCI Express. We could also think of adding USB controller by default.-- Asa-dev mailing list -- asa-dev@op-lists.linaro.org To unsubscribe send an email to asa-dev-leave@op-lists.linaro.org