October 2020 - Stratos-dev - op-lists.linaro.org

[App-services] 2020-10-27 OpenAMP App-services call recording, notes, slides, and action items (fwd)

by Stefano Stabellini

Hi Alex, Arnd, Jean-Philippe, and all, During the OpenAMP App-services call this week WindRiver gave a couple of extremely interesting presentations, see attached slides. Dan Milea discussed the usage of virtio between heterogeneous clusters, i.e. virtio frontends on the Cortex-R cluster and virtio backends on the Cortex-A cluster. They used a setup based on pre-shared memory to make it work, which I believe is similar to the swiotlb approach we discussed in Stratos. They noted that the last version of the virtio spec on github had something in regards to virtio and pre-shared memory regions that might help us cover this use-case from a spec perspective. Dan, would you be able to share a pointer to it for clarity? So far, we acted on the assumption that the virtio spec doesn't allow for this architecture today; it would be fantastic if it turns out that it already does. The other very interesting presentation by Joshua Pincus was about virtio-mmio and MSIs. They did excellent measurements on the performance of virtio-mmio and they found out that the single source of notifications (one interrupt) is the bottleneck. Adding MSIs vastly improved performance. You can see the detailed breakdown on slide #8 of "OpenAMP Virt I/O MMIO w/ MSI". This analysis really points in the direction of adding MSIs to virtio-mmio. Cheers, Stefano ---------- Forwarded message ---------- Date: Wed, 28 Oct 2020 18:41:09 +0000 From: Nathalie Chan King Choy via App-services <app-services(a)lists.openampproject.org> Reply-To: Nathalie Chan King Choy <nathalie(a)xilinx.com> To: "app-services(a)lists.openampproject.org" <app-services(a)lists.openampproject.org> Subject: [App-services] 2020-10-27 OpenAMP App-services call recording, notes, slides, and action items Hi all, The notes from yesterday’s OpenAMP App-services call can be found at: https://github.com/OpenAMP/open-amp/wiki/OpenAMP-Application-Services-Subgr… The link to the Webex recording is in the notes. I am not sure how long before the recordings expire or I will hit my storage limit, so if you need to catch up by watching the recording, please download it in the next couple weeks. Please find attached the slides from Dan & Josh. Action items: * Dan & Josh to send slides (DONE) * Stefano to start a thread w/ the folks who are working on shared memory & VirtIO Best regards, Nathalie C. Chan King Choy Program Manager focused on Open Source and Community -- App-services mailing list App-services(a)lists.openampproject.org https://lists.openampproject.org/mailman/listinfo/app-services

3 years, 5 months

3
2
0 0

AGL Demo KVM Command Lines

by Alex Bennée

Hi, I think I've mentioned them in other threads but to bring it all in one place. Using the images from: https://download.automotivelinux.org/AGL/release/jellyfish/10.0.0/qemuarm64… You just need the kernel image and the agl-demo-platform-crosssdk-qemuarm64.ext4 filesystem. The initrd is a slimmed down rootfs in a RAM disk. To run with a simple VGA-like frame buffer: ./aarch64-softmmu/qemu-system-aarch64 -cpu host -enable-kvm \ -smp 2 -machine type=virt -serial mon:stdio \ -netdev user,id=unet,hostfwd=tcp::2222-:22 \ -device virtio-net-device,netdev=unet,id=virt-net \ -drive id=disk0,file=agl-demo-platform-crosssdk-qemuarm64.ext4,if=none,format=raw \ -device virtio-blk-device,drive=disk0 \ -soundhw hda \ -device qemu-xhci \ -device usb-tablet \ -device usb-kbd \ -object rng-random,filename=/dev/urandom,id=rng0 \ -device virtio-rng-pci,rng=rng0 \ -device virtio-serial-device \ -chardev null,id=virtcon -device virtconsole,chardev=virtcon -m 4096 \ -kernel ~/images/agl/Image \ -append "console=ttyAMA0 root=/dev/vda" \ -display gtk,show-cursor=on \ -device VGA,edid=on,xmax=1024,ymax=768 Running with virtio-gpu should give a similar 2D frame buffer experience, making the last two lines: -display gtk,show-cursor=on \ -device virtio-gpu-pci And finally enabling VirGL for accelerated pass-through you get: -display gtk,show-cursor=on,gl-on \ -device virtio-gpu-pci,virgl=on although obviously if your host ends up sending it to softpipe in the end the results won't be as impressive. I've also replicated the config (find it in /boot) and re-built my own kernel current stable kernel and it worked fine. -- Alex Bennée

3 years, 5 months

2
1
0 0

Linaro Stratos Agenda for this weeks call 29th Oct 2020

by Mike Holmes

Hi All We have only a small agenda so far [1] with some updates on the demo progress and a posed question. Do we have any other agenda topics? We are planning a Huawei presentation for the following call; Salil will highlight their interest in the topics touched on by the Stratos project. If you want to catch Salili he is also presenting at the KVM forum this week https://kvmforum2020.sched.com/event/eE4m/challenges-in-supporting-virtual-… Mike [1] https://collaborate.linaro.org/display/STR/2020-10-29+Project+Stratos+Sync+… -- Mike Holmes | Director, Foundation Technologies, Linaro Mike.Holmes(a)linaro.org <mike.holmes(a)linaro.org> "Work should be fun and collaborative, the rest follows"

3 years, 5 months

1
0
0 0

[PATCH v1 0/4] add guest-loader (for direct Xen boot)

by Alex Bennée

Hi, This is a refinement of the Xen loader I posted a few weeks ago. Rather than complicate the generic loader with extra options I went for the more expedient approach of adding a completely new device called the guest-loader. As it didn't need to deal with any of the subtleties of the generic-loader it also worked out somewhat simpler. Instead of allowing the user to hand hack FDT blobs we simply add syntactic sugar that understands the difference between a kernel and initrd. This could be expanded in future if we want although at the moment I don't know what else you would add. The new syntax is now simpler: -device guest-loader,addr=0x42000000,kernel=Image,bootargs="console=hvc0 earlyprintk=xen" \ -device guest-loader,addr=0x47000000,initrd=rootfs.cpio So any objections? It would be nice to get in this cycle but we only have a week left until soft-freeze. Alex Bennée (4): hw/board: promote fdt from ARM VirtMachineState to MachineState hw/riscv: migrate fdt field to generic MachineState device_tree: add qemu_fdt_setprop_string_array helper hw/core: implement a guest-loader to support static hypervisor guests hw/core/guest-loader.h | 34 ++++ include/hw/arm/virt.h | 1 - include/hw/boards.h | 1 + include/hw/riscv/virt.h | 1 - include/sysemu/device_tree.h | 17 ++ hw/arm/virt.c | 322 ++++++++++++++++++----------------- hw/core/guest-loader.c | 140 +++++++++++++++ hw/riscv/virt.c | 18 +- softmmu/device_tree.c | 26 +++ hw/core/meson.build | 2 + 10 files changed, 398 insertions(+), 164 deletions(-) create mode 100644 hw/core/guest-loader.h create mode 100644 hw/core/guest-loader.c -- 2.20.1

3 years, 5 months

3
8
0 0

Re: [Stratos-dev] My current status of Xen on QEMU TCG (-m virt)

by Stefano Stabellini

On Thu, 22 Oct 2020, Julien Grall via Stratos-dev wrote: > On 22/10/2020 09:55, Masami Hiramatsu wrote: > > 2020年10月22日(木) 17:11 Julien Grall <julien(a)xen.org>: > > > On 22/10/2020 08:09, Masami Hiramatsu wrote: > > > > Hi, > > > > Now I'm trying to use vhost-user-gpu. > > > > > > > > I built qemu and xen and installed user /usr/local/. > > > > Xen installed it's qemu in /usr/local/lib/xen/bin, > > > > Qemu was installed in /usr/local/bin, so those are not conflict. > > > > > > > > If I added device_model_args= option to the domU.conf file, it didn't > > > > work (failed to exec qemu). > > > > > > What is the path used for QEMU? > > > > It should be /usr/local/lib/xen/bin/qemu-system-i386, because I > > replaced it with a wrapper shell script to record the parameters :) > > > > With the device_model_override, I used locally installed > > /usr/local/bin/qemu-system-i386. > > > > > > > > > I also added device_model_override= but it didn't change neither. > > > > But in both cases, it seems vhost-user-gpu got access from qemu. > > > > > > > > To clarify what argument passed to the qemu, I hacked > > > > /usr/local/lib/xen/bin/qemu-system-i386, and found that the arguments > > > > passed via device_model_args=[] were just added to the original > > > > arguments. > > > > > > > > OK, now I'm trying to test the qemu-system-i386 itself as below > > > > > > > > mhiramat@develbox:/opt/agl$ sudo > > > > /usr/local/lib/xen/bin/qemu-system-i386.bin -xen-domid 11 -no-shutdown > > > > -chardev > > > > socket,id=libxl-cmd,path=/var/run/xen/qmp-libxl-11,server,nowait > > > > -mon chardev=libxl-cmd,mode=control -chardev > > > > socket,id=libxenstat-cmd,path=/var/run/xen/qmp-libxenstat-11,server,nowait > > > > -mon chardev=libxenstat-cmd,mode=control -nodefaults -no-user-config > > > > -xen-attach -name agldemo -vnc none -display none -nographic -object > > > > memory-backend-file,id=mem,size=512M,mem-path=/dev/shm,share=on -numa > > > > node,memdev=mem -chardev socket,path=/opt/agl/vgpu.sock,id=vgpu > > > > -device vhost-user-gpu-pci,chardev=vgpu -machine xenpv -m 513 > > > > > > I am a bit confused what you are trying to do. QEMU has to be started > > > after the domain is created by the toolstack (aka xl). > > > > > > Can you describe a bit more the steps you are using? > > > > This command is solely executed without xl, because I would like to check > > what error happens. With xl, I just got the errors below. > > > > libxl: error: libxl_exec.c:434:spawn_middle_death: domain 14 device > > model [20073]: unexpectedly exited with exit status 0, when we were > > waiting for it to confirm startup > > libxl: error: libxl_dm.c:3102:device_model_spawn_outcome: Domain > > 14:domain 14 device model: spawn failed (rc=-3) > > libxl: error: libxl_dm.c:3322:device_model_postconfig_done: Domain > > 14:Post DM startup configs failed, rc=-3 > > libxl: error: libxl_create.c:1836:domcreate_devmodel_started: Domain > > 14:device model did not start: -3 > > libxl: error: libxl_aoutils.c:646:libxl__kill_xs_path: Device Model > > already exited > > libxl: error: libxl_domain.c:1182:libxl__destroy_domid: Domain > > 14:Non-existant domain > > libxl: error: libxl_domain.c:1136:domain_destroy_callback: Domain > > 14:Unable to destroy guest > > libxl: error: libxl_domain.c:1063:domain_destroy_cb: Domain > > 14:Destruction of domain failed > > > > Is there any way to get the qemu's error output? > > IIRC the qemu's log should be in /var/log/xen/*. As you have a wrapper shell > script, you may be able to re-direct them somewhere different if you can't > find them. Replying to Masami. Before adding any additional parameters like "-device vhost-user-gpu-pci,chardev=vgpu", have you double-checked that you can start regular DomUs successfully? I.e. domU without any special device_model_args? > > > > qemu-system-i386.bin: -object > > > > memory-backend-file,id=mem,size=512M,mem-path=/dev/shm,share=on: > > > > -mem-path not supported with Xen > > > > > > > > If I use the qemu in xen, it doesn't support -mem-path option. Hmm, > > > > memory backend support is not enabled? > > > > > > This is expected. QEMU is only providing emulated/PV device for Xen > > > guest. The memory management is done by Xen itself. When QEMU requires > > > to access the guest memory, then it will have to issue an hypercall. > > > > I got it. That is the most different point from KVM. > > (I'm not sure Xen can overcommit memory for the domains) > > Xen doesn't support overcommitting memory. > > [...] > > > I'm using the Xen and Qemu master branch, just add a fix for uninstall > > error on Xen. > > I am afraid this is not going to be enough to test virtio on Xen on Arm. For > Xen, you would at least need [1]. > > [1] IOREQ feature (+ virtio-mmio) on Arm Yes, exactly. Before you can get something like vhost-user-gpu-pci to work you need the following things: 1) ioreq infrastructure in Xen 2) enable ioreq infrastructure in QEMU (is already there, but unused on ARM) 3) add vhost-user-gpu-pci to QEMU 1) is covered by "IOREQ feature (+ virtio-mmio) on Arm", see https://marc.info/?l=xen-devel&m=160278030131796 2) no patch series has been posted to the list for this, but EPAM mentioned they have a series to enable ioreqs in QEMU for ARM, they used it for testing. As part of this, you might also have to change the machine type to something other than -M xenpv, because xenpv doesn't come with ioreqs or any virtio devices. (More on this below.) 3) I don't know if you can find a way to add the vhost-user-gpu-pci emulator to qemu-system-i386 (--target-list=i386-softmmu), and certainly you cannot easily add it to the xenpv machine because it doesn't come with a virtio transport. But this is something EPAM might have already done as part of their unpublished QEMU series. > For QEMU, I don't think you would be able to use qemu i386 because it is going > to expose x86 device. > > You most likely going to need to enable Xen for qemu arm > and create a new machine that would match Xen layout. Just as a clarification, the issue is not much that qemu i386 is "x86", because there is no x86 emulation being done. The issue is that QEMU comes with two machines we can use: "xenpv" and "xenfv". xenpv is not tied to x86 in any way but also doesn't come with any emulated devices, only Xen PV devices. So unless you add a virtio bus transport to xenpv, it wouldn't work for vhost-user-gpu-pci. xenfv is tied to x86 because it comes with a PIIX emulator. It would be easy to add Virtio devices to it, but we can't really use it on ARM. Either way something needs to be changed. It is probably not a good idea to extend xenpv, because it is good as it is now, without any emulators. Which means that we need to introduce something like xenfv for ARM, maybe based on the aarch64 'virt' generic virtual platform. In which case, it also means that we need to use a different --target-list for QEMU. So far we only used --target-list=i386-softmmu on ARM and x86.

3 years, 5 months

2
2
0 0

Re: [Stratos-dev] Limited memory sharing investigation

by Srivatsa Vaddagiri

* Arnd Bergmann via Stratos-dev <stratos-dev(a)op-lists.linaro.org> [2020-10-12 15:56:01]: > Yes, my rough idea would be to stay as close as possible to the existing > virtio/virtqueue design, but replace the existing descriptors pointing to guest > physical memory with TLV headers describing the data in the ring buffer > itself. This might not actually be possible, but my hope is that we can get > away with making the ring buffer large enough for all data that needs to > be in flight at any time, require all data to be processed in order, I had the impression that virtio allows out-of-order processing of requests i.e requests can be completed not strictly in FIFO order. I think your proposed virtqueue design should still allow that? Also I think reserving 512kB of memory per virtqueue may not be acceptable in all cases. Some device virtqueue could be more busy than other at a given time - the busy virtqueue fills up fast and stalls for more memory to become available, while there is free memory available with other device virtqueues. A global (swiotlb) pool shared among multiple devices would allow for better sharing of available memory in that sense. For memory constrained configurations, a global shared pool may be desirable solution (vs dedicating per-virtqueue buffers of fixed size). -- QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

3 years, 5 months

2
2
0 0

Current status of Xen on MachiatoBin

by Alex Bennée

Hi Julian, Another data point, this time on real HW: Xen 4.15-unstable (c/s Wed Sep 30 12:25:05 2020 +0100 git:e680dde6fd) EFI loader Xen 4.15-unstable (XEN) Xen version 4.15-unstable (alex@lan) (gcc (Debian 8.3.0-6) 8.3.0) debug=y Fri Oct 2 16:18:39 BST 2020 (XEN) Latest ChangeSet: Wed Sep 30 12:25:05 2020 +0100 git:e680dde6fd (XEN) build-id: 05ec585524e05f933d11662012b43576ab57eb63 (XEN) Processor: 410fd081: "ARM Limited", variant: 0x0, part 0xd08, rev 0x1 (XEN) 64-bit Execution: (XEN) Processor Features: 0000000000002222 0000000000000000 (XEN) Exception Levels: EL3:64+32 EL2:64+32 EL1:64+32 EL0:64+32 (XEN) Extensions: FloatingPoint AdvancedSIMD (XEN) Debug Features: 0000000010305106 0000000000000000 (XEN) Auxiliary Features: 0000000000000000 0000000000000000 (XEN) Memory Model Features: 0000000000001124 0000000000000000 (XEN) ISA Features: 0000000000011120 0000000000000000 (XEN) 32-bit Execution: (XEN) Processor Features: 00000131:00011011 (XEN) Instruction Sets: AArch32 A32 Thumb Thumb-2 Jazelle (XEN) Extensions: GenericTimer Security (XEN) Debug Features: 03010066 (XEN) Auxiliary Features: 00000000 (XEN) Memory Model Features: 10201105 40000000 01260000 02102211 (XEN) ISA Features: 02101110 13112111 21232042 01112131 00011142 00011121 (XEN) Using SMC Calling Convention v1.1 (XEN) Using PSCI v1.1 (XEN) SMP: Allowing 4 CPUs (XEN) enabled workaround for: ARM erratum 1319537 (XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 25000 KHz (XEN) GICv2: DT overriding v2m hardware setting (base:160, num:32) (XEN) GICv2m extension register frame: (XEN) gic_v2m_addr=00000000f0280000 (XEN) gic_v2m_size=0000000000001000 (XEN) gic_v2m_spi_base=160 (XEN) gic_v2m_num_spis=32 (XEN) GICv2: DT overriding v2m hardware setting (base:192, num:32) (XEN) GICv2m extension register frame: (XEN) gic_v2m_addr=00000000f0290000 (XEN) gic_v2m_size=0000000000001000 (XEN) gic_v2m_spi_base=192 (XEN) gic_v2m_num_spis=32 (XEN) GICv2: DT overriding v2m hardware setting (base:224, num:32) (XEN) GICv2m extension register frame: (XEN) gic_v2m_addr=00000000f02a0000 (XEN) gic_v2m_size=0000000000001000 (XEN) gic_v2m_spi_base=224 (XEN) gic_v2m_num_spis=32 (XEN) GICv2: DT overriding v2m hardware setting (base:256, num:32) (XEN) GICv2m extension register frame: (XEN) gic_v2m_addr=00000000f02b0000 (XEN) gic_v2m_size=0000000000001000 (XEN) gic_v2m_spi_base=256 (XEN) gic_v2m_num_spis=32 (XEN) GICv2 initialization: (XEN) gic_dist_addr=00000000f0210000 (XEN) gic_cpu_addr=00000000f0220000 (XEN) gic_hyp_addr=00000000f0240000 (XEN) gic_vcpu_addr=00000000f0260000 (XEN) gic_maintenance_irq=25 (XEN) GICv2: Adjusting CPU interface base to 0xf022f000 (XEN) GICv2: 352 lines, 4 cpus, secure (IID 0200143b). (XEN) XSM Framework v1.0.0 initialized (XEN) Initialising XSM SILO mode (XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2) (XEN) Initializing Credit2 scheduler (XEN) load_precision_shift: 18 (XEN) load_window_shift: 30 (XEN) underload_balance_tolerance: 0 (XEN) overload_balance_tolerance: -3 (XEN) runqueues arrangement: socket (XEN) cap enforcement granularity: 10ms (XEN) load tracking window length 1073741824 ns (XEN) Allocated console ring of 32 KiB. (XEN) CPU0: Guest atomics will try 12 times before pausing the domain (XEN) Bringing up CPU1 (XEN) CPU1: Guest atomics will try 11 times before pausing the domain (XEN) CPU 1 booted. (XEN) Bringing up CPU2 (XEN) CPU2: Guest atomics will try 10 times before pausing the domain (XEN) CPU 2 booted. (XEN) Bringing up CPU3 (XEN) CPU3: Guest atomics will try 10 times before pausing the domain (XEN) Brought up 4 CPUs (XEN) CPU 3 booted. (XEN) Data Abort Trap. Syndrome=0x5 (XEN) Walking Hypervisor VA 0x800440000000 on CPU0 via TTBR 0x00000000aac72000 (XEN) 0TH[0x100] = 0x00400000aac70f7f (XEN) 1ST[0x11] = 0x0000000000000000 (XEN) CPU0: Unexpected Trap: Data Abort (XEN) ----[ Xen-4.15-unstable arm64 debug=y Not tainted ]---- (XEN) CPU: 0 (XEN) PC: 000000000025ed4c strlen+0x10/0x84 (XEN) LR: 00000000002030f8 (XEN) SP: 00000000002ffcf0 (XEN) CPSR: 60000249 MODE:64-bit EL2h (Hypervisor, handler) (XEN) X0: 000080043ee00000 X1: 0000800440000000 X2: ffffffffffffffff (XEN) X3: ffffffffffffffff X4: 0000000000000001 X5: 0000000000000000 (XEN) X6: 0000000000000000 X7: fefefefefefefefe X8: ffffffffffffffff (XEN) X9: fefefefefefefefe X10: ffffffffffffffff X11: 0101010101010101 (XEN) X12: 0000000000000038 X13: 00000000002883b8 X14: 00000000002ffb48 (XEN) X15: 0000000000000020 X16: 0000000000000000 X17: 00000000002b1000 (XEN) X18: 00000000002b2000 X19: 000080043ee00000 X20: 000000000029cb40 (XEN) X21: 000080042ffde5d0 X22: 0000000000000001 X23: 0000000000000000 (XEN) X24: 00000000002a23d0 X25: 0000000440000000 X26: 000000000000002d (XEN) X27: 00000000002e0910 X28: 00000000003fe440 FP: 00000000002ffcf0 (XEN) (XEN) VTCR_EL2: 80000000 (XEN) VTTBR_EL2: 0000000000000000 (XEN) (XEN) SCTLR_EL2: 30cd183d (XEN) HCR_EL2: 0000000000000038 (XEN) TTBR0_EL2: 00000000aac72000 (XEN) (XEN) ESR_EL2: 96000005 (XEN) HPFAR_EL2: 0000000000000000 (XEN) FAR_EL2: 0000800440000000 (XEN) (XEN) Xen stack trace from sp=00000000002ffcf0: (XEN) 00000000002ffd20 000000000020404c 00000000002a5448 0000000000000001 (XEN) 00000000002ffd40 f11e576000204150 00000000002ffd50 00000000002c417c (XEN) 00000000002b25c0 0000000000000001 00000000002b2620 000080042ffde5d0 (XEN) 00000000002ffd90 00000000002bdbe0 000080042ffde5d0 0000000000000001 (XEN) 00000000002ddc68 0000000000000000 0000000000000004 00000000ffffffc8 (XEN) 00000000002ffdd0 00000000002bf09c 0000000000000004 0000000000000004 (XEN) 00000000002b0580 000000000033e430 0101010101010101 0000000000000000 (XEN) 00000000002ffdf0 00000000002cbaec 0000000000000004 0000000000000004 (XEN) 000000003f8790a0 00000000002001b8 00000000aab35000 00000000aa935000 (XEN) 00000000b2273000 0000000000000000 0000000000400000 00000000b22ad018 (XEN) 00000000aabe7138 0000000000000001 0000000000000001 8000000000000002 (XEN) 0000000000000000 00000000002e1948 00000000b2273000 0000000000009000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000300000000 (XEN) 0000000000000000 00000040ffffffff 00000000ffffffff 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 (XEN) Xen call trace: (XEN) [<000000000025ed4c>] strlen+0x10/0x84 (PC) (XEN) [<00000000002030f8>] dt_device_is_compatible+0x48/0x84 (LR) (XEN) [<000000000020404c>] dt_match_node+0x70/0x10c (XEN) [<00000000002c417c>] device_init+0x90/0xdc (XEN) [<00000000002bdbe0>] iommu_hardware_setup+0x5c/0x188 (XEN) [<00000000002bf09c>] iommu_setup+0x30/0x188 (XEN) [<00000000002cbaec>] start_xen+0xac4/0xc88 (XEN) [<00000000002001b8>] arm64/head.o#primary_switched+0x10/0x30 (XEN) (XEN) (XEN) **************************************** (XEN) Panic on CPU 0: (XEN) CPU0: Unexpected Trap: Data Abort (XEN) **************************************** (XEN) (XEN) Reboot in five seconds... On the real HW it does actually reboot ;-) -- Alex Bennée

3 years, 5 months

3
12
0 0

Re: [Stratos-dev] [virtio-comment] [PATCH] content: Reserve virtio-watchog device ID

by Alex Bennée

Rob Bradford <robert.bradford(a)intel.com> writes: > Reserve an ID for a watchdog device which may be used to ensure that the > guest is responsive. This is equivalent of a hardware watchdog device > and will trigger the reboot of the guest if the the host does not > periodic ping from the the guest. Out of interest is your watchdog device also going to allow for straight power/reset control? The use case I'm thinking of is QEMU's -m virt machine which is a purely virtual machine which is a target for a number of firmware/bootcode projects. QEMU provides a firmware emulation that will do the right thing if you issue PSCI supervisor calls and you have directly loaded a kernel. However if you are testing firmware - for example EDK2 or Arm Trusted Firmware it is expected to make the final hw appropriate twiddle to reboot the machine and of course that is undefined for -m virt. > Signed-off-by: Rob Bradford <robert.bradford(a)intel.com> Reviewed-by: Alex Bennée <alex.bennee(a)linaro.org> -- Alex Bennée

3 years, 5 months

1
0
0 0

Re: [Stratos-dev] My current status of Xen on QEMU TCG (-m virt)

by Masami Hiramatsu

Hi, Now I'm trying to use vhost-user-gpu. I built qemu and xen and installed user /usr/local/. Xen installed it's qemu in /usr/local/lib/xen/bin, Qemu was installed in /usr/local/bin, so those are not conflict. If I added device_model_args= option to the domU.conf file, it didn't work (failed to exec qemu). I also added device_model_override= but it didn't change neither. But in both cases, it seems vhost-user-gpu got access from qemu. To clarify what argument passed to the qemu, I hacked /usr/local/lib/xen/bin/qemu-system-i386, and found that the arguments passed via device_model_args=[] were just added to the original arguments. OK, now I'm trying to test the qemu-system-i386 itself as below mhiramat@develbox:/opt/agl$ sudo /usr/local/lib/xen/bin/qemu-system-i386.bin -xen-domid 11 -no-shutdown -chardev socket,id=libxl-cmd,path=/var/run/xen/qmp-libxl-11,server,nowait -mon chardev=libxl-cmd,mode=control -chardev socket,id=libxenstat-cmd,path=/var/run/xen/qmp-libxenstat-11,server,nowait -mon chardev=libxenstat-cmd,mode=control -nodefaults -no-user-config -xen-attach -name agldemo -vnc none -display none -nographic -object memory-backend-file,id=mem,size=512M,mem-path=/dev/shm,share=on -numa node,memdev=mem -chardev socket,path=/opt/agl/vgpu.sock,id=vgpu -device vhost-user-gpu-pci,chardev=vgpu -machine xenpv -m 513 qemu-system-i386.bin: -object memory-backend-file,id=mem,size=512M,mem-path=/dev/shm,share=on: -mem-path not supported with Xen If I use the qemu in xen, it doesn't support -mem-path option. Hmm, memory backend support is not enabled? mhiramat@develbox:/opt/agl$ sudo /usr/local/bin/qemu-system-i386 -xen-domid 11 -no-shutdown -chardev socket,id=libxl-cmd,path=/var/run/xen/qmp-libxl-11,server,nowait -mon chardev=libxl-cmd,mode=control -chardev socket,id=libxenstat-cmd,path=/var/run/xen/qmp-libxenstat-11,server,nowait -mon chardev=libxenstat-cmd,mode=control -nodefaults -no-user-config -xen-attach -name agldemo -vnc none -display none -nographic -object memory-backend-file,id=mem,size=512M,mem-path=/dev/shm,share=on -numa node,memdev=mem -chardev socket,path=/opt/agl/vgpu.sock,id=vgpu -device vhost-user-gpu-pci,chardev=vgpu -machine xenpv -m 513 qemu-system-i386: -xen-domid 11: Option not supported for this target With the qemu which I built, it seems it doesn't support xen... but I'm sure the qemu has to be configured to support xen as below mhiramat@develbox:~/ksrc/qemu$ grep XEN build/config-host.mak CONFIG_XEN_BACKEND=y CONFIG_XEN_CTRL_INTERFACE_VERSION=41500 XEN_CFLAGS=-I/usr/local/include XEN_LIBS=-L/usr/local/lib -lxenctrl -lxenstore -lxenguest -lxenforeignmemory -lxengnttab -lxenevtchn -lxendevicemodel -lxentoolcore Thanks, -- Masami Hiramatsu

3 years, 5 months

3
6
0 0

My current status of Xen on QEMU TCG (-m virt)

by Alex Bennée

Hi, As it's Friday I thought I'd better summarise where I've gotten to with booting Xen. TLDR, got something working with GICv2, GICv3 seems broken. Platform: QEMU with: -cpu cortex-a57 -machine type=virt,virtualization=on,gic-version=3 \ -device virtio-net-pci,netdev=unet,id=virt-net,disable-legacy=on \ -device virtio-scsi-pci,id=virt-scsi,disable-legacy=on \ -blockdev driver=raw,node-name=hd,discard=unmap,file.driver=host_device,file.filename=/dev/zen-disk/debian-buster-arm64 \ -device scsi-hd,drive=hd,id=virt-scsi-hd \ -drive if=pflash,file=/usr/share/AAVMF/AAVMF_CODE.fd,format=raw,readonly \ -drive if=pflash,file=/home/alex/models/qemu-arm64-efivars,format=raw The firmware is the stock Debian EFI: Package: qemu-efi-aarch64 Status: install ok installed Installed-Size: 133154 Maintainer: Debian QEMU Team <pkg-qemu-devel(a)lists.alioth.debian.org> Architecture: all Source: edk2 Version: 0~20181115.85588389-3+deb10u1 Description: UEFI firmware for 64-bit ARM virtual machines qemu-efi-aarch64 is a build of EDK II for 64-bit ARM virtual machines. It includes full support for UEFI, including Secure Boot. Homepage: http://www.tianocore.org I've also turned on semihosting (-semihosting) to get earlycon output. Xen: Current master with the following patches: e680dde6fd (HEAD -> review/arm-xen-acpi-changes) xen/arm: gic-v2: acpi: Use the correct length for the GICC structure a34a93acb9 xen/arm: acpi: add BAD_MADT_GICC_ENTRY() macro f66afeb018 (linaro/review/arm-xen-acpi-changes) xen/arm: acpi: relax FADT revision requirements to 5.1 72f3d495d0 xen/arm: Introduce fw_unreserved_regions() and use it 8d447fa48a xen/arm: Check if the platform is not using ACPI before initializing Dom0less fa03236a08 xen/arm: acpi: The fixmap area should always be cleared during failure/unmap 505b82a8be xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory() I've tried various kernels: - The default buster kernel (4.19.0-11) would panic in memblock_alloc_base - The backports kernel (5.8.0-0.bpo.2) didn't panic AFAICT but never came up either In the end I built my own image so I could enable semihosting support and finally I was able to get some output from the kernel: ing Linux on physical CPU 0x0000000000 [0x411fd070] Linux version 5.8.11-ajb-00010-ge059ce7b131b (alex@zen) (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0, GNU ld (GNU Binutils for Debian) 2.31.1) #42 SMP Fri Oct 2 14:19:57 ST 2020 earlycon: smh0 at I/O port 0x0 (options '') printk: bootconsole [smh0] enabled Xen XEN_VERSION.XEN_SUBVERSION support found efi: EFI v2.50 by Xen efi: ACPI 2.0=0xf7e90328 cma: Reserved 16 MiB at 0x00000000bf000000 ACPI: Early table checksum verification disabled ACPI: RSDP 0x00000000F7E90328 000024 (v02 BOCHS ) ACPI: XSDT 0x00000000F7E902D0 000054 (v01 BOCHS BXPCFACP 00000001 01000013) ACPI: FACP 0x00000000F7E90000 00010C (v05 BOCHS BXPCFACP 00000001 BXPC 00000001) ACPI: DSDT 0x0000000138520000 0014A2 (v02 BOCHS BXPCDSDT 00000001 BXPC 00000001) ACPI: APIC 0x00000000F7E90110 000194 (v03 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: GTDT 0x00000001384F0000 000060 (v02 BOCHS BXPCGTDT 00000001 BXPC 00000001) ACPI: MCFG 0x00000001384E0000 00003C (v01 BOCHS BXPCMCFG 00000001 BXPC 00000001) ACPI: SPCR 0x00000001384D0000 000050 (v02 BOCHS BXPCSPCR 00000001 BXPC 00000001) ACPI: STAO 0x00000000F7E902A8 000025 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: SPCR: console: pl011,mmio,0x9000000,9600 Zone ranges: DMA [mem 0x0000000040000000-0x000000007fffffff] DMA32 [mem 0x0000000080000000-0x00000000ffffffff] Normal [mem 0x0000000100000000-0x000000013856ffff] Movable zone start for each node Early memory node ranges node 0: [mem 0x0000000040000000-0x00000000bfffffff] node 0: [mem 0x00000000f7e90000-0x00000000f7e90fff] node 0: [mem 0x00000001384d0000-0x000000013852ffff] node 0: [mem 0x0000000138550000-0x000000013856ffff] Initmem setup node 0 [mem 0x0000000040000000-0x000000013856ffff] psci: probing for conduit method from ACPI. psci: PSCIv1.1 detected in firmware. psci: Using standard PSCI v0.2 function IDs psci: Trusted OS migration not required psci: SMC Calling Convention v1.1 ACPI: [APIC:0x0b] Invalid zero length percpu: Embedded 22 pages/cpu s51288 r8192 d30632 u90112 Detected PIPT I-cache on CPU0 CPU features: detected: ARM erratum 832075 CPU features: detected: GIC system register CPU interface CPU features: detected: ARM erratum 834220 CPU features: detected: EL2 vector hardening CPU features: detected: Speculative Store Bypass Disable CPU features: detected: ARM errata 1165522, 1319367, or 1530923 Built 1 zonelists, mobility grouping on. Total pages: 516222 Kerel command line: placeholder root=/dev/sda2 ro console=hvc0 earlycon=smh Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) mem auto-init: stack:off, heap alloc:off, heap free:off software IO TLB: mapped [mem 0x7bfff000-0x7ffff000] (64MB) Memory: 1927524K/2097668K available (8700K kernel code, 536K rwdata, 3280K rodata, 1536K init, 248K bss, 153760K reserved, 16384K cma-reserved) SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1 rcu: Hierarchical RCU implementation. rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2. rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2 NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0e] Invalid zero length ACPI: [APIC:0x0b] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length Kernel panic - not syncing: No interrupt controller found. CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.8.11-ajb-00010-ge059ce7b131b #42 Call trace: dump_backtrace+0x0/0x1a8 show_stack+0x18/0x28 dump_stack+0xc8/0x128 panic+0x154/0x30c init_IRQ+0xd0/0xe8 start_kernel+0x260/0x400 ---[ end Kernel panic - not syncing: No interrupt controller found. ]--- If I boot the same kernel with: -cpu cortex-a57 -machine type=virt,virtualization=on,gic-version=2 We get a lot further: Booting Linux on physical CPU 0x0000000000 [0x411fd070] Linux version 5.8.11-ajb-00010-ge059ce7b131b (alex@zen) (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0, GNU ld (GNU Binutils for Debian) 2.31.1) #42 SMP Fri Oct 2 14:19:57 BST 2020 earlycon: smh0 at I/O port 0x0 (options '') printk: bootconsole [smh0] enabled Xen XEN_VERSION.XEN_SUBVERSION support found efi: EFI v2.50 by Xen efi: ACPI 2.0=0xf7e90308 cma: Reserved 16 MiB at 0x00000000bf000000 ACPI: Early table checksum verification disabled ACPI: RSDP 0x00000000F7E90308 000024 (v02 BOCHS ) ACPI: XSDT 0x00000000F7E902B0 000054 (v01 BOCHS BXPCFACP 00000001 01000013) ACPI: FACP 0x00000000F7E90000 00010C (v05 BOCHS BXPCFACP 00000001 BXPC 00000001) ACPI: DSDT 0x0000000138520000 0014A2 (v02 BOCHS BXPCDSDT 00000001 BXPC 00000001) ACPI: APIC 0x00000000F7E90110 000174 (v03 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: GTDT 0x00000001384F0000 000060 (v02 BOCHS BXPCGTDT 00000001 BXPC 00000001) ACPI: MCFG 0x00000001384E0000 00003C (v01 BOCHS BXPCMCFG 00000001 BXPC 00000001) ACPI: SPCR 0x00000001384D0000 000050 (v02 BOCHS BXPCSPCR 00000001 BXPC 00000001) ACPI: STAO 0x00000000F7E90288 000025 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: SPCR: console: pl011,mmio,0x9000000,9600 Zone ranges: DMA [mem 0x0000000040000000-0x000000007fffffff] DMA32 [mem 0x0000000080000000-0x00000000ffffffff] Normal [mem 0x0000000100000000-0x000000013856ffff] Movable zone start for each node Early memory node ranges node 0: [mem 0x0000000040000000-0x00000000bfffffff] node 0: [mem 0x00000000f7e90000-0x00000000f7e90fff] node 0: [mem 0x00000001384d0000-0x000000013852ffff] node 0: [mem 0x0000000138550000-0x000000013856ffff] Initmem setup node 0 [mem 0x0000000040000000-0x000000013856ffff] psci: probing for conduit method from ACPI. psci: PSCIv1.1 detected in firmware. psci: Using standard PSCI v0.2 function IDs psci: Trusted OS migration not required psci: SMC Calling Convention v1.1 percpu: Embedded 22 pages/cpu s51288 r8192 d30632 u90112 Detected PIPT I-cache on CPU0 CPU features: detected: ARM erratum 832075 CPU features: detected: ARM erratum 834220 CPU features: detected: EL2 vector hardening CPU features: detected: Speculative Store Bypass Disable CPU features: detected: ARM errata 1165522, 1319367, or 1530923 Built 1 zonelists, mobility grouping on. Total pages: 516222 Kernel command line: placeholder root=/dev/sda2 ro console=hvc0 earlycon=smh printk: log_buf_len individual max cpu contribution: 4096 bytes printk: log_buf_len total cpu_extra contributions: 12288 bytes printk: log_buf_len min size: 16384 bytes printk: log_buf_len: 32768 bytes printk: early log buf free: 12432(75%) Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) mem auto-init: stack:off, heap alloc:off, heap free:off software IO TLB: mapped [mem 0x7bfff000-0x7ffff000] (64MB) Memory: 1927316K/2097668K available (8700K kernel code, 536K rwdata, 3280K rodata, 1536K init, 248K bss, 153968K reserved, 16384K cma-reserved) SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 rcu: Hierarchical RCU implementation. rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=4. rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 rcu: Offload RCU callbacks from CPUs: (none). random: get_random_bytes called from start_kernel+0x2a4/0x400 with crng_init=0 arch_timer: cp15 timer(s) running at 62.50MHz (virt). clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x1cd42e208c, max_idle_ns: 881590405314 ns sched_clock: 56 its at 62MHz, resolution 16ns, wraps every 4398046511096ns Console: colour dummy device 80x25 printk: console [hvc0] enabled printk: bootconsole [smh0] disabled which completes the Buster boot and allows me to ssh into Dom0 - I haven't yet tried to boot a DomU. -- Alex Bennée

3 years, 6 months

4
10
0 0

2024

2023

2022

2021

2020

Stratos-dev October 2020