Hi Alex and all,

you might find useful a little project called ImageBuilder that generates a U-Boot boot.scr script to load Xen, Dom0 and Dom0less VMs (if specified) via TFTP or MMC/SATA:

https://gitlab.com/ViryaOS/imagebuilder

See the example in the README file. It makes working with Xen, Dom0less and U-Boot a lot easier! It can also be called from Yocto. (I know we use it from Yocto at Xilinx but I am not sure if upstream Yocto does the same.)

An example config file looks like this:

--- MEMORY_START="0x0" MEMORY_END="0x80000000" XEN="xen" DOM0_KERNEL="Image" DEVICE_TREE="mpsoc.dtb" DOM0_RAMDISK="rootfs.cpio.gz"

XEN_CMD=console="dtuart dtuart=serial0 dom0_mem=1600M dom0_max_vcpus=2 vwfi=native sched=null" DOM0_CMD="console=hvc0 earlycon=xen earlyprintk=xen clk_ignore_unused root=/dev/ram0"

NUM_DOMUS="0" UBOOT_SOURCE="boot.source" UBOOT_SCRIPT="boot.scr" ---

Then you can generate boot.scr as follows:

bash imagebuilder/scripts/uboot-script-gen -t tftp -d . -c config

Cheers,

Stefano

On Fri, 15 Oct 2021, Alex Bennée wrote:

Hi,

I've dumped a bunch of files on my Stratos file share:

https://fileserver.linaro.org/s/jC3Xq2q297j5bQ8

The first directory (xenpi4.boot) is the result of a Yocto demo build (as per Christopher Clark's instructions) and contains a known good version of the firmware, u-boot and with the rest of the image would boot a Yocto distro image.

As I like understand the full process and I wanted to have a full self hosted distro set up for development I set up a TFTP based setup where u-boot gets the hypervisor, kernel and DTB over the network. Most of the work it turned out was in finding the correct u-boot runes to both subtly modify the DTB to account for HW variations (for some reason I couldn't get u-boot to modify the master DTB from the GPU firmware). We also need to make some modifications to /chosen nodes to ensure Xen boots up out DomO kernel.

The source of the script (ajb.configs/boot-xenpi.source) is a heavily commented boot script that will give you the blow-by-blow details. With this I can boot my own custom hypervisor and kernel into a Debian Bullseye install. You can find known working configs for those components in the directory as well.

My distro is running a xen-upstream.deb built from the Xen source tree:

make -j9 debball CROSS_COMPILE=aarch64-linux-gnu- XEN_TARGET_ARCH=arm64

I've also hacked the distro a little:

• removed the kernels and raspi-firmware packages (so as not to mess with /boot/firmware)
• don' auto-mount /boot.firmware
• hand built a QEMU which is pointed to by QEMU_XEN in /etc/default/xencommons

Be aware the Dom0 has 1G of the RAM by default, if you plan to do some heavy building you might want to give it a bit more memory to play with.

I'll upload my test image and xen config in due course but this should give something that is fairly easy to iterate with. I've not yet run rustup on the pi itself though ;-)

-- Alex Bennée