- Stratos-dev - op-lists.linaro.org

[RFC QEMU] docs: vhost-user: Add custom memory mapping support

by Viresh Kumar

The current model of memory mapping at the back-end works fine with Qemu, where a standard call to mmap() for the respective file descriptor, passed from front-end, is generally all we need to do before the front-end can start accessing the guest memory. There are other complex cases though, where we need more information at the backend and need to do more than just an mmap() call. For example, Xen, a type-1 hypervisor, currently supports memory mapping via two different methods, foreign-mapping (via /dev/privcmd) and grant-dev (via /dev/gntdev). In both these cases, the back-end needs to call mmap() followed by an ioctl() (or vice-versa), and need to pass extra information via the ioctl(), like the Xen domain-id of the guest whose memory we are trying to map. Add a new protocol feature, 'VHOST_USER_PROTOCOL_F_CUSTOM_MMAP', which lets the back-end know about the additional memory mapping requirements. When this feature is negotiated, the front-end can send the 'VHOST_USER_CUSTOM_MMAP' message type to provide the additional information to the back-end. Signed-off-by: Viresh Kumar <viresh.kumar(a)linaro.org> --- docs/interop/vhost-user.rst | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst index 3f18ab424eb0..f2b1d705593a 100644 --- a/docs/interop/vhost-user.rst +++ b/docs/interop/vhost-user.rst @@ -258,6 +258,23 @@ Inflight description :queue size: a 16-bit size of virtqueues +Custom mmap description +^^^^^^^^^^^^^^^^^^^^^^^ + ++-------+-------+ +| flags | value | ++-------+-------+ + +:flags: 64-bit bit field + +- Bit 0 is Xen foreign memory access flag - needs Xen foreign memory mapping. +- Bit 1 is Xen grant memory access flag - needs Xen grant memory mapping. + +:value: a 64-bit hypervisor specific value. + +- For Xen foreign or grant memory access, this is set with guest's xen domain + id. + C structure ----------- @@ -867,6 +884,7 @@ Protocol features #define VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS 14 #define VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS 15 #define VHOST_USER_PROTOCOL_F_STATUS 16 + #define VHOST_USER_PROTOCOL_F_CUSTOM_MMAP 17 Front-end message types ----------------------- @@ -1422,6 +1440,20 @@ Front-end message types query the back-end for its device status as defined in the Virtio specification. +``VHOST_USER_CUSTOM_MMAP`` + :id: 41 + :equivalent ioctl: N/A + :request payload: Custom mmap description + :reply payload: N/A + + When the ``VHOST_USER_PROTOCOL_F_CUSTOM_MMAP`` protocol feature has been + successfully negotiated, this message is submitted by the front-end to + notify the back-end of the special memory mapping requirements, that the + back-end needs to take care of, while mapping any memory regions sent + over by the front-end. The front-end must send this message before + any memory-regions are sent to the back-end via ``VHOST_USER_SET_MEM_TABLE`` + or ``VHOST_USER_ADD_MEM_REG`` message types. + Back-end message types ---------------------- -- 2.31.1.272.g89b43f80a514

1 year, 9 months

1
0
0 0

[Discussion] Xen grants and access permissions

by Viresh Kumar

Hi Oleksandr, As you already know, I am looking at how we can integrate the Xen grants work in our implementation of Rust based Xen vhost frontend [1]. The hypervisor independent vhost-user backends [2] talk to xen-vhost-frontend using the standard vhost-user protocol [3]. Every memory region that the backends get access to are sent to it by the frontend as memory region descriptors, which contain only address and size information and lack any permission flags. I noticed that with Xen grants, there are strict memory access restrictions, where a memory region may be marked READ only and we can't map it as RW anymore, trying that just fails. Because the standard vhost-user protocol doesn't have any permission flags, the vhost libraries (in Rust) can't do anything else but try to map everything as RW. I am wondering how do I proceed on this as I am very much stuck here. -- viresh [1] https://github.com/vireshk/xen-vhost-frontend [2] https://github.com/rust-vmm/vhost-device [3] https://qemu.readthedocs.io/en/latest/interop/vhost-user.html [4] https://qemu.readthedocs.io/en/latest/interop/vhost-user.html#memory-region…

1 year, 9 months

3
2
0 0

Xen grants setup

by Viresh Kumar

Hi Oleksandr, Finally I am getting around Xen grants and haven't got a running setup yet. There are few questions I have at the moment: - Xen's libxl_arm.c creates the iommu nodes only if backend isn't in Dom0. Why are we forcing it this way ? I am not running my backend in a separate dom as of now, as they need to share a unix socket with dom0 (with vhost-user-fronend (our virtio-disk counterpart)) for vhost-user protocol and am not sure how to set it up. Maybe I need to use "channel" ? or something else ? - I tried to hack it up, to keep backend in Dom0 only and create the iommu nodes unconditionally and the guest kernel is crashing in drivers/iommu/iommu.c:332 iommu_dev = ops->probe_device(dev); Since grant_dma_iommu_ops have all the fields set to NULL. - Anything else you might want to share ? -- viresh

1 year, 10 months

2
4
0 0

Looking for examples of vhost-user backends using VHOST_USER_SLAVE_IOTLB_MSG

by Alex Bennée

Hi, As part of looking at implementing vhost-user daemons which don't have complete access to a guests address space we noticed the vhost-user spec has a definition for VHOST_USER_SLAVE_IOTLB_MSG's. Specifically the message VHOST_IOTLB_ACCESS_FAIL looks like it could be used to delegate the mapping of memory by the master/frontend so the backend can access it. As far as I can see no backends currently use this message and the specific handling of VHOST_IOTLB_ACCESS_FAIL in QEMU doesn't do much more than report an error. The VHOST_IOTLB_MISS does have some handling but the commentary seems to imply this is needed for the in kernel vhost kernel support (perhaps when real hardware is filling in a buffer being forwarded to a VirtIO device?). Can anyone point to any backends that implement these messages? We have implemented a Xen Vhost User Frontend: https://github.com/vireshk/xen-vhost-frontend which currently uses a lightly hacked Xen privcmd device to map all of the guests memory. We want to investigate using the stricter gntdev device where buffers for individual transactions can be mapped into the backend domains before being released at the end of the transaction. We want to keep the hypervisor specific code in the frontend so the backend can stay portable between different hypervisors. Thanks, -- Alex Bennée Virtualisation Tech Lead @ Linaro

1 year, 10 months

1
1
0 0

[PATCH] libxl: virtio: Fix build error for 32-bit platforms

by Viresh Kumar

The field 'base' in 'struct libxl_device_virtio' is defined as uint64, while we are printing it with '%lu', which is 32bit only 32-bit platforms. And so generates a error like: libxl_internal.h:4388:51: error: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'uint64_t' {aka 'long long unsigned int'} [-Werror=format=] Fix the same by using PRIx64 instead. Now that the base name is available in hexadecimal format, prefix it with '0x' as well, which strtoul() also depends upon since base passed is 0. Fixes: 43ba5202e2ee ("libxl: add support for generic virtio device") Signed-off-by: Viresh Kumar <viresh.kumar(a)linaro.org> --- Couldn't test on 32-bit platforms yet, but works fine for 64 bit one. tools/libs/light/libxl_virtio.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/libs/light/libxl_virtio.c b/tools/libs/light/libxl_virtio.c index 6a38def2faf5..2217bda8a253 100644 --- a/tools/libs/light/libxl_virtio.c +++ b/tools/libs/light/libxl_virtio.c @@ -45,12 +45,12 @@ static int libxl__set_xenstore_virtio(libxl__gc *gc, uint32_t domid, const char *transport = libxl_virtio_transport_to_string(virtio->transport); flexarray_append_pair(back, "irq", GCSPRINTF("%u", virtio->irq)); - flexarray_append_pair(back, "base", GCSPRINTF("%lu", virtio->base)); + flexarray_append_pair(back, "base", GCSPRINTF("0x%"PRIx64, virtio->base)); flexarray_append_pair(back, "type", GCSPRINTF("%s", virtio->type)); flexarray_append_pair(back, "transport", GCSPRINTF("%s", transport)); flexarray_append_pair(front, "irq", GCSPRINTF("%u", virtio->irq)); - flexarray_append_pair(front, "base", GCSPRINTF("%lu", virtio->base)); + flexarray_append_pair(front, "base", GCSPRINTF("0x%"PRIx64, virtio->base)); flexarray_append_pair(front, "type", GCSPRINTF("%s", virtio->type)); flexarray_append_pair(front, "transport", GCSPRINTF("%s", transport)); -- 2.31.1.272.g89b43f80a514

1 year, 12 months

4
8
0 0

[PATCH V10 0/3] toolstack support for generic virtio devices on Arm

by Viresh Kumar

Hello, This patchset adds toolstack support for I2C, GPIO and generic virtio devices. This is inspired from the work done by Oleksandr for the Disk device. This is developed as part of Linaro's Project Stratos, where we are working towards Hypervisor agnostic Rust based backends [1]. This is based of Xen's master branch. V9->V10: - Swap I2C/GPIO compatible strings, they were wrongly added earlier. - Arrange tags in timely order and add new ones. V8->V9: - Drop changes to tools/ocaml/libs/xl/genwrap.py file. - Replace GPIO with I2C in a comment. - Add Reviewed-by tags. V7->V8: - Use macros for compatible string names. - Use strcmp() instead of strncmp() at several places. - Rename "virtio,devices" to "virtio,device" in commit log. - Remove extra call to fdt_end_node(). - Disallow "unknown" in xenstore transport. - Use libxl__strdup(). - Update documentation. - Remove DEVICE_ADDREMOVE(virtio). V6->V7: - Support generic virtio devices too. They are passed with type=virtio,device, and we only create the MMIO DT node for them. - Add links to DT bindings of I2C and GPIO, in code and documentation. - Call libxl__device_add() for all hypervisor types. - Add (0, "UNKNOWN") for libxl_virtio_transport. - Removed libxl_virtio_backend and libxl_virtioinfo, as they were unused. - Remove unnecessary stuff from libxl__virtio_from_xenstore() and add support for type and transport. - Add backend=domid in documentation and replace compatible with type. - Improved commit logs. V5->V6: - The cleanup patches are sent separately [2]. - We don't add I2C or GPIO specific device changes anymore, rather we create generic "virtio" devices. The "type" of a virtio devices helps us identify the right device, and create an entry in the DT node. The same can be used for all Virtio devices now. - Update man page xl.cfg. V4->V5: - Fixed indentation at few places. - Removed/added blank lines. - Added few comments. - Added review tags from Oleksandr. - Rebased over latest staging branch. V3->V4: - Update virtio_enabled independently of all devices, so we don't miss setting it to true. - Add iommu handling for i2c/gpio and move it as part of make_virtio_mmio_node_common(), which gets backend_domid parameter as a result. V2->V3: - Rebased over latest tree and made changes according to changes in Oleksandr's patches from sometime back. - Minor cleanups. V1->V2: - Patches 3/6 and 4/6 are new. - Patches 5/6 and 6/6 updated based on the above two patches. - Added link to the bindings for I2C and GPIO. - Rebased over latest master branch. Thanks. -- Viresh [1] https://lore.kernel.org/xen-devel/20220414092358.kepxbmnrtycz7mhe@vireshk-i… Viresh Kumar (3): libxl: Add support for generic virtio device xl: Add support to parse generic virtio device docs: Add documentation for generic virtio devices docs/man/xl.cfg.5.pod.in | 33 +++++ tools/libs/light/Makefile | 1 + tools/libs/light/libxl_arm.c | 100 +++++++++++++++ tools/libs/light/libxl_create.c | 4 + tools/libs/light/libxl_internal.h | 6 + tools/libs/light/libxl_types.idl | 18 +++ tools/libs/light/libxl_types_internal.idl | 1 + tools/libs/light/libxl_virtio.c | 144 ++++++++++++++++++++++ tools/xl/xl_parse.c | 81 ++++++++++++ 9 files changed, 388 insertions(+) create mode 100644 tools/libs/light/libxl_virtio.c -- 2.31.1.272.g89b43f80a514

1 year, 12 months

4
8
0 0

[PATCH V9 0/3] toolstack support for generic virtio devices on Arm

by Viresh Kumar

Hello, This patchset adds toolstack support for I2C, GPIO and generic virtio devices. This is inspired from the work done by Oleksandr for the Disk device. This is developed as part of Linaro's Project Stratos, where we are working towards Hypervisor agnostic Rust based backends [1]. This is based of Xen's master branch. V8->V9: - Drop changes to tools/ocaml/libs/xl/genwrap.py file. - Replace GPIO with I2C in a comment. - Add Reviewed-by tags. V7->V8: - Use macros for compatible string names. - Use strcmp() instead of strncmp() at several places. - Rename "virtio,devices" to "virtio,device" in commit log. - Remove extra call to fdt_end_node(). - Disallow "unknown" in xenstore transport. - Use libxl__strdup(). - Update documentation. - Remove DEVICE_ADDREMOVE(virtio). V6->V7: - Support generic virtio devices too. They are passed with type=virtio,device, and we only create the MMIO DT node for them. - Add links to DT bindings of I2C and GPIO, in code and documentation. - Call libxl__device_add() for all hypervisor types. - Add (0, "UNKNOWN") for libxl_virtio_transport. - Removed libxl_virtio_backend and libxl_virtioinfo, as they were unused. - Remove unnecessary stuff from libxl__virtio_from_xenstore() and add support for type and transport. - Add backend=domid in documentation and replace compatible with type. - Improved commit logs. V5->V6: - The cleanup patches are sent separately [2]. - We don't add I2C or GPIO specific device changes anymore, rather we create generic "virtio" devices. The "type" of a virtio devices helps us identify the right device, and create an entry in the DT node. The same can be used for all Virtio devices now. - Update man page xl.cfg. V4->V5: - Fixed indentation at few places. - Removed/added blank lines. - Added few comments. - Added review tags from Oleksandr. - Rebased over latest staging branch. V3->V4: - Update virtio_enabled independently of all devices, so we don't miss setting it to true. - Add iommu handling for i2c/gpio and move it as part of make_virtio_mmio_node_common(), which gets backend_domid parameter as a result. V2->V3: - Rebased over latest tree and made changes according to changes in Oleksandr's patches from sometime back. - Minor cleanups. V1->V2: - Patches 3/6 and 4/6 are new. - Patches 5/6 and 6/6 updated based on the above two patches. - Added link to the bindings for I2C and GPIO. - Rebased over latest master branch. Thanks. -- Viresh [1] https://lore.kernel.org/xen-devel/20220414092358.kepxbmnrtycz7mhe@vireshk-i… Viresh Kumar (3): libxl: Add support for generic virtio device xl: Add support to parse generic virtio device docs: Add documentation for generic virtio devices docs/man/xl.cfg.5.pod.in | 33 +++++ tools/libs/light/Makefile | 1 + tools/libs/light/libxl_arm.c | 100 +++++++++++++++ tools/libs/light/libxl_create.c | 4 + tools/libs/light/libxl_internal.h | 6 + tools/libs/light/libxl_types.idl | 18 +++ tools/libs/light/libxl_types_internal.idl | 1 + tools/libs/light/libxl_virtio.c | 144 ++++++++++++++++++++++ tools/xl/xl_parse.c | 81 ++++++++++++ 9 files changed, 388 insertions(+) create mode 100644 tools/libs/light/libxl_virtio.c -- 2.31.1.272.g89b43f80a514

1 year, 12 months

4
9
0 0

[PATCH V7 0/3] toolstack support for generic virtio devices on Arm

by Viresh Kumar

Hello, This patchset adds toolstack support for I2C, GPIO and generic virtio devices. This is inspired from the work done by Oleksandr for the Disk device. This is developed as part of Linaro's Project Stratos, where we are working towards Hypervisor agnostic Rust based backends [1]. This is based of Julien's for-next/4.18 branch [2]. V6->V7: - Support generic virtio devices too. They are passed with type=virtio,device, and we only create the MMIO DT node for them. - Add links to DT bindings of I2C and GPIO, in code and documentation. - Call libxl__device_add() for all hypervisor types. - Add (0, "UNKNOWN") for libxl_virtio_transport. - Removed libxl_virtio_backend and libxl_virtioinfo, as they were unused. - Remove unnecessary stuff from libxl__virtio_from_xenstore() and add support for type and transport. - Add backend=domid in documentation and replace compatible with type. - Improved commit logs. V5->V6: - The cleanup patches are sent separately [2]. - We don't add I2C or GPIO specific device changes anymore, rather we create generic "virtio" devices. The "type" of a virtio devices helps us identify the right device, and create an entry in the DT node. The same can be used for all Virtio devices now. - Update man page xl.cfg. V4->V5: - Fixed indentation at few places. - Removed/added blank lines. - Added few comments. - Added review tags from Oleksandr. - Rebased over latest staging branch. V3->V4: - Update virtio_enabled independently of all devices, so we don't miss setting it to true. - Add iommu handling for i2c/gpio and move it as part of make_virtio_mmio_node_common(), which gets backend_domid parameter as a result. V2->V3: - Rebased over latest tree and made changes according to changes in Oleksandr's patches from sometime back. - Minor cleanups. V1->V2: - Patches 3/6 and 4/6 are new. - Patches 5/6 and 6/6 updated based on the above two patches. - Added link to the bindings for I2C and GPIO. - Rebased over latest master branch. Thanks. -- Viresh [1] https://lore.kernel.org/xen-devel/20220414092358.kepxbmnrtycz7mhe@vireshk-i… [2] https://xenbits.xen.org/gitweb/?p=people/julieng/xen-unstable.git;a=shortlo… Viresh Kumar (3): libxl: Add support for generic virtio device xl: Add support to parse generic virtio device docs: Add documentation for generic virtio devices docs/man/xl.cfg.5.pod.in | 28 +++++ tools/libs/light/Makefile | 1 + tools/libs/light/libxl_arm.c | 103 ++++++++++++++++ tools/libs/light/libxl_create.c | 4 + tools/libs/light/libxl_internal.h | 1 + tools/libs/light/libxl_types.idl | 18 +++ tools/libs/light/libxl_types_internal.idl | 1 + tools/libs/light/libxl_virtio.c | 144 ++++++++++++++++++++++ tools/ocaml/libs/xl/genwrap.py | 1 + tools/ocaml/libs/xl/xenlight_stubs.c | 1 + tools/xl/xl_parse.c | 80 ++++++++++++ 11 files changed, 382 insertions(+) create mode 100644 tools/libs/light/libxl_virtio.c -- 2.31.1.272.g89b43f80a514

2 years

2
7
0 0

[PATCH V6 0/3] Virtio toolstack support for I2C and GPIO on Arm

by Viresh Kumar

Hello, This patchset adds toolstack support for I2C and GPIO virtio devices. This is inspired from the work done by Oleksandr for the Disk device. This is developed as part of Linaro's Project Stratos, where we are working towards Hypervisor agnostic Rust based backend [1]. This is based of origin/staging (e61a78981364 xen/arm: add iounmap after initrd has been loaded in domain_build) and the earlier posted cleanup patches [2]. V5->V6: - The cleanup patches are sent separately [2]. - We don't add I2C or GPIO specific device changes anymore, rather we create generic "virtio" devices. The "type" of a virtio devices helps us identify the right device, and create an entry in the DT node. The same can be used for all Virtio devices now. - Update man page xl.cfg. V4->V5: - Fixed indentation at few places. - Removed/added blank lines. - Added few comments. - Added review tags from Oleksandr. - Rebased over latest staging branch. V3->V4: - Update virtio_enabled independently of all devices, so we don't miss setting it to true. - Add iommu handling for i2c/gpio and move it as part of make_virtio_mmio_node_common(), which gets backend_domid parameter as a result. V2->V3: - Rebased over latest tree and made changes according to changes in Oleksandr's patches from sometime back. - Minor cleanups. V1->V2: - Patches 3/6 and 4/6 are new. - Patches 5/6 and 6/6 updated based on the above two patches. - Added link to the bindings for I2C and GPIO. - Rebased over latest master branch. Thanks. -- Viresh [1] https://lore.kernel.org/xen-devel/20220414092358.kepxbmnrtycz7mhe@vireshk-i… [2] https://lore.kernel.org/all/cover.1662626550.git.viresh.kumar@linaro.org/ Viresh Kumar (3): libxl: Add support for generic virtio device xl: Add support to parse generic virtio device docs: Add documentation for generic virtio devices docs/man/xl.cfg.5.pod.in | 21 ++++ tools/libs/light/Makefile | 1 + tools/libs/light/libxl_arm.c | 89 +++++++++++++++ tools/libs/light/libxl_create.c | 5 + tools/libs/light/libxl_internal.h | 1 + tools/libs/light/libxl_types.idl | 29 +++++ tools/libs/light/libxl_types_internal.idl | 1 + tools/libs/light/libxl_virtio.c | 127 ++++++++++++++++++++++ tools/ocaml/libs/xl/genwrap.py | 1 + tools/ocaml/libs/xl/xenlight_stubs.c | 1 + tools/xl/xl_parse.c | 84 ++++++++++++++ 11 files changed, 360 insertions(+) create mode 100644 tools/libs/light/libxl_virtio.c -- 2.31.1.272.g89b43f80a514

2 years

3
18
0 0

[PATCH V6 0/3] libxl: arm: Make generic code independent of disk device

by Viresh Kumar

Hello, This was earlier sent as part of a patch series [1] adding support for GPIO/I2C virtio devices. The device specific patches would require some rework and possibly several versions, and so this series separates out the generic independent patches into a series of their own. This series makes some of the generic code independent of the disk device, since it can be used for other device types later on. Rebased over staging branch from today. V5->V6: - Separated into a patch series of their own. - Updated commit log of 1st patch to cover all changes. - Rename make_virtio_mmio_node_simple() as make_virtio_mmio_node(). - New patch 3/3, separated code from device specific patch. -- Viresh Viresh Kumar (3): libxl: arm: Create alloc_virtio_mmio_params() libxl: arm: Split make_virtio_mmio_node() libxl: arm: make creation of iommu node independent of disk device tools/libs/light/libxl_arm.c | 83 +++++++++++++++++++++++++----------- 1 file changed, 57 insertions(+), 26 deletions(-) -- 2.31.1.272.g89b43f80a514 -- Viresh [1] https://lore.kernel.org/all/cover.1661159474.git.viresh.kumar@linaro.org/

2 years

4
14
0 0

2024

2023

2022

2021

2020

Stratos-dev