October 2021 - Stratos-dev - op-lists.linaro.org

Xen Rust VirtIO demos work breakdown for Project Stratos

by Alex Bennée

Hi, The following is a breakdown (as best I can figure) of the work needed to demonstrate VirtIO backends in Rust on the Xen hypervisor. It requires work across a number of projects but notably core rust and virtio enabling in the Xen project (building on the work EPAM has already done) and the start of enabling rust-vmm crate to work with Xen. The first demo is a fairly simple toy to exercise the direct hypercall approach for a unikernel backend. On it's own it isn't super impressive but hopefully serves as a proof of concept for the idea of having backends running in a single exception level where latency will be important. The second is a much more ambitious bridge between Xen and vhost-user to allow for re-use of the existing vhost-user backends with the bridge acting as a proxy for what would usually be a full VMM in the type-2 hypervisor case. With that in mind the rust-vmm work is only aimed at doing the device emulation and doesn't address the larger question of how type-1 hypervisors can be integrated into the rust-vmm hypervisor model. A quick note about the estimates. They are exceedingly rough guesses plucked out of the air and I would be grateful for feedback from the appropriate domain experts on if I'm being overly optimistic or pessimistic. The links to the Stratos JIRA should be at least read accessible to all although they contain the same information as the attached document (albeit with nicer PNG renderings of my ASCII art ;-). There is a Stratos sync-up call next Thursday: https://calendar.google.com/event?action=TEMPLATE&tmeid=MWpidm5lbzM5NjlydnA… and I'm sure there will also be discussion in the various projects (hence the wide CC list). The Stratos calls are open to anyone who wants to attend and we welcome feedback from all who are interested. So on with the work breakdown: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ STRATOS PLANNING FOR 21 TO 22 Alex Bennée ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Table of Contents ───────────────── 1. Xen Rust Bindings ([STR-51]) .. 1. Upstream an "official" rust crate for Xen ([STR-52]) .. 2. Basic Hypervisor Interactions hypercalls ([STR-53]) .. 3. [#10] Access to XenStore service ([STR-54]) .. 4. VirtIO support hypercalls ([STR-55]) 2. Xen Hypervisor Support for Stratos ([STR-56]) .. 1. Stable ABI for foreignmemory mapping to non-dom0 ([STR-57]) .. 2. Tweaks to tooling to launch VirtIO guests 3. rust-vmm support for Xen VirtIO ([STR-59]) .. 1. Make vm-memory Xen aware ([STR-60]) .. 2. Xen IO notification and IRQ injections ([STR-61]) 4. Stratos Demos .. 1. Rust based stubdomain monitor ([STR-62]) .. 2. Xen aware vhost-user master ([STR-63]) 1 Xen Rust Bindings ([STR-51]) ══════════════════════════════ There exists a [placeholder repository] with the start of a set of x86_64 bindings for Xen and a very basic hello world uni-kernel example. This forms the basis of the initial Xen Rust work and will be available as a [xen-sys crate] via cargo. [STR-51] <https://linaro.atlassian.net/browse/STR-51> [placeholder repository] <https://gitlab.com/cardoe/oxerun.git> [xen-sys crate] <https://crates.io/crates/xen-sys> 1.1 Upstream an "official" rust crate for Xen ([STR-52]) ──────────────────────────────────────────────────────── To start with we will want an upstream location for future work to be based upon. The intention is the crate is independent of the version of Xen it runs on (above the baseline version chosen). This will entail: • ☐ agreeing with upstream the name/location for the source • ☐ documenting the rules for the "stable" hypercall ABI • ☐ establish an internal interface to elide between ioctl mediated and direct hypercalls • ☐ ensure the crate is multi-arch and has feature parity for arm64 As such we expect the implementation to be standalone, i.e. not wrapping the existing Xen libraries for mediation. There should be a close (1-to-1) mapping between the interfaces in the crate and the eventual hypercall made to the hypervisor. Estimate: 4w (elapsed likely longer due to discussion) [STR-52] <https://linaro.atlassian.net/browse/STR-52> 1.2 Basic Hypervisor Interactions hypercalls ([STR-53]) ─────────────────────────────────────────────────────── These are the bare minimum hypercalls implemented as both ioctl and direct calls. These allow for a very basic binary to: • ☐ console_io - output IO via the Xen console • ☐ domctl stub - basic stub for domain control (different API?) • ☐ sysctl stub - basic stub for system control (different API?) The idea would be this provides enough hypercall interface to query the list of domains and output their status via the xen console. There is an open question about if the domctl and sysctl hypercalls are way to go. Estimate: 6w [STR-53] <https://linaro.atlassian.net/browse/STR-53> 1.3 [#10] Access to XenStore service ([STR-54]) ─────────────────────────────────────────────── This is a shared configuration storage space accessed via either Unix sockets (on dom0) or via the Xenbus. This is used to access configuration information for the domain. Is this needed for a backend though? Can everything just be passed direct on the command line? Estimate: 4w [STR-54] <https://linaro.atlassian.net/browse/STR-54> 1.4 VirtIO support hypercalls ([STR-55]) ──────────────────────────────────────── These are the hypercalls that need to be implemented to support a VirtIO backend. This includes the ability to map another guests memory into the current domains address space, register to receive IOREQ events when the guest knocks at the doorbell and inject kicks into the guest. The hypercalls we need to support would be: • ☐ dmop - device model ops (*_ioreq_server, setirq, nr_vpus) • ☐ foreignmemory - map and unmap guest memory The DMOP space is larger than what we need for an IOREQ backend so I've based it just on what arch/arm/dm.c exports which is the subset introduced for EPAM's virtio work. Estimate: 12w [STR-55] <https://linaro.atlassian.net/browse/STR-55> 2 Xen Hypervisor Support for Stratos ([STR-56]) ═══════════════════════════════════════════════ These tasks include tasks needed to support the various different deployments of Stratos components in Xen. [STR-56] <https://linaro.atlassian.net/browse/STR-56> 2.1 Stable ABI for foreignmemory mapping to non-dom0 ([STR-57]) ─────────────────────────────────────────────────────────────── Currently the foreign memory mapping support only works for dom0 due to reference counting issues. If we are to support backends running in their own domains this will need to get fixed. Estimate: 8w [STR-57] <https://linaro.atlassian.net/browse/STR-57> 2.2 Tweaks to tooling to launch VirtIO guests ───────────────────────────────────────────── There might not be too much to do here. The EPAM work already did something similar for their PoC for virtio-block. Essentially we need to ensure: • ☐ DT bindings are passed to the guest for virtio-mmio device discovery • ☐ Our rust backend can be instantiated before the domU is launched This currently assumes the tools and the backend are running in dom0. Estimate: 4w 3 rust-vmm support for Xen VirtIO ([STR-59]) ════════════════════════════════════════════ This encompasses the tasks required to get a vhost-user server up and running while interfacing to the Xen hypervisor. This will require the xen-sys.rs crate for the actual interface to the hypervisor. We need to work out how a Xen configuration option would be passed to the various bits of rust-vmm when something is being built. [STR-59] <https://linaro.atlassian.net/browse/STR-59> 3.1 Make vm-memory Xen aware ([STR-60]) ─────────────────────────────────────── The vm-memory crate is the root crate for abstracting access to the guests memory. It currently has multiple configuration builds to handle difference between mmap on Windows and Unix. Although mmap isn't directly exposed the public interfaces support a mmap like interface. We would need to: • ☐ work out how to expose foreign memory via the vm-memory mechanism I'm not sure if this just means implementing the GuestMemory trait for a GuestMemoryXen or if we need to present a mmap like interface. Estimate: 8w [STR-60] <https://linaro.atlassian.net/browse/STR-60> 3.2 Xen IO notification and IRQ injections ([STR-61]) ───────────────────────────────────────────────────── The KVM world provides for ioeventfd (notifications) and irqfd (injection) to signal asynchronously between the guest and the backend. As far a I can tell this is currently handled inside the various VMMs which assume a KVM backend. While the vhost-user slave code doesn't see the register_ioevent/register_irqfd events it does deal with EventFDs throughout the code. Perhaps the best approach here would be to create a IOREQ crate that can create EventFD descriptors which can then be passed to the slaves to use for notification and injection. Otherwise there might be an argument for a new crate that can encapsulate this behaviour for both KVM/ioeventd and Xen/IOREQ setups? Estimate: 8w? [STR-61] <https://linaro.atlassian.net/browse/STR-61> 4 Stratos Demos ═══════════════ These tasks cover the creation of demos that brig together all the previous bits of work to demonstrate a new area of capability that has been opened up by Stratos work. 4.1 Rust based stubdomain monitor ([STR-62]) ──────────────────────────────────────────── This is a basic demo that is a proof of concept for a unikernel style backend written in pure Rust. This work would be a useful precursor for things such as the RTOS Dom0 on a safety island ([STR-11]) or as a carrier for the virtio-scmi backend. The monitor program will periodically poll the state of the other domains and echo their status to the Xen console. Estimate: 4w #+name: stub-domain-example #+begin_src ditaa :cmdline -o :file stub_domain_example.png Dom0 | DomU | DomStub | | : /-------------\ : | |cPNK | | | | | | | | | | /------------------------------------\ | | GuestOS | | |cPNK | | | | | EL0 | Dom0 Userspace (xl tools, QEMU) | | | | | /---------------\ | | | | | | |cYEL | \------------------------------------/ | | | | | | +------------------------------------+ | | | | | Rust Monitor | EL1 |cA1B Dom0 Kernel | | | | | | | +------------------------------------+ | \-------------/ | \---------------/ -------------------------------------------------------------------------------=------------------ +-------------------------------------------------------------------------------------+ EL2 |cC02 Xen Hypervisor | +-------------------------------------------------------------------------------------+ #+end_src [STR-62] <https://linaro.atlassian.net/browse/STR-62> [STR-11] <https://linaro.atlassian.net/browse/STR-11> 4.2 Xen aware vhost-user master ([STR-63]) ────────────────────────────────────────── Usually the master side of a vhost-user system is embedded directly in the VMM itself. However in a Xen deployment their is no overarching VMM but a series of utility programs that query the hypervisor directly. The Xen tooling is also responsible for setting up any support processes that are responsible for emulating HW for the guest. The task aims to bridge the gap between Xen's normal HW emulation path (ioreq) and VirtIO's userspace device emulation (vhost-user). The process would be started with some information on where the virtio-mmio address space is and what the slave binary will be. It will then: • map the guest into Dom0 userspace and attach to a MemFD • register the appropriate memory regions as IOREQ regions with Xen • create EventFD channels for the virtio kick notifications (one each way) • spawn the vhost-user slave process and mediate the notifications and kicks between the slave and Xen itself #+name: xen-vhost-user-master #+begin_src ditaa :cmdline -o :file xen_vhost_user_master.png Dom0 DomU | | | | | | +-------------------+ +-------------------+ | | |----------->| | | | vhost-user | vhost-user | vhost-user | : /------------------------------------\ | slave | protocol | master | | | | | (existing) |<-----------| (rust) | | | | +-------------------+ +-------------------+ | | | ^ ^ | ^ | | Guest Userspace | | | | | | | | | | | IOREQ | | | | | | | | | | | v v V | | \------------------------------------/ +---------------------------------------------------+ | +------------------------------------+ | ^ ^ | ioctl ^ | | | | | | iofd/irqfd eventFD | | | | | | Guest Kernel | | +---------------------------+ | | | | | +-------------+ | | | | | | | | virtio-dev | | | Host Kernel V | | | | +-------------+ | +---------------------------------------------------+ | +------------------------------------+ | ^ | | ^ | hyper | | | ----------------------=------------- | -=--- | ----=------ | -----=- | --------=------------------ | call | Trap | | IRQ V | V | +-------------------------------------------------------------------------------------+ | | ^ | ^ | | | +-------------+ | | EL2 | Xen Hypervisor | | | | +-------------------------------+ | | | +-------------------------------------------------------------------------------------+ #+end_src [STR-63] <https://linaro.atlassian.net/browse/STR-63> -- Alex Bennée

2 years, 3 months

8
15
0 0

[PATCH V7] gpio: virtio: Add IRQ support

by Viresh Kumar

This patch adds IRQ support for the virtio GPIO driver. Note that this uses the irq_bus_lock/unlock() callbacks, since those operations over virtio may sleep. Reviewed-by: Linus Walleij <linus.walleij(a)linaro.org> Signed-off-by: Viresh Kumar <viresh.kumar(a)linaro.org> --- Bartosz, The spec changes are close to merging now, I will let you know once the ballot is closed and the spec changes are merged. You can then pick this patch for 5.16. V6->V7: - Use generic_handle_domain_irq. - Drop check for IRQ_TYPE_NONE, dead code. - Avoid breaking line to fit into 80 columns. V5->V6: - Sent it separately as the other patches are already merged. - Queue the buffers only after enabling the irq (as per latest spec). - Migrate to raw_spinlock_t. drivers/gpio/Kconfig | 1 + drivers/gpio/gpio-virtio.c | 302 ++++++++++++++++++++++++++++++- include/uapi/linux/virtio_gpio.h | 25 +++ 3 files changed, 324 insertions(+), 4 deletions(-) diff --git a/drivers/gpio/Kconfig b/drivers/gpio/Kconfig index fae5141251e5..bfa723ff8e7c 100644 --- a/drivers/gpio/Kconfig +++ b/drivers/gpio/Kconfig @@ -1674,6 +1674,7 @@ config GPIO_MOCKUP config GPIO_VIRTIO tristate "VirtIO GPIO support" depends on VIRTIO + select GPIOLIB_IRQCHIP help Say Y here to enable guest support for virtio-based GPIO controllers. diff --git a/drivers/gpio/gpio-virtio.c b/drivers/gpio/gpio-virtio.c index d24f1c9264bc..aeec4bf0b625 100644 --- a/drivers/gpio/gpio-virtio.c +++ b/drivers/gpio/gpio-virtio.c @@ -16,6 +16,7 @@ #include <linux/kernel.h> #include <linux/module.h> #include <linux/mutex.h> +#include <linux/spinlock.h> #include <linux/virtio_config.h> #include <uapi/linux/virtio_gpio.h> #include <uapi/linux/virtio_ids.h> @@ -28,12 +29,30 @@ struct virtio_gpio_line { unsigned int rxlen; }; +struct vgpio_irq_line { + u8 type; + bool disabled; + bool masked; + bool queued; + bool update_pending; + bool queue_pending; + + struct virtio_gpio_irq_request ireq ____cacheline_aligned; + struct virtio_gpio_irq_response ires ____cacheline_aligned; +}; + struct virtio_gpio { struct virtio_device *vdev; struct mutex lock; /* Protects virtqueue operation */ struct gpio_chip gc; struct virtio_gpio_line *lines; struct virtqueue *request_vq; + + /* irq support */ + struct virtqueue *event_vq; + struct mutex irq_lock; /* Protects irq operation */ + raw_spinlock_t eventq_lock; /* Protects queuing of the buffer */ + struct vgpio_irq_line *irq_lines; }; static int _virtio_gpio_req(struct virtio_gpio *vgpio, u16 type, u16 gpio, @@ -186,6 +205,238 @@ static void virtio_gpio_set(struct gpio_chip *gc, unsigned int gpio, int value) virtio_gpio_req(vgpio, VIRTIO_GPIO_MSG_SET_VALUE, gpio, value, NULL); } +/* Interrupt handling */ +static void virtio_gpio_irq_prepare(struct virtio_gpio *vgpio, u16 gpio) +{ + struct vgpio_irq_line *irq_line = &vgpio->irq_lines[gpio]; + struct virtio_gpio_irq_request *ireq = &irq_line->ireq; + struct virtio_gpio_irq_response *ires = &irq_line->ires; + struct scatterlist *sgs[2], req_sg, res_sg; + int ret; + + if (WARN_ON(irq_line->queued || irq_line->masked || irq_line->disabled)) + return; + + ireq->gpio = cpu_to_le16(gpio); + sg_init_one(&req_sg, ireq, sizeof(*ireq)); + sg_init_one(&res_sg, ires, sizeof(*ires)); + sgs[0] = &req_sg; + sgs[1] = &res_sg; + + ret = virtqueue_add_sgs(vgpio->event_vq, sgs, 1, 1, irq_line, GFP_ATOMIC); + if (ret) { + dev_err(&vgpio->vdev->dev, "failed to add request to eventq\n"); + return; + } + + irq_line->queued = true; + virtqueue_kick(vgpio->event_vq); +} + +static void virtio_gpio_irq_enable(struct irq_data *d) +{ + struct gpio_chip *gc = irq_data_get_irq_chip_data(d); + struct virtio_gpio *vgpio = gpiochip_get_data(gc); + struct vgpio_irq_line *irq_line = &vgpio->irq_lines[d->hwirq]; + + raw_spin_lock(&vgpio->eventq_lock); + irq_line->disabled = false; + irq_line->masked = false; + irq_line->queue_pending = true; + raw_spin_unlock(&vgpio->eventq_lock); + + irq_line->update_pending = true; +} + +static void virtio_gpio_irq_disable(struct irq_data *d) +{ + struct gpio_chip *gc = irq_data_get_irq_chip_data(d); + struct virtio_gpio *vgpio = gpiochip_get_data(gc); + struct vgpio_irq_line *irq_line = &vgpio->irq_lines[d->hwirq]; + + raw_spin_lock(&vgpio->eventq_lock); + irq_line->disabled = true; + irq_line->masked = true; + irq_line->queue_pending = false; + raw_spin_unlock(&vgpio->eventq_lock); + + irq_line->update_pending = true; +} + +static void virtio_gpio_irq_mask(struct irq_data *d) +{ + struct gpio_chip *gc = irq_data_get_irq_chip_data(d); + struct virtio_gpio *vgpio = gpiochip_get_data(gc); + struct vgpio_irq_line *irq_line = &vgpio->irq_lines[d->hwirq]; + + raw_spin_lock(&vgpio->eventq_lock); + irq_line->masked = true; + raw_spin_unlock(&vgpio->eventq_lock); +} + +static void virtio_gpio_irq_unmask(struct irq_data *d) +{ + struct gpio_chip *gc = irq_data_get_irq_chip_data(d); + struct virtio_gpio *vgpio = gpiochip_get_data(gc); + struct vgpio_irq_line *irq_line = &vgpio->irq_lines[d->hwirq]; + + raw_spin_lock(&vgpio->eventq_lock); + irq_line->masked = false; + + /* Queue the buffer unconditionally on unmask */ + virtio_gpio_irq_prepare(vgpio, d->hwirq); + raw_spin_unlock(&vgpio->eventq_lock); +} + +static int virtio_gpio_irq_set_type(struct irq_data *d, unsigned int type) +{ + struct gpio_chip *gc = irq_data_get_irq_chip_data(d); + struct virtio_gpio *vgpio = gpiochip_get_data(gc); + struct vgpio_irq_line *irq_line = &vgpio->irq_lines[d->hwirq]; + + switch (type) { + case IRQ_TYPE_EDGE_RISING: + type = VIRTIO_GPIO_IRQ_TYPE_EDGE_RISING; + break; + case IRQ_TYPE_EDGE_FALLING: + type = VIRTIO_GPIO_IRQ_TYPE_EDGE_FALLING; + break; + case IRQ_TYPE_EDGE_BOTH: + type = VIRTIO_GPIO_IRQ_TYPE_EDGE_BOTH; + break; + case IRQ_TYPE_LEVEL_LOW: + type = VIRTIO_GPIO_IRQ_TYPE_LEVEL_LOW; + break; + case IRQ_TYPE_LEVEL_HIGH: + type = VIRTIO_GPIO_IRQ_TYPE_LEVEL_HIGH; + break; + default: + dev_err(&vgpio->vdev->dev, "unsupported irq type: %u\n", type); + return -EINVAL; + } + + irq_line->type = type; + irq_line->update_pending = true; + + return 0; +} + +static void virtio_gpio_irq_bus_lock(struct irq_data *d) +{ + struct gpio_chip *gc = irq_data_get_irq_chip_data(d); + struct virtio_gpio *vgpio = gpiochip_get_data(gc); + + mutex_lock(&vgpio->irq_lock); +} + +static void virtio_gpio_irq_bus_sync_unlock(struct irq_data *d) +{ + struct gpio_chip *gc = irq_data_get_irq_chip_data(d); + struct virtio_gpio *vgpio = gpiochip_get_data(gc); + struct vgpio_irq_line *irq_line = &vgpio->irq_lines[d->hwirq]; + u8 type = irq_line->disabled ? VIRTIO_GPIO_IRQ_TYPE_NONE : irq_line->type; + unsigned long flags; + + if (irq_line->update_pending) { + irq_line->update_pending = false; + virtio_gpio_req(vgpio, VIRTIO_GPIO_MSG_IRQ_TYPE, d->hwirq, type, + NULL); + + /* Queue the buffer only after interrupt is enabled */ + raw_spin_lock_irqsave(&vgpio->eventq_lock, flags); + if (irq_line->queue_pending) { + irq_line->queue_pending = false; + virtio_gpio_irq_prepare(vgpio, d->hwirq); + } + raw_spin_unlock_irqrestore(&vgpio->eventq_lock, flags); + } + + mutex_unlock(&vgpio->irq_lock); +} + +static struct irq_chip vgpio_irq_chip = { + .name = "virtio-gpio", + .irq_enable = virtio_gpio_irq_enable, + .irq_disable = virtio_gpio_irq_disable, + .irq_mask = virtio_gpio_irq_mask, + .irq_unmask = virtio_gpio_irq_unmask, + .irq_set_type = virtio_gpio_irq_set_type, + + /* These are required to implement irqchip for slow busses */ + .irq_bus_lock = virtio_gpio_irq_bus_lock, + .irq_bus_sync_unlock = virtio_gpio_irq_bus_sync_unlock, +}; + +static bool ignore_irq(struct virtio_gpio *vgpio, int gpio, + struct vgpio_irq_line *irq_line) +{ + bool ignore = false; + + raw_spin_lock(&vgpio->eventq_lock); + irq_line->queued = false; + + /* Interrupt is disabled currently */ + if (irq_line->masked || irq_line->disabled) { + ignore = true; + goto unlock; + } + + /* + * Buffer is returned as the interrupt was disabled earlier, but is + * enabled again now. Requeue the buffers. + */ + if (irq_line->ires.status == VIRTIO_GPIO_IRQ_STATUS_INVALID) { + virtio_gpio_irq_prepare(vgpio, gpio); + ignore = true; + goto unlock; + } + + if (WARN_ON(irq_line->ires.status != VIRTIO_GPIO_IRQ_STATUS_VALID)) + ignore = true; + +unlock: + raw_spin_unlock(&vgpio->eventq_lock); + + return ignore; +} + +static void virtio_gpio_event_vq(struct virtqueue *vq) +{ + struct virtio_gpio *vgpio = vq->vdev->priv; + struct device *dev = &vgpio->vdev->dev; + struct vgpio_irq_line *irq_line; + int gpio, ret; + unsigned int len; + + while (true) { + irq_line = virtqueue_get_buf(vgpio->event_vq, &len); + if (!irq_line) + break; + + if (len != sizeof(irq_line->ires)) { + dev_err(dev, "irq with incorrect length (%u : %u)\n", + len, (unsigned int)sizeof(irq_line->ires)); + continue; + } + + /* + * Find GPIO line number from the offset of irq_line within the + * irq_lines block. We can also get GPIO number from + * irq-request, but better not to rely on a buffer returned by + * remote. + */ + gpio = irq_line - vgpio->irq_lines; + WARN_ON(gpio >= vgpio->gc.ngpio); + + if (unlikely(ignore_irq(vgpio, gpio, irq_line))) + continue; + + ret = generic_handle_domain_irq(vgpio->gc.irq.domain, gpio); + if (ret) + dev_err(dev, "failed to handle interrupt: %d\n", ret); + }; +} + static void virtio_gpio_request_vq(struct virtqueue *vq) { struct virtio_gpio_line *line; @@ -210,14 +461,15 @@ static void virtio_gpio_free_vqs(struct virtio_device *vdev) static int virtio_gpio_alloc_vqs(struct virtio_gpio *vgpio, struct virtio_device *vdev) { - const char * const names[] = { "requestq" }; + const char * const names[] = { "requestq", "eventq" }; vq_callback_t *cbs[] = { virtio_gpio_request_vq, + virtio_gpio_event_vq, }; - struct virtqueue *vqs[1] = { NULL }; + struct virtqueue *vqs[2] = { NULL, NULL }; int ret; - ret = virtio_find_vqs(vdev, 1, vqs, cbs, names, NULL); + ret = virtio_find_vqs(vdev, vgpio->irq_lines ? 2 : 1, vqs, cbs, names, NULL); if (ret) { dev_err(&vdev->dev, "failed to find vqs: %d\n", ret); return ret; @@ -225,11 +477,23 @@ static int virtio_gpio_alloc_vqs(struct virtio_gpio *vgpio, if (!vqs[0]) { dev_err(&vdev->dev, "failed to find requestq vq\n"); - return -ENODEV; + goto out; } vgpio->request_vq = vqs[0]; + if (vgpio->irq_lines && !vqs[1]) { + dev_err(&vdev->dev, "failed to find eventq vq\n"); + goto out; + } + vgpio->event_vq = vqs[1]; + return 0; + +out: + if (vqs[0] || vqs[1]) + virtio_gpio_free_vqs(vdev); + + return -ENODEV; } static const char **virtio_gpio_get_names(struct virtio_gpio *vgpio, @@ -325,6 +589,30 @@ static int virtio_gpio_probe(struct virtio_device *vdev) vgpio->gc.owner = THIS_MODULE; vgpio->gc.can_sleep = true; + /* Interrupt support */ + if (virtio_has_feature(vdev, VIRTIO_GPIO_F_IRQ)) { + vgpio->irq_lines = devm_kcalloc(dev, ngpio, sizeof(*vgpio->irq_lines), GFP_KERNEL); + if (!vgpio->irq_lines) + return -ENOMEM; + + /* The event comes from the outside so no parent handler */ + vgpio->gc.irq.parent_handler = NULL; + vgpio->gc.irq.num_parents = 0; + vgpio->gc.irq.parents = NULL; + vgpio->gc.irq.default_type = IRQ_TYPE_NONE; + vgpio->gc.irq.handler = handle_level_irq; + vgpio->gc.irq.chip = &vgpio_irq_chip; + + for (i = 0; i < ngpio; i++) { + vgpio->irq_lines[i].type = VIRTIO_GPIO_IRQ_TYPE_NONE; + vgpio->irq_lines[i].disabled = true; + vgpio->irq_lines[i].masked = true; + } + + mutex_init(&vgpio->irq_lock); + raw_spin_lock_init(&vgpio->eventq_lock); + } + ret = virtio_gpio_alloc_vqs(vgpio, vdev); if (ret) return ret; @@ -357,7 +645,13 @@ static const struct virtio_device_id id_table[] = { }; MODULE_DEVICE_TABLE(virtio, id_table); +static const unsigned int features[] = { + VIRTIO_GPIO_F_IRQ, +}; + static struct virtio_driver virtio_gpio_driver = { + .feature_table = features, + .feature_table_size = ARRAY_SIZE(features), .id_table = id_table, .probe = virtio_gpio_probe, .remove = virtio_gpio_remove, diff --git a/include/uapi/linux/virtio_gpio.h b/include/uapi/linux/virtio_gpio.h index 0445f905d8cc..d04af9c5f0de 100644 --- a/include/uapi/linux/virtio_gpio.h +++ b/include/uapi/linux/virtio_gpio.h @@ -5,12 +5,16 @@ #include <linux/types.h> +/* Virtio GPIO Feature bits */ +#define VIRTIO_GPIO_F_IRQ 0 + /* Virtio GPIO request types */ #define VIRTIO_GPIO_MSG_GET_NAMES 0x0001 #define VIRTIO_GPIO_MSG_GET_DIRECTION 0x0002 #define VIRTIO_GPIO_MSG_SET_DIRECTION 0x0003 #define VIRTIO_GPIO_MSG_GET_VALUE 0x0004 #define VIRTIO_GPIO_MSG_SET_VALUE 0x0005 +#define VIRTIO_GPIO_MSG_IRQ_TYPE 0x0006 /* Possible values of the status field */ #define VIRTIO_GPIO_STATUS_OK 0x0 @@ -21,6 +25,14 @@ #define VIRTIO_GPIO_DIRECTION_OUT 0x01 #define VIRTIO_GPIO_DIRECTION_IN 0x02 +/* Virtio GPIO IRQ types */ +#define VIRTIO_GPIO_IRQ_TYPE_NONE 0x00 +#define VIRTIO_GPIO_IRQ_TYPE_EDGE_RISING 0x01 +#define VIRTIO_GPIO_IRQ_TYPE_EDGE_FALLING 0x02 +#define VIRTIO_GPIO_IRQ_TYPE_EDGE_BOTH 0x03 +#define VIRTIO_GPIO_IRQ_TYPE_LEVEL_HIGH 0x04 +#define VIRTIO_GPIO_IRQ_TYPE_LEVEL_LOW 0x08 + struct virtio_gpio_config { __le16 ngpio; __u8 padding[2]; @@ -44,4 +56,17 @@ struct virtio_gpio_response_get_names { __u8 value[]; }; +/* Virtio GPIO IRQ Request / Response */ +struct virtio_gpio_irq_request { + __le16 gpio; +}; + +struct virtio_gpio_irq_response { + __u8 status; +}; + +/* Possible values of the interrupt status field */ +#define VIRTIO_GPIO_IRQ_STATUS_INVALID 0x0 +#define VIRTIO_GPIO_IRQ_STATUS_VALID 0x1 + #endif /* _LINUX_VIRTIO_GPIO_H */ -- 2.31.1.272.g89b43f80a514

2 years, 8 months

3
4
0 0

Minutes for 28/10

by Alex Bennée

Hi, I've posted the meeting minutes the Stratos project pages: https://linaro.atlassian.net/wiki/spaces/STR/pages/28655222830/2021-10-28+P… As I was quite involved in the conversation and didn't have my usual background note taker could I ask participants to give it a quick once over while the conversation is fresh to make sure I didn't miss anything out. Thanks, -- Alex Bennée KVM/QEMU Hacker for Linaro

2 years, 9 months

1
0
0 0

Any topics for tomorrows sync-up?

by Alex Bennée

Hi, Any topics to cover tomorrow? I know Peter has made progress on virtio-video but is out on holiday this week. I don't think there is much to report on the rist-vmm work yet but perhaps a quick catch-up? Anything else? -- Alex Bennée

2 years, 9 months

1
0
0 0

[PATCH] i2c: virtio: Add support for zero-length requests

by Viresh Kumar

The virtio specification received a new mandatory feature (VIRTIO_I2C_F_ZERO_LENGTH_REQUEST) for zero length requests. Fail if the feature isn't offered by the device. For each read-request, set the VIRTIO_I2C_FLAGS_M_RD flag, as required by the VIRTIO_I2C_F_ZERO_LENGTH_REQUEST feature. This allows us to support zero length requests, like SMBUS Quick, where the buffer need not be sent anymore. Signed-off-by: Viresh Kumar <viresh.kumar(a)linaro.org> --- Hi Wolfram, Please do not apply this until the spec changes [1] are merged, sending it early to get review done. I will ping you later once the spec is merged. [1] https://lists.oasis-open.org/archives/virtio-dev/202110/msg00109.html drivers/i2c/busses/i2c-virtio.c | 56 ++++++++++++++++++--------------- include/uapi/linux/virtio_i2c.h | 6 ++++ 2 files changed, 36 insertions(+), 26 deletions(-) diff --git a/drivers/i2c/busses/i2c-virtio.c b/drivers/i2c/busses/i2c-virtio.c index f10a603b13fb..1ed4daa918a0 100644 --- a/drivers/i2c/busses/i2c-virtio.c +++ b/drivers/i2c/busses/i2c-virtio.c @@ -62,35 +62,33 @@ static int virtio_i2c_prepare_reqs(struct virtqueue *vq, for (i = 0; i < num; i++) { int outcnt = 0, incnt = 0; - /* - * We don't support 0 length messages and so filter out - * 0 length transfers by using i2c_adapter_quirks. - */ - if (!msgs[i].len) - break; - /* * Only 7-bit mode supported for this moment. For the address * format, Please check the Virtio I2C Specification. */ reqs[i].out_hdr.addr = cpu_to_le16(msgs[i].addr << 1); + if (msgs[i].flags & I2C_M_RD) + reqs[i].out_hdr.flags |= cpu_to_le32(VIRTIO_I2C_FLAGS_M_RD); + if (i != num - 1) - reqs[i].out_hdr.flags = cpu_to_le32(VIRTIO_I2C_FLAGS_FAIL_NEXT); + reqs[i].out_hdr.flags |= cpu_to_le32(VIRTIO_I2C_FLAGS_FAIL_NEXT); sg_init_one(&out_hdr, &reqs[i].out_hdr, sizeof(reqs[i].out_hdr)); sgs[outcnt++] = &out_hdr; - reqs[i].buf = i2c_get_dma_safe_msg_buf(&msgs[i], 1); - if (!reqs[i].buf) - break; + if (msgs[i].len) { + reqs[i].buf = i2c_get_dma_safe_msg_buf(&msgs[i], 1); + if (!reqs[i].buf) + break; - sg_init_one(&msg_buf, reqs[i].buf, msgs[i].len); + sg_init_one(&msg_buf, reqs[i].buf, msgs[i].len); - if (msgs[i].flags & I2C_M_RD) - sgs[outcnt + incnt++] = &msg_buf; - else - sgs[outcnt++] = &msg_buf; + if (msgs[i].flags & I2C_M_RD) + sgs[outcnt + incnt++] = &msg_buf; + else + sgs[outcnt++] = &msg_buf; + } sg_init_one(&in_hdr, &reqs[i].in_hdr, sizeof(reqs[i].in_hdr)); sgs[outcnt + incnt++] = &in_hdr; @@ -191,7 +189,7 @@ static int virtio_i2c_setup_vqs(struct virtio_i2c *vi) static u32 virtio_i2c_func(struct i2c_adapter *adap) { - return I2C_FUNC_I2C | (I2C_FUNC_SMBUS_EMUL & ~I2C_FUNC_SMBUS_QUICK); + return I2C_FUNC_I2C | I2C_FUNC_SMBUS_EMUL; } static struct i2c_algorithm virtio_algorithm = { @@ -199,15 +197,16 @@ static struct i2c_algorithm virtio_algorithm = { .functionality = virtio_i2c_func, }; -static const struct i2c_adapter_quirks virtio_i2c_quirks = { - .flags = I2C_AQ_NO_ZERO_LEN, -}; - static int virtio_i2c_probe(struct virtio_device *vdev) { struct virtio_i2c *vi; int ret; + if (!virtio_has_feature(vdev, VIRTIO_I2C_F_ZERO_LENGTH_REQUEST)) { + dev_err(&vdev->dev, "Zero-length request feature is mandatory\n"); + return -EINVAL; + } + vi = devm_kzalloc(&vdev->dev, sizeof(*vi), GFP_KERNEL); if (!vi) return -ENOMEM; @@ -225,7 +224,6 @@ static int virtio_i2c_probe(struct virtio_device *vdev) snprintf(vi->adap.name, sizeof(vi->adap.name), "i2c_virtio at virtio bus %d", vdev->index); vi->adap.algo = &virtio_algorithm; - vi->adap.quirks = &virtio_i2c_quirks; vi->adap.dev.parent = &vdev->dev; vi->adap.dev.of_node = vdev->dev.of_node; i2c_set_adapdata(&vi->adap, vi); @@ -270,11 +268,17 @@ static int virtio_i2c_restore(struct virtio_device *vdev) } #endif +static const unsigned int features[] = { + VIRTIO_I2C_F_ZERO_LENGTH_REQUEST, +}; + static struct virtio_driver virtio_i2c_driver = { - .id_table = id_table, - .probe = virtio_i2c_probe, - .remove = virtio_i2c_remove, - .driver = { + .feature_table = features, + .feature_table_size = ARRAY_SIZE(features), + .id_table = id_table, + .probe = virtio_i2c_probe, + .remove = virtio_i2c_remove, + .driver = { .name = "i2c_virtio", }, #ifdef CONFIG_PM_SLEEP diff --git a/include/uapi/linux/virtio_i2c.h b/include/uapi/linux/virtio_i2c.h index 7c6a6fc01ad6..acf3b6069136 100644 --- a/include/uapi/linux/virtio_i2c.h +++ b/include/uapi/linux/virtio_i2c.h @@ -11,9 +11,15 @@ #include <linux/const.h> #include <linux/types.h> +/* Virtio I2C Feature bits */ +#define VIRTIO_I2C_F_ZERO_LENGTH_REQUEST 0 + /* The bit 0 of the @virtio_i2c_out_hdr.@flags, used to group the requests */ #define VIRTIO_I2C_FLAGS_FAIL_NEXT _BITUL(0) +/* The bit 1 of the @virtio_i2c_out_hdr.@flags, used to mark a buffer as read */ +#define VIRTIO_I2C_FLAGS_M_RD _BITUL(1) + /** * struct virtio_i2c_out_hdr - the virtio I2C message OUT header * @addr: the controlled device address -- 2.31.1.272.g89b43f80a514

2 years, 9 months

4
5
0 0

[PATCH V6] gpio: virtio: Add IRQ support

by Viresh Kumar

This patch adds IRQ support for the virtio GPIO driver. Note that this uses the irq_bus_lock/unlock() callbacks, since those operations over virtio may sleep. Reviewed-by: Linus Walleij <linus.walleij(a)linaro.org> Signed-off-by: Viresh Kumar <viresh.kumar(a)linaro.org> --- Bartosz, The spec changes are close to merging now, I will let you know once the ballot is closed and the spec changes are merged. You can then pick this patch for 5.16. V5->V6: - Sent it separately as the other patches are already merged. - Queue the buffers only after enabling the irq (as per latest spec). - Migrate to raw_spinlock_t. drivers/gpio/Kconfig | 1 + drivers/gpio/gpio-virtio.c | 310 ++++++++++++++++++++++++++++++- include/uapi/linux/virtio_gpio.h | 25 +++ 3 files changed, 332 insertions(+), 4 deletions(-) diff --git a/drivers/gpio/Kconfig b/drivers/gpio/Kconfig index fae5141251e5..bfa723ff8e7c 100644 --- a/drivers/gpio/Kconfig +++ b/drivers/gpio/Kconfig @@ -1674,6 +1674,7 @@ config GPIO_MOCKUP config GPIO_VIRTIO tristate "VirtIO GPIO support" depends on VIRTIO + select GPIOLIB_IRQCHIP help Say Y here to enable guest support for virtio-based GPIO controllers. diff --git a/drivers/gpio/gpio-virtio.c b/drivers/gpio/gpio-virtio.c index d24f1c9264bc..73fbe2eda4b9 100644 --- a/drivers/gpio/gpio-virtio.c +++ b/drivers/gpio/gpio-virtio.c @@ -16,6 +16,7 @@ #include <linux/kernel.h> #include <linux/module.h> #include <linux/mutex.h> +#include <linux/spinlock.h> #include <linux/virtio_config.h> #include <uapi/linux/virtio_gpio.h> #include <uapi/linux/virtio_ids.h> @@ -28,12 +29,30 @@ struct virtio_gpio_line { unsigned int rxlen; }; +struct vgpio_irq_line { + u8 type; + bool disabled; + bool masked; + bool queued; + bool update_pending; + bool queue_pending; + + struct virtio_gpio_irq_request ireq ____cacheline_aligned; + struct virtio_gpio_irq_response ires ____cacheline_aligned; +}; + struct virtio_gpio { struct virtio_device *vdev; struct mutex lock; /* Protects virtqueue operation */ struct gpio_chip gc; struct virtio_gpio_line *lines; struct virtqueue *request_vq; + + /* irq support */ + struct virtqueue *event_vq; + struct mutex irq_lock; /* Protects irq operation */ + raw_spinlock_t eventq_lock; /* Protects queuing of the buffer */ + struct vgpio_irq_line *irq_lines; }; static int _virtio_gpio_req(struct virtio_gpio *vgpio, u16 type, u16 gpio, @@ -186,6 +205,244 @@ static void virtio_gpio_set(struct gpio_chip *gc, unsigned int gpio, int value) virtio_gpio_req(vgpio, VIRTIO_GPIO_MSG_SET_VALUE, gpio, value, NULL); } +/* Interrupt handling */ +static void virtio_gpio_irq_prepare(struct virtio_gpio *vgpio, u16 gpio) +{ + struct vgpio_irq_line *irq_line = &vgpio->irq_lines[gpio]; + struct virtio_gpio_irq_request *ireq = &irq_line->ireq; + struct virtio_gpio_irq_response *ires = &irq_line->ires; + struct scatterlist *sgs[2], req_sg, res_sg; + int ret; + + if (WARN_ON(irq_line->queued || irq_line->masked || irq_line->disabled)) + return; + + ireq->gpio = cpu_to_le16(gpio); + sg_init_one(&req_sg, ireq, sizeof(*ireq)); + sg_init_one(&res_sg, ires, sizeof(*ires)); + sgs[0] = &req_sg; + sgs[1] = &res_sg; + + ret = virtqueue_add_sgs(vgpio->event_vq, sgs, 1, 1, irq_line, GFP_ATOMIC); + if (ret) { + dev_err(&vgpio->vdev->dev, "failed to add request to eventq\n"); + return; + } + + irq_line->queued = true; + virtqueue_kick(vgpio->event_vq); +} + +static void virtio_gpio_irq_enable(struct irq_data *d) +{ + struct gpio_chip *gc = irq_data_get_irq_chip_data(d); + struct virtio_gpio *vgpio = gpiochip_get_data(gc); + struct vgpio_irq_line *irq_line = &vgpio->irq_lines[d->hwirq]; + + raw_spin_lock(&vgpio->eventq_lock); + irq_line->disabled = false; + irq_line->masked = false; + irq_line->queue_pending = true; + raw_spin_unlock(&vgpio->eventq_lock); + + irq_line->update_pending = true; +} + +static void virtio_gpio_irq_disable(struct irq_data *d) +{ + struct gpio_chip *gc = irq_data_get_irq_chip_data(d); + struct virtio_gpio *vgpio = gpiochip_get_data(gc); + struct vgpio_irq_line *irq_line = &vgpio->irq_lines[d->hwirq]; + + raw_spin_lock(&vgpio->eventq_lock); + irq_line->disabled = true; + irq_line->masked = true; + irq_line->queue_pending = false; + raw_spin_unlock(&vgpio->eventq_lock); + + irq_line->update_pending = true; +} + +static void virtio_gpio_irq_mask(struct irq_data *d) +{ + struct gpio_chip *gc = irq_data_get_irq_chip_data(d); + struct virtio_gpio *vgpio = gpiochip_get_data(gc); + struct vgpio_irq_line *irq_line = &vgpio->irq_lines[d->hwirq]; + + raw_spin_lock(&vgpio->eventq_lock); + irq_line->masked = true; + raw_spin_unlock(&vgpio->eventq_lock); +} + +static void virtio_gpio_irq_unmask(struct irq_data *d) +{ + struct gpio_chip *gc = irq_data_get_irq_chip_data(d); + struct virtio_gpio *vgpio = gpiochip_get_data(gc); + struct vgpio_irq_line *irq_line = &vgpio->irq_lines[d->hwirq]; + + raw_spin_lock(&vgpio->eventq_lock); + irq_line->masked = false; + + /* Queue the buffer unconditionally on unmask */ + virtio_gpio_irq_prepare(vgpio, d->hwirq); + raw_spin_unlock(&vgpio->eventq_lock); +} + +static int virtio_gpio_irq_set_type(struct irq_data *d, unsigned int type) +{ + struct gpio_chip *gc = irq_data_get_irq_chip_data(d); + struct virtio_gpio *vgpio = gpiochip_get_data(gc); + struct vgpio_irq_line *irq_line = &vgpio->irq_lines[d->hwirq]; + + switch (type) { + case IRQ_TYPE_NONE: + type = VIRTIO_GPIO_IRQ_TYPE_NONE; + break; + case IRQ_TYPE_EDGE_RISING: + type = VIRTIO_GPIO_IRQ_TYPE_EDGE_RISING; + break; + case IRQ_TYPE_EDGE_FALLING: + type = VIRTIO_GPIO_IRQ_TYPE_EDGE_FALLING; + break; + case IRQ_TYPE_EDGE_BOTH: + type = VIRTIO_GPIO_IRQ_TYPE_EDGE_BOTH; + break; + case IRQ_TYPE_LEVEL_LOW: + type = VIRTIO_GPIO_IRQ_TYPE_LEVEL_LOW; + break; + case IRQ_TYPE_LEVEL_HIGH: + type = VIRTIO_GPIO_IRQ_TYPE_LEVEL_HIGH; + break; + default: + dev_err(&vgpio->vdev->dev, "unsupported irq type: %u\n", type); + return -EINVAL; + } + + irq_line->type = type; + irq_line->update_pending = true; + + return 0; +} + +static void virtio_gpio_irq_bus_lock(struct irq_data *d) +{ + struct gpio_chip *gc = irq_data_get_irq_chip_data(d); + struct virtio_gpio *vgpio = gpiochip_get_data(gc); + + mutex_lock(&vgpio->irq_lock); +} + +static void virtio_gpio_irq_bus_sync_unlock(struct irq_data *d) +{ + struct gpio_chip *gc = irq_data_get_irq_chip_data(d); + struct virtio_gpio *vgpio = gpiochip_get_data(gc); + struct vgpio_irq_line *irq_line = &vgpio->irq_lines[d->hwirq]; + u8 type = irq_line->disabled ? VIRTIO_GPIO_IRQ_TYPE_NONE : irq_line->type; + unsigned long flags; + + if (irq_line->update_pending) { + irq_line->update_pending = false; + virtio_gpio_req(vgpio, VIRTIO_GPIO_MSG_IRQ_TYPE, d->hwirq, type, + NULL); + + /* Queue the buffer only after interrupt is enabled */ + raw_spin_lock_irqsave(&vgpio->eventq_lock, flags); + if (irq_line->queue_pending) { + irq_line->queue_pending = false; + virtio_gpio_irq_prepare(vgpio, d->hwirq); + } + raw_spin_unlock_irqrestore(&vgpio->eventq_lock, flags); + } + + mutex_unlock(&vgpio->irq_lock); +} + +static struct irq_chip vgpio_irq_chip = { + .name = "virtio-gpio", + .irq_enable = virtio_gpio_irq_enable, + .irq_disable = virtio_gpio_irq_disable, + .irq_mask = virtio_gpio_irq_mask, + .irq_unmask = virtio_gpio_irq_unmask, + .irq_set_type = virtio_gpio_irq_set_type, + + /* These are required to implement irqchip for slow busses */ + .irq_bus_lock = virtio_gpio_irq_bus_lock, + .irq_bus_sync_unlock = virtio_gpio_irq_bus_sync_unlock, +}; + +static bool ignore_irq(struct virtio_gpio *vgpio, int gpio, + struct vgpio_irq_line *irq_line) +{ + bool ignore = false; + + raw_spin_lock(&vgpio->eventq_lock); + irq_line->queued = false; + + /* Interrupt is disabled currently */ + if (irq_line->masked || irq_line->disabled) { + ignore = true; + goto unlock; + } + + /* + * Buffer is returned as the interrupt was disabled earlier, but is + * enabled again now. Requeue the buffers. + */ + if (irq_line->ires.status == VIRTIO_GPIO_IRQ_STATUS_INVALID) { + virtio_gpio_irq_prepare(vgpio, gpio); + ignore = true; + goto unlock; + } + + if (WARN_ON(irq_line->ires.status != VIRTIO_GPIO_IRQ_STATUS_VALID)) + ignore = true; + +unlock: + raw_spin_unlock(&vgpio->eventq_lock); + + return ignore; +} + +static void virtio_gpio_event_vq(struct virtqueue *vq) +{ + struct virtio_gpio *vgpio = vq->vdev->priv; + struct device *dev = &vgpio->vdev->dev; + struct vgpio_irq_line *irq_line; + int irq, gpio, ret; + unsigned int len; + + while (true) { + irq_line = virtqueue_get_buf(vgpio->event_vq, &len); + if (!irq_line) + break; + + if (len != sizeof(irq_line->ires)) { + dev_err(dev, "irq with incorrect length (%u : %u)\n", + len, (unsigned int)sizeof(irq_line->ires)); + continue; + } + + /* + * Find GPIO line number from the offset of irq_line within the + * irq_lines block. We can also get GPIO number from + * irq-request, but better not to rely on a buffer returned by + * remote. + */ + gpio = irq_line - vgpio->irq_lines; + WARN_ON(gpio >= vgpio->gc.ngpio); + + if (unlikely(ignore_irq(vgpio, gpio, irq_line))) + continue; + + irq = irq_find_mapping(vgpio->gc.irq.domain, gpio); + WARN_ON(!irq); + + ret = generic_handle_irq(irq); + if (ret) + dev_err(dev, "failed to handle interrupt: %d\n", ret); + }; +} + static void virtio_gpio_request_vq(struct virtqueue *vq) { struct virtio_gpio_line *line; @@ -210,14 +467,15 @@ static void virtio_gpio_free_vqs(struct virtio_device *vdev) static int virtio_gpio_alloc_vqs(struct virtio_gpio *vgpio, struct virtio_device *vdev) { - const char * const names[] = { "requestq" }; + const char * const names[] = { "requestq", "eventq" }; vq_callback_t *cbs[] = { virtio_gpio_request_vq, + virtio_gpio_event_vq, }; - struct virtqueue *vqs[1] = { NULL }; + struct virtqueue *vqs[2] = { NULL, NULL }; int ret; - ret = virtio_find_vqs(vdev, 1, vqs, cbs, names, NULL); + ret = virtio_find_vqs(vdev, vgpio->irq_lines ? 2 : 1, vqs, cbs, names, NULL); if (ret) { dev_err(&vdev->dev, "failed to find vqs: %d\n", ret); return ret; @@ -225,11 +483,23 @@ static int virtio_gpio_alloc_vqs(struct virtio_gpio *vgpio, if (!vqs[0]) { dev_err(&vdev->dev, "failed to find requestq vq\n"); - return -ENODEV; + goto out; } vgpio->request_vq = vqs[0]; + if (vgpio->irq_lines && !vqs[1]) { + dev_err(&vdev->dev, "failed to find eventq vq\n"); + goto out; + } + vgpio->event_vq = vqs[1]; + return 0; + +out: + if (vqs[0] || vqs[1]) + virtio_gpio_free_vqs(vdev); + + return -ENODEV; } static const char **virtio_gpio_get_names(struct virtio_gpio *vgpio, @@ -325,6 +595,32 @@ static int virtio_gpio_probe(struct virtio_device *vdev) vgpio->gc.owner = THIS_MODULE; vgpio->gc.can_sleep = true; + /* Interrupt support */ + if (virtio_has_feature(vdev, VIRTIO_GPIO_F_IRQ)) { + vgpio->irq_lines = devm_kcalloc(dev, ngpio, + sizeof(*vgpio->irq_lines), + GFP_KERNEL); + if (!vgpio->irq_lines) + return -ENOMEM; + + /* The event comes from the outside so no parent handler */ + vgpio->gc.irq.parent_handler = NULL; + vgpio->gc.irq.num_parents = 0; + vgpio->gc.irq.parents = NULL; + vgpio->gc.irq.default_type = IRQ_TYPE_NONE; + vgpio->gc.irq.handler = handle_level_irq; + vgpio->gc.irq.chip = &vgpio_irq_chip; + + for (i = 0; i < ngpio; i++) { + vgpio->irq_lines[i].type = VIRTIO_GPIO_IRQ_TYPE_NONE; + vgpio->irq_lines[i].disabled = true; + vgpio->irq_lines[i].masked = true; + } + + mutex_init(&vgpio->irq_lock); + raw_spin_lock_init(&vgpio->eventq_lock); + } + ret = virtio_gpio_alloc_vqs(vgpio, vdev); if (ret) return ret; @@ -357,7 +653,13 @@ static const struct virtio_device_id id_table[] = { }; MODULE_DEVICE_TABLE(virtio, id_table); +static const unsigned int features[] = { + VIRTIO_GPIO_F_IRQ, +}; + static struct virtio_driver virtio_gpio_driver = { + .feature_table = features, + .feature_table_size = ARRAY_SIZE(features), .id_table = id_table, .probe = virtio_gpio_probe, .remove = virtio_gpio_remove, diff --git a/include/uapi/linux/virtio_gpio.h b/include/uapi/linux/virtio_gpio.h index 0445f905d8cc..d04af9c5f0de 100644 --- a/include/uapi/linux/virtio_gpio.h +++ b/include/uapi/linux/virtio_gpio.h @@ -5,12 +5,16 @@ #include <linux/types.h> +/* Virtio GPIO Feature bits */ +#define VIRTIO_GPIO_F_IRQ 0 + /* Virtio GPIO request types */ #define VIRTIO_GPIO_MSG_GET_NAMES 0x0001 #define VIRTIO_GPIO_MSG_GET_DIRECTION 0x0002 #define VIRTIO_GPIO_MSG_SET_DIRECTION 0x0003 #define VIRTIO_GPIO_MSG_GET_VALUE 0x0004 #define VIRTIO_GPIO_MSG_SET_VALUE 0x0005 +#define VIRTIO_GPIO_MSG_IRQ_TYPE 0x0006 /* Possible values of the status field */ #define VIRTIO_GPIO_STATUS_OK 0x0 @@ -21,6 +25,14 @@ #define VIRTIO_GPIO_DIRECTION_OUT 0x01 #define VIRTIO_GPIO_DIRECTION_IN 0x02 +/* Virtio GPIO IRQ types */ +#define VIRTIO_GPIO_IRQ_TYPE_NONE 0x00 +#define VIRTIO_GPIO_IRQ_TYPE_EDGE_RISING 0x01 +#define VIRTIO_GPIO_IRQ_TYPE_EDGE_FALLING 0x02 +#define VIRTIO_GPIO_IRQ_TYPE_EDGE_BOTH 0x03 +#define VIRTIO_GPIO_IRQ_TYPE_LEVEL_HIGH 0x04 +#define VIRTIO_GPIO_IRQ_TYPE_LEVEL_LOW 0x08 + struct virtio_gpio_config { __le16 ngpio; __u8 padding[2]; @@ -44,4 +56,17 @@ struct virtio_gpio_response_get_names { __u8 value[]; }; +/* Virtio GPIO IRQ Request / Response */ +struct virtio_gpio_irq_request { + __le16 gpio; +}; + +struct virtio_gpio_irq_response { + __u8 status; +}; + +/* Possible values of the interrupt status field */ +#define VIRTIO_GPIO_IRQ_STATUS_INVALID 0x0 +#define VIRTIO_GPIO_IRQ_STATUS_VALID 0x1 + #endif /* _LINUX_VIRTIO_GPIO_H */ -- 2.31.1.272.g89b43f80a514

2 years, 9 months

4
11
0 0

[PATCH V10] virtio-gpio: Add support for interrupts

by Viresh Kumar

This patch adds support for interrupts to the virtio-gpio specification. This uses the feature bit 0 for the same. Fixes: https://github.com/oasis-tcs/virtio-spec/issues/114 Cc: Marc Zyngier <maz(a)kernel.org> Cc: Thomas Gleixner <tglx(a)linutronix.de> Reviewed-by: Linus Walleij <linus.walleij(a)linaro.org> Signed-off-by: Viresh Kumar <viresh.kumar(a)linaro.org> --- V9 -> V10: - Mandatory for the interrupt to be enabled before being unmasked. - Remove unnecessary stuff from Device Initialization and clean it up. - Minor cleanup (s/enabled/negotiated, etc). V8 -> V9: - The patch for base GPIO specification is already merged, sending this separately now. - Differentiate properly between enabling/disabling and masking/unmasking of the interrupt. - Specify how a trigger type should be changed, i.e. by disabling interrupt first. - No fixed sequence for enabling/unmasking of the interrupt, any of them can be done first. The interrupt is only delivered once it is enabled and unmasked. - Use normative text only in normative sections. - Guest side Linux driver's IRQ implementation: https://lore.kernel.org/linux-gpio/96223fb8143a4eaa9b183d376ff46e5cd8ef54b4… conformance.tex | 2 + virtio-gpio.tex | 245 +++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 246 insertions(+), 1 deletion(-) diff --git a/conformance.tex b/conformance.tex index c52f1a40be2d..64bcc12d1199 100644 --- a/conformance.tex +++ b/conformance.tex @@ -310,6 +310,7 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets} \begin{itemize} \item \ref{drivernormative:Device Types / GPIO Device / requestq Operation} +\item \ref{drivernormative:Device Types / GPIO Device / eventq Operation} \end{itemize} \conformance{\section}{Device Conformance}\label{sec:Conformance / Device Conformance} @@ -568,6 +569,7 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets} \begin{itemize} \item \ref{devicenormative:Device Types / GPIO Device / requestq Operation} +\item \ref{devicenormative:Device Types / GPIO Device / eventq Operation} \end{itemize} \conformance{\section}{Legacy Interface: Transitional Device and Transitional Driver Conformance}\label{sec:Conformance / Legacy Interface: Transitional Device and Transitional Driver Conformance} diff --git a/virtio-gpio.tex b/virtio-gpio.tex index 3c614ec97b92..55c553f92a14 100644 --- a/virtio-gpio.tex +++ b/virtio-gpio.tex @@ -11,11 +11,17 @@ \subsection{Virtqueues}\label{sec:Device Types / GPIO Device / Virtqueues} \begin{description} \item[0] requestq +\item[1] eventq \end{description} +The \field{eventq} virtqueue is available only if the \field{VIRTIO_GPIO_F_IRQ} +feature is offered by the device. + \subsection{Feature bits}\label{sec:Device Types / GPIO Device / Feature bits} -None currently defined. +\begin{description} +\item[VIRTIO_GPIO_F_IRQ (0)] The device supports interrupts on GPIO lines. +\end{description} \subsection{Device configuration layout}\label{sec:Device Types / GPIO Device / Device configuration layout} @@ -46,6 +52,9 @@ \subsection{Device Initialization}\label{sec:Device Types / GPIO Device / Device \begin{itemize} \item The driver configures and initializes the \field{requestq} virtqueue. + +\item The driver configures and initializes the \field{eventq} virtqueue, if the + \field{VIRTIO_GPIO_F_IRQ} feature has been negotiated. \end{itemize} \subsection{Device Operation: requestq}\label{sec:Device Types / GPIO Device / requestq Operation} @@ -105,11 +114,20 @@ \subsection{Device Operation: requestq}\label{sec:Device Types / GPIO Device / r #define VIRTIO_GPIO_MSG_SET_DIRECTION 0x0003 #define VIRTIO_GPIO_MSG_GET_VALUE 0x0004 #define VIRTIO_GPIO_MSG_SET_VALUE 0x0005 +#define VIRTIO_GPIO_MSG_SET_IRQ_TYPE 0x0006 /* GPIO Direction types */ #define VIRTIO_GPIO_DIRECTION_NONE 0x00 #define VIRTIO_GPIO_DIRECTION_OUT 0x01 #define VIRTIO_GPIO_DIRECTION_IN 0x02 + +/* GPIO interrupt types */ +#define VIRTIO_GPIO_IRQ_TYPE_NONE 0x00 +#define VIRTIO_GPIO_IRQ_TYPE_EDGE_RISING 0x01 +#define VIRTIO_GPIO_IRQ_TYPE_EDGE_FALLING 0x02 +#define VIRTIO_GPIO_IRQ_TYPE_EDGE_BOTH 0x03 +#define VIRTIO_GPIO_IRQ_TYPE_LEVEL_HIGH 0x04 +#define VIRTIO_GPIO_IRQ_TYPE_LEVEL_LOW 0x08 \end{lstlisting} \subsubsection{requestq Operation: Get Line Names}\label{sec:Device Types / GPIO Device / requestq Operation / Get Line Names} @@ -270,6 +288,73 @@ \subsubsection{requestq Operation: Set Value}\label{sec:Device Types / GPIO Devi \hline \end{tabularx} +\subsubsection{requestq Operation: Set IRQ Type}\label{sec:Device Types / GPIO Device / requestq Operation / Set IRQ Type} + +This request is allowed only if the \field{VIRTIO_GPIO_F_IRQ} feature has been +negotiated. + +The interrupt configuration is divided into two steps, enabling or disabling of +the interrupt at the device and masking or unmasking of the interrupt for +delivery at the driver. This request only pertains to enabling or disabling of +the interrupt at the device, the masking and unmasking of the interrupt is +handled by a separate request that takes place over the \field{eventq} +virtqueue. + +The driver sends the \field{VIRTIO_GPIO_MSG_SET_IRQ_TYPE} message over the +\field{requestq} virtqueue to enable or disable interrupt for a GPIO line at +the device. + +The driver sends this message with trigger type set to any valid value other +than \field{VIRTIO_GPIO_IRQ_TYPE_NONE}, to enable the interrupt for a GPIO line, +this doesn't unmask the interrupt for delivery at the driver though. For edge +trigger type, the device should latch the interrupt events from this point +onward and notify it to the driver once the interrupt is unmasked. For level +trigger type, the device should notify the driver once the interrupt signal on a +line is sensed and the interrupt is unmasked for the line. + +The driver sends this message with trigger type set to +\field{VIRTIO_GPIO_IRQ_TYPE_NONE}, to disable the interrupt for a GPIO line. The +device should discard any latched interrupt event associated with it. In order +to change the trigger type of an already enabled interrupt, the driver should +first disable the interrupt and then re-enable it with appropriate trigger type. + +The interrupts are masked at initialization and the driver unmasks them by +queuing a pair of buffers, of type \field{virtio_gpio_irq_request} and +\field{virtio_gpio_irq_response}, over the \field{eventq} virtqueue for a GPIO +line. A separate pair of buffers must be queued for each GPIO line, the driver +wants to configure for interrupts. Once an already enabled interrupt is unmasked +by the driver, the device can notify the driver of an active interrupt signal on +the GPIO line. This is done by updating the \field{struct +virtio_gpio_irq_response} buffer's \field{status} with +\field{VIRTIO_GPIO_IRQ_STATUS_VALID} and returning the updated buffers to the +driver. The interrupt is masked automatically at this point until the buffers +are available again at the device. + +The interrupt for a GPIO line should not be unmasked before being enabled by the +driver. + +If the interrupt is disabled by the driver, by setting the trigger type to +\field{VIRTIO_GPIO_IRQ_TYPE_NONE}, or the interrupt is unmasked without being +enabled first, the device should return any unused pair of buffers for the GPIO +line, over the \field{eventq} virtqueue, after setting the \field{status} field +to \field{VIRTIO_GPIO_IRQ_STATUS_INVALID}. This also masks the interrupt. + +\begin{tabularx}{\textwidth}{ |l||X|X|X| } +\hline +\textbf{Request} & \field{type} & \field{gpio} & \field{value} \\ +\hline +& \field{VIRTIO_GPIO_MSG_SET_IRQ_TYPE} & line number & one of \field{VIRTIO_GPIO_IRQ_TYPE_*} \\ +\hline +\end{tabularx} + +\begin{tabularx}{\textwidth}{ |l||X|X| } +\hline +\textbf{Response} & \field{status} & \field{value} \\ +\hline +& \field{VIRTIO_GPIO_STATUS_*} & 0 \\ +\hline +\end{tabularx} + \subsubsection{requestq Operation: Message Flow}\label{sec:Device Types / GPIO Device / requestq Operation / Message Flow} \begin{itemize} @@ -313,6 +398,20 @@ \subsubsection{requestq Operation: Message Flow}\label{sec:Device Types / GPIO D \item The driver MAY send multiple messages for same or different GPIO lines in parallel. + +\item The driver MUST NOT send IRQ messages if the \field{VIRTIO_GPIO_F_IRQ} + feature has not been negotiated. + +\item The driver MUST NOT send IRQ messages for a GPIO line configured for + output. + +\item The driver MUST set the IRQ trigger type to + \field{VIRTIO_GPIO_IRQ_TYPE_NONE} once it is done using the GPIO line + configured for interrupts. + +\item In order to change the trigger type of an already enabled interrupt, the + driver MUST first disable the interrupt and then re-enable it with + appropriate trigger type. \end{itemize} \devicenormative{\subsubsection}{requestq Operation}{Device Types / GPIO Device / requestq Operation} @@ -344,4 +443,148 @@ \subsubsection{requestq Operation: Message Flow}\label{sec:Device Types / GPIO D \item The device MUST discard all state information corresponding to a GPIO line, once the driver has requested to set its direction to \field{VIRTIO_GPIO_DIRECTION_NONE}. + +\item The device MUST latch an edge interrupt if the interrupt is enabled but + still masked. + +\item The device MUST NOT latch an level interrupt if the interrupt is enabled + but still masked. + +\item The device MUST discard any latched interrupt for a GPIO line, once + interrupt is disabled for the same. +\end{itemize} + +\subsection{Device Operation: eventq}\label{sec:Device Types / GPIO Device / eventq Operation} + +The \field{eventq} virtqueue is used by the driver to unmask the interrupts and +used by the device to notify the driver of newly sensed interrupts. In order to +unmask interrupt on a GPIO line, the driver queues a pair of buffers, +\field{struct virtio_gpio_irq_request} (filled by driver) and \field{struct +virtio_gpio_irq_response} (to be filled by device later), to the \field{eventq} +virtqueue. A separate pair of buffers must be queued for each GPIO line, the +driver wants to configure for interrupts. The device, on sensing an interrupt, +returns the pair of buffers for the respective GPIO line, which also masks the +interrupts. The driver can queue the buffers again to unmask the interrupt. + +\begin{lstlisting} +struct virtio_gpio_irq_request { + le16 gpio; +}; +\end{lstlisting} + +This structure is filled by the driver and read by the device. + +\begin{description} +\item[\field{gpio}] is the GPIO line number, i.e. 0 <= \field{gpio} < + \field{ngpio}. +\end{description} + +\begin{lstlisting} +struct virtio_gpio_irq_response { + u8 status; +}; + +/* Possible values of the interrupt status field */ +#define VIRTIO_GPIO_IRQ_STATUS_INVALID 0x0 +#define VIRTIO_GPIO_IRQ_STATUS_VALID 0x1 +\end{lstlisting} + +This structure is filled by the device and read by the driver. + +\begin{description} +\item[\field{status}] of the interrupt event, + \field{VIRTIO_GPIO_IRQ_STATUS_VALID} on interrupt and + \field{VIRTIO_GPIO_IRQ_STATUS_INVALID} to return the buffers back to the + driver after interrupt is disabled. +\end{description} + +\subsubsection{eventq Operation: Message Flow}\label{sec:Device Types / GPIO Device / eventq Operation / Message Flow} + +\begin{itemize} +\item The virtio-gpio driver is requested by a client driver to enable interrupt + for a GPIO line and configure it to a particular trigger type. + +\item The driver sends the \field{VIRTIO_GPIO_MSG_SET_IRQ_TYPE} message, over + the \field{requestq} virtqueue, and the device configures the GPIO line for + the requested trigger type and enables the interrupt. The interrupt is still + masked for delivery though. The device shall latch the interrupt from now + onward for edge trigger type. + +\item The driver unmasks the interrupt by queuing a pair of buffers to the + \field{eventq} virtqueue for the GPIO line. The driver can do this before + enabling the interrupt as well, though the interrupt must be both unmasked + and enabled to get delivered at the driver. + +\item The driver notifies the device of the presence of new buffers on the + \field{eventq} virtqueue. The interrupt is fully configured at this point. + +\item The device, on sensing an active interrupt on the GPIO line, finds the + matching buffers (based on GPIO line number) from the \field{eventq} + virtqueue and update its \field{struct virtio_gpio_irq_response} buffer's + \field{status} with \field{VIRTIO_GPIO_IRQ_STATUS_VALID} and returns the + pair of buffers to the device. This results in masking the interrupt as + well. + +\item The device notifies the driver of the presence of returned buffers on the + \field{eventq} virtqueue. + +\item If the GPIO line is configured for level interrupts, the device ignores + any further interrupt signals on this GPIO line, until the interrupt is + unmasked again by the driver by making the buffers available to the device. + Once the interrupt is unmasked again and the interrupt on the line is still + active, the device shall notify the driver again. + +\item If the GPIO line is configured for edge interrupts, the device latches + the interrupt received for this GPIO line, until the interrupt is unmasked + again by the driver by making the buffers available to the device. Once the + interrupt is unmasked again and an interrupt was latched earlier, the + device shall notify the driver again. + +\item The driver on receiving the notification from the device, processes the + interrupt. The interrupt is masked at the device until the buffers are + queued again by the driver. + +\item In a typical guest operating system kernel, the virtio-gpio driver + notifies the client driver, that is associated with this GPIO line, to + process the event. In the case of a level triggered interrupt, the client + driver shall fully process and acknowledge the event at its source to return + the line to its inactive state before the interrupt is unmasked again to + avoid a spurious interrupt. + +\item Once the interrupt is handled, the driver may queue a pair of buffers for + the GPIO line to unmask the interrupt again. + +\item The driver can also disable the interrupt by sending the + \field{VIRTIO_GPIO_MSG_SET_IRQ_TYPE} message, with + \field{VIRTIO_GPIO_IRQ_TYPE_NONE} trigger type. In that case, the device + shall return the unused pair of buffers for the GPIO line after setting the + \field{status} field with \field{VIRTIO_GPIO_IRQ_STATUS_INVALID}. +\end{itemize} + +\drivernormative{\subsubsection}{eventq Operation}{Device Types / GPIO Device / eventq Operation} + +\begin{itemize} +\item The driver MUST both enable and unmask the interrupt in order to get + notified for the same. + +\item The driver MUST enable the interrupt before unmasking it. + +\item To unmask the interrupt, the driver MUST queue a separate pair of buffers + to the \field{eventq} virtqueue for each GPIO line. + +\item The driver MUST NOT add multiple pairs of buffers for the same GPIO line + on the \field{eventq} virtqueue. +\end{itemize} + +\devicenormative{\subsubsection}{eventq Operation}{Device Types / GPIO Device / eventq Operation} + +\begin{itemize} +\item The device MUST NOT send an interrupt event to the driver for a GPIO + line unless the interrupt has been both unmasked and enabled by the + driver. + +\item On receiving \field{VIRTIO_GPIO_MSG_SET_IRQ_TYPE} message, with + \field{VIRTIO_GPIO_IRQ_TYPE_NONE} trigger type, the device MUST return the + buffers, if they were received earlier, after setting the \field{status} + field to \field{VIRTIO_GPIO_IRQ_STATUS_INVALID}. \end{itemize} -- 2.31.1.272.g89b43f80a514

2 years, 9 months

2
1
0 0

Xen Configs for booting Xen with Linux Dom0 via u-boot/tftp

by Alex Bennée

Hi, I've dumped a bunch of files on my Stratos file share: https://fileserver.linaro.org/s/jC3Xq2q297j5bQ8 The first directory (xenpi4.boot) is the result of a Yocto demo build (as per Christopher Clark's instructions) and contains a known good version of the firmware, u-boot and with the rest of the image would boot a Yocto distro image. As I like understand the full process and I wanted to have a full self hosted distro set up for development I set up a TFTP based setup where u-boot gets the hypervisor, kernel and DTB over the network. Most of the work it turned out was in finding the correct u-boot runes to both subtly modify the DTB to account for HW variations (for some reason I couldn't get u-boot to modify the master DTB from the GPU firmware). We also need to make some modifications to /chosen nodes to ensure Xen boots up out DomO kernel. The source of the script (ajb.configs/boot-xenpi.source) is a heavily commented boot script that will give you the blow-by-blow details. With this I can boot my own custom hypervisor and kernel into a Debian Bullseye install. You can find known working configs for those components in the directory as well. My distro is running a xen-upstream.deb built from the Xen source tree: make -j9 debball CROSS_COMPILE=aarch64-linux-gnu- XEN_TARGET_ARCH=arm64 I've also hacked the distro a little: - removed the kernels and raspi-firmware packages (so as not to mess with /boot/firmware) - don' auto-mount /boot.firmware - hand built a QEMU which is pointed to by QEMU_XEN in /etc/default/xencommons Be aware the Dom0 has 1G of the RAM by default, if you plan to do some heavy building you might want to give it a bit more memory to play with. I'll upload my test image and xen config in due course but this should give something that is fairly easy to iterate with. I've not yet run rustup on the pi itself though ;-) -- Alex Bennée

2 years, 9 months

2
3
0 0

Dump of my current Xen notes

by Alex Bennée

Hi Viresh/Mathieu, This is a dump of the current state of my notes for cross-building and running Xen via QEMU. It's a little bit of a stream of consciousness as I wrote things down but hopefully it proves useful in getting things started. ━━━━━━━━━━━━━━━━━━━ XEN RELATED NOTES Alex Bennée ━━━━━━━━━━━━━━━━━━━ Table of Contents ───────────────── 1. Tasks [0/1] :tasks: .. 1. TODO Xen SMC/HVC pass through ([STR-41]) 2. Notes .. 1. Building Xen .. 2. Cross Build .. 3. Cross Packages .. 4. Configure .. 5. Build ..... 1. Just the Hypervisor .. 6. Running Dom0 .. 7. MachiatoBin Issues .. 8. Running DomU .. 9. View console of DomU guest 1 Tasks [0/1] :tasks: ═════════════ [STR-41] <https://projects.linaro.org/browse/STR-41> 1.1 TODO Xen SMC/HVC pass through ([STR-41]) ──────────────────────────────────────────── > sstabellini_: that tracks [ 1.420679] mvebu-comphy f2120000.phy: > unsupported SMC call, try updating your firmware [ 1.420720] > mvebu-comphy f2120000.phy: Firmware could not configure PHY 4 with > mode 15 (ret: -95), trying legacy method sstabellini_: is that a > boot option? <sstabellini_> stsquad: as an example see [19:55] > xen/arch/arm/platforms/xilinx-zynqmp-eemi.c:forward_to_fw > <sstabellini_> stsquad: we don't have a boot option for that, you > need to add a few lines of code. Actually this issue comes up often > enough that a boot option would be really useful! [STR-41] <https://projects.linaro.org/browse/STR-41> 2 Notes ═══════ 2.1 Building Xen ──────────────── As Xen's out-of-tree support is a little sporadic I've ended up going for using git-worktree to split out the various builds. 2.2 Cross Build ─────────────── 2.3 Cross Packages ────────────────── ┌──── │ apt install libpython3-dev:arm64 libfdt-dev:arm64 libncurses-dev:arm64 └──── Or as I do on hackbox do everything in a container: ┌──── │ docker run --rm -it -u (id -u) -v $HOME:$HOME -w (pwd) alex.bennee:xen-arm64 /usr/bin/fish └──── built from [my dockerfile] [my dockerfile] <https://github.com/stsquad/dockerfiles/blob/master/crossbuild/xen-arm64/Doc…> 2.4 Configure ───────────── ┌──── │ ./configure --build=x86_64-unknown-linux-gnu --host=aarch64-linux-gnu \ │ --disable-docs --disable-golang --disable-ocamltools \ │ --with-system-qemu=/usr/bin/qemu-system-i386 └──── The location of system QEMU can be tweaked after the fact by editing /etc/default/xencommons 2.5 Build ───────── You need to have the compilers in the command line: ┌──── │ make -j9 dist CROSS_COMPILE=aarch64-linux-gnu- XEN_TARGET_ARCH=arm64 └──── To build an Debian package which can install all the tools and binaries (replacing any distro requirements) then: ┌──── │ make -j9 debball CROSS_COMPILE=aarch64-linux-gnu- XEN_TARGET_ARCH=arm64 └──── This builds a package which you can find in ./dist/xen-upstream-4.15-unstable.deb which you can dpkg -i on your dom0 environment. 2.5.1 Just the Hypervisor ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌ ┌──── │ cd xen │ set -x CROSS_COMPILE aarch64-linux-gnu- │ make XEN_TARGET_ARCH=arm64 └──── 2.6 Running Dom0 ──────────────── To avoid complications with broken grub and bios firmware I'm currently just using my direct boot: ┌──── │ ./qemu-system-aarch64 -machine virt,virtualization=on \ │ -cpu cortex-a57 -serial mon:stdio \ │ -nic user,model=virtio-net-pci,hostfwd=tcp::2222-:22 \ │ -device virtio-scsi-pci \ │ -drive file=/dev/zvol/hackpool-0/debian-buster-arm64,id=hd0,index=0,if=none,format=raw \ │ -device scsi-hd,drive=hd0 \ │ -display none \ │ -m 8192 \ │ -kernel ~/lsrc/xen/xen.build.arm64-xen/xen/xen \ │ -append "dom0_mem=2G,max:2G dom0_max_vcpus=4 loglvl=all guest_loglvl=all" \ │ -device guest-loader,addr=0x46000000,kernel=$HOME/lsrc/linux.git/builds/arm64/arch/arm64/boot/Image,bootargs="root=/dev/sda2 console=hvc0 earlyprintk=xen" \ │ -smp 8 └──── Care has to be taken to avoid the guest-loader address clashing with anything else and ending up corrupting the DTB. Currently QEMU doesn't do anything avoid those issues as it doesn't have visibility of where the kernel gets loaded. You can see by examining the Xen output: ┌──── │ (XEN) Loading zImage from 0000000046000000 to 0000000050000000-0000000050eb2200 │ (XEN) Loading d0 DTB to 0x0000000058000000-0x0000000058001ce8 └──── The initial xl list will check Xen and it's tools are up and running and the hypervisor and userspace ABI's are in sync. It's important that the hypercall ABI and it's userspace are in sync. ┌──── │ 13:13:43 [root@buster:~] # xl list │ Name ID Mem VCPUs State Time(s) │ Domain-0 0 4096 4 r----- 215.8 └──── 2.7 MachiatoBin Issues ────────────────────── • upgraded to Bullseye for latest Grub • built with 8250 serial (32 bit, shift = 2) 2.8 Running DomU ──────────────── The with a config like ┌──── │ # =====================================================================# Example PV Linux guest configuration │ # ===================================================================== │ # │ # This is a fairly minimal example of what is required for a │ # Paravirtualised Linux guest. For a more complete guide see xl.cfg(5) │ │ # Guest name │ name = "xenpv-initrd-guest" │ │ # 128-bit UUID for the domain as a hexadecimal number. │ # Use "uuidgen" to generate one if required. │ # The default behavior is to generate a new UUID each time the guest is started. │ #uuid = "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX" │ │ # Kernel image to boot │ kernel = "/home/alex/arm64-minkern" │ # kernel = "/home/alex/arm64-kernel" │ # kernel = "/home/alex/arm64-defconfig" │ │ # Ramdisk (optional) │ #ramdisk = "/boot/initrd.gz" │ │ # Kernel command line options │ extra = "console=hvc0" │ │ # Initial memory allocation (MB) │ memory = 4096 │ │ # Maximum memory (MB) │ # If this is greater than `memory' then the slack will start ballooned │ # (this assumes guest kernel support for ballooning) │ maxmem = 4096 │ │ # Number of VCPUS │ vcpus = 2 │ │ # Network devices │ # A list of 'vifspec' entries as described in │ # docs/misc/xl-network-configuration.markdown │ # vif = [ ] │ │ # Disk Devices │ # A list of `diskspec' entries as described in │ # docs/misc/xl-disk-configuration.txt │ # disk = [ '/dev/vg/guest-volume,raw,xvda,rw' ] │ # disk = [ ] └──── ┌──── │ xl create simple-guest.conf │ xl list │ Name ID Mem VCPUs State Time(s) │ Domain-0 0 4096 4 r----- 221.4 │ xenpv-initrd-guest 1 4095 2 -b---- 5.3 └──── 2.9 View console of DomU guest ────────────────────────────── And to get console: ┌──── │ xl console -t pv -n 0 xenpv-initrd-guest └──── to exit you need *Ctrl-[* -- Alex Bennée

2 years, 9 months

6
11
0 0

[PATCH V9] virtio-gpio: Add support for interrupts

by Viresh Kumar

This patch adds support for interrupts to the virtio-gpio specification. This uses the feature bit 0 for the same. Fixes: https://github.com/oasis-tcs/virtio-spec/issues/114 Cc: Marc Zyngier <maz(a)kernel.org> Cc: Thomas Gleixner <tglx(a)linutronix.de> Reviewed-by: Linus Walleij <linus.walleij(a)linaro.org> Signed-off-by: Viresh Kumar <viresh.kumar(a)linaro.org> --- V8 -> V9: - The patch for base GPIO specification is already merged, sending this separately now. - Differentiate properly between enabling/disabling and masking/unmasking of the interrupt. - Specify how a trigger type should be changed, i.e. by disabling interrupt first. - No fixed sequence for enabling/unmasking of the interrupt, any of them can be done first. The interrupt is only delivered once it is enabled and unmasked. - Use normative text only in normative sections. - Guest side Linux driver's IRQ implementation: https://lore.kernel.org/linux-gpio/96223fb8143a4eaa9b183d376ff46e5cd8ef54b4… conformance.tex | 2 + virtio-gpio.tex | 248 +++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 249 insertions(+), 1 deletion(-) diff --git a/conformance.tex b/conformance.tex index c52f1a40be2d..64bcc12d1199 100644 --- a/conformance.tex +++ b/conformance.tex @@ -310,6 +310,7 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets} \begin{itemize} \item \ref{drivernormative:Device Types / GPIO Device / requestq Operation} +\item \ref{drivernormative:Device Types / GPIO Device / eventq Operation} \end{itemize} \conformance{\section}{Device Conformance}\label{sec:Conformance / Device Conformance} @@ -568,6 +569,7 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets} \begin{itemize} \item \ref{devicenormative:Device Types / GPIO Device / requestq Operation} +\item \ref{devicenormative:Device Types / GPIO Device / eventq Operation} \end{itemize} \conformance{\section}{Legacy Interface: Transitional Device and Transitional Driver Conformance}\label{sec:Conformance / Legacy Interface: Transitional Device and Transitional Driver Conformance} diff --git a/virtio-gpio.tex b/virtio-gpio.tex index 3c614ec97b92..fba7ec400bb3 100644 --- a/virtio-gpio.tex +++ b/virtio-gpio.tex @@ -11,11 +11,17 @@ \subsection{Virtqueues}\label{sec:Device Types / GPIO Device / Virtqueues} \begin{description} \item[0] requestq +\item[1] eventq \end{description} +The \field{eventq} virtqueue is available only if the \field{VIRTIO_GPIO_F_IRQ} +feature is enabled by the device. + \subsection{Feature bits}\label{sec:Device Types / GPIO Device / Feature bits} -None currently defined. +\begin{description} +\item[VIRTIO_GPIO_F_IRQ (0)] The device supports interrupts on GPIO lines. +\end{description} \subsection{Device configuration layout}\label{sec:Device Types / GPIO Device / Device configuration layout} @@ -46,6 +52,14 @@ \subsection{Device Initialization}\label{sec:Device Types / GPIO Device / Device \begin{itemize} \item The driver configures and initializes the \field{requestq} virtqueue. + +\item The driver checks the presence of \field{VIRTIO_GPIO_F_IRQ} feature + before initiating any IRQ related messages. + +\item The driver configures and initializes the \field{eventq} virtqueue. + +\item The device configures all GPIO lines in \field{VIRTIO_GPIO_IRQ_TYPE_NONE} + trigger type state. \end{itemize} \subsection{Device Operation: requestq}\label{sec:Device Types / GPIO Device / requestq Operation} @@ -105,11 +119,20 @@ \subsection{Device Operation: requestq}\label{sec:Device Types / GPIO Device / r #define VIRTIO_GPIO_MSG_SET_DIRECTION 0x0003 #define VIRTIO_GPIO_MSG_GET_VALUE 0x0004 #define VIRTIO_GPIO_MSG_SET_VALUE 0x0005 +#define VIRTIO_GPIO_MSG_SET_IRQ_TYPE 0x0006 /* GPIO Direction types */ #define VIRTIO_GPIO_DIRECTION_NONE 0x00 #define VIRTIO_GPIO_DIRECTION_OUT 0x01 #define VIRTIO_GPIO_DIRECTION_IN 0x02 + +/* GPIO interrupt types */ +#define VIRTIO_GPIO_IRQ_TYPE_NONE 0x00 +#define VIRTIO_GPIO_IRQ_TYPE_EDGE_RISING 0x01 +#define VIRTIO_GPIO_IRQ_TYPE_EDGE_FALLING 0x02 +#define VIRTIO_GPIO_IRQ_TYPE_EDGE_BOTH 0x03 +#define VIRTIO_GPIO_IRQ_TYPE_LEVEL_HIGH 0x04 +#define VIRTIO_GPIO_IRQ_TYPE_LEVEL_LOW 0x08 \end{lstlisting} \subsubsection{requestq Operation: Get Line Names}\label{sec:Device Types / GPIO Device / requestq Operation / Get Line Names} @@ -270,6 +293,74 @@ \subsubsection{requestq Operation: Set Value}\label{sec:Device Types / GPIO Devi \hline \end{tabularx} +\subsubsection{requestq Operation: Set IRQ Type}\label{sec:Device Types / GPIO Device / requestq Operation / Set IRQ Type} + +This request is allowed only if the \field{VIRTIO_GPIO_F_IRQ} feature is enabled +by the device. + +The interrupt configuration is divided into two steps, enabling or disabling of +the interrupt at the device and masking or unmasking of the interrupt for +delivery at the driver. This request only pertains to enabling or disabling of +the interrupt at the device, the masking and unmasking of the interrupt is +handled by a separate request that takes place over the \field{eventq} +virtqueue. + +The driver sends the \field{VIRTIO_GPIO_MSG_SET_IRQ_TYPE} message over the +\field{requestq} virtqueue to enable or disable interrupt for a GPIO line at +the device. + +The driver sends this message with trigger type set to any valid value other +than \field{VIRTIO_GPIO_IRQ_TYPE_NONE}, to enable the interrupt for a GPIO line, +this doesn't unmask the interrupt for delivery at the driver though. For edge +trigger type, the device should latch the interrupt events from this point +onward and notify it to the driver once the interrupt is unmasked. For level +trigger type, the device should notify the driver once the interrupt signal on a +line is sensed and the interrupt is unmasked for the line. + +The driver sends this message with trigger type set to +\field{VIRTIO_GPIO_IRQ_TYPE_NONE}, to disable the interrupt for a GPIO line. The +device should discard any latched interrupt event associated with it. In order +to change the trigger type of an already enabled interrupt, the driver should +first disable the interrupt and then re-enable it with appropriate trigger type. + +The interrupts are masked at initialization and the driver unmasks them by +queuing a pair of buffers, of type \field{virtio_gpio_irq_request} and +\field{virtio_gpio_irq_response}, over the \field{eventq} virtqueue for a GPIO +line. A separate pair of buffers must be queued for each GPIO line, the driver +wants to configure for interrupts. Once the interrupt is unmasked by the driver +and the interrupt is also enabled at the device, the device can notify the +driver of an active interrupt signal on the GPIO line. This is done by updating +the \field{struct virtio_gpio_irq_response} buffer's \field{status} with +\field{VIRTIO_GPIO_IRQ_STATUS_VALID} and returning the updated buffers to the +driver. The interrupt is masked automatically at this point until the buffers +are available again at the device. + +When the interrupt is disabled by the driver, by setting the trigger type to +\field{VIRTIO_GPIO_IRQ_TYPE_NONE}, the device should return any unused pair of +buffers for the GPIO line, over the \field{eventq} virtqueue, after setting the +\field{status} field to \field{VIRTIO_GPIO_IRQ_STATUS_INVALID}. This also masks +the interrupt. + +The driver can enable and unmask the interrupt in any order, i.e. it can enable +the interrupt first and then queue the buffers or queue the buffers first and +then enable the interrupt. + +\begin{tabularx}{\textwidth}{ |l||X|X|X| } +\hline +\textbf{Request} & \field{type} & \field{gpio} & \field{value} \\ +\hline +& \field{VIRTIO_GPIO_MSG_SET_IRQ_TYPE} & line number & one of \field{VIRTIO_GPIO_IRQ_TYPE_*} \\ +\hline +\end{tabularx} + +\begin{tabularx}{\textwidth}{ |l||X|X| } +\hline +\textbf{Response} & \field{status} & \field{value} \\ +\hline +& \field{VIRTIO_GPIO_STATUS_*} & 0 \\ +\hline +\end{tabularx} + \subsubsection{requestq Operation: Message Flow}\label{sec:Device Types / GPIO Device / requestq Operation / Message Flow} \begin{itemize} @@ -313,6 +404,20 @@ \subsubsection{requestq Operation: Message Flow}\label{sec:Device Types / GPIO D \item The driver MAY send multiple messages for same or different GPIO lines in parallel. + +\item The driver MUST NOT send IRQ messages if the \field{VIRTIO_GPIO_F_IRQ} + feature is not enabled by the device. + +\item The driver MUST NOT send IRQ messages for a GPIO line configured for + output. + +\item The driver MUST set the IRQ trigger type to + \field{VIRTIO_GPIO_IRQ_TYPE_NONE} once it is done using the GPIO line + configured for interrupts. + +\item In order to change the trigger type of an already enabled interrupt, the + driver MUST first disable the interrupt and then re-enable it with + appropriate trigger type. \end{itemize} \devicenormative{\subsubsection}{requestq Operation}{Device Types / GPIO Device / requestq Operation} @@ -344,4 +449,145 @@ \subsubsection{requestq Operation: Message Flow}\label{sec:Device Types / GPIO D \item The device MUST discard all state information corresponding to a GPIO line, once the driver has requested to set its direction to \field{VIRTIO_GPIO_DIRECTION_NONE}. + +\item The device MUST latch an edge interrupt if the interrupt is enabled but + still masked. + +\item The device MUST NOT latch an level interrupt if the interrupt is enabled + but still masked. + +\item The device MUST discard any latched interrupt for a GPIO line, once + interrupt is disabled for the same. +\end{itemize} + +\subsection{Device Operation: eventq}\label{sec:Device Types / GPIO Device / eventq Operation} + +The \field{eventq} virtqueue is used by the driver to unmask the interrupts and +used by the device to notify the driver of newly sensed interrupts. In order to +unmask interrupt on a GPIO line, the driver queues a pair of buffers, +\field{struct virtio_gpio_irq_request} (filled by driver) and \field{struct +virtio_gpio_irq_response} (to be filled by device later), to the \field{eventq} +virtqueue. A separate pair of buffers must be queued for each GPIO line, the +driver wants to configure for interrupts. The device, on sensing an interrupt, +returns the pair of buffers for the respective GPIO line, which also masks the +interrupts. The driver can queue the buffers again to unmask the interrupt. + +\begin{lstlisting} +struct virtio_gpio_irq_request { + le16 gpio; +}; +\end{lstlisting} + +This structure is filled by the driver and read by the device. + +\begin{description} +\item[\field{gpio}] is the GPIO line number, i.e. 0 <= \field{gpio} < + \field{ngpio}. +\end{description} + +\begin{lstlisting} +struct virtio_gpio_irq_response { + u8 status; +}; + +/* Possible values of the interrupt status field */ +#define VIRTIO_GPIO_IRQ_STATUS_INVALID 0x0 +#define VIRTIO_GPIO_IRQ_STATUS_VALID 0x1 +\end{lstlisting} + +This structure is filled by the device and read by the driver. + +\begin{description} +\item[\field{status}] of the interrupt event, + \field{VIRTIO_GPIO_IRQ_STATUS_VALID} on interrupt and + \field{VIRTIO_GPIO_IRQ_STATUS_INVALID} to return the buffers back to the + driver after interrupt is disabled. +\end{description} + +\subsubsection{eventq Operation: Message Flow}\label{sec:Device Types / GPIO Device / eventq Operation / Message Flow} + +\begin{itemize} +\item The virtio-gpio driver is requested by a client driver to enable interrupt + for a GPIO line and configure it to a particular trigger type. + +\item The driver sends the \field{VIRTIO_GPIO_MSG_SET_IRQ_TYPE} message, over + the \field{requestq} virtqueue, and the device configures the GPIO line for + the requested trigger type and enables the interrupt. The interrupt is still + masked for delivery though. The device shall latch the interrupt from now + onward for edge trigger type. + +\item The driver unmasks the interrupt by queuing a pair of buffers to the + \field{eventq} virtqueue for the GPIO line. The driver can do this before + enabling the interrupt as well, though the interrupt must be both unmasked + and enabled to get delivered at the driver. + +\item The driver notifies the device of the presence of new buffers on the + \field{eventq} virtqueue. The interrupt is fully configured at this point. + +\item The device, on sensing an active interrupt on the GPIO line, finds the + matching buffers (based on GPIO line number) from the \field{eventq} + virtqueue and update its \field{struct virtio_gpio_irq_response} buffer's + \field{status} with \field{VIRTIO_GPIO_IRQ_STATUS_VALID} and returns the + pair of buffers to the device. This results in masking the interrupt as + well. + +\item The device notifies the driver of the presence of returned buffers on the + \field{eventq} virtqueue. + +\item If the GPIO line is configured for level interrupts, the device ignores + any further interrupt signals on this GPIO line, until the interrupt is + unmasked again by the driver by making the buffers available to the device. + Once the interrupt is unmasked again and the interrupt on the line is still + active, the device shall notify the driver again. + +\item If the GPIO line is configured for edge interrupts, the device latches + the interrupt received for this GPIO line, until the interrupt is unmasked + again by the driver by making the buffers available to the device. Once the + interrupt is unmasked again and an interrupt was latched earlier, the + device shall notify the driver again. + +\item The driver on receiving the notification from the device, processes the + interrupt. The interrupt is masked at the device until the buffers are + queued again by the driver. + +\item In a typical guest operating system kernel, the virtio-gpio driver + notifies the client driver, that is associated with this GPIO line, to + process the event. In the case of a level triggered interrupt, the client + driver shall fully process and acknowledge the event at its source to return + the line to its inactive state before the interrupt is unmasked again to + avoid a spurious interrupt. + +\item Once the interrupt is handled, the driver may queue a pair of buffers for + the GPIO line to unmask the interrupt again. + +\item The driver can also disable the interrupt by sending the + \field{VIRTIO_GPIO_MSG_SET_IRQ_TYPE} message, with + \field{VIRTIO_GPIO_IRQ_TYPE_NONE} trigger type. In that case, the device + shall return the unused pair of buffers for the GPIO line after setting the + \field{status} field with \field{VIRTIO_GPIO_IRQ_STATUS_INVALID}. +\end{itemize} + +\drivernormative{\subsubsection}{eventq Operation}{Device Types / GPIO Device / eventq Operation} + +\begin{itemize} +\item The driver MUST both enable and unmask the interrupt in order to get + notified for the same. + +\item To unmask the interrupt, the driver MUST queue a separate pair of buffers + to the \field{eventq} virtqueue for each GPIO line. + +\item The driver MUST NOT add multiple pairs of buffers for the same GPIO line + on the \field{eventq} virtqueue. +\end{itemize} + +\devicenormative{\subsubsection}{eventq Operation}{Device Types / GPIO Device / eventq Operation} + +\begin{itemize} +\item The device CAN ONLY send an interrupt event to the driver for a GPIO line, + if the interrupt is both unmasked and enabled by the driver. + +\item On receiving \field{VIRTIO_GPIO_MSG_SET_IRQ_TYPE} message, with + \field{VIRTIO_GPIO_IRQ_TYPE_NONE} trigger type, the device MUST return the + buffers, if they were received earlier, after setting the \field{status} + field to \field{VIRTIO_GPIO_IRQ_STATUS_INVALID}. \end{itemize} -- 2.31.1.272.g89b43f80a514

2 years, 9 months

3
7
0 0

2024

2023

2022

2021

2020

Stratos-dev October 2021