Hi,
This is an initial implementation of a vhost-user backend for the
VirtIO RPMB device. The device is currently in the draft of the next
VirtIO specification and describes block device which uses combination
of a key, nonce, hashing and a persistent write counter to prevent
replay attacks (hence Replay Protected Memory Block).
It is implemented as a vhost-user device because we want to experiment
in making portable backends that can be used with multiple
hypervisors. We also want to support backends isolated in their own
separate service VMs with limited memory cross-sections with the
principle guest. This is part of a wider initiative called project
Stratos for which you can find information here:
https://collaborate.linaro.org/display/STR/Stratos
I mention this to explain the decision to duplicate some of the
utility functions (specifically iov and hmac handling) and write the
daemon as a fairly pure glib application that just depends on
libvhost-user. As it happens I ended up having to include libqemuutil
as libvhost-user requires qemu_memfd_alloc. Whether this is an
oversight for libvhost-user or it means we should split these daemons
into a separate repository is a discussion I would like to have with
the community. Now I have a working reference implementation I also
want to explore how easy it is to write a Rust version of the backend
which raises similar questions about where such a project should live.
The current Linux kernel doesn't support RPMB devices in the vanilla
tree so if you want to test you will need to look at my testing tree
which is based on Thomas Winkler's original patches although somewhat
cut down and pared back to just support the JDEC style frames of the
upstream spec and the simple chardev based userspace interface. You
can find my kernel testing tree here:
https://git.linaro.org/people/alex.bennee/linux.git/log/?h=testing/virtio-r…
The above branch includes a simple test script with the rpmb userspace
tool which I've used to exercise the various features. I'm unsure if
there will ever be a push to upstream support for RPMB to the kernel
as access to these sorts of devices are usually the preserve of
firmware living in the secure world. There is currently work underway
to support this device in uboot and I suspect eventually there will be
support for OPTEE as well.
Any review comments gratefully received as well as discussion about if
we should consider creating some new projects for housing these sort
of vhost-user backends.
Alex Bennée (19):
tools/virtiofsd: add support for --socket-group
hw/block: add boilerplate for vhost-user-rpmb device
hw/virtio: move virtio-pci.h into shared include space
hw/block: add vhost-user-rpmb-pci boilerplate
virtio-pci: add notification trace points
tools/vhost-user-rpmb: add boilerplate and initial main
tools/vhost-user-rpmb: implement --print-capabilities
tools/vhost-user-rpmb: connect to fd and instantiate basic run loop
tools/vhost-user-rpmb: add a --verbose/debug flags for logging
tools/vhost-user-rpmb: handle shutdown and SIGINT/SIGHUP cleanly
tools/vhost-user-rpmb: add --flash-path for backing store
tools/vhost-user-rpmb: import hmac_sha256 functions
tools/vhost-user-rpmb: implement the PROGRAM_KEY handshake
tools/vhost-user-rpmb: implement VIRTIO_RPMB_REQ_GET_WRITE_COUNTER
tools/vhost-user-rpmb: implement VIRTIO_RPMB_REQ_DATA_WRITE
tools/vhost-user-rpmb: implement VIRTIO_RPMB_REQ_DATA_READ
tools/vhost-user-rpmb: add key persistence
tools/vhost-user-rpmb: allow setting of the write_count
docs: add a man page for vhost-user-rpmb
docs/tools/index.rst | 1 +
docs/tools/vhost-user-rpmb.rst | 102 +++
docs/tools/virtiofsd.rst | 4 +
include/hw/virtio/vhost-user-rpmb.h | 46 ++
{hw => include/hw}/virtio/virtio-pci.h | 0
tools/vhost-user-rpmb/hmac_sha256.h | 87 ++
tools/virtiofsd/fuse_i.h | 1 +
hw/block/vhost-user-rpmb-pci.c | 82 ++
hw/block/vhost-user-rpmb.c | 333 ++++++++
hw/virtio/vhost-scsi-pci.c | 2 +-
hw/virtio/vhost-user-blk-pci.c | 2 +-
hw/virtio/vhost-user-fs-pci.c | 2 +-
hw/virtio/vhost-user-input-pci.c | 2 +-
hw/virtio/vhost-user-scsi-pci.c | 2 +-
hw/virtio/vhost-user-vsock-pci.c | 2 +-
hw/virtio/vhost-vsock-pci.c | 2 +-
hw/virtio/virtio-9p-pci.c | 2 +-
hw/virtio/virtio-balloon-pci.c | 2 +-
hw/virtio/virtio-blk-pci.c | 2 +-
hw/virtio/virtio-input-host-pci.c | 2 +-
hw/virtio/virtio-input-pci.c | 2 +-
hw/virtio/virtio-iommu-pci.c | 2 +-
hw/virtio/virtio-net-pci.c | 2 +-
hw/virtio/virtio-pci.c | 5 +-
hw/virtio/virtio-rng-pci.c | 2 +-
hw/virtio/virtio-scsi-pci.c | 2 +-
hw/virtio/virtio-serial-pci.c | 2 +-
tools/vhost-user-rpmb/hmac_sha256.c | 331 ++++++++
tools/vhost-user-rpmb/main.c | 880 +++++++++++++++++++++
tools/virtiofsd/fuse_lowlevel.c | 6 +
tools/virtiofsd/fuse_virtio.c | 20 +-
MAINTAINERS | 5 +
hw/block/Kconfig | 5 +
hw/block/meson.build | 3 +
hw/virtio/trace-events | 7 +-
tools/meson.build | 8 +
tools/vhost-user-rpmb/50-qemu-rpmb.json.in | 5 +
tools/vhost-user-rpmb/meson.build | 12 +
38 files changed, 1956 insertions(+), 21 deletions(-)
create mode 100644 docs/tools/vhost-user-rpmb.rst
create mode 100644 include/hw/virtio/vhost-user-rpmb.h
rename {hw => include/hw}/virtio/virtio-pci.h (100%)
create mode 100644 tools/vhost-user-rpmb/hmac_sha256.h
create mode 100644 hw/block/vhost-user-rpmb-pci.c
create mode 100644 hw/block/vhost-user-rpmb.c
create mode 100644 tools/vhost-user-rpmb/hmac_sha256.c
create mode 100644 tools/vhost-user-rpmb/main.c
create mode 100644 tools/vhost-user-rpmb/50-qemu-rpmb.json.in
create mode 100644 tools/vhost-user-rpmb/meson.build
--
2.20.1
Hi Ulf,
Ilias told me you were the eMMC guru so your might be best placed to
advise on this problem.
It has been pointed out that a lot of RPMB devices are handled in eMMC
storage stacks. A naive implementation of a virtio-rpmb device driver
has to fake-up a number of responses for higher in the stack to convince
the stack there is a zero length eMMC block device which provides an
RPMB partition. One potential approach would be to pass-through the eMMC
commands typically done during such probing to the virtio-rpmb backend.
This would avoid each individual driver making up different things but
at the cost of the backend having to fake up a bunch of information
instead.
It could also be argued that this just indicates the virtio-rpmb driver
is sitting in the wrong place of the guest OS stack. This may be the
view of the virtio-dev developers and indeed earlier drafts of the spec
suggested handling of JDEC and NVME style frames which eventually got
dropped before being adopted into the current draft.
However to test that view we need to come up with a draft for the
extension. So far my draft is something like:
--8<---------------cut here---------------start------------->8---
virtio-rpmb.tex: define optional EMMC probe pass through support
Guest drivers commonly exist in an existing eMMC stack and although
the virtio-rpmb device itself does not support normal eMMC storage it
makes driver design simpler if start-up probe commands can be handled
by the backend. This avoids having guest drivers invent their own fake
data for the stack in a way that might differ from guest to guest.
Signed-off-by: Alex Bennée <alex.bennee(a)linaro.org>
1 file changed, 25 insertions(+), 3 deletions(-)
virtio-rpmb.tex | 28 +++++++++++++++++++++++++---
modified virtio-rpmb.tex
@@ -18,7 +18,9 @@ \subsection{Virtqueues}\label{sec:Device Types / RPMB Device / Virtqueues}
\subsection{Feature bits}\label{sec:Device Types / RPMB Device / Feature bits}
-None.
+\begin{description}
+\item[VIRTIO_RPMB_F_EMMC (0)] Device handles EMMC probe command packets.
+\end{description}
\subsection{Device configuration layout}\label{sec:Device Types / RPMB Device / Device configuration layout}
@@ -64,12 +66,14 @@ \subsection{Device Operation}\label{sec:Device Types / RPMB Device / Device Oper
#define VIRTIO_RPMB_REQ_DATA_WRITE 0x0003
#define VIRTIO_RPMB_REQ_DATA_READ 0x0004
#define VIRTIO_RPMB_REQ_RESULT_READ 0x0005
+#define VIRTIO_RPMB_REQ_EMMC_CMD 0x0040
/* RPMB Response Types */
#define VIRTIO_RPMB_RESP_PROGRAM_KEY 0x0100
#define VIRTIO_RPMB_RESP_GET_COUNTER 0x0200
#define VIRTIO_RPMB_RESP_DATA_WRITE 0x0300
#define VIRTIO_RPMB_RESP_DATA_READ 0x0400
+#define VIRTIO_RPMB_RESP_EMMC_CMD 0x4000
\end{lstlisting}
\begin{description}
@@ -96,6 +100,16 @@ \subsection{Device Operation}\label{sec:Device Types / RPMB Device / Device Oper
It is used following with VIRTIO_RPMB_REQ_PROGRAM_KEY or VIRTIO_RPMB_REQ_DATA_WRITE
request types for a returned result in one or multiple RPMB frames. If it's not
requested, the device will not return result frame to the driver.
+
+ \item[\field{VIRTIO_RPMB_REQ_EMMC_CMD}] passes through an eMMC probe command.
+ This is intended to simplify drivers which are already part of an
+ eMMC driver stack. Rather than have the device driver fake
+ responses to an eMMC probe it can pass the commands to the backend
+ to respond on it's behalf. Normal eMMC read/writes will not be
+ possible and access to the RPMB storage MUST only use the existing
+ VIRTIO_RPMB_REQ commands.
+
+ (confirm do all eMMC commands have implicit responses?)
\end{description}
@@ -106,7 +120,10 @@ \subsubsection{Device Operation: Request Queue}\label{sec:Device Types / RPMB De
\begin{lstlisting}
struct virtio_rpmb_frame {
- u8 stuff[196];
+ union {
+ u8 stuff[196];
+ u8 emmc_cmd_resp[8]
+ } stuff;
u8 key_mac[32];
u8 data[256];
u8 nonce[16];
@@ -126,11 +143,16 @@ \subsubsection{Device Operation: Request Queue}\label{sec:Device Types / RPMB De
#define VIRTIO_RPMB_RES_WRITE_FAILURE 0x0005
#define VIRTIO_RPMB_RES_READ_FAILURE 0x0006
#define VIRTIO_RPMB_RES_NO_AUTH_KEY 0x0007
+#define VIRTIO_RPMB_RES_EMMC 0x0040
#define VIRTIO_RPMB_RES_WRITE_COUNTER_EXPIRED 0x0080
\end{lstlisting}
\begin{description}
-\item[\field{stuff}] Padding for the frame.
+\item[\field{stuff}] For normal RPMB operations this is simply padding
+ for the frame. When the VIRTIO_RPMB_F_EMMC feature has been
+ negotiated this contains the embedded EMMC probe command (when
+ \field{req_resp} is VIRTIO_RPMB_REQ_EMMC_CMD) or the response (when
+ \field{req_resp} is VIRTIO_RPMB_RESP_EMMC_CMD).
\item[\field{key_mac}] is the authentication key or the message
authentication code (MAC) depending on the request/response type.
--8<---------------cut here---------------end--------------->8---
However I'm wary of adding an open ended pass-through definition,
especially anything that might tempt an implementer to start trying to
read and write data using eMMC commands instead of the proper virtio
commands.
- What eMMC commands are needed for a probe?
- What are the bounds of frame sizes for those commands?
- I'm currently overloading the wasted stuff[196] bytes to the
encapsulation and it would get complicated if we extended into the
used fields.
- Should we mandate certain responses?
- e.g. 0 "normal" size, eMMC version 4.1 etc
Thanks,
--
Alex Bennée
Hi,
Recently the openEuler community published the StratoVirt VMM solution (for
coincidence, the name is quite similar to "Stratos"). StratoVirt is based
on Rust programming language, and the goal is to support both lightweight
VM and normal VM (by integrating some good design concepts from QEMU).
Currently it only supports lightweight virtualization, and the plan is to
publish the support for standard virtualization in openEuler 21.09 (end of
Sep. 2021). Hope it can be interesting to you.
The repo: https://gitee.com/openeuler/stratovirt
The design document:
https://gitee.com/openeuler/stratovirt/blob/master/docs/design.md
Regards,
Jammy
Hi all,
Looking into the details of the newly created STR-15, I realized it is
very similar to STR-14. I think that STR-14 and STR-15 have to be
designed together from the start. It might be detrimental to implement
one without the other. Below the details.
Historically we started with the Xen LBI (LBI-24), from it we derived
STR-9 for the generic virtio work, such as new virtio protocols.
There is a security/safety issue with virtio which has been overlooked
so far and it becomes important when introducing virtio to Xen and other
type1s. As we discussed a few times during the calls, it is about the
virtio backends having to be privileged (having to be in Dom0.) In a
typical Xen deployment today, using Xen PV drivers, that is not the case.
To address this concern we created STR-14. (Please ignore the non-virtio
deliverables of STR-14 for now.) We decided to go with STR-14, a new
card, instead of adding it to STR-9. One of the reasons is that the work
is expected to be done using Xen as reference for STR-14 while STR-9 is
generic.
The main points covered by STR-14 are:
- Virtio spec changes for unprivileged backends
- changes required for pre-shared memory setups
- enable dynamical sharing of memory between frontends and backends
- pre-shared memory implementation in Linux, Xen, QEMU
- dynamically shared memory implementation in Linux, Xen, QEMU
In this context:
a) Xen: implementation of the static and dynamic memory sharing
b) Linux: virtio frontend, e.g. drivers/net/virtio_net.c
c) QEMU (or other virtio backend, I'll use QEMU as example): virtio
backend, e.g. hw/net/virtio-net.c
Both STR-14 and STR-15 are about explicitly sharing memory between VMs
using virtio. STR-15 seems to correspond to the "using pre-shared memory
and memcpys" deliverable of STR-14.
If STR-15 is going to need a new virtio interface, it is likely that the
virtio committee will ask us to come up with a single technical solution
that works for both STR-14 and STR-15. We need a close collaboration
between the two efforts and design it together. In fact, I wonder if it
makes sense to just do the work under STR-14.
It is worth noting that the development of STR-14 and STR-15 won't be
hypervisor neutral. The Linux virtio frontend changes (b) and QEMU
virtio backend changes (c) should be mostly reusable. But there is also
a Xen specific component (a). A prototype based on KVM won't work on Xen
out of the box.
>From a reference implementation perspective, STR-14 has Xen as reference
also because the companies that requested it (ARM, Qualcomm, Xilinx)
were interested in Xen and other type-1s. I wonder if it is possible to
keep Xen as reference implementation for this work, or raise its
priority.
Happy to discuss more. I am also happy to join a call on the topic.
Cheers,
Stefano
All
There were some interesting topics on the KVM forum schedule [1]. The event
is October 28-30, and the details have been added to the Stratos calendar
[2] they are also listed below
28th
- Virtualization for the Masses: Exposing KVM on Android
29th
- Evaluate Implementation Options of KVM-based Type1 (or 1.5) Hypervisor
- Speeding Up VM’s I/O Sharing Host's io_uring Queues With Guests
- Virtual Versus Physical: Virtio Changes for New Hardware
30th
- Virtio-(balloon|pmem|mem): Managing Guest Memory
Also a reminder that we have the Stratos call on the 29th, agenda items
welcome [3]
Mike
[1] https://events.linuxfoundation.org/kvm-forum/
[2] https://collaborate.linaro.org/display/STR/calendars
[3]
https://collaborate.linaro.org/display/STR/2020-10-01+Project+Stratos+Sync+…
--
Mike Holmes | Director, Foundation Technologies, Linaro
Mike.Holmes(a)linaro.org <mike.holmes(a)linaro.org>
"Work should be fun and collaborative, the rest follows"
Good morning all,
In a Stratos meeting from a few weeks ago we talked about transporting
SCMI over virtIO as a potential area of involvement. It seems like
someone has already identified the need to do just that and has been
working on a patchset[1]. I haven't investigated the proposed
solution but I thought it was worth sharing in case someone is looking
into it.
Best regards,
Mathieu
Good morning all,
In a Stratos meeting from a few weeks ago we talked about transporting
SCMI over virtIO as a potential area of involvement. It seems like
someone has already identified the need to do just that and has been
working on a patchset[1]. I haven't investigated the proposed
solution but I thought it was worth sharing in case someone is looking
into it.
Best regards,
Mathieu
[1]. https://lkml.org/lkml/2020/9/18/869
Hi,
After the QEMU team migrated to TCWG the channel became pretty dead as
most team based chat was done in #linaro-tcwg. Now we have a few more
Linaro engineers starting to look at Stratos related things we thought
we might as well make #linaro-virtualisation the hang-out location for
real-time chat on Stratos topics.
So if you like to IRC likes it's 1988 see you there on Freenode.
--
Alex Bennée
Hi
AGL is looking for all virtio devices needed to virtualize an infotainment
solution.
It would be nice to have "authoritative" vision by members on things like:
- virtualizing a radio; what are the virtio devices needed ?
virtio-tuner...?
- virtualized access to DRM solution (openCDM, MS PlayReady, Google
Windvine...) ?
- other devices? (virtio-wall-clock...)
Probably a good discussion to extend to Stratos.
Cheers
FF
--
François-Frédéric Ozog | *Director Linaro Edge & Fog Computing Group*
T: +33.67221.6485
francois.ozog(a)linaro.org | Skype: ffozog
All
There was a quick scratch sync to discuss extending the demo plans to the
Socionext platform
https://collaborate.linaro.org/display/STR/2020-09-16+Project+Stratos+Demo+…
Masami has a 5 min slot in the call agenda tomorrow along with Alex just to
talk about the updated thinking.
Mike
--
Mike Holmes | Director, Foundation Technologies, Linaro
Mike.Holmes(a)linaro.org <mike.holmes(a)linaro.org>
"Work should be fun and collaborative, the rest follows"
