Hello,
This patch series implements a pure-capability vDSO which purecap userspace applications can directly use. It also fixes the vDSO selftests so that they can be built as purecap binaries, therefore allowing the new purecap vDSO to be tested. The existing aarch64 vDSO implementation is unchanged - it has simply been redefined as the compat vDSO. Therefore, processes using both the standard and purecap ABIs will have a vDSO to use.
Main changes in v3: * Series has now grown to 5 patches. The first patch duplicates an unmerged upstream change (I forgot to include this in v2). Once it is merged upstream and the fork is rebased this patch may disappear from the log. * Patch 3 adds some purecap-only code to linux/types.h to enable the vDSO, which uses kernel headers, to be built for purecap. * v2 removed the vdso-purecap/ directory, as it needlessly replicated the code in vdso/. vdso/Makefile was instead changed so it could build both vDSOs (regular and purecap) from the same source. This is achieved by calling the Makefile twice with BUILD_PURECAP_VDSO=[y/n]. v3 further changes this Makefile; now all purecap generated files are placed in a separate subdirectory (vdso/purecap.)
Aditya Deshpande (5): selftests: vDSO: fix Makefile so that it uses lib.mk correctly selftests/vDSO: Add support for purecap vDSO testing linux/types.h: Redefine uintptr_t to __uintcap_t under purecap arm64: vdso: Build a pure-capability vDSO arm64: vDSO: Provide a purecap vDSO to userspace purecap programs
arch/arm64/Makefile | 6 ++ arch/arm64/include/asm/elf.h | 24 +++++--- arch/arm64/include/asm/vdso.h | 1 + arch/arm64/include/asm/vdso/gettimeofday.h | 58 +++++++++++++++--- arch/arm64/kernel/Makefile | 2 + arch/arm64/kernel/vdso-purecap-wrap.S | 22 +++++++ arch/arm64/kernel/vdso.c | 55 ++++++++++++++++- arch/arm64/kernel/vdso/Makefile | 60 ++++++++++++++++--- fs/compat_binfmt_elf.c | 24 ++++---- include/linux/compiler.h | 3 +- include/linux/types.h | 4 ++ tools/testing/selftests/vDSO/Makefile | 16 ++--- tools/testing/selftests/vDSO/parse_vdso.h | 16 +++++ tools/testing/selftests/vDSO/vdso_test_abi.c | 5 +- .../selftests/vDSO/vdso_test_gettimeofday.c | 6 +- 15 files changed, 252 insertions(+), 50 deletions(-) create mode 100644 arch/arm64/kernel/vdso-purecap-wrap.S
kselftest.rst states that flags must be specified before including lib.mk, but the vDSO selftest Makefile does not follow this order. As a result, changes made by lib.mk to flags and other variables are overwritten by the Makefile. For example, it is impossible to pass CFLAGS to the compiler via make.
Rectify this by including lib.mk after assigning flag values.
Also change the paths of the generated programs from absolute to relative paths as lib.mk will now correctly prepend the output directory path to the program name as intended.
Signed-off-by: Aditya Deshpande aditya.deshpande@arm.com --- tools/testing/selftests/vDSO/Makefile | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/tools/testing/selftests/vDSO/Makefile b/tools/testing/selftests/vDSO/Makefile index d53a4d8008f9..19145210d044 100644 --- a/tools/testing/selftests/vDSO/Makefile +++ b/tools/testing/selftests/vDSO/Makefile @@ -1,16 +1,15 @@ # SPDX-License-Identifier: GPL-2.0 -include ../lib.mk - uname_M := $(shell uname -m 2>/dev/null || echo not) ARCH ?= $(shell echo $(uname_M) | sed -e s/i.86/x86/ -e s/x86_64/x86/)
-TEST_GEN_PROGS := $(OUTPUT)/vdso_test_gettimeofday $(OUTPUT)/vdso_test_getcpu -TEST_GEN_PROGS += $(OUTPUT)/vdso_test_abi -TEST_GEN_PROGS += $(OUTPUT)/vdso_test_clock_getres +TEST_GEN_PROGS := vdso_test_gettimeofday +TEST_GEN_PROGS += vdso_test_getcpu +TEST_GEN_PROGS += vdso_test_abi +TEST_GEN_PROGS += vdso_test_clock_getres ifeq ($(ARCH),$(filter $(ARCH),x86 x86_64)) -TEST_GEN_PROGS += $(OUTPUT)/vdso_standalone_test_x86 +TEST_GEN_PROGS += vdso_standalone_test_x86 endif -TEST_GEN_PROGS += $(OUTPUT)/vdso_test_correctness +TEST_GEN_PROGS += vdso_test_correctness
CFLAGS := -std=gnu99 CFLAGS_vdso_standalone_test_x86 := -nostdlib -fno-asynchronous-unwind-tables -fno-stack-protector @@ -19,7 +18,8 @@ ifeq ($(CONFIG_X86_32),y) LDLIBS += -lgcc_s endif
-all: $(TEST_GEN_PROGS) +include ../lib.mk + $(OUTPUT)/vdso_test_gettimeofday: parse_vdso.c vdso_test_gettimeofday.c $(OUTPUT)/vdso_test_getcpu: parse_vdso.c vdso_test_getcpu.c $(OUTPUT)/vdso_test_abi: parse_vdso.c vdso_test_abi.c
Replace getauxval() with getauxptr() when building the tests for purecap. By building the vDSO selftests as purecap binaries, we can test the purecap vDSO.
Signed-off-by: Aditya Deshpande aditya.deshpande@arm.com --- tools/testing/selftests/vDSO/parse_vdso.h | 16 ++++++++++++++++ tools/testing/selftests/vDSO/vdso_test_abi.c | 5 +++-- .../selftests/vDSO/vdso_test_gettimeofday.c | 6 +++--- 3 files changed, 22 insertions(+), 5 deletions(-)
diff --git a/tools/testing/selftests/vDSO/parse_vdso.h b/tools/testing/selftests/vDSO/parse_vdso.h index de0453067d7c..54ad67f463cc 100644 --- a/tools/testing/selftests/vDSO/parse_vdso.h +++ b/tools/testing/selftests/vDSO/parse_vdso.h @@ -4,6 +4,7 @@ #define PARSE_VDSO_H
#include <stdint.h> +#include <sys/auxv.h>
/* * To use this vDSO parser, first call one of the vdso_init_* functions. @@ -28,4 +29,19 @@ void *vdso_sym(const char *version, const char *name); void vdso_init_from_sysinfo_ehdr(uintptr_t base); void vdso_init_from_auxv(void *auxv);
+/* + * Under PCuABI, pointers in the auxiliary vector can no longer be represented + * as unsigned long as they are now capabilities. Get the capability to the + * vDSO using getauxptr() instead, which returns a a capability instead of + * unsigned long. + */ +static inline uintptr_t get_sysinfo_ehdr() +{ +#ifdef __CHERI_PURE_CAPABILITY__ + return getauxptr(AT_SYSINFO_EHDR); +#else + return (uintptr_t)getauxval(AT_SYSINFO_EHDR); +#endif +} + #endif diff --git a/tools/testing/selftests/vDSO/vdso_test_abi.c b/tools/testing/selftests/vDSO/vdso_test_abi.c index 883ca85424bc..091efeaa3172 100644 --- a/tools/testing/selftests/vDSO/vdso_test_abi.c +++ b/tools/testing/selftests/vDSO/vdso_test_abi.c @@ -20,6 +20,7 @@
#include "../kselftest.h" #include "vdso_config.h" +#include "parse_vdso.h"
extern void *vdso_sym(const char *version, const char *name); extern void vdso_init_from_sysinfo_ehdr(uintptr_t base); @@ -175,7 +176,7 @@ static inline void vdso_test_clock(clockid_t clock_id)
int main(int argc, char **argv) { - unsigned long sysinfo_ehdr = getauxval(AT_SYSINFO_EHDR); + uintptr_t sysinfo_ehdr = get_sysinfo_ehdr();
ksft_print_header(); ksft_set_plan(VDSO_TEST_PLAN); @@ -190,7 +191,7 @@ int main(int argc, char **argv)
printf("[vDSO kselftest] VDSO_VERSION: %s\n", version);
- vdso_init_from_sysinfo_ehdr(getauxval(AT_SYSINFO_EHDR)); + vdso_init_from_sysinfo_ehdr(sysinfo_ehdr);
vdso_test_gettimeofday();
diff --git a/tools/testing/selftests/vDSO/vdso_test_gettimeofday.c b/tools/testing/selftests/vDSO/vdso_test_gettimeofday.c index e411f287a426..33c6881aca95 100644 --- a/tools/testing/selftests/vDSO/vdso_test_gettimeofday.c +++ b/tools/testing/selftests/vDSO/vdso_test_gettimeofday.c @@ -37,15 +37,15 @@ const char *name = "__vdso_gettimeofday";
int main(int argc, char **argv) { - unsigned long sysinfo_ehdr = getauxval(AT_SYSINFO_EHDR); + uintptr_t sysinfo_ehdr = get_sysinfo_ehdr(); if (!sysinfo_ehdr) { printf("AT_SYSINFO_EHDR is not present!\n"); return KSFT_SKIP; } - - vdso_init_from_sysinfo_ehdr(getauxval(AT_SYSINFO_EHDR)); + vdso_init_from_sysinfo_ehdr(sysinfo_ehdr);
/* Find gettimeofday. */ + typedef long (*gtod_t)(struct timeval *tv, struct timezone *tz); gtod_t gtod = (gtod_t)vdso_sym(version, name);
This change is made in preparation for the introduction of a purecap vDSO. While the vDSO is userspace shared library, it does use some kernel headers. Therefore, under purecap, any pointers in those headers must be compiled to capabilites.
Note that this does not make any part of the actual kernel purecap.
Signed-off-by: Aditya Deshpande aditya.deshpande@arm.com --- include/linux/types.h | 4 ++++ 1 file changed, 4 insertions(+)
diff --git a/include/linux/types.h b/include/linux/types.h index 897c134c1e8a..5e7a6c948618 100644 --- a/include/linux/types.h +++ b/include/linux/types.h @@ -46,7 +46,11 @@ typedef __kernel_gid32_t gid_t; typedef __kernel_uid16_t uid16_t; typedef __kernel_gid16_t gid16_t;
+#ifdef __CHERI_PURE_CAPABILITY__ +typedef __uintcap_t uintptr_t; +#else typedef unsigned long uintptr_t; +#endif
#ifdef CONFIG_CHERI_PURECAP_UABI typedef __uintcap_t user_uintptr_t;
This change is made in preparation for the introduction of a purecap vDSO. While the vDSO is userspace shared library, it does use some kernel headers. Therefore, under purecap, any pointers in those headers must be compiled to capabilites.
Note that this does not make any part of the actual kernel purecap.
Signed-off-by: Aditya Deshpande aditya.deshpande@arm.com --- include/linux/compiler.h | 2 +- include/linux/types.h | 4 ++++ 2 files changed, 5 insertions(+), 1 deletion(-)
diff --git a/include/linux/compiler.h b/include/linux/compiler.h index d7779a18b24f..c2f87d9f3e1d 100644 --- a/include/linux/compiler.h +++ b/include/linux/compiler.h @@ -222,7 +222,7 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val, */ static inline void *offset_to_ptr(const int *off) { - return (void *)((unsigned long)off + *off); + return (void *)((uintptr_t)off + *off); }
#endif /* __ASSEMBLY__ */ diff --git a/include/linux/types.h b/include/linux/types.h index 897c134c1e8a..5e7a6c948618 100644 --- a/include/linux/types.h +++ b/include/linux/types.h @@ -46,7 +46,11 @@ typedef __kernel_gid32_t gid_t; typedef __kernel_uid16_t uid16_t; typedef __kernel_gid16_t gid16_t;
+#ifdef __CHERI_PURE_CAPABILITY__ +typedef __uintcap_t uintptr_t; +#else typedef unsigned long uintptr_t; +#endif
#ifdef CONFIG_CHERI_PURECAP_UABI typedef __uintcap_t user_uintptr_t;
Modify the vDSO Makefile such that it can also build a vDSO for the purecap ABI and change the fallbacks to use capability registers when running in purecap. Therefore, under PCuABI, vDSOs will be built for both aarch64 (COMPAT64) and a purecap.
Note that this change only implements the build of the purecap vDSO. Mapping of this vDSO by the kernel into user address space will be done in a subsequent patch.
Signed-off-by: Aditya Deshpande aditya.deshpande@arm.com --- arch/arm64/Makefile | 6 +++ arch/arm64/include/asm/vdso/gettimeofday.h | 58 ++++++++++++++++++--- arch/arm64/kernel/vdso/Makefile | 60 +++++++++++++++++++--- include/linux/compiler.h | 3 +- 4 files changed, 111 insertions(+), 16 deletions(-)
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index f5ac0cb0147e..839a522d83b8 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -177,6 +177,8 @@ install zinstall: PHONY += vdso_install vdso_install: $(Q)$(MAKE) $(build)=arch/arm64/kernel/vdso $@ + $(if $(CONFIG_CHERI_PURECAP_UABI), \ + $(Q)$(MAKE) BUILD_PURECAP_VDSO=y $(build)=arch/arm64/kernel/vdso $@) $(if $(CONFIG_COMPAT_VDSO), \ $(Q)$(MAKE) $(build)=arch/arm64/kernel/vdso32 $@)
@@ -204,6 +206,10 @@ prepare: vdso_prepare vdso_prepare: prepare0 $(Q)$(MAKE) $(build)=arch/arm64/kernel/vdso \ include/generated/vdso-offsets.h arch/arm64/kernel/vdso/vdso.so +ifdef CONFIG_CHERI_PURECAP_UABI + $(Q)$(MAKE) BUILD_PURECAP_VDSO=y $(build)=arch/arm64/kernel/vdso \ + include/generated/vdso-purecap-offsets.h arch/arm64/kernel/vdso/purecap/vdso.so +endif ifdef CONFIG_COMPAT_VDSO $(Q)$(MAKE) $(build)=arch/arm64/kernel/vdso32 \ include/generated/vdso32-offsets.h arch/arm64/kernel/vdso32/vdso.so diff --git a/arch/arm64/include/asm/vdso/gettimeofday.h b/arch/arm64/include/asm/vdso/gettimeofday.h index 764d13e2916c..17151df887d4 100644 --- a/arch/arm64/include/asm/vdso/gettimeofday.h +++ b/arch/arm64/include/asm/vdso/gettimeofday.h @@ -11,22 +11,36 @@ #include <asm/barrier.h> #include <asm/unistd.h> #include <asm/sysreg.h> +#include <linux/stringify.h>
#define VDSO_HAS_CLOCK_GETRES 1
+/* + * Inline Assembly Macros for Purecap + * PTR_REG(n) will expand to "cn" under purecap, and "xn" under non-purecap + * PTR_REG_OP will expand to "C" under purecap and "r" under non-purecap. + */ +#if defined(__CHERI_PURE_CAPABILITY__) +#define PTR_REG(n) "c" __stringify(n) +#define PTR_REG_OP "C" +#else +#define PTR_REG(n) "x" __stringify(n) +#define PTR_REG_OP "r" +#endif /* __CHERI_PURE_CAPABILITY__ */ + static __always_inline int gettimeofday_fallback(struct __kernel_old_timeval *_tv, struct timezone *_tz) { - register struct timezone *tz asm("x1") = _tz; - register struct __kernel_old_timeval *tv asm("x0") = _tv; + register struct timezone *tz asm(PTR_REG(1)) = _tz; + register struct __kernel_old_timeval *tv asm(PTR_REG(0)) = _tv; register long ret asm ("x0"); register long nr asm("x8") = __NR_gettimeofday;
asm volatile( " svc #0\n" : "=r" (ret) - : "r" (tv), "r" (tz), "r" (nr) + : PTR_REG_OP (tv), PTR_REG_OP (tz), "r" (nr) : "memory");
return ret; @@ -35,7 +49,7 @@ int gettimeofday_fallback(struct __kernel_old_timeval *_tv, static __always_inline long clock_gettime_fallback(clockid_t _clkid, struct __kernel_timespec *_ts) { - register struct __kernel_timespec *ts asm("x1") = _ts; + register struct __kernel_timespec *ts asm(PTR_REG(1)) = _ts; register clockid_t clkid asm("x0") = _clkid; register long ret asm ("x0"); register long nr asm("x8") = __NR_clock_gettime; @@ -43,7 +57,7 @@ long clock_gettime_fallback(clockid_t _clkid, struct __kernel_timespec *_ts) asm volatile( " svc #0\n" : "=r" (ret) - : "r" (clkid), "r" (ts), "r" (nr) + : "r" (clkid), PTR_REG_OP (ts), "r" (nr) : "memory");
return ret; @@ -52,7 +66,7 @@ long clock_gettime_fallback(clockid_t _clkid, struct __kernel_timespec *_ts) static __always_inline int clock_getres_fallback(clockid_t _clkid, struct __kernel_timespec *_ts) { - register struct __kernel_timespec *ts asm("x1") = _ts; + register struct __kernel_timespec *ts asm(PTR_REG(1)) = _ts; register clockid_t clkid asm("x0") = _clkid; register long ret asm ("x0"); register long nr asm("x8") = __NR_clock_getres; @@ -60,7 +74,7 @@ int clock_getres_fallback(clockid_t _clkid, struct __kernel_timespec *_ts) asm volatile( " svc #0\n" : "=r" (ret) - : "r" (clkid), "r" (ts), "r" (nr) + : "r" (clkid), PTR_REG_OP (ts), "r" (nr) : "memory");
return ret; @@ -99,6 +113,32 @@ static __always_inline u64 __arch_get_hw_counter(s32 clock_mode, return res; }
+#if defined(__CHERI_PURE_CAPABILITY__) +static __always_inline +const struct vdso_data *__arch_get_vdso_data(void) +{ + const struct vdso_data *vd; + asm(".hidden _vdso_data\n\t" + "adrp %0, _vdso_data\n\t" + "add %0, %0, #:lo12:_vdso_data" + : "=C"(vd)); + return vd; +} + +#ifdef CONFIG_TIME_NS +static __always_inline +const struct vdso_data *__arch_get_timens_vdso_data(const struct vdso_data *vd) +{ + const struct vdso_data *td; + asm(".hidden _timens_data\n\t" + "adrp %0, _timens_data\n\t" + "add %0, %0, #:lo12:_timens_data" + : "=C"(td)); + return td; +} +#endif /* CONFIG_TIME_NS */ + +#else /* !__CHERI_PURE_CAPABILITY__ */ static __always_inline const struct vdso_data *__arch_get_vdso_data(void) { @@ -111,7 +151,9 @@ const struct vdso_data *__arch_get_timens_vdso_data(const struct vdso_data *vd) { return _timens_data; } -#endif +#endif /* CONFIG_TIME_NS */ + +#endif /* __CHERI_PURE_CAPABILITY__ */
#endif /* !__ASSEMBLY__ */
diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile index fe7a53c6781f..185eafcf709b 100644 --- a/arch/arm64/kernel/vdso/Makefile +++ b/arch/arm64/kernel/vdso/Makefile @@ -13,7 +13,6 @@ obj-vdso := vgettimeofday.o note.o sigreturn.o
# Build rules targets := $(obj-vdso) vdso.so vdso.so.dbg -obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
btildflags-$(CONFIG_ARM64_BTI_KERNEL) += -z force-bti
@@ -33,6 +32,29 @@ ldflags-y += -T ccflags-y := -fno-common -fno-builtin -fno-stack-protector -ffixed-x18 ccflags-y += -DDISABLE_BRANCH_PROFILING -DBUILD_VDSO
+# When building for PCuABI, this Makefile will be called twice: first to build +# the regular arm64 vDSO, and second to build the purecap vDSO. We must change +# certain variables/flags depending on which vDSO we are building. +ifdef BUILD_PURECAP_VDSO + # Specify building for purecap + PURECAP_FLAGS := -mabi=purecap + # Remove this flag as capability registers are not considered + # as general purpose registers by the compiler. + REMOVE_FLAGS := -mgeneral-regs-only + ccflags-y += $(PURECAP_FLAGS) + asflags-y := $(PURECAP_FLAGS) + ccflags-remove-y := $(REMOVE_FLAGS) + asflags-remove-y := $(REMOVE_FLAGS) + + # We want the purecap vDSO in its own directory (kernel/vdso/purecap) + output-dir := $(obj)/purecap + # Put the offsets for the purecap vDSO into their own header file + offsets := include/generated/vdso-purecap-offsets.h +else + output-dir := $(obj) + offsets := include/generated/vdso-offsets.h +endif + # -Wmissing-prototypes and -Wmissing-declarations are removed from # the CFLAGS of vgettimeofday.c to make possible to build the # kernel with CONFIG_WERROR enabled. @@ -48,6 +70,9 @@ KCOV_INSTRUMENT := n
CFLAGS_vgettimeofday.o = -O2 -mcmodel=tiny -fasynchronous-unwind-tables
+CFLAGS_REMOVE_purecap/vgettimeofday.o = $(CFLAGS_REMOVE_vgettimeofday.o) +CFLAGS_purecap/vgettimeofday.o = $(CFLAGS_vgettimeofday.o) + ifneq ($(c-gettimeofday-y),) CFLAGS_vgettimeofday.o += -include $(c-gettimeofday-y) endif @@ -58,13 +83,34 @@ GCOV_PROFILE := n targets += vdso.lds CPPFLAGS_vdso.lds += -P -C -U$(ARCH)
+obj-vdso := $(addprefix $(output-dir)/, $(obj-vdso)) + +#Make the purecap directory under vdso/ +$(output-dir): +ifdef BUILD_PURECAP_VDSO + @mkdir -p $(output-dir) +endif + +# Explicit build rules are required for .o files when building for purecap, +# as the path to the output directory for built objects no longer matches +# the path to the source files. +# Regular: $(output-dir)/*.o == $(src)/*.[c|S] -> builds with implicit rule +# Purecap: $(output-dir)/*.o != $(src)/*.[c|S] -> needs an explicit rule +ifdef BUILD_PURECAP_VDSO +$(output-dir)/%.o: $(srctree)/$(src)/%.c FORCE | $(output-dir) + $(call if_changed_rule,cc_o_c) + +$(output-dir)/%.o: $(srctree)/$(src)/%.S FORCE | $(output-dir) + $(call if_changed_rule,as_o_S) +endif + # Link rule for the .so file, .lds has to be first -$(obj)/vdso.so.dbg: $(obj)/vdso.lds $(obj-vdso) FORCE +$(output-dir)/vdso.so.dbg: $(obj)/vdso.lds $(obj-vdso) FORCE $(call if_changed,vdsold_and_vdso_check)
# Strip rule for the .so file -$(obj)/%.so: OBJCOPYFLAGS := -S -$(obj)/%.so: $(obj)/%.so.dbg FORCE +$(output-dir)/%.so: OBJCOPYFLAGS := -S +$(output-dir)/%.so: $(output-dir)/%.so.dbg FORCE $(call if_changed,objcopy)
# Generate VDSO offsets using helper script @@ -72,7 +118,7 @@ gen-vdsosym := $(srctree)/$(src)/gen_vdso_offsets.sh quiet_cmd_vdsosym = VDSOSYM $@ cmd_vdsosym = $(NM) $< | $(gen-vdsosym) | LC_ALL=C sort > $@
-include/generated/vdso-offsets.h: $(obj)/vdso.so.dbg FORCE +$(offsets): $(output-dir)/vdso.so.dbg FORCE $(call if_changed,vdsosym)
# Actual build commands @@ -81,9 +127,9 @@ quiet_cmd_vdsold_and_vdso_check = LD $@
# Install commands for the unstripped file quiet_cmd_vdso_install = INSTALL $@ - cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@ + cmd_vdso_install = cp $(output-dir)/$@.dbg $(MODLIB)/vdso/$@
-vdso.so: $(obj)/vdso.so.dbg +vdso.so: $(output-dir)/vdso.so.dbg @mkdir -p $(MODLIB)/vdso $(call cmd,vdso_install)
diff --git a/include/linux/compiler.h b/include/linux/compiler.h index d7779a18b24f..7864350d0edb 100644 --- a/include/linux/compiler.h +++ b/include/linux/compiler.h @@ -3,6 +3,7 @@ #define __LINUX_COMPILER_H
#include <linux/compiler_types.h> +#include <linux/types.h>
#ifndef __ASSEMBLY__
@@ -222,7 +223,7 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val, */ static inline void *offset_to_ptr(const int *off) { - return (void *)((unsigned long)off + *off); + return (void *)((uintptr_t)off + *off); }
#endif /* __ASSEMBLY__ */
Map the purecap vDSO into userspace application address space, therefore providing purecap userspace applications a purecap vDSO to use. Redefine the aarch64 vDSO as the compat vDSO, therefore allowing both regular arm64 and purecap binaries to use a vDSO under PCuABI.
Signed-off-by: Aditya Deshpande aditya.deshpande@arm.com --- arch/arm64/include/asm/elf.h | 24 ++++++++---- arch/arm64/include/asm/vdso.h | 1 + arch/arm64/kernel/Makefile | 2 + arch/arm64/kernel/vdso-purecap-wrap.S | 22 +++++++++++ arch/arm64/kernel/vdso.c | 55 ++++++++++++++++++++++++++- fs/compat_binfmt_elf.c | 24 ++++++------ 6 files changed, 107 insertions(+), 21 deletions(-) create mode 100644 arch/arm64/kernel/vdso-purecap-wrap.S
diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h index f01f1f99cf03..df8dcfb3feaf 100644 --- a/arch/arm64/include/asm/elf.h +++ b/arch/arm64/include/asm/elf.h @@ -188,21 +188,26 @@ do { \ NEW_AUX_ENT(AT_IGNORE, 0); \ } while (0)
+#define ARCH_HAS_SETUP_ADDITIONAL_PAGES +struct linux_binprm; +extern int aarch64_setup_additional_pages(struct linux_binprm *bprm, + int uses_interp); + #ifdef CONFIG_CHERI_PURECAP_UABI +extern int purecap_setup_additional_pages(struct linux_binprm *bprm, + int uses_interp); +#define arch_setup_additional_pages purecap_setup_additional_pages /* - * TODO [PCuABI]: In Transitional PCuABI, AT_SYSINFO_EHDR is passed as NULL - * as there is no purecap vDSO yet. + * TODO [PCuABI]: Look into restricting the bounds of this capability to just + * the vDSO pages, as currently the bounds are of the root user capability. */ -#define ARCH_DLINFO SETUP_DLINFO(0) +#define ARCH_DLINFO SETUP_DLINFO(uaddr_to_user_ptr_safe( \ + (elf_addr_t)current->mm->context.vdso)) #else /* !CONFIG_CHERI_PURECAP_UABI */ +#define arch_setup_additional_pages aarch64_setup_additional_pages #define ARCH_DLINFO SETUP_DLINFO((elf_addr_t)current->mm->context.vdso) #endif /* CONFIG_CHERI_PURECAP_UABI */
-#define ARCH_HAS_SETUP_ADDITIONAL_PAGES -struct linux_binprm; -extern int arch_setup_additional_pages(struct linux_binprm *bprm, - int uses_interp); - /* 1GB of VA */ #ifdef CONFIG_COMPAT #define STACK_RND_MASK (test_thread_flag(TIF_32BIT) ? \ @@ -244,6 +249,9 @@ typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG];
#define COMPAT_ARCH_DLINFO SETUP_DLINFO((elf_addr_t)current->mm->context.vdso)
+#define compat_arch_setup_additional_pages \ + aarch64_setup_additional_pages + #else /* !CONFIG_COMPAT64 */
/* PIE load location for compat arm. Must match ARM ELF_ET_DYN_BASE. */ diff --git a/arch/arm64/include/asm/vdso.h b/arch/arm64/include/asm/vdso.h index 83a50071a85e..709b5bbec255 100644 --- a/arch/arm64/include/asm/vdso.h +++ b/arch/arm64/include/asm/vdso.h @@ -28,6 +28,7 @@
extern char vdso_start[], vdso_end[]; extern char vdso32_start[], vdso32_end[]; +extern char vdso_purecap_start[], vdso_purecap_end[];
#endif /* !__ASSEMBLY__ */
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile index 12eb4c55234e..28f80ffc9ed9 100644 --- a/arch/arm64/kernel/Makefile +++ b/arch/arm64/kernel/Makefile @@ -72,12 +72,14 @@ obj-$(CONFIG_ARM64_MTE) += mte.o obj-$(CONFIG_ARM64_MORELLO) += morello.o obj-y += vdso-wrap.o obj-$(CONFIG_COMPAT_VDSO) += vdso32-wrap.o +obj-$(CONFIG_CHERI_PURECAP_UABI) += vdso-purecap-wrap.o obj-$(CONFIG_UNWIND_PATCH_PAC_INTO_SCS) += patch-scs.o CFLAGS_patch-scs.o += -mbranch-protection=none
# Force dependency (vdso*-wrap.S includes vdso.so through incbin) $(obj)/vdso-wrap.o: $(obj)/vdso/vdso.so $(obj)/vdso32-wrap.o: $(obj)/vdso32/vdso.so +$(obj)/vdso-purecap-wrap.o: $(obj)/vdso/purecap/vdso.so
obj-y += probes/ obj-y += head.o diff --git a/arch/arm64/kernel/vdso-purecap-wrap.S b/arch/arm64/kernel/vdso-purecap-wrap.S new file mode 100644 index 000000000000..715477cda574 --- /dev/null +++ b/arch/arm64/kernel/vdso-purecap-wrap.S @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2023 ARM Limited + */ + +#include <linux/init.h> +#include <linux/linkage.h> +#include <linux/const.h> +#include <asm/assembler.h> +#include <asm/page.h> + + .globl vdso_purecap_start, vdso_purecap_end + .section .rodata + .balign PAGE_SIZE +vdso_purecap_start: + .incbin "arch/arm64/kernel/vdso/purecap/vdso.so" + .balign PAGE_SIZE +vdso_purecap_end: + + .previous + +emit_aarch64_feature_1_and diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c index c9d961249894..3557b7f35307 100644 --- a/arch/arm64/kernel/vdso.c +++ b/arch/arm64/kernel/vdso.c @@ -32,6 +32,7 @@ enum vdso_abi { VDSO_ABI_AA64, VDSO_ABI_AA32, + VDSO_ABI_PURECAP, };
enum vvar_pages { @@ -64,6 +65,13 @@ static struct vdso_abi_info vdso_info[] __ro_after_init = { .vdso_code_end = vdso32_end, }, #endif /* CONFIG_COMPAT_VDSO */ +#ifdef CONFIG_CHERI_PURECAP_UABI + [VDSO_ABI_PURECAP] = { + .name = "vdso_purecap", + .vdso_code_start = vdso_purecap_start, + .vdso_code_end = vdso_purecap_end, + }, +#endif /* CONFIG_CHERI_PURECAP_UABI */ };
/* @@ -143,6 +151,10 @@ int vdso_join_timens(struct task_struct *task, struct time_namespace *ns) #ifdef CONFIG_COMPAT_VDSO if (vma_is_special_mapping(vma, vdso_info[VDSO_ABI_AA32].dm)) zap_vma_pages(vma); +#endif +#ifdef CONFIG_CHERI_PURECAP_UABI + if (vma_is_special_mapping(vma, vdso_info[VDSO_ABI_PURECAP].dm)) + zap_vma_pages(vma); #endif }
@@ -411,6 +423,47 @@ int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp) } #endif /* CONFIG_COMPAT32 */
+#ifdef CONFIG_CHERI_PURECAP_UABI +enum purecap_map { + PURECAP_MAP_VVAR, + PURECAP_MAP_VDSO, +}; + +static struct vm_special_mapping purecap_vdso_maps[] __ro_after_init = { + [PURECAP_MAP_VVAR] = { + .name = "[vvar]", + .fault = vvar_fault, + }, + [PURECAP_MAP_VDSO] = { + .name = "[vdso]", + .mremap = vdso_mremap, + }, +}; + +static int __init purecap_vdso_init(void) +{ + vdso_info[VDSO_ABI_PURECAP].dm = &purecap_vdso_maps[PURECAP_MAP_VVAR]; + vdso_info[VDSO_ABI_PURECAP].cm = &purecap_vdso_maps[PURECAP_MAP_VDSO]; + + return __vdso_init(VDSO_ABI_PURECAP); +} +arch_initcall(purecap_vdso_init); + +int purecap_setup_additional_pages(struct linux_binprm *bprm, int uses_interp) +{ + struct mm_struct *mm = current->mm; + int ret; + + if (mmap_write_lock_killable(mm)) + return -EINTR; + + ret = __setup_additional_pages(VDSO_ABI_PURECAP, mm, bprm, uses_interp); + mmap_write_unlock(mm); + + return ret; +} +#endif /* CONFIG_CHERI_PURECAP_UABI */ + enum aarch64_map { AA64_MAP_VVAR, AA64_MAP_VDSO, @@ -436,7 +489,7 @@ static int __init vdso_init(void) } arch_initcall(vdso_init);
-int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp) +int aarch64_setup_additional_pages(struct linux_binprm *bprm, int uses_interp) { struct mm_struct *mm = current->mm; int ret; diff --git a/fs/compat_binfmt_elf.c b/fs/compat_binfmt_elf.c index b6b453b35946..631f8cff8d36 100644 --- a/fs/compat_binfmt_elf.c +++ b/fs/compat_binfmt_elf.c @@ -120,18 +120,6 @@ #define ELF_PLAT_INIT COMPAT_ELF_PLAT_INIT #endif
-#ifdef compat_arch_setup_additional_pages -#define COMPAT_ARCH_SETUP_ADDITIONAL_PAGES(bprm, ex, interpreter) \ - compat_arch_setup_additional_pages(bprm, interpreter) -#endif - -#ifdef COMPAT_ARCH_SETUP_ADDITIONAL_PAGES -#undef ARCH_HAS_SETUP_ADDITIONAL_PAGES -#define ARCH_HAS_SETUP_ADDITIONAL_PAGES 1 -#undef ARCH_SETUP_ADDITIONAL_PAGES -#define ARCH_SETUP_ADDITIONAL_PAGES COMPAT_ARCH_SETUP_ADDITIONAL_PAGES -#endif - #ifdef compat_elf_read_implies_exec #undef elf_read_implies_exec #define elf_read_implies_exec compat_elf_read_implies_exec @@ -177,6 +165,18 @@ }) #endif
+#ifdef compat_arch_setup_additional_pages +#define COMPAT_ARCH_SETUP_ADDITIONAL_PAGES(bprm, ex, interpreter) \ + compat_arch_setup_additional_pages(bprm, interpreter) +#endif + +#ifdef COMPAT_ARCH_SETUP_ADDITIONAL_PAGES +#undef ARCH_HAS_SETUP_ADDITIONAL_PAGES +#define ARCH_HAS_SETUP_ADDITIONAL_PAGES 1 +#undef ARCH_SETUP_ADDITIONAL_PAGES +#define ARCH_SETUP_ADDITIONAL_PAGES COMPAT_ARCH_SETUP_ADDITIONAL_PAGES +#endif + /* * We share all the actual code with the native (64-bit) version. */
This change is made in preparation for the introduction of a purecap vDSO. The vDSO includes compiler.h; when building for purecap unsigned long will no longer be suitable for storing a pointer value. Replace it with uintptr_t, which can have a variable definition based on the ABI.
Signed-off-by: Aditya Deshpande aditya.deshpande@arm.com --- include/linux/compiler.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/include/linux/compiler.h b/include/linux/compiler.h index d7779a18b24f..7864350d0edb 100644 --- a/include/linux/compiler.h +++ b/include/linux/compiler.h @@ -3,6 +3,7 @@ #define __LINUX_COMPILER_H
#include <linux/compiler_types.h> +#include <linux/types.h>
#ifndef __ASSEMBLY__
@@ -222,7 +223,7 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val, */ static inline void *offset_to_ptr(const int *off) { - return (void *)((unsigned long)off + *off); + return (void *)((uintptr_t)off + *off); }
#endif /* __ASSEMBLY__ */
On 23/11/2023 15:35, Aditya Deshpande wrote:
Hello,
This patch series implements a pure-capability vDSO which purecap userspace applications can directly use. It also fixes the vDSO selftests so that they can be built as purecap binaries, therefore allowing the new purecap vDSO to be tested. The existing aarch64 vDSO implementation is unchanged - it has simply been redefined as the compat vDSO. Therefore, processes using both the standard and purecap ABIs will have a vDSO to use.
Main changes in v3:
- Series has now grown to 5 patches. The first patch duplicates an unmerged upstream change (I forgot to include this in v2). Once it is merged upstream and the fork is rebased this patch may disappear from the log.
- Patch 3 adds some purecap-only code to linux/types.h to enable the vDSO, which uses kernel headers, to be built for purecap.
- v2 removed the vdso-purecap/ directory, as it needlessly replicated the code in vdso/. vdso/Makefile was instead changed so it could build both vDSOs (regular and purecap) from the same source. This is achieved by calling the Makefile twice with BUILD_PURECAP_VDSO=[y/n]. v3 further changes this Makefile; now all purecap generated files are placed in a separate subdirectory (vdso/purecap.)
Aditya Deshpande (5): selftests: vDSO: fix Makefile so that it uses lib.mk correctly selftests/vDSO: Add support for purecap vDSO testing linux/types.h: Redefine uintptr_t to __uintcap_t under purecap arm64: vdso: Build a pure-capability vDSO arm64: vDSO: Provide a purecap vDSO to userspace purecap programs
Now that the CI is back on track, I have merged these patches in next, thanks!
I made a few amendments along the way:
- Part of patch 4 is replaced by "linux/compiler.h: Replace unsigned long with uintptr_t" (posted by Aditya in reply to this series, as I suggested to split this change out of patch 4). - Patch 4, vDSO Makefile: * Fixed the paths of files in the targets variable in purecap, so that files only get rebuilt when needed. * Renamed the installed purecap vDSO to vdso-purecap.so (this is installed under $INSTALL_MOD_PATH when running make vdso_install). * Moved the purecap/vgettimeofday.o CFLAGS / CFLAGS_REMOVE definition after the final assignment to CFLAGS_vgettimeofday.o for clarity. - Patch 5: fixed the signal trampoline offset in purecap by including the appropriate generated header in signal.c (the only place a generated offset is used). The alternative, renaming the generated macro in purecap, turned out to be too complicated to be worth it. - Patch 5: amended the bootstrap Morello kselftest so that it expects AT_SYSINFO_EHDR to be set to a valid capability. - Minor whitespace and comments tweaks, and update to patch 5's commit message.
With that, the Morello kselftests are happy and so is LTP. Worth noting that the clock_gettime04 LTP test still doesn't see the vDSO, because libltpvdso uses getauxval(AT_SYSINFO_EHDR), which returns an error as AT_SYSINFO_EHDR is a pointer (i.e. getauxptr(AT_SYSINFO_EHDR) must be used in purecap). This means that clock_gettime04 still passes, even though libltpvdso needs to be modified for purecap (pointer manipulation, etc.).
Kevin
arch/arm64/Makefile | 6 ++ arch/arm64/include/asm/elf.h | 24 +++++--- arch/arm64/include/asm/vdso.h | 1 + arch/arm64/include/asm/vdso/gettimeofday.h | 58 +++++++++++++++--- arch/arm64/kernel/Makefile | 2 + arch/arm64/kernel/vdso-purecap-wrap.S | 22 +++++++ arch/arm64/kernel/vdso.c | 55 ++++++++++++++++- arch/arm64/kernel/vdso/Makefile | 60 ++++++++++++++++--- fs/compat_binfmt_elf.c | 24 ++++---- include/linux/compiler.h | 3 +- include/linux/types.h | 4 ++ tools/testing/selftests/vDSO/Makefile | 16 ++--- tools/testing/selftests/vDSO/parse_vdso.h | 16 +++++ tools/testing/selftests/vDSO/vdso_test_abi.c | 5 +- .../selftests/vDSO/vdso_test_gettimeofday.c | 6 +- 15 files changed, 252 insertions(+), 50 deletions(-) create mode 100644 arch/arm64/kernel/vdso-purecap-wrap.S
linux-morello@op-lists.linaro.org
-
Aditya Deshpande -
Kevin Brodsky