Application-level mobile virtualization
Harvey Tuch, Staff Engineer, Mobile Virtualization Platform
January 25th 2012
Sponsored by MIT and VMware Academic Programs
VMware: www.vmware.com
VMware Labs: labs.vmware.com
© 2010 VMware Inc. All rights reserved
Agenda




2
Mobile hypervisor distribution
Virtualization at user-level on Linux
Putting it together: Android VMM app
Programming exercise
Mobile hypervisor distribution
3
Mobile hypervisor distribution
 Not all mobile hypervisor components can be typically provisioned
by app store
• System software provisioned by OEM and/or carrier
• Maintenance of “baked on” bits
• Time to market (TTM)
• Market coverage
 What if we move system (privileged) components to application
(user) level?
• Distribute entire hypervisor like a regular app, via app store
• Relax distribution constraints
• Performance, fidelity tradeoffs
4
Mobile hosted architecture
Host world
Guest/monitor world
VM control, device backends
Guest
User
Privileged
Host kernel
5
Kernel modules
Monitor
Deprivileged hosted architecture
Host world
Guest + monitor
User
Privileged
Host kernel
6
App store distribution constraints
 Mobile app security models, e.g. Android
• User-level, deprivileged execution
• Kernel sandboxing, restricted access to:
• File system
• Other applications
• Services (e.g. SMS, GPS, network)
• Native components via JNI
• Unique UID + data directory for each application
7
Agenda




8
Mobile hypervisor distribution
Virtualization at user-level on Linux
Putting it together: Android VMM app
Programming exercise
Virtualization at user-level on Linux
9
User-level virtualization
 Run guest as an application on host OS
• Guest user ISA runs 1:1
• How to handle guest user exceptions? (syscall traps, page faults, etc.)
• How to execute guest privileged ISA? (e.g. on ARM MSR/MRS/MCR/MRC..)
• How to map guest adddress space to application address space on host?
10
User-level virtualization
 Run guest as an application on host OS
• Guest user ISA runs 1:1
• How to handle guest user exceptions? (syscall traps, page faults, etc.)
• How to execute guest privileged ISA? (e.g. on ARM MSR/MRS/MCR/MRC..)
• How to map guest adddress space to application address space on host?
11
Guest user ISA runs 1:1
Guest app
(Angry
Pelicans)
User-level
hypervisor app
Guest kernel
(Linux)
User mode
Host kernel (Linux)
12
Privileged mode
Guest user ISA runs 1:1
mov r0, r3
add r2, r3, #42
str r0, [r2, r8]
app
Guest
(Angry
Pelicans)
Hypervisor
thread(s)
Guest kernel
(Linux)
User mode
Host kernel (Linux)
13
Privileged mode
User-level virtualization
 Run guest as an application on host OS
• Guest user ISA runs 1:1
• How to handle guest user exceptions? (syscall traps, page faults, etc.)
• How to execute guest privileged ISA? (e.g. on ARM MSR/MRS/MCR/MRC..)
• How to map guest adddress space to application address space on host?
14
Guest user syscall exception
mov r0, r3
add r2, r3, #42
str r0, [r2, r8]
app
swi
Guest
(Angry
Pelicans)
Hypervisor
thread(s)
Guest kernel
(Linux)
User mode
Host kernel (Linux)
15
Privileged mode
Guest user syscall exception
mov r0, r3
add r2, r3, #42
str r0, [r2, r8]
app
swi
Guest
(Angry
Pelicans)
Hypervisor
thread(s)
Guest kernel
(Linux)
User mode
Host kernel (Linux)
16
Privileged mode
Guest user syscall exception
mov r0, r3
add r2, r3, #42
str r0, [r2, r8]
app
swi
Guest
(Angry
Pelicans)
Hypervisor
thread(s)
Guest kernel
(Linux)
User mode
Host kernel (Linux)
17
Privileged mode
Guest user syscall exception
mov r0, r3
add r2, r3, #42
str r0, [r2, r8]
app
swi
Guest
(Angry
Pelicans)
Hypervisor
thread(s)
Guest kernel
(Linux)
User mode
Host kernel (Linux)
18
Privileged mode
ptrace system call
 Linux system call allowing one process to control/monitor another
• Used by gdb, strace
• Parent can inspect/modify child’s:
• Register file
• Memory
• Parent can intercept child signals
• Exceptions (including syscall traps) reflected in Unix at user-level as signals
• Parent can single step and inject signals into child
19
ptrace based virtualization
 Hypervisor thread parent
• ptrace child thread representing guest
• Use ptrace:
• Intercept all signals (exceptions)
• Intercept system calls
• Context switch child thread between guest kernel/user
 Performance penalties
• Each exception requires switching between guest thread, host kernel,
hypervisor thread, host kernel and guest thread
• ptrace originally only intended for debug, but now optimized to support Usermode Linux (see also Fiasco-UX)
20
ptrace exception handling
while (1) {
waitpid(guestPID, &status, __WCLONE | WUNTRACED);
...
struct user_regs regs;
ptrace(PTRACE_GETREGS, vmm->currentGuestPID, NULL, &regs);
int sig = WSTOPSIG(status);
switch (sig) {
case SIGTRAP: {
...
}
21
Guest user syscall exception
mov r0, r3
add r2, r3, #42
str r0, [r2, r8]
app
swi
Guest
(Angry
Pelicans)
Hypervisor
thread(s)
Guest kernel
(Linux)
User mode
Host kernel (Linux)
22
Privileged mode
User-level virtualization
 Run guest as an application on host OS
• Guest user ISA runs 1:1
• How to handle guest user exceptions? (syscall traps, page faults, etc.)
• How to execute guest privileged ISA? (e.g. on ARM MSR/MRS/MCR/MRC..)
• How to map guest adddress space to application address space on host?
23
Guest privileged ISA
Guest app
(Angry
Pelicans)
Hypervisor
thread(s)
ldr r0, [r3]
msr cpsr_f, r0
Guest kernel
(Linux)
User mode
Host kernel (Linux)
24
Privileged mode
Guest privileged ISA
 Sensitive + privileged instructions
• E.g. MCR/MRC on ARM
• Trap+emulate with ptrace
 Sensitive + non-privileged instructions
• E.g. MSR/MRS on ARM
• Paravirtualization
• Hypercall traps to hypervisor parent process via ptrace
25
User-level virtualization
 Run guest as an application on host OS
• Guest user ISA runs 1:1
• How to handle guest user exceptions? (syscall traps, page faults, etc.)
• How to execute guest privileged ISA? (e.g. on ARM MSR/MRS/MCR/MRC..)
• How to map guest adddress space to application address space on host?
26
Guest address space mapping
4GB
0
Guest kernel
27
Guest user process
Guest address space mapping
4GB
0
Guest kernel
Guest user process
4GB
0
Host kernel
28
Host user process
Guest address space mapping
4GB
0
Guest kernel
Guest user process
4GB
0
Host kernel
Host user process
• Reduced guest address space
• Guest kernel modifications required
• Guest application visible
29
Agenda




30
Mobile hypervisor distribution
Virtualization at user-level on Linux
Putting it together: Android VMM app
Programming exercise
Putting it together: Android VMM app
31
Android .apk anatomy
Meta-data
Resources
DEX (Java)
.apk
Native code (JNI)
32
Certificates
Android VMM application
 Java frontend
• VM lifecycle management
• Display VM framebuffer contents
• Touchscreen input
• Invoke native code via JNI
 Native code (JNI)
• ptrace-based VMM
 Resources
• Guest kernel + applications (LBS VM image)
33
Agenda




34
Mobile hypervisor distribution
Virtualization at user-level on Linux
Putting it together: Android VMM app
Programming exercise
Programming exercise
 http://labs.vmware.com/academic/mit-iap-2012-mobile-virtualization
 Implement parts of deprivileged mobile hypervisor
• Context switching
• Paravirtualized hypercall handling
 Run on your Android smartphone (or SDK emulator)
 Complete by 5th February 2012 to enter draw to win iPad
 Questions and clarification to: htuch@vmware.com
35