Sharing and protection in
Multics
Landon Cox
February 5, 2016
Layer 4
User programs
Layer 3
I/O devices
Layer 2
Console
Superviso
r code
Layer 1
Pager
Layer 0
Scheduler
User/kernel translation data
4GB
Kernel data
(same for all
page tables)
3GB
(0xc0000000)
User data
(different for
every process)
0GB
Virtual
memory
Where to store translation data?
1. Could be kept in physical memory
• How the MMU accesses it
• On x86, PTBR (aka CR3 register) refers to physical memory
2. Could be in kernel virtual address space
• A little weird, but what is done in practice
• How to keep translation data in a place that must be
translated?
• Translation for user address space is in kernel virtual memory
• Kernel’s translation data always in physical memory (pinned)
• Does anything else need to be pinned (and why)?
• Kernel’s page fault handler code also has to be pinned.
Where to store translation data?
• How does kernel access data in user’s address
space?
• If for currently loaded process, access using current page table
• If for another process, have to change page table
Kernel vs. user mode
• Who sets up the data used by the
MMU?
• Can’t be the user process
• Otherwise could access anything
• Only kernel is allowed to modify any memory
• Processor must know to allow kernel
• To update the translator
• To execute privileged instructions (halt, do
I/O)
Kernel vs. user mode
• How does machine know kernel is
running?
• This requires hardware support
• Process supports two modes, kernel and user
• Mode is indicated by a hardware register
• Mode bit
Protection
GBs
of protected data
1. All memory accesses go through a
translator
• Who can modify translator’s data?
2. Only kernel can modify translator’s data
• How do we know if kernel is running?
3. Mode bit indicates if kernel is running
• Who can modify the mode bit?
One bit
of protected data
Making progress: the amount of protected data is down to a
Protecting the mode bit
• Can kernel change the mode bit?
• Yes. Kernel is completely trusted.
• Can user process change the mode
bit?
• Not directly
• User programs need to invoke the kernel
• Must be able to initiate a change
When to transition from user to kernel?
1. Exceptions (interrupts, traps)
• Access something out of your valid address space
• (e.g., segmentation fault)
• Disk I/O finishes, causes interrupt
• Timer pre-emption, causes interrupt
• Page faults
2. System calls
• Similar in purpose to a function call
• Kernel as software library
Example system calls
• Process management
• Fork/exec (start a new process), exit, getpid
• Signals
• Kill (send a signal), sigaction (set up handler)
• Memory management
• Brk (extend the valid address space), mmap
• File I/O
• Open, close, read, write
• Network I/O
• Accept, send, receive
• System management
• Reboot
System call implementation
• Syscalls are like functions, but different
• Implemented by special instruction
• syscall
• Execution of syscall traps to kernel
• Processor safely transfers control to kernel
• Hardware invokes kernel’s syscall trap
handler
System call implementation
• Libc wraps systems calls
• C++/Java make calls into libc
Tracing through “cin >> a;”
C++: cin >> a Java: in.read()
C: read ()
libc: syscall(SYS_read,filenum,offset,numbytes)
Processor: kernel’s syscall handler
kernel: sys_read
kernel: // perform I/O
Crucial
step
Kernel trap details
• Hardware must atomically
1.
2.
3.
4.
Set processor mode bit to “kernel”
Save current registers (SP, PC, general purpose)
Change execution stack to kernel (SP)
Jump to exception handler in kernel (PC)
• What does this look a lot like?
•
Switching between threads (see previous lecture)
Kernel trap details
• User process can initiate mode switch
• But only transfers control in limited way
• (i.e., to predefined kernel code)
• How does processor know where to
jump?
• Stored in hardware “interrupt vector table”
• Who can update the interrupt vector
table?
• Kernel does this at startup
• Code that runs first gets control of machine
Syscall arguments, return
values
• For args passed in memory, in which address space do they
reside?
• User (programs can’t access kernel’s portion of address space)
• Linux passes parameters via registers
• Before syscall
• Push ebp, edx, ecx vals onto user stack
• Copy esp (stack pointer) into ebp
•
•
•
•
Each process has kernel stack
CPU knows kernel stack via “Task Segment State”
CPU sets esp using TSS
Kernel finds user stack in ebp register
Tracing through read()
C: read (int fd, void *buf, size_t size)
libc: push fd, buf, size onto user’s stack
Note: pointer!
kernel: sys_read () {
verify fd is an open file
Verify
verify [buf – buf+size) valid, writable argument
read file from data (e.g. disk)
s!
find physical location of buf
copy data to buf’s physical location
}
Layer 4
User programs
Layer 3
I/O devices
Layer 2
Console
Layer 1
Pager
Layer 0
Scheduler
THE
• Were THE processes protected from each other?
• Yes, processes had private segment spaces
• Those spaces were isolated via the compiler and Layer 1
• Isolation/protection was the motivation for
layering
• What is the flip side of protection?
• Sharing!
• There are often many benefits to sharing (i.e., “features”)
• Sharing also creates complexity
• Ongoing issue: balancing features and
complexity
THE
• Could THE processes share in-core data?
• Only within the structure of the hierarchy
• Lower layers were like shared libraries
• User processes could not share with each other
• What about persistent data?
• Not really, “no common data base.”
THE
• Important idea not supported by THE
• The notion of a user
• “not intended as a multi-access system”
• What is a “user” in a computer system?
• Identifier assigned to processes and storage objects
• Unit of access control in most systems
• Often associated with a person, but doesn’t have to
• Do modern machines support users?
• Desktops/servers do, mobile devices kind of do
Multics
• Multi-user operating system
• Primary goal was to allow efficient, safe sharing btw users
•
What was the central data abstraction in Multics?
•
•
•
•
A segment
All data was contained within a segment
No distinction between files and memory
Means everything is persistent
• What is a segment?
• Named range of contiguous data + associated meta-data
• Accessed through loads/stores in memory
• Think of a segment as an mmapped region of memory
Multics segments
• Implications of segments
• Difference between a program and process?
• We think of a process as an execution of a program
• In Multics, there was very little distinction
• What does this force programmers to do?
•
•
•
•
Must explicitly manage segments
Process cannot just exit and return to a “known” state
Places a lot of burden on programmers
Beware, complexity creep …
Segment access
Descriptor Base
Register
Address [s,i]
Descriptor Segment
Core|Ld
s
Segment
Descriptor Word
Segment
Ld
Core|Ls|Acc|F
Lookup algorithm?
i
Ls
word of data
if(DBR.L < S) fault
sdw = DW[s]
if(sdw.F || sdw.L<i) fault
if(!verify(sdw.Acc)) fault
return sdw.Core + i
What about paging?
• How do we make sure that segments are incore?
• Segments are broken into fixed-size pages (1k)
• Another structure to map parts of a segment to pages
• Page tables describe where pieces are in
core
• Descriptor segment is a segment too
• Must locate the pages on which it is located
• Can then walk data structure to locate data
(1) Find page
table for
descriptor
segment
Paged segment access
(2) Page holds
part of descriptor
segment
[s,i]  [sp,sw,ip,iw]
(3) Use
descriptor
segment to find
page table for
data segment
Page sp of DS
Segment
Descriptor Word
Page Table of
Segment S
Page ip of
Segment S
Core|L
Core|
L
sp
sw
Core|L|Acc|F
What’s the problem with this?
Core|L
ip
word of data
Page Table of
Descriptor Segment
Descriptor Base
Register
i
w
(4) Use page
table to locate
data we want to
access
This is a lot of memory accesses
How do we make it faster?
Caching in hardware: TLB
Multics supervisor code
• Location of THE supervisor state
• In separate processes at low-level layers
• In Multics, where did supervisor state
reside?
• In segments mapped into every process
• Supervisor segments were at top of address space
• How was supervisor state protected?
•
•
•
•
Hardware protection provided by processor
Hardware supported 8 protection rings
Idea was to enforce layering via hardware
Exactly like mode bit (kernel/user) you are used to
Multics supervisor code
• How did programs invoke the supervisor?
• Just a procedure call
• Calls always reside in segments at top of addr. sp.
• Just jump into code in those segments
• What else has to happen?
• Have to change hardware protection mode
• Who/what changed protection modes?
• This was hardware enforced
• All segments were assigned a ring level (including code)
• Mode for an instruction was set in descriptor on which it resided
Segment sharing
• To facilitate sharing need common namespace for segments
• Multics uses file hierarchy to name all segments
• Populating address space like a bunch of mmap calls
• Why is this a nice abstraction?
• Maps human-readable names (easy to program) to data
• Makes it easy to specify what data you want to operate on
• Who manages the namespace?
• Must be privileged code
• Common data structure used by all processes
• Only code trusted by all should be allowed to modify directly
Segment sharing
• What operations can a process perform on the namespace?
•
•
•
•
•
Create a segment
Delete a segment
Change a segment’s name
Change the access policy of a segment
Read the content of a directory
• Who prevents collisions in the namespace?
• The supervisor code
• On a request to create a segment, checks to see if it already exists
Segment sharing
• How does a process populate its address space?
•
•
•
•
•
Must know or compute parts of the namespace
Invokes the OS to map named segment into memory
OS updates the Known Segment Table for process
KST maps segments to pathnames
OS returns beginning of segment, length to process
• Once mapped, how is data loaded into memory?
•
•
•
•
Data is demand loaded
Process accesses address (e.g., [s,i])
This triggers a page fault and a trap to the OS
The OS uses the KST to locate data on disk
Discussion
• Are Multics segments a good idea or bad?
• Programmers must do a lot of garbage collection
• Temporary state must be re-initialized by hand
• Adds complexity
• Danger of accessing persistent data via
loads/stores?
• Buggy programs can lead to stray writes
• Stray writes become permanent
• Why are permanent stray writes bad?
• Important data structures could become corrupted
• “Restarting” program just puts you back in a bad state
Discussion
• How do most modern systems present persistent storage?
• Must access persistent storage through file system
• File system interface: open, read, write
• Why is an explicit file system interface safer?
•
•
•
•
Write system call is similar to a “commit”
An explicit acknowledgement that data is ready to be made persistent
Buggy programs much less unlikely to generate a spurious write call
Spurious loads and stores are very common in buggy programs
• We actually see this observation in Multics itself. Where?
•
•
•
•
In how it handles the namespace
Important, persistent data structure
Can only be modified in a controlled way, through narrow interface
Nice to offer same protections for user data
Topic we will revisit
• Interfaces to persistent storage
• Periodic source of new research
• In the 90s: battery-backed RAM
• Memory persists across reboots
• Rio-Vista from Michigan (SOSP ‘97)
• In the 00s: phase-change memory
• Fast, persistent substrate
• BPFS from Microsoft and UCLA (SOSP ‘09)