CSCI1600: Embedded and
Real Time Software
Lecture 27: Real Time Linux
Steven Reiss, Fall 2015
Conventional Operating Systems

Memory model
 Independent address spaces
 Allocation and deallocation
 Stack management

Device support
 Abstraction based on descriptors
 Clock and other basic devices built in
 High-level I/O support (e.g. networking)

Thread / process model (with scheduling)

IPC support (pipes, sockets, shared memory)
 Synchronization primitives
Conventional OS Problems

Size (for embedded)




Micro-kernel based architectures help here
Unpredictable (and poor) latency

Slow interrupt handlers

Unpreemptible kernel code

Latency might be built into hardware

System management interrupts on x86

DMA bus mastering; USB handling; VGA console; Page faults
Unpredictable Scheduling

Desktop/server heuristics

Scheduling overhead grows with workload
Poor support for precise timing

Timing accuracy related to tick overhead
Are the Problems Fixable?
 Want to retain the bulk of prior work
 Hundreds of device drivers
 Many abstractions make programming easier
 TCP/IP, File systems, IPC, Memory
 How can we do this?
 Two approaches to this are used
 Fixing the problems
 Sidestepping the problems
RT-Linux
 Assume most of the program is non-real time
 There is a small hard (soft) real-time component
 Attempt to side step the problem
 Run the OS as a real time task
 Allow other RT tasks (outside the OS)
 Provide the benefits of an OS and still have a real time
environment
 Except that Linux tasks are now sporadic
Linux on a RTOS
 We’ve seen this before
 Scheduling sporadic tasks
 Bandwidth servers (Total and Constant)
 Group all the sporadic tasks together
 Provide a real-time task (with fixed bandwidth)
 This executes the various sporadic tasks
 Using their priorities
 This is basically RT-Linux
 Linux is “ported” to use RTOS primitives instead of hardware
 Like running on a virtual machine
RT Linux
RT-Linux Details
 RTLinux core
 Non-preemptible
 Hard real time scheduler
 Basic OS functionality
 Deterministic interrupt-latency ISRs run in core
 Others are transferred to Linux via a FIFO
 Primitive tasks
 Only statically allocated memory, no virtual memory, fixed
priority
 Run as real time threads
RT-Linux Details
 Real time tasks
 No address space protection
 Run with disabled interrupts
 FIFO (message queue)
 Connects real time tasks with LINUX
 Shared memory synchronization
 Non-real time tasks
 Run as Linux processes
RT-Linux Pros and Cons
 Pros
 Hard real-time guarantees (low tens of microseconds)
 OS code is virtually unchanged
 Applies equally well to other OS’s
 Supports BSD as well as Linux
 Cons
 Real time task lose the convenience of a mature wellunderstood API
 Communication with Linux processes is ad-hoc
 Message queues – but user needs to define the messages
Fixing the Problems
 Problems to tackle
 Scheduling, latency, timing
 Techniques
 Priority scheduling and priority inheritance
 Limit un-preemptible code
 Offer high-resolution timers
Scheduling

Real time schedulers are actually simpler

Support for fixed (absolute) priority

Ability to bind tasks to specific processors (cores)

Create new classes of processes (RR, FIFO)

Round robin runs for a fixed time slice

FIFO is never guaranteed

Enhance mutexes to support priority (PIP,PCP)

Problem: scheduling latency

Support fixed priorities

But scheduling decisions might be deferred
 Interrupt handlers cannot be preempted
 Spin-locked code can not be preempted
Minimizing Latency

Simple Solution:


Threaded interrupt handling

Move interrupt handlers into kernel threads; true ISR is now very short

Handling can be preempted
Preemptible spin locks


Turn spin locks into mutexes
This almost works

Timer interrupts need to be handled directly

Individual handlers are still atomic

Spinlocks not always equivalent to mutexes

Programming devices, modifying page tables, in the scheduler

Read-copy-update synchronization in Linux make non-preemption assumptions

New type: raw_spinlock_t
Other Improvements
 Dynamic tick
 Ask for timer IRQ as needed, not periodically
 High resolution timers
 O(1) scheduler
 Robust mutexes
 O(1) kernel memory allocation
 Ability to lock pages into memory (avoid page faults)
 Most problems can be minimized
 But retrofitting them to an OS make hard real time unlikely
Solaris
 Attempt to fix the problems
 Starting with the initial OS design
 Not retrofitted
 Features of Solaris for real time
 Interrupts are handled by threads
 Kernel is fully preemptible and multithreaded
 Real time scheduling is supported
 Deterministic with real time priorities
 Fixed priority scheduling supported
Solaris Real Time Threads
 Separate classes of threads
 Fixed-priority (non-varying)
 But lower priority than Operating System
 Real-Time (fixed priority with fixed quantum)
 Higher priority than operating system
 Real time threadss
 Lock their pages in memory
 Can call OS (but then run at lower priority)
Solaris Features
 Priority inheriting synchronization primitives
 Processor sets
 The ability to isolate one or more CPUs for scheduling
 Ability to lock tasks to a particular core
 Full support for the Posix real time standard
 Real time threads
 High-precision timers and clocks
 Fast TCP/IP (zero copy)
 Memory locking
 Early binding of shared libraries
Solaris
 Has been used as a hard real time environment
 But not as an embedded environment
 Too large
Embedded Linux
 Linux is a common tool for embedded systems
 Linux itself is relatively small
 Linux is fairly modular
 It can run in 1M (or less) of memory
Requirements on RT/Embedded
 Safety Requirements
 The car won’t crash
 Trains won’t crash
 Both traffic lights won’t be green at the same time
 You won’t blow a fuse
 The system won’t deadlock
 The heat and air conditioning won’t be on at the same time
More Requirements: Timing
 Forms
 X will occur within k ms of event Y
 X will occur every 10 +/- 1 ms
 Examples
 The solenoid will trigger within 10 ms of the bumper being
hit
 The switches will be sensed every 20 ms
 The heat will come on within 5 minutes of detecting the
room is too cool
More Requirements: Fairness
 Forms
 Eventually X will occur
 If X occurs, then eventually Y will occur
 X will occur infinitely often
 As long as Y is true, X will occur infinitely often
 Examples
 Eventually you will add K to the score when you hit Y
 As long as the heating system is on, the room will
eventually be comfortable
Requirements
 Are often mandatory (hard)
 Failure is not a viable option
 Car or plan crashes
 Broken devices
 Heart stops pumping
 Water is supplied unfiltered
 The nuclear plant does not shut down
 This is what hard real time means
What Are Your Requirements?
Problems
 How do you show the system is safe
 How do you get confidence in the system’s safety
 When can you reasonably deploy the system
 Can the system be used in a medical device
 Can peoples lives depend on the system
Exercise
 Suppose you wanted to sell your tic-tac-toe box
 What would you want as requirements
 How would you check these?
 Take the ankle-mounted obstacle finder example
 What would be the requirements
 How would you check these?
Helpful Techniques: SE

Good requirements analysis


Understanding all the possible problems and solutions
Good specifications

Accurate modeling

Showing the models are correct
 Petri net and FSA validation

Design for security

Defensive coding

Building monitors as part of the code
 Enter a safe state if the monitor detects anything wrong
 Enter a safe state if the monitor detects anything unusual

Checking tools: lint, compiler, modeling analysis
Homework
 Read chapter 13