CRITERIA AND AN OBJECTIVE APPROACH TO SELECTING COMMERCIAL
REAL-TIME OPERATING SYSTEMS BASED ON PUBLISHED INFORMATION
Phillip A. Laplante1
Penn State University
Abstract
Matching a commercial real-time operating system to a particular application is a problem for
which there is no obvious solution strategy. While factors such as estimated production volume,
likely processor involved, mission criticality, cost, reliability, software support, ease-of-use, and
maintainability need to be considered, oftentimes the selection is merely based on the preferences
of the design team, management, or intensity of vendor marketing.
In this paper the selection of real-time operating systems is examined. A set of criteria and an
associated metric are presented that rely only on published marketing information. Then, to
demonstrate the application of this metric, information from several existing commercial realtime operating systems is collected and analyzed. Five hypothetical applications are then
matched with the most appropriate real-time operating system, using the criteria and metric.
The contribution of this paper is a set of criteria, a metric, and a methodology for matching realtime operating systems to applications based on marketing information, without the need for
intensive performance testing.
Keywords: metrics, real-time operating systems, selection criteria
1. Introduction
There is surprisingly little work on metrics for evaluating real-time systems. A literature search
turned up nothing for evaluating commercial real-time operating systems (RTOS2) in journals. A
number of reports are available either from workshops or trade magazines but these were difficult
to find.
The dearth of published work on evaluation of RTOS is surprising because there is both an
engineering and business advantage in some formalized methodology for evaluating candidate
systems. In any case, some metrics that are documented for RTOS only involve scheduling
features. For example, Buttazzo gives average response time, total completion time, weighted
sum of completion times, maximum lateness and maximum number of late tasks [1]. Other
metrics are based on extensive experimental procedures, [2], [3], [4], [5], [6]. Unfortunately, these
are fairly limited in scope and the results of a given test cannot be necessarily imputed beyond the
experimental platform. Hence, they provide only partial guidance in the selection of a commercial
RTOS solution.
1
Associate Professor of Software Engineering, Penn State Great Valley School of Graduate Professional Studies, 30 Swedesford Road, Malvern,
PA 19355-1443; (610) 725-5314; fax (610) 889-1334; - plaplante@psu.edu. The Author wishes to acknowledge Rick Bruno, Sandip Patel,
Nutthinee Katapituk, Jason Poleski, Sebastian Lesniak, Eric Shoup, from Lockheed Martin Naval Electronics and Surface Systems for assisting
with the data collection for this paper.
2
“RTOS” or “OS” are used as short hand for “real-time operating system” or “operating systems,” depending on the context.
1
Unfortunately, unless a comprehensive experience base exists using several alternative
commercial real-time operating systems in multiple, identical application domains, there are only
two ways to objectively determine the fitness of a product for a given application. The first is to
rely on third party reports of success or failure. These abound and are published widely on the
Web (e.g. [3]). The second is to compare alternatives based on manufacturer’s published
information from brochures, technical reports, and Web sites. Therefore, there should be some
way to incorporate manufacturer’s published data into the decision-making process, especially
when independent performance tests are unavailable and impractical to conduct.
The contribution of this paper, then, is an objective and flexible technique for comparing
commercial real-time operating systems based on marketing information. This technique can be
used, however, in conjunction with supplemental information from actual experience and third
party reports.
1.1 Build Versus Buy
A common question that is asked at the time of systems requirements specification is “should a
commercial real-time solution be used or should one be built from scratch?” While the answer
depends on the situation, commercial kernels3 are frequently chosen because they generally
provide robust services, are easy to use, and may be portable.
While commercial RTOS provide flexibility in the number of tasks supported and scheduling
discipline, there are drawbacks in their use. For example, they are usually slower than the hardwired interrupt driven model because tremendous overhead is incurred in implementing the taskcontrol block model, which is the typical architecture for Commercial RTOS [7], [8].
Furthermore, commercial solutions tend to suffer from featureitis – too many unneeded features
are incorporated in order for the product to have the widest appeal. The run-time and storage
costs of these features may be excessive. Finally, manufacturers may be tempted to make
misleading claims, or give best case performance figures. The worst-case response times, which
are the most informative, can generally not be known. If they are known, they are typically not
published because they would place the product in an unfavorable light.
For embedded systems, when the per-unit charge for commercial products is too high, when
desired features are unavailable, or when the overhead is too high, the only alternative is to write
the real-time kernel. But this is not a trivial task [7], [8]. Therefore, commercial RTOS should be
considered wherever possible.
1.2 Commercial Real-Time Operating Systems
There are many commercial solutions available for real-time systems, but deciding which one is
most suitable for a given application is difficult. Many features of embedded real-time operating
systems must be considered including cost, reliability, and speed. But there are many other
characteristics that may be as important or more important, depending on the application. For
example, the real-time operating system largely resides in some form of ROM and usually
3
The term “kernel” is used to mean that part of the operating system that provides task scheduling and intertask synchronization and
communication.
2
controls hardware that will not tolerate many faults; therefore, the RTOS should be fault-tolerant
[9]. Also, the hardware needs to be able to react to different events in the system very quickly;
therefore, the OS should be able to handle multiple processes in an efficient manner [10]. Finally,
because the hardware on which the operating system has limited memory, the operating system
must also require a reasonable amount of memory in which to run [7].
In fact, there are so many functional and non-functional attributes of any commercial RTOS,
that evaluation and comparison must be a subjective endeavor. Some structure, however, should
be used in the heuristic decision-making. Using a standard set of criteria provides such structure.
2. Criteria for selecting real-time systems
What makes a good RTOS? In short, it depends on the standard definition of a real-time system,
namely, a system in which requirement satisfaction (logical correctness) is based on the
correctness of the system’s behavior in terms of data and time [7]. In essence, the RTOS must
have correct and bounded behavior under all system load scenarios.
From a business perspective, there is concern about compatibility, especially in an environment
where there are heterogeneous and legacy systems. Furthermore, it is critical that the RTOS
support newer or standard network technologies for flexibility of product development and
maintenance.
Mainstream real-time researchers and engineering practitioners seem to agree on the following
desirable characteristics for real-time systems, these are:





timeliness
design for survival under peak load (schedulability)
predictability
fault-tolerance
maintainability [1]
Therefore the selection criteria should reflect these desiderata.
Consider thirteen selection criteria, m1 m13 , each having a range mi  [0,1] where unity
represents the highest possible satisfaction of the criterion and zero represents complete nonsatisfaction. Recognizing that the importance of individual criterion will differ depending on the
application, a weighting factor, wi  [0,1] , will be used for each criterion mi , where unity is
assigned if the criterion has highest importance, and zero if the criterion is unimportant for a
particular application. Then a fitness metric, M [0,1] , is formed as
13
M
w m
i
i
.
i 1
Clearly, a higher value of M means that the RTOS is well suited to the application, while a
lower value means that the RTOS is not well suited for the application.
3
(1)
While selection of the values for mi and wi will be subjective for any given RTOS and any
given application, the availability of this heuristic metric, provides a handle for objective
comparison, historical perspective and other uses. The following sections describe the criteria.
2.1 Minimum Interrupt Latency
The minimum interrupt latency, m1 , measures the time between the occurrence of hardware
interrupt and when the interrupt’s service routine begins executing. A low value of m1 represents
relatively high interrupt latency, while a high value of m1 represents low latency.
This criterion is important because if the minimum latency is greater than that required by the
embedded system, a different operating system must be selected.
2.2 Total Number of Tasks Supported
This criterion, m2 , defines the most concurrent processes the operating system can simultaneously
support. Even though the operating system may support a large number of tasks, this metric may
be further limited by available memory. This criterion is important for systems that need
numerous simultaneous processes. A relatively high number of tasks supported would result
in m2  1 , while few task supported would suggest a lower value for m2 .
2.3 Memory Requirements
Criterion m3 specifies the system memory required to support the OS. It does not include the
amount of additional memory required to run the system’s application software. m3  1 suggests a
minimal memory requirement, while m3  0 would represent a larger memory requirement.
2.4 Scheduling Mechanism
The scheduling mechanism criterion, m4 , enumerates whether preemptive, round-robin, or some
other task scheduling mechanism is used by the operating system. Round-robin scheduling is
used to allow tasks at the same priority level to share the resources of the system. Preemptive
scheduling allows high priority tasks to begin immediately by preempting the current task
running. Most PC operating systems use preemptive task scheduling because an indeterminate
amount of processes will be running, each with their own unique CPU requirements. However,
when the number of processes is fixed, as in the case of many real-time systems, preemptive task
scheduling is not always a more efficient algorithm than round-robin scheduling [10].
Kernel preemption is another feature that reduces response times and increases system
schedulability. If the kernel is not preemptable, then a low priority task that is in the middle of a
kernel service call would inherit the highest priority to the low priority task, creating a priority
inversion. Kernel preemptability alleviates this situation.
If many scheduling mechanisms are supported and if the kernel is preemptable, then a high
value would be assigned to m4 .
2.5 Intertask Synchronization Mechanism
4
Criterion m5 refers to the available methods the operating system has to allow processes to
communicate with each other. Among possible choices are mutual exclusion (mutexes), binary
and counting semaphores, POSIX pipes, message queues, shared memory, FIFO buffers, control
sockets, and signals and scheduling. Each mechanism has advantages and disadvantages [11],
[12]. Let m5  1 if the RTOS provides all desired scheduling mechanisms. A lower value for m5
implies that less synchronization mechanisms are available.
2.6 Context Switch time
Criterion m6 refers to the time it takes for the kernel to save the context when it needs to switch
from running one task to another. The context may include one or more of the following: PC
pointer, stack pointer, heap pointer, I/O pointer, floating point registers, temporary storage
registers, and any MMU registers. This information is needed so that the OS can return to the
point of interruption without loss of information [13]. A relatively short context switch time
would result in a higher value for m6 .
2.7 Programming Environment
Application availability, m7 , refers to the amount of software available (either that ships with the
operating system or is available elsewhere) to develop applications to run on the operating system.
For example, RTLinux is supported by the GNU suite of software, which includes the gcc C
compiler and many freely available software debuggers, and other supporting software. This is an
important consideration, especially when using an unfamiliar operating system. This criterion also
takes into consideration the number of other operating systems that are compatible with the given
RTOS.
Let m7  1 if a large amount of software were available and compatible, while 0 would mean
that little or none was available or compatible.
2.8 Portability
Criterion m8 refers to the different processors that the OS supports. This is important in terms of
portability and compatibility with off-the-shelf hardware and software. This criterion also
encompasses the range of peripherals that the OS can support, such as video, audio, SCSI, and
such. A high value for the criterion represents a highly portable and compatible RTOS.
2.9 Source Code Availability
Criterion m9 refers to whether the code of the operating system will be available to the developer,
for tweaking or changes. The source also gives insight to the RTOS architecture, which is quite
useful for debugging purposes and systems integration. Setting m9  1 would suggest open source
code or free source code, while a lower value might be assigned in proportion to the purchase
price of the source code. Let m9  0 if the source code were unavailable.
5
2.10 Software Support (warranty)
Criterion m10 refers to the after-sale support a company puts behind its product. Most vendors
offer some sort of free technical support for a short period of time after the sale, with the option of
purchasing additional support if required. Some even offer on-site consultation. A high value
might be assigned to a strong support program, while m10  0 if no support is provided.
2.11 Cost
This criterion is directly related to the cost of the RTOS including the development license cost as
well as the per target license costs. This consideration is critical because for some systems, the
RTOS cost may be disproportionately high. In any case, a relatively high cost would be assigned
a very low value, while a low cost would merit a higher value for m11 .
2.12 Royalty Fee
Frequently, a per target license or royalty fee is charged for each delivered system. In the case
where few targets are intended, this can be inconsequential. However, when a large number of
targets are intended then the per target licensing fee can be considerable. This criterion, m12 , is
high if a relatively low per unit royalty fee is charged.
2.13 Networking Support
This criterion, m13 , is based on a listing of what networks and network protocols are supported by
the given RTOS. A high value for the criterion represents a relatively large number of networks
supported.
3. Evaluation of Commercial Real-Time Operating Systems
A number of commercial RTOS are now examined based on the criteria introduced. Although the
data is real, the manufacturer names are omitted as the intention is not to imply a recommendation
of any product. These systems were selected based on both the availability of published
information on the Web, and the knowledge that these systems have fairly widespread use. Of
course, the scope of the study could have been expanded to consider many other viable
commercial products. But the intent of this work is to provide a methodology and some examples
of its use – not to completely assess the RTOS marketplace.
Since this work was done, clearly some of the systems performance characteristics may have
changed. One must realize that any analysis is only as accurate as the data provided by the
manufacturers at the time of the study4. This is typical of a real-world situation where there is
little opportunity to test the accuracy of a manufacturer’s claim prior to making a buy decision.
However, this constraint does not diminish the value of this work, which is to provide a
methodology for assessment of goodness of fit for commercial real-time solutions and a particular
application.
In the sections that follow, the following assumptions are made:
4
The data were collected during early 2002.
6

For all the sample RTOS, assume that the calculations for the number of interrupt, the
minimum time that it takes, and other system analysis based on the metrics chosen are
performed under the same conditions, i.e. sampling, time constraints, number of
processors, and etc.

Maximum or minimum of tasks refers to the OS object, such as the MMU, device drivers,
and other system tasks.

Assume that interrupt refers to “hardware interrupt”. “Software interrupts”, together with
hardware interrupts and other vectoring mechanisms provided by the processor, are
referred to as “ exception handling”.

Thread switching time is equivalent to the measurement of context switching time.
In the cases where a criterion value can be assigned, this is done. Where the criteria are
“processor dependent” or indeterminate absent a real-application, assignment of a rating is
postponed, and a value of * is given. This “uncertain” value is fixed at the time of application
analysis. Note too that the values between tables need to be consistent. So, for example, if a 6
microsecond latency yields m1  1 for RTOS X, the same 6 microsecond latency should yield
m1  1 for RTOS Y.
Consider commercial RTOS A. Table 1 summarizes the criteria and ratings. The rationale for
the ratings are described in the following paragraphs.
.
7
Table 1: Summary data for real-time operating system A.
Criterion
m1
Description
Minimum Interrupt Latency
Rating
*
Comment
CPU dependent
m2
Total Number of Tasks Supported
0.5
32 Thread Priority Levels
m3
Memory Requirements
0.7
ROM: 60k
m4
m5
0.25
0.5
Preemptive
Direct Message Passing
m6
Scheduling Mechanism
Intertask Synchronization
Mechanism
Context Switch Time
m7
Programming Environment
m8
Portability
m9
Source Code Availability
m10
Software Support (Warranty)
0.5
Paid phone support
m11
m12
m13
Cost
Royalty Fee
Networking Support
0.7
0
1
approximately $2500
no
TCP/IP, FTP, SMTP, SNMP, PPP, ATM, ISDN,
X25, Telnet, Bootp, http-server
*
na
1
Compilers: Greenhills, ADS, ARM, Diab, GNU,
Tools: Mentor Graphics, Greenhills, SDS, ARM,
ADS
PowerPC, ARM, MIPS, DSP’s, Network Processors
0.8
1
yes
The product literature indicated that the minimum interrupt latency is CPU dependent,
therefore a * value is assigned here (which will be later resolved as 0.5 for the purposes of
evaluating the metric). Context switch time is not given and so a * is also indicated. The RTOS
supports 32 thread priority levels, but it is not known if there is a limit on the total number of
tasks, so a value of 0.5 is assigned. The RTOS itself requires 60K of memory, which is somewhat
more than some of the alternatives so a value of 0.7 is assigned. The system provides only one
form of scheduling, preemptive priority, so a lower value, 0.25, is assigned here than if other
forms such as round-robin were available. Intertask synchronization and communication is
available only through message passing so a relatively low m5  0.5 is assigned.
The company provides paid phone support, which is not as “generous” as other companies, so
a value of m10  0.5 is assigned. There is a royalty cost for each unit so a zero was assigned.
Finally, there is a wide range of software support for the product, including network protocols,
and the source is available so values of one are given for these three criteria.
Using the same kind of reasoning, ratings tables are obtained for commercial RTOS B, C, D
and E, shown in tables 2 through 5 respectively.
8
Table 2: Summary data for commercial RTOS B.
Criterion Description
Rating
Comment
m1
Minimum Interrupt Latency
0.8
Less then 15 us (Typical)
m2
Number of Tasks Supported
*
CPU Dependent
m3
Memory Requirements
0.2
Less then 4 MB
m4
m5
Scheduling Mechanism
Intertask Synchronization Mechanism
0.5
1
m6
Context Switch Time
*
Preemptive and round-robin
Mutual exclusion, POSIX pipes, message queues,
shared memory, signals, sockets, fifos
Dependent on CPU
m7
Programming Environment
0.75
GNU Development Tools (gcc)
m8
Portability
0.5
X86, PowerPC, MIPS, Alpha
m9
Source Code Availability
1
Open source available
m10
Software Support (Warranty)
0.5
Support available for additional cost
m11
m12
m13
Cost
Royalty Fee
Networking Support
0.5
1
1.0
$5000
Yes
Full TCP/IP Suite
Table 3: Summary data for commercial RTOS C.
Criterion
m1
Description
Minimum Interrupt Latency
Rating
1
m2
m3
Number of Tasks Supported
Memory Requirements
*
*
m4
m5
Scheduling Mechanism
Intertask Synchronization Mechanism
0.1
0.5
m6
Context Switch Time
0.5
m7
Programming Environment
m8
Portability
m9
Source Code Availability
0
Source not available
m10
Software Support (Warranty)
1
Full support available
m11
m12
m13
Cost
Royalty Fee
Networking Support
0.2
1
1
$20000
Yes
All POSIX Compliant
1
0.2
9
Comment
8 microsecond
Multiple cooperating tasks
POSIX Memory mapping, memory locking
Preemptive task scheduling
Inter-task communication through shared memory
space
“fast”
POSIX compliant software supported
PowerPC
Table 4: Summary data for Commercial RTOS D.
Criterion Description
Rating
m1
Minimum Interrupt Latency *
Comment
CPU Dependent
m2
m3
Number of Tasks Supported 1
Memory Requirements
1
“Unlimited “
2K memory required
m4
m5
Scheduling Mechanism
Intertask Synchronization
Mechanism
1
1
m6
Context Switch Time
0.5
Preemptive, round-robin and time slicing
Binary, counting, and mutual exclusion semaphores with
priority inheritance, message queues, POSIX pipes, counting
semaphores, signals and scheduling, control sockets and
shared memory
14 micro second switch time
m7
Programming Environment
1
m8
Portability
1
m9
Source Code Availability
0.25
Tornado II, more then 1800 APIs available, powerful file
system and I/O management, C++ and other standard runtime support
x86, PowerPC, ARM, MIPS, 68K, CPU 32, ColdFire,
MCORE, Pentium,i960,SH, SPARC, NEC V8xx, M32 R/D,
RAD6000, ST 20, TriCore
Source available for $120,000
m10
Software Support
(Warranty)
Cost
Royalty Fee
Networking Support
0.8
Phone Tech support available
0.1
1
1
$23,000
Yes
Full TCP/IP Suite
m11
m12
m13
10
Table 5: Summary data for commercial RTOS E.
Criterion
m1
Description
Rating
Minimum Interrupt Latency
1
Comment
6 microseconds
m2
Number of Tasks Supported
32,000
m3
Memory Requirements
0.9
m4
Scheduling Mechanism
0.25
m5
Inter-task Synchronization
Mechanism
1
m6
Context Switch Time
*
m7
Programming Environment
0.5
m8
Portability
0.2
m9
Source Code Availability
m10
Software Support
(Warranty)
Cost
Royalty Fee
Networking Support
m11
m12
m13
1
1
1
kernel = 6.2Kb
file management = + 32Kb
peripherals = + 200Kb
Prioritized, preemptive scheduling.
Equal priority tasks are scheduled on a time-sliced, roundrobin basis
Semaphore event flags, queuing libraries, binary
semaphores, timing semaphores, physical semaphores, and
counting semaphores.
Dependent on Processor
Editor, disk utilities, file utilities, and a virtual port that
allows the user to run, monitor, and switch between
multiple task output screens.
68k architecture family
Yes
Full support
0.7
1
0.6
$2,880 – 5,850
Yes
Provided by BSDnetTM, which provides Ethernet, and
TCP/IP support, telnet, and ftp.
Admittedly, these assignments were somewhat subjective. Numerous arguments can be made
that a higher or lower number should be assigned in almost every case. But, the idea is that there
is some consistency in how the ratings are assigned so that like attributes can be compared
objectively. This represents the power of this methodology over heuristic or even emotionally
based approaches.
4. Matching the Application to the Operating System Using the Criteria
In order to demonstrate the use of the criteria and associated metric, a set of five hypothetical
systems are analyzed to demonstrate how comparative metrics can be used to match an application
to the appropriate real-time operating system. These systems are loosely described as





a hand-held game application,
a controller application for the fuel injection system of typical passenger car,
an animatronic robot used in the filming of a science fiction movie,
a navigation system for a fighter aircraft, and
a medical device that reduces the time needed for a medical imaging scan.
11
These systems were selected because they are representative of a large band of the real-time
embedded application spectrum. Moreover, pedestrian familiarity with the basics of what these
systems do, avert the need for complex systems requirements specifications.
To do the comparison, a tabular format shown in tables 6 through 10 is used. All uncertain
mi are set to 0.5 where information is still unavailable for the purposes of evaluating the fitness
metric M . This is not the ideal manner for dealing with uncertain information [14], but
consideration of this aspect is left for further study.
The result is a set of M values that reflect the weighted importance of each criterion for a
given application, and the known information about the candidate operating system. A
quantitative selection of the appropriate RTOS can then be made.
In the forgoing discussions, tables 6 through 10 were obtained by setting up a set of spreadsheets
in which the weighting factors are input in appropriately named column. Then the objective
metric component values are input for the various candidate operating systems. In this way, metric
M can be computed rapidly and correctly, and the weighting factors can be adjusted as necessary
for sensitivity analysis.
4.1 Hand-held game application
For a hand-held game application similar any criterion related to cost to produce and performance
are critical. Hence m1 , m3 , m5 , m9 , m11 , m12 are given unity weights, as shown in table 6. The number
of tasks in such a system is likely to be relatively low (e.g. one task per actuator, one for video
refresh, one for sound), therefore, m2  0.5 . The scheduling mechanism is not as important because
this application is likely going to be a preemptive priority system, which is supported by every
RTOS. Therefore m4  0.1 .
12
Table 6: Decision Table for Hand-held Game Application
Criterion Description
Weight, wi
1
m1
Minimum
Interrupt
Latency
0.5
m2
Total Number of
Tasks Supported
m3
m4
m5
m6
m7
m8
m9
m10
m11
m12
m13
M
Memory
Requirements
Scheduling
Mechanism
Intertask
Synchronization
Mechanism
Context Switch
Time
Programming
Environment
Portability
Source Code
Availability
Software
Support
(Warranty)
Cost
Royalty Fee
Networking
Support
A
0.5
B
0.8
C
1
D
0.5
E
1
0.5
0.5
0.5
1
1
1
0.7
0.2
0.5
1
0.9
0.1
0.25
0.5
0.25
1
0.25
1
0.5
1
0.5
1
1
1
0.5
0.5
0.5
1
0.5
0.5
1
0.75
1
1
0.5
0.1
0.5
0.8
1
0.5
1
0.2
0
1
0.4
0.2
1
0.5
0.5
0.5
1
0.8
1
1
1
0.2
0.5
0
1
0.5
1
1
0.1
1
1
0.1
1
1
0.7
1
0.6
4.51
5.68
5.10
6.60
7.02
Since the application will run on a special purpose device, portability, and network support are
of low importance, and m8 , m13 reflect this. Finally, software support and source availability are
relatively mild since this application is not as sensitive to the cost of the source as it is to the per
unit royalty cost (because of anticipated production volume). Hence, m9 m10 m12  0.5
Applying the weighted criteria to the five candidate systems yields the decision matrix shown
in table 6. From this table it is clear that RTOS E emerges as the choice for this application.
4.2 Controller application for the fuel injection system
13
A fuel injection system controls the fuel/air mix in an internal combustion engine and reacts to
events to promote correct and efficient operation. The main processor, or Powertrain Control
Module (PCM) reads various input signals and controls the fuel and emissions systems to try to
obtain the best performance while maintaining good mileage and emissions output [14]. Each of
these inputs and outputs is critical to the overall operation of the entire system.
The RTOS should be able to manage multiple processes quickly and reliably, therefore high
weights were assigned to m1 , m3 , m4 , m5 . The operating system must have a low unit cost because of
anticipated production volume, and the source code must be available for fine-tuning.
Therefore, m9  m12  1 . Processor compatibility is not critical in this highly embedded system,
there is potential for using a standard bus for sharing data. Hence m8  0.1 and m13  0.5
The other criteria ratings follow the same logic as in the previous example. A summary of the
ratings is given in table 7, along with the associated decision matrix.
Table 7: Decision Matrix for Fuel-Injection System.
Criterion Description
Weight, wi
1
m1
Minimum
Interrupt Latency
0.1
m2
Total Number of
Tasks Supported
1
m3
Memory
Requirements
0.75
m4
Scheduling
Mechanism
0.5
m5
Intertask
Synchronization
Mechanism
1
m6
Context Switch
Time
0.5
m7
Programming
Environment
0.1
m8
Portability
Source Code
1
m9
Availability
0.5
m10
Software Support
(Warranty)
1
m11
Cost
1
m12
Royalty Fee
0.5
m13
Networking
Support
M
A
0.5
B
0.8
C
1
D
0.5
E
1
0.5
0.5
0.5
1
1
0.7
0.2
0.5
1
0.9
0.25
0.5
0.25
1
0.25
0.5
1
0.5
1
1
0.5
0.5
0.5
1
0.5
1
0.75
1
1
0.5
0.8
1
0.5
1
0.2
0
1
0.4
0.2
1
0.5
0.5
1
0.8
1
0.5
0
1
0.5
1
1
0.1
1
1
0.1
1
1
0.7
1
0.6
5.02
6.10
5.11
6.85
6.96
The metric suggests that RTOS E is again a good fit.
4.3 An animatronic robot used in the filming of Star Wars
14
These robots are complex electromechanical systems, which require sophisticated control
software, and at the same time must be easily programmable by special effects creators. A typical
robot is likely to include numerous limbs with multiple degrees of freedom. These limbs will be
controlled by a variety of actuators. Consequently there is a need for a scalable embedded system
to control such complex machine in a predictable and reliable fashion. The decision matrix is
given in table 8.
Table 8: Decision Matrix for Animatronic Robot.
Criterion Description
m1
Minimum
Interrupt Latency
m2
Total Number of
Tasks Supported
m3
Memory
Requirements
m4
Scheduling
Mechanism
m5
Intertask
Synchronization
Mechanism
m6
Context Switch
Time
m7
Programming
Environment
m8
Portability
Source Code
m9
Availability
m10
Software Support
(Warranty)
m11
Cost
m12
Royalty Fee
m13
Networking
Support
M
Weight, wi
1
A
0.5
B
0.8
C
1
D
0.5
E
1
1
0.5
0.5
0.5
1
1
0.5
0.7
0.2
0.5
1
0.9
0.5
0.25
0.5
0.25
1
0.25
0.5
0.5
1
0.5
1
1
0.1
0.5
0.5
0.5
1
0.5
0.5
1
0.75
1
1
0.5
0.5
0.5
0.8
1
0.5
1
0.2
0
1
0.4
0.2
1
0.5
0.5
0.5
1
0.8
1
1
0.1
0.1
0.5
0
1
0.5
1
1
0.1
1
1
0.1
1
1
0.7
1
0.6
4.03
4.28
3.58
5.00
5.34
The criteria ratings, and rationale for the robot are similar to those for the fuel-injection system
with a few exceptions. The likely number of tasks to be supported is likely high due to the many
articulated joints that need to be controlled, therefore m2  1 . Memory requirements are not as
critical, because it is likely that a robot can more easily contain a larger memory module than the
fuel injection system, so, m3  0.5 . The scheduling mechanism is not quite as important in the
robot, in that all tasks are likely to be preemptive priority, while in the fuel-injection system there
is possibly going to be additional scheduling flexibility needed. Finally, the hardware
compatibility is slightly more critical for the robot because of the possibility of reuse of existing
components from other robots and the source code is somewhat less important because the
mission criticality is lower. Therefore, m8  0.5 . The decision metric again suggests RTOS E.
15
4.4 A fighter jet inertial navigation system
The software controlling a navigation system requires substantial input/output processing which
inherently causes large amounts of system interrupts. This is a highly reactive, and mission
critical system that requires fast context switching, minimal interrupt latency, a high degree of
synchronization and a well-supported and reliable system. Therefore
m1  m2  m5  m9  m10  m11  m12  1 . Hardware compatibility is not critical because there is little
need to port the system and the number of tasks supported is relatively low, therefore m8  0.2 .
The other criteria are set to 0.4 or 0.5 because they are only moderately important. The ratings
assigned are summarized in Table 9.
Table 9: Decision Matrix for Navigation System.
m1
Minimum
Interrupt
Latency
m2
Total Number
of Tasks
Supported
m3
Memory
Requirements
Scheduling
Mechanism
Intertask
Synchronization
Mechanism
m4
m5
m6
m7
m8
m9
Context Switch
Time
Programming
Environment
Portability
m10
Source Code
Availability
Software
Support
(Warranty)
m11
Cost
m12
Royalty Fee
m13
Networking
Support
M
1
0.5
0.8
1
0.5
1
0.1
0.5
0.5
0.5
1
1
1
0.7
0.2
0.5
1
0.9
0.5
0.25
0.5
0.25
1
0.25
1
0.5
1
0.5
1
1
1
0.5
0.5
0.5
1
0.5
1
1
0.75
1
1
0.5
0.1
0.8
0.5
0.2
1
0.2
1
1
1
0
0.4
1
1
0.5
0.5
1
0.8
1
0.4
0.5
0.5
0.1
0.1
0.7
0
1
1
1
1
1
1
1
1
0.6
5.66
5.80
5.24
6.94
6.73
0.5
The metric suggests that RTOS D is the best match for this system.
16
4.5 A medical device that reduces the time needed for an MRI scan
Now consider a real-time operating system to be implemented within a device used to reduce the
time needed for a medical imaging scan, such as an MRI. The following discussion summarizes
the ratings shown in Table 10.
Table 10: Decision Table for Medical Imaging System.
m1
m2
m3
m4
m5
m6
m7
m8
m9
m10
m11
m12
m13
M
Minimum
Interrupt
Latency
Total Number of
Tasks Supported
Memory
Requirements
Scheduling
Mechanism
Intertask
Synchronization
Mechanism
Context Switch
Time
Programming
Environment
Portability
Source Code
Availability
Software Support
(Warranty)
0.5
0.5
0.8
1
0.5
1
1
0.5
0.5
0.5
1
1
0
0.7
0.2
0.5
1
0.9
0.5
0.25
0.5
0.25
1
0.25
1
0.5
1
0.5
1
1
0.5
0.5
0.5
0.5
1
0.5
1
1
0.75
1
1
0.5
0.1
1
0.8
1
0.5
1
0.2
0
1
0.4
0.2
1
1
0.5
0.5
1
0.8
1
Cost
Royalty Fee
Networking
Support
0.1
0
1
0.5
0
1
0.5
1
1
0.1
1
1
0.1
1
1
0.7
1
0.6
5.26
0
5.75
4.91
6.56
6.07
Such a system should be relatively fast, to reduce patient exposure to possibly harmful
radiation. Since the system is likely to be large and expensive, there are no constraints on the
amount of memory used and cost. High reliability and manufacturer support for the software is
likely to be at a premium, and because the system is likely to be part of a networked environment,
good networking support is of high value.
Based on the setting the criterion accordingly, the metric again suggests that RTOS D is a good
candidate for this application.
4.6 Summary of Case Studies and Sensitivity Analysis
Table 11 summarizes the RTOS recommendations rendered by metric analysis.
17
Table 11: Applications and metrics derived from decision matrices. Choices are shaded.
Application
A
B
C
D
E
Fuel injection system
4.51
5.68
5.10
6.60
7.02
Hand-held game
5.02
6.10
5.11
6.85
6.96
Fighter navigation system
4.03
4.28
3.58
5.00
5.34
Animatronic robot
5.66
5.80
5.24
6.94
6.73
Medical imaging device
5.26
5.75
4.91
6.56
6.07
It is important to note that these conclusions do not represent an endorsement or criticism of
any particular commercial solution. Indeed, the conclusions are sensitive to changes in any
number of factors. This reflected in the weightings choices – the metric is very sensitive to
changes in the designers’ priorities as reflected by these weightings.
For example, in the fuel injection system, changing the scheduling mechanism weighting from
0.75 to 1.0 would have caused the selection of RTOS D instead of RTOS E. Similarly, if the
unknown value, “*” were given a value of 0 instead of 0.5, the outcome of the analysis would be
quite different.
In any case, the power of the metric is that it can be used to reflect the needs, tastes and desires
of its users. One might argue with the ratings assigned, or the weights, but the methodology and
metric are flexible enough to accommodate these differences of opinion. Moreover, the designer
is forced to justify and document his or her choices, which will benefit the system throughout its
life cycle.
5. Summary and Conclusions
In this paper a set of subjective criteria was introduced to be used in selecting commercial RTOS
to be used to match to a specific application. The formulation of a goodness-of-fit metric based
on these criteria and subjective/quantitative data provided by commercial RTOS manufactures
was discussed. Finally, a flexible heuristic approach using this metric that can be fine tuned
depending on the users needs was presented. The advantage of this metric is that it can be used
before the system has been built, in an environment where the experience base is limited, and can
be used in conjunction with other information from technical and experience reports.
This approach was applied to five examples to show how this might work in the real world.
18
Suggested future work includes studying the sensitivity of the metric and examining the effect
of uncertainty on the decision making and modifying the metric to handle uncertain or unavailable
information using the extended fuzzy pointing set [14].
6. References
[1] G. Buttazzo, Hard Real-Time Computing Systems: Predictable Scheduling Algorithms and
Applications, Kluwer Academic Publishers, 2000.
[2] M. Botlo, J. Zalewski, “An Evaluation of VxWorks Version 5.0,” CERN Mini & Micro
Computer Newsletter, No. 32, pp. 15-18, October 1991.
[3] Dedicated Systems, “Comparison between VXWorks/X86 5.3.1, QNZ 4.25 and PSOS
SYSTEM/X86 2.2.5”, Technical Report, June 11, 2001, accessed from www.DedicatedSystems.com.
[4] K. Low et al., “Overview of Real-Time Kernels at the Superconducting Super Collider
Laboratory,” Report SSCL-397, Dallas, TX, May 1991.
[5] M. Timmerman, “RTOS Evaluations Kick Off.” Real-Time Magazine, pp. 6-10, Issue 98-3,
1998.
[6] J. Zalewski, “An Independent Comparison of Three Real-Time Kernel Standards,” IEEE
Posix 1003.4 Working Group Meeting, New Orleans, LA, January 7-11, 1991, Paper No.
P1003.4-N0289.
[7] P. A. Laplante, Real-Time Systems Design and Analysis: An Engineer's Handbook, Third
Edition, John Wiley/IEEE Press, to appear 2003.
[8] J. J. Labrosse, MicroC/OS-II: The Real-Time Kernel, 2nd Edition, CMP Books, 2002.
[9] J. K. Muppala, Steven Woolet, Kishor S. Trivedi, “Real-Time Systems Performance in the
Presence of Failures,” IEEE Computer, May 1991, pp. 37-47.
[10] J. A. Stankovic, Marco Spuri, Marco DiNatale and Giorgio Buttazzo, “Implications of
Classical Scheduling Results for Real-Time Systems,” IEEE Computer, June 1995, pp. 16-25.
[11] IEEE, POSIX Standard, http://standards.ieee.org/reading/ieee/std_public/description/posix/,
last accessed 8/5/02.
[12] B. O. Gallmeister, Posix. 4: Programming for the Real World, O’Reilly & Associates, 1995.
[13] B. Furht, et al, Real-Time Unix Systems: Design and Application Guide, Kluwer Academic
Publishers, 1991.
19
[14] P. A. Laplante and D. Sinha, “Handling Uncertain Data in Imaging Systems," IEEE
Transactions on Systems, Man and Cybernetics, Volume 26, No. 1, February 1996, pp. 21-28.
[15] S. Raman, N. Sivashankar, W. Milam, W. Stuart and S. Nabi,” Design and implementation of
HIL simulators for powertrain control system software development,“ Proceedings of the 1999
American Control Conference, Volume: 1 , 1999 Page(s): 709 -713 .
20