Project Report A concise original critique paper comparing Chorus Operating System with Unix Operating System. Vani R Doraisamy 999-29-4889 CS-550 Fall 2001 Main Campus Project Report CS-550 Fall 2001 Table of Contents 1. Introduction -----------------------------------------------------------------3 2. Design principle -----------------------------------------------------------4 3. Programmer Interface -----------------------------------------------------5 4. User interface ---------------------------------------------------------------6 5. Process management ------------------------------------------------------7 6. Memory management ------------------------------------------------------7 7. File system -------------------------------------------------------------------8 8. Inter-process Communication --------------------------------------------10 9. Protection & security ------------------------------------------------------11 10. Miscellaneous topics ------------------------------------------------------12 11. Conclusions & future directions -----------------------------------------13 Appendix: Bibliography Vani R Doraisamy 2 Project Report CS-550 Fall 2001 Introduction Older operating systems are monolithic, that is, the whole operating system is a single a.out file that runs in kernel mode. This binary contains the process management, memory management, file system and the rest. Examples of such systems are UNIX, MS-DOS, VMS, MVS, OS/360 and MULTICS. UNIX was originally developed in a laboratory at AT&T’s Bell Labs (now an independent corporation known as Lucent Technologies). Unix is a mature, technically superior group of operating systems with almost thirty years of continual development, performed often by volunteers who believe in what they’re doing, has produced a group of operating systems and extremely powerful multiprocessor server hardware tailor-made to its needs, whose performance is still unparalleled by Intel hardware. The alternative is a micro kernel-based system, in which most of the OS run as separate processes, mostly outside the kernel. They communicate by message passing. The kernel's job is to handle the message passing, interrupt handling, low-level process management, and possibly the I/O. Examples of this design are the Chorus, Mach, RC4000 and Amoeba Chorus was a research project on distributed systems at INIRA in France. It was bought by Sun Microsystems in 1997. The Chorus operating system is a highly scalable and reliable embedded operating system that has established itself among top telecom suppliers. The Chorus operating system is used in public switches and PBXs, as well as within access networks, cross-connect switches, voice-mail systems, cellular base stations, web-phones and cellular telephones. An open and flexible solution, the Chorus operating system also allows developers to rapidly respond to customer needs and market conditions by quickly and cost-effectively creating and deploying new services and mission-critical applications. With enhanced networking features, the Chorus operating system seamlessly supports third-party protocol stacks, legacy applications, real-time and Java technology-based applications simultaneously on a single hardware platform. In this paper, I discuss the relative merits and demerits of the two operating systems. A sub-section on Analysis and Evaluation of operating systems in comparison is provided at the end of each chapter. Vani R Doraisamy 3 Project Report CS-550 Fall 2001 Design Principle A Chorus system is composed of a small nucleus and a set of system servers, which cooperate in the context of subsystems. Chorus Nucleus is not the core of a specific operating system; rather it provides generic tools designed to support a variety of host subsystems, which can coexist on top of the Nucleus. This structure supports application programs, which already run on an existing operating system, by reproducing the operating system’s interface within a subsystem. This classic idea of separating the functions of the operating system into groups of services provided by autonomous servers is central to the Chorus philosophy. In monolithic systems, these functions are usually part of the “kernel”. This separation of functions increases modularity, and therefore the portability of the overall system. Unix is a layered operating system. The innermost layer is the hardware that provides the services for the OS. The operating system, referred to in Unix as the kernel, interacts directly with the hardware and provides the services to the user programs. These user programs don't need to know anything about the hardware. They just need to know how to interact with the kernel and it's up to the kernel to provide the desired service. One of the big appeals of Unix to programmers has been that most well written user programs are independent of the underlying hardware, making them readily portable to new systems. User programs interact with the kernel through a set of standard system calls. These system calls request services to be provided by the kernel. Such services would include accessing a file: open close, read, write, link, or execute a file; starting or updating accounting records; changing ownership of a file or directory; changing to a new directory; creating, suspending, or killing a process; enabling access to hardware devices; and setting limits on system resources. Unix is a multi-user, multi-tasking operating system. You can have many users logged into a system simultaneously, each running many programs. It's the kernel's job to keep each process and user separate and to regulate access to system hardware, including cpu, memory, disk and other I/O devices. Vani R Doraisamy 4 Project Report CS-550 Fall 2001 Analysis and Evaluation: Unix was designed to be a time-sharing system. It has a simple standard user interface (shell) that can be replaced. The file system has multilevel tree-structured directories. The kernel supports files as unstructured sequences of bytes. High priority was given to making the system interactive, and providing facilities for program development. On the other hand Chorus was designed to improve process control, network communication and emulate different Unix operating systems. Chorus was designed to use the capabilities of distributed systems while focusing on real-time applications and integrating OOP into code. Programmer Interface Analysis and Evaluation: Many proprietary operating systems have a simplified view of application behavior. The typical application reads some data from disk, tape or a terminal and does some processing. Output is produced onto disk, tape, terminal, or printer. The operating systems generally provide easy to use well-implemented facilities to support these types of facilities. System Calls System calls define the programmer interface to UNIX The set of systems programs commonly available defines the user interface. The programmer and user interface define the context that the kernel must support. Roughly three categories of system calls in UNIX. o File manipulation (same system calls also support device manipulation) o Process control o Information manipulation When UNIX was distributed, users could write applications in C and easily make use of all of the operating system facilities. This allowed application developers to quickly develop much more sophisticated applications using these facilities. The pattern of development in UNIX when adding new features such as networking is to provide an application program interface from the C language to access the new features. Vani R Doraisamy 5 Project Report CS-550 Fall 2001 In general UNIX system developers and application developers program in the same language using the same application-programming interface. In typical proprietary operating systems, the operating systems programmers are programming in assembly language and have access to a many capabilities, which are not available to the application developer. The application program interface allows many different types of applications to be easily implemented under UNIX without writing assembly language. These applications are relatively portable across multiple vendor hardware platforms. Third party software vendors can save costs by supporting a single UNIX version of their software rather than four completely different vendor specific versions requiring four times the maintenance. The UNIX editor is called 'vi' which stands for visual editing. Most people find vi a very difficult editor to learn compared to many of the microcomputer based editors today. Programmer Interface for Chorus is similar to that of Unix. User Interface Analysis and Evaluation: Programmers and users mainly deal with already existing systems programs; the needed system calls are embedded within the program and do not need to be obvious to the user. The most common systems programs are file or directory oriented. Directory:mkdir,rmdir,cd,pwd File:ls,cp,mv,rm Other programs relate to editors (e.g.,emacs,vi), text formatters (e.g., troff, TEX), and other activities. Shell – the user process, which executes programs (also called command interpreter). Called a shell, because it surrounds the kernel. The shell indicates its readiness to accept another command by typing a prompt, and the user types a command on a single line. A typical command is an executable binary object file. The shell travel through the search path to find the command file, which is then loaded and executed. The directories /bin and /usr/bin are almost always in the search path. Typical search path on a BSD system: (./home/prof/avi/bin /usr/local/bin /usr/ucb /bin /usr/bin ) So far, there has been no mention of the user interface for UNIX. UNIX is a good operating system for experienced programmers. The operating system was designed and implemented by experienced programmers so everything, which the experienced programmer needs, is present but not much else. A perfect example of this is the on-line documentation called "man-pages" or manual pages. The material is completely reference oriented with very little tutorial information. Experienced programmers find the man pages very useful but the beginning user often finds them overwhelming. In the last few years, there has been extensive work to improve the user interface to UNIX. The most dramatic effort has been the addition of windowing interfaces on top of UNIX such as X-windows, Suntools, NextStep, Motif, OpenLook, etc. These windowing interfaces do not change UNIX itself but are built on top of UNIX to provide a more intuitive interface to UNIX. Each of the different user interfaces has some advantages and some disadvantages. Currently intensive development effort is being done on all of these Graphical User Interfaces (GUIs). The User Interface for Chorus is similar to that of Unix. Vani R Doraisamy 6 Project Report CS-550 Fall 2001 Process Management A program in execution is called process in UNIX environment. Under UNIX system any number of process can exist for a given program. Every process running under UNIX is identified by a process- id (pid), an integer value. Each process is identified with the owner of the process, simply called uid. Processes are arranged in a tree structure link. So except the root, all other process is attached with its parent. Unix is a timesharing system, which means that the processes take turns running. Each turn is a called a timeslice; on most systems this is set at much less than one second. The reason this turns-taking approach is used is fairness: We don't want a 2-second job to have to wait for a 5-hour job to finish, which is what would happen if a job had the CPU to itself until it completed. A process in Chorus defines a protected address space, which encapsulates the following resources: a set of threads that share the resources of the process, a virtual memory context, and a set of ports for communication with other processes. Every process has a protection identifier associated with it. If the process forks, the children inherit the same ID. Protection identifiers provide a mechanism for authentication. There are three kinds of processes in Chorus, each having different execution privileges. Supervisor processes execute in the same address space as the micro kernel and are permitted to directly execute kernel instructions. They may also execute privileged I/O instructions. System processes are permitted to execute kernel operations but may not execute privileged I/O instructions. Unlike supervisor processes, system processes execute in a private address space. User processes may execute neither kernel operations nor privileged I/O instructions. They run in a private address space. Many threads can execute concurrently within a process. Each thread is characterized by the state of the processor. The scheduling scheme is very flexible: although the basic scheme is priority-based, Chorus also supports time-slicing and priority degradation on a per-thread basis. Analysis and Evaluation: A process in Chorus with one thread is like a standard Unix process. Although Chorus can support multiple simultaneous processes, it is not possible to migrate a process and its threads to another site on a distributed system. Chorus supports Unix like fork() and exec() system calls for creating new processes. Threads are synchronized using mutexes, semaphores, or spin locks. Memory Management Abstract memory objects are typically provided to protect regions of memory. Virtual memory management may be implemented as part of the kernel or user-level process. The Unit of data abstraction in Chorus is called the segment. Segments generally represent some form of secondary storage such as a file. Similarly to other abstractions in Chorus, segments are global and are identified by capabilities. Each process address space is divided into regions. A region is a contiguous range of virtual addresses within a process, which maps a portion of a segment to a given virtual address. Associated with each mapping is a set of access rights. System processes known as mappers are responsible for mapping segments onto regions. If a process makes a request to read or modify data within a region, the mapper returns the appropriate segment Vani R Doraisamy 7 Project Report CS-550 Fall 2001 containing the data. Segments are swapped on a demand basis by a user level process called the External Mapper. UNIX Memory Management All memory assigned as pages Each user process works in a virtual 4GB memory space Multi-User nature of UNIX requires that no memory be allocated to absolute locations Unix memory management is based on Demand-paged virtual memory system. It employs the Least Recently Used algorithm. Each process has a separate virtual address space. This virtual memory is implemented via demand paging. Pages in physical memory are selected for replacement using a modified working-set policy. A page that has not been referenced in a certain length of time is considered no longer to be in the working set of any process; thus it becomes a candidate for replacement. It is possible that the working sets of the processes being run might require more pages than can fit into memory. If this happens, one or more of the processes is temporarily suspended. Pages being used by these processes are removed form memory to make more room available. Analysis and Evaluation: Unix operating system use some form of complex memory management algorithm, which makes use of dynamic binding in the form of virtual addresses. This requires that the translation from virtual to physical address must occur at least once per instruction. If this translation were to be done using software it would be the system very, very slow. This means that address translation must be done in hardware. The Chorus operating system has a separate portable model for virtual memory management in the Nucleus. There are three memory management models. The model used is determined by the settings of the virtual address space and on demand paging features. Protected memory management model is targeted at real-time applications able to benefit from the edibility and protection offered by memory management units, address translation and separate address spaces. No swap or demand paging is provided. No mapper interface is provided. Application-specific file servers must do accesses to programs and data stored on secondary devices. Virtual memory management model supports full virtual memory with swapping in and out on secondary devices. It has been specifically designed to implement distributed UNIX subsystems on top of the kernel. The Unit of data abstraction in Chorus is called the segment while it is called a page in Unix. File System A file system is a logical method for organizing and storing large amounts of information in a way that makes it easy manage. The file is the smallest unit in which information is stored. The UNIX file system has several important features. Types of Files in Unix: To the user, it appears as though there is only one type of file in UNIX, the file that is used to hold your information. In fact, the UNIX file system contains several types of file. Ordinary files: This type of file is used to store your information, such as some text you have written or an image you have drawn. This is the type of file that you usually work with. Directories: A directory is a file that holds other files and other directories. You can create directories in your home directory to hold files and other sub-directories. Vani R Doraisamy 8 Project Report CS-550 Fall 2001 Special files: This type of file is used to represent a real physical device such as a printer, tape drive or terminal. It may seem unusual to think of a physical device as a file, but it allows you to send the output of a command to a device in the same way that you send it to a file. Pipes: UNIX allows you to link commands together using a pipe. The pipe acts a temporary file that only exists to hold data from one command until another reads it. Structure of the file system: The UNIX file system is organized as a hierarchy of directories starting from a single directory called root which is represented by a / (slash). Imagine it as being similar to the root system of a plant or as an inverted tree structure. Immediately below the root directory are several system directories that contain information required by the operating system. The file holding the UNIX kernel is also here. /(root) | -------------------------------------------------------------| | | | | | | | /bin /dev /etc /home /lib /tmp /usr kernel file Chorus Operating System supports the following file systems: Unix File System (UFS) The UNIX file system option provides support for a disk-based file system, that is, the file system resides on physical media such as hard disks. The UNIX file system option supports drivers for the following types of physical media: o o o SCSI disks IDE disks RAM disks MS DOS file system Network File System (NFS) The NFS permits transparent access of remote files on most Unix systems and many non-Unix systems, including Microsoft Windows. This facility can be used, for instance, to dynamically load applications from the host on the target. It Includes: NFS Client, which gives access to remote files from Chorus. NFS Server, which allows to mount local file systems from a remote system. Applications have access to The standard POSIX.1 file I/O interface. The ANSI-C stdio(3) The BSD file I/O The C++ iostream. Vani R Doraisamy 9 Project Report CS-550 Fall 2001 Analysis and Evaluation: The file system in UNIX is clearly the simplest and easiest file system to use today. The UNIX file system is one of the simplest tree structured file systems to use. There are no syntax extensions for accessing system files, files belonging to other users, or files on network mounted file systems. The file system is one place where UNIX has a friendlier user interface than most systems. The Chorus file system is very familiar to the Unix file system but it provides a file manager. The file manager acts as a mapper for the Chorus kernel and also performs Unix file system management. The Chorus file system has the standard Unix code, code to support Chorus segment mapper, and codes to catch hardware interrupts and map them onto hardware device drivers. The Original Unix design was not well suited to separation into a number of components and so the file manager also contains code that simulates non-file related aspects of Unix kernel and provides this information to the file system components to trick them into believing they are part of the single monolithic kernel. The original Unix file system code is bundled up into a Chorus actor, and executed unchanged in conjunction with a set of library routines developed by Chorus Systems that simulate the usual environment within which the file system code operates. Inter-process Communication IPC is a set of programming interfaces that allow you to create and manage individual program processes that can run concurrently in an operating system. This allows a program to handle many user requests at the same time. The various IPC methods are: Pipes and named pipes Message queuing Semaphores Shared memory Sockets RPC IPC is provided for the following reasons: In-order delivery of data - so that the receiving process reads messages in the order that they are sent. Unduplicated delivery of data - so that the receiving process only gets each message once. Reliable delivery of data - so that no data is lost. Preservation of message boundaries - so that individual messages are not mixed up into one. Support for out-of band messages - so that "emergency" situations can be handled on the same channel. Connection oriented communication - so that a connection can be established between two independent processes before any other communication takes place. Communication services provided by Chorus are: Asynchronous message passing. Synchronous RPC. Asynchronous Message-passing does not guarantee delivery of message. During asynchronous message transfer no notification is sent to the sender. Asynchronous message passing reduces network traffic. It has higher performance and provides better scaling to large or busy networks. Synchronous RPC is based on the client-server model. It guarantees delivery of message by sending notification to the sender. It is a simple concept, which is easy to understand. It is easy to handle in case of errors or crashes. Vani R Doraisamy 10 Project Report CS-550 Fall 2001 Processes in Unix based systems communicate between themselves through IPC constructs. Among these are: Pipes FIFO (named pipes) Streams (Files) Sockets Messages: Exchange messages with any process or server. Semaphores: Allow unrelated processes to synchronize execution. Shared memory: Allow unrelated processes to share memory. Analysis and Evaluation: In Chorus IPC allows applications to be distributed across multiple machines. It allows applications to run in a heterogeneous environment that comprises hardware and software with stark operational and programming incompatibilities. IPC recognizes whether a given process is available locally, or is installed on a remote system. When a process is accessed, the IPC identifies the shortest path and quickest execution time that can be used to reach it, and communicates in a manner that makes the location entirely transparent to the application. The Unix IPC routines messages, shared memory and semaphores are not implemented in Chorus. Chorus messages make the exchange of information explicit. Messages in Chorus provide well-defined points for isolating the state. Messages are easier to manage than shared memory. Using messages, the grain of information exchange is bigger, is better defined and its costs can be more accurately calculated. It increases in scale more easily and can be applied more efficiently in loosely coupled environments. In Unix, whereas an elementary form of naming service (binding between service names and process ports) is provided, the binding scope is only local to the machine. In Chorus, port names are global but when a machine failure occurs the localization service broadcasts a request in order to find the new locations of the ports. Because this mechanism is too expensive, we need a concept of reliable ports. TCP is the reliable and ordered communication service in both Unix and Chorus. In Unix, there are no group functionality i.e. there are only broadcast messages. The Chorus micro-kernel offers group communication primitives with several addressing mode through the concept of port groups. Protection and Security Security and Protection is provided for the following reasons: Secrecy: Information is accessible for reading by authorized parties only. Integrity: Information is accessible for modification by authorized parties only. Availability: Computer system assets are available to authorized parties only. Types of security threats are: Interruption Threat to Availability Interception Threat to Secrecy Modification Threat to Integrity Fabrication Threat to Integrity Vani R Doraisamy 11 Project Report CS-550 Fall 2001 Security in Chorus is offered in the application level. There is no security in the Chorus nucleus. Chorus has provisions for the development of secure distributed systems through support for access control and authentication. Security is offered in Chorus by the concept of unique identifiers. Knowledge of Unique Identifier gives full privileges to manipulate the object. The types of security provided in Unix are: Physical Security Console Security Data Security Users practice secure measures NO welcome banner on site Network Security Filtering Prevent Spoofing FTP Security Modem Security Account Security Password Security Root Accounts Guest Accounts User Accounts File System Security NFS Security Device Security Script Security Program Security General Security Measures Analysis and Evaluation: Chorus provides support for the development of fault tolerant computer networks. It also has provisions for the development of secure distributed systems through support for access control and authentication. Security enforcement in Chorus can take advantage of modularity and encapsulation of resource management. Security is costly and therefore must be provided at appropriate levels. Unlike Unix, the Chorus nucleus does not control the propagation of unique identifiers; it is the responsibility of individual subsystems. The construction methods used offer a cheap and basic level of protection that is suitable for many circumstances. The protection and security in Unix is expensive and is in the system kernel. In Chorus the user can define his level of security and can implement it at appropriate levels. Miscellaneous Topics One interesting aspect of UNIX, which is different from many operating systems, is that ordinary user can type the system administrator commands and look at system configuration information. Ordinary users generally cannot change the system configuration files but it is often interesting looking at system configuration files. Nearly all of the system configuration information is available to ordinary users and man pages are included. This is one of the reasons that UNIX is very popular as a tool for teaching how operating systems work. This also simplifies the work of system administrators because the system administration tools are often just the normal text editor. Vani R Doraisamy 12 Project Report CS-550 Fall 2001 The ChorusOS operating system incorporates several features that successfully address the needs of this demanding market, including: Memory Protection Hot Restart Dynamic Reconfiguration Memory Protection Different applications can run in different memory address spaces protected from one another. If one application fails, it can corrupt only its own data but cannot corrupt the data of other applications, or of the system itself. This mechanism confines errors and prevents their propagation. Hot Restart An important benefit of the ChorusOS operating system is its hot restart capability, which provides one of the fastest mechanisms available in the industry today for restarting applications or entire systems if a serious error or failure occurs. Dynamic Reconfiguration The dynamic process management feature of the ChorusOS operating system allows processes to be loaded dynamically, from either disk or the network, without first halting the system Conclusion & Future Direction The current version of Chorus has some restrictions on the implementation of some of the features described previously. Many of the Chorus system calls do not work between sites, i.e. they do not allow an actor on a site to manipulate an object on another site. Full copy on write message passing has not been implemented. Sharing segments between actors on different sites is not currently possible. The Chorus code is not heavily optimized. Performance has been achieved by designing efficient algorithms, but no serious attempt ahs been made to squeeze every last microsecond out of the implementation of these algorithms. Chorus does not provide as good a test bed for closely coupled symmetric multiprocessor machines as desired. The provisions of backward compatibility of Chorus increase its complexity. The Chorus documentation provided is of high quality. However a number of pieces of documentation are unfinished. Unix is mainly written in C with a small amount of assembly code. C is a poor language for large projects, or for complex systems. Chorus is written in C++ with a small amount of C and assembly code. Chorus systems estimate that it will take a few hours to remove these limitations. We know that Unix is used in many companies for various applications. Chorus is being used for telecom manufacturing and data communication in system manufacturing. A few companies that use Chorus are: Agranat Systems Telelogic Nokia Telecommunication Huges Software Systems Sun Microsystems Vani R Doraisamy 13 Project Report CS-550 Fall 2001 Appendix Links: http://www.cis.temple.edu/~ingargio/old/cis307s96/readings/docs/ipc.html http://users.actcom.co.il/~choo/lupg/tutorials/multi-process/multi-process.html http://www.spy.net/~jeeb/unix_security.html http://www.tssp.co.uk/Literature/Supplements/unixipc.htm http://compute.cnr.berkeley.edu/unixhelp/concepts/ http://www.yahoo.com/Computers_and_Internet/Software/Operating_Systems/Unix/ http://compute.cnr.berkeley.edu/unixhelp/concepts/ http://facility.berkeley.edu/pdf_files/UNIX.pdf http://www.cs.berkeley.edu/~gribble/osprelims/summaries/Chorus.html http://www.cse.ucsc.edu/~darrell/classes/111/slides/silbershatz/mod21.1.pdf http://wks.uts.ohio-state.edu/unix_course/intro-1.html http://guir.berkeley.edu/projects/osprelims/summaries/osstructure.html http://uwsg.iu.edu/UAU/uau.html http://faculty.juniata.edu/rhodes/os/ch3c.htm http://handu.handtech.com/dpec/courses/un7/frt001.htm http://www.usenix.org/publications/library/proceedings/ana97/full_papers/shirriff/shirrif.html http://www.cas.mcmaster.ca/~cs4cc3/week10.txt -- memory and process mgmt Books: Distributed Operating Systems Concepts & Practice by Doreen L. Galli Distributed Systems Concepts and Design by George Coulouris Distributed Operating Systems by Andrew S. Tanenbaum Operating Systems – William Stallings. Design of Unix Operating System – Marice J Bach, Maurice Bach. Research Papers: Micro-kernel based Operating Systems: Moving Unix onto Modern System Architecture – Michel Gien, Lori Grob. Initial Investigation of Mach and Chorus – Gordan Irlam Overview of the Chorus Distributed Operating Systems – M.Rozier, V.Abrossimov, F.Armand, I.Boule, M.Gien. Microkernel Operating System: A Key to Modern Operating Systems Design – Michel Gien. Web Sites: Sun Microsystems; www.docs.sun.com Creation of Unix Operating System – www.Bell-labs.com Vani R Doraisamy 14