THE DESIGN OF A HIGH PERFORMANCE SPARC BUS INTERFACE by Diana Fung-Yee Wong Submitted to the Department of Electrical Engineering and Computer Science in Partial Fulfillment of the Requirements for the Degrees of Bachelor of Science in Electrical Science and Engineering and Master of Engineering in Electrical Engineering and Computer Science at the Massachusetts Institute of Technology February 1994 Copyright Diana Fung-Yee Wong 1994. All rights reserved. The author hereby grants to M.I.T. permission to reproduce and to distribute copies of this thesis document in whole or in part, and to grant others the right to do so. Ui,~.. // . Author ,4: a q -~ _ ~ Department of Elecrical Engineering and Computer Science February 4, 1994 Certified by - -Lg-=.6, 0J 1 .J~t ' + Dr. William J. Dally (jginie.ing and Ctpurer Science Associate ProfessreoElectrical 4,<~~~~~~~~~ A~rvisor i I 1.~ Accepted by Dr. Leric R. Morgenthaler Chairman, Departmental Committee on Graduate Sents /"k: ... t jJ-a \t "' iE ; go_ l~...\ 9 4 ) ..... S ,.-.......... THE DESIGN OF A HIGH PERFORMANCE SPARC BUS INTERFACE by Diana Fung-Yee Wong Submitted to the Department of Electrical Engineering and Computer Science February 4, 1994 In Partial Fulfillment of the Requirements for the Degrees of Bachelor of Science in Electrical Science and Engineering and Master of Engineering in Electrical Engineering and Computer Science Abstract With the advance in VLSI and packaging technologies, the power of microprocessors has been increasing at a stunning rate and the computing power of a workstation is now edging close to that of a mainframe computer. While workstations are already commonly connected by networks, the computational power of these workstations can be greatly increased if they can be configured as an integrated system for distributed computation. Unfortunately, the bandwidth, latency, reliability and cost of current network technologies are inadequate to allow such distributed network organization to be of widespread use. In this thesis, we designed and simulated a low cost SPARC Bus (SBus) interface board which connects workstations into a high performance tightly-coupled network. The interface provides high bandwidth communication channels between workstationis and routers supporting high speed routing for both multiprocessor and multicomputer systems. Thesis Supervisor: Dr. William J. Dally Title: Associate Professor of Electrical Engineering and Computer Science Acknowledgment I would like to thank Bill Dally for his supervision and support in this thesis. Special thanks is due to Larry Dennison for being so helpful and patient. His guidance and encouragement have made this thesis possible. There were so many times that when I hit the brick walls he was there to rescue me. To Kinhong Kan, Duke Xanthopoulos, Ed Ouellette, Andre Dehon, Andrew Chang and the whole 6th floor crew who have made the AI Lab a fun and friendly working environment, I would like to express my appreciation. I am also grateful to Nate Osgood, Kathy Knobe and John Keen for so kindly reading my indecipherable draft and providing me with invaluable comments. My thanks go to Ronson for his companionship throughout my undergraduate years; to Cliff for teaching me how to think and how to care. I thank him for everything he has done for me. I give my deepest gratitude to my family: Dad, Mom, Mary, Winnie and Shirley for their unfailing support and encouragement throughout the years. It is them who teach me how to become a real human being. Finally, I praise the Lord for His mercy and grace. Without His love, I certainly will not be here today. "I lift up my eyes to the hill where does my help come from? My help comes from the Lord, the Maker of heaven and earth." Psalm 121:1 To Dad, Mom, Mary, Winnie and Shirley Contents 1 Introduction 1 2 Design Overview 4 3 Board Modules 8 3.1 3.2 Modules .......... · . . . . ·... . . . . . ... . 3.1.1 Memory Module 3.1.2 Transmit Module . 3.1.3 Receive Module . . ····· . . . . . · . 8 8 . . . . . . . . . . . . . . . . . . .. . . . . .. .. . . 9 118...... 3.1.4 SBus Module . . . . . . . . . . . .. .. .. .. .. .. .. .. .. .. .. .. .. .. ....... . 0912 Submodules ........ . . . . . . . . 3.2.1 Bus Exchange Unit (Bux) . . . . . 3.2.2 ENDEC 3.2.3 EDS 3.2.4 3.2.5 . . .... .............. .......... 12 . . . . . . . .. 12 12 Cyclic Redundancy Checks (CRC) . .. .. .. .. . . .. .. .. . . .. .. .. . . . .. 14 12 FIFOs ..... .... .... .. 14 ........ . . .. .... . . . . . . . . . . . . .............................. ..... .... .... . . . 4.2.1 The Slave Cycle . 4 Board Operation 16 4.1 TDMA plan ............................... 16 4.2 The SBus Operation 16 ........................... . . . . . . . . . . . . . . . . . . . . . . . . . i 18 4.2.2 4.3 Service for Interrupt Request . . Message .................. 4.3.1 Sending and Receiving a Message ............... 4.4 Flow-Control 4.5 Interface to the SunOS ......... 5 Simulation 5.1 Generation of Simulation Results 5.2 Simulation .... . . . . . . . . . . . . Results 5.2.1 Verification of TDMA Plan . . . 5.2.2 Verification of Flow-Control . . . ................ ................ ................ ................ ................ ................ ................ ................ ................ 6 Performance Evaluation and Design Options 6.1 21 22 23 23 23 24 25 27 6.1.1 Throughput ........... 27 Latency Design Options . . . . . . . . . . . . ..... 29 .............. 31 Conclusion 34 A SBus Interface Board Specification A.1 Device Components ....... A.2 Device Timing .......... A.3 Board Interface Signaling .... A.4 Intermodular Signaling ...... A.5 Format of Message Frame A.6 20 27 . 7 19 Performance Evaluation ........ 6.1.2 6.2 19 Memory A.7 FIFO. Module .... . . . . . . . . . . ............... ii 38 .................... .................... .................... .................... .................... .................... .................... 38 38 39 39 39 41 42 A. Transmit Module . Receive M odule 51 A.8 Next FSM ........................... 43 A.9 SBus Module ........................ 45 10 11 . . . . . . . . . . . . . . . . . . . . . . . . . . . .48 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B Module Codes 53 iii List of Figures 1.1 Interface Board Connecting Workstations to Routers......... 2.1 Block Diagram for the SBus Board Design 3.1 Transmit Module Signaling ... 3.2 Receive Module Signaling 3.3 SBus Module Signaling ...... 3.4 1 12 CRC-12: Check Polynomial G(X) = X + X +X 3 +X 2 3.5 CRC Logic Block ......... 4.1 SRAM TDMA Cycles 4.2 Next FSM Signaling . .- ..... 3 ............... 5 9 ... 10 11 +X+1 13 13 ..... 17 17 A.1 SBus Interface Signals 39 A.2 Intermodular Signaling 40 A.3 Next FSM ....... A.4 SBus Module ..... A.5 Transmit Module . . . A.6 Receive Module .... .......................... .......................... .......................... .......................... iv 44 46 49 52 Chapter 1 Introduction As VLSI and packaging technology improve, the computing power delivered by a single processor has been increasing rapidly. The current generation of workstations can deliver more power than mainframe computers a decade ago and in a more cost-effective manner. The productivity of these workstations could be increased even further by harnessing the aggregate power of multiple workstations connected in a network, thereby enabling researchers to develop new ways of using groups of these low cost and high performance workstations to deliver unprecedented computing power. However, current network technologies cannot offer such options owing to the followinglimitations: 1. Bandwidth. The most common local area network technology is implemented by Ethernet. While Ethernet can deliver a maximum bandwidth of 10 Mbits/sec, the actual throughput is typically between 2 to 3 Mbits/sec [6]. Such a low bandwidth makes communication extremely expensive and seriously limits the capabilities of networked workstations. 2. Latency. Many interactive or real time applications require low latency. The high overhead of packet-switching protocols, such as the circuit set-up and teardown time required by ATM, preclude their use in environments where communication latency is critical. 3. Cost. There are numerous high speed network architectures being proposed and developed. Unfortunately, most of these architectures (for instance, FDDI 1 and ATM) have exceedingly high costs that prevent their widespread use in low budget situations. This thesis provides a possible solution to the above stated problems by presenting the design of an interface board which connects a workstation to a router to form a low latency and high bandwidth network. The prototype board is a SPARC Bus (SBus) board connected to a SPARCStation. The use of SBus is motivated by the SBus' status as a popular industrial standard which provides low cost, compact form factor and high bandwidth transfer between the mother board and the card. The SBus system design also permits the use of a large installed base of SPARCStations at the AI Lab as a testbed. To provide a clear picture of the functionality of the SBus board, let us consider the following scenario: Suppose there is a network of SPARCStations as shown in Figure 1.1. Each of the SPARCStations is connected to an NxN mesh of routers through its own SBus board. If workstation A (host) wants to send a message to workstation B (destination) on the network, it writes to its SBus board which relays the message to the router via fiberoptic cable. The router delivers the message through the mesh network to the SBus board on B which then forwards the message to the kernel. While the primary design objective of this thesis is to solve the above mentioned limitations of current network technologies, we also need to keep the following design issues in mind: 1. Transparency and Modularity. The board is intended to be an interface connecting workstations to networks and it is important that this interface be well-defined and transparent (i.e. has a small number of handshake signals). 2. Small Form Factor. All the components must fit on a single-width SBus expansion board (146.70mm x 83.82mm) given by the SBus specification. Thus, the parts count should be minimized to make the design more compact. 2 NxN Mesh Routers Figure 1.1: Interface Board Connecting Workstations to Routers. 3. Ease of Implementation. Due to constraints of time and effort, ease of implementation is clearly an important consideration. While we may choose to cut corners to increase the throughput or to lower the cost, it is important to ensure that the design remains robust. The design must effectively trade off constraints of speed, size, cost, ease of implementation and maintenance. The thesis is organized into 7 chapters. Chapter 2 presents an overview of the SBus interface board and describes features of the system. Chapter 3 describes the various modules on the board while Chapter 4 turns to discuss the operation of the board and provides a detailed example of message flow. Chapter 5 describes the simulation used to verify the design. Chapter 6 presents the performance evaluation and the various design options of the work. Finally, Chapter 7 presents the conclusions. 3 Chapter 2 Design Overview The SBus interface board is designed for a host-based system with a 32-bit I/O interface between the SBus system on a SPARCStation and a remote end connected to networks. A typical SBus system consists of three participants: an SBus master that initiates the transfer request, a slave that performs the operation and an SBus controller that oversees the whole SBus transaction. The SBus board in this thesis is implemented as an SBus slave that responds with an acknowledgment to Address Strobe and Slave Select asserted by the controller. However, the slave is not capable of initiating an SBus transaction, except indirectly by generating an Interrupt Request. The words SBus interface board, SBus board and SBus slave will be used interchangeably in this thesis. A detail specification of the SBus board is provided in Appendix A and the overall design is as follows: * Clocking: The SBus board operates at 25 MHz. This clock frequency provides a good balance between high performance and ease of system design and integration. It also satisfies the restriction imposed by the SBus clock which specifies the board to run between 16.67 MHz and 25 MHz. The FDDI ENcoder DECoder (ENDEC), due to its FDDI property however, performs at a frequency of 12.5 MHz. All of the modules on the SBus board are synchronized to the Nibble Clock (NCLKOUT) of the ENDEC. 4 I I I To Optical Transmitter I I To SBus Con I I I : From Optical Receiver : Figure 2.1: Block Diagram for the SBus Board Design. * Logic Partitioning: The SBus board consists of the following major logic blocks (refer to Figure 2.1): 1. Memory Module. The Memory module provides storage for messages delivered among the SBus module, the Transmit module, and the Receive module. 2. SBus Module. The SBus module serves as an interface between the SBus controller on the SPARCStation and the SBus board. It provides handshake signals to initiate transfer of data between the CPU master on the host machine and the SBus board. 3. Transmit Module. The Transmit module (Tx) is responsible for sending messages and relinquishing buffers that have been transmitted. It also 5 generates a header word and a checksum for each message, and sends flowcontrol information to the Receive module at the remote end. 4. Receive Module. The Receive module (Rx) is the dual of the Transmit module. It is responsible for requesting buffers to handle incoming packets, decoding the header packets, verifying the checksum, and relaying incoming flow-control signals from the remote end to the Transmit module. *TDMA plan: The SBus board is operated under the Time Division Multiplex Access (TDMA) plan. Under this protocol, access to the SRAM is time-multiplexed among the three logic blocks, namely.the SBus module, the Tx and the Rx. The SRAM cycles are divided into the following cycles, during when a specific logic block has exclusive access to the SRAM - Receive Cycle (Rx.cycle) - SBus Write Cycle (SWr.cycle) - Transmit Cycle (Tx-cycle) - SBus Read Cycle (SRdcycle) The 12.5 MHz ENDEC constitutes a bottleneck of the system. Since 8 clock cycles are required to transmit and to receive a 32-bit word to and from the ENDEC, each logic block can only effectively access the SRAM once every 8 clock cycles. There are 4 SRAM cycles and thus, the optimal partitioning required for each SRAM cycle is 80 ns. The cycles on the board are orchestrated by three different Finite State Machines (FSMs), namely the Transmit FSM (Txfsm), the Receive FSM (Rxfsm) and the SBus FSM (SBusfsm). These three FSMs are coordinated by a Next FSM (Nxtfsm) which enforces the TDMA plan. The simplicity of the interfaces among the FSMs makes the SBus board easy to implement and debug. Most of the FSM controls and interface logic are implemented in an Altera Erasable Programmable Logic Device (EPLD) for design customization and cost minimization. 6 While the design can be implemented using existing hardware components, such an approach will cause the board size to exceed the SBus form factor specification. Although one may resort to a double-sided card using surface mount technology, this would increase both the cost and the complexity of the design. Instead, we chose the Altera EPLD because it provides high-density logic integration, greater cost-effectiveness, shorter development cycles, a smaller board area requirement and better ease of modification. 7 Chapter 3 Board Modules This chapter presents the details of the logic blocks described in the previous chapter. The functionalities of the Transmit module (Tx), the Receive module (Rx), the SBus module and the Memory module are discussed. Furthermore, the submodules inside each logic block are examined. The block diagrams and signal definitions of the modules are included in Appendix A. 3.1 Modules The SBus interface board consists of the following four modules: 3.1.1 Memory Module The SRAM memory is accessed by three logic blocks (SBus module, Tx and Rx) under the TDMA plan. It is a 4 megabit (128Kx32) memory cell and is divided into 512 1KByte buffers where each buffer can hold a maximum of 256 32-bit words. The 17-bit SRAM address is composed of a buffer pointer (9 bits) and an offset (8 bits) into the buffer. 8 Clk /Reset i> next[ 1:0] phase /EFfull ful2tx[16:01 txin[31:0] tx_crc_out[7:0] Transmit Module M- i> /txcrc_reset /txcrc_hold tx_crcsel[2:0] /ren_full /wenjrecycle 5- _'V tx2crc[7:0] tx2endec[10:0] i- /txsram_oe tx_sramaddr[16:0] i-:.. /AEempty /remoteready Figure 3.1: Transmit Module Signaling 3.1.2 Transmit Module The Tx is responsible for obtaining an address pointer to a full SRAM buffer and sending messages to the remote end. It reads a 32-bit word from the SRAM during the Tx-cycle, and transmits the word to the TxENDEC during the subsequent 8 clock cycles. Upon completing the transmission of the message, it puts the buffer pointer into a recycle FIFO and asserts an Interrupt signal to the SBus controller, which then returns the buffer pointer to the free-pointer queue in the kernel. The Tx is composed of the followings: * Multiplexor for selecting whether data, control bytes, or checksum is sent across the link. * Parity generator for outputting the odd-parity bit to the TxENDEC. * Registers for synchronizing the inputs and outputs to and from the Tx. * TxENDEC for encoding 4B/5B code. 3.1.3 Receive Module The Rx receives messages from the remote end and writes them to the SRAM during the Rx.cycle. It consists of a 2-deep 32-bit-wide FIFO which allows data to be written 9 /rxcrc_reset Clk /Reset next[l :0] Receive /rxrc_hold Module /renempty /wenempty S[2:01 endec2rx[7:0] rcl /wen_done rxout[3 1:0] /rxsram_we rx_sramaddr[16:0] /remoteready rcu empty2rx[8:0] rx_crczero[3:0] I Figure 3.2: Receive Module Signaling to the SRAM and to be read from the RxENDEC simultaneously. Upon completion of message reception, if there is no error in the received message, an Interrupt Request is generated to the SBus controller for transferring the message to the host machine. If an error is encountered during the receiving process, the Rx simply relinquishes the buffer by putting the buffer pointer back to an empty FIFO for later use. The Rx is composed of the followings: * FIFO consisting of two registers: - Register A for storing data from RxENDEC. - Register B for outputting data to SRAM. * RxENDEC performing 4B/5B decoding of received data. * RxEDS (ENDEC Data Separator) for recovering clock and data from the FDDI bit stream. 3.1.4 SBus Module The SBus module consists of the SBusWrite submodule and the SBusRead submodule. The SBusWrite submodule is responsible for writing messages from the controller 10 Clk /Reset i- next[ 1:0] . SBus Module phase PA[20:17] /AS /Sel Siz[2:0] Rd /EF_recycle /EF_done ER- :O 3- :i- R- i:R- D[31:0] portsram[3 1:0] /IntReq[7:6] /Ack[2:0] /sw_sram_we /sr_sram_oe /wen_full /wen_empty /renrecycle :O /rendone done_out[l16:0] recycleout[8:0] Figure 3.3: SBus Module Signaling to the SRAM and the SBusRead submodule is responsible for reading messages from the SRAM to the controller. Since the traffic between the SBus controller and the slave SBus module is bidirectional, the datapath between the two employs tristate logic to avoid contention. The SBus module is composed of the followings: * sram2sbus tristate buffer for driving the SRAM data onto the SBus. * sram2sbus register for synchronizing the bus input from the SRAM. * sbus2sram tristate buffer for driving the SBus data to the SRAM. * sbus2sram register for synchronizing the bus input from the SBus. 3.2 Submodules Within each module just described, there are submodules that complement the functions of the logic blocks. 11 3.2.1 Bus Exchange Unit (Bux) The Bus Exchange Unit is a multiple bus exchange device that time-multiplexes the SRAM datapath among the SBus module, the Tx and the Rx. The unit is organized as two I/O ports, one Input port and one Output port, where each port can be used as either a source or destination under independent control to implement a digital cross-point switch. This device facilitates the communication among the four ports: data can be routed from the SRAM to the Tx, from the Rx to the SRAM and between the SBus and the SRAM. Each port has a 16-bit databus. To form a 32-bit interface, two Buxes are cascaded together. This module is implemented in an Altera EPLD. 3.2.2 ENDEC In both the Tx and the Rx, an ENDEC is used to perform 4B/5B encoding and decoding of data to and from the FDDI link. The TxENDEC enables line states to be forced onto the link for control purposes and the RxENDEC decodes line-state information from data that has been received from the fiberoptic link. The data frame is transmitted and is received in the form of 8-bit bytes accompanied by two control characters and one odd-parity bit. The ENDEC, which operates at 12.5 MHz, transmits or receives a byte of data every other clock, thus delivering a transfer rate of 100 Mbits/sec. 3.2.3 EDS The ENDEC Data Separator recovers clock and data from an FDDI bit stream. It extracts the receive bit clock from the serial frames on the fiberoptic line. The EDS also provides timing information for the RxENDEC to perform 4B/5B decoding before sending data to the Rx. 3.2.4 Cyclic Redundancy Checks (CRC) The checksum mechanism is performed by 4 block check registers (BCRs) using the CRC-12 cyclic code [14]. The CRC-12 code provides error detection of all burst errors 12 Input data stream I - Figure 3.4: CRC-12: Check Polynomial G(X) = X12 + X" + X 3 + X 2 + X + 1 > Clk /Reset Crc_ln[7:0] /Clear 2 - 1 ZI 2~~~~~~~~1 > / BCRO BCR1 2 BCR2 BCR3 8 1 ll~~1 . > CrcOut[7:0] 4/ Zero[3:0] /Hold CrcSel[2:0] Figure 3.5: CRC Logic Block of length 12 or less. The data length can be arbitrary. Thus, redundancy and coverage probability change with the data length. Each BCR is implemented by a serial shift XOR circuit (Figure 3.4). Since a byte of data is transmitted every 2 clock cycles, 8 bits of data need to be serially shifted through the BCR circuit in 2 clocks. However, to avoid using a higher frequency clock, 4 BCR circuits are used in parallel, where each BCR circuit performs a checksum on 2 bits of data (as shown in Figure 3.5). In operation, each TxBCR (Transmit BCR) is preloaded with an zero initial value. The data is simultaneously transmitted and fed to the input of the TxBCRs. Once the last data bit has been transmitted, the TxBCRs contain the check bits of the code word which are then transmitted following the data. Identical BCRs are used at the receiving end (also initialized with the same value as that used in the TxBCRs). 13 The output of the RxENDEC is fed to the RxBCRs' (Receive BCRs') input. Similar to the TxBCRs, the received check bits are fed into the RxBCRs followingthe data bits. When there is no error in the message, the result in the receiver is zero. Notice that checksum is performed only when data is transmitted from the SBus interface board to the remote end and when data is received from the remote end to the board. The link between the SBus controller and the SBus slave is assumed to be reliable and no checksum is performed. This module is implemented in an Altera EPLD. 3.2.5 FIFOs The memory is partitioned into 512 1KByte buffers. Upon initialization, half of the buffers are allocated for message transmission and half of the buffers are allocated for message reception. The status of the buffers is monitored by both the kernel and the SBus slave. The SBus slave keeps track of the SRAM buffer status by means of the four FIFOs on board. Each buffer pointer can be in one of the following five states: * Tx-Empty: ready to be written by kernel. * Tx-Full: ready to be transmitted to the remote end. * RxEmpty: ready to receive a message from the remote end. * Tx-Recycle: ready to be reclaimed by the kernel. * RxDone: ready to be transmitted to the kernel. 1. Full FIFO The Full FIFO contains the pointers and lengths of allocated SRAM buffers whose contents are ready for transmission. The SBus controller writes the buffer information into the FIFO after transferring data to the SRAM. The Tx then reads the pointers and transmits data to the remote end. 14 2. Empty FIFO The Empty FIFO contains pointers to free buffers which are ready for reception of data from the remote end. When a message arrives, the Rx retrieves a free pointer from the Empty FIFO to store the message. The buffer pointer is returned to the Empty FIFO when: (a) the SBus controller finishes reading the received message. (b) the received message contains errors. (In this case, the incoming message is discarded.) 3. Done FIFO The Done FIFO contains pointers to buffers which are ready to be transmitted from the SBus slave to the host machine. The Rx writes to the FIFO after receiving an error-free message. The FIFO is then read by the SBus controller to transfer data to the host machine. 4. Recycle FIFO After the Tx finishes sending a message, it writes the transmitted buffer pointer to the Recycle FIFO. The Recycle FIFO is read by the SBus controller to reclaim the buffer for another data transmission. 15 Chapter 4 Board Operation This chapter describes the operation of the logic blocks and details the means by which a message is sent to and received from the SBus interface board. 4.1 TDMA plan Under the Time Division Multiplex Access (TDMA) plan, the SRAM is accessed by the Tx, the Rx and the SBus module during Tx-cycle, Rx-cycle, SRd-cycle and SWr.cycle (Figure 4.1). These four cycles are automated by a Next FSM performing SRAM cycle partition and databus/addressbus arbitration. The Next FSM selects which module drives the SRAM address and datapath, asserts (or deasserts) /WEand /OE SRAM signals accordingly, and enables tristates and registers at the module interface to guarantee that there is only one module driving the bus at any particular time (Figure 4.2). 4.2 The SBus Operation The SBus slave in the design connects the host machine to the remote end. There are 8 handshake signals associated with data transfer between the SBus controller and the slave: 16 System Clock -<- 40 ns -jI - 80 ns -i I TDMA Cycles -: 3 Rx_Cycle Swr_Cycle ti TxCycle I Tic SrdCycle Figure 4.1: SRAM TDMA Cycles Clk /Reset tx_sram_addr[ 16:0] rx_sram_addr[ 16:0] :O sram_addr[ 16:0] ii'- Next FSM i- /sramnoe /sramwe sb_sram_addr[ 16:0] /txsram_oe next[ 1:0] /rxsramwe phase /srsram_oe /swsram_we /sbwenempty /wenempty 3- i- /rxwen empty Figure 4.2: Next FSM Signaling 17 empty_in[8:0] I · /AS (Address Strobe) : used by the SBus controller to initiate a slave cycle. * /Sel (Slave Select): The SBus is a geographically-addressed bus. Each SBus slave receives an unique unary encoded /Sel address signal. The assertion of this signal indicates that the given slave is addressed. * Rd (Read): signals whether the SBus controller will read data from the selected slave (Rd asserted) or write data to the slave (Rd unasserted). * PA[27:0] (Physical Address): PA[16:0]is used to address the SRAM. PA[20:17] are used as flags to signal special conditions. * Siz[2:0] (Size): used to indicate the number of bytes to be transferred to and from the SBus controller. * D[31:O](Data): transfer data and buffer pointers. * /Ack[2:0] (Transfer Acknowledge): indicates the types of data that have been transferred (Data Ack), or that the current slave cycle should be terminated (Error Ack), or both. * IntReq[7:1] (Interrupt Request): used by the SBus slave to request service from the host machine. 4.2.1 The Slave Cycle The slave cycle occurs when there is data transfer between the host machine and the slave. It ends only after the slave acknowledges the last word of data, or issues an Error Ack in response to a transfer size that it is unable to support. The slave cycle can be either one of the following two types: 1. SBusWrite Cycle When writing data to a slave, the SBus controller drives the first datum onto the bus in the clock cycle when AS is asserted. At a later time (subject to timeout), the slave generates a Data Ack for one clock cycle. 18 2. SBusRead Cycle When reading data from a slave, the slave can generate a Data Ack at any time (subject to timeout), beginning with the clock cycle following the assertion of /AS. The data corresponding to the acknowledgment is driven onto the databus for exactly one clock cycle during the clock cycle immediately following the Data Ack. 4.2.2 Service for Interrupt Request Interrupt Request (/IntReq[7: 1]) provides a mechanism for the SBus slave to interrupt the CPU. When the not-empty flag of the Done FIFO is asserted, the SBus slave asserts one of the Interrupt lines to asynchronously signal the CPU to read from the slave. After an Interrupt has been serviced, the SBus controller clears the asserted ,/IntReq line. The SBus slave, however, cannot deassert an Interrupt until polled by the controller. The SBus has a total of seven shared Interrupt lines which a slave can assert at any time. 4.3 Message Each message consists of a start delimiter, data, an end delimiter and a checksum codeword. The kernel is responsible for assembling the header information (such as destination and message length) into the data frame before handing the message to the SBus. The SBus board is responsible for providing a start delimiter and appending an end delimiter and a checksum at the end of each message frame. The message frame format is composed of the followings: 1. Idle bytes (16 or more bytes). 2. Start delimiter (4 control bytes). 3. Data (1 to 256 32-bits-word). 4. End delimiter (2 control bytes). 19 5. Checksum (6 bytes of CRC checksum) 4.3.1 Sending and Receiving a Message The transmission of a message from a host begins with the assertion of Address Strobe (/AS) and Slave Select (/Selo) by the SBus controller. The buffer pointer to the SRAM is provided by the kernel of the host machine on the Physical Address (PA) lines. SBus starts writing the message to the SRAM. After the entire message is written, the buffer pointer and the message length are added to the Full FIFO. During the Tx-cycle, the Txfsm reads the pointer and message length from the Full FIFO and starts message transmission. Upon completion, the Txfsm writes the buffer pointer to the Recycle FIFO and asserts an Interrupt Request signal. When the SBus controller services the Interrupt Request, it returns the buffer pointer to the kernel for later use. The reception of a message begins with the arrival of the start delimiter of a message frame. A free buffer pointer is retrieved from the Empty FIFO to store the received message. Upon receiving the end of a message, if there is no error in the message, the buffer pointer is written to the Done FIFO and an Interrupt Request is signaled. The SBus controller services the Interrupt by reading the received message from SRAM to host and freeing the buffer by returning it to the Empty FIFO. We now turn to look at how a particular message flows among the different logic blocks on the SBus board. 1. Message Flow from the SBus Module to the Transmit Module The Txfsm controls the message flow from the SBus module to the Tx. When the Tx has no message to send, the TxENDEC outputs idle bytes over the fiberoptic link to the remote end to indicate no valid data. At least 16 idle bytes are transmitted before any valid data is sent in order to synchronize the source TxENDEC and the destination RxENDEC. After completing the idle transmission, if the Full FIFO is not empty, the Txfsm sends a start delimiter followed by data and checksum. If the Full FIFO is empty, the Txfsm will return to the idle state and wait for the next message arrival. 20 2. Message Flow from the Receive Module to the SBus Module The Rxfsm controls the message flow from the Rx to the SBus Module. The Rxfsm spins in the idle state until the start delimiter of a message is received. Data bytes are first put into Register A until a word is composed which will then be shifted into Register B. Register A is ready for receiving the next word and Register B is ready to be read by the SRAM. This process continues until the end of message is indicated. If there is no error in the message, the received buffer will be sent to the SBus module. If the received message contains an error, the buffer will be ignored and the buffer pointer will be reused for further message reception. 3. Message Flow between the SBus Module and the SBus Controller The SBusfsm controls the message flow from the SBus module to the SBus controller. The assertion of Address Strobe (/AS) and Slave Select (/Sel) by the controller indicates the beginning of a slave cycle when data is transferred between the SBus slave and the host machine. The physical address PA[27:0], Rd, Siz[2:0] and D[31:0] (if performing a write) at this time are driven onto the bus. During an SBusWrite, a free buffer pointer is read from the kernel. Data is transferred from the controller to the slave and is written to the SRAM during SWR-cycle. An SBusRead is performed in response to an Interrupt requested by the slave. Such Interrupts can serve for either reading data from the slave during the SRd-cycle or for returning transmitted buffer pointers back to the kernel. A Data Ack from the slave terminates the slave cycle. 4.4 Flow-Control To prevent message overflow, a simple flow-control mechanism applies back pressure to the transmitting end when the receiving end is running out of buffers. The Empty FIFO, which holds pointers to empty Rx buffers, asserts an almostLempty flag when the number of pointers in the FIFO is less then a predetermined number 21 (7, in our case). When the Tx detects the almosLempty flag, TxENDEC sends halt bytes in place of the normal control sequence. At the remote end, when the Rx detects halt bytes in the incoming stream, it deasserts the remote-ready flag, causing the remote Tx to cease sending messages. Eventually, when the SBus controller finishes reading the received messages from the SRAM, it returns the freed buffer pointers to the Empty FIFO, causing the almost-empty flag to be deasserted. When the Tx detects the deassertion of the almostLempty flag, it stops sending halt bytes and sends the normal byte sequence. When the remote Rx detects the normal control bytes in the message stream, it asserts remote-ready to enable the remote Tx to resume message transmission. 4.5 Interface to the SunOS The software in the SPARCStation kernel is responsible for the followings: * Assembling any header information (such as destination and message length) required by the routers in data frames. * Keeping a list of free-buffer pointers. When there is a message to send, it obtains a free buffer pointer and generates the corresponding SRAM address on the address lines (PA[16:0]). * Asserting PA[17] to indicate End Of Write after a message is sent to the slave. * Asserting PA[18] to indicate End of Read after a message sent from the slave is read. The freed buffer pointer is returned to the Empty FIFO for later use. * Asserting PA[19] when servicing Interrupt Request[7] (/IntReq[7]), the request for message transfer from the Rx to the SBus controller. * Asserting PA[20] when servicing Interrupt Request[6] (/IrtReq[6), for reclaiming a freed buffer pointer from the Recycle FIFO. 22 the request Chapter 5 Simulation 5.1 Generation of Simulation Results To verify the correctness of the SBus interface board and to obtain timing information, modules were written in Verilog and Altera Hardware Description Language (AHDL) to model the components' behavior on the board. The models of the SBus controller, the SRAM, the FIFOs and the ENDEC were written in Verilog. To reflect the real behavior of each component, timing constraints (such as signal hold-time and clock-to-data-valid latency) as specified in the data sheet were modeled as closely as possible in each of these modules. The Txfsm, the Rxfsm, the SBusfsm, the Nxtfsm and the CRC submodule were written in AHDL and compiled into Edif files using the Altera MaxplusII software. The Edif files were converted to Verilog for simulation. Each of the above modules was simulated, debugged and modified with separate drivers and test vectors in isolation before all modules were integrated for the final simulation. 5.2 Simulation Results A loopback simulation was used to validate the operation of the SBus interface board. The operation of sending and receiving a message, as described in the previous chap23 ters, was verified in the following loop: 1. SBus Controller to SRAM: Messages are generated by the SBus controller connected to the SBusfsm. The SBusfsm writes the data from the controller into a SRAM buffer specified by the controller. When the End ofWrite (EOW) is detected, the buffer pointer and length are written to the Full FIFO. 2. SRAM to Tx: The Tx constantly monitors the Full FIFO for message arrival. When the Full FIFO is not empty, the Tx fetches the first buffer pointer in the queue and transmits the message to the TxENDEC. 3. Rx to SRAM: To form a loop, the TxENDEC is looped back to the RxENDEC. When the Rx detects the start delimiter of a message, it retrieves a buffer pointer from the Empty FIFO for the incoming message. When an error-free message is received, the Rx writes the pointer to the Done FIFO. 4. SRAM to SBus Controller: The SBusfsm monitors the Done FIFO for message arrival. When the Done FIFO is not empty, the SBusfsm asserts an Interrupt Request to the SBus controller. The SBus controller then reads the message from the SRAM and verifies the correctness of the message. To avoid any a priori assumptions, the message lengths, contents and arrival times are generated by random number generators provided in Verilog, allowing the message arrival at the SBus controller to be at a random time, of random length and with random content. 5.2.1 Verification of TDMA Plan Since each module of the SBus board operates concurrently, the SBus controller can be writing a message to the memory module while the Tx is transmitting a second message and the Rx is receiving a third message. The proper functioning of the TDMA was verified by examining the multiplexed signals which drive the memory module and the interleaving reads and writes initiated by the various modules. 24 5.2.2 Verification of Flow-Control In order to verify the flow-control mechanism, the size of the Empty FIFO was adjusted to simulate a low-memory condition. The Empty FIFO was initialized to contain 8 buffer pointers. The flow-control mechanism was tested as follows: 1. SBus controller writes a message to SRAM. 2. Tx transmits the message to Rx (due to the looping of messages in simulation). 3. Rx detects the message and retrieves a buffer pointer from Empty FIFO, causing the almost-empty flag to be asserted. 4. Tx detects the almost-empty flag and sends halt bytes in place of idle bytes (if in the idle state), or in place of the start control bytes (if transmitting the start delimiter) or in place of the CRC control bytes (if transmitting the CRC sequence). 5. Rx detects halt bytes in the incoming bit stream and deasserts remote-ready. 6. Tx (after sending its current message) detects the deassertion of remote-ready and sends halt bytes, even if there are pending messages in the Full FIFO. 7. After reading the message from SRAM, the SBus controller frees the buffer by returning its pointer to Empty FIFO. 8. Empty FIFO deasserts the almost-empty flag (because there are now more than 7 buffer pointers in the FIFO). 9. Tx detects the deassertion of almost-empty and sends the normal control bytes, instead of the halt bytes. 10. Rx detects the normal control bytes in the incoming bit stream and asserts remote-ready. 11. Tx detects the assertion of remote-ready and resumes message transmission if there is any pending message in the Full FIFO. 25 Thus far, no mention has been made of the simulation of the SBus interface board under different network traffic conditions. The network traffic can be modeled in Verilog simulation. This can be accomplished by using queueing models which use predicted job creation and processing rates to simulate the operation of the system under varying loads. The jobs' arrival rates and processing times can be represented by random functions which approximate the real world. Meanwhile, dynamic information concerning the current and maximum queue length, mean inter-arrival time and average waiting time can be gathered. Verilog provides random number generators which return integer values distributed according to standard probabilistic functions, such as Exponential, Poisson, and Erlang functions. Although the analysis remains to be done, this kind of stochastic network traffic analysis offers the potential for insights into the performance ramifications of the board design. 26 Chapter 6 Performance Evaluation and Design Options Previous chapters explained the details of the physical components and operation of the SBus interface board. This chapter presents a performance evaluation of the board. 6.1 6.1.1 Performance Evaluation Throughput The SBus board adopts a host-based design supporting only single word transfer (i.e. non-burst mode). Operating at 25 MHz, the peak transfer rate of the SBus controller is 100 MBytes/s. With single word transfer mode between the SBus controller and the board, the effective I/O bandwidth is approximately 33 MBytes/s, since each SBus transaction requires three cycles: address strobe, data transfer and acknowledgment. lToevaluate the performance of the Tx and the Rx, we define the protocol efficiency, , as follows: V length of data length of message 27 For data length equation: and protocol overhead r, the efficiency r is given by the following 1 = r +l The overhead, , included in every message is fixed at 7 words (4 words of control bytes, 1 word of start delimiter and 2 words of end delimiter and checksum). Due to the lack of real application statistics concerning message data lengths, the efficiency is computed under several assumptions regarding the probability distribution of data lengths (in words), Pr(l): 1. Pr(l) is uniform or Gaussian with the expected data length, E[l], equals 128. The efficiency 128 i about 95%. 2. Pr(l) is a shifted Gaussian with the expected data length, E[l], equals 64. The efficiency 764 is about 90%. 3. Pr(l) is a shifted Gaussian with the expected data length, E[l], equals 32. The efficiency 7732 is about 80%. Thus, after excluding the protocol overhead, the Tx and the Rx are capable of transrnitting and receiving at an average rate of 80-95 Mbits/s, when data lengths are longer than 32 words. The bandwidth is significantly better than those of the existing LAN architectures. For example, the peak transfer rate of Ethernet is 10 Mbits/s [6], while the peak transfer rate of Token Ring is either 4 or 16 Mbits/s [7]. The transfer rate of the SBus board interface is comparable to that of FDDI, since both of them operate in the range of 80-95 Mbits/s [16]. However, FDDI and our intended network design use different network topologies and exhibit different scaling properties. The FDDI network is a ring network. The addition of workstations will degrade the performance of the ring and will eventually saturate the network. On the other hand, the SBus board is intended to be used in a network with a fast centralized switching hub whose performance will be less sensitive to an increase in network size. 28 The ATM packet-switching architecture is composed of 53-byte cells where each cell consists of 5 control bytes. Hence the efficiency of ATM network is approximately 90%, which is comparable to 7764 calculated above. ATM is a scalable architecture permitting Gbits/s data transfer. The most common rate that is being used currently is OC-3 (155.52 Mbits/s) which delivers a throughput of 140 Mbits/s. One can see that the ATM offers a better performance when compared to the SBus board. However, the higher throughput is achieved at a higher cost, since the ATM chip sets are fairly expensive. 6.1.2 Latency Another performance metric by which the SBus board can be judged is message latency, which we define as the time elapsed between the first word written to the SBus board by the sender and the last word read from the destination SBus board by the receiver. As described in Chapter 4, a message transfer from host A to host B (each equipped with an SBus board) requires the following steps: * Step 1: Host A to Memory on SBus board A. The message is written from host A to its SBus board memory. * Step 2: Memory to Tx. The message is retrieved by the Tx and sent to the router. * Step 3: Routing. The router delivers the message to the SBus board at host B. * Step 4: Rx to Memory. The message is written to the SBus board memory by the Rx at host B. * Step 5: Memory on SBus board B to host B. The message is read by host B from the SBus board. To derive the latency due to the SBus board in the process above, the routing time is ignored since it is independent of our board design. We define board latency, T, as 29 the amount of time that data resides on the SBus board, i.e. message latency minus routing time. Assuming that the length of a data frame is words long, the steps in a message transfer involving the SBus board require the following numbers of clock cycles: * Step 1: 8 x 1. A word is written from host A to the SBus board A once every 8 clock cycles (during SWR-cycle). * Step 2: 8 x ( + r), where r is the protocol overhead (section 6.1.1). The TxENDEC transmits one word every 8 clock cycles. * Step 4: 8 x ( + r). The RxENDEC receives one word every 8 clock cycles. * Step 5: 8 x 1. A word is read from the SBus board B by host B once every 8 clock cycles (during SRdcycle). The protocol overhead r was calculated to be 7 words in section 6.1.1. For the purpose of latency calculation, step 4 can be considered as overlapping in time with step 2 (which can be seen if the routing time is assumed to be 0 for example), the board latency of message length 1, T, is given by the following equation: T = 81+8(l+7)+8+81 = 241+64 In order to see the order of magnitude of the board latency, T is computed for three different values of l: * l = 32: T equals 832 cycles or 33 psec. The shortest possible time required by a 32-word transfer over a fiberoptic link is approximately 10 usec. * I = 64: T equals 1600 cycles or 64 sec. The shortest possible time required by a 64-word transfer over a fiberoptic link is approximately 21 /ILsec. * I = 128: T equals 3136 cycles or 125 sec. The shortest possible time required by a 128-word transfer over a fiberoptic link is approximately 41 30 sec. The latency figures above underscore the fact that the board latency is largely due to the buffering of messages between the host machine (or more precisely the SBus controller) and the board. The buffering is necessary due to the following two considerations: 1. The host machine may compose several messages simultaneously. In deriving the board latency, it is assumed that the host writes the data words of a message consecutively. However, the host machine may in fact compose several messages at the same time. For example, the board memory may be directly mapped into the host machine's address space (to avoid copying) and several applications may simultaneously use the mapped memory to compose their messages. The supp6rt of such a technique of message creation requires buffering. 2. The SBus controller may take an arbitrary amount of time to service a request asserted by the SBus board. Because the SBus is generally shared by multiple SBus devices, buffering of messages is essential. The SBus controller may need to service other SBus devices on the bus, and therefore may take an arbitrary amount of time to service requests asserted by the SBus board. 6.2 Design Options The SBus system described in this thesis can be improved in several ways. Some of the more important of these possibilities are described below: 1. Higher Throughput As the memory module can deliver 100 MBytes/s I/O bandwidth and greatly exceeds the current requirement of the SBus module, the Tx and the Rx, the bandwidth of the board can be easily increased by two-fold. With the TDMA plan, one-fourth of the SRAM bandwidth is available to the Tx, another one-fourth to the Rx, and the remaining to the SBus module. The 31 Tx and the Rx can double their throughputs and fully utilize their SRAM bandwidth by each making use of a pair of ENDECs. To complement this change, the SBus module can also increase its throughput to 50 MBytes/s by using double word burst mode transfer. Assuming that the number of messages sent from the board roughly equals the number of messages received, the throughputs of the SBus controller, the Tx and the Rx will be balanced in this case (in the sense that none of the subsystems presents a bottleneck for others). While the throughput can be easily improved by a factor of two, the options for attaining a yet higher performance are not as straight-forward. Such a goal would require significant redesign to improve the bandwidth of all parts of the board. One way to achieve the goal is to separate the Tx from the Rx by requiring each block to have its own SRAM and FSM logic. While increasing the transfer rate, such a technique may require two SBus expansion slots instead of one. This option limits the flexibility of the SBus expansion system and may not be a practical solution. Notice that the peak data rate of an SBus system is 100 MBytes/s. A hostbased SBus provides burst transfers of up to 16 words with two clock cycles of overhead, yielding a burst transfer rate of 88 MBytes/s. If this throughput of the SBus system is judged to be inadequate, a different computer architecture could be exploited for higher data bandwidth. For instance, the Alpha architecture delivers better performance and higher throughput than a SPARCStation and an increasing number of system developers are adopting the Alpha architecture. Unfortunately, the design of a high bandwidth Alpha interface would require very careful engineering. 2. Error Detection and Frame Resend In the present system, when the Rx detects an error in a received frame, the message is immediately dropped. Therefore, the responsibility of retransmission is delegated to the application software and no resend mechanism is provided. The performance of the interface can be improved by having a link-level retry if an error frame is received. However, since fiberoptic cables, which have very low error rate (approximately 10-12), are used as the transmission media, the 32 usefulness of a link-level retry mechanism may not be worthwhile. 33 Chapter 7 Conclusion This thesis presented the design of a SPARC Bus interface board. The board is intended as an expansion board in any SBus system. It requires a minimal number of handshaking signals and features a very simple flow-control protocol. The modular design allows the system to be treated as a black box which is capable of connecting any SBus system into a network of arbitrary size. The current design effectively trades off constraints in speed, size, cost and ease of implementation. As mentioned in Chapter 6, it is possible to adopt a system with higher performance in terms of speed and bandwidth, but only at the cost of dealing with a more complex and expensive design. In summary, the design objectives stated in the Introduction were fulfilled in the following ways: * High Bandwidth: The peak transfer bandwidth that the SBus board supports is 100 Mbits/s. This is permitted by the use of the FDDI protocol and a fiberoptic link. * Compact Form Factor: Most of the logic on the board is implemented in an Altera EPLD, permitting a densely integrated system. In addition to the EPLD, only a handful of devices are required on the board: memory devices, FDDI Encoder/Decoders and optical transmitter/receiver. The components will readily fit on the specified size of the SBus board. 34 * Ease of Implementation and Maintenance: The board provides transparency among modules and facilitates modification and debugging. It is designed in a modular fashion (SBus module, Tx and Rx), and each module needs only to know the interface signals (rather than the internal operation) of other modules. This gives the designer the flexibility to modify and debug each subsystem independently, as was done during the simulation and verification phase of the board. * Low Protocol Overhead: We have adopted a simple protocol which incurs little overhead in transferring messages. For the average message lengths between 32 to 128 bytes, the overhead is 5% to 20%. As described in Chapter 4, flow-control handshaking information is piggybacked onto the control bytes in the message stream, thereby incurring no extra overhead. Several lessons were learned from this project. First, one should start with a simple design and improve upon it. The original design objectives were ambitious in terms of transfer rate, flow-control and link-level error recovery mechanisms, which made the project difficult to implement. While the revised objectives for the project were more humble, they allow us to come up with a design which can be improved upon readily (as mentioned in the previous chapter). The second major lesson learned was to expect bugs in software. The Altera MaxplusII compiler has numerous problems in compiling big modular designs (for instance, Txfsm) and some of them were very hard to detect. (For example, the compiler could not compile correctly when the module had a certain number of states.) In conclusion, this thesis presented the design of a prototype interface board which represents a stepping stone towards the goal of developing local area supercomputing. It serves as an indicator that the goal of having powerful and tightly-coupled networks can be realized in the future. 35 Bibliography [1] Advanced Micro Devices. The SUPERNET Family for FDDI. 1991/1992 World Network Data Book. [2] Advanced Micro Devices. Ethernet/IEEE 802.3 Family 1992 World Network Data Book. [3] Altera. Application Handbook, 1992. [4] Altera. Data Book, 1993. [5] Altera. MAX+PLUS II Text Editor & AHDL, 1991. [6] Computer Design, High-performance networks challenge Ethernet. July 1992, P77-92. [7] Dimitri Bertsekas and Robert Gallager. Data Networks. Second Edition, Prentice Hall, 1992. [8] Robert H. Halstead, Jr. 6.033 Notes on Networking and Communication, 1987. [9] Integrated Device Technology, Inc. Specialized Memories and Modules. 1992 Data Book. [10] James D. Lyle. SBus, Information, Applications, and Experience. SpringerVerlag New York, Inc., 1992. [11] Robert M. Metcalfe and David R. Boggs. Ethernet: Distributed Packet Switching for Local Computer Networks. Communications of the ACM, C-19(7); 395404, July 1976 36 [12] Craig Partridge. Gigabit Networking. Addison-Wesley, 1994. [13] Michael D. Schroeder et al. Autonet: A High-Speed, Self-Configuring Local Area Network Using Point-to-Point Links. IEEE Journal on Selected Area sin Communications. Vol.9, No.8; 1318-1335, October 1991. [14] Daniel P. Siewiorek and Robert S. Swarz. Reliable Computer Systems, Design and Evaluation. Second Edition, Digital Press, 1992. [15] Sun Microsystems. SBus Specification B.0, 1990. [16] Andrew S. Tanenbaum. Computer Networks. Second Edition, Prentice Hall, 1989. 37 Appendix A SBus Interface Board Specification A.1 Device Components * FDDI ENDEC: Part Number AM7984A (Advanced Micro Device). * FDDI EDS: Part Number AM7985A (Advanced Micro Device). * SRAM: 4 megabit memory cell (128K x 32). Part Number IDT7M4013 (Integrated Device Technology). * FIFO: 256 x 9-bit parallel SyncFIFO with clocked read and write controls. Part number IDT72201 (Integrated Device Technology). * Altera EPLD: MAX 7000 Programmable Logic. Part Number EPM7192 (A1- tera). A.2 Device Timing The timing specification of each component is as follows: * System Clock: 25 MHz, aligned to the Nibble Clock of the ENDEC. * ENDEC Clock: 12.5 MHz. 38 To and From SBus Controller Clk /Reset /AS /Sel PA[20:0] Siz[2:0] Rd D[31:0] /Ack[2:0] To Optical Transmitter \ SBus Tx Ty Interface Board c< Rx Ry / From Optical Receiver /IntReq[7: 11 Figure A.1: SBus Interface Signals * SRAM: 15 ns access time. * FIFO: Asynchronous read/write with 10 ns access time. * Altera: EPM7192, with 12 ns propagation delay. A.3 Board Interface Signaling The SBus board interface signals are shown in Figure A.1. A.4 Intermodular Signaling The top level module interconnection among SRAM, Tx, Rx, and SBus module is shown in Figure A.2. A.5 Format of Message Frame The format of a message frame is closely related to that of the FDDI Encoding Table [1]. During normal operation when there is no back pressure applied to the Tx, the message frame format is as follows: 39 Figure A.2: Intermodular Signaling 40 1. Idle bytes (16 or more IDLE-bytes). 2. Start delimiter (a JK-byte followed by 3 ZERO.bytes). 3. Data (1 to 256 32-bits word). 4. End delimiter (2 JK.bytes). 5. Checksum (6 bytes of CRC checksum). When the flow-control mechanism applies back pressure to the Tx, the message frame format becomes the followings: 1. Halt bytes (16 or more HALT-bytes). 2. Start delimiter (a JK.byte followed by two ZERO-bytes and a HALT-byte). 3. Data (1 to 256 32-bits words). 4. End delimiter (a JK.byte followed by a HALT-byte). 5. Checksum (6 bytes of CRC checksum). A.6 Memory Module Name addr[16:0] io[31:0] /we /oe 11/0 I I/O I I Description Address lines. Data lines for SRAM content. Write enable. Output enable. The SRAM module (implemented in Verilog) has 128K x 32-bit memory cells. The addr[16:0] addresses a particular cell whose content is output when oe is asserted, or the cell is written with data on io[31:0] when /we is asserted. 41 A.7 FIFO There are 4 FIFOs on the SBus board, namely: 1. Empty FIFO (256 x 9-bit) 2. Recycle FIFO (256 x 9-bit) 3. Full FIFO (256 x 17-bit) 4. Done FIFO (256 x 17-bit) Empty FIFO and Recycle FIFO are used to keep track of buffer pointers, so a 9bit port is sufficient. Full FIFO and Done FIFO are used to record both the buffer pointers and length of messages, hence a 17-bit port is required. Each FIFO has the following I/O signals (where W is the respective width of each FIFO): Name ] I/O Description Wclk Rclk /reset I I I Write Clock. Read Clock. System reset. in[W:O] I FIFO input. out[W:O] 0 FIFO output. /ren /wen I I Read enable. Write enable. /ef /ff 0 0 Empty flag. Full flag. /ae /af 0 0 Almost-empty flag. Almost-full flag. The FIFOs are capable of asynchronous Read/Write, i.e. it can service both Read and Write simultaneously. The empty flag (/eJ) and the full flag (/ff) are asserted when the FIFO is empty and full respectively. The programmable flags almost-empty (/ae) and almost-full (/af) are asserted when the FIFO contains (7) or (Full - 7) items respectively. 42 A.8 Next FSM Name I/O sysclk I System clock. /reset tx sramaddr[16..0] rx-sramaddr[16..0] sbsramaddr[16..0] /rxsramwe /swsramwe /txsramoe /srsramoe sramaddr[16..0] /sramoe /sramwe /sbwenempty /rxwen _empty /wenempty I I I I I I I I O O O I I O System reset. SRAM address driven by Tx. SRAM address driven by Rx. SRAM address driven by SBus module. SRAM write enable driven by Rx. SRAM write enable driven during a SBusWrite. SRAM output enable driven by Tx. SRAM output enable driven during a SBusRead. SRAM address. SRAM output enable. SRAM write enable. Empty FIFO write enable driven by SBus module. Empty FIFO write enable driven by Rx. Empty FIFO write enable. emptyin[8..0] O Empty FIFO input. next[1..0] O Current TDMA cycle. phase O Phase (0 or 1).of the current TDMA cycle. Description The Nxtfsm is the coordinator for the SBusfsm, the Txfsm and the Rxfsm. It generates next[l:0] and phase signals to indicate the current TDMA cycle. During a particular TDMA cycle, inputs from the specified FSM are driven onto sramaddr, /sramoe and /sram-we. The Next FSM also coordinates the Writes from both the Rxfsm and the SBusfsm to the Empty FIFO. 43 *next signal is generated internally in Next FSM Clk /reset Tx_sram_addr[ 16:0] Rx_sram_addr[ 16:0] > sram_addr[16:0] Sw_sram_addr[16:0] Sr_sram_addr[16:0] - next[l:0] - phase *next /rxsram_we /swsram_we ------- ----- /sramoe *next /txsram_oe /srsram_oe 0a1i41iwr , - "' *next i- /sbwenempty /rxwenemtpy empty_in[8:0] /wenemtpy I__ *next Figure A.3: Next FSM 44 A.9 SBus Module Name sysclk /reset I/O / Description I I System clock. System reset. next[1:0] I Current TDMA cycle. phase /ef-recycle /ef-done doneout[16:0] I I I I recycle-out[8:0] I PA[20:17] /AS /Sel Rd /IntReq[7:6] I I I I O /Ack[2:01 O Phase (0 or 1) of current TDMA cycle. Recycle FIFO empty flag. Done FIFO empty flag. Input from the Done FIFO (containing buffer pointer and length of received messages). Input from the Recycle FIFO (containing buffer pointer to be reclaimed by SBus controller). Control lines signalled by SBus controller. SBus Address Strobe, asserted when device is selected. SBus Slave Selected, asserted when device is selected. SBus Read, asserted when current cycle is SBusRead. SBus Interrupt Requests, asserted when slave requires service from controller. SBus Acknowledgment, asserted after read/write request has been serviced by slave. SBus data lines, for writing/reading data to/from interface board. D[31:0] I/O portsram[31:0] I/O SRAMdata lines, for writing/reading data to/from SRAM. /swsramwe /srsramoe /wenfull O O O /wenempty O /ren-recycle O /rendone O SRAM write enable for SBusWrite. SRAM output enable for SBusRead. Full FIFO write enable, asserted after SBus finishes writing message to SRAM. Empty FIFO write enable, asserted after SBus finishes reading message from SRAM. Recycle FIFO read enable, asserted when SBus services /IntReq[6], i.e. reclaiming buffer pointer. Done FIFO read enable, asserted when SBus services /IntReq[7], i.e. reading pointer and length of received buffer. The SBusfsm serves as an interface between the SBus controller and the rest of the board. It responds to read and write requests from the controller. Notice that /IntReq[7] and /IntReq[6] are connected to the empty flags of the Done FIFO and the Recycle FIFO respectively. If all of the control bits (PA[20:17]) are deasserted, 45 D(3 1:01 /IntReq[7:6] /Ack[2:0] Clk /swsramwe /swsramoe /reset next[1:0] phase PA[20:17] /AS Siz[2:0] Rd /wenfull /wenempty /renrecycle /rendone portsram[31:0] /EFrecycle /EFdone doneout[16:0] recycle_out[8:01 Figure A.4: SBus Module the request is deemed to be a normal read or write: 1. SBusRead (/AS, /Sel and Rd asserted): When an SBusRead is requested, the SBusfsm waits for a SRd-cycle. During the SRd-cycle, the SBusfsm generates an acknowledgment, enables SRAM output by asserting /rx-sram oe and drives data onto D[31:0]. 2. SBusWrite (/AS, /Sel and Rd deasserted): When an SBusWrite is requested, the SBusfsm waits for a SWr.cycle before asserting the SRAM write enable (/sw.sramwe), driving data from the SBus (D[31:0]) onto the SRAM (sram.io[31:0) and generating an acknowledgment to the SBus controller. During both SBusRead and SBusWrite cycles, the Physical Address (PA[16:0) driven by the SBus controller is used directly to address the SRAM. If one of control bits (PA[20:17]) is asserted, the following actions are taken: 46 1. PA[17]: signals an End of Message Write from SBus to SRAM. PA[16:0], which contains the buffer pointer and the length of the last written message, is written to the Full FIFO, followed by an acknowledgment from the slave to the SBus controller. 2. PA[18]: signals an End of Message Read from SRAM to SBus. PA[16:8], which points to the buffer read, is written to the Empty FIFO. Since the Empty FIFO is also addressed by the Rx, the SBusfsm waits for a SBus Cycle (SRdcycle or SWr-cycle) before asserting /wen empty. 3. PA[19]: signals Interrupt Service for /IntReq[7], which is asserted when there are messages ready to be read by the host machine (i.e. when Done FIFO is not empty). When the controller services the request, an acknowledgment is generated by the slave. The Done FIFO read enable is asserted and the output (which contains buffer pointer and length of received message) is driven onto the SBus data lines (D[31:0]). 4. PA[20]: signals Interrupt Service for /IntReq[6], which is asserted when there are buffer pointers to be reclaimed by the host machine (i.e. when the Recycle FIFO is not empty). In this case, an acknowledgment is generated by the slave, the Recycle FIFO read enable is asserted and the output (which contains buffer pointer ready to be reclaimed by the kernel) is driven onto the SBus data lines (D[31: o). 47 A.10 Transmit Module Name [ I/O Description sysclk I System clock. /reset /ef-full I I /aeempty I System reset. Full FIFO empty flag, asserted when Full FIFO is not empty. Empty FIFO almost-empty flag, asserted when Empty FIFO is almost empty (i.e. with less than 7 pointers in the FIFO.) fu112tx[16:0] I txin[31:0] I Full FIFO output, contains pointers and lengths of buffers ready for transmission. Data input from SRAM. next[1..0] I Current TDMA cycle. /remote-ready I txcrcout[7:0] /txcrcreset /txcrc-hold txcrcsel[2:0] /renfull /wen-recycle tx2endec[10..0] /txsramoe tx sramaddr[16:0] I O O O O O O O O Remote ready flag, asserted by Rx when remote end is ready for receiving messages. Input from Tx_CRC. Signal to reset TxCRC. Signal to freeze output of TxCRC. Select one of the six Tx_CRC bytes. Full FIFO read enable. Recycle FIFO write enable. Data, control and parity bits from Tx to ENDEC. SRAM output enable. SRAM address. After initialization, the Tx enters the idle state when idle bytes are sent. When the Full FIFO is not empty, the Tx fetches a pointer and message length from the Full FIFO, transmits a start delimiter followed by data, an end delimiter and a checkcode. When the Tx is done transmitting the message, it writes the transmitted pointer into the Recycle FIFO. Since the TxENDEC sends one byte of data once every 2 clocks, the Tx latches a 32-bit word from the SRAM and sends the word during the subsequent 8 clock cycles before proceeding to read another word. Due to the propagation delay inherent in an Altera EPLD, the parity bit (an XOR function with 9 terms) cannot be computed in one clock. To circumvent the problem, the SRAM word is latched 1 cycle before its first byte is sent. Then, during the cycle when each byte is sent, the XOR terms of 48 txin[31:24] txin[23:161 /txcrcreset /txscrchold txin[15:8] tx_crc_sel[2:0] Tx,Ty txin[7:0] Nclkout /ren full tx_crc_out[7:0] /wenrecycle Clk /Reset /txsram_oe next[ 1:01 tx_sram_addr[16:0] /EFfull /aeempty /remoteready full2tx[ 16:0] Figure A.5: Transmit Module 49 the lower and upper nibbles are computed and the odd parity bit is generated when the data byte is latched out to the TxENDEC. Notice that since the Tx can only access the SRAM during a Tx-cycle under the TDMA plan, the Tx spins in the idle state until it ensures that the SRAM read falls on a Tx-cycle. Tx.crc-reset is assertedjust beforethe first data byte is transmitted to the TxENDEC and tx.crc-hold is asserted after the first JK pair of the CRC sequence is transmitted. Flow control is implemented by asserting back pressure to the remote transmitter. The assertion of the almost-empty flag (/ae) of the Empty FIFO indicates that the remote receiver is running out of receiving buffers. The Tx will then send halt bytes and wait for the remote ready flag (/remote-ready) to be asserted before transmitting new messages. 50 A.11 Receive Module Name [ I/O Description sysclk /reset S[2:0] endec2rx[7:0] rcl rcu empty2rx[8:0] next[l:0] rxcrc zero[3:0] I I I I I I I I I /rx_crcreset /rxcrc-hold 0O RxCRC reset. O RxCRC hold. /ren_empty /wenempty /wendone rxout[31:0] /rxsramwe O O O O O Empty FIFO read enable. Empty FIFO write enable. Done FIFO write enable. Tri-stated output to SRAM. SRAM write enable. rx-sramaddr[16:0] O SRAM address. /remote.ready O Remote ready flag. System clock. System reset. ENDEC line status. Data input from ENDEC. Ctl bit for lower nibble of ENDEC input. Ctl bit for upper nibble of ENDEC input. Output of Empty FIFO, containing free buffer pointers. Current TDMA cycle. Zero flag for each of the BCR in RxCRC. After reset, the Rx enters the idle state and waits for the arrival of the start delimiter of a message. When the first byte of the start delimiter is detected, the Rx retrieves a buffer pointer from the Empty FIFO to hold the incoming message. When an error-free message is received, the pointer and the length of the buffer are written to the Done FIFO. If errors appear in the message, the buffer pointer is returned to the Empty FIFO, essentially discarding the message. Since the Rx can only write to the SRAM during a Rx.cycle under the TDMA plan and a message can arrive at the RxENDEC at any arbitrary time, the Rx has a 2-deep 32-bit-wide FIFO consisting of Register A and Register B. Each byte from the RxENDEC is first written to Register A. When a 32-bit word is composed in Register A, it is shifted to Register B. Register A is ready to receive the next incoming byte from the RxENDEC, while Register B is ready to be written to the SRAM during the next Rx-cycle. 51 rxcrczero[3:01 empty2rx[7:0] Rx Ry Tx Ty Cardet next[l:0] Figure A.6: Receive Module The Rx is also responsible for asserting and deasserting the /remote-ready flag. After reset, /remote-ready is asserted; it is deasserted when the Rx detects halt bytes in the incoming bit stream and reasserted when normal control sequence is detected. 52 Appendix B Module Codes Codes are written in Verilog and Altera Hardware Description Language (AHDL) to model the behaviors of the components on the SBus interface board. The modules of the SBus controller, SRAM, FIFO and ENDEC are written in Verilog while the Txfsm, Rxfsm, SBusfsm, Nextfsm and CRC logic block are written in AHDL. 53 -40- .. . . a . 4 0M. a ^ ^ un b I u u u I 0 0 0* o _ .. - l - 4I 0 ~ U ~ ~ 0N4 U, OU" 11 -. . ~i1 I2 ' - -o u 0 U 1 0 - o C O .0- " 1 1 1l l l 11 11 l lII V 0 ~ o o o U0 l4 W14 0 U o 0 W U 0 4O o o 4 4 1 0 0 0 U011 11 44141 11 e 4111 O O U I 0 <#> 00 77 ° 1 -40,° . I O 4 _- 0 1U 4 O 4w 11 11 H4H " @,,gll 04 U1 U U1 U 0 0 0 ^0I00| I0 U' <#> U u 11> V?~~~ 0 VV0,,Yl 7 19 oy I 00 0 UVt 4 44 xxn1 727 ^0 <D 0 <#> 0 01U1111 * 11 H 11 11 4 > @ N 4 4 H )Hl U U U UIU UI I .0 .107 |1, , 7 o UI - 0U . 4.4 _0 I0I0I .4 0 . l - 4 04 I .11 _0 0.I e UU U VE 0V> VE 0000 VE VE 0 0> ^VtVE 0 I4 V oV -u 0 .. ' 1 U11 C0000 u ,. .I. u _ 0. 0.4 4 U U U 0 0 7 ffi 0 0 u Vt N 1 441 1 1 11 m 4 N U U U 1 1 0 4r0 1 0 0 -407> ^ - 4 - 4r 0440r 1 1 1 1 4 0 0 411 H 01 e 0 0 4. 0' 11 I- . 0 i~~~~~~~~ o @11 4 . 0o . U ZU N -4 a -m 00 14 0 0fl *- u C X1 0co Ef u~~o0 0 qlo am- 0 e4J4 o.-. ~ A0 f l 0 . 34 4 R ._ O u 0l 0 V E. U CO U ~~~o 0Z _ _ 0 oc U -^_EzE _ 4- COa U- I 0 C _ .... h >-l ala . 1. W h -4-4 1 o C C I Il Il hl -I I "II, . -0 ,. 0 UI"1 0 .... u uO O O a) _ 41 a 4 C L. _. * . . 0. e 0 z zul_z a. R0 o V 0 e I 'c - 0 u.0-0 v. A- W. U U U i .0 .o IO ucII oIu u. I I a0 4U a1 1. a. . _ U - 0.1 04 n 0 II eI . .. 0 . -a 0 00 4 4 U4 U - III r _ u .% C _ . - 0 ' u 4 l.. .II _ -1 0.0. 0 , r.I I . , u uI 40 4 uu U uU uU U A I 0 u CO 1. u 4 M LI C M_ C CCS M_U CO f 0 U Z III I~Il UO -1 z ' zoooo -UUO oO -UUU ~ ~ ~ ~ I _ ~ I I u u U U U IUU U~~~~~~~ a ~~0 - - UF U I I, I a, __,,'__,4 ,.,,, 0 0s co _u N~~~~~~~~~1 ONU11 O_ - 0 0 - U U D~~~~~~~~~~~~~~~~~~1uQ I U ~~ ·~~~~~~~~ 0 I~ U U U°l2uU U U o3 ~ ~ ~ * 00 00 .. I i I Z 1 o~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~0Oo I I I I1 I I I 13 000 . . .. ~6 ~~~~~~~~~~~~~~1 0u /~ 6u S &' I ClN 00 |s~l 0 0 U IN 0 Ol I.- l.. lN0 i- 11 11011 .. 031 I Iz - I I.. 31-. . . 01 I 1... Wo 10 113 u 11 11 11 -NC1N - N[JN 0 .. 03 11 0 ]N -Na 10 1... 31.- 11 11 . . uu uu I U C to1 I I I * - 1 000oo 00 0 I 0I I ,_ _ 0.HU UX@-UUUU ,, a . 1.0 0 .o *" z ,, J , , - 1.0 0 I ,,, N ^o J_ -NJ o Uuuuuuuuuuuuuu U1 r0 n v~~~~~~~~~~~~~~~~~~~~~~~ C C EC C_C C Wf I C ' C ~~ ~ ~ ~ ~ . A > .CD. 0 CA 33 > - E '--C .--. ODOD DC U . C.DVC -- -C iI' A CA~~~~~~~~~~~~~~D A03 'x 0 C'1 a CA4 C-, '0 1-0~~~~~~~~~~~~~~~~~1 1 0 0 CD | - CD ot 0 1. 00I C..C~ CDnEECC-~ 1 > -E O CC.. A * 0 .... . -. AC.CA 0C v~ X A 0C . 0 CC, I III 000000000~~~"00 v 00 ~E-E ~ ~ ~ ~ ~ ~ ~ ~~ ~~~~~~~~~~~~~~~~~~~~~= I~~ ~0 ~~~~~~~~~~~~~~~~~~~~ ~ ~ ~ ,CC .. oAC o 0 0 v ~ v...ACAAAAAA - W v OI D D DD _A . 2D v 4J C- vE CC. E- l,CAAC n * .. 0 CA O' 0 0 CC, D00$z I WI n _0 D. v o~~~~~~~ e A CAO s: I 1 0' A -D I .n O. I n 'a0C 0~~~~~~~~~C a 0a, DCI D " 0 >,. AA . _ h D II Di iC DCC C 23 - 3DDC OX C.DCCnC CDI _ ._ IC e DIh.CCI -CC ACC ., -"> h D DU CDA 0 D C O O.CDC 3CC4 C D C v ..f D C4 U D 1.. D 'C >r 0'- V - I 4 - .. I I .0 6 00 . 0 0 L 0 11I 0 I I I X I 0""v U 0 v v 00 0.s 4C. Xu "P I X 0 X I ", 0. Ix . .. ~U I Ez 0 00 D ja 10 I I D1l m- lQ v vY - X C X DC Cc C 3 '- ~ ~ a4 0 4 1 v 0 v a Iw U] X - I X UI e X - D C h-.' L >, '40 01 )L e 4 C 0 31 Ca 0 3. X -3 Q 10 I I 00- I 1 -.1 'CO 0- X I.0 I X 0 _ 0'' 0 0 vP vD t03E C A Q 0- 0I I~ O. WV 00I 00 3> 0- D '-.0 I C I E 0v Ol 0 0 I C 0 n^ '.1 Q C 0.> 0 0 14 I - C1( v 30 -.- X 0l 4_ 000l QXC C 0 D 0 x C ,, 001 >. 10DQ XO.D 00. C OM UI .00 .0 Z 0 0 *- v O I. vo s 0" e "I I I I 0 3°l XX C X e 4 X3 I D v LiX C D C I a C i 1. 01 v'I e 0 w I X P~ 0 a: a c' 0% 0.0 O'C e ot 0 0 I.C I co C I aD v X .. 'C. 4 . 00 0.0 I moi 00 e I XLg .I0 muu - _ 3 .. 0 D - . 11 U' : 2 I i II I0 (0 Iu I . 4.) Z m II v u.4 u I ·- 2 X v D D 1 o m0 m 00 Q L Q a m 3 IL. Ie - i0 6 v 4m0 X Z 4 0 - U fl 3 1e oo 30 X "I. 0II W 0 I I 40 ) O 4 v 0 D " I" 31 I 0 E 4 e)1 v . U2I-r I'3Z , m .01 3 - D I o D .a3 0 e E X C vU 000 "3 0> U-A If > D 1l 11 4 - v 11 0I D e~'-. I 3 C D eP ED 3E 10 H_ Ioo ID 1m I X c X cr pt v Xv >r 1 0 X ca 3r D c Je s4 e ^ N 4 X3 0 3 I'- 0 30 ILX.( 0 I 1-' UU I 'C3 01D 0 v s >* v 0 'C § v X C e I .. L'. ac UL X C _ 'C1 ( a I X ::1 _z C1 ¢ X I 11 ^. > z v e v IX X4 u <;C uu Aa 10 4 0. 0D 1 'C1 04 I DI D v vm - D -m 0 T 3I 41 .111 =: 011 1010 111 4 aD E I I C c Iv.' 000 c0. 050.O~- 00 E OL. L. 0eC 300c . 30m C C 3 3 Ca " - L4 3.0 X 4 '0 4 e'41 ,0' O -'. *I 30 40 O 30 00 ll O -c C , fl 0 c a C s i ! aa-c4, ,, I Wa 301 . 0 XX 4,000 al P~ 4D . C ,.''3' ,.3I , olo 3 I : I0 rp : aL~ Pli Ll · r11 I ce 11 31 0)') Im 11 C a) 4 I0'IELaC.0'03103U. DV 21 t ra - D 4 Ia lI 3 4@, sr ItZ I I V I EI a * iL a. - 4 o I a 0 a _ o 0I 10.0. _ _ uP 033 I 2)4), I I XXX b4 b I 03. 3.0 b JP l a : :1: .5 1 I n 1 .. m I t D 0 4 a '04 S O D II b_,' O 44 I Im I *'03. -'0 a X ·l I - a 'a . '0'0lr 3.0 .. 0 1 DX hX 1. Ul ' 11r z0O X O D 0O 01i 3 L D UU X aOe .3. 11D;3o4, I 18 m am m C 1s3X ~IW If : | ,,3 I C D '0I 0'' sI 3 . 0 4 ds r4 . D, ' 3 C __ $ D~ V C C 4 C D D 0.0. 00 4, e U C O- 0 a 00 0% II o 0, T I alP u I U I ffi C V c ; QO a 0..- la0 Ie r s N e _ v ID 1.3 "v I ->3 C - oo : (0 1 I 03 02. i o I 3.Z 2. 3 3. O . w w _ D O 8"100 w I o I , I I1 I1 1 eme x___ X CmLV E , ZZ I a 0 2. , V000E 0 4)~.0. e.. ae b O) O 30 a' 3.4,~ I D U - I00 3 E· b.) 01.. c- Oe flaX 410Y be I C C 2EC v 10.0la '1 O U 3 I w _V clu _. 0 4230 42w 1:34 U O u X E C w I CI XI I : V U4 b 0^ 03. CO UUOU E·z o 00 0)00 .. EE ZZZ Z ..2_.... ..4 __ .. .. Z .. .._ w~ D)'O XC 3.o b C Pa b,, 3.33 0. .., W ,__D v.0 V eqI a3 r: D· u u c . >" 42.o - I uo 00 3 v Go tn: a: w, o o U 0 C)42 Z ZC B 000 z 04). D C Y XI I ' > . o0 w t -i , I 0 ' u 00 0 M 3 e 2 Z 44O4O O Z OOO 1 a 2.3 m b 2.3 c WI ::D 00 - V - . ,1 - 0 IW I D 0.Xx aa·C Dx D, Ot I Il bX3X. C o~~~~0 >,:, " I0.I0v.0 D3D3 X vi v.4 ^I b b3.Xb II313 I 0 0 _ 1 . c) -a .. ) -a) .U .. Ma _ M I Uci , 104 '- _ aa)' a) - at1 I1 a>> 4 1 Pa Ma) a) U 4 .. ..C 1. . cc a s m I c.4 a ? __ - - ..0a>o u* u ua r1 44 4 MMMI . w u i aa 4 0 a - . e M 'A : Cm m en z S : I . a U _ a 0-. a) b a.0. aC. UUC. U aP MSC C> -vCz_ 4 nuru _ _ E L4 XL i Ch ch 14 1 3,N 0% - M-4--M0MVM I III z O 4( a ' a zze zra a Ia 144 10 q4 IE Xi 4. I ' d!d I I 0 0~ o I r NNNNNNNN co .0 E4 i 14 > _~ . _ . 4M. > , 03V303V3 1 )1 NN11 NN_ NNNNN 1 S i i i .a . u, II I a . eXXeX3X i I q Iu o 7 U i u _i Xi,i 1 Xi,c,c U u UUUUU UUU UUUC ~·g Xu u C >n _II.I _ MMM ~ _ _a UUX _M _0 XI-U oe C' -- >0 4 _ Na a 5 ci X _ X_. 4 4 c44 C uI 0I UO. . 0a >1 _eo-2 a .0 _# a) _'Ai U . a- 00000 010101010101~01 -w_ _ X F _ _8 'c 660 aa N N XU "b 4 U 1) v 4 %e s O O O 0 V I 0 1 0 0 P CC 6 II C V V ta C m>0 0 0 : D > a -v Ae0> I0@ XAn VU I 0 C US 1 Z CA* L3 0e aQ z0ii u . . . , _, I I bI l 3 I Lb.~ e 00 .4. VVc~ X XX 1140-AS DL2 I fiX 0e 4 I 6 II Si @ b i I I N .4 Co 'a i; b~ UB ^I I V 0 XC .4 b88 U Cl- XI e r ceX 0 _ C ,, 5080 3 5 1H UUU -i .8 .4 -m ~uWCu Csm ,e 4!-;! X _ 8 .4 ~ 0 4N " 2 e- 3 X 4;! XDl 8 O 'nn " w ^ An C A8CS uu ef .4 II C *1a . -0 W I fiy w C 3e ff B X8 J w I a" ,4_ Ib I U - _ I.. ,, , U bC ai Y -4 I1_Ii -a 88 * 8 ,, I S 0 0U C e X -·--' .40 U o1 .4 dP I I u " -V4 Y U U _ 3 Ub-nOd :4" : I . 0 0 Vb C .4k 0 " O -4 c OLNE 88 UC n 0 Ia8 ·o ·· c i. m b 1 m OU Ae88 A L- A 43 C 0>Uamo L b * ^_ > >A.. e oC Xb I I ws - a 8 A.4CCSe( IC> ' A e b4l8 Ukk ICL o or @ b oh 'm e CD in 08U A a CC CC a C C C 0 6 n woE 39 _, 2C 3@ z 32 i c k. S u 0- aco 8 U 6 , _ A L.U O~ al O0 a S > c a - d 30 c " u -S 08>. rlL e V0> L) r u >8 U II 8 bUO 8 e ea b---- 0 -8. Xl116 Y C488 3 80 6 -4 '0 'I u o C I S4 L1 8I 8 8W@ _ 88 0.C ^ 58 8I 8 0' 8 qA lp A II v-,a 111E bl X,$P u S b _~s b U 8I8 U.4 8a48 8 SOy 8i ~1 C C~ OI b L. k81 Uk N e "e e"s eY 7 w8 e a u c U0 1 84 I E r° I ss$" I iD b w eL m -s Ao 0 pSo 1 ceOm ... ePa e i Oz o A bS i Sn ICI1 ,e 4 . L. a a _ . PPaw@@ a b 4 C488C O 8 e .. @ > 3a, 0* AW _l 8.OAS 88I b @0808 0--IL 88-X b 080 I. I I aU -u 0 !2 za > 10 v, V z c . 1. A a .405 . ZV I 0.0 > I .U IC 741 I0C o>8 u e> 4c 6 - 1S e 3ib I" , C. s -oc IY UL D | I I I eX X .4. -55. UCO_ I , A 'C 0 88 . I4 I la '- .. I0 '. A.. 1 . I a i I 4i SI i 10. ., Co -_ 0C 0 I .4 n -' C 00 w 5 Pk 0 0 rr0 C b. O _f 5- -- bw 6 n ^ CC ILI 'm 11 U $64 5 0>I-'S 8o i, I 0bI C 11 3 40 I*8 86L .- 88 N8 XC , I Au 50 X V. § 01 . X I a z 2 oul >I~I WO WO U ' CCC -Cfo a u u L 0. 0 QUO OA0[ CC.. 0 I ~ UCL. CU _U . U~r 4L.A QW[Aa · lO ,i C. LS UB s > U.C 0 A ' OL. n ACA V -l I3 - I IC A O L. CIP AAWL~or o. I A ou Z W O Z uAL.3 oUAL -C C ~e _ 11 V 3b0.0 V x4 1 A ~l3 . A" V _ OZ CO L.o Wx - I^L I H OOW a uA (SIC A A AL A oL I - 4L0H clIL. Ax . L. V I Ll.A JJC.o0iV@6CAeSf CA [A au X O 8A L 3A I -C L.CX - ^COC w CA [A A. [A w w U Iu S a V) 01 ' V A 0l O I I O ' O , t,.73 - t v*4I C o eE CC- C PIo ~ ~ 3 ia - o C CCI-" E2 , .Y I0 o 1 I_ u CC ,8 c s3 UAO) uI 0 [A Uv ~4 e l r xe $ -I- A Aom ' Ud - c C .01~al- I- O Iu unr I a~ E _ a, > A V zQ 4L A I ' XUC U I ' U 'AL. w1 VI 4 L, I - _ U I 3 U l V L4 r0 U U I a, XI ^ L1 - U4 ,. u I OA 11 u~ Au u u A 314 u '"A L, u11 0 uX Vo . I AI X I A cu uo l o XI 3f ,aA3 I lA I A ~ ,A A .,,Al ia u aa !UW L I11r 1 XI 1. I 4X 4 X Ie 3 AJ x u 4X A A A L. L. A 4 X 3e u A L. UL. U14U o1 H X~ Ix A 4. 3 ,;:, ql 3 u I A V LQ. - , 4C0 I n ... 3 h .. U.-, 11 Ur _1 14~ u1 e 31l a1l uv 4X X u Lz nx u1 11 H 1 1 v u 12 aa xx 0 ~~~ ~~~~I~ ' 0 ~~ ' ~ 0 aV_. L.~ UL .LA0 L~~Z*E m a 3 i ICY OL a , 0r ~~ I I i I I I , X 0 Cl .C1: OV iU I I I I en-a X - 4' 0 D 'U _ .. i, O 0' I' I .0 O ' '0 uC am , 0' U , ' ' ' 4 u I ,,, CD 54 ,, , >^x Xe I, 433n 0' OX , , ' I, bI O '0'a a C ' 8' . ' _4 4 _. a , II I 'I 4uuu3 -o I C 4 4 _ _ e 43 J1 | | s X , en ' ' Ola e, {D , I ,,' ': ' Z V I V 43i' C I l l v 'V 'U V 'V I D o 4- a e Of h |1 ,l I h a I D 'a La b3 u $iu e t 11e.enaee- e ee4 h I U I 1> 'm b t"~ n r.J IA I I | I I I' - -.· 3 I enen - O en O eD aD D ._, 01 l X 4444 U O e1 O -r 30a:, C '-0 III U III1 ' .. ' a '.Latr en I I3I D S m X D en vm a g _4 *b ' 1 4h a cn ._, cd D w ,, I La LaD3 > u cen: .j o a.'ZU a. Dci a nv .- '3' D eP Ie ul > e'o z > 4. (3 C>> 'a a0 > ID > >1 e f I u bI ., 41 1 I , X aNn: pl 4m I h _ cn LaO C v b1O1O1N v1 I:I C lfi * I evv 03 'env eI gI X I 'ad ae '0C a' C I 'en'La VI ' li l 0. , 10 * enI' X . I , aa ' ~ X I V Il ~ , _ a _ Iv Wv x D~Ce . X X C - b XI T n: o0o XX I uI C41 dl VV a s z 0 '00C '0~00 l I mm eI' 000O S , LIU :9? I I 3 ~ C u I ts 0- I aa 02 ~P PPQ I~~~ s e , mm D mm VV -0.0.0-0..02.. 02.. ..0.02 4~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Ji.-.E 02 mw' L 020'. 02 ,-'-.-Z:Ol'.'.102 0222 '02i a. 4~~~~~~~~~~~~~~~~~~~~~)4 r ~3w 1 0 II0 , n I I 0 w02 a m 00 Ul X im 0 20 10 01 .. u im i i) * 88 1I I~~~~~~DD=1= , : DDDD 1= - ... E I' 0200 0 I 23~~~~~~~ 2 , D ,~~~~~~~~~~~~~ n~~~~~ Da Ei V.' X l~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~l I D I o . s I ~~~ ~~ ~ ~ ~ ~ ~ ~~ ~ ~ ~ ~ ~ ~ 3o 0 ~ OJ , E; E: 1.I 0- 11 en 1o -. E 3, 4) C 0 02 . E;E -E V L, 0t E s0 ; E dP S0 d on 0 m ·0, as O 3 024 'Ou _02i 0C Ii > O a O I ,i02>, m~~~~0 0o 0 V ~ ~ as S01 _ 3J m02 ~ X: OEm a: C _S:U 0 fi _020 X -E NC..ow~O OX 0y.02 C S. D D O vl tD v C. D , 0 v v '.2 '.0'| ' 0N D >,>u am 0 00 U Z w O~ O 02 2: a 3 Z '.t02' , , ,I X u ,OI,.. .... 02-02 oc_ o- Y o *: 02] E . w° U a E (o '.* 0 02 0 020 0 C'. w * Z I 4 00X E-024 0 20 20 2n 0202 V 0D O.-.-.w V 4 iXx <0023ww00 0 U a Z ZZ ww ZwwZZ bll C a Z b 0 DD; D oaoosm'.Q a a o· 0 o :"I ; _ 0 _t _ u .00 ;i -W_D. 0.002 * _ v, ,W . . '0 W 02 e0220 30 33 44 .............. 11 11 W 02'a:11 X e-' EO0O.O2.0 --Z 02( 020m0 W ZwwZZ 02000 0 C :I '3=1 w V)U 3 0 02 u . 3 0 C .^ _ .. .......... (0 0 '-"- . 02DuC - O :e.-C.O -u _0.0D0D 00 0 ,0. S 4D: e 0 >.>0 m2 o OW . o f DO -".0 04 D a O cU ~o-0.>.02y ~1 ~O : 02 D w m ZwwZ e-.. 1 ~ 41e oV02'- 0-- 43 w 0000 OXw 10 >.l. .002-. 1020 uow0.0. 3w0-.2E C.. D Dco i C D m 0-0-r 0 :I 41 m '. w 020 a o00 t O 00 0 0 'O 0UO ' O D 02.0-.DViO02 042 00 DD CO P ^. >s. 0 _m 0 X .u z .2 a O ] X V 3 4 m 0 0 02 U 1I -.. u Id 02" v (D C 0. a:11 m- 0CIL a:2 0220 -· C 0> 0 IO >u I u . 3 020o 3 2 0 - II-v rT D JTN e 1 I -42230 vfi e: DF 1FF E D F0000'- 3 D- E I * II ~~~C 10 V 3 I : V -I I 11 ,~ a a 0 Co I0 C : 0 - I I a ali i o,, C0 V, I, 3 ol '.0> w ' ''w IU * l: I o w w. u o- CI- _ I ~- ; w 00~ -I '0 I DO' I 1 I , ,4 -00J- O > , 'IC 0I 3 cn CI II. 'IV VO w OoO oo0-I 0 - ._ I ltd I W :I I i 0.o 0 _ ~ oI wcouw . Co V* V I o ' * J1 0 -0 .0W X O 0 _ too ti I I -_ -LX _ I -- 0 30 II L 0 Z m wI Pm D 1I ' ° cqa 4 O fi , dC"· 4 *-· o~ X~ U r lto 3>0 cn IC) - ' 1 31aJ I :,I W -^XO , c .I 0I o I., o I~~~~~ I - I sC4 II _ W OCO 0-. m 1 U- 00-0 _. 'I'u ' s · . 4.o . C- 101 u I | mI m10 .0000, 00 E- *oo 1 ""n a ~~ , r 4U , ~-~ -~ 0- 0- - 11110 oo a1 Lu uIL.. u _ m_ POwU0 4e-t7W Lu cn . 3a'IIS> L| 4 EE .> u- u L QCX: s zm WWWW Il 0010 00. c. . m X N " au s w SXww m P 3 m ui N X ua =:: n >m _D 4 4 44 N rs:,. 0m, (O> 4444 . I 011DWW3 P *~~~'· 1 0000o Q:~4N wIIw w ww I O O 1lIl 3 * 0 m m o 2 Dn X , nI ~ m l S D u m X 1 X t a _ I 1 ,4, - aar ,p 0 I-3 Z: ll uI 0s a, - C I Cl .. mi roX , : V ;7. :: 1 11 > _ >,; C * O a: ,= -C :4 Z 0l d u' c ' 11 I~U ts I L,m31N rn S I 000 Z *'a L1 nom ZI U I.-~ mum o-- 00.. , I WO. hm _ 1 I PC0 0 I I 9, mr 1 I- _ L0' *ta 3 'LC., I OZ _ O i c DJO S ab ffi_I I W 2l _) ae l I Il WW ', II C .~0. - I @ I 0 ,I 11n - C fi 'Vl L> I I Zo I EzI ,3 :, I ~~I 0 Zlr X a 0}O V C. Q 3 . . O oc ) ov * I CV -V * I v I '2 _ a) u AC cOW m m L Wa-a *1 CC 1 a-I 41 E C 'C w i'" g, Q V 5 I | U C C 'a N I,, 0 ·- L4 a a.] o ~~~~O -sa J ' O , o 0. C1 m I Cu LW C Z Wl 3 1C N CO LI _, u a C C Sm uV - U xx 0C -v I C I a 'CC' u u C Cv - 0 u v *- *- 'C ,~ 0 CD a D w~~~~~~~~~~~~~~~~~~~~~~~~~~~~ o 40 o 4- o vc *ra * IC 0 1 ,~'VI ' IC XV -- m O CU SN ) - u '( '-I a- a) - *D 4 ) za "~ · t 'CCI * -_ I CX Im , W I~ v . I X | Q I- * fia o'c O C) I II .. .0 U _W I ^ , I a u a A4 cV a) WV O A . 33 pa[3 DI 1 'C' 1 4 _ 0 ': Va-WC(C I, X X I'I - a- C 4.. < I t a C.] a- Ca , Uc~ O a) a) a) a) m I. va v a-.(C ( o v . v fl - f2I Z I 2 ' C(C a- DaU 4) Cn h} ILl w Ia C Z X CoC S s (C -- U. -a a)VY -'C I ..Io D-um I eC C D 4 I) ) V aX --. e -UHaC (O C aQ, a I I (a) 00 Va-A1 ' I C ' 'e Iv- XIw a A,ca 1 C-'1 cn 1@ 1 1en a I*., a C . 'Z C ' I I , ts ' _g' m 'C' I z a- . _ b u 1I'4-' 3 *i a - Ca) ( r 4 I o ~~~au ~~~~~~~~~~~a ~~ .. t b C le -4 ,, v o o CI (C 4L 4 4a 01 ra ) rn 3r_ .u I 1.II I! . 0I6 i5 > yg u' C Iv N11 ^ ii oa C U"C Ca) U41 D rx I D IC >D a 4 a*aYevWnm (Ca.4 ( 1 L4 aea- C U¢ a - wab 2 AC f Pa^r 1 I * ,r I 0\ a.a I aI rl -a. - n H i ,.. N 0 0 0 a o o m 0I --a .0 0 q- a a U o l l o u t - C.z u _ s *0 ZD , · 4 a o an a m'm .- II , OP Rp u 1 b b .9'"- b m .. U . . 0-10 o 7- anz , oa n-fl I,. f 11 DCL C4 cm -a l1 C. I I II II ,1 m , ' ie Im m , Ii , ,a a- · WoW CU4 IY - m > o c% D> a 10O C a i C X u 3 m· r al _ a . 0 DI >,_~ a ,a _:: m m f fi l m m ri D ., w Ia ( cO fi Wa w_ '"3 mu aL--Cm OwCi a-a -I: nn a m I .I mo fi as > Ca C I ,, u u _N lraw o 0o onn _ U ar -.aflrs O -U _i u I em45 a) a, Cc. aC1. - I E .. -a D D i -wD aC 0)0 an a , I 4, a> .. a 0 , a m cz v m~oi ,,rt mraI fxmm _ _I .I mmf cc X CO h E E J X , >, m m I _ r I I Ir ;0u . I. . ... : : ~ ;111. 1. 111. 1 . . : : N4 I 1. 111. .3 1~4- 41;- ;--1 ; 1~1; I,. N4 I. 1 . . . . I. I I I I I. C)P 2I - - 24 O---- C ^ 3_ ai m xl xl Xl 3 43~ -a .34343 l_ c ""9T I .11 I$ 3 _i u. I 1 Oo _O01 Xl Xl xl xl X X x x x x x xl Xx X X oio| O - 4;ii 4 0s Ia C IZ S 0 C .0 C C oa o o el io ; O b. IC 114 .3 43 . . uU . -.u5.1 _ k Su I 'U -3u I a! 43 0 3 , es l 1: . ;I: 'lll I'l I '. 343 I t , , . 00 V 0 n z 00 . I J ' I 31 l 'A . .. 43 a 4 a C- C3 r 4 o3 -I - q.° a 0 i UP - Iob OI x I _ I U I 0 O I o .0 E4 -k l 0rI i f0 g c m O I w A. l o I0 lup 1. 4X 13 (I' 0 - >, IC ,4 0a' _a 1 I >1 o t 41 ' ..2 0 W I o0 3o C43 0 mo ul .. 0' 'I X 0 . .. tNE . 0 . V 'd 4) 0 0. 1S E-' 3. 2 3. EI WI 2 3. M 4C3 i III UXXI 0 A1 '3b 9 22 ID 4 1 'uI4>ZI I 4E 340-Ec ffi fi CL x 2222 W0 ,aC I I 0 4| -. u I *- W1 I4.00 ' f I # d O 3.3.13 A la I '¢U 4 i Io 0U I £ 43 %4 ,0 W 03 II I I II II II 0 rzO IC I'c 1 1 2 I 0 443Y O Oha .0 . U Alr C fi34, O C0 a. 0.11 UL z~~~ N x U ffi 0i .fi ' D ..Iu . 0 "C -0 C1 ~0 _ cq I '* 14 r S U W 0 gE 2 _ ~I ,z 0 In wi - , s m ,c m V a0 5 ol- I s _.. 4 N S ,Ir> O L 110 C @ 43. I ll 1 4 X @ I W I. U ·I m , O , 4 .. .. -3.4 ,;P. I I XI D 43) ID 1343D # e I -o - 3 ,- . 0 uzooo 00 .^ _ ._ _ S 4 @ V W 1.1W IX "I1 _eX U ,. , I fi I ' _ O *_ : 4b I 0Lx 0141 aU 0 141 a 1 E 11 3 a, P1 w o I -X 4-1 V 4~ 2C XIII 0) OW .0 0 Cj J _ I I W* ., 14. za 0 W-.0 IIO-1 4 C oV V4 4 'I 00 O AI 0 j' X OX .0 -OL LI o Vc urcu_4. mI ow, -4W> 0* 10 .0 O E _4 0 . 3 -I m Xjji i w C .- 4 Ea ,,vZ VO - > UU ·a l x au a EH - .. a11 I J ) L) _Eq m a A 1[ N cu 6 eu x D l3vI - I.14 U ear H X vI1I U II 0. X V Z ,' iS X " .1 i, AY 4 V i . A 11 ,> - '1 v - 0 - IIV . vx c iD *11 41 v e v 3Q Z ,^ a ] oo I v _ D, . x v U i , iz U a _ C_ vI- C n: H v A m 11 WI 1, 3I _ WW *cW _ m _ ,l 00 D- lu c ) _- 4 I Z o Iu Z 30 XN I1.ul 1o mn U O -l v s v .. II -3 .. O C v xl xxxV , v rD es Z#v ,- 4 WI VI X' 3v "f 1 .v Il v I 00 44 I i <XN ,x x X v 0' :I: I or , 01 I , 1, oI :D r m .. 1 -1 0 1 '>1 I f 4 l IW , 0 r c O _r 44a 1 >11, ... . 2, _ 4 u.. i I) r0 2 E4I ~~ OI I I . 4I 4.) , C I' D is 'iW ~ -1O0 0 I u>e D J=" 4u Oe m:_ D OP W 4 kX )urn4 {3 cm i W . )ov C :o ,,,- . *p 4 -01 94 O 00 4 0 ;D .u -- Oix 0 -- 3. Yu pn Orl " C 14 _ _ _- rn E oo o - 000.4 nI C) 141 0 --u u -V V -1,1,1, Df 4 4 _ 0.0.3 _ D1 -v _ 01 ' a 11 , -. _e I _ 0 .. , I I I . z * r xiW~x C o u _.l1,-A _I4C C u c L IV Ir 11 IXII 41._III_ 0. 1.01 _ VWDX Xu ,3 .1 4 I x -1 m - DI" I . N an o ->I v) .o"I >10; "I41 I .4 1t tn _xs~V 4,11 - nx 1I 011 04 1111 u 44 ... -- 0101 r 04u _ 00 44 00 0 44 D .-> ..I , W_ 0 O I a44VII I .fWi.-0 *. u r>0 >0 0 U0 0* --0 uDO C _ _ rD rD D 4 44 4 XxX .4 IX II I I uD v fi p. x Drn . 0 ' 4I - o x _ I , ' x 4 v- - 0c u 0g 0 xx X X X X CIII C. 0 - Uw W .o * A,n V N W o o a xx 2 xx go C Z-* 11 * I - ,1U ~- WW N _ lu.,* Q? C o " ow X U .N _ b% aX X U A_s 11 xx In__ in OCV- 9 b a v>v b ainiiU Li XiU * A4fr1S a-I cti0 tow 3 Usii I~eD 1 Xi--_ A U gUy aa UI I a L 9 a n --a U C _4ui a > _ , w l U U82 J 01W 0W 0"1l1 a Li a uI ", _I Iola b b 1O A ini x 0x 01t Ul ,, I -N in i-inai-i o-- X nu 1l II a II~~~~~~~C~ X xH 0 a 1C aeiii e a X f Z _0 C U aU Ub uVU C n c> wCi1 enC IlA a U a ao gau-nc Cb S X u acA5 i g b X ZS ggg U U U Is .. z~ Za .C.. 11 I i uuZ = . _. 111 IY5 U S u Y , U 1 guon8 o8 ,o N I~o UE1 x b _ _tA 1IN-xto wx I ., aIIxA ffi d n X 0a' . am - u u u 3 0 8 aC W- 01 a Ul Zr a o - > D 01 1 _t 0014 a> Sf J a I 01 ZM XIPI - I II An NCZSN > X~ _ V .M ax _C N goa CC m-n Cxxx X aaW WX ai __0:aa Ca1 010101 a a 4 b .L, O xbge xa - a .0. dr. XX -o _I Loia 0- _ u w m gInU on Un i XAXo aiUii 01 No Li-n>U *0VLw 11 CA C X a X 3 I I W W vb XW e_ 4 I a *J * cn u a01i eIm~ :94 z (4go- m >,VV * 01-nUC CNN ebX x ia ca a 5 W b bL q X X If I S X m _ _ II N4 m__ xx u C )c Xo __ @ o .a " W a IX _ X_u A $l _ WW 1 U .. 01 a° X~ C N 11 a PI Snuaaa, go: a U >,gola_ <S 4_ a C iZ CA axV a V CZ 01 -1 C 14 _ _ C w D9acm~>U wc ffi · I N 1 NC0:1 c 3 o cS a i i PILX b4 X X 1LI .. _ D .. s. CU C m i n8LI~~~I _" ZY A , Xu LiC x 01 fi aa CC C.a C O 0 CO .4 X >aa-AU Liii C 2: c II gog c On bna E4 a h E1-v 010 w 1h S CV 013c-. a. C>._ g 0 0 - 1- Ouu O O _ o 11bS ~U)a X 01 C,}- fi XX 5iniUC C) anY WOC&aC1 aC-iCUW Uy i N .0 IP I X ma CI XI 1 N a 0.i a X X I I Li I la '11 x I o X H a~~~~~~~~~~~ Wa.crl f C.U El~~~~~~~~~~ L.1 41 I I o I0 -- *1. W2 4~~~~~~~~ I I ~~~~~~~~~> I > 'IO u UU CC 12U 1. 1-l "I ~ffI IC C I IC. u I *II . CI I IUIL--- 1 flUV uo '( I I WIC) ol~~~~~~~- , 2) I WW, U, ... CC 2 ' . I I X .4 ul u iC oJ II ~U C1 C~~~a uf A -, . . 2- 'C - 'o"~~~,,,,,, C2)2 2)2 H C 0~~~~~~~~~~~~~~~~~~~cu · 0 u ~~~~ ' ~~~,-i" .1. >12~ C)I 10 II I ^. -Cu Ix> C~~~~~~~~~~~~~~~~~~ x 1 ""WOI 00 00 ~ l >I,1. C 0 N- U . ... . fl., C .- ,> >51 3 xZ 'l " t N .12, ~ux'x~ R N 'a a 2f W- uu . W j :: ~' x , 0 u~ u I~~~~~~~~~~~~~~~~~~~~~~~~~~4 C.12Nfl *X W LI C. X or ~ii! u 0 ' W w - uI 2) ux C (0 fUI 1W14 U la Cx LILI C" ,, IC C. C Y 5~~~~~~~~~~~~~~~ ,.4~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~l 0 . LI~~~~~~~ rlm r~ I -r o X O N - a m O ' ' ·· ° 3 :: C- _m~ t r m E -- 4 3' a' -ae e , _ 3s O 11 Z V) _O I u w 1 peq o 0aLrl to Pc uo '- 1,a 4 EC I zo 01 u0 0 a~~~~~~~~~~0 uC c - _ u a 0 o e I a 3 : > t4 _ _ u a~~~~~~~~~~~~~~~~~~~C S ° Oue w a m e cl o Y o .. IA · .4 11 a *-tn -. I 01 vv I UW-4>. boaI -3 3 O3oW .. -. -OVV 0V W···I IDV I1W vC- W u t-. : : a 0bt>.-u- 0 · P .- i oa W o 0. ~ ~ -. l tOOI I- ~0 0.. . ... 1>. t.I( >,U vUa v. - U - V>I I 0 1.m-tWO o..lu~0Wat~av0Wmna0-0WW.-.b-t eW-_l. .b.WaO..0W I .IL. 0 0.-t zw -. ,. I i flIt- 00. oI I ._11 _ a.-.- r 11: 0 -vv 100 -0 - u ou1 -L *b.I_ e-a D-O I.0.0.4A- o t=X ,:o V U I v 4 tt-t**tb b...-..blb-. u > qH.. V.. u > u 1X_ lpe' l 0. c u *X @X W I ua.-W-a -. 0O V VW > U u >. I.a lob.( H.. X:: _Z. -W U ; - I WWV _u > '- 00 0% co a VA - V Im 0ID .. 11-4 rl o4 a n . o o a40. aC; I ce V . @4 . mm I~ e -.111 I- 0o ,,3 a - _0 I I- a -n on -1 I . -1 t..i Il I-4 I U) 4 _Y H - o .. 0*0 a* -- 0- a C ._4 .0. 0 o-aa 3 1 0000000 V:::V- i C ^ o b , 0 I-b 4 0 0 0 · C a4 II O m 0I 4 1 0 b n·^ nn4 aaaaaa 4 uale---- a 30 II -0 no at. V VV n n OO o C _ i C ^ b.. .-.33t.C I I Ia 33 ._e _ 4 .^ _ a ,, _ 00 10 mu - -3 a P) 4nnn ue ea I - 0 E1 -E _ _t .aa u bO O-3 0 P v- W b..ba-I I I r( _at1 *-..-,4 IIII 4 r. __ a 0 . a uX a ea n 10 _ 3 Ju a 4 - I _e( m 4 *| .1 - _-to -t 0 e _ 14 b. O I 0 - _a a en I_ n 0 a 0 a I ·I - 4 @ 4 .0- ... - I~~~am~ P -o 4o . e s1 a, C 14 I e D 4X , I, ,4 4 a .. 40 o4t a4- W W W WX r era lb b. 0 ; onI IIN -0 lll.a o aI a I a I ;P I 11t a CC11^ ' ,8,,,-ao- - f I a- ab.( Ua U I en a__ 0 a 3 ec~lu~- n 0 aI ei 0- c 04. 0 0 ;z 0 - - 11 v.Z - 11 II o IIV OL 1 0,4 0. 3-- v- 11 a Baw > aw >w c e "X OA _ > u eW~ - ac .i 0 a I V V i-e 1 VV >S -IL. I I 0aa I _ 00 0 W]Wv^ -n ea ,~ O~ b-:A I ,_VC 403343,, (14 a' .4 o : . 4.- a 0, * II Po ce- A _' Cl * . u 0 · a 4.. o E R Y: ,U 0 e I ,u u ,-o A O bd. _ 41 - . U v 0 Ie _.. - A nf4 ' q _ A 0 A 40 . .- C _I- .._.. >: Ub % eq -_ . v -. -o A Y # v _ _ s >w~ b _ _ e a* ,cX - i oe_ b0..4 U .a ., _ 1b _i- - m 4 I I v A 4. A C. PIool l 1luul es l l oe A%_.o V U u I N~~~~~~ O1 c' e crv . o *` , 1 u A Z 0 I~ .4-4 + I A ...0 I I444.4l0 v - ..|b__ b U I-4u0 "u | v |1 " · b .. _g ~,,g : 0 44 II c n I00n0.-- I ,4.0 _ ~ >" . n l.. 04 v 1/,. l VVU . 110g qu"" _14. NO v 4. b _ 14 I b_ 4.b 4.*.4 4. l-oi :t l -._0 _111 OOOV b1 000-001 4 1e I. 4 4.00I~ 3.0Yr 0 041 S > -4-1 0--1-40. -~ 0 U u >g _e be 4 a oa I. o - pio 0 0i IJ~ o 0 ot 0 s0 3 * a 3 ~ -U402 .. - 0 0 00 01 C 03 0 08 -l .A 0. 0 Y '0_- o 0O V~ i . 0o> Uf E · 0I 4 O~ I 0 0 * 0*-* . 0tuL0.. 0 0)X 000 0 E u~ a4 _ 0. _ 0 OC"0 u _oo= X tS --o ..r a. o o.. . 0 E E E E u0. 0 0 V 0. I I 0w3-. 0 -o >.^4-1 z t0L 0v D C C .. ..- 400 000~ *.-.CL. 00000 00000 00000~ -o S · XI · 0 4c E Eu at fi U ." . s 0 - c * _o .a0 00 ~ *. - -, O .... oo C C u ."I2)C . -'_ *.Uo-o 0 X w ~ .. o-05 CU O E0cII II - mnLI X c a - ' I_ 0..I _ W E - X -i@ ~ - _r 0 3- c 0 0 V *0-. _ sqX to-aw_*seu E a-c uaD . 0-n .. c - _C * E .. r I -nL 0.E 0uXo^ J II & C 0- 00 o 0 V- - -2 00, m0t I 0--V 4 u O tt E. u~ I 0- 4 11 _>) oC " _ II _ a ~~~0 ~~ V mI at O ~ c 02~~~~~~~~~~u ~ -i ·- 0 0 - s m- u a. c4 0 , a c 3m C 39 e.~~~~~ ~~~ sC~~~~ m '0 0 D .0 Q m210n C c01 c - 00 00oo 000 0 00 E a . o00 -- .- 00~~~~~~~~~' .D-' .o .o000 ^ ^W -o o o 0 · C 01.* oa, . 00 . e 0.1. ·. 3 II -1 i _- Ii I w av 00 C w . I o 1 1 6 . 0-I 1 wo 00 0 w w_ - _ _ *. 0. ww .c .fiw C It V 0 0t- .0OOVVM ~~~~ ~~ ~~~~~ ~ ~ 11 . _ v0 v v I I1 UP) - o C - Iol m W I 0 L 00060 .m I .- .0 w. ^c 0 .-.. .0 V^ Cc~ a I a - X >s I 10-- 3 t3 >s 3 "c D W3 a X W W c m ii P1~ 6,00_0-~l - 0Vo E_. -. .. fi 0. He1 I- -fi~l *l0 Ol;o -A s _ J6 I 0 ,,U e 01 a I u II n : at r4 .. -1 0 -1 0 0 ... 0 , 0I .. .Q 0 O 0 0 O I I u 01. 0 a.0 o ol IE aO w o 14 .a2 > O r·a . 1 4' 0. .. . O. _ X* -o · a .00 00V a I 00. Z.0 C _1-.D 0 a Vu a- e . 0 0 ' V 0 1. E C. C0. D O . D " S _ Io _ V X: o _ k oI~i L) Q 00000~ o .. . 0 0 0 >:v 0_ 0 .0 1. 4 - '0 W1 0 0, 0 - aa D -. 0 0 P C V V1V u 0 30 -0D _ tO m - ~ I 0ii 0 o 0... -000 O 0-00m.0l01.V 0O0.0 0.0 s Ic_ Ic _5 b. 1 4~ _ sg 4. " _ .- a b F· II oo N a O 01 II II C C Vmlu 0'C 00"0 OL C C 14 V 0V_ H 'I I o r o W ~ _ 0*-,,' D _ ; L, ;, I I .. I I , ID a . ii 4; ·e " .0I 0--. a- I 4I I I I 4 I I a D _u~ a, u·0 D e I n 0 la E wf O tC 0* 00000 > C 0 Va Q, 0 o VI _ _ , _--·- .. -0_ ' < - 0-0-- I C 0 _0 H06.1 o-2V naI r al o 0-0 oo 0 *. 'Q C - · 0. *o Du~ O a ahC11, u 0 0'0 a3 3 o . .., .... ~ 1 I . 0I 0I .0 11. 0~ v E- 0 ---- . 0 rI - O0 o o t 6> 6- .... 0-00 0 -. W I mr I U . e 0 t4 0 *)O Es .... _00 o S a w. . .. . 0 00000ld sV e X 3>3 - " . -~~~~n X U 4 V In I-D' - I In e IO -In uo w 4LI I :uu riX -0o I- D n D b4 - In D N I u ... NI X Xg I fi n .0 . I o fi -- 4 1 *c 4 13 3 4 -i 3 3I 00 · "I 3 0I _u .1 ( l0.-n I 4wD 4 4" ..40In4n03 In >lO10n 0 >-0' In D X 3n o _ a4 n_ I X - :: 30 lIID n 4X I-In 00. X In n I W 44 4 33 3 ¢3 I-I -In N 0- 0n I 0 In Cl-00 XI In u3 O n '3C n0In n I > C I I, _ i u-~ uI _ .00D 3 000nu u I1x I >.In I-IIxfu >. In - - 1I003 I I I I ~ IDn In I* nDnDnDI4 3 nI In C c I Xn '3U 0I Xn D .II Ia X-I '' N _-_0 _I wI-n I o D -' IO-1CI 00 I 0 Iw I 0 D X 00 L, -. -. 0_OO I. n Cn I0 I I 0i n _ I 0 1 0 _ 0 I In I I L, C~w I~ -X 0 1 0 D10 I IP_ _ 000 - Iw I u0 XX Q4 X e0 e- I13D1 000 O w ; O _ · U 4 I W) . - I-.-_ 0 * 30o0 I-_ n.--_ D3-O w Ie 0n 3 · X -I ~~u . n 30. u o In0 D; 1z. 01_ ^ X 0l 3:1 n 0 II .0In 3nIn -. I 0 -In I I 0 - I In u-0 C 0 .V3-I .0 0 1.-n I 3 .*- ._ I n lO -In ~- 0 .0 on3 .0- - 04 , u. n -_ ox :t J0 0.3 10 -0 e _ DI · 0041 30I nD D n > ._. I u~ -33D0 - fi I - I4 I-~ I In -3DIn - -X _ II I 44 3 no IIW X _ .c _ . .. 33_3333 3 u 3n 03- _33_3 44 a 4> -0InIn W E4 1 0I u X4 I C1- I ,CI~0 e~lo 4 4 u uO~ In - en -0 u 00O DnID 00 . . >. . a 0.0 - n n I.U u lI e N00 14X _ InCln _^n IInX ,, 1u Wn~ I In I X In X 0 U I U -. InI D10 gn .0 I 3 lo3 n I >. 0 -. o04 eI _. >-u Oo 4 0u _x I -- _ - DI 3 -0 10O In D C InI Inr e3 -333D I uXIn X >, I U 3 0..0 C0 aC 0 I w IX O - ·. W 1 0 '-4 0 In *0 0% H - Ch O m .. 0 >r u 0. ID u In In c n A a o - 0: P n oP. 0 01 uu x10 l N 0 O I I I X e, 0 31 _ -u _ c a 0 O I - c33333 0 -^O 11 33 *- ,.-0 I 1t _~ w w-I-- 4 > > In oI ,w , 0sIn .4 I k. la I In D v EO . In D .0 0. _ -. 0 .I 0 IawI I.*-OI I u D3u 3n 010 00 I n X0-S 0D 3- In 0000DD 10 -33303 > n1 3 3330 I 3 IW.- o rr Da I u O --00Dc -33333 3 0 0 0 C oC I 00 ~o o --- 0 E~ EE4 D DC _~ _ o _D e o o~ I ...... CI - 4 - - I- :>.>, 00 InD Du 3010 - 3333 -3 In …... .. _---0D 3333 D C3.1 D W D D > 4 a _ 33 . . 33 3 33333 0 3- >i I 33333---33 In0. :s, CIOR0 H 33 ~ .^ 33 I In>. 13 Iu v In I I >w> 0 2 W _ -- 0I .H . .I -. I X 4 'l I E1 *-4 Xl eI e11 IIn---0 n E1.I 3i - :·o e E II1 ElE In00 In1 InI D~~~n -33 D o D 3 e e X I 0In -v 30 In D 0 -- 03 D 0I vI 3 0u I I Inn 333x e D0 0.c - In I InE E -'DD0 00 43I X 3 o n I Z-. <- - 00Cu D un OX-303Cx 00 -- u- o0 Inn~ D -- -- - 33 CX -: ., 3 3 3 - 33 3 3 3 1414141414144 i I I I 0 010 I140101 I AAAAA in.0. NVNNNiOIN o M33ni3 o3 A"114 MNI I In 00M0 I I 0 000M I e 14 X x i X 4 4 L .00 * I ,, . _r .o . W 4 W 4 . zr s . ·4 an; > 4 4 i > @ W Xi L I 1 I I iN N N N - _ @4I rm apeuc s.m X . Y 44 X" .. S Wu i ar eX e w 3 I dS N ee d 0 de V 1V la XX"" I * X la a V. ., X 0 0 010101 *i0 i IDI i, 1~L21 VJ- I 1 1 I 0 II I I d , W.. .0 I 41 C3 0 *2 -r U . 0 .C a _ P . fi nl. 1 e I4 I .0 P ' I-. 4 Ue I 0 1 1 0 1 * >. * . -01 I 4 . 01 1 0 1 14 1 ° I 110. . 54 54 0 0 4 . 0l O . 0 X) C in 14 c xe 1 P o @ l0 _ I aO>. la , C - 19 a~ 01 l..1401 1430100 ~~~~~~* I.1... .. ....... OO O l. e o .4 I. - os m o)(DPO o . 0 .nD 0 e x O 01010101010 4 0 in X 01010101010101 - X 14141 C441 I o . ,, .. 0 14 0 1 l114u H 1 4.. V S V, 0 in I ela 000000 0.010100114 I I " I IeI W e >.>.>.->.o 001 1a4 1 up 1 1 > _ X n *1 V~ A m >,01.014~· 4 . * s 2E-. C.4 . 014 01- 4 in , , . 4 , .9 . .X 0-0-0x * . - e) VV -" I W Ul 01 01 0. . 11I 4. 0 11 1 04 .0 10 ci n Q-~Y .>..0 . .i in .V- in ml X _0 C 0 1 01 01 in444444444444n4i 1 4 >. U. I 14 0 I x x0 :J v 0. °C (l 1414 X '^ 44 LJ0 0 I XI XI XI 4 WWW 1 1 V 0 1 0 1 0 1 0 1 0 1 0I10L~* 01101100 141414144141 .1 01e * u eC e X X in 0I 41.in141441144 1 01 1 4 c S X 14 .0 .14 010 V.. - 3 0 X 4 . i0 .n- 0 01 14 HW >1 __N e VI> _ 4 D_ C X 0 0 N N N1 1 41 0 101001011011010010 uI111111000000 V 0 01 0 0 oU 0 0. Cw ' 0.CC O 0 01141414 3 .0 _4 14a4 e X ,-X4 0 U PI I_ - 0 0 51 0 iw e100. 1 001 J ~~~c '0 IW .. 01 4 4 in 0 0 .10f .i 0.a 10-0 0 I 0 .n0 II I '14494300 I I I I I 0 0 0 0 0 0 0 * >011 014V u 00 0 .. 0 I XX V II H - . Ee eE a 0>*s _-· _ S gil a,>I gX I V ,, 0. l>.>.I .l. I C010 1 3 3 11 . * . W. WIGe~L . 01 u u 0 _ 0000000 * . I. ~i _ 0~ . .4~_ H I 1.40001 I 1 @ @ ' 1 01 10. 4X II 0 "I 1414. | | u u-^^ I I 143W 0 e@ V- 4 I I IL1 0 .. 0 0 0 - -I- o 00 c o Io I8 10 i= . AL. - U U U IVIVI010 01 A U- . 0 s . 1 w X" .. D a. .. 10 44 Q3 ... .WUWW0 I N S 44 " I NN e e ez ee E E lSe e ee ee 11 ... ' e* 01 z 3 1 3 la3 3 3aX X X31X 4 W X3 . 1 . l Ql 0101010101010101010101 1E1010E101EE10101 n01001O-E N Ih .tm4 * rC@ *W 4 _N .l - _m l D ._ . S Ul L0 WinO - . wH @ 0010 l0 W 3I W_ ~O I 14 O0s1WaW0W.. . OMito I... ml a, W I WI inI 01 W II in WS 01II 01 W * PP ._ Sm -r 0IM ... ~U S 8. O 001 01Win_- i X o * i XI in ll I I ininio 4 @ @ * X 3 WA a a W0 414 * IIE a MA 01uo 41 X 4 010104W01b1W *1 X X X 4 .e40 .0114. CO. I o . O. .000 .. 3 0 : ol xuI, . o0 10 .0 o 0% ,,u -; L. 0 uI U. U I I U XUlI U U X X X u XUU oo . 0 H Ot in u . . .0 u .~X X X P IOU .u 3 . _I U.N'N II · .. u ~,1 -o -1 UI; I o "° ~ XU · U I I U U UO X Ut U u UIL) . U. XLXU XU X XI uXI oUx> uI I I UII xUI xU XX X U goU IU u I JX H LzUU U uu z; 3 LiU XU L YlU X . . a, *.. )O uXlU -0 OU Li X U a, ok .< 3 . oo0'. X o I LlU . .0 00 . o U N 0 d .S U o U U U U 1^ IUl U UU X U U L) U UU U U X X Y . U U l " U .0.0 P Ul U X Ul *X 4 *0.. I I . Xx. . . . .0 .0 - J.l elp - 14 _ :; aam frn - - r 0 I I - '.'. .0 I .· · *-1 -. -_· Oa W> ^ :5 X1 0 0.0~~~a .- I00.0 .0.0.0~.0 .01 -. 0. "U 0 . i .00.0 ~~~~ - -U *UO I I II xx . O: x u .· U O X I x x xx .10 .0.' I I U v> 0" 00 . 0 -c .' rr O~~~~~~~~~~~~~~~~~. I . "Now to Him who is able to do immeasurably more than all we ask or imagine, according to His power that is at work within us, to Him be glory in the church and in Christ Jesus throughout all generations, for ever and ever! Amen." Ephesians 3:20-21