MUHAMMAD AL-XORAZMIY NOMIDAGI TOSHKENT AXBOROT TEXNOLOGIYALARI UNIVERSITETI Assignment - 4 Group: SRM401-1 Student: Matyoqubov Fazliddin Teacher: Xoldorov Shohruhmirzo Toshkent 2023 Software of parallel computers - Architecture of parallelization systems, MIMD architecture Parallel computing involves using multiple central processing units (CPUs) to solve one computational problem. For many years researchers have used parallel computing when working on computationally challenging problems such as nuclear weapons simulation. More recently, economists have applied parallel computing to large-scale matrix problems (Nagurney and Eydeland, 1992; Chabini et al.,1994), the solution of non-linear dynamic models (Coleman, 1993), and simulation (Liu and Rubin, 1996). Nagurney (1996) provides an overview of the application of parallel computing to economics. In addition to these applications, parallel computing can be used to reduce the time it takes to solve estimation problems in econometrics. While this review will not discuss the details of the estimation process, a simple illustration is helpful.1 Consider estimating the parameters of a maximum likelihood model with data on 4800 independent observations. Suppose that a four processor Unix workstation is available to solve the problem, and let Ts be the amount of time it takes to solve the problem in serial. If the data can be broken up into 1200 person subsets so that View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by The University of North Carolina at Greensboro all four processors are used to solve the problem at the same time, the time it takes to find a solution may be approximately one fourth of Ts. Von Neumann’s architecture was first published by John von Neumann in 1945. It is based on the stored-program computer concept where computer memory is used to store both program instructions and data. Program instructions tell the computer to do something. Central Processing Unit: o the electronic circuit responsible for executing the instructions of a computer program. Memory Unit: o consists of random access memory (RAM) Unlike a hard drive (secondary memory), this memory is fast and directly accessible by the CPU. Arithmetic Logic Unit: o carries out arithmetic (add, subtract etc) and logic (AND, OR, NOT etc) operations. Control unit: o reads and interprets instructions from the memory unit. o controls the operation of the ALU, memory and input/output devices, telling them how to respond to the program instructions. Registers o Registers are high-speed storage areas in the CPU. All data must be stored in a register before it can be processed. Input/Output Devices: o The interface to the human operator, e.g. keyboard, mouse, monitor, speakers, printer, etc. Parallel computers still follow this basic design, just multiplied in units. The basic, fundamental architecture remains the same. Parallel computers can be classified based on various criteria: number of data & instruction streams computer hardware structure (tightly or loosely coupled) degree of parallelism (the number of binary digits that can be processed within a unit time by a computer system) Today there is no completely satisfactory classification of the different types of parallel systems. The most popular taxonomy of computer architecture is the Flynn’s classification. SISD Conventional single-processor von Neumann’s computers are classified as SISD systems. Examples: Historical supercomputers such as the Control Data Corporation 6600. SIMD The SIMD model of parallel computing consists of two parts: a front-end computer of the usual von Neumann style, and a processor array. The processor array is a set of identical synchronized processing elements capable of simultaneously performing the same operation on different data. The application program is executed by the front-end in the usual serial way, but issues commands to the processor array to carry out SIMD operations in parallel. All modern desktop/laptop processors are classified as SIMD systems. MIMD Multiple-instruction multiple-data streams (MIMD) parallel architectures are made of multiple processors and multiple memory modules connected via some interconnection network. They fall into two broad categories: shared memory or message passing. Processors exchange information through their central shared memory in shared memory systems, and exchange information through their interconnection network in message-passing systems. MIMD shared memory system A shared memory system typically accomplishes inter-processor coordination through a global memory shared by all processors. Because access to shared memory is balanced, these systems are also called SMP (symmetric multiprocessor) systems MIMD message passing system A message-passing system (also referred to as distributed memory) typically combines the local memory and processor at each node of the interconnection network. There is no global memory, so it is necessary to move data from one local memory to another employing message-passing. This is typically done by a Send/Receive pair of commands, which must be written into the application software by a programmer. MISD In the MISD category, the same stream of data flows through a linear array of processors executing different instruction streams. In practice, there is no viable MISD machine In summary, MIMD architecture forms the basis for parallel computers, and the associated software infrastructure includes programming models, compilers, libraries, operating systems, development tools, middleware, and more. Effective parallel programming requires careful consideration of both the hardware architecture and the software tools and techniques employed.
