Distributed System: Definition

advertisement
Distributed System: Definition
A distributed system is a piece of software that ensures that:
A collection of independent computers that
appears to its users as a single coherent system
Two aspects: (1) independent computers and
middleware.
(2) single system
Machine A
Machine B
Machine C
Distributed applications
Middleware service
Local OS
Local OS
Local OS
Network
01 – 1
Introduction/1.1 Definition
Goals of Distributed Systems
Connecting resources and users
Distribution transparency
Openness
Scalability
01 – 2
Introduction/1.2 Goals
Background
Developing Collaborative applications over a collection of mobile
heterogeneous devices and data stores.
• Autonomous and mobile data stores
• Wireless (or wired) networks of various characteristics
• Devices of varying capabilities (pagers, cell phones, PDAs, PCs etc.)
Limitations of Current Technology
• Explicit and tedious (data and network) programming of applications on each
device.
• Multiple types of heterogeneity of data stores
• Poor support for maintaining global consistency of data stores
• Poor Middleware support
• Difficult peer-to-peer interaction (no data serving capabilities)
• Poor or no support for dis-connectivity, location independence, group
collaboration, atomic transaction, QoS
Confidential & Proprietary – All Rights Reserved
Internal Distribution, April 2003.
System on Mobile Devices (SyD)
An Integrated Programming and Deployment Platform
• Uniform Web Service view of device, data & network
• persistent object-view of mobile data and services.
• Rapid development of reliable and portable group applications
• high-level programming and deployment environment
• Leverage off existing server applications
•
•
•
•
Peer-to-peer and distributed applications
Group creation, maintenance, and manipulation
Quality of Service, while handling mobility and dis-connectivity.
Footprint: 112 KB, only 42 KB device resident
Confidential & Proprietary – All Rights Reserved
Internal Distribution, April 2003.
SyD Kernel Architecture and Interactions
1. Lookup
SyDAppO
Server
•SyD Kernel modules developed in Java.
SyDDirectory
• SyD Directory provides user, group and
service publishing, lookup service, and
intelligent proxy management.
3. Register
Globally
SyDListener
2. Invoke
2. Lookup
2. publish
SyD Kernel
SyDEngine
Client UI
1. Execute
SyDEventHandler
3. Remote Invoke
SyDLink
TCP/IP
Web Services
SyDAppO
SyDAppO
• SyD Engine allows users to execute services
(can be group) remotely and aggregate results.
• SyD Event Handler handles local and global
events.
1.Invoke
SyDAppO
• SyD Listener sitting on device enables
devices to act as servers by listening to remote
invocation requests.
SyDAppO
• SyD Link enables an application to create and
enforce interdependencies, constraints, and
automatic updates among groups of SyD
entities
Confidential & Proprietary – All Rights Reserved
Internal Distribution, April 2003.
Distribution Transparency
Transparency
Description
Access
Hides differences in data
representation and invocation
mechanisms
Location
Hides where an object resides
Migration
Hides from an object the ability of a
system to change that object’s location
Relocation
Hides from a client the ability of a
system to change the location of an
object to which the client is bound
Replication
Hides the fact that an object or its
state may be replicated and that
replicas reside at different locations
Concurrency
Hides the coordination of activities
between objects to achieve
consistency at a higher level
Failure
Hides failure and possible recovery of
objects
Persistence
Hides the fact that an object may be
(partly) passivated by the system
Note: Distribution transparency may be set as a goal,
but achieving it is a different story.
01 – 3
Introduction/1.2 Goals
Degree of Transparency
Observation: Aiming at full distribution transparency
may be too much:
Users may be located in different continents; distribution is apparent and not something you want
to hide
Completely hiding failures of networks and nodes
is (theoretically and practically) impossible
– You cannot distinguish a slow computer from
a failing one
– You can never be sure that a server actually
performed an operation before a crash
Full transparency will cost performance, exposing
distribution of the system
– Keeping Web caches exactly up-to-date with
the master copy
– Immediately flushing write operations to disk
for fault tolerance
01 – 4
Introduction/1.2 Goals
Openness of Distributed Systems
Open distributed system: Be able to interact with
services from other open systems, irrespective of the
underlying environment:
Systems should conform to well-defined interfaces
Systems should support portability of applications
Systems should easily interoperate
Achieving openness: At least make the distributed
system independent from heterogeneity of the underlying environment:
Hardware
Platforms
Languages
01 – 5
Introduction/1.2 Goals
Policies versus Mechanisms
Implementing openness: Requires support for different policies specified by applications and users:
What level of consistency do we require for clientcached data?
Which operations do we allow downloaded code
to perform?
Which QoS requirements do we adjust in the face
of varying bandwidth?
What level of secrecy do we require for communication?
Implementing openness: Ideally, a distributed system provides only mechanisms:
Allow (dynamic) setting of caching policies, preferably per cachable item
Support different levels of trust for mobile code
Provide adjustable QoS parameters per data stream
Offer different encryption algorithms
01 – 6
Introduction/1.2 Goals
Scale in Distributed Systems
Observation: Many developers of modern distributed
system easily use the adjective “scalable” without making clear why their system actually scales.
Scalability: At least three components:
Number of users and/or processes
(size scalability)
Maximum distance between nodes
(geographical scalability)
Number of administrative domains
(administrative scalability)
Most systems account only, to a certain extent, for
size scalability. The (non)solution: powerful servers.
Today, the challenge lies in geographical and administrative scalability.
01 – 7
Introduction/1.2 Goals
Techniques for Scaling
Distribution: Partition data and computations across
multiple machines:
Move computations to clients (Java applets)
Decentralized naming services (DNS)
Decentralized information systems (WWW)
Replication: Make copies of data available at different machines:
Replicated file servers (mainly for fault tolerance)
Replicated databases
Mirrored Web sites
Large-scale distributed shared memory systems
Caching: Allow client processes to access local copies:
Web caches (browser/Web proxy)
File caching (at server and client)
01 – 8
Introduction/1.2 Goals
Scaling – The Problem
Observation: Applying scaling techniques is easy,
except for one thing:
Having multiple copies (cached or replicated),
leads to inconsistencies: modifying one copy
makes that copy different from the rest.
Always keeping copies consistent and in a
general way requires global synchronization
on each modification.
Global synchronization precludes large-scale
solutions.
Observation: If we can tolerate inconsistencies, we
may reduce the need for global synchronization.
Observation: Tolerating inconsistencies is application dependent.
01 – 9
Introduction/1.2 Goals
Distributed Systems:
Hardware Concepts
Multiprocessors
Multicomputers
Networks of Computers
01 – 10
Introduction/1.3 Hardware Concepts
Multiprocessors and
Multicomputers
Distinguishing features:
Private versus shared memory
Bus versus switched interconnection
Private memory
Shared memory
P
M
P
P
M
P
P
P
Processor
01 – 11
M
M
M
P
P
P
P
M
M
M
M
P
P
P
P
P
M
P
M
Switch-based
M
M
Bus-based
M
P
M
Memory
Introduction/1.3 Hardware Concepts
Networks of Computers
High degree of node heterogeneity:
High-performance parallel systems (multiprocessors as well as multicomputers)
High-end PCs and workstations (servers)
Simple network computers (offer users only network access)
Mobile computers (palmtops, laptops)
Multimedia workstations
High degree of network heterogeneity:
Local-area gigabit networks
Wireless connections
Long-haul, high-latency connections
Wide-area switched megabit connections
Observation: Ideally, a distributed system hides these
differences
01 – 12
Introduction/1.3 Hardware Concepts
Distributed Systems:
Software Concepts
Distributed operating system
Network operating system
Middleware
System
DOS
NOS
Middleware
01 – 13
Description
Tightly-coupled OS for
multiprocessors and
homogeneous
multicomputers
Loosely-coupled OS for
heterogeneous
multicomputers (LAN and
WAN)
Additional layer atop of
NOS implementing
general-purpose services
Main goal
Hide and manage
hardware
resources
Offer local services
to remote clients
Provide distribution
transparency
Introduction/1.4 Software Concepts
Distributed Operating System
Some characteristics:
OS on each computer knows about the other computers
OS on different computers generally the same
Services are generally (transparently) distributed
across computers
Machine A
Machine B
Machine C
Distributed applications
Distributed operating system services
Kernel
Kernel
Kernel
Network
01 – 14
Introduction/1.4 Software Concepts
Multicomputer Operating System
Harder than traditional (multiprocessor) OS: Because memory is not shared, emphasis shifts to processor communication by message passing:
Often no simple global communication:
– Only bus-based multicomputers provide hardware broadcasting
– Efficient broadcasting may require network interface programming techniques
No simple systemwide synchronization mechanisms
Virtual (distributed) shared memory requires OS
to maintain global memory map in software
Inherent distributed resource management: no central point where allocation decisions can be made
Practice: Only very few truly multicomputer operating
systems exist (example: Amoeba)
01 – 15
Introduction/1.4 Software Concepts
Network Operating System
Some characteristics:
Each computer has its own operating system with
networking facilities
Computers work independently (i.e., they may even
have different operating systems)
Services are tied to individual nodes (ftp, telnet,
WWW)
Highly file oriented (basically, processors share
only files)
Machine A
Machine B
Machine C
Distributed applications
Network OS
services
Kernel
Network OS
services
Kernel
Network OS
services
Kernel
Network
01 – 16
Introduction/1.4 Software Concepts
Distributed System (Middleware)
Some characteristics:
OS on each computer need not know about the
other computers
OS on different computers need not generally be
the same
Services are generally (transparently) distributed
across computers
Machine A
Machine B
Machine C
Distributed applications
Middleware services
Network OS
services
Kernel
Network OS
services
Kernel
Network OS
services
Kernel
Network
01 – 17
Introduction/1.4 Software Concepts
Need for Middleware
Motivation: Too many networked applications were
hard or difficult to integrate:
Departments are running different NOSs
Integration and interoperability only at level of primitive NOS services
Need for federated information systems:
– Combining different databases, but providing
a single view to applications
– Setting up enterprise-wide Internet services,
making use of existing information systems
– Allow transactions across different databases
– Allow extensibility for future services (e.g., mobility, teleworking, collaborative applications)
Constraint: use the existing operating systems,
and treat them as the underlying environment (they
provided the basic functionality anyway)
01 – 18
Introduction/1.4 Software Concepts
Middleware Services (1/2)
Communication services: Abandon primitive socketbased message passing in favor of:
Procedure calls across networks
Remote-object method invocation
Message-queuing systems
Advanced communication streams
Event notification service
Information system services: Services that help manage data in a distributed system:
Large-scale, systemwide naming services
Advanced directory services (search engines)
Location services for tracking mobile objects
Persistent storage facilities
Data caching and replication
01 – 19
Introduction/1.4 Software Concepts
Middleware Services (2/2)
Control services: Services giving applications control over when, where, and how they access data:
Distributed transaction processing
Code migration
Security services: Services for secure processing
and communication:
Authentication and authorization services
Simple encryption services
Auditing service
01 – 20
Introduction/1.4 Software Concepts
Comparison of DOS, NOS,
and Middleware
1:
2:
3:
4:
5:
6:
7:
Degree of transparency
Same operating system on each node?
Number of copies of the operating system
Basis for communication
How are resources managed?
Is the system easy to scale?
How open is the system?
Item
1
2
3
4
5
6
7
01 – 21
Distributed OS
multiproc.
multicomp.
Very High
High
Yes
Yes
1
N
Shared
Messages
memory
Global,
Global,
central
distributed
No
Moderately
Closed
Closed
Network
OS
Low
No
N
Files
Per node
Middleware DS
High
No
N
Model
specific
Per node
Yes
Open
Varies
Open
Introduction/1.4 Software Concepts
Client–Server Model
Basic model
Application layering
Client–Server architectures
01 – 22
Introduction/1.5 Client–Server Model
Basic Client–Server Model (1/2)
Characteristics:
There are processes offering services (servers)
There are processes that use services (clients)
Clients and servers can be distributed across different machines
Clients follow request/reply model with respect to
using services
Wait for result
Client
Request
Reply
Server
Provide service
01 – 23
Time
Introduction/1.5 Client–Server Model
Basic Client–Server Model (2/2)
Servers: Generally provide services related to a shared
resource:
Servers for file systems, databases, implementation repositories, etc.
Servers for shared, linked documents (Web, Lotus Notes)
Servers for shared applications
Servers for shared distributed objects
Clients: Allow remote service access:
Programming interface transforming client’s local
service calls to request/reply messages
Devices with (relatively simple) digital components
(barcode readers, teller machines, hand-held phones)
Computers providing independent user interfaces
for specific services
Computers providing an integrated user interface
for related services (compound documents)
01 – 24
Introduction/1.5 Client–Server Model
Application Layering (1/2)
Traditional three-layered view:
User-interface layer contains units for an application’s user interface
Processing layer contains the functions of an application, i.e. without specific data
Data layer contains the data that a client wants to
manipulate through the application components
Observation: This layering is found in many distributed
information systems, using traditional database technology and accompanying applications.
01 – 25
Introduction/1.5 Client–Server Model
Application Layering (2/2)
User-interface
level
User interface
HTML page
containing list
Keyword expression
HTML
generator
Query
generator
Database queries
Ranked list
of page titles
Ranking
component
Database
with Web pages
01 – 26
Processing
level
Web page titles
with meta-information
Data level
Introduction/1.5 Client–Server Model
Client-Server Architectures
Single-tiered: dumb terminal/mainframe configuration
Two-tiered: client/single server configuration
Three-tiered: each layer on separate machine
Traditional two-tiered configurations:
Client machine
User interface
User interface
User interface
User interface
Application
User interface
Application
Application
Database
User interface
Application
Application
Application
Database
Database
Database
Database
Database
(d)
(e)
Server machine
(a)
01 – 27
(b)
(c)
Introduction/1.5 Client–Server Model
Alternative C/S Architectures (1/2)
Observation: Multi-tiered architectures seem to constitute buzzwords that fail to capture many modern
client–server organizations.
Cooperating servers: Service is physically distributed across a collection of servers:
Traditional multi-tiered architectures
Replicated file systems
Network news services
Large-scale naming systems (DNS, X.500)
Workflow systems
Financial brokerage systems
Cooperating clients: Distributed application exists by
virtue of client collaboration:
Teleconferencing where each client owns a (multimedia) workstation
Publish/subscribe architectures in which role of
client and server is blurred
01 – 28
Introduction/1.5 Client–Server Model
Alternative C/S Architectures (2/2)
Essence: Make distinction between vertical and horizontal distribution
Front end
handling
incoming
requests
Replicated Web servers each
containing the same Web pages
Requests
handled in
round-robin
fashion
Disks
Internet
Internet
01 – 29
Introduction/1.5 Client–Server Model
uÜXØ dß ËÛ ÿ Ü ÞXÛNÚ]8 ýf Þ¿Ù>²Ø OÜ Û Ý¢Û ý Ø uÿ ÿ( qÛ¢ÞXØÄÜ3?ØyÜÞ
r1(* S0 b^ ØnþgyØÄÜ ÛuØ ÛuÞ#)˜ •ý ÜØÄÝD ÜXØ yØÄÝÜØuguØÄÞ32eÜÛ Þ3OØÄÜ~yÛB ý Þ ÿuÿ( qÛÞXØÄÜ3?ØÄÜ~ AUØÄÞ
ØuÛUÜ ²Û uÞO+²ØS(¢ÞXØÄÜq?>(*< ý ؽÜØuguØÄÞ3&
Ú ý •ý (Üf( ÜØ ÞXØÄÞO ý ؽÜØÄÞXÿ‡ÛuÞXØ'(Ø#
n~–‡ØnÙÛ uP
Ý ›w Û uØ Û uÞ ²Ûd3Þ Û' ý غÜØÄÝqÜØ OØÄÝ ÜXØuguØÄÞ3yÞByÛ ý Ø ÿuÿ+ Û ÞXØÄÜ3?ØÄÜ
Ù €Þ qÜ *dØ (*¹ÜXØ Ø OØÄË
Ý eÛdÜ  Û uØ qÛ # üDý Ø Œ ý TØ 3Þ OØÄ2Ü ` üD'ü a ÞXØÄqÜ ?AØÄÜ-Ú +(DÿuÜXÛ ØÄÞXÞ
Þ3 3Ü yÞÇÿuÜÛ ØÄÞXCÞ +*S ý Ø­ÜXØ u6uØÄ€Þ ¨Û ÂÛ uØ Û  ý Ø­ÜXØ +D+*SSÚÛ ÿuÿ ( qÛ ÂÞXØÄqÜ ?AØÄÜXÞ # üDý ÞÇÜXØÄÝ ØÄÞ
ý زÜXØÄÞXÿ®Û uÞXS
Ø (²2Ø eÛU'Ü ý زÜXØ uguØÄ3Þ &.Þ + SØ ý ØÄÞXØ ÿuÿ ( qÛ ’Þ§ØÄqÜ ?AØÄÜÞ ÜUØ u/Û -Þ qÛ ÝuØÄÝ bÞ ý Ø
Ú ý Øngþ yØÄÜ d Û Ý # üDý Ø²Þ 2Ø q'Þ ý زÜXØ ÞXÛ eÛdÜ Û u3Þ >ÜØÄÞXÿ‡Û uÞXˆØ +²ØÄÞ
ÿuÿ + Û ¹ÞXØÄqÜ ?AØÄÜ¾Ü _w Û uØ Û uÞ #
tOØÄ'
˜p?AØ ’(– ý Ø ÞXØ¢Û Áݳ>
Ù 2 T (*dÛUÜ 8 ý ²Þ &ÁÜØÄÞ§ÿ®Û ÞXØ (TØ + ÜXØ ÞXØÄUÞ eÛd2Ü Û•ÚØÄ2Ü 6B®ØÄÜ²Û Øngþ yØÄÜ Û uØ Û uÞ #T‡=yØÄ'Ü ý qÞ­ÿ®Û (S ý ØÜXØÄÞXÿ®Û uÞXˆØ (²ØÜXØ Šý ØÄÞ T Û u3Þ >3Þ yØ # üDý ]Þ qÞ
®Ø uÞXØ " ەÚØÄd
Ü g®ØÄÜÛ ‡Øn6þ OØÄÜ Û uØ qÛ uCÞ &Þ§Û ²ØDÛ 6 ý Ø2ÜØ uguØÄ3Þ OÞÚØyÜØDÿuÜXÛ ØÄÞXÞXØÄÝ²Û ˆ ý Ø
Û ÝØÄÝ ÿuÿ ( qÛ ’Þ§ØÄqÜ ?AØÄÜ­ÜXØÄÞ +D* [( ÜXØ ÞXUØ (f ý ØÜXØÄÞXÿ®Û uÞXSØ +²Ø #[email protected] yØÄbÜ ý qÞ ÿ®Û +&
ý Þ ÿ 3Ü Ü ÿÿ + Û ¹ÞXØÄ3Ü ?Øy]
Ü -Þ uØ ?ØÄÜ Šý ÛUÞXØ !OÛtÿ qÜ qÿ O2Ø (! ý 2Ø qÛ 
Ý (*#‡=+
ý ؽÜXØ uguØÄ3Þ yÞ ÜØ­ÿuÜXÛ ØÄÞXÞXØÄÝtÛ ý ؽÜXØ (+*ˆœÚÛ ÿuÿ + qÛ tÞXØÄ3Ü ?ØÄÜXÞ # üËý qÞ>
Ù Ø qÝuÞ 2 Û u€Þ ÜXØÄÞXÿ®Û uÞ[email protected]Ø (SØ eÛUÜ +À ý ØÄާؽØn6þ OØÄÜ ; Û ÝuÞ #
üËý qXÞ v*uÜXØ qÞXÛ Þ **UØÄ3Þ O;Þ ý =monjk§ÜØÄÞXÿ‡Û uÞXØ +²Ø2ÿ®ØÄÜ eÛdÜ XÞ ‡Ø OØÄpÜ ý S ý eØ monjk§ÞXØÄ3Ü ?ØÄXÜ Û Ý &
+Ý (*< ý b ý ؽÞXØÄqÜ ?AØÄ'
Ü qÛ fÝ eÞ uMÛ *UÛ¶Û\Ý Šý Û SØ eÛU'Ü qÛ fÝ Ø OÜ #
,/š
uÜØ dß ËÛ ÿ ÜqÞXÛtÛ
rO+* ]. b^ (™
ý Ø +*UÛdÜ ý Þ
ü &j®Ø uÞXبÛOÞÁÿuÜبݳÙ> uÜØ #[email protected]&j®Ø ÞXØ¨Û ØÄÞXÞ +*UÛdÜ ý 2 ÛC?AØÄÜ ý Ø Ý
uÝ ØÄÞ§Þ@uØ œÚÛdÜAÛC?AØÄÜ ý Ø Ý ý +* ý ØÄܾÿ®Û+(+*(OØÄÜ3? qÞ&>monjk ×e× k ×e× +*UÛdÜ ý ÛD Ý‡Ø ®ØÄÞ3
Ø eÛUÜ Þ Ø½ÞXÛ + Û #
ÛÿuÜXÛ2 ÞXS
monjk ×e× k ×
tù>õ¾ù¤öp ÷¢ö ¨ô¤ø ! ÷tô¤úeÐyùR¡^ûX÷tôR¡
Û qÝ ®Ø Û (uÝuØÄÝ] ý X ý ØÇØ qØ Ùtۙ ý ØÇÝqÞ3OÜ(yØÄÝ]
Ø>ÞXØÄÜq?AØÄÜÞ¿Ù9Þ3OØÞ*Ú- ý G Þ3yØÄÜ3k
h ?Ø Ü Šý OØ uÜX،qÞtÝuØÄÿ®ØuÝuØ Û²ÿqØ yØÆÙ0ÛI ý ØÞ Šý ØÄÝ(+* +*UÛdÜ ý Û Šý Û Ø# ×e× k
Þ Šý ØÄÝ(+*ºÿuÜXÛ2 ÞXØÄÞ*UÛ\Û¶Ý ÿ®ØÄÜ ÛUÜ Ø fqÛ§ÚÛqÛ ÝuÞ uÝImonjk§ÜØÄÞXÿ‡ÛuÞXØB(؞ÿuÜÛ2 ÞXØÄÞ*dÛ¶Û¶Ý
ÿ®ØÄÜ eÛdÜ Ø ý (* ý Û ÝuÞC#[email protected]ÙËuÞ(*·ÛuØ»Û1 ý Ø ý Ù>uÜqÝ +*UÛdÜ ý ²Þ*UÛ¶Û\Ý ÿ®ØyÜeÛdÜ Ø
"
Þ * Ü yØÄØÄ2Ý ý ÜXÛ * ý Û #=n¾Ø uÞ§Ø­Û O ØÄÞXÞ (*dÛUÜ 8 ý 2 CÛ ?AØÄÜ ý Ø Ý uTÝ ØyÞXeÞ uØ œÚÛdÜ AªCÛ ?AØÄÜ ý Ø Ý
Ø eÛdÜ Þ qØ
ý (* ý ØÄÜDÿ®Û +(+*2(OØÄqÜ ? qÞ&;monjk ×e× k ×e× +*UÛdÜ ý Û qÝf®Ø ‡ØÄ3Þ Û ÿÜÛ qÞXS
ÞXÛ + Û #
rÛUÜ ²ÛdÜXØ»ÜXÛ u€Þ ªÞ qØ»ÞXÛ + Û &ËÛ uØ Û $
Ý uÞX)Ø monjk ×e× k × È
ü (*dÛUÜ 8 ý 2&^-Ú ý Û ý Ø
+*dÛUÜ ý CÛ ?AØÄÜ ý Ø Ý #J) ý Ø ÞXØNÛ Xmonjk ×e× k × F
ü +*UÛdÜ ý &‡ØnÙÛ u)Ý ý =Ø ý ÜXØÄÞ ý Û ÝºÜXØÄÞXÿ®Û uÞXØ
+²ˆ
Ø ²Ûd3Þ ­CÞ qØ.ÚÛdÜ AdØÄ'Ü qÞËÞ§Ø Ø OØÄ!Ý ORÙ vÝ +*< ý ~Ø ý ØÞ ?AØ.-Ú ý (+D ÜØÄÞXÿ‡Û uÞ[email protected]Ø (Ø #
r1(uÝ (*·Þ ?AØ Ú ý (+BÑqÛ %
Ý qÞ R Ûd€Þ ÆÙ Ûdÿ®ØÄÜ Û ý ꨯ y=Þ ý ØªÞ (+8Sæ
Ù (%monjk
×e× k × ü ޿ه€Þ yØ ²Þ #–n€’
Ù A6(*2 ý Ø>ÞXØ Ü Šý eÛd-Ü 2+(\ Û ÝØÄݒÞXØÄ3Ü ?ØÄÜ ­ “ Û *T• +*UÛdÜ ý &
®ØÄ3Þ ÿ®ØÄÜ ÛUÜ Ø R®Ø ý Ü uØÄÞXÞ§ØÄ
Ý eÜXÛ ý BØ monjk ×e× k × ü Þ¿Ù93Þ OØ Þ #rAÛUXÜ ý ÞÛ uØ Û RÝ uÞ§Ø ý qØÄÜ Ü Šý ÙtÛ Þ ?=Ø uÛ¶ÝuØÄÞ ÜXÜ *dØÄ)Ý ( yÜXØÄØÇ3Þ OÜ uÜXØÇÜ ý ØÄoÜ ý »ÿ +* (i ý ØÁÞ ?ØÄ~Þ (] ý Ø
³ J
,/_
Download
Random flashcards
Arab people

15 Cards

Radiobiology

39 Cards

Pastoralists

20 Cards

Radioactivity

30 Cards

Create flashcards