System Design
1)
to
Try
break
2) Talk
about
( No
the problem
( Top
modules
Basics
down
simpler
approach )
the trade offs
-
solution
is
perfect )
calculate
the impact
on
the
constraints
and
all
into
①
system
based
the end test
on
cases
heproblem
]
T
Abstract
questions
]
finding
Requirements )
←
Focus
on
interviewer 's
intentions
.
Ask
( constraints
&
Functionality
bottlenecks
idudas
.
System Design
D
Architectural
2)
How
3)
Utilization
-
f-
-
-
-
-
-
-
1-
these
Basics
ccontd )
pieces / resources
work
resources
&
available
together
Tradeoffs
Hashing
consistent
CAP
Theorem
Load
balancing
✓
✓
queues
caching
-
Replication
59L
vs
No
-
SQL
Indexes
I
✓
Proxies
Data
Partitioning
✓
②
Load
③
Balancing
Types of distribution
-
f
<
( Distributed system )
Random
Round
-
robin
Random ( weights for
CPU
To
utilize
D
User
2) Web
full scalability &
¥
server
memory
cycles)
redundancy
,
3
add
LB
web server
¥
App
server 1
( Internal
3) Internal platform
DB
Cache
Server
platform )
.
#W
\er
Client
DB
T ,,
&
LB
LB
Smart
clients
Takes a pool of service hosts & balances load
→
detects
hosts that are not responsive
→
recovered
→
addition
Load
*
Attractive
( small
As
Hardware
g
.
functionality
but
to
Large companies
system
Intra
high
trivial
it
solution
to serve
load
tend
to
.
.
avoid
this
point or contact
&
user
requests
uses
smart
( Next
balancing
→
performance
configure
servers)
:
1st
as
network
.
( standalone
load Balancers
Net scaler
use
DB ( cache Service
developers
LBS
→
Citrix
Not
or
for
to
systems)
grows
Expensive
e.
hosts
new
solution
scale
system
hosts
of
balancing
.
clients /
page)
traffic
for
.
config
to
.
their
hybrid
Software
Load Balancers
[
of
'
No
pain
No
cost
of
hybrid
of
creation
purchasing
smart
dedicated
client
hardware
approach
HA Proxy
Load
OSS
balancer
-4
1)
Running
'
client
on
2
( locally bound port)
e
-
g
.
machine
local host
:
9000
F-
I
( with
2) Running
on
[
HA Proxy
efficient
management
of requests on the port )
intermediate server Proxies running beth
:
dirt
HA Proxy
Manages
by
managed
server
health checks
removal &
addition
of machines
balances
requests
alc
pools
.
side components
Wortdof
Databases
=
S9Lvs.NoS÷
iaa
1) structured
2)
Predefined
3) Data
in
Row
One
column
D
schema
3)
Into
¥
Separate data points
9sa¥fl
mysar
Oracle
Postgres
Maria DB
.
2) distributed
rows & columns
Entity
Unstructured
dynamic
schema
's::L:L: stores
wait:p : :3 ;
'
DB
÷ ÷ ÷ ÷ ÷ ÷ ÷ ÷÷÷÷÷÷÷÷÷÷
an
E
o
e
r
f÷ ÷¥÷ ÷ ÷ ÷ ÷
O
e
I
s
E
th
e
is
I 3
to
O
of =
6
'
U
E
O
s
I
E
O
cs
s
s
ai
g
G
S
IT
S
O
J
Os
u
O
Et
-
s
U
O
e
o
te
O
G
O
e
:
-
'
①
O
E
O
T
e
I
E
or
I
E E
o
o
E
tr
S
J
is
or
-
g
I
ET
± .
or
e
o
E
E
J
s
E
-
÷EE
.
if÷ ÷f ÷ i ÷ :f÷: f÷ i ÷
.
E
u
w
O
ch
O
→
E
or
j
or
Cs
d
-
s
d
O
f 6
S
u
-
of
Z t
&
-
-
w
-
-
s
D
t
d
O
g
o
as
a
E
g.
E
o
U
6
O
U
n
ou
w
E
-
-
c
O
s
>
O
s
u
@
o
@
z
cry
at
Ed
O
E
O
-
to
so I
•
'
E
I
I
e
.
-
.
O
u
s
U
a
G
U
og
-
u
we
E
e
si
I
it
E
T
t
O
0
8 of
I
w
y
s
⑤
E
G
&
u
O
E
u
•
u
on
Nos
O
←
>
so
or
I
N
E
n
§
e
s
t
E
-
-
w
t
E
O
s 't
I
O
U
ou
Notre
E
KE u
s
E
s
too
o
-
ou
S
c
e
88
W
o
=
so
o
ng
-
O
z
8
s
w
J
S
or
O
-
J
y
is
s
a
o
E
u
s
s
OE
O
is
-
s
Dw
O
E
8
-
us
u
E
-
it's
s
S
O
o
OS
J
.
88
n
O
o
g
d
g
I
E
e
.
R
b
g
Es
S e
EE
by
o
A
v.
-
s
U
E
Id
gu
s
O
S
o
o
d
O
d
T
cs
o
o
.
E -3
E
-
s
f
s
er
Ess
u
d
-
Er
G
n
883
T
8
r
i
t
O
I
E
o
°
3
d
d
G
is
o
u
E
O
to
In
i
•
N
-
u
O
si
t
N
w
3
8
Fa
c
s
T
Z 's
n
f
U
s
O
G
O
o
es
y
d
-
T
I
O
1-
I
E
u
o
-
u
06
§
I
6
03
O
S
T
89
E
Es
u
o
Tu
@ u
U
y
-
z
o
S
°
-
G
o
-
w
L
d
or
E
n
's
o
I
62
u
nd
t
g-
o
- ooo
o
x
'
o
-
I
d. E
w
T
8-
o
J
U
O
b
as
u
w
u
>
o
u
or
6
'
,
88
+
as
as
#
D
g
U
d
Eire E
•
'
Es
EEE -
G
es
s
E
od
f-
s
O
o
N
#
I
O
'
I
O
E
O
6
s
or
go.IT
g
s
E E
=
a
E
IT IT
IT
o
u
D
t
'
-
w
s
.-
I
e
El
q
8
Ev
0
or
u
or
d
s
on
if
J
u
us
o
g
¥4
Reasonstouses.cl#BJ
1) You need
ACID
to
:
anomalies
Protects
for
ACID
2) Your
of
data is
→
No
→
data
or
is not
sudden
requirements
is
of
.
"
app
unchanging
.
experiencing
changes
more
servers
consistent
there's
to support
database
.
structured &
business
rapid growth
the
commerce & financial
-
compliant DB
first choice
the
your
then
integrity
E
many
is
If
compliance
compliance
Reduces
→
ACID
ensure
no
variety
reason
to
of data
use
&
system
high
design
traffic
.
Reasonstouse.NO#IB
When
components of system are fast
→
bottleneck
querying & searching for data
NoSQL
being bottleneck
prevent data from
all other
.
.
NoSQL
large success for
large volumes of data C little Ino
Big data
1)
.
To store
No
limit
Document
type of data
on
Stores
DB
(
2)
No
.
all data in
of
type of data)
to
the
cloud
& storage
Excellent
cost
saving
solution
.
across
OR
commodity
No
&
NoSQL
hlw
headache
DBS
on
Useful for
site
servers
( affordable
centers
rapid 1 agile development
will
slow
iterations
you down
.
,
to scale up)
smaller )
Stw
designed
multiple data
making quick
.
spread of data
multiple
like Cassander
If you're
SQL
fullest
( Easy
or additional
across
3)
place
one
need
Using
structure)
to
scale
out
of
on
schema
the box
.
.
CAP-heore€
Achieved
by
updating
several
before
\tenyC
allowing
reads
nodes
Hr
at
"
[ cassandra
.
Couch DB]
System
different
continues to work
despite message
Failure
servers
sufficiently replicated
combination
networks
to
time)
Hr
keep
of
the
loss ( partial
.
( can
- resulting
across
same
,
sustain
of network
Data is
data
%%%
( success ( failure)
across
same
e.
Not
Achieved by replicating
data
see
"
Every request gets
response
nodes
pareieiontdera£
g)
E
g
Availability
All
"
"
.
any
amount
failure without
in failure
of entire
network )
nodes /
system
up
.
.in:6#eo:::i:::::ia::i:n:::ans'
We
cannot
build
a
data store
which is
:
continually available
2)
sequentially consistent
D
3)
failure
partition
Because
tolerant
.
,
To be
consistent
all
nodes should
set
of updates
network
suffers
partition,
update
in
partition
might
other
partitions
But
if
↳
one
client
reads
data from
from
the
the
same
in
not
make it
same
order
to
out-of-date partition
After
having
read
Solutions
stop
serving requests
partition
↳
see
up-to-date partition
from
out of date
-
-
.
service
100%
is
no
longer
available
.
.
Redundancy&ReplicationJ
Duplication of critical data &
↳
For
copies 1 versions
servers
of system
increasing reliability
services & data
critical
services
1 databases
that
ensure
simultaneously
running
are
.
multiple
on
different
.
node failures
Secure
against
single
Provides
backups
if needed
in
crisis
.
.
Deter Dow
Primary
secondary
server
2
- a
Replication
2
&
Data
-
Ig
Active
# Service
Every node
[
server
data
Mirrored
Redundancy
independent
More
to
:
.
shared
No
failures
Nosing
nothing architecture
-
central
resilient ←
data
service
New
managing
servers
addition
←
without
special conditions
.
state
.
Helps in
]
scalability
Caching
Load
balancing
caching
I
I
of
Locality
:
almost
in
Used
horizontally
Scales
reference
every
Application
Server
cache
Placing
cache
directly on
↳
a
storage of
Local
principle
layer of
computing
.
:
request layer
a
node
.
response
Requests
miss
in€.÷
.
data
response
#
Caches on one Request layer made
Tcatedy
✓
Memory (very
fast)
Node 's local disk
( faster
##
our:c
Bottleneck
!m£
by
:
If
LB
distributes
↳ same
than going
requests
request
D
Global
2)
Distributed
caches
caches
network
randomly
different
d
More
to
nodes
storage)
Distributed
Gimme
⑤gµeautts.-€#
-
Divided
③
using consistent hashing function
##
Easy to
##
Disadvantage
by adding
Resolving a missing node
increase cache
:
staring multiple copies of
data
on
hordes
more
I can be handled
by
different hordes
likes
# # Even
space
making it
more
complicated
if node disappears
pull data from Origin
request
can
.
.
flabellate
#
↳
#
#
GEE
%gz
Single cache space
Adding
a
cache
for all the hacks
source
Difficult to manage if
effective if
.
I file store ( faster than original store)
no
af clients I request
Y
fixed dataset that needs to be cached
2)
special Hlw
Forms
of global
{
fast
cache
Ho
"
-
.
:
Database
%fbtae.mn?hdEtahhYm
database
contains
hot data at
Global
{
App
increases
cache
Database
logic understands the eviction strategy that
better
than
cache
.
spoils
CDN : content
4- Cache store
of
Distribution
for sites that
static media
network
saves
large amount
.
if not
CDN
Request
if
available
-
-
End
server
L
a
available
Baek
(
local
( static Media)
storage
Lf the site isn't large enough to have its
own
CDN
4µA
transition
static media
using separate subdomain
- ( static
yaursuuice.com
using lightweight Nginse serves
Some
.
↳ entrance
to
DNS
a
from your sauce
CDN
later
Invalidation
Cache
# Cached data
Lf
#
3
data in
schemes
needs to be coherent with the database
DB
modified
invalidate the cached data
.
:
=
I
Write through
-
cache
:
cache
Data
DB
+
+
-
2)
is
same
time in
hath
cache & DB
DB)
Complete data consistency C cache
Fault tolerance in case of failure Club data loss)
=
high latency in writes
Write around
2
write operations
cache
Cache
Data
DB
flooding foe writes
read request for newly written data
+ No cache
-
written
Data
higher latency
miss
d
.
B)
Write
back
cache
:
DB
cache
Data
£
client
after some
interval
some
as
under
sepciifird conditions
data is written to DB
from cache
+
-
law
latency
Data
loss
#
Cache
D
FIFO
2)
LIFO
3)
LRV
4)
MRU
5)
LF U
Random
&
TT
high throughput
bae write
( only
in caches
Eviction
Ae
FILO
Replacement
one
copy
Policies
intensive app
"
-
11 Data Partitioning
S harding
#
Data
Partitioning splitting up DD I table
multiple
&
machines
manageability performance availability LB
:
across
,
.
**
After
a
certain scale paint
,
it is cheaper and more
horizontally by adding money instead
vertical
scaling by adding beefiness
to scale
# Methods
1)
of
Partitioning
feasible
of
:
Horizontal Partitioning Different rows into diff tables
Range based shading
g
staring locations by zip
different
:
.
e.
.
Table
1
:
Table
2
:
Lips with
and
* * come
e.
g
.
with
Lips
so
L
100000
7
Loo ooo
ranges in
different tables
on
if the value of the range not chosen carefully
leads to
unbalanced
Table
can
I
servers
have more data than table
2
.
Vertical
Feature wise distribution of data
#
↳
e.
Partitioning
g
different servers
in
Instagram
.
-
.
DB
sauce
1
DB
sauce
2
DB
server
3 :
user
:
:
info
followers
photos
to implement
* A
straightforward
* A
lane impact on app
if app → additional growth
.
-
-
need to partition
(e g
-
all
.
feature specific DB across various sources
it would not be possible for a single sewer
metadata queries
for Lo
Directory
billion photos
based
to handle
by 140 mill users
.
partitioning
loosely coupled approach to work around issues
mentioned in above two partitioning
**
Create lookup service
current partitioning scheme
A
.
& abstracts it
away from
Mapping l tuple key
→
the DB access code
.
D8 sauce)
Easy to add DD towers or change partitioning
scheme
.
Partitioning
Criteria
key or Hash based partitioning
D
Kay
atte
-
af
function
Hash
the data
#
:
→
number
the total number
fines
of
Effectively
So
if
we
Partition
add
new source I
downtime
sauces 1partitions
partition To
because
of
change in hash function
d
redistribution
↳
2)
List
Partitioning
:
Each
Solution
partition
is
:
consistent Hashing
assigned a list of
values
Nuo
record
→
.
hookup
for
key
→
stare the
record
( partition based on the key)
3) Round Robin
Partitioning
:
uniform data distribution
with
'
partitions
n .
the
'
is
tuple
assigned to partition
is
( i mad n )
4
Composite Partitioning
combination
flashing t List
:
af above partitioning schemes
consistent Hashing
Hr
key space to
Hash reduces the
size
#
Common
Problems
Iharded DB
:
that
of Shouting
Entree
be listed
can
a
.
:
constraints
on
the
diff operations
.
Hr
operations
across
-
multiple tables
multiple rains in the same table
no
in
or
7
longer running
single severe
.
Denoumalizatiom
" Jains A
:
straightforward
single sauce
* not
shrouded tables
feasible to perform joins
↳
Less efficient C data needs to be compiled from
Jains
on
tables
on
.
on
multiple
#
Workaround
(
so
Denarmalip
queries
that
from
single
table
Perils
of denavmalizatiom
coins
↳
2)
the DB
that the
a
data
Referential integrity
:
previously read jains
.
Mast
of the RDBMS
be
performed
inconsistency
Foreign keys
om
does
not
shrouded D8
difficult
support
foreign keys
stranded DB
#
can
.
↳
*
servers)
"
If app demands referential integrity
↳
enforce it in app
-
om
clean up
"
on
.
shrouded DB
code
C SOL
jobs to
dangling references)
3)
Rebalancing
:
Reasons to
a)
horn
b)
non
-
-
change sharking scheme
uniform distribution C data wise )
uniform laced balancing C request wise)
Workaround
:
Y add
:
2)
rebalance
{
We
can
↳
DB
new
use
↳
change in partitioning
↳
scheme
data monument
↳
downtime
directory based partitioning
-
highly complex
↳
single
(
paint of failure
lookup
service 1 table)
Indexes
Well known
-
because
speed
Increased
storage
↳
auerhead
write
Can be created
&
#
Update
the index
using one or more columns
lookups
random
efficient
data
the
↳
Rapid
.
writes
Shauna
*
of databases
of retrieval
Improves
-
-
access
of ordered records
.
Data structure
column
→
to whale
raw
different views of the same data
↳
data sets
very good for filtering / sorting of large
Create
-
.
↳
#
Painter
→
no
Used foe datasets
spied
over
physical
several
devices
need to
( TB
in
I
→
create
size) &
We
small
need
physical
additional
copies
.
.
payload ( KB)
some
way to find
location
i. e.
the correct
Indexes
useful under high load
Peonies
-
if
we
have
limitcdcaehing
↳ batches several
client
>
>
client
situations
>
Peony
requests into one
MY
Backend
> sauce
Source
>
^
a
filters requests
-
log
requests
transform
add / remain
-
encryption / decryption
frequently
used
headers
compression
request
resources
co
-
ordination
( request traffic optimization
T
we can
also
use
←
Collapse
same
data
locality
request into
collapsing requests
for data that is spatially
collapsed
spatial
↳
Ipsfminimize
access
one
.
forwarding
reads
from origin
-
.
Queues
Effectively manages requests in large scale distributed system
-
→
In small
In
→
systems
→
complex systems
→
writes
are
To
achieve
↳
#
writes take more time
high performance & availability
system
Synchronous
.
high incoming load
4- individual
*
fast
needs
behaviour
to
be
↳
Queue
→
asynchronous
degrades performance
d
&
fair
difficult for
balanced
ye
Load
balancing
distribution
server
gag
{
C
,
T,
-
T2
r
C,
-13
T,
,
↳
Queue
Cu
T2
#
Queues
asynchronous communication protocol
:
↳
sends
client
↳
gets
[
#
Limit
# Queue
[
on
from queue lecccipt )
ACK
I
continues its work
client
the
sispafeeguest
&
number
of requests
fault
-
↳
in
reference
.
queue
tolerance
protection from
retry failed
Enforces Quality
↳
service
service
outage /failure
request
of Service guarantee
L Does NOT expose clients
Queues
↳
as
highly robust
[
#
serves
for the results in future
Provides
:
task
to
outages )
distributed communication
:
Open
↳
source
implementations
Rabbit ma
,
Zoeoma
,
Active MQ
,
BeanstalkD
.
Consistent
#
Distributed
#
hash
=
Suppose
we're
with
n
Drawbacks
-
function
hash
key)
caching system
servers
function
.
(
key %
n
)
:
scalable
horizontally
NOT
↳
addition
of
↳
2)
C
distributed
designing
cache
↳
1)
Table
Hash
index
Hashing
l because
results in
change all existing mapping
( downtime
of system )
need
af
-
to
uniform distribution of data )
1-
non
Some
caches
:
Other
caches
:
How to tackle
server
balanced
load
NOT
new
hat & saturated
idk &
about
Consistent
-
empty
problems ?
flashing
.
Hashing ?
Very useful strategy for distributed
What
→
→
consistent
is
minimizes reorganization
only
→
How
k
total number
n
number
it
up / dawn
.
.
of keys
all
suppose outputs
hashing
of these integers
[0
•
placed on
are
0
of
•
,
253
in
•
2
"
,
±
have
3
.
2567
,
255
we
.
of servers
254
&
outs
works ?
consistent
imagine
scaling
keys needs to be remapped
kin
Typical hash function
In
in
caching &
servers
:
A
,
B & C
.
a
ring
.
1)
Given
a
of servers
list
hash
,
255
to
them
integers in
0
A
C
B
2)
Map
key to a serum
it
a)
Hash
b)
Mane
c)
to
CLK
:
single integer
until
wise
key to that
map
255
you find
server
.
0
hL key
"
of
h (
key
-
2)
A
'
B
-
1)
Laura
the
range
.
Adding
a
'
server
new
.
will
255
result in
-
the
morning
key -2 to 'D
'
0
hL key
-
1)
"
C
a
h (
key
-
A
D
'
2)
B
Removing
server
IA
'
,
will result in
255
morning
O
①
h ( key
a
h (
key
-
D
2)
the
'
B
•
-
1)
-
key
-1
'
to II
Consider
real world scenario
data
randomly distributed
→
↳
How
to
cactus
.
handle this issue ?
Virtual
Instead
we
unbalanced
Replicas
of mapping each node to
-
map it to multiply
↳ ( more number
↳
paints
a
single paint
.
of replicas
more
equal distribution
↳
255
balancing )
load
good
0
D
C
•
h ( key
A
-
1)
a-
C
B
B
C
AD
h [ Key 2)
-
D
^
A
B
Long Palling
tikbsoekrts
vs
-
Serves
us
Sent
-
Events
'
↳
#
Client
Protocol
HTTP
-
Protocols
Senior Communication
NJ
:
request
client
<
>
Serum
prepare Remorse
Response
.
#
AJAX
Patting
:
repeatedly palls servers for data
Clients
similar to
↳
Drawbacks
HTTP protocol
requests
sent
to
at
regular intervals ( 0.5sec)
:
Client
keeps asking the source
↳
Lot
of
uspomsgy.au
HTTP
Request
.
now data
'
↳
empty
Overhead
\
'
.
)
.
4
Request
.
Client
screen
a
Response
>
sooner
response
\
Request
>
.
<
Response
#
☐
2)
3)
4)
5)
Long Patting
HTTP
Sauce
does
Pushes
response
•
:
Hanging GET
send
NOT
makes
Server
delays response
HTTP
until
until
When
update
Client
sends
d)
after a
timeout
latency period
periodically
due to timeouts
My
Request
.
<
.
a
.
acceptable
7
.
<
LP
full response
sends
.
reconnect
LP Request
Client
.
receiving response
pause to allow
needs to
LP
.
-
immediately after
has
response
long poll request
now
a)
Client
occurs
the
is available
update
server
→
.
only when new data is available
4- waits
for
Request
time-out
or
Each request
empty response
to clients
Client
'
full
Response
>
full
Showa
Response
Request
>
full Response
Wet Sockets
→
duplex communication
→
Provides persistent
'
( client a
→
channel
communication
serum
can
at
Handshake
Success
anytime ]
Response
'
Client
<
i.
is
:
.
Source
}
Communication
channel
.
→
→
Lower
overheads
Real time
.
Request
<
-
TCP connection
always open channel
Handshake
socket
pep
in
single
'
send data
communication
bidirectional
over
data
transfer
.
Server
client
* *
( SSE )
-
this
uses
Events
sent
persistent & long term connection with sauna
establishes
server
-
to
connection
send data
to
client
If client wants to send data to server
↳
Requires another technology / protocol
data
.
request
using regular
HTTP
v
'
ing
i.
Client
<
.
•
<
<
=
Always
-
Source
open unidirectional
communication
channel
.
responses
→
best
OR
when
server
we
is
will be
whenever
need real time data
-
data available
from soever
generating data in a loop
sending multiple
now
events
to client
&
to the
client
.