20-755: The Internet
Lecture 6: Programming the Internet
David O’Hallaron
School of Computer Science and
Department of Electrical and Computer Engineering
Carnegie Mellon University
Institute for eCommerce, Summer 1999
Lecture 6, 20-755: The Internet, Summer 1999
1
Today’s lecture
•
•
•
•
Internet architecture (30 min)
Packet filtering (10 min)
Break (10 min)
Client-server programming model (40 min)
– Berkeley sockets interface
– example clients and servers
•
Domain Naming System (DNS) (20 min)
Lecture 6, 20-755: The Internet, Summer 1999
2
Basic Internet components
•
•
•
•
•
An Internet backbone is a collection of
routers (nationwide or worldwide) connected
by high-speed point-to-point networks.
A Network Access Point (NAP) is a router that
connects multiple backbones (sometimes
referred to as peers).
Regional networks are smaller backbones
that cover smaller geographical areas (e.g.,
cities or states)
A point of presence (POP) is a machine that is
connected to the Internet.
Internet Service Providers (ISPs) provide dialup or direct access to POPs.
Lecture 6, 20-755: The Internet, Summer 1999
3
The Internet circa 1993
•
In 1993, the Internet consisted of one
backbone (NSFNET) that connected 13 sites
via 45 Mbs T3 links.
– Merit (Univ of Mich), NCSA (Illinois), Cornell Theory
Center, Pittsburgh Supercomputing Center, San Diego
Supercomputing Center, John von Neumann Center
(Princeton), BARRNet (Palo Alto), MidNet (Lincoln, NE),
WestNet (Salt Lake City), NorthwestNet (Seattle),
SESQUINET (Rice), SURANET (Georgia Tech).
•
Connecting to the Internet involved
connecting one of your routers to a router at
a backbone site, or to a regional network that
was already connected to the backbone.
Lecture 6, 20-755: The Internet, Summer 1999
4
The Internet backbone
(circa 1993)
Lecture 6, 20-755: The Internet, Summer 1999
5
Current NAP-based
Internet Architecture
•
•
•
•
In the early 90’s commercial outfits were building
their own high-speed backbones, connecting to
NSFNET, and selling access to their POPs to
companies, ISPs, and individuals.
In 1995, NSF decommissioned NSFNET, and fostered
creation of a collection of NAPs to connect the
commercial backbones.
Currently in the US there are about 50 commercial
backbones connected by ~12 NAPs (peering points).
Similar architecture worldwide connects national
networks to the Internet.
Lecture 6, 20-755: The Internet, Summer 1999
6
Internet connection hierarchy
NAP
Backbone
POP
NAP
Backbone
POP
POP
NAP
Backbone
POP
Backbone
POP
POP
POP
T3
Regional net
POP
POP
T1
ISP (for individuals)
ISP
POP
POP
T1
Small Business
Lecture 6, 20-755: The Internet, Summer 1999
Big Business
POP
POP
dialup
Pgh employee
POP
dialup
DC employee
7
Network access points (NAPs)
Note: Peers in this context are
commercial backbones..droh
Lecture 6, 20-755: The Internet, Summer 1999
Source: Boardwatch.com
8
Lecture 6, 20-755: The Internet, Summer 1999
Source: Boardwatch.com
9
MCI/WorldCom Global Backbone
Lecture 6, 20-755: The Internet, Summer 1999
Source: Boardwatch.com
10
Lecture 6, 20-755: The Internet, Summer 1999
Source: Boardwatch.com
11
Relative backbone performance
Lecture 6, 20-755: The Internet, Summer 1999
Source: Boardwatch.com
12
Comparison of
T1 average
monthly costs
MCI/WorldCom
Lecture 6, 20-755: The Internet, Summer 1999
Source: Boardwatch.com
13
Cost of Frame Relay connections
56 Kbps frame relay:
Availability: All U.S. backbone cities
Setup: $495
Monthly: $595
Recommended Equipment:
Cisco 2524 router with 5IN1 Card &
Kentrox 56K CSU/DSU: Total $2,395
Lecture 6, 20-755: The Internet, Summer 1999
Source: Boardwatch.com (MCI/Worldcom)
14
Cost of T1 connections
Burstable 1.544 Mbps T-1 service:
Monthly charge based on 95 percent usage level
Availability: All U.S. backbone cities
Average Installation Time: 4-6 weeks
Setup: $5,000
Recommended Equipment: Cisco Integrated T-1
CSU/DSU - $995, Cisco 2524 router - $1,950
Bandwidth
0-128 Kbps
128 Kbps-256 Kbps
256 Kbps-384 Kbps
384 Kbps-512 Kbps
512 Kbps-1.544 Mbps
Monthly
$1,295
$1,895
$2,495
$2,750
$3,000
Lecture 6, 20-755: The Internet, Summer 1999
Source: Boardwatch.com (MCI/Worldcom)
15
Cost of T3 connections
Burstable 45 Mbps T-3 service:
Monthly price based on 95th percentile usage level.
Availability: All U.S. backbone cities
Average Install Time: 8-10 weeks
Setup: $6,000
Bandwidth
up to 6 Mbps
6.01 Mbps-7.5 Mbps
7.51 Mbps-9 Mbps
9.01 Mbps-10.5 Mbps
10.51 Mbps-12 Mbps
12.01 Mbps-13.5 Mbps
3.51 Mbps-15 Mbps
15.01 Mbps-16.5 Mbps
16.51 Mbps-18.01 Mbps
18.01 Mbps-19.5 Mbps
19.51 Mbps-21 Mbps
21.01 Mbps-45 Mbps
Monthly
$12,000
$14,000
$17,000
$19,000
$22,000
$26,000
$29,000
$32,000
$37,000
$43,000
$48,000
$55,500
Recommended Equipment: Cisco 7204 router
Lecture 6, 20-755: The Internet, Summer 1999
Source: Boardwatch.com (MCI/Worldcom)
16
Programming the Internet
•
•
We’ll look at two extreme examples of
manipulating the Internet
Low-level example
– packet filtering
» firewalls
» user-level protocol implementations
» user-level tunneling
» password sniffing!
•
Higher-level example
– client-server computing via the sockets interface
» underlies most Internet services
» relies on DNS to translate domain names to IP
addresses
Lecture 6, 20-755: The Internet, Summer 1999
17
Internet Protocol Stack
Packet Filter Interface
Berkeley Sockets Interface
firewalls, user-level
protocols, tunneling
Applications
UDP
TCP
IP
Berkeley Packet
Filter (BPF)
Network interface
Lecture 6, 20-755: The Internet, Summer 1999
18
Packet filtering
P4
P2
output
packets
application
packet
filter
input
packets
P3
P5
P1
Packet filters allow user
programs to directly read
and write network packets
(e.g., ethernet packets)
network
adapter
packet stream
Network
Lecture 6, 20-755: The Internet, Summer 1999
... P5
P4
P3
P2
P1
19
Unix packet filters
•
•
•
Unix uses files in the /dev
directory as user-level
abstractions for I/O
devices.
To access a device directly
from a user program, open
the appropriate /dev/* file,
read/write to/from it, and
close it.
Four of these files (called
packet filters) provide
direct user-level access to
network adapters:
/
dev
home
jmcc cbs
pf0 pf1 pf2 pf4
– /dev/pf{0,1,2,3}
Lecture 6, 20-755: The Internet, Summer 1999
20
Example of using a Unix packetfilter
/* Find an available input packet filter */
fd = openinpf(interface);
/* read and decode raw ethernet packets */
while (1) {
p = readpacket(fd, &packetsize);
printpacket(p, packetsize);
}
/* Read an ethernet packet from the network */
struct packet *readpacket(int fd, int *packetsize) {
struct packet *p;
p = (struct packet *)malloc(sizeof(struct packet));
*packetsize = read(fd, p, sizeof(struct packet);
return p;
}
Lecture 6, 20-755: The Internet, Summer 1999
21
Packet filtering implications
•
Crucial for important applications
–
–
–
–
•
firewalls
user-level tunneling
protocol research
network monitoring on routers
However, mechanism can be easily abused
– Anybody with a PC can set up a packet filter
– Can be used to sniff passwords
» scan input packets for strings “Login” and
“Password”
•
Should be wary of sending password over the
network in clear text.
Lecture 6, 20-755: The Internet, Summer 1999
22
Break time!
Lecture 6, 20-755: The Internet, Summer 1999
23
Today’s lecture
•
•
•
•
Internet architecture (30 min)
Packet filtering (10 min)
Break (10 min)
Client-server programming model (40 min)
– Berkeley sockets interface
– example clients and servers
•
Domain Naming System (DNS) (20 min)
Lecture 6, 20-755: The Internet, Summer 1999
24
Client-server
programming model
•
•
•
•
Client + server =
distributed computing
Client & server are both
processes
Server manages a resource
Client makes a request for
a service
client
request
server
process
request
client
server
– request may involve a
conversation according to some
server protocol
•
Server provides service by
manipulating the resource
on behalf of client and then
returning a response
Lecture 6, 20-755: The Internet, Summer 1999
client
response
server
25
Internet Servers
•
Servers are long-running processes (daemons).
– Created at boot-time by the init process
– Run continuously until the machine is turned off.
•
Each server waits for either TCP connection
requests or UDP datagrams to arrive on a wellknown port associated with a particular service.
– port 7: echo server
– port 25: mail server
– port 80: http server
•
A machine that runs a server process is also
often referred to as a “server”.
Lecture 6, 20-755: The Internet, Summer 1999
26
Server examples
•
Web server (port 80)
– resource: files/compute cycles (CGI programs)
– service: retrieves files and runs CGI programs on behalf
of the client
•
FTP server (20, 21)
– resource: files
– service: stores and retrieve files
•
Telnet server (23)
– resource: terminal
– service: proxies a terminal on the server machine
•
Mail server (25)
– resource: email “spool” file
– service: stores mail messages in spool file
Lecture 6, 20-755: The Internet, Summer 1999
27
Server examples (cont)
•
DNS name server (53)
– resource: distributed name database
– service: distributed database lookup
•
Whois server (430
– resource: second level domain name database (e.g.
cmu.edu)
– service: database lookup
•
Daytime (13)
– resource: system clock
– service: retrieves value of system clock
•
DHCP server (67)
– resource: IP addresses
– service: assigns IP addresses to clients
Lecture 6, 20-755: The Internet, Summer 1999
28
Server examples (cont)
•
X server (177)
– resource: display screen and keyboard
– service: paints screen and accepts keyboard input on
behalf of a client
•
AFS file server (7000)
– resource: subset of files in a distributed filesystem (e.g.,
AFS, NFS)
– service: retrieves and stores files
•
Kerberos authentication server (750)
– resource: “tickets”
– service: authenticates client and returns tickets
•
/etc/services file gives a comprehensive list
for Linux machines.
Lecture 6, 20-755: The Internet, Summer 1999
29
Berkeley Sockets Interface
•
•
•
•
•
Created in the early 80’s as part of the original
Berkeley distribution of Unix that contained
the TCP/IP protocol stack.
Often referred to as the “sockets interface”
Modified somewhat by Microsoft in early 90’s
(Winsock interface).
The sockets interface is powerful but
somewhat complicated.
More convenient abstractions provided by
Perl and Java and various C libraries.
Lecture 6, 20-755: The Internet, Summer 1999
30
Client-side socket interface to TCP
• socket() : creates a socket on the local host and
returns a file descriptor for that socket.
– int sockfd = socket(INET, STREAM, 0);
• connect(): connects to a remote socket specified as
an IP address and a port at that address.
– connect(sockfd, &servaddr, sizeof(servaddr));
– servaddr is a structure that contains the IP address and the port
• read() : reads bytes from a connected socket.
– n = read(sockfd, inbuf, MAXLINELEN);
• write() : writes bytes to a connected socket.
– n = write(sockfd, outbuf, MAXLINELEN);
• close(): closes a socket and its associated connection
– close(sockfd);
Lecture 6, 20-755: The Internet, Summer 1999
31
Server-side socket interface to TCP
• socket():
creates a “listening socket” on the local
machine and returns a file descriptor for that socket.
– listenfd = socket(INET, STREAM, 0);
• bind(): binds a socket to fixed port on local machine.
– bind(listenfd, &servaddr, sizeof(servaddr));
– here servaddr identifies the port number (e.g., 13 for timeofday)
• listen(): puts socket in “listening mode” to listen for
connection requests from any client.
– listen(listenfd, LISTENQ);
Lecture 6, 20-755: The Internet, Summer 1999
32
Server-side socket interface to TCP
• accept(): waits for a connection request to
arrive at a listening socket, and returns a file
descriptor for a “connected socket” that the
server can read and write from.
– connfd = accept(listenfd, NULL, NULL);
• read() : reads bytes from a connected socket.
– n = read(connfd, inbuf, MAXLINELEN);
• write() : writes bytes to a connected socket.
– n = write(connfd, outbuf, MAXLINELEN);
• close(): closes socket and its associated
connection.
– close(connfd);
Lecture 6, 20-755: The Internet, Summer 1999
33
TCP sockets client-server interaction
Client
Server
Create a socket::
socket()
Create a master socket which is
ready to accept connection requests on port p
from a client:
socket(), bind(), listen()
Create a connection between client
and server socket. The server socket
is identified by an address/ port pair.
connect()
Wait for a connection request to arrive on
the master socket and create a “slave
socket” once a request has arrived.
accept()
Read and write to/from socket.
read() and write()
Read and write to/from slave socket .
read() and write()
Close the client socket: close()
Close the slave socket: close()
Lecture 6, 20-755: The Internet, Summer 1999
34
Example C client code:
daytime(13)
This client gets the time from another machine and prints it
char recvline[MAXLINELEN+1];
/* allocate an internet (INET) stream (STREAM) socket */
int sockfd = socket(INET, STREAM, 0)
/* connect to the remote server
servaddr is a structure containing both the server IP
address and a port at that address (13 for time) */
connect(sockfd, &servaddr, sizeof(servaddr))
/* read the time from the remote server and print */
int n = read(sockfd, recvline, MAXLINELEN);
recvline[n] = 0;
/* null terminate the line */
printf(recvline); /* print the line */
/* take down the connection */
close(sockfd);
Lecture 6, 20-755: The Internet, Summer 1999
35
Example C server code:
daytime(13)
/* create a socket */
listenfd = socket(INET, STREAM, 0);
/* bind it to a local port.
Here servaddr specifies the local port number 13. */
bind(listenfd, &servaddr, sizeof(servaddr));
/* put socket into “listening” mode to receive request for
connection from anyone */
listen(listenfd, LISTENQ);
while (1) {
/* wait and accept next connection from a client
this creates a new “slave socket” connfd */
connfd = accept(listenfd, NULL, NULL);
ticks = time(NULL);
/* get the time of day */
sprintf(buff, “%s\n”, ctime(&ticks)); /* print as a string */
write(connfd, buff, strlen(buff));
/* write to client */
close(connfd);
/* close connection */
}
Lecture 6, 20-755: The Internet, Summer 1999
36
Example Perl5 client code:
#!/usr/local/bin/perl5 -w
use IO::Socket;
# daytime client
$port = “8000”;
$host = "euro.ecom";
# send TCP connection request
$sockfd = IO::Socket::INET->new(
Proto => "tcp",
# use tcp
PeerAddr => $host, # server name
PeerPort => $port) # and port number
or die "Couldn't connect to port $port on $host: $@\n";
# wait for the response from the server and print to stdout
$time = <$sockfd>;
print $time;
close $sockfd;
Lecture 6, 20-755: The Internet, Summer 1999
37
Example Perl5 server code
#!/usr/local/bin/perl5 -w
use IO::Socket;
# daytime server
$port = 8000;
# create a TCP listening socket file descriptor
$listenfd = IO::Socket::INET->new(
LocalPort => $port,
# port to listen on
Type => SOCK_STREAM, # use TCP
Reuse => 1,
# reuse addr right away
Listen => 10)
# buffer at most 10 requests
or die "Couldn't listen on port $port: $@\n";
# loop forever, waiting for client requests
while(1) {
$connfd = $listenfd->accept(); # wait for client request
$time = "The time is ".`date`; # compose message
$connfd->print($time);
# send to client
close $connfd;
# close socket
}
Lecture 6, 20-755: The Internet, Summer 1999
38
Client-server debugging hints
•
Debug on the local machine
– Client and server can both be running on the same
machine.
– By default, the local machine can be referenced by:
» domain name: localhost
» IP address: 127.0.0.1
•
Develop and debug the server first
– Use telnet as the universal client
– e.g., to debug a web server:
» % telnet localhost 80
•
Use print statements liberally to track the
server’s state and progress.
Lecture 6, 20-755: The Internet, Summer 1999
39
Hierarchical domain name space
•
•
•
•
unnamed root
Until 198x, domain name/IP
address mapping maintained
in HOSTS.TXT file at SRI.
Each new host manually
entered and copied to
backbone routers.
Explosive growth rendered
HOSTS.TXT approach
impractical.
Replaced by Domain Name
System in 198x.
Lecture 6, 20-755: The Internet, Summer 1999
com
edu
mit
cmu
gsia
cs
gov
berkeley
ece
cmcl
lb
sahara
www
128.2.185.40
mil
128.2.209.79
40
DNS
•
•
•
Worldwide distributed system for mapping
names to addresses (and vice versa).
Implemented as a collection of cooperating
servers called name servers.
name servers are accessed by DNS clients
– user programs
– nslookup
» stand-alone client with command line interface
Lecture 6, 20-755: The Internet, Summer 1999
41
Zones
•
•
Domains are partitioned
into zones.
Each zone has multiple
name servers that store
info about about names in
that zone.
unnamed root
com
edu
mit
cmu
gsia
cs
gov
mil
berkeley
– CS Zone has 4 servers
•
One server is authoritative
– the others get copies of the
authoritative server’s data
CS Zone in red
LB Zone in blue
Lecture 6, 20-755: The Internet, Summer 1999
cmcl
other cmcl sahara
names 128.2.185.40
ece
lb
other cs
names
other lb
128.2.209.79 names
42
www
Zone databases
•
•
Each name server keeps a database with
information about each name in its zone.
Examples of info (Type:Description)
–
–
–
–
–
–
A: IP address
NS: name servers for zone
SOA: “start of authority” indicates authoritative server
WKS: well known services running on that host
HINFO: host info (OS and machine type)
PTR: domain name ptr (if this subdomain has its own
server)
Lecture 6, 20-755: The Internet, Summer 1999
43
Zone transfers
•
Copying the contents of a zone database is
called a zone transfer.
– all info of a particular type or types (A, NS, etc) for the
entire zone.
Example: zone transfer of cs.cmu.edu (Types A & PTR)
(note: this is the default for nslookup)
...
SAHARA.CMCL
…
LB
LB
…
POSTOFFICE
...
128.2.185.40
server = ALMOND.SRV.CS.CMU.EDU
server = PECAN.SRV.CS.CMU.EDU
128.2.181.62
Lecture 6, 20-755: The Internet, Summer 1999
44
Zone transfers (cont)
Example: zone transfer of cs.cmu.edu (Type HINFO)
...
SAHARA.CMCL
…
AMEFS.SRV
...
DEC-600-5/333 UNIX
INTEL-486 UNIX
Note: no HINFO for POSTOFFICE or LB
Lecture 6, 20-755: The Internet, Summer 1999
45
Hierarchical DNS name
resolution
2. R
root name server
3. PTR to edu name server (ns)
1. sahara.cmcl.cs.cmu.edu (R)
client
4. R
name
server
edu name server
5. PTR to cmu.edu ns
6. R
10. 128.2.185.40
7. PTR to cs.cmu.edu ns
cmu.edu name server
8. R
9. 128.2.185.40
cs. cmu.edu name server
Lecture 6, 20-755: The Internet, Summer 1999
46
DNS Caching
•
•
•
Servers cache (keep a copy of) of information
they receive from other servers as part of the
name resolution process.
This greatly reduces the number of queries.
E.g. In our previous example, the next query
for sahara.cmcl can be answered immediately
because the server kept a copy of the
address.
1. sahara.cmcl.cs.cmu.edu (R)
client
name
server
10. 128.2.185.40
Lecture 6, 20-755: The Internet, Summer 1999
47