Network Programming – TCP/IP Socket Programming
Chapter-1
Network Programming – An Overview
Introduction:
Network programming is the act of using computer code to write programs or processes that
can communicate with other programs or processes across a network. Programmers use various
programming languages, code libraries, and protocols to do the work. The world is full of such type
of programs: for example, the web browser you are using to read this website is a piece of software
that connects to a remote computer where the data is stored and grabs the text content to display on
your screen. The browser and the web server can do their networking job thanks to the operating
systems they run on, where all the necessary network protocols have been implemented. The operating
system's parts that provide network functionality are called sockets.
A socket is an abstraction over a communication flow. Concretely, sockets are programming
objects provided by the operating system that allow your programs to send and receive data.
1. network sockets — they are used to exchange data between programs over a network, or in
other words between two remote hosts.
2. Unix Domain sockets — also known as UNIX sockets, they are used to exchange data
between programs running on the same machine (i.e. in the same host). This is a form of InterProcess Communication (IPC).
Python plays an essential role in network programming. The standard library of Python has
full support for network protocols, encoding, and decoding of data and other networking concepts,
and it is simpler to write network programs in Python than that of C++.
There are two levels of network service access in Python. These are:
•
•
Low-Level Access
High-Level Access
In the first case, programmers can use and access the basic socket support for the operating
system using Python's libraries, and programmers can implement both connection-less and
connection-oriented protocols for programming.
Application-level network protocols can also be accessed using high-level access provided by
Python libraries. These protocols are HTTP, FTP, etc.
1
Network Programming – TCP/IP Socket Programming
Why Python is Preferred for Network Programming and not others?
The question is rather vague. Python is a general-purpose programming language; it has
libraries that facilitates writing network-based applications. Hence it can be used for "network
programming". So can Java, C#, Ruby etc.
The question you might be trying to ask is perhaps "Why use Python for network programming
instead of X?"
If that is the case, these are valid reasons:
1. The network programmer might prefer Python over another language.
2. Python has some really good libraries that make network programming easier, e.g. Twisted,
AsyncIO
Other reasons include:
•
•
•
•
Python for Efficient Documenting
Python to Collect MAC Addresses
Python to Speed Up Processes
Python Increases Network Reliability
2
Network Programming – TCP/IP Socket Programming
Important Python Libraries used for Network programming & Automation :
•
Asyncio (Python standard library) Asynchronous I/O, event loop, coroutines and tasks)
This module provides infrastructure for writing single-threaded concurrent code using
coroutines, multiplexing I/O access over sockets and other resources, running network
clients and servers, and other related primitives.
•
NAPALM – Cross-vendor API to manipulate network devices.
NAPALM (Network Automation and Programmability Abstraction Layer with
Multivendor support) is a Python library that implements a set of functions to interact
with different router vendor devices using a unified API.
•
Pulsar – Event-driven concurrent framework for Python.
Pulsar’s goal is to provide an easy way to build scalable network programs. Pulsar uses
the asyncio module from the standard python library and it can be configured to run in
multi-processing mode.
•
Diesel – Greenlet-based event I/O Framework for Python.
It provides a clean API for writing network clients and servers. TCP and UDP
supported. It bundles battle-tested clients for HTTP, DNS, Redis, Riak and MongoDB.
It makes writing network applications fun.
Python Module
httplib, urllib,
xmlrpclib
nntplib
ftplib, urllib
smtplib
poplib
imaplib
telnetlib
gopherlib, urllib
Protocol
Function
Port
Number
For HTTP protocol
It deals with web pages
80
For NNTP protocol
For FTP protocol
For SMTP protocol
For POP3 protocol
For IMAP4 protocol
For Telnet protocol
It deals with gopher
protocol
It deals with Usenet news
It deals with the transfer of files
It deals with sending mails
It is used for fetching mails
It is used for fetching mails
It deals with command lines
It deals with
transferring documents
119
20
25
110
143
23
70
3
Network Programming – TCP/IP Socket Programming
Chapter-2
Networking Essentials:
1. IP Addressing:
An IP address (short for Internet Protocol address) is a label which is used to
identify one or more devices on a computer network, such as the internet. It can be
compared to a postal address. An IP address is a long number written in binary. Since such
numbers are difficult to communicate, IP addresses are usually written as a set of numbers
in a given order. Devices using IP addresses use the internet protocol to communicate.
The address is made up of 32 binary bits, which can be divisible into a network
portion and host portion with the help of a subnet mask. The 32 binary bits are broken into
four octets (1 octet = 8 bits). Each octet is converted to decimal and separated by a period
(dot). For this reason, an IP address is said to be expressed in dotted decimal format (for
example, 172.16.81.100). The value in each octet ranges from 0 to 255 decimal, or
00000000 - 11111111 binary.
Internet Protocol version 4 (IPv4) defines an IP address as a 32-bit number.
However, because of the growth of the Internet and the depletion of available IPv4
addresses, a new version of IP (IPv6), using 128 bits for the IP address, was standardized
in 1998. IPv6 deployment has been ongoing since the mid-2000s.
4
Network Programming – TCP/IP Socket Programming
2.
IPv4 Vs IPv6:
IPv4
IPv6
IPv4 has a 32-bit address length
IPv6 has a 128-bit address length
It Supports Manual and DHCP address
configuration
It supports Auto and renumbering address configuration
In IPv4 end to end, connection integrity is
Unachievable
In IPv6 end to end, connection integrity is Achievable
It can generate 4.29×109 address space
Address space of IPv6 is quite large it can produce
3.4×1038 address space
The Security feature is dependent on
application
IPSEC is an inbuilt security feature in the IPv6 protocol
Address representation of IPv4 is in
decimal
Address Representation of IPv6 is in hexadecimal
Fragmentation performed by Sender and
forwarding routers
In IPv6 fragmentation performed only by the sender
In IPv4 Packet flow identification is not
available
In IPv6 packet flow identification are Available and uses
the flow label field in the header
In IPv4 checksum field is available
In IPv6 checksum field is not available
It has broadcast Message Transmission
Scheme
In IPv6 multicast and anycast message transmission
scheme is available
In IPv4 Encryption and Authentication
facility not provided
In IPv6 Encryption and Authentication are provided
IPv4 has a header of 20-60 bytes.
IPv6 has header of 40 bytes fixed
IPv4 can be converted to IPv6
Not all IPv6 can be converted to IPv4
IPv4 consist of 4 fields which are separated
by dot (.)
IPv6 consist of 8 fields, which are separated by colon (:)
IPv4’s IP addresses are divided into five
IPv6 does not have any classes of IP address.
different classes. Class A , Class B, Class C
, Class D , Class E.
IPv4 supports VLSM(Variable Length
subnet mask).
IPv6 does not support VLSM.
Example of IPv4: 66.94.29.13
Example of IPv6:
2001:0000:3238:DFE1:0063:0000:0000:FEFB
5
Network Programming – TCP/IP Socket Programming
3. IPv4 Notation, Classful and Classless addressing:
•
Classful Addressing:
The 32-bit IP address is divided into five sub-classes. These are:
•
•
•
•
•
Class A
Class B
Class C
Class D
Class E
Each of these classes has a valid range of IP addresses. Classes D and E are reserved for
multicast and experimental purposes respectively. The order of bits in the first octet determines the
classes of IP address. IPv4 address is divided into two parts:
•
•
Network ID
Host ID
The class of IP address is used to determine the bits used for network ID and host ID and the
number of total networks and hosts possible in that particular class. Each ISP or network administrator
assigns IP address to each device that is connected to its network.
•
Classless Addressing:
To reduce the wastage of IP addresses in a block, we use sub-netting. What we do is that we
use host id bits as net id bits of a classful IP address. We give the IP address and define the number of
bits for mask along with it (usually followed by a ‘/’ symbol), like, 192.168.1.1/28. Here, subnet mask
is found by putting the given number of bits out of 32 as 1, like, in the given address, we need to put
28 out of 32 bits as 1 and the rest as 0, and so, the subnet mask would be 255.255.255.240.
6
Network Programming – TCP/IP Socket Programming
Subnetting: Dividing a large block of addresses into several contiguous sub-blocks and
assigning these sub-blocks to different smaller networks is called subnetting. It is a practice that is
widely used when classless addressing is done.
4. IPv6:
IPv6 or Internet Protocol Version 6 is a network layer protocol that allows
communication to take place over the network. IPv6 was designed by Internet Engineering Task Force
(IETF) in December 1998 with the purpose of superseding the IPv4 due to the global exponentially
growing internet users.
7
Network Programming – TCP/IP Socket Programming
5. Ports:
In computer networking, a port or port number is a number assigned to uniquely identify
a connection endpoint and to direct data to a specific service. At the software level, within an
operating system, a port is a logical construct that identifies a specific process or a type of network
service. A port at the software level is identified for each transport protocol and address
combination by the port number assigned to it. The most common transport protocols that use
port numbers are the Transmission Control Protocol (TCP) and the User Datagram Protocol
(UDP); those port numbers are 16-bit unsigned numbers.
A port number is always associated with a network address of a host, such as an IP
address, and the type of transport protocol used for communication. It completes the destination
or origination address of a message. Specific port numbers are reserved to identify specific
services so that an arriving packet can be easily forwarded to a running application. For this
purpose, port numbers lower than 1024 identify the historically most commonly used services
and are called the well-known port numbers. Higher-numbered ports are available for general use
by applications and are known as ephemeral ports.
Port numbers are part of the addressing information that helps identify senders and
receivers of information and a particular application on the devices. Port numbers consist of 16bit numbers.
For example, a user request for a file transfer from a client, or local host, to a remote
server on the internet uses File Transfer Protocol (FTP) for the transaction. Both devices must be
configured to transfer files via FTP. To transfer the file, the Transmission Control Protocol (TCP)
software layer in local host identifies the port number of 21, which, by convention, associates
with an FTP request -- in the 16-bit port number integer that is appended to the request.
At the server, the TCP layer will read port number 21 and forward the request to the FTP program
at the server.
8
Network Programming – TCP/IP Socket Programming
Port
Number
7
20
Service name
Echo
FTP-data
Transport
protocol
TCP, UDP
TCP, SCTP
21
FTP
22
SSH-SCP
23
Telnet
TCP, UDP,
SCTP
TCP, UDP,
SCTP
TCP
25
SMTP
TCP
53
69
80
DNS
TFTP
HTTP
TCP, UDP
UDP
TCP, UDP,
SCTP
FTPS Protocol (data), FTP over TLS/SSL
110
135
POP3
Microsoft
EPMAP
TCP
TCP, UDP
137
NetBIOS-ns
TCP, UDP
139
143
NetBIOS-ssn
IMAP4
TCP, UDP
TCP, UDP
381
383
443
HP Openview
HP Openview
HTTP over
SSL
TCP, UDP
TCP, UDP
TCP, UDP,
SCTP
464
465
Kerberos
SMTP over
TLS/SSL,
SSM
SMTP
Microsoft
DCOM
TCP, UDP
TCP
LDAP over
TLS/SSL
MS Exchange
VMware
Server
FTP over SSL
TCP, UDP
989
Secure Shell, secure logins, file transfers (scp,
sftp), and port forwarding
Telnet protocol—unencrypted text
communications
Simple Mail Transfer Protocol, used for email
routing between mail servers
Domain Name System name resolver
Trivial File Transfer Protocol
Hypertext Transfer Protocol (HTTP) uses TCP
in versions 1.x and 2.
TCP, UDP
TCP, UDP
TCP
691
902
File Transfer Protocol (FTP) control connection
TCP
unofficial
Kerberos
Iso-tsap
636
Echo service
File Transfer Protocol data transfer
HTTP/3 uses QUIC, a transport protocol on top
of UDP
Network authentication system
ISO Transport Service Access Point (TSAP)
Class 0 protocol
Post Office Protocol, version 3 (POP3)
Microsoft EPMAP (End Point Mapper), also
known as DCE/RPC Locator service, used to
remotely manage services including DHCP
server, DNS server, and WINS. Also used by
DCOM
NetBIOS Name Service, used for name
registration and resolution
NetBIOS Session Service
Internet Message Access Protocol (IMAP),
management of electronic mail messages on a
server
HP data alarm manager
HP data alarm manager
Hypertext Transfer Protocol Secure (HTTPS)
uses TCP in versions 1.x and 2. HTTP/3 uses
QUIC, a transport protocol on top of UDP.
Kerberos Change/Set password
Authenticated SMTP over TLS/SSL (SMTPS),
URL Rendezvous Directory for SSM (Cisco
protocol)
Email message submission
HTTP RPC Ep Map, Remote procedure call
over Hypertext Transfer Protocol, often used by
Distributed Component Object Model services
and Microsoft Exchange Server
Lightweight Directory Access Protocol over
TLS/SSL
MS Exchange Routing
VMware ESXi
88
102
587
593
Description
TCP
TCP, UDP
9
Network Programming – TCP/IP Socket Programming
990
993
FTP over SSL
IMAP4 over
SSL
POP3 over
SSL
Microsoft
RPC
TCP, UDP
TCP
TCP
Microsoft operating systems tend to allocate one
or more unsuspected, publicly exposed services
(probably DCOM, but who knows) among the
first handful of ports immediately above the end
of the service port range (1024+).
1194
1337
1589
1725
2082
2083
OpenVPN
WASTE
Cisco VQP
Steam
cPanel
radsec, cPanel
TCP, UDP
unofficial
TCP, UDP
UDP
unofficial
TCP, UDP
2483
Oracle DB
TCP, UDP
2484
Oracle DB
TCP, UDP
2967
3074
3306
3724
TCP, UDP
TCP, UDP
TCP
TCP, UDP
6665
6669
6881
Symantec AV
XBOX Live
MySQL
World of
Warcraft
Google
Desktop
PostgreSQL
RFB/VNC
Server
IRC
IRC
BitTorrent
OpenVPN
WASTE Encrypted File Sharing Program
Cisco VLAN Query Protocol (VQP)
Valve Steam Client uses port 1725
cPanel default
Secure RADIUS Service (radsec), cPanel
default SSL
Oracle database listening for insecure client
connections to the listener, replaces port 1521
Oracle database listening for SSL client
connections to the listener
Symantec System Center agent (SSC-AGENT)
Xbox LIVE and Games for Windows – Live
MySQL database system
Some Blizzard games, Unofficial Club Penguin
Disney online game for kids
Google Desktop Search
6999
BitTorrent
unofficial
6970
8086
8087
8222
unofficial
TCP
UDP
TCP, UDP
9100
Quicktime
Kaspersky AV
Kaspersky AV
VMware
Server
PDL
10000
BackupExec
unofficial
12345
NetBus
unofficial
27374
18006
Sub7
Back Orifice
unofficial
unofficial
995
1025
4664
5432
5900
TCP, UDP
unofficial
TCP
TCP, UDP
TCP
TCP
unofficial
TCP
FTPS Protocol (control), FTP over TLS/SSL
Internet Message Access Protocol over TLS/SSL
(IMAPS)
Post Office Protocol 3 over TLS/SSL
PostgreSQL database system
virtual Network Computing (VNC) Remote
Frame Buffer RFB protocol
Internet Relay Chat
Internet Relay Chat
BitTorrent is part of the full range of ports used
most often
BitTorrent is part of the full range of ports used
most often
QuickTime Streaming Server
Kaspersky AV Control Center
Kaspersky AV Control Center
VMware Server Management User Interface
(insecure Web interface).
PDL Data Stream, used for printing to certain
network printers[1
Webmin, Web-based Unix/Linux system
administration tool (default port)
NetBus remote administration tool (often Trojan
horse).
Sub7 default
Back Orifice 2000 remote administration tools
10
Network Programming – TCP/IP Socket Programming
Chapter - 3
SOCKETS and Socket Programming:
Sockets:
A socket can be thought of as an endpoint in a two-way communication channel. Socket
routines create the communication channel, and the channel is used to send data between application
programs either locally or over networks. Each socket within the network has a unique name
associated with it called a socket descriptor—a full word integer that designates a socket and allows
application programs to refer to it when needed. Using an electrical analogy, you can think of the
communication channel as the electrical wire with its plug and think of the port, or socket, as the
electrical socket or outlet, as shown
This figure shows many application programs running on a client and many application
programs on a server. When the client starts a socket call, a socket connection is made between an
application on the client and an application on the server.
Another analogy used to describe socket communication is a telephone conversation. Dialling
a phone number from your telephone is similar to starting a socket call. The telephone switching unit
knows where to logically make the correct switch to complete the call at the remote location. During
your telephone conversation, this connection is present and information is exchanged. After you hang
up, the connection is broken and you must start it again. The client uses the socket() function call to
start the logical switch mechanism to connect to the server.
Types of Sockets : There are two types of Sockets: the datagram socket and the stream
socket.
11
Network Programming – TCP/IP Socket Programming
1. Datagram Socket : This is a type of network which has connection less point for sending and
receiving packets. It is similar to mailbox. The letters (data) posted into the box are collected
and delivered (transmitted) to a letterbox (receiving socket).
2. Stream Socket In Computer operating system, a stream socket is type of interprocess
communications socket or network socket which provides a connection-oriented, sequenced,
and unique flow of data without record boundaries with well-defined mechanisms for creating
and destroying connections and for detecting errors. It is similar to phone. A connection is
established between the phones (two ends) and a conversation (transfer of data) takes place.
Function Call
Socket()
Bind()
Listen()
Connect()
Accept()
Write()
Read()
Close()
Description
To create a socket
It’s a socket identification like a telephone number to contact
Ready to receive a connection
Ready to act as a sender
Confirmation, it is like accepting to receive a call from a sender
To send data
To receive data
To close a connection
To understand python socket programming, we need to know about three interesting topics Socket Server, Socket Client and Socket. So, what is a server? Well, a server is a software that waits
for client requests and serves or processes them accordingly. On the other hand, a client is requester
of this service. A client program request for some resources to the server and server responds to that
request. Socket is the endpoint of a bidirectional communications channel between server and client.
12
Network Programming – TCP/IP Socket Programming
Sockets may communicate within a process, between processes on the same machine, or between
processes on different machines.
Direct / Bind Connections:
Direct connection is a method to remotely access another PC using your own computer
using a command shell or terminal.
Procedure for direct connection:
•
•
•
we create a socket on our computer.
Then we bind our code and host to the socket and send a request to the other computer on
that computer’s IP address.
If the other PC accepts the connection request, we can remotely access that PC through
our command prompt/terminal.
Why direct Connection is not preferred?
•
•
•
It is difficult to get IP address
Even if we get his IP address the IP address is Dynamic IP so its always changing
Even we could get regular updates to the dynamic IP address the computer has bunch of
firewalls which makes it impossible to get to his computer.
13
Network Programming – TCP/IP Socket Programming
Reverse Shell:
In a typical remote system access scenario, the user is the client and the target machine is the
server. The user initiates a remote shell connection and the target system listens for such
connections. With a reverse shell, the roles are opposite. It is the target machine that initiates the
connection to the user, and the user’s computer listens for incoming connections on a specified
port.
The primary reason why reverse shells are often used by attackers is the way that most
firewalls are configured. Attacked servers usually allow connections only on specific ports. For
example, a dedicated web server will only accept connections on ports 80 and 443. This means that
there is no possibility to establish a shell listener on the attacked server.
On the other hand, firewalls usually do not limit outgoing connections at all. Therefore, an attacker
may establish a server on their own machine and create a reverse connection. All that the attacker
needs is a machine that has a public (routable) IP address and a tool such as netcat to create the
listener and bind shell access to it.
In general, a reverse shell on its own is not malicious and can also be used for legitimate
purposes, for example, for remote server administration. If you don’t need to use reverse shells, you
can try to limit the possibility to create them but it is very difficult:
•
•
You can impose strict control of outgoing connections. However, this is only possible for
very specialized servers. And there is nothing to stop the attacker from opening a listener on
a common port such as 80. In such a case, all connections would have to be monitored for
content as well.
You can disable most tools that make it possible to create a reverse shell, but again this is only
possible for very specialized servers. As you can see above, reverse shells can be created using
different tools and languages. Therefore, you may make it more difficult for the attacker, but
not impossible.
14
Network Programming – TCP/IP Socket Programming
Even if you succeed in avoiding reverse shells, there are other methods that the attacker can use
to establish control over the system. For example, in some cases, they may use web shells instead.
Reverse shells on their own are always a result of some other kind of attack, for example, an SQL
Injection attack. Therefore, the best way to avoid reverse shells is to protect against attacks that allow
impostors to gain shell access in the first place.
Multi-Client Reverse Shell
In multi-client Reverse shell, a single server can control multiple client systems through reverse
shelling. It requires modification of single client reverse shell python program so that it can handle
multiple clients. Also, it can be enhanced using multi-threading.
Why Multithreading?
Multithreading is a way to run more than one command at a time. Where PowerShell normally uses a
single thread, there are many ways to use more than one to parallelize your code. The primary benefit
of multithreading is to decrease the runtime of the code. This time decrease is at the tradeoff a higher
processing power requirement.
So, by having the shell process multiple client connections through multiple threads, we are saving
runtime.
The tradeoff being processing power is tackled in high end server workstations which control multiple
PC’s using server-level CPU’s which have more than 8 cores.
15