LECTURE 16
Twisted
A SIMPLE TCP SERVER
To begin our introduction to Twisted, we’ll build a simple TCP server.
REACTOR BASICS
At the core of any Twisted application is the reactor loop. The reactor loop works by
waiting for events to occur and then calls the relevant event handler (“dispatches the
event”). The simplest Twisted program is the following, which accesses the
reactor object and starts it.
from twisted.internet import reactor
reactor.run()
REACTOR BASICS
At the core of any Twisted application is the reactor loop. The reactor loop works by
waiting events to occur and then calls the relevant event handler (“dispatches the
event”). The simplest Twisted program is the following, which accesses the
reactor object and starts it.
from twisted.internet import reactor
reactor.run()
# Not created explicitly! Just import it.
The reactor loop runs in the main and only thread. Once we run the reactor, it
assumes control of the program. If there is nothing to do (as is the case here), the
reactor loop just sits idle, not consuming CPU resources.
REACTOR BASICS
The reactor provides basic interfaces to a number of services, including network
communications, threading, and event dispatching. There are several reactor options,
each specialized for a particular application. The default reactor on most systems is
a Select-based reactor.
To use a different reactor, you must “install” it.
from twisted.internet import pollreactor
pollreactor.install()
# When reactor is imported, it will be a pollreactor
from twisted.internet import reactor # Always import reactor right before you run it
reactor.run()
REACTOR BASICS
The core required functionality of a reactor is defined in
twisted.internet.interfaces.IReactorCore. These include:
• run() – move the reactor to a “running” state. Starts the main loop.
• stop() – shutdown the reactor.
• callWhenRunning() – call a function when the reactor is running.
• addSystemEventTrigger() -- add a function to be called when a system event occurs.
• fireSystemEvent() -- fire a system-wide event.
There are additional methods defined in IReactorProcess, IReactorTCP, IReactorSocket,
IReactorThreads, IReactorSSL, etc…
The choice of reactor determine which/how these methods are implemented.
REACTOR BASICS
Here’s an example Twisted program
which registers a function call to be
made when the reactor starts running.
Twisted has a basic logging mechanism
defined in twisted.python.log.
import sys
from twisted.python import log
log.startLogging(sys.stdout)
def func(x):
log.msg("In func as event handler")
log.msg(str(x))
log.msg(“Shutting down now!")
reactor.stop()
from twisted.internet import reactor
reactor.callWhenRunning(func, “Hello!")
reactor.run()
REACTOR BASICS
$ python twist.py
2015-02-17 11:46:16-0500 [-] Log opened.
2015-02-17 11:46:16-0500 [-] In func as event handler
2015-02-17 11:46:16-0500 [-] Hello!
2015-02-17 11:46:16-0500 [-] Shutting down now!
2015-02-17 11:46:16-0500 [-] Main loop terminated
import sys
from twisted.python import log
log.startLogging(sys.stdout)
def func(x):
log.msg("In func as event handler")
log.msg(str(x))
log.msg(“Shutting down now!")
reactor.stop()
from twisted.internet import reactor
reactor.callWhenRunning(func, “Hello!")
reactor.run()
REACTOR BASICS
Another example using the callLater scheduling method defined in IReactorTime.
import sys, time
from twisted.python import log
log.startLogging(sys.stdout)
def func(x):
log.msg(str(x))
now = time.localtime(time.time())
log.msg(str(time.strftime("%y/%m/%d %H:%M:%S", now)))
log.msg(“Shutting down now!")
reactor.stop()
now = time.localtime(time.time())
log.msg(str(time.strftime("%y/%m/%d %H:%M:%S", now)))
from twisted.internet import reactor
reactor.callLater(5, func, “Hello after 5 seconds!”)
reactor.run()
REACTOR BASICS
import sys, time
from twisted.python import log
log.startLogging(sys.stdout)
$ python twist.py
2015-02-17 11:49:10-0500
2015-02-17 11:49:10-0500
2015-02-17 11:49:15-0500
2015-02-17 11:49:15-0500
2015-02-17 11:49:15-0500
2015-02-17 11:49:15-0500
[-] Log opened.
[-] 15/02/17 11:49:10
[-] Hello after 5 seconds!
[-] 15/02/17 11:49:15
[-] Shutting down now!
[-] Main loop terminated.
def func(x):
log.msg(str(x))
now = time.localtime(time.time())
log.msg(str(time.strftime("%y/%m/%d %H:%M:%S", now)))
log.msg(“Shutting down now!")
reactor.stop()
now = time.localtime(time.time())
log.msg(str(time.strftime("%y/%m/%d %H:%M:%S", now)))
from twisted.internet import reactor
reactor.callLater(5, func, “Hello after 5 seconds!”)
reactor.run()
REACTOR BASICS
import sys, time
from twisted.python import log
log.startLogging(sys.stdout)
Let’s create a custom self-esteem boosting event.
def func(x):
log.msg(str(x))
now = time.localtime(time.time())
log.msg(str(time.strftime("%y/%m/%d %H:%M:%S",now)))
log.msg("Fire FireworkEvent after 3 seconds…")
reactor.callLater(3, reactor.fireSystemEvent, ‘FireworkEvent')
def Firework_CustomEventHandler():
now = time.localtime(time.time())
log.msg(str(time.strftime("%y/%m/%d %H:%M:%S",now)))
log.msg(“BABY YOU’RE A FIIIIIIIIREWOOOOOORK")
reactor.stop()
now = time.localtime(time.time())
log.msg(str(time.strftime("%y/%m/%d %H:%M:%S",now)))
from twisted.internet import reactor
reactor.callLater(5, func, "func called after 5 sec")
reactor.addSystemEventTrigger('during', ‘FireworkEvent', Firework_CustomEventHandler)
reactor.run()
REACTOR BASICS
import sys, time
from twisted.python import log
log.startLogging(sys.stdout)
Let’s create a custom self-esteem boosting event.
addSystemEventTrigger can be done
‘before’, ‘during’, or ‘after’ event.
def func(x):
log.msg(str(x))
now = time.localtime(time.time())
log.msg(str(time.strftime("%y/%m/%d %H:%M:%S",now)))
log.msg("Fire FireworkEvent after 3 seconds…")
reactor.callLater(3, reactor.fireSystemEvent, ‘FireworkEvent')
def Firework_CustomEventHandler():
now = time.localtime(time.time())
log.msg(str(time.strftime("%y/%m/%d %H:%M:%S",now)))
log.msg(“BABY YOU’RE A FIIIIIIIIREWOOOOOORK")
reactor.stop()
now = time.localtime(time.time())
log.msg(str(time.strftime("%y/%m/%d %H:%M:%S",now)))
from twisted.internet import reactor
reactor.callLater(5, func, "func called after 5 sec")
reactor.addSystemEventTrigger('during', ‘FireworkEvent', Firework_CustomEventHandler)
reactor.run()
REACTOR BASICS
import sys, time
from twisted.python import log
log.startLogging(sys.stdout)
$ python twist.py
2015-02-17 11:53:43-0500 [-] Log opened.
2015-02-17 11:53:43-0500 [-] 15/02/17 11:53:43
2015-02-17 11:53:48-0500 [-] func called after 5 sec
2015-02-17 11:53:48-0500 [-] 15/02/17 11:53:48
2015-02-17 11:53:48-0500 [-] Fire FireworkEvent after 3 seconds...
2015-02-17 11:53:51-0500 [-] 15/02/17 11:53:51
2015-02-17 11:53:51-0500 [-] BABY YOU'RE A FIIIIIIIIREWOOOOOORK
2015-02-17 11:53:51-0500 [-] Main loop terminated.
def func(x):
log.msg(str(x))
now = time.localtime(time.time())
log.msg(str(time.strftime("%y/%m/%d %H:%M:%S",now)))
log.msg("Fire FireworkEvent after 3 seconds…")
reactor.callLater(3, reactor.fireSystemEvent, ‘FireworkEvent')
def Firework_CustomEventHandler():
now = time.localtime(time.time())
log.msg(str(time.strftime("%y/%m/%d %H:%M:%S",now)))
log.msg(“BABY YOU’RE A FIIIIIIIIREWOOOOOORK")
reactor.stop()
now = time.localtime(time.time())
log.msg(str(time.strftime("%y/%m/%d %H:%M:%S",now)))
from twisted.internet import reactor
reactor.callLater(5, func, "func called after 5 sec")
reactor.addSystemEventTrigger('during', ‘FireworkEvent', Firework_CustomEventHandler)
reactor.run()
REACTOR BASICS
Let’s back up for a second and make sure we understand what’s going on.
• Our callback code runs in the same thread as the Twisted loop.
• When our callbacks are running, the Twisted loop is not running. And vice versa.
• The reactor loop resumes when our callback returns.
Our code blocks the reactor loop so it should return asap.
Before we continue writing our TCP server, we need to learn about a couple of
abstractions.
TRANSPORT BASICS
The Transport abstraction is defined in twisted.internet.interfaces.ITransport.
A Twisted Transport represents a single connection that can send/receive data.
The Transport abstraction represents any such connection and handles the details of
asynchronous I/O for whatever sort of connection it represents. The methods defined in
Itransport are:
• write(data) – send some data.
• writeSequence(list_of_data) – send a sequence of data.
• loseConnection() – close connection.
• getPeer() – get remote address of other side of connection.
• getHost() – get address of this side of connection.
TRANSPORT BASICS
You may have noticed that there are no methods for reading data. The Transport
object will make a callback when it receives data – we don’t have to explicitly make
it happen.
Also, note that these methods are really just suggestions. Remember, Twisted is in
control, not us. So when we tell the Transport to write some data, we’re really asking
it to write some data whenever it is able to.
PROTOCOLS
The Protocol abstraction is defined in twisted.internet.interfaces.IProtocol.
Protocols implement protocols. This could be one of the built-in protocols in
twisted.protocols.basic or one of your own design.
Strictly speaking, a Protocol instance implements the protocol for a single connection.
This connection is represented by a Transport object.
So every connection requires its own Protocol instance (and therefore, Protocol is a
good candidate for storing stateful information about the connection).
PROTOCOLS
Methods of the IProtocol class include:
• dataReceived(data) – called whenever data is received (transport’s callback!)
• connectionLost(reason) – called when connection is shut down.
• makeConnection(transport) – associates transport with protocol instance to make
connection.
• connectionMade() – called when a connection is made.
PROTOCOL FACTORIES
The Protocol Factory object is defined in twisted.internet.interfaces.IProtocolFactory.
Protocol Factories simply create Protocol instances for each individual connection. Of
interest is just one method:
• buildProtocol(addr): The buildProtocol method is supposed to return a new Protocol
instance for the connection to addr. Twisted will call this method to establish Protocols
for connections.
SIMPLE TCP SERVER
from twisted.internet.protocol import Protocol, Factory
class Echo(Protocol):
def dataReceived(self, data):
self.transport.write(data)
f = Factory()
f.protocol = Echo
from twisted.internet import reactor
reactor.listenTCP(9000, f)
reactor.run()
# reactor will call makeConnection with current transport instance.
# when transport executes callback with data, just echo
# f.buildProtocol() will create an instance of f.protocol
# listenTCP defined in IReactorTCP, which the default reactor inherits
SIMPLE TCP SERVER
from twisted.internet.protocol import Protocol, Factory
class Echo(Protocol):
def dataReceived(self, data):
self.transport.write(data)
f = Factory()
f.protocol = Echo
from twisted.internet import reactor
reactor.listenTCP(9000, f)
reactor.run()
Try “telnet localhost 9000”.
Anything you send will be echoed back to you.
The only thing we’re doing explicitly is running
the server. Everything that is executed from that
point forward is handled and scheduled by Twisted.