Our ‘nic.c’ module
We create a ‘character-mode’
device-driver for the 82573L NIC
to use in future experiments
No way to communicate
SOFTWARE
Linux operating system kernel
(no knowledge of the NIC)
These
components
lack a way
to interact
HARDWARE
Network interface controller
(no knowledge of the OS)
Role for a ‘device driver’
SOFTWARE
Linux operating system kernel
(no knowledge of the NIC)
device driver module
(knows about both the OS and the NIC)
HARDWARE
Network interface controller
(no knowledge of the OS)
Character devices
• The concept of a ‘character mode’ device is that
it provides a ‘stream’ of data bytes that an
application-programs can access by using
standard Linux system-calls:
–
–
–
–
–
‘open()’
‘write()’
‘read()’
‘close()’
Maybe other ‘methods’ (depending on the device)
Requirement
• The character-mode devices are known to
a Linux system by their ‘names’ and by an
associated pair of ID-numbers (major and
minor) normally set up by a ‘superuser’
root# mknod /dev/nic c 97 0
root# chmod a+rw /dev/nic
Driver module
nic.c
my_proc_rx()
this module’s
collection of
‘payload’
functions
my_proc_tx()
isr()
read()
write()
fops
required pair of
module
administration
functions
module_init
module_exit
character-mode
device-driver’s
required
‘file-operations’
data-structure
Overview
user space
standard
runtime
libraries
kernel space
Linux
operating system
Kernel
file subsystem
application
program
networking subsystem
char driver module
hardware
We bypass use
of the TCP/IP
protocol stack
for our demo
Transmit operation
user space
kernel space
Linux OS kernel
runtime library
write()
file subsystem
nic device-driver
my_write()
user data-buffer
copy_from_user()
packet buffer
DMA
application program
hardware
Receive operation
user space
kernel space
Linux OS kernel
runtime library
read()
file subsystem
nic device-driver
my_read()
user data-buffer
copy_to_user()
packet buffer
application program
DMA
hardware
We ‘tweak’ our packet-format
• Our ‘nictx.c’ and ‘nicrx.c’ modules elected
to use the Ethernet-header’s Type/Length
field for storing the number of data-bytes
• But this allows our driver to be misled by
packet’s which don’t follow that scheme
• So we added a ‘signature’ and relocated
our ‘count’ field to the ‘payload’ region
0
6
destination MAC-address
12
source MAC-address
actual bytes of user-data
Type/Len
14
count
Using ‘echo’ and ‘cat’
• Our device-driver module ‘nic.c’ is intended
to allow two programs that are running on a
pair of our classroom PCs to communicate
via their ‘secondary’ Ethernet interfaces
Transmitting…
$ echo Hello > /dev/nic
$_
Receiving…
$ cat /dev/nic
Hello
_
Exploring RX Descriptor FIFO
Timeout for an in-class demonstration
‘push’ versus ‘pull’
Host memory
transmit
packet
buffer
Ethernet controller
push
transmit-FIFO
to/from
LAN
receive
packet
buffer
pull
receive-FIFO
The ‘write()’ routine in our ‘nic.c’ driver can transfer some data anytime there
is at least one Tx-descriptor that is not “owned” by the network hardware, but
the ‘read()’ routine in our ‘nic.c’ driver will have to wait for some data to arrive.
So to avoid doing any wasteful busy-waiting, our ‘nic.c’ driver can utilize the
Linux kernel’s sleep/wakeup mechanism – if it enables the NIC’s interrupts!
Sleep/wakeup
• We will need to employ a wait-queue, we
will need to enable device-interrupts, and
we will need to write and install the code
for an interrupt service routine (ISR)
• So our ‘nic.c’ device-driver will have a few
additional code and data components that
were absent from our ‘nictx.c’ and ‘nicrx.c’
kernel-module demos
How NIC’s interrupts work
• There are four interrupt-related registers
which are essential for us to understand
0x00C0
ICR
Interrupt Cause Read
0x00C8
ICS
Interrupt Cause Set
0x00D0
IMS
Interrupt Mask Set/Read
0x00D8
IMC
Interrupt Mask Clear
Interrupt-event types
31
30
18 17 16 15 14
reserved
10
9
8
7
6
5
4
2
1
reserved
31: INT_ASSERTED (1=yes,0=no)
17: ACK (Rx-ACK Frame detected)
16: SRPD (Small Rx-Packet detected)
15: TXD_LOW (Tx-Descr Low Thresh hit)
9: MDAC (MDI/O Access Completed)
7: RXT0 ( Receiver Timer expired)
6: RXO (Receiver Overrun)
4: RXDMT0 (Rx-Desc Min Thresh hit)
2: LSC (Link Status Change)
1: TXQE( Transmit Queue Empty)
0: TXDW (Transmit Descriptor Written Back)
82573L
0
Interrupt Mask Set/Read
• This register is used to enable a selection
of the device’s interrupts which the driver
will be prepared to recognize and handle
• A particular interrupt becomes ‘enabled’ if
software writes a ‘1’ to the corresponding
bit of this Interrupt Mask Set register
• Writing ‘0’ to any register-bit has no effect,
so interrupts can be enabled one-at-a-time
Interrupt Mask Clear
• Your driver can discover which interrupts
have been enabled by reading IMS – but
your driver cannot ‘disable’ any interrupts
by writing to that register
• Instead a specific interrupt can be disabled
by writing a ‘1’ to the corresponding bit in
the Interrupt Mask Clear register
• Writing ‘0’ to a register-bit has no effect on
the interrupt controller’s Interrupt Mask
Interrupt Cause Read
• When interrupts occur, your driver’s ‘interrupt
service routine’ (isr) discovers which specific
conditions triggered the interruption if it reads
the NIC’s Interrupt Cause Read register
• In this case your driver can clear any selection of
these bits (except bit #31) by writing ‘1’s to them
(writing ‘0’s to this register will have no effect)
• If case no interrupt has occurred, reading this
register may have the side-effect of clearing it
Interrupt Cause Set
• For testing your driver’s interrupt-handler,
you can artificially trigger any particular
combination of interrupts by writing ‘1’s
into the corresponding register-bits of this
Interrupt Cause Set register (assuming
your combination of bits corresponds to
interrupts that are ‘enabled’ by ‘1’s being
present for them in the Interrupt Mask)
Our interrupt-handling
• We decided to enable only a subset of the
possible interrupt causes (those related to
transmits, receives, and link-changes):
#define INTR_MASK
(1<<0)|(1<<1)|(1<<15)|(1<<2)|(1<<4)|(1<<6)|(1<<7)
transmit-related
receive-related
link-status changed
• The ones we take action on are TXDW,
RXT0, and RXDMT0
NIC “owns” some Rx-descriptors
RDBAH/RDBAL
RDLEN
=0x80
0
1
2
3
4
5
6
7
Our ‘static’ variables
rxhead
pickup
descriptor 0
descriptor 1
descriptor 2
descriptor 3
RDH
This register gets
initialized to 0, then
gets changed by the
controller as new
packets are received
descriptor 4
RDT
descriptor 5
This register gets
advanced by 4 when
the NIC’s supply of
descriptors drops
below a programmed
threshold
descriptor 6
descriptor 7
Keeps track of where packet-data resumes
In-class exercise
• Login to a pair of classroom machines
• Install our ‘nic.ko’ module on both hosts
• Try transferring a textfile from one of the
machines to the other, by using ‘cat’:
hrn23501$ cat textfile > /dev/nic
hrn23502$ cat /dev/nic > recv1000.out
• How large a textfile can you successfully
transfer using our Linux device-driver?
• Any problems encountered doing this?