What’s needed to transmit?
A look at the minimum steps
required for programming our
anchor nic’s to send packets
Access to PRO/1000 registers
• Device registers are hardware mapped to
a range of addresses in physical memory
• You obtain the location (and the length) of
this memory-range from a BAR register in
the nic device’s PCI Configuration Space
• Then you request the Linux kernel to setup
an I/O ‘remapping’ of this memory-range to
‘virtual’ addresses within kernel-space
i/o-memory remapping
Local-APIC
IO-APIC
nic
registers
APIC registers
nic registers
vram
1-GB
kernel code/data
vram
user
space
dynamic
ram
physical address-space
‘virtual’ address-space
3-GB
Enhanced ‘82573.c’ module
• Originally we created this module to view
the PCI Configuration Space for our nic
• Today we’ve added two character-mode
device-driver ‘methods’ allowing us to see
values in all of the nic’s internal registers
• You can use our ‘fileview’ utility to ‘read’
and ‘seek’ in the ‘/dev/82573’ device-file
(or you could write your own application)
An observation
• We notice that the 82573L device retains
the values in many of its internal registers
• This fact reduces the programming steps
that will be required to operate our nic on
the anchor cluster machines, since Intel’s
own Linux device driver (‘e1000.ko’) has
already initialized many nic registers
• But we WILL need to bring ‘eth1’ down!
Use ‘/sbin/ifconfig’
• You can use the ‘/sbin/ifconfig’ command
to find out whether the ‘eth1’ interface has
been turned off:
$ /sbin/ifconfig eth1
• If it is still operating, you can turn it off with
the (privileged) command:
$ sudo /sbin/ifconfig eth1 down
Our ‘nicping.c’ demo
• We created this module to help us identify
all of the programming steps that would be
needed for our nic to ‘transmit’ a packet
• The action of displaying this module’s
pseudo-file (e.g., with the ‘cat’ command)
will trigger transmission of a ‘broadcast’
message to all the other stations on our
anchor cluster’s LAN (if they are ‘listening’)
Tx-Desc Ring-Buffer
0x00
TDBA
base-address
0x10
0x20
TDH (head)
0x30
TDLEN
(in bytes)
0x40
0x50
0x60
TDT (tail)
0x70
0x80
= owned by hardware (nic)
= owned by software (cpu)
Circular buffer (128-bytes minimum)
How ‘transmit’ normally works
List of Buffer-Descriptors
descriptor0
descriptor1
descriptor2
descriptor3
0
0
0
0
Buffer0
Buffer1
Buffer2
We setup each data-packets that we want
to be transmitted in a ‘Buffer’ area in ram
We also create a list of buffer-descriptors
and inform the NIC of its location and size
Then, when ready, we tell the NIC to ‘Go!’
(i.e., start transmitting), but let us know
when these transmissions are ‘Done’
Buffer3
Random Access Memory
How ‘transmit’ works here
List of Buffer-Descriptors
TDH
TDT
descriptor0
descriptor1
descriptor2
descriptor3
descriptor4
descriptor5
descriptor6
descriptor7
Packet
Buffer
We setup just one packet-buffer, and we
let all of our descriptors pointing to it
We retain ‘ownership’ of all our descriptors,
so nothing gets transmitted, until we have
prepared all our data in the packet-buffer
Then, when ready, we tell the NIC to ‘Go!’
(i.e., start transmitting), by giving the nic
‘ownership’ of only the next descriptor
Random Access Memory
Essential ‘transmit’ registers
enum
{
E1000_CTRL
E1000_STATUS
E1000_IMC
E1000_TCRL
E1000_TDBAL
E1000_TDBAH
E1000_TDLEN
E1000_TDH
E1000_TDT
E1000_RA
};
0x0000, // Device Control
0x0008, // Device Status
0x00D8, // Interrupt Mask Clear
0x0400, // Transmit Control
0x3800, // Tx Descriptor Base Address Low
0x3804, // Tx Descriptor Base Address High
0x3808, // Tx Descriptor Length
0x3810, // Tx Descriptor Head
0X3818, // Tx Descriptor Tail
0x5400, // Receive address-filter Array
Programming challenge
• Can you see how use the code shown in
this example-module to implement a TX
capability in our earlier ‘netframe.c’ driver?