Dr A Sahu
Dept of Comp Sc & Engg.
IIT Guwahati
•
•
•
•
PCI Devices
NIC Cards
NIC card architecture
Access to NIC register
–PCI access
• RealTek (Example RTK 8169S)
• Broadcom (Example BCM5751)
• Intel (Example 82573L NIC)
• Download manual from vender site
–Or search “Realtek 8169 pdf” in google
• Peripheral devices in the early PCs used fixed i/oports and fixed memory-addresses, e.g.:
–
–
–
–
–
–
–
–
Video memory address-range: 0xA0000-0xBFFFF
Programmable timer i/o-ports: 0x40-0x43
Keyboard and mouse i/o-ports: 0x60-0x64
Real-Time Clock’s i/o-ports: 0x70-0x71
Hard Disk controller’s i/o-ports: 0x01F0-01F7
Graphics controller’s i/o-ports: 0x03C0-0x3CF
Serial-port controller’s i/o-ports: 0x03F8-0x03FF
Parallel-port controller’s i/o-ports: 0x0378-0x037A
• It became clear in the 1990s that there would
be contention among equipment vendors for
‘fixed’ resource-addresses, which of course
were in limited supply
• Among the goals that motivated the PCI
Specification was the creation of a more
flexible scheme for allocating addresses that
future peripheral devices could use
A non-volatile parameter-storage area
for each PCI device-function
PCI Configuration Space Header
(16 doublewords – fixed format)
64
doublewords
PCI Configuration Space Body
(48 doublewords – variable format)
16 doublewords
31
0 31
Status
Command
Register
Register
Cache
Header Latency
Line
BIST
Type
Timer
Size
0 Dwords
Device
Vendor
ID
ID
Class Code
Revision
Class/SubClass/ProgIF
ID
1- 0
3- 2
Base Address 1
Base Address 0
5- 4
Base Address 3
Base Address 2
7- 6
Base Address 5
Base Address 4
9- 8
CardBus CIS Pointer
11 - 10
Expansion ROM Base Address
13 - 12
reserved
15 - 14
Subsystem
Device ID
reserved
Subsystem
Vendor ID
capabilities
pointer
MaximumMinimum Interrupt Interrupt
Latency Grant
Pin
Line
accessed using a large variety of processor
instructions (mov, add, or, shr, push, etc.)
and virtual-to-physical address-translation
memory
space
(4GB)
accessed only by using the processor’s
special ‘in’ and ‘out’ instructions
(without any translation of port-addresses)
i/o space
(64KB)
PCI
configuration
space
(16MB)
i/o-ports 0x0CF8-0x0CFF dedicated to accessing PCI Configuration Space
PCI Configuration Space Address Port (32-bits)
31
CONFADD
( 0x0CF8)
E
N
23
reserved
16 15
bus
(8-bits)
11 10 8 7
device
(5-bits)
function doubleword
(3-bits)
(6-bits)
2 0
00
Enable Configuration Space Mapping (1=yes, 0=no)
PCI Configuration Space Data Port (32-bits)
31
CONFDAT
( 0x0CFC)
0
• Step one: Output the desired longword’s
address (bus, device, function, and dword)
with bit 31 set to 1 (to enable access) to the
Configuration-Space Address-Port
• Step two: Read the designated data from the
Configuration-Space Data-Port
• Already discussed PCI-probes pciprobes.c
– Lect 29..Showing vram, pciprobe.cpp
• We can identify the network interface controller in
PC’s by class-code 0x02
• The subclass-code 0x00 is for ‘ethernet’
• We can identify the NIC from its VENDOR and DEVICE
identification-numbers:
• VENDOR_ID = 0x14E4
• DEVICE_ID = 0x1677
• You can use the ‘grep’ command to search for these
numbers in this header-file:
</usr/src/linux/include/linux/pci_ids.h>
• The VENDOR-ID 0x14E4 belongs to the
Broadcom Corporation
• Information about this firm may be learned
from the corporation’s website
• The DEVICE-ID 0x1677 is used to signify
Broadcom’s BCM5751 ethernet product
packet
main
memory
TX FIFO
buffer
B
U
S
CPU
nic
RX FIFO
transceiver
LAN
cable
• Network Interface’s hardware needs to
implement ‘filtering’ of network packets
• Otherwise the PC’s memory-usage and
processor-time will be wasted handling
packets not meant for this PC to receive
network packet’s layout
Destination-address (6-bytes)
Source-address (6-bytes)
Each data-packet begins with the 6-byte device-address
of the network interface which is intended to receive it
• You can see the Hardware Address of the
ethernet controller on your PC by typing:
$ /sbin/ifconfig
• Look for it in the first line of screen-output
that is labeled ‘eth0’, for example:
eth0
Link encap: Ethernet HWaddr 00:11:43:C9:50:3A
• Lets write a kernel module that lets users see
certain register-values which pertain to the
network interface in your system :
– (1) the PCI Configuration Space registers
– (2) the Media Access Controller’s address
• It also shows your machine’s node-name (in
case you want to save the information)
• We do not have NIC’s programming datasheet
-- but we do have Linux source code for the
‘nic_pci_info.c’ device-driver, which includes a
header-file ‘tg3.h’ found here:
</usr/src/linux/drivers/net/>
• If you scroll through the #define directives you
will see the offset where the hardware
address is stored in the memory-mapped
register-space of the ‘nic_pci_info.c’ interface
16 doublewords
31
0
Status
Register
Header
BIST
Type
Command
Register
Cache
Latency
Line
Timer
Size
31
0
DeviceID
VendorID
0x1677
0x14E4
Class Code
Revision
Class/SubClass/ProgIF
ID
Dwords
1- 0
3- 2
Base Address 1
Base Address 0
5- 4
Base Address 3
Base Address 2
7- 6
Base Address 5
Base Address 4
9- 8
CardBus CIS Pointer
11 - 10
Expansion ROM Base Address
13 - 12
reserved
15 - 14
Subsystem
Device ID
reserved
Subsystem
Vendor ID
capabilities
pointer
Maximum Minimum Interrupt Interrupt
Latency
Grant
Pin
Line
#include <linux/pci.h>
struct pci_dev *devp;
unsigned int iomem_base, iomem_size;
void
*io;
devp = pci_get_device( 0x14E4, 0x1677, NULL );
if ( !devp ) return –ENODEV;
iomem_base = pci_resource_start( devp, 0 );
iomem_size = pci_resource_len( devp, 0 );
io = ioremap( iomem_base, iomem_size );
if ( !io ) return -EBUSY;
Broadcom network interface storage-addresses
0x0410 0x0411 0x0412 0x0413 0x0414 0x0415 0x0416 0x0417
mac
1
mac
0
mac
0
mac
5
mac
1
mac
2
mac
3
mac
4
mac
4
Intel IA-32 character-array storage
mac
3
mac
5
mac
2
Intel network interface storage-addresses
0x5400 0x5401 0x5402 0x5403 0x5404 0x5405 0x5406 0x5407
mac
0
mac
1
mac
2
mac
3
mac
0
mac
1
mac
2
mac
4
mac
3
mac
5
mac
4
Intel IA-32 character-array storage
mac
5
• For Intel NICs :
#define VENDOR_ID 0x8086 // Intel Corp
#define DEVICE_ID 0x109A // 82573L NIC
• Intel’s filter-register at offset 0x5400 uses the
‘little endian’ storage-convention
host-1
host-2
host-3
HUB
“Collision Domain”
CSMA/CD = “Carrier Sense Multiple Access/Collision Detection”
host-4
•
•
•
•
•
PCI = Peripheral Component Interconnect
MAC = Media Access Controller
Phy = Physical-layer functions
AMT = Active Management Technology
LOM = LAN On Motherboard
•
•
•
•
•
•
•
32K configurable RX and TX packet FIFO
IEEE 802.3x Flow Control support
Host-Memory Receive Buffers 16K/256K
IEEE 802.3ab Auto-Negotiation
TCP/UDP checksum off-loading
Jumbo-frame support (up to 16KB)
Interrupt-moderation controls
MDI interface
10/100/1000 PHY
MDIO
interface
SM Bus
interface
LED
indicators
S/W Defined
pins
GMII/MII
interface
MAC/Controller
EEPROM
Flash
interface
PCI/PCI-e Bus
• Device registers are hardware mapped to a
range of addresses in physical memory
• You obtain the location (and the length) of this
memory-range from a Base Add register in the
nic device’s PCI Configuration Space
• Then you request the Linux kernel to setup an
I/O ‘remapping’ of this memory-range to
‘virtual’ addresses within kernel-space
Local-APIC
IO-APIC
nic
registers
APIC registers
nic registers
vram
1-GB
kernel code/data
vram
user
space
dynamic
ram
physical address-space
‘virtual’ address-space
3-GB
• Linux provides device-driver writers with some
macros for accessing i/o-memory:
#include <asm/io.h>
unsigned int
datum;
iowrite32( datum, address );
datum = ioread32( address );
#include <linux/pci.h>
#include <asm/io.h>
#define E1000_STATUS 0x0008
unsigned int iomem_base, iomem_size;
void
*io;
// remap the device’s i/o-memory into kernel space
devp = pci_get_device( VENDOR_ID, DEVICE_ID, NULL );
if ( !devp ) return –ENODEV;
iomem_base = pci_resource_start( devp, 0 );
iomem_size = pci_resource_len( devp, 0 );
io = ioremap_nocache( iomem_base, iomem_size );
if ( !io ) return –ENOSPC;
// read and display the nic’s STATUS register
device_status = ioread32( io + E1000_STATUS );
printk( “ Device Status Register = 0x%08X \n”, status );
31
?
30
29
28
0
0
27
0
26
0
25
0
24
23
22
21
0
0
0
0
0
11
10
9
8
7
20
19
18
17
16
0
GIO
Master
EN
0
0
0
some undocumented functionality?
15
0
14
0
13
0
12
0
0
PHY
reset
ASDV
I
S
L
SPEEDL
O
S
U
6
5
0
4
TX
OFF
FD = Full-Duplex
LU = Link Up
TXOFF = Transmission Paused
SPEED (00=10Mbps,01=100Mbps, 10=1000Mbps, 11=reserved)
ASDV = Auto-negotiation Speed Detection Value
3
2
Function
ID 0
1
0
L
F
D
0U
82573L