System Enhancements, Optimization
and Compiling the Kernel
Tutorial Index:







# Optimize hard drive access performance
# Kernel Tweaks
# Compiling the Kernel
# Kernel Source and Patches
# Tips
# Info
# Books
This tutorial covers optimization techniques to make your system run faster or
support your needs better. This may include tweaking configuration
specifications, turning off un-needed processes, and compiling a new kernel
tweaked for your system. Note that graphics device drivers are NOT compiled
into the kernel although support for interfacing with their particular chipset can be
compiled in. This is because X-Windows is an application running on Linux and
not integral to the kernel.
Optimize hard drive access performance:
Hdparm:
The following optimizations techniques can be applied to (E)IDE drives to utilize
32 bit I/O and DMA more effectively.




Get info: hdparm /dev/hda
Get more info: hdparm -I /dev/hda
Test Speed: hdparm -tT /dev/hda
Increase Speed: hdparm -d1 -c3 -u1 /dev/hda
o -d1: Turn on DMA (0: off)
o -c3: IDE chipset support
 0: Disable
 1: Enable
 3: 32 bit transfer
o -u1: Get/set interrupt umask. Driver ignores other interrupts during
processing of disk interrupts. (0: off)
Test Speed Again: hdparm -tT /dev/hda

The Red Hat 7.1+/8.0 configuration uses /etc/rc.sysinit to call scripts
/etc/sysconfig/harddisks. (Earlier systems used:
/etc/sysconfig/harddiskhda, /etc/sysconfig/harddiskhdb, etc) to run hdparm
upon boot.
File: /etc/sysconfig/harddisks:
USE_DMA=1
EIDE_32BIT=1
Turn off hdparm DMA access: hdparm -d0 /dev/hda
Links:



hdparm man page.
Optimizing hard drive performance with hdparm.
ATA, UDMA and hdparm
Ramdisk:
Go even faster by using ramdisk - Memory mapped as a disk. See: How to use
a Ramdisk for Linux
Also see: /usr/src/linux-2.4/Documentation/ramdisk.txt
Tools:


Measure hared drive performance with IBM Drive Fitness Test (DFT)
utility. Create a boot floppy with DFT image (use dd) and boot. Measures
SCSI and IDE performance as well as some repair utilities.
IoMeter: Server I/O benchmark tool.
Kernel Tweaks:
Direct kernel to specify processor affinity for a process:
Set (or retrieve) the CPU affinity. The scheduler attempts to keep the process
on the specified CPU. This is especially effective for threaded applications to
improve cache coherancy on SMP computers. It is also usefull for software
licensed to a CPU or for a single CPU system when it is in fact running on an
SMP system. (i.e. Oracle)


taskset: retrieve or set a processes's CPU affinity
(Part of package "schedutils")
sched_setaffinity / sched_getaffinity: set and get a process's CPU affinity
mask
Kernel enhancements to optimize SCSI for scanner access:


See man page for sane: man sane-scsi
Excerp from man page:
First, make sure your kernel has SCSI generic support
enabled. In ``make xconfig'', this shows up under ``SCSI
support->SCSI generic support''.
To keep scanning times to a minimum, it is strongly recom�
mended to use a large buffer size for the generic SCSI
driver.
By default, Linux uses a buffer of size 32KB.
This works, but for many cheaper scanners this causes
scanning to be slower by about a factor of four than when
using a size of 127KB. Linux defines the size of this
buffer
by
macro
SG_BIG_BUFF
in
header
file
/usr/include/scsi/sg.h.
Unless a system is seriously
short on memory, it is recommended to increase this value
to the maximum legal value of 128*1024-512=130560 bytes.
After changing this value, it is necessary to recompile
both the kernel (or the SCSI generic module) and the SCSI
backends.
AUTHOR
David Mosberger
13 May 1998
Kernel enhancements to increase the per process limit of open
files:
Change the definition of NR_OPEN in local files:



/usr/src/linux/include/linux/limits.h (Local file: kernel 2.2 RH 7.0-) (default
1024 typ.)
/usr/include/limits.h (Local file) (default 1024 typ.)
/usr/src/linux/include/linux/fs.h (Local file: kernel 2.2 RH 7.0-)
Compiling the Kernel:
The default Red Hat Kernel comes with as much stuff compiled into the Kernel
as they could fit. This allows it to support a vast array of devices and system
needs. If you must add support for something new or if you wish to streamline
your kernel with only what is needed so that it uses less memory, operates
faster, and is optimized for your processor, then one must recompile the kernel.
To find out about the latest kernel, issue the command: finger
@finger.kernel.org
Typically the source code for the Red Hat installation is located in:



- patches and kernel will be placed here by kernel
source RPM. (Fedora Core 3)
(bzip2 -d linux-2.6.9.tar.bz2, tar xf linux-2.6.9.tar)
/usr/src/linux/ - (Soft linked to /usr/src/linux-2.2.xx) Kernel 2.2 and
older. (Red Hat 6.x, 7.0)
/usr/src/linux-2.4/ - (Soft linked to /usr/src/linux-2.4.xx) Kernel 2.4. (Red
Hat 7.1)
/usr/src/redhat/SOURCES/
This separation to a new directory for kernel 2.4 is to accommodate the
differences between the C library and the kernel.
If you get new kernel source download to a new directory. i.e. /usr/src/linux2.2.16 then ln -s /usr/src/linux-2.2.16 /usr/src/linux
Typically the kernel RPM (kernel-source-XXX.i386.rpm) will install the source
and generate the appropriate links.
Perform the following steps: (as root)
1. Setup:
o
o
o
The source may already be installed as part of the installation or
from an RPM (See /usr/src/).
OR
You may also download the latest source from
http://www.kernel.org/pub/linux/kernel/.
Move the downloaded file linux-2.X.XX.tar.gz to /usr/src/.
Uncompress file: tar -xzf linux-2.X.XX.tar.gz which will install
everything in a subdirectory linux/. (To view contents use
command tar -tzf linux-...tar.gz)
If using the file with the bz2 extension: bunzip2 linux2.X.XX.tar.bz2
cd /usr/src/linux/ (or /usr/src/linux-2.4/)
less README (Read the documentation. It tells you
everything. Use
space bar to view next page)
(Local links to: /usr/src/linux/README or /usr/src/linux2.4/README)
o
Update /usr/include/
NOTE: PERFORM THIS STEP FOR KERNEL 2.2.x AND OLDER.
DO NOT DO THIS FOR KERNEL 2.4.
The header files for Kernel 2.4 and the C library header files may
no longer match and thus must remain separated.





cd
rm
ln
ln
ln
/usr/include
-fr asm linux scsi
-s /usr/src/linux/include/asm asm
-s /usr/src/linux/include/linux linux
-s /usr/src/linux/include/scsi scsi
The soft links (ln -s) to the include directories are important to
software developers who need to compile with the new include
files.
Note: The sym link is optional and some don't do it due to
potential differences in the header files used by glibc and the
kernel. (A response from Linus)
o
o
cd /usr/src/linux (or /usr/src/linux-2.4)
make distclean (Cleans up junk from old compiles)
o
removes .config)
make mrproper (build the source tree)
(Warning:
At this time the sources are correctly installed. This also resets the
/usr/src/linux/.config file to the system default.
SuSE setup:
o
o
For SuSE (9.3), there are two kernel-source RPMs. One RPM
resides on the "Source" DVD-2 and the other resides on the
"Installation" DVD-1. Use the kernel-source RPM found on the
"Installation" DVD-1. This one will register in the RPM database
and creates the directories expected
Check with the command: rpm -q kernel-source
Kernel source code location: /usr/src/linux
Docs location: /usr/share/doc/packages/kernel-source
The SuSE kernel-source RPM will not produce a ".config" file
for the kernel compilation.
Produce the default ".config" file by: zcat /proc/config.gz >
/usr/src/linux/.config
2. Configuration:
Use one of the following tools to create the .config file. This gives you
the chance to choose what goes into the kernel. You can choose support
for many of the latest capabilities, device drivers, and can tune the
kernel for particular uses. i.e. you can also optimize the compilation for
the Pentium. (Default for RH 5.2 was a 386 with no math co-processor!)
The default config file used to compile the default distribution kernel can
be found in the source directory. (i.e. Fedora Core 3:
/usr/src/redhat/SOURCES/kernel-2.6.9-i686.config,
/usr/src/redhat/SOURCES/kernel-2.6.9-x86_64-smp.config, ... and all
the others for the various hardware platforms.)
Pick one of the following:
o
o
o
make config (Bash shell script)
make menuconfig (uses text window curses)
make xconfig (uses tck and wish) - recommended
o
.config
make oldconfig Build a
in current .config file.
due to online
help feature and intuitive interface. Save configuration to file:
configuration file based on defaults found
menuconfig Notes: "< > options can be compiled as a module.
o
o
<*> = Compile into kernel (y)
<M> = Compile as a module (m)
Recommendation for Pentium owners: General Setup choose :
o
o
o
o
o
Kernel math emulations: N
Processor type: Pentium
Development, experimental or kernel debugging features: N
If using netfilter capabilities, do not enable the kernel fastswitching option. That code will permits fast switching at a lower
level in the IP stack than netfilter and thus will bypass the netfilter
capabilities.
Note: When requiring gobs of memory (> 1Gb), there is a kernel
compile option which sets the limit on the amount of user memory
one can use: "Processor type and features / Maximum Virtual
Memory".
3. Compile:
Next: (from /usr/src/linux)
o
o
o
(make dependencies. Older kernels only. Attempt and if
not needed, make will return a message stating that this step is
un-necessary. Not required for 2.6+ kernels.)
make clean (At this point you have all the source gathered)
Choose one of the following to create the new kernel and then
make dep
move it to /boot/vmlinux:
 make zImage (compressed image)
 make bzImage (very compressed image good for recovery
floppies - slower to boot because of time to uncompress)


make image
make bzdisk
(Boot floppy with no LILO)
4. Install:
o
mv /usr/src/linux/arch/i386/boot/zImage /boot/vmlinuzrelease
ln -s /boot/vmlinuz-release /boot/vmlinuz
(If "make zImage" was chosen)
or
mv /usr/src/linux/arch/i386/boot/bzImage /boot/vmlinuzrelease
ln -s /boot/bzImage-release /boot/vmlinuz
(If "make bzImage" was chosen)
Also move/copy the file /usr/src/linux.../System.map to
/boot/System.map-2.X.XX
OR
make install
o
o
o
o
o
(Preferred. Potential Pitfall: On my SCSI system I had to execute
this command after issuing the mkinitrd command below.)
Modules documentation: (not that helpful)
 /usr/src/linux-2.4/Documentation/modules.txt
 /usr/src/linux/Documentation/modules.txt
make modules
rm /lib/modules/Kernel_release (i.e. /lib/modules/2.2.16-3
ln -s /lib/modules/new_modules_directory
/lib/modules/Kernel_release
make modules_install
Generates the file: /lib/modules/2.X.XX/modules.dep
)
5. Configure Boot Loader:
(Grub or Lilo)
o
Configure grub:
File: /etc/grub.conf
default=1
- Kernel to boot (2nd in list)
timeout=5
splashimage=(hd1,0)/boot/grub/splash.xpm.gz
- Boot image location
(hd1, 1st partition)
hiddenmenu
title Fedora Core (2.6.12-1.1372_FC3)
root (hd1,0)
- Root of GRUB
reference. Same as /boot
kernel /boot/vmlinuz-2.6.12-1.1372_FC3 ro root=LABEL=/ rhgb
quiet
initrd /boot/initrd-2.6.12-1.1372_FC3.img
title Fedora Core (2.6.11-1.14_FC3)
root (hd1,0)
kernel /boot/vmlinuz-2.6.11-1.14_FC3 ro root=LABEL=/ rhgb
quiet
initrd /boot/initrd-2.6.11-1.14_FC3.img
Description:






default: Default kernel to boot. All options displayed but
one will boot by default.
timeout: Time for user to select kernel. After "timeout"
seconds, default kernel chosen and boot begins.
splashimage: Background image shown by Grub and
location (hard drive hd1 and "0" or first partition (/boot).
root: Grub root directory. In this example, drive hd1 and "0"
or first partition (/boot).
kernel: Kernel to boot, read only. Note that this is the same
as the result of cat /proc/cmdline
initrd: Initial ramdisk for a Linux format boot image. Sets
parameters in the Linux setup area in memory.
See:


o
Grub Home page
grub shell man page.
Configure lilo: Lilo must point to the new kernel. Edit
/etc/lilo.conf. Add a new image statement to point to the new
kernel. Keep the old as backup in case you need to boot it.
SCSI systems must perform an additional step to load drivers
upon system boot:

Execute the following command which creates the file used
to let the system boot:

mkinitrd
/boot/initrd-2.2.16-3.img
2.2.16-3
The second argument is the name of the sub-directory of
the modules under the directory /lib/modules/. (In this
case /lib/modules/2.2.16-3/
This step is required by systems using modular SCSI
support. If the SCSI support is compiled into the kernel and
is NOT a module, then this step may not be necessary.
The file /boot/initrd-2.2.16-3.img is specified as the
RAM disk image used before the file system is available.
(initrd = initial ram disk)
Run /sbin/lilo -v to configure the master boot record with data
from lilo.conf.
Sample lilo.conf:
boot=/dev/sda
map=/boot/map
boot
install=/boot/boot.b
prompt
timeout=50
linear
default=linux
- Location of the kernel used for
- Specific to SCSI configurations
image=/boot/vmlinuz-2.2.16-3
- New kernel
label=linux
- Name to be displayed by Lilo boot
manager
initrd=/boot/initrd-2.2.16-3.img
- Specific to SCSI boot
drives.
read-only
root=/dev/sdb6
image=/boot/vmlinuz-2.2.14-5.0
label=OldLinux
initrd=/boot/initrd-2.2.14-5.0.img
read-only
root=/dev/sdb6
- SCSI specific.
When the system boots and gives you the "Lilo:" prompt, you can
type "?" to see your boot choices as defined by the above labels
in lilo.conf. Type the label name of the kernel or system you
wish to boot. A carriage return will choose the default.
For a full list of boot parameters see: bootparam man page
Also see the Boot Prompt Howto.
Kernel version verification: uname -rv will print the compile date of the kernel
currently running.
Modules:
Modules are used to reduce the amount of memory used to hold the kernel.
There is a slight penalty for the time taken to load and unload the module.
If the code is required for general operation of the kernel or is needed often or
required by the boot process, it is best to compile it into the kernel and it should
NOT be compiled as a module.
Module commands:
The kernel will use the modprobe utility to determine if the module is compatible
with the kernel.
The program used is specified by a proc file:
cat /proc/sys/kernel/modprobe
/sbin/modprobe
- Result of cat operation
Modules are loaded by init scripts which call insmod/rmmod to load/unload
modules.
Command Description
lsmod
List loaded modules
[ /root]# lsmod
Module
3c59x
(unused)
autofs
emu10k1
soundcore
aic7xxx
Size
19716
9120
45688
2596
137400
Used by
0 (autoclean)
1
1
4
3
(autoclean)
[emu10k1]
insmod
Inserts a module into the active kernel
rmmod
Remove a loaded module. Just specify the module name.
No ".o" or path necessary.
modprobe
High level handling of loadable modules.
Loads module and dependencies.
depmod
Creates dependencies file for a module (used by
modprobe)
modinfo
Display information about a kernel module
List of modules held in /etc/modules.conf (or for older systems:
/etc/conf.modules)
Sample:
alias scsi_hostadapter aic7xxx
alias eth0 eepro100
alias eth1 eepro100
alias parport_lowlevel parport_pc
alias sound-slot-0 emu10k1
alias char-major-81 bttv
Other useful commands:



Display system status gathered from /proc: /usr/bin/procinfo -a
List Processor type: cat /proc/cpuinfo
List devices:
o
o



cat /proc/devices
/sbin/lsmod
List IO ports (devivce address used drivers): cat /proc/ioports
List DMA channels: cat /proc/dma
View interrupts used by the system:
o
o
cat /proc/interrupts
procinfo -i
IRQ
Description
0
Timer channel 0
1
Keyboard
2
Cascade for controller 2 (which controls IRQ 8-15)
3
Serial Port 2 (COM2)
4
Serial Port 1 (COM1)
5
Parallel Port 2 (LPT2)
6
Floppy Diskette Controller
7
Parallel Port 1 (Printer LPT1)
8
Real-time Clock
9
Redirected to IRQ2
10
--
11
--
12
PS/2 Mouse
13
Math Co-processor
14
Hard Disk Controller (IDE)
15
--

Also see file:
o
o
for a list of protected IRQ's which will not be
allocated to ISA Plug and Play device.
Command: pnpdump
/etc/isapnp.gone
This command scans ISA PnP cards and can be used to generate
o
/etc/isapnp.conf
pnpdump > /etc/isapnp.conf
Command: isapnp -C /etc/isapnp.conf
Uses file /etc/isapnp.gone and /proc files
to avoid IRQ's already
allocated and to determine safe IRQ's.
Home page for "isapnptools"
Plug and Play ISA Specification
Create devices. Run script: /dev/MAKEDEV
Display kernel version in current use: /proc/version
Display boot messages: dmesg
Display sound driver status: cat /dev/sndstat
o




Kernel Source and Patches:
Linux kernel naming convention:
Version.MajorRelease.MinorRelease.PatchNumber
(In Linux kernel vernacular:
KernelVersion.PatchLevel.Sublevel.Extraversion)
Patch File Description: The patch files are file "diffs" (differences from the
stated source file) which can be applied to create a newer updated file.
Downloading kernel source and applying a patch:

Obtain kernel source:
ftp://ftp.us.kernel.org/pub/linux/kernel/v2.X/linux-2.X.X.tar.gz
Uncompress: tar xzf linux-2.X.X.tar.gz
(Or: tar xjf linux-2.X.X.tar.bz2)
This creates the local directory: linux/...

OR
use the source from the kernel-source RPM package released by your
distribution. Using the source released by your distribution will include
patches and source code needed to support all the features of your
distribution. Of greatest importance is the default ".config" file which
describes the features used by your distribution. i.e. Red Hat / Fedora
Core use the ext3 file system, devfs and turn on many features by
default.
Download patch:
o Download patch:
ftp://ftp.us.kernel.org/pub/linux/kernel/v2.X/patch-2.X.X
(Place patches in directory (create it first): /usr/src/KernelPatches/)

Apply patch: Apply patches from default source directory
/usr/src/linux (linked to actual source directory):
o gzip -cd ../KernelPatches/patch-file.gz | patch -p0
o bzip2 -dc ../KernelPatches/patch-file.bz2 | patch -p0
o If patch file resides in directory /usr/src/linux:
zcat patch-file.gz | patch -p0
Use script patch-kernel if applying multiple patches at once: (i.e. patch2.6.11.5 to patch-2.6.11.7)
(Script located in directory linked to kernel source)
o
o
Place all patches in /usr/src/KernelPatches/ (.gz or .bz2)
Apply all the patches to kernel (i.e. 2.6.11.4)
/usr/src/linux/scripts/patch-kernel linux KernelPatches
(Execute from directory /usr/src)
o
Or apply all the patches to kernel to bring it up to specified level.
/usr/src/linux/scripts/patch-kernel linux KernelPatches
2.6.11.7

(patch-kernel kernel-source-directory-to-act-on <KERNELPATCH-DIRECTORY< i>highest-version to include)
Set-up directory links:
o mv linux linux-2.X.XX (only required on some older RPMs)
o
o
o
o
ln -s linux-2.X.XX linux
/usr/include/asm should point to /usr/src/linux/include/asmi386
/usr/include/linux should point to
/usr/src/linux/include/linux
/usr/include/scsi should point to /usr/src/linux/include/scsi
Example: Applying the devfs patch to kernel 2.2.20

Download kernel: http://www.kernel.org/pub/linux/kernel/v2.2/linux2.2.20.tar.gz





mv linux-2.2.20.tar.gz /usr/src
cd /usr/src/
Remove previous source: rm linux (May use rm -Rf linux/)
Unload to subdirectory linux/: tar xzf linux-2.2.20.tar.gz
Download patch: http://www.atnf.csiro.au/~rgooch/linux/kernelpatches.html
Download patch file devfs-patch-current to /usr/src/
cd /usr/src
Test: patch -p0 --dry-run < devfs-patch-current
Apply patch: patch -p0 < devfs-patch-current






Now go through regular kernel build as described above. (From the
beginning) make distclean; make mrproper
The configuration has been altered by the patch thus make xconfig will
have options not seen in the original 2.2.20 kernel.
o "Code maturity level" option
o Select "development and/or incomplete code/drivers"
This will provide lots of options for kernel.
o Go to Filesystems - check /dev file support "Experimental"
This feature was introduced by the patch. If it is not selected the
devfs feature introduced by the patch will not be included.
Kernel source must be in /usr/src/linux/ (From kernel.org) or
/usr/src/linux-2.2.20/ for the patch command as given, to work. The patch is
pre-configured for this.
Patch command options:
Option
Description
-DMACRONAME
Define pre-processor macro
-c
Use diff file format.
-d
directory
Change to directory.
--dry-run
Don't modify file but output results.
-s
Silent mode.
-f
Force change.
-n
Using patch file which is a regular diff file.
-R
Reverse or undo patch. (i.e. if patch rejected.)
-p0
Do not strip directory path prefix. Path intact. i.e.
/usr/src/linux/...
-p1
Strip first "/" in directory path. i.e. usr/src/linux/...
-p2
Strip first directory in path. i.e. src/linux/...
-p#
Directory path "/" prefixes to strip (# - 1).
Look at the patch file to see the directory path you will reference.
View patch file: zcat patch-file.gz | more
patch man page.