lsu
Server’s Ordinary Configuration
for FreeBSD
Useful message for server admins
ccui
6/22/2009
Server’s Ordinary Configuration for FreeBSD 2009
File system
1. FFS works best with SCSI and SAS drives due to the robustness of the SCSI
architecture. FFS also works as well as the ATA architecture allows, with one
critical exception: Many modern IDE drives support write caching.
If you care about your data, disable write caching by adding the following to
/boot/loader.conf:
hw.ata.wc=0
(AB FreeBSD, Page: 221)
2. Useful data generator:
# dd if=/dev/zero of=filesystem.file bs=1m count=1k // 1m*1k = 1GB of data
(AB FreeBSD, Page: 236)
More easier data generator:
# truncate –s100m /path/to/file
//-s flag dictates the size of the file to create,
//k for KB, m for MB, G for GB
(AB FreeBSD, Page: 565)
3. change file or directory ownership:
boss# chown -Rv root:wheel target_file_or_directory
4. Sorting files in certain directory:
du -d 1 | sort -n | cut -f2 | xargs du -sh
//in ascending order
du -d 1 | sort -nr | cut -f2 | xargs du -sh
//in descending order
5. find modified files by time, date:
find . -mtime -1 -ls
find . -mtime -1 -print
//-print can be replaced with -ls if you want a directory-listing-type response.
To find all files with regular file types only, and modified in the last 24 hours (last
full day) in current directory and its sub-directories:
find /directory_path -type f -mtime -1 -print
To find all files that are modified today only (since start of day only, i.e. 12
am), in current directory and its sub-directories:
Page 2
Server’s Ordinary Configuration for FreeBSD 2009
touch -t `date +%m%d0000` /tmp/$$
find /tmefndr/oravl01 -type f -newer /tmp/$$
rm /tmp/$$
The first command can be modified to specify other date and time, so that the
commands will return all files that have changed since that particular date and
time.
6. show files changed on a certain date:
If you need to find a file you changed on a certain date, this handy one liner will
do it.
ls -lt * | grep 'May 8' | awk '{print $9}'
Optimizing software compiling
1. The /etc/make.conf does not exist on new systems, but you can copy
/usr/share/examples/etc/make.conf to /etc/make.conf
Edit the file, and look for the line starting: #CPUTYPE=
Also in the file, look for the line #CFLAGS= -O -pipe
Uncomment this line, and change it to: CFLAGS= -O2 -pipe -funroll-loops
Optimizing network performance
1. Cancel network card speed auto-negotiate:
The network speed can either be set with ifconfig at run time, or in /etc/rc.conf for boot
time. Here are examples for /etc/rc. conf for the nVidia nfe network drivers:
ifconfig_nfe0=”inet 192.168.0.2 netmask 255.255.255.0 media 1000baseTX mediaopt full-duplex”
ifconfig_nfe1=”inet 192.168.0.3 netmask 255.255.255.0 media 1000baseTX mediaopt full-duplex”
ifconfig_nfe2=”inet 192.168.0.4 netmask 255.255.255.0 media 1000baseTX mediaopt full-duplex”
ifconfig_nfe3=”inet 192.168.0.5 netmask 255.255.255.0 media 1000baseTX mediaopt full-duplex”
2. Enable DEVICE_POLLING
If you want to take advantage of DEVICE_POLLING, you need to compile two
options into your kernel:
 options DEVICE_POLLING
 options HZ=1000
If you want to enable it at boot, add this line to the end of your /etc/sysctl.conf:
 kern.polling.enable=1
And enable it at network card in /etc/rc. conf:
ifconfig_nfe1=”inet 192.168.0.2 netmask 255.255.255.0 media 1000baseTX mediaopt full-duplex polling”
3. Restart the network service:
Page 3
Server’s Ordinary Configuration for FreeBSD 2009
/etc/rc.d/netif restart
4. Manual method using ifconfig
To stop network card (NIC) on-fly:
# ifconfig network-interface down
5. renew DHCP IP address: dhclient em0
/e.g. em0 is the network port number
6. In Ubuntu 9.04 server AMD64, to restart networking service, use:
sudo /etc/init.d/networking restart
Optimize kernel building:
1. Edit your favor kernel configure file in /usr/src/sys/i386/conf/
2. #cd /usr/src
3. Compile the kernel:
#make -j9 buildkernel KERNCONF=PAE
//PAE is a configure file
4. Install the new kernel:
#make -j9 installkernel KERNCONF=PAE
5. And you can check the building time by add the “time” command
#time make -j9 buildkernel KERNCONF=PAE
#time make -j9 installkernel KERNCONF=PAE
Mounting USB and CD Rom drive
1. For Microsoft NTFS format USB drives, use:
#mount -t msdosfs /dev/da0s1 /mnt
//in FreeBSD 6.x, would try da0 and
//da0s1 repeatedly
2. To Mount a CD disk, use:
#mount –t cd9660 /dev/acd0 /cdrom
Enable root SSH login
1. Open up /etc/ssh/sshd_config and set "PermitRootLogin" to "yes". (Your ISP probably
set it to "without-password"); or disable it as “PermitRootLogin no”
2. You also need to restart the sshd process. This is done by killing the existing one (use ps
-aux|grep sshd to get the process ID, then use kill to zap it), then restarting
/usr/sbin/sshd
or:
/etc/rc.d/sshd onerestart
Booting an Alternate Kernel
1. Get the loader prompt.
2. ok unload
Page 4
Server’s Ordinary Configuration for FreeBSD 2009
3. ok load /boot/kernel.good/kernel
4. ok load /boot/kernel.good/acpi.ko
5. ok boot
Forcefully un-mount mounted partition.
What happens basically, is that Linux / UNIX will not allow you to unmount a device that is busy.
There are many reasons for this (such as program accessing partition or open file) , but the most
important one is to prevent data loss. Linux/FreeBSD comes with fuser command to kill
forcefully mounted partition.
Try the following command to find out what processes have activities on the device/partition. If
your device name is /dev/sda1, enter the following command as root user:
# lsof | grep '/dev/sda1'
Output: vi 4453
vivek 3u
BLK
8,1
8167 /dev/sda1
Above output tells that user vivek has a vi process running that is using /dev/sda1. All you have
to do is stop vi process and run umount again. As soon as that program terminates its task, the
device will no longer be busy and you can unmount it with the following command:
# umount /dev/sda1
Linux fuser command to forcefully unmount a disk partition
Suppose you have /dev/sda1 mounted on /mnt directory then you can use fuser command as
follows:
# fuser -km /mnt
Where

-k : Kill processes accessing the file.

-m : Name specifies a file on a mounted file system or a block device that is mounted. In
above example you are using /mnt
Linux umount command to unmount a disk partition
You can also try umount command with –l option:
# umount -l /mnt
Where,

-l : Also known as Lazy unmount. Detach the filesystem from the filesystem
hierarchy now, and cleanup all references to the filesystem as soon as it is not
busy anymore. This option works with kernel version 2.4.11+ and above only.
Page 5
Server’s Ordinary Configuration for FreeBSD 2009
If you would like to unmount a NFS mount point then try following command:
# umount -f /mnt
Where,

-f: Force unmount in case of an unreachable NFS system
Caution: Using these commands or option can cause data loss for open files; programs
which access files after the file system has been unmounted will get an error.
Configure the Network Time Protocol to synchronize system time.
The ntpd utility is an operating system daemon which sets and maintains the system time of day
in synchronism with Internet standard time servers. Ordinarily, ntpd reads the ntp.conf(5)
configuration file at startup time in order to determine the synchronization sources and
operating modes.
FILES
/etc/ntp.conf the default name of the configuration file
/etc/ntp.drift the default name of the drift file
/etc/ntp.keys the default name of the key file
In /etc/ntp.conf add the follows:
server 0.us.pool.ntp.org
server 1.us.pool.ntp.org
server 2.us.pool.ntp.org
# Use NTP to synchronize system time, added by ccui in /etc/rc.conf
ntpd_enable="YES"
ntpd_sync_on_start="YES"
Clear the CMOS setting in the SUN Fire X4240 motherboard
1. Locate the jumper header J1802.
Access the J1802 jumper on the rear of the motherboard next to the SATA connector,
below PCIe slot 0, PCIe riser 0.
2. Place the jumper across the 2 pins of the header.
3. Power on the server and boot until the message about NVRAM has been cleared.
4. Power off the server, and remove AC power cables.
5. Remove the jumper from J1802.
6. Return the server to operation (cover it).
Update the locate database
1. Probably the best way to do it is to become root using su(1) or sudo(1),
and then type:
/etc/periodic/weekly/310.locate
Page 6
Server’s Ordinary Configuration for FreeBSD 2009
2. The weekly script that updates the locate database can be run manually by logging in
and issuing the command:
/etc/periodic/weekly/310.locate
This script will safely update without generating any of those 'updating locate database
as root' security warnings. To run the command directly uses the following command:
/usr/libexec/locate.updatedb
Install nfe-nVidia network adapter in FreeBSD 6.3
1. Copy “e1000phyreg.h v 1.7 2009/06/02 00:30:30” into /usr/src/sys/dev/mii/
2. Copy “e1000phy.c v 1.14.2.4.2.2 2008/10/27 00:23:09” into /usr/src/sys/dev/mii/
3. Add the optional options “DEVICE_POLLING” and “HZ=1000” into the kernel configure
file, for the purpose of enable “device polling”.
4. Build the new kernel and install the new kernel
5. Extract the nfe-20071124.tar.gz somewhere and edit the if_nfe.c to uncomment the “#
define DEVICE_POLLING”.
6. Type “make” in the extracted directory, and find if_nfe.ko, then copy it into
/boot/modules/
7. Type “kldload if_nfe” or add if_nfe_load=”yes” into /boot/loader.conf and then reboot
mxge -- Myricom Myri10GE 10 Gigabit Ethernet adapter driver
Installation(10G-PCIE-8AL-S)
1. To compile this driver into the kernel, place the following lines in your kernel
configuration file:
device firmware
device mxge
2. In FreeBSD 6.3
Alternatively, to load the driver as a module at boot time, place the following lines in
loader.conf(5):
if_mxge_load="YES"
mxge_ethp_z8e_load="YES"
mxge_eth_z8e_load="YES"
reference:
http://www.freebsd.org/cgi/man.cgi?query=mxge&sektion=4&manpath=FreeBSD+6.3-RELEASE
3. In FreeBSD 6.4
if_mxge_load="YES"
mxge_ethp_z8e_load="YES"
mxge_eth_z8e_load="YES"
mxge_rss_ethp_z8e_load="YES"
mxge_rss_eth_z8e_load="YES"
Page 7
Server’s Ordinary Configuration for FreeBSD 2009
4. In FreeBSD 7.2
Alternatively, to load the driver as a module at boot time, place the following lines in
loader.conf(5):
if_mxge_load="YES"
mxge_ethp_z8e_load="YES"
mxge_eth_z8e_load="YES"
mxge_rss_ethp_z8e_load="YES"
mxge_rss_eth_z8e_load="YES"
reference:
http://www.freebsd.org/cgi/man.cgi?query=mxge&sektion=4&manpath=FreeBSD+7.2-RELEASE
5. In Ubuntu 9.04 server AMD64 edition
The default MTU for the Linux Myri10GE driver has changed to 1500 bytes.
The Myri10GE Linux driver version 1.5.0 and later configures the
Myri-10G NIC with a 1500-byte MTU. It is still possible to use a 9000
byte default MTU by building the driver with the new MYRI10GE_JUMBO
option:
% cd myri10ge/linux
% make MYRI10GE_JUMBO=1
% make clean
% make
% su root
# make install-only
To load the Myricom 10GbE driver, type the command
# modprobe -v myri10ge myri10ge_initial_mtu=9000
//to remove the module, use: modprobe -rv myri10ge
//to check driver version, use: sudo lshw | grep "myri10ge"
A new ethernet interface, having a MAC address beginning with 00:60:DD,
should now appear in the output of ifconfig -a . For example:
# ifconfig -a | grep 00:60:dd
eth4 Link encap:Ethernet HWaddr 00:60:dd:46:9e:0b
eth5 Link encap:Ethernet HWaddr 00:60:dd:46:9e:0a
Restart DNS service:
/etc/rc.d/named restart
Restart DHCP service:
Page 8
Server’s Ordinary Configuration for FreeBSD 2009
/usr/local/etc/rc.d/isc-dhcpd restart
Finding a File containing a particular text string in Linux server
1. For example search for a string called redeem reward in all text files located in
/home/tom/*.txt directory, use
$ grep "redeem reward" /home/tom/*.txt
2. Search all subdirectories recursively
You can search for a text string all files under each directory, recursively with –r option:
$ grep -r "redeem reward" /home/tom
3. Only print filenames
By default, “grep” command prints the matching lines. You can pass -H option to print
the filename for each match.
$ grep -H -r “redeem reward” /home/tom
4. To just print the filename use cut command as follows:
$ grep -H vivek /etc/* -R | cut -d: -f1
Mailing service configuration: Sendmail in FreeBSD
1. configuring sendmail for 4 most important files:
/etc/mail/access: The access file lets you set per-host and per-domain access controls
for mail server
/etc/mail/aliases: The aliases file contains a map of email redirections for the local host.
/etc/mail/mailertable: The mailertable allows you to override MX records for this mail
server.
/etc/mail/relay-domains: The relay-domains file lists domain names and addresses that
your server will relay email for.
To update the access and mailertable files, go to the /etc/mail directory and type make
maps
# cd /etc/mail
# make maps
To rebuild the aliases database, either run newaliases or make aliases:
# cd /etc/mail
# make aliases
Restart the mailing service system:
# /etc/rc.d/sendmail restart
2. edit the Smart Hosts under /etc/mail/ops.cron.cct.lsu.edu.mc, as follows:
dnl Dialup users should uncomment and define this appropriately
define(`SMART_HOST', `mail.cct.lsu.edu')
then:
# cd /etc/mail
Page 9
Server’s Ordinary Configuration for FreeBSD 2009
# make all
# make install
Issues on configuring the BOSS server network:
1. There was one problem when I configured the second network port of BOSS server to
connect to the manage port of control switch (cisco 3560E). The wireless access to the
BOSS server does not work, cannot access any kind of resources in BOSS, like www,
remote login. The reason is that the LSU wireless network space has confliction with my
setting, (previously set as default 10.0.0.2). So I consulted Adam and from whom a guy
whose name is Aaron the LSU network analyst told me to change the IP to 10.100.x.x.
After that, this problem has been solved.
2. The mailing service now in BOSS has to relay with mail.cct.lsu.edu. All the testbed
mailing transaction has to contact with the cct mail server to get into outside.
3. Basically, watching the console in a window that is saving the output during the entire
create node process is a very good idea, especially when you are fist getting a testbed
started up. You can rerun the created node process by:
boss> wap deletenode pcXXX -f -b
and then pushing the power button on the node. It will go back through the newnode
path.
Git installation on CRON system
GIT is a version control software like CVS. It is required by Utah Emulab system as they migrated
from CVS to GIT. The packages installation for our CRON is as follows:
Even though it is not yet in the ports collection or on the
Emulab wiki, the official recommended version of Git (for now)
will be 1.7.0.2, which is the most current stable release.
You should just be able to do './configure; gmake; gmake install'.
Make sure to remove the 1.5.3.2 (old) package first.
Branch maintenance under git
Page
10
Server’s Ordinary Configuration for FreeBSD 2009
Ubuntu: change user group:
http://www.cyberciti.biz/faq/howto -linux-add-user-to-group/
usermod example - Add a existing user to existing group
Add existing user tony to ftp supplementary/secondary group with usermod command
using -a option ~ i.e. add the user to the supplemental group(s). Use only with -G option :
# usermod -a -G ftp tony
Change existing user tony primary group to www:
# usermod -g www tony
root@pc2:~# usermod -G admin,adm,dialout,cdrom,plugdev,sambashare,lpadmin ccui
SCP: in both Linux and FreeBSD
ccui@pc1:~$ scp -r ccui@ops:/z/users/ccui/Drivers ~/
Configure proxy in client nodes under Ubuntu 9.04 server AMD64
Set up proxy server in ops.cron.cct.lsu.edu
ops:
cd /usr/ports/net/3proxy
make install ; make clean
ops# proxy -p9000 -i192.168.0.171
Configure your apt-get to get updated and install from shell
edit this file with sudo /etc/bash.bashrc with (vim or gedit)
and add:
#proxy
export http_proxy=http://ops.cron.cct.lsu.edu:9000/
export ftp_proxy=http://ops.cron.cct.lsu.edu:9000/
Note: Synaptic is a graphical front-end to apt, the package management system in Ubuntu. So
no need to configure Synaptic under command line in Ubuntu server edition.
Page
11
Server’s Ordinary Configuration for FreeBSD 2009
Then, try to update:
sudo aptitude update
sudo aptitude install linux-headers-$(uname -r)
sudo aptitude install build-essential
Set up wget through proxy:
sudo gedit /etc/wgetrc
uncomment the line with the http_proxy like this:
# You can set the default proxies for Wget to use for http and ftp.
# They will override the value in the environment.
http_proxy = http://ops.cron.cct.lsu.edu:9000/
ftp_proxy = http://ops.cron.cct.lsu.edu:9000/
# If you do not want to use proxy at all, set this to off.
use_proxy = on
Configure Jumbo Frame MTU in Cisco Nexus 5000
Configuring Type Network QoS Policies, turn to page. 438. The minimum MTU is 2240
bytes and the maximum MTU is 9216 bytes.
For NX-OS 4.1, the commands to enable jumbo frames are:
switch(config)# policy-map type network-qos jumbo
switch(config-pmap-nq)# class type network-qos class-default
switch(config-pmap-c-nq)# mtu 9216
switch(config-pmap-c-nq)# exit
switch(config-pmap-nq)# exit
switch(config)# system qos
switch(config-sys-qos)# service-policy type network-qos jumbo
Page
12