System Test of Endcap Peripheral crate and chamber
electronics (STEP)
User’s guide
Alex Madorsky, Victor Barashko, University of Florida
madorsky@phys.ufl.edu, barashko@phys.ufl.edu
System diagram (note the acronyms – they will be used in the instructions):
FED crate
control computer
(FEDC)
DAQ computer
(DAQC)
regular connection
to VME controller
(CAEN or other)
DAQ “spy” fiber
from DDU
custom socket
connection via
regular network
VME control
computer
(VMEC)
regular connection
to OSU crate
controller
FED crate with
DDU installed
(FED Crate)
Regular DAQ fiber
connection from
DMB to DDU
Peripheral crate
with OSU crate
controller
(PCrate)
STEP is designed to work on existing slice test hardware setup without any reconfiguration, and hopefully on final CMS Endcap hardware configuration as well. For
the correct operation, all components shown above must be present and connected as
shown. No TTC system, Trigger crate or anything else is required, but having them
connected as usual will not interfere with STEP.
DAQC, VMEC and FEDC can be 3 separate computers, or a single computer may
perform functions of any two or three of them. STEP will operate correctly in any such
combination. However, during tests at UF we’ve discovered that merging DAQC and
VMEC into one computer creates software conflicts, probably between DAQ readout and
VME control kernel-mode drivers, so such merging is not recommended.
Initial requirements
Before STEP can be installed and compiled, the following is required:
DAQC:
1. CERN Scientific Linux 3
2. DAQkit installed (including source code) and compiled.
3. DAQ readout kernel-mode driver installed
4. Passwordless access to VMEC via ssh key mechanism
VMEC:
1. CERN Scientific Linux 3
2. DAQkit installed (including source code) and compiled.
3. OSU VME controller kernel-mode driver installed
FEDC:
1. XDAQ service with DDU configuration page must be running
Installation
On both DAQC and VMEC the installation sequences are almost identical. Most of the
steps below are performed under the same account that was used to install and compile
DAQkit (USER in the text below). Some steps have to be performed as root (each case
mentioned individually). Steps that are required only on DAQC are specifically marked.
Instructions shown below assume that you use bash as your default shell. If not, you’ll
have to replace some commands with equivalents, or start using bash.
1. Unzip the source code from step.zip into your home directory. This will
create a directory named “step”, with 3 directories inside:
bin
data
fast_daq
Once you did that, any new versions of the STEP source code can be updated
from CVS. Password for CVS can be provided upon request.
2. cp ~/step/fast_daq/step_env ~
cp ~/step/fast_daq/xdaqenv.sh ~
This is your site-specific environment settings. It is important to have these files
in your home directory, since they are used by some of the STEP scipts.
3. Edit ~/step_env on both DAQC and VMEC. Change the following
parameters:
(a) STEPHOME, to make it point to the location of “step” directory on your
local disk. Example:
export STEPHOME=/home/daq/step
(b) JAVADIR must point to your Java Development Kit. Example:
export JAVADIR=/usr/local/java/jdk
(c) VME_SERVER_ADDRESS must show the address of VMEC. Example:
export VME_SERVER_ADDRESS=cms-dqm01.phys.ufl.edu
(d) VME_XML_FILE must specify the name of the default XML
configuration file on VMEC (see XML configuration files section below)
(e) DAQSCHAR must specify the schar device name that is used to access the
DAQ readout driver:
export DAQSCHAR=-scharX (where X is a number 1,2 3 etc.)
(f) VME_STARTUP_DELAY_SEC specifies a number of seconds that STEP
will wait for the VME server startup. Default of 2 seconds should be
enough for most systems. However, this number may need to be increased
if your ssh connection with VMEC is slow.
4. Edit ~/xdaqenv.sh on both DAQC and VMEC. Change one parameter:
(a) XDAQ_ROOT must point to the TriDAS directory of the DAQKit.
5. Add the following line to your .bashrc file, so the STEP environment is
automatically configured in each new console:
source ~/step_env
6. As root, install the following rpm packages:
xforms-0.89-5
xforms-devel-0.89-5
castor-lib-2.1.2-4.i386.rpm (CERNLIB needs it)
CERNLIB-2004-4 (make sure to take version for CERN Scientific Linux 3)
7. As root:
ln –s /usr/libexec/CERNLIB /cern
cd /usr/src
ln –s linux-2.4 linux
8. As root:
check that the following file is available:
/usr/lib/libshift.so
if this file is not available on your system, find a file with a name similar to this
one:
/usr/lib/libshift.so.x.x.x.x (where x.x.x.x is a version number)
and do this:
cp /usr/lib/libshift.so.x.x.x.x /usr/lib/libshift.so
9. As USER:
cd
source step_env
cd step/fast_daq
make
That should make the entire package. You’ll see some warnings during compilation,
please ignore them. If the compilation is not finished because of errors, please let us
know.
Additional steps required on DAQC:
1. As root, add the following line to your /etc/syslog.conf file:
local1.* /var/log/emu_daq
2. As root, restart syslog:
/etc/init.d/syslog restart
3. As root:
touch /var/log/emu_daq
chmod 644 /var/log/emu_daq
4. As root, create data file storage directory:
mkdir /csc_data
chown USER /csc_data
chmod 755 /csc_data
This directory must have enough space to store your data files. At least 20 GB is
recommended. If necessary, a symbolic link may be created with the same name,
pointing to the actual storage area, as shown below:
ln –s DATA_STORAGE_DIR /csc_data
XML files
XML configuration files reside on VMEC, in step/data/xml directory. They are
used by vme_control program to configure the EMU system for STEP use.
vme_control is executed remotely by STEP software during startup process, and
terminated when STEP terminates on DAQC.
Several example files are available in step/data/xml directory. However, for each
chamber and for each test performed on that chamber the STEP operator must create an
individual XML file. The XML file creation process is outlined below:
1. Locate the XML configuration file created for normal Slice test operation. That file
should include all chambers available in the system, and should configure them for
ALCT-CLCT coincidence trigger. If you’re not sure where this file is located, one of
the individuals listed below should be able to help you:
• Frank Geurts
• Greg Rakness
2. Copy that file into step/data/xml directory on VMEC, and rename it. We
suggest that the new name should include test number, chamber number on disk and
serial number of the chamber that you’re going to test.
3. In the new file, locate the PeripheralCrate section for the crate that your
chamber is connected to. Remove all other PeripheralCrate sections.
4. Inside the PeripheralCrate section:
(a) Remove MPC section
(b) Locate the CSC section for the chamber you’re going to test. Remove all other
CSC sections.
5. The resulting file must contain only one PeripheralCrate section, with only one
CSC section and one CCB section inside.
6. Modify the parameters as shown in Table 1, according to the test that you’re creating
this XML file for.
7. One of the newly created XML files must be declared a “default” configuration file.
XML file created for any test can be used as default, but it is important that it
describes the chamber that you’re going to test. Set the default XML file on DAQC
via ~/step_env script, VME_XML_FILE variable. Example:
export VME_XML_FILE=step_4110.xml
8. If a working XML file for the chamber you’re going to test is not available, an XML
file from similar chamber can be used. In addition to the parameters shown above, the
following parameters need to be adjusted in it:
(a)
(b)
(c)
(d)
VME controller address and port
CFEB and ALCT cable delays (measured with utilities provided in DAQKit)
TMB and DMB slot numbers
Crate ID parameter (in DMB section)
Don’t forget – all XML files are created and used on VMEC.
Test
Section
11, 15
Modified Parameter
TMB
trgmode=”2”
TMB
trgmode=”2”
trig_mode=”0”
12
fifo_pretrig=”12”
ALCT
fifo_tbins=”8”
l1a_delay=”115”
send_empty=”1”
16a
TMB
trgmode=”1”
trgmode="1"
18
TMB
hs_pretrig_thresh="1"
ds_pretrig_thresh="1"
Table 1 XML file parameter modification for STEP tests. (More tests will be added as they become
available)
DDU configuration
For tests 11 to 26, the DDU must be configured for pass-through mode as shown below:
1. Locate the DDU that receives the fiber from DMB that is connected to chamber under
test. Note the fiber number for that particular DMB (fn in the text below, ranges
from 1 to 16, fiber #1 is on top of DDU front panel).
2. Open XDAQ configuration page for the selected DDU
3. Modify the following parameters:
a. Kill Fiber = (1 << (fn-1)) + 0x70000
// must be in hex
b. Gbe prescale = f0f0
c. Fake L1A req = 8787
4. Press “DDU sync reset via VME” button.
Running STEP
Test 9
•
•
Make sure VME_XML_FILE variable is set
Type “test9 chamber_number chamber_type <Enter>”.
Parameters:
chamber_number is the serial number of chamber under test
chamber_type is:
0 - ME1/1
1 - ME1/2
2 - ME1/3
3 - ME2/1
4 - ME2/234
5 - ME3/1
6 - ME4/1
Tests 11 to 26
1. Type step <Enter>. The following components should start:
• vme_control is started on VMEC in a separate xterm (this program is
accepting commands from STEP and controlling boards via VME)
• ddu_b_daemon is started on DAQC in a separate xterm (this program is
reading the DAQ data from Gigabit Ethernet card and transfers them into STEP
shared memory area)
• Message window (GUI)
• Event display (GUI)
• Event number monitor (GUI)
• Run control window (GUI)
2. Select the test number in Run control window (see Figure 1)
3. Specify the name of XML file (without path) in Run control window (see Figure 1).
Make sure that file exists on VMEC, and is located in step/data/xml directory.
4. Press INIT button, wait until the START button appears
5. Press “DDU sync reset via VME” button (in XDAQ web-page)
6. Press START button.
The data collection should begin. Event number monitor will show increasing numbers,
as the events are being collected.
The events can be viewed in real time using Event Display:
1. Press “File” button
2. Select “Open Event Shared Memory”(Run number)
Once the run is finished, the data file can be processed as it is done normally at FAST
sites, with lisas_test_manager.
Select test number here
Type XML
configuration file name
here before pressing
INIT button
Figure 1 STEP Run Control window