Getting Started With T4
Tom Cafarella
OpenVMS Boot Camp
September, 2011
© 2004 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice
Outline
•
T4 History
•
What is T4?
•
What are its components?
•
How is it distributed?
•
What is its support status?
•
Advanced Topics
•
How to install it
•
How to run it
April 7, 2015
2
T4 History
•
Idea conceived by Steve Lieman in 2001
− For troubleshooting customer performance problems.
•
Visual element missing from existing free tools
•
After-market products difficult to access inside HP
•
Prototype written in DCL by Tom Cafarella in 2001
•
Most T4 original code and TLViz visualizer written by Ian
Megarity.
•
Now maintained by Kernel Tools team and Office of
Customer Programs.
− See Mandar Chitale this week with questions
•
Used heavily by Performance Team
− See Rafiq Ahamed K and Kevin Jenkins this week.
April 7, 2015
3
What is T4 ?
•
It’s the preferred system-level data collector for
OpenVMS Engineering customer performance
engagements.
•
And a low-overhead, basic performance tool for
our customers.
•
Data collector only!
− To visualize data, use its PC companion tool, TLViz
− TLViz and Case Studies sessions after this one
•
It’s an “umbrella collector”
− Uses other low-level data collectors
April 7, 2015
4
What is T4 ?
•
Runs on OpenVMS Integrity and Alpha only
− Versions V7.3 through V8.4 and beyond
•
A single PCSI kit contains all versions
•
It is officially UNSUPPORTED
− but many of our customers depend on it
− and so do we, when performance issues arise
April 7, 2015
5
T4’s Niche
•
Mainly a performance troubleshooting tool
− Both within OpenVMS Engineering and at our customer
sites.
•
Run it all the time
− Keep history
− You never know when issues will arise
− Always need to know what the baseline is
• So we can see what changed.
•
Not designed for capacity planning
− But could be used for that in a basic way.
April 7, 2015
6
T4 Data  Powerful Visual Stories
Dramatic proof
of :higher
Node(s)
ALCOR throughput
and ALCOR
following
upgrade
9,500
9,500
9,000
9,000
8,500
8,500
8,000
8,000
7,500
7,500
7,000
7,000
6,500
6,500
6,000
6,000
5,500
5,500
5,000
5,000
4,500
4,500
4,000
4,000
3,500
3,500
3,000
3,000
2,500
2,500
08:30:00
(6-May-2003)
09:00:00
(6-May-2003)
09:30:00
(6-May-2003)
10:00:00
(6-May-2003)
10:30:00
(6-May-2003)
b [MON.SYST]Buffered I/O Rate(# 1) g
c
d
e
f
g
b [MON.SYST]Buffered I/O Rate(# 2)
c
d
e
f
April 7, 2015
7
How do you run T4?
•
@T4$SYS:T4$CONFIG
•
Submits batch job that runs for one full day
− … and schedules batch job for the next day
•
Spawns a process for each component collector
•
At end of day, converts data files to CSV files
•
CSV files packaged into few .ZIP’s for xfer to PC
•
Data files packaged into a separate .ZIP file
•
Analysis done on PC using TLViz
April 7, 2015
8
T4 Collection
•
Intent is to provide a low-overhead
comprehensive, but not exhaustive overview of
system statistics.
•
Comprised of several individual collectors
•
And your own!
April 7, 2015
9
Worrying about T4 Overhead?
It’s Not a Problem
•
T4 coexists with other high powered performance
management and capacity planning tools
•
T4 defaults  low overhead – typically less than
1/2 of 1% with 60 second sampling
•
High resolution possible at 10 seconds or less
with reasonable overhead
April 7, 2015
10
T4 Collectors
•
MONITOR or TDC
•
XFC (SHOW MEM / CACHE)
•
Dedicated lock manager (Busy %, Request)
•
TCPIP traffic system wide (pkts, mbs)
•
Network adapter (0,1, or more)
•
Login/Logout extraction from accounting log
•
FCMon (if Fibrechannel disks present)
•
VevaMon (if EVA present)
•
SPL (Spinlock Tracing) (if T4$EXPERT non-zero)
April 7, 2015
11
T4 Components
•
Collectors written especially for T4
− T4$LCK73_MON.EXE
• Lock Management statistics
− T4$NET_MON.EXE
• Data-link level network statistics
− T4$XFC_MON, T4$XFC_POST and T4$XFC_STAT
• XFC statistics
April 7, 2015
12
T4 Components
•
Collectors written especially for T4 (continued)
− T4$ACLG.EXE
• Accounting login statistics
− T4$FC_MONITOR.EXE
• Fibrechannel statistics
• Runs if Fibrechannel devices present on system
− T4$SPL_TL.EXE
• Spinlock usage statistics
• Runs when the logical name T4$EXPERT defined to any nonnull value
April 7, 2015
13
T4 Components
Collectors provided by other engineering groups
− TCPIP$EXAMPLES:TCPIP$TCP_MON.EXE
• TCP network statistics
− VEVAMON.EXE
• EVA performance statistics
• Runs if EVA is located on the system
• VEVAMON_V2_3_ALPHA.EXE and
VEVAMON_V2_3_IA64.EXE
April 7, 2015
14
T4 Components
•
Other collectors
•
T4$RDB2CSV.EXE
− Rdb statistics
− Designed to run if Rdb present on the system
− Tried it for a while, but was “not ready for prime time”
• You may see the image, but it has been disabled.
•
T4MEM.EXE
− Memory usage statistics.
− Being considered for future inclusion
April 7, 2015
15
Advanced T4 Topics
•
Add your own collector
•
Use T4$EXPERT logical name
•
Multiple concurrent collections
•
Run a “one-off” collection starting NOW
•
Use @T4$NOW to collect today’s stats “so far”
•
Use T4ABORT to cleanly stop collection ahead of
schedule
•
Keep a historical archive of collected data
•
Customize T4 command procedures
•
Use T4EXTR to convert any MONITOR file to T4 format
•
Collecting RMS statistics
April 7, 2015
16
T4 Style Collectors
Add your own!
•
Easy to add your own collector to T4 and have
it display along with existing stats.
•
Simply have it output a CSV file in “T4 style”
•
Slides describing this technique will be posted
soon at T4 website.
•
Then use T4APRC utility to combine your data
with other system data.
− Follow T4APRC example in T4$COLLECT.COM
April 7, 2015
17
Advanced T4 Collection
•
Sample rate
− 60 seconds default, 10 second minimum
•
Can go down to 1 second if T4$EXPERT
logical name set to non-null value.
− DEFINE /SYSTEM /EXEC T4$EXPERT 1
− Will also enable SPL (Spinlock Tracing) collection if
desired.
April 7, 2015
18
Multiple T4 collections
•
Overlap 2 or more T4 collections
•
Ad hoc trouble shooting at higher resolution
− Runs for short duration,
− Launch only when needed
•
Or, zoom in on known peak period
− E.g. you know the system tends to get especially busy
between 9:30 and 11:30
April 7, 2015
19
Special T4 Collections
•
Start “NOW” & do not resubmit
− Input parameters when launching T4 session
− Quick response to slowdown
− Running quick ad hoc Benchmarking tests
− Getting the feel for T4 launch procedure
− Estimating data storage requirements
− Couple this with higher resolution time
April 7, 2015
20
Process Accumulated Statistics
NOW
•
Use the @T4$NOW feature to capture stats from
currently running T4 collection.
•
Creates a composite CSV file entitled
*_xxxx.CSV, where xxxx is time of day.
•
T4 collection continues uninterrupted.
•
Lets you see what’s happened so far today.
•
May also want to do a MONITOR playback on the
T4 MONITOR.DAT file for a quicker look.
•
Restriction: Only composite file created
− No DISK, SCS, FCM or EVA data
•
Issue: @T4$SYS:T4$NOW
April 7, 2015
21
Stopping T4 Gracefully
•
T4ABORT (utility included in kit)
− Stops collection
− Produces standard CSV files
− Gracefully terminates all collectors
− Let’s you select which T4 sessions to stop
• Stop high resolution
• Leave daily history running
− Issue:
• $ SET COMMAND T4$SYS:T4$ABORT.CLD
• $ T4ABORT /ID=pid
April 7, 2015
22
Initiating T4 Historical Archive
•
Define system logical name T4$DATA
•
Choose disk & directory for output files carefully –
make sure sufficient disk quota
•
Avoid using the system disk
•
Test run to gauge size of output
April 7, 2015
23
T4$CONFIG.COM and
T4$COLLECT.COM
•
DCL – so it’s customizable to your needs
− Can comment out collectors
− Can add your own integrated collectors
• Response, throughput, database, …
− Can adjust who receives mail
− Can transfer resulting files (e.g. FTP)
− Can change the MONITOR command
• Change Processes list
April 7, 2015
24
RMS statistics
•
Turn on RMS statistics for selected files
•
MONITOR RMS
•
Then T4EXTR /CLASS=(NOALL,RMS)
•
Recommend keeping these separate
April 7, 2015
27
Where to get T4 and TLViz kits
•
OpenVMS Website:
− http://h71000.www7.hp.com/openvms/products/t4/
− T443-1.zip file. It contains:
• T4 V4.3-1PCSI kit
• T4_README.TXT
•
Starting with V8.4, same files available in
SYS$ETC:T4_V43_KIT.EXE
− Self-extracting image file. Simply RUN it to produce
above files.
− Contains an extra fix to correct FCMON problem.
April 7, 2015
28
Quick “How To Install & Run T4”
Guide
•
Obtain T4 kit
•
PRODUCT INSTALL T4
•
DEFINE/SYS/EXEC T4$SYS
SYS$SYSDEVICE:[VMS$COMMON.T4$SYS]
•
DEFINE/SYS/EXEC T4$DATA disk:[directory]
− Not system disk!
•
Create batch queue
•
@T4$SYS:T4$CONFIG.COM
− Answer prompts
April 7, 2015
29
Questions and Feedback
•
Thank you for your attention!
•
Send questions/feedback/suggestions to
T4@HP.COM
•
Working on V4.4!
April 7, 2015
30