Progress System
Tables
Dan Foreman
BravePoint, Inc.
Email: danf@prodb.com
BravePoint
Introduction - Dan Foreman
•
•
Progress User since 1984 (V2.1)
Presenter at lots of Progress
conferences dating back to 1990
BravePoint
Introduction - Dan Foreman
Author of:
• Progress Performance Tuning Guide
• Progress Database Admin Guide
• Progress Virtual System Tables
• Pro D&L - Dump/Load with short down time no
matter how large the Database is
• ProMonitor – DB monitoring tool (uses VSTs)
• ProCheck – Monitor AppServers & WebSpeed
BravePoint
What are Virtual System Tables
Virtual System tables (VSTs) provide
access to Database Shared Memory data
that was previously only accessible
through promon
It’s possible to customize the user
interface
Accessible Locally or Remotely via the
4GL or SQL
BravePoint
What are Virtual System Tables
VSTs are Schema Tables
They do not interfere with Application
tables and Indexes
VSTs are not part of the empty Database in
V8
Enable with proutil <db> -C enablevst
Are part of the V9/V10/V11 empty Database
Are visible in the Dictionary (must view
‘Hidden’ tables)
BravePoint
What are Virtual System Tables
Some new tables and fields are not in
the “native” empty DB unless:
proutil –C updatevst is run
Example: New VST fields added in
V10.2B SP6
The feature must be enabled:
Auditing: proutil enableauditing
TDE
Multi-Tenancy
BravePoint
What are Virtual System Tables
VSTs in the Standard empty DB
V8.2
30 tables
V8.3
35 tables
V9.0-V9.1C
39 tables
V9.1D
42 tables
V10
45 tables
V10.1B
47 tables
BravePoint
What are Virtual System Tables
promon can lag behind VSTs;
Examples:
_TableStat & _IndexStat (V8.3)
_AreaStatus (V9 and later)
Conversely VSTs can lag behind also:
_Startup
BravePoint
Resource Impact of VSTs
There is no performance penalty for
accessing VSTs with 2 exceptions that
will be covered later
VSTs take up no additional disk space
except for the VST schema
VSTs take up a small amount of
memory to cache the additional
schema
Except –tablerangesize & -indexrangesize
can increase DB Shared Memory by a
surprising amount
BravePoint
VST Implementation
Whenever a record is requested, the
table is checked to see if it’s a VST
table.
VST tables have table numbers between
-16385 to -32768.
The check is simple arithmetic which takes
place regardless of whether VSTs have
been enabled or not
BravePoint
VST Implementation
If a requested record is a VST record,
the fields are populated from shared
memory structures instead of from a
database record
BravePoint
VST Basics
Each VST has one, unique index on the
‘Id’ field which is a ‘virtual’ RECID
Each VST record is numbered starting
at 1 and continues sequentially (some
exceptions)
It is not possible to add new indexes or
triggers to VSTs
BravePoint
VST Basics
Some VST Record Counts are
controlled by startup parameters
_connect
-n & -Mn
_LockReq
-n & -Mn
_UserIO
-n & -Mn
_UserLock
-n & -Mn
_UserStatus
-n & -Mn
_Trans
-n
_Servers
-Mn
_Lock
-L
BravePoint
VST Basics
The tables on the last slide are prepopulated at Database Broker startup
time
The empty slots have “?” values
To avoid displaying the empty slots:
FOR EACH _connect
WHERE _connect-usr <> ?
The <> operator is not a performance
problem since VSTs are not really
‘indexed’
BravePoint
Which Database
If a Client is connected to multiple
Databases, the VST Statistics will be
based upon the Current Working
Database
BravePoint
Updatable VSTs
_Startup-spin
_Startup-APWQTime
_Startup-APWSTime
_Startup-APWBuffs
_Startup-APWMaxWrites
_Startup-
BravePoint
-spin
-pwqdelay
-pwsdelay
-pwscan
-pwwmax
Updateable in V10.2B SP6
_Startup._Startup-LRU-Skips
_Startup._Startup-LRU2-Skips
_Startup._Startup-NmsgWait
_Startup._Startup-Pefetch-Delay
_Startup._Startup-Prefetch-Factor
_Startup._Startup-Prefetch-Num-Recs
_Startup._Startup-Prefetch-Priority
BravePoint
Updatable VSTs
_MyConnect-NumSeqBuffers
Added in V9
Same as –Bp parameter
Number of Sequential Read Buffers
Makes it possible to run a large report
with a less severe impact on the -B
Buffer Cache
Example on the next slide
BravePoint
Updatable VSTs
DO FOR _MyConnect TRANSACTION :
FIND FIRST _MyConnect.
_myconnect-numseqbuffers = 32.
END.
RUN killer_report.p.
DO FOR _MyConnect TRANSACTION :
FIND FIRST _MyConnect.
_myconnect-numseqbuffers = 0.
END.
BravePoint
Updatable VSTs
_StatBase._TableBase
_StatBase._IndexBase
V9 and later
Used to set the window size of Tables
and Indexes monitored using the
_TableStat and _IndexStat VSTs or to
reset (zero out) the statistics by
moving the “window” up and then
down
BravePoint
Table & Index I/O VSTs
Note that _TableStat and _IndexStat are
available in V8.3 and very helpful for
Storage Area Planning
Use _IndexStat to identify unused
indexes; COMPILE/XREF is only good
for finding indexes that are defined but
not used in the code; COMPILE/XREF
is useless for Dynamic Queries
BravePoint
VST Problems
Potential Performance Killers
_Block ( V8, useless in V9/V10/V11 )
_Lock ( FOR EACH _lock WHILE … )
_UserLock ( DO loops of 512 iterations )
Too Much Data for One Screen
_Block-block
_UserLock-*
Missing Data
_DBStatus
BravePoint
x(1024)
Several arrays of 512
VST Problems
Storage
VSTs are defined as INTEGERs which
limits the maximum value to about 2
billion – prior to V10.1B
Poorly documented (see my System
Table Guide for better
documentation)
Stale Data
_Lock (total # of records) compared to
_DbStatus-NumLocks
BravePoint
System Tables - MetaSchema
_File
_Field
_Index
_Index-Field
_File-Trig
_Field-Trig
_Db
_Sequence
BravePoint
System Tables - Other
SQL
Authentication (GRANTs & REVOKEs)
Views
Constraints
Query Optimizer statistics
Auditing
Encryption
Multi-Tenancy
BravePoint
Usefulness
VSTs are useful for two types of
Progress users
Application Developers
Database Administrators
Who else is there?
BravePoint
For Developers
Can view the lock status for an
individual record
_Lock VST shows all active locks and
who has the lock
We can easily find the Table name of the
record being locked by doing a join to
the _File table
_Trans VST shows all active
transactions
BravePoint
For Developers
Deadly Embrace (Deadlock) Detection
_Lock
_Connect
“Delinquent” Active Transactions that
grow the BI File to an abnormally large
size
_Connect
_Trans
BravePoint
For Developers
Improve code efficiency by measuring:
User DB Accesses, Reads, etc.
Reads, Writes, Creates, Deletes by Table
_TableStat
Unused indexes; Logical Scatter Factor
_IndexStat
BravePoint
For Developers
Name & attributes of a Field
find _file “customer” no-lock.
for each _field of _file no-lock :
display
_field-name
_data-type
_format
_extent.
end.
BravePoint
DBAs - Tables of interest
Monitoring the BI High Water Mark
_DbStatus._dbstatus-bisize
Available in V8.3 and later
V8: BI Size in Bytes
V9 & later: BI Size divided by BI Block
Size
BravePoint
DBAs - Tables of interest
Monitoring Database Size
promon doesn’t have the correct data
Each Storage Area has a separate
High Water Mark
_AreaStatus
_AreaStatus-AreaName
_AreaStatus-HiWater
_AreaStatus-TotBlocks
_AreaStatus-LastExtent
_AreaStatus-Extents
BravePoint
DBAs - Tables of Interest
Record Fragmentation
_Record._Record-RecRead
_Record._Record-FragRead
dbanalys/tabanalys only shows
how many Fragments exist, not
how often the Fragments are
accessed
BravePoint
DBAs - Tables of Interest
What Client is running what Program
_Connect-Cache*
Also Line# for 4GL Code
Or SQL Statement
Since V10.1C
What Tables is a Client Hitting
_UserTableStat
Since V10.1B
BravePoint
For More Information
Progress System Tables Guide
200+ pages
Covers V8 through V11
Covers VSTs and System Tables
Has lots of undocumented information
BravePoint
ProMonitor
Written using (mostly) VSTs
Source Code provided for most Programs
Threshold Alerts warn of problems
Monitor an unlimited number of DBs
Monitors more than just VST Statistics
Session Temp File I/O
Client Memory Utilization
DB Log File Monitoring
DB Fragmentation/Scatter Monitoring
AppServers, WebSpeed, AdminServer, etc
BravePoint
Conclusion
Questions?
Dan Foreman
danf@prodb.com
www.BravePoint.com
BravePoint