Cross-Platform Batch Reports
VUGM 2001
Saturday, April 21
Part of
Session 40,
Customized Reporting Options
Presented by:
Roy Zimmer
Programmer/Analyst
Office of Information Technology
Western Michigan University
Cross-Platform Batch Reports – Presentation Outline
VUGM 2001
Presented by Roy Zimmer, Programmer Analyst,
Office of Information Technology, Western Michigan University
Why We Did This
initially no access to Access with the ODBC stuff, didn’t know about it, couldn’t get any
help
got more familiar with Unix, Endeavor supplied some training, figured out where
SQL*PLUS was
started exploring SQL
created several reports, the library liked them; had to run them manually, and got tired of:
remembering to run them, ftp them to a PC, format them in Word, print them, and
deliver them to various locations
we also felt they should be printed using library resources
An overview of the automated process immediately follows this outline.
When to Choose SQL, PL/SQL, or Perl
SQL
can be invoked from within SQL*PLUS
it’s sufficient to create most reports that we’ve needed
allows easy and painless control of page size, line size, and page formatting
it takes control and does a lot of the work for you
output can be to screen or to a file
PL/SQL
can be invoked from within SQL*PLUS or can be embedded in or combined with
SQL
it’s SQL with programming constructs added, so you’ve got a lot more power and
control
functionality is between that of SQL and Perl
output can only be to screen; Oracle allows for file output, but the Voyager
installation does not have this enabled
you can capture the screen output to a file, but you’re limited to a maximum of a
million bytes per “capture” session
Perl
invoked from the operating system level, but you could call it from within SQL if
you needed to
you have control over everything, which means:
you can do just about anything you want
you have to do all the work
output is to screen or file, of course
2
The Batch PC
set up a machine dedicated to this task
can be an older machine, we’re using a 133 MHz Pentium
this machine has 32 meg RAM, and the 1.5 gig hard disk is more than adequate
for O.S., we’re running Windows 95
the WinBatch software is what enables the batch aspect, see www.windowsware.com for
more information
The Batch Software
when you install the WinBatch software, it sets itself so that it has a process running in
the background and leaves an icon on the task bar
WinBatch scripts typically have a .WBT extension
double click on a script to run it
it will either run right away or appear on the task bar, awaiting its time
you’ll want to set up scripts so that their trigger and run times do not overlap
despite referring to application windows by their names, as evidenced in their title bars,
WinBatch appears to get confused and sometimes sends keystrokes to the wrong
window
don’t break up long lines in your scripts – WinBatch expects them to be contiguous
Brief Presenter Bio
working at WMU for 16 years
started out working with VAX for many years
spent several years struggling with IBM mainframe and NOTIS
supporting Voyager for the last 3 years
picked up some Oracle, SQL, PL/SQL, Perl, and Unix along the way
For more information, contact Roy Zimmer at 616.387.3885 or zimmer@wmich.edu
3
4
[This site is available to the world, so code that was part of
this document for the presentation has been removed here. This
code involved details on Voyager’s proprietary schema.]
5
-- example of accessing the blob; bib records in this case
-- setup to only show first 240 characters of a tag's data for example brevity
-- (dbms_output.put_line has several limits,
-- one of which is 255 characters per call)
-- remember to: set serveroutput on size 1000000
-- (yes, I know, this code could be written better...)
declare
cursor marcrec is
select bib_id,
seqnum,
record_segment
from bib_data
where bib_id = 985555
order by bib_id asc, seqnum desc;
m_bibid number(22);
m_seqnum number(22);
m_recseg char(990);
marc
marclen
baseaddr
strptr
tagid
taglen
offset
tagaddr
tagdata
idx
strlen
mchar
subfldchar
marceof
fmt4
fmt5
tempstr
varchar2(17000);
integer(5);
integer(5);
integer(5);
varchar2(3);
integer(4);
integer(5);
integer(5);
varchar2(9999);
integer(5);
integer(5);
char(1);
char(1):= '|';
boolean:= false;
varchar2(4);
varchar2(5);
varchar2(5);
begin
open marcrec;
loop
marc:= '';
fetch marcrec into m_bibid, m_seqnum, m_recseg;
exit when marcrec%notfound;
-- assemble blob into 1 piece
while m_seqnum <> 1
loop
marc:= m_recseg || marc;
fetch marcrec into m_bibid, m_seqnum, m_recseg;
if marcrec%notfound then
marceof:= true;
exit;
end if;
end loop;
exit when marceof;
-- get segment #1
marc:= m_recseg || marc;
-- determine info from header
marclen:= substr(marc, 1, 5);
baseaddr:= substr(marc, 13, 5) + 1;
-- loop through tag directory, get tag parameters, then tag's data
strptr:= 25;
6
while strptr < baseaddr-1
loop
tagid:= substr(marc, strptr, 3);
taglen:= substr(marc, strptr+3, 4);
offset:= substr(marc, strptr+7, 5);
tagaddr:= baseaddr + offset;
tagdata:= substr(marc, baseaddr+offset, taglen-1);
-- set up to pretty print subfield indicators
tagdata:= translate(tagdata, chr(31), subfldchar);
idx:= 2;
strlen:= length(tagdata);
while idx < strlen
loop
mchar:= substr(tagdata, idx, 1);
if mchar = subfldchar then
tagdata:= substr(tagdata, 1,
idx-1) || ' ' ||
substr(tagdata, idx, 2
) || ' ' ||
substr(tagdata, idx+2
);
idx:= idx + 2;
strlen:= length(tagdata);
end if;
idx:= idx + 1;
end loop;
-- set up to pretty print tag parameters
tempstr:= to_char(taglen);
fmt4:= lpad(tempstr, 4, '0');
tempstr:= to_char(tagaddr);
fmt5:= lpad(tempstr, 5, '0');
dbms_output.put_line(tagid || ':' || fmt4 || ':' || fmt5 || ':' ||
substr(tagdata,1,240));
strptr:= strptr + 12;
end loop;
end loop;
close marcrec;
end;
/
7
#!/usr/local/bin/perl
### lb2510.pl print to file table and column schema for Voyager database
### print tables in alphabetical order and within each table:
###
print xxx_id columns alphabetically first, then
###
all other columns alphabetically
### also, don’t split table output over page boundaries
### Word settings: MonoSpacDL821 9pt portrait, 1" T&B, .9" L&R margins
use DBI;
$pagesize = 60;
$dash3 = "---";
$dash5 = "-----";
$dash6 = "------";
$dash8 = "--------";
$dash30 = "------------------------------";
$firstheading = 1;
@tablename = ();
### connect to database
$dbh = DBI->connect('DBI:Oracle:host=voyager.library.wmich.edu;sid=YOURSID',
'dbname', 'password')
or die "connecting: $DBI::errstr";
# get distinct list of table names into an array
$sqlquery = sprintf("select table_name from user_tab_columns order by table_name");
$sth = $dbh->prepare($sqlquery) or die "preparing query statement";
$rc = $sth->execute;
$oldtablename = "";
$numtables = 0;
$loopctr = 0;
while (@row = $sth->fetchrow_array)
{
if ($row[0] ne $oldtablename)
{
$numtables++;
$tablename[$numtables] = $row[0];
$oldtablename = $row[0];
}
$loopctr++;
}
$sth->finish;
# get number of columns per table
$idx = 1;
while ($idx <= $numtables)
{
$sqlquery = sprintf("select count(*) from user_tab_columns
where table_name = '%s'", $tablename[$idx]);
$sth = $dbh->prepare($sqlquery) or die "preparing query statement";
$rc = $sth->execute;
@row = $sth->fetchrow_array;
$tablecolumns[$idx] = $row[0];
$idx++;
}
$sth->finish;
8
# generate the report
$fileout = ">lb2510.rpt";
$fopen = sprintf("Cannot open %s for output\n", $fileout);
open(fileout, $fileout) or die $fopen;
doheading();
$idx1 = 1;
while ($idx1 <= $numtables)
{
$sqlquery = sprintf("select table_name,
lower(column_name),
lower(data_type),
nvl(data_precision, data_length),
data_scale,
nullable
from user_tab_columns
where table_name = '%s'
order by column_name", $tablename[$idx1]);
$sth = $dbh->prepare($sqlquery) or die "preparing query statement";
$rc = $sth->execute;
# get data from query, mark rows where the Oracle column name ends in "_id"
# and store in array
$numrows = 0;
while (@row = $sth->fetchrow_array)
{
$isid = 0;
$col = $row[1];
if (substr($col, length($col)-3, 3) eq '_id')
{
$isid = 1;
}
$rowstr = join '|', $row[0], $row[1], $row[2], $row[3], $row[4], $row[5], $isid;
@tbldata[$numrows] = $rowstr;
$numrows++;
}
# output report data (from temporary array)
$firstone = 1;
# output "_id" columns first
$idx2 = 0;
while ($idx2 < $numrows)
{
@row = split /\|/, $tbldata[$idx2];
# if fieldname ends with "_id"
if ($row[6] == 1)
{
if ($firstone)
{
$firstone = 0;
if (($linectr + $tablecolumns[$idx1]) > $pagesize) {doheading();}
printf fileout ("%-30.30s %-30.30s %-8.8s %6.6s %3.3s %1s\n",
$row[0], $row[1], $row[2], $row[3], $row[4], $row[5]);
}
else
{
printf fileout ("%-30.30s %-30.30s %-8.8s %6.6s %3.3s %1s\n",
" ", $row[1], $row[2], $row[3], $row[4], $row[5]);
}
$linectr++;
}
$idx2++;
}
# now output non "_id" columns
9
$idx2 = 0;
while ($idx2 < $numrows)
{
@row = split /\|/, $tbldata[$idx2];
# if fieldname does not end with "_id"
if ($row[6] != 1)
{
if ($firstone)
{
$firstone = 0;
if (($linectr + $tablecolumns[$idx1]) > $pagesize) {doheading();}
printf fileout ("%-30.30s %-30.30s %-8.8s %6.6s %3.3s %1s\n",
$row[0], $row[1], $row[2], $row[3], $row[4], $row[5]);
}
else
{
printf fileout ("%-30.30s %-30.30s %-8.8s %6.6s %3.3s %1s\n",
" ", $row[1], $row[2], $row[3], $row[4], $row[5]);
}
$linectr++;
}
$idx2++;
}
if ($linectr < ($pagesize-3))
{
print fileout "\n";
$linectr++;
}
$idx1++;
}
close(fileout);
### clean up at end
$sth->finish;
$dbh->disconnect;
sub doheading
{
if (not $firstheading) {print fileout "\f";}
else {$firstheading = 0;}
printf fileout ("Table Name
Column
Type
Length Dec Null?\n");
printf fileout ("%s %s %s %s %s %s\n\n", $dash30, $dash30, $dash8, $dash6, $dash3, $dash5);
$linectr = 3;
}
10
; lb2600 - Serials: titles, PO#s, details, by fund and dept - WinBatch script
; this WinBatch file starts at a certain time, ftp's the SQL report file from
; Voyager, brings it into Word, formats it, and prints it in the Library's
; Serials printer
; should run all the time
; starting this script:
;
start after main loop checking time if today is the day the report is to
;
be printed and it has already been printed
;
else can start any time and any day
; Word settings:
;
7pt Monospac 821DL, landscape,
;
.5" margins
;
print on first floor Serials printer
;*** CONSTANTS
unixpath = "/usr/local/rpt/"
pcpath = "c:\librpts\"
rptid = "lb2600"
rptext = ".rpt"
pwfilename = "c:\librpts\batchpw.psi"
pwfile = fileopen(pwfilename, "read")
pw = fileread(pwfile)
fileclose(pwfile)
pwstr = strcat(pw, "~")
;*** BUILD COMPLETE UNIX AND PC FILENAMES
unixfile = strcat(unixpath, rptid, rptext)
;message("",unixfile)
pcfile = strcat(pcpath, rptid, rptext)
;message("",pcfile)
ftpfile = strcat("c:\program files\winbatch\batch files\", rptid, "ftp.wbt")
;*** MAIN LOOP STRUCTURE
;*********
:starthere
;*********
timenow = strsub(timeymdhms(), 12, 8)
modaynum = strsub(timeymdhms(), 9, 2)
if (modaynum == "02") || (modaynum == "16") then
if (timenow >= "08:23:00") && (timenow <= "08:45:00") then
ftppwstr = strcat('sendkeysto("ftp","', pwstr, '")')
ftpgetstr = strcat('sendkeysto("ftp",', '"get ', unixfile, ' ', pcfile, '~~")')
hftp = fileopen(ftpfile, "write")
filewrite(hftp, 'run ("c:\windows\ftp.exe","")')
filewrite(hftp, 'winwaitexist("ftp", 20)')
filewrite(hftp, 'sendkeysto("ftp","open voyager.library.wmich.edu~")')
filewrite(hftp, 'sendkeysto("ftp","zimmer~")')
filewrite(hftp, ftppwstr)
filewrite(hftp, 'sendkeysto("ftp","ascii~")')
filewrite(hftp, ftpgetstr)
filewrite(hftp, 'sendkeysto("ftp","bye~~")')
fileclose(hftp)
run(ftpfile,"")
timedelay(60)
if fileexist(pcfile) then
; ***format and print it
run ("c:\program files\mso97\office\winword.exe", pcfile)
; delete leading formfeed
sendkey("{DEL}")
; set document size and orientation
11
sendkey("!f{TAB 7}{ENTER}!s!c!m!t.5!b.5!f.5!g.5{TAB 5}{ENTER}")
; set font and font size
sendkey("!e{TAB 8}{ENTER}!o{ENTER}")
sendkey("Monospac821 DL{TAB}{TAB}7")
sendkey("!k!k!p!p!w!w!o!o!m!m!h!h{TAB 2}{ENTER}")
; now send to printer
;
this sendkey string may change whenever the
;
available printer pool changes,
;
or if the default printer is changed
sendkey("^p{TAB 7}!n{DOWN 4}{TAB 9}{ENTER}")
; wait for doc to print
timedelay(60)
; now close Word (leading CR to get rid of extra message)
sendkey("~!{F4}!n")
; wait until 0823 tomorrow so can exit processing loop
now = timeymdhms()
tomorrow_at_823 = timeadd(strcat(strsub(now,1,11),"00:00:00"), “0000:00:01:08:23:00")
timewait(tomorrow_at_823)
exitvalue = 1
;
else> hook for email notification: failure, .rpt file not found
endif
else
; wait 10 minutes and try again
timedelay(600)
exitvalue = 1
endif
else
; not desired day, so wait a day
; wait until 0823 tomorrow
now = timeymdhms()
tomorrow_at_823 = timeadd(strcat(strsub(now,1,11),"00:00:00"), "0000:00:01:08:23:00")
timewait(tomorrow_at_823)
exitvalue = 1
endif
if exitvalue == 1 then goto starthere
; this would be a good location to hook into failure notification via email
errmsg = strcat("Contact programmer.%@crlf%Script for ", rptid, rptext, " exited loop.")
message("ERROR", errmsg);
exit
===================================================================================
; example from an other script showing how to restrict
; execution for specific days
timenow = strsub(timeymdhms(), 12, 8)
a = timeymdhms()
b = timejulianday(a)
c = (b + 5) mod 7
weekday = itemextract(c+1, "Sun Mon Tue Wed Thu Fri Sat", " ")
if weekday != "Sun" then
if (timenow >= "07:53:00") && (timenow <= "08:20:00") then...
12