Voyage meets MeLCat:
MC’ing the Introductions
MeLCat extract sequences
Voyager
Pmarcexport
bibout.pl
patout.pl
today’s
extract
yesterday’s
extract
patdiff.pl
patron extract
bib extract
to MeLCat
Disclaimer
Our MeLCat implementation is unfortunately
still a work in progress.
The information presented here may be
subject to change.
The information is as current as possible,
given deadline constraints.
Notes
The bib extract and patron extract programs,
and the patron differences program, are all
written in Perl and are designed to run on
your Voyager box.
A Unix box is assumed. Minimal modifications
should be required in order to run on a
Windows box.
Notes
User requirements:
Familiarity with a text editor, file names, and
paths on your machine.
You’ll need to be able to query your database
(SQL provided).
You will be editing the setup section at the
beginning of the programs to customize
settings for your institution.
MeLCat extract sequences
Voyager
Pmarcexport
bibout.pl
patout.pl
today’s
extract
yesterday’s
extract
patdiff.pl
patron extract
bib extract
to MeLCat
Bib Extract, Pmarcexport
/m1/voyager/YOURdb/sbin/Pmarcexport \
-oOUTPUTFILE \
-mBtoday-N \
-ttoday-N \
-i
The export command is in script form in a file. Thus
the parts of the command have been broken out. The
trailing “\” indicates the command continues on the
next line.
Bib Extract, Pmarcexport
/m1/voyager/YOURdb/sbin/Pmarcexport \
-oOUTPUTFILE \
-mBtoday-N \
-ttoday-N \
-i
Substitute your own values for words in upper case.
Bib Extract, Pmarcexport
/m1/voyager/YOURdb/sbin/Pmarcexport \
-oOUTPUTFILE \
-mBtoday-N \
-ttoday-N \
-i
The mode, m, is B, which means to select records by
both create and update dates.
The export target, t, gives the date range.
The last parameter, i, says to ignore suppressed bib
records.
Bib Extract, Pmarcexport
/m1/voyager/YOURdb/sbin/Pmarcexport \
-oOUTPUTFILE \
-mBtoday-N \
-ttoday-N \
-i
today-N specifies to look back N days from today
At WMU, we used a value of 10,000 for our initial
load. For ongoing loads, we use a value of 1, since
we run our MeLCat bib extract daily.
Bib Extract, Pmarcexport problems
We had problems with Pmarcexport.
Our initial large bib export resulted in two
files, not one!
This turned out to be expected, but
previously undocumented, behavior.
Bib Extract, Pmarcexport problems
We had problems with Pmarcexport.
Our initial large bib export resulted in two
files, not one!
This turned out to be expected, but
previously undocumented, behavior.
Had problems with record counts when a
split occurred.
Bib Extract, Pmarcexport problems
We had problems with Pmarcexport.
Our initial large bib export resulted in two
files, not one!
This turned out to be expected, but
previously undocumented, behavior.
Had problems with record counts when a
split occurred.
Also had mysterious SQL error which
didn’t(?) seem to affect the output.
When we finalized this presentation, these
issues had not yet all been resolved.
MeLCat extract sequences
Voyager
Pmarcexport
bibout.pl
patout.pl
today’s
extract
yesterday’s
extract
patdiff.pl
patron extract
bib extract
to MeLCat
Bib Extract, Usage
bibout.pl
voy_bib_export_file
melcat_bib_file
voy_bib_export_file is created by Voyager’s
Pmarcexport program
melcat_bib_file is the resulting bib file ready
to be sent to MeLCat
Both files are in MARC format.
Bib Extract, Usage
Typically, Pmarcexport will create one output
file. For large amounts of data, you may
have two or more files.
In that case, you can run bibout on each file,
or possibly concatenate the Pmarcexport files
into one for bibout to process.
In this presentation, and in bibout.pl,
Pmarcexport’s output is referred to as one
file.
Bib Extract, Usage
Two stages:
test extracts and initial loads
ongoing production extracts and loads
The exact usage is determined via a
Pmarcexport script parameter and the
corresponding setting for bibout.
Bib Extract – What does it do?
bibout reads the file created by Pmarcexport.
For each bib, find and scan its mfhd(s) for
fields 866, 867, and 868. These fields, if
present, are added as is to the bib record.
Get mfhd and item information associated
with this bib record, or just mfhd data as a
limited fallback when there is no item record.
Bib Extract – What does it do?
Append a new field to the bib record that
contains the following data:
|a
|b
|c
|d
|e
|f
|g
|h
item id (mfhd id if no item)
item barcode
item status
item type
call number
location
enum info
number of recalls
This is field 999 by default.
Bib Extract – What does it do?
|a item id
When there is no item id, |a contains a mfhd
id. This is indicated with a leading “M” in this
field.
Still to come: handling deleted records
Bib Extract, database access
bibout needs to know how to connect to your
Voyager database.
At WMU, we’ve standardized on storing this
information in a file, allowing for easier
updates and better security. Any programs
that access Voyager do not have the specific
database access information in the program.
Bib Extract, database access
The information you need for this is your:
machine name
database name (may be username)
database login (password)
Oracle SID [VGER | LIBR]
Voyager 5’s SID is VGER; the SID for previous
versions is LIBR. Use the right one for your
version.
Bib Extract, database access
Put this information in the file, all on one
line:
your.machine.name.edu/namedb/db_login/sid
(namedb may be username in your situation)
The slashes, “/”, belong in there, and the bib
extract program will read this data so it can
access Voyager. Make sure that this file contains
your information, including the SID.
Then we need to tell bibout where this file is,
by specifying its path and the file name.
Bib Extract - which database?
bibout needs to know from which database
we are extracting the data.
Since the database name may not equate to
the user, we specify the database name
separately.
For example, you may be connecting to your
database as a read-only user, so you would
have to specify the database.
Bib Extract
bibout also needs to know how many days to
look back.
Give it the same value, N days, that you
used in the export script.
This makes sure that everyone is on the
same page.
Bib Extract – Which locations?
We wanted to restrict, by location, the items
showing up in MeLCat.
Here’s a sample query to see your location
codes:
Select the location_codes you want, and put
them in a text file, one per line.
Bib Extract – Which locations?
After creating the location codes file, you
need to tell bibout where this file is (the
path), and the file name.
Bib Extract
By default, the new field that contains the
item/mfhd information is 999.
If you need a different tag, you’ll need to tell
bibout what that is.
MeLCat extract sequences
Voyager
Pmarcexport
bibout.pl
patout.pl
today’s
extract
yesterday’s
extract
patdiff.pl
patron extract
bib extract
to MeLCat
Patron Extract, Usage
patout.pl
patron_extract_file
patron_extract_file is the double quoted,
comma separated value format patron data
retrieved from Voyager
Patron Extract – What does it do?
Similar to bibout, the patron extract program
(patout.pl) also looks back so many days to
get patron data.
Voyager is queried and the patron extract
data for MeLCat is put in CSV format.
Patron Extract, Output
last name, first name, middle name,
patron barcode, patron group,
patron exp date,
WIN (WMU ID number), patron ID (Voyager
internal number),
delete indicator (D=delete, blank to retain),
blocked (Y/N),
email address, email exp date
patron exp date is 2382.12.31, if indefinite
Patron Extract, blocking
Patron is blocked when:
patron record expired before today
lost items count exceeds lost limit defined in
the extract
recalls count exceeds recall limit defined in
the extract
fees due amount exceeds the maximum
balance for the patron group used in the
extract
email address record expired before today
Patron Extract, assumptions
The extract uses the patron’s most recent
active patron group as the patron group of
record.
Patron Extract, database access
The patron extract gets its Voyager database
access information from the same file
mentioned earlier in the bib extract part of
this presentation…
Database Access
Put this information in the file, all on one line:
your.machine.name.edu/namedb/db_login/sid
(namedb may be username in your situation)
The slashes, “/”, belong in there, and the bib extract
program will read this data so it can access Voyager.
…then we need to tell patout where this file is, by specifying its path and
the file name.
Patron Extract
Again, like with bibout, we need to tell
patout how many days back to look to get
patron data.
This value will probably depend on how often
your patron data is updated.
Patron Extract
We discovered setting up limits by patron
group to be pretty much impossible. This is
because a patron group can be in multiple
circ groups, with each circ group able to
have different limits.
So we went with across-the-board limits, and
you’ll have to use a global value for the
lost-limit and the recall-limit.
Patron Extract, patron groups
There will probably be some patron groups
that should not be in MeLCat.
You’ll need to tell patout which patron groups
are allowed. This is done via the patron
group ID.
Here’s how to get these ID numbers:
Patron Extract, patron groups
Now you have a listing of patron group IDs
and names. Note that fine/fee limits can be
found in the Systems Administration client by
clicking on Circulation, then on
Policy Definitions.
Record your patron group limits at each
corresponding patron group ID.
Remember that in Voyager, money values
(U.S.) are stored in cents, so the values you
enter here, divided by 100, equal the dollar
amounts.
Patron Extract
We also need a representative circ group for
patout to do its thing.
From your Sys…Admin… client, pick a
circulation policy group with the most
appropriate lending attributes. Now run a
query to get its ID number:
and let patout know what it is.
Patron Extract
Finally, patout needs to know where to write
the output. Specify the file path and file
name.
The filename will automatically have a date
stamp appended.
For example, if you’re using a file name of
“patron.out”, you’ll end up with file names
like patron.out.20060825
patron.out.20060826
etc.
MeLCat extract sequences
Voyager
Pmarcexport
bibout.pl
patout.pl
today’s
extract
yesterday’s
extract
patdiff.pl
patron extract
bib extract
to MeLCat
Patron Differences Extract
patdiff.pl looks at the current and the
previous patron extract and puts the
differences into the patron extract to be sent
to MeLCat.
The extract comparison is based on patron
ID.
Patron Differences Extract
The extract gets:
new records
records no longer showing up; they are
marked for deletion
the current version of any records with
differences in them
Patron Differences Extract
patdiff needs to know where to write its
output file. Specify the file path and the file
name.
Extracts ready!
Your bib and patron extracts are now ready
to be sent to MeLCat.
Thanks for listening.
Questions?
roy.zimmer@wmich.edu
269.387.3885
Picture © 2005 by Roy Zimmer