IBM ^
z/VM
Module 8: CMS Pipelines
© 2004 IBM Corporation
IBM ^
Objectives
 Describe the fundamental concepts behind CMS Pipelines
 Explain what device drivers are and how they work
 Explain the difference between running the PIPE command
from a command prompt and from a REXX EXEC
 Describe filters and how they are used within a CMS Pipeline
© 2004 IBM Corporation
IBM ^
Objectives continued
 Describe how to use the LOCATE and FIND commands to
select records
 Explain how to create your own filters using REXX
 Show how to use pipelines in EXECs and XEDIT macros in
different subcommand environments
 Describe the commands needed to create a SPOOL file to hold
information
© 2004 IBM Corporation
IBM ^
Objectives continued
 List the commands needed to read information from a SPOOL
file
 Describe the CP commands that control the virtual reader
 Describe what multistream pipelining is and how to use it
© 2004 IBM Corporation
IBM ^
What is a Pipeline?
© 2004 IBM Corporation
IBM ^
Device Drivers
© 2004 IBM Corporation
IBM ^
The PIPE Command
 PIPE:
© 2004 IBM Corporation
IBM ^
Some Popular Device Drivers
 The console filter reads from the terminal and types on it; for
example:

Pipe console | console
 CONSOLE can provide two functions:

Read input, when it is first in a pipeline specification

Type the input it gets, when it is not first
 A device driver that writes to a device also writes the output to
the pipeline.
© 2004 IBM Corporation
IBM ^
Some Popular Device Drivers, continued
 Reading and writing disk files:
– ‘<‘ to read a file
– ‘>’ to create or replace a file
– ‘>>’ to append to or create a file
 LITERAL:
– A literal creates a record with the argument string and writes to a
pipeline
 Combining input drivers:
– This allows the programmer to create a file at one location and
append, copy, or overwrite the file later in the pipeline.
© 2004 IBM Corporation
IBM ^
Using the PIPE Command and Other Commands
 The PIPE command is not part of the pipeline itself, therefore it is not
considered a stage.
 In the reading and writing example, the left-hand stage reads the file
from disk and the right-hand stage appends to the file specified.
 In the second example, we use the CONSOLE command to print the
file to the screen. This is helpful when you want to see the file
quickly.
© 2004 IBM Corporation
IBM ^
Filters
 A filter is an application in a pipeline that takes its input from the stage to the
left and passes its output to the stage to the right.
 The filters that are supplied with CMS Pipelines have many general-use
functions.
 A function can be anything.
© 2004 IBM Corporation
IBM ^
Filters: XLATE - Change Characters
XLATE:

This filter translates data passing through the pipeline on a
character-by-character basis.

Some sample pipes:
– Pipe literal ABCDEFG | xlate 1-* lower | console
– Pipe literal abcdefghi | xlate c-g = e e | console

This filter is also capable of altering multiple characters and
character ranges in a single stage or pipe.
© 2004 IBM Corporation
IBM ^
Filters: XLATE - Change Characters
XLATE:
/*
Replace “%” with “.” and
*/
/*
Replace “!” with “0” and
*/
/*
Replace “4” with “0” and
*/
/*
Replace “+” with “-”
*/
“Pipe Literal Don’t forget that (2+2) / 10 = 40%! |”,
‘Xlate 1-* % . ! 0 4 0 + - |’,
‘Console’
________________________________________
 Don’t forget that (2-2) / 10 = 00.0
© 2004 IBM Corporation
IBM ^
Making Records Longer or Shorter
 CHOP truncates each record after a column.
 PAD fills each record to the specified length with a pad character (the
default is a blank).

Example:
– Pipe disk Unknown Data | pad 256 | chop 256 | > Demo output A
 You can combine chop and pad to create fixed format records.
 Strip removes blanks from both the beginning and the end of records.
© 2004 IBM Corporation
IBM ^
Reformat Records
 SPLIT creates an output record for each blank-delimited word
in its input record.
 JOIN creates a single record from one or more input records.
 FBLOCK reformats the input stream to fixed length records.
© 2004 IBM Corporation
IBM ^
Edit and Rearrange Contents of Records
© 2004 IBM Corporation
IBM ^
Buffer Filters
 A filter that buffers a file reads all input records before writing
output records.
 The SORT filter must buffer the file by the nature of its
processing.
 Use BUFFER when a file must be buffered but not reordered.
 Examples:

Pipe Disk INPUT FILE | Split | Sort unique | Console

Pipe console | buffer | stack
© 2004 IBM Corporation
IBM ^
Selecting Records
© 2004 IBM Corporation
IBM ^
Looking for Labels - FIND and NFIND
© 2004 IBM Corporation
IBM ^
Discarding and Keeping Records
 Use TAKE and DROP to retain or discard a specified number of
records from the beginning or end of the file.
 TAKE and DROP make it easy to select records based on their
position in the file.
 The DROP filter is the converse of TAKE, which allows you to
delete the first or last n lines.
© 2004 IBM Corporation
IBM ^
Writing REXX Filters
 There are two types of REXX
programs you can write to run in a
pipeline:
One
reads input records and writes
output records, just like all other
programs
The
other kind is a subroutine pipeline
 The REXX program ‘COPY REXX’
copies its input to the output. It can be
used as a prototype for more complex
filters.
© 2004 IBM Corporation
IBM ^
Pipeline Input and Output in a REXX Program
 Two of the most important REXX interface commands are
READ and WRITE.
 The READTO command has a single argument: the name of
the variable you wish to be set to the contents of the next input
record.
 The OUTPUT command is followed by the data you wish to
write; you can compute the output data as a REXX expression
or you can write a literal.
© 2004 IBM Corporation
IBM ^
How to Use Filters
© 2004 IBM Corporation
IBM ^
How to Use Filters, continued
© 2004 IBM Corporation
IBM ^
Using Pipelines in EXECs
and XEDIT Macros
© 2004 IBM Corporation
IBM ^
Using Pipelines in EXECs and XEDIT Macros continued
 VAR handles the special case where you want to read and write
a single record with the contents of a REXX variable.
 When you run the pipeline shown in the notes, the SCRIPT
macro reads a line from the stack and inserts it in the
document.
© 2004 IBM Corporation
IBM ^
Getting Information about Files
 STATE and STATEW provide information about selected files.
 STATEW only searches minidisks that are accessed in write mode.
 The underlying CMS commands allow asterisks for components of a file name, so you
can find the first occurrence of a certain type of file.
© 2004 IBM Corporation
IBM ^
Subcommand Environments
 There can be several subcommand environments active in your session, such
as XEDIT, CMS, and ISPF.
 The SUBCOM device driver takes as an argument the name of a
subcommand environment, which is used for execution and displaying output.
© 2004 IBM Corporation
IBM ^
Accessing Files in XEDIT
© 2004 IBM Corporation
IBM ^
More REXX Interface Commands
© 2004 IBM Corporation
IBM ^
More REXX Interface Commands
 CALLPIPE replaces a stage with a pipeline.
 The asterisk followed by a colon, which seems to be a stage by itself,
is called a connector.
 Connectors are the magic that tells CALLPIPE to take the input and
output for the stage issuing the command and connect it to a new
pipeline.
 CALLPIPE returns when all stages of the new pipeline have
completed.
© 2004 IBM Corporation
IBM ^
Output Device Driver - Creating a SPOOL File
 Three device drivers write lines on unit record devices.
 Some output device drivers include:

Printmc  For a virtual printer

Punch  For a virtual punch

Uro
 Can write to either device
 For complete control, you must issue SPOOL, TAG, and
CLOSE commands as required.
© 2004 IBM Corporation
IBM ^
Unit Record Input - Reader SPOOL Files
 SPOOL files in your virtual reader can come from several sources that
have different formats; for example:

virtual card punch

virtual printer

CP-generated SPOOL files, such as a VMDUMP

read from a real card reader
 A reader reads a file and writes a line to the pipeline for each CCW in
the SPOOL file.
© 2004 IBM Corporation
IBM ^
CP Commands to Control the Virtual Reader
 The characteristics of a virtual reader can be set by the CP command
SPOOL, which can control:

CLASS

NOCOUNT & CONT

NOHOLD & HOLD
 A printer file is probably easier to handle than a punch file because
you often want to retain the carriage control provided by the printer.
 Punch files are more complicated because a punch file usually has
more than plain data records.
© 2004 IBM Corporation
IBM ^
Example: Process Reader Files
© 2004 IBM Corporation
IBM ^
Multistream Pipelines
© 2004 IBM Corporation
IBM ^
Examples of Multistream Pipelines
© 2004 IBM Corporation
IBM ^
Specifying Multistream Pipelines
© 2004 IBM Corporation
IBM ^
Combining Streams
© 2004 IBM Corporation
IBM ^
Combining Streams continued
© 2004 IBM Corporation
IBM ^
Pipeline Stalls
© 2004 IBM Corporation
IBM ^
Conclusion
 Each section of this module deals with an aspect of CMS Pipelines.
 The pipeline concepts discussed were:

Device drivers

Filters

Selecting records

Writing REXX filters

Using pipeline macros

Unit record input and output
 These topics help students learn the necessary elements for creating
their own pipelines.
© 2004 IBM Corporation
IBM ^
Glossary
CHOP: – truncates each record after a column
FBLOCK: – reformats the input stream to fixed length records
Filter: – a stage in a pipeline that takes its input from the stage to the
left of it and passes its output to the stage to the right of it
JOIN: – creates a single record from one or more input records
© 2004 IBM Corporation
IBM ^
Glossary
LOCATE: – writes only the records that contain a specific string
NLOCATE: – writes all records that do not contain the string specified
as the argument
PAD: – fills each record to the specified length with a pad character
Pipeline: – a series of programs that data passes through.
SPLIT: – creates an output record for each blank-delimited word in its
input records
© 2004 IBM Corporation
IBM ^
Glossary
SORT: – orders the input in ascending or descending order
UNPACK: – converts a file from the packed format supported by
COPYFILE and XEDIT to plain records
VAR: – handles the special case where you want to read and write a
single record with the contents of a REXX variable
XLATE: – translates data passing through the pipeline on a character by
character basis
© 2004 IBM Corporation
IBM ^
References
Hartmann, J., L. Kraines, and J. Lynn. CMS Pipelines Tutorial.
GG66-3158-00, February 1990.
IBM. z/VM: CMS Pipelines User’s Guide. SC24-5970-00,
February 2001.
IBM. z/VM: CMS Command and Utility Reference. SC24-601002, May 2002.
© 2004 IBM Corporation