0
Home
Datastage Related
More Next Blog»
Datastage Training
Create Blog Sign In
Big Data
Unix
Database
Interview Related
Certifications
Discussion Forum
Feedback
Many thanks for visiting my Blog..!!Please share this blog using below share buttons and leave your Comments/Feedback/Appreciations on SEARCH YOUR PROBLEMS SOLUTION IN THIS BLOG
Search
Like
0
Tweet
0
Share
0
S
0
10 Share This Blog..!!
TRANSLATE THIS BLOG
Datastage Transformer Stage Looping concept
LOOPING CONCEPT IN DATASTAGE 8.5 OR 8.7 TRANSFORMER STAGE...!
VISITOR'S VIEW COUNT
8 0 6 2 5 1
Select Language ​
▼
Log In
You may like this link : Click here to know Transformer Looping Functions For Pivoting
Loop example: Aggregating Data in Transformer
You can use the Transformer stage to add aggregated information to output rows.
Aggregation operations make use of a cache that stores input rows. You can monitor the number of entries in the
cache by setting a threshold level in the Loop Variable tab of the Stage Properties window. If the threshold is
reached when the job runs, a warning is issued into the log, and the job continues to run.
ABOUT ME : CLICK ON G+ BUTTON TO FOLLOW ME
Devendra Kumar Yadav
Follow
181 followers
INPUT ROW GROUP AGGREGATION INCLUDED WITH INPUT ROW DATA
You can save input rows to a cache area, so that you can process this data in a loop.
For example, you have input data that has a column holding a price value. You want to add a column to the
output rows. The new column indicates what percentage the price value is of the total value for prices in all
rows in that group. The value for the new Percentage column is calculated by the following expression.
(price * 100)/sum of all prices in group
In the example, the data is sorted and is grouped on the value in Col1.
Col1 Col2 Price
OTHER DATASTAGE QUESTIONS SOLUTIONS
1000 abc
100.00
1000 def
20.00
▼ 2013 (48)
1000 ghi
60.00
► Dec (8)
1000 jkl
20.00
2000 zyx
120.00
► 2014 (34)
► Nov (15)
► Oct (12)
▼ Jan (13)
2000 wvu 110.00
IBM DataStage 8.5 New Features
2000 tsr
Use of Stages in Datastage 8.5 or 8x Series
170.00
New Debug feature in DataStage 8.5
The percentage for each row in the group where Col1 = 1000 is calculated by the following expression.
DS Parallel Processing & Partition Techniques
Datastage Transformer Stage Looping concept
(price * 100)/200
Data Modeling Concept in Datawarehouse
DW&BI Concepts Interview Questions and Answers
Datastage 8x Engine/Tiers Start and Stop Process
The percentage for each row in the group where Col1 = 2000 is calculated by the following expression.
(price * 100)/400
RT_SCTEMP Error : Not able to run Datastage Jobs
Parameters Using Parameter/Value Set/Value File
Delete DataStage jobs from the command line
MY MOST POPULAR FREQUENTLY ACCESSED POSTS
Datastage Slowly Changing Dimensions
Datastage 8.5, 8.7 and 9.1 Differences
Disk Space Estimation
The output is shown in the following table.
Col1 Col2 Price
Percentage
Datastage Transformer Stage Looping concept
► 2012 (4)
Data partitioning & collecting methods Examples
1000 abc
100.00 50.00
IBM Datastage 9.1 Newly Added features
1000 def
20.00
10.00
DATASTAGE Performance Tuning Tips V1.1
1000 ghi
60.00
30.00
Surrogate Key Generator Implementation
1000 jkl
20.00
10.00
2000 zyx
120.00 30.00
Parameters Using Parameter/Value Set/Value File
Use of Stages in Datastage 8.5 or 8x Series
2000 wvu 110.00 27.50
2000 tsr
170.00 42.50
Datastage Interview Questions and Answers V1.4
Datastage Scenario Based Question/Answer :1
This scenario uses key break facilities that are available on the Transformer stage. You can use these facilities
to detect when the value of an input column changes, and so group rows as you process them.
LIST OF VISITOR'S COUNTRIES
This scenario is implemented by storing the grouped rows in an input row cache and processing them when the
value in a key column changes. In the example, the grouped rows are processed when the value in the column
named Col1 changes from 1000 to 2000. Two functions, SaveInputRecord() and GetSavedInputRecord(), are
used to add input rows to the cache and retrieve them. SaveInputRecord() is called when a stage variable is
evaluated, and returns the count of rows in the cache (starting at 1 when the first row is added).
GetSavedInputRecord() is called when a loop variable is evaluated.
To implement this scenario in the Transformer stage, make the following settings:
Stage variable
Define the following stage variables:
NumSavedRows
SaveInputRecord()
IsBreak
LastRowInGroup(inlink.Col1)
TotalPrice
IF IsBreak THEN SummingPrice + inlink.Price ELSE 0
SummingPrice
IF IsBreak THEN 0 ELSE SummingPrice + inlink.Price
NumRows
IF IsBreak THEN NumSavedRows ELSE 0
Loop condition
Enter the following expression as the loop condition:
@ITERATION <= NumRows
The loop continues to iterate for the count specified in the NumRows variable.
Loop variables
Define the following loop variable:
SavedRowIndex
GetSavedInputRecord()
Output link metadata and derivations
Define the output link columns and their derivations:
Col1 ‐ inlink.Col1
Col2 ‐ inlink.Col2
Price ‐ inlink.Price
RECENTLY VISITED USER'S LOCATION
Live Traffic Feed
A visitor from Mumbai, Maharashtra left
"DEV'S DATASTAGE
TUTORIAL,GUIDES,TRAINING AND
ONLINE HELP 4 U. UNIX, ETL, DATABASE
RELATED SOLUTIONS: Datastage Related
A visitor from Hyderabad, Andhra Pradesh
Problems and Solutions" via datastageinfoguid
viewed "DEV'S DATASTAGE
e.blogspot.com 1 min ago
TUTORIAL,GUIDES,TRAINING AND
ONLINE HELP 4 U. UNIX, ETL, DATABASE
RELATED SOLUTIONS: Datastage Jobs Best
A visitor from Hyderabad, Andhra Pradesh left
Practices for Tuning" 2 mins ago
"DEV'S DATASTAGE
TUTORIAL,GUIDES,TRAINING AND
ONLINE HELP 4 U. UNIX, ETL, DATABASE
RELATED SOLUTIONS: Search results for
A visitor from Bangalore, Karnataka viewed
funnel stage" via datastageinfoguid
"DEV'S DATASTAGE
e.blogspot.com.au 2 mins ago
TUTORIAL,GUIDES,TRAINING AND
ONLINE HELP 4 U. UNIX, ETL, DATABASE
RELATED SOLUTIONS: Sequential File Best
A visitor from Mumbai, Maharashtra left
Performance Tips/Settings" 7 mins ago
"DEV'S DATASTAGE
TUTORIAL,GUIDES,TRAINING AND
ONLINE HELP 4 U. UNIX, ETL, DATABASE
RELATED SOLUTIONS: Datastage Related
A visitor from Mumbai, Maharashtra viewed
Problems and Solutions" via datastageinfoguid
"DEV'S DATASTAGE
e.blogspot.com 13 mins ago
TUTORIAL,GUIDES,TRAINING AND
ONLINE HELP 4 U. UNIX, ETL, DATABASE
RELATED SOLUTIONS: Conductor
A visitor from Hyderabad, Andhra Pradesh
Node,Section Leaders and Players" 24 mins ago
viewed "DEV'S DATASTAGE
TUTORIAL,GUIDES,TRAINING AND
ONLINE HELP 4 U. UNIX, ETL, DATABASE
RELATED SOLUTIONS: Datastage Jobs Best
A visitor from Grand Rapids, Michigan viewed
Practices for Tuning" 26 mins ago
"DEV'S DATASTAGE
TUTORIAL,GUIDES,TRAINING AND
ONLINE HELP 4 U. UNIX, ETL, DATABASE
RELATED SOLUTIONS: IBM Datastage 9.1
A visitor from India viewed "DEV'S
Newly Added features" 30 mins ago
DATASTAGE
TUTORIAL,GUIDES,TRAINING AND
Percentage ‐ (inlink.Price * 100)/TotalPrice
SaveInputRecord() is called in the first Stage Variable (NumSavedRows). SaveInputRecord() saves the current
input row in the cache, and returns the count of records currently in the cache. Each input row in a group is
saved until the break value is reached. At the last value in the group, NumRows is set to the number of rows
stored in the input cache. The Loop Condition then loops round the number of times specified by NumRows,
calling GetSavedInputRecord() each time to make the next saved input row current before re‐processing each
input row to create each output row. The usage of the inlink columns in the output link refers to their values
in the currently retrieved input row, so will change on each output loop.
TUTORIAL,GUIDES,TRAINING AND
ONLINE HELP 4 U. UNIX, ETL, DATABASE
RELATED SOLUTIONS: Datastage 8.5, 8.7
A visitor from Bangalore, Karnataka viewed
and 9.1 Differences" 33 mins ago
"DEV'S DATASTAGE
TUTORIAL,GUIDES,TRAINING AND
ONLINE HELP 4 U. UNIX, ETL, DATABASE
RELATED SOLUTIONS: DATASTAGE
Real­time view · Menu
CACHING SELECTED INPUT ROWS
You can call the SaveInputRecord() within an expression, so that input rows are only saved in the cache when
the expression evaluates as true.
For example, you can implement the scenario described, but save only input rows where the price column is
not 0. The settings are as follows:
Stage variable
Define the following stage variables:
IgnoreRow
IF (inlink.Price = 0) THEN 1 ELSE 0
NumSavedRows
IF IgnoreRecord THEN SavedRowSum ELSE SaveInputRecord()
IsBreak
LastRowInGroup(inlink.Col1)
SavedRowSum
IF IsBreak THEN 0 ELSE NumSavedRows
TotalPrice
IF IsBreak THEN SummingPrice + inlink.Price ELSE 0
SummingPrice
IF IsBreak THEN 0 ELSE SummingPrice + inlink.Price
NumRows
IF IsBreak THEN NumSavedRows ELSE 0
Loop condition
Enter the following expression as the loop condition:
@ITERATION <= NumRows
Loop variables
Define the following loop variable:
DAILY HOROSCOPES
Daily Horoscopes
♈ Aries
♎ Libra
3/21­4/19
9/23­10/22
♉ Taurus
♏Scorpio
4/20­5/20
10/23­11/21
♊
Gemini
5/21­6/21
♐ Sagittarius
11/22­12/21
♋ Cancer
♑ Capricorn
6/22­7/22
12/22­1/19
♌ Leo
♒ Aquarius
7/23­8/22
1/20­2/18
♍Virgo
♓
8/23­9/22
Pisces
2/19­3/20
Made by albinoblacksheep.com Terms
MY BLOG POSTS
DEV'S DATAWAREHOUSING HELP GUIDE
DataStage Parallel jobs fail with failure code 11
SavedRowIndex
GetSavedInputRecord()
Output link metadata and derivations
Define the output link columns and their derivations:
Col1 ‐ inlink.Col1
Col2 ‐ inlink.Col2
Price ‐ inlink.Price
Percentage ‐ (inlink.Price * 100)/TotalPrice
This example produces output similar to the previous example, but the aggregation does not include Price
values of 0, and no output rows with a Price value of 0 are produced.
OUTPUTTING ADDITIONAL GENERATED ROWS
This example is based on the first example, but, in this case, you want to identify any input row where the
Price is greater than or equal to 100. If an input row has a Price greater than or equal to 100, then a 25%
discount is applied to the Price and a new additional output row is generated. The Col1 value in the new row
has 1 added to it to indicate an extra discount entry. The original input row is still output as normal.
Therefore any input row with a Price of greater than or equal to 100 will produce two output rows, one with
the discounted price and one without.
The input data is as shown in the following table:
Col1 Col2 Price
1000 abc
100.00
1000 def
20.00
1000 ghi
60.00
1000 jkl
20.00
2000 zyx
120.00
2000 wvu 110.00
2000 tsr
170.00
The required table is shown in the following table:
Col1 Col2 Price
Percentage
1000 abc
100.00 50.00
1001 abc
75.00
50.00
1000 def
20.00
10.00
1000 ghi
60.00
30.00
1000 jkl
20.00
10.00
2000 zyx
120.00 30.00
2001 zyx
90.00
30.00
2000 wvu 110.00 27.50
2001 wvu 82.50
27.50
2000 tsr
170.00 42.50
2001 tsr
127.50 42.50
To implement this scenario in the Transformer stage, make the following settings:
Stage variable
Define the following stage variables:
NumSavedRowInt
SaveInputRecord()
AddRow
IF (inlink.Price >= 100) THEN 1 ELSE 0
NumSavedRows
IF AddRow THEN SaveInputRecord() ELSE NumSavedRowInt
IsBreak
LastRowInGroup(inlink.Col1)
TotalPrice
IF IsBreak THEN SummingPrice + inlink.Price ELSE 0
SummingPrice
IF IsBreak THEN 0 ELSE SummingPrice + inlink.Price
NumRows
IF IsBreak THEN NumSavedRows ELSE 0
Loop condition
Enter the following expression as the loop condition:
@ITERATION <= NumRows
The loop continues to iterate for the count specified in the NumRows variable.
Loop variables
Define the following loop variables:
SavedRowIndex
GetSavedInputRecord()
AddedRow
LastAddedRow
LastAddedRow
IF (inlink.Price < 100) THEN 0 ELSE IF (AddedRow = 0) THEN 1 ELSE 0
Output link metadata and derivations
Define the output link columns and their derivations:
Col1 ‐ IF (inlink.Price < 100) THEN inlink.Col1 ELSE IF (AddedRow = 0) THEN
inlink.Col1 ELSE inlink.Col1 + 1
Col2 ‐ inlink.Col2
Price ‐ IF (inlink.Price < 100) THEN inlink.Price ELSE IF (AddedRow = 0) THEN
inlink.Price ELSE inlink.Price * 0.75
Percentage ‐ (inlink.Price * 100)/TotalPrice
SaveInputRecord is called either once or twice depending on the value of Price. When SaveInputRecord is
called twice, in addition to the normal aggregation, it produces the extra output record with the recalculated
Price value. The Loop variable AddedRow is used to evaluate the output column values differently for each of
the duplicate input rows.
RUNTIME ERRORS
The number of calls to SaveInputRecord() and GetSavedInputRecord() must match for each loop. You can call
SaveInputRecord() multiple times to add to the cache, but once you call GetSavedInputRecord(), then you must
call it enough times to empty the input cache before you can call SaveInputRecord() again. The examples
described can generate runtime errors in the following circumstances by not observing this rule:
If your Transformer stage calls GetSavedInputRecord before SaveInputRecord, then a fatal error
similar to the following example is reported in the job log:
APT_CombinedOperatorController,0: Fatal Error: get_record() called on
record 1 but only 0 records saved by save_record()
If your Transformer stage calls GetSavedInputRecord more times than SaveInputRecord is called,
then a fatal error similar to the following example is reported in the job log:
APT_CombinedOperatorController,0: Fatal Error: get_record() called on
record 3 but only 2 records saved by save_record()
If your Transformer stage calls SaveInputRecord but does not call GetSavedInputRecord, then a
fatal error similar to the following example is reported in the job log:
APT_CombinedOperatorController,0: Fatal Error: save_record() called on
record 3, but only 0 records retrieved by get_record()
If your Transformer stage does not call GetSavedInputRecord as many times as SaveInputRecord,
then a fatal error similar to the following example is reported in the job log:
APT_CombinedOperatorController,0: Fatal Error: save_record() called on
record 3, but only 2 records retrieved by get_record()
Recommend this on Google
Like (1)
Reactions: Useful (1)
Dislike (0)
WHAT'S THIS?
AROUND THE WEB
TheFix.com
Stack
Lose Belly Fat With 6
Stand­Up Exercises
Alcohol and Energy
Drinks A Dangerous
Combo, Study Says
ALSO ON DK.DSXCHANGE ­ DWBI TUTORIAL
DEV'S DATASTAGE TUTORIAL,GUIDES,TRAINING
AND ONLINE HELP 4 U. UNIX, ETL, DATABASE
RELATED SOLUTIONS: Datastage Online Training ­
DEV'S DATASTAGE TUTORIAL,GUIDES,TRAINING
Web Sessions at Home 5 comments
AND ONLINE HELP 4 U. UNIX, ETL, DATABASE
RELATED SOLUTIONS: Data Modeling Concept in
Datawarehouse 1 comment
3 Comments
DK.DSXchange ­ DWBI Tutorial
Sort by Newest
Join the discussion…
DEV'S DATASTAGE TUTORIAL,GUIDES,TRAINING
AND ONLINE HELP 4 U. UNIX, ETL, DATABASE
RELATED SOLUTIONS: Datastage Jobs Best
DEV'S DATASTAGE TUTORIAL,GUIDES,TRAINING
Practices for Tuning 1 comment
AND ONLINE HELP 4 U. UNIX, ETL, DATABASE
RELATED SOLUTIONS: Your Comments Feedback
and Appreciations 13 comments

Login
Share ⤤ Favorite ★
Join the discussion…
kiran • 6 months ago
what is unicode in datastage and what is the use of it
△ ▽
• Reply • Share › Devendra Kumar Yadav Mod > kiran •
5 months ago
Hi Kiran,
While doing the View Data shows you "un­American" characters. Use a viewer that can display the Euro symbol, accented characters, etc. properly.
We are using "UNICODE" to handle LATIN source data.
Apply stringtoUstring(columnvalue,'UTF­8') and make column datatype as nvarchar View data wont give the correct result ,please check the final out put (file or table)
△ ▽
• Reply • Share › kiran > Devendra Kumar Yadav • 4 months ago
thanks kiran nice info about unicode
△ ▽
✉
Subscribe
d
• Reply • Share › Add Disqus to your site
Newer Post

Privacy
Home
Older Post
Subscribe to: Post Comments (Atom)
DISCLAIMER
All content provided on this http://datastageinfoguide.blogspot.in blog is for informational purposes only. The owner of this blog makes no representations as to the accuracy or completeness
of any information on this site or found by following any link on this site.The owner of http://datastageinfoguide.blogspot.in will not be liable for any errors or omissions in this information nor for
the availability of this information. The owner will not be liable for any losses, injuries, or damages from the display or use of this information.
.
FIND YOUR PROBLEMS SOLUTION HERE ..!
Search