Session Task - Communities

advertisement
INFORMATICA
POWERCENTER 8.1.0
WORKFLOW
Content
 WORKING WITH POWERCENTER 8
WORKFLOW MANAGER
 WORKING WITH POWERCENTER 8
WORKFLOW MONITOR
integration * intelligence * insight
2
Workflow Manager
 Workflow Manager
In the Workflow Manager, we can define a set of instructions to execute tasks, such as
sessions, emails, and shell commands. This set of instructions is called a workflow. It
has the following tools to help we develop a workflow.
• Task Developer
It creates tasks that we want to accomplish in the
workflow.
• Worklet Designer
It creates a worklet in the Worklet Designer. A
worklet is an object that groups a set of tasks.
Worklet is similar to a workflow, but without
scheduling information. We can nest worklets inside
a workflow.
• Workflow Designer
It creates a workflow by connecting tasks with links
in the Workflow Designer. We can also create tasks
in the Workflow Designer as we develop the
workflow.
integration * intelligence * insight
Workflow Manager Windows
3
Workflow Manager Tools
 Workflow Designer
• Maps the execution order and
dependencies of Sessions, Tasks and
Worklets, for the Informatica Server
 Task Developer
• Create Session, Shell Command and
Email tasks
• Tasks created in the Task Developer
are reusable
 Worklet Designer
• Creates objects that represent a set of
tasks
• Worklet objects are reusable
integration * intelligence * insight
4
Workflow Structure
 A Workflow is set of instructions for the Informatica Server to perform data transformation
and load
 Combines the logic of Session Tasks, other types of Tasks and Worklets
 The simplest Workflow is composed of a Start Task, a Link and one other Task
Link
Start
Task
Session
Task
integration * intelligence * insight
5
Working With Links
Use the links to connect each workflow
task. We can specify conditions with links
to create branches in the workflow. The
Workflow Manager does not allow us to
use links to create loops in the workflow.
Each link in the workflow can run only
once and Workflow Manager does not
allow us to create a workflow that
contains a loop.
By using the following procedure we can
link tasks in the Workflow Designer or the
Worklet Designer. To link two tasks.
 In the Tasks toolbar, click the Link
Tasks button.
 In the workspace, click the first task
you want to connect and drag it to the
second task.
 A link appears between the two
tasks.
If we want to link multiple tasks
concurrently, we may not want to connect
each link manually.
integration * intelligence * insight
6
Working With Links
To link tasks concurrently

In the workspace, click the first task you want to
connect.

Ctrl-click all other tasks you want to connect.

Note: Do not use Ctrl+A or Edit > Select All to choose
tasks.

Click Tasks > Link Concurrent.

A link appears between the first task you selected and
each task you added. The first task you selected links
to each task concurrently.
If we have a number of tasks that we want to link
sequentially, we may not wish to connect each link
manually
To link tasks sequentially

In the workspace, click the first task you want to
connect.

Ctrl-click the next task you want to connect. Continue
to add tasks in the order you want them to run.

Click Tasks > Link Sequential.
Links appear in sequential order between the first task
and each subsequent task you added.
integration * intelligence * insight
7
Workflow Tasks
Workflow tasks
Workflow tasks are instructions the Integration Service executes when running a
workflow. Workflow tasks perform functions supplementary to extracting, transforming,
and loading data. Workflow tasks include commands, decisions, timers, and email
notification.
We can create the following types of tasks in the Workflow Manager:
 Assignment
Assigns a value to a workflow variable.
 Command
Specifies a shell command to run during the workflow.
 Control
Stops or aborts the workflow.
 Decision
Specifies a condition to evaluate.
 Email
Sends email during the workflow.
 Event-Raise
Notifies the Event-Wait task that an event has occurred.
 Event-Wait
Waits for an event to occur before executing the next task.
 Session
Runs a mapping we create in the Designer.
 Timer
Waits for a timed event to trigger.
integration * intelligence * insight
8
Task Developer
We can create the following three types of tasks in the Task Developer.
 Command
 Session
 Email
integration * intelligence * insight
9
Creating a Task in the Task Developer
 In the Task Developer, click Tasks >
Create.
 The Create Task dialog box appears.
 Select the task type we want to create,
Command, Session, or Email.
 Enter a name for the task.
 For session tasks, select the mapping we
want to associate with the session.
 Click Create.
 The Task Developer creates the workflow
task.
 Click Done to close the Create Task dialog
box.
integration * intelligence * insight
10
Session Task
Server instructions to runs the logic of ONE specific Mapping
e.g. - source and target data location specifications, memory allocation, optional Mapping
overrides, scheduling, processing and load instructions
Becomes a component of a Workflow (or
Worklet)
If configured in the Task Developer,
Session Task is reusable (optional)
the
Session Task (continued)
Double click on the Session object
Valid Mappings are displayed in the dialog box.
Session Task tabs
General
Properties
Config Object
Mapping
Components
Metadata Extensions
integration * intelligence * insight
11
Session Task
Session Task-General
Session Task-Properties
Session Task-Config Object
integration * intelligence * insight
12
Session Task
Session Task Properties
 Enable Test Load With a test load, the Integration Service
reads and transforms data without writing to targets.
 $Source Connection Value Enter the database
connection we want the Integration Service to use for the
$Source variable. Select a relational or application
database connection. We can also choose a
$DBConnection parameter. Use the $Source variable in
Lookup and Stored Procedure transformations to specify
the database location for the lookup table or stored
procedure.
 $Target Connection Value Enter the database
connection we want the Integration Service to use for the
$Target variable. Select a relational or application
database connection. We can also choose a
$DBConnection parameter. Use the $Target variable in
Lookup and Stored Procedure transformations to specify
the database location for the lookup table or stored
procedure.
 Treat Source Rows As If the mapping for the session
contains an Update Strategy transformation or a Custom
transformation configured to set the update strategy, the
default option is Data Driven.
 Commit Interval In conjunction with the selected commit
interval type, indicates the number of rows. By default, the
Integration Service uses a commit interval of 10,000 rows.
This option is not available for user-defined commit.
integration * intelligence * insight
Session Task-Properties
13
Session Task
Session Task-Config Object
 Line Sequential Buffer Length Affects the way the
Integration Service reads flat files. Increase this setting
from the default of 1024 bytes per line only if source
flat file records are larger than 1024 bytes.
 On Stored Procedure Error Required if the session
uses pre- or post-session stored procedures.
• If we select Stop Session, the Integration Service
stops the session on errors executing a presession or post-session stored procedure.
• If we select Continue Session, the Integration
Service continues the session regardless of errors
executing pre-session or post-session stored
procedures.
 On Pre-Post SQL Error Required if the session uses
pre- or post-session SQL.
• If we select Stop Session, the Integration Service
stops the session errors executing pre-session or
post-session SQL.
• If we select Continue, the Integration Service
continues the session regardless of errors
executing pre-session or post-session SQL.
integration * intelligence * insight
Session Task-Config Object
14
Session Task
Session Task-Mapping
 Connection
Before the Integration Service can access a source or target
database in a session, we must configure the database connections
in the Workflow Manager. When we create or modify a session that
reads from or writes to a relational database, we can select
configured source and target database connections.
When we create a connection, we must have the following
information available:
• Database name
Name for the connection.
• Database type
Type of the source or target database.
• Database user name
Name of a user who has the appropriate database permissions
to read from and write to the database. To use an SQL override
with pushdown optimization, the user
must
also
have
permission to create views on the source or target database.
• Password
Database password (7-bit ASCII only).
• Connect string
Connect string used to communicate with the database.
• Database code page
Code page associated with the database.
integration * intelligence * insight
15
Session Task
 Partition Points Partition points mark the boundaries between threads in a pipeline. The
Integration Service redistributes rows of data at partition points. We can add partition points
to increase the number of transformation threads and increase session performance
• Add Partition Point
Click to add a new partition point. When we add a partition point, the transformation
name appears under the Partition Points node.
• Delete Partition Point
Click to delete the selected partition point. We cannot delete certain partition points.
• Edit Partition Point
Click to edit the selected partition point. This opens the Edit Partition Point dialog box.
$$PushdownConfig Mapping
Depending on the database workload, we may want to use source-side,
target-side, or full pushdown optimization at different times. For example,
we might want to use partial pushdown optimization during the peak
hours of the day, but use full pushdown optimization from midnight until
2 a.m. when activity is low.
 To use different pushdown optimization configurations at different times, use the
$$PushdownConfig mapping parameter. The parameter lets we run the same session using
the different types of pushdown optimization.
 Complete the following steps to configure the mapping parameter:
 Create $$PushdownConfig in the Mapping Designer.
integration * intelligence * insight
16
Session Task
 When we add the $$PushdownConfig mapping parameter in the Mapping Designer, use the
following values:
 Field Value Name $$PushdownConfig Type Parameter Datatype String Precision or Scale
10 Aggregation n/a Initial Value None Description Optional
 When we configure the session, choose $$PushdownConfig for the Pushdown Optimization
attribute.
 Define the parameter in the parameter file.
 Enter one of the following values for $$PushdownConfig in the parameter file:
• None. The Integration Service processes all transformation logic for the session.
• Source. The Integration Service pushes part of the transformation logic to the source
database.
• Source with View. The Integration Service creates a view to represent the SQL
override value, and it runs an SQL statement against this view to push part of the
transformation logic to the source database.
• Target. The Integration Service pushes part of the transformation logic to the target
database.
• Full. The Integration Service pushes all transformation logic to the database.
• Full with View. The Integration Service creates a view to represent the SQL override
value, and it runs an SQL statement against this view to push part of the
transformation logic to the source database. The Integration Service pushes any
remaining transformation logic to the target database.
integration * intelligence * insight
17
Session Task
Session Task-Transformations
Allows
overrides
of
some
transformation properties Does not
change the properties in the Mapping
Session Task-Components
 Pre-Session Command
Shell commands that the Integration
Service performs at the beginning of
a session.
 Post-Session Success Command
Shell commands that the Integration
Service performs after the session
completes successfully
 Post-Session Failure Command
Shell commands that the Integration
Service performs after the session if
the session fails.
integration * intelligence * insight
18
Event-Task
Event-Raise task
The Event-Raise task represents the location
of a user-defined event. A user-defined event
is the sequence of tasks in the branch from
the Start task to the Event-Raise task. When
the Integration Service runs the Event-Raise
task, the Event-Raise task triggers the userdefined event.
To use an Event-Raise task
 In the Workflow Designer workspace,
create an Event-Raise task and place it in
the workflow to represent the userdefined event we want to trigger.
 A user-defined event is the sequence of
tasks in the branch from the Start task to
the Event-Raise task.
 Double-click the Event-Raise task to open
it.
 Click the Open button in the Value field
on the Properties tab to open the Events
Browser for user-defined events.
 Choose an event in the Events Browser.
 Click OK twice to return to the workspace.
integration * intelligence * insight
19
Event-Task
Event-Wait task
The Event-Wait task waits for a predefined
event or a user-defined event. A predefined
event is a file-watch event. When we use the
Event-Wait task to wait for a predefined
event, we specify an indicator file for the
Integration Service to watch. The Integration
Service waits for the indicator file to appear.
Once the indicator file appears, the
Integration Service continues running tasks
after the Event-Wait task.
To wait for a user-defined event
 In the workflow, create an Event-Wait
task and double-click the Event-Wait
task to open it.
 In the Events tab of the task, select
User-Defined.
 Click the Event button to open the
Events Browser dialog box.
 Select a user-defined event for the
Integration Service to wait.
 Click OK twice.
integration * intelligence * insight
20
E-Mail Task
We can send email during a workflow using the Email task on the Workflow Manager. We
can create reusable Email tasks in the Task Developer for any type of email. Or, we can
create non-reusable Email tasks in the Workflow and Worklet Designer.
Use Email tasks in any of the following locations
 Session properties
We can configure the session to send email when the session completes or fails.
 Workflow properties
We can configure the workflow to send email when the workflow is interrupted.
 Workflows or worklets
We can include an Email task anywhere in the workflow or worklet to send email
based on a condition we define.
integration * intelligence * insight
21
E-Mail Task
We can create Email tasks in the Task Developer, Worklet
Designer, and Workflow Designer.
To create an Email task in the Task Developer:
 In the Task Developer, click Tasks > Create. The Create
Task dialog box appears.
 Select an Email task and enter a name for the task. Click
Create.
The Workflow Manager
creates an Email task in the workspace.
 Click Done.
 Double-click the Email task in the workspace. The Edit
Tasks dialog box appears.
 Click Rename to enter a name for the task.
 Enter a description for the task in the Description field.
 Click the Properties tab.
 Enter the fully qualified email address of the mail recipient
in the Email User Name field.
 Enter the subject of the email in the Email Subject field. Or,
we can leave this field blank.
 Click the Open button in the Email Text field to open the
Email Editor.
 Enter the text of the email message in the Email Editor. We
can leave the Email Text field blank. Note: We can
incorporate format tags and email variables in a postsession email. However, we cannot add them to an Email
task outside the context of a session.
 Click OK twice to save the changes.
integration * intelligence * insight
22
Worklet


A worklet is an object that represents a set of tasks that we
create in the Worklet Designer. Create a worklet when we
want to reuse a set of workflow logic in more than one
workflow.
To run a worklet, include the worklet in a workflow. The
workflow that contains the worklet is called the parent
workflow. When the Integration Service runs a worklet, it
expands the worklet to run tasks and evaluate links within the
worklet. It writes information about worklet execution in the
workflow log.
To create a reusable worklet
 In the Worklet Designer, click Worklet > Create. The
Create Worklet dialog box appears.
 Enter a name for the worklet.
 Click OK.
 The Worklet Designer creates a Start task in the worklet.
To create a non-reusable worklet:
 In the Workflow Designer, open a workflow.
 Click Tasks > Create.
 For the Task type, select Worklet.
 Enter a name for the task.
 Click Create. The Workflow Designer creates the worklet
and adds it to the workspace.
 Click Done.
integration * intelligence * insight
23
Workflow Scheduler
Workflow Scheduler Objects
Setup
reusable
schedules
to
associate with multiple WorkflowsUsed in Workflows and Session Tasks
integration * intelligence * insight
24
Server Connections
Configure Server data access connections- Used in Session Tasks
integration * intelligence * insight
25
Relational Connections (Native)
Creating a relational (database)
connection
 Instructions to the Server to
locate relational tables
 Used in Session Tasks
Relational Connections Properties
Define native relational (database)
connection
integration * intelligence * insight
26
FTP Connection
Creating an FTP connection
 Instructions to the Server to ftp flat files
 Used in Session Tasks
integration * intelligence * insight
27
Workflows Design
Sample Workflow
Developing Workflow
integration * intelligence * insight
28
Workflows Design
Building Workflow Components
 Add Sessions and other Tasks to
the Workflow
 Connect all Workflow components
with Links
 Save the Workflow
 Start the Workflow
NOTE: Sessions in a workflow can
be executed independently
integration * intelligence * insight
29
Workflow Monitor
 Workflow Monitor
We can monitor workflows and tasks in the Workflow Monitor. View details about a
workflow or task in Gantt Chart view or Task view. We can run, stop, abort, and
resume workflows from the Workflow Monitor. We can view sessions and workflow log
events in the Workflow Monitor Log Viewer.
The Workflow Monitor displays workflows that have run at least once. The Workflow
Monitor continuously receives information from the Integration Service and Repository
Service. It also fetches information from the repository to display historic information.
The Workflow Monitor consists of the following windows.
• Navigator window
It displays monitored repositories, servers, and repositories
objects.
• Output window.
It displays messages from the Integration Service and
Repository Service.
• Time window
It displays progress of workflow runs.
• Task view
It displays details about workflow runs in a report format.
• Gantt Chart view
It displays details about workflow runs in chronological
format.
integration * intelligence * insight
Workflow Monitor Windows
30
Workflow Monitor
The Workflow Monitor displays Workflows that have been run at least once
we can monitor a Server in two modes: online or offline
Online mode
Workflow Monitor continuously receives information from the
Informatica Server and the Repository Server
Offline mode
Workflow Monitor displays historic information about past Workflow
runs by fetching information from the Repository
Monitoring Workflow
Perform the following operations in the Workflow Monitor
Restart -- restart a Task, Workflow or Worklet
Stop -- stop a Task, Workflow, or Worklet
Abort -- abort a Task, Workflow, or Worklet
Resume -- resume a suspended Workflow after a failed task is corrected
 View Session and Workflow logs
 Abort has a 60 second timeout
If the Server has not completed processing and committing data
during the timeout period, the threads and processes associated with
the Session are killed
integration * intelligence * insight
31
Workflow Monitor
Monitoring Workflow
Monitoring Window Filtering
Task View provides filtering
Truncating Monitor Logs
integration * intelligence * insight
32
The End
integration * intelligence * insight
33
Download