Informatica
Create a crash dump using “Debug Diag”
HOW-TO | CIF CONSULT | Redouane BELBAHRI
Agenda
1
Terminology
Terminology

ISP – Informatica Services Platform
– 9.x - The ISP is the administrative framework for Informatica services. It allows monitoring, start/stop,
and failover of services

Model Repository Service (MRS)
– Model Reposiory Service (MRS) manages the model Repository which contains the metadata for Data
Services.

Data Integration Service (DIS)
– The Data Integration Service (DIS) is a container for all data integration functionalities. Plug-ins for the
DIS provide different data integration functionalities.
Terminology

Data Transformation Manager (DTM)
– DTM Interprets and executes mappings.
– It is re-engineered to be lightweight and embeddable. It allows repository-less execution: In previous
versions of PowerCenter, DTM connects to the repository to fetch the plan. However, now the
mapping can be run without reading from the repository, like mapping stored to disk.
– DTM enables concurrent execution semantics: multiple DTM instances can coexist in the same
process.
– It allows different execution modes: single threaded and pipelined execution modes.

Profiling Service
– The Profiling Service is a component integrated into Informatica Data Services. It includes the Profiling
Plug-in and Profiling Warehouse.
Terminology

Mapping Service Module
– The Mapping Service Module is the component service in the Data Integration Service that manages
requests to preview target data and run mappings.

SQL Service Module
– The SQL Service Module is the component service in the Data Integration Service that manages SQL
queries sent to an SQL data service from a third party client tool.
– When the Data Integration Service receives an SQL request from a third party client tool, the SQL
Service Module starts a DTM instance to run the SQL query against the virtual tables in the SQL data
service.
Terminology – Nodes and Domains
When you install and run the Informatica services, the installation is known as a node. The node becomes
part of an Informatica domain. A domain is a grouping of one or more nodes.

Domain Basics
– The most basic description of an Informatica domain is this: There is a node, and there is a domain
database. The node is where stuff runs. The database is where the node keeps stuff.

The Node
– The node is a logical representation of the Informatica installation on a server. It is run by Apache
Tomcat.
– Two Kinds of Nodes:

Worker Nodes: A worker node runs a Service Manager process, and it can run application services.

Gateway Nodes: A gateway node can run application services, and it runs its own Service Manager
Process. When you start the master gateway node, the management services log their startup progress to
node.log and catalina.out. You can find both of these log files in INFA_HOME/tomcat/logs.
 Master Gateway Node: Any gateway node can also be a master gateway node.
Terminology – Nodes and Domains

Node Communication
– The master gateway node is the one node in the domain that communicates with the domain
database, and it does so through JDBC. It periodically performs a heartbeat check against the domain
database.
Following is the architectural diagram of Informatica 9.1

Green components are
added in Informatica 9.x

Blue components are from
existing PowerCenter.
Informatica 9.X Service-Oriented Architecture
Agenda
2
Log management
Catalina.out & ispLogs.log

Catalina.out contains
– Core services initial startup events
– Tomcat’s ERROR + event until 9.1HF3
– ISP process’s stdout & stderr

ispLogs.log (introduced in 9.1HF3)
– Tomcat’s ERROR + events in 9.1HF3
– Tomcat’s INFO + event until 9.1HF4+
– Not generated on windows

Logging level modified through isplogging.properties in ./tomcat/bin location
Node.log

Contains
– Master Election & Re-election
– Some errors during inter-node communication

Error severity Level controlled in AdminConsole from Node’s properties
– WARNING by default. Change it to INFO for information of master re-election
Exceptions.log

Logs exceptions faced in the ISP process
– Any exception message seen in Node.log is typically logged with stack trace to exceptions.log
ServiceFramework.log (until 9.1.x)

Tomcat/bin
– JSF logs to this file.
– All Mercury service processes, ISP process, Admin Console process write to the same
– Critical for domain/node issues as it contains most critical errors

Isp/bin
– Infacmd commands write to this file (infacmd generate this file in whichever directory you execute the
command)
– Contains errors related to infacmd to node/service communication.
Domain Logs (exported from Adminconsole)

Contains some more log files by each node and domain
– On missing heartbeat from node
– When a service process terminated unexpectedly
– When a service process wrote to stdout/stderr before crashing (stack trace)
AdminConsole’s log files


Tomcat/temp/_AdminConsole/logs/*
Services/AdministratorConsole/Administrator.log
Application Service Logs



./tomcat/logs/MRS/<MRS_NAME>.log
./tomcat/logs/as/analyst.log
./tomcat/bin/disLogs/<DIS_NAME>.log
Agenda
3
Windows diagnostic tools
Debug Diagnostic Tool

The Debug Diagnostic Tool (DebugDiag) is designed to assist in troubleshooting issues
such as hangs, slow performance, memory leaks or fragmentation, and crashes in any
user-mode process. The tool includes additional debugging scripts focused on
Internet Information Services (IIS) applications, web data access components, COM+
and related Microsoft technologies.

Supported Operating System
– Windows 7
– Windows 2003-Windows XP • Windows 2008-Windows Vista • Windows 2008 R2- Windows7
Installation steps

The Installation is strait forwards.
Create a stack trace from a hanging session or server
process

1.
2.
When session or a server process hangs on Windows, use one of the following methods
to obtain a stack trace and send it to Informatica technical support:
Determine the process ID (PID) of the hanging process
Dump the process stack
Agenda
4
Determine the process ID (PID) of the hanging process
Introduction to Process Explorer



Process Explorer is a lot like Task Manager, which is the program you access via the
following keys on your keyboard: "CTRL+ALT+DELETE". Task Manager is used to
display process information such as the process name, memory usage and other
application information. However, the information that Task Manager displays is
rather limited compared to what Process Explorer can show you. With Process
Explorer one can see the entire process tree for a particular applications, which
consists of all other processes that were started by the original process, or parent, in
the tree.
Process Explorer has many different uses such as examining what process are active
and what processes are making connections to outside computers. In this guide I will
demonstrate the kinds of information one can gleam from using Process Explorer as
opposed to using task manager.
The tool can be downloaded for free from: http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx
Determine the process ID (PID)

Open Process Explorer
– The top window always shows a list of the currently active processes
– Highlight the process. The Services tab of the process properties dialog shows the list of services
running within a process.
Names of the PowerCenter, Data Quality and Data
Explorer processes on Windows

When PowerCenter node is started up, you should see the below Tomcat Processes:
– infasvcs.exe
– Java.exe
– If you start Informatica service from command line, then you may not see infasvcs.exe in the list.

pmserver.exe
– This is the process for Integration Service

pmrepagent.exe
– This is the process for Repository Agent, will show when Repository Service has started up.

pmdtm.exe
– This is the session process started by the Integration Service.
Names of the PowerCenter, Data Quality and Data
Explorer processes on Windows

Data Explorer
If you are using Data Explorer you will see the following processes:
– Idelistener.exe
– Ide_server.exe

Data Quality
When Data Quality Server is running, you should see the following process from Task Manager:
– ImplRepo_Service
– Naming_Service
– ImR_Activator
Agenda
5
Dump the process stack
Using Debug Diag


Open Debug Diag and add a crash rule.
Select Crash
Select the process ID


Select Target Type as A specific process
Select the process ID (Where PID is the process id obtained in the first step) and
check This process instance only

Configure the following next parameters:
Create a Rule



Add an event for Child Process Exit and set the Action Type to Full Userdump
9.Save and close.
Click Next > Finish
Achievements

Following is achieved after performing the steps above:
– The Child process in this context is the process being debugged (pmserver), not its children.
– When the debugged process exits, debug diag collects the full user dump.
– Run crash analysis on the dump and collect the stack of the process during the time of its exit.
– Core dump analysis can also be performed on the full user dump if required (space consuming)
Crash/Hang Analyzers

If a crash occurs, open the Debug Diag and do as follows:
– Click on Advanced Analysis
– Click on Add Data Files
– Browse to the location for the datafiles, and click Open

Once you have selected the datafile for the dump, click Start Analysis and send the
report to Global Customer Support (GCS).
Crash/Hang Analyzers


You can also right click the dump created dump file
Start Analysis and send the report to Global Customer Support (GCS).
Using dump file generated by Dr Watson

Dr. Watson ( drwtsn32.exe ) is a default component of the Windows operating system.
Crash/Hang Analyzers (from Dr Watson)

Once you have selected the datafile for th dump, click Start Analysis and send the
report to Global Customer Support (GCS).
java.lang.OutOfMemoryError: Java heap space

When a Java program starts, the Java Virtual Machine (JVM) gets some memory from the operating
system. The JVM uses this memory for all its needs and part of this memory is called Java heap
memory. When a Java application needs memory, the Java application gets it from Java Heap Space
and when the object ceases to exist (garbage collected), the memory returns to Java Heap Space.

When JVM starts, JVM heap space is the initial size of Heap specified by the -Xms parameter. As the
application progresses, objects are created and JVM expands Heap space in Java to accommodate new
objects. JVM also runs garbage collector to reclaim memory back from objects that cease to exist. JVM
expands Heap in Java close to the Maximum Heap Size specified by -Xmx and if there is no memory left
for creating new objects in Java heap, JVM results in "java.lang.outofmemoryerror" and the application
ends.

The default Heap size in Java is 128 MB. You can increase the size of Java Heap space based on what
the Java application needs. To increase the Java heap space, use the parameters Xms (denotes starting
size of Heap) and Xmx (denotes maximum size of Heap in Java).
ANNEX
Many Thanks for you time today!