24 June 2014 ISACA Professional Guidance Webinar
IBM-DuPont COBIT PAM Assessment
Attendee Questions & Answers
On 26 June 2014, ISACA presented a 60-minute webinar on IBM-DuPont COBIT PAM Assessment. It will
be available on archive until June 2015; please visit
https://webinars.isaca.org/webinar.aspx?webinarid=110293 to access.
Our Speakers John W. Lainhart, Dr. Zhiwei Fu and James Aliquo have been able to respond to the many
of questions that were asked by attendees. Below is a recap:
#
1
2
QUESTION
"Roughly, how many person hours did
the assessment take for the full team
(excluding responder activities).
Could you break out the rough %
allocated to each major phase of the
review?
How do you track results associated with
your improvements and how do they
track to your financial health?
"Hi Just to clarify on the Class 3.
COBIT assessor says Class 3 is not a selfassessment as well as it required
evidence.
3
Is it conflicting?
4
5
6
Your Mock-Up report addresses
"recommended priority". What about
"sequence of implementation”; and can
you explain the color codes, please?
In sheet 19, Jim presents a mock-up of
the PAM result scores. Can he give me a
hint what the numbers in red, yellow
and green block respond / refer to?
Do you know if there are many not-forprofit entities implementing COBIT
PAM?
ANSWER
The man hours of the assessment, excluding responder
activities, mainly depends on business requirements for
the assessment, number of in-scope governance and
management processes, their current states, and type of
assessment, among others.
Too early to tell but again the idea is approach this from
a win-win perspective. The idea being benefits realized
using the most effective and least cost solution residing
with the Third Party or DuPont. The PAM provides a
pathway overtime to realize the benefits of both.
COBIT assessor states that Class 3 is not a selfassessment. However, given no independence is not
required for Class 3, it is considered for a status check or
progress monitoring purpose, therefore not an
independent process capability assessment. A key
determining factor is essentially who performs the
assessment, i.e. self or independent entities, while what
is the basis of assessment, i.e. factual evidence or
subjective opinions is also important to consider.
Red are the areas to fix first or areas requiring significant
improvement. Yellow is medium or areas that could have
a significant impact if not fixed over time. Green is
specific to areas not requiring immediate attention. The
sequence of implementation is very much impacted by
the reported service levels. Service areas where users
are reporting problem come first. The focus is always to
maintain the current level of service or better.
See number 4 above.
COBIT 5 and the PAM were designed for use by not-forprofit entities implementing to assess their process
capability assessment and continuous improvement. We
know of several not-for-profits who are using COBIT
1
24 June 2014 ISACA Professional Guidance Webinar
IBM-DuPont COBIT PAM Assessment
Attendee Questions & Answers
Does your internal audit use your PAM
rating as part of their audit scope?
7
8
How does COBIT/ PAM apply to
measuring the effectiveness of an
Information Security Program? Any
specific COBIT documentation you can
refer me to?
Is PAM a self-assessment activity?
9
10
11
12
13
It would be interesting to have an
indication of the number of
activities/tasks and related work
products per process that were
examined.
What specific COBIT documents did you
find most helpful when customizing the
PAM assessment model?
It's a little confusing to determine the
capacity level from level 2 and level
5...some tips please?
What degree of accuracy was developed
in defining the process/activities/tasks
required to achieve the specifically
selected outcomes prior to starting the
assessment?
PAM to assess and continuously improve their IT
processes.
Currently included in the Audit scope from a risk
perspective. The PAM was openly shared with Internal
Audit in an effort to reduce any duplication in effort. The
reflection back to Audit should be “we know where we
are in the process and are working on the following.”
Audit will certainly provide input as it relates to the risk
relationship and the implications of fixing one thing
before another. The bottom line should be that we have
base lined and this is where we are at. The
communications must be very transparent if you expect
to approach this as a win-win.
Please refer to ISACA for “COBIT 5 for Information
Security” – it can be used similarly to the approach we
took for the PAM assessment discussed on our webinar,
taking advantage of the COBIT 5 PAM assessor’s guide.
I would not call this a self-assessment activity. A
significant difference between this and a self-assessment
is the usage of artifacts or finding factual evidence that
the process is working as intended. Doing this removes
any individual perceptions with regard to execution. If
the process is working in conjunction with the
framework the outputs should be X. Not a lot of room
for individual perceptions when the outputs have been
identified.
The COBIT 5 Enabling processes served as a guide to the
PAM assessment here. While most activities/tasks were
examined, there were a few exceptions of these and
related work products, given the unique characteristics
of some processes in the DuPont environment.
COBIT 5 PAM assessor’s guide, as well as COBIT 5
Framework and Enabling processes.
Please refer to ISACA for the COBIT 5 PAM assessor
guide and relevant COBIT PAM training materials.
The framework has very defined inputs and outputs. The
tolerance level expected in both simply cannot be
flexible if you expect the framework to build efficiencies
and therefore reduce costs or move to win-win. This is a
quality exercise in that hitting the appropriate levels, a 2
may be ideal for one company and a 4 for another.
There are cost penalties for overshooting or
undershooting. It just depends on where your company
2
24 June 2014 ISACA Professional Guidance Webinar
IBM-DuPont COBIT PAM Assessment
Attendee Questions & Answers
14
15
What are the top 3 IT processes that a
company should apply the PAM too?
This process had a duration of 1 year?
16
What tools do you use to track and
present the metrics? Who has visibility
to these?
17
Was the process automated?
18
Can you explain what the numbers in
the middle column on slide 19 mean?
19
Do you find that cultural resistance is
the greatest inhibitor to managing
changes?
20
While the PAM refers to generic
practices, work products and resources
for levels 2-5 in performing an
assessment, it is necessary to define the
assessment input prior to the data
collection phase. To what level of detail
were the processes and work products
in this assessment defined prior to
conducting the assessment?
Are you expecting your service providers
to comply with specific process
attributes? Do you state these
requirements in contracts?
21
wants to be on the continuum.
It really depends on the company’s business
requirements and current state of their process
capabilities.
Yes annual execution and reexamination for the life of
the contract.
We have developed some IBM proprietary toolsets to
facilitate the evidence gathering and process capability
assessment. These toolsets and processes have been
explained to the assessment stakeholders. The
appropriate assessment participants have access to the
usage of these toolsets.
The process is a combination of some manual reviews of
evidence, automated toolsets, workshops, etc. as
suggested by the techniques used to perform the
assessment.
The process improvement opportunities identified
through the COBIT PAM assessment have been
numbered, as in the middle column on slide 19. While
the numbering has no indication of priority, the coloring
of red, yellow, and green do indicate the recommended
priority for implementation. Also see number 4 above.
Actually I would say that it’s individual perception. For
the most part people are open to change as long as they
understand the end point. The framework provides that
for them. It clearly outlines the target and lets them
define the pathway for improvement. It becomes more
of a strategy exercise if you present in a way that says
“hey, have you thought about this? What’s the
probability of getting there in X?” Using everyone’s
experience to get to the solution or endpoint is ideal and
sells framework buy in.
All the inputs, processes and work products were
determined and defined in detail prior to the data
collection phase.
A major strength of using a third party contractor is
specific to approaching this from a win-win perspective.
Yes you are paying them to execute but understand the
iron fist generally results in either an early departure
from the contract or limiting services to shield from
3
24 June 2014 ISACA Professional Guidance Webinar
IBM-DuPont COBIT PAM Assessment
Attendee Questions & Answers
22
Wonder how you can say Largely
achieved if the score stands at 51%
23
During the session you mentioned many
few companies are at Level 5. Since
COBIT PAM is very new... would like to
know how many organizations are
moved to COBIT 5 Model and what is
their Capability Level
24
Is there a benefit to performing a PAM
Assessment if the assessment team is
new to COBIT and if so, how would you
recommend the assessment be
performed and/or what would be best
to focus on? Thank you
25
What does PAM stand for? I missed it
inadvertently.
litigation perspective. Neither position is beneficial to a
long term business relationship. The PAM is about saying
these are your processes and these are ours. Which one
is more effective or efficient given the general level of
current processing and where do we want to be.
Requirements for the most part are set at the industry
standard level (ISO). Just like any company, we have are
sticking points that are not negotiable. The ideal
situation is that the areas of non-negotiability need to be
the exception not the rule.
The associated percentages for the rating scales of
largely achieved, as well as not achieved, partially
achieved and fully achieved, should be used as a
reference point – it requires the COBIT assessor’s
professional judgment on the actual evidence. It should
not be, and never meant to be, used as scientific
thresholds.
The COBIT 5 PAM is an evidence based assessment
model based on the COBIT 5 and ISO/IEC 15504. It is a
capability assessment, not a maturity assessment. It is
based on facts, not subjective opinion as COBIT 4 CMM
or prior assessments. While there is no hard count of
how many companies are currently at PAM Level 5, the
empirical data have clearly indicated many companies
are at lower levels such as PAM Level 1 or 2, rather than
PAM Level 4 or 5.
My recommendation would be to start with the Align
Plan and Organize processes, more specifically Quality,
Risk and Security. Starting here will help you define just
where the company is. If there are significant pieces of
any APO process missing you can bet that the process
execution underneath is lacking or ad hoc. Understand
it’s not necessarily bad for these pieces to be missing.
Your company may be new or newly acquired, reforming
or just not focused on a specific area for various reasons.
This being said this will likely impact the other areas of
the framework. As you walk your way through the PAM
don’t be frustrated as you bump into these missing APO
processes. This is just PAM’s way of communicating your
priorities. Identify it put it on your list of priorities and
move on. All part of the process.
Process Assessment Model – it is based on the
internationally recognised ISO/IEC 15504
Software Engineering—Process Assessment standard.
4