Process Operability Class Materials
Refining Trouble Shooting Skills
Basic flowsheet
LAH
LAL
Design with Operability
L
2
LC
1
LC
1
FC
1
FC
1
TC
2
TC
1
F
4
fuel
T
10
T
12
T
13
Copyright © Thomas Marlin 2013
The copyright holder provides a royalty-free license for use of this material at non-profit
educational institutions
T
11
Refining our Trouble Shooting Skills
Now that we have learned the basics,let’s review
some issues that have challenged students
•
What is a Fact?
•
Multiple Root Causes
•
Looking for the Root Cause: “Changes” vs. “Usual”
•
Iteratively finding the Root Cause through
intermediate causes (Fishbone Diagrams)
•
Priorities for the Diagnostic Actions
•
Diagnostic Actions – Disturbing the process
PROCESS TROUBLESHOOTING
What is a
fact?
What is the best statement – the most representative of the situation?
•
The temperature is 55 °C.
•
The sensor measures 55 °C.
•
The meter shows 55 °C.
•
We do not have any idea what is going on – I’m out of here!
We need to make clear statements of the evidence so that we do not
confuse others (or ourselves).
PROCESS TROUBLESHOOTING
What is a
fact?
v8
F
2
F
1
T
1
v3
T
3
T
4
F
5
T
5
F
3
L
1
v1
T
6
v5
v2
v6
T
7
L
2
v7
T
2
Hot
Oil
F
4
P
1
T
9
T
8
Hot
Oil
F
6
What is the best statement – the most representative of the situation?
•
Sensor T7 uses a thermocouple to measure temperature.
•
SensorT7 measures the temperature after the flash valve.
•
Sensor T7 is shown to be located after the flash valve on the drawing.
•
I do not trust T7-I’m going to feel the pipe!
PROCESS TROUBLESHOOTING
Multiple
root causes
We are sure that the cooling
valve is fully open.
Let’s look elsewhere for the
problem.
Situations with several independent root causes are much
more challenging to trouble shoot.
Many of the recent major industrial catastrophes involved
multiple faults. The plant operating personnel were not able
to consider all of the possibilities.
Often, one of the faults (e.g., a faulty sensor) leads us to
incorrectly eliminate the true root cause of the problem.
PROCESS TROUBLESHOOTING
Multiple
root causes
We have inconsistent
information about the opening
of the cooling valve..
We better look into this
immediately!
Situations with several independent root causes are much
more challenging to trouble shoot.
We must try to identify critical situations during design and
provide sufficient redundant equipment (e.g., sensors) to
ensure that people can trouble shoot them.
Training is also important.
PROCESS TROUBLESHOOTING
“Change”
vs
“Usual”
Two major categories of situations exist for the trouble
shooter
•
“Change” when significant changes have occurred
that likely (but not certainly) have caused the
problem. Examples are new plant startup, restart
after maintenance, & changes to operating
conditions.
Since the equipment has not been tested completely, major
faults might exist, such as hand valves being improperly
open/closed, pipes being blocked, motors failing, sensors
incorrectly calibrated, etc.
PROCESS TROUBLESHOOTING
“Change”
vs
“Usual”
Two major categories of situations exist for the trouble
shooter
•
“Usual” when no significant, obvious change has
occurred. The root cause must be determined from
evidence and applying our knowledge of process
principles.
Since the equipment has operated well up to the problem, the
change has occurred because of a recent change to
equipment or inadvertent action by a person.
The likelihood of “change” faults are very low.
PROCESS TROUBLESHOOTING
Iteratively
find root
cause
What is the cause?
Professor
The temperature is high
What is the cause?
The cooling water flow is
low
What is the cause?
Oh, for heavens sake,
when does this stop?
Student
PROCESS TROUBLESHOOTING
Iteratively
find root
cause
Many potential
root causes
Symptom(s)
Real world
Cause to effect
Trouble shooter must work “in reverse”, from
effect to cause. It is difficult to jump all the way to
each root cause in one step.
PROCESS TROUBLESHOOTING
Iteratively
find root
cause
What is the cause?
The temperature is high
Symptom(s)
What is the cause?
The cooling water flow is low
Many potential
root causes
What is the cause?
The pump head is too small to
achieve the needed flow.
Rxn
Engineering
Heat transfer
Fluid
mechanics
Most trouble shooters benefit from breaking
problem into smaller segments, each with its causeeffect relationship.
Let’s learn through a workshop: The symptom is “High level in
V-30”. The process is a distillation tower. Please find all
possible root causes and document in a fishbone diagram.
Depropanizer, C-8
CW
By-product
to fuel,
mostly C1
and C2
PAH
PC-10
LAH
V-30
LC-3
LAL
FC-4
FC-4
reflux
This is a simplified drawing of one part of Figure PID-2A
from Wood’s Process Design and Engineering Practice
Tower bottoms stream
Product
mostly C3
mostly C4+
PROCESS TROUBLESHOOTING
Iteratively
find root
cause
Building a Fishbone Diagram: What could cause the
symptom of “high liquid level in V-30”?
Start the fishbone diagram with the most fundamental
causes of the symptom; do not try to jump to the root cause
(we’ll get there).
Some ???
???
???
High level
in V-30
???
• Sensor error
• Too much liquid
into tank
• Too little liquid
leaving tank
PROCESS TROUBLESHOOTING
Iteratively
find root
cause
For each of the three intermediate causes in the diagram,
find one cause, which could be, but does not have to be, a
root cause.
Connection between sensor and
vessel is plugged
Sensor Error
Too much
liquid into
tank
Tower feed rate has been increased
High level
in V-30
Too little
liquid
leaving tank
Pump motor has stopped
PROCESS TROUBLESHOOTING
Iteratively
find root
cause
What could cause the symptom of “high liquid level in V-30”?
Complete all paths to root causes for the three initial causes
in the diagram.
Connect between sensor and vessel is
plugged
Sensor Error
Too much
liquid into
tank
Tower feed rate has been increased
High level
in V-30
Too little
liquid
leaving tank
Pump motor has stopped
Delta pressure sensor calibrated incorrectly (reading higher level than actually exists)
Connection point (tap) blocked/corroded
(level measurement is constant causing controller to make an incorrect action)
Controller
sensor
error
Extra
condensation
Cooling water temperature becomes much colder
Steam valve fails open (unsafe)
Too much liquid
into the tank
Extra vapor
overhead
Increased feed rate (level controller will lower level in
time)
Increased % propane in feed (level controller will lower
level in time)
High level
in V-30
Reflux or product flow valve failed closed (safe)
Valve
malfunction
Too little liquid
leaving the tank
Reflux or product flow valve stuck , not responding to control
signal
Power loss (motor failure or coupling break)
Poor
feedback
control
Pump
malfunction
Cavitation
Vortex (unlikely with high level)
Magnitude of feedback controller gain (Kc) is too small
Distillate liquid product valve saturation
PROCESS TROUBLESHOOTING
Iteratively
find root
cause
Building a Fishbone Diagram
Class Exercise: The fishbone diagram on the previous page has
many root causes. Add at least one additional root cause to
each of the major branches (sensor error, too much liquid in,
and too little liquid out).
Building a Fishbone Diagram
Class Exercise: The fishbone diagram on the previous page has
many root causes. Add at least one additional root cause to
each of the major branches (sensor error, too much liquid in,
and too little liquid out).
One answer for “sensor failure”:
The alarm level sensor is improperly calibrated. An alarm
occurs (audible signal and blinking light) when the actual level
is not above the level high alarm value (which might be ~85%).
No process excursion has occurred. Solution is to calibrate the
alarm sensor, which is different from the level sensor used for
control.
Building a Fishbone Diagram
Class Exercise: The fishbone diagram on the previous page has
many root causes. Add at least one additional root cause to
each of the major branches (sensor error, too much liquid in,
and too little liquid out).
One answer for “too much liquid in”:
The cooling water flow rate has increased, resulting in more
liquid being condensed; perhaps, an additional cooling water
pump has been turned on. The level controller will ultimately
increase the product flow rate.
No process excursion has occurred. The solution is to wait,
while monitoring that the level control functions properly.
Building a Fishbone Diagram
Class Exercise: The fishbone diagram on the previous page has
many root causes. Add at least one additional root cause to
each of the major branches (sensor error, too much liquid in,
and too little liquid out).
One answer for “too little liquid out”:
One or more of the block (manual) valves in line with the
product control valve could be partially closed. This would
reduce the maximum flow rate. This cause would yield the
symptom that the product flow valve is 100% open, while the
flow rate is much lower than expected.
The solution is to check the block valves and open those in line
with the product line to 100% open.
PROCESS TROUBLESHOOTING
Iteratively
find root
cause
Advantages of Fishbone Diagrams
1. Picture of thought process
for learning and avoiding
confusion.
2. Documenting all causes for
fast trouble shooting in plant.
When we disprove this cause,
we eliminate all subsequent
root causes connected to the
branch!
3. Using information to quickly
“trim the tree”, without
checking every root cause.
Advantages of Fishbone Diagrams
Factoid: The blue (solid) root causes yield safe conditions; the
red (stripped) root causes yield hazardous conditions.
Class exercise: Where will you concentrate your efforts when
collecting additional information?
Collect information
about this branch; try
to confirm or disprove.
Symptom
Where do Fishbone Diagrams fit in the Trouble
Shooting Method?
1. Engage
2. Define
3. Explore
Concept will help you pose good questions
4. Plan
You will use FB diagrams to generate a
complete set of hypotheses and evaluate
how the information (initial and collected
as part of TS) affects the truth of each
hypothesis.
5. Implement
6. Evaluate
PROCESS TROUBLESHOOTING
Iteratively
find root
cause
This scenario is GOOD, if we manage the process
and achieve the root causes as candidates.
Professor
What is the cause?
The temperature is high
What is the cause?
The cooling water flow is
low
What is the cause?
The pump head is too
small. A root cause!
Student
PROCESS TROUBLESHOOTING
Diagnostic
Priorities
We need to tailor our diagnostic actions to the process priorities.
Start
Y
Safety concerns?
Follow defined procedures
N
Equipment
protection?
Y
These could be
extreme, including
shutting down the
process!
Follow defined procedures
N
Product quality
degraded?
Y
Y
Large $ impact?
Follow defined procedures
N
N
Production rate
reduction?
Y
Y
Large $ impact?
N
N
1
Follow defined procedures
PROCESS TROUBLESHOOTING
Diagnostic
Priorities
1
Consider many factors in setting the sequence of actions
•
Time required for an action
•
Cost for delaying the solution of the problem
•
Cost for the action
•
Probability of each hypothesis being correct
•
Ability of a single action to eliminate many root causes
We consider these factors when determining the sequence that has the best
expected outcome – solving the problem with the lowest cost.
We recognize that our decisions are based on imperfect knowledge, but we must
use the best information and experience available when trouble shooting.
PROCESS TROUBLESHOOTING
Diagnostic
Actions
In some cases, a small change to the process will help to test
hypotheses.
For example, many variables are oscillating; what is the cause?
v8
F
2
F
1
T
1
v3
T
3
T
4
F
5
T
5
F
3
L
1
v1
F
4
v5
v2
v6
v7
T
2
Hot
Oil
T
6
T
9
T
8
Hot
Oil
F
6
T
7
P
1
L
2
PROCESS TROUBLESHOOTING
Diagnostic
Actions
In some cases, a small change to the process will help to test
hypotheses.
For example, many variables are oscillating; what is the cause?
Several
variables
are
oscillating
• Cycling in all
variables stops
Place one
controller
in manual
• Cycling is
unchanged
• Cycling
changes, but
continues
PROCESS TROUBLESHOOTING
Diagnostic
Actions
In some cases, a small change to the process will help to test
hypotheses.
We try to avoid shutting down a process and opening
equipment for inspection. This is the last action.
Non-invasive testing can be used some times.
We shutdown for safety and to protect equipment, even
though it is expensive!
Refining our Trouble Shooting Skills
These are issues that the instructor is aware of
•
What is a Fact?
•
Multiple Root Causes
•
Looking for the Root Cause: “Changes” vs. “Usual”
•
Iteratively finding the Root Cause through intermediate causes
(Fishbone Diagrams)
•
Priorities for the Diagnostic Actions
•
Diagnostic Actions – Disturbing the process
What additional issues are you finding challenging?
How can we confirm that everyone in the class has
mastered this important skill?