CRG-TM, June 1st, 2015
A. Perin, TE-CRG
A. Perin, TE/CRG, 01.06.2014
Page 2
Outline
•
Project scope and preparation
•
Project execution
•
The resistive current leads of the DFBXs
•
Conclusions
A. Perin, TE/CRG, 01.06.2014
Page 3
Project scope
• Consolidation of the splices, CL line, clamps in SHM, safety disks
+ additional NC discovered during LS1:
• Insulation problems: between HCM an LCM, on one pigtail
• Multilayer bellows failure in 4 DFBAs shuffling modules
• About 20 persons, 8 CERN groups, 2.5 MCHF, integrated into the SMACC
project
• About 1200 tasks identified and tracked during LS1
A. Perin, TE/CRG, 01.06.2014
Page 4
Project preparation
• Project management: tools defined and operational several month before
project start
• Main risks identified early and explicitly: technical difficulty + coordination
•
Specific actions were taken for technical development and team training (e.g.
mockup for SHM-HCM interconnection and model for clamps development)
•
Personnel for coordination and execution follow-up defined and identified early.
• Effort started early (2011) on technical development
• Design and production readiness review held in February 2013 + QA review in
July 2013.
• Big effort to prepare the work:
•
•
•
•
•
•
Procedures, technologies, systematic feedback and approval
Quality assurance, documentation prepared in advance, web site, storage space, etc.
Detailed schedule, information to all teams involved
Procurement of tooling
Personnel training
Etc.
A. Perin, TE/CRG, 01.06.2014
Page 5
DFBA splice consolidations for LS1: the dashboards
Resources boost
Resource decision
SHM bellows
Resource problem
Broaching
• Dashboards were a very useful tool to
communicate and give a synthetic
view of the project
A. Perin, TE/CRG, 01.06.2014
Page 6
Feedback on project execution phase 1/2
• Coordination with all teams worked extremely well thanks to among other
factors:
•
•
Presence of a clearly identified execution coordination person
Weekly meeting and timely update of the schedule
• Main difficulties: unexpected problems
•
•
The key “broaching” technique was found to have serious problems despite
extensive testing before.
Faulty multilayer bellows were found in the SHM cold mass.
• Solutions found with:
1.
2.
Rescheduling of tasks and re-allocation of resources
Work with all stakeholders and optimization of resources on a CERN wide base
Main “lessons learned”
• Unplanned problems do happen!! It’s useful to “prepare for the unplanned”
• Specifically identified person for execution coordination is extremely useful.
A. Perin, TE/CRG, 01.06.2014
Page 7
Feedback on project execution phase 2/2
• Personnel
•
Training of external personnel proved very useful, but followup by expert CERN
staff is mandatory for critical tasks.
•
The decrease of external personnel when approaching the end of the project
created some difficulties: quality, coordination, etc.
• Useful to more specifically plan & track the end of project, but it is not easy as
typically it is difficult to know precisely the end dates…
• Quality assurance and quality of work
•
During the “main” phase mostly no problem (in particular when clearly defined
procedures and controls were defined).
•
Difficulties encountered at the end of the project in “smaller” and “less challenging” tasks
at the end of the project: incorrectly installed equipment, leaks, etc.
•
Often found after handover of DFBs to OA: direct impact on commissioning!!
•
Frequently operations where no dedicated procedure was deemed necessary. Only a
check list was used.
In some cases, systematic or random inspection was missing
•
• Main lessons learned
• Systematic or random controls should be implemented even on “simple”
tasks, particularly in the final phases.
A. Perin, TE/CRG, 01.06.2014
Page 8
Project closing and commissioning
•
Project closing & documentation: essentially ok, but still a small percentage to be
completed because the personnel in charge was shifted to other high priority tasks…
•
Some production tracking documents still not uploaded (need clanup), etc.
•
ECR and and built for clamps & shuffling module, …
• Commissioning
• No NC linked to splices, CL line, busbar, welding found.
• Several non-conformities found during commissioning: all linked to
“routine” maintenance tasks (leaks, etc.)
•
Inspection (systematic / random) shall be planned for all tasks.
•
A formal “handover” of the equipment shall be performed.
A. Perin, TE/CRG, 01.06.2014
Page 9
The resitive current leads of the DFBXs
•
The control system for 200 resistive current leads of the DFBXs was improved by
adding 200 flowmeters, remotely installed in non radiatiion affected areas. Mainly
piping + control system upgrade. Installation performed on time and with no major
problems reported.
•
Serious issues during commissioning, with 15/32 heaters to repair, local diagnostic of
all current leads (50 swapped channels), replacement of several heating elements and
thermometers, several leaks and two clogged pipes, requiring more than 60
interventions over 6 months. Most of the time was taken for the correction of NCs,
actual tuning of parameters took about 3 days.
•
Several identified causes of problems:
•
«legacy» of the DFBXs not fully diagnosed: e.g. heater problems were most probably present but not
identified, some channel swapping was also probably present before LS1
•
Specification documents should have been more widely checked and circulated
•
Insufficient control of the quality of work performed (leaks, clogged tubes, swapped channels)
•
Very limited time to perform initial «debugging» due to late availability of sectors
Lessons learned
•
More diagnostic shall be performed on complex equipement before consolidation.
•
Controls shall be implemented for all tasks.
•
More tests should be planned before the actual commissioning at cold.
A. Perin, TE/CRG, 01.06.2014
Page 10
Conclusions
•
The consolidation of the DFBAs during LS1 encountered mainly technical
problems that could not be foreseen before. These were overcome by a
combination of schedule adapation and resource re-allocation.
•
Thanks to a detailed preparation phase, the coordination for the DFBA
consolidation was excellent, adapting smoothly to unexpected events.
Main lessons learned
A detailed preparation is essential (but not always sufficient) for technically
challenging tasks
•
The presence of a person in charge of the execution coordination was
extremely useful
•
Unplanned problems do happen! One should prepare (as far as possible) for
the unplanned problems (identify who can act, etc.)
•
Controls/inspections shall be performed even on tasks considered routine or
simple.
•
A formal handover of equipment to the operation team shall be performed
and a sequence of «pre-commissioning» shall be defined.
A. Perin, TE/CRG, 01.06.2014
Page 11