D C entral S Central DCS Tutorial -Shutdown editionMade by : Etiennette Auffray Andrea Barisone Last Revision : June 27, 2013 1 Central DCS Team D C entral S Central DCS Expert Team: Frank Glege (71688, 163919) Robert Gomez-Reino Garrido (71654, 162366) Lorenzo Masetti (165820) Giovanni Polese (165807) DCS expert on Call : 162229 Central DCS Operation: Central DCS desk : 75140 Etiennette Auffray >> Shift organisation, Tutorial, Twiki : (75844, 163646) Andrea Barisone >> Tutorial, Twiki Central DCS shift requirements: • CERN ID card and access to P5 and USC ( request done by EDH) • All safety trainings up-to-date, including the Biocell training. For more information, please visit: http://cms-safety.web.cern.ch ; • At least the token. 2 D C entral S CMS Slice View 3 DCS Workstation : Screens D C entral S Cooling Panel Racks Panel Sensors 4 CMS Panel DSS Screens Main DCS Panel Cavern Access GTPM Panel DCS alarm Panel Subdetectors Panel 4 Cavern Access : Screens D C entral S Cavern Access Displays WP Display Intercom Experimental Cavern Doors Telephone Speaker for the UXC and USC Cavern 5 D C entral S DSS Screens UXC 55 USC 55 6 DCS Mobile Phone D C entral S DCS Mobile Phone 7 Central DCS General Guideline D C entral S As a general guideline all variables controlled and monitored by central DCS should be GREEN If not green, a corresponding alarm will be raised and will appear in the central DCS Alarm Screen; Therefore it is very important to monitor the alarm screen during your shift and have prompt action in case of any alarm; In the next transparencies, the central DCS shifter tasks will be visited one by one, keeping in mind this general guideline. 8 Main DCS Tasks D C entral S 1. What to do when a DCS alarm occur during the Shift (Concerned Screen : DCS Alarm Panel ) 3. What to do when a DSS alarm occur during the Shift (Concerned Screen : DSS Alarm Panels) 4. Monitoring Operation: Services (Racks and Cooling) 5. Underground Cavern Access Management 6. DCS Elog : (When and How use it) 9 DCS Alarm Screen D C entral S This screen should be : Checked regularly Empty Each time an important problem occurs (subdetectors, services, etc…) an alarm (together with an audible alert) should appear on this screen 10 DCS Alarm Screen Below you find a row where there are several information regarding the alarm : select and right click on it and a pop up window will appear D C entral S On the first line of the page you can see which subdetector is affected by the problem : the alarm will continue blinking until handled by the DCS shifter Read Instructions to correctly handle the alert : In case you don’t know how to solve it call the responsible If there is no instruction send an email to us 11 DCS Alarm Screen SUBDETECTORS MAIN LINE: Colors meaning list : LIGHT BLUE : No errors RED: Fatal alert ORANGE: Error YELLOW: Warning GREY: Alert acknowledgeable D C entral S Time stamp (when shifter has clicked on Alarm Help) Direction box : CAME : The condition which triggered the alarm is still present WENT : The condition which triggered the alarm has disappeared 12 D C entral S Monitor Services 13 Service Screens (Rack Panel ) D C entral S Racks summary Racks Cooling Station. Rack Power Status per underground area S1-S4 = Service Cavern X0-X5 = Exp. Cavern If you click on 1 rack you get details on it Take actions on racks. Follow the instructions Color code: Grey – Not used Green – OK -----------------------Blue – OFF -> Red – ERROR -> requires intervention! 14 Service Screens (Rack Panel ) D C entral S Rack details 15 TOP PART OF THE SCHEME entral Service Screens (Cooling) Cooling Panel D CS Outside working hours hours:(8h30/17h30) no matter which During working TOP PART of the screen: doubt or change of color: CMS cooling support, 163021 ORANGE, write on DCS e-log, RED - Call CCC 72201 BOTTOM PART of the screen (flow & tank levels) : ORANGE : 17h30/24h00 call 165000 24h00/7h00 ignore, RED, call 165000 Color code: Green – OK -------------------------Yellow – warning Orange – Error Red – FATAL ERROR Blinking Values requires intervention! BOTTOM PART OF THE SCHEME Copper circuit leak Warning: 1 l/h > 60 min Alarm: 4 l/h >30 min or 8 l/h >1 min Aluminum circuit leak Warning: 1 l/h > 60 min Alarm: 4 l/h >30 min min or 8 l/h >1 min Rack circuit leak Warning: 2 l/h > 60 min Alarm: 6 l/h >30 min min or 8 l/h >1 min Service Screens (Cooling) D C entral S Blinking values • If a value is blinking: you should verify at http://dipg.cern.ch/dipg if there are invalid CMS cooling values. In case the bullet corresponding to CMS Cooling is red, it is very likely to be a problem on the CCC side. You should call the CCC (72201) to inform them there is a problem with their publications and point to them what you see from the link above. If there are no invalid values in the link above, call the central DCS expert and inform about the problem. The interface from above is slow, so you should wait about 10 minutes to make sure it is updating, before reporting the problem. 17 DSS :Detector Safety System Front End warning light : should be always green. If not, call Alain Meynet, between 7:00 and 22:00. If red during the night, call Alain Meynet in following the morning. If there is a DSS Alarm with Siren and light, the DCS shifter should immediately call the Technical coordination number 165000 DSS Alive. Should blink. User should always be set to ‘user’. If not, click on Logout. The panel will restart automatically. Stop Siren button: stops the siren in the DSS hardware panel when there is an alarm. Only works from the DSS USC55 panel! D C entral S Right-click on the alarm and select ‘Show’ to obtain more information. If the alarm’s “Triggered” state is red, it cannot be acknowledged. To acknowledge an alarm: 1. click on the ALARM column, on the DSS ALARMS list; 2. Then click on the ACTIONS column of the DSS ACTIONS list. DSS :Detector Safety System D C entral S In case of alarms: read the alarm description and contact the responsible person (e.g. Magnet, subdetectors, etc.); In case the alarm description is not clear: during working hours: call DSS expert, Alain Meynet: 163146; out of working hours: call Christoph Schaefer: 160202 or Wolfram Zeuner: 162541 and at last 165000 (in case the other numbers failed). Check the DSS display is Alive: These buttons should blink. If not, restart the panel (instructions in the DSS binder). If restart does not work, call : 162082 19 DSS :Detector Safety System D C entral S DSS Hardware Piquet: 162082 20 Underground Cavern Access D C entral S Every person going underground (Service and Experimental caverns) is required to fill in the paper version of the 'Access to Experimental and Service Caverns' list that can be found in a binder by the Shift Leader's desk; The central DCS shifter has to check that the person fills this form when he/she goes into the cavern and when he/she returns; The person has to have a mobile phone; If after some time the person is not back in the control room, call him/her to check everything is ok. You can find detailed information on underground access in the cDCS twiki: https://twiki.cern.ch/twiki/pub/CMS/OnlineWBCentralDCSInstructions/DCScavernAccess1.3.pdf 21 Cavern Access : Screens D C entral S Cavern Access Displays WP Display Intercom Experimental Cavern Doors Telephone Speaker for the UXC and USC Cavern 22 Underground Cavern Access 1. D C entral S Each person that wants to access the experimental cavern has to be properly registered in a WP; 2) The Access management software Work Packages (WP) is reachable from the CMS Online Page ( clicking on ACT ) or directly from this url : http://cmsonline.cern.ch/portal/ page/portal/CMS%20online%20s ystem/ACT Underground Cavern Access 3) Once inside DCS Shifter has to select : “Document Database “ D C entral S Underground Cavern Access 4. D C entral S go up on the left side of the page and select “Shifter Page”) 4 25 Underground Cavern Access 5. 6. 7. 8. D C entral S Enter the name of the person willing to enter of the cavern The list of assigned WP will appear Select the WP to see its status. To start the WP Select in the menu status : Ongoing 5 8 6, 7 26 Underground Cavern Access 9. D C entral S Turn the dosimeter ON; (Push the black button three times : once for each message will pop up on its little screen (CHANGE >> ENTER >> 1mv) : then a red light will flash few seconds and also you will hear a “bip”) 10. Select the dosimeter number in the ‘Operational Dosimeter’ field; 11. Read the initial dose from the dosimeter and fill it in the appropriate field of the Shift Operation page; 12. Click ‘Ok’ to grant access to the person; Point 10 to 13 13. By the return of the dosimeter, fill in the final dose and click ‘Ok’; 14. Turn the dosimeter OFF (simply push the black button on top of the dosimeter, the word “CHANGE” will appear, keep on holding the black button until the word “GO OFF” shows up. Then immediately release the black button. If one waits for too long to release the black button, the 27 dosimeter will not switch off). Underground Cavern Access 9-14 Button 9. Dosimeter number D C entral S Turn the dosimeter ON; (to do that, push the black button three times : once for each message will pop up on its little screen (CHANGE >> ENTER >> 1mv) : then a red light will start flashing) 10. Select the dosimeter number in the ‘Operational Dosimeter’ field; 11. Read the initial dose from the dosimeter and fill it in the appropriate field of the Shift Operation page; 12. Click ‘Ok’ to grant access to the person; 13. By the return of the dosimeter, fill in the final dose and click ‘Ok’; Red flashing light Dosimeter display 9-14 After an access: DCS shifter has to make sure the lights in UXC are switched OFF. 14. Turn the dosimeter OFF (simply push the black button on top of the dosimeter, the word “CHANGE” will appear, keep on holding the black button until the word “GO OFF” shows up. Then immediately release the black button. If one waits for too long to release the black button, the 28 dosimeter will not switch off). Underground Cavern Access D C entral S 15. Click on edit button 16. The WP summary page will appear 17. In the list of name click on edit 16 15 17 29 Underground Cavern Access 18. 19. 20. 21. D C entral S Insert the name of the person you want to register (previously authorized by the WP leader) Select his role save changes The new name will appear in the name list 19 18 9 21 20 30 Underground Cavern Access D C entral S Once the authorized person has received his dosimeter (s)he can finally access to the cavern, and in the two displays here below the DCS Shifter can check all the cameras pointing at the different doors in the cavern : When the person reach the entrance (21) there are still few steps to be execute … 21 Underground Cavern Access The name will appear here …. 23 22 25 D C entral S … he needs to pass his badge over the card reader in the yellow underground cabinet (22) : Then on the DCS screen his name will appear in the dedicated window (23 and the DCS Shifter has first to select the line and then click the button “Give the Key” (24) in order to let the drawer in the underground rack opening (25) and delivering the access key to him. 24 Once selected the name the button will became clickable DCS Elog D C entral S DCS ELOG address : cmsonline > Common > Elog > Subsystems > DCS At the beginning the shift: write an elog with the summary status of all the central DCS systems; During your shift : Write only a short elog with specific information when you have any problem with subdetectors, cooling, racks, etc... with indication on your elog Title of the problem At the end of the shift: Write an elog with a summary of all the relevant events that happened during your shift. Shift Instructions are available online at: https://twiki.cern.ch/twiki/bin/view/CMS/OnlineWBCentralDCSIn structions 33 Useful Contacts (1) • Emergency, accident, fire : 74444 •Any technical problem (Siren Alarm): 165000 • Rack Problem : Rack becomes off or in error : 1. Tracker (Pixel) : 165502 2. Tracker (Strip) : 165503 3. ECAL : 70130 or 165504 4. HCAL : 165505, 5. CSC :165509 6. DT : 165507 7. RPC :165508 During the day you can also call Silverio : 163706 • Cooling Problem : 1. ECAL cooling: 70130/165504 2. One (other the ECAL) of the box is not green : During the day : Cooling piquet 163021 Outside working hours : • Top part : call CCC 72201 • Bottom part : call 165000 if red • Screen blinking : Call CCC 72201 D C entral S Magnet Problem : Call Magnet piquet 162082 • Cryogenic problem : Call Cryogenic piquet 160124 • Gaz Problem : Call Gaz Piquet 162516 • DCS Problem: • Central DCS :Call DCS expert on call 162229 • Subdetector DCS : 1. Tracker (Pixel) : 165502 2. Tracker (Strip) : 165503 3. ECAL :72606 or 165504 4. HCAL : 165505, 5. CSC :165509 6. DT : 165507 7. RPC :165508 • DSS Problem : • Alarm : Alain Meynet during the day 163146 Christoph Schaefer 160202 • Technical Problem : 165000 • Hardware Problem : 162082 Useful Contacts (2) TS Technical control room (CCC): 72201 IN CASE OF ANY TECHNICAL PROBLEM CALL: 16 5000 Backup: Austin BALL: 160408, Christoph SCHAEFER:160202, Wolfram ZEUNER: 162541, Martin GASTAL: 163702 DCS expert on call : 16 2229 Backup : Frank : 163919, Bobby : 162366, Lorenzo : 165820 DSS expert on call : 162082 CASTOR: 162875 TOTEM (Green Barrack): 77534 BRM/BCM: 165501 Fireman at P5 (working hours ) : 164812 (still call directly the 74444 in case of DSS alarms) Online Web Contact List : https://twiki.cern.ch/twiki/bin/viewauth/CMS/OnlineWBContactList D C entral S