NFS Portal Testing Progress Ollie Ollewagen 7 August 2013 Problems Identified • • • • • Stability of solution IRTMS Helpdesk not available Fault Reporting cumbersome No feedback on items Reported Test Heads maintenance issues © Telkom 2011 | NFS Portal Feedback | 2 NFS Portal System Flow Diagram IRTMS TaskMate II Web Portal Portal User Testing xAct Application Tech ITBM_INFO Xsuite IP Activator LoopCare DSL_INFO Xsuite Internet authentications Warehouse SMS Gateway Fault History SMS messaging Clarify xAct Customer Info DSLAM update / reset CBS Network Info / update T-Agent EN update / reset © Telkom 2011 | NFS Portal Feedback | 3 EAI – Enterprise Application Integration Stability of solution • ITS Have done the following • The server was spilt from 1 to 4 to manage additional load • Memory and tread leaks on the Application were identified and resolved • In-&-out bound Interface (EAI) problems were resolved • Implement “Fog-light” to monitor and report major incidents • ITS created a task team for the NFS Portal to ensure stability and manage incidents, the IT Executive responsible is immediately notified by the monitoring tools when the service degrades • The development team did a complete code re-write in the application to handle the load • Developed a method to provide better messages to technicians • Continuously monitoring the application for improvements • ITS enhanced test results to discriminate between Test head failures and Soft failures also providing better instructions to the technician on actions required. – 19 July 2013 • • Conditional Pass on test head failures with No retry Will Indicate which failures require a manual retry © Telkom 2011 | NFS Portal Feedback | 4 IRTMS Helpdesk not available • Part of ANTC and have long answer times due to staff also performing other functions • ANTC staff do not have system skills to analyse problems • Support Function moved to NFS System Support – 1 August 2013 © Telkom 2011 | NFS Portal Feedback | 5 Fault Reporting cumbersome • Technicians can only telephonically report faults • Call centers have no means of Reporting faults. – Implemented a email with automatic tracking and feedback. - Done – Implement a reporting function via IPPACMAN - Done – Logged CR 10293 to provide Email function from NFS Portal screen Waiting for impact study and delivery date from ITS • Developed a report that can be extracted indicating all network and test head failures. - Done • Investigate the possibility to use Foglight for identification of Network and test head failures – To complete investigation in next month • Fault management process implemented between NFS and NCO – Continues improvement will apply to improve repair of Test heads © Telkom 2011 | NFS Portal Feedback | 6 No feedback on items Reported • Technicians never received feedback on faults reported. • With move to NFS System Support technicians will receive updates via email. • Have logged CR 10305 for a notice board functionality to give technicians updates on events and highlight problems – Waiting for ITS to do impact study and to provide feedback • Have logged CR 10306 document distribution to assist techs to get hold of documents to assist them – Waiting for ITS to do impact study and to provide feedback © Telkom 2011 | NFS Portal Feedback | 7 Test Heads not functional or maintained • IRTMS repair takes long due to spares availability – Have implemented an escalation and tracking process • No formal Fault management process did exist for LoopCare. – Process now implemented with NNOC and NCO • Work on effective routine maintenance on test heads to reduce faults. – Proactive monitoring – Identify problematic sites for investigation – Doing of regular 6 month routine on test heads © Telkom 2011 | NFS Portal Feedback | 8 Testing Availability • IRTMS / LoopCare testing integrated in the following systems – Unibase – NFS Portal – CPH – Clarify for Call Centers • All MSAN equipment is implemented with LoopCare testing capabilities. © Telkom 2011 | NFS Portal Feedback | 9 Implementation • OPS Managers trained on how to use NFS Portal fully • Technicians trained by OPS Managers • Included in Functional MSAN training • Included in CoE and ADSL refreshing material • Training material developed for Call Centers • Call Center Staff trained to use Upfront testing © Telkom 2011 | NFS Portal Feedback | 10 Next Steps • Implement improvement drive for stability and up time for the test heads. • Improve proactive reporting process to identify problems before it impacting operations. © Telkom 2011 | NFS Portal Feedback | 14 Thank You! Ollie Ollewagen