TOPIC: Use of Auto Audit OFFICE: Auditor STATE: OK DATE: 11/15/12 QUESTION / ISSUE: Does your office use the AutoAudit product from Thomson Reuters? If so, would you recommend it to other state audit offices? Based on your experience, what are the pros and cons of this product? State Arkansas Comments The AR Division of Legislative Audit uses AutoAudit for our electronic working paper program. We are still in the process of implementing the application statewide, but at this point are very pleased with the program. Our “roll out” of the program has been ongoing. We have all of our templates in AutoAudit, copied from Word and Excel, directly into the program. It is also easy to place pdf copies into the program workpapers. Our field staff really like the application program and think that, once used in a couple of audits, it saves a lot of time and is easy to use. The review process for supervisors and up has been a little more difficult to implement mainly because of the change from the printed copy. Thomson Reuters has been good to work with, responding very promptly to our questions. The new version coming out soon – Version 5.7 – will be even more responsive to our needs because the application will now look just like the Word and Excel “ribbons” for ease of moving around. We have been very impressed with both the program and Thomson Reuters. We looked at several programs before purchasing AutoAudit. Our decision to use this program was because of the ability to move our templates into the program and continue on without much slowing down of the process. But, we have discovered that using the “Activity Checklist” within AutoAudit – where we copy and paste our audit programs into the Checklists – actually makes completing those workpapers even easier. We are in the process of completing that transition. The program has a feature called Snap! Reporter. It is a great tool, but requires detailed “info in” to produce the report you need. We have about mastered that process and believe that it will help us in the future to monitor progress, produce our reports, compile data from all of our audit reports for analysis, and even communicate with our auditees. The program has ability to work “off the server” and then upload the audit work. It’s called Travel Replica. We have had a few problems with that part of the program that mainly stems from connection problems at the audit site more than staff usage. We were able to utilize the servers that we already had although, since then, we have purchased a new dedicated server. Other than that, we have not had to purchase additional hardware. The only “con” I can think of is setting up the security rights. With 275 staff, that was time consuming and has had to be tweaked several times. However, I think that would be time-consuming and take a lot of thought with any program you purchase. Connecticut We are very excited about the upgrade that’s coming, but even as we’re working with AutoAudit now, we are realizing many efficiencies, particularly with the work on the CAFR and Single Audit! It’s been good for us. We do not use AutoAudit and I am not familiar with it. I went on-line and noted that it is described as, an "automated solution for internal audit departments," and an Use of Auto Audit 1 State Comments "audit management software package." We recently contracted for an electronic workpaper software package. Our RFP drew four responses from: CCH ProSystem fx; CCH TeamMate; CaseWare; and MetricStream We ended up selecting the CCH ProSystem fx product and have been generally happy with it. Idaho Maryland Minnesota I believe the product you are referring to may not be similar to the products above, but if you want me to explain our approach and rollout, I would be happy to do so. Idaho currently does not use electronic audit programs or workpapers, but we are interested in converting. We have started the research process and had a demo of one product. We have been using CCH’s TeamMate for a little over 6 years. Overall, our experience has been pretty good; however, there have been some rough spots along the way. By way of background, the MN OLA implemented AutoAudit in 2007- about when I started with the office as a staff. I am now the IT Audit Manager. We had significant challenges, mostly on our end because we had not done a proper analysis of our business requirements and processes to really understand how implementing an application would impact how we did our work. In short, we used the application for glorified content management. I come from outside government, so I had experience at Target Corp. executing testing for the inaugural year of SarbanesOxley which really became the genesis for all GRC type software. Because of that experience, I have a vision for us to move forward to finally achieve some of the ROI we had hoped for. The only reason I say that, is since government has not been subject to SOX it is somewhat hard to see how all these bits fit together. GRC can be really complicated if you don't have an idea of what the endpoint is SUPPOSED to look like. I am at this very time working with a team of our auditors to complete a business case for an upgrade from our current AutoAudit version to the most recent available. It is very detailed because our Legislative Auditor has really challenged us on the business value of the application. It includes a SWOT (strengths, weaknesses, opportunities, threats) analysis, risk analysis, cost benefit analysis (to be used in our benefits realization plan), and ultimately an ROI projection. Ours will look much different than yours if you are evaluating different applications where as we are evaluating an upgrade from a prior version of the application to the current version- but some of it might be insightful. I could give you a stripped down version of our business case (taking out the actual dollars and some of our more sensitive information) in the next two weeks. If you could provide a bit of information, I could help you a little more. These questions are aimed at determining if you need more of Content Management tool (i.e., SharePoint with some tweaks), a project management tool (sooo many available), or a more true GRC tool (i.e., AuditAudit/Paisley). All "audit" software is not created equal: Content management is less complex, Project Management is more integrated, and GRC is the most integrated. So as you get feedback from other states make sure you are comparing apples and apples and not apples to oranges. Content management solutions can be really cheap and are less complex but they offer less long term benefits. Be aware that it is easier to decide to first Use of Auto Audit 2 State Comments implement a content management solution and then maybe in a few years a project management software and maybe ultimately a GRC tool. It is MUCH more difficult to start with a GRC solution and then work backwards. A GRC tool is a longer term commitment. 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. Missouri North Carolina Are you responsible primarily for municipalities or for state agencies or both? Do you have an IT audit function? What is your office's current use of electronic workpapers? Does your office have "Program Evaluation" responsibilities? Do you perform Financial Statement? Single Audit? Performance Audits? PCI? HIPPA? Do you currently have a risk assessment process for determining which audits to perform? How do you track your staff's time or not? Do you currently provide aggregate reporting of any kind to the Legislative or Executive Branch(s)? Who is responsible for monitoring follow-up on reportable findings? Would you describe your current audit process as being more standardized (i.e. all auditors use the same general payroll audit as a starting point that they may modify) or more flexible (the office relies on experienced auditors to create audit programs) How many auditors do you have? What do benefits or efficiencies do you hope to gain from a "audit software"? Near term? Long term? Do you have staff with excellent project management skills to assist in internally managing the implementation or would you need to rely totally on the vendor? You don't have to email me the results of those questions- but I wanted to get you thinking! :) I hope this helps! No, we do not use an automated audit management package. The Microsoft Office software suite is used for preparation of audit documentation and reports. Accounting for audit time and costs is handled through an internally developed system. We don’t have any knowledge of AutoAudit. We have been using Audit System 2 (AS/2) from Deloitte. However, since AS/2 is scheduled to be discontinued, we have recently evaluated several products and selected CCH’s ProSystem FX Engagement, which we will implement fully this spring. We were specifically looking for a combination of engagement management software and financial audit methodology (audit programs and workpapers). We were searching for an alternative to maintaining our own standard audit programs and workpapers. We evaluated three products closely: CCH’s ProSystem FX Engagement and Knowledge Coach audit methodology. PPC’s (Thomson Reuters) Engagement CS and audit methodology (smart audit suite) McGladrey’s audit methodology packaged with CaseWare engagement management software. We found there to be advantages and disadvantages to each product, but overall found the CCH pairing to be the best match for us. We found the integration between the engagement management software and audit methodology to be more complete than PPC, we liked the diagnostics offered in Knowledge Coach (tells you Use of Auto Audit 3 State Puerto Rico Comments quickly when portions of workpapers are not complete), and we liked the ease of use of the engagement management software. Overall, we preferred the audit methodology offered in Knowledge Coach, especially for single audit work. Finally, we already subscribe to CCH’s Accounting Research Manager, so adding these products provided some consistency. At the Office of the Comptroller of Puerto Rico we have been using TeamMate since 2003 with Version R6. As of today we have over 450 licenses and users of Version R10 which we acquired last year. We are very happy with the program, especially now that we have all modules (suites) in use. Our office has 20 TeamMate Champions who are up to date in all trainings and help other auditors achieve maximum potential of this tool. TeamMate is used in 150 remote sites across the island. I have listed pros and cons below and have included other information. Should you need any additional information or feedback, please let me know. Pros: Very intuitive and easy to use for the auditors Excellent maintenance and customer service experience Great TeamMate community interaction Annual user conference We like TeamMate's commitment to continually improving their product. This includes improving the seamlessness and flow between Suite Items (Modules) It has a strong built-in security Microsoft Office Suite integration Cons: License & maintenance costs Use of Auto Audit 4