RCPAQAP Training and Competency Assessment Program
FINAL REPORT
JANUARY 2015
Protecting patients – A project funded under the Australian Government’s Quality Use of
Pathology Program
1
Contents
Contents
2
Executive Summary
Background
Project Officer
Learning Management System (LMS)
Development of Modules
Evaluation of the online modules
Summary
Appendix A
3
4
6
7
9
14
20
22
2
Executive Summary
Training and competency assessment of scientific staff within pathology laboratories is critical to
providing a quality service and is an accreditation requirement for pathology services in Australia.
Currently each pathology service prepares training and competency assessment for all key tasks
within the laboratory; however, this is not currently performed in a structured or systematic
manner.
Providing training and competency assessment on a web-based platform and in structured
modules will deliver a standardised approach to training and competency assessment for scientists
and meets both accreditation requirements and national competency based standards, as
developed by the Australian Government (2009). It will also significantly reduce resources needed
to provide the training and reduce duplication of effort in each pathology service.
The modules are designed to support training conducted in the workplace and act as a resource
for laboratories in continuing education. These modules are intended to provide ongoing training
and competency assessment for laboratory personnel, at the basic and senior levels. Certificates
of completion provide evidence of training and competency for employers and regulators.
A new Learning Management System was developed, and included the capability for managers
responsible for workplace training, to monitor staff progress and identify training needs for their
pathology service. This has all been embedded within current RCPAQAP systems to ensure a
streamlined experience for all customers.
The project was launched on 1 September 2014 with five (5) online modules;

Quality Assurance and Quality Control – suitable for multi-test analysers in Chemical
Pathology/Haematology/Immunology – (Basic scientists level)

Snake and snake bite: the role of the pathology laboratories

Pre-Analytical Module - Select safe sites for venepuncture

Cytopathology Module 1 - Anatomy of the Female Genital Tract

Cytopathology Module 2 - Cytological Identification of normal epithelial cells and other
cellular components of cervical and vaginal smears
RCPAQAP has processed 65 enrolments in the first 11 weeks since launching the modules. Early
feedback is very positive with scores of 8-9/9.
3
Background
Pathology services within Australia are required to prepare and deliver training and competency
assessment, of all scientific staff, within their organisation in order to meet accreditation
requirements and provide a quality service. Pathology services usually consist of several
pathology sites, both within metropolitan and rural areas. Senior scientific staff may have
responsibility for as many as 30 different sites/pathology laboratories.
Senior scientific staff (also sometimes referred to as Training Managers) within a pathology
service, are responsible for ensuring scientific and technical staff receive appropriate training and
are assessed for competency. Each pathology service develops training activities/material and
ensures that competency is assessed once the training is completed. This represents a significant
amount of duplication across pathology services in Australia and is a very inefficient use of limited
resources for delivering training and competency assessment.
Training is delivered face-to-face thus requiring staff to regularly travel to each pathology
laboratory to deliver this service. Alternatively, staff may be required to travel to the main
laboratory of the pathology group to attend training for several days or weeks. This method of
delivering training has a significant impact on both staff and budget resources for the pathology
service.
The online training and competency assessment modules are a cost-effective and efficient method
for delivering this service to all staff in all laboratories, regardless of geographical location.
There is a need to provide training in an efficient and easy to access platform, especially for those
scientists in rural areas, where training may be difficult to achieve. Online modules can be
accessed easily and quickly, at any time and best of all, the modules can be undertaken at the
individuals own pace.
Delivery of training and competency assessment via a web-based platform will assist in reducing
the burden of increased workloads on senior scientific staff within pathology services, and also
decrease administrative loads required to meet accreditation requirements. A recent survey of
the pathology workforce (by the Department of Health and Ageing, May 2011) showed that more
than a third of employed scientists were 50 years of age and older, indicating that a large number
of qualified staff would be leaving the profession in the foreseeable future. In addition, over 80%
of senior scientists were based in metropolitan areas. These are the staff who usually deliver the
4
training and competency assessment to junior staff. This survey also found that training of staff
was one of the major secondary issues of concern, after workforce shortages.
The target audience of the program is scientific staff, although modules for Point-of-Care (POC)
and collection of pathology specimens will be aimed at other sections of medicine such as nurses,
doctors and allied health staff that collect pathology specimens or use POC devices. The pilot
module on ‘correctly identifying the patient’ when collecting a pathology specimen was very well
received and requests for this module included allied health staff and collection staff. Similarly the
POC module was popular and interest exists for more modules in this area.
While there are some organisations that currently deliver some training online they do not
address the broad needs of the Australian pathology industry and competency assessment is
limited.
The RCPAQAP online modules are being developed to meet the specific requirements of Australian
pathology services across all disciplines of pathology. The modules are specifically designed to
meet accreditation requirements, Australasian Professional Acknowledgement of Continuing
Education (APACE) and professional development requirements and the national competency
based standards for Australian scientists.
The modules should enhance the training and competency assessment protocols of laboratories
and compliment training that is provided for technical activities. The modules will act as an
adjunct to existing training and will assist in increasing the resources available for the training and
competency of scientists.
Ensuring pathology services are provided by a well trained and competent scientific workforce will
provide both consumers (the patient) and referrers (the doctors) with the confidence that
pathology laboratories will meet their needs and provide a quality service.
Government, as major funders of pathology in Australia, will also benefit from this initiative, as
they will be assured that quality pathology services are being provided by a highly skilled scientific
workforce. This will be particularly relevant in the next 5-10 years when it is expected that a
significant number of highly trained senior scientists plan to retire and pathology services will
experience workforce shortages. It is expected that remaining scientific staff will be under
significant workload pressures as the increase in demand for pathology will come from an ageing
population.
5
Project Officer
Recruitment of an Education Project Manager began in July 2013; a Position Description was
prepared and the position was advertised on Seek.com. Unfortunately, no suitable candidates
were identified that met the selection criteria. The position advertised was a one year contract
and this may have deterred suitable candidates from applying.
RCPAQAP then conducted a search for an internal staff member that may be suitable for this
position. A suitable internal candidate was identified but she had not applied because the position
was a short term contract. This staff member was seconded into the role of Project Officer for one
year with the option of moving back into her previous position after the end of the contract.
Fifin Intan commenced in the role of Education Project Manager on 26 August 2013. Fifin
commenced training to gain proficiency in Articulate, the authoring tool software. Fifin comes
with experience she gained as part of the pilot team that developed the original LMS. She
produced the modules, instructional documentation and Frequently Asked Questions. Fifin is a
great addition to the project and has helped drive the project forward.
6
Learning Management System (LMS)
An LMS Provider was identified and engaged to develop a general LMS for RCPAQAP; RCPAQAP
had well documented requirements for the development of the LMS. In planning the
development of the RCPAQAP LMS, consideration was given to implementing this system within
the current customer portal, where participants/customers currently conduct all activities
including processing sales (the enrolment process), accessing information and documents,
customer relationship management (CRM) system as well as reports. The sale of the online
modules had to be implemented within existing systems, as the target audience is predominantly,
existing customers. Integration with the current system (customer portal) was essential.
The first stage of preparing online modules is the development of a learning management system
(LMS), a system that delivers the online modules and their content. The development of the LMS
was a two-fold process, firstly developing the LMS itself then development for Training Manager
Functionality, within the LMS.
Some technical issues were identified during the early development of the LMS, and these were
related to the customer portal. This resulted in delays in the delivery of the LMS. The LMS
development continued and testing was conducted January-February 2014. Communication with
the US based LMS supplier was difficult and was exacerbated by the time differences between the
US and Australia and this caused further delays. An alternate supplier was engaged to provide the
training manager or administrator functionality required in the LMS. Requirements for the
Administration (Training Manager) functionality were correctly documented.
As a result of the poor communication with the US based team, the development phase had fallen
behind. Some technical issues were causing delays and proved difficult to resolve. The main
difficulty was that the functionality would not work within the RCPAQAP environment. The best
way to move forward with the project was to engage current developers to develop the Training
Manager functionality. Despite some staffing and further technical issues, the delivery of the LMS
with Training Manager functionally was an excellent outcome.
This delay impacted on the delivery dates of the modules and although the impact was minimal it
meant RCPAQAP was not able to meet the deadlines set in the Work Plan submitted to the
Department and the project was extended by six months.
7
The administration section of the LMS provides a powerful monitoring tool for Training Managers.
It allows them to allocate modules and to monitor the progress, completion and pass rates of all
staff under their management. This functionality will assist managers to determine training needs
within their organisation.
The LMS was thus developed within the RCPAQAP customer portal (myQAP) and is displayed as
the ‘Training’ tab.
The LMS and Training Manager Functionality were completed late July 2014 and User Acceptance
Testing (UAT) was conducted August 2014. Although some difficulties were encountered causing
delay in deployment, the end product has met all expectations.
8
Development of Modules
Training and competency assessment delivered in an efficient and standardised manner supports
laboratory requirements, and ensures staff are trained and competent to perform key tasks. The
modules are being developed to meet the requirements of the National Competency Based
Standards for Scientists in Australia (2009), which have been developed by government in
consultation with the pathology industry.
The 2012 pilot results were vital in determining what modules were needed and these provided a
good starting point and direction for RCPAQAP to plan and develop the first modules to launch as
part of this project. Once the modules were decided, subject matter experts/authors were
identified for each module and negotiations ensued.
The content provided by each author was in PowerPoint format. The content provided the basis
for the online module but required significant time and effort to transition from PowerPoint slides
to an interactive module with audio, and finally to the test environment. Continuous
communication between authors and the Project Manager ensured the modules had the look and
feel that each author was aiming for and that the final product met both the RCPAQAP and
author’s expectations.
Identifying and engaging subject matter experts (SME)
RCPAQAP has been in negotiations with several subject matter experts to development content
for the online modules. Feedback from the pilot phase formed the basis of the planning phase and
as well as decisions relating to which modules would meet the need of scientists and would be
received well. In addition, some experts contacted RCPAQAP expressing an interest in working on
this project.
Several SMEs were contacted to develop specialised modules in some areas. An expert from the
University of Adelaide developed the Snake Venom Detection module; an expert from Bond
University developed the QC/QA module; an expert from the Australian Pre-Analytical Network
(APaN) developed the pre-analytical module and a cytology expert developed the two
Cytopathology modules.
Each author was provided with guidelines (prepared by RCPAQAP) to ensure they had a clear
understanding of the expectations of RCPAQAP, and that modules were consistent in terms of
presentation, content elements, assessment element presentation and length of the module. The
9
guidelines also ensured the learning modules were delivered in a structured and consistent
manner and met the requirements of the Competency Based standards.
RCPAQAP experienced delays in receiving the content material on some of the modules.
Predominantly the delay was a result of heavy workloads of the subject matter expert. To ensure
better management of the subject matter experts and the expectations of the RCPAQAP, the
guidelines will be strengthened to include deadlines for each stage of the process, development of
the module, beta testing, and adjustments to various elements including the assessment element
of the module. This will provide a more realistic guide for the timelines involved in developing and
producing an online module from start to finish. Consideration is also being given to
implementing service level agreements with each new SME.
Engaging an external provider to develop modules
Engagement of an external provider to produce two online modules was conducted in late 2013.
The use of an external provider to develop modules enabled RCPAQAP to utilise experts using high
quality graphics and animations to enhance the quality of the module and produce a highly
professional product. The external supplier has unlimited resources both in staff expertise and
software products to produce high resolution graphics and audio content.
The Snake Venom and Pre-analytical modules (safe site for venepuncture) were most suited to the
external provider as there were significant graphics utilised in these modules. The Snake Venom
Detection module was ideal for this development as high resolution images or videos of the
various snakes were added, as well as high quality animations showing effects on the body and
explanations on how test results are impacted.
An external provider was selected and worked with the Project Officer to ensure optimum results.
The Project Officer monitored progress of the module development work and testing. The
modules were completed on time and to a high standard.
Two modules were chosen to be developed by an external provider for two reasons: firstly to
facilitate a faster production time of the modules and secondly to test whether a module
developed externally by experts was significantly superior to those produced in house. The two
modules were of a high quality and consideration will be given to the utilisation of an external
provider in future.
10
Allocation of professional development points and recognition of modules
During the pilot phase of the training and competency assessment project in 2012, feedback was
gathered which provided significant input into the development of the current modules. One
major factor that was important to scientists was that they were awarded professional
development points for successful completion of the modules.
The Australasian Professional Acknowledgement of Continuing Education (APACE) scheme
recognises continuing education and training that contributes to professional development of
scientist. Further information can be found at the Australasian Professional Acknowledgement of
Continuing Education (APACE) webpage in the ‘Overview’ link. APACE is administered by the
Australian Institute of Medical Scientists (AIMS), the professional association representing all
disciplines of pathology.
Organisations which produce training material/activity are required to submit the material to the
APACE committee, a panel of experts that allocate the relevant professional development points
to the material/training activity. As a guide, APACE allocates 1 APACE point to a one hour online
learning activity. Where that activity also has a competency/assessment element a further 5
APACE points may be allocated.
RCPAQAP submitted the following three modules to the APACE committee for review and
allocation of professional development points in August 2014;

Quality Assurance and Quality Control (Basic scientists level) – suitable for Chemical
Pathology, Haematology and Immunology scientists

Snake and snake bite: the role of the pathology laboratories

Pre-Analytical Module - Select safe sites for venepuncture
Each module was awarded 6 APACE points upon successful completion of the module. Each
module has been assigned a pass mark by the subject matter expert.
The two Cytopathology modules were reviewed by the Australian Society of Cytology as follows:

Cytopathology Module 1 - Anatomy of the Female Genital Tract

Cytopathology Module 2 - Cytological Identification of normal epithelial cells and other
cellular components of cervical and vaginal smears
11
These modules are eligible for professional development points according to the Australian
Society of Cytology (ASC) Continuing Education for Cytologists (CEC) Program for non-medical
members. Refer to Category 2 “Computer based learning”.
All five modules were successfully awarded the highest possible professional development points
by the relevant association/society.
NATA
National Association of Testing Authorities (NATA) recognition of the online modules was sought,
as this was considered an important issue by laboratory scientists as indicated by the feedback
during the pilot phase. NATA, as Australia’s accreditation body, cannot endorse any particular
product and as such cannot endorse the modules prepared by RCPAQAP.
While each pathology service must decide what type of competency assessments are appropriate,
NATA does acknowledge that online training programs and competency assessments may have a
role in the overall training & competency procedures that can be utilised to demonstrate technical
competence. Such activities may also allow pathology services to demonstrate the ability to
comply with ISO and NPAAC requirements.
Many pathology services utilise online products in the assessment of theoretical and knowledge
based competency assessments and the use of these are assessed as part of the accreditation
process.
It is noted that the online modules produced by RCPAQAP have been prepared by highly
regarded experts within Australian pathology.
Any On-line competency product used by pathology services needs to be assessed in the context
of suitability in meeting that facility’s training and competency needs. Further online competency
assessment is usually only part of the process and other technical demonstrations are generally
also required.
Other work activities
While work was in progress on the LMS and associated functionality, a significant body of work
was completed on;

developing the Training and Competency website
12

establishing an enrolment process for this new customer group

developing a process to assign the modules to the relevant trainee

preparing instructions and other relevant documentation e.g. enrolment forms

setting prices and fees
Launching the modules
Preparations for the launch of the modules included a significant amount of work and included
developing the website, establishing an enrolment process for this new customer group,
developing a process to assign the modules to the relevant trainee and preparing instructions and
other relevant documentation e.g. enrolment forms and setting prices.
The five modules developed for this project were launched on 1 September 2014 at the opening of
the 2015 RCPAQAP enrolments;

Quality Assurance and Quality Control – suitable for multi-test analysers in Chemical
Pathology/Haematology/Immunology – (Basic scientist level)

Snake and snake bite: the role of the pathology laboratories

Pre-Analytical Module - Select safe sites for venepuncture

Cytology module 1 - Anatomy of the Female Genital Tract

Cytology module 2 - Cytological Identification of normal epithelial cells and other cellular
components of cervical and vaginal smears
13
Evaluation of the online modules
Evaluation of the online training and competency assessment modules consists of several activities
including monitoring the sales of each module, seeking feedback from customers who complete
the modules and assessment by professional bodies allocating professional development points.
All of the above evaluation techniques are currently employed by RCPAQAP to evaluate and
determine the usefulness and appropriateness of each module when compared to expected
outcomes.
Customer feedback is vital in determining what modules are useful and relevant for scientists.
This feedback will form the basis upon which decisions are made in the development of future
modules to ensure customer’s needs are being met. The feedback will also serve to identify any
amendments or updates that are required to ensure the content remains consistent with any
changes in standards and industry. The modules will be reviewed annually to ensure any required
changes are implemented.
Marketing and advertising the modules
A concerted and planned marketing strategy to launch the modules commenced in the first week
of September 2014. It included bulk email notification to all customers in the RCPAQAP customer
portal using a Flyer to announce the launch of the modules. Refer Appendix A.
The customer portal was targeted, as this is the most visited section of the RCPAQAP website.
Information on the modules was also available on the public RCPAQAP website under the
Discipline tab with the page titled ‘Training and Competency Assessment’, to capture visitors who
are not currently enrolled in RCPAQAP programs and may be seeking these services.
The Flyer was also distributed to international agents who manage various regions overseas and
include large numbers of RCPAQAP enrolled participants. In addition, advertising of the modules
with major scientific bodies is in progress; Australian Association of Clinical Biochemists (AACB),
Australian Institute of Medical Scientists (AIMS) and Australian Society of Cytology (ASC).
Future marketing plans include advertising the online modules on professional bodies’ websites or
newsletters. The modules will also be promoted at major pathology/scientific conferences such as
Pathology Update.
14
RCPAQAP advertises the modules in the company Product Catalogue which is available in the
customer portal as well as the company website. The modules are also actively promoted to
international markets through agents collaborating with RCPAQAP.
Feedback from customers using the modules
A feedback survey has been prepared seeking feedback on various aspects of the modules,
including relevance of material, navigation of the module, appropriateness of the content and
improvement suggestions. Customers who complete the modules are invited to participate in the
feedback survey.
Feedback will be most useful not only to evaluate the modules but to assist with planning future
modules. Some of the feedback from the pilot, such as including assessment elements throughout
the module rather than a single quiz at the end of the module, was a significant improvement and
ensured the learner was engaged for the duration of the online module. This format also served
to reinforce the learning material.
In the short time since the modules were launched, the feedback has been very positive and the
modules have been highly rated. Despite a good number of sales, only a small number of
customers have completed their training and only a small percentage of those completed the
feedback survey (n=8), which is disappointing. Below is an example of the some of the survey
feedback;
Navigation & Instructions
15
Learning Objectives
16
Content & Presentation
17
Future Planning
Comments
‘Easy to follow’
‘These online training and assessments were the most helpful I have had to do, I wish there were
more of this standard’
‘Easy to use and informative’
‘User friendly’
Although the feedback was from a small number of customers, it is worthwhile to gauge customer
satisfaction.
18
Monitoring sales of each module
Monitoring the sales of the modules is critical in determining not only the usefulness and
appropriateness of the modules but also will assist RCPAQAP in determining if the modules have
been priced at a level the market deems appropriate.
Sales of training modules will be measured and monitored on a regular basis to evaluate the
usefulness and appropriateness of each module when compared to expected outcomes. Modules
are being developed to meet participant needs, so it is imperative to monitor sales to determine if
those needs are being met.
Sales will be monitored throughout the year, and where expected sales figures are not met,
follow-up will include communication with participants to determine the main cause of poor sales,
i.e. inappropriate or perceived lack of need for a particular module.
Since the launch of the modules (in the first three months) sales are as follows;

Quality Assurance and Quality Control – suitable for multi-test analysers in Chemical
Pathology/Haematology/Immunology – (Basic scientist level), 32 modules sold

Snake and snake bite: the role of the pathology laboratories, 9 modules sold

Pre-Analytical Module - Select safe sites for venepuncture, 15 modules sold

Cytology module 1, 2 modules sold

Cytology module 2, 2 module sold
19
Summary
The support from the Quality Use of Pathology Program has enabled RCPAQAP to develop a
Learning Management System which delivers online Training and Competency Assessment
modules for scientists, with Training Manager capability included. This facilitates monitoring of
training and competency assessment of staff and identifies training needs within a pathology
service.
The five modules launched 1 September 2014 have been well received with good initial sales to
date and positive feedback. RCPAQAP continues to receive a significant number of inquiries about
the online modules and with further advertising and marketing of these products, sales are
expected to increase.
In addition to the five modules initially launched, other modules are currently being developed for
QC/QA and Pre-analytical modules for senior scientists and should be available in the first quarter
2015. Negotiations are also underway to develop modules for Transfusion and Microbiology.
Negotiations with an external organisation are almost complete. This is to develop a Haematology
Morphology course (Red cells, White Cells and Platelets) suitable for both junior and senior
scientists, as well as modules on Paediatric morphology for senior scientists. This course will be
supplemented with Case Studies to further assess competence on an ongoing basis.
Problems Encountered
During the development of the LMS within the RCPAQAP customer portal, delays were
experienced with the software product selected to establish the LMS functionality. This set back
resulted in a 6 month delay in the delivery of the project. The main issue encountered was the lack
of expected, off-the-shelf capability of the software with the licenses RCPAQAP held (despite the
functionality being available normally). The solution offered by the software team was a total redevelopment of the LMS and the required Training Manager functionality. This solution would
have been costly and time consuming. RCPAQAP was able to identify an alternate solution and
developed the functionality with an alternate software development team.
Some delays were also experienced with subject matter experts unable to deliver the content as
per agreed deadlines. It is acknowledged that many of the experts with whom RCPAQAP
collaborates with are very busy people and the work they undertake for RCPAQAP is usually
completed in their own time. This will require better management in future, possibly with the use
of service level agreements which include requirements to meet deadlines for the provision of the
20
content and for beta testing of the online modules prior to release. An alternative may be to
request documented work plans and even consider an alternate subject matter expert in some
cases, where a particular expert’s schedule is too restrictive.
Lessons Learnt
In relation to the delays and problems encountered with software development, in future
RCPAQAP will undertake a more rigorous and thorough assessment process of new software, prior
to committing to any work activities.
Subject matter experts present a particularly unusual and delicate management issue, as these
individuals have busy full time jobs, leaving them little time to dedicate to extracurricular activities
such as collaboration with RCPAQAP to develop online training and competency assessment
module or modules. In future the RCPAQAP may consider engaging SMEs under service level
agreements to endure the timely delivery of completed modules.
Final Comments
This is an exciting venture for RCPAQAP that will help to ensure laboratory training managers
provide quality pathology services delivered by a trained scientific workforce, which will ensure
confidence to patients, requesters of pathology and government. The online training and
competency assessment modules also address the current workforce shortages by providing,
efficient and easy to access training material that compliments on the job technical training. It
focuses on meeting the Competency Based standards for scientists in structured and standardised
manner and offers continuing professional development credits.
Easy access to the online modules regardless of geographical location is especially important for
rural areas where training can be difficult to access. The online solution is particularly ideal in
these cases.
21
Appendix A
Training and Competency Assessment –2014
The Training and Competency Assessment Project was successfully piloted in 2012 and further
modules have been developed since then.
The online Training & Competency Assessment modules are intended to support laboratory
technical training and provide a resource for continuing education. RCPAQAP is working with
experts to ensure the modules meet participants learning needs. Web-based delivery offers
flexible learning where learners can access the modules anywhere and at anytime.
Modules available for 2015 are;

Chemical Pathology – Quality Assurance and Quality Control (Basic scientists level) *

Snake and snake bite: the role of the pathology laboratories *

Pre-Analytical Module - Select safe sites for venepuncture *

Cytopathology Module 1 - Anatomy of the Female Genital Tract **

Cytopathology Module 2 - Cytological Identification of normal epithelial cells and other
cellular components of cervical and vaginal smears **
* These modules are eligible for Australian Institute of Medical Scientists (AIMS) APACE
accreditation - 6 APACE points upon successful completion of each module.
**These modules are eligible for the Australian Society of Cytology (ASC) Continuing Education for
Cytologists (CEC) Program for non-medical members. Refer to Category 2 “Computer based
learning”.
RCPAQAP acknowledges the Department of Health and Ageing for its support of this project via
funding under the Australian Government’s Quality Use of Pathology Program.
Participants will be able to enrol in these modules via the Enrolments tab in the myQAP (customer
portal) and then view the modules in the Training tab once payment is received.
For further details please Contact RCPAQAP:
Phone (+ 612) 9045 6000
Email: elearn@rcpaqap.com.au
22