Evaluation Study of Online Continuing Education*
*Poster Presentation, 2005 Medical Education Conference, Saskatoon, SK
Vernon Curran1, L. Fleet1, F. Kirby1, J. Lockyer2, Joan Sargeant3
1Memorial
University, St. John’s; 2University of Calgary, Alberta; 3Dalhousie University, Halifax
MDcme is a not-for-profit Web portal which was developed through a consortium of fourteen
Canadian university-based CME units. Other partners include federal/provincial governments,
medical associations, private industry companies, and physicians. MDcme serves as the home of a
number of MainPro-M1 and MainPro-C accredited Web-based CME courses. MDcme courses
include instructional activities comprising multimedia-enhanced learning tutorials, self-assessment
activities and quizzes, and links to online resources. As part of their online learning experiences,
participants also have the opportunity to interact with a facilitator and other physicians in casebased discussions using an asynchronous computer-mediated conferencing system.
A summative evaluation of fourteen (N = 14) MDcme courses was conducted to review overall
learning outcomes. The purpose of this summative evaluation review was to evaluate physicians’
satisfaction with Web-based CME participation; identify the characteristics of physicians who
participate in Web-based CME activities; and evaluate outcomes of participation in Web-based
CME with respect to participants’ knowledge, confidence and self-reported practice changes.
The majority of respondents reported general satisfaction with MDcme course content and the
design of courses. Respondents found the MDcme courses to be a convenient and flexible means
for participating in CME. An equal number of male (N = 88, 51.5%) and female (N = 83, 48.5%)
participants completed MDcme courses during the study period. The majority (79.1%) were family
physicians. As well, a slight majority (55.7%) reported practicing in a rural area (i.e. an area with a
population of less than 10,000).
Pre and post knowledge and confidence surveys were constructed for each MDcme course.
Participants were asked to complete both surveys immediately prior to and after completion of a
course, respectively. The overall mean pre-knowledge and post-knowledge assessment scores were
3.2 and 4.3 (out of 5), respectively. There was a significant pre to post knowledge increase for seven
of 10 courses at the p <.05 level. The overall mean pre-confidence and post-confidence scores were
14.8 and 20.3 (out of 25), respectively. The pre to post-confidence scores for 8 of 10 courses also
showed a significant difference (p < .05).
A retrospective skill/ability change survey was also constructed for each course in order to evaluate
participants’ self-reported practice change as a result of participation in a Web-based CME course.
Each survey was comprised of 5 practice-based statements which were linked to the learning
objectives of the respective course. Participants were asked to indicate their performance ‘before
participating in CME’ and ‘after participating in CME’ using a Likert scale of 1 = ‘to no extent’ to 5
= ‘to a large extent’. Surveys were distributed approximately 6-8 weeks after course completion.
The pre to post-practice change scores for 12 of 14 courses showed a significant difference (p <.05).
Respondents indicated that computer conferencing was an important feature of the MDcme courses
and enabled opportunities for interaction with colleagues and course facilitators. However, the
quality and level of interaction varied depending on the course and/or the facilitator. The evaluation
findings indicate a greater amount of discussion (as measured by the number of overall discussion
postings) when there were greater numbers of participants. However, while overall discussion
postings appeared to increase with a larger number of participants, the level of individual
participation as measured by individual discussion postings did not. Participant postings were also
found to be associated with facilitator participation. The number of postings made by the facilitator
was related to the level of participation by individual participants in the MDcme.ca courses.
(Editor’s emphasis) This finding suggests that facilitator participation may be an important factor in
enhancing interaction.
Funding and support for this evaluation study was provided from the Atlantic Innovation Foundation (AIF), Atlantic
Canada Opportunities Agency (ACOA), and the Government of Canada.