Medical Writing Certification:
Where Are We Now?
Thomas Gegeny, MS, ELS (Chair)
Karen Potvin Klein, MA, ELS (Chair-Elect)
2012 Annual Conference Open Session #38 – Medical Writing Certification: Where are we now?
1
Today’s Objectives
• Provide a brief history of AMWA’s certification-related
activities since the 1990s
• Describe recent and ongoing activities by AMWA to
develop a medical writing certification program
• Provide information on the current processes and
planned timelines
• Review results from AMWA’s recent Job Analysis Survey,
with data from more than 1,000 respondents
• Share next steps and opportunities to contribute
• Conduct question-and-answer session with panelists
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
2
Brief History of AMWA’s Initiative
on Medical Writing Certification
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
3
Moving Toward Certification
1996: AMWA Certification Task Force
•
•
Issued preliminary report
Recommended to continue research
2007−2008: Long-Range Planning Committee Report
•
•
Noted perceived weakness for AMWA as “lack of certification in a competitive market”
Recommended development of credentialing process be added to AMWA’s professional
development offerings
2009: Medical Writing Certification Task Force
•
•
•
Included AMWA members who also represented BELS, DIA, and ISMPP
Researched certification programs of allied groups (BELS, ACCME, ISMPP, CSE,
others); survey of employers; AMWA membership survey (2010)
Recommended that AMWA pursue certification
2011: Certification Commission established by
AMWA Board of Directors
1990s
2000s
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
2010s
4
2010 AMWA Membership Survey
No (34.4%)
Is professional certification
with a competency examination
desirable for the medical
communication profession?
Yes (65.6%)
N = 1339
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
5
Certification vs Certificate
Certification is a credential granted to an individual after the individual has met
established criteria for competency, usually through an examination.
A certificate is a document attesting to an individual’s participation in an
educational program designed to help him or her fulfill specific learning objectives.
Certification
Certificate
Granted after an assessment process
Granted after an educational process
Usually requires experience
Open to individuals regardless of
experience
Granted by associations or institutions
Granted by an independent, standardsetting body
Indicates knowledge and/or competency Indicates completion of course work
through an examination
AMWA offers an educational certificate
Note: Table based in part on table available at http://www.cfre.org/pdf/Certificate-vs-%20Certification.pdf. Accessed Sept10, 2012.
6
Education & Certification
• AMWA’s educational programming is core to its
mission and will continue to be a cornerstone of its
activities. AMWA continues to invest in its
educational programs and resources.
• Medical Writing Certification will be a separate
activity with dedicated budget and resources. It will
not be directly tied to AMWA educational certificates
(though these can be one means of preparing for an
examination).
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
7
Medical Writing
Certification Commission
•
Overview
– Established to initiate, evaluate, maintain, and oversee the credentialing program for
medical writers
– Seeks to represent the diversity that exists within the profession and serve as a voice for
stakeholders who have an interest in maintaining high standards in medical writing
•
Authority and Responsibility
– Charged with overseeing matters pertaining to certification of medical writers
– Makes recommendations on certification policy and quality improvement
– Oversees the application, examination, recertification, and appeals processes
•
Mission
To maintain a quality certification program for the medical writing profession by:
1. Overseeing the certification process and monitoring policies
2. Developing and revising the certification examination through exam assembly
committees and item development
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
8
Current Composition of the
Certification Commission
Chairperson
(2-year term)
ChairpersonElect
AMWA Executive
Director
(ex officio)
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
At-Large
Members (4+)*
Representative
from AMWA
Leadership
(ex officio)
*Number may change over time;
serve 2-year staggered terms 9
How Will the Commission Fit Into
AMWA’s Organizational Structure?
•
•
•
Fulfill organizational mission
(education and professional development)
Oversight of AMWA business activities
(finances, legal, etc)
Full staff support (programs,
administration, etc.
AMWA Leadership
(Officers, Board of
Directors)
AMWA
Certification
Commission
•
•
•
•
•
Fiscal oversight by AMWA
Separate budget (revenue, expenses)
Infrastructural resources (eg, technology)
Certification-specific staff support
Outside governance structure of AMWA
Chapters
Departments and
Committees
Associated
Committees
Commission and
subcommittee
membership will
include but not be
limited to AMWA
members
Note: Once the certification process is in place,
Commission member candidates will be
elected annually, with staggered 2-year terms.
10
What Has the Commission
Accomplished So Far?
Commission
Formed:
•
•
•
•
•
•
•
•
•
B. Bonk
D. Clemow
B. Gastel
T. Gegeny
S. Hudson
K. Klein
S. Krug*
M. Mallia
B. Snyder*
Profession-related materials and
references gathered to develop a draft
listing of knowledge, skills, and abilities
(KSAs) for competent medical writers:
• Job descriptions
• Educational curricula
(including AMWA wkshps)
Final KSA review
• Competency model (DIA)
• Articles, other publications (commission, panel,
& volunteers)
• Guideline documents
*ex officio
Job Analysis Panel teleconference:
•
•
•
•
Review survey results with vendor
Give input for final report
Verify next steps to communicate data
Plan for exam development
KSA Survey
Data Report
finalization
Review & BETA test
of online survey
2011
2012
Dec
Jan
First
Telecon
Vendor contracted
KSA list review
(volunteers)
Draft KSA review
(volunteers)
Feb
Mar
Job Analysis
Panel formed to
finalize KSA list
and design survey
Apr
Survey
May
Jun
Job Analysis Panel
meeting (2 days):
Jul
Aug
Sep
Oct
Annual Conference
Open Session (OS-38)
Six commission members
(DC, BG, TG, KK, SK, MM)
and L. Alexander, N. Taylor, F. Witte
11
What Lies Ahead?
Development of policies and procedures:
• Eligibility and application for exam
• Governance for commissioning body
• Resources and exam preparation materials
New members join
Certification Commission
(staggered terms)
Nov
2012
2013
Dec
Jan
Item
Writing
Feb
Mar
1st exam administration:
Late 2013/Early 2014
• Item analysis
• Cut score study
• Raw score determination
• Final score roster
• Draft test form
• Map items to
content outline
Apr
May
Jun
Jul
Aug
Sep
Oct
Article for AMWA
Journal, other venues
•
•
Call for volunteers (Sept 2012)
Enlisting of Subject Matter Experts (SMEs)
to assist with item writing for exam
Test form review
and QC
12
So…Where Are We Now?
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
13
Why Was A Job Analysis
Survey Done?
• A job analysis survey is a part of the certification process that helps
define medical writing practices
– Provides uniform standard
– Identifies those who meet the standard
– Ensures minimum level of competence
• Specifically, the job analysis establishes a source of evidence from
input gathered across a large number of survey participants
– Defines the profession and is defensible
– Establishes foundation of testing program
• Establishes assessment content
• Provides link between assessment and practice
• Provides legal defensibility
– Determines test design
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
14
Who is the certification vendor and
how were they chosen?
• A competitive Request for Proposals (RFP) process was conducted in 2011
• Schroeder Management Technologies (SMT) was selected from among 3
potential vendors (considering price, technology, and experience)
– SMT was founded in 1995 by testing industry leader Dr. Lee Schroeder
– SMT employs more than 30 testing industry professionals with specialized
affiliation that include American Statistical Association (ASA), the Association
of Test Publishers (ATP), the American Educational Research Association
(AERA), the National Council on Measurement in Education (NCME), the
Council on Licensure Enforcement and Regulation (CLEAR), and the Institute
for Credentialing Excellence (ICE)
• SMT offers a variety of special exam development and implementation
services including

Job Analysis

Item Writing

Exam Assembly & Administration

Item Banking

Item Analysis

Standard Setting & Scoring
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
15
Job Analysis Survey
• Provides information related to the expertise and
expectations of a (minimally) competent medical writing
professional
• To be included, a task or KSA must be weighed with the
following considerations:
–
–
–
–
–
Is it important to competency across all medical writing settings?
Is it performed by “practitioners” regularly?
Is it clearly written?
Are there errors or omissions?
Is it assigned to the appropriate area (“domain”) within the
outline?
– Is it testable?
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
16
Establishing Domains: Looking to
AMWA’s Definition of the Profession
•
AMWA’s definition of a medical communicator
( http://www.amwa.org/default.asp?id=420 )
Medical communicators write, edit, or develop materials about medicine and
health. They do this by gathering, organizing, interpreting, and presenting
information in a manner appropriate for the target audience.
Materials prepared by medical communicators (collaborating with others, or
working independently) include the following:
– Patient education brochures, news articles, Web content, and books for the general
public
– Journal articles and continuing education monographs for health care professionals
– Regulatory documents for government agencies
– Grant proposals for research scientists and institutions
– Sales training and marketing materials for the pharmaceutical industry
Professional medical communicators have communication expertise, awareness
of ethical standards, and health care knowledge.
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
17
Five Medical Writing Domains
Gathering
Organizing
Evaluating
ethics
Interpreting
Presenting
18
Medical Writing Knowledge, Skills,
& Abilities (KSAs) by Domain (1)
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
19
Medical Writing Knowledge, Skills,
& Abilities (KSAs) by Domain (2)
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
20
Medical Writing Knowledge, Skills,
& Abilities (KSAs) by Domain (3)
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
21
Medical Writing Knowledge, Skills,
& Abilities (KSAs) by Domain (4)
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
22
Medical Writing Knowledge, Skills,
& Abilities (KSAs) by Domain (5)
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
23
Do The Domains/KSAs Apply Across the
Spectrum of Medical Writing Careers?
Example of Medical Writing Environments
Domain/KSA
Patient
Info/Education
Pharmaceutical,
Regulatory
Clinical or Basic
Research
Continuing Education
(CME, etc)
Gathering:
interviews with clinical
experts, patient
educators, dieticians,
patients, etc.
medical reviewers,
statistical support,
legal/regulatory
reviewers
principal
investigators,
co-authors,
biostatisticians
authors/contributors,
review committees
(content/activity
accreditation)
ensure level of
information is suitable
and useful for patients
and/or caregivers
evaluate content for
appropriate level of
evidence and
suitability for
intended context
ensure collected
references are
relevant to line of
evidence and
support logic
confirm content meets
educational objectives
for intended health
care audience
factsheet, Q&A, news
article format;
health literacy
recommendations
clinical trial report,
common technical
document, product
label; FDA
IMRAD paper, case
report, poster;
ICMJE, HIPAA,
CONSORT
needs assessment,
learning objectives,
slide kits, test
questions; ACCME
summarize key
information on
treatment/condition;
inform patients
summarize key
safety and efficacy
conclusions
communicate how
study hypothesis
was or was not
supported by data
integrate clinical
guidelines and current
evidence to address
educational needs
Elicit information from
collaborators, stakeholders
Evaluating:
Evaluate collected
information with regard to
content, audience, context
Organizing:
Apply templates and
guidelines to documents
Interpreting:
Synthesize and integrate
information
Presenting:
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
Develop clear, concise prose
A
L
W
A
Y
S !
24
Medical Writing Job Analysis:
Survey Results Summary
July 2012
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
25
Survey
Distribution
Job
Analysis
Survey and Response
April 2012 – Call for participation sent directly* to approximately
14,200 potential respondents:
• AMWA (5283 current, 4109 previous members)
• Drug Information Association, DIA (2687 members, specifically the
medical writing special interest area)
• Board of Editors in the Life Sciences, BELS (988 members)
• Society for Technical Communication, STC (207 members)
• International Society for Medical Publication Professionals, ISMPP (930
members)
Available online for approximately 6 weeks (May 28-July 8, 2012).
A total of 1,177 individuals completed the survey. (Return rate =
8.0%, an average return rate for an unsolicited survey.)
*Note: Survey was also announced via social media, e-mail lists, and personal networks.
26
Survey
RatingSurvey
Scales
Job
Analysis
The following performance and importance frequency rating
scales were used for the job domains section of the survey:
Performance:
0 = Not Performed
Importance:
1 = Of No Importance
2 = Of Little Importance
3 = Moderately Important
4 = Very Important
5 = Extremely Important
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
27
Demographics:
Experience
Job
Analysis Survey
40
38.0
35
Percentage
30
27.6
25
21.3
20
15
13.1
10
5
0
0-5
6-14
15-24
25+
Years of Experience
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
28
Demographics:
Region
Job
Analysis Survey
50
45
44.1
40
Percentage
35
30
25
20.9
18.7
20
16.3
15
10
5
0
North
South
Midwest
West
U.S. Region
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
29
Demographics:
Outside US
Job
Analysis Survey
90
80
79.22
70
Percentage
60
50
40
30
20
11.84
10
0.34
3.07
3.49
0.43
0.17
1.11
0.34
Oceania
Other*
0
U.S.
Africa
Asia
Cntrl. or So.
Am.
Europe
Middle East No. Am.
(non-U.S.)
All Regions
* Four "Other" respondents provided: China, Entire US (independent writer), India, Many US Clients, several in UK, Japan, Eastern Europe
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
30
Demographics:
Work Setting
Job
Analysis Survey
35
28.7
Percentage
30
25
22.8
20
15
10
5
13.4
8.7
7.1
2.6
4.2
3.2
4.4
5.0
0
Work Setting
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
31
Demographics:
Primary Role
Job
Analysis Survey
50
40
36.1
Percentage
32.9
30
20
11.8
11.1
10
3.2
4.9
0
Writer
Editor
Writer and
editor
Researcher Supervisor or
manager
Other*
Role
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
32
Demographics:
Level of Education
Job
Analysis Survey
Data from 2010 AMWA
Membership survey:
50
40
Bachelor’s: 27.0%
Master’s: 29.8%
PhD: 26.5%
MD or DO: 3.4%
PharmD: 3.6%
37.1
Percentage
30.7
30
21.4
20
10
4.3
0.8
3.1
0.4
0.3
1.9
0
Education
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
33
Demographics:
Field of Study
Job
Analysis Survey
50
41.7
Percentage
40
30
20
13.5
12.2
11.9
10
5.1
6.4
5.1
4.0
0
English or other Journalism or
humanities communication
Medical,
technical, or
professional
writing
Health care
professions
Life sciences
Other natural
sciences (e.g.,
chemistry,
physics)
Other social
sciences
Other
Field of Study
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
34
Demographics:
Job
Analysis Survey
Professional
Organizations
80
74.0
70
60
Percentage
50
40
30
20
14.5
11.6
10
10.4
5.6
1.3
8.1
3.1
0.3
3.3
3.1
1.9
0.5
0
Memberships
Note: More than one choice was allowed
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
35
Demographics:
Job
Analysis Survey
Professional
Designations
20
Percentage
12.6
11.5
10
4.8
1.1
0.5
0.3
0.2
0
CHES
CMPP
CPTC
ELS
ELS(D)
RAC
Other
Designation
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
36
Demographics:
Sex
Job
Analysis Survey
Male, 20.1%
Data from 2010 AMWA
Membership survey:
Female, 79.9%
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
Female: 84.4%
Male: 15.6%
37
Demographics:
Age
Job
Analysis Survey
Data from 2010 AMWA
Membership survey:
40
Median age: 47 years
29.8
30
28.1
Percentage
24.0
20
11.6
10
5.1
1.4
0
20s
30s
40s
50s
60s
70+
Age Range
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
38
Demographics:
Ethnicity
Job
Analysis Survey
100
90
87.3
80
Percentage
70
60
50
40
30
20
7.0
10
1.7
1.6
1.2
1.1
0
White, NonHispanic
Black or
African
American
Hispanic or Asian or Indian Two or more
Latino
subcontinent
races
Other*
Ethnicity
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
39
Survey
Coverage
of Medical Writing
Job
Analysis
Survey
Inadequately,
6.4%
Completely,
32.2%
More than 93% of the
respondents indicated
that the survey either
completely or adequately
described the critical
tasks required of
competent practice of the
medical writer.
Adequately,
61.4%
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
40
Scale
and Respondent
Job
Analysis
Survey Reliability
Cronbach’s coefficient alpha reliability estimate was calculated to
evaluate the internal consistency of the instrument (Crocker &
Algina, 1986).
This statistic is bound between 0 and 1, with values closer to 1. This
indicates higher instrument reliability, showing that the tasks work
well together to measure the knowledge, skills, and abilities
requisite in the medical writing field.
The coefficient alpha was calculated using the SPSS© statistical
program, version 17.0.
For this survey, the coefficient alpha reliability estimate was 0.97
for the importance statements. This estimate indicates that the
survey is a reliable instrument.
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
41
Mean
ScoringSurvey
of Item Importance
Job
Analysis
1
2
3
4
5
Rating Scale
Of No Importance
Of Little Importance
Moderately Important
Very Important
Extremely Important
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
42
Mean
ScoringSurvey
of Item Importance
Job
Analysis
Lowest-ranking
items still fall
between moderately
to very important
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
43
Mean
ScoringSurvey
of Item Importance
Job
Analysis
53 out of 69 line
items were ranked at
least “very important”
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
44
Mean
ScoringSurvey
of Item Importance
Job
Analysis
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
45
Subgroup
analyses
Job
Analysis
Survey
Subgroup analyses were run based on the demographic questions related to Years
Experience, Geographical Region, and Work Setting. Significant differences were
identified at P=0.05 and P=0.01 levels.
Subgroup
Category
Years Experience
Geographic Location
Work Setting
# of Significant # of Elements
Differences
within Sign. Diff.
26
6
53
20
5
24
Lowest Mean Rating
within Sign. Diff.
3.82
3.71
3.59
These findings indicate that although the differences may be significant, the
lowest average rating of 3.59 is within the “Moderate Importance.”
While these data help clarify the relative importance of items to different
subgroups, overall ranked importance of items is still high enough to warrant
inclusion in examination design.
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
46
Summary
(1)Survey
Job
Analysis
• As part of an initiative to pursue and establish a certification credential
for professional medical writers, AMWA conducted a survey to assess
the importance of knowledge, skills, and abilities (KSAs) for those who
practice in the field.
• The results of this survey will be used to help develop a content outline
and relative weighting for an examination. These results also further
characterize the type of knowledge and skills used by medical writers
across the spectrum of the profession. More detailed results will be
published in the December 2012 issue of the AMWA Journal.
• The examination will test minimal competency (not aptitude or skill
level) and may accompany the submission of written materials as part
of the application process or other criteria such as time spent in field.
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
47
Summary
(2)Survey
Job
Analysis
• AMWA’s Certification Commission is working according to a
timeline, with the aim to have the first examination developed by
early 2014. The commission will maintain responsibility over
certification as a semi-independent governing body.
• As opportunities to contribute expand over time, volunteers are
encouraged to become involved and offer any subject matter
expertise (whether in areas of medical writing or in exam
development).
• AMWA’s leadership in this area helps further define the medical
writing profession and establishes both a roadmap for those
entering the profession and a benchmark for baseline competency
expectations, regardless of area of specialization.
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
48
Thanks
& Acknowledgment
Job
Analysis
Survey
Thanks to the following individuals who reviewed this presentation for
content, clarity, and editorial accuracy:
• Lori Alexander, MTPW, ELS
• David Clemow, PhD
• Barbara Gastel, MD, MPH
• Doug Haneline, PhD
• Sue Hudson
• Marianne Mallia, ELS
• Barbara Snyder, MA
• Nancy Taylor, PhD, ELS
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
49
Q&AAnalysis Survey
Job
Questions?
Additional Panelists
Barbara Gastel, MD, MPH
Texas A&M University
Marianne Mallia, ELS
Scientific Publications, Texas Heart Institute
2012 Annual Conference Open Session – Medical Writing Certification: Where are we now?
50