Conducting a Job Analysis to
Establish the Examination
Content Domain
Patricia M. Muenzen
Associate Director of Research Programs
Professional Examination Service
What is Job Analysis?
• Set of procedures to used to describe a job or
profession
• Foundation for licensure and certification
examination development
• Key source of validity evidence for the
interpretation of test scores
Job Analysis Process
• Create initial description of the job or
profession
• Develop and administer job analysis survey
• Analyze survey data to identify the
examination content domain
• Create written report of job analysis
procedures and findings
Create Initial Description
of the Job or Profession
• Collect information about the job from a variety
of sources
–
–
–
–
subject-matter expert committee
written documents
observations
individual and group interviews
• Translate information into domains, tasks, and
knowledge, skills, and abilities (KSAs)
Develop Job Analysis Survey
• Develop rating scales for domains, tasks, and
KSAs. Rating scales might include:
– Domains: time spent and criticality to public
protection
– Tasks: how frequently performed, criticality to
public protection, performance at entry-level
– KSAs: frequency of use, criticality, required at
entry level
• Create demographic questionnaire
Administer Job Analysis Survey
• Conduct pilot test of survey instrument
• Select survey sample - typically job
incumbents, but could also include educators,
supervisors
• Distribute survey: invitation, cover letter,
follow-up mailing
Analyze Survey Data
• Review characteristics of respondents
• Calculate means, standard deviations,
frequency distributions for rating scales
• Compare ratings of subgroups based on
demographic variables of interest, such as
practice setting, academic background
Identify the Examination
Content Domain
• Create decision rules to identify key tasks
– for example, tasks rated as (1) performed at entry
level and (2) at least moderately critical to public
protection
• Create decision rules to identify key KSAs
– for example, KSAs rated as (1) used frequently
and (2) necessary at entry level
• Calculate test weightings for domains, key
tasks and/or KSAs using questionnaire data
The Job Analysis
Written Report
• The job analysis written report documents the
process by which the examination content
domain was identified. Thus, it is an important
source of evidence for the content validity of
the exam.
• The examination content domain/test plan
documented in the written report guides all
item writing and test construction activities.