How MyPsychLab scores your auto-graded writing assignments
The ten MyPsychLab writing assignments are presented in class with a video
and a situation to write about. Student groups, and your instructor, will work in
class to help you organize your ideas and develop your written response. After
class, you write your response in 150 words or more and click “Get Essay Feedback”.
The software then gives you feedback to make your essay better. You are free to
adjust your essay and, when you are happy with it, click, “FINISH: Submit for
grading” Within 8 – 10 seconds you will receive a score. If you are still not happy
with your score, you may repeat the process and resubmit your essay a second time
for grading, and receive the higher of the two scores. You will receive feedback on
each of your submissions to improve your score.
The scoring system:
The writing assessment engine employs latent semantic analysis, a data-based
computational system for lexical content. The database for the prompts in MyPsychLab
consists of hundreds of real student essays written to the prompts and scored by expert
human scorers. MyPsychLab evaluates new essays written to the prompts on the basis of
these data. Thus, the system is anchored in the real-world teaching and learning of
college psychology students across the country.
Each writing prompt has a corresponding scoring rubric that specifies the criteria
on which essays are evaluated. The system applies a score of 1 through 4 on each of five
essay traits. The traits are: Development of Ideas, Focus and Coherence, Organization,
Mechanics, and Voice. Historical content falls under the Development of Ideas trait.
The Focus and Coherence trait evaluates how effectively the essay argues and illustrates
ideas. The other three traits are writing traits. MyPsychLab calculates an overall score
by taking a weighted average of the trait scores. The weights are:
Development of Ideas: 50%
Focus and Coherence: 20%
Organization: 10%
Mechanics: 10%
Voice: 10%
Apart from the scoring, essays receive two forms of feedback. Students may view
categorical feedback on the trait scores for their essays. This feedback describes the
general criteria that apply to the trait score. Students may also view constructive
feedback designed to help them write more effective essays.
Typing your response:
When you log into MyPsychLab to type your response, you will see the screen
below.
There is an introductory paragraph, a paragraph discussing your response and a
link for more information on how your response will be scored.
After you view the video again and are ready to give a response, click “Start”.
You are then directed to the response screen below.
You may type your answer directly into the space, or compose it is a Word
document to check spelling and grammar, then copy and paste it into the response
space.
After submitting your response, you will receive your score as well as
trait-specific feedback within 8-10 seconds. This feedback can help you
improve your response. Once you are happy with your work, click
“Finish: Submit for Grading” in the top right hand corner, and results flow to
the instructor gradebook.
After submitting your response, you will receive your score as well as
trait-specific feedback within 8-10 seconds. This feedback can help you
improve your response. Once you are happy with your work, click
“Finish: Submit for Grading” in the top right hand corner, and results flow to
the instructor gradebook.
Feedback
Questions about the automated grading:
Q. How does the MyPsychLab automatic essay grader work?
The Pearson MyPsychLab auto-graded writing assessments use the Knowledge Analysis
Technologies (KAT) engine, a patented technology based on over twenty years of
research and development. The KAT engine is based on the mathematical approach
known as Latent Semantic Analysis (LSA), which provides a sophisticated computer
analysis of text. The program assesses the total content of an essay as well as the
correlation between the essay's content and that of training essays previously scored by
expert human readers. MyPsychLab assigns a score to each essay based in part on the
similarity of the content of the essay to that of the training essays.
Q. What is the KAT™ engine?
The KAT engine evaluates the meaning of text by examining whole passages. The KAT
engine is based on Pearson's unique implementation of Latent Semantic Analysis, an
approach that infers semantic similarity of words and passages by analyzing large bodies
of relevant text. LSA can then understand the meaning of text in much the same way as a
human reader.
Q. Does the MyPsychLab automatic essay grader use keywords in its evaluation?
A common misconception is that the MyPsychLab automatic essay grader scores by
recognizing keywords. It does not. If two or more different terms accurately describe a
concept, The MyPsychLab automatic essay grader will recognize either or both and score
them equally as well. For example, The MyPsychLab automatic essay grader measures:
"Understanding is very important when you reading something you want to know about"
as very similar in meaning to "Comprehending is essential" even though there are no
literal words in common. Keyword based similarity would find nothing in common
between these two sentences.
Q. What research has been conducted on the Pearson MyPsychLab auto-graded
writing assessments? Pearson has conducted extensive research on the reliability and
validity of the essay component of the MyPsychLab writing assessments. The correlation
between the scoring of this component and human graders has been shown to be as high
or higher than that between two independent human graders in dozens of studies with
over 200 prompts of every type. The correlation and agreement rates of the scores
produced by the MyPsychLab automatic essay grader are better the more expert and
reliable the human scores. Each of the MyPsychLab writing prompts have been tested
and trained using over 400 student essays.
Q. Are students required to type their responses directly into MyPsychLab ?
Students may type their responses directly into MyPsychLab if they choose; however,
many students choose to compose their responses in a word-processing program and then
copy and paste them into the MyPsychLab interface.
Q. How long does it take to score a response?
Most responses are scored within 8 - 10 seconds. Keep in mind that a slow or busy
Internet connection may delay the scoring.
Q. How does the MyPsychLab auto-grader recognize a good essay?
The auto-grader, using the KAT engine, assesses the content of an essay, as well as more
mechanical aspects of writing. When a student submits an essay for scoring, the system
immediately measures the meaning of the essay. It then compares the essay to the
training essays, looking for similarities and assigns a holistic score in part by placing the
essay in a category with the most similar training essays. Analytic scoring occurs in much
the same way. For each trait, the system assesses that trait in the student essay, compares
it to the training essays, and then categorizes the trait in question.
Q. How does the MyPsychLab auto-grader scoring compare to the way instructors
grade writing?
The MyPsychLab auto-grader approach mirrors the way that instructors grade essays. For
example, when instructors evaluate a student's essay, they look for characteristics that
identify an essay as an A or C paper. Their expectations are likely based on their previous
experience as a grader and on criteria for the assignment in question. In other words,
instructors search for a match between the essay itself and the criteria for a particular
grade or score. The Essay Component is trained to mimic this process.
Q. How does the MyPsychLab auto-grader score essays with highly unusual writing
styles?
An essay with a highly unique writing style or unusual construction may receive an
advisory message along with a score. If an essay is off-topic, written in a language other
than English, too brief or too repetitive, a written refusal to write, or otherwise
incomprehensible, a student will receive an advisory that his or her essay can not be
scored. These advisory messages ask the student to discuss the essay and all feedback