Paul D. Camp Community College
Shock-Tucker Assessment of General Education (STAGE)
2011 Degree Graduates vs. 1st Time Students
INTERVENTION:
The intervention is to assess all general education goals annually and to identify areas of value-added
skills and/or knowledge, as well as, areas needed for improvement.
INTRODUCTION:
The search for a viable measurement of the general education of college students began more than a
decade after the Virginia General Assembly mandated the measurement of student outcomes. Often,
multiple measures are being used and may include student opinion surveys, analysis of writing samples,
and objective tests.
One measure of general education that was adopted by many of Virginia’s community colleges is an
objective test call Shock-Tucker Assessment of General Education (STAGE). This measurement was first
developed by faculty at Mountain Empire Community college in response to many colleges’ displeasure
with commercial objective tests. External validation and analysis of STAGE’s reliability turned this test
into a collaborative effort among many of the VCCS colleges. Over the past few years the Virginia
Community College System (VCCS) has modified its general education goals. As a result, PDCCC has
made some slight modifications to its quantitative reasoning and information literacy items.
The College’s general education goals include: (1) communication, (2) critical thinking, (3) quantitative
reasoning, (4) information literacy, (5) cultural/social, (6) scientific reasoning, and (7) personal
development. The STAGE test is used to assess all of the gen-ed objectives. It is, however, only one of
the assessments instruments used to analyze general education skills. The purpose of this assessment is
to show value-added.
Unlike nationally developed tests, STAGE was based on the VCCS definition of general education.
METHODOLOGY:
Paul D. Camp Community College plan is to engage in a cross-sectional approach to determine the value
that is added to degree graduates in all seven general educations areas identified by the Virginia
Community College System (VCCS). These include: (1) communication, (2) critical thinking, (3)
quantitative reasoning, (4) information literacy, (5) cultural/social, (6) scientific reasoning, and (7)
personal development. Value added will be measured through pre and post assessments of the STAGE
test.
A cross-sectional approach, which permits for college-level flexibility in administration, was selected to
demonstrate the value added to PDCCC’s degree graduates while accommodating the non-linear course
taking and enrollment patterns, and varied times-to-degree of community college students. The cross-
sectional approach will be used to draw inferences about the differences, between pre and post
populations assessed and not changes in individual student learning.
In spring 2011, Paul D. Camp Community College (PDCCC) administered the STAGE test to the 2011
degree graduates. It began testing new student in the SDV classes fall term as a pre-test group consisting
of program-placed students in their first semester of college work. The same STAGE instrument was
used as the post-test instrument each spring term to the PDCCC degree-seeking students prior to
graduation.
The pre-test was administered to first-time students enrolled in SDV classes. The post-test was
administered spring term to upcoming degree graduates at the college. The use of a larger absolute
value of students (i.e. first-time students versus graduates) will reflect a larger number of first-time
students relative to graduates.
Independent samples t-tests is planned to compare the pre- and post-test mean scores in order to
determine where significant differences might be found.
RESULTS:
The STAGE instrument contained items pertaining to all of the general education goals for Paul D. Camp
Community. For the analysis of each general education area, a number of the Virginia community
college met and evaluated each test items and categorized them into the general education areas. Paul
D. Camp Community College revised some of the items for quantitative reasoning and information
literacy in 2010 to better evaluate each of the sub scores. Using the blueprint created by VCCS faculty
members participating in the summer 2008 backward translation workshop, sub scores for each student
learning outcomes objective were calculated by summing the number of items mapped to that objective
that were answered correctly. Some test items mapped to more than one objective. In addition, an
overall score was calculated by adding the number correct on each item reflected in one or more of the
sub scores. For the overall score, each item was counted once as opposed to adding each of the seven
sub scores together.
The results definitely showed value-added in all areas when comparing new 1st time students with
graduates. The strongest areas of general education were in personal development (6.48),
communication (6.24), and information literacy (6.17). The weakest areas were in quantitative reasoning
(3.99) and cultural/social (4.41). The weakest sub-score in quantitative reasoning was objective 6
(Graduates will recognize and communicate the appropriate applications of mathematical and statistical
models when solving problems). The weakest sub-score in cultural/social was objective 1 (The graduate
will demonstrate the ability to assess the impact that social institutions have on individuals, and culture,
past, present, and future).
As for the motivational scale, 65.3% agreed and 2.6% disagreed that they gave their best effort on the
STAGE test.
Paul D. Camp Community College STAGE Testing
2011 Degree Graduates vs. 1st Time Students
General Education Goal
2011 Graduates 1st Time Students
N=82
N=166
Communication
6.24
5.75
Critical Thinking
4.70
4.39
Quantitative Reasoning
3.99
3.57
Information Literacy
6.17
4.92
Cultural/Social
4.41
3.54
Scientific Reasoning
5.16
4.48
Personal Development
6.48
4.97
Composite Score
37.15
31.62
Test Items
5-14
15-24
25-34
35-44
45-54
55-64
65-74
5-74
Note: Composite Score maximum=70. Each general education score ranges from a minimum of 0 to a maximum
of 10.
Motivational Score: To what extent do you agree or disagree with the following statement: I
gave my very best effort on this test.
Motivational Scale
65.3% agree or strongly agree
2.6% disagree or strongly disagree
SA=19
25.3%
A=30
40.0%
Neutral=24
32.0%
D=1
1.3%
SD=1
1.3%
Note: Score used to show test motivation.
Objective
1.
2.
3.
4.
5.
6.
Mapping of STAGE Test Items to Communication SLO Objectives
Communication Goal
Test Items
% Correct
2011 Graduates
The graduate will demonstrate the ability to
understand and interpret complex materials.
The graduate will demonstrate the ability to
assimilate, organize, develop, and present an idea
formally and informally.
The graduate will demonstrate the ability to use
standard English.
The graduate will demonstrate the ability to use
appropriate verbal and non-verbal responses in
interpersonal relations and group discussions.
The graduate will demonstrate the ability to use
listening skills.
The graduate will demonstrate the ability to
recognize the role of culture in communication.
7, 9
30.8%, 73.1%
5, 7, 9
85.9%, 30.8%, 73.1%
7, 8, 9, 10, 12
30.8%, 90.9%, 73.1%, 73.7%, 32.1%
6, 9, 11, 13
73.1%, 73.1%, 70.1%, 91.0%
6, 9, 11
73.1%, 73.1%, 70.1%
9, 14
73.1%, 34.6%
Objective
1.
2.
3.
4.
5.
6.
Mapping of STAGE Test Items to Critical Thinking SLO Objectives
Critical Thinking Goal
Test Items
% Correct
2011 Graduates
The graduate will demonstrate the ability to
discriminate among degrees of credibility,
accuracy, and reliability of inferences drawn from
given data.
The graduate will demonstrate the ability to
recognize parallels, assumptions, or
presuppositions in any given source of
information.
The graduate will demonstrate the ability to
evaluate the strengths and relevance of arguments
on a particular question or issue.
The graduate will demonstrate the ability to weigh
evidence and decide if generalizations or
conclusions based on the given data are
warranted.
The graduate will demonstrate the ability to
determine whether certain conclusions or
consequences are supported by the information
provided.
The graduate will demonstrate the ability to use
problem solving skills.
18
16, 22, 24
17, 22
73.3%
47.4%, 80.3%, 14.5%
57.9%, 80.3%
17, 19, 21
57.9%, 52.6%, 13.5%
20, 21, 23
66.7%, 13.5%, 36.0%
15, 23
67.1%, 36.0%
Mapping of STAGE Test Items to Quantitative Reasoning SLO Objectives
Objective
Quantitative Reasoning Goal
Test Items
% Correct
2011 Graduates
1.
2.
3.
4.
5.
6.
7.
The graduate will use logical and mathematical
reasoning within the context of various disciplines.
The graduate will interpret and use mathematical
formulas when instructed.
The graduate will interpret mathematical models
such as graphs, tables, and schematics, and draw
inferences from them when they appear on test.
The graduate will use arithmetic, algebraic,
geometric, and statistical models in problems
solving.
The graduate will estimate and consider answers
to mathematical problems in order to determine
reasonableness.
Graduates will recognize and communicate the
appropriate applications of mathematical and
statistical models when solving problems.
Graduates will represent mathematical
information numerically, symbolically, and visually
using graphs and charts when given standardized
test.
29
39.2%
25, 27
34.7%, 50.7%
30
66.7%
26,31,34
44.6%, 20.8%, 56.0%
28
42.7%
32
20.0%
33
65.3%
Objective
1.
2.
3.
4.
5.
Mapping of STAGE Test Items to Information Literacy SLO Objectives
Information Literacy Goal
Test Items
% Correct
2011 Graduates
The information literate student determines
the nature and extent of the information
needed.
The information literate student accesses
needed information efficiently and effectively.
The information literate student evaluates
information and its sources critically, and
incorporates selected information into his or
her knowledge base and value system.
The information literate student, individually
or as a member of a group, uses information
effectively to accomplish a specific purpose.
The information literate student understands
many of the economic, legal, and social issues
surrounding the use of information resources
and accesses and uses information ethically
and legally.
42
35, 37, 41, 42,
43, 44
36
73.3%
54.7%, 73.3%, 56.8%, 73.3%,
57.3%, 61.3%
60.0%
39
61.3%
38, 40
89.3%, 88.0%
Mapping of STAGE Test Items to Cultural and Social Understanding SLO Objectives
Objective Cultural and Social Understanding Goal
Test Items
% Correct
2011 Graduates
1.
2.
3.
4.
5.
The graduate will demonstrate the ability to assess
the impact that social institutions have on
individuals and culture—past, present, and future.
The graduate will demonstrate the ability to
describe their own as well as others’ personal
ethical systems and values within social
institutions.
The graduate will demonstrate the ability to
recognize the impact that arts and humanities
have upon individuals and cultures.
The graduate will demonstrate the ability to
recognize the role of language in social and
cultural contexts.
The graduate will demonstrate the ability to
recognize the interdependence of distinctive
world-wide social, economic, geo-political, and
cultural systems.
45, 46,47, 48,
49, 52, 53, 54
53.3%, 30.7%, 29.3%, 82.7%,
64.9%, 58.7%, 5.3%, 28.4%
49,54
64.9%, 28.4%
49, 50
64.9%, 62.2%
51,
45, 47, 48, 51,
52, 53
69.3%
53.3%, 29.3%, 82.7%, 69.3%,
58.7%, 6.7%
Mapping of STAGE Test Items to Scientific Reasoning SLO Objectives
Objective
1.
2.
3.
4.
5.
Scientific Reasoning Goal
The graduate will demonstrate the ability to
generate an empirically evidenced and logical
argument.
The graduate will demonstrate the ability to
distinguish a scientific argument from a nonscientific argument.
The graduate will demonstrate the ability to
reason by deduction, induction and analogy.
The graduate will demonstrate the ability to
distinguish between causal and correlational
relationships.
The graduate will demonstrate the ability to
recognize methods of inquiry that lead to
scientific knowledge.
Test Items
% Correct
2011 Graduates
59
28.0%
61, 63
62.2%, 74.7%
56, 58, 60, 64
48.0%, 57.3%, 28.0%, 77.3%
62
70.7%
55, 57
88.0%, 30.7%
Mapping of STAGE Test Items to Personal Development SLO Objectives
Objective
Personal Development Goal
Test Items
% Correct
2011 Graduates
1.
2.
The graduate will demonstrate the ability to
develop and/or refine personal wellness goals.
The graduate will demonstrate the ability to
develop and/or enhance the knowledge, skills, and
understanding to make informed academic, social,
personal, career, and interpersonal decisions.
68, 69, 70
68.9%, 57.3%, 64.0%
65, 66, 67, 71,
72, 73, 74
89.3%, 49.3%, 81.3%, 67.1%,
82.4%, 81.1%, 72.0%