Program/Discipline/Course Assessment Report
Discipline: Astronomy and Physics
Course Number: AST 104, CHEM 100, PHYS 100, 151, 180, 181
School/Unit: School of Sciences
Submitted by: Daniel Loranz
Contributing Faculty: Listed in Attachments
Academic Year: 2009/2010
Complete and submit your assessment report electronically to your Academic Dean. As needed, please attach supporting documents and/or a narrative description of the assessment activities in your program or
discipline.
Program , Discipline or
Course Outcomes
In the boxes below,
summarize the outcomes
assessed in your program or
discipline during the last
year.
Outcome #1:
AST 104
Content Mastery:
Actively engaged students
will gain an introductory
knowledge of modern
astronomy.
Outcome #2:
AST 104, CHEM100
& PHYS 100
Nature of Science:
Actively engaged students
will demonstrate an
understanding of scientific
theories, demonstrate an
ability to use the scientific
method, understand and
appreciate scientific
phenomena, and understand
scientific and technical
developments.
Assessment Measures
Assessment Results
Use of Results
In the boxes below,
summarize the methods
used to assess program
or discipline, or course
outcomes during the last
year.
Students complete the
Light & Spectroscopy
Concept Inventory as a
pre-course and postcourse survey. The
Normalized Hake Gain
is used as a measure of
student learning gains.
Students complete a precourse/post-course
concept inventory
regarding scientific
thinking. The
Normalized Hake Gain
is used as a measure of
student learning gains.
In the boxes below, summarize the
results of your assessment
activities during the last year.
In the boxes below,
summarize how you are or
how you plan to use the
results to improve student
learning.
Fall 2009
D01 (Case) <g> = .07
N01 (Case) <g> = .06
E50 (Loranz) <g> = .26
Assessment data will be
shared with part-time
instructor Case.
Light & Spectroscopy Concept
Inventory is a nationally validated
intro astronomy content diagnostic.
I will continue to use this survey.
Scientific Thinking must
be explicitly addressed. I
have been revising my
section of AST 104 to do
this. So far results are
encouraging that mitigating
student miscomprehensions about scientific
thinking can be done.
I am developing a Nature of
Science concept inventory. I will
use the results from this year to
refine the questions asked in this
survey.
Spring
D01 (Loranz) <g> = .38
N01 (Case) <g> = .07
Students appear to enroll in our
non-majors science courses with
almost no understanding of
scientific thinking. In addition,
unless specifically addressed as
part of the course curriculum,
students do not show any
improvement in their
understanding.
Effect on the Program,
Discipline or Course
Based on the results of this
assessment, will you revise your
outcomes? If so, please summarize
how and why in the boxes below.
Outcome #3:
PHYS 151
Content Mastery:
Actively engaged students
will gain a working
knowledge of algebra based
Newtonian Mechanics.
Administer Force
Concept Inventory (FCI)
as pre/post test.
Measure learning gains
by calculating the Hake
gain.
Fall 2009
D01 (Jensen) – no data
N01 (Porter) <g> = .43
Outcome #4
PHYS 180 & PHYS 181
Content Mastery:
Actively engaged students
will gain a working
knowledge of calculus based
Newtonian Mechanics and
Maxwellian Electromagnetic
Theory
Students solve exam
problems closely tied to
course learning
objectives.
Please see attached summary.
Jensen and Porter have the
FCI data for their sections.
I am not aware of how they
will use this data.
Our use of the FCI is giving us
results consistent with those of
other colleges and universities
across the nation. I will continue
to use the FCI as a benchmark for
student learning gains in PHYS
151.
Please see attached
summary.
This assessment approach provides a
good summary of student
performance. However, it would
also be useful to measure learning
gains. I am still looking for a
reasonable pre-test/post-test
diagnostic for use in PHYS 180 and
PHYS 181.
Spring 2010
D01 (Jensen) <g> = .28
N01 (Porter) <g> = .38
For Discipline Assessment Reports:
I have reviewed this report:
________________________________________________
Dean
_______________________________________________
Vice President for Academic Affairs & Student Services
Date_______________
Date_______________
Physics and Astronomy Assessment for 2008-2009
Results for AST 104
FACULTY INVOLVED
Daniel Loranz (tenured faculty) and Clint Case (part-time instructor)
SUMMARY OF ASSESSMENT ACTIVITIES
I used the Light and Spectroscopy Concept Inventory (LSCI) to assess student learning gains in introductory astronomy. The LSCI is a
nationally validated diagnostic used in many intro astronomy courses across the nation. The LSCI was administered as a pre-test and post-test
and these results were used to calculate the normalized Hake gain.
WHAT WERE THE RESULTS?
Fall 2009 – AST 104 D01
Instructor: Clint Case
Average Hake Gain for the class = 0.07
Fall 2009 – AST 104 N01
Instructor: Clint Case
Average Hake Gain for the class = 0.06
Fall 2009 – AST 104 E50
Instructor: Daniel Loranz
Average Hake Gain for the class = 0.26
Spring 2010 – AST 104 D01
Instructor: Daniel Loranz
Average Hake Gain for the class = 0.38
Spring 2010 – AST 104 N01
Instructor: Clint Case
Average Hake Gain for the class = 0.07
SPECIFICALLY, HOW ARE YOU OR HOW DO YOU PLAN TO USE THE RESULTS TO IMPROVE STUDENT LEARNING?
Examining individual questions, students in my sections appear to need more help understanding the following concepts. (Data available upon
request.)
- Understanding the different sources for emission spectra, absorption spectra and continuous spectra.
- Understanding that color and spectral lines are not linked.
- Being able to apply Doppler shift concepts.
- Being able to rank star sizes based upon their luminosity plots.
Clint Case will also have his LSCI data. I am not aware of how he will use this information.
BASED ON THE RESULTS OF THIS YEAR, WILL YOU REVISE YOUR ASSESSMENT PLAN?
HOW AND WHY?
The LSCI is a nationally validated diagnostic for intro astronomy, and it has provided useful data this year. I will continue to use the LSCI as a
benchmark.
Results for AST 104, CHEM 100 and PHYS 100
FACULTY INVOLVED
Tenured Faculty: Daniel Loranz
Part-time Faculty: Rio Andaya, Clint Case, John Hadder, Cullen Jones, Steve Kohl, Cindy Porter, Ileana Tibuleac, Robert Weise
SUMMARY OF ASSESSMENT ACTIVITIES
During the 2008/2009 academic year, I collected student responses to the question “What is Science”. Using these responses, I have written a
draft survey on scientific thinking.
For 2009/2010, this draft survey was used to assess student learning gains on understanding scientific thinking. The survey was given as a pretest/post-test to all face-to-face sections of AST 104, CHEM 100, and PHYS 100.
In addition, all instructors of face-to-face AST 104, CHEM 100 and PHYS 100 sections were asked to complete a short self-evaluation
regarding whether or not they directly address scientific thinking in their courses. Only three instructors responded to this request. Based upon
these responses and other anecdotal stories, most instructors of these courses are apparently NOT directly addressing scientific thinking in their
classes. Instead they are focusing on discipline specific content and hoping that students will learn about scientific thinking as a side benefit of
engaging with the specific scientific models presented in the course.
My hypothesis is that this approach is not working, and I have begun a significant revision to my section of AST 104 to directly address
scientific thinking as a fully integrated part of the course. I began these revisions in my Fall 2009 AST 104 E50 section. These revisions were
further refined in my Spring 2010 AST 104 D01 section, and will be refined yet again next year.
WHAT WERE THE RESULTS?
Spring 2010
AST 104 N01
CHEM 100 D01, D02, D03, N01
PHYS 100 D01, D02, N01
Spring 2010
AST 104 D01
Aggregate Scores
Ave. Pre-Score = 69%
Ave. Post-Score = 71.5%
Ave Hake Gain = .07
Ave. Pre-Score = 69%
Ave. Post-Score = 80.6%
Ave. Hake Gain = .37
The data suggests that students start with a poor understanding of the scientific thinking, and that experiences in AST 104, CHEM 100, and
PHYS 100 predominantly have no impact on improving student understanding of this topic. This lack of affect is not surprising, given that
currently the AST 104, CHEM 100 and PHYS 100 sections predominantly do NOT explicitly address the scientific thinking.
The exception is for my revised section of AST 104, where I do explicitly address scientific thinking. In that case, students start on par with
other sections but experience an average learning again of .37 and have an average final score of 80.6% on the post-course survey.
SPECIFICALLY, HOW ARE YOU OR HOW DO YOU PLAN TO USE THE RESULTS TO IMPROVE STUDENT LEARNING?
As we should perhaps have expected, indirectly addressing scientific thinking (e.g. by example) appears to have almost no effect on improving
student of this topic. And yet, scientific thinking is a learning objective for all these non-majors science courses.
Encouragingly, directly addressing scientific thinking does appear to significantly improve student understanding of this topic. As a result, I
will continue to refine my curriculum to specifically address this topic. As this curriculum begins to take shape, I will then happily share it
with other instructors in these non-majors science courses.
BASED ON THE RESULTS OF THIS YEAR, WILL YOU REVISE YOUR ASSESSMENT PLAN? HOW AND WHY?
The pre/post survey described here is still undergoing refinement. I will use student responses from this year to further improve the survey.
Results for PHYS 151
FACULTY INVOLVED
Lars Jensen (tenured faculty) and Cindy Porter (part-time instructor)
SUMMARY OF ASSESSMENT ACTIVITIES
We continue to use the Force Concept Inventory (FCI) to assess PHYS 151 student learning gains in the Analytical Thinking learning outcome
in the context of Newtonian Mechanics. The FCI was administered as a pre-test and post-test and these results were used to calculate the
normalized Hake gain.
WHAT WERE THE RESULTS?
Fall 2009 – PHYS 151 D01
Fall 2009 – PHYS 151 N01
Instructor: Lars Jensen
Instructor: Cindy Porter
No Data. Jensen decided to not participate in
Average Hake Gain for the class = 0.43
this assessment activity – perhaps because only
3 students were left in his section by the end of
the semester.
Spring 2010 – PHYS 151 D01
Instructor: Lars Jensen
Average Hake Gain for the class = 0.28
Spring 2010 – PHYS 151 N01
Instructor: Cindy Porter
Average Hake Gain for the class = 0.38
For comparison, nationally reported Hake gains from the FCI are …
- about 0.23 +/- 0.04 in traditional lecture style physics courses
- about 0.48 +/-0.14 in active-learning style physics courses.
SPECIFICALLY, HOW ARE YOU OR HOW DO YOU PLAN TO USE THE RESULTS TO IMPROVE STUDENT LEARNING?
Jensen and Porter both have their FCI data. I am not aware of how they will use this information.
BASED ON THE RESULTS OF THIS YEAR, WILL YOU REVISE YOUR ASSESSMENT PLAN?
HOW AND WHY?
The FCI has now been used successfully for many semesters to measure student learning gains in analytical thinking within a Newtonian
Mechanics context. We will continue to use this nationally validated diagnostic tool.
Results for PHYS 180 and PHYS 181
FACULTY INVOLVED
Daniel Loranz
SUMMARY OF ASSESSMENT ACTIVITIES
- Students answer fifteen exam questions during each semester. These exam questions are carefully written to address the specific learning
objectives for the course.
- Each question is scored on a scale from 0 to 15 points.
- Scores of 10 thru 15 points indicate student success on the learning objective.
(After the questions are scored, students are allowed to submit corrections. An initial score of 10 points means students can still possibly at least a “B” grade on the
question after submitting corrections.)
WHAT WERE THE RESULTS?
Fall 2009 PHYS 180 D01
Question #
Exam 1 Q1
Graphing 1 Dim. Kinematics
Exam 1 Q2
Solving 2 Dim. Kinematics Using Vectors
Exam 1 Q3
Newton’s Laws and Free Body Diagrams
Exam 1 Q4
Application of Newton’s Laws for Constant Forces
Exam 1 Q5
Application of Newton’s Laws for Centripetal Motion
Exam 2 Q1
Work and Energy for a Varying Force
Exam 2 Q2
Work and Energy: Interpreting F vs distance graphs
Exam 2 Q3
Momentum and Impulse: Solving Collision Problems
Exam 2 Q4
Rotational Kinematics
Exam 2 Q5
Moment of Inertia and Center of Mass
Exam 3 Q1
Ave
Score
% of students scoring
10 or more points
77.2%
75%
80%
83%
85%
92%
71%
67%
69.7%
64%
86.7%
100%
94.4%
92%
67.8%
67%
57.6%
27%
51.3%
25%
54.5%
45%
Rotational Dynamics
Exam 3 Q2
Static Equilibrium
Exam 3 Q3
Rotational Work and Energy
Exam 3 Q4
Analysis of Translational and Rotational Motion
Exam 3 Q5
Simple Harmonic Oscillators
65.3%
50%
58%
60%
55.6%
44%
31.1%
0%
Spring 2010 PHYS 181 D01
Question #
Exam 1 Q1
Using Electric Potential to Solve Work & Energy Problems
Exam 1 Q2
Find E-field & Force for a Collection of Point Charges
Exam 1 Q3
Find Electric Potential for a Collection of Point Charges
Exam 1 Q4
Application of Guass’ Law
Exam 1 Q5
Calculating Electric Potential from an E-field
Exam 2 Q1
Using Matrix Algebra to Solve Simple DC Circuits
Exam 2 Q2
Conceptual Understanding of Simple DC Circuits
Exam 2 Q3
Solving RC Circuits
Exam 2 Q4
Solving Equivalent Capacitance
Exam 2 Q5
Calculating Magnetic Field for a Collection of Currents
Exam 3 Q1
Understanding Generators
Exam 3 Q2
Application of Faraday’s Law
Exam 3 Q3
LR Circuits
Exam 3 Q4
Solving for Parameters of a 1-D Harmonic Wave
Exam 3 Q5
Solving for Parameters of a Harmonic EM Plane Wave
Ave
Score
87.3%
% of students scoring
10 or more points
100%
70%
70%
80.7%
100%
62.7%
70%
54.1%
56%
76.7%
70%
54.7%
30%
77.3%
70%
83%
78%
91.7%
100%
58.5%
56%
86%
90%
79.2%
88%
86.7%
89%
65.2%
56%
SPECIFICALLY, HOW ARE YOU OR HOW DO YOU PLAN TO USE THE RESULTS TO IMPROVE STUDENT LEARNING?
In general, PHYS 180 and PHYS 181 students appear to be successfully learning the course content, with some noticeable exceptions. These
are addressed below.
PHYS 180
- Ex2 Q4, Ex2 Q5, and Ex3 Q1: Rotational Kinematics and Rotational Dynamics.
Students have difficulty transitioning from linear mechanics to rotational mechanics, especially in grappling with rotation axes, center
of mass, and moment of inertia. Possible remedies might include an additional lab experience on this set of topics and/or the possibility
of deleting the vector nature of rotation from the curriculum.
-
Ex3 Q4: Combined Analysis of Translational and Rotational Motion.
This is an end-of-semester, big-picture application problem. Even though students may have mastered all parts needed to solve this
comprehensive problem, they still struggle with assembling all the pieces for a complete solution. Given the introductory nature of this
course, I am not surprised that students have difficulty with this sort of comprehensive problem.
-
Ex3 Q5: Simple Harmonic Oscillators
Data on this problem is an outlier. Review of scores show that students generally did just fine with the physics of this problem but
panicked when the problem presented the binomial expansion and asked students to use the expansion to simplify their solutions.
Bottom line – students did fine with the physics part of this problem, and only balked unnecessarily at the math.
PHYS 181
-
Exam 2 Q2: Conceptual Understanding of Simple DC Circuits
Students are not making the connection that knowledge of power usage in an Ohmic circuit gives rankings for both current and
potential. This link should be made more explicit in the lab for this topic.
BASED ON THE RESULTS OF THIS YEAR, WILL YOU REVISE YOUR ASSESSMENT PLAN?
HOW AND WHY?
I will continue to use this approach. The assessment method described for PHYS 180 and PHYS 181 provides a reasonable summary of
student performance on all major course learning objectives.
Unfortunately, the current assessment methodology does not measure learning gains (only final student performance). So I will also examine
the utility of multiple-choice style pre-test/post-test diagnostic tools for use in PHYS 180 and PHYS 181. It would be valuable to know what
student learning gains are.