M. Chantale Damas, Physics Department
Assessment Institute, Spring 2014
Final Report
This report describes the final assessment of two astronomy courses in the physics
department at QCC. The goal of this project was to assess students’ understanding of key
concepts in astronomy. Two courses were assessed for this project: 1) Principles of Astronomy
& Space (PH110); and 2) Space, Astronomy, and our Universe (PH111). PH110 is a 4-credit
course that meets five hours per week (3-hr lecture) and includes a mandatory weekly two-hour
laboratory session. PH111 is a 3-credit course that meets 3 hours a week (lecture) and unlike
PH110, the laboratory session is taken as a separate course (PH112). Both courses were taught
with little to no math. As of this coming fall semester, PH110 will no longer be offered and will
be replaced by PH111.
Students were administered a pre and post Astronomy Diagnostic Test (ADT). ADT is a
multiple-choice test that has been designed with common misconceptions in astronomy in mind.
ADT has been field-tested and is normed with thousands of students at diverse institutions. Both
pre and post ADT were scored in order to assess the following:




Conceptual gains of students in both PH110 and PH111 as a whole.
Compare the conceptual gains of students in PH110 to those in PH111
Identify concepts that are weak areas of student understanding
Identify misconceptions students bring as prior knowledge to a course
Major Results of Project
Overall, students’ conceptual understanding of astronomy increased, albeit modestly for both
courses. Some of the challenges encountered for this project were the following:
1. Many of the questions were left blank, both pre and post
2. Faculty teaching to the survey, thus biasing results
3. Multiple answers given to one question
4. Unable to get copies of finals from participating faculty
5. Unable to get students’ grades
General Education Objective and Student Learning Outcomes
1. Communicate effectively through reading, writing, listening and speaking
2. Use analytical reasoning to identify issues or problems and evaluate evidence in order to
make informed decisions
3. Reason quantitatively and mathematically as required in their fields of interest and in
everyday life
4. Use information management and technology skills effectively for academic research and
lifelong learning
5. Work collaboratively in diverse groups directed at accomplishing learning objectives
6. Employ concepts and methods of the natural and physical sciences to make informed
judgments
7. Integrate knowledge and skills in their program of study
8. Differentiate and make informed decisions about issues based on multiple value systems
1
M. Chantale Damas, Physics Department
Assessment Institute, Spring 2014
General
PH110 & 111 Student Learning Outcomes
Education
Objectives
2
•Students will be able to differentiate between facts, assumptions, and conclusions in
the formulation of a proposed solution or answer (e.g., Astronomy vs. Astrology)
•Students will be able to demonstrate an understanding of and use some crucial
astronomical quantities. (such as mass, force, energy, luminosity, distance)
•Students will be able to describe the appropriate physical laws. (such as Newton’s
laws of motion and gravitation)
3
•Student will relate some subjects from physics (e.g., gravity and electromagnetic
radiation) to astronomy.
•Students will use mathematics to solve simple problems involving physical laws.
(e.g. using Kepler’s third law to calculate masses)
6
•Students will demonstrate an understanding of the nature, scope, and evolution of
the Universe, and where the Earth and Solar system fit in
•Students will be able to demonstrate an understanding of the notion that physical
laws and processes are universal, that the world is knowable, and that we are coming
to know it through observations, experiments, and theory (the nature of progress in
science).
• Students will be able to describe topics related to the history of astronomy and the
evolution of scientific ideas (science as a cultural process).
Results of Pre and Post ADT
The sample size consisted of 146 students from six sections of both PH110 and PH111.
Students from all sections were administered ADT at the end of the semester. Two sections were
removed because the instructor for those sections coached the students for the post ADT.
Courses and sections included the following:
1
2
3
4
5
6
7
8
9
Course
PH110
PH110
PH110
PH111
PH111
PH111
PH111
PH111
PH111
Section
G124
D124
M13
H24
C35
D24A
D24B
X13
X24
The following five common core topics covered by all instructors1 were used to assess
students gain, and were compared both pre and post for all sections. The table below shows the
results of the five questions, both pre and post for both courses respectively. In addition,
students’ confidence was also assessed and the results are also listed below. The correct answers
1
Instructors were asked which topics in the syllabi (PH110 & PH111) they covered during the semester.
2
M. Chantale Damas, Physics Department
Assessment Institute, Spring 2014
are bolded and the percentage change from pre to post for the correct answers are also shown
below.
Astronomy Diagnostic Test Results
PH 110
Pre Post %change
5. How does the speed of radio waves compare to
the speed of visible light?
A. Radio waves are much slower.
B. They both travel at the same speed.
C. Radio waves are much faster.
27
12
13
22
18
10
13
18
18
11
8
10. Which of the following lists is correctly
arranged in order of closest-to-most-distant from
the Earth?
A. Stars, Moon, Sun, Pluto
B. Sun, Moon, Pluto, stars
C. Moon, Sun, Pluto, stars
D. Moon, Sun, stars, Pluto
E. Moon, Pluto, Sun, stars
Pre
PH 111
Post %change
50
27
6
9
15
8
5
33
38
17
10
41
17
4
8
19
4
1
13
1
4
1
7
7
22
19
10
8
4
23 4.5
13
2
5
8
20
7
2
5
4
14 -30
3
1
13. The hottest stars are what color?
A. Blue
B. Orange
C. Red
D. White
E. Yellow
16
11
8
6
9
36
2
4
7
1
14
1
8
11
8
16
3
5
4
2
15. Global warming is thought to be caused by the
A. Destruction of the ozone layer.
B. Trapping of heat by nitrogen.
C. Addition of carbon dioxide.
41
7
4
32
10
4
20
8
8
15
3
9
8. Where does the Sun’s energy come from?
A. The combining of light elements into heavier
elements
B. The breaking apart of heavy elements into lighter
ones
C. The glow from molten rocks
D. Heat left over from the Big Bang
3
125
0
30
12.5
M. Chantale Damas, Physics Department
Assessment Institute, Spring 2014
16. In general, how confident are you that your
answers to this survey are correct?
A. Not at all confident (just guessing)
B. Not very confident
C. Not sure
D. Confident
E. Very confident
8
22
13
8
1
0
14
21
17
3
-100
-36
61.5
112.5
200
5
10
17
6
1
6
10
10
Overall, students from both courses showed a gain in the post test; however students from
PH110 showed a higher gain in general than students in PH111. This could be due to the fact
that some of these topics were reinforced in the lab, which was mandatory for all PH110
students. This was particularly true for question 13 (color of stars), in which PH110 students’
gain was much greater than students in PH111. Students in PH111 are not required to take
PH112, the lab course, unless they are transferring outside of CUNY. It should also be noted
that PH110 students’ confidence increased by over 100% at the end of the semester, though the
percentage of those who were not sure also increased by more than 60%. Although confidence
levels increased, students still were unsure of themselves (an increase of 61.5%).
Demographic data were also collected (see Appendix). Overall, the majority of the
students were male and Hispanic. Most students have taken Algebra; however, a great number
of them reported not being comfortable with math. In general, students from both sections
reported being comfortable at science, much more than math. On the other hand, many also
reported not being comfortable at science. Students were generally young (< 24years old), and
for the majority of students, this was their first astronomy course at the college level.
Conceptual Understanding
a. Weak areas of student understanding—It is clear from the results that students have
problems understanding what electromagnetic (EM) waves are and that all EM waves
travel at the same speed. Additionally, students have difficulty with identifying distances
of planets in the solar system (question 10). Question 10 showed a small gain for PH110
students, yet saw a decrease for PH111 students. Distances in the solar system are
measured in astronomical units (AU). The book also describes the solar system and
shows the distances of each planet from the Sun. However, the majority of students do
not buy the book, and some have difficulty with the language of science. Some terms
used in astronomy are difficult for students, e.g., epicycles, especially for English
learners.
b. It should also be noted that students maintain that global warming is caused by the
destruction of the ozone, and not by carbon dioxide. Although CO2 levels have been
discussed in the media, the majority of students seem to have no clue as to what causes
global warming.
Conclusion
Confronting students’ weak areas of understanding and misconceptions is a very big
challenge. This past year, some faculty members in the department began using Lecture-based
Astronomy Tutorials in the classroom to help student overcome these difficulties. Students are
given worksheets in class that reinforce what they have just learned in lecture. They usually
4
80
-40
-41
67
M. Chantale Damas, Physics Department
Assessment Institute, Spring 2014
work in teams, while the instructor circulates around the room and engage students in Socratic
dialogue. The questions are then reviewed in class allowing instructors to address students’
misconceptions or lack of understanding immediately. This method has shown to help students at
the 4-year level increase their conceptual understanding in astronomy. It would be instructive to
do a study on whether classroom based tutorials are effective with students at the community
college level.
5
M. Chantale Damas, Physics Department
Assessment Institute, Spring 2014
APPENDIX
Demographic data
Pre
17. What is your college major (or current area of
interest if undecided)?
A. Business
B. Education
C. Humanities, Social Sciences, or the Arts
D. Science, Engineering, or Architecture
E. Other
PH 110
Post %change
Pre
13
1
12
8
14
12
1
10
1
13
31
10
3
3
2
24
7
1
4
2
19. What is your age?
A. 0-20 years old
B. 21-23 years old
C. 24-30 years old
D. 31 or older
E. Decline to answer
15
15
12
13
1
20
8
4
20. Which best describes your home community
(where you attended high school)?
A. Rural
B. Small town
C. Suburban
D. Urban
E. Not in the USA
7
7
7
21
6
9
1
10
11
3
21. What is your gender?
A. Female
B. Male
21
24
17
18
18. What was the last math class you completed
prior to taking this course?
A. Algebra
B. Trigonometry
C. Geometry
D. Pre-Calculus
E. Calculus
F. Arithmetic
6
PH 111
Post %change
M. Chantale Damas, Physics Department
Assessment Institute, Spring 2014
4
C. Decline to answer
22. Which best describes your ethnic
background?
A. Black/African-American
B. Asian-American
C. Native-American
D. Hispanic-American
E. None of the above (see question 29 below)
9
7
1
17
14
6
5
1
12
10
23. Which best describes your ethnic
background?
A. Black (not American)
B. Asian (not American)
C. White, non-Hispanic
D. Multicultural
E. None of the above (see question 28 above)
7
14
7
12
6
4
6
4
14
5
24. How comfortable at math are you?
A. Extremely comfortable
B. Very comfortable
C. Comfortable
D. Not very comfortable
6
5
14
23
1
10
15
9
25. How comfortable at science are you?
A. Extremely comfortable
B. Very comfortable
C. Comfortable
D. Not very comfortable
2
7
25
18
1
2
20
13
26. Which best describes the level of difficulty
that you expect from this course?
A. Extremely difficult for me
B. Difficult for me
C. Unsure
D. Easy for me
E. Very easy for me
5
10
30
6
0
0
4
21
4
27. How many astronomy courses at the college
level have you taken?
7
M. Chantale Damas, Physics Department
Assessment Institute, Spring 2014
A. I'm re-taking this course.
B. This is my first college-level astronomy course.
C. This is my second college-level astronomy
course.
D. I've completed more than two other college-level
astronomy courses.
8
6
26
0
32
2
1
0