An Analysis of NYS Regents
Physics Exam Issues
J. Zawicki
Science Education, BSC
STANYS DAL, Physics Mentor
Additional Researchers





2
Michael Jabot, SUNY Fredonia
Kathleen Falconer, SUNY BSC
Dan MacIsaac, SUNY BSC
David Henry, SUNY BSC
Rita Fischer, Elba Central School
NYS Regents Physics -- Zawicki
9/25/03
Presentation Overview




3
Issues from the field
Exam findings
Issues and events
Future directions…
NYS Regents Physics -- Zawicki
9/25/03
Issues From the Field
What are issue(s) are uppermost in your mind
at this time?
4
NYS Regents Physics -- Zawicki
9/25/03
Exam Findings



5
Call for student papers, STANYS, NYS Section
AAPT
>2000 student papers were submitted
Results from across NYS
NYS Regents Physics -- Zawicki
9/25/03
Initial Distribution of Papers
NYC (& Manhattan)
Catskill-Leatherstocking
Central-Western
Eastern
Mohawk Valley
Nassau
North Central
Northeastern
Northwestern
Southeastern
Southern
Southwestern
Suffolk
Westchester
Western
6
0
5
74
74
80
324
17
24
114
126
71
271
18
21
473
NYS Regents Physics -- Zawicki
9/25/03
Rasch Analysis


Parts A and B-1; B-2 and C presented together
Concepts
–
–
–
7
Difficulty – how well did the students do?
Discrimination – how well did an item distinguish
between high and low performing students?
Rasch transformation – log transformation
NYS Regents Physics -- Zawicki
9/25/03
8
NYS Regents Physics -- Zawicki
9/25/03
“Easy” Items – A, B-1
9
Item
Key
R1
R2
R3
R4
%C
N
DE
Q45
3
57
26
860
15
89.8
958
-2.17
Q40
3
40
3
807
106
84.2
956
-1.69
Q36
2
122
777
38
21
81.1
958
-1.46
Q38
3
12
54
759
131
79.2
956
-1.35
Q3
2
129
749
38
42
78.2
958
-1.28
NYS Regents Physics -- Zawicki
9/25/03
10
NYS Regents Physics -- Zawicki
9/25/03
11
NYS Regents Physics -- Zawicki
9/25/03
12
NYS Regents Physics -- Zawicki
9/25/03
13
NYS Regents Physics -- Zawicki
9/25/03
“Difficult” Items – A, B-1
14
Item
Key
R1
R2
R3
R4
%C
N
DE
Q15
1
324
13
42
576
33.8
955
0.67
Q11
3
69
76
352
458
36.7
955
0.54
Q4
4
313
20
237
387
40.4
957
0.39
Q7
2
95
394
320
148
41.1
957
0.36
Q34
4
122
371
46
418
43.6
957
0.25
NYS Regents Physics -- Zawicki
9/25/03
15
NYS Regents Physics -- Zawicki
9/25/03
16
NYS Regents Physics -- Zawicki
9/25/03
17
NYS Regents Physics -- Zawicki
9/25/03
18
NYS Regents Physics -- Zawicki
9/25/03
“Easy Items” B-2, C
19
Item
Key
R0
R1
R2
%C
N
DE
Q59
1
16
458
--
96.6
474
-3.35
Q54
2
27
51
398
89.0
476
-2.84
Q65
1
27
447
--
94.3
474
-2.81
Q58
1
34
440
--
92.8
474
-2.56
Q69
2
23
60
391
91.0
474
-2.32
NYS Regents Physics -- Zawicki
9/25/03
20
NYS Regents Physics -- Zawicki
9/25/03
21
NYS Regents Physics -- Zawicki
9/25/03
22
NYS Regents Physics -- Zawicki
9/25/03
23
NYS Regents Physics -- Zawicki
9/25/03
“Difficult” Items B-2, C
24
Item
Key
R0
R1
R2
%C
N
DE
Q62
1
266
207
--
43.8
473
0.25
Q48
1
253
223
--
46.8
476
0.13
Q53
1
246
230
--
48.3
476
0.07
Q74
1
243
231
--
48.7
474
0.05
Q56
1
242
233
--
49.1
475
0.04
NYS Regents Physics -- Zawicki
9/25/03
25
NYS Regents Physics -- Zawicki
9/25/03
26
NYS Regents Physics -- Zawicki
9/25/03
27
NYS Regents Physics -- Zawicki
9/25/03
Reading Level


28
McLaughlin-SMOG
Additional Analysis Pending
NYS Regents Physics -- Zawicki
9/25/03
Initial Analysis, Reading Levels
29
Exam Administration
Grade Equivalent
June 2000
8th
June 2001
10th (Low)
June 2002
10th (High)
June 2003
11th
NYS Regents Physics -- Zawicki
9/25/03
Conceptual Level

Modification of Bloom
–
–
–
–
30
Knowing
Using 1
Using 2
Integrating
NYS Regents Physics -- Zawicki
9/25/03
Exam Conceptual Level
31
Exam Administration
Conceptual Level
June 2000
1.61
June 2001
1.59
June 2002
1.89
June 2003
1.74
NYS Regents Physics -- Zawicki
9/25/03
Exam Format
32
ExamAdministration
June 2000
January 2001
MC W
75 11
75 10
P
3
3
June 2001
January 2002
June 2002
August 2002
75
75
45
47
11
11
24
21
4
3
12
8
January 2003
June 2003
50
47
27
29
10
16
NYS Regents Physics -- Zawicki
9/25/03
AP Correlation


33
Approximately 0.69
Precise value varied from year to year
NYS Regents Physics -- Zawicki
9/25/03
NYS Physics/AP B
100. 0
90. 0
80. 0
70. 0
60. 0
bef or e 2002
2002 & 2003
50. 0
40. 0
30. 0
20. 0
10. 0
0. 0
1
34
2
3
4
NYS Regents Physics -- Zawicki
5
9/25/03
Listserv Comments


35
Latency -- Incomplete/unfinished tests
Power Tests – assessments where time does
not play a significant role in the score
NYS Regents Physics -- Zawicki
9/25/03
Issues
Impact of Scaled Scoring on student scores in
the standards-based exams
Direction of future exams
36
NYS Regents Physics -- Zawicki
9/25/03
Sessions






37
NYSUT Session with Commissioner Mills
Science Education Consortium – August
Regents Session – September
Senate Hearings
NYS AAPT Meeting (SUNY Binghamton)
Physics Summit (Albany Area)
NYS Regents Physics -- Zawicki
9/25/03
Additional Directions?
Where do we go from here?
38
NYS Regents Physics -- Zawicki
9/25/03