CALIFORNIA
SOCIAL
U N I V E R S I T Y
O F
S C H O O L
WO RK
EDUCATION
C A L I F O R N I A ,
O F
S O C I A L
CENTER
B E R K E L E Y
W E L FA R E
Regional Report: Placement and Permanency Knowledge Test
IUC, March 2009
RESULTS
Calendar Year 2007
Test Data:
IUC
 188 complete test pairs
o IUC 1
N=45
o IUC 2
N=53
o IUC 3
N=37
o IUC 4
N=53
 Test versions 1.0 and 1.1
 Poorly functioning items omitted
Statewide
 N=878 complete pairs
 Test versions 1.0 and 1.1
 Poorly functioning items omitted
Pretest – Posttest Change:
Placement and Permanency Pretest
Posttest Change
1.4

1.2
Mean in Logits
IUC


Average gain of .67 logits
Raw score change from 15.6 to 20.1
out of 24 points
Statistically significant t=13.276
d.f.=187, p<.001
1
0.8
Statewide
IUC
0.6
0.4
0.2
0
Pretest
Posttest
Statewide
 Average gain of .46 logits
 Raw score change from 17.0 to 20.6
out of 24 points
 Statistically significant t=21.528
d.f.=877, p<.001
MARCHANT BUILDING, SUITE 420  6701 SAN PABLO  BERKELEY, CA 94720-7420
(P) 510.642.9272  (F) 510.642.8573  (W) http://calswec.berkeley.edu
Calendar Year 2008
Test Data:
IUC
 148 complete test pairs
o IUC 1
N=47
o IUC 2
N=31
o IUC 3
N=39
o IUC 4
N=31
 Test versions 1.1
 Poorly functioning items omitted
Statewide
 N=742 complete pairs
 Test versions 1.1 and 1.15
 Poorly functioning items omitted
Pretest – Posttest Change:
Placement and Permanency Pretest
Posttest Change
1.4

1.2
Mean in Logits
IUC


Average gain of .77 logits
Raw score change from 17.3 to 20.6
out of 24 points
Statistically significant t=13.01
d.f.=147, p<.001
1
0.8
Statewide
IUC
0.6
0.4
0.2
0
Pretest
Posttest
Statewide
 Average gain of .52 logits
 Raw score change from 18.3 to 20.9
out of 24 points
 Statistically significant t=22.996
d.f.=741, p<.001
Demographics for 2007 and 20081:




Demographic forms were available for 256 out of a total of 336 test pairs (76.2 %).
Since many of the individual demographic variables are interrelated (e.g. carrying a caseload and concern
for time away from the office, or educational level and IV-E status), consideration of the relationship of a
single demographic variable to test scores can be misleading. To minimize this potential, a multiple
regression analysis was done to assess the effects of a demographic variable on posttest score over and
above the effects of other related variables.
Pretest is included as a predictor in this model to control for level of knowledge prior to the training. This
allows an assessment of the relationship of background variables to learning (and training effectiveness)
rather than to the overall level of knowledge shown on the posttest.
Amount learned did not differ significantly by age2, race, gender, experience, educational level or type of
degree, participation in the Title IV-E program, ESL, carrying a caseload or expectations of core training.
All groups of trainees made gains from pre to posttest.
1
The timeframe for this report has been expanded in order to capture all backlogged data. Numbers for the two years are
sufficient to do regional demographic analyses.
IUC Report for Placement and Permanency Knowledge Test
Semi Annual Report, March 2009
2

A few statistically significant differences in posttest score level were noted in this analysis :
o Trainees with a BSW and those with an MSW who had completed the IV-E program scored
significantly higher on the posttest (p<.05 and p<.001, respectively).
o Trainees who indicated coming into Core unsure of what to expect or with negative expectations
scored higher on the posttest than those who indicated being excited about attending Core
(p<.05).
Next Steps:
 Pilot all items identified by the CDOG+ Team as relevant.
 Enter well functioning items that are identified by the CDOG+ Team as relevant to the revised curriculum
into the Examiner item bank.
 Work to increase the percentage of participants whose pre and posttest scores can be matched and
whose test scores can be matched with their demographic data.
 Conduct further item analyses to identify any poorly functioning items or areas of differential functioning.
2
Variables with multiple categories were recoded as dummy/contrast variables.
IUC Report for Placement and Permanency Knowledge Test
Semi Annual Report, March 2009
3