CALIFORNIA
SOCIAL
U N I V E R S I T Y
O F
S C H O O L
WO RK
EDUCATION
C A L I F O R N I A ,
O F
S O C I A L
CENTER
B E R K E L E Y
W E L FA R E
Regional Report: Case Planning Knowledge Test
IUC, April 2009
RESULTS
Calendar Year 2007
Test Data:
IUC
 148 complete test pairs
o IUC 1
N=47
o IUC 2
N=28
o IUC 3
N=21
o IUC 4
N=52
 86 pre and posttests could not be matched.
 Test versions 1.0, 1.25 and 1.26
 Poorly functioning items omitted
Pretest – Posttest Change:
IUC


Case Planning Pretest Posttest Change

1.2

1
Mean in Logits
Statewide
 N=802 complete pairs
 Test versions 1.0, 1.25 and 1.26
 Poorly functioning items omitted

0.8
Statewide
IUC
0.6
0.4
0.2
0
Pretest
Posttest
Average gain of .57 logits
Raw score change from 13.7 to 15.7 out of 24
points for version 1.0
Raw score change from 15.5 to 19.2 out of 24
points for version 1.25
Raw score change from 16.0 to 17.7 out of 25
points for version 1.26
Statistically significant t=8.115 d.f.=147, p<.001
Statewide
 Average gain of .53 logits
 Raw score change from 13.8 to 15.3 out of 24
points for version 1.0
 Raw score change from 15.5 to 18.2 out of 25
points for version 1.25
 Raw score change from 15.7 to 18.0 out of 25
points for version 1.26
 Statistically significant t=20.610 d.f.=801, p<.001
MARCHANT BUILDING, SUITE 420  6701 SAN PABLO  BERKELEY, CA 94720-7420
(P) 510.642.9272  (F) 510.642.8573  (W) http://calswec.berkeley.edu
Calendar Year 2008
Test Data:
IUC
 146 complete test pairs
o IUC 1
N=70
o IUC 2
N=36
o IUC 3
N=40
 43 pre and posttests could not be matched.
 Test version 1.26
 Poorly functioning items omitted
Pretest – Posttest Change:
Case Planning Pretest Posttest Change
1.4
IUC



1.2
Mean in Logits
Statewide
 N=743 complete pairs
 Test versions 1.25 and 1.26
 Poorly functioning items omitted
1
0.8
Statewide
IUC
0.6
0.4
0.2
0
Pretest
Posttest
Average gain of .51 logits
Raw score change from 16.2 to 18.6 out
of 25 points for version 1.26
Statistically significant t=8.582 d.f.=145,
p<.001
Statewide
 Average gain of .60 logits
 Raw score change from 16.1 to 18.7 out
of 25 points for version 1.25
 Raw score change from 15.6 to 17.6 out
of 25 points for version 1.26
 Statistically significant t=22.070
d.f.=742, p<.001
Demographics for 2007 and 20081:

Demographic forms were available for 189 of the 294 pairs of pre and posttest tests overall (64.3%).

Since many of the individual demographic variables are interrelated (e.g. carrying a caseload and concern
for time away from the office, or educational level and IV-E status), consideration of the relationship of a
single demographic variable to test scores can be misleading. To minimize this potential, a multiple
regression analysis was done to assess the effects of a demographic variable on posttest score over and
above the effects of other related variables.

Pretest is included as a predictor in this model to control for level of knowledge prior to the training. This
allows an assessment of the relationship of background variables to learning (and training effectiveness)
rather than to the overall level of knowledge shown on the posttest.
1
The timeframe for this report has been expanded in order to capture all backlogged data. Numbers for the two years are
sufficient to do regional demographic analyses.
IUC Report for Case Planning Knowledge Test
Semi Annual Report, April 2009
2

Amount learned did not differ significantly by age2, race, gender, ESL status, experience, educational level
or type of degree, carrying a caseload, or feelings about attending Core.

Several statistically significant differences were also noted in this analysis :
 Gains from pretest to posttest were significantly greater for participants who had heard that Core was
worthwhile than for those who had not (p<.05). These trainees also scored higher at posttest;
however, this difference did not reach statistical significance.
 MSW trainees who had participated in Title IV-E made significantly smaller gains from pretest to
posttest (p<.05). These trainees scored higher at pretest than those with other educational
backgrounds. Differences in posttest scores were not statistically significant.
 Posttest scores varied significantly by race. African American and Hispanic /Latino trainees scored
lower at posttest (p<.05 and p<.01, respectively) although the amount of pre to posttest change was
not significantly different for either group from the gains experienced by Caucasian trainees. Posttest
score differences remained after controlling for ESL status in the analysis.
 Differences associated with demographic groups should be viewed with caution due to the relatively
high percentage of scores that could not be matched with demographic data and relatively small
numbers of trainees in some demographic categories.
Next Steps:
 Pilot all items identified by the CDOG+ Team as relevant.
 Enter well functioning items that are identified by the CDOG+ Team as relevant to the revised curriculum
into the Examiner item bank.
 Work to increase the percentage of participants whose test scores can be matched with their
demographic data.
 Conduct further item analyses to identify areas of differential functioning with regard to race/ethnicity,
gender, region, and ESL. areas of differential functioning.
2
Variables with multiple categories were recoded as dummy/contrast variables.
IUC Report for Case Planning Knowledge Test
Semi Annual Report, April 2009
3