Construct Validation of a Web-based Imagination Assessment
Su-Pin Hunga* Hung-Yu Huangb, Po-Hsi Chenc
a Center of Teacher Education, National Cheng Kung University
Tainan, Taiwan
b Department of Psychology and Counseling, University of Taipei
Taipei, Taiwan
c Department of Education and Counseling, National Taiwan Normal University Taipei,
Taiwan
*Corresponding Author: suping0612@gmail.com
Abstract
Imagination contributes the development of humanity civilization. Education
should encourage children to imagine future life rather than impede their abstractive
thinking. Examining the efficiency of imagination training program requires an
appropriate assessment however, because of “imagination” is an abstract concept
and thus enhances the difficulty of measuring. To date, rarely measurements are
developed to assess imagination. The previous imagination assessments are relying
on paper-based thus spent a lot of time on scoring and coding (e.g., Karwowski, 2008;
Ward, 1994; Ward, Patterson & Sifonis, 2004; Ward & Wickes, 2009). Furthermore,
scoring indices (i.e., originality, flexibility and fluency) which applied in these
imagination assessments are similar with those applied in traditional creativity
performance assessments. Since the construct of creativity and imagination are
different. Consequently, more specific scoring indices (i.e., expansion imagination;
correlation imagination) should be included to assess imagination. Hence, in the
present study the meaning of imagination is clearly defined and scoring indices are
proposed to assess imagination. Finally, a web-based imagination assessment is then
developed to measure one’s imagination and examined its construct validity.
There are four subsystem included in the web-based imagination assessment,
including: subject interface, automatic rating program, rater interface, management
interface. For the subject interface, two sections are included. Section 1: participants
are required to imagine the circumstance on earth after 1000 years. Section 2:
participants are required to do follow up imagination after accomplish section 1.
Participants are asked to typing possible function, design and related components in
detail of transportation that will need in future (after 1000 years) separately in the
follow up layers. Overall, we adopted 11 scoring indices in the newly developed
assessment. These scoring indices including: the amounts of responses in level 1;
average of category by computing the average of categories that appeared in each
level which are intended to assess divergent thinking; average of originality by
computing average of originality that appeared in each level; the amounts of
responses in level 2; the amounts of responses in level 3; the amounts of responses
in level 4; the average of levels; the maximum levels which are intended to assess
expansion imagination; connection of level 1 and level 2; connection of level 2 and
level3 and connection of level 3 and level 4 which are intended to assess correlation
imagination. Only 4 rating indices rely on rating.
The sample consisted of 1022 high school students and 644 undergraduates.
Informed consent was obtained from every participant. Participants were informed
of the type of study as well as the aims, and method to be used. All participants were
invited to attend group assessment sessions in their classrooms to complete the
newly developed imagination assessment. The imagination task had a time limit of
18 minutes, and students were required to finish the test within that limit, as fast
and effectively as possible.
To assure construct validity of our imagination assessment, confirmatory analysis
is applied to analyze two data pools high school students and undergraduates. As
listed in the Table 1, the results from confirmatory factor analysis suggested that the
three factors with 10 indices structure perform better than another one, similar
results also can be found in undergraduates. The three factors are divergent thinking
which including numbers of level 1, average numbers of classes, uniqueness of
imagination; expansion imagination which including numbers of level 3 and level 4,
numbers of maximum level and numbers of average level; and correlation
imagination which including relationship between each level and it’s follow-up level.
Table 1 Summary of confirmatory analysis
High school
N=1022
df
χ2
SRMR
GFI
NFI
CFI
Model1
1：L1 N. of Class
Uni
2：L2~L4
max_level
Average_level
3：L1L2 L2L3L3L4
41
822.04
0.098
0.87
0.88
0.89
Model2
1：L1 N. of Class
Uni
2：L3~L4
max_level
Average_level
3：L1L2 L2L3L3L4
32
637.79
0.095
0.89
0.90
0.91
University
N=644
Model1
1：L1 N. of Class Uni
2：L2~L4
max_level
Average_level
3：L1L2 L2L3L3L4
41
658.15
0.11
0.84
0.82
0.83
Model1
1：L1 N. of Class
Uni
2：L2~L4
max_level
Average_level
3：L1L2 L2L3L3L4
32
490..07
0.11
0.87
0.86
0.86
Standardized parameter estimates for the model are presented in Table 2. As shown,
model parameters were all significant (p<.01) and explained substantial amounts of
item variance (R2 ranged from 0.08 to 0.83). Among the three factors, expansion
imagination and correlation imagination were significantly correlated in the two
samples ( r=.43 and r=.37). The result also indicated that the three factor model of
imagination is supported in both groups.
Table 2 Standardized parameter estimates for the three-factor with 10 indices model
University
F1
Indicators
N. of L1
F3
0.93
N. of L3
N. of L4
Average-level
Max-level
N. of class
Uniqueness
L1L2
L2L3
L3L4
F2
0.59
0.77
0.87
0.94
High school
R2
F1
0.14
0.89
0.65
0.41
0.24
0.11
0.84
0.69
0.72
0.96
0.73
0.30
0.52
0.48
0.08
0.47
F2
F3
R2
0.20
0.41
0.71
0.77
0.88
0.83
0.49
0.40
0.22
0.86
0.78
0.80
0.96
0.78
0.27
0.39
0.36
0.08
0.40
Note: F1= Divergent thinking, F2=Expansion imagination, F3= Correlation imagination
In sum, in contrast of the previous creative imagination measures, the newly
developed imagination assessment has many advantages. First of all, the newly
developed imagination assessment is a web-based assessment tool; responses of
participants can be collecting on line so that saving time to recording. Second, most
of the scoring procedures are automatically proceed thus reduce the laborious
scoring demand. Third, in contrast to the previous assessment may restricted in
specific domain (i.e., make up a story), the newly measurement we design can be
applied in various field. For example, in the present study, the condition we set is
asking participant to think about transportation in future. However, the demand
could be change very easily and will not affect scoring. Forth, the scoring indices
included in the present study are more general and appropriate to assess
imagination. Finally, result of construct validity asserted that the scoring indices
proposed in the present study met our expectation.
Keywords: Construct Validity; Confirmatory Factor Analysis; Web-based Imagination
Assessment