Analysis of Information & Communication Technology (ICT)
Literacy Assessment Data
Summer 2006
General Information on Test Group:
 639 individuals were tested. Analyses were conducted on those students identified as Freshman,
Sophomores, Juniors, or Seniors (N=616)
 Mostly Juniors & Seniors (73%)
 Similar distribution of male (49%) and female (51%) students
 38% Asian, followed by 30% white, and 16% Mexican/Hispanic
 Similar distribution of native (47%) and transfer (50%) students
 94% US citizenship or permanent resident
 92% identified English as the language they know best (17% equally English and another language)
 93% consider themselves full time students
We had hoped to be able to disaggregate data by major, however the test recorded only general areas of
study. Our campus liaison with ETS is presently working with them to insure that actual major will be
identified in the fall administration of the test.
Executive Summary
The Information & Communication Technology (ICT) Assessment developed by ETS is comprised of 7
components (matched to the Information literacy standards developed by the American Library
Association). These 7 test components assess students' skill in defining, accessing, evaluating,
managing, integrating, creating, and communicating information. More information is available on the
ETS web site.
Our students' overall ICT Scores were below the average of all students taking the ICT test early this
past Spring 2006. Our Native Juniors/Seniors' scores were stronger than their counterparts who
transferred into SJSU.
With respect to test components, our students appear to have the greatest difficulty in the areas of
integration and evaluation of information. Though below the average of the reference group,
management of information appears to be a stronger area for SJSU students.
Overall Performance
Test scores can range between 400 and 700. The average for all that had taken the test at the same time
or earlier in 2006 as did SJSU students was 550. The range of scores for SJSU students was 455 to
625. The distribution of scores was normal with a mean of 541 (below the average of other test takers).
Performance by Class Status
The strongest group was comprised of Juniors followed by Sophomores, Seniors, and Freshman. Note:
the size of the Freshman and Sophomore groups were notably smaller than the Juniors and Seniors. To
increase the size of these groups for the fall testing, efforts will be made to specifically recruit students
from the English 1a and 1b courses.
ICT Reported Score
class in college
Freshman
Sophomore
Junior
Senior
Total
Mean
535.77
540.25
544.96
540.41
541.84
Median Minimum Maximum Std. Deviation
530.00
455
615
35.934
545.00
465
615
36.905
545.00
470
625
34.938
537.50
465
625
36.404
540.00
455
625
35.798
N
84
80
282
170
616
Performance by Entry Status
Native and transfer students scores were comparable. This serves as an excellent baseline for future
comparison. With our GE program specifically identifying information literacy (IL) as a student
learning outcome, activities in our GE courses designed to improve IL should begin to affect the
performance of future Juniors and Seniors and perhaps show differences in the scores of native
compared to transfer students as well as Junior/Senior scores compared to native freshman/sophomores
who have not yet completed their core GE.
ICT Reported Score
Status at SJSU
Mean
Median
Native
Transfer
544.00
539.95
545.00
540.00
Minimum Maximum
455
465
625
625
Std. Deviation
N
35.245
35.889
289
307
Performance by Entry and Class Status
This analysis was done to further examine the Native/Transfer scores among just our Juniors and
Seniors. The spread of scores is more defined with Native Juniors/Seniors scoring better than transfer
students.
ICT Reported Score
Status at SJSU
Native
Transfer
class in college
Junior
Senior
Junior
Senior
Total
Mean
552.82
547.36
541.66
536.18
539.95
Median Minimum Maximum Std. Deviation
N
555.00
480
625
32.919 94
545.00
465
620
31.238 55
540.00
480
620
35.194 184
532.50
465
625
37.547 106
540.00
465
625
35.889 307
Performance by Gender
Male students scored slightly higher (greater spread of scores as well) overall than female students. It
will be interesting to track this as our GE program puts in place curricular materials designed to
strengthen our students' information competency skills.
ICT Reported Score
GENDER
female
male
Total
Mean
539.43
544.62
541.95
Median
540.00
550.00
540.00
Minimum
465
455
455
Maximum
620
625
625
Std. Deviation
33.947
37.387
35.726
N
316
299
615
Performance on Separate Components of the Information Competency Test
The raw component data from ETS was not available with this administration of the exam. We have
communicated that the disaggregated information is essential to us and will continue to work with both
the Chancellor's office and ETS to obtain this level of detail in the future.
However, ETS did provide us with overall group performance in each area. Student performance was
calculated using the students' raw skill area score as a percentage of the possible points for that skill
area. Two forms of the exam were administered. Therefore, what we have are component scores for
each form (1A and 2A).
Notes: The gray box indicates the range of student achievement for the middle half of the SJSU
students tested (75th to 25th percentile). The median is indicated by the horizontal line within the box.
The vertical lines extend to the highest and lowest student achievement levels. The horizontal line at
0% indicates the reference group (N=982) average.
While performance on the different forms varied, our students appear to have the greatest difficulty in
the areas of integration and evaluation of information. Though below the average of the reference
group, management of information appears to be a stronger area for SJSU students.
For a better sense of the nature of the tasks in each area, the following provides a description of what
students were asked to do.
Area:
Define
Access
Evaluate
Manage
Integrate
Create
Communicate
Students were asked to:
Narrow a customer's particular needs; Identify appropriate
features for a product to solve an office problem.
Analyze the possible reasons for an internet search's poor
results; Search a database to obtain information
Evaluate emails from customers to determine whether they
contained complete information for further action; Evaluate
extent to which particular web sites are useful for a research
project.
Fill in an organizational chart to reflect the structure of a
department; Reduce the size of an email in box by deleting
some and saving some attachments to a hard drive.
Complete a table comparing potential roommates according
to specific criteria; Fill out a spreadsheet in order to
determine the season records of teams in an ultimate frisbee
league.
Edit and format a document about llama farms; Create a
presentation slide arguing in favor of a particular position.
Format a document about a soap opera; Select and organize
slides for two distinct presentations to different audiences.
Feedback to students
ETS sends directly to students a very complete report of their performance overall and on each
component. A sample (without student identification) is provided here for review.