Resources for Measuring Constructs Related to 1:1 Technology Programs
By Morgan Grotewiel, MA
August 27, 2014
This document describes resources available for measuring constructs related to 1:1 technology
programs. These constructs are divided into five broad categories, chosen based on feedback received
at the June 5th, 2014 KC-AERC Local Steering Committee and the July 29th 1:1 Workshop: 1:1
implementation, 21st century skills, student engagement, teacher engagement, and technological
literacy. For many of these categories, evaluation instruments exist for the district, school,
administration, teacher, and/or student-level. They employ a variety of evaluation techniques, including
self-report, supervisor and peer observation, and formative and summative assessments.
In this document, when possible, I have provided links to publications that have already
compiled several measures related to a category of interest. These publications are especially good
resources because they include comparisons of multiple instruments as well as discussion of issues
related to measurement and assessment. For constructs for which these pre-existing compilations were
not available, I selected academic or commercial measures that appear to have good psychometric
properties and/or widespread use and support.
1:1 Implementation
Friday Institute for Educational Innovation (n.d.). Technology innovation.
https://eval.fi.ncsu.edu/instruments-2/technology-innovation-instruments/
This webpage from the William & Ida Friday Institute for Educational Innovation provides easy access to
instruments for measuring concerns related to 1:1 implementation at the administration, teacher, and
student levels.
Measurement subject(s): districts, administrators, teachers, students
Constructs measured: district-wide strengths in technology innovation, district-wide challenges for
technology innovation, 1:1 implementation at the building level, 1:1 implementation at the system level,
administrators’ perceptions of the 1:1 environment, 1:1 device repair needs, school-level statistics,
technology integration at the classroom level, school-level technology needs, administrators’ confidence
with technology use, teachers’ perceptions of the 1:1 environment, exemplary lesson content and
format by discipline, teachers’ confidence with technology use, teacher and student use of software and
hardware, students’ perceptions of the 1:1 environment, technology skills for grades 3-5, technology
skills for grades 6-12
21st Century Skills
Soland, J., Hamilton, L. S., & Stecher, B. M. (2013). Measuring 21st century competencies: Guidance for
educators. New York, NY: Asia Society, Global Cities Education Network. Retrieved from
http://asiasociety.org/files/gcen-measuring21cskills.pdf
This report from the Asia Society’s Global Cities Education Network and the RAND Corporation describes
the current state of 21st century skills assessment, provides an extensive list of student-level measures
of 21st century skills, and offers guidance to educators for selecting the measures that best fit their
needs. It includes reviews of multiple choice, open response, self-report, performance, portfolio, and
cross-cutting measures that assess cognitive, interpersonal, and intrapersonal 21st century skills for
formative and summative purposes.
Measurement subject(s): K-12 students
Constructs measured: academic mastery, critical thinking, creativity, communication and collaboration,
leadership, global awareness, growth mindset, learning how to learn, intrinsic motivation, grit
Partnership for 21st Century Skills (2009). The MILE Guide: Milestones for Improving Learning &
Education. Tucson, AZ: Partnership for 21st Century Skills. Retrieved from
http://www.p21.org/storage/documents/MILE_Guide_091101.pdf
This publication from the Partnership for 21st Century Skills describes a framework for implementing 21st
century skills in the curriculum at the district level and includes a district-level self-assessment to
evaluate current implementation of 21st century skills.
Measurement subject(s): district
Constructs measured: student knowledge and skills, education support systems, education leadership,
policy making, partnering, continuous improvement
Student Engagement
Carter, C. P., Reschly, A. L., Lovelace, M. D., Appleton, J. J., & Thompson, D. (2012). Measuring student
engagement among elementary students: Pilot of the Student Engagement Instrument—
Elementary Version. School Psychology Quarterly, 27(2), 61-73. doi:10.1037/a0029229
This article describes the development and validation of the Student Engagement Instrument (SEI)—
Elementary Version, a student self-report measure of cognitive and affective engagement with school.
The original SEI (Appleton et al., 2006) was designed for use with middle and high school students;
Carter et al. (2012) adapted the SEI for use with elementary students by modifying survey items to be
developmentally appropriate and by reanalyzing the factor structure.
Measurement subject(s): elementary students
Constructs measured: teacher-student relationships, peer support for learning, future goals and
aspirations, family support for learning
Fredricks, J., McColskey, W., Meli, J., Mordica, J., Montrosse, B., and Mooney, K. (2011). Measuring
student engagement in upper elementary through high school: a description of 21 instruments.
(Issues & Answers Report, REL 2011–No. 098). Washington, DC: U.S. Department of Education,
Institute of Education Sciences, National Center for Education Evaluation and Regional
Assistance, Regional Educational Laboratory Southeast. Retrieved from
http://ies.ed.gov/ncee/edlabs
This report from REL Southeast describes 21 instruments designed to measure student engagement in
upper elementary through high school. For each instrument, it describes what is measured, purpose and
use, and information on psychometric properties. It includes student self-reports, teacher reports, and
observational measures of behavioral, emotional, and cognitive engagement. Measurement subject(s):
mostly middle and high school students, but some measures are completed by teachers or other
observers; a few measures are for middle and high school classrooms
Constructs measured: behavioral engagement, emotional engagement, cognitive engagement,
participation and effort, positive affect, school belonging, school value, academic engagement, selfregulation, deep cognitive strategy use, shallow cognitive strategy use, persistence, effort, attention,
behavioral disaffection, emotional disaffection, lack of effort, withdrawal from learning activities,
motivated involvement during learning activities, withdrawal during learning, perceptions of effort
toward learning, actions in extracurricular activities, feelings of connection to school, self-belief, learning
focus, planning, study management, disengagement, self-sabotage, anxiety, failure avoidance, uncertain
control, ongoing engagement with school, reaction to challenge, engagement with homework,
attendance, classroom attention, classroom concentration, trouble-avoidance, teacher-student
relationships, peer support for learning, family support for learning, control and relevance of
schoolwork, future aspirations and goals, extrinsic motivation for learning, engagement with reading,
active engagement, passive engagement, nonengagement, staying on task, participating in class,
expressing excitement, positive engagement behaviors, neutral engagement, inappropriate behavior,
student-engaged instruction, teacher-directed instruction
Teacher Engagement
Klassen, R. M., Yerdelen, S., & Durksen, T. L. (2013). Measuring teacher engagement: Development of
the Engaged Teachers Scale (ETS). Frontline Learning Research, 2, 33-52. Retrieved from
http://journals.sfu.ca/flr/index.php/journal/article/view/44
This article describes the development and validation of the Engaged Teachers Scale (ETS), a teacher
self-report measure of cognitive, emotional, and social engagement with school.
Measurement subject(s): teachers
Constructs measured: emotional engagement, social engagement with colleagues, social engagement
with students, cognitive engagement
Technological Literacy
Hohlfeld, T. N., Ritzhaupt, A. D., & Barron, A. E. (2010). Development and validation of the Student Tool
for Technology Literacy (ST2L). Journal of Research on Technology in Education, 42(4), 361-389.
This article describes the development and validation of the Student Tool for Technology Literacy (ST2L),
an objective, student-level low-stakes measure of technology literacy for middle school students.
Domains assessed include essential operational skills, constructing and deconstructing knowledge,
communication and collaboration, independent learning, and ethical, legal, and safety issues. Skills
assessed include software use and file manipulation; ethics, safety, and acceptable use; graphics,
presentation, and video editing; spreadsheets; browser use and email; and word processing and
flowcharts. The ST2L is currently only available for use in Florida, but a similar measure could be
constructed using this article. (See also the Florida Department of Education, 2009:
http://st2l.flinnovates.org/index.aspx)
Measurement subject(s): middle school students
Constructs measured: essential operational skills, constructing knowledge, deconstructing knowledge,
communication and collaboration, independent learning, ethical issues, legal issues, and safety issues
Kent State University Libraries. (2014). TRAILS: Tools for Real-time Assessment of Information Literacy
Skills. http://www.trails-9.org/index.php
TRAILS is a free, Web-based, multiple choice assessment of information literacy skills based for the 3rd,
6th, 9th, and 12th grade levels.
Measurement subject(s): 3rd, 6th, 9th, and 12th grade students
Constructs measured: developing topics; identifying potential sources; developing, using, and revising
search strategies; evaluating sources and information; recognizing how to use information responsibly,
ethically, and legally