1.
Applicant Information
Principal applicant:
Janice Patterson, Assistant Director
Centre for Teaching and Learning
University of Toronto Scarborough
1265 Military Trail
Toronto, ON M1C 1A4
janicecp@utsc.utoronto.ca
416.287.7504
Co-applicants:
Dr. Sarah King, Senior Lecturer, Coordinator, The Writing Centre
Centre for Teaching and Learning
University of Toronto Scarborough
1265 Military Trail
Toronto, ON M1C 1A4
sking@utsc.utoronto.ca
416.287.7480
Dr. Allyson Skene, Coordinator, Facilitated Study Groups
Centre for Teaching and Learning
University of Toronto Scarborough
1265 Military Trail
Toronto, ON M1C 1A4
askene@utsc.utoronto.ca
416.208.2897
1
2.
Overview
Descriptive Title – Data scaffolding for data novices: a strategic approach to
supporting educational developers and faculty in educational research
quantitative analysis.j
Intended Outcomes –
a. We will:
• produce a document that staff in our unit and other instructors at UTSC and
outside of UTSC, can use to plan and improve educational research, and that
will facilitate communication with colleagues with expertise in quantitative
analysis;
• enhance dialogue between instructors and educational developers from a
range of disciplines;
• enhance program assessment.
b. Our timeline 
September 2011: hire graduate student

October – December: analysis of data collection process for two peer
programs

January 2012: draft guidelines

February – April 2012 guidelines implemented, reflection and assessment of
guidelines

May – June 2012: conference presentations of results and guidelines

July – August 2012: final preparation of report and guidelines
Rationale –
Our department, the Centre for Teaching and Learning, like similar units at other
institutions, includes several programs (e.g. Writing Centre, Facilitated Study
Groups, English Language Development) that support student learning through
peer programming. Coordinators of these programs have dual roles: managing
2
centres, which includes developing and assessing their peer programs (e.g study
groups, peer coaches, peer facilitators); and engaging in educational
development work that supports their own teaching and that of their colleagues
on the UTSC campus. While some people who fill this role have a background in
education research, many come from diverse disciplinary backgrounds.
To more effectively support the Scholarship of Teaching and Learning (SoTL),
we are asking for a grant that would enable us to establish general guidelines
and best practices for program coordinators around defining, conducting,
analyzing and disseminating educational research results, in accordance with
SoTL principles. Our intent is to share the latter guidelines and best practices
with colleagues in our unit and on our campus and others outside of the
University of Toronto. For example, we know of instructor colleagues facing
similar challenges, e.g. collecting and analyzing data to assess pedagogical
initiatives for a range of purposes e.g. teaching portfolios, curriculum design or
budget requests.
Our goal is to equip our coordinators (and others) with a flexible toolkit to enable
them to begin (or further enhance) educational research regarding their program
responsibility, advance their professional skills in conducting educational
research, and facilitate communication with experts in statistical analysis who
might be supporting their work. Specific challenges program coordinators face
include: a) identifying efficient means of collecting non-credit course data; b)
uncertainties about appropriate measures for educational research; c) lack of
expertise with data management software.
Our plan is to:
1. hire a graduate student with knowledge of educational research and expertise
in statistical analysis, who would be provided with a thorough introduction to
the student programming in our various units in order to understand our core
activities and specific data needs and interests. This will include
3
understanding how data gathering can be difficult or awkward to obtain,
manipulate and analyze in each of our areas.
2. have this graduate student do an analysis of those needs and interests in
order to make recommendations to enhance existing and develop new
methods of gathering, analyzing and presenting data for internal use and
sharing externally
3. extract generalizable principles and use them to create general guidelines for
educational developers on designing assessment of educational programs,
identifying key terms and communicating with experts in statistical analysis
who may not have expertise in educational research
4. we will network with educators and educational developers who have done
work in this area for their expertise and insight into the inherent challenges
and opportunities
5. test guidelines through implementation and reflective practice
6. share guidelines with our colleagues on campus, and with other teaching
centres.
This will advance the EDC Living Professional Development Plan by creating a
framework that might be used by other educational developers trained from a
range of disciplines.
Scholarship - Quantitative analysis is recognized as one of the three major
types of analyses done for SoTL (Weimer, 2006), and quantitative data are often
regarded as essential for convincing senior administrators of the value of
programs. Educational developers are expected to assess their own programs,
as well as support other faculty in developing assessment. However, developers
come from diverse disciplinary backgrounds (McDonald, 2010), and there is no
single training process or certification (Hoessler et al, 2010). Faculty engaged in
SoTL are similarly diverse. Educational Developers and many faculty have little
to no background in statistical analysis. These individuals find themselves at a
disadvantage when it comes to investigating the impact of their programs.
4
While guidelines for quantitative analysis in education written with educational
developers in mind exist (e.g. Hohmann, 2006), and educational researchers with
expertise in statistical analysis offer consultations and share best practices at
conferences (e.g. Hoessler, 2007), these are necessarily limited. Furthermore,
given faculty workload, it is unreasonable to expect faculty to become experts in
quantitative analysis. Educational research is often conducted in teams, and so it
is reasonable to prepare developers to work with experts in working with
quantitative data.
We must note here that our intention is not to favour quantitative analysis over
qualitative; we recognize that each paradigm serves distinct purposes in
educational research. However, given the size of the EDC grants, we propose to
start with the development of quantitative research guidelines. In subsequent
years, we would look for ways to expand our project to include qualitative
educational research guidelines for educational developers and instructors.
Dissemination – We will share the results of this project with our instructor
colleagues on our campus through professional development activities the
Centre for Teaching and Learning provides, publishing on the CTL website, and
taking advantage of other opportunities at the University of Toronto. We will also
share our results at conferences, for example, STLHE and/or the Educational
Developers Caucus.
3. Budget –
• 51 hours x $40.04/hr ($35/hr plus benefits) for a graduate student = $2,042.04.
• 8 hours x $55/hr for technical support through our Information and Instructional
Technology Services = $440
TOTAL = $2,482.04
5
4. References cited
Hoessler, C., Britnell, J., and Stockley, D. (2010). Assessing the Impact of
Educational Development Through the Lens of the Scholarship of Teaching
and Learning. New Directions for Teaching and Learning 122, 81-89.
doi: 10.1002/tl.400
Hoessler, C. (2007). Demonstrating Impact in Scholarship of Teaching and
Learning with Statistics. STLHE 2007 Evolving Scholarship, June 13-16.
McDonald, J. (2010). Charting Pathways into the Field of Educational
Development. New Directions for Teaching and Learning 122, 37-45.
doi: 10.1002/tl.396
Hohmann, U. Quantitative Methods in Education Research. Originally prepared
by Professor John Berry. Centre for Teaching Mathematics, University of
Plymouth, 2005. Retrieved from
http://www.edu.plymouth.ac.uk/resined/Quantitative/quanthme.htm
Weimer, Maryellen. (2006). Enhancing Scholarly Work on Teaching and
Learning: Professional Literature that Makes a Difference. San Francisco,
CA : Jossey-Bass, 2006.
6