Webinar:
Managing Research Requests in a
Local Education Agency
April 9, 2014
WebEx Instructions
2
WebEx Instructions
Attendees can
provide nonverbal
feedback to
presenters
utilizing the
Feedback tool.
3
WebEx Instructions
Feedback
options:
4
WebEx Instructions
Responses to poll questions
can be entered in the Polling
Panel.
Remember to click “Submit”
once you have selected your
answer(s).
5
WebEx Instructions
Attendees should utilize the “Q&A”
feature to pose questions to the
speaker, panelists, and/or host.
The host will hold all questions
directed toward the speaker or
panelists, and they will be answered
during a Q&A session at the end of
each discussion.
6
Welcome and Overview
Jane Alexander
Senior Research Specialist
REL Appalachia
Agenda
1.
Welcome and Overview
•
What is a REL?
•
REL Appalachia’s Mission
•
Introductions and Webinar Goals
2.
Forum Guide presentation
3.
Perspectives of an LEA representative
4.
Presentation of REL NEI toolkit
5.
Wrap-up
6.
Stakeholder Feedback Survey
8
What is a REL?
• A REL is a Regional Educational Laboratory.
• There are 10 RELs across the country.
• The REL program is administered by the U.S. Department of Education,
Institute of Education Sciences (IES).
• A REL serves the education needs of a designated region.
• The REL works in partnership with the region’s school districts, state
departments of education, and others to use data and research to
improve academic outcomes for students.
9
What is a REL?
10
REL Appalachia’s Mission
• Meet the applied research and technical assistance needs of Kentucky,
Tennessee, Virginia, and West Virginia.
• Conduct empirical research and analysis.
• Bring evidence-based information to policy makers and practitioners:
– Inform policy and practice – for states, districts, schools, and other
stakeholders.
– Focus on high-priority, discrete issues and build a body of knowledge over
time.
http://www.RELAppalachia.org
Follow us! @REL_Appalachia
11
Introductions and Webinar Goals
Jane Alexander
Senior Research Specialist
REL Appalachia
12
Speakers
Christina Tydeman, Ph.D.
Hawaii Department of Education
Robert Rodosky, Ph.D.
Jefferson County Public Schools
Julie Kochanek, Ph.D.
REL Northeast and Islands
13
Webinar Goals
• Provide an introduction to the National Forum on Education Statistics’
(NFES) Forum Guide to Supporting Data Access for Researchers: A Local
Education Agency Perspective.
• Increase understanding of how participants can promote research that
aligns with their LEA's strategic plan.
• Provide best practices and tools that can increase the effectiveness and
efficiency of processing research requests to reduce the burden on the
education agency.
14
Supporting Data Access for Researchers: An
LEA Perspective
Christina Tydeman
Hawaii Department of Education
Forum Purpose
To improve the quality, comparability, and utility
of elementary and secondary education data.
http://nces.ed.gov/forum
16
Background
• Forum Guide to Supporting Data Access for Researchers: A
State Education Agency Perspective
17
Problems
• Responding to and servicing researcher requests for data
access requires a significant amount of LEA staff time and
effort.
• Requests often fail to align with the agency’s strategic priorities
and, as a result, research findings infrequently support the LEA
in its efforts to educate students.
18
LEA Data Access Working Group
• Chair
• Christina Tydeman, Hawaii Department of Education
• Members
• Sheri Ballman, Princeton City Public Schools (OH)
• Marilyn King, Bozeman School District #7 (MT)
• Linda Rocks, Bossier Parish School System (LA)
• Arleen Thompson, Boston Public Schools (MA)
• David Weinberger, Yonkers Public Schools (NY)
• Project Officer
• Ghedam Bairu, National Center for Education Statistics (NCES)
• Consultants
• Abby Loughrey, Quality Information Partners
19
Document Purpose
• Helps the LEA to more efficiently and responsibly support
access to data.
• Recommend policies, practices, and templates that can
be adapted/adopted.
• Results in findings that enhance the agency’s efforts to
educate its students.
20
Audience
• Primary
• Local education agency policymakers
• Local education agency data managers
• Secondary
• Research community
21
Types of Data Access Requests
Permission to Access
New Data
Permission to Access
Existing Data
(i.e., primary research, or simply,
research)
(i.e., secondary research, or data
sharing)
•
•
•
•
•
Not previously collected
Available through research
subjects
Requires interactions with
staff and/or students
•
Previously collected
Available in data systems or
hard data sets
Does not require interactions
with staff and/or students
22
Foundation: The Research Agenda
The research agenda:
• Aligns with the LEA’s strategic plan.
• Guides researchers in designing projects that meet the needs
of the LEA.
• Is modified as the agency identifies new goals and priorities.
23
CORE PRACTICES:
Steps for Handling Data Access Requests
24
Core Practices
1. Help Researchers Understand Agency Data and the Data
Request Process
2. Create Effective Request Forms for Researchers
3. Review Data Requests Strategically
4. Manage the Data Request Process Efficiently
5. Release Data Appropriately
6. Monitor Data Use
7. Use Research Findings in the LEA
25
1. Inform Researchers
• Create, promote, and use a research agenda.
• Develop policies about training topics and requirements for
researchers.
• Identify or develop resources (e.g., training materials) to help
researchers better understand, request, and access new and
existing data.
• Determine when communications would be most useful to
researchers during the data request/access/use timeline.
26
2. Create Effective Request Forms for Researchers
• Identify the information needed by the LEA to evaluate data
access requests; design forms that are likely to generate this
information in an efficient manner.
• Consider the use of a Preliminary Data Access Request form to
minimize the burden on prospective researchers while still
providing adequate information for an initial review of the
request.
• Design and implement a Full Data Access Request form that
requires all of the information needed for the request to be
evaluated.
27
3. Review Requests Strategically
• Determine and post a process for channeling requests to the
appropriate office or offices in the agency.
• Assign responsibility for each incoming request.
• Standardize a review method or methods.
• Determine the participants and frequency of review team
meetings.
• Establish a method for handling requests that cannot wait for a
review team meeting.
• Develop guidelines for handling requests that involve data
from other agencies.
28
4. Manage the Request Process Efficiently
• Establish and clearly communicate eligibility criteria.
• Develop a searchable system (for internal use) that tracks the
entire data access process, from receiving requests to data
destruction.
• Determine how the LEA will communicate with researchers
and share formative information about the review status.
• Advise researchers about ways in which the request process
can be streamlined.
29
5. Release Data Appropriately
• Create a checklist for review prior to releasing data to verify
that the researcher has completed all mandated training, paid
any required fees, and signed all required agreements.
• Establish procedures for providing the researcher with relevant
metadata.
• If providing access to new data, ensure that an LEA staff
member—who is qualified to evaluate and enforce the legality
and ethics of research activities—monitors all interactions
between students and researchers (according to local policy).
• If providing access to existing data, determine the format and
media for releases and engage in data protection activities.
• Record all disclosures in compliance with expectations set forth
in FERPA.
30
6. Monitor Data Use
• Confirm researcher adherence to agreements through project
monitoring and data security audits.
• Consider reasonable modifications to the Data Accessing
Agreement when the researcher submits the appropriate forms
and justifies any modifications to the original request.
• Ensure that approved modifications are stored and tracked with
the original request.
• Review research outcomes to prevent any unintended
disclosure of personally identifiable information.
• Review any recordkeeping by the researcher to ensure that all
privacy rights afforded to students and staff will be guaranteed.
• Confirm project completion and data destruction.
31
7. Use Research Findings in the LEA
• Solicit results once research projects are complete.
• Encourage researchers to effectively communicate findings in
ways that are easily disseminated to a range of stakeholders
including policymakers, administrators, practitioners, and
parents.
• Use the research findings to inform the work of the LEA.
• Build mutually beneficial relationships with researchers
through ongoing communication.
32
Core Practices Review
1. Help Researchers Understand Agency Data and the Data
Request Process
2. Create Effective Request Forms for Researchers
3. Review Data Requests Strategically
4. Manage the Data Request Process Efficiently
5. Release Data Appropriately
6. Monitor Data Use
7. Use Research Findings in the LEA
33
Jefferson County Public Schools (KY) Model of Success
• Partnership with the University of Louisville’s College of
Education and Human Development.
• LEA teachers and administrators pursue doctoral degrees that
culminate in research around the agency’s strategic plan.
• Request process is automated through LEA website.
• Doctoral program’s leadership includes an LEA representative
who also sits on the university’s IRB (expedites LEA approval
process and strengthens partnership).
• Steps for informing and training researchers are embedded in
the college’s coursework.
• Monitoring is a coordinated effort between the LEA and
university.
34
Additional Resources
• Download the guide from the Forum website.
• http://nces.ed.gov/forum/pub_2014801.asp
• Refer to the SEA version of the guide.
• http://nces.ed.gov/forum/pub_2012809.asp
35
Questions?
36
Supporting Data Access for Researchers:
A Local Education Agency Perspective
Robert J. Rodosky, Ph.D.
Chief of Data Management, Planning, and Program Evaluation
Jefferson County Public Schools (Louisville, Kentucky)
Creating a Robust Data Access Framework in JCPS
• Key for creating a robust access framework was to implement the 7
Core Practices—together, the Core Practices are comprehensive in
nature.
• A well-developed districtwide Strategic Plan guiding the JCPS
research agenda was essential.
• You can prioritize data requests based on alignment with your
Strategic Plan.
38
Creating a Robust Data Access Framework in JCPS (cont.)
• It is important to have Internal Review Board (IRB) approval of
the researcher’s organization (particularly for Expedited and Full
Reviews).
• Assess the value of the data you will gain versus the burden of
collecting the data.
• Disruption to the data management and business processes
of JCPS.
• Disruption to instructional time in schools.
• Use a framework as guidance for developing a data request
tracking system—our JCPS Data Request Management System.
39
Impact of Creating a Data Access Framework in JCPS
• Researchers can advance our JCPS Strategic Plan research agenda:
– Goal 1 – Increased learning
– Goal 2 – Graduation and success beyond graduation
– Goal 3 – Stakeholder involvement/engagement
– Goal 4 – Safe, resourced, supported, and equipped schools
40
Impact of Creating a Data Access Framework in JCPS (Cont.)
• The process of managing data requests is more efficient and
effective.
• We are supporting the JCPS research capacity (we are faced with
relatively limited personnel to handle the data and research needs).
• It is more likely that the research findings will be used in our
system.
• We reserve the right to decline requests, even after IRB approval.
School principals can decide to decline after the central office
approves a request.
41
Impact of Creating a Data Access Framework in JCPS (Cont.)
• Better protection of personally identifiable information (PII) and
compliance with established federal regulations:
• Family Educational Rights and Privacy Act (FERPA)
• Protection of Pupil Rights Amendment (PPRA)
• National School Lunch Act (NSLA)
• No one is exempt from following the JCPS protocol for research.
• Legal counsel is used only when necessary.
• Meshes well with IRB requirements.
• Promotes the use of consent forms when necessary.
• Promotes careful and appropriate releasing of data (e.g., website
rather than e-mail, de-identification, anonymization).
42
Working With Researchers: Lessons Learned
• We need to agree that JCPS’ key goal is to support useful, timely,
valid/reliable, and ethical research to improve student
achievement.
• If focus is on student achievement, the external research actually
helps reduce JCPS Data Management and Research workload.
• We distinguish between permission to access existing data and
permission to access new data, while balancing research benefits
with the burden placed on schools.
• We need to emphasize the requirements to comply with
regulations associated with FERPA, PPRA, and NSLA.
• We need to be clear to researchers that we need actionable
research that can be easily understood to support us in making
data-driven decisions.
43
Working With Researchers: Lessons Learned (cont.)
• Timing is important in reporting data—we need to focus on the right
window of opportunity.
• Theoretical implications are OK, but practical implications are better.
• Collaboration with top authorities from local universities is key to
this process.
• We are disproving the longstanding association between large
organizations and the phrase, “data rich, but information poor.”
• We reserve the right to prioritize our data services to our priority
schools; then we assemble the data needed by external researchers.
• If data accessing agreements are involved, we make sure to connect
with our Legal Office for guidance.
• When working in collaborative research endeavors, we emphasize
that a true partnership is a two-way street.
44
Suggestions Based on Our Experience in JCPS
Suggestions for creating a data access framework with few resources:
• Design a workflow for managing data requests
– Data Request Management System
• Develop a prioritization process
– Use your Strategic Plan
– Use a framework like the Program Evaluation Standards (1981; 1994;
2011) developed by the Joint Committee, which highlight utility,
feasibility, accuracy, and propriety (access this at
http://www.jcsee.org/program-evaluation-standards)
• Protect instructional time in schools (assess the value of data vs.
burden of collecting the data)
• Maximize the impact by developing guidelines on reporting processes
– Executive Summary, Managerial Report, and Technical Report
45
Questions?
Dr. Robert Rodosky, Chief
Data Management, Planning, and Program Evaluation Services
Jefferson County Public Schools (Louisville, KY)
Phone: (502) 485-3036
robert.rodosky@jefferson.kyschools.us
46
REL Northeast & Islands
School District and State Tools for
Working With External Researchers
Julie Kochanek, Director of Research
REL Northeast & Islands
REL NEI Tools for Working With External Researchers
• Product developed by REL Northeast & Islands for the Urban School
Improvement Alliance.
• Alliance research agenda focuses on data use in school and district
decision-making.
• Alliance member needs identified:
• How to leverage requests for data to serve district research priorities?
• How to review, prioritize, and manage research requests?
• Toolkit created with advisory committee of alliance members.
48
District and State Tools for Working with External Researchers
• Identifying District or State Research Priorities
• [Insert link to published product.]
• Vetting Research Requests
• [Insert link to published product.]
• Recruiting External Researchers
• [Insert link to published product.]
• Disseminating Research Results Externally
• [Insert link to published product.]
49
Wrap Up and Closing Remarks
Stakeholder Feedback Survey
Jane Alexander
Senior Research Specialist
REL Appalachia
Stakeholder Feedback Survey Access Information
Please visit:
https://checkbox.cna.org/ResearchRequestsLEA.aspx
to provide feedback on today’s webinar event.
51