USED Monthly Pat B Report, C3 IIS, North Carolina, May 2014
1
Race to the Top
Monthly Report - C3
Application sub-criterion: (C)(3): Using data to improve instruction
STATE’s goals for this sub-criterion:
• Develop and implement a statewide Instructional Improvement System (IIS).
• Provide professional development and communications to prepare the teachers, students,
parents and administrators for implementing the IIS.
Part B:
1. Is the State on-track to implement the activities and meet the goals and performance
measures that are included in its approved scope of work for this sub-criterion? If
so, explain why. If not, explain why not.
General Progress
Yes, the state is on track to implement the activities and meet the goals and performance
measures that are included in our approved scope of work. To date, NCDPI has
implemented the Student Information System (based on Pearson PowerSchool), the
Instructional Improvement System (based on Pearson Schoolnet) to provide curriculum &
instruction, classroom & benchmark assessment, and data analysis tools, a collaboration
and course management tool (based on Pearson OpenClass), the Educator Evaluation
System for both teachers and principals (based on Truenorthlogic Educator Evaluation
software), and the Professional Development (PD) System (based on Truenorthlogic
Professional Learning Management System software) which currently offers stateprovided professional development opportunities. The NCDPI also began piloting the use
of the PD system for the creation and administration of local professional development
and began planning for the deployment of the Summative Assessment delivery platform
(based on Pearson TestNav).
The NCDPI has continued to add high quality, NC standards-aligned instructional
resources and classroom and benchmark assessment items on a rolling basis as they are
developed or purchased. The NCDPI worked with Pearson recently to integrate
OpenClass with the NC Learning Object Repository (LOR), enabling educators to search
for resources available in the LOR and to easily add them to their courses that are
managed through OpenClass.
While there have been some challenges during implementation of the Home Base
applications, there have been many accomplishments as well.
USED Monthly Pat B Report, C3 IIS, North Carolina, May 2014
2
The NCDPI and vendor teams continue to grow the functionality of Schoolnet, with each
new version release meeting more of the requirements North Carolina requested. A recent
upgrade brought North Carolina into the 15.4 version, which added expanded search
capabilities including North Carolina standard dot notation, new technology enhanced
assessment item types, including drag and drop, co-authoring folders which fosters
collaborative item authoring, and the ability for students to leave comments after each
test item as well as at the end of the assessment, to name just a few new features.
The NCDPI continued to roll out and train on OpenClass, Home Base’s collaboration and
course management tool, which will allow teachers and students to collaborate with one
another and for teachers to manage instructional resources and activities. LEAs and
charter schools began implementing the system in waves, based on when they expressed
interest in using the tool. As of the end of April 2014, NCDPI had trained 77 LEAs on
OpenClass. Sixty-seven LEAs are integrated with PowerSchool, and 36 are fully set up
and ready to use the system. Most of these LEAs, including both districts and charter
schools, are piloting the use of OpenClass in select schools, or in some cases with select
teachers within schools.
The Educator Evaluation system is fully implemented. The NCDPI has also completed
the development of the professional development tools in Home Base, completed pilots
and launched the PD system. NCDPI continues to work with the vendor to migrate
existing modules and professional development resources to the new tool in Home Base.
NCDPI has continued planning for the build of the final tool in Home Base, TestNav,
which will allow for delivery of summative assessments. NCDPI expects to have the
build for this tool complete by the summer of 2014 and the system will be ready to pilot
in the fall of 2014.
In addition to the progress made with the systems within Home Base, NCDPI also
continues to streamline the process for the Home Base Support Center (HBSC) protocols.
The support center has continued to make great strides to improve its capacity to support
Home Base users in the field. The Support Center staff who specialize in the instructional
components were recently trained on the more technical, back end side of OpenClass so
that when vendor implementation teams phase out, NC has the capacity to continue to do
the work of creating new campuses and integrating them with PowerSchool, to name one
example. For weekly updates on the systems and the Home Base Support Center is
making, please visit http://www.ncpublicschools.org/homebase/status/.
Reset for Sustainability
The NCDPI has done work internally to ensure that the organizational structure of the
Department is one that can sustain Home Base well past the end of the RttT grant. The
NCDPI is in the process of identifying current and future needs, including deliverables,
knowledge transfer and human resources needs, and creating and implementing plans that
address those needs.
USED Monthly Pat B Report, C3 IIS, North Carolina, May 2014
3
a) Functionality
Home Base is still on track to have all systems implemented by the end of the 2014-15
school year.
b1)
Student Information System (PowerSchool)
Although PowerSchool is not an RttT-funded deliverable, some of the
requirements of the Instructional Improvement System (IIS) contract, such as the
attendance, grade book and parent and student portals, are satisfied by the
PowerSchool component and the integration between PowerSchool and
Schoolnet. The “PowerSchool for Schoolnet” requirements that have been
delivered are functioning as intended.
b2)
Instructional Improvement System (Schoolnet)
The Instructional Improvement System is made up of essentially three main parts:
curriculum and instruction tools, assessment tools, and reporting tools. The
majority of the required functions are in operation and available to all LEAs,
while a few of the system requirements are still under development. For example,
there is still development underway to provide the ability for educators to rate
instructional content and enter reviews and comments as part of the vetting
process. Currently educators can recommend a resource, with a running count of
the times users have recommended the resource showing on the resource, but the
ability to rate a material and enter reviews and comments as a part of the vetting
process is forthcoming. There is also development work occurring to allow
parents to identify activities and resources aligned with content standards that
their child has not yet mastered. The requirements that the vendor is still
developing were clearly stated as such in the initial contract, and overall progress
is on track.
The performance issues that users experienced early in the school year have
improved. The state made considerable progress and has seen marked
improvement in performance and availability across the state, as indicated from
both performance monitoring reports and communications directly with LEAs.
b3)
Educator Evaluation for teachers and principals (Truenorthlogic EE)
The Home Base Educator Evaluation System is functioning as intended and is
being used by all LEAs to complete teacher and principal evaluations. NCDPI
has listened to input from LEAs and had made a number of changes in the system
throughout the year to make the system more efficient and user-friendly.
b4)
Professional Development (Truenorthlogic PLMS)
The Professional Development (PD) System has rolled out and provides a
platform for professional development content for educators and integrates
seamlessly with the Educator Evaluation System. The PD system will include all
USED Monthly Pat B Report, C3 IIS, North Carolina, May 2014
4
state-sponsored PD courses. Upcoming work on the PD system includes the use
of this tool for local professional development which will enable LEAs to create
and manage district/charter-level instructor led and self-paced courses in a
common catalog. It will also enable districts and charter schools that opt in to
utilize expanded functionality to manage both State and local-PD content,
tracking, reporting, etc.
b5)
Collaboration and Course Management System (OpenClass)
The Home Base collaboration and course management system (OpenClass) is
continuing to roll out to LEAs in waves. OpenClass has now been integrated with
the NC LOR so that teachers and students may use curriculum and instructional
resources that are stored in the LOR.
b6)
Summative Assessment delivery platform (TestNav)
NCDPI has continued to plan for the build and rollout of the Summative
Assessment delivery platform, which is the final tool in Home Base. NCDPI
expects to pilot this tool to deliver some of the NC final exams in the fall of 2014.
b7)
Identity Access Management System (IAMS)
The IAMS service has gone live and currently integrates with 5 cloud-based
applications that are commonly used in the LEAs. The state continues
discussions with Cloud team regarding integration of the Identity Access
Management System with Home Base applications and is holding weekly
technical meetings to identify technical integration options.
Ensuring High-Quality, Useful Content in Home Base
Curriculum and instructional resources were made available to teachers and students
from the first day of the IIS rollout, and the work to enhance the content continues (see
General Progress subsection above).
Materials from the North Carolina Department of Cultural Resources are now available in
the IIS. A new collection of 2,100 cultural resources from PK-12 has been added to the
IIS bank of materials. It includes photos, blue prints, lesson and research guides, online
activities and exhibits, field trip planners, articles and multimedia produced by the North
Carolina's 27 historic sites, seven museums of history and two museums of art in addition
to contributions from the N.C. Arts Council, N.C. Symphony, State Archives, State
Library and state offices of archaeology and historic preservation.
In addition to state-provided resources, districts, schools, and teachers have uploaded or
created many of their own. As of May 14, 2014, there are over 5,500 instructional
resources that have been added by districts or schools.
USED Monthly Pat B Report, C3 IIS, North Carolina, May 2014
5
The teachers who will be a part of the Governor’s Teachers Network will also contribute
North Carolina teacher-created instructional materials that will be shared across the state
through the IIS platform.
b) Stakeholder Guidance and Engagement
NCDPI continues to communicate with LEAs and charters to discuss and demonstrate
portions of the system.
d1)
LEA Engagement
NCDPI continued communication and stakeholder engagement efforts to assist districts
and charter schools with the transition to Home Base and to continue gathering input
regarding educator needs and preferences related to Home Base functionality and
content; this communication and stakeholder engagement took the form of face-to-face
meetings, trainings, webinars, weekly newsletter updates, providing a website that
remains up to date with current issues and resolutions, updates, calendars and other
materials, and weekly status updates posted on the website for each system within Home
Base.
The NCDPI also takes into account the needs expressed by the LEAs and works with the
works with the vendor(s) to take actions to meet those needs. For example, the IIS is not
natively configured to allow users to have the ability to review and approve instructional
materials without also having the role of Curriculum Manager. We heard from many
small LEAs that they had select teachers responsible for this work in their schools. They
did not need to provide those teachers with the ability to schedule lessons for all teachers,
create or edit curricula and curricular units, and map courses to curriculum. They simply
wanted a select group of lead teachers or department heads to create materials and share
and the school level and to review the materials submitted to the school materials banks.
North Carolina added new roles by parsing out the specific functionality needed and
creating the new roles that addressed LEA needs.
d2)
Stakeholder Meetings
NCDPI continues to hold meetings with the Home Base Stakeholder Advisory
Committee. This group meets approximately quarterly to discuss Home Base. Members
include staff from LEAs, education agencies, and the Office of State Budget and
Management (OSBM).
d3)
Working with partner sites
The NCDPI is finalizing plans to expand the work of the partner sites. The partner site
coordinators will be connecting with “power users” of the opt-in components of Home
Base at the district, school and teacher levels to learn and share best practices with both
these early adopters and with districts that are not currently using the opt-in components.
The objective of this activity is to discover and understand potential obstacles to broader
adoption and to share lessons learned. More information about the upcoming work of the
partner sites group is attached as Appendix A.
USED Moonthly Pat B Report,
R
C3 IIIS, North Caroolina, May 20014
6
c) Communicat
C
tions
NCDPI contin
N
nues issuingg newsletterss to numerouus listservs, including
i
thee Biweekly
U
Updates
listseerv on whichh there are people who have
h
self-subbscribed. Thee biweekly also
a
goes out to teechnical directors, princippals, teacherrs, superintendents and more.
m
Staff
coontinues to keep
k
the Hom
me Base webbsite fresh with
w timely and
a pertinentt informationn.
The Home Baase communnications team
T
m has also begun
b
work on
o a “good news”
n
video
seeries of 3-5 minute
m
shortts that will highlight
h
teacchers and theeir use of Hoome Base inn the
cllassroom. Th
hese videos are a part off a larger cam
mpaign to shhare the Hom
me Base and Race
too the Top succcess storiess across the state.
s
d) Training
T
and
d Profession
nal Development
Professional developmen
d
nt for Home Base
B
has beeen available in every reggion on an
onngoing basiss and supporrts state, disttrict, and chaarter staff off varying rolees.
Those LEAs that
T
t opt in will
w receive additional
a
traaining on thee systems wiithin Home Base.
B
T
There
have allready been 27 LEAs thaat have receiived Schoolnnet certificattion training,
based on the expressed
e
coommitment to
t opt in to Home
H
Base. Planning for upcoming
seessions of th
hat training iss currently underway.
u
Training docu
T
uments havee continued to
t be createdd and made available
a
to LEAs.
L
Thosee
m
materials
can
n be found inn the IIS (Schhoolnet) as resources
r
under the subjeect, “Home Base
B
T
Training.”
Training mateerials can alsso be found on the Hom
T
me Base webssite at
htttp://www.n
ncpublicschools.org/hom
mebase/traininng/materialss/.
There continu
T
ue to be webbinars held foor targeted needs
n
based on
o responsess the state
reeceives from
m the districtss and charterr schools. Foor example, the state held a 2-part
w
webinar
seriees on roles annd permissioons in Schooolnet. In ordeer for an eduucator to havve
USED Monthly Pat B Report, C3 IIS, North Carolina, May 2014
7
access to the tools he or she needs, and to only those tools needed for a particular role, the
roles and permissions must be correctly set. An archived recording of the recent webinars
can be found at http://www.ncpublicschools.org/homebase/resources/videos/webinars/.
The NCDPI also identified the need for additional staffing to create more documentation
for end users, such as user guides, additional training materials, and process documents.
Two new positions were created to address this need and the hiring process has begun.
e) Support Center
The Home Base Support Center has continued to help users in the field receive answers
to their questions and resolutions for their issues.
f) Opting In
The formal opt in process was shared with Superintendents. The NCDPI created a simple
web form that allows Superintendents or their designees to opt in, and NCDPI set a
deadline of July 15, 2014 for opting into Home Base. As of May 20, forty-one districts
and seven charter schools have already formally opted in to use Home Base, and many
others have expressed their intent to opt in.
2. Does the State have evidence indicating the quality of implementation for this subcriterion? What is/has the State doing/done as a result of this information?
a) Quality of Implementation
Implementation work will include continued training and support opportunities for state,
district, and charter staff, as referenced in the training and professional development
sections of this report. In addition, partnerships with the LEAs will benefit both those
volunteer districts and the state in the implementation process.
In order to analyze the success of the implementation, the state has requested and will
collect data on system performance, usage, and issues.
i. System performance
NCDPI and Pearson continue to run and monitor system performance monitoring
reports using a tool called “Keynote” to report the average system response time
and the system up-time or availability. These reports include metrics as defined in
the service level agreement (SLA), and they offer a relatively easy way to ensure
that the Schoolnet application performs within the parameters defined in the SLA.
This tool may also be used to conduct real-time performance monitoring to
identify the sources of performance issues at the time in which they are occurring.
Schoolnet has been performing well consistently throughout 2014, though after
upgrading to version 15.4 on the weekend of May 10, the NCDPI and Pearson
teams have needed to apply some bug fixes to improve performance. The reports
of the performance issues related to the 15.4 upgrade were not widespread and
were resolved by May18 The NCDPI and vendor teams also continue their work
USED Monthly Pat B Report, C3 IIS, North Carolina, May 2014
8
to improve overall performance of Home Base and increase the number of
concurrent users supported.
ii. Usage
The state is also running reports that monitor usage of the IIS. The reports
include information such as current enrollment system-wide, the number of
successful logins, failed logins, assessments created, assessments administered,
lesson plans created, and the number of resources at the state level. NCDPI is in
the early stages of that data collection and is also working with evaluation team to
determine what data to capture to further explore stakeholder use and satisfaction
with the tool.
As of May 15, LEAs and charter schools have developed and delivered 521,792
classroom assessments using the IIS (Schoolnet). 417,492 of these classroom
assessments (80%) were delivered online. LEAs and charter schools have
developed and delivered 911,001 benchmark assessments using Schoolnet.
569,019 of these benchmark assessments (62.5%) were delivered online.
iii. Issues
The state has requested and received reports on the issues reported for the IIS and
Educator Evaluation System components of Home Base. Those reports come
from multiple sources, including the Home Base Support Center, direct contact
from LEAs, and from the vendors when issues have been escalated to tier 2 and
tier 3 support. The project team meets regularly to discuss, analyze, prioritize, and
troubleshoot the main issues reported.
3. What obstacles and/or risks could impact the State’s ability to meet its goals and
performance measures related to this sub-criterion?
Risk
Strategy to Mitigate
Single Sign-On
•
Pearson’s single sign-on (SSO)
solution, which allows users to
access both Schoolnet for
benchmarking and classroom
assessment and Truenorthlogic for
educator evaluation has, at times,
failed to grant users access to the
applications. Access to Schoolnet is
currently unavailable during
PowerSchool maintenance windows.
The NCDPI project staff members are
working with the Cloud Identity
Access Management team toward a
long-term SSO solution. The current
projection is to test and implement that
functionality in the 2014-15 school
year.
USED Monthly Pat B Report, C3 IIS, North Carolina, May 2014
Risk
Performance
NCDPI continues to monitor
performance issues. Late in 2013,
there were issues related to long
page loads and time outs,
particularly as usage increased. The
IIS has seen marked improvement,
but the project team continues to
monitor and make improvements
until the system performance and
availability comply with our Service
Level Agreement.
Data Validation
Data validation processes have been
delayed to ensure that data move
accurately from the local LEA
instance of PowerSchool, to the
PowerSchool Operational Data Store
(ODS), and into Schoolnet.
Lack of a Combined Training Site
The Training environments for
PowerSchool and Schoolnet are not
integrated in the way they are in the
Home Base production environment.
The vast majority of functionality is
still available in each application,
but the integrated functionality
between the two applications is not
available in the Training
environment. For example the ability
to demonstrate how students can
take an assessment in Schoolnet and
post the grade to the PowerSchool
gradebook is not available in the
Training environment but is
available in Production.
Adoption and Sustainability Plans
Because Home Base has many
components that are not required,
NC must create a suite of tools that
provide a combination of high
quality resources and superior value
compared to the tools available in
the open market. Many LEAs and
Strategy to Mitigate
• Performance improvements have been
consistently evident for the last several
months. System monitoring and
corrective action continues. The
NCDPI and Pearson continue to make
infrastructure and configuration
improvements and system code
changes to improve IIS system
performance. End-users will continue
to see incremental improvements over
the next few months
•
•
•
•
•
Data validation and quality assurance
testing are underway. NCDPI has
requested additional Pearson resources
to address the data validation testing,
and Pearson has committed to provide
the resources to get the data validation
testing completed.
The migration of the Schoolnet
environments to the new data center in
Iowa City, Iowa was delayed due to
focused efforts on improving
performance. The move is scheduled
to take place in the summer. In the
new data center, Pearson will build an
integrated training environment that
will mimic the Production
environment.
Support documents and trainings have
been developed to support users in
understanding how the actual site
functions.
Created an integrated suite of tools
that provide curriculum and
instructional tools, classroom and
benchmark assessment tools, and
professional development tools that
offer high degrees of functionality and
integration.
Created a cost-sharing model designed
9
USED Monthly Pat B Report, C3 IIS, North Carolina, May 2014
Risk
schools have expressed that the
potential of Home Base is powerful,
but they are cautious about replacing
extant tools or expanding to Home
Base.
Strategy to Mitigate
to bring significant cost-savings to
districts and create a sustainable costsharing model ($4 per student).
• Developed a training plan to ensure
users to have sufficient training for
implementation.
• Continue to gather user feedback to
ensure tools meets educator needs.
• Conducting LEA and charter outreach,
including tracking current dispositions
toward opt-in.
Communications
NCDPI must maintain complete
transparency with LEAs and charters
and must frequently and accurately
communicate status and issues
regarding the Home Base Suite of
applications. NCDPI must set
accurate expectations for user
communities in the use and
capabilities of the Home Base suite
of applications and must accurately
communication costs in time for
LEAs and charters to plan for Home
Base adoption.
•
•
•
•
10
New weekly Project Status updates on
Home Base website (see
communications section above for
details)
Carefully designing key message
Widely distributing biweekly updates
Engaging in speaking engagements
and meetings across the state to talk
about Home Base
Evaluation: Based on the responses to the previous question, evaluate the State’s
performance and progress to date for this sub-criterion (choose one)
Red (1)
Orange (2)
Yellow (3)
Green (4)1
1
Red – substantially off-track and/or has significant quality concerns; urgent and decisive action is required; Orange –off-track
and/or there are quality concerns; many aspects require significant attention; Yellow –generally on-track and of high or good
quality; only a few aspects require additional attention; Green – on-track with high quality.