Race to the Top
Progress Update - C3
Application sub-criterion: (C)(3): Using data to improve instruction
STATE’s goals for this sub-criterion:
 Develop and implement a statewide Instructional Improvement System (IIS).
 Provide professional development and communications to prepare the teachers, students,
parents and administrators for implementing the IIS.
Part B:
1. Is the State on-track to implement the activities and meet the goals and performance
measures that are included in its approved scope of work for this sub-criterion? If
so, explain why. If not, explain why not.
a) General Progress
The student information (PowerSchool), core IIS (Schoolnet) and educator evaluation
components of Home Base were implemented across the state at the start of school for
both year-round and traditional calendared schools for SY2013-14, as planned. Also as
planned, the NCDPI provided single sign-on access to the suite of tools that are currently
available, using PowerSchool as the portal.
While the suite of tools has been implemented on schedule, there have been a number of
issues through which the state and vendor teams continue to work. Because the
instructional improvement and educator evaluation systems, both included within the
(C)(3) portion of the Race to the Top project, have been integrated with the student
information system, PowerSchool, and because PowerSchool is the portal into all
systems, there is a dependency on the accuracy of data in PowerSchool and on the correct
configuration between all systems and the state’s Operational Data Store (ODS) in order
for Home Base to function as intended. In addition, each system has its own set of issues
that one might expect with the rollout of such large enterprise information systems. The
project team has worked to troubleshoot and resolve issues as they arise. Details about
the Home Base issues are included below in the answer to question 3 of this report.
While there have been some challenges with the “big bang” rollout approach for all of the
Home Base applications, there have been many accomplishments as well. In the last
several months, the NCDPI and vendor staff have rolled out the Student Information
System (PowerSchool), the Instructional Improvement System (Schoolnet), and the
Educator Evaluation System (Truenorthlogic Performance Management System). The
scope of the Educator Evaluation system includes both teacher evaluation and principal
evaluation; the teacher evaluation tool was available at the start of the school year and the
principal evaluation tool became available on October 1, 2013. The team has kicked off
the Professional Development (PD) System project (an additional module of the
Truenorthlogic Performance Management System) and has begun work piloting the first
PD course registration and delivery. NCDPI has also begun the work of rolling out the
NCDPI / NC RttT PMO
Page 1 of 14
11/27/2013
Home Base’s learning management system, OpenClass. In addition to the progress made
with the systems within Home Base, NCDPI also put in place the Home Base Support
Center which was created, staffed, and made available to support the LEAs. The support
center has continued to make great strides to improve its capacity to support Home Base
users in the field.
b) Functionality
Home Base functionality is still on track to roll out in stages throughout 2013-14 and
2014-15.
b1)
PowerSchool
In the IIS contract, some of the requirements for the IIS are satisfied by
PowerSchool and the integration between PowerSchool and Schoolnet. Some
examples of functionalities satisfied by PowerSchool are attendance, gradebook
and parent and student portals. Many of the functional requirements are already in
place, and others are in development.
b2)
Schoolnet
Schoolnet is made up of essentially three main parts: curriculum and instruction
tools, assessment tools, and reporting. The majority of the required functions are
operating and available to all LEAs, while some are under development. Overall,
the functionality is in place and working as intended, though in some instances
performance issues have caused some users to not fully utilize the system. For
example, we have seen instances in which system performance degraded enough
to cause user sessions to time out and instances in which students had trouble
taking an assessment because of slow system response time. NCDPI and vendor
staff are working to resolve system performance issues and anticipate better
performance by the end of the calendar year when the training, test and
production environments are moved to a new, more robust hosting data center.
There are also some issues, in the assessment admin component for example, that
will be addressed by the newest release of Schoolnet.
b3)
Educator Evaluation for teachers and principals
NCDPI and Truenorthlogic launched the Home Base Educator Evaluation System
in July 2013 to provide functionality for teacher and principal evaluations. This
system fully supports the North Carolina teacher and principal evaluation rubric.
There has been a fairly steep learning curve for teachers and principals with this
system because of the workflow built in to support the NC rubric. Initially, there
were some issues with staff showing as active in schools in which they no longer
worked. This issue was caused by inaccurate data in some of the source systems
used to populate the system. The great majority of these issues have been
successfully addressed. The system has also had performance issues that at times
made it difficult for teachers and principals to complete self-assessments,
observations, and other evaluation activities. NCDPI has met with
Superintendents over the last few weeks and had developed an action plan to
NCDPI / NC RttT PMO
Page 2 of 14
11/27/2013
address LEA concerns. All-in-all, the system is functional and is being used by
all LEAs to complete teacher and principal evaluations.
b4)
Professional Development
In August, the state and vendor teams began the planning meetings for the
development of the Professional Development System that will provide a platform
and professional development content for educators and will integrate seamlessly
with the Educator Evaluation System.
The new online Professional Development (PD) system pilot launched in
November with a four-week, facilitated course Introduction to Universal Design
for Learning. This pilot course is offered in two sections of 30 participants each
and runs Nov. 11 through Dec. 15. There is exceptional interest in the
Professional Development System pilot on behalf of the LEAs, as evidenced by
the fact that the two sections filled up within two days of registration opening.
The purpose of this initial pilot is to test the functionality of the new Online
Professional Development System. As of today, we know that the enrollment
function and wait list features work well.
b5)
OpenClass
OpenClass, Home Base’s learning management system, is rolling out to LEAs in
waves.
OpenClass is an open source product that will become a part of the Home Base
suite of tools. Integration between PowerSchool and OpenClass enables automatic
generation of teachers, students and courses from PowerSchool into OpenClass.
Teachers will be able to log in and see all of their courses and students without
having to manually enter or import them. Another benefit is the single-sign on
integration between PowerSchool, Schoolnet, and OpenClass. Students and
teachers can log into OpenClass from other applications in Home Base simply by
clicking a link.
NCDPI and Pearson staff are currently working with the first wave of eight
districts and charter schools to set up their OpenClass campuses (meaning that the
sync between PowerSchool and OpenClass is in place and that the courses are in
the system).
c) Ensuring High-Quality, Useful Content in Home Base
NCDPI and Pearson staff have successfully loaded content into both the Schoolnet and
NC Learning Object Repositories. Rich resources were available to teachers and students
from the first day of rollout, and the work to enhance the content continues.
Currently, there are 9,457 state-vetted instructional resources, 1,137 SAS Curriculum
Pathways instructional resources, 25,411 state-created assessment items, and 8,652 statepurchased assessment items available in Home Base. In addition to these state-provided
NCDPI / NC RttT PMO
Page 3 of 14
11/27/2013
resources, districts, schools, and teachers have uploaded or created many of their own.
See Appendix A for a more detailed view of the numbers for the state-provided content
within Home Base as of November 1, 2013.
While there are currently thousands of instructional resources and assessment items that
the NCDPI is providing to educators across state through Home Base, there are still areas
where additional content is needed to support every standard in every grade level. For
example, there are far fewer resources in social studies and the arts than in math and
science. To that end, Learning Systems content analysts are continuing to find, create,
align and upload new resources and items. Learning Systems will also bring on new
content analysts within the next month or two to contribute to this work.
d) Stakeholder Guidance and Engagement
NCDPI continues to communicate with LEAs and charters to discuss and demonstrate
portions of the system.
d1)
LEA Engagement
The state and vendor staffs hold weekly meetings for planning LEA engagement work.
This work includes LEA outreach, such as targeted webinars for LEAs and surveys for
educators using Home Base.
For example, knowing that staff in small districts have different needs than those in larger
districts, staff organized a webinar entitled "Many Hats & Home Base: Small District
Implementation." During the webinar, some of North Carolina’s smallest LEAs, Lee,
Perquimans, and Hoke, shared their lessons learned and experiences in the following
areas: curriculum management, assessments, parent access, and identifying funding for
opting-in.
The team is also currently putting together a survey for teachers to gauge their interest
and hear about their experiences with using the instructional resources in Home Base.
The staff is also communicating with districts and charter schools to learn about how
implementation is going in LEAs and have created a tracking device by which the state
can monitor which districts have expressed an interest in opting in for the optional pieces
of Home Base.
d2)
Stakeholder Meetings
NCDPI continues to hold meetings with the Home Base Stakeholder Advisory
Committee. This group meets approximately every other month to discuss Home Base.
Members include staff from LEAs, education agencies, and the Office of State Budget
and Management (OSBM).
d3)
Working with partner sites
NCDPI / NC RttT PMO
Page 4 of 14
11/27/2013
NCDPI has partnered with eight LEAs during the implementation of Home Base. They
include the following:
•
•
•
•
•
•
•
•
Hertford County Schools (Region 1)
Onslow County Schools (Region 2)
Franklin Academy, Charter School, Wake Forest (Region 3)
Lee County (Region 4)
The College Preparatory and Leadership Academy, Charter School in High
Point (Region 5)
KIPP Charter School, Charlotte (Region 6)
Avery County Schools (Region 7)
Rutherford County Schools (Region 8)
The NCDPI staff, working with the partner sites, are aiding LEAs as they determine their
top three initiatives to achieve this year. They are working together to create a strategic
plan for each partner LEA. Those plans should be completed by the end of this calendar
year.
e) Communications
NCDPI continues issuing newsletters to numerous listservs, including the Weekly
Updates listserv on which there are people who have self-subscribed. The weekly also
goes out to technical directors, principals, teachers, superintendents and more. Previously
this newsletter was issued on a biweekly basis, but because of the need for more timely
information, the team began issuing it weekly after the initial rollout of Home Base in
July, 2013. Staff continues to keep the Home Base website fresh with timely and
pertinent information. Items available on the website include training materials and
information, a calendar of Home Base webinars and presentations with links to
registration, answers to frequently asked questions (FAQs), archives of all biweekly
updates, timelines, status updates, and resources such as presentation slides and videos.
f) Training and Professional Development
The Professional Development for Home Base has been available in every region on an
ongoing basis and supports state, district, and charter staff of varying roles.
The first round of face-to-face training in the districts for both the Educator Evaluation
and the Core IIS components of Home Base was completed in the early summer in June
and July, and additional training has followed and continues to be available.
Following the June and July training, Home Base was featured in the NCDPI Summer
Institutes. There were three 3-hour design studios that focused on Home Base. One was
an overview that allowed participants to explore the demonstration site as teachers; one
focused on building instructional capacity using Home Base; and the other was on the
OER rubric used to vet and align content that is included in Home Base.
NCDPI / NC RttT PMO
Page 5 of 14
11/27/2013
The state has put together and begun implementation of a comprehensive, ongoing
professional development plan for Home Base. It includes both face-to-face sessions and
webinars hosted by both NCDPI staff and vendor staff. Target audiences include NCDPI
staff, Support Center staff, LEAs and charter schools.
Typically, there are weekly webinars on both the educator evaluation system and on
Schoolnet with additional webinars on OpenClass offered each month. There are also
additional webinars planned for targeted needs based on responses the state receives from
the districts and charter schools. For example, the state offered a webinar on the ScanIt
application after receiving numerous questions regarding use of the scanning feature of
Home Base, and the “Many Hats” webinar when smaller LEAs shared information about
the need for guidance with small district implementation.
There are also face-to-face refresher trainings happening across the state in each region.
These sessions offer LEAs the opportunity to learn more about Schoolnet and build up
the LEAs knowledge base from the foundation of the original LEA trainings.
Districts that have indicated early that they intend to opt in to the instructional
components of Home Base have received certification training in Schoolnet. Some
districts have contacted the NCDPI for additional training beyond the planned sessions
and webinars that have been offered, and a core group of certified DPI trainers as well as
other PD leads and District and School Transformation staff have provided that requested
training.
g) Support Center
The Home Base Support Center has continued to help users in the field receive answers
to their questions and resolutions for their issues. There have been challenges associated
with tracking logged tickets from the LEAs and response times for identifying and
relaying resolutions to LEAs; some of the issues were related simply to setting up
processes and training both state and vendor staff on those processes. The support center
and implementation teams are working together to address these and other issues to
improve the Support Center.
h) Staffing
Five new staff members were hired to support the LEA Partnership work. One new staff
member was hired to work on training on the instructional components of Home Base. In
addition, the Learning Systems division interviewed candidates to fill three content
specialist positions. Those content analysts will find, vet, and align resources primarily in
Social Studies, Math and English Language Arts.
2. Does the State have evidence indicating the quality of implementation for this subcriterion? What is/has the State doing/done as a result of this information?
NCDPI / NC RttT PMO
Page 6 of 14
11/27/2013
a) Quality of Implementation
Implementation work will include continued training and support opportunities for state,
district, and charter staff, as referenced in the training and professional development
sections of this report. In addition, partnerships with the LEAs will benefit both those
volunteer districts and the state in the implementation process.
In order to analyze the success of the implementation, the state has requested and will be
collecting data on system performance, usage, and issues.
i. System performance
NCDPI and Pearson are in the process of developing system performance
monitoring reports, using a tool called “Keynote” to report the average system
response time and the system up-time or availability. This report will include
metrics as defined in the service level agreement (SLA), and it will make it
relatively easy to monitor to ensure that the Schoolnet application performs within
the parameters defined within the SLA. This tool may also be used to conduct
real-time performance monitoring to identify the sources of performance issues at
the time in which they are occurring.
Truenorthlogic already provides a weekly performance monitoring report that
tracks the average response times to ensure that the system operates within the
performance parameters defined within the service level agreement.
ii. Usage
The state is also running or receiving reports that include information such as
current enrollment system-wide, the number of successful logins, failed logins,
assessments created, assessments administered, lesson plans created, and the
number of resources at the state level. NCDPI is in the early stages of that data
collection.
iii. Issues
The state has requested and received reports on the issues reported for the
Schoolnet and Educator Evaluation System components of Home Base. Those
reports come from two sources—the Home Base Support Center and from the
vendors when issues have been escalated to tier 2 and tier 3 support. The project
team meets at a minimum once a week to discuss, analyze, prioritize, and
troubleshoot the main issues reported.
3. What obstacles and/or risks could impact the State’s ability to meet its goals and
performance measures related to this sub-criterion?
Risk
Strategy to Mitigate
Single Sign-On
•
Pearson’s single sign-on (SSO)
NCDPI / NC RttT PMO
Ongoing work with vendors to isolate
source of problems. In some cases,
issues have been caused by Pearson
Page 7 of 14
11/27/2013
Risk
Strategy to Mitigate
solution, which allows users to
access both Schoolnet for
benchmarking and classroom
assessment and Truenorthlogic for
educator evaluation has, at times,
failed to grant users access to the
applications.
•
single sign-on solution, and in other
cases, the cause has been
Truenorthlogic handling of the login
requests. NCDPI continues to work
with the vendors to try to find the root
causes for the SSO integration
problems.
The Cloud Identity Access
Management System will be piloted in
January 2014 and will begin rolling
out in the spring of 2014 to provide
login integration between Home Base
components by back to school 201415.
Performance
•
Performance issues have surfaced
(unacceptably long page loads and
time outs) particularly as use has
increased. This challenge was most
evident during weeks of increased
use – during weeks of heavy
benchmarking and during weeks
where many schools were engaged
in teacher observation and
evaluation.
Data Validation
Vendors and DPI are taking steps to
isolate the source of performance
problems which currently may be
connected to inefficient database query
coding and proper scaling of hardware
and software to match the usage
demand. Testing is currently underway
towards increasing reliability of tools
at peak use times.
•
Data validation processes have been
delayed to ensure that data move
accurately from the local LEA
instance of PowerSchool, to the
PowerSchool Operational Data Store
(ODS), and into Schoolnet.
Lack of a Combined Training Site
Data validation and quality assurance
testing are underway. NCDPI has
requested additional Pearson resources
to address the data validation testing,
and Pearson has committed to provide
the resources to get the data validation
testing completed.
•
Pearson is in the process of migrating
to a new data center in Iowa City,
Iowa. In the new data center, Pearson
is building an integrated training
environment that will mimic the
Production environment.
Support documents and trainings have
been developed to support users in
understanding how the actual site
functions.
The Training environments for
PowerSchool and Schoolnet are not
integrated in the way they are in the
Home Base production environment.
The vast majority of functionality is
still available in each application,
but the integrated functionality
between the two applications is not
NCDPI / NC RttT PMO
•
Page 8 of 14
11/27/2013
Risk
Strategy to Mitigate
available in the Training
environment. For example the ability
to demonstrate how students can
take an assessment in Schoolnet and
post the grade to the PowerSchool
gradebook is not available in the
Training environment but is
available in Production.
Adoption and Sustainability Plans •
Because Home Base has many
components that are not required,
NC must create a suite of tools that
provide a combination of high
quality resources and superior value
compared to the tools available in
the open market. Many LEAs and
schools have expressed that the
potential of Home Base is powerful,
but they are cautious about replacing
extant tools or expanding to Home
Base.
•
•
•
•
Created an integrated suite of tools
that provide curriculum and
instructional tools, classroom and
benchmark assessment tools, and
professional development tools that
offer high degrees of functionality and
integration.
Created a cost-sharing model designed
to bring significant cost-savings to
districts and create a sustainable costsharing model ($4 per student).
Developed a training plan to ensure
users to have sufficient training for
implementation.
Continue to gather user feedback to
ensure tools meets educator needs.
Conducting LEA and charter outreach,
including tracking current dispositions
toward opt-in.
Communications
•
Carefully designing key message
NCDPI must maintain complete
transparency with LEAs and charters
and must frequently and accurately
communicate status and issues
regarding the Home Base Suite of
applications. NCDPI must set
accurate expectations for user
communities in the use and
capabilities of the Home Base suite
of applications and must accurately
communication costs in time for
LEAs and charters to plan for Home
Base adoption.
•
Daily communication of specific
issues and resolutions
•
Communicating realistic target dates
for milestones
•
Publishing frequently asked questions
(FAQ)
•
•
Widely distributing biweekly updates
Engaging in speaking engagements
and meetings across the state to talk
about Home Base
Offering Home Base sessions in
conferences across the state
NCDPI / NC RttT PMO
•
Page 9 of 14
11/27/2013
Evaluation: Based on the responses to the previous question, evaluate the State’s
performance and progress to date for this sub-criterion (choose one)
Red (1)
Orange (2)
Yellow (3)
Green (4)1
1
Red – substantially off-track and/or has significant quality concerns; urgent and decisive action is required; Orange –off-track
and/or there are quality concerns; many aspects require significant attention; Yellow –generally on-track and of high or good
quality; only a few aspects require additional attention; Green – on-track with high quality.
NCDPI / NC RttT PMO
Page 10 of 14
11/27/2013
Appendix A
Home Base Instructional Resources by Subject as of November 1, 2013
(Schoolnet: State Bank)
Grade
TOTAL*
ELA
Math
Science
Social
Studies
Healthful
Living/PE
World
Languages
Arts
English
as a
Second
Language
PK
38
-
3
-
-
-
-
3
-
KG
753
115
161
73
50
148
89
65
20
1
899
133
158
105
51
189
101
88
32
2
962
191
173
121
73
181
64
87
40
3
1272
149
261
215
73
315
102
89
36
4
1537
116
259
291
75
478
139
99
48
5
1667
98
294
253
72
595
159
112
52
6
1991
170
289
306
64
729
220
138
43
7
2161
144
238
281
62
1019
229
122
34
8
2430
131
264
401
65
1046
281
173
37
9
3537
186
702
773
376
890
392
156
30
10
2782
188
702
836
376
29
431
158
30
11
2747
148
702
838
376
28
441
152
30
12
2725
134
701
838
377
28
442
145
28
9,457
* Some resources cross various grade levels
** Common Core ELA also tagged to Social Studies and Science standards
NCDPI / NC RttT PMO
Page 11 of 14
11/27/2013
Home Base Instructional Resources in State Bank: Totals as of November 1, 2013
Subject/Focus Area
Actuals in Schoolnet
English Language Arts*
1,018
Mathematics
2,500
Science
1,888
Social Studies
Healthful Living/Physical Education
804
1,859
World Languages
639
Arts Education
586
English as a Second Language
131
Home Base Training Materials
32
STEM**
11
Formative Assessment**
TOTAL
183
9,457
* Common Core ELA also tagged to Social Studies and Science standards
** Listed as keywords, not as subjects in Schoolnet
Home Base Instructional Resources from SAS Curriculum Pathways: Totals as of
November 1, 2013
Subject/Focus Area
Actuals in Schoolnet
English Language Arts
283
Mathematics
249
Science
210
Social Studies
264
Spanish
127
TOTAL
1,137
NCDPI / NC RttT PMO
Page 12 of 14
11/27/2013
Home Base Classroom and Benchmark Items: Formerly in ClassScape*
Subject/Focus Area
3-8 ELA
Actuals in Schoolnet
5,453
English II
593
3-8 Math
11,186
Algebra I / Integrated I
2,376
5 Science
1,871
8 Science
2,297
Biology
1,635
TOTAL**
25,411
* These items are split into a bank for teachers to use daily for classroom assessment purposes (Publisher = NCDPI_Classroom) and a bank that is
secure at the district level for benchmark purposes (Publisher = NCDPI_Benchmark).
** Development is ongoing.
NCDPI / NC RttT PMO
Page 13 of 14
11/27/2013
Home Base Classroom (non-secure) Purchased Items: NWEA
Social Studies
Actuals in Schoolnet
Grade 7
43
Grade 8
17
Global Geography
131
World History
524
American History I
124
American History II
517
Civics and Economics
617
TOTAL
1,973
Science
Actuals in Schoolnet
Grade 3
152
Grade 4
633
Grade 5
352
Grade 6
1,006
Grade 7
880
Grade 8
659
Physical Science
603
Biology
1,030
Chemistry
596
Physics
174
Earth and Environmental
594
TOTAL
NCDPI / NC RttT PMO
6,679
Page 14 of 14
11/27/2013