Transforming a
Human Research Protection Program
July 16, 2014
YOUR MISSION | OUR SOLUTIONS
© Huron Consulting Group Inc. All Rights Reserved.
Huron is a management consulting firm and not a CPA firm, and does not provide attest services, audits, or other engagements in accordance with the AICPA's Statements on Auditing Standards.
Huron is not a law firm; it does not offer, and is not authorized to provide, legal advice or counseling in any jurisdiction.
Agenda
• Introduction
• Background
• HRPP Transformation at Penn State
•
•
•
Process and Procedures
Organizational Structure
System Implementation
• Results
• Questions
2
Presenters
Kathleen Hay
Director, Human Subjects Protection Office
Penn State College of Medicine, Penn State Hershey Medical Center
khay@hmc.psu.edu
Sara Horn
Assistant Director, Office for Research Protections
The Pennsylvania State University, State College Campus
sjh246@psu.edu
Frank Conte
Manager
Huron Consulting Group
fconte@huronconsultinggroup.com
3
Introduction
Introduction
The Pennsylvania State University is a large, multi-campus
system located throughout the state of Pennsylvania, including:
•
•
•
•
24 campuses throughout the State
98,097 total student enrollment
42,976 Faculty, Staff and Employees
$848 million in Total Research Expenditures
5
Introduction
Office for Research Protections
(State College)
• Over 3500 IRB Submissions
Annually
Human Subjects Protection Office
(Hershey )
• Over 3400 IRB Submissions
Annually
• 1583 New
• 973 Modifications
• 976 Continuing Reviews
• 722 New
• 1318 Modifications
• 1345 Continuing Reviews
• 8 IRB Specific Staff Members
• 11 Staff Members
• *1 Director/IO (Oversees COI, Research
Misconduct, Dive Safety, Information
Systems, IRB, IACUC)
• 1 Assistant Director
• 6 IRB Analysts
• 1 Program Coordinator
•
•
•
•
6
1 Director
1 Manager/Education Coordinator
3 IRB Analysts
6 IRB Coordinators
Background
Background
PRAMS
Both IRB Offices were using PRAMS, a homegrown electronic
submission system, built in 1999, released in 2002
• Robust database with strong reporting
• Electronic IRB submission used by one of the IRB Offices (esubmission released in 2009)
• No Meeting Management functionality
• No Unanticipated Problem/Reportable New Information functionality
• Not user friendly
• 800+ possible questions – review-level based
• Significant back and forth with researchers
8
Background
Turnaround Times – Before Transformation
90
80
70
60
50
2012
2013
AAHRPP
40
30
20
10
0
Exempt
Expedited
9
Full
Background
Leadership Decision for Change
Consulting
• Penn State engaged Huron to conduct an
analysis of IRB Processes and System
functionality
• Leadership decision to transform the HRPP
at Penn State
Consulting
Benchmarking
Benchmarking
• Penn State used available data and
communication with peer institutions to
determine benchmarks
Governance
Governance
• Executive Committee, Advisory Committee,
Key User Group and Project Implementation
Team were formed
10
Background
Leadership Decision for Change
Metrics Tracking
• Metrics, such as staff response to change, staff performance, sprint
performance, issue handling, earned value, and budget allocation
were all tracked and managed at regular intervals.
Implementation
• Decisions on implementing the Huron HRPP Toolkit, Click IRB
software, timelines, and process were made by all governance bodies
11
Background
Implementation Phases
Phase I - During system design and development
• Initial HRPP Toolkit customization and implementation
• IRB Staff and Leadership training and mentoring
• Organizational infrastructure discussions and planning
1
Phase II - In parallel with the system rollout
• Ongoing mentoring with staff and IRB leadership
• QA of Toolkit use
• Developed investigator-facing materials
2
Phase III – Post system rollout
• Developed internal audit SOPs and tools
• Training of IRB Staff to conduct internal QA/Audit
12
3
• Business
Process and
Organizational
Structure
• System Rollout
and InvestigatorFacing Materials
• Ongoing
Support and
Quality
Assurance
HRPP Transformation:
Process & Procedures
HRPP Transformation
Process and Procedures – Why the HRPP Toolkit?
• HRPP Toolkit chosen
•
•
•
•
•
•
Policies and Procedures (SOPs) – define processes within IRB
offices
Checklists, worksheets and templates
Integrated in Click IRB
Back to the regulations
Allows a more efficient accreditation submission process
Provides a complete set of tools to meet regulatory and
accreditation requirements
14
HRPP Transformation
1
Process and Procedures – Phase I
Business Process
and Organizational
Structure
• Phase I implementation of the HRPP Toolkit – March 2013
•
•
Documents were customized for Penn State
Initial review and training with staff
• Started to use parts of the HRPP Toolkit before
implementation of the Click IRB software – April/May 2013
•
•
•
•
Conducted “Six Hats” session with our Advisory committee to make the
decision to do so
Checklists and worksheets
Minutes template and letter format
Reportable New Information SOP, forms and review process
• Allowed researchers, IRB staff and IRB members to
become familiar with the Toolkit before system rollout
15
HRPP Transformation
1
Process and Procedures - Challenges
Business Process
and Organizational
Structure
Challenges faced during preparation and implementation:
• Takes a significant amount of time on the part of leadership
to identify institutional needs and build these into the
existing documents
• Challenge in taking two sites who had differing policies and
procedures and joining these
•
•
•
Required regular contact between leadership at the two sites
Creation of the Responsibility Matrix Team to make decisions
about revisions to toolkit documents
Toolkit “Keeper” who makes final changes to all toolkit
documents
16
HRPP Transformation
2
Process and Procedures – Phase II
System Rollout and
Investigator-Facing
Materials
Phase II – Full implementation of the HRPP Toolkit – December
2013 – when Click IRB software went live
• Impact on Investigators
•
•
Investigators required to use protocol templates and consent
form templates for new study submissions
New studies for 3 Colleges/Campuses submitted in the new
system
–
–
•
Short application
Relies on document uploads to explain the research
Studies active in old system - partially converted to new system
–
–
All future submissions occurred in CATS IRB
Conversion process completed at next modification or continuing review
17
HRPP Transformation
2
Process and Procedures – Phase II
System Rollout and
Investigator-Facing
Materials
Phase II implementation (continued)
• Impact on IRB staff – transitioned from a paperbased/electronic system to a completely electronic system
•
•
•
Full use of the HRPP SOPs, checklists, worksheets, templates
Electronic reviews
Electronic meeting management functionality
• Impact on IRB members – transitioned to an all electronic
review system using documents and comments in new Click
Commerce system
18
HRPP Transformation
3
Process and Procedures – Phase III
Ongoing Support and
Quality Assurance
Phase III
• To ensure consistent use of the system and Toolkit, Huron
worked with Penn State to develop a set of SOPs and tools
to assist with ongoing quality assurance.
• Training will be provided to IRB staff to conduct QA
assessments
19
HRPP Transformation:
Organizational Structure
Organizational Structure - HSPO
Structure Before Transformation
11 Staff Members
• 1 Director and 1 Manager/Education Coordinator
• 5 IRB Coordinators – conducted non-committee reviews and
prepared committee review materials
• Each IRB Coordinator responsible for one type of submission (e.g., continuing
reviews, expedited modifications, committee modification, new studies)
• 3 IRB Assistants – coordinated board meetings, printed study
materials, and performed administrative duties
• 1 Computer Specialist – provided database and computer support
• All staff reported either directly to the IRB Director or the IRB
Manager
21
Organizational Structure - HSPO
Advantages and Disadvantages of Legacy Structure
Advantages
• People were specialized and efficient
• Researchers knew who to contact based on
type of submission
Disadvantages
• Little cross training
• No backups for vacations or leaves of
absence
• Bottlenecks if we received more of one type
of submission
• No career ladders
22
Organizational Structure - HSPO
After HRPP Transformation
11 Staff Members
• Same people – different job descriptions
23
Organizational Structure - HSPO
After HRPP Transformation
• Submission are assigned to teams based on the principal
investigator’s last name
• Each team reviews all types of submissions
• Team Leader triages reviews based on team member’s
experience and training
• Study stays with the assigned team for the life of the study
24
Organizational Structure - HSPO
Advantages and Disadvantages of Revised Structure
Advantages
• Everyone is cross-trained
• Continuous coverage for vacations and
leaves of absences
• Career ladders
Disadvantages
• Human Resources did not recognize
some of the changes in job descriptions
• Structure does not account for office
administrative duties or database and
computer support
25
Organizational Structure - HSPO
Changes Since Implementation
Adjustments in HSPO organization during the conversion
process:
• Teams were overwhelmed with conversion process and tracking IRB
expiration dates plus reviewing new studies
• Pulled 2 IRB Compliance Specialists from teams to do all continuing
reviews for one year during conversion process
• This allowed the teams to focus on reviews of new study and
modification submissions
• After all studies have converted to new system – plan is to return to
Team structure
26
Organizational Structure - ORP
Structure Before Transformation
8 Staff Members
• *1 Associate Director of the ORP who was also leading several other
compliance areas
• 2 Team Leaders/Coordinators- Supervision and leadership of program
in addition to the below review responsibilities
• 5 IRB Coordinators – conducted non-committee reviews and
prepared committee review materials
• 1 working on Exempts only, 2 working with full review submissions, 4 working on
expedited submissions
• 1 IRB Support Assistant – Record Retention, Phone calls, meeting
minutes
27
Organizational Structure - ORP
Advantages and Disadvantages Before Transformation
Advantages
• At the expedited level,
submissions were distributed
accordingly to PI’s last name
which worked very well
Disadvantages
• Little to no cross training across
review levels
• Quickly identified a need for
direct leadership of the HRPP
28
Organizational Structure - ORP
After Transformation
8 Staff Members
•
•
•
•
1 Assistant Director of the ORP, HRPP
2 Senior Program Leaders/Analysts
4 IRB Analysts
1 Program Coordinator
29
Organizational Structure - ORP
After Transformation
• Two teams-each team works with one of the IRBs and
consists of 2 IRB Analysts and a Senior Program Leader
• Analysts and Senior Program Leaders conduct noncommittee reviews, reviews of RNI, and prepare committee
reviews
• All submissions distributed by the last name of the PI
30
Organizational Structure - ORP
Advantages and Disadvantages After Transformation
Advantages
• All individuals are cross trained and can review all
types of submissions
• Investigators work consistently with a single IRB
Analyst providing some consistency
• Direct Leadership of the IRB by the Assistant
Director who reports directly to the Institutional
Official
Disadvantages
• Team structure and efficiency when working with
the committees is still challenging one year after
re-structure
31
HRPP Transformation:
System Implementation
System Implementation
Timeline
December 2013
July 2014
Go live date for the Penn State
College of Medicine and the
Penn State Hershey Medical
Center and for 2 colleges at UP
Represented 60% of overall
submission load
Go live date for the remainder of
colleges and campuses at Penn
State
System Implementation
Progress
As of go-live date
• New studies submitted in new system
•
•
Short application
Relies on document uploads to explain the research (protocol,
consent form, recruitment materials, etc.)
• Active studies in old system partially converted to new
system
•
•
All future submissions in new system
At time of next mod or CR – investigators needed to complete
the conversion process – complete the application form in new
system and upload current approved documents
34
System Implementation
System Customization
Customized some aspects of the system
• Added functionality:
•
•
•
•
•
Set Study Initiator to identify investigator-written research
Set ClinicalTrials.gov Information to capture the registration #
Manage Training to add or update training requirements
View Training to check current training of study team members
Create Printable Materials for study coordinators
• Use the Manage Ancillary Reviews functionality for reviews
conducted by other committees such as the radiation safety
committee, conflict of interest committee, anatomic pathology,
biosafety committee
35
System Implementation
Conversion Process
Conversion Process
• Biggest challenge for investigators and IRB staff
• Instructions were not completed at go-live date due to time
constraints
• Initially, IRB staff were not fully trained on conversion process
• Unable to answer researcher questions
• Process is slightly different for each office so instructions were
unclear
• Complicated and time-consuming process for both investigators and
IRB staff
36
System Implementation
Staff Training and Education
Started having weekly HSPO staff meetings
•
•
•
•
System updates from functional analyst
System and HRPP Toolkit training
Review of procedures for consistency
Discussion of problems or issues with the system, Toolkit or
submissions
Both offices conduct IRB Member training at each committee
meeting
• Review system procedures for viewing agenda items
• Review Toolkit worksheets and checklists periodically
37
System Implementation
Impact and Benefits
• For the two IRB Offices, the new electronic system has had the
biggest impact on efficiency
•
•
•
•
Electronic meeting management features
Electronic submission of all types of submissions
Electronic workflow routing and review
Elimination of the need for paper study files
• So far investigators report the following benefits:
• Clarity in review process
• Ability to directly comment in the system
• Ease of submitting new studies, modifications, continuing reviews, and RNI,
though the conversion process for existing studies can be complicated and
time consuming for some
38
Results
Results
Turnaround Times – After Transformation
90
80
70
60
2012
2013
AAHRPP
CATS-IRB
50
40
30
20
10
0
Exempt
Expedited
40
Full
Results
What Went Well
• Effectively evaluated and supported a need for change and created a
solid governance structure for the project
• Hired a dedicated team of Project Coordinator, Functional Analysts,
Programmers to work specifically on the project
• Created the Responsibility Matrix Team to lead the process of making
revisions to the Toolkit documents
• Leadership and staff went into the process open minded,
compromised on some of our needs and accepted change at each
site
• Engaged Huron frequently along the way
41
Results
Areas for Growth
• Toolkit implementation and system development timing created
development bottlenecks
• Testing with key users started late, not as effective as it could have
been
• Dealing with sponsor-written clinical trial issues
• Procedures for converting studies from old system to new not
effectively communicated
• Communicate effectively with users about significant changes that
would come based on the HRPP Toolkit. For instance, at UP, student
investigators are not often doing industry or federally sponsored
studies therefore they were not familiar with the idea of writing a
protocol
42
Results
If We Could Go Back…
• Put bigger gap between start of system development and
implementation of Toolkit
• Didn’t know what we needed until development was in process
• Improve communications about conversion process and
about the Toolkit in general
• Train IRB staff to use live system well before rollout so they
can answer researchers questions
43
Results
Additional Results
• Decreased use of office supplies, especially paper
• Decreased use of copiers
• Emptying of file cabinets and shelving - freed up space
• More work stations
• Conference room
• Improved clarity in review process
• Increased morale in IRB Offices
• Positive experience for researchers
Thank you!
Thank you!
Kathleen Hay
Director, Human Subjects Protection Office
Penn State College of Medicine, Penn State Hershey Medical Center
khay@hmc.psu.edu
Sara Horn
Assistant Director, Office for Research Protections
The Pennsylvania State University, State College Campus
sjh246@psu.edu
Frank Conte
Manager
Huron Consulting Group
fconte@huronconsultinggroup.com
46