What’s New at The CITI
Program
Paul Braunschweiger Ph.D.
Professor Radiation Oncology
University of Miami Miller School of Medicine
www.citiprogram.org
Talking points
•
•
•
•
•
•
•
•
Historical perspective
Organizational structure
New materials through CITI
New curriculum structure opportunities.
New registration, enrollment and navigation aids
New administrator utilities
PRIM&R Member Access
New look for 2009
CITI Founders
CITI Program is a web based bioethics education program designed to promote
the responsible conduct of research.
Karen Hansen,
Fred Hutchinson Cancer Research Center,
Seattle, WA., USA
Paul Braunschweiger Ph.D.
University of Miami
Miami, Fl. USA
June 2000 DHHS Mandates
Training in Human Subjects
Protection
A. TERMS OF THE FEDERALWIDE ASSURANCE (FWA) FOR INSTITUTIONS WITHIN
THE UNITED STATES. Announced Dec. 2000; implemented, 2005
12. Educational Training
OHRP strongly recommends that the Institution and the designated IRB(s)
establish educational training and oversight mechanisms (appropriate to the
nature and volume of its research) to ensure that research investigators, IRB
members and staff, and other appropriate personnel maintain continuing
knowledge of, and comply with, the following: relevant ethical principles;
relevant federal regulations; written IRB procedures; OHRP guidance; other
applicable guidance, state and local laws; and institutional policies for the
protection of human subjects. Furthermore, OHRP recommends that a) IRB
members and staff complete relevant educational training before reviewing
human subjects research; and b) research investigators complete appropriate
institutional educational training before conducting human subjects research.
CITI – Program
11-2008
Participating Institutions and Organizations (~1150)
CITI Developer Group (~60)
RCR
CITI Executive Advisory Committee
Founders
L.A.W.
CITI Editorial Board (15)
Intl.
HSRP
HIPS
GCP
www.citiprogram.org
CITI Administration – Office of Research Education, University of Miami
CITI Support Staff
CITI Program Milestones
• 10 Participating Institutions in 2000.
– CITI - Collaborative IRB Training Initiative
• New Courses added 2004.
• ~ 1150 Participating Institutions in 11- 2008.
– CITI - Collaborative Institutional Training Initiative
• Lab Animal Welfare Course – 5-2007
•
•
•
•
~ 35,000 Learners per month. 40- 60% students
>105,000 Learners since 1-1-2009.
> 12,000 Lab Animal Course learners since 1-1-2009
>1,000,00 people have completed a CITI course since 9-1-2000.
• Subscriber Profile
– Universities, colleges, medical centers, community hospitals, societies,
government, commercial IRBs, industry
CITI Program Milestones
• International Initiatives
– CITI - Collaborative International Training Initiative
• Multi-language Course Site. 11-2007 updated 2-2009
– Spanish
– Portuguese
– Chinese, Japanese, Russian, Mongolian, French, Thai
CITI Worldwide Footprint
World wide n > 1150
International n = 49sites / 35 countries
CITI Program Courses
• Human Subject Protection
– Good Clinical Practice (15)
– Refresher Courses
• Health Information Privacy and Security (HIPS)
– Basic
– Refresher
• The Responsible Conduct of Research.
– 5 Disciplines
– Text, cases, quizzes.
• Laboratory Animal Welfare (L.A.W.)
– Basic
– Refresher
Lab Animal Welfare Program
• 2004 - Collaboration between CITI and Dr, M. Fallon at the
Atlanta, VA.
• 2007 – VA decided to close the www.ResearchTraining.org
LAW course site.
• 5-2007 CITI offered the CITI Version of RT.org to our
participating institutions.
• 10-2007 – CITI Lab Animal Welfare Working Group.
• 2008 - 2009 - Comprehensive review and updating of the
course content.
• 2009 CITI International Collaborating Centers
RCR 12-29-07
CITI L.A.W. Course Library
• CITI Basic Courses.
– Working with the IACUC
– Essentials for IACUC Members
• Special Courses
– Aseptic Technique
– Minimizing Pain and Distress in Rodents.
RCR 12-29-07
CITI L.A.W. Course Library
Species / model Specific Courses
Working With Amphibians In The Research Setting
Working With Mice In The Research Setting
Working With Rats In The Research Setting
Working With Guinea Pigs In The Research Setting
Working With Hamsters In The Research Setting
Working With Gerbils In The Research Setting
Working With Rabbits In The Research Setting
Working With Cats In The Research Setting
Working With Dogs In The Research Setting
Working With Swine In The Research Setting
Working With NHP In The Research Setting
RCR 12-29-07
Establishing an Institutional
Curriculum
• Each institution can establish their own
curriculum from the L.A.W. Content Library.
• Curriculum can be customized according to:
– Role in LAW research.
– Species / models used.
• Basic courses and refresher courses.
– Reminder notifications
RCR 12-29-07
CITI L.A.W Course site usage
384 institutions
Course Completions x 1000
45
40
35
30
25
20
15
10
5
0
2007
2008
2009 3 months
Top 10 Institutional Users
Institution / Organization
Veterans Affairs**
Emory University
University of Miami / Jackson Memorial Hospital
Wayne State University-Detroit MI
Georgia Institute of Technology
Binghamton University
Clemson University
University of Maryland
West Virginia University
University of Texas Health Science Center San Antonio
University of California Irvine
*First 3 months of 2009
** 151 VA sites have access to the CITI Program.
Total for
2009*
3533
681
739
255
501
559
495
492
591
587
551
2008 CITI L.A.W. Welfare Course Usage.
n=384 institutions
Course
Working with the IACUC
Working with Mice
Working with Rats
Reducing Pain and distress
Essentials for IACUC Members
Aseptic surgery
Working with Rabbits
Working with Swine
Working with NHP
Working with Amphibians
Working with Dogs
Working with Guinea pigs
Working with Hamsters
Working with Cats
Working with Gerbils
RCR 12-29-07
# Completed
11909
8180
5022
4699
1954
1001
930
849
803
490
470
412
341
266
139
CITI L.A.W. Working Group
Name
Institution
Tracy Arwood, MS
Paul Braunschweiger, Ph.D.
Mark Christensen, MA
Mike Fallon DVM, Ph.D.
Cynthia Gillett, DVM
Molly Greene
Susan Harper, DVM, MS
Mike Mann, Ph.D.*
Randall J. Nelson, DVM
Gwenn Oki, MS
Ernie Prentice, Ph.D.
Susan Silk, MS
Kathy Wadsworth, MS
Thomas Zimmerman, DVM
Clemson University
University of Miami
Lourdes College
Veterans Affairs
University of Minnesota
Michigan State University
Veterans Affairs
Emeritus, University of Nebraska Medical Center
University of Memphis
City of Hope Medical Center
University of Nebraska Medical Center
Office of Laboratory Animal Welfare (OLAW)
University of California at Los Angeles
SUNY Stony Brook
*CITI L.A.W. Working group leader
L.A.W. Working Group
• Review learner comments
– Modify the content if appropriate.
• New Content
– Susan Silk MS, Director of the Division of Policy and Education,
OLAW
• “Using Animals in Research: Information for PHS-funded Investigators.” will
provide a comprehensive overview of the regulations and guidelines for PHSfunded researchers including institutional and investigator responsibilities,
applying for funding, receiving an award and post award responsibilities.
– Mark Christensen, Prof. of Ethics, Lourdes College
• 5 module course on the Ethics of conducting research with Lab animals.
• Ethical considerations related to the use of live animal subjects in scientific
research. In this module we will briefly highlight some philosophical theories
about ethics and moral standing and their relation to the use of animals as
research subjects.
– Conducting Field Studies Responsibly.
– Case based refresher course.
RCR 12-29-07
L.A.W. Working Group
• Recently completed updating and editing course
content for:
– Working With the IACUC
– Essentials for IACUC Members
• Quiz questions Reviewed and updated.
– Feedback for all of the questions in all of the modules.
– Posting in June 2009.
• Antibody production
– New Course on antibody production
RCR 12-29-07
Course Site and Software Update
Multilanguage Course Site
Curriculum design
• Curriculum design is institutional responsibility
• Stratified Curriculum to meet educational needs.
– Research Focus
• Model / species specific modules.
– Role in research
• IO, IACUC Member / staff, PI, Investigators, research staff,
students.
– Institution specific modules
• Policies and Procedures.
• Materials not covered by CITI Courses
– Special species or model systems
– Field studies.
– Lab Animal Research Ethics.
RCR 12-29-07
Curriculum Development
Required vs. Optional vs. Elective
• Required
– Learner is required to complete modules in this category.
• Elective
– Institution can designate several “Elective” modules (e.g., 10) and
require that the learner complete a specific number (e.g., 8) that are
of interest.
• Optional
– Optional modules are made available to the learner after all
required and elective modules are completed.
• Passing score set by the institution
– Learner must attain an aggregate score that meets the institutional
requirement to get a Completion Report
RCR 12-29-07
Curriculum design
•
•
CITI Course order forms are available at www.citiprogram.org
Institutional Administrator menu
Curriculum Modifications
•
•
CITI Course order forms are available at www.citiprogram.org
Institutional Administrator menu
Quiz Questions Utilities
• Integrity Assurance.
• New presentation paradigms.
– Randomized questions from a pool
– Using a case study as a stem.
• New psychometric tools to evaluate
quiz questions.
RCR 12-29-07
Institutional Modules
Developed by organizations
specifically for their research
community.
Examples of Institutional L.A.W.
Modules
•
•
•
•
•
•
•
•
•
•
Medical Surveillance Program
General Zoonotic Training
Farm Animals
Wild Animals
Fish, Amphibians and Reptiles
Introduction to Surgical Research
Antibody Production
Occupational Health and Safety
Other models (e.g., Birds)
CU Policy on the Care and Use of Animals in Research
and Teaching.
• Conducting Potentially Painful/Distressful Procedures in
Lab Animals At The University of Miami
RCR 12-29-07
Registration, Enrollment and
Navigation
REGISTRATION
Add Customizable fields
Enrollment Questions
Curriculum Selection
• New page to page
branching logic
enrollment process
– Learner self-identifies
research areas.
– Chooses topics
• Choices determine the
next series of choices
Enrollment Questions
• Choose the most
appropriate research
focus or role in research
• Software then shows the
next series of questions
appropriate for the
learner.
Choose the Animal Models
Special Invasive Techniques
Special Course Enrollment
Course Summary
Learner Menu
Grade Book
Integrity Statement
Integrity Statement
X
Utilities for Learners and
Administrators
CITI Program Knowledge Base
http://citiprogram.supportcenterpro.com/knowledgebase/
or
CITI Home Page, Contact Us
• The CITI Program Knowledge Base provides helpful
information to assist in the use of the CITI Program
website.
• Provide support, resources, hints, tips, links and answers
to frequently asked questions about the CITI Program, all
in one place.
• We welcome contributions and feedback from learners and
CITI institutional administrators.
• Please use the “Submit New Ticket” ticket link to the right
of the screen to share comments, questions, and feedback
with the CITI Support team.
CITI Program Knowledge Base
Please use the “Submit New Ticket” ticket link to the right of the screen to share comments, questions, and
feedback with the CITI Support team.
Admin Access to User Data
• All CITI Institutional
administrators have
access to user data
• Can be restricted
according to admin
role.
– Receive completion
reports
– Responsible for
segments of the training
program
•
•
•
•
Human subjects
RCR
Lab Animal
HIPS
Institutional
Administrator Menu
Course listing
• Review a completion Report
Training Data
Retrieval
View the data
Links to download the data in various formats
Look up a specific learner
Learner’s Records
Other data management
options available
• Bulk data download
– CITI will provide institution specific URL to download the
institution’s training data.
– Username and password protected.
– Data for each course (HSR, lab Animal) can be downloaded
together or separately
– The data fields to be included in the download is customizable.
– Suitable format to upload to data management system.
– Contact CITI Helpdesk - 305 243-7970; citisupport@med.miami.edu
https://www.citiprogram.org/members/siteadministrators/manageinstitutions/downloadaccesscodes.asp?st
rKeyID=XXXXXXXXXXXXXXXXXXXXXXXXX73
Other data management
options
• CITI can now be placed behind your institutional Portal
System.
–
–
–
–
User authentication
Username and password conservation.
Increased data and password security.
Data can be automatically added to training database.
• Contact CITI helpdesk
• 1 time cost for programming and implementation.
PRIM&R Member Access
PRIM&R Member Access
• Go to Main Menu
• “Affiliate with another
institution”
• “Register as PRIM&R
Member”
Contact Information for PRIM&R Member
X
New initiatives for 2009
• Courses and modules
– Software updates
•
•
•
•
•
“Look back” Utility
Non-English authoring utilities.
Knowledge Base.
Username and Password security
Verification and Portal pass through.
– Human subjects (Summer 2009)
•
•
•
•
•
ESCRO (1 modules)
New Refresher Modules
Community Member Module
Research Coordinators
CBPR
New initiatives for 2009
• Courses and modules
– RCR
• RCR for Engineers.
• RCR for Arts and Humanities. (Fall 2009)
– Biosafety and Bio-security (Summer 2009)
• 15 module course
– Lab animal
• Ethics and Lab Animal Research
• Field Studies
• Zebra Fish
Download this Slide Show
Slide show available
back
Announcements and FAQ
CITI L.A.W. Satisfaction Survey
The current course site presents the materials in a
way that is easy for me to understand and learn
% of responders n =5467
40%
35%
30%
25%
20%
15%
10%
5%
0%
1
2
Strongly Disagree
3
4
5
6
7
8
9
Strongly Agree
% of responders n =5467
80%
70%
60%
50%
40%
30%
20%
10%
0%
% of responders n =3381
If you have previously completed a Lab Animal Welfare
course, how does this CITI Course compare with others you
have completed?
30%
25%
61%
20%
15%
10%
5%
0%
1
2
Strongly Disagree
3
4
5
6
7
8
9
Strongly Agree
Although training in Laboratory Animal Welfare issues is mandated
by the Federal government and by my institution, I believe that this
training is really not very useful for new faculty and students?
40%
% of responders n =5369
35%
30%
25%
20%
15%
10%
5%
0%
1
2
3
Strongly Disagree
4
5
6
7
8
9
Strongly Agree
It was easy to register and enroll in the
appropriate course.
45%
% of responders n =5343
40%
35%
30%
25%
20%
15%
10%
5%
0%
1
2
Strongly Disagree
3
4
5
6
7
8
9
Strongly Agree
The quiz questions addressed the relevant issues
in the modules
35%
% of responders n =5343
30%
25%
20%
15%
10%
5%
0%
1
2
3
Strongly Disagree
4
5
6
7
8
9
Strongly Agree
Now that I have completed the CITI Lab Animal
Welfare Course, I feel that the time I invested was
time well spent.
% of responders n =5340
25%
20%
15%
10%
5%
0%
1
2
Strongly Disagree
3
4
5
6
7
8
9
Strongly Agree
After completing this instruction, I now have a better
understanding of how to apply the "3Rs" to insure that my lab
animal studies are conducted to the highest ethical standards?
35%
% of responders n =5343
30%
71%
25%
20%
15%
10%
5%
0%
1
2
Strongly Disagree
3
4
5
6
7
8
9
Strongly Agree
Now that I have completed the course, I am more confident in
my ability to advise a student or a colleague on how to apply
the "3Rs" to insure that their lab animal studies are conducted
to the highest ethical standards?
30%
% of responders n =5289
68%
25%
20%
15%
10%
5%
0%
1
2
3
Strongly Disagree
4
5
6
7
8
9
Strongly Agree
Now that I have completed this course on the ethical conduct of
studies with laboratory animals, I intend to take a more active role in
assuring that lab animal research at my institution is conducted to
the highest ethical standards by pursuing professional certification,
by joining an ethics committee or an Institutional Animal Care and
Use Committee (IACUC)
% of responders n =5280
30%
25%
60%
20%
15%
10%
5%
0%
1
2
Strongly Disagree
3
4
5
6
7
8
9
Strongly Agree
The "Essentials for IACUC Members" Course has provided
we with the knowledge base and confidence to take a more
active role in the business of the IACUC.
% of responders n =2889
25%
64%
20%
15%
10%
5%
0%
1
2
Strongly Disagree
3
4
5
6
7
8
9
Strongly Agree
About how many hours did it take you to complete
your Lab Animal Welfare Course requirements?
% of responders n =5321
30%
25%
20%
15%
10%
5%
0%
6
more
than 6
less
than 1
5
4
3
2
The presentation model used by CITI permits you to scroll through
the content as if you were reading a book or a journal. Other web
based models require that the learner click from "page to page” in
order to read the material in smaller segments.
I DO NOT like the current presentation model another approach should be
implemented.
2%
I prefer the "page to page" approach with interactive games or activities to
5%
break up the text.
The "page to page" is better because I could pause and reflect on
important points in the content.
6%
I DO NOT recommend the "Page to Page" model as it would increase the
time to complete the course.
10%
The materials presented in the course are new to me. I like the current
presentation model.
15%
I am familiar with the content. The current approach permits an efficient
review of the material.
22%
The current approach enabled me to read through the content as if I
where reading a book or journal.
40%
Please visit the CITI exhibit