VT’s University Libraries
Assessing Technology-based
Projects for Faculty Evaluations
Some Survey Results
and comparisons with ARL Academic Libraries
Technology-based Projects include
• Publication of articles in ejournals
• Development of Web-based materials
e.g., Web pages, tutorials,
digitization
• Development of video/audio tapes
• Development of computer software
e.g., CD-ROMs
74% of library faculty responded
to the survey
28 of 38 library faculty on LIBFAC
– Every category represented: administration
(including heads, directors), CLs, Collection
Management, DLA, ILL, Instruction,
Reference, Technical Services
– Tenured (59%), non-tenured (33%), and
not on tenure track (7%)
– 60% ≤ 10 years, 40% ≥ 11 years
Q1.
68% of the library
faculty have
developed techbased projects in
the last 3 years
that they
considered
appropriate for
evaluation
purposes.
Q2.
93% of the
library faculty
felt they had
been
encouraged to
develop techbased projects.
Q3.
Do librarians
feel that they
are supported
in their
development
of tech-based
projects?
yes
78%
no
22%
• Yes: Equipment,
software, training
provided.
• No: Support is
lacking for time to
development.
Q4. Why create technology-based projects?
10 0%
90 %
80 %
70 %
60 %
50 %
40 %
30 %
20 %
10 %
0%
meet needs of
lib/uni
improve lib
instruction
VT libraries
ARL
needs of
external
constituents
professsional
recognition
ARL w/faculty status
opportunity
for
scholarship
meet needs of
profession
ARL w/out faculty status
Q5. Technology-based projects librarians developed
in last 3 years appropriate for evaluation
VT Libraries
ARL
ARL w/Faculty Status
ARL w/out Facult Status
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
subj/res
. web
maj.
web
proj.
pers.
web
page
article
in ej
softwar
e
listserv
80%
50%
40%
25%
15%
10%
5%
ARL
100%
88%
34%
47%
24%
38%
28%
ARL w/Faculty Status
100%
91%
45%
55%
30%
36%
38%
ARL w/out Facult Status
100%
81%
24%
31%
13%
35%
19%
VT Libraries
video/au
dio
Q5. Other types of technology-based
projects for evaluations (comments)
•
•
•
•
Team-taught, online course
Timelines
Oral history interviews
Database design: input, testing, etc.
Q6 and Q7 Within the past 3 years has
there been an increase in the number of
tech-based projects
Q6.
...on which you
collaborated?
Q7.
...that you initiated?
no
33%
no
41%
yes
67%
yes
59%
Q6. Within the past 3 years, has there been an
increase in the number of tech-based projects on
which you collaborated?
19 librarians collaborated on 30 projects.
with others in
library outside
department
with others in
department
with others in
university outside
library
16.7%
26.3%
40.0%
63.2%
43.3%
68.4%
Q8. The merit of a tech-based project may be assessed
through its quality, relevance, or contribution to the
profession. Indicate which materials you might evaluate.
VT Libraries
ARL w/out faculty status
ARL w/faculty status
ARL members
product
itself
project
summ.
project
descrip.
end-user
comments
93%
94%
96%
95%
56%
88%
70%
75%
33%
73%
68%
73%
78%
53%
55%
58%
peer review
70%
59%
55%
57%
nonrefereed
print comp.
project
procedures
refereed
print comp.
7%
27%
38%
34%
30%
33%
24%
26%
37%
7%
38%
26%
Q8. Other materials to evaluate when
deciding on the merit of a tech-based project
• Benefit to general public
• Usefulness to me or its users
• Meets one of more needs of its target
audience
• 41%: decreases staff hours required since
using technology
• VT: usefulness • ARL: awards
Q9. Criteria used to determine merit
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
devel op.
ide as
co mple x
des ign
reviewed
prio r
devel op.
67%
37%
19%
26%
88%
75%
53%
50%
25%
83%
73%
77%
67%
43%
57%
81%
78%
73%
62%
48%
45%
uni que/
cre ative
schola rly
co ntrib ut
recog-local
VT Libra ries
63%
93%
74%
ARL w/o ut facu lty status
81%
76%
ARL w/facu lty status
83%
ARL m embe rs
83%
recog-nat'l/i'n a
tio nal
Q9. Other criteria that should be used to
determine merit of tech-based projects
•
•
•
•
Enhances distance learning initiatives
Operations run better; more more efficiently
Usefulness
Contribution to the University
• Serves the needs of the end-user
Q10. Should P&CA Guidelines specifically
address tech-based projects?
100%
88%
90%
83%
yes
80%
no
70%
60%
50%
48%
52%
40%
30%
17%
20%
12%
10%
0%
VT
ARL
ARL w/Faculty
Status
Q10. Should P&CA Guidelines specifically
address tech-based projects? Yes.
• Clarification about types of materials and how
they will be evaluated.
• Reminder that content and quality of the
project is key, not the format.
• So that it is clearly understood that such
projects are worthy of recognition.
• To make sure that everyone knows
achievement in this domain is valued.
Q10. Should P&CA Guidelines specifically
address tech-based projects? No.
• Acceptable projects should be defined by
these guidelines
• Treat like other projects: does it serve a real
need?
• Any innovative project should be considered
• Tell when and when not to include Web pages
in dossier
Q11. Do tech-based projects deserve more credibility?
Yes
No (same + less)
69%
65%
61%
49%
51%
39%
35%
31%
VT
ARL
ARL
w/Faculty
Status
ARL w/out
Faculty
Status
Q11. Do tech-based projects deserve more
credibility?
• If yes, what would give them more credibility?
–
–
–
–
Creativity, accuracy, relevance, originality
Standardize procedures and documentation
Financial recognition in annual salary
Motivate people who could do more
• If they deserve less, why?
– Technology is a process not a “product”
– Technology is a tool. The real determining factor is:
has it met a need and is it helping those in its target
audience?
Q12. Will tech-based projects ever be
sufficient evidence of scholarship?
VT
ARL w/ Faculty Status
10 0%
90 %
80 %
70 %
60%
60 %
50%
50%
50 %
40%
40 %
30 %
20 %
10 %
0%
Yes
No