VT’s University Libraries Assessing Technology-based Projects for Faculty Evaluations Some Survey Results and comparisons with ARL Academic Libraries Technology-based Projects include • Publication of articles in ejournals • Development of Web-based materials e.g., Web pages, tutorials, digitization • Development of video/audio tapes • Development of computer software e.g., CD-ROMs 74% of library faculty responded to the survey 28 of 38 library faculty on LIBFAC – Every category represented: administration (including heads, directors), CLs, Collection Management, DLA, ILL, Instruction, Reference, Technical Services – Tenured (59%), non-tenured (33%), and not on tenure track (7%) – 60% ≤ 10 years, 40% ≥ 11 years Q1. 68% of the library faculty have developed techbased projects in the last 3 years that they considered appropriate for evaluation purposes. Q2. 93% of the library faculty felt they had been encouraged to develop techbased projects. Q3. Do librarians feel that they are supported in their development of tech-based projects? yes 78% no 22% • Yes: Equipment, software, training provided. • No: Support is lacking for time to development. Q4. Why create technology-based projects? 10 0% 90 % 80 % 70 % 60 % 50 % 40 % 30 % 20 % 10 % 0% meet needs of lib/uni improve lib instruction VT libraries ARL needs of external constituents professsional recognition ARL w/faculty status opportunity for scholarship meet needs of profession ARL w/out faculty status Q5. Technology-based projects librarians developed in last 3 years appropriate for evaluation VT Libraries ARL ARL w/Faculty Status ARL w/out Facult Status 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% subj/res . web maj. web proj. pers. web page article in ej softwar e listserv 80% 50% 40% 25% 15% 10% 5% ARL 100% 88% 34% 47% 24% 38% 28% ARL w/Faculty Status 100% 91% 45% 55% 30% 36% 38% ARL w/out Facult Status 100% 81% 24% 31% 13% 35% 19% VT Libraries video/au dio Q5. Other types of technology-based projects for evaluations (comments) • • • • Team-taught, online course Timelines Oral history interviews Database design: input, testing, etc. Q6 and Q7 Within the past 3 years has there been an increase in the number of tech-based projects Q6. ...on which you collaborated? Q7. ...that you initiated? no 33% no 41% yes 67% yes 59% Q6. Within the past 3 years, has there been an increase in the number of tech-based projects on which you collaborated? 19 librarians collaborated on 30 projects. with others in library outside department with others in department with others in university outside library 16.7% 26.3% 40.0% 63.2% 43.3% 68.4% Q8. The merit of a tech-based project may be assessed through its quality, relevance, or contribution to the profession. Indicate which materials you might evaluate. VT Libraries ARL w/out faculty status ARL w/faculty status ARL members product itself project summ. project descrip. end-user comments 93% 94% 96% 95% 56% 88% 70% 75% 33% 73% 68% 73% 78% 53% 55% 58% peer review 70% 59% 55% 57% nonrefereed print comp. project procedures refereed print comp. 7% 27% 38% 34% 30% 33% 24% 26% 37% 7% 38% 26% Q8. Other materials to evaluate when deciding on the merit of a tech-based project • Benefit to general public • Usefulness to me or its users • Meets one of more needs of its target audience • 41%: decreases staff hours required since using technology • VT: usefulness • ARL: awards Q9. Criteria used to determine merit 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% devel op. ide as co mple x des ign reviewed prio r devel op. 67% 37% 19% 26% 88% 75% 53% 50% 25% 83% 73% 77% 67% 43% 57% 81% 78% 73% 62% 48% 45% uni que/ cre ative schola rly co ntrib ut recog-local VT Libra ries 63% 93% 74% ARL w/o ut facu lty status 81% 76% ARL w/facu lty status 83% ARL m embe rs 83% recog-nat'l/i'n a tio nal Q9. Other criteria that should be used to determine merit of tech-based projects • • • • Enhances distance learning initiatives Operations run better; more more efficiently Usefulness Contribution to the University • Serves the needs of the end-user Q10. Should P&CA Guidelines specifically address tech-based projects? 100% 88% 90% 83% yes 80% no 70% 60% 50% 48% 52% 40% 30% 17% 20% 12% 10% 0% VT ARL ARL w/Faculty Status Q10. Should P&CA Guidelines specifically address tech-based projects? Yes. • Clarification about types of materials and how they will be evaluated. • Reminder that content and quality of the project is key, not the format. • So that it is clearly understood that such projects are worthy of recognition. • To make sure that everyone knows achievement in this domain is valued. Q10. Should P&CA Guidelines specifically address tech-based projects? No. • Acceptable projects should be defined by these guidelines • Treat like other projects: does it serve a real need? • Any innovative project should be considered • Tell when and when not to include Web pages in dossier Q11. Do tech-based projects deserve more credibility? Yes No (same + less) 69% 65% 61% 49% 51% 39% 35% 31% VT ARL ARL w/Faculty Status ARL w/out Faculty Status Q11. Do tech-based projects deserve more credibility? • If yes, what would give them more credibility? – – – – Creativity, accuracy, relevance, originality Standardize procedures and documentation Financial recognition in annual salary Motivate people who could do more • If they deserve less, why? – Technology is a process not a “product” – Technology is a tool. The real determining factor is: has it met a need and is it helping those in its target audience? Q12. Will tech-based projects ever be sufficient evidence of scholarship? VT ARL w/ Faculty Status 10 0% 90 % 80 % 70 % 60% 60 % 50% 50% 50 % 40% 40 % 30 % 20 % 10 % 0% Yes No