Materials linked from the Computing Resources Committee 2011‐2012 Annual Report. THE OREGON STATE UNIVERSITY
BLACKBOARD TOOLS EVALUATION SURVEY
Brief Summary
2012 Background The Oregon State University Blackboard Tools Evaluation Surveys were conducted by the Survey Research Center at Oregon State University (OSU‐SRC) during Spring Quarter 2008. The purpose of this study was to ask undergraduate students and instructors at Oregon State University about the use of OSU Blackboard and to identify aspects of the Blackboard system and related infrastructure that may be improved to better serve the OSU educational community. This survey is part of a review initiated by the Faculty Senate Computing Resources Committee in cooperation with OSU Information Services. Methodology Every instructor who taught a course anytime between spring quarter 2007 and winter quarter 2008 received an invitation from the OSU Survey Research Center to participate in the online survey. In addition, every student that was enrolled at OSU during winter quarter 2008 had the chance to be included in this study. A random sample of 2,500 graduate and undergraduate students was compiled by Frank Kessel of OSU Enterprise Computing. An email was sent to the selected instructors and students with a link to the survey followed up by a paper survey. Response Rates The response rates for the surveys were 27% for the students and 33% for the instructors. Demographics The instructor/faculty respondents were primarily from the College of Science followed by the College of Liberal Arts. They were primarily tenured professors or instructors. Table 1: Demographics (Instructors) College College of Science College of Liberal Arts HHS Engineering Agricultural Sciences 

19%
19%
13%
11%
10%
Status
Tenured professors Instructors
Tenure–track Other1
30% 23% 12% 10 % The average total credit hours taught by the respondents between spring quarter 2007 and winter quarter 2008 was 13.5 hours (Question 3). The average credit hours not using Blackboard during this same time period was 5.3 hours while the average credit hours using Blackboard during this same time period was 10.4 hours Instructors, on average had 115 students in their courses using Blackboard. 
1
Primarily courtesy faculty, graduate assistants, emeritus and professional faculty, and program directors. Report prepared December 2010 1 The student responses represented all class levels from freshman to non‐degree seeking students to post‐baccalaureate students. Most of the respondents were upper‐division undergraduates. Table 2: Student demographics Class standing Senior Junior Freshman Sophomore MA/Doctorate Percent
24%
21%
15%
13%
22%


84% of the students were on the Corvallis campus. 2% were at the Cascades campus and 13% were Ecampus students. Most of the students used Blackboard in over half of their classes. Table 3: Blackboard use in classes Percentage of classes using Blackboard
100% 51‐99% 50% 1‐49% 0% Percent
34%
36%
11%
11%
8%
Results Instructors and students were asked about the importance of certain features in Blackboard and their use of Blackboard as well as their perceived skill at using Blackboard. In some cases, a comparison between the two populations is valuable. Use Instructors were asked to describe their uses of Blackboard to engage in certain activities, such as communication and assessment. Instructors are most likely to use Blackboard to communicate with their students and to continue to do so. They are least likely to use Blackboard to evaluate the level of student knowledge, understanding or progress because they did not know how to use the tool, effectively for this purpose. This could indicate an instructional need. Have used and plan to continue Have used do not plan to continue Have not used because does not meet my needs Have never used because not familiar with the tools Other Communicate with students Evaluate the level of student knowledge, understanding or progress Allow students to communicate with one another outside of class 94%
1%
.20%
33%
4%
18%
42%
4%
8%
1%
36%
31%
3%
10%
14%
Value Report prepared December 2010 2 Instructors were asked what processes or activities they felt most important in their teaching and learning. These are all processes that can be supported by different Blackboard tools. Instructors value the ability to assess their students learning to help them prepare future lessons as the most important process. The instructors value most of the other processes highly. The ability to post course material and the ability to communicate with the students are the most important processes. The least valued is the students’ ability to communicate outside of class. The ability to communicate with students as a group and/or by means other than face‐to‐face The ability of students to communicate with one another outside the classroom The ability to post course materials online The ability to post grades and scores online The ability to evaluate what students know, understand, or have yet to learn, so that you may plan appropriate activities for class meetings/activities Very Important Somewhat Slightly important important important Not at all important 56% 23% 11% 5% 3% Not sure/No basis for opinion .95% 20% 24% 24% 11% 14% 7% 70% 15% 6% 3% 5% .63% 45% 17% 11% 7% 17% 2% 34% 26% 14% 8% 11% 7% Students responded to a question about similar processes and how important the Blackboard tools were for their learning processes. For them, the primary value of Blackboard is having access to online course materials followed by having access to grades. The ability to communicate with instructors by means other than face‐to‐
face The ability to communicate with other students by means other than face‐to‐
face Having online access to course materials Having online access to your course grades and scores Very Important Somewhat Slightly important important important Not at all important 40% 36% 15% 6% 2% Not sure/No basis for opinion 0.59% 19% 26% 27% 18% 9% 1% 74% 71% 19% 20% 5% 6% 0.44% 1% 0.29% 1% 0.29% 2% While instructors value the ability to post materials online as much as students, they do not value using the online grade book as much. The contrast between the value of the grade book as perceived by instructors (45%) and students (71%) indicates an important issue to further investigate. Effective Use Instructors were asked to rate themselves and their students on their effective use of Blackboard. Students, they found, are effective to very effective in their use of Blackboard. Students rate themselves more effective in their use of Blackboard than their instructors do. Report prepared December 2010 3 In your opinion, how effective are your students in making use of the OSU Blackboard system 27% 45% 23% 3% 0.59% 2% Very effective Effective Somewhat effective Slightly effective Not at all effective No response Overall, how effective do you rate your own ability to use OSU Blackboard (students) 38% 48% 11% 2% ‐‐ 3% Skill Students and instructors were asked to rate their skill levels at using Blackboard. Instructors consider themselves to be somewhat effective to effective with only a few who rate themselves as highly effective. Students rate themselves as highly effective or effective users of the system. How effective do you rate your own ability to use OSU Blackboard? (Instructors) 9% 46% 35% 8% 2% 0.39% Very effective Effective Somewhat effective Slightly effective Not at all effective No response Overall, how effective do you rate your own ability to use OSU Blackboard? 38% 48% 11% 2% ‐‐ 3% Feedback Most students report being given no opportunity to provide faculty with feedback regarding their use of the Blackboard system. Instructors indicate they provide these opportunities with more frequency that the students do. It is possible that instructors provide these opportunities but students either do not see them or take advantage of them. To a great extent To a moderate extent Very little Not at all No response To what extent do you ask for feedback from your students regarding your uses of OSU Blackboard for instruction? 9% 32% 34% 25% 0% 100% (All) 51‐99% (Most) 50% (Half) 1‐49% (Less than half) 0% (None) No response Of those instructors who use OSU Blackboard this quarter, what percentage ask for your feedback regarding their uses of OSU Blackboard for instruction? 6% 7% 7% 16% 61% 3% Assisting instructors in providing students with feedback opportunities about Blackboard, or increasing student awareness of those opportunities, will contribute to more consistent perceptions about the value of the system. Report prepared December 2010 4