Undergraduates' Perceptions & Use of a LMS 1 Running Head: Undergraduates' Perceptions & Use of a LMS Investigating Undergraduates' Perceptions and Use of a Learning Management System: A Tale of Two Campuses Steven Lonn University of Michigan Stephanie D. Teasley University of Michigan Andrew E. Krumm University of Michigan Please address all correspondence to: Steven Lonn University of Michigan Digital Media Commons & School of Education 2281 Bonisteel Blvd., Ann Arbor, MI 48109-2094 Email: slonn@umich.edu Paper presented at the Annual Meeting of the American Educational Research Association San Diego, California, April 16, 2009 Undergraduates' Perceptions & Use of a LMS 2 Investigating Undergraduates' Perceptions and Use of a Learning Management System: A Tale of Two Campuses Steven Lonn1, Stephanie D. Teasley2, & Andrew E. Krumm1 University of Michigan Digital Media Commons, School of Education1, and School of Information2 Abstract: Learning Management Systems (LMS) are web-based systems allowing instructors and/or students to share materials and interact online. We explored the differences between undergraduate students at a large residential campus with students at a smaller commuter campus who used the same LMS. We examined students’ survey responses, focusing on items related to materials management and interactive teaching and learning. We also examined aggregated log data to see if students’ use was consistent with their beliefs. Students from both campuses rate highly all activities and tools within the LMS. Findings suggest, however, that residential students value activities and tools for materials management more than commuter students, while commuter students value activities and tools fostering interactive teaching and learning more than residential students. The use of technology for teaching and learning in higher education has grown exponentially in the past decade, particularly the use of web-based systems. A recent report (Hawkins & Rudy, 2007) showed that over 90% of all responding universities and colleges have established one or more Learning Management Systems (LMS) to allow instructors and students to share instructional materials, make class announcements, submit and return course assignments, and communicate with each other online. In prior research (Lonn & Teasley, 2008), we coded LMS use into two categories that distinguished whether the LMS was used for efficient instructor communication (e.g., posting online readings) or interactive teaching and learning (e.g., students commenting on each other's work). We found that both instructors and students more highly valued LMS for efficient instructor communication than for interactive teaching and learning uses. In the study presented here, we asked if this pattern was specific to a particular type of campus or if the results were related to the fact that the previous data set was based on the experiences of undergraduate students at a large, mainly residential campus. We therefore extended our previous work by examining the differences in students’ attitudes about LMS use and actual experience with a LMS for undergraduates at a large residential campus and undergraduates at a smaller commuter campus who were using the same LMS product. We examined students’ survey responses, focusing on items specifically related to materials management versus more interactive teaching and learning practices. Aggregated log data from the LMS were also examined to see if students’ system use was consistent with their beliefs as shown by the survey results. Background Research investigating the differences between commuter and residential students has found that commuters tend to be the first generation to attend college in their family, their parents are more frequently "blue collar" workers with less education than residential students, and, overall, commuter students are less prepared for the academic demands of college (Chickering, 1974). Despite these differences in background and preparation, Chickering found Undergraduates' Perceptions & Use of a LMS 3 that commuter students tend to perform academically as well as residential students. Later studies have found whether students lived on campus was a very important factor of educational impact, stating that residential students, compared with commuters, participate in more extracurricular activities, interact more frequently with faculty and peers, are more satisfied, are more likely to graduate from college, and exhibit greater gains in educational goals and outcomes (Pascarella, Terenzini, & Blimling, 1994). Overall, while some research in this area posits that commuter students are at a disadvantage and face several challenges in creating developmental experiences comparable to residential students (Smith, 1989), other researchers argue that the most important factor affecting what students gain from college is the quality of effort they exert in academic and social activities, regardless of their residential status (Pace, 1979; 1984). In this study, we applied this area of research in our exploration of differences between residential and commuter students with regards to their attitudes and use of LMS. When LMS first began to gain popularity, the early adopters cited efficiency affordances of the technology more frequently than teaching and learning activities (Morgan, 2003). Other survey-based studies have similarly found that instructors and students value materials management and housekeeping activities over more interactive uses of LMS (e.g., Ansorge & Bendus, 2003; Herse & Lee, 2005; Holm, Röllinghoff, & Ninck, 2003; Yohon, Zimmerman, & Keeler, 2004). However, a study by Hanson & Robson, (2004) suggests that connecting perception with actual use is not easy. They examined the use of two commercial LMS (WebCT and Blackboard) and found that more students than faculty reported that these systems "improved learning." When asked to select the specific benefits of using these systems, however, both instructors and students chose "saves time" more often than "improves learning." Therefore it is important to examine both perceptions about use and actual use of these systems. Although there have been some multi-campus studies of LMS (e.g., Harrington, Gordon, & Schibik, 2004; Morgan, 2003), none have focused on students’ perceptions of use nor on the possible differences in use between residential and commuter institutions. While researchers who focus on commuter students and their experiences note that information and communication technologies are a significant force affecting the nature of student interaction with each other and with their instructors (Krause, 2007), there continues to be little scholarly exploration about the nature of the commuter student experience (Pascarella, 2006; Pascarella, Duby, & Iverson, 1983) particularly with respect to the role of IT in their educational experience. Our study serves to fill several gaps in the literature by examining students' perceptions and actual use of the same LMS at a residential campus and a commuter campus, focusing specifically on activities related to materials management and interactive teaching and learning practices. Methods Data Sources We administered an online survey in the spring of 2008 at the main campus of a large American Midwestern research university (the residential campus) and at a smaller satellite campus of the research university (the commuter campus). According to the Carnegie Undergraduates' Perceptions & Use of a LMS 4 Classification of Institutions of Higher Education (http://www.carnegiefoundation.org/classifications/), the residential campus is a large, public, four-year research university with very high research activity and a majority undergraduate enrollment. The commuter campus is a medium, public, four-year master's university with a large overall program and a high undergraduate enrollment. We used the system's event logs to create aggregated data representing the activity in the course sites in which the student participants were enrolled during the survey period (Winter term 2008). Event logs capture when a user takes a particular action with a specific tool, such as downloading a document or posting an announcement. Participants All students from the commuter campus and a random sample (stratified by school/college) of 25% of students from the residential campus were invited to participate in the survey. The sample for this study included only undergraduate students who participated in the online survey and were enrolled in at least one course that had an active site within the LMS (residential campus, n=1,686; commuter campus, n=898). The demographics for the two campuses were very similar. For gender, 58% of the residential students were female compared with 63% of the commuter students. For ethnicity, 65% of the residential students identified themselves as white compared with 67% of the commuter students. For the number of years in school, 26% of the residential students were firstyear undergraduates, 19% were second-year, 25% were third-year, and 30% were undergraduates for four or more years. On the commuter campus, 19% of the students were first-year undergraduates, 19% were second-year, 27% were third-year, and 35% were undergraduates for four or more years. Finally, when looking at students' majors (see Table 1), the most popular category was social sciences on both the residential campus (30%) and the commuter campus (28%). Table 1: Category of Students' Majors by Campus 16% Natural Sciences 19% Social Sciences 30% Professional School 8% 13% 18% 28% 24% Campus Engineering Humanities Residential 18% Commuter 13% Undeclared 9% 4% Nearly all courses at the residential campus are conducted in traditional lecture sections where students have direct face time with the course instructor. The commuter campus offers three types of course experiences, so we asked those students to identify their main experience: traditional face-to-face courses (83%), blended courses that have some face-to-face components and some online components (16%), and virtual courses conducted solely online (1%). Design & Procedure Undergraduates' Perceptions & Use of a LMS 5 The LMS examined in this study is based on the Sakai community-source architecture (http://sakaiproject.org). This environment is comparable to other popular systems such as Blackboard (http://blackboard.com) and Moodle (http://moodle.org). A total of 1,584 course sites from the residential campus and 248 sites from the commuter campus were analyzed in this study. Within LMS course sites, instructors may elect to use any of the following tools: • • • • • • • • • • • • • • • Announcements: Non-threaded, asynchronous messages for all site participants to read. Email notifications may be sent as well. Assignments: Allows students to upload and submit assignments and projects and instructors to grade and comment on students' submissions. Chat: Synchronous conversation tool. All messages are automatically saved and are viewable for all site participants. Content Sharing: File storage space. Any file type may be stored; URLs to other websites may be created, and shared citation lists may be created. Email notifications of new content may be sent as well. Discussion: Threaded, asynchronous messages for all site participants to read. Drop Box: Private file storage space for sharing between individual students and the instructor. Email Archive: All email sent via a site-specific email address is delivered to all participants and also saved online for archival and searching. Forums: A threaded, asynchronous messaging tool, similar to the Discussion tool but with more finely grained options and permission settings. Gradebook: An online grading tool that allows the instructor to grade any assignment or exam and share that information with students, who view only their own scores. Messages: A non-threaded, asynchronous messaging tool that can either send messages to groups of students or individuals. News: Allows site participants to view RSS feeds from external sources. Schedule: A shared calendar used to post deadlines, due dates, etc. Syllabus: Instructors may use this tool to post their syllabus as HTML or an attachment. Web Content: Allows site participants to view external websites. Wiki: A collaborative document-writing tool. Any site participant may add or modify additional pages and a history of changes is automatically recorded. Our analysis of the survey data examined questions that could be categorized as either materials management or interactive teaching & learning items. These results were then compared to patterns of use shown in the system log data that reported aggregated user actions in the LMS sites for the courses in which the student participants were enrolled. Given the different sample sizes, we also analyzed the effect sizes of the significant statistical results, typically generating a small effect size (Cohen's d greater than .15 and less than .40). Results Materials Management and Interactive Teaching & Learning Activities Our analysis for this study categorized several survey items about specific activities within the LMS as either supporting "materials management" or "teaching & learning" (see Undergraduates' Perceptions & Use of a LMS 6 Table 2). These items asked the students to rate their agreement that the LMS was valuable for the various activities on a 5-point Likert scale from 1 = Strongly Disagree to 5 = Strongly Agree. Table 2: Survey Items by Category Materials Management • • • • • • Interactive Teaching & Learning Access a syllabus online Receive messages or notifications from the instructor Access online readings & supplementary course materials Access sample exams & quizzes for learning purposes Access lecture outline or notes after the lecture View calendar / schedule of activities • • • • • • Students turn in assignments online Instructors return assignments to students online with comments & grade Visit online office hours Students post questions before lecture Students post questions after lecture Students work together on a task or assignment In general, all students rated the activities provided by the LMS quite positively as almost all items received a rating at 4.0 or above (see Tables 3 & 4). There were some significant differences in how residential and commuter students rated the materials management items (see Table 3). The residential students rated two of the items significantly higher than the commuter students: Access online readings (t (1,298) = 4.907, p < .001; Cohen's d = .27) and Access lecture outline after the lecture (t (1,222) = 2.840, p = .005; Cohen's d = .16). The commuter students rated the View calendar / schedule item significantly higher than the residential students: (t (1,690) = 4.230, p < .001; Cohen's d = .21). Table 3: Mean Differences between Students' Answers on the Materials Management Survey Items Item Access a syllabus online Residential Students Commuter Students Mean Difference 4.53 4.48 (n=1,470) (n=784) .05 Receive messages / notifications from instructor 4.38 4.31 (n=1,460) (n=780) Access online readings & supplementary materials 4.54 4.37 (n=1,467) (n=780) Access sample exams & quizzes 4.47 4.37 (n=1,462) (n=778) Access lecture outline or notes after the lecture 4.57 4.46 (n=1,464) (n=781) View calendar / schedule of activities 3.89 4.14 (n=1,462) (n=781) Note: * p < .001 .07 .17* .10 .09** .25* ** p < .05 There were also significant differences in how residential and commuter students rated several of the interactive teaching & learning items (see Table 4). The commuter students rated two of the items significantly higher than the residential students: Instructors return assignments Undergraduates' Perceptions & Use of a LMS 7 to students online (t (1,622) = 4.101, p < .001; Cohen's d = .20) and Post questions after lecture (t (1,742) = 1.992, p = .047; Cohen's d = .10). Table 4: Mean Differences between Students' Answers on the Interactive Teaching & Learning Survey Items Item Residential Students Commuter Students Mean Difference .04 Students turn in assignments online 4.27 4.31 (n=1,457) (n=778) Instructors return assignments to students online 4.01 4.25 (n=1,461) (n=780) Visit online office hours Post questions before lecture Post questions after lecture Students work together on a task or assignment Note: * p < .001 4.35 4.42 (n=1,461) (n=778) 4.21 4.32 (n=1,462) (n=779) 4.27 4.38 (n=1,456) (n=777) 4.08 4.16 (n=1,458) (n=778) .24* .07 .11 .11* .08 ** p < .05 Student Attitudes About LMS Tools The activities described above occur within several LMS tools. We therefore categorized the tools available to students into materials management or interactive teaching and learning based on the tools' predominant mode of use. The students were asked to rate their agreement that the LMS tools were valuable for their course-related activities on a 5-point Likert scale from 1 = Strongly Disagree to 5 = Strongly Agree. Like the activities described in Table 1, all students rated the tools within the LMS quite positively as almost all items received a rating at 4.0 or above (see Table 5). For the materials management tools, the residential students rated the Announcements tool significantly higher than the commuter students (t (1,383) = 2.237, p = .025; Cohen's d = .12) and the commuter students rated two tools significantly higher than the residential students: Drop Box (t (1,509) = 3.276, p = .001; Cohen's d = .17) and Schedule (t (1,819) = 4.427, p < .001; Cohen's d = .21). The commuter students also rated four interactive teaching and learning tools significantly higher than the residential students: Chat Room (t (1,603) = 6.858, p < .001; Cohen's d = .34), Discussion (t (1,786) = 8.018, p < .001; Cohen's d = .38), Email Archive (t (1,551) = 2.656, p = .008; Cohen's d = .13), and Forums (t (1,739) = 2.318, p = .021; Cohen's d = .06). Undergraduates' Perceptions & Use of a LMS 8 Table 5: Mean Differences between Students for Attitudes about LMS Tools Category Tool Announcements Drop Box Gradebook Materials Management News Content Sharing Schedule Syllabus Assignments Chat Room Discussion Interactive Teaching & Learning Email Archive Forums Messages Wiki Note: * p < .001 Residential Students Commuter Students Mean Difference 4.50 4.43 (n=1,454) (n=783) .07** 4.19 4.37 (n=1,454) (n=778) 4.27 4.47 (n=1,454) (n=779) 4.42 4.45 (n=1,448) (n=775) 4.56 4.50 (n=1,448) (n=779) 4.22 4.45 (n=1,452) (n=776) 4.46 4.51 (n=1,452) (n=777) 4.44 4.39 (n=1,454) (n=782) 3.81 4.23 (n=1,455) (n=782) 3.83 4.28 (n=1,454) (n=781) 4.25 4.42 (n=1,451) (n=781) 4.30 4.45 (n=1,452) (n=775) 4.32 4.40 (n=1,449) (n=778) 4.53 4.60 (n=1,436) (n=772) .18** .20* .03 .06 .23* .05 .05 .42* .45* .17** .15** .08 .07 ** p < .05 LMS Event Log Data We investigated LMS event logs to see if actual system use was consistent with preferences expressed in the survey data. We began this phase of the analysis by comparing whether each LMS tool was turned on across all course sites in which the student participants were enrolled in order to help determine if the students were exposed to the same tools on both campuses. This preliminary analysis revealed that tool activation was consistent across campuses. To better understand the use of these tools, we calculated the average percentage of total events for each LMS tool by aggregating the events for all LMS course sites in which the student participants were enrolled (see Table 6). Tool use was generally consistent across campuses for Undergraduates' Perceptions & Use of a LMS 9 the materials management tools except for the Content Sharing / Drop Box tools which had a significantly higher average percentage of use by the residential campus sites than the commuter campus sites (t (301) = 4.013, p < .001; Cohen's d = .46). The residential campus sites also had a significantly higher average percentage of use than the commuter campus sites for one interactive teaching and learning tool, Email Archive (t (1,725) = 4.199, p < .001; Cohen's d = .20). The commuter campus sites had a significantly higher average percentage of use than the residential campus sites for three interactive teaching and learning tools: Assignments (t (298) = 2.688, p = .008; Cohen's d = .31), Discussion (t (274) = 4.164, p < .001; Cohen's d = .50), and Forums / Messages (t (286) = 2.109, p = .036; Cohen's d = .25). Table 6: Mean Percentage (of Total Site Events) Differences Between Institutions for LMS Tool Use Based on Event Logs Category Materials Management Interactive Teaching & Learning Note: * p < .001 Residential Campus (n=1,584 sites) Commuter Campus (n=248 sites) Mean Difference Announcements 2.49% 2.74% .25 Gradebook 1.25% 0.9% .35 News 0.39% 0.39% 0 Content Sharing / Drop Box 83.43% 75.64% 7.79* Schedule 0.35% 0.39% .04 Syllabus 0.42% 0.67% .25 Assignments 6.00% 9.16% 3.16** Chat Room 1.12% 1.36% .24 Discussion 1.43% 4.56% 3.13* Email Archive 0.75% 0.06% .69* Forums / Messages 2.16% 4.05% 1.89** Wiki 0.20% 0.06% .14 Tool ** p < .05 Student Opinions of LMS Affect of Instructors' Use of In-Class Time After examining students' perceptions of activities and specific tools, we wanted to explore if LMS use changed the nature of their in-class experiences. On our survey, students from both campuses were asked a qualitative question: "Do you think using (the LMS) has Undergraduates' Perceptions & Use of a LMS 10 affected instructors' use of in-class time? If so, how?" Each student's answer was coded by two raters with an inter-rater reliability of α = .870 (n=375). There were a total of 1,168 comments (n=1,100 students) from the residential campus and 639 comments (n=621 students) from the commuter campus. We organized the comments into three categories (eliminating comments coded as "not applicable"): Positive Effect, No Effect, and Negative Effect (see Table 7). In general, the majority of students on both campuses believed that the LMS had a positive effect on the use of class time, although more of the residential students reported a positive effect. More of the commuter students reported no effect than did the residential students. Very few students on either campus stated a negative effect regarding class time. There was a significant difference in the type of student comments from each campus (F (1,1,726) = 4.947, p = .026; Cohen's d = .10). Table 7: Categorical Breakdown by University of Student Comments About LMS Effect of Instructors' Use of In-Class Time University Positive Effect No Effect Negative Effect Residential Students 64.8% 29.7% 5.5% (n=733) (n=336) (n=62) Commuter Students 58.6% 35.3% 6.0% (n=350) (n=211) (n=36) In order to understand why students believed the LMS had a positive effect of class time, we further coded their comments into materials management or interactive teaching and learning categories (see Table 8). Although there was no significant difference between residential and commuter students for the global materials management versus interactive teaching and learning categories, there were significant differences between students for the individual coded categories (X2(11, N=1,032) = 33.291, p < .001). For both types of students, most of the positive effects in the classroom were related to materials management issues. Specifically, the use of the LMS led to more efficient use of class time was the most frequent answer offered by students on both campuses, although somewhat higher for commuter students. There were fewer positive comments about interactive teaching and learning effects, although students on both campuses said that class time was improved because the posting of lecture slides allowed more focus during lectures. Undergraduates' Perceptions & Use of a LMS 11 Table 8: Percentages of Student Comments About LMS Effect of Instructors' Use of In-Class Time by University Categorized as Positive Effect Category SubCategory Efficiency Code Example Quote Residential Students % of Positive Effect More efficient use of class time "Typically it reduces time needed to pass out notes or other documents, etc., so more time can be used to teach." 53.4% 57.0% 10.0% 4.4% 1.3% 0.6% 8.3% 10.9% 0.8% 3.1% 1.3% 0.9% 8.0% 5.9% 2.0% 1.2% 85.1% 84.1% 9.3% 8.1% 2.8% 1.9% 2.7% 4.0% 0.1% 1.9% 14.9% 15.9% "I think it has. They go over lecture slides much quicker, which allows them to put more on the tests and cover more material in a class." "Yes. Instructors can post power-point lectures so that students can be more attentive during class & not worry about taking so many notes." "(Instructors) can instruct students to view the material and comment or just view the material for the next class." Cover more material Less Note Taking To materials Materials Management Access To assignments To supplementary materials Better Preparation By students for class By instructors for class "Yes, they post assignments online and give detailed descriptions, so they don't have to describe the assignments in class." “Professors can put supplemental material or information on (the LMS) and therefore give the student the ability to enrich their learning without spending valuable class time on it." "I think it is helpful in that the instructors can provide handouts and assignments before class, so the students can come in already prepared for lecture." "Yes it does, using (the LMS) helped my instructors plan ahead to optimize the use of class time for more important matters." SUBTOTAL More Focused Interactive Teaching & Learning Communication Instruction / lecture In-class discussion Improved overall Better follow-up discussions "Instructors who post lecture slides before/after class tend to be more focused in their lectures and more clear in their content." " We can come in focused and continue a discussion that was already started online, using (the LMS)." "Yes, (the LMS) has allowed instructors to connect with their students more fully outside of class. (It) permit(s) instructors to keep students updated on what will be happening in class, and allow(s) students to comment on/shape the structure of the class to best suit their learning needs." "Yes, I believe that is a great tool for questions or answers that could not be covered during class time." SUBTOTAL Commuter Students % of Positive Effect Discussion Overall, students on both residential and commuter campuses rated course-related activities within the LMS very highly. They likewise rated the LMS tools supporting those activities quite highly. When asked about the effect of the LMS on their instructors' use of inclass time, a majority of students (65% of the residential and 59% of commuter students) reported a positive effect, particularly with regards to improved efficiency and access to Undergraduates' Perceptions & Use of a LMS 12 materials. Although 30% of residential student and 35% of commuter students felt there was no effect on class time, very few students from either campus reported a negative effect of the LMS. In general, students enjoy the use of the LMS for their courses and view their activities using the system as valuable and positive. When we specifically examined the reasons underlying the positive ratings of the activities and tools in the LMS, a distinctive pattern emerged: residential students rated materials management activities more highly than commuter students, and commuter students rated interactive teaching and learning activities more highly than residential students. For materials management, this pattern is most pronounced in the activities of accessing online readings and supplementary materials and accessing lecture outlines, replicating findings found in previous LMS studies for both instructors (Hanson & Robson, 2004; Holm et al., 2003; Yohon et al., 2004) and students (Herse & Lee, 2005; Maslowski, Visscher, & Collis, 2000; Parker, Bianchi, & Cheah, 2008). For interactive teaching and learning, the commuter campus students rated activities such as posting questions after lecture and receiving assignments from instructors significantly higher than residential students. While this pattern was not evident in the tool ratings for materials management tools, the commuter students rated four interactive tools higher than residential students. The log data supported these findings showing higher activity in the most heavily used materials management tools for the residential campus and higher activity in three of the interactive teaching and learning tools for the commuter campus. One explanation for our results is that commuter students may have relied on interactive tools within the LMS to communicate with instructors and students with whom they do not otherwise have opportunities for face-to-face interaction. This may indeed be why we saw that the LMS sites at the commuter campus had a higher use of discussion board tools. The use of these tools may help compensate for a previous lack of interaction with instructors by commuter students, as compared with residential students (Pascarella et al., 1994) Furthermore, instructors at the commuter campus may have structured their courses to include more student interaction through the LMS as a consequence of diminished face-to-face time that is likely to be more easily accomplished at the residential campus. Thus, instructors and students at the two campuses may use the same tools, but in different ways and for different purposes. Our study highlights the need for further exploration into the differences between the use of LMS technologies at residential and commuter campuses. Our results indicate that while all students rate LMS activities and tools quite highly, statistically significant differences in use exist between campuses. We argue that this difference is attributable, at least in part, to different patterns of interaction between students and instructors at residential and commuter campuses. Further study into exactly how residential and commuter courses structure the use of the LMS for course-related activities outside the classroom may help instructors and students at both types of campuses use this technology to its fullest potential. By looking more deeply at the potential ways to use LMS within the context of all university courses, researchers can help instructors and students realize the full benefits of Learning Management Systems. Undergraduates' Perceptions & Use of a LMS 13 References Ansorge, C. J., & Bendus, O. (2003). The pedagogical impact of course management systems on faculty, students, and institution. In R. H. Bruning, C. A. Horn, & L. M. PytlikZillig (Eds.), Web-based learning: What do we know? Where do we go? (pp. 169-190)., Greenwich, CT: Information Age Publishing Hanson, P., & Robson, R. (2004). Evaluating course management technology: A pilot study. Boulder, CO: EDUCAUSE Center for Applied Research, Research Bulletin, Issue 24. Harrington, C. F., Gordon, S. A., & Schibik, T. J. (2004). Course management system utilization and implications for practice: A national survey of department chairpersons. Online Journal of Distance Learning Administration, 7(4) http://www.westga.edu/~distance/ojdla/winter74/harrington74.htm Hawkins, B. L., & Rudy, J. A. (2007). Educause core data service: Fiscal year 2006 summary report. Boulder, CO: Educause. Herse, P., & Lee, A. (2005). Optometry and WebCT: A student survey of the value of web-based learning environments in optometric education. Clinical and Experimental Optometry, 88(1), 46-52. Holm, C., Röllinghoff, A., & Ninck, A. (2003). WebCT and elearning in Switzerland. Proceedings of the 5th International Conference on New Educational Environments, Luzerne. 139-143. Koszalka, T. A., & Ganesan, R. (2004). Designing online courses: A taxonomy to guide strategic use of features available in course management systems (CMS) in distance education. Distance Education, 25(2), 243-256. Krause, K-L.D. (2007). Social involvement and commuter students: The first-year student voice. Journal of the First-Year Experience & Students in Transition, 19(1), 27-45. Lonn, S. & Teasley, S. D. (2008). Investigating Perceptions and Uses of Learning Management Systems: Saving Time or Innovating Practice? Paper presented at the Annual Meeting of the American Educational Research Association. New York, March 26. Maslowski, R., Visscher, A. J., & Collis, B. (2000). The formative evaluation of a web-based course-management system within a university setting. Educational Technology, 40(3), 5-19. Pace, C. R. (1979). Measuring outcomes of college: Fifty years of findings and recommendations for the future. San Francisco: Jossey-Bass. Pace, C. R. (1984). Measuring the quality of college student experiences. Los Angeles: Higher Education Research Institute, Graduate School of Education, University of Los Angeles. Parker, R. E., Bianchi, A. & Cheah, T. Y. (2008). Perceptions of instructional technology: Factors of influence and anticipated consequences. Educational Technology & Society, 11(2), 274-293. Pascarella, E. (2006). How college affects students: Ten directions for future research. Journal of College Student Development, 47(5), 508-520. Pascarella, E., Duby, P. B., & Iverson, B. K. (1983). A text and reconceptualization of a theoretical model of college withdrawal in a commuter institution setting. Sociology of Education, 56(2), 88-100. Pascarella, E., Terenzini, P. T., & Blimling, G. (1994). The impact of residential life on students. In C. Schoeder, P. Mable, and Associates (Eds.), Realizing the educational potential of residence life on students. San Francisco: Jossey-Bass. Undergraduates' Perceptions & Use of a LMS 14 Smith, B. M. (1989). The personal development of the commuter student: What is known from the comparisons with resident students? An ERIC review. Community College Review, 17, 47-56. Yohon, T., Zimmerman, D., & Keeler, L. (2004). An exploratory study of adoption of course management software and accompanying instructional changes by faculty in the liberal arts and sciences. Electronic Journal of e-Learning, 2(2), 313-320.