Students’ Perceptions of Terrascope, A Project-Based Freshman Learning Community The MIT Faculty has made this article openly available. Please share how this access benefits you. Your story matters. Citation Lipson, Alberta, Ari Epstein, Rafael Bras, and Kip Hodges. 2007. Students’ Perceptions of Terrascope, A Project-Based Freshman Learning Community. Journal of Science Education and Technology 16, no. 4: 349-364. doi:10.1007/s10956-007-9046-6. As Published http://dx.doi.org/10.1007/s10956-007-9046-6 Publisher Springer Netherlands Version Author's final manuscript Accessed Thu May 26 06:09:56 EDT 2016 Citable Link http://hdl.handle.net/1721.1/49458 Terms of Use Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. Detailed Terms STUDENTS’ PERCEPTIONS OF TERRASCOPE, A PROJECT-BASED FRESHMAN LEARNING COMMUNITY Alberta Lipson1, Ari W. Epstein2*, Rafael Bras3 and Kip Hodges4 Suggested running head: Students’ Perceptions of Terrascope, a Project-Based Freshman Learning Community Corresponding Author: Ari W. Epstein Terrascope MIT Room 16-177 Cambridge, MA 02139 (617) 253-3666 awe@mit.edu 1 Teaching and Learning Laboratory, Massachusetts Institute of Technology, Cambridge, Massachusetts Terrascope Program, Office of Experiential Learning, Massachusetts Institute of Technology, Cambridge, Massachusetts * To whom correspondence should be addressed: Ari W. Epstein, Terrascope, MIT Room 16-177, Cambridge, MA 02139; awe@alum.mit.edu 3 Department of Civil and Environmental Engineering, Massachusetts Institute of Technology, Cambridge, Massachusetts 4 Department of Earth, Atmospheric and Planetary Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts (Present Affiliation: School of Earth and Space Exploration, Arizona State University, Tempe, Arizona) 2 1 ABSTRACT We present a descriptive case study of Terrascope, an innovative, year-long, project-based learning community at MIT. Each year, Terrascope students study a particular environmental or Earth-system problem from a multidisciplinary perspective. Terrascope includes both academic and non-academic components; this paper focuses on the academic components. The objectives of the academic subjects, and of the program as a whole, involve helping students develop their team-building, communication, problemsolving, and self-regulatory learning skills. This study focuses on cohorts of students from the first and second years of the program (2002-2003 and 2003-2004);it is based on end-of-semester surveys and focus groups, and on additional focus groups conducted when these students were upperclassmen. Students felt Terrascope helped them make significant improvements in their ability to work in teams and to take on complex, multidisciplinary problems. They felt that the program’s two-semester structure gave them an opportunity to develop and nurture these skills, and that the program prepared them well for their later work at MIT. They also felt that being engaged, as freshmen, in a distinct learning community, significantly eased their transition into MIT. We describe lessons learned in the development of Terrascope and offer suggestions for other institutions planning to develop similar programs. Keywords: project-based learning, experiential learning, teamwork, assessment, selfdirected learning, interdisciplinary learning, freshman, environmental education Introduction In 1998 the Task Force on Student Life and Learning at the Massachusetts Institute of Technology (MIT) issued a final report that highlighted as problems the lack of enthusiasm and excitement among freshmen and the lack of inquiry or project-based learning opportunities in the freshman curriculum. Following the release of this report, a number of recommendations on ways to invigorate the freshman year were made. One proposal concerned the introduction of a project-based learning experience as a unique addition to the standard freshman teaching format of large lecture classes, smaller recitations, problem-set homework, and exams. 2 This paper presents a descriptive case study of a project-based learning community called Terrascope (web.mit.edu/terrascope) that was introduced into the freshman curriculum in 2002. It discusses the evolution of the program and examines the immediate, end-of-semester perceptions of freshmen who participated in the program during the first year it was offered. It also presents retrospective reflections from those students and from the next class of Terrascopers (i.e. those who participated in the program’s second year), as gathered during focus-group sessions conducted when they were upperclassmen. In addition, the paper summarizes changes that have taken place over time, discusses some lessons learned in the process, and offers concrete suggestions for others who may wish to create similar programs. Origins of Terrascope Terrascope originated as one element of the educational arm of MIT’s Earth System Initiative, which facilitates and supports interdisciplinary research and educational efforts that focus on Earth system science and engineering. The founders of ESI chose to build on the educational philosophy of a subject called Solving Complex Problems, which was created in response to the report of the Task Force on Student Life and Learning. This subject was first offered in 2000-2001 with a broad thematic focus, not one centered on Earth systems. Terrascope incorporated the subject as one component of a year-long experience that includes both academic and non-academic elements. With its formation in 2002-03, Terrascope became the newest of four alternative learning communities available to MIT freshmen. 3 Description of the Program The Terrascope program as a whole is described in detail elsewhere (Epstein et al. 2006a, Epstein et al. 2006b); here we shall provide a brief overview. Terrascope is at its heart a community within which students engage in team-based, project-oriented, selfdirected learning focused on environmental and Earth system-related topics. Every year the program is centered on a particular complex, real-world problem that involves scientific, technical, social, economic and political aspects. In the fall subject (MIT Subject 12.000, Solving Complex Problems), the class as a whole develops solutions to the year’s core problem; at the end of the semester the students present and defend their solution before an international panel of experts convened for the purpose. In the spring subject (MIT Subject 1.016, Communicating Complex Environmental Issues: Designing and Building Interactive Museum Exhibits), Terrascopers design, engineer and build interactive museum exhibits through which the general public can learn about issues the students have explored all year; the exhibits are opened to the public, and many of them have later been adopted for use by established aquariums and science museums. In addition, some students in the spring take an optional subject (MIT subject SP.360, Terrascope Radio) in which they develop and produce a radio program on the year’s core topic; the program is broadcast on the MIT campus radio station and is then made available for use by public radio stations nationwide. In addition to its academic components, Terrascope includes a number of elements specifically focused on building and maintaining a community of learners. For example, all Terrascope students are provided with a freshman advisor affiliated with the program, as well as one or several undergraduate ―associate advisors‖ (upperclassmen 4 who participated in the program as freshmen). Terrascopers have exclusive access to a space on campus that includes a classroom, kitchen, lounge and computer cluster, which is open (and often in use) twenty-four hours a day. The program also sponsors weekly lunches and occasional outings. During spring break, Terrascopers participate in an optional field trip to a region closely connected with the year’s core problem. In addition, there are a number of ways in which upperclass alumni of Terrascope can stay actively engaged in the community throughout their time at MIT. In this paper we focus primarily on the academic components of Terrascope, particularly the two required classes. We also discuss the spring field trip, which is integral to both the academic and non-academic aspects of the program. The pedagogies of the two classes are complementary. The fall subject is more theoretical and less structured, whereas the spring one is more structured, emphasizes a hands-on approach to learning, and introduces students to the process of engineering design and development. Faculty members function as facilitators to encourage self-regulatory learning on the part of students and teams. Both subjects rely on upperclass students, called undergraduate teaching fellows (UTFs), to work with each team. They facilitate the students’ work, providing encouragement and guidance. In both classes, librarians who are specialists in the content areas are assigned to assist the teams. The fall class also includes mentors, generally alumni who are experts in the topic under study. The class environment encourages close, collegial relationships among faculty, students, undergraduate teaching fellows, and alumni. During the first week of the Fall semester, students brainstorm about the year’s core problem and the project is subdivided into topic areas that can be assigned to 5 individual teams. Students are encouraged to explore a variety of resources (e.g. books, journals, websites, GIS databases) to find information, and an extensive syllabus of readings is available on the class website. Students are expected to keep a personal journal and contribute to a message board on the class website, and each team is expected to maintain a web-based team journal. The class meets three times a week. Additionally, teams often meet outside of class. Teams are responsible for collaborating with one another to produce a coordinated solution to the problem. This is often the most challenging aspect of the class. The culmination of the class’s efforts is the presentation and defense of the mission plan to an expert panel; the presentation is webcast live and archived for later viewing. INSERT FIGURE 1 HERE In the spring class, as students conceptualize, design, and build their exhibits, they deepen their understanding of the problem they studied in the fall, sometimes choosing to approach it from a totally different perspective. Students brainstorm a number of exhibit ideas and are divided into teams, with each team in charge of developing a specific exhibit. Students do not have to work on an exhibit related to the topic they researched during the fall term, nor do they generally remain with the same teammates. Although both fall and spring classes give students a fair degree of autonomy, the spring class provides more structure in the form of performance milestones. Approximately every two weeks teams are required to complete some type of task, such as a preliminary design proposal, development of a working prototype, and a final design proposal. In addition to 6 the team-based performance milestones, individual students each keep a Developer’s Journal, in which they chronicle their experience in the class; the journal also gives them a regular opportunity to reflect on their work and progress, and on their own experience as learners. Students email their journal entries to the instructors, who read them and give extensive feedback; journals thus provide a powerful method of direct communication between students and faculty, an important element in a class that includes so much team-based work. The journals also enable instructors to track students’ progress throughout the semester, and to anticipate any issues that might arise within teams. The completed exhibits are installed in a public space at MIT for viewing by museum professionals, invited high school students, and the general public. INSERT FIGURE 2 HERE The spring-break field trip gives students the opportunity to see at first hand the problems they have been grappling with, to conduct additional research and collect artifacts for their exhibits, and to speak to scientists, officials, and ordinary citizens who deal with these problems on a day-to-day basis. Their immersion into the geographic area is designed to give them an understanding of the issues in a way that classroom experiences cannot and to help them to see ways in which their thinking up to that point (including their proposed solution from the Fall class) may have missed complexities that can be appreciated only through direct experience. Literature Review 7 Project-based learning is one of the newer learner-centered pedagogical approaches. It is gaining in popularity and widespread use because it endeavors to equip students with life-long learning skills (how to learn, how to problem-solve, how to apply what they have learned to new situations) as well as the life skills of teamwork and communication (Buck Institute for Education, 1999; Frank et al., 2003: Stinson and Miller, 1996). It is an outgrowth of the experiential, constructivist tradition that encourages students to be active rather than passive learners and to construct their own knowledge and understanding (Pascarella and Terenzini, 2005; Smith et al., 2005). This approach was first introduced into medical education in the late 60’s, and so much of the earlier research about project-based learning has come from medical schools (Barrows, 1996). During the 80’s, this approach spread to other professional schools, and since the 90’s its reach has extended to all types of school populations and disciplinary areas (Savin-Baden and Major, 2004). However, more has been written about its use in primary and secondary education than in higher education (Pascarella and Terenzini, 2005), and there is a lack of research about its effects on undergraduate science and engineering courses (Barak and Dori, 2004; Dym et al., 2005; Smith et al., 2005). The terms ―project-based learning‖ and ―problem-based learning‖ have been used interchangeably in the literature. We shall use ―project-based learning‖ here. Projectbased learning refers to an array of approaches that includes many variations but is based on a core of common characteristics (Barrows, 1996; Gijselaers, 1996; Major and Palmer, 2001; Saven-Baden, 2000; Shelton and Smith, 1998; Thomas, 2000). It takes on different forms depending on the specific curricular situation (Saven-Baden, 2000; Thomas, 2000). As the previously-cited authors have described, project-based learning is contextual 8 learning. Students are presented with an authentic, challenging, real-world problem that has no single or simple solution. Solving this problem is the central focus of the class. Students work together in small, cooperative groups to investigate the problem and formulate a solution that is then presented in a culminating experience. Boundaries between disciplines are crossed, as the problem often requires a multi-disciplinary solution. This curricular approach places new demands on instructors and students, since both groups play roles that differ sharply from their traditional ones. Instructors facilitate rather than lecture, and students are encouraged and given the tools to be self-directed learners. (See Table I.) INSERT TABLE I HERE Within undergraduate science and engineering classes in particular, there are many variations in the ways project-based learning is implemented, owing to differences among academic domains, learner populations, educational contexts, pedagogical decisions, availability of appropriate instructional resources, and the problems/projects themselves (Allen et al., 1996; Savin-Baden, 2000; Taconis et al., 2001). These variations have an impact on learning outcomes (Thomas, 2000). In addition, studies of projectbased learning differ in terms of research methods, data collection instruments, and outcome measures. A number of studies of science and engineering undergraduates who participate in project-based learning classes have focused on improvements in life skills and life-long learning skills (e.g. problem-solving, teamwork, communication, self-directed learning). 9 This is not a simple task, because the skills that project-based pedagogies seek to impart are difficult to measure with traditional measures of assessment such as grades and examinations (Dori, 2003; Major and Palmer, 2001; Reeves, 2000). Some studies present descriptive, anecdotal data as outcome measures (Ramsier, 2001; Duffield and Grabiner, 1997; Striegel and Rover, 2002). Others use alternative assessment measures (selfreports, observations, portfolios, learning journals, focus groups and interviews) as the major source of evidence for improvement (Bauer, 2003; Frank et al., 2003; Ram, 1999; Shelton and Smith, 1998; Wright and Boggs, 2002). These studies have generally not included an examination of longitudinal data that would reveal the longer-term impact of this pedagogy on students. In our ongoing study of the Terrascope program we are gathering and examining such data; in this study we present some of our preliminary results in that area, as well as more established results based on data gathered while students were engaged in the program and just after they had completed it. Methodology The Terrascope assessment focused on the following research questions: (1) How did students experience project-based teamwork? (2) How did they perceive the instructional environment? (3) What were some of the major benefits of the program? (4) What were the major challenges students faced? (5) How satisfied with the experience were students, and would they recommend it to incoming freshmen? (6) How well did the first- and second-semester classes complement one another? 10 Data Collection To answer these questions, we used two forms of inquiry—surveys and focus groups. We examined students who participated in the program in AY 2002-2003 at two points in time – as freshmen and as juniors. We also examined, when they were sophomores, students who had participated in Terrascope in AY 2003-2004. Freshmen We administered surveys to freshmen who participated during AY 2002-2003, as part of an ongoing effort that includes the administration of surveys to every class of Terrascopers. (Further results from this ongoing assessment effort are reported in Epstein et al., 2006a). One survey was administered at the end of the first semester, and another was administered at the end of the second semester. Some students in the fall class were not enrolled in the Terrascope program (unlike other elements of Terrascope, this class is open to any MIT freshman), so survey results about the first semester program include data from Terrascope and non-Terrascope students. Among the 54 students who took the fall class, 44 responded, yielding an 81 percent response rate. Among the 37 students in the spring class, 24 responded, giving a 64 percent response rate. Although the surveys covered some similar areas, they were designed for two distinct classes, so there were few overlapping questions. Two focus groups were conducted with the first group of Terrascope students during spring 2003; a total of 10 students attended. Alumni (Terrascope sophomores and juniors). During spring 2005, three focus groups and two individual interviews were held with Terrascope 2002-2003 juniors and 2003-2004 sophomores. Thirteen students participated—4 juniors and 9 sophomores. Given that such a small number of students 11 participated in these focus groups, the results cannot be generalized; it is probably the case that the more committed and/or vocal students participated. Freshman and alumni data are reported in separate sections. First, we report on the freshman data. Results First-Semester Experience Prior High-School Experiences Since students’ perceptions are often influenced by prior experiences, the fallsemester survey asked students about their prior experience working in teams and solving complex problems. Slightly over 60 percent of the students entered the class having had some prior high-school teamwork experience: some had participated in speech and debate teams and short-term high-school projects, and others had participated in athletics. Not surprisingly, far fewer (25 percent) had the experience of solving open-ended complex problems, and those who had this type of experience had generally worked on individual rather than group projects. A little over one-fifth had participated in classes that emphasized self-directed learning. Few (18 percent) had worked on team projects that required a long-term coordinated effort The Project-based Teamwork Experience Teams made their own decisions about team structure and the ways members would work together. Half of the teams had rotating leaders; 16 percent had one leader during the entire semester; and approximately one-third never had a leader. Students occupied a variety of structured or semi-structured team roles during the semester: the 12 secretary who took meeting notes; the liaison who communicated with mentors and other teams; and the website specialist. Sixteen percent said their team had a prepared agenda; 27 percent said their agenda was set during the meeting itself; and 50 percent said it varied – sometimes there was a prepared agenda, while at other times there was no agenda. Students were asked to rate the effectiveness of their team’s structure. Thirtynine percent thought their team’s structure was ―very effective‖ or ―effective;‖ 52 percent said it was ―somewhat effective‖ and 9 percent said it was ―only slightly effective.‖ Each team had its own unique personality and challenges; some were more focused, while others floundered. Some had to deal with conflicts among members because everyone wanted to be a leader and each member had different ideas about how to proceed. Others had to determine how to handle team members who were slacking off and avoiding their responsibilities. Separate questions on the same survey asked students to rate their teamwork experiences at two points in time, during the first and last months of class. As Table II shows, there was statistically significant improvement for ―clear understanding of team goals,‖ ―efficiency of meetings,‖ ―ability of teams to reach consensus on decisions,‖ ―inter-team coordination,‖ and ―student preparation for meetings.‖ Student ratings of their overall team performance showed some improvement, but this was not statistically significant. Equal sharing of workload and motivation decreased slightly over time. INSERT TABLE II HERE 13 Fifty percent of students were ―satisfied‖ or ―very satisfied‖ with their teamwork experience; 27 percent were ―ambivalent;‖ and 23 percent were either ―dissatisfied‖ or ―very dissatisfied.‖ In their open-ended remarks, those who were satisfied commented that they learned a lot about how to work in a group as well as how to deal with a variety of different personalities, and their team worked well together. Those who were ―ambivalent‖ or ―dissatisfied‖ mentioned such things as the unequal workload distribution, the difficulties involved in working together as a team, and the challenges they faced in trying to coordinate with other teams. However, teamwork problems apparently did not diminish the students’ appreciation of the experience’s educational value. In spite of the fact that only half the students were satisfied with their teamwork experience, 85 percent agreed that working in a group had a beneficial effect on their learning; 89 percent acknowledged that individual team members brought valuable qualities, skills, or abilities to their team; and 75 percent enjoyed working with members of their team. The Instructional Environment The academic support structure consisted of undergraduate teaching fellows (UTFs), alumni/ae mentors, two teaching assistants, the instructor, and library liaisons. We were interested in examining whether it was important for a team to have a UTF with special expertise in the team’s topic, so students were asked whether their UTF had some familiarity with their team’s topic and, if so, whether this was useful. Most (93 percent) said their UTF had some familiarity with their team’s topic. Among this group, only 16 percent said it was either ―useful‖ or ―very useful;‖ 18 percent said it was ―somewhat useful;‖ and 66 percent said it was ―only slightly useful‖. 14 Students were also asked to rate the usefulness of various types of assistance their teams received from UTFs. The two types of assistance students found most useful were: ―gave constructive feedback‖ and ―encouraged the team to set deadlines.‖ (See Figure 3.) INSERT FIGURE 3 HERE Mentors were also part of the support structure, and the role they played varied by team. Early in the semester, one or more mentors were assigned to each team, and a student/mentor get-together was held to facilitate face-to-face meetings; however, some mentors were not local, and thus a number of students were unable to meet their mentors personally. Although mentor contact was not mandatory, students were encouraged to initiate contact and use mentors as sounding boards. Eighty-six percent of responding students had some contact with their mentors during the semester. Among those who had contact, 75 percent had email contact; 50 percent had one or more face-to-face meetings; 16 percent had contact via an electronic discussion program; and 2 percent had telephone contact. Among the types of assistance students received from mentors, those most frequently mentioned included, ―asked questions that stimulated the team to investigate other areas,‖ mentioned by 85 percent; ―answered questions,‖ mentioned by 83 percent; and ―referred team to source materials on the web or in the library,‖ mentioned by 80 percent. Students were asked about amount of direction, guidance, or coaching they received from the various instructional resources available to the class. As Table III 15 shows, students were most satisfied with the amount of direction they received from their UTFs. On the other hand, approximately one-half to two-thirds would have liked more guidance from each of the other sources. INSERT TABLE III HERE As noted earlier, few students had previous experience with self-regulated learning or with solving complex problems in a group structure, so it is not surprising that many wished they could have had more structure, especially in light of the fact that they were new to MIT and to the demands of the college environment. The Benefits of Taking the Fall-Semester class Unlike the fall-semester Terrascope class, most of the required freshman-year classes had a traditional format of a large lecture, a smaller recitation section, and weekly problem-set homework. Oftentimes, freshmen complained that they were not engaged and stimulated by these subjects. When asked about their level of engagement in the Terrascope class, 89 percent said they were engaged and stimulated by the overall topic, and 86 percent said they were engaged and stimulated by the topics on which they did research. In regard to their skill improvement as a result of having taken this subject, the largest gains were reported for ability to break complex problems into smaller units and ability to work collaboratively with a diverse group of students. The smallest gains were reported for research skills. (See Figure 4.) 16 INSERT FIGURE 4 HERE We were interested to learn how students compared this class to their other fallterm subjects. Eighty percent or more rated the Terrascope subject higher than their other subjects in such aspects as: encouraging them to be creative, giving them an opportunity for independence and autonomy, improving their ability to work productively in a group, and encouraging them to feel as if they had accomplished something because they could see a final product. In addition, between 60 and 70 percent thought they had accomplished something by developing a final product; their confidence in their ability to handle a complex and ambiguous task had improved; they were exposed to MIT resources they would not have been exposed to otherwise in the fall term of their freshman year; and they were actively involved in what they were learning. (See Table IV.) INSERT TABLE IV HERE Students who responded positively in their open-ended comments mentioned that their confidence in their ability to successfully confront any problem had increased; they were exposed to students majoring in different areas; they had the experience of being able to apply classroom learning to ―real world‖ problems; they learned teamwork skills that will play an important role in their future endeavors; and they learned how to direct their own learning. Those responding negatively mentioned the lack of class structure; 17 the heavy workload; the lack of motivation among some of the students; and problems dealing with students who wanted to take charge of the entire process. Endorsement One of the last questions in the survey asked, ―Knowing what you know now about 12.000, would you recommend this subject to incoming first year students?‖ Sixtyfour percent said they ―definitely would‖ or ―probably would;‖ 25 percent said ―maybe;‖ and 12 percent said ―definitely not‖ or ―probably not.‖ The following quote represents a good summary of the reasons why students would recommend 12.000 to incoming freshmen: I think that (12.000) was an amazing experience. Despite the stress that was involved at the end of the semester, I felt an incredible sense of accomplishment by the end. However, the main reason why I would recommend this to future freshmen is because it gives you a chance to do something that few other freshmen get to do. I believe that the experiences I took from the class can play a vital role for all my future teamwork endeavors. Also, the class gave me a refreshing break from mainstream learning techniques. The class really provides a good supplement to the day-to-day class/lecture/problem-set mode. The thoughts of students who would not recommend the class can be summed up by this quote: I felt the class could be better organized. A lot of things could be improved by this. A lot of time is wasted in the first two weeks just figuring out what to do, and then an important amount of work is done during the last two weeks before the presentation. I think that what the teams need to reach their goal should be traced out more clearly. Second-Semester Experience and Field Trip Project-Based Teamwork Experience As in the first-semester survey, separate questions in the second-semester survey asked students to rate their team’s performance during two time periods: the first and last 18 month of the semester. As Table V shows, the largest improvements in performance related to the procedures of working together as a team (running efficient meetings and being able to reach consensus on decisions). Students thought their overall team’s performance and individual team characteristics significantly improved from the first to the last month in every area except ―equal sharing of workload.‖ In that case, there was a perceived decrease in equal workload participation among team members. INSERT TABLE V HERE When asked ―Which were the biggest problems you encountered in working together as a team?‖ the three most frequently mentioned problems were: communication, unequal sharing of workload, and conflicting schedules. The Instructional Environment The spring semester focused on the development of an interactive museum exhibit that would be opened to the public at the end of the semester. The survey asked students about their satisfaction with various aspects of their exhibit-development activities, such as developing and testing an exhibit prototype (which occurred mid-semester), building the final exhibits (which occupied the last half of the semester) and opening the exhibits to the general public. Table VI shows some representative results. In general, students’ satisfaction with the class increased markedly towards the end of the semester, as they began to see their earlier work bearing fruit. INSERT TABLE VI HERE 19 Students felt that the class, as a whole, significantly increased their skills: they learned how to convey information to people who do not have a strong technical or scientific background; they acquired a new perspective about the complexity of creating museum exhibits; and they gained an appreciation for the challenges involved in participating in a large-scale design/construction effort—creating an appropriate design, acquiring the right materials, working within a budget, and devising and adhering to a strict timeline. Although class time was largely spent working on projects in individual teams, students also participated in some conventional academic activities, and we asked them to indicate their satisfaction with these activities. As can be seen in Figure 5, roughly 80 percent or more were satisfied with the Developer’s Journals, instructors’ proposal feedback, and the guidance received from undergraduate teaching fellows, instructors, and teaching assistants. There was less satisfaction with the instructors’ journal feedback and with the lectures. INSERT FIGURE 5 HERE Benefits of Taking the Spring Class It should be noted that by the end of the spring semester many students strongly identify the term ―Terrascope‖ with the spring-semester subject, and such identification 20 may color their survey responses. Therefore, it is not always possible to differentiate between students’ opinions of the subject itself and their opinions of Terrascope as a whole. In constructing the surveys, we worked to minimize this confusion, for example by asking students to separately assess gains from the class itself and from Terrascope as a whole. The survey presented students with a series of statements about the learning objectives of the class and asked students to indicate whether, from their personal point of view, these objectives had been met. The learning objectives fell into three main areas: process and content learning, skill improvement, and personal development. As Table VII shows, over 80 percent reported gains in understanding design and engineering processes, learning from the hands-on experience, being part of an intensive design experience, teamwork skills, and feeling that their increased environmental knowledge will likely influence them in the future. Slightly over 70 percent reported improvement in problemsolving skills and in learning to develop their own creativity. Less than 50 percent cited improved oral communication skills, and written communication skills were the least frequently mentioned. INSERT TABLE VII HERE Students were also asked whether the class had any influence on the way they think about themselves, their major, their academic choices, or their future career. Sixtysix percent said ―yes.‖ Some students mentioned that the class confirmed their interest in 21 majoring in a related discipline and pursuing a career focused on environmental issues. Others said it increased their awareness of the environmental impact of political decisions. Still others noted that the program had made them think more carefully about the role they play in groups, and it gave them more confidence in their leadership abilities. The Field Trip A major highlight of the Terrascope experience is the field trip. Although not specifically linked to the spring-semester class, it takes place in the middle of the semester and thus has a major impact on the students’ experience in the spring class. In 2003, students went to the Brazilian Amazon (the year’s core problem involved devising methods for monitoring and reversing degradation of the Amazon rainforest). Satisfaction with the field trip experience was extremely high: 83 percent were ―very satisfied,‖ and 17 percent were ―generally satisfied.‖ Ninety-six percent were able to apply some of the insights/information they gained on the trip to their exhibit. As one student commented, ―The trip really helped my group to find out what research stations in the Amazon are really like. Staying at the research camp and interviewing scientists made this possible.‖ Another said, ―We interviewed local citizens about music, school, and life in general. This information coupled with photos from the trip was instrumental in our exhibit.‖ In addition to the educational impact of the trip, there was a large social impact. Students emphasized the important role the trip made in strengthening connections within the community (students, UTFs, faculty and staff). Learning Community 22 We were interested in the extent to which a learning community developed over the course of the year. We defined a community as a group of people whom one knew and liked, and with whom one felt one shared a common bond. Approximately onequarter of students said Terrascope felt like a community during the first semester, whereas everyone said it felt like a community during the second semester. Although being together for a second semester with students who share common interests helped to develop a community feeling, students cited the field trip as the major contributing factor. The value students placed on the community aspect of their experience was also highlighted when, in response to the question of whether the Spring class should be open to any students who would like to enroll (rather than just Terrascope students) 92 percent said ―no.‖ They thought Terrascope, in general, created a cohesive community and opening that class to others might remove this important aspect of the experience. Endorsement Overall, students were very positive about Terrascope. When asked ―Knowing what you know now about Terrascope, would you recommend it to incoming first year students who share your interests?‖ 67 percent said ―definitely would,‖ 29 percent said ―probably would,‖ 4 percent said ―maybe.‖ The following are some representative quotations: Terrascope has been one of the best things this year.I feel we have come a long way. A great way to start MIT. I would recommend anyone motivated for more original work and (interested) in the environment to join. It‟s fun. It‟s a nice break from regular institute classes, and you‟ll learn a lot and get to work with awesome professors. It‟s very literally a once-in-a-lifetime opportunity, especially for a freshman. The students who were ambivalent about the program mentioned one particular theme – the workload and time commitment. For example, one student said, “It was an amazing 23 experience, but it took a lot of time away from my other classes that I have more intellectual interest in.” Whether Fall and Spring Classes Complement Each Other The majority of the students thought the teamwork, problem-solving, and projectmanagement skills that they had learned in the Fall class could be applied to the Spring class and were enhanced by taking that class. Most also said the knowledge acquired in the fall could be applied in the spring. Fewer thought the specific subject knowledge acquired in the fall was enhanced by taking the spring class, probably because individual students’ second semester research topics differed from their first semester exhibit topics. And fewer than half indicated that the research skills they had acquired first semester were enhanced by taking the spring class, a result of the fact that the spring class focused less on research than the fall class did. A majority of students also said that the coordination between the two subjects was ―all right as it is‖ (vs. ―should be more closely coordinated‖) and that a combination of a less structured first semester class and a more structured second semester class was useful. (See Table VIII.) INSERT TABLE VIII HERE In the freshman focus groups conducted in spring 2003, students commented on the impact of a less structured first-semester experience and a more structured second- 24 semester experience. They explained that the trial-and-error experience of first semester and the mistakes they had made gave them an appreciation for the importance of good communication within and between teams and for the value of deadlines. About deadlines, one student explained it this way: Coming into 1.016, we realized why we needed deadlines, and we felt good about the deadlines, as opposed to coming into the class and having instructors say, „This is due now,‟ and you feel „Why is this due now?‟ And, another, commenting on the looser structure of first semester said, So, in retrospect, I don‟t think we liked the free form (first semester class), but it was a valuable learning experience for second semester. Alumni Reflections: Focus Groups with Terrascope ’03-’04 sophomores and ’02-’03 juniors We conducted focus groups to document the views of upperclass students about their freshman year Terrascope experience. We wanted students to reflect on the influence this program may have had on their growth a year or more after the experience ended. Benefits Students highlighted a number of benefits of participation. They thought Terrascope provided a good transition to MIT. Through Terrascope, they got to know other freshmen, faculty, and upperclass students, and during the course of conducting their research and working on their exhibits, they became acquainted with other MIT faculty and MIT alumni. Terrascope, they said, helped reduce some of the stress they felt as freshmen. As one student explained, ―Freshman year is terrifying to most people. Terrascope makes it so much better because you’re doing something different and 25 instructors are there to help you.‖ Not only did Terrascope offer a unique freshman academic experience that differed from the traditional lecture/recitation/problem-set format, but it also provided a more exciting academic experience, which encouraged students to be creative: You come to MIT with all these great ideas about what you‟re going to do and you get totally bogged down with the problem-set routine.... And (Terrascope) definitely improved how I felt about my academic freshman year because it was challenging in an intellectual sense ... you have to think creatively as opposed to dealing with numbers and problems. Students emphasized that a learning community developed during the Terrascope experience. The structure of the classes required them to work in small teams and enabled them to get to know other freshmen as well as the upperclassmen who were UTFs. As one student explained, ―The fact that you had all these people doing the same thing you were doing and you had to get along with them created this awesome community where there were friends and people to support you.‖ The field trip was mentioned by all as one of the most worthwhile educational experiences because they were able to see the actual effect of the processes they were studying on the local people and on the environment. They said the sense of community came into its own as a real entity during the trip. In addition, students said Terrascope provided a more intimate academic experience than their other classes, where many felt as if they were just a number. A number of skills were developed during this experience. Those most frequently mentioned were teamwork, communication and research skills. Some students noted a carryover to their sophomore- and junior-year classes, since they were more comfortable working in groups and more apt to take group leadership roles both in school and in 26 outside activities. One student summed it up by saying, ―It gave me a lot more confidence about how to approach a team project and gauge other people’s skills. And it made me a lot more willing to take on a larger role in class and get involved with activities on campus.‖ Students said they currently felt more comfortable about their research skills, and this has helped them when they have taken other classes requiring research. As one student described, ―(Without this type of introduction) I would have been lost going to libraries and using the web pages. I’ve used these skills so much in my other classes, and the way they taught us was helpful.‖ Students mentioned other skills such as webpage design, oral presentation, construction, and problem-solving skills. However, they found it difficult to articulate what they had learned about problemsolving, saying such things as ―you’re so focused on what you’re doing that you don’t think about how you’re doing it.‖ Students felt the Terrascope experience was empowering. One student reflected, ―You feel like you can handle a lot more.‖ Students particularly valued the fact that they were responsible for their own learning, something that never happened before for many of them, since they had always been told what to study and how to study: ―The onus was on us to organize it any way we wanted … and it was great to be a part of that.‖ Many said Terrascope influenced their subsequent academic careers. The Terrascope lunches where faculty presented their ―cutting-edge‖ research gave them an opportunity to learn about current research being carried on by faculty and enabled them to follow up on the research that interested them. Some said these led to opportunities to participate in undergraduate research and provided topics for their undergraduate theses. Others said the lunches as well as the entire Terrascope experience influenced their choice of major 27 and career interests. For example, a student who planned to major in mechanical engineering said he had decided to focus his career on the development of products for developing countries as a result of the field trip. Challenges Students commented that learning how to work effectively together as a team was a challenging experience. By the end of each semester, most felt their teams had learned to work successfully together, but some felt their teams still had a long way to go before their performance could be judged successful. In addition, time management was also a key issue. Since Terrascope classes currently do not satisfy any of the General Institute Requirements (core classes required of all undergraduates), students often had to weigh their priorities and decide whether to focus their efforts on a problem set, an upcoming exam, or an assigned task for one of their Terrascope classes. In addition, although exhibits were a focal point of second semester and provided students with a sense of accomplishment, some respondents were less enthusiastic about the exhibit focus. These students had joined Terrascope because of their interest in solving environmental problems. Even though they were able to see the connection between the ability to communicate about environmental issues and the ability to solve environmental problems, it was not as satisfying to these students as the first semester focus. Discussion The Terrascope program consists of so many elements that it is not possible to discuss all the key results, so in this section we will select a few of the most salient ones. Prior Learning Experiences and Transition to MIT 28 Freshmen joining Terrascope had little experience with project-based teamwork, solving complex problems, and self-regulatory learning. Most came from traditionally structured high-school environments where students had little autonomy and carried out specific assignments with definite goals. If they worked together in groups with an academic focus, these were generally short-lived, informal working groups, and, if they had experience with complex problems, they usually worked on them individually. Although adjusting to the MIT environment and the innovative setting of the Terrascope community presented huge challenges, students said their participation provided an excellent transition to MIT because they were able to interact with other freshmen, upperclass students, and instructors in a small-group educational setting, unlike the large settings of their other classes. They felt this helped reduce the stress that freshmen normally experience when they enter the MIT environment. Project-based Teamwork During first semester, students noted improvement in a number of teamworkrelated areas: they ran more efficient meetings, were better prepared for meetings, and were able to coordinate with other teams. Nevertheless, overall team performance did not show statistically significant improvement. During second semester, however, students noted statistically significant improvement in overall team performance from the first to the last month of class. Although teamwork was not specifically ―taught‖ to freshmen during first semester, the trial and error experiences of first semester probably contributed to improvement in students’ teamwork skills. The Instructional Environment 29 During the first semester, the majority of students thought they did not receive enough direction, guidance, or coaching from their instructor, the teaching assistants, and mentors and other experts. That is to be expected, since the class is designed to challenge students to take responsibility for their own learning in a way they have not experienced before. Indeed, in spite of the fact that students evidenced a longing for more structure during the first semester, by the end of the year most said a combination of a lessstructured first semester experience and a more-structured second semester experience had been helpful. Their comments revealed that the fairly flexible first-semester structure helped them learn to rely on themselves to provide structure rather than rely on the instructional staff. A number of students said they learned to appreciate the importance of deadlines and structure because of the relatively loose structure of first semester. During second semester, there was markedly more satisfaction with guidance from instructors and teaching assistants, probably due, in part, to the tighter structure, but also partly due to students’ increased experience and confidence with the project-based teamwork environment. There was also satisfaction with the Developer’s Journals that each student submitted on a weekly basis. Not only did this give students an opportunity to reflect on their experiences, it also enabled instructors to monitor students’ learning experiences and team dynamics, and to communicate directly with students, in ways that would otherwise not have been possible in a class where so much of the work is done in group meetings or work sessions outside of class time. Benefits of Terrascope The first-semester data showed that students were engaged with the topic and appreciated the advantages the fall class provided compared to their other, more 30 traditional first-semester subjects. Both first- and second-semester data indicated that students thought these subjects encouraged creativity, gave them autonomy, improved their group skills, encouraged them to use their own initiative, and gave them selfconfidence in their abilities to tackle difficult subjects. The Terrascope experience also helped some students to clarify their thoughts about their majors and future careers. Terrascope provided students with a close-knit learning community. Students developed strong relationships with other freshmen (generally this happens primarily through living groups), upperclass students (again, this usually happens primarily in living groups), and instructors (this generally does not happen in the freshman year). This type of experience gave students a strong support system that they could call upon in times of stress, both as freshmen and as upperclass students. It also provided a number of important educational and experiential benefits, since it gave them an understanding of how what they learned in the classroom could be put to use in the ―real world.‖ The field trip is a unique experience that, of course, cannot be replicated in many other project-based learning classes. Still, many elements of the experience – formal and informal working groups, hands-on experiences, close collaboration with upperclass students and instructors – can be replicated. And many of the community-building benefits of the field trip could be provided, at least in part, by smaller-scale outings and activities, particularly if they involved overnight stays off-campus and participation by all or nearly all members of the program. The fact that the experience carried over into a second semester and did not end abruptly after one semester seems to have been important, since students had time to improve their teamwork skills and build a more solid community. Indeed, students said a 31 number of skills they learned first semester were enhanced second semester and could be applied to their second-semester work. Also, alumni focus groups revealed that their Terrascope experiences (the combination of first- and second-semester research and exhibit-development experiences; discussions with faculty, mentors, and upperclass students) gave them time to explore their interests and often led to undergraduate research opportunities and senior thesis topics. Lessons Learned Terrascope classes give students a great deal of autonomy and control over the direction and structure of their work. That provides them with a strong sense of empowerment and an equally strong sense of responsibility, not only to the classes but to the program as a whole. Students feel a powerful need to help shape the program based on their own experiences. In keeping with the overall philosophy of the program, Terrascope faculty and staff encourage, value and make use of students’ input, both formal and informal. A number of changes have been implemented in response to students’ feedback, and a number of changes that were under consideration have been abandoned, again due to students’ feedback. For example, during the early years of the program, students expressed occasional frustration over the amount of time it took instructors to read and respond to Developer’s Journals. For instructors, reading and commenting on journals is one of the most timeintensive and challenging aspects of the class; carefully reading and commenting on one week’s journals can easily require a full day or more of concentrated work. Nevertheless, students’ comments made it clear that if journal feedback was not received very soon after the journals were submitted, it was far less useful to them. As a result, we have 32 structured journal due dates, and our own schedules, so that we can provide feedback as rapidly as possible. Assessment data from the past few years (not included in this study) show that this has greatly increased students’ sense of the journals’ usefulness and relevance. Another change, and one that has come in direct response to students’ feedback on surveys and in focus groups, has to do with the exhibits produced in the spring class. Students in the early Terrascope years sometimes felt as if a semester’s work vanished too quickly and with too little impact on people outside the program. As a result, the program has found public exhibit space—an area with substantial foot traffic—and has committed to keeping the exhibits open for a longer period. Terrascope staff have also worked with museums and aquariums, both locally and nationwide, to find productive uses for the students’ work after the official exhibits have closed. In the most recent Terrascope year (2005-2006), the great majority of exhibit elements were accessioned by museums, for use either as prototypes or in guided exhibit experiences. Students have responded well to the increased exposure their exhibits receive, and it is clear that the public nature of the final exhibit is a major driving force in their work. One of the most interesting trends in Terrascope has to do with the role played by UTFs. Over the four years of the program’s existence, the UTF’s role has evolved considerably. Originally seen as facilitators and aides for individual teams, UTFs are now engaged generally in the progress of the Terrascope classes; major class-wide issues (e.g. the students’ level of engagement, overall team dynamics) are discussed at UTF meetings, and many of the solutions implemented come directly from UTFs’ suggestions. UTFs have also been given increased responsibility for logistical aspects of the spring 33 field trip, particularly in day-to-day organizational issues. The UTFs clearly see themselves as having a stake in the program, and their participation is indeed a key to its success. The instructional staff has also responded to students’ concerns about the Terrascope workload, by increasing the amount of credit given to students for the spring class. (It is interesting, however, that students identify workload as an issue, since nearly all the work they do is self-assigned and self-motivated. Students generally work much harder on their projects than is required, and they seem to be motivated more strongly by the responsibility they are given than by grades alone.) The program has also introduced an optional Terrascope class called Terrascope Radio, which gives students the opportunity to use some of the subject knowledge they have gained to satisfy MIT’s Communication Intensive requirement. These changes and others, and the ways in which they were made, lead us to offer the following notes to other institutions hoping to develop similar programs: The public nature of the students’ final presentation, whether in the fall public presentation or the spring exhibits, is key to engaging the students in the work, and it also provides much deeper satisfaction with the final product than any faculty member’s assessment could. The instructional staff has found it best to err on the side of giving students too much, rather than too little, responsibility for the nature and structure of the work they do, and of the final product. Students generally rise to the challenge, producing work that is better and more comprehensive than they had originally thought possible. 34 It has been extremely useful to the program to engage upperclass students in the community, and to involve them in teaching and advising. End-of-semester and end-of-year surveys, and other methods of collecting students’ feedback and suggestions, are key in developing and maintaining a program that relies so much on students’ motivation and interest. It is extremely important for such a program to be flexible and to be able to respond rapidly to students’ needs and wants. Acknowledgments We wish to acknowledge the contributions of Debra Aczel and Ruth Weinrib, who handle the logistics of the Terrascope program and who continually work to find ways to make the students happier, more comfortable and more fulfilled. We are also grateful for the assistance of Stephen Rudolph of the MIT Department of Civil and Environmental Engineering, who works closely with students during the spring semester, talking them through their designs and then helping bring their projects into being. We also thank Maria Shkolnik for administrative assistance. Terrascope has been generously supported through the offices of the MIT Chancellor and Provost, and much of the program’s development was made possible by MIT’s Alex and Brit D’Arbeloff Fund for Excellence in Education. The Terrascope field trip is supported by the Henry Luce Foundation. References 35 Allen, D.E., Duch, B.J., & Groh, S.E. (1996). The power of problem-based learning in teaching introductory science courses. In L. Wilkerson & W.H. Gijselaers (Eds.), Bringing problem-based learning to higher education theory and practice. New Directions for Teaching and Learning, 68, 43-52. San Francisco, CA: Jossey Bass. Barak, M., & Dori, Y.J. (2004). Enhancing undergraduate students’ chemistry understanding through project-based learning in an IT environment. Science Education, 89, 117-139. Barrows, H. (1996). Problem-based learning in medicine and beyond: A brief overview. In L. Wilkerson & W.H. Gijselaers (Eds.), Bringing problem-based learning to higher education theory and practice. New Directions for Teaching and Learning, 68, 3-12. San Francisco, CA: Jossey Bass. Bauer, G. (2003). Summary report: Qualitative assessment activities for Pew grant on problem-based learning (PBL). Available at: http://www.udel.edu/pbl/Final-Report-PewPBL.pdf/ (May 2006). Buck Institute for Education Project-Based Handbook. (1999). Available at: http://www.bie.org/pbl/pblhandbook/intro.php#benefits (April 2006). Dori, Y. (2003). A framework for project-based assessment in science education. In M. Segers, F. Dochy, & E. Cascallar (Eds.), Optimising new modes of assessment: In search of qualities and standards, 89-118. Dordrecht, The Netherlands: Kluwer. Duffield, J., & Grabinger, R.S. (1997). Problem-based learning at the University of Colorado at Denver, ERIC, 409834, 31-34. Dym, C.L., Agogino, A.M., Ozgur, E., Frey, D., & Leifer, L.J. (2005). Engineering design thinking, teaching, and learning. Journal of Engineering Education, 94 (1), 103120. Epstein, A.W., Lipson, A., Bras., R. & Hodges, K. (2006a). Terrascope: A project-based, team-oriented freshman learning community with an environmental/Earth system focus. Proceedings of the American Society for Engineering Education Annual Conference, June 2006, paper 2006-435. American Society for Engineering Education, Washington, D.C. Available on line at: http://asee.org/acPapers/code/getPaper.cfm?paperID=10456 Epstein, A.W., Bras, R., Hodges, K., & Lipson, A., (2006b). Team-oriented, projectbased learning as a path to undergraduate research. In K.K. Karukstis and T. Elgren (Eds.), Designing, Implementing, and Sustaining a Research-Supportive Undergraduate Curriculum: A Compendium of Successful Curricular Practices from Faculty and Institutions Engaged in Undergraduate Research, Council on Undergraduate Research, Washington, D.C. (in press). 36 Frank, M., Lavy, I., & Elata, D. (2003). Implementing the project-based learning approach in an academic engineering course. International Journal of Technology and Design Education, 13, 273-288. Gijselaers, W.H. (1996). Connecting problem-based Practices with educational theory. In L. Wilkerson & W.H. Gijselaers (Eds.), Bringing problem-based learning to higher education theory and practice. New Directions for Teaching and Learning, 68, 13-22. San Francisco, CA: Jossey Bass. Major, C. H. & Palmer, B. (2001). Assessing the effectiveness of problem-based learning in higher education: Lessons from the literature. Spring, 4-9. Pascarella, E.T. & Terenzini, P. (2005). How college affects students. V. 2, San Francisco, CA: John Wiley. Ram, P. (1999). Problem-based learning in undergraduate education. Journal of Chemical Education, August, 1122-1126. Ramsier, R.D. (2001). A hybrid approach to active learning. Physics Education, 36 (2), 124-128. Reeves, T. C. (2000). Alternative assessment approaches for online learning environments in higher education. Journal of Educational Computing Research, 23 (1), 101-111. Savin-Baden, M. (2000). Problem-based learning in higher education: Untold stories. Buckingham England: The Society for Research into Higher Education and Open University Press. Savin-Baden, M. & Major, C.H. (2004). Foundations of problem-based learning. England: Society for Research into Higher Education and Open-University Press. Shelton, J.B. & Smith, R.F. (1998). Problem-based learning in analytical science undergraduate teaching. Research in Science and Technology Education, 16(1), 19-28. Smith, K., Sheppard, S., Johnson, D. & Johnson, R.T. (2005). Pedagogies of engagement: Classroom-based strategies. 94 (1), 87-101. Stinson, J.E. & Miller, R.G. (1996). Problem-based learning in business education: Curriculum design and implementation issues. In L. Wilkerson & W.H. Gijselaers (Eds.), Bringing problem-based learning to higher education theory and practice. New Directions for Teaching and Learning, 68, 33-42. San Francisco, CA: Jossey Bass. Striefel, A. & Rover, D.T. (2002). Problem-based learning in an introductory computer engineering course. 32nd ASEE/IEEE Frontiers in Education Conference. November 6-9, Boston, MA, FIG-7 to FIG-12. 37 Taconis, R., Ferguson-Hessler, M.G.M. & Broekkamp, H. (2001). Teaching science problem-solving: An overview of experimental work. Journal of Research in Science Teaching, 38 (4), 442-468. Thomas, J.W. (2000). A review of research on problem-based learning. Available at: http://www.bobpearlman.org/Bestpractices/PBL_Research.pdf or www.edutopia.org/php/resources.php?id=Item_217264. (April, 2006). Wright, R. and Boggs, J. (2002). Learning cell biology as a team: A project-based approach to upper division biology. Cell Biology Education, 1, 145-153. 38 Figure Captions 1. A Terrascope team brainstorming ideas. 2. Terrascope students constructing an exhibit during the spring-semester class. 3. Percentage of respondents indicating that certain kinds of assistance from UTFs were ―Very Useful‖ or ―Useful.‖ 4. Percentage of respondents indicating that their skills improved ―A Great Amount‖ or ―A Fair Amount‖ in certain areas. 5. Percentage of respondents indicating that they were ―Very Satisfied‖ or ―Generally Satisfied‖ with certain teaching and learning activities. 39 Figure 1 [FOR REFERENCE ONLY—JPEG FILE PROVIDED] 40 Figure 2 [FOR REFERENCE ONLY—JPEG FILE PROVIDED] 41 Figure 3 72% Encouraged team to set deadlines Gave constructive feedback 70% Discussed team functioning 63% Showed an interest in the research topic 63% 61% Encouraged and supported team members Asked questions that stimulated discussion 61% Suggested appropriate resources 56% 0% 20% 40% 60% 80% 100% 42 Figure 4 Ability to divide unstructured problem into manageable components 73% Ability to collaborate with students of diverse backgrounds, cultures, and levels of expertise 63% Ability to identify tasks needed to solve unstructured problem 55% Ability to communicate effectively with teammates 54% Ability to find scholarly reference sources at library or via library website 45% Ability to develop a systematic research plan 35% 0% 20% 40% 60% 80% 100% 43 Figure 5 Guidance from instructors 96% Guidance from undergraduate teaching fellows 91% Guidance from teaching assistants 88% Instructors’ proposal feedback 88% Developer's journals 79% Museum visits 71% Instructors’ journal feedback 63% 58% Lectures about museum exhibit design 54% Lectures about scientific issues 0% 20% 40% 60% 80% 100% 44 Table I. Major Differences Between Traditional and Project-Based Instruction Traditional Instruction Project-Based Instruction Curriculum focus Fixed or standard curriculum Learning focus Acquisition of knowledge Project/driving question Life skills (problem-solving, teamwork, communication) as well as knowledge Curriculum materials Textbook, assigned readings Developed by students through their own discovery process Role of student Passive observer; teacherdirected learner Active problem-solver; selfdirected learner Role of instructor Expert; source of knowledge Facilitator Social context of classroom Individual learners Community of learners who collaborate with one another Scope of subject Narrow, focused (usually) Broad, interdisciplinary 45 Table II. Mean Ratings of Team Characteristics: First Month vs. Last Month First Month Last Month Standard Deviation t ratio Sig (2tailed) Overall team performance 2.63 2.90 .95 1.91 n.s. Clear understanding of team goals 1.80 3.00 .96 7.98 <.001 Student preparation for meetings 2.48 2.89 1.13 2.41 <.05 Efficiency of meetings 1.91 2.82 1.05 5.73 <.001 2.59 3.07 .85 3.73 <.01 Equal sharing of workload 2.52 2.30 80 1.54 n.s. Motivation 3.05 2.89 1.12 0.94 n.s. Supportiveness 3.09 1.90 3.02 2.43 1.02 0.98 0.44 n.s. 3.56 <.01 (4-point scale: 1=poor; 4= excellent) Ability to reach consensus on decisions Inter-team coordination 46 Table III. Direction, Guidance, or Coaching Received from Various Sources Too Much Just Right Too Little Undergraduate Teaching Fellows 5% 57% 39% Instructor 5% 42% 54% Teaching Assistants 39% 46% Mentors or other experts 15% __ 37% 63% Other Teams 9% 27% 65% Library Staff Liaison __ 32% 68% 47 Table IV. Benefits of Fall Terrascope Class Relative to Other Fall Subjects Scale: 1= much less than most subjects; 5 = much more than most subjects ―Much More than Most‖ or ―More than Most‖ ―About the Same‖ ―Less than Most‖ or ―Much Less than Most‖ Encouraged me to be creative 91% 2% 7% 4.02 0.87 Gave me an opportunity to have some independence and autonomy 88% 7% 5% 4.23 0.87 Improved my ability to work productively in a group Encouraged me to use my own initiative 81% 79% 9% 12% 9% 9% 3.98 4.05 1.01 1.07 70% 14% 16% 3.81 1.22 65% 14% 21% 3.56 1.16 Exposed me to MIT resources (faculty, alumni mentors, heads of laboratories, etc.) 65% 35% -- 3.88 0.76 Actively involved me in what I was learning 61% 26% 14% 3.72 1.26 Made me feel as if I had accomplished something because there was a final product Increased my confidence in my ability to handle a complex and ambiguous task Mean SD 5-point scale 48 Table V. Mean Ratings of Team Characteristics: First Month vs. Last Month (4-point scale: 1=poor; 4= excellent) First Month Last Month Standard Deviation t ratio Sig (2tailed) Overall team performance 2.62 3.35 .92 3.87 <.001 Supportiveness 3.00 3.50 1.02 2.40 <.05 Motivation 2.75 3.52 1.38 2.72 <.05 Equal sharing of workload 2.67 2.46 1.10 0.93 n.s. Ability to reach consensus on decisions 2.50 3.46 1.20 3.92 <.001 Efficiency of meetings 2.42 3.63 .98 6.06 <.001 49 Table VI. Satisfaction with Exhibit Development Activities ―Very Satisfied‖ or ―Generally Satisfied‖ Prototype Development Designing 46% Building 46% Displaying 54% ―Ambivalent‖ ―Dissatisfied‖ or ―Very Dissatisfied‖ 25% 29% 29% 29% 25% 17% 3.29 3.38 3.50 1.16 1.06 1.18 8% 4% 4% 4% --- 4.21 4.63 3.96 .78 .58 .21 Mean SD 5-point scale Final Museum Exhibit Development Designing Building Displaying 88% 96% 96% 50 Table VII. Self-Reported Learning Gains ―Strongly Agree‖ or ―Agree‖ ―Neutral‖ ―Disagree‖ or ―Strongly Disagree‖ Process/Content Learning Gained appreciation of the processes that go into design and engineering 96% 4% -- 4.54 .59 Learned from the hands-on experience 92% 8% -- 4.38 .65 83% 13% 4% 4.38 1.01 63% 33% 4% 3.63 .82 44% 35% 22% 3.48 1.12 42% 33% 25% 3.25 1.07 Experience what it was like to be part of an intensive design team Experience of teaching others helped me learn material more thoughtfully and deeply. Gained greater appreciation of global environmental problems and the science behind them. Gained knowledge about the scientific, economic, and political issues Mean SD 5-point scale Skill Improvement Improved teamwork skills 88% 4% 8% 4.25 .90 Improved problem-solving skills Learned how to encourage my creativity Improved oral communication skills Improved written communication skills 71% 71% 50% 13% 29% 21% 17% 38% --33% 50% 4.04 4.04 3.17 2.5 .81 1 1.05 .89 Personal Development Increased understanding of environmental issues will likely influence me in the future (no matter what major or career I choose). 83% 17% -- 4.21 .72 -- 51 Table VIII. Student Perceptions of Integration of Fall and Spring Subjects Percent Saying Skills and Knowledge Learned in the Fall Were Enhanced by Taking the Spring Class Teamwork 84% Problem-solving 72% Research 47% Project Management 86% Knowledge 67% Percent Saying Skills and Knowledge Learned in the Fall Class Could be Applied to the Spring Class Teamwork 91% Problem-solving 81% Research 73% Project Management 89% Knowledge 67% Percent Saying Coordination Between the Two Classes Is "All Right As It Is" 82% Percent Saying a Combination of a Less Structured 1st Semester Experience and a More Structured 2nd Semester Experience is Useful 79% 52