SUMMER INSTITUTE SURVEY REPORT
1 DESCRIPTIVES
Is this your First, Second, or Third time at the Summer Institute?
Count
st
1
2nd
3rd
No Response
30
13
3
6
%
65
28
7
-
Have you Taught any Applied Learning Courses before?
Yes
No
No Response
How many ETEAL Proposals for Funding have you submitted?
0
1
2
3+
No Response
25 76
8 24
19
-
25 58
8 19
7 16
3 7
9
-
How many of your ETEAL Proposals have been funded?
0
1
2
3+
No Response





28 54
8 15
4 8
3 6
9
-
30 Departments represented across the survey population
52 total Survey Respondents
44 Participants completed the Pre-Institute Survey
39 Participants completed the Post-Institute Survey
~75% of participants completed both pre and post surveys
For the majority of participants, this was their very first time at the Summer Institute, though of those
about half were accompanied by colleagues from the same department who had been once or even
twice before. Most participants who attended had taught an Applied Learning course before, though
this question had a high rate of non-response. This could be because there were many staff members
who did not teach at all present on the first day, but it could also be because of a typo on the survey
itself which made the question read “I have taught _____ Applied Learning” rather than “I have taught
_____ Applied Learning Courses” This also led to confusion among respondents about whether they
should be checking the blank as a yes/no question or filling in a specific number.
While around half of all respondents reported having taught Applied Learning courses in the
past, just as many respondents reported never having submitted a proposal for ETEAL funding. From the
data we collected, it seems that among Summer Institute attendees, the chances of being funded if you
submit an ETEAL proposal are fairly high. This is generally similar to the overall rate of ETEAL funding and
the rates of funding over the past year, though it is still on the higher end of that range. In general, since
fall of 2013 ETEAL proposals have been funded at a rate of about two out of every three, while the rate
of funding was just over 70% among survey respondents.
This in itself is not terribly significant, but when we look closer we can see that only three
attendees had submitted proposals but never received funding. The rest of the respondents had all
received funding at least once, even if they were rejected on other occasions. Put more plainly, only
three respondents who had submitted proposals but never received funding were in attendance. As we
will see in the next section, the Summer Institute appeared to have a significant positive impact on how
confident respondents felt in their ability to submit a worthwhile proposal for funding, so it would
benefit those who have previously not received funding the most.
2 PRE AND POST TESTING
Change in Attitudes, Opinions, and Plans between day 1 and day 2
n
Initial Survey
Mean
4.15
Exit Survey
Mean
4.30
Difference
in Means
0.15
I feel that I know how to design an Applied Learning 30
Experience
4.06
4.48
0.42
I have a clear understanding of how applied learning 32
impacts students and instructors
3.45
4.45
1.00
I feel that I know exactly what I need to do to submit 30
a proposal for ETEAL funding
32
4.73
4.91
0.19
To what Extent do you think Applied Learning
Experiences can enhance student learning?
3.78
3.97
0.18
How Well do you feel you could Implement an ETEAL 28
Applied Learning Experience in one of your courses?
31
4.78
4.84
0.07
I feel that Applied Learning Experiences provide
Students with Valuable Skills and Experience
3.69
4.19
0.5
I have all of the Resources and Information I need to 28
Carry out an Applied Learning Experience within the
next year
28
4.65
4.94
0.29
How Likely do you think you are to plan an Applied
Learning Experience within the next year?
Difference in Means calculated using a Paired T-test, cases with missing pair data excluded
“I don’t know” coded as missing data, cases not reflected here, see discussion below
1
Significant at the 0.01 level
2 Significant at the 0.1 level, not substantively significant
Is it
Significant?
No
Yes1
Yes1
Yes2
No
No
No
No
Right away, we can see that some opinions and attitudes changed more than others over the
course of the two days of the Summer Institute. The differences in means column lists the results of a
paired t-test used to determine the overall average difference between the pre-survey and post-survey
answers for each respondent. This tells us how much their answers changed over the course of the
Institute, as well as which direction that change was in. By far the most significant change, both
substantively and statistically, is the shift in confidence regarding the submission of ETEAL funding
proposals. Most respondents reported feeling like they knew substantially more about how to submit an
ETEAL proposal by the end of the Institute. Among all the survey questions, this item showed the biggest
overall increase between the pre and post surveys, meaning that overall, respondents felt like they
gained a noticeable amount of knowledge and/or confidence regarding the submission of ETEAL
proposals.
The next item of note is the seventh question, regarding a respondent’s feeling that they have
the information and resources needed to carry out an Applied Learning Experience within the next
academic year. The change here was not as drastic as it was in the case of the ETEAL proposals, but by
the end of the Institute respondents did, on average, appear to feel more confident that they had the
resources and understanding needed to engage students in Applied Learning. This is, however, contrary
to the overall lower average confidence attendees had in their ability to implement an ETEAL funded
initiative in one of their courses. The change here was not statistically significant; in fact, there was
almost no change at all, and this is perhaps a cause for concern. By the end of the Institute, respondents
felt no more confident in their ability to implement an ETEAL funded initiative.
Respondent confidence in their ability to design Applied Learning Experiences rose over the
course of the institute, as did their perceived understanding of how Applied Learning impacted students
and instructors and how Applied Learning could enhance student learning. When we look more closely
at the responses for these items, we can also see that while most of the increase is the result of
respondents who initially answered “Disagree” or “Neither Agree nor Disagree” changing their answers
to either “Agree” or “Strongly Agree” in the exit survey, there were also a small number of respondents
whose opinions decreased by the end of the Institute. Although there were not many and it is difficult to
make any definitive claims about what it would mean given such a small sample size, there were two
respondents who initially felt that they knew how to design Applied Learning Experiences at the start of
the Institute and then by the end were not that certain. Given the impact that every instructor has on
their students, we should pay attention to these cases, few though they are, to make sure that the
Summer Institute does not have a negative impact on any of its participants.
Overall, it should be noted that the net effect of the Summer Institute appears to be fairly
positive on all fronts, with most respondents showing slight gains in their opinion of Applied Learning,
their confidence in implementing it, and their perceived knowledge about Applied Learning and
submitting ETEAL proposals. Although there was a roughly 75% response rate overall, there were seven
respondents who never completed their exit surveys and six who never completed initial surveys,
combined with those who left individual questions incomplete this leaves us with thirteen respondents
whose experiences are not accurately represented here.
This brings us to the issue of Significance. Several items were rated as statistically significant,
two at the 0.01 level and one at the 0.1 level, but this may be misleading. Substantively, there are only
three questions in which attendees appeared to have relevant and noteworthy gains. If we look at the
actual beginning and ending means, we can see that for several items such as question 4 (regarding the
extent to which attendees felt that Applied Learning enhanced student learning) that on average
respondents either marked “Agree” or “Strongly Agree” and by the end of the Institute, there was
simply a slight shift toward “Strongly Agree”. We can argue then that this is not substantively significant
because the shift between “Agree” and “Strongly Agree” is not likely to be a dramatic or noticeable
change. There were also a number of respondents who answered items with “I don’t Know” or “I’m not
sure” during their initial surveys and these cases were excluded from the overall paired-t tests, but they
are still important cases. Both Attendees who answered that they Didn’t Know how they felt about
Applied Learning design at the start of the Institute changed their answers to “Agree” or “Strongly
Agree” by the end. For most all attendees, this was the case no matter what the question and in all but
one case, attendees moved from “I don’t know” to “agree” or an equivalent answer. One attendee
maintained their “I don’t know” answer throughout the institute, and two others who answered “I don’t
know” did not complete their exit surveys.
Finally, we should note that while there was clearly a noticeable and significant change in
Attendee attitudes and opinions by the end of the Institute, we cannot make any claims of causation.
Not only are we limited by the relatively small sample size and the low rate of survey completion, these
surveys could not accurately capture any information about which part of the Summer Institute would
have affected these changes. In fact, we cannot be sure that these changes are the result of the Institute
at all, only that they occurred between the time when the Institute began and when it ended. Still, this
provides us with valuable insight into the areas that attendees improved in the most, as well as letting
us know where they already felt confident and competent.
3 EXPECTATIONS AND EXPERIENCES
The final question on both the initial and exit surveys was an open ended item designed to be a sort
of reflective piece, a parallel to the intention and critical reflection prompts used in Applied
Learning in ETEAL funded courses. At the start of the Summer Institute, respondents were asked
what their goals were for the institute. At the end, we asked them what they felt they gained by
attending. Compared to the body of the survey, the response rate on this final question was much
lower and although almost everyone answered at least one of the two reflections, only 21
respondents answered both. There are several possible reasons for this, though the primary culprit
is likely time considering that it takes much longer to think about and write a reflective response
than to check off boxes. Still, despite the low response rate, several distinct patterns emerged that
can tell us a lot about the expectations and experiences of the Summer Institute Attendees.
Among all of the participants who expressed clear expectations for the Summer Institute there
were four common themes but no single expectation represented the majority. The most common
theme by far was a desire to gain information or learn more about Applied Learning or how to
implement it in a course, which was the goal of at least 19 attendees. Many (11) respondents also
indicated a desire for new ideas and information in general which may or may not have been
synonymous with the previous point, based on their understanding of the framing of the Summer
Institute. Following that, 8 respondents were concerned with learning more about AL concepts and
theories themselves as opposed to just ideas for courses, 7 were focused on networking and
collaborative opportunities, and 6 were looking for support or information about things that supported
AL such as critical reflection and measuring student learning outcomes. Other goals and expectations
included learning about using Applied Learning in online courses, learning more about ETEAL funding
proposals, and generally improving teaching skills.
Overall, expectations were largely centered on the exchange of ideas, information, and
developing a knowledge base that would help individual instructors with their own courses. After that,
we start to see themes of cooperation and collaboration, but when compared with all of the
expectations that focus more on an individualized teaching goal or practice, the desire for collaboration
and networking is relatively small.
Breaking away from the more individually focused goals indicated on the initial survey, more
participants reported networking, collaboration, and the exchange of ideas as their primary gain from
the Summer Institute. This is not only a valuable effect and insight but also a source of even more
important questions; if the Summer Institute has the capacity to build camaraderie and increase
interdisciplinary communication and cooperation, then which part of the Institute is responsible? Was it
the round table discussions? Was it the Speaker-facilitated activities? Was it just lunch? If we can find
out which aspects of the Institute were most effective for networking and for developing
interdepartmental communication, then perhaps we could distill that and find a way to have other,
similar one-day events that focus specifically on interdisciplinary networking within the framework of an
Applied Learning marketplace of ideas and skills. Also, with any event that builds interdisciplinary
bridges, we’d need to establish some form of at least short-term monitoring to see how much
interdisciplinary communication and collaboration there is after these events. It’s entirely possible that
the exchange of ideas and interdisciplinary communication are only really happening in these events
and not outside of them.
Several participants also had specific comments about the guest speakers at the Summer
Institute. Peg Wessinger, the speaker on Day one, received the most praise from participants and her
topics and activities were well received among those who spoke of her, though it should be noted that
only six participants mentioned her at all. There was an equal mix of positive and negative commentary
on Laura Cruz, the speaker of the second day. It’s worth noting that the criticisms she received were
fairly targeted while the positive comments she received were often indirect or lumped in with general
positive statements such as “I enjoyed all of the guest speakers”. Because of the small number of
participants who commented on Laura Cruz, however, it’s hard to know the overall impression she left
on all participants.
Among those who responded to the final question on both the entry and exit survey, 15
reported achieving the goals they had at the start of the institute or even achieving something
additional. 5 Did not mention their original goals but instead reported that they had gained something
entirely different than what they set out to find. While this hints at a high level of satisfaction and
efficacy, we must be careful about making such a declaration. Not all participants who attended the
institute completed an exit survey and among the participants who did, 12 declined to provide a final
statement about what they felt they gained by attending the Summer Institute.
4 THE EFFECT OF ATTENDANCE AND RECOMMENDATIONS
From the results of the initial and exit surveys, we can see that the Summer Institute clearly had
an effect on those who attended and in a majority of cases this effect was entirely positive. With few
exceptions, every respondent reported feeling either better or the same about their understanding of
Applied Learning, its impact on students and instructors, its design and implementation, and its overall
benefit. Attendees also rated their understanding of how to submit an ETEAL proposal much higher by
the end of the Institute and generally felt that they had a better grasp of the resources and information
they’d need to carry out an Applied Learning Experience within the next year. Although the Institute was
not significantly likely to change an attendee’s plans to design a new Applied Learning Experience, most
attendees were already planning on that at the start of the Institute.
The exchange of ideas and the collaborative atmosphere of the Institute was also reported as a
common experience and benefit, though subsequent monitoring will be necessary to determine how
resilient the connections formed at the Summer Institute are. Regardless of longevity though, the
Summer Institute produced a distinct feeling of collaboration and camaraderie among attendees which
has value in itself. We can only speculate about which aspect of the Institute is responsible for this sense
of collaboration and mutual exchange at present, but upcoming follow-up interviews with attendees
should clarify the matter. It should also be noted that there were a large number of academic
departments not represented at the Institute and so there were perspectives that attendees did not
have direct access to. In the future, increasing both the representation of each academic department
and facilitating further collaboration and contact could benefit the Applied Learning Teaching
Community overall.
One avenue for potential improvement of the Summer Institute lies in continuing to refine it as
an applied learning exercise. The Institute already has most of the components of an Applied Learning
Experience, especially when it comes to the critical discussions that faculty have with one another and
the speakers, but it is still possible to make the Institute even more applied. The final question on both
the initial and exit surveys, for example, could be expanded to serve as Intention and Critical reflections.
An applied learning course creation activity could be added for the whole group with group discussion,
critique, and speaker feedback built in to further integrate the applied practice of designing an Applied
Learning Experience.
Another option for improvement, based on the results we obtained, would be to allow previous
ETEAL funded instructors to introduce themselves to the group at the start of the institute so that they
could be identified as potential collaborators or mentors by those looking for advice. There was such an
introduction in this institute, but it came at the end of the second day after many people were either
preparing to leave or had already gone. If networking and forming collaborative partnerships was one of
the greatest gains that attendees felt they made, then this could help them spend their time networking
more efficiently. Ideally, introductions for every Attendee would maximize this benefit but that is not
likely to be feasible given the size of the Summer Institute. It may also be beneficial to nurture the
connections made at the Summer Institute throughout the fall semester by holding shorter events
focused specifically on faculty and staff networking and collaboration.
As a final note and recommendation, the response rate on future Summer Institute surveys
might be increased by translating the paper survey to an electronic form that respondents could answer
via clicker. This would serve the dual purpose of gather data and introducing attendees to technology
that they could use in their own classrooms.