THE IMPACT SEMINAR EVALUATION
The IMPACT seminar was evaluated on the basis of annual reports that the seminar
participants were obliged to produce by the end of each year. Thus qualitative information
collected through the annual reports provided the data used by the evaluator to determine the
multiple aspects of change initiated by the seminar with respect to the participants, university
and community. The seminar participants were asked to structure their reports to include the
short description of the following elements:

strategic professional and personal goals,

short-term professional plans and revision of such if needed,

summary of their achievements,

personal change as an effect of IMPACT seminar

benefits of participation in IMPACT seminar (at personal and university/community
levels)

suggestions For Future IMPACT classes
The material has been processed through the QSR Nudist software. The evaluator has
created the following categories (nodes) to detect different concepts (constructs) mentioned in
IMPACT fellows’ reports:
1. Time/Stress Management. In reflection over the seminar topic about the strategies that
may be helpful in improving the quality and quantity of productive activity, a number of
IMPACT fellows stressed the importance of new techniques that they learned while
approaching the issues of overwhelming workload. New ways of structuring and planning
their workday are highlighted by several fellows as useful strategies they learned.
1
2. Perception of Self (Self-Awareness). This node refers to the changes in personal and
social identity as a result of attending the seminar. It is an axiom of cultural anthropology that
we are not endowed at birth with conscious self-awareness, but we have the propensity to
attain it (Foster, 2002; Herder, 1800; Rousseau and Herder, 1986). Therefore, this node is
critical in assessment of how and why IMPACT seminar triggered change at the levels of
social identity and, perhaps, even worldview. The change in self-awareness that a number of
IMPACT fellows attribute to the seminar is described as positive and empowering, a result of
continuous professional maturation and rise in self-esteem.
3. Learning (New Skills). An openness to learn new ways of doing things is a central motif
that passes through all IMPACT fellows’ reports and therefore is important as a criterion of
IMPACT success. Not only does this node reflect real progress in secondary socializations to
the norms of the institution, as is also evidenced by the two aforementioned nodes, but, more
importantly, it captures the openness of the seminar participants to new modes of thinking and
behavior, as well as their adaptability and flexibility to new social environments.1 Therefore,
this node has a strong nexus with “Attitude Change” node (see Table 1).
4. Attitude Change. The evaluator used the definition of attitude as a complex of beliefs and
feelings that people have about specific ideas, events, or other people. This definition is based
primarily on Cohen’s (1964) and Muzafer and Hovland’s (1961) work. Some change of
attitude is expected to take place during the course because the seminar stimulates cognitive
and psychological adjustment to new information flows by allowing participants observe their
own behavior in a new social setting (Bem, 1967). The intensive seminar is designed to foster
an environment in which participants begin to trust their new colleagues enough to analyze
1
Here openness serves as a prerequisite for the enhancement of cognitive ability and, therefore, is a latent
construct that cannot form a separate node.
2
publicly their own beliefs about the institution and their role in it. Such reflections are
encouraged throughout their year of participation in monthly team meetings. The IMPACT
seminar provides fellows with the resources that enable them to perform at a higher level and
achieve academic success. The “Attitude Change” is important because a number of fellows
note change in their attitude towards themselves and others around them (see Table 1).
5. Empowerment. This node is a reflection of the fact that many IMPACT fellows attest to
being empowered in consequence of the positive experiences and encounters developed in the
course of the seminar. At the level of personality development empowerment means a
culmination of self-realization (Ozer and Bandura, 1990). It means that an individual has
come to an understanding of his or her own worth and through the resources available through
the program, self-realization becomes possible. Note that empowerment does not necessarily
imply involvement of only an individual agent. In practice, agency may be implemented at the
individual level or in groups. At the institutional level empowerment is critical for
involvement of faculty in decision-making. Because institutions of higher education generally
are hierarchical organizations, faculty and administration often have different professional
agendas. Hence, faculty members are often skeptical about the university administration
decisions. Empowerment is the positive personal change that contrasts with the distrust that
some faculty have towards upper administration. Empowerment in the sense of institutional
transformation leads to the increased interest in the decision-making process.
6. Increased Productivity. As already stipulated earlier, time and stress management was not
only set as a priority by many IMPACT participants, but also a number of the program
seminars and workshops are routinely allocated to address this particular problem. Keeping
time/stress management in focus, IMPACT participants were encouraged to develop an
3
action plan for improvement with accountability and benchmarks. The action plans allowed
IMPACT participants compare the results of their research and teaching activity before and
after attending the seminar. The better results in a very limited framework of the seminar (1
year) are reported almost unanimously by the IMPACT participants. There are signs of
continuous improvement in productivity not only in terms of research and publication record,
but also in terms of more effective teaching techniques, etc.
7. Integration of Personality. One of the main benefits of the seminar that has been
constantly stressed by its participants was learning the practical skills of how to become an
integrated personality. Here lies the importance of IMPACT in creating a conducive social
environment which serves as a source of the external stimuli which, in turn, bring about the
integration. Social environment and significant others (including role models) provide us with
the stimuli to imitate which results in creating common patterns of behavior (Dewey, 1916).
Although the integration of personality usually manifests itself through a variety of behaviors,
an integrated personalities self-identify themselves as “being together”, “being whole”.
Several IMPACT fellows reported becoming “whole” as a result of attending the seminar (see
Table 1).
8. Integration of Teaching, Research and Service. Among the personal changes that
IMPACT fellows ascribe to the positive influence of the seminar is the necessity to effectively
incorporate their teaching into their research and service. Apart from “keeping the focus” and
“staying in focus” which are repeated motifs in the fellows’ reports, the integration of
research and teaching is perceived as an important strategy to be employed by the junior
faculty in the process of their professional development. It is by utilizing this strategy that the
4
continuous enhancement of the scholarly work of teaching and research becomes a virtually
limitless opportunity (Cross, and Steadman, 1996).
9. Leadership. One of the positive changes that the seminar has brought about in IMPACT
fellows is genuine interest in leadership. As one of the recurrent themes in fellows’ annual
reports, interest in leadership is seen as not just a personal enterprise but as a collaborative
effort. Indeed, one of the main goals of the seminar was to discuss and explore the challenges
and opportunities of academic leadership. This effort should lead to the formation of a cadre
of leaders who may take up the sustainability of institutional change. Availability of role
models is one of the most important factors that affect leadership as a process. The formation
of integrated personality, in theory, relies on the exposure to role models that show a
successful career is possible (Henry et al., 1994; Tierney and Bensimon, 1996; Wunsch,
1994). Several of IMPACT fellows expressed their desire to serve as role models to other
junior faculty, a change which is attributed to attending IMPACT.
10. Sense of Community. This node captures the ways IMPACT fellows experience the
feeling of belongingness to a University as a community (McMillan and Chavis, 1986;
Sarason, 1974).2 A sense of community manifests itself in the interest in being a member and
sharing responsibility for community action (Sarason, 1974). Thus, by showing a strong
interest in community action, IMPACT fellows perceive the benefits that might acrue from
their participation in community.
11. Collegiality. Collegiality in the context of an educational institution means an ability on
the part of individual faculty to effectively participate in decision making given the existing
For parsimony, it is suffice to integrate two concepts – sense of community and sense of belongingness in the
community. Although the use of these concepts has a divergent path in the community psychology literature
(see, for example, Chavis and Pretty, 1999), the empirical value of this report is not undermined by the insistence
on conceptual clarity.
2
5
diversity of opinions (Bess, 1988). A mutual definition of a problem and a shared process of
decision making are important components of collegiality and are essential for professional
development. Collegiality dwells upon the ethical code of bahavior which is based on mutual
respect and allows collective understanding of common problems (Cahn, 1993; Baldridge,
1971; Johnson, 1997). Therefore, this node includes sub-node “Diversity” to capture the value
and significance that IMPACT fellows place on differences among and between them.
Beyond its intrinsic value, collegiality has been referred to by IMPACT fellows as a variety of
strategies UTEP as an institution of higher education has to employ in order to address the
consequences of historically pervasive white-man bias in academia. Not only is the
persistence of the aforementioned bias evident from the fact that the majority of academia
until the 1970s has been disproportionately white men (Cahn, 1993), but it is also manifested
in the homogeneity of views and opinions clustered around the viewpoint of non-Hispanic
white men. Collegiality was mentioned by IMPACT as an important skill gained through
participation in the seminar by IMPACT fellows 15 times (the highest frequency among the
university benefits), a result reflecting increased awareness of and newer thinking about the
building of a university-wide community.
12. Networking. This node refers to junior faculty’s ability to build ties with their colleagues.
Networking fosters professional development through access to information and resources
(social capital) that support scholarly growth.3 As shown in Table 1, several IMAPCT fellows
reported that creation of a network of associates as one of the most visible benefits of the
seminar.
3
The concept of social capital was developed by social scientists to describe the intangible resources available to
individuals through their membership in networks (Portes, 1998; Putnam, 2000).
6
13. Certification (Institutionalization). A number of IMPACT fellows supported the view
that IMPACT seminar should become institutionalized. Moreover, some even proposed that it
should become mandatory for all junior faculty, given the perceived benefits of the program.
Nodes 1-8 were further merged into a tree node “Personal Benefits of IMPACT
Participation” and nodes 9-13 were combined into a tree node “Institutional Benefits of
IMPACT Participation”. The node tree is shown in Figure 1. This aggregation of individual
nodes is provisional because of a certain degree of overlap between them. For example, the
“Leadership” node can be interpreted as being beneficial for junior faculty as individuals as
well as for UTEP as an educational institution. The institutional benefit, in the evaluator’s
opinion, is more important because leadership is a collective asset through which the
dissemination of best practices and transmission of knowledge and skills is sustainable
(Tierney and Bensimon, 1996). The summary of nodes and correspondent quotes from
IMPACT fellows’ reports is presented in Table 1.
The comparison of the annual IMPACT fellow reports across years revealed that
earlier cohorts of IMAPCT fellows tended to underscore the less concrete outcomes, such as
attitude change. There is a notable shift in the reports of more recent cohorts towards tangible
outcomes, such as publications, presentations, etc. Also noteworthy is the tendency of more
recent cohorts to adopt a more realistic attitude towards their own abilities and endeavors.
Bibliography
Baldridge, J. V. 1971. Power and Conflict in the University. New York: John Wiley.
Bem, D. J. 1967. Self-Perception: An Alternative Interpretation of Cognitive
Dissonance Phenomena. Psychological Review 74: 183–200.
7
Bess, J. 1988. Collegiality and Bureaucracy in the Modern University: the Influence of
Information and Power on Decision-Making Structures. New York, NY: College Press.
Cahn, S. M. 1993. Affirmative Action and the University: A Philosophical Inquiry.
Philadelphia, PA: Temple University Press.
Chavis, D. M., and G. Pretty. 1999. Sense of Community: Advances in Measurement
and Application. Journal of Community Psychology 27(6): 635-642.
Cohen, A. R. 1964. Attitude Change and Social Influence. New York: Basic Books.
Cross, K. P. and M. H. Steadman. 1996. Classroom Research: Implementing the
Scholarship of Teaching. San Francisco, CA: Jossey-Bass.
Dewey, J. 1916 Democracy and Education. An Introduction to the Philosophy of
Education. New York: Free Press.
Forster, M. N. 2002. Herder: Philosophical Writings. Cambridge: Cambridge
University Press.
Henry, J.S, Stockdale, M.S., Hall, M., and W. Deniston. 1994. A Formal Mentoring
Program for Junior Female Faculty: Description and Evaluation. Initiatives 56(2): 37-45.
Herder, J. G. 1800. Outlines of a Philosophy of the History of Man, translated by T.
Churchill, London: J. Johnson.
Johnson, A. G. 1997. The Gender Knot: Unraveling Our Patriarchal Legacy.
Philadelphia, PA: Temple University Press.
McMillan, D. W., and D. M. Chavis. 1986. Sense of Community: A Definition and
Theory. American Journal of Community Psychology 14(1): 6-23.
Muzafer, S., and C. I. Hovland. 1961. Social Judgment. New Haven, CT: Yale
University Press.
8
Ozer, E., and A. Bandura. 1990. Mechanisms Governing Empowerment Effects: a
Self-Efficacy Analysis. Journal of Personality and Social Psychology 58:3: 472-486.
Portes, A. 1998. Social Capital: its Origins and Applications in Modern Sociology.
Annual Reviews of Sociology 24:1-24.
Putnam, R. D. 2000. Bowling Alone. New York, NY: Simon and Schuster.
Ross, E. A. 1901. Social Control. New York: Macmillan.
Rousseau, J. J., and J. G. Herder. 1986. On the Origin of Language, translated by J. H.
Moran and A. Gode. Chicago: The University of Chicago Press.
Sarason, S. B. 1974. The Psychological Sense of Ccommunity: Prospects for a
Ccommunity Psychology. San Francisco: Jossey-Bass.
Tierney, W. G., and E. M. Bensimon. 1996. Promotion and Tenure: Community and
Socialization in Academe. Albany, NY: State University of New York Press.
Wunsch, M. A. 1994. Giving Structure to Experience: Mentoring Strategies for
Women Faculty. Initiatives, 56 (1): 1-10. Washington, D.C.: National Association for Women
in Education.
9
TABLE 1. Quotes of IMPACT Participants and Nodes Created in QSR Nudist.
Node
Selected Quotes
Personal Benefits of IMPACT Participation
“IMPACT … helped to reduce my stress level greatly, solved some real
problems and tooled me to avoid these problems in the future.” “Closing
my door for a couple of hours a day has helped tremendously in that it
Time/Stress
provides me with uninterrupted time. I will most definitely maintain this
1 Management
initiative.” “Following the seminar last May, I purchased a timeTotal quotes: 19
management system which I have been using continuously. This, perhaps,
has been one of the most visible and welcome changes in my life.”
[need a line between these two like you did for the other categories]
“Self Development – am I utilizing my network of people and am I
listening? Gotta self reflect on this on a regular basis…” “Reflection
came about more subtly; assessment became a normal part of my routine,
even though I didn’t really realize that I was assessing my progress.”
Perception of
“The importance of self-reflection to truly understand my own mission to
Self (Self-
students, colleagues, UTEP, and El Paso/Cd. Júarez community. This
Awareness)
understanding helps sharpen diplomacy when it comes to saying “no” to
Total quotes: 11
committee work that has little or nothing to do with my mission. Most of
2
all, since service work must be done, knowing my mission allows me to
be proactive as to which committees and other serves activities to
participate in order to increase my circle of influence for changes that I
would like to see take place.”
10
Node
Selected Quotes
“… able to benefit from the new ideas I have learned…” “Learned more
Learning (New
about the tenure and promotion process at UTEP and in other
3 Skills)
institutions.” “…I left with new ideas, suggestions and reassurance from
Total quotes: 16
colleagues who had dealt successfully with similar challenges.”
“Attitude is perhaps the major change I have experienced this year as an
IMPACT fellow…Thus this last year, more then others, I have challenged
myself to keep focus on the issues where my chances of creating change
Attitude Change
are greater—yet without compromising my ethics” “Another challenge
4
Total quotes: 7
with change is busyness, loss of focus, overworked. This is combated by
encouraging people to plan well, manage time.”
“One of the main changes in perspective that I am trying to maintain is to
see myself as being in control of my situation instead of being buffeted
Empowerment
about by forces out of my control.” “I now see myself as a “facilitator”,
Total quotes: 6
whereas before I saw myself as expendable, part of the herd, a follower.”
5
“I now see myself as a manager and not as a teacher or researcher.”
“Stronger and integrated faculty (integrated within themselves
Integration of
6 Personality
Total quotes: 7
and between colleges) leads to a stronger institution.” “…I could see
myself more as a “whole person” and member of a whole campus
community, not just my own department or college.”
11
Node
Selected Quotes
“I have seen a large improvement in my productivity over the past year. I
expect that productivity increases will continue into the near future. That
increased productivity will help to make me a more valuable asset to
Increased
UTEP through improved scholarly work, teaching, and service.”
7 Productivity
“More writing was accomplished as a function of time. For example, three
Total quotes: 4
major articles were published in this period.”
“The program has allowed me to focus more clearly on obtaining funding
by targeting relevant funding opportunities…”
“The integration (R/T/S) and the creation of synergy were very important
lessons. This does not have to be an artificial add-on, nor does it have to
be done as an afterthought.” “… the integration of research and teaching
became a natural progression.” “…improved my skills in integrating
teaching, research, and service (kill
Integration of
as many birds as possible in one stone)” “Since IMPACT is designed for
Teaching,
faculty to develop ways of integrating teaching, research, and service once
8 Research and
we begin a proactive approach
Service
towards such integration, I feel the major benefit to the University is
Total quotes: 8
that it will have more efficient faculty members who won’t burn out
within a few years.”
12
Node
Selected Quotes
Institutional Benefits of IMPACT Participation
“The greatest impact that I see the IMPACT Fellowships having is in
increasing collegiality among the faculty.” “… the fact that the
participants get to meet colleagues from various colleges and departments
9
Collegiality
helps in creating bonds of friendship and possibilities to collaborate on
Total quotes:
multi-disciplinary and inter-disciplinary endeavors.” “The different
17
points of view about the university operation are extremely important,
especially for young faculty.”
“… by hearing experiences of people outside my department…, I gained
very helpful perspective about what facets of my experience at UTEP are
fairly “universal”…” “…all of us, assistant, associate, and full professors
are experiencing many of the same issues.” “Almost all untenured faculty
10
Sense of
shared most of my feelings and concerns. This was tremendously
Community
encouraging …”
Total quotes: 5
“The IMPACT program … exposes the faculty member to the University
process as a whole”
13
Node
Selected Quotes
“Impact can help cultivate future leadership at UTEP by providing faculty
members who have thought holistically about teaching, research, service
and leadership and have some training in integration.” “Participation in
IMPACT has stimulated an interest in leadership issues in me.” “I plan to
continue my studies in leadership, and to incorporate lessons learned in
Leadership
my professional and private life.” “I do see myself as a role model, and as
11
Total quotes:
a person who will help others achieve tenure.” “…I have selected certain
11
people, whom I view as role models and emulate some of their practices.”
“There is something to learn from everyone’s leadership style and I made
it a point to try to pick and choose certain characteristics from people that
I met, adopt some of them, tailor some of them to meet my needs and
certainly to avoid others.”
“We have developed a new support network for women in engineering at
UTEP. The network has been established to promote success of
Networking
women…” “…rely on this network of people for help, advice, mentoring,
Total quotes: 9
and encouragement.”
12
14
Node
Selected Quotes
“I strongly believe that having a certificate (yes, a paper certificate) for
each fellow who completes the IMPACT program is extremely
important, particularly if such a certificate has the signature of
Certification
someone from the upper administration – such as the provost – who cares
(Institutionali-
about the success of this program. I am willing to personally pay for the
zation)
forms, and for the administrative time to prepare such certificates (for our
Total quotes: 4
cohort).” “… I would encourage the university to make IMPACT
13
mandatory, as part of new faculty orientation.” “I would recommend that
UTEP make IMPACT mandatory for all new faculty, and offer it as part
of new faculty orientation.”
15
Figure 1. Node Tree.
Personal Benefits
Perception
Integration
Attitude
of Self
of T/R/S
Change
Time/Stress
Management
Empowerment
Institutional Benefits
Sense of
Learning
Integration
of
Personality
Leadership
Collegiality
Community
Increased
Productivity
16
Networking
Certification