Sample Evaluation Project Plan

advertisement
BODY LANGUAGE INSTITUTE INTERNSHIP
EVALUATION PLAN
JERUSALEM MERKEBU
GEORGE MASON UNIVERSITY
NOVEMEBR 20, 2012
Submitted To: Janine M. Driver
Program Organization: The Body Language Institute
Start Date: September, 5, 2012
Report Delivery Date: December, 20, 2012
Total Fees:0
1
EVALUATION PLAN
Information about the Evaluation Request
Introduction
The Body Language Institute (BLI) offers nonverbal communication training to
executives across various industries. The institute has numerous service areas that are run
simultaneously, i.e., media promotions, book publication/projects, and corporate speaking
engagements. The mission of the internship program is to bring in interns that will support these
service areas in an effective and efficient manner. The program is said to provide interns with a
dynamic opportunity to assist the CEO, Janine Driver, with media interviews in Washington, DC
and New York City; where they work closely with producers and editors from major
networks/national interest magazines (yourefferral.com). Furthermore, interns coordinate with
Janine’s management company, Traci Allen Inc. to arrange her corporate speaking events; as an
incentive the opportunity to travel may be possible. The fundamental existence of the internship
program is to provide support the service areas mentioned above and will be thoroughly
discussed in the program components section of this evaluation. Although, interns are expected
to complete administrative tasks, the vibrant environment promises to be inspiring and
challenging. The target audience includes all who are interested in working in a creative
environment.
Request from Program Officer
The program officer, Janine Driver has expressed the Body Language Institute’s need to
create a sustainable internship program. The program officer has stated the lack commitment
from interns as they tend to leave the institute prematurely. The organization had obtained a total
2
of 27 interns over a four year period. 18 of the 27 left the organization in less than a three month
period. This constant turnover has caused dissatisfaction with the program. Consequently, the
program officer is interested in discovering what can be done to increase the longevity of interns.
She is interested in receiving feedback from the evaluation in order to implement the
recommendations on the next attempt to recruit interns in 2013. Currently, formal processes are
not in place to support the internship program.
Background Information
Janine Driver is the founder, president, and lead instructor for the Body Language
Institute (BLI). As a retired industry operations investigator with the Bureau of Alcohol,
Tobacco, Firearms, and Explosives (ATF), where she held top-secret clearance for all of her 16years with the Bureau, she progressively transitioned into the role of a key note speaker and
became a media personality. In this new role, she conducted training across various industries
from the Central Intelligence Agency to numerous Fortune 500 companies such as Coca Cola,
Disney, Proctor and Gamble etc... The Body Language Institute was birthed in 2007 out of
Janine Driver’s desire to transfer the knowledge, abilities, and skills she had amassed to help
executives, sales personnel, and future leaders build executive presence, develop their sales
skills, and create and deliver winning business presentations. It was principally conceptualized
to create a space, being the only one of its kind, where all can come to learn about nonverbal
communication in greater depth. The institutions mission is “to boost the confidence, credibility,
and careers” of individuals in professional and interpersonal settings, by exposing them to a
wealth of knowledge about nonverbal communication, in an interactive classroom environment.
Taking it to the next level, the Body Language Institute offers multiple courses
exclusively designed for those demonstrating mastery in detecting deception and nonverbal
3
communication skills. These intense certification programs are anticipated to equip professionals
with the appropriate tools needed to achieve the high level of proficiency, required to train others
in reading nonverbal cues. Hence, students will be able to calibrate and norm others, detect
deviations which serve as indicators of deception, and ask purposeful questions to obtain truth.
Additionally, they’ll be equipped to effectively instruct nonverbal communication courses across
in their domains. The major learning objectives include, helping individuals teach how to
decipher fact from fiction, and gain new insight into body language, by breaking mindreading
myths. Instructional methods include decoding videos, statement and picture analysis, reviewing
case studies of notorious convicted con artists, and class discussions. Furthermore, through
experiential learning individuals will interact with subject matter experts in voice analysis and
training, proxemics (the study of social spaces), etiquette, graphology (the study of handwriting),
facial expressions, and decoding deception. Accordingly, students are said to master the ability to
detect deceptive signals, illicit purposeful information through interviewing techniques,
effectively communicate and read masked information displayed by others. This holistic
approach is deemed to create instructors that will exude confidence and deliver powerful
sessions as they receive a video tape of their journey and a one-on-one evaluation/coaching with
subject matter experts.
Body Language Institute Program
The BLI internship program was designed to recruit interested individuals who can assist
in all the programs that the institute offers, as mentioned above. The internship program is
designed to benefit both the interns and the organization. While interns provide support in the
various service areas of the organization, the interns will be exposed to nonverbal
communication content and develop their business skills. The internship program provides
4
interns with a free scholarship to participate in any of the courses offered by the institute. They
are expected to assist in all of the courses that the institute houses. The program promises to give
interns an opportunity to work directly with the subject matter expert and New York Times bestselling author, media personality, Janine Driver. Interns are typically presented with an
opportunity to travel as they will be assisting in the CEO’s endeavors. They are to coordinate all
logistics with the instructors and students; essentially prepare workbooks, assessments, and
anything needed by the organization. Unlike the average office environment, the organizations
informal workspace is comprised of flexible work hours and casual dress code. The nature of the
organization being fast paced multiple projects i.e. media, books, classes etc… will be
undertaken simultaneously. Interns are given the flexibility of taking on more of the tasks that
they feel is relevant to their long-term goals. The internship is unpaid and for those seeking 3040 hours of work. Interns are not required to have previous work experience.
Description of Procedure for Gathering Information Included in the Evaluation Plan
The procedure for collecting information for the evaluation plan included gathering and
reviewing, the two books written by Janine Driver, and all the materials (comprised by subject
matter experts who teach at the BLI) presented to the students of the institute. Some of the
documents were received through email from the program officer. Upon Janine Driver’s request
the website was also pertinent document for the evaluation. No formal documents for the
internship program exist. The only available supporting document is the advertisement
previously posted, on youferral.com and craigslist.com. Consequently, antidotal information that
was retrieved via phone, from the program officer, regarding the number of interns that had
previously taken part in the internship program, was also incorporated.
5
Audience
The program officer and CEO of the Body Language Institute, Janine Marie Driver, will be the
only audience of this evaluation.
Evaluation Utilization
The program officer and CEO of the Body Language Institute, Janine Marie Driver is the
greatest stakeholder. She has expressed that she is interested in having the evaluation conducted
in order to increase the retention rate of the interns in the program. She is seeking to make the
appropriate changes that will contribute to enhance the quality of the internship program; by
creating the appropriate organizational conditions. In other words, the organization may need to
create an infrastructure that will support the internship program. The evaluation will pose critical
questions that will assess the existence or lack thereof a robust selection process. The
organization plan and process theory will be reevaluated in order to create an atmosphere where
interns are selected through a systematic review for the internship program. Their interests must
be aligned to that of the Body Language Institutes. Ultimately, with the hope of obtaining a good
fit for the both the individual and the organization. Decisions will be made with the goal of
decreasing the turnover rate. Changes that are likely to be implemented include: creating a
formal process and structure for the internship program, while re-conceptualizing the programs
theory. In reference to budgeting, providing incentives to motivate those joining the internship
program may be taken into consideration.
Information about the Program
Narrative Description of the Program
6
The internship program was birth out of Janine Driver’s desire to receive assistance in
handling her multifaceted business. The internship program was fully launched in the summer of
2009. At the release of her first book, You Say More Than You Think, the CEO, along with her
manager Traci Allen, decided recruiting a large number of interns would be ideal for the
organization. During this demanding season, where media interviews and book promotions had
skyrocketed, the internship program was conceived to meet the new demands placed on the
institute. To successfully recruit interns an advertisement with the following header precisely,
“NBC's TODAY SHOW'S BODY LANGUAGE EXPERT SEEKS INTERN,” was posted on
youferral.com and craigslit.com. Those who are interested reply with their resumes, to the email
address provided. Next, a mass email is sent out by the CEO, asking those individuals that are
interested to attend the institute when classes are in procession. The individuals are provided
with the date, time, and location of the organization. After the last session, all those that attend
are verbally offered the internship program. From that point forward this process became the
informal standard operating procedure for recruiting interns. The interviews are also kept lite and
informal. The CEO’s comment is that she finds value in observing their interactions with one
another, and the executives in the classroom. Her observations and dialogue with her manager
Traci Allen are used to accept the individuals. No formal documents are signed. The CEO being
flexible presents the interns with an opportunity to verbally select the times and days they are
available for work.
It is verbally communicated to new interns that this special position grants them with the
opportunity to work with various media personnel, write for well-recognized entertainment
magazines, and essentially will boost their resume. The nature of the organization being fast
paced and unstructured, it’s assumed to be a thrilling environment. Due to the unknown
7
demands that are likely to arise on a day-to-day basis, interns are told how a day will go remains
uncertain. It can range from working on a simple administrative task to developing content for
publishing in a contemporary magazine like Psychology Today. If a call is received from the
media it must be acted upon immediately and the interns will accompany the CEO to the
interview. Furthermore, the various services offered by the institute are projected to allow
interns to explore their creativity in various ways such as social media, acting; creating ads
etc…The informal atmosphere is expected to provide interns with much flexibility and reduced
the negative effects of the more mundane corporate office. Consequently, the underlying theory
is that the interns, placed in such a fun and vibrant setting, will be motivated to effectively and
efficiently, support the various needs of the institute.
However, the unfortunate feedback from the CEO is that, interns are steadily dropping
out of the internship program. For example, in a single recruitment session, where ten interns
selected only two remained for a period of time. The interns failed to commit to the organization.
Due to such challenges, that have been consistent in the past four years, it may be in the best
interest of the organization to re-conceptualize the internship program theory and description
(Rossi et al, 2004). As a collaborative effort the CEO Janine Driver will be asked to participate
in taking surveys that will help to clarify the goals and objectives of the internship program with
the intention of restructuring program components. This process will also aid in identifying or
measuring the intervening variables on which outcomes may depend. Correspondingly, being
able to explain what the deficiencies are of the internship program and seeking out the variables
that are contributing to heighten the lack of productive outcomes of interest.
List of Program Documents
8
The Body Langue Institute internship program is advertised onYouferral.com and
Craigslist.com. The captivating advertisement begins by informing potential interns that they
will have an opportunity to work side by side with the CEO “Janine who leads a team of some of
the world’s greatest thought leaders, including Emmy award winning television producer…”
furthermore, no previous work is required. The only requirements listed include: happy
personality, strong writing, research, communication and interpersonal skills, knowledge of
Microsoft products, being a quick learner, self-motivated, and comfortable in a fast paced
environment that gets national media attention. While, interns are expected to complete
administrative tasks such as log in receipts, organize files, create a client folder for each of
Janine's events, pack up and unpack speaking supplies, photocopy BLI workbooks and handouts
at Kinkos, print and frame certificates, pick up supplies from Staples etc… the vibrant
environment promises to be inspiring and challenging. The advertisement does communicate the
various aspects of work involved within the organization from coordinating speaking
engagements to assisting with media interviews, working on the author’s book projects, and
assisting with the courses offered. The program is designed to have interns participate in all the
functions of the institute. The internship program exists to support all the different aspects of
BLI. Each of the components and their documents will be individually described below.
The critical documents of the program that the interns will take part include two books
written by Janine Driver:

Driver, J. (2012). You Can’t Lie to Me by Harper One

Driver, J. (2010) You Say More Than You Think by Crown Publishing Group.
You Can’t Lie to Me
9
You Can’t Lie to Me is a book that draws from the research conducted by Dr. Paul Ekman
who is recognized for his seminal work in the 7 universal emotions: Happiness, Sadness, Anger,
Surprise, Fear, Disgust, and Contempt. In addition to describing how each appears on the human
face Janine Driver in her book, discusses how to recognize these emotions when they are being
disguised or appear as micro-expressions. The purpose of the book is to help individuals
decipher fact from fiction, by a) getting a person’s baseline or normal behavior and b) asking the
right questions to decode if someone is trying to be convincing i.e. lying or is simply conveying
information. The interns are expected to learn the fundamental assumptions of the book for
promotion and course development purposes.
You Say More Think
You say more than you think is the first book published by the President of the Body
Language Institute Janine Driver. The book focuses on non-verbal communication that occurs in
professional and interpersonal settings. This is described as a two-fold process in the book 1)
how an individual is reading others 2) how others are perceiving the individual. The author
conveys gestures and positions that individuals can use to appear more powerful or persuasive in
any context, from the boardroom to the bedroom, in order to elicit the responses they are
seeking. The book provides various methods for manipulating positions and gestures to convey
power. The purpose of the book is to help individuals appear more powerful in seven seconds, by
quickly employing gestures to take control in social settings. The interns will be provided with
the book and will need to learn the material in order to write various blogs or support student
learning at the institute.
Course Documents Overview
10
The individuals in the internship program heavily support all the courses offered by the
institute, from assisting in content development to creating certificate for those becoming
certified. The students in the Body Language Institute coursers receive a comprehensive
facilitators program manual and resource kit which includes: reproducible customized handouts
and marketing materials i.e. The Dining Etiquette, The power of communication: Decoding
Personality and Character/ The Power of Communication power point slides, handwriting
analysis case studies, and practical video exercises which they are encouraged to utilize when
teaching futures sessions. Each of these critical components that the internship program is
intended to support will be described below.
The Dining Etiquette
The Dining Etiquette is a handbook developed by Michele Pollard Patrick, who is a
certified Protocol Officer and business etiquette consultant/ image coach since 1989. As a
subject matter expert in business etiquette and protocol, she teaches professionals break bread
with clients and colleagues. The interns are expected to support Michelle and therefore, to be
familiar with the contents in the handbook. The content includes: appropriate posture at the table,
when to being eating, place setting, handling flatware, distinguishing American and Continental
styles of eating, eating soup properly, using finger bowl/chopstick properly, managing specific
foods gracefully, dining do’s and don’ts, appropriate table talk, excusing yourself, toasting, and
tipping etiquette. The second module focused on business etiquette confident introduction (i.e.
making eye contact, proper handshake, smile etc…) rules of social/ business introductions e.g. “a
person of lessor importance is introduced to one of greater importance; clients take precedence
over business associates.” Furthermore, business card, speaker phone, cell phone, email, Ipod,
business meeting, and thank you notes etiquettes are all discussed. The interns maybe asked to
11
create a formal assessment drawing from the content or assist Michelle in this endeavor as
needed.
The power of communication: Decoding Personality and Character document
Frank Marsh is a former Marine Corps Intelligence Officer and is a certified handwriting
analyst who has been called upon by law enforcement as an expert witness. At the Body
Language Institute his course on Decoding Personality and Character raises one’s awareness of
handwriting or graphology. He contends this subject has been studied longer and more in-depth
than other personality assessment tools as revealed in Forensic Handwriting Identification.
Graphology is defined as the study of handwriting, especially when used as a way of analyzing
character. In this section student assess handwriting traits i.e. clear, consistent, spontaneous,
good form, style, pressure, speed, and rhythm. Any deviations from normal styles of writing can
be attributed to physical health such as being tired or drunk, emotional health i.e. anger or
sadness, and psychological health i.e. personality disorders. This content is derived from
scholarly research as is evident in the references section of his presentation.
The second module is on the power of communication especially words and its ability to
shape one’s world. This inspirational PowerPoint is designed to convey the significance of
communication as a Harvard study on success demonstrated 85% was positively correlated with
communication skills and only 15 % with knowledge (The Power of Communication BLI). The
presentation highlights that what a person sees, says, and thinks are determined by one’s choice.
Therefore, by focusing on the present and utilizing positive words, one can choose to have
success in the future. The interns will be asked to support Frank Marsh in threading the content
together for creating a formal assessment or as needed.
Reconnect to Your Body and Push past Your Fear
12
This document is created by Chris Ulrich and it provides meditation, movement, and
improvisation techniques to move beyond fear and demonstrate. The objective of the breathing
and quieting the mind exercise listed is to shift focus toward productive engagement and
effective decision making. The document contains practical use of meditation, basic movement
exercises, and improvisational exercises such as laughing. The improvisation games are said to
help individuals work from a place of awareness, focus, and ease, increasing the ability to
perform at an optimal level in all settings. The games listed such as “Yes And” increase listening
skills, teaches acceptance with “yes and” and help to develop mental quickness. The activities
listed include, Zip, Zap, Zop and Imaginary Object, Music in the Middle, Crazy Eights, 10
Things, Sitting, Standing, Bending Yes And, Taxi Driver, Alphabet Game and concludes with a
reflection. The interns are expected to participate in the activities in order to support student
learning at various capacities, from providing students with props to physically engaging in the
games. The interns will work with Chris Ulrich to make the experience a meaningful one for the
students.
The Body Language Institute .com
The website thebodylanguageinsitute.com is filled with much content that emphasizes the
need to be aware of nonverbal communication techniques especially detecting deception. The
website contains information about BLI programs, techniques to boost sales, book information
which promises professionals and executives with cutting-edge, scientifically-based
communication and body language tools, media interviews, etc… The website is filled with
fundamental information that the interns must comprehensively know, in order to effectively
support the institute in media, books, course, and corporate speaking engagements. The interns
are expected to assist in each of these sectors as needed. The internship advertisement highlights
13
graphic design and social media skills as a bonus due to the CEO’s interest in having the interns
maintain the website. Interns are expected to develop website content if the demand presents
itself. They are encouraged to become familiar with the multifaceted website. Therefore, the
website serves as a fundamental document of the internship program.
The internship program is informally designed to have the interns interact with the
documents listed here at various capacities, depending on their knowledge, skills, and abilities
and the needs of the institute. Apart from the documents discussed in this section, there are no
documents that specifically exist to serve or elucidate the mission, goals, and objectives of the
internship program. The lack of formal documents and program design may serve as critical
evidence of the fundamental issues thwarting the internship program. Consequently, it will
contribute to driving the program evaluation.
Key Program Components
14
Interns support all
book related
promotions and
activities
Interns support all
courses offered by the
institute i.e. formative
assessments
Interns support all
media related tasks
and website activities
The Body Language
Institute Internship
Program
Interns must develop
content knowledge:
dining etiquette,
power of
communication:
Interns coordinate
corporate speaking
engagements
Interns perform
administrative tasks
on a day to day basis
Decoding
Personality and
Character/ The
Power of
Communication
power point slides,
handwriting
analysis case
studies
This image illustrates all the service areas and the overlapping nature of all the tasks. These
are all key program components that the internship program exists to serve. However, due to the
plethora of service areas and shifting needs of the Body Language Institute, the evaluation will to
take into consideration: 1) what can be done to minimize the competing demands placed on the
interns?; and 2) what can be done to get the interns fully engaged when they need to attend to
disparate service areas?
List of Program Clients
15
Individuals registered for the Body Language Institute courses (sales executives, corporate
leaders, etc…)
BLI instructors
Anyone affiliated or working on a project with BLI
List of Program Stakeholders
Janine Marie Driver, CEO and President
Interns
BLI instructors
Information about the Evaluation
Purpose for the Evaluation
The purpose of this evaluation is to enhance the quality of the Body Language Institute’s
internship program operation. The evaluation will thoroughly assess the underlying
conceptualization and design of the internship program. The processes involved in carrying out
the internship will also be explored in order to help decrease the steadily increasing turnover rate.
The objective of the evaluation is to provide insightful information that the program officer will
use to re-conceptualize and restructure the internship program when it’s recommenced in January
2013.
Evaluation Focus or Type
Consistent with the program officer’s request, this formative evaluation aspires to provide
valuable information that will bolster the internship program to operate optimally; thus, meeting
the need of the institute and the interns in the program. Given the feedback that the interns are
not committing and no formal structure is in place to support the internship program, it will be
16
meaningful for the evaluation to focus on assessing the program’s theory and operation/process.
The evaluation will seek to assess the fundamental challenges by posing critical questions to
extract the underlying assumptions at work. Subsequently, the answers will serve as building
blocks for creating a sustainable internship program at the Body Language Institute.
The
evaluation seeks to gain a more thorough understanding of the interrelationship of the programs
and input from the program officer, interns, and the instructors will be analyzed.
Evaluation Questions
Questions about the internship program
Interview Questions for management
design


Does the Body Language Institute
have the infrastructure to support the
internship program?
 What resources are needed to
create the appropriate
conditions to house interns?
Are the interns selected a good fit for
the internship and is the institute a
good fit for the interns?
 Can selection criteria be
developed?
 Is the program theory of
creating a vibrant
environment adequate enough
to sustain the program?

What resources are in place to create
the appropriate conditions to house
interns?
 Please provide concrete
examples

Please describe your positive
experiences with the support that you
received from the internship program?

What processes are in place to bolster
the learning experiences of interns?
Interview Questions for interns

Please describe your positive
experiences in the internship program?

Please describe your satisfaction level
with the structure of the BLI internship
program?

Please describe the value you obtained
from participating in the internship
program?
Questions about internship program
operation/process

Does the Body Language Institute
have processes to effectively identify
the needs of each intern?
 Are the needs of the interns
being met?
 Is the dynamic nature of the
17
organization having an
adverse effect on the interns?
 Are the interns satisfied with
the internship program?
Methodology
The overall evaluation methodology will consist of four phases (1) Data collection (2)
Results, Interpretation, and Discussion (4) Implications (5) Recommendations and (6)
Conclusions with Appreciative Inquiry embedded in each phase.
Size of sample
The organization consists of approximately ten geographically dispersed members and
two interns. All of the twenty-seven interns who had at one time or another participated in the
internship program, as well as the current two, will receive an email to participate in the
individual interview, focus group, or survey according to their preference. Out of the ten
employees five will be randomly selected to participate in the individual interviews, and the
other five left will be asked to participate in the focus group. The Satisfaction with BLI scale
will be randomly administered at to members at all levels.
Appreciative Inquiry
This evaluation will begin by facilitating a focus group in which previous interns will
explore the collective images that are resident at the Body Language Institute’s internship
program by utilizing Appreciative Inquiry methodology. This appreciative model, as opposed to
18
deficiency models that approach organizations from a problem solving perspective, seeks to
explore what was working well in the internship program and focuses on heightening those
factors (Thatchenkery, 2010). Accordingly, Weick (1982) highlights strong “affirmations” are
likely to reveal defects than critiques. Appreciative Inquiry method begins with identifying what
is positive in an organization. It recognizes that an organization depends on its human capital to
bring its vision to life. It’s a methodology that acknowledges, the individual and collective
images its members hold, will be projected to create a reality (Thatchenkery, 2010). Appreciative
Inquiry intrinsically calls for the program evaluation to seek, identify, locate and heighten the
core values of the Body Language Institute’s internship program. This process of inquiry begins
and ends with valuing that which gives life to the institution; therefore, this evaluations goal is to
discover and illuminate the values that give life to the internship program and the organizational
factors that bolster it.
Through this collaborative process the interns will have an opportunity to collectively and
proactively explore what was working best in the internship program and while envisioning what
it ought to be. Consequently, they will begin to self-generate new knowledge that is applicable
and critical for this evaluation process. Furthermore, the program manager and the instructors of
the institute who have received support from interns will also engage in this affirmative process.
The purpose of this focus group is to unfold the shared purpose among the leadership in
implementing a cohesive internship program as it’s discussed in a meaningful and authentic
manner. The information extracted from the focus group and interviews will be pertinent in
reinforcing the alignment needed between leaders and interns.
Appreciative Interviews
19
As an evaluation method to answer the questions listed above, the appreciative focus
group will be followed by conducting individual interviews of key participants that were not in
the focus groups. The CEO, manager, and the instructors that are available will be randomly
selected and interviewed to provide deeper insights. The purpose of the interviews is to
contextualize progress, including successes and challenges in meeting the goals and deliverables
of the internship program in relationship to stakeholder expectations. It aims to identify factors
that are affecting the success and failures of the internship program overall. The interviews will
be transcribed. After transcribing the interviews a thematic analysis will be conducted.
Data Collection Devices/ Instruments
The data collection techniques that the evaluation plans to use include questionnaires,
surveys, interviews, observation, catalytic dialogue/conversation and, document analysis. The
instruments will also adhere to proven design techniques that demonstrate reasonableness and
expertise on the subject matter and infuse the client with confidence through the process. For
example, The Satisfaction with Life Survey is a widely used survey due to its “favorable
psychometric properties” (Diener, 1993), and has been adapted for the purpose of this evaluation
to reflect the satisfaction level of the internship program. The purpose of this survey is to gather
the level of satisfaction in the internship program. The distribution of satisfaction levels among
leadership and interns will provide valuable insight. It will be administered randomly to those
who have worked at some capacity with or in the Body Language Institute’s internship program;
willing to participate in this evaluation process. This preliminary diagnostic survey will be
critical in assessing the correlation between manager/ instructors and interns and conversely, the
satisfaction level of the individuals within the internship program. This survey will be
20
administered prior to participating in the focus group along with a consent form. Please view
Appendix A and Appendix B for the detailed instruments.
This evaluation seeks to thoroughly address stakeholder concerns by carefully exploring
the Body Language Institute’s Internship program descriptions and theory. The implications
from research and practice will be utilized to analyze the soundness of the of the program’s
theory description as a plan. The plausibility of the organizational component of the internship
programs process theory will be taken into great consideration through observations and
interviews. The magnitude of discrepancy between the program theory and reality will be easy to
identify in this collaborative process of filling in the gaps (Rossi et al, 2004). Additionally, the
evaluation will assess the lack of systematic review standards and the role of the many
competing demands in the internship program. The logic/ assumptions of the internship program
as intended and the actual current procedures in place have been amalgamated into a process
chart in order to effectively capture the reality of the institute. This chart is simply a pictorial
representation of some of the current logic, theories, and deficiencies guiding the BLI
internship program.
21
Input: High demands placed
on BLI contributed to the
development of the BLI
Internship program
Interns provide value by
increasing the efficiency of
the organization
All Interns are selected based
on course attendance and
interaction/sociability
Output:Interns are
positioned to assist in all
service areas supported by
the organization.
•Implicit theory: internship program will appease the
demands placed on the organization while giving interns
an opportunity to learn
•No formal program design, missions, goals, or objectives
are in place. Informal processes dictate the operation of
the organization
•Attractive advertisements are created. The interns
would be given the invaluable experience of
working with the NBC Today show Body Language
Expert and other media personnel while boosting
their resumes
•Fill in the gaps
•What organization factors are currently in place to
support the internship program? Are resources
limited?
•Decision to retain interns is made by the CEO
•No systematic selection processes in place
•No documents are signed. A psychological contract
is established between interns and CEO/ program
manager Janine Driver
•Fill in the gaps
•Defined roles and responsibilities, timelines,
policies, guidelines?
•The skills of interns vary:Lack of Knowledge, skills, and
abilities
•On the job training is the form of training available for
the interns
•Challenge
•Unintended result has been the steady turnover rate of
interns in the program
22
Body Language Institute Internship Program Logic Model
Activity/Output
Input:High
demands placed
on BLI to
manage all the
different service
areas
Recruit a large
number of
interns by
inviting them to
work with NY
Times Best
Selling Author
and NBC media
contributor
Interns will
recieve a free
scholarship to
develop their
nonverbal
communication
content
knowledge
The dynamic
enviroment
motivates
interns to apply
their KSA's in all
the service
areas supported
by BLI
Outcome:Interns
provide value by
increasing the
efficiency/
effectiveness of
the institute
23
Body Language Institute Internship Program Logic Model
Recruit a large body
of interns and
develop their
nonverbal
communication
content knowledge
Collective impact
of interns will
enhance the
efficiency and
effectiveness of
BLI
The interns will support in
media promotions, book
sales, BLI courses, website,
administrative tasks,
coordinate corporate
speaking engagements
Does content
knowledge necessairly
sustain interns to
perform in these
service areas? Do they
have the KSA's?
24
Procedures
As described in the methods section of this report one of the methods used for gathering
data was to conduct an interview. Twenty-five previous interns and two current interns were sent
an email requesting their participation. However, only one was responsive to the email.
Therefore, they were contacted via phone; and only seven were responsive and willing to
participate in this interview. Due to time constraints leaders and interns were unavailable or not
willing to come together for a focus group. A total of seven appreciative interviews were
conducted via phone; six previous interns and once current intern. Although, initially scheduled
to complete the interview, the CEO of the company opted out of the interview. As a project
manager of the internship program she felt that the information that the interns provided is much
more valuable for the purposes she seeks to accomplish. In this evaluation there had been a
previous coworker relationship between the evaluator and the interviewees. Accordingly, Descombe
(2003) cautions about "the interviewer effect, where research on interviewing has demonstrated
fairly conclusively that people respond differently depending on how they perceive the person asking
the questions” (p.169). The evaluator remained sensitive as to how this relationship might influence
the interns’ likeliness to openly explore issues during the interview. Accordingly, former BLI interns
were fully informed that their names would be kept confidential. Thus, allowing the evaluator to
speak candidly with the interns in a safe environment.
Results
Data Analysis
The appreciative interviews allowed interns to express their experiences and what they
hope would be improved at the Body Language Institute in a positive light. By meticulously
25
analyzing general patterns of answers, narratives, and articulated statements the evaluator began
to search for themes. Through a funneling process, broad themes were defined and categorized
according to their relevance. At the command of Appreciative Inquiry data analysis is typically
conducted using qualitative methods of Thematic Analysis. Accordingly, Braun and Clarke
(2006) articulate that Thematic Analysis "provides a flexible and useful research tool, which can
potentially provide a rich and detailed, yet complex, account of data” (p.78).
Data analysis consisted of classifying data by categorizing similar themes into categories by
analyzing the story of the questions presented. The weaker thematic categories were identified
and their respective narratives were further analyzed. During this process, the following thematic
categories were surfaced as weaker and needed further analysis. All of the interviews were
analyzed using the thematic categories defined according to this model. During the initial
qualitative data analysis of the interviews, 21 thematic categories surfaced; which translated into
nine unique thematic categories. The themes will be discussed in-depth according to research and
how they will best support BLI organizational context.
This chart below displays the proportion of the nine thematic categories that were surfaced
and grouped together during interview data analysis:
26
Themes
Role validation
Transparency
Recognition
Incentives
Specialty
Support
Commitment
Communication
Flexibility
Experience
Professionalism
Next, thematic categories were grouped to form high-level aggregated categories. After
careful qualitative analysis, the five thematic categories were reduced / consolidated into three
higher-level aggregated categories. This shows the frequency of the thematic categories that
were grouped to form high-level aggregated categories. These higher-level themes were finally
identified as the core values identified by previous interns of the internship program.
Aggregated Category
Core Values
Category 1
Role Validation/Appreciation
Frequency (# of times it
appeared in the
interview)
18
Category 2
Transparent Leadership
13
Category 3
Professional Structure
09
Results revealed that interns Core Values consisted of:
27

Being appreciated, recognized, affirmed. The interviews suggest being able to
take trips, verbal affirmations from the CEO/ faculty, informal emails giving them
kudos contributed to their positive experiences at BLI.
o I was supper happy that I got to go to New York in the green room at
NBC. That was a fun experience that made me feel special.

Knowing what was going on in the institute from leaders made interns feel more
connected to BLI. For example one participant states;
o “I enjoyed that we would have one on one conversations and plan the
week and major events, but would have liked for that to be the case at all
times…”

Interns appear to enjoy the unofficial office environment where dress code and
work hours are extremely liberal.
o “I liked having a flexible schedule, but it was odd not filling out a time sheet
and being there for over 9 hours a day past midnight”
Interpretation and Discussion
Role Validation/Affirmation
At the Body Language Institute where interns are unpaid their desire to be affirmed in
their role is clearly confirmed when looking at the Role Validation breakdown in the Data
Analysis portion of this report. Boulding (1966) remarks, “One of the basic theorems of the
theory of image is that it is the image which in fact determines what might be called the current
28
behavior of any organism or organization. The image acts as a field. The behavior consists in
gravitating toward the most highly valued part of the world” (p. 155). Interviews revealed that
several interns who were provided with an opportunity to travel or attend a social media event
felt appreciated. Perhaps, the interns have a need to feel affirmed by their supervisors and that in
turn provides them with an image of themselves that is internally congruent with the image of the
organization and his/her leadership. Interns want to know that they are providing value and
fulfilling their positions well. As the elaborate expression states “valuing is creating: hear it, ye
creating ones! Valuation is itself the treasure and jewel of valuating things” (Rader, 1973, p. 12).
By sustaining a culture that nurtures valuation the institute will increase living out that reality.
In his article Determinants of Job Satisfaction the author asserts, “positive feedback and
emotion result from information, feedback, and situations that affirms or reinforces the
individual's self-worth and self-concept, while negative affect is evoked by invalidating
situations. Self-worth is validated when individuals feel accepted as valuable members of the
organization and their competencies and core values are affirmed. When individuals are in a
positive affect state while working, they are likely to evaluate the organization positive manner”
(Scholl, 2003). Recognition plays a dual role in this matter. The appreciation and recognition
system most alive in the Body Language Institute enhances the success and motivation within its
interns. One intern clearly stated, “It was nice that I got recognized in front of everyone and got
treated for lunch because I was able to successfully coordinate the FOX interview via Skype on
such short notice.” There is a win-win benefit both for BLI and the interns being validated in
their role. Scholl (2003) states, “The campus carries out its mission through the individual and
collective contributions of its employees. To do their best, staff members need to know that those
contributions will be recognized and acknowledged.” The interviews reveal that something
29
personal and specific is very meaningful to them. Especially, since they are not receiving
financial compensation for the work that they are completing. This revelation cannot be
emphasized enough as it was central to the responses provided by interns. Accordingly, Harris
(2010) declared the need to consistently provide positive feedback by telling interns how their
contribution is significant and giving them a sense of purpose so that they feel fulfilled.
The evaluator also discovered a strong positive correlation between autonomy and role
affirmation. Several interns revealed the concept of autonomy reoccurring. In their article, Job
autonomy and job satisfaction: new evidence the authors argue that autonomy is expected to be
associated with greater job satisfaction as individuals are given liberty to determine their own
effort and work (Nguyen, 2003). It’s evident that not being micromanaged made BLI interns
feel that their supervisor trusted them to perform their tasks.
Transparent Leadership
The interns revealed their admiration and desire for transparency from leadership.
Whether being honest about the ambiguity of implementing a new process, providing clear
expectations about the internship, and having an open dialogue about how to best develop the
interns skills could have contributed to making the them feel connected. One individual
expressed being more invested as a result of taking part in these processes. Accordingly, Kerfoot
(2004) avows, transparent leaders are motivated to honestly disclose and to enter into extensive
dialogue to explain issues/decisions, and to the best of their ability. Unfortunately, because there
are no formal procedures e.g. procedures in place to govern the internship program interns
expressed feeling “neglected,” “in the dark,” “uncertain/unaware” about what was generally
going on. Limited examples were available of feeling incorporated into what was going on in the
leadership. For example one intern who has been there for 8 months stated, “it was normal not to
30
know what was going to take place I sort of adapted to finding things out on the fly….it was nice
knowing when what was going to take place.” To enhance the level of transparency among
interns Baum, (2005) highlights “The first step in the process of creating a transparent
organization is to define what it stands for in writing. Once it’s set in black and white, you’ll
have the road map you need to move forward. A cultural mission statement helps you
communicate your core values in a way that’s not preachy or imposing, and if the employees
help create it; it becomes a lifestyle rather than a mandate from above” (Baum, 2005). BLI
internship is in its earliest stages therefore applying Baum’s advice would support the institute in
creating a sustainable internship program. To have such an organizational infrastructure would
heighten the effects of this and all the Life Giving Forces discovered. When leadership is all
inclusive interns sense it and are likely to display positive citizenship behavior. Setting up the
right organizational conditions is a key to enhancing the core values. It interconnects in a
convoluted manner to create a life giving dynamic.
Professional Structure
Scholar Ann Baker in Catalytic Conversations declares, “Organizational contexts must
be intentionally created; it must be congruent with and not contradict the organizational purpose
and the people involved” (p. 97). Some of the interns’ ephemeral stay leads one to question the
fragmented culture of the organization. Some of the interns revealed that they would appreciate
consistent information and hinted their frustration about being told one thing and clearly
experiencing another. “When words and behavior are congruent, the authenticity of the
environment is palpable. There must be congruence between broad messages and daily behavior”
(p. 96-97). One interns who left in two weeks expressed he heard words and experienced the
incongruence and therefore left the institute in great dissatisfaction (Baker, 2010). Leadership
31
has great influence and responsibility in creating a comfortable atmosphere where relevant
information concerning members can surface. The role of leadership in shaping organizational
culture is immense. “A focus on the links between leadership style and corporate culture often
provides key insights into why organizations work the way they do” (p. 131). Is the theory or
idea of getting interns a merely “short term intervention” to get help quick? Whether BLI has the
infrastructure to support interns has to be taken into account. In Good Communication that
Blocks Learning the author Chris Argyris emphasizes that, “asking one dimensional question to
elicit one-dimensional answer is single loop learning. A double loop process might ask why
getting a horde of interns as a opposed to one or two full time employees was chosen in the first
place; seeking to understand the reason and motive behind the objective facts (Argyris, 2000).
One interns commented, “I liked having a flexible schedule but it was odd not filling out a time
sheet and being there for over 9 hours a day past midnight.” Structuring the institute to best serve
the interns requires asking these multidimensional questions such as: How can the organization
be best equipped in order to optimally facilitate the internship program?
The institutional structure needs to foster the sort of mentorship and guidance needed to
help interns achieve the work at hand. The interns’ desire to serve the company is evident in that
he put in between 20-60 hours a week. Such dedication from members is vital information for
the organization to acknowledge. Argyris authentically captures these occurrences as he
highlights, “But their behavior and the behavior of the CEO in ignoring this dimension of the
problem-combined with everyone’s failure to examine his or her individual behavior blocked the
kind of learning that is crucial to organizational effectiveness (Argyris, 2000). Furthermore,
author Ann Baker in Catalytic Conversations articulates, “the challenge therefore is to create
more congruent organizational environments to support the verbalized priorities, values, and
32
missions” (p. 97). Engaging interns in an, “ongoing dialogue, which is the medium through
which people work through differences, short comings, mistakes, and misunderstandings” could
in the future preserve their longevity with the institute (Baker, 2010 p. 99).
Recommendation
In order to maintain an appreciative climate organizational factors must be intentionally
and positively correlated with life giving forces. According to the core values, perhaps as Rossi
et al. (2004) contend the programs theory can be can be reconceptualized by clarifying goals and
objectives and restructuring program components to increase the longevity of inters by
incorporating core values (p.165). The authors highlight the need to create a well-articulated
credible program theory (Rossi et al., 2004).
The organizational factors include organizational structure, leadership, organizational
practices, communication, incentives, and decision-making. Accordingly, Appreciative Inquiry
intrinsically calls for BLI to incorporate role validating, transparent leadership, and professional
structures by creating a new framework that is supported by organizational infrastructure factors
and these life-giving-forces (core values). AI methods aim to find ways that will enhance the
identified core values so that they become continuous, sustainable, long-term activities at BLI
(Thatchenkery, Strategic, Appreciative Sharing of Knowledge: Leveraging Knowledge
Management for, 2005, pp. 15, 16). Thus, creating an appreciative environment has the potential
to enhance BLI’s internship program. The image below captures the dynamic between the
organization and the core values.
33
Life giving forces
(core values)
Body Language Institute
infrastructure
Interrelationship between Core Values and Organization Factors. Key interrelationships cannot exist without both being
present.
Limitation
The qualitative analysis was limited to seven interns. Furthermore, calibrating the data
analysis against the needs of the client BLI was challenging at times because the evaluator
recognized that different approaches could have potentially been used. This experience was both
a challenge that required the researcher to stretch beyond comfort zone and yet provided an
impactful learning experience.
34
Evaluation Team Qualifications
Ms. Merkebu has significant research experience in organizational behavior, where she explored
diverse constructs that foster collaboration, increase effectiveness, and decrease
counterproductive performance in the work place. Ms. Merkebu has been trained in Analytical
Interviewing to obtain accurate reading and interpretation of verbal and non-verbal
communication, recognize masked emotions, use techniques to detect deception, identify microexpressions, and establish rapport with others. As a reflective practitioner, she communicates at
an exceptional capacity across professions, paradigms, and cultures.
Qualifications for Evaluation Organization: George Mason University
35
Logistics
Evaluation Timeline
Task
Deliverable/notes
Responsible
Date
Project Initiation: Initial program evaluation meeting with the
program manager, Janine Driver, mutual agreement on key
points
Essential documents were provided for evaluation via email for
conducting document analysis
Program evaluation request of Train the Trainer program
retention rate was placed by the program manager
The program manager put in a request to change program
evaluation from Train the Trainer to BLI internship program
Discussion of progress/ Interview Questions developed
N/A
JM
09/ 05
N/A
JM
09/12
N/A
JM
09/14
N/A
JM
10/15
N/A
JM
10/19
Focus group recruitment attempt
Pending status
JM
10/27
Interview with Participant A and B
Interview with the Participant C
Interview with Participant D, E, and F
Interview with program manager/CEO
Interview with Manager
Identification of Life Giving Forces and expansion of BLI
organizational factors
Conducting Thematic analysis
Consensual validation of propositions
Interpretation/discussion: creating an implementation plan
Recommendation and Implementation: Final report will be
delivered to the program manager, (the only material the client
or program manager is interested in receiving is the final report
that reveals the gaps and recommendations for enhancing the
quality of the internship program.)
N/A
N/A
N/A
Declined
N/A
N/A
JM
JM
JM
JM
JM
JM
11/02
11/08
11/12
12/03
10/26
12/10
N/A
N/A
JM
JM
Hard copy of final
report
JM
12/11
12/13
12/15
12/28
36
References
Argyris, C. (1994). Good Communication That Blocks Learning. Harvard Business Review
Baker, A. (2010). Catalytic Organizations: Organizational Communication and Innovation. New
York: M.E. Sharpe, Inc.
Baum, H. (2005). Transparent Leadership. Retrieved from
http://ehis.ebscohost.com.mutex.gmu.edu/ehost/pdfviewer/pdfviewer?vid=8&hid=23&si
d=eaf5fb68-5b9a-4556-aad1-8a3f6e7cd242%40sessionmgr112
Braun, V. and Clarke, V. (2006) Using thematic analysis in psychology. Qualitative Research in
Psychology, 3 (2). pp. 77-101.
Denscombe, M. (2003). The Good Research Guide (2nd ed.). Maidenhead: Open
University Press.employee. Retrieved 12 8, 2010, from URI College of
BusinesAdministration :ftp://ftp.cba.uri.edu/Classes/Beauvais/HPR412/Ramlall_2004.pdf
Kerfoot, K. (2004). The transparent organization: leadership in an open organization. Nursing
Economics, 22(1), Retrieved from http://ehis.ebscohost.com.mutex.
gmu.edu/ehost/pdfviewer/pdfviewer?vid=4&hid=115&sid=eaf5fb68-5b9a-4556-aad18a3f6e7cd242%40sessionmgr112
Nguyen, A., Taylor, J & Bradley, S. (2003) Job autonomy and job satisfaction: new evidence
Retrieved from http://ideas.repec.org/p/lan/wpaper/000192.html
Rossi P, Lipsey M, Freeman H Evaluation: a systemic approach (7th ed ) Thousand Oaks (CA): Sage;
2004
Scholl, W. R. (2003) Determinants of Job satisfaction Schmidt Labor Research Center
37
Thatchenkery, T. (Fall 2010). A Work-Book to Appreciative Organizational Analysis. Fairfax,
VA, USA: George Mason University.
Weick, K.(1982). Affirmation as inquiry. Small Group Behavior, 13, 441-442
38
Appendix A
Satisfaction with BLI Internship Scale
The SWLS is a short, 5-item instrument designed to measure global cognitive judgments of
one's lives. The scale usually requires only about one minute of respondent time. The scale is not
copyrighted, and can be used without charge and without permission by all professionals
(researchers and practitioners). The scale takes about one minute to complete, and is in the public
domain. A description of psychometric properties of the scale can be found in Pavot and
Diener’s (1993) Psychological Assessment.
Survey Form
Below are five statements that you may agree or disgree with. Using the 1 - 7 scale below
indicates your agreement with each item by placing the appropriate number on the line preceding
that item. Please be open and honest in your responding.







7 - Strongly agree
6 - Agree
5 - Slightly agree
4 - Neither agree nor disagree
3 - Slightly disagree
2 - Disagree
1 - Strongly disagree
____ In most ways the internship program at the Body Language Institute is close to
my ideal.
____ The conditions of the internship program are excellent.
____ I am satisfied with the Body Language Institute Internship program.
____ I have gotten the important things I want from the internship program.
____ If I could participate in the internship program I would change almost nothing.
Appendix B
INFORMED CONSENT FORM
RESEARCH PROCEDURES
This evaluation is being conducted in order to enhance the quality of the Body Language
Institute Internship program. The questionnaire and focus group will provide a better
understanding of the interns and managements perspectives about the conditions of the internship
program. If you agree to participate in this evaluation, you will be asked to fill out a short survey
and participate in a focus group. You will fill out the survey via the Internet, phone or mail after
participating in the focus group.
RISKS
There are no foreseeable risks for participating in this research.
BENEFITS
Your participation in this evaluation will provide valuable insight that will be taken into great
consideration when reconstructing the Body Language Internship program for optimal
performance.
CONFIDENTIALITY
The data in this study will be confidential. As soon as we enter your responses into a computer,
the surveys will be destroyed. All data will be stored securely on one computer to which only the
primary researchers will have access and no individual results will be released. A
PARTICIPATION
Individuals who were previously and are currently in the Body Language Institute Internship
program may participate. Your participation is completely voluntary, and you may withdraw
from the study at any time for any reason.
If you choose to participate in this study, please sign and date below.
CONSENT
I have read this form and agree to participate in this study.
Printed Name _____________________________
Signature
_____________________________
Today’s Date _____________________________
40
41
Download