What’s Happening to Our
“Freshout” Engineers?
 Steven W. Villachica
 Anthony W. Marker
 Donald Plumlee
 Amy Chegash
Project Worldwide Out
of the Box
https://sites.googboisestate.edu/faculty/sv
illachica.htmle.com/a/boisestate.edu/ieeci/
e2r2p
2
Engineering Education
Research to Practice (E2R2P)
Portions of this material are
based upon work supported by
the National Science Foundation
under Grant No. 1037808.
The Research Team
 Don Plumlee, PhD.
 Steve Villachica, PhD.
 Tony Marker , PhD.
Any opinions, findings, and
conclusions or recommendations
expressed in this material are
those of the author(s) and do
not necessarily reflect the views
of the National Science
Foundation.
 Linda Huglin, PhD.
 Shannon Rist
 Amy Chegash
 Lorece Stanton
 Jessica Scheufler
3
Agenda
 Share our research
 Ask for your help interpreting the data we’ve
collected
 Wrap up
4
Why Should You Care?
 64% engineering employers are “somewhat
satisfied” with quality of new hires.
(Trevelyan & Tilli, 2008; Trevelyan, 2010; Blom & Sakei,
2011)
 Professional skills for the engineering
workplace include teamwork, communication,
coordination, data analysis and problem
solving.
(Hoey & Gardner, 1999; Jonassen et al., 2006; Grant &
Dickson, 2006; Korte, Sheppard, & Jordan, 2008; Trevelyan,
2007, 2008; Borrego & Bernhard, 2011; Passow, 2012; ASEE
& NSF, 2013)
5
Research Design
Communication, teamwork, networks, critical thinking, global understanding,
perspective, organizational culture, project management, etc.
Many Disciplines ME Many Systems
Deep at least one system
Deep at least one discipline
What sort of
engineers do
engineering
firms REALLY
want to hire?
Boundary Crossing Competencies
(c.f. Brown, 2005; Spohrer, 2010;
ASEE & NSF, 2013)
6
Literature Review
Company Costs
Desired
Competency
Performance
Promotion!
Actual
Competency
New Task/Project
Leave University/Enter Workforce
$
$
$
$
$
$
$
Training
Errors
Mentoring
Salary
Opportunity
Other projects
Others?
Time
REDUCE
CO$T
{
}
• Improve Starting Skills
• Change Performance Curve
• Make Boundaries Porous
7
Literature Review
There is a significant disconnect between
engineering education and engineering practice.
(Bucciarelli & Kuhn, 1997; NAE, 2005; Jonassen et al., 2006;
Spinks et al., 2006; Korte et al., 2008; Trevelyan, 2008, 2010;
McCrohon & Gibson, 2009; Sheppard et al., 2009; Morgan &
O’Gorman, 2010; Anderson et al., 2009, 2010; Duderstadt,
2010; Stump et al., 2011; ASEE & NSF, 2013; Winters et al.,
2013)
8
Literature Review
The time for freshout engineers to fit into their
jobs and perform them competently is a
significant workplace cost.
 2 to 5 years ramp-up.
(Trevelyan & Tilli, 2008; Jonassen et al., 2006)
 Socialization and onboarding are long-lived.
(Dai & De Meuse, 2007; Bradt & Vonnegut, 2009; Roethle,
2012; Jones, 2013)
 The engineering workplace supports
socialization and onboarding variably well.
(Montesano, 2007; Roethle , 2012; Korte & Lin, 2013)
 Bad onboarding is co$tly.
(Ramlall, 2004; Rollag et al., 2005; Snell, 2006; Kowtha,
2008; Lindo, 2010; Roethle, 2012; Korte & Lin, 2013)
9
Literature Review
Academics, industry, and government agencies
own this shared problem, and it requires a
systemic solution. Unfortunately, we know little
about
 Engineering practice for freshout engineers.
(Kowtha, 2008; Trevelyan, 2007, 2008; Brunhaver et al.,
2010, in press; Winters et al., 2013)
 What freshout engineers do successfully and
unsuccessfully in the workplace.
(Trevelyan & Tilli, 2008; Trevelyan ,2008, 2009)
 Barriers to desired workplace performance.
(Korte et al., 2008; Atman et al., 2010; Brunhaver et al.,
2010, in press; Anderson et al., 2010)
E2R2P is an opportunity to collaborate systemically to decrease
ramp-up time to competent performance.
10
Research Design
Decrease Ramp-up Time to Competent Job
Performance in the Engineering Workplace
Research Questions
• What are newly graduated and hired “fresh out” engineers doing/not
doing in the workplace that they should?
• What are the consequences of performance/non-performance in the
workplace?
• What are the root causes of workplace nonperformance?
Mixed Design: Focus Groups & Surveys
• Engineering managers, engineering leads, HR personnel, and technical
scientists who work with fresh out engineers
• Fresh out engineers
• Professional engineering organizations
11
Thanks to Our Professional
and Industry Sponsors!
 Practicing
engineers at ISPE
 BSU COEN
Advisory Council
 Focus Groups at
local engineering
firms
12
Method
Practicing Engineer Survey
Short survey measuring:
 Types of work assigned to
freshouts.
 Typical time to competence
 Costs and risks that
organizations incur when
freshouts can’t perform to
standards
 Typical project organization
for groups of engineers
 Organizational support for
freshouts
ISPE (2012), n = 23
13
Time to Competency for
"Fresh-Outs" (in months)
10
2
Six to Nine
3
Ten to Twelve
4
4
Thirteen to
Sixteen
Seventeen to
Twenty-Four
More than
Twenty-Four
14
Typical Assignment Size
Large/Complex Projects
Small/Simple Projects
6
2
2
Design
Analysis
2
Drawing
Drafts
Project
Management
19
19
14
3
Design
Analysis
Drawing
Drafts
Project
Management
15
Organizational Support
20
15
9
11
8
10
3
Formal
Training
Formal
Mentoring
Informal
Mentoring
Orientation Performance
Feedback
SOPs
Other
16
Method
Focus Groups
Collect Incidents
Critical Incident Technique
(Flanagan, 1954)
Nominal Group Technique
(Delp et al., 1975)
Generate Categories
Negotiate Categories
Select Incidents of
Non-performance
Rank Categories
Group Incidents
Under a Root Cause
Cause Analysis
Group Incidents
under Categories
17
Collect Incidents
Critical Incident Technique
(Flanagan, 1954)
18
Focus Group Results
Performance Categories
Category
Communication
Technical
Fundamentals
Unsuccessful
9
Successful
8
3
11
Business
Systems
Design
Motivation
Problem Solving
Initiative
Positive Attitude
Work Ethic
Circuit Debug
5
7
5
6
6
2
3
2
3
6
5
5
8
4
5
3
Freshout-Defined Categories
n = 10
Competency
Unsuccessful Successful
Design
14
18
Communication
18
10
Analysis
10
14
Motivation
8
10
Technical
3
12
Fundamentals
Problem Solving
7
6
Software
3
10
Business
5
7
Systems
Initiative
2
8
Leadership
4
5
Process
4
3
Knowledge
Positive Attitude
3
4
Manager-Defined Categories
n = 20
19
Focus Group Results
Consequences
Successful Performance
# of Events
Consequences
Met Schedule
19
Freed up Senior Staff
18
Saved Resources
18
Improved Processes
15
Developed New Tool
14
Developed Skill / Knowledge
12
Saved Time
12
Gained Client Confidence
10
Gained Employer Confidence
10
Stayed Within Budget
7
Increased Productivity
6
Unsuccessful Performance
Consequences
Lost Time
Rework
Additional Staff Support
Missed Deadline
Increased Costs
Wasted Budget
Lost Employer Confidence
Stressed Staff
Job Unfinished
Lost Client Confidence
Exceeded Budget
# of Events
34
22
17
12
10
8
5
5
4
4
3
20
Categories, Activities and
Consequences
Top-Ranked Categories
Source
1. Communication /
Teamwork
Freshout
“The first time I had to write up an
engineering report--I struggled doing it. I
never had to submit something to IDQ
before, and I wasn’t confident”
“It just took me a lot of my personal time
and a senior had to review it and it wasn’t
good.”
2. Skill Development / Learning
Freshout
“I observed a lot of nuclear test procedures
and started evolutions on board and naval
vessels so after you learned we went out to
the vessels and basically were like buddy
buddy with a qualified test engineer to learn
the procedures and observe what was going
on…”
“I think I was pretty successful. You had to
regurgitate what you learned. You had a
qualification check off sheet and so you’d have
to go back with that test engineer and
regurgitate what you saw and how you thought
the procedure went then he would sign off
your sheet and you’d go on to the next.”
3. Work Ethic
Manager
“We had a junior engineer. I think he had
only been here a couple of weeks. Got him
onto a team for ongoing projects and gave
him minimal amount of direction…”
“I didn’t consider that activity [resolving
drawing issues] successful because I was
not aware or did not have enough
knowledge about manufacturing processes
in general to really be effective and resolve
those issues in a short time span. It took
me a lot longer than it could have...”
“…He immediately started contributing more
than I think any of us envisioned…within a
few days he was coordinating with several
other staff inside and outside our company.”
“As a result it there were a lot of late
deliveries to those revisions and caused us to
do multiple rework and multiple parts…”
4. Business Systems
/ Processes
Freshout
Activity
Consequence
21
Categories, Activities and
Consequences (con’t)
Top-Ranked Categories Source
Activity
5. Problem Solving
Manager
“There was an issue that was found on the
floor and it was in the CAB. The CAB group
lead came to this person [the new engineer]
and asked him to go figure out what the
problem was.”
6. Analysis
Freshout
“The engineer that was doing it [analysis] had
a lot going on. Field work and stuff so [he]…
passed it off to me. “
7. Technical
Fundamentals
Manager
“…basically he [the new engineer] didn’t know
how to use the software and didn’t have the
general, multiple CAD system-type training.
8. Design
Manager
“I had a young engineer who was tasked with
developing a draw bar for semi trailers.”
Consequence
“That person went out there, they figured it
out, they investigated it…found out that it
was in fact a design error, and they went and
took care of it…ended up fixing the design
error correctly…It allowed production to keep
going and we had minimal down time”
The biggest impact was probably time and
cost for doing it slower than he was or
would’ve and … a couple hours of
coaching…So there was a learning curve to
it..”
“He was focused all on one CAD software when
he was in school and so although he could
model in 3-D, he couldn't psych out how this
software thought and how it behaved and that
it was different than what he had done
before…”
“He [the new engineer] was able to, in about a
four-month period, generate a single CAD
model that could then be driven through a
family table to automatically generate new
designs and drawings. He took a typically 8
hour to 12-hour project down to 30 minutes.
The impact was huge.”
22
Categories, Activities and
Consequences (con’t)
Top-Ranked Categories Source
9. Software
11. Leadership
Freshout
Activity
“We had switched over to this other software
and for 9 months I don’t think we produced
any usable products. “
[When I joined the design team] “I was
immediately put in charge of an entire CAB
design for one of our contracts. Essentially
what that entails is facilitating the whole design
process.”
Consequence
“.Literally everything we did in the software for
six months was never used. It was never
useable. Had to be trashed. It was an
extremely frustrating part of my career. “
“I don’t feel like I was ill prepared to do that.”
23
Root Cause Analysis
PERSON
ENVIRONMENT
Instrumentation
INFORMATION
TOOLS
MOTIVATION
•
•
•
•
Data
Expectations
Feedback
Standard Operating
Procedures
•
•
•
•
Resources
Software
Tools
Support
•
•
•
Incentives
Rewards
Consequences
•
•
Knowledge
Skills
•
•
•
•
Physical Capacity
Mental Capacity
Flexibility
Resilience
•
•
•
•
Motives
Affect
Work Habits
Drive
24
Root Cause Analysis
Results
Freshouts
Managers
11
Information
10
2
0
8
1 2
6
Tools
Incentives
7
Knolwedge
Capacity
24
Motivation
22
0
25
Root Cause Analysis
Results
Dean (1997)
Combined
13
Information
16
Tools
8
9
0
Incentives
Knolwedge
Capacity
46
Motivation
8%
6%
35%
11%
11%
29%
26
Limitations
 Validity and reliability of the Practicing
Engineer Survey is unknown.
 Small exploratory study using a convenience
sample of local engineering firms.
 No post-focus group data checking with
participants and their company sponsors
(managers).
27
Conclusions
 Decreasing time to engineering workplace




competency is a shared problem.
Freshout engineers are variably prepared to
enter the workplace.
The engineering workplace supports freshout
performance variably well.
Socialization and onboarding involves a lot of
self-reported learning.
We don’t know about the extent to which
fixing the workplace environment and
introducing it to students sooner would
decrease ramp-up time.
28
Next Steps
 Seek funding to expand research.
– Include other engineering populations.
– Regional, national, or international sample.
– Scale up and automate processes.
 Investigate research questions about:
– Blurring traditional academic and industry
boundaries.
– The extent to which a smarter workplace
environment introduced in academics could
decrease ramp-up time.
29
Summary
 Research questions and importance
 Literature review
 Method
 Results
 Limitations
 Conclusions
What are your lessons
learned?
How might you apply
them back on the job?
30
References
Anderson, K., Courter, S., McGlamery, T., Nathans-Kelly, T., & Nicometo, C. (2009, June). Understanding the
current work and values of professional engineers: Implications for engineering education. Paper presented
at the American Society for Engineering Education, Austin, TX.
Anderson, K.J.B., Courter, S.S., McGlamery, T., Nathans-Kelly, T.M., & Nicometo, C.G. (2010). Understanding
engineering work and identity: A cross-case analysis of engineers within six firms. Engineering Studies,
2(3), 153-174. doi: 10.1080/19378629.2010.519772
ASEE & NSF. (2013). Transforming undergraduate education in engineering: Phase I--Synthesizing and
integrating industry perspectives. Arlington, VA: American Society for Engineering Education. Retrieved
from http://www.asee.org/TUEE_PhaseI_WorkshopReport.pdf
Atman, C.J., Sheppard, S.D., Turns, J., Adams, R.S., Fleming, L.N., Stevens, R., . . . Lund, D. (2010). Enabling
engineering student success: The final report for the center for the advancement of engineering education.
San Rafael, CA: Center for the Advancement of Engineering Education. Retrieved from
http://www.engr.washington.edu/caee/CAEE%20final%20report%2020101102.pdf
Blom, A., & Saeki, H. (2011). Employability and skill set of newly graduated engineers in India. The World Bank
South Asia Region Education Team. Retrieved from
http://elibrary.worldbank.org/docserver/download/5640.pdf?expires=1337278148&id=id&accname=guest&
checksum=E557A65C13E9CA650433F9E5798C4242
Borrego, M., & Bernhard, J. (2011). The emergence of engineering education research as an internationally
connected field of inquiry. Journal of Engineering Education, 100(1), 14-47.
Bradt, G.B., & Vonnegut, M. (2009). Onboarding: How to get your new employees up to speed in half the time.
Hoboken, NJ: John Wiley & Sons.
Brown, T. (2005). Strategy by design. Fast Company, (June 1). Retrieved from
http://www.fastcompany.com/52795/strategy-design
Brunhaver, S., Korte, R., Lande, M., & Sheppard, S. (2010, June). Supports and barriers that recent engineering
graduates experience in the workplace. Paper presented at the American Society for Engineering Education,
Vancouver, British Columbia.
Brunhaver, S.R., Korte, R., Barley, S.R., & Sheppard, S.D. (in press). Bridging the gaps between engineering
education and practice.
Bucciarelli, L.L., & Kuhn, S. (1997). Engineering education and engineering practice: Improving the fit. In S. R.
Barley & J. E. Orr (Eds.), Between craft and science: Technical work in the United States (pp. 210-229).
Ithica, NY: Cornell University Press.
Dai, G., & De Meuse, K.P. (2007). A review of onboarding literature. Minneapolis, MN: Lominger Limited.
Retrieved from http://boardoptions.com/onboardingevidence.pdf
31
References
Dean, P.J. (1997). Thomas f. Gilbert, ph. D: Engineering performance improvement with or without training. In
P. J. Dean & D. E. Ripley (Eds.), Performance improvement series: Vol. 1. Performance improvement
pathfinders: Models for organizational learning systems (pp. 45-64). Silver Spring, MD: International
Society for Performance Improvement.
Delp, P., Thesen, A., Motiwalla, J., & Seshardi, N. (1977). Nominal group technique. In P. Delp (Ed.), Systems
tools for project planning (pp. 14-18). Bloomington, IN: International Development Institute.
Duderstadt, J.J. (2010). Engineering for a changing world. In D. Grasso & M. Burkins (Eds.), Holistic engineering
education: Beyond technology (pp. 17-35). New York, NY: Springer. Retrieved from
http://milproj.dc.umich.edu/pdfs/2009/Engineering%20for%20a%20Changing%20World.pdf
Flanagan, J.C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327-358. doi:
10.1037/h0061470
Grant, C.D., & Dickson, B.R. (2006). Personal skills in chemical engineering graduates: The development of
skills within degree programmes to meet the needs of employers. Education for Chemical Engineers, 1(1),
23-29. doi: http://dx.doi.org/10.1205/ece.05004
Hoey, J.J., & Gardner, D.C. (1999). Using surveys of alumni and their employers to improve an institution. New
Directions for Institutional Research, 1999(101), 43-59. doi: 10.1002/ir.10103
Jonassen, D., Strobel, J., & Chwee Beng, L. (2006). Everyday problem solving in engineering: Lessons for
engineering educators. Journal of Engineering Education, 95(2), 139-151. doi: 10.1002/j.21689830.2006.tb00885.x
Jones, K. (2013). Virtual onboarding for today’s global workforce. Oakland, CA: Bersin by Deloitte. Retrieved
from http://communication.on24.com/virtual-onboarding-for-todays-global-workforce
Korte, R., & Lin, S. (2013). Getting on board: Organizational socialization and the contribution of social capital.
Human Relations, 66(3), 407-428. doi: 10.1177/0018726712461927
Korte, R., Sheppard, S., & Jordan, W. (2008). A qualitative study of the early work experiences of recent
graduates in engineering. Paper presented at the American Society for Engineering Education, Pittsburgh,
PA.
Kowtha, N.R. (2008). Engineering the engineers: Socialization tactics and new engineer adjustment in
organizations. IEEE Transactions on Engineering Management, 55(1), 67-81. doi:
10.1109/TEM.2007.912809
32
References
Lindo, D.K. (2010). New employee orientation is your job! SuperVision, 71(9), 11-15.
McCrohon, M., & Gibson, P. (2009, December). Student experiences in the direct applicability of their
engineering education to professional practice. Paper presented at the 20th Annual Conference for the
Australasian Association for Engineering Education, Adelaide, Australia.
Montesano, A. (2007). Orienting new employees for career success: Effective onboarding ensures smoother
integration and faster productivity. Canadian HR Reporter, 20(18), 23-24.
Morgan, M., & O’Gorman, P. (2010, July). Developing industry-ready engineers: A regional university
perspective. Paper presented at the International Conference on Engineering Education ICEE-2010, Gliwice,
Poland.
National Academy of Engineering. (2005). Educating the engineer of 2020: Adapting engineering education to
the new century. Washington, DC: National Academies. Retrieved from
http://www.nap.edu/catalog.php?record_id=11338#toc
Passow, H.J. (2012). Which ABET competencies do engineering graduates find most important in their work?
Journal of Engineering Education, 101(1), 95-118. doi: 10.1002/j.2168-9830.2012.tb00043.x
Ramlall, S. (2004). A review of employee motivation theories and their implications for employee retention
within organizations. The Journal of American Academy of Business, Cambridge 5(1/2), 52-63.
Roethle, J. (2012). Developing a new employee onboarding program in a small engineering department.
Milwaukee School of Engineering, Milwaukee, WI.
Rollag, K., Parise, S., & Cross, R. (2005). Getting new hires up to speed quickly. MIT Sloan Management
Review, 46(2), 35-41.
Sheppard, S., Macatangay, K., Colby, A., & Sullivan, W.M. (2009). Educating engineers: Designing for the future
of the field (Vol. 9): Jossey-Bass San Francisco, CA.
Snell, A. (2006). Researching onboarding best practice: Using research to connect onboarding processes with
employee satisfaction. Strategic HR Review, 5(6), 32-35.
Spinks, N., Silburn, N., & Birchall, D. (2006). Educating engineers for the 21st century: The industry view.
London: The Royal Academy of Engineering.
Stump, G.S., Hilpert, J.C., Husman, J., Chung, W.-t., & Kim, W. (2011). Collaborative learning in engineering
students: Gender and achievement. Journal of Engineering Education, 100(3), 475-497.
33
References
Trevelyan, J. (2007). Technical coordination in engineering practice. Journal of Engineering Education, 96(3),
191-204.
Trevelyan, J. (2008). A framework for understanding engineering practice. Paper presented at the American
Society for Engineering Education, Pittsburgh, PA.
Trevelyan, J., & Tilli, S. (2008). Longitudinal study of Australian engineering graduates: Perceptions of working
time. Paper presented at the American Society for Engineering Education, Pittsburgh, PA.
Trevelyan, J.P. (2009). Engineering education requires a better model of engineering practice. Paper presented
at the Proceedings of the Research in Engineering Education Symposium, Palm Cove, QLD, Australia.
Trevelyan, J.P. (2010). Mind the gaps: Engineering education and practice. Paper presented at the 21st Annual
Conference for the Australasian Association for Engineering Education, Sydney, Australia.
Winters, K.E., Matusovich, H.M., Brunhaver, S., Chen, H.L., Yasuhara, K., & Sheppard, S. (2013, June). From
freshman engineering students to practicing professionals: Changes in beliefs about important skills over
time. Paper presented at the American Society for Engineering Education, Atlanta, GA.
34