Assessing the Impact of Education:
Are We Moving the Needle?
How Would We Know?
(and what does all this really mean?)
Norris Krueger
Entrepreneurship Northwest
@entrep_thinking ; norris.krueger[at]gmail.com
Sturgeon’s Law?
• Do you want to be:
• Above-average?
• Top 10%?
• Top 1%?
• Are you sure?  [“stand up”]
Assessing Impact
• What are we looking for?
– Show that something changed?
– Something positive? 
– Or actual, useful feedback on our performance?
– (hint: this is what the best programs do)
– (and in fact, assessment is part of course design)
What do the best programs do?
• What does anyone do???
• Usual suspects? (What do YOU do?)
What could we do?
• Outputs?
• Throughputs?
• Behaviors (ours and theirs)
• Cognitive or “non-cognitive”
– Knowledge; Skills
– Life skills/attributes
• DEEP cognitive change?
But…..
• What IS the “entrepreneurial mindset”?
• How would we know if it is changing?
Sorry… there’s homework 
• For now…
• When you think about the entrepreneurial
mindset – not skills, not info… Think DEEPER.
How would you know someone had a more
expert mindset?
• What is different specifically?
Copernican Revolution(s)?
•
Converging today….
How do humans actually learn important stuff?
Practitioners leading the way
Online/blended tools
Moving from Novice to Expert
(don’t you want to create experts?)
Assessment!
Experts’ Obsessions
• Obsess over developing a truly entrepreneurial
mindset
– Not “stuff”
– Not “skills”
– Business models NOT plans
• Obsess over deep co-immersion with the
entrepreneurial community (a/k/a “Ecosystem”)
What drives great things in…
• Startup Weekend, Lean Launchpad, TechStars?
•  exactly these twin obsessions
• Best technology commercialization programs?
 exactly this
• The most successful entrepreneurial ecosystems? (e.g.
Boulder or SV)
•  exactly this
• The A++ Entrepreneurship Programs
 exactly this
What the best programs do…
•
•
•
•
1. Focus on mindset.. but not as buzzword
2. Connecting students to ecosystem
3. Connecting ourselves even more deeply
4. And make connections two-way
• “Engagement” –> Immersion
• Immersion -> CO-immersion!
• ACTION!
• Uh… maybe we should measure these?
Change in WHAT we
know (content)
Novice
Entrepreneurial
Mindset
Change in HOW we know it
(knowledge structures)
Change in Deep Beliefs
Critical Developmental
Experiences
Expert
Consider “lean startup”
• Validate biz model… sure, but also..
• Change mindset at a deep level
• Startup Weekend, TechStars
– Full immersion
• Peer support
• Personal reflection
– Co-Immersion with ecosystem
• Expert mentors
• Ecosystem feedback
– Who & what is guiding all this?
– And how do we assess ?
10,000 hours?
WB Yeats
“Learning is not about filling a pail;
it’s about lighting a fire.”
(but where does the spark come
from?)
Two Ways to Learn
• Behavioristic
• - fill the bucket
• -define the buckets
•
• Constructivistic
• - light the fire (or fan it!)
• - assume knowledge structures evolve
• (often discontinuously)
Learners create their own
buckets…
– through…
•
•
•
•
personal reflection,
peer mentoring,
expert mentoring
….based on authentic questions
• “Experience is not what happens to you, it is
what you do with what happens.”
-Epictetus
the best programs…
•
•
•
•
Provide multiple CDEs
Make sure learners take away the right lessons
CCL example
Why not for entrepreneurship?
What we are should be
doing
• Skills
• Knowledge
• Mindset
• Changing Deep
Beliefs
• Novice 
Expert
• Improving Intent
• Improving
ACTION
So what IS this
“entrepreneurial mindset” anyway??
• Crowdsourced list
• Thoughts? Which are the hardest to change?
– (More important for here... what can we measure?)
Seeing opportunities, yes, but…
• Seeing self as entrepreneur (role identity)
• Strategic thinking
• Dot-connecting / pattern recognition
• Effectual thinking (bricolage, improv)
• Optimist (attributional)
• Metacognition
• Counterfactual thinking
• Bias for action (vs. FoF!)
• Value creation as sine qua non (Market-pull?)
the “new” role of instructors
MORE homework 
• Remember that one change that contributes
to a more entrepreneurial mindset – what’s
different?
• How would you know if it changed?
• How would you measure that?
….and…
• What CDE would YOU give learners the
opportunity to make that change?
• And how would you know that learners took
away the right lessons?
Accelerating Learning
• Problem-Based Learning (“authentic”
questions)
• Two overlapping themes:
• IMMERSION – Action before thinking
• MENTORING – Learn the right lessons
–
–
–
Expert mentors [and I do mean expert]
Peer mentors
Or THREE? Supportive ecosystem?
–
–
Social infrastructure; cognitive infrastructure
http://bit/ly/EcoSys
…. If you do it right, you
will be called….
•
•
•
•
A trouble-maker
A bomb-thrower
A boat-rocker
A…. Well, what do they say in your country?
• But NO, you are not. You are a….
• Bildung Brandstifter!!
If you want to get involved
• in projects – research or applied?
• Norris Krueger:
– norris.krueger@gmail.com
– @entrep_thinking
– Facebook
– LinkedIn
Immersion/Mentoring
•
•
•
•
•
Chalmers/Gothenburg
NU Singapore
The Foundry (Utah)
TechStars (www.techstars.org)
Startup Weekend
– www.startupweekend.org
• Lean Startup
– http://www.slideshare.net/sblank/successfulentrepreneurship-1
We’re Not Teaching
Entrepreneurship
(or Shouldn’t Be!)
We Are Growing…
Entrepreneurial
Brains
Norris Krueger, PhD
–
More ed psych!
Neergaard, Tanggaard, Krueger & Robinson (2013) Pedagogical
Interventions in Entrepreneurship: from behaviourism to
existential learning?