Measuring Skills Using Games
What would you rather do …
Take a test
or
play a game?
The Problem and Mission
The value of Build-It-Yourself will be largely measured by how
well we can prove that our students learn valuable skills.
Tests are the most obvious way to measure skills.
The Problem:
No one likes to take tests!
Our Mission:
Create playful games that measure
building skills quantitatively.
The Problem and Mission
Choose an avatar
Target audience:
Boys and girls 8 – 13. (playful)
Parents (constructive)
Game Attributes:
1) Takes less than 10 minutes
to play.
2) Funky, game-like, inspired
by Lemonade Stand and
other Tycoon games.
Choose an avatar. Your
avatar will be presented
with a problem. You must
solve the problem.
“There must be millions of
people all over the world who
never get any love letters . . .
I could be their leader.”
Charlie Brown
“Honey, Where’s my super suit?”
Mr. Incredible
“Go in peace my daughter. And
remember that, in a world of
ordinary mortals, you are a Wonder
Woman.” Queen Hippolyte
Problem Solving Skills
We want to measure how well kids know the problem solving
process exercised in BIY workshops.
1) Define and focus on a problem.
2) Research and build on the ideas of others.
Don’t reinvent the wheel.
3) Teamwork.
4) Break a solution into simple parts.
5) Document.
If we cannot measure how well kids can execute the problem solving process in a short
game, is it meaningful to measure how well they know a problem solving process?
Lab books, finished projects and presentations are a reflection of execution.
Definitions of problem solving skills.
QP1
QP2
1) Focus on problem / mission
Describe the problem / mission clearly, concisely and convincingly.
Use technology constructively to solve or better understand a social issue.
QR1
QR2
2) Research
Find appropriate images in Google and embed them in PPT.
Build on solutions created by others. Find previous art. Incorporate the ideas of others.
QB1
QB2
3) Break a solution into simple parts
Know the benefits of modular construction. (quick to build, easy to edit, reusable)
Define and assign tasks to complete a project.
QT1
QT2
QT3
4) Teamwork
Share ideas, supplies and tools.
Help other teammates and / or teach other teammates.
Team goals are more important that individual goals.
Use collaboration tools??? Perhaps we should add this to our curriculum.
QD1
QD2
5) Documentation
Create a PPT lab book that invites people to read it.
Define concepts clearly, concisely and convincingly. Can you read a message quickly from a
distance?
Measuring Skills Using Games
Skill Game Storyline Prototypes
Problem / Mission:
1) An important robot is broken. Fix it. (Benjamin)
2) Someone is in trouble. Rescue them. (Carolyn)
3) The economy is in trouble. Stimulate it. (Qin En)
Focus on a problem
1) View an animated clip that shows 5 things happening.
and then select the best definition of the problem.
(5 gold coins if you make the proper selection.)
Research Skills
1) Filter useful information from useless information.
(15 of 45 bits of information are useful.)
2) Connect useful information
(Decision is made that requires knowledge of two
independent bits of information.)
3) Know what information is necessary to solve a problem.
(Decision requires finding information in Google)
Teamwork Skills
1) Value team before individual success
(Select 1 resume out of 10 that focuses on team before
individual accomplishments.)
2) Communicate effectively
(Select 1 resume out of 10 that communicates clearly,
concisely and convincingly.)
3) Listen, be open minded, compromise, brainstorm
(Select a decision that reflects compromise or win – win.)
•
•
•
•
skilled / talented
offer help
contribute ideas
if you pick bad teammates it becomes very hard to solve or your teammates will slow you down.
Break into simple parts
1) Break solution into simple, focused modules and tasks
2) Identify series vs parallel tasks
Documentation Skills
1)
2)
3)
4)
Clear
Well organized, easy to find specific info
Concise, Relevant, Accurate
Document results and edits
Select user’s manuel that has 1, 2, 3 and / or 4 qualities.
Skill Measurement Matrix
Focus on a problem (20%)
1) Observe and define the problem
Research a problem (20%)
1) Filter useful information
0
No part of the problem
identified.
10
Part of the problem selected.
May include irrelevant
information.
0
Select useless information
4
Selection reflects connecting 2
bits of information
Calculate correct answer. Must
find fact on Google.
3) Find critical information
Break a solution into simple parts (20%)
1) Choose project plan tasks and parts
0
Teamwork (20%)
1) Communicate convincingly
0
4
Select 5 or 20 critical parts and
or tasks that are simple,
focused components of a
solution.
6
3) Listen and compromise
3) Concise, relevant, accurate info
8
Select 1 of 10 resumes that
reflects good communication
Select 1 of 10 resumes that
reflects team over individual.
2) Value team over individual success
2) Well organized, easy to find info
6
Select 2 sections of useful
information.
2) Connect information
Document (20%)
1) Clear
20
Problem and focus selected.
Select plan that reflects
compromise and win - win.
0
6
8
Select 1 of 10 user's manuels
that is clear.
Select 1 of 10 user's manuels
that is well organized
Select user's manuel that is
concise, relevant and accurate.
Project Plan
Skill Measuring Game Project Plan
Tasks
PPT Final Presentation (10 slides)
Design specification (skill definitions, scoring)
Test finished prototype (survey, summarize results)
Storyline content for:
opening trailer (must be spectacular)
problem definition
research
break problem into parts
teamwork
documentation
Graphic content (characters, backgrounds, props)
opening trailer
problem definition
research
break problem into parts
teamwork
documentation
Sound effects
Scratch program
HTML for research section
Meetings
Documentation
Total
Total Weeks Feb 20 - Aug 10
Weeks Remaining
Hours / week required to complete project
4
4
3
20-Feb
Status
Hours
Remaining
2
4
3
3
1
4
2
4
4
2
1
4
2
4
4
6
2
2
4
4
2
3
12
4
24
4
96
5
2
2
4
4
2
3
12
4
23
2
89
24
24
3.7
as of
Est hrs
20-Feb
Research
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.117.8503&rep=rep1&type=pdf
Definition of problem solving skills
1) Identify the problem (Focus on problem)
2) Define the problem through thinking about it and sorting out the relevant information (Research)
3) Explore solutions through looking at alternatives, brainstorming, and checking out
different points of view (Teamwork)
4) Act on the strategies (Break solution into simple parts)
5) Look back and evaluate the effects of your activity (Document)
http://www.ehow.com/how_2120954_teach-problem-solving-skills.html
The University of Phoenix NO FUN way.
Examples:
http://www.lemonadestandgame.com/
Simple, intuitive, no graphics, makes you think strategically.
Skill Game Storyline Prototypes
Problem / Mission:
1) An important robot is broken. Fix it. (Benjamin)
2) Explore a curious object on a remote planet. (Rahul)
3) Someone is in trouble. Rescue them. (Carolyn)
4) The economy is in trouble. Fix it. (Qin En)
John’s notes for Game Graphics and Storylines:
RPG / Tycoon Style Game:
You are a famous “Problem Solver.” (Trouble Shooter)
Your fee is 50 gold coins to start plus
A bonus (up to 50 GC) based on how well you solve a problem.
You must make decisions that solve the problem.
Each decision has a cost / benefit ratio.
Your goal is to solve the problem at the lowest cost.
(i.e. maximize your profit.)
John’s notes for Game Graphics and Storylines:
RPG / Tycoon Style Game:
A trailer presents a problem.
You explore an RPG space and select:
1. Definition of the problem and goal.
2. The most important research info.
3. A strategy that solves the problem.
4. Teammates needed to solve the problem.
5. Documentation that enables your teammates to work together.
John’s notes for Game Graphics and Storylines:
Intro trailer:
You are …
Mr. Problem Solver
Ms. Trouble Shooter
Click one
John’s notes for Game Graphics and Storylines:
Intro trailer:
The economy is in the toilet!
Many kids do not have enough money
to buy the candy or computer games
they need.
John’s notes for Game Graphics and Storylines:
Intro trailer:
Some young BIY entrepreneurs have set out to build a
robot that will provide a service, put people to work and
make many on their team rich and famous beyond their
wildest dreams.
John’s notes for Game Graphics and Storylines:
Intro trailer:
However, their business is failing.
Your mission is to help them find the problem, fix it, and earn
a fortune?
John’s notes for Game Graphics and Storylines:
Intro trailer:
You can go to …
The Computer Cove to find information and document.
Headquarters
Bank
John’s notes for Game Graphics and Storylines:
Possible problems:
The socialists say, "There are too many greedy,
scurrilous scoundrels and vile villains." A vile
competitor stole your customers.
The capitalists say, "Too many lazy no-gooders are
looking for free rides. Not enough people are
building.“
A vile employee is embezzling your profits.
The robot never worked.
Nobody wants to pay 25 cents.
The robot costs more to run than it can earn.
The robot was doing a robust business then it broke.
www.build-it-yourself.com
1.
2.
3.
4.
Financial history.
Parts life cycle.
People in the news.
Testimonials.
Bank Galaxy
$
M1
$
110,500
$
59,500
$
51,000
1,000,000 $ 1,051,000
M2
$
$
$
$
35,750
340,500
(304,750)
746,250
M3
$
550,000
$
150,250
$
399,750
$ 1,146,000
M4
$
$
$
$
53,250
250,000
(196,750)
949,250
M5
$
350,500
$
50,250
$
300,250
$ 1,249,500
M6
$
475,000
$
90,500
$
384,500
$ 1,634,000
Rocket Works Financial History
$2,000,000
$1,800,000
$1,600,000
$1,400,000
$1,200,000
Dollars
Start
$1,000,000
$800,000
$600,000
$400,000
$200,000
John’s notes for Game Graphics and Storylines:
Intro trailer:
You can go to …
Headquarters to recruit teammates and build.
Headquarters
Bank
Headquarters
Candidates
Kitchen
Game Room
Value team before individual success
Communicate effectively
Listen, be open minded, compromise
Wellesley Student
$500/day
Successful negotiator.
Community organizer.
Applicants
Harvard Student
$1000/day
Can sell.
Claims to be the best.
MIT Student
$2000/day
Techno-guru.
Hero is Ghandi.
Value team before individual success
Communicate effectively
Listen, be open minded, compromise
High School Graduate
$10/day
Follows instructions and
keeps it simple.
Talks from the heart.
Applicants
???
BU Stud
You design your own
$400/day
Teammate
Doesn’t get bogged
down with philosophy or
politics.
World Solitaire Champ
Headquarters
Candidates
Kitchen
Game Room
Kitchen
$10/day/employee
25% of prospects
like the food. You
can’t recruit good
people with perks.
$25/day/employee
50% of prospects
like the food. Food
is not a reason for
joining your team.
$50/day/employee
100% of prospects
like the food.
Many people want
to work for you.
John’s notes for Game Graphics and Storylines:
Intro trailer:
You can go to …
Bank Galactica to monitor your gold coins.
Headquarters
Bank
Bank Galaxy
$56,735,000
Balance Sheet
$
Specifications
$
Research
$
Recruiting
$
Construction
$
Documentation
$
Balance
Gold Coins Earned