Enterprise Open Innovation
WWW.TOPCODER.COM
AGENDA
Theme: Community, by Community
• A little about TopCoder/Appirio
• Crowdsourcing
• Contests
• TopCoder Crowdsourcing
• When Communities Build Communities
• Contributing to the Asteroids Mission
TOPCODER
ENGAGING TOP MINDS TO SOLVE ENTERPRISE NEEDS
ENTERPRISE
CUSTOMERS
592K COMMUNITY
TOP DESIGNERS, DEVELOPERS
AND ALGORITHM / ANALYTIC EXPERTS
FRONT-END DESIGN
& INNOVATION
SOFTWARE
DEVELOPMENT
ALGORITHM/
ANALYTICS
KEY AREAS:
IA/UX Design, Development
& Algorithms / Analytics
Design Concepts
User Interface Design
Wireframes
Storyboards
Prototype
Architecture
Development
Assembly
Testing
Bug Races
Marathon Matches
Single-Round Match
TOPCODER
FRONT-END
DESIGN & INNOVATION
Subjective
Client Decides Winner
SOFTWARE
DEVELOPMENT
Objective
Rigorous Scorecards
Decide Winner
ALGORITHM
ANALYTICS
Performance
Best solution against
pre-determined metrics wins
What Customers are Building
Web & Mobile Apps
Websites, E-Commerce
Platform & Strategic Assets
Legacy System Integration
& Cloud Utility Tools
Analytics
CROWDSOURCING
Lots accomplished with Crowdsourcing:
• Linux
• Java
• Wikipedia
• fold.It
• NatGeo – Field Expedition: Mongolia
CONTESTS
Lots accomplished with Contests:
• Longitude Contest
• White House Portico Design
• Canned food!
• Lindbergh’s Flight
• Ansari X-Prize
• And a few things on TopCoder…
SELECT ACADEMIC RESEARCH ON COMMUNITIES
•
•
•
•
•
•
•
•
•
•
•
Lakhani, Karim, David Garvin, and Eric Lonstein. "Topcoder (a): Developing software through
crowdsourcing." Harvard Business School General Management Unit case 610-032 (2010).
Malone, Thomas W., Robert J. Laubacher, and Tammy Johns. "The age of hyperspecialization." Harvard
Business Review 89.7-8 (2011): 56-+.
Wu, Wenjun, Wei-Tek Tsai, and Wei Li. "An evaluation framework for software crowdsourcing."
Frontiers of Computer Science: 1-16.
Archak, Nikolay. "Money, glory and cheap talk: analyzing strategic behavior of contestants in
simultaneous crowdsourcing contests on TopCoder. com."Proceedings of the 19th international
conference on World wide web. ACM, 2010.
Begel, Andrew, Jan Bosch, and Margaret-Anne Storey. "Social Networking Meets Software Development:
Perspectives from GitHub, MSDN, Stack Exchange, and TopCoder." Software, IEEE 30.1 (2013): 52-66.
Boudreau, Kevin J., Nicola Lacetera, and Karim R. Lakhani. "Incentives and problem uncertainty in
innovation contests: An empirical analysis."Management Science 57.5 (2011): 843-863.
Kaganer, Evgeny, et al. "Managing the Human Cloud." MIT SLOAN MANAGEMENT REVIEW 54.2 (2013):
23-32.
Eisenstein, Michael. "Crowdsourced contest identifies best-in-class breast cancer prognostic." Nature
biotechnology 31.7 (2013): 578-580.
Boudreau, Kevin J., and Karim R. Lakhani. "How to Manage Outside Innovation." MIT SLOAN
MANAGEMENT REVIEW
Lakhani, Karim R., et al. "Prize-based contests can provide solutions to computational biology
problems." Nature biotechnology 31.2 (2013): 108-111.
Zhang, Yin, and Ruoyu Lu. "The case study on innovation contest modes of Internet companies." Service
Systems and Service Management (ICSSSM), 2011 8th International Conference on. IEEE, 2011.
600+ academic articles (source: Google Scholar)
Nearly two dozen+ patents on crowdsourcing and software development
CROWDSOURCING WITH CONTESTS
TopCoder Crowdsourcing
• NASA Tournament Lab
• MegaBLAST
• Medicaid!
• Community Sites
TopCoder CROWDSOURCING WITH CONTESTS
NASA Tournament Lab (NTL)
FIT (Food Intake Tracker)
iPad Application
The application can identify the Astronaut,
track all dietary intake (food and beverages),
and provide a timestamp of when the food
was consumed on space.
International Space Station:
Solar Panel Alignment
Pipeline Protection
Detection Algorithm
A detection algorithm to spot the presence of
wheeled vehicles in proximity to a pipeline
with the ability to distinguish between a
security threat an un-authorized vehicle and
a non-threat such as wild-life or a downed
tree limb.
Planetary Data Systems
Web API, iOS and Android Apps
4000+ Registrants, $30,000 Total
Develop and deploy an API to homogenize
search results across a federation of PDS
nodes. Demonstrate the value of the API
with a suite of mobile consumer apps.
The goal of the challenge is to position the solar
collectors on the International Space Station to
generate as much power as possible during the most
difficult orbital positions.
NASA and Harvard University have established the NASA Tournament Lab (NTL), to leverage the TopCoder
community and allow for competitions to create the most innovative, most efficient, and most optimized
solutions for specific, real-world challenges being faced by NASA researchers
Winning solution performs 120x faster
Improve on NIH MegaBlast algorithm
for nucleotide sequence alignment
ANTIBODY SEQUENCE ANNOTATION
122
654
89
5
CODERS SUBMITTED
SOLUTIONS
DIFFERENT APPROACHES TO
SOLVE PROBLEM IDENTIFIED
WINNING COUNTRIES
RUSSIA, FRANCE, EGYPT, BELGIUM & US
TOPCODER CROWDSOURCING
Modernizing Medicaid and Medicare
• Implement with > 130 individual contests
• Design and build a new single Medicaid module for screening providers for use by all
states
• Single proposal was submitted for RFP for 3 years $120M+ for 80 modules for 1 state
• Game change outcome for US with $100Ms in savings possible
• Allows for late-entry stakeholder participation
• States can engage at any time through conceptualization contests
• TopCoder analyst competitors review state requirements and wish-lists
• TopCoder competitors organize results for changes, additions, priorities
• Stakeholder steering committee can assign requirements to later phases
• Contests allow all voices to be heard and managed
• Directly tests HHS Architecture Compliance
• Specification contests will explore and document requirements
• “Bug Hunts” will interrogate designs and code for government compliance
TOPCODER CROWDSOURCING
Portal: Innovation and Cost Savings
$8,000,000
$7,800,000
$7,600,000
$7,400,000
$7,200,000
$7,000,000
$6,800,000
$6,600,000
$6,400,000
$6,200,000
$6,000,000
$5,800,000
$5,600,000
$5,400,000
$5,200,000
$5,000,000
$4,800,000
$4,600,000
$4,400,000
$4,200,000
$4,000,000
$3,800,000
$3,600,000
$3,400,000
$3,200,000
$3,000,000
$2,800,000
$2,600,000
$2,400,000
$2,200,000
$2,000,000
$1,800,000
$1,600,000
$1,400,000
$1,200,000
$1,000,000
$800,000
$600,000
$400,000
$200,000
$-
$7,458,013
$6,004,924
$1,520,076
$1,433,806
$735,549
$717,540
$59,220
Development Contractor
$27,050
Federal Development Project
Support
TRADITIONAL MODEL
Federal Infrastructure Support
CROWD MODEL
Total Costs
COMMUNITIES
When Communities build Communities
• The Pilgrims
• Linux
• Java
• Ghost
By WordPress engineers who
wanted to focus on Blogs
+ Part-time devs they trusted
+ Kickstarter funding!
Contribution Communities
COMMUNITIES: @TOPCODER
When Communities build Communities
(with TopCoder & others)
• TopCoder.com
• NoNameSite.com
• ZeroRobotics
• Verigames.com
• Expedition Mongolia
Responsive Collaboration
Communities
• Online Collaboration Centers
• With Infrastructure and Instrumentation
• For Incentives & Innovation
COMMUNITIES: @TOPCODER
TopCoder.com
COMMUNITIES: @TOPCODER
COMMUNITIES: @TOPCODER
ZeroRobotics.com
• Spherical satellites move with independent control of
all six degrees of freedom in zero gravity (inside the
international space station)
• These “spheres” travel within a defined area to
complete tasks in a game scenario developed by
researchers at MIT
• High School students compete against one another to
earn the most points using a high-level language
environment
• Virtual zero-gravity and visual simulation is used to
allow online competition before the finals are executed
on the space station
•Zero Robotics infrastructure and web site was built by
TopCoder
COMMUNITIES: @TOPCODER
COMMUNITIES: @TOPCODER
Verigames.com
Perform formal software verification
through games accessible to
unspecialized players.
•
•
•
•
A community platform
Provides profiles, badges, forums
Hosts 3rd Party Games
Provides storage & display for:
• Scores
• Achievements
• History
COMMUNITIES: @TOPCODER
Verigames.com
COMMUNITIES: @NATGEO
National Geographic Explorer
COMMUNITIES: ASTEROIDS
Advancing the Asteroid Initiative
• Build communities around specialized challenges
• Encourage collaboration, and fun competition
• Connect the “Carbon Crowd” to the “Silicon Cloud”?
Algorithmic Automation
Human Crowd Perception
Can an algorithm
rank image analysis
tasks in order of
potential?
SIMILAR PROMISING RESULTS: Collective Minds Challenge
510 Registrants
394 Solutions (~10 per competitor)
4 Winners
9 Submissions in Idea Generation Contest
21 day challenge
$15k Prize Purse
1st and 2nd place submissions were good-quality generic-solution algorithms. These algorithms are comparable in
scoring, and clearly outscore the third and fourth places.
Approaches:
1.
Random forest approach to a reasonably
short list of features, mostly concentrating
on edges and circles. Features have been
collected on areas of different sizes.
Performance continues to improve with
more training data.
2.
Random forest approach to large number
of features, mostly based on color-gradient
analysis. Features have been analyzed on
areas of different sizes. Performance
saturates after a few thousand training
samples.
Illustration 3: Test1 Ancient Structures
0.9
0.8
0.7
0.6
Algo 1
0.5
Algo 2
0.4
Algo 3
0.3
Algo 4
0.2
0.1
0
500
1000
2000
4000
6000
8000
Credit: Dr. Rinat Sergeev, NASA Tournament Lab, Harvard IQSS
CAPSTONE THOUGHTS
Open Innovation via Communities is the practice of
finding a way to do something new through access
to people you normally wouldn’t be engaged with.
Concentrate on the real goals
- What are you trying to do?
- What problem are you trying to solve?
- What do you want your audience to be able to accomplish?
- Stay focused – do not get technical
Learn to let go, allow creativity to flourish
- Define goals but don’t define how!
- Communicate! – be honest with your opinion
- Allow yourself to be surprised and open to new ideas
Iteration
- Fail! Be prepared to fail and learn from it.
- Plan to review multiple submissions, look for “gems” within
- Provide feedback!
Thank you!