Can AI Match Human
Ingenuity in Creative
Problem-Solving?
Generative AI handles a variety of business
tasks, but can it develop creative solutions
to problems? Yes, although some of the
best ideas emerge when humans and
machines work together, according to
research by Jacqueline Ng Lane, Karim
Lakhani, Miaomiao Zhang, and colleagues.
August 26, 2024
Jacqueline N. Lane
Assistant Professor at Harvard Business School and co-Principal Investigator of the
Laboratory for Innovation Science at the Digital Data Design Institute (D^3) at Harvard
Karim R. Lakhani
Dorothy and Michael Hintze Professor of Business Administration at Harvard Business
School
Miaomiao Zhang
Doctoral candidate at the Technology & Operations Management Unit
at Harvard Business School
W
hen ChatGPT and other large language
models began entering the mainstream
two years ago, it quickly became apparent the
technology could excel at certain wbusiness
functions, yet it was less clear how well artificial
intelligence could handle more creative tasks.
Sure, generative AI can summarize the content of
an article, identify patterns in data, and produce
derivative work—say, a song in the style of Taylor
Swift or a poem in the mood of Langston Hughes—
but can the technology develop truly innovative
ideas?
Specifically, Harvard Business School Assistant
Professor Jacqueline Ng Lane was determined to
find out “how AI handled open-ended problems
that haven’t been solved yet—the kind where you
need diverse expertise and perspectives to make
progress.”
In a working paper published in the journal
Page 2
Organization Science, Lane and colleagues
compare ChatGPT’s creative potential to
crowdsourced innovations produced by people.
Ultimately, the researchers found that both humans
and AI have their strengths—people contribute
more novel suggestions while AI creates more
practical solutions—yet some of the most promising
ideas are the ones people and machines develop
together.
Lane cowrote the paper with Léonard Bouissioux,
assistant professor at the University of Washington’s
Foster School of Business; Miaomiao Zhang, an
HBS doctoral student, Karim Lakhani, the Dorothy &
Michael Hintze Professor of Business Administration
at HBS; and Vladimir Jacimovic, CEO and founder
of ContinuumLab.ai and executive fellow at HBS.
Crowdsourcing people for ‘moonshots’
Any innovation process usually starts with
brainstorming, says Lane, whose research has long
looked at how creative ideas are produced.
“It’s like a funnel,” she says. “You start with defining
the problem, then you generate ideas, then
you evaluate them and choose which ones to
implement.”
Research has shown that crowdsourcing can be an
effective way to generate initial ideas. However, the
approach can be time-consuming and expensive.
Creative teams typically offer incentives to
respondents for their ideas. Then teams often must
wait for input and then comb through ideas to
come up with the most promising leads.
An off-the-shelf large language model such as
ChatGPT, however, is free or low cost for end users,
and can generate an infinite number of ideas
quickly, Lane says. But are the ideas any good?
“You start with defining the problem,
then you generate ideas, then you
evaluate them and choose which ones
to implement.”
To find out, Lane and her fellow researchers asked
people to come up with business ideas for the
sustainable circular economy, in which products
are reused or recycled to make new products.
They disseminated a request on an online platform,
offering $10 for participating and $1,000 for the
best idea. Here’s part of their request:
We would like you to submit your circular economy
idea, which can be a unique new idea or an existent
idea that is used in the industry. Here is an example:
Car sharing in order to reduce the carbon footprint
Page 3
associated with driving.…
Submit your real-life use cases on how companies
can implement the circular economy in their
businesses. New ideas are also welcome, even if
they are “moonshots.”
Seeking creative ideas from ChatGPT
The researchers asked for ideas that would involve
“sharing, leasing, reusing, repairing, refurbishing
[or] recycling existing materials and products as
long as possible.” Suggestions would be scored for
uniqueness, environmental benefits, profitpotential,
and feasibility.
Some 125 people replied with contributions,
offering insights from a variety of industries and
professional backgrounds. One, for example,
proposed a dynamic pricing algorithm for
supermarkets to cut down on food waste, while
another suggested a mobile app that could store
receipts to reduce paper waste.
At the same time, the research team employed
prompt engineering techniques to craft a variety
of AI prompts. Using these carefully designed
prompts, they generated several hundred additional
solutions through ChatGPT. The team strategically
modified their prompts to:
• Challenge the model to create more ideas.
• Mimic the perspective of someone from a
particular industry, job title, and place—a persona.
• Remind the model to provide ideas that reflect the
scoring criteria.
The team then recruited some 300 evaluators
well-versed in the circular economy to evaluate a
randomized selection of the ideas based on the
scoring criteria.
People are creative, but AI ideas are more feasible
The evaluators judged the human solutions as more
novel, employing more unique “out of the box”
thinking. However, they found the AI-generated
ideas to be more valuable and feasible.
For example, one participant from Africa proposed
creating interlocking bricks using foundry dust and
waste plastic, creating a new construction material
and cutting down on air pollution at the same time.
“The evaluators said, ‘Wow, this is really innovative,
but it would never work,’” Lane says.
“We were surprised at how powerful
these technologies were.”
One ChatGPT response, meanwhile, created an
idea to convert food waste into biogas, a renewable
energy source that could be used for electricity and
fertilizer. Not the most novel idea, the researchers
noted, but one that could be implemented and
might show a clear financial return.
“We were surprised at how powerful these
technologies were,” Lane says, “especially in these
early stages in the creative process.”
How to reach the best solutions
The “best” ideas, Lane says, may come from those
in which humans and AI collaborate, with people
engineering prompts and continually working with
AI to develop more original ideas.
“We consistently achieved higher quality results
when AI would come up with an idea and then we
Page 4
had an instruction that said: Make sure before you
create your next idea, it’s different from all the ones
before it,” Lane explains.
Additional prompts increased the novelty of the
ideas, generating everything from waste-eating
African flies to beverage containers tracked by
smart chips that instantly pay consumers for
recycling them.
Based on the findings, the researchers suggest
business leaders keep a few points in mind when
implementing AI to develop creative solutions:
• Knowing how to ask the right questions is
important. Organizations might want to invest
in cultivating an “AI-literate” workforce that can
understand the capabilities and limitations of AI to
generate the most successful ideas.
• Organizations should resist the temptation to rely
excessively on AI. That could “dumb down” the
overall level of creative output over time, leading
to more incremental improvements than radical
breakthroughs, the team says.
• People should view generative AI models as
collaborative tools. In a sequential approach,
humans could brainstorm solutions, then submit
them to AI to refine them and increase their value
and feasibility. Alternatively, humans could work
more iteratively with AI, constantly shaping and
improving the ideas it provides.
The most productive way to use generative AI, the
research suggests, is to combine the novelty that
people excel at with the practicality of the machine.
Says Lane, “We still need to put our minds toward
being forward-looking and envisioning new things
as we are guiding the outputs of AI to create the
best solutions.”