The Art of Asking Questions
Karen N. Johnson
Test Bash 2015
The Art of Asking Questions © Karen N. Johnson, 2015
KAREN N. JOHNSON
• New position in 2015. Director of Mobile Quality at Orbitz
Worldwide, Inc.
• Published Author (Beautiful Testing)
• Teach Software Testing
• Speak at conferences
• Co-founder of WREST, the Workshop on Regulated Software
Testing
• Website: www.karennicolejohnson.com or www.karennjohnson.com
• Twitter: @karennjohnson
The Art of Asking Questions © Karen N. Johnson, 2015
What is the purpose of this talk?
The Art of Asking Questions © Karen N. Johnson, 2015
“The scientist is not a person who
gives the right answers, he's one
who asks the right questions.”
- Claude Lévi-Strauss
The Art of Asking Questions © Karen N. Johnson, 2015
What do I think are the two
most important elements
of asking a question?
The Art of Asking Questions © Karen N. Johnson, 2015
Timing
When you ask a question
can impact what you get for
a response.
Where you ask a question,
can impact what you get for
a response. The where you
ask a question, factors into
timing.
You can ask the right
person the right question
but ask at the “wrong” time.
Timing matters.
The Art of Asking Questions © Karen N. Johnson, 2015
“The right thing at the wrong
time is the wrong thing.”
- Joshua Harris
The Art of Asking Questions © Karen N. Johnson, 2015
Tone
How you ask a question can
impact what you get for a
response.
You can ask the right
person the right question at
the right time but ask in a
tone that can affect not just
the response but your
ongoing relationship with
the person.
Tone matters.
The Art of Asking Questions © Karen N. Johnson, 2015
“We often refuse to accept an idea
merely because the tone of voice
in which it has been expressed
is unsympathetic to us.”
- Friedrich Nietzsche
The Art of Asking Questions © Karen N. Johnson, 2015
“Humble Inquiry is
the fine art of
drawing someone
out, of asking
questions to which
you do not already
know the answer, of
building a
relationship based
on curiosity and
interest in the other
person.”
The Art of Asking Questions © Karen N. Johnson, 2015
“The best scientists and explorers have
the attributes of kids! They ask questions
and have a sense of wonder. They have
curiosity, “Who, what, where, why, when
and how! They never stop asking
questions, and I never stop asking
questions, just like a five year old.”
- Sylvia Earle
The Art of Asking Questions © Karen N. Johnson, 2015
The journalist’s six
basic questions.
A short look at the
basic journalism
questions.
The Art of Asking Questions © Karen N. Johnson, 2015
Who?
Find the right source
for information.
The Art of Asking Questions © Karen N. Johnson, 2015
What?
Ask questions of
substance.
The Art of Asking Questions © Karen N. Johnson, 2015
Where?
Find a good location for
a discussion.
The Art of Asking Questions © Karen N. Johnson, 2015
When?
Timing matters.
The Art of Asking Questions © Karen N. Johnson, 2015
Why?
Know what you want to
learn from your questions.
The Art of Asking Questions © Karen N. Johnson, 2015
How?
The tone of how you
ask a question
influences the outcome.
The Art of Asking Questions © Karen N. Johnson, 2015
"What people think of as the
moment of discovery is really the
discovery of the question."
- Jonas Salk
The Art of Asking Questions © Karen N. Johnson, 2015
The Five Whys
The concept of using the
five whys is that by
persisting to ask why, the
underlying issue, the cause
of the issue will be
discovered. Harvard
Business Review has a
great short video that
explains this.
See:
https://www.youtube.com/w
atch?v=JmrAkHafwHI
The Art of Asking Questions © Karen N. Johnson, 2015
THE FIVE WHYS
• Harvard Business Review,
http://blogs.hbr.org/2010/04/the-five-whys-forstartups/
• Harvard Business Review,
https://www.youtube.com/watch?v=JmrAkHafwHI
The Art of Asking Questions © Karen N. Johnson, 2015
The Phoenix
Checklist
The Phoenix checklist is a
list of questions developed
by the United States
Central Intelligence
Agency. The list is
comprised of questions
arranged in two sections:
the problem and the
solution. The questions are
meant to help an
investigator ask questions.
The questions are
intentionally context-free.
The Art of Asking Questions © Karen N. Johnson, 2015
THE PHOENIX CHECKLIST
•
•
•
•
•
•
Malbon, Ben. Blog: How the CIA define problems and plan solutions: The Phoenix
Checklist http://bbh-labs.com/how-the-cia-define-problems-plan-solutions-thephoenix-checklist/
Tinkham, Andy, blog post: Context-free questions
http://testerthoughts.com/2003/07/16/context-free-questions/
Strazzere, Joe, blog post: http://www.allthingsquality.com/2010/04/phoenixchecklist.html
Lambert, Rob blog post: http://thesocialtester.co.uk/professional-skepticsdispeller-of-illusions-and-questioner/ and http://thesocialtester.co.uk/thephoenix-checklist-mind-map/
Kaner, Cem and Bach, James Black Box Software Testing Special Edition
www.testingeducation.org/k04/documents/specBased.ppt
Bolton, Michael blog: Context-free Questions for Testing.
http://www.developsense.com/blog/2010/11/context-free-questions-for-testing/
The Art of Asking Questions © Karen N. Johnson, 2015
The Socratic
Method
The “teacher,” or
leader of the dialogue, asks
probing questions in an
effort to expose the values and
beliefs which frame
and support the thoughts and
statements of the
participants in the inquiry.
- Source:
http://www.stanford.edu/dept/
CTL/Newsletter/socratic_meth
od.pdf
The Art of Asking Questions © Karen N. Johnson, 2015
The Paper Chase
In a well-known movie, The
Paper Chase, a scene between
a student and professor
demonstrates the Socratic
Method.
While the Socratic Method
may be a favored method of
questioning for knowledge.
• Intimidation
• Fear
• Being put on the spot
Are some of the negative
reactions to using the method.
The Art of Asking Questions © Karen N. Johnson, 2015
SOCRATIC METHOD
• The Paper Chase, YouTube clip:
https://www.youtube.com/watch?v=qx22TyCge7w
• Socratic Method applied to teaching:
http://www.stanford.edu/dept/CTL/Newsletter/socratic
_method.pdf
• Socratic teaching and critical thinking:
http://www.criticalthinking.org/pages/socraticteaching/606
• Socratic method and scientific method compared,
http://www.niu.edu/~jdye/method.html
The Art of Asking Questions © Karen N. Johnson, 2015
Roadblocks
to asking questions and to getting answers
The Art of Asking Questions © Karen N. Johnson, 2015
Not everyone
shares
In the book, “Questions
That Work” the author
Andrew Finlayson claims
there are question killers.
“Question killers can also
stifle curiosity and
openness with obtuse or
bureaucratic answers.”
The Art of Asking Questions © Karen N. Johnson, 2015
Relying on only one
source for
information
Sometimes, you might ask
the wrong person for
information.
The Art of Asking Questions © Karen N. Johnson, 2015
Fear &
Intimidation
"He who asks a question is
a fool for five minutes; he
who does not ask a question
remains a fool forever."
- Chinese proverb
*
“It can be intimidating to
ask questions. Sometimes
you have to be humble and
yet brave to ask questions.”
- Karen N. Johnson
The Art of Asking Questions © Karen N. Johnson, 2015
“And I like asking questions, to
keep learning; people with big egos
might not want to look unsure.”
- Heston Blumenthal
The Art of Asking Questions © Karen N. Johnson, 2015
Not doing your
homework
Don’t expect people to tell
you everything, instead
learn as much as you can
ahead of asking questions.
The Art of Asking Questions © Karen N. Johnson, 2015
A Few Tips
on
Asking Questions
The Art of Asking Questions © Karen N. Johnson, 2015
Set
a Constructive
Atmosphere
Create the right
atmosphere to ask
questions.
The “right” atmosphere
won’t just be the location
but the mood and tone of
the conversation.
The Art of Asking Questions © Karen N. Johnson, 2015
“Creating the right
questioning
atmosphere means
adapting your
presentation to the
situation and the
other’s personality.”
The Art of Asking Questions © Karen N. Johnson, 2015
Learn
Different
Types of Questions
There are many types of
questions: open-ended,
rhetorical, leading –
learning about questions is
a worthwhile “self
schooling.”
The Art of Asking Questions © Karen N. Johnson, 2015
Give people the
tools and space to
give visual answers
Some people think more
visually. Asking visual
people to explain the inner
workings of software
without having tools to
draw can be frustrating for
the person.
The Art of Asking Questions © Karen N. Johnson, 2015
Allow answers to
come in the form of
a story
Some people answer
questions best when they
can tell a story, walk you
through an entire process
instead of having a Q&A
session. Give the person
answering the ability to
answer in the format that
works for them.
The Art of Asking Questions © Karen N. Johnson, 2015
Don’t get lost
It’s ok to let the other
person lead but at intervals,
review your questions and
notes and make sure your
questions get answered.
The Art of Asking Questions © Karen N. Johnson, 2015
Look for
spontaneous pickup
of information at
unexpected
moments
Realize you may pickup
some information “on the
fly.” Be ready and be open
to it.
The Art of Asking Questions © Karen N. Johnson, 2015
“I love the early process of asking
questions about a story and deciding
which questions matter most.”
- Diane Sawyer
The Art of Asking Questions © Karen N. Johnson, 2015
Collect the artifacts
of investigation
Gather together the
information, including your
notes from interviews and
see what you have.
The Art of Asking Questions © Karen N. Johnson, 2015
Separate facts from
opinions
Journalists are taught early
on to distinguish between
fact and fiction. Sometimes
you can receive the wrong
information or conflicting
information and you have to
be able to discern those
gaps and misleading of
information.
The Art of Asking Questions © Karen N. Johnson, 2015
Make Sense of what
you’ve gathered
You may need to mentally
reassemble what you have
gathered and even mentally
reformulate what you
believed or understood as
information is gathered,
consumed and understood.
The Art of Asking Questions © Karen N. Johnson, 2015
Can you walk me through the process of the
software?
I like this question because:
• It gives the other person space to answer in a variety
of formats, including drawing.
• It assumes little.
• It is an open-ended question.
The Art of Asking Questions © Karen N. Johnson, 2015
Can you tell me what you’re most concerned
about with this application?
I like this question because:
• It gets to the heart of risk.
• It gives a developer the chance to open up about where
coding may have been challenging.
• It gives a developer an opportunity to discuss what
has been minimally tested or is minimally working.
• It enables product owners to articulate their concerns
which I can turn into test ideas and follow-up.
• It speaks directly to high-level executives who want
their concerns heard and addressed.
The Art of Asking Questions © Karen N. Johnson, 2015
“Humble Inquiry is
the fine art of
drawing someone
out, of asking
questions to which
you do not already
know the answer, of
building a
relationship based
on curiosity and
interest in the other
person.”
The Art of Asking Questions © Karen N. Johnson, 2015
“Creating the right
questioning
atmosphere means
adapting your
presentation to the
situation and the
other’s personality.”
The Art of Asking Questions © Karen N. Johnson, 2015
“… the talking,
listening, thinking,
analyzing, selecting
and editing
segments that all
good interviewers
incorporate,
whether they
realize it or not.”
The Art of Asking Questions © Karen N. Johnson, 2015
"Judge a man by his questions rather than
his answers."
– Voltaire
The Art of Asking Questions © Karen N. Johnson, 2015
“Maybe some of the best questions
we ask, are the questions we ask
ourselves, the internal questions,
the ones that keep us exploring.”
- Karen N. Johnson
Questions?
The Art of Asking Questions © Karen N. Johnson, 2015