Exploring the future of verbatim coding with
ChatGPT
NewMR Webinar June 2023
Tim Brandwood & Damien Gouriet
Digital Taxonomy
Sponsors
WHO WE ARE….
Founded
in 2016
Verbatim
Coding Software
Efficiency through
cutting-edge tech
THE PURPOSE OF CODING
Open-ended Comments
I like the orange taste,
but it’s too sweet
The sweetness is a bit much
I think the orange colour is
too pale and artificial looking
I don’t like the bits of
orange in it
I don’t like the idea of
artificial sweeteners
Analysable Quantitative Measures
Taste
(871)
87%
Too Sweet
Tastes Artificial
Doesn’t taste like real orange
(632)
(421)
(237)
63%
42%
24%
Appearance
(423)
42%
Colour – too light/pale
Colour – too artificial looking
Colour – too dark
Colour – other comments
(302)
(150)
(100)
(40)
30%
15%
10%
4%
Ingredients
(408)
41%
Artificial Sweeteners
Real Orange
Orange bits
(300)
(150)
(50)
30%
15%
5%
Packaging
(387)
39%
(223)
(150)
(87)
(60)
22%
15%
8%
6%
Bottle too big
Difficult to hold
Label hard to read
Dislike bottle shape
Requires: Accuracy and Precision
CAN CHATGPT DO CODING THEN?
Complete Coding:
Performs extremely well:
Generates rich, human-like themes
Extracting Themes
“Chat” interactivity allows you to ask follow-up questions,
and dig deeper
Complete Coding:
Struggles to accurately and precisely quantify themes. Issues include:
Quantifying Themes
Incomplete coverage
Miscoding
Repeatability
20% verbatims coded
80% accuracy
PEOPLE STILL NEED TO BE
INVOLVED IN THE PROCESS
ChatGPT is an electric bike for your mind.
Go further, quicker …but you still need to pedal!
-Richard Bowman (https://prompt.mba)
CUSTOM MACHINE LEARNING
Still has a role to play in the coding Use Case
Autocoding
Accuracy
ChatGPT
20%
80%
Custom ML Model
60%
90%
FUTURE DEVELOPMENTS
4
Fine-Tuning LLMs
GPT4, Claude, Bard etc…
Interactivity
CONCLUSION
ChatGPT is a big step forward – but only truly useful for coding up to a point
Human oversight and quality tools needed
Custom ML models outperform ChatGPT
Together these developments point towards better, more enriched research
Q&A
Tim Brandwood
Digital Taxonomy
Damien Gouriet
Digital Taxonomy
Ray Poynter
NewMR
Sponsors