Journal Club notes – March 2024 NEJM AI article on informed consent
Basic idea: want to see if AI Large Language Models (LLM) could come up with a more patient friendly and
understandable consent form for surgical procedure
Background:
“As of 2020, 54% of Americans were estimated to read below the sixth-grade level.”
Informed consent is enshrined in modern biomedical ethics as a core tenet of “respect for persons,” as
outlined in the 1978 Belmont Report by the National Commission for the Protection of Human Subjects of
Biomedical and Behavioral Research.
Written documents can serve as a framework for physician–patient conversations and as take-home
references for patients. If the comprehension level required to read such documents is too high,
approaches to optimize the text should be undertaken to achieve truly informed consent.
They did previous work: analyzed surgical consent forms from a nationally representative cohort of 15
hospitals, found that they required reading level of a college sophomore.
Setting: At Lifespan, Rhode Island’s largest health care system and the primary medical teaching affiliate of Brown
University
they have tons of surgeries so they have a universal surgical consent form first
undergoes periodic review to incorporate the latest best practices.
April 2023, a modification of this form was scheduled but the updated form had reading level that
necessitated literacy skills equal to those of a college freshman
Methods:
Question: Can Chat GPT4 make a better consent?
They fed Lifespan’s preliminary update of the surgical consent form into GPT-4 and provided a succinct,
14-word prompt: “While preserving content and meaning, convert this consent to the average American
reading level.”
Results:
Made one at slightly less than 7th grade reading level. Can see example of two side by side comparisons in
Figure 2 of the pdf
Some issues:
o The first concerned the potential introduction of biases into the output, such as stereotypes and
discriminations, that may have been absorbed from the large, human-generated data sets used
to build and train LLMs. We addressed this concern by ensuring that multiple human experts
from various medical and nonmedical domains reviewed the consent forms before their use in
patient-facing scenarios.
o The second issue pertained to the authority responsible for approving the wording of the
consent form. We determined that the final approval in our institution would come from the risk
management team given that they oversee policy regarding consent forms.
o The third issue was if a similar initiative would be necessary for other patient literature.
Conclusions
This is a task current AI technology is well-suited for. LLMs are trained on a large text corpus of
heterogeneous writing styles and reading levels. For a task that may take a proofreader hours or
days, GPT-4 simplified a 947-word document in less than 1 minute.
Always need the human in the loop (for now): use of human review to confirm that the simplified content
did not sacrifice medicolegal rigor
Don’t reinvent the wheel/generalizable: Replicating this process across the 15 institutions we previously
studied resulted in a significant decrease in reading level by five grades and reading time by 26%.
o Could create repositories of this with JCAHO or other organizations that we (the medical field
“we”) regulate and monitor and update