HOW TO ESTABLISH AND
EVALUATE CLINICAL
PREDICTION MODELS
An Academic presentation by
Dr. Nancy Agnes, Head, Technical Operations, Statswork
Group www.statswork.com
Email: info@statswork.com
TODAY'S DISCUSSION
Outline
Introduction
Clinical Prediction Model
Steps to establishing a clinical prediction model
Clinical prediction models CODE
Future Scope:
INTRODUCTION
The use of a parametric/semi-parametric/nonparametric mathematical model to estimate the
probability that a subject currently has a certain
condition or the possibility of a certain outcome in the
future is referred to as a clinical predictive model.
Various regression analysis approaches are used to
model clinical prediction models, and the statistical
nature of regression analysis is to find "quantitative
causality."
To put it another way, regression analysis is a quantitative assessment of how much
X impacts Y.
Multiple linear regression models, logistic regression models, and Cox regression
models are all widely used approaches.
The secret to statistical analysis, data modelling, and project design is assessing
and verifying prediction models' efficacy.
It is also the most difficult aspect of data analysis technology.
CLINICAL
PREDICTION
MODEL
A clinical prediction model is a tool used in healthcare to
measure estimates of the likelihood of the future course of
a specific patient outcome using multiple clinical or nonclinical predictors.
A realistic checklist for developing a valid prediction
model is presented in a clinical prediction model.
A clinical prediction model can be used in various clinical
contexts, including screening for asymptomatic illness,
forecasting future events such as disease, and assisting
doctors in their decision-making and health education.
Contd...
Despite the positive effects of clinical prediction models on practice, prediction
modelling is a difficult process that necessitates meticulous statistical analysis and
sound clinical judgments.
Contd...
STEPS TO
ESTABLISHING
A CLINICAL
PREDICTION
MODEL
There exist several types of research detailing the
methods to construct clinical prediction models.
However, there is no proper method to construct the
prediction model in medicine.
The construction and evaluation of prediction models are
classified into five steps.
Contd...
STEP 1: GATHERING THE IDEATIONS AND QUESTIONS FOR ENHANCING THE
MODEL.
It incorporates structuring the research questions, such as finding the target
variable for predicting which age group of the targeted people you want to predict.
etc.
For instance, gathering one patient details and use it as a trained data set to test
the other data set of another patient’s details. [1].
Contd...
STEP 2: SELECTION OF DATA
Data collection is a vital part of statistical or clinical research.
Nevertheless, the perfect data and a perfect model can't exist. It would be nice to look
for the most appropriate.
The primary dataset with the endpoint of the study and all key predictors may not be
available at all the time.
Secondary or administrative data sources are mandatory.
Contd...
Based on the various data types of datasets, prediction models can be utilized.
[2] For instance, the epidemiology study is based on the Data Mining systematic
approach.
STEP 3: WAYS TO HANDLE VARIABLES
Most of the time, researchers may face challenging situations where the variables
are highly correlated to each other, excluded in the study.
Variables don't show statistical significance or the petite effect size.
But it will contribute to the predictive model. Researchers will handle the missing
data problems, categorical data, etc., before getting the interference.
Contd...
CLINICAL
PREDICTION
MODELS CODE
Contd...
The Bayesian network was implemented to manipulate the independent variables of
some diseases in the crucial stage of treatment.
This model predicts and offers a way to handle the disease along with preventive
measures [3].
Contd...
STEP 4: GENERATING MODEL
There are no proper rules to select a particular model for the statistical analysis.
There are some standard methods to build a model using Linear
regression analysis, logistic regression analysis, and Cox models.
Sometimes the clinical data encounters over-fitting of the model and its results in as
estimates.
This over-fitting issue can be detected using Akaike Information Criteria or Bayesian
Information Criteria.
Contd...
The smaller AIC and BIC values result in a good fit for the model.
[4] Using Multivariate prediction models for analyzing the different characteristics of
various patients.
Contd...
STEP 5: EVALUATION AND VALIDATION OF THE MODEL
After building the model, it is necessary to evaluate and validate the predictive
power of the model.
The key components that evaluate the model are calibration which plots the
proportion, and discrimination classifies the events like success or failure.
There are two types of data validation, namely internal and external validation of the
model.
Internal validation evaluates the model within the data, whereas external validation
can be done using the re-sampling technique, usually through bootstrapping.
Contd...
It means you are creating or generating new data sets with similar characteristics to
the original data and validating the study's method through the newly created or
bootstrapped data.
Further, there are several statistical measures to evaluate the model.
Some of them are ROC curve, AUC curve, sensitivity and specificity, likelihood ratio,
R square value, calibration plot, c-index, Hosmer-Lemeshow test, AIC, BIC, etc.
Contd...
Figure 1: Slope of Calibration plot – Source: Stevens and Poppe (2020)
Contd...
Besides, Stevens and Poppe (2020) suggested the Cox- calibration slope using a
logistic regression model instead of using the predictive model's calibration slope.
This suggestion has been made after the scrutiny of around 33 research articles
and found that most of the validation is external validation and identified the validity
using the calibration slope.
Contd...
Contd...
Figure 2: This flow diagram illustrates the progress through the various phases of
the CARDAMON phase II clinical trial, including the impact of COVID‐19 on the 70
patients on maintenance K across the two treatment arms at the start of the
lockdown period.
The 15 patients who stopped K maintenance joined the 170 patients who were
already on long‐term follow‐up on 24 March 2020, bringing the number up to a total
of 185. SCT, stem cell transplantation; K, carfilzomib; C, cyclophosphamide; d,
dexamethasone [6].
Contd...
FUTURE
SCOPE
Based on the patient details, we can predict the further severe
causation of disease in the future.
By gathering the data from a single patient may help to predict other
similar patients for better treatment.
Big data support for manipulating vast amounts of clinical trials,
without complexity simultaneously with high accuracy.
Contd...
TABLE 1 Concepts and Techniques of Clinical prediction models:
Contact Us
UNITED KINGDOM
+44-1143520021
INDIA
+91-4448137070
EMAIL
info@statswork.com