Quantitative Structure-Activity Relationships
Quantitative Structure-Property-Relationships
SAR/QSAR/QSPR modeling
Alexandre Varnek
Faculté de Chimie, ULP, Strasbourg, FRANCE
SAR/QSAR/QSPR models
• Development
• Validation
• Application
Classification and Regression
models
• Development
• Validation
• Application
Development of the models
•
•
•
Selection and curation of experimental data
Preparation of training and test sets (optionaly)
Selection of an initial set of descriptors and their
normalisation
Variables selection (optionally)
Selection of a machine-learning method
•
•
Validation of models
•
•
Training/test set
Cross-validation
-
internal,
external
Application of the Models
•
Models Applicability Domain
Development the models
•
•
•
•
Experimental Data: selection and cleaning
Descriptors
Mathematical techniques
Statistical criteria
Data selection: Congenericity problem
• Congenericity principle is the assumption that « similar
compounds give similar responses ». This was the basic
requirement of QSAR. This concerns structurally
homogeneous data sets.
• Nowdays, experimentalists mostly produce structurally
diverse (non-congeneric) data sets
Data cleaning:
•
•
•
•
•
Similar experimental conditions
Dublicates
Structures standardization
Removal of mixtures
…..
The importance of Chemical Data Curation
Dataset curation is crucial for any cheminformatics analysis (QSAR
modeling, clustering, similarity search, etc.).
Currently, it is uncommon to describe procedures used for curation
in research papers; procedures are implemented or employed
differently in different groups.
We wish to emphasize the need to create and popularize
standardized curation strategy, applicable for any ensemble of
compounds.
What about these structures? (real examples)
Why duplicates are unsafe for QSAR ?
Duplicates are identical compounds present in a given dataset.
OH
CH3
HO
CH3
CH3
CH3
OH
H3C
H3C
CH3
OH
CH3
OH
CH3
OH
ID = 256
ID = 879
ID = 2346
Manual identification of duplicates is practically impossible especially when the dataset is large.
Activity analysis of duplicates is also highly important to identify cases where one occurrence is
identified as ‘active’ and another one as ‘weak active’ or ‘inactive’.
CH3
CH3
HO
OH
H3C
ACTIVE
CH3
INACTIVE
H3C
OH
OH
CH3
Structural standardization
For a given dataset, chemical groups have to be written in a standardized way, taking
into account critical properties (like pH) of the modeled system.
Aromatic compounds
OH
OH
These two different representations of the same
compound will lead to different descriptors, especially
with certain fingerprint or fragmental approaches.
Cl
Cl
CH3
CH3
Carboxylic acids, nitro groups etc.
O O
HO
X
O– O
O O
X
X
O O
OH
N
N+
X
X
For a given dataset, these functional groups have to be written in a consistent way to
avoid different descriptor values for the same chemical group.
Normalization of carboxylic, nitro groups, etc.
removal of inorganics
All inorganic compounds must be removed since our QSAR
modeling strategy includes the calculation of molecular
descriptors for organic compounds only.
This is an obvious limitation of the approach. However the total fraction of
inorganics in most available datasets is relatively small.
To detect inorganics, several solutions are
available:
- Automatic
identification
using
in
combination Jchem (ChemAxon, cxcalc
program) to output the empirical formula
of all compounds and simple scripts to
remove compounds with no carbon;
- Manual
inspection
of
compounds
possessing no carbon atom using
Notepad++ tools.
removal of mixtures
Fragments can be removed according to the number of
constitutive atoms or the molecular weight.
removal of mixtures
However, some cases are particularly difficult to treat.
Examples from DILI - BIOWISDOM dataset:
ID=172
CLEANED FORM BY CHEMAXON
The two eliminated compounds
could be active !
.
INITIAL FORM
MANUAL INSPECTION/VALIDATION IS STILL CRUCIAL
ID=1700
INITIAL FORM
CLEANED FORM BY CHEMAXON Ok.
removal of salts
Options Remove Fragments, Neutralize and Transform of Chemaxon
Standardizer. have to be used simultaneously for best results.
Aromatization and 2D cleaning
ChemAxon Standardizer offers two ways to aromatize benzene rings,
both of them based on Hűckel’s rules.
“General Style”
O
CH3
NH
OH
“Basic Style”
CH3
O
NH
OH
Most descriptor calculation
packages recognize the “basic
style” only.
http://www.chemaxon.com/jchem/marvin/help/sci/aromatization-doc.html
Preparation of training and test sets
Building of structure property models
Training set
Initial data set
Test
10 – 15 %
Splitting of an initial
data set into training
and test sets
Selection of the best
models according to
statistical criteria
“Prediction” calculations
using the best structure property models
Recommendations to prepare a test set
• (i) experimental methods for determination of activities in the training
and test sets should be similar;
•
(ii) the activity values should span several orders of magnitude, but
should not exceed activity values in the training set by more than 10%;
•
(iii) the balance between active and inactive compounds should be
respected for uniform sampling of the data.
References: Oprea, T. I.; Waller, C. L.; Marshall, G. R. J. Med. Chem. 1994, 37, 2206-2215
Descriptors
• Variables selction
• Normalization
molecules
descriptors
Pattern matrix
Selection of descriptors for QSAR model
QSAR models should be reduced to a set of descriptors which is
as information rich but as small as possible.
Objective selection (independent variable only)
Statistical criteria of correlations
Pairwise selection (Forward or Backward Stepwise selection)
Principal Component Analysis
Partial Least Square analysis
Genetic Algorithm
……………….
Subjective selection
Descriptors selection based on mechanistic studies
Preprocessing strategy for the derivation of models
for use in structure-activity relationships (QSARs)
1. identify a subset of columns (variables) with significant
correlation to the response;
2. remove columns (variables) with zero (small) variance;
3. remove columns (variables) with no unique information;
4. identify a subset of variables on which to construct a model;
5. address the problem of chance correlation.
D. C. Whitley, M. G. Ford, D. J. Livingstone
J. Chem. Inf. Comput. Sci. 2000, 40, 1160-1168
Descriptors Normalisation
descriptors
n
molecules
m j  (1/ n) xij*
i 1
n
s  (1/ n) ( xij*  m j ) 2
2
j
i 1
Pattern matrix
Normalisation 1 (Unit Variance scaling):
Normalisation 2 (Mean Centring Scaling):
xij  x  m j
*
ij
xij 
xij*  m j
sj
Data Normalisation
Initial
descriptors
Norm. 1
Norm. 2
Machine-Learning Methods
Fitting models’ parameters
Y = F(ai , Xi )
Xi - descriptors (independent variables)
ai
- fitted parameters
The goal is to minimize Residual Sum of Squared (RSS)
N
RSS   ( yexp,i  ycalc,i )
i 1
2
Multiple Linear Regression
Activity
Descriptor
Y1
X1
Y2
Y2
…
…
Yn
Xn
Yi = a0 + a1 Xi1
Y
X
Multiple Linear Regression
y=ax+b
Residual Sum of
Squared (RSS)
N
RSS   ( yi  ycalc,i )
i 1
2
b
a
Multiple Linear Regression
Activity
Descr 1
Descr 2
…
Descr m
Y1
X11
X12
…
X1m
Y2
X21
X22
…
X2m
…
…
…
…
…
Yn
Xn1
Xn2
…
Xnm
Yi = a0 + a1 Xi1 + a2 Xi2 +…+ am Xim
kNN (k Nearest Neighbors)
Activity Y assessment calculating a weighted mean of the
activities Yi of its k nearest neighbors in the chemical space
TRAINING SET
Descriptor 1
Descriptor 2
A.Tropsha, A.Golbraikh, 2003
Biological and Artificial Neuron
Multilayer Neural Network
Neurons in the input layer correspond to descriptors, neurons in the output
layer – to properties being predicted, neurons in the hidden layer – to nonlinear
latent variables
SVM: Support Vector Machine
 w, x  b  1
 w, x  b  0
 w, x  b  1
2
w
Support Vector Classification (SVC)
SVM: Margins
The margin is the minimal
distance of any training point to
the separating hyperplane
Margin
1

w
Support Vector Regression
ε-Insensitive Loss Function
Only the points outside the εtube are penalized in a linear
fashion
 0 if   
  : 
    otherwise
Kernel Trick
In low-dimensional
input space
K ( x, x)  ( x), ( x) 
In high-dimensional
feature space
Any non-linear problem (classification, regression) in the original input space can be
converted into linear by making non-linear mapping Φ into a feature space with
higher dimension
QSAR/QSPR models
• Development
• Validation
• Application
Preparation of training and test sets
Building of structure property models
Training set
Initial data set
Test
10 – 15 %
Splitting of an initial
data set into training
and test sets
Selection of the best
models according to
statistical criteria
“Prediction” calculations
using the best structure property models
Validation
Estimation of the models predictive performance
5- Fold Cross Validation
All
compounds
of the
dataset are
predicted
Dataset Fold1 Fold2 Fold3 Fold4 Fold5
Leave-One Out Cross-Validation
N- Fold Internal Cross Validation
• Cross-validation is performed AFTER variables selection on the entire dataset.
• On each fold, the “test” set contains only 1 molecule
Statistical parameters for Regression
42
Fitting vs validation
Stabilities (logK) of Sr2+L complexes in water
LogKcalc
LogKpred
12
LOO
12
Fit
9
6
0
9
9
6
6
3
R2 = 0.886
RMSE = 0.97
3
12
3
R2= 0.826
RMSE = 1.20
0
5-CV
R2 = 0.682
RMSE = 1.62
0
-3
0
3
6
9
12
15
0
3
6
9
12
15
3
6
9
12
15
LogKexp
All molecules were used for
the model preparation
Each molecule was
“predicted” in internal CV
Each molecule was predicted
in external CV
Regression Error Characteristic (REC)
REC curves are widely used to compare of the performance of different models.
The gray line corresponds to average value model (AM). For a given model, the
area between AM and corresponding calculated curve reflects its quality.
Statistical parameters for Classification
Confusion Matrix
Classification Evaluation
sensitivity = true positive rate (TPR) = hit rate = recall
TPR = TP / P = TP / (TP + FN)
false positive rate (FPR)
FPR = FP / N = FP / (FP + TN)
specificity (SPC) = True Negative Rate
SPC = TN / N = TN / (FP + TN) = 1 − FPR
positive predictive value (PPV) = precision
PPV = TP / (TP + FP)
negative predictive value (NPV)
NPV = TN / (TN + FN)
accuracy (ACC)
ACC = (TP + TN) / (P + N)
balanced accuracy (BAC)
BAC = (sensitivity + sensitivity ) / 2 = (TP / (TP + FN) + TN / (FP + TN)) /2
Receiver Operating Characteristic (ROC)
TPR
Plot of the sensitivity vs (1 −
specificity) for a binary classifier
system as its discrimination threshold
is varied.
The ROC can also be represented
equivalently by plotting the fraction of
true positives (TPR = true positive
rate) vs the fraction of false positives
(FPR = false positive rate).
FPR
Ideally, Area Under Curve (AUC) => 1
ROC (Receiver Operating Characteristics)
100%
TP 0
FP a
1 2 3
b c d
4 5 6 7 8 9 e f g h i j
FN 0
TN a
1 2 3
b c d
4 5 6 7 8 9 e f g h i j
TP%
Ideal model:
AUC=0.84
AUC=1.00
j g
Useless model:
AUC=0.50
0%
FP%
0
a
1
b
h
100%
2
3
d
6
e
f
i
5
c 8
4
7 9
When a model is accepted ?
Regression Models
Classification Models
3 classes
Determination coefficient R2 > R02
Here, R02 = 0.5
49
BA > 1/q for q classes
“Chance correlation” problem
2,000
1
1,500
0.75
1,000
0.5
1965
1970
1975
year
1980
a model MUST be validated on new independent
data to avoid a chance correlation
Y-Scrambling
(for methods without descriptor selection)
X1
X2
Y1
Y2
Y2
Y5
X3
Y3
Y4
X4
Y4
Y6
X5
Y5
Y1
X6
Y6
Y7
X7
Y7
Y3
R2
0.0
1.0
Y-Scrambling
(for methods without descriptor selection)
X1
X2
Y1
Y2
Y4
Y1
X3
Y3
Y5
X4
Y4
Y2
X5
Y5
Y6
X6
Y6
Y3
X7
Y7
Y7
R2
0.0
1.0
Y-Scrambling
(for methods without descriptor selection)
X1
X2
Y1
Y2
Y7
Y6
X3
Y3
Y3
X4
Y4
Y5
X5
Y5
Y4
X6
Y6
Y1
X7
Y7
Y2
R2
0.0
1.0
QSAR/QSPR models
• Development
• Validation
• Application
Test compound
QSPR Models
Prediction Performance
Robustness of QSPR models
- Descriptors type;
- Descriptors selection;
- Machine-learning methods;
- Validation of models.
Applicability domain of models
Is a test compound similar
to
the
training
set
compounds?
Applicability domain of QSAR models
Descriptor 2
The new compound will be predicted by
the model, only if :
Di ≤ <Dk> + Z × sk
with Z, an empirical parameter (0.5 by default)
TRAINING SET
Descriptor 1
= TEST
INSIDE THE DOMAIN
OUTSIDE THE DOMAIN
Will be predicted
Will not be predicted
COMPOUND
Applicability
domain of QSAR models
Applicability
Domain Approaches
Fragment –based methods
 Fragment Control (FC)
Density based methods

1-SVM
 Model’s Fragment Control
(MFC)
Distance –based methods
 zkNN
Range –based methods
 Bounding Box (BB)
Ensemble modeling
Hunting season …
Single hunter
Hunting season …
Many hunters
Ensemble modelling
Ensemple modeling
Y1
Y2
1 n
Consensus = Y i
n i 1
Y3
Screening and hits selection
Database
O
COOH
Cl
Br
OH
N
OH
Virtual
Sreening
N
OH
QSPR model
N
COOH
Useless
compounds
O
Br
Hits
Experimental
Tests