Support Vector Machines and Extensions
for Toxicological Evaluation of Nanoparticles
Erhun Kundakcioglu
Department of Industrial Engineering
University of Houston
Texas A&M University
College Station, TX
March 30, 2011
Outline
1
Introduction
2
Support Vector Machine Classifiers
3
Heat Induced Cellular Death
4
Toxicological Evaluation of Titania Nanoparticles
5
Current & Future Research
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
1 / 45
Introduction
Motivation
Cancer causes about 13% of all human deaths
7.6 million deaths worldwide due to cancer in 2007
Current Treatments
Radiation
Toxic for other tissues
Chemotherapy
High toxicity
Surgery
Complication and high cost
Gene/bone marrow transplantation
High cost
The cost of the therapy is estimated to $18,000 - $37,000 per patient in a
total of $72B annually.
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
2 / 45
Introduction
Motivation
Toxicity
A shop in Beijing, China: Polyacrylic coating application on
polystyrene boards
Coatings used by the workers contain nanoparticles
Seven women workers (aged 18-47) were identified with shortness of
breath
Clinical examinations reveal hypoxemia (low oxygen saturation in
blood), pleural effusion (fluid in thoracic cavity), and pericardial
effusion (fluid around heart)
Severe skin rash from intense itching of their faces, hands, and forearms
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
3 / 45
Introduction
Motivation
Toxicity
A shop in Beijing, China: Polyacrylic coating application on
polystyrene boards
Coatings used by the workers contain nanoparticles
Seven women workers (aged 18-47) were identified with shortness of
breath
Clinical examinations reveal hypoxemia (low oxygen saturation in
blood), pleural effusion (fluid in thoracic cavity), and pericardial
effusion (fluid around heart)
Severe skin rash from intense itching of their faces, hands, and forearms
Two other women from the same shop died (ages 19 and 29)
Pathology samples from the workers lungs identified 30 nm
(nanometer) in diameter particles
Song Y, Li X, Du X. Exposure to nanoparticles is related to pleural effusion, pulmonary fibrosis and granuloma.
European Respiratory Journal, Aug 2009.
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
3 / 45
Introduction
Motivation
Criticizms
“I think the paper should never have been published without characterizing the
exposure and the toxicological reactivity of the nanoparticles before blaming the
effects on them.”
“The real tragedy here is that these workers could have been protected if a
conventional chemical hygiene plan had been implemented that included a
working ventilation system and personal protective equipment. Preventing
inhalation of 30-nm nanoparticles can be as simple as the proper use of an
inexpensive mask sold by your neighborhood home improvement store.”
These researchers are funded by DuPont, Intel, L’oreal, Lockheed Martin,
Proctor&Gamble, Unidym.
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
4 / 45
Introduction
Cell Death Discrimination
Investigate in-situ the mechanism that the particles are inducing by
Titania nanoparticles and heat
Lung epithelia cancer cells (A549 from ATCC) are grown for 24 hours
785nm high power near infrared diode laser for spectroscopy analysis
(this light does not damage cells even after 60min at 115mW power)
Cells have been grown on MgF2 crystal for minimizing background
radiation
After reaching 80% confluency, cells are treated with the testing
material
Drug or heat exposure
Recording for the next 24 hours
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
5 / 45
Introduction
Cell Death Discrimination
Helps understanding pathological processes
induced by a disease
induced by pharmaceutical treatments such as anti-cancer drugs
Raman spectroscopy is employed to assess the potential toxicity of
chemical substances
noninvasive and does not require chemicals or markers
measurements can be taken rapidly and in real time
it is possible to analyze the health of either a single cell or the entire
population
has no interference with water and CO2
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
6 / 45
Introduction
Raman Spectroscopy
Observes vibrational, rotational, and other low-frequency modes
Relies on Raman scattering of monochromatic light from a laser
Laser light interacts with molecular vibrations, phonons or other excitations
The energy levels of the laser photons are shifted up or down
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
7 / 45
Introduction
Raman Spectroscopy (cont’d)
Drawback: complexity of the obtained spectra
Traditionally, peak fitting has been used to analyze Raman spectra
time consuming due to large number of overlapping peaks
series of mathematical procedures to remove the baseline, the
fluorescence, to normalize the spectra, and calculate the average and
the standard deviation
prior knowledge needed on which peaks are discriminant with limited
interference from background noise
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
8 / 45
Support Vector Machine Classifiers
Outline
1
Introduction
2
Support Vector Machine Classifiers
3
Heat Induced Cellular Death
4
Toxicological Evaluation of Titania Nanoparticles
5
Current & Future Research
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
9 / 45
Support Vector Machine Classifiers
An Example: Flu Patients
Patient
Age
Gender
Blood type
Body Temperature
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
Flu
March 30, 2011
10 / 45
Support Vector Machine Classifiers
An Example: Flu Patients
Patient
1
Age
25
Gender
F
Blood type
A(-)
Body Temperature
99
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
Flu
-
March 30, 2011
10 / 45
Support Vector Machine Classifiers
An Example: Flu Patients
Patient
1
2
Age
25
26
Gender
F
M
Blood type
A(-)
AB(+)
Body Temperature
99
104
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
Flu
+
March 30, 2011
10 / 45
Support Vector Machine Classifiers
An Example: Flu Patients
Patient
1
2
3
Age
25
26
30
Gender
F
M
M
Blood type
A(-)
AB(+)
B(+)
Body Temperature
99
104
103
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
Flu
+
+
March 30, 2011
10 / 45
Support Vector Machine Classifiers
An Example: Flu Patients
Patient
1
2
3
4
Age
25
26
30
32
Gender
F
M
M
M
Blood type
A(-)
AB(+)
B(+)
0(+)
Body Temperature
99
104
103
98
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
Flu
+
+
-
March 30, 2011
10 / 45
Support Vector Machine Classifiers
An Example: Flu Patients
Patient
1
2
3
4
5
Age
25
26
30
32
23
Gender
F
M
M
M
F
Blood type
A(-)
AB(+)
B(+)
0(+)
B(-)
Body Temperature
99
104
103
98
105
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
Flu
+
+
?
March 30, 2011
10 / 45
Support Vector Machine Classifiers
An Example: Flu Patients
Patient
1
2
3
4
5
Age
25
26
30
32
23
Gender
F
M
M
M
F
Blood type
A(-)
AB(+)
B(+)
0(+)
B(-)
Body Temperature
99
104
103
98
105
Flu
+
+
?
Patients 1-4: Training Set
Patient 5: Test Set
Goal: Minimize the generalization error
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
10 / 45
Support Vector Machine Classifiers
Support Vector Machines
111
000
000
111
000
111
000
111
000
111
111
000
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
Feature 1
1111
0000
0000
1111
0000
1111
1111
0000
0000
1111
0000
1111
0000
1111
0000
1111
0000
1111
111
000
000
111
0000
1111
000
111
0000
1111
0000
1111
Control Cell
Death Cell
1111
0000
0000
1111
0000
1111
1111
0000
0000
1111
0000
1111
Feature 2
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
11 / 45
Support Vector Machine Classifiers
Support Vector Machines
111
000
000
111
000
111
000
111
000
111
111
000
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
Feature 1
1111
0000
0000
1111
0000
1111
1111
0000
0000
1111
0000
1111
0000
1111
0000
1111
0000
1111
?
111
000
000
111
0000
1111
000
111
0000
1111
0000
1111
Control Cell
Death Cell
1111
0000
0000
1111
0000
1111
1111
0000
0000
1111
0000
1111
Feature 2
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
11 / 45
Support Vector Machine Classifiers
Support Vector Machines
111
000
000
111
000
111
000
111
000
111
111
000
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
Feature 1
1111
0000
0000
1111
0000
1111
1111
0000
0000
1111
0000
1111
0000
1111
0000
1111
0000
1111
?
111
000
000
111
0000
1111
000
111
0000
1111
0000
1111
Control Cell
Death Cell
1111
0000
0000
1111
0000
1111
1111
0000
0000
1111
0000
1111
Feature 2
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
11 / 45
Support Vector Machine Classifiers
Support Vector Machines
111
000
000
111
000
111
000
111
000
111
111
000
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
Feature 1
1111
0000
0000
1111
0000
1111
1111
0000
0000
1111
0000
1111
0000
1111
0000
1111
0000
1111
?
111
000
000
111
0000
1111
000
111
0000
1111
0000
1111
Control Cell
Death Cell
1111
0000
0000
1111
0000
1111
1111
0000
0000
1111
0000
1111
Feature 2
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
11 / 45
Support Vector Machine Classifiers
Support Vector Machines
111
000
000
111
000
111
000
111
000
111
111
000
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
Feature 1
1111
0000
0000
1111
0000
1111
1111
0000
0000
1111
0000
1111
0000
1111
0000
1111
0000
1111
111
000
000
111
0000
1111
000
111
0000
1111
0000
1111
Control Cell
Death Cell
1111
0000
0000
1111
0000
1111
1111
0000
0000
1111
0000
1111
Feature 2
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
11 / 45
Support Vector Machine Classifiers
Support Vector Machines
111
000
000
111
000
111
000
111
000
111
111
000
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
Feature 1
margin
1111
0000
0000
1111
0000
1111
1111
0000
0000
1111
0000
1111
0000
1111
0000
1111
0000
1111
111
000
000
111
0000
1111
000
111
0000
1111
0000
1111
Control Cell
Death Cell
1111
0000
0000
1111
0000
1111
1111
0000
0000
1111
0000
1111
Feature 2
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
11 / 45
Support Vector Machine Classifiers
Support Vector Machines
111
000
000
111
000
111
000
111
000
111
111
000
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
Feature 1
1111
0000
0000
1111
0000
1111
1111
0000
0000
1111
0000
1111
0000
1111
0000
1111
0000
1111
111
000
000
111
0000
1111
000
111
0000
1111
0000
1111
Control Cell
Death Cell
1111
0000
0000
1111
0000
1111
1111
0000
0000
1111
0000
1111
Feature 2
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
11 / 45
Support Vector Machine Classifiers
Support Vector Machines
111
000
000
111
000
111
000
111
000
111
111
000
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
Feature 1
n
rgi
ma
1111
0000
0000
1111
0000
1111
1111
0000
0000
1111
0000
1111
0000
1111
0000
1111
0000
1111
111
000
000
111
0000
1111
000
111
0000
1111
0000
1111
Control Cell
Death Cell
1111
0000
0000
1111
0000
1111
1111
0000
0000
1111
0000
1111
Feature 2
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
11 / 45
Support Vector Machine Classifiers
Support Vector Machines
111
000
000
111
000
111
000
111
000
111
111
000
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
Feature 1
1111
0000
0000
1111
0000
1111
1111
0000
0000
1111
0000
1111
0000
1111
0000
1111
0000
1111
?
111
000
000
111
0000
1111
000
111
0000
1111
0000
1111
Control Cell
Death Cell
1111
0000
0000
1111
0000
1111
1111
0000
0000
1111
0000
1111
Feature 2
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
11 / 45
Support Vector Machine Classifiers
Support Vector Machines (cont’d)
Developed by Vladimir Vapnik in 1995
State-of-the-art supervised learning methods
Classify two linearly/nonlinearly separable sets of pattern vectors
Kernel trick
Quadratic convex optimization problem
Efficiently solved
Can also be formulated as a SOCP problem
Minimizes the generalization error
Widely used; e.g., pattern recognition, text categorization,
biomedicine, brain-computer interface, financial applications.
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
12 / 45
Support Vector Machine Classifiers
Hard Margin Classifiers
Given the data xi ∈ Rd and labels yi ∈ {−1, +1} for i = 1, . . . , n
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
13 / 45
Support Vector Machine Classifiers
Hard Margin Classifiers
Given the data xi ∈ Rd and labels yi ∈ {−1, +1} for i = 1, . . . , n
Separating hyperplane is defined as hw, xi + b = 0
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
13 / 45
Support Vector Machine Classifiers
Hard Margin Classifiers
Given the data xi ∈ Rd and labels yi ∈ {−1, +1} for i = 1, . . . , n
Separating hyperplane is defined as hw, xi + b = 0
The geometric margin between the hyperplane and the nearest data
point x∗ is
|hw, x∗ i + b|
kwk
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
13 / 45
Support Vector Machine Classifiers
Hard Margin Classifiers
Given the data xi ∈ Rd and labels yi ∈ {−1, +1} for i = 1, . . . , n
Separating hyperplane is defined as hw, xi + b = 0
The geometric margin between the hyperplane and the nearest data
point x∗ is
|hw, x∗ i + b| → functional margin
kwk
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
13 / 45
Support Vector Machine Classifiers
Hard Margin Classifiers
Given the data xi ∈ Rd and labels yi ∈ {−1, +1} for i = 1, . . . , n
Separating hyperplane is defined as hw, xi + b = 0
The geometric margin between the hyperplane and the nearest data
point x∗ is
|hw, x∗ i + b| → functional margin
kwk
⇓
max
subject to
geometric margin
functional margin = 1
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
13 / 45
Support Vector Machine Classifiers
Hard Margin Classifiers
Given the data xi ∈ Rd and labels yi ∈ {−1, +1} for i = 1, . . . , n
Separating hyperplane is defined as hw, xi + b = 0
The geometric margin between the hyperplane and the nearest data
point x∗ is
|hw, x∗ i + b| → functional margin
kwk
⇓
max
subject to
min
w,b
subject to
geometric margin
functional margin = 1
⇓
1
kwk2
2
yi (hw, xi i + b) ≥ 1
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
i = 1, . . . , n
March 30, 2011
13 / 45
Support Vector Machine Classifiers
Soft Margin Classifiers
111
000
000
111
000
111
000
111
000
111
111
000
000
111
000
111
000
111
000
111
0000
1111
000
111
0000
000 1111
111
0000
1111
000
111
000
111
000
111
000
111
000
111
000
111
Feature 1
1111
0000
0000
1111
0000
1111
1111
0000
0000
1111
0000
1111
0000
1111
0000
1111
0000
1111
111
000
000
111
0000
1111
000
111
0000
1111
0000
1111
Control Cell
Death Cell
1111
0000
0000
1111
0000
1111
1111
0000
0000
1111
0000
1111
Feature 2
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
14 / 45
Support Vector Machine Classifiers
Soft Margin Classifiers
111
000
000
111
000
111
000
111
000
111
111
000
000
111
000
111
000
111
000
111
0000
1111
000
111
0000
000 1111
111
0000
1111
000
111
000
111
000
111
000
111
000
111
000
111
Feature 1
er
1111
0000
0000
1111
0000
1111
r(
ξ)
1111
0000
0000
1111
0000
1111
0000
1111
0000
1111
0000
1111
ro
111
000
000
111
0000
1111
000
111
0000
1111
0000
1111
Control Cell
Death Cell
1111
0000
0000
1111
0000
1111
1111
0000
0000
1111
0000
1111
Feature 2
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
14 / 45
Support Vector Machine Classifiers
Soft Margin Classifiers
111
000
000
111
000
111
000
111
000
111
111
000
000
111
000
111
000
111
000
111
0000
1111
000
111
0000
000 1111
111
0000
1111
000
111
000
111
000
111
000
111
000
111
000
111
Feature 1
1111
0000
0000
1111
0000
1111
1111
0000
0000
1111
0000
1111
0000
1111
0000
1111
0000
1111
error
(ξ )
111
000
000
111
0000
1111
000
111
0000
1111
0000
1111
Control Cell
Death Cell
1111
0000
0000
1111
0000
1111
1111
0000
0000
1111
0000
1111
Feature 2
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
14 / 45
Support Vector Machine Classifiers
Soft Margin Classifiers
min
w,b,ξ
subject to
n
1
CX 2
2
kwk +
ξi
2
2
i=1
yi (hw, xi i + b) ≥ 1 − ξi
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
i = 1, . . . , n
March 30, 2011
15 / 45
Heat Induced Cellular Death
Outline
1
Introduction
2
Support Vector Machine Classifiers
3
Heat Induced Cellular Death
4
Toxicological Evaluation of Titania Nanoparticles
5
Current & Future Research
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
16 / 45
Heat Induced Cellular Death
Experiment / Data Set
Each row represents a lung epithelia cell
Each column represents the Raman outputs at different energy levels
There are ∼ 1300 columns
Some cells are subject to either Etoposide or Triton X-100
Etoposide is a strong chemotherapeutic drug, used for Ewing’s
sarcoma, lung cancer, testicular cancer, lymphoma, non-lymphocytic
leukemia, and glioblastoma multiforme
Triton X-100 raptures the cellular membrane and results in the necrotic
death of the cells
The effect of abnormal heat (45 ℃ for 2 hours)
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
17 / 45
Heat Induced Cellular Death
Effect of Abnormal Heat - Heat Induced Cellular Death
Training Set:
Etoposide vs. control cells
Triton X-100 vs. control cells
Triton X-100 vs. Etoposide
Test Set: Cells that are subject to 45 ℃ for 2 hours
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
18 / 45
Heat Induced Cellular Death
Distance from the Hyperplane
Computational Results for Death Cell Discrimination
Sample ID’s
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
19 / 45
Heat Induced Cellular Death
Distance from the Hyperplane
Computational Results for Death Cell Discrimination
Sample ID’s
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
20 / 45
Heat Induced Cellular Death
Distance from the Hyperplane
Computational Results for Death Cell Discrimination
Sample ID’s
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
21 / 45
Heat Induced Cellular Death
Remarks
Heat treatment can be used as a form of programmed cell death
Foundation for developing diagnostic tools for cancer or other genetic
diseases
Powerful monitoring of the cellular response and disease treatment
with chemotherapy and radiation
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
22 / 45
Toxicological Evaluation of Titania Nanoparticles
Outline
1
Introduction
2
Support Vector Machine Classifiers
3
Heat Induced Cellular Death
4
Toxicological Evaluation of Titania Nanoparticles
5
Current & Future Research
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
23 / 45
Toxicological Evaluation of Titania Nanoparticles
Experiment / Data Set
Each row represents a lung epithelia cell
Each column represents the Raman outputs at different energy levels
There are ∼ 1300 columns
Some cells are subject to either Etoposide or Triton X-100
Etoposide is a strong chemotherapeutic drug, used for Ewing’s
sarcoma, lung cancer, testicular cancer, lymphoma, non-lymphocytic
leukemia, and glioblastoma multiforme
Triton X-100 raptures the cellular membrane and results in the necrotic
death of the cells
The effect of TiO2
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
24 / 45
Toxicological Evaluation of Titania Nanoparticles
Titania Nanoparticles
Claim: Titania Nanoparticles / Anatase (i.e., TiO2 ) is potentially
toxic
Training Set:
Etoposide vs. control cells
Triton X-100 vs. control cells
Test set: Cells that are subject to TiO2
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
25 / 45
Toxicological Evaluation of Titania Nanoparticles
Results: Cross Validation
4
Apoptotic
Healthy
Functional Distance / Arbit. Units
3.5
3
2.5
2
1.5
1
0.5
0
ï0.5
ï1
ï1.5
0
2
4
6
8
10
12
14
16
18
20
Sample ID
Figure: Cross-validation results with leave-one-out method: apoptotic vs. healthy.
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
26 / 45
Toxicological Evaluation of Titania Nanoparticles
Results: Cross Validation
5
Necrotic
Healthy
Functional Distance / Arbit. Units
4
3
2
1
0
ï1
ï2
ï3
ï4
0
2
4
6
8
10
12
14
16
18
20
Sample ID
Figure: Cross-validation results with leave-one-out method: necrotic vs. healthy.
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
27 / 45
Toxicological Evaluation of Titania Nanoparticles
Results: Apoptotic-Healthy
Figure: Progression from healthy to apoptotic along the course of the 36 hrs.
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
28 / 45
Toxicological Evaluation of Titania Nanoparticles
Results: Necrotic-Healthy
Figure: Inconclusive since the distances from the separation plane are scattered along
both regions.
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
29 / 45
Toxicological Evaluation of Titania Nanoparticles
Remarks
Classification between apoptotic and healthy cells is in agreement
with previous literature
There is a progression from healthy to apoptotic for Titania exposed
cells in 36 hrs
Classification between necrotic and healthy cells did not show any
particular trend
The MTT assay shows the cytotoxicity of Titania nanoparticles,
which is in agreement with other reports claiming that titania
nanoparticles are potentially toxic
MTT assay: Laboratory test for measuring the activity of enzymes that
reduce MTT to formazan giving a purple color
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
30 / 45
Current & Future Research
Outline
1
Introduction
2
Support Vector Machine Classifiers
3
Heat Induced Cellular Death
4
Toxicological Evaluation of Titania Nanoparticles
5
Current & Future Research
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
31 / 45
Current & Future Research
Research Directions
Observation: SVM classifier is extremely sensitive to outliers
Robust techniques: SVMs with hard margin loss
Minimize the number of misclassified points
n
min
w,b,z
subject to
X
1
kwk2 + C
zi
2
i=1
yi (hw, xi i + b) ≥ 1
if zi = 0, i = 1, . . . , n
zi ∈ {0, 1}
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
i = 1, . . . , n
March 30, 2011
32 / 45
Current & Future Research
SVMs with Hard Margin Loss
The formulation can accommodate nonlinear kernel functions
Convex quadratic IP formulation
Proven to be N P-hard
More robust (i.e., less sensitive to outliers compared to Hinge loss
function)
Universally consistent
A classifier is consistent if the probability of misclassification converges
in expectation to a Bayes optimal rule as sample size is increased
A classifier is universally consistent if it is consistent for all distributions
of data
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
33 / 45
Current & Future Research
SVMs with Hinge Loss
min
w,b,ξ
subject to
n
C X 2
1
2
kwk +
ξi
2
2
i=1
yi (hw, xi i + b) ≥ 1 − ξi
i = 1, . . . , n
Feature 1
Control Cell
Death Cell
Feature 2
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
34 / 45
Current & Future Research
SVMs with Hinge Loss
min
w,b,ξ
subject to
n
C X 2
1
2
kwk +
ξi
2
2
i=1
yi (hw, xi i + b) ≥ 1 − ξi
i = 1, . . . , n
Feature 1
Control Cell
Death Cell
Feature 2
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
34 / 45
Current & Future Research
SVMs with Hinge Loss
min
w,b,ξ
subject to
n
C X 2
1
2
kwk +
ξi
2
2
i=1
yi (hw, xi i + b) ≥ 1 − ξi
i = 1, . . . , n
Feature 1
Control Cell
Death Cell
Feature 2
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
34 / 45
Current & Future Research
SVMs with Hinge Loss
min
w,b,ξ
subject to
n
C X 2
1
2
kwk +
ξi
2
2
i=1
yi (hw, xi i + b) ≥ 1 − ξi
i = 1, . . . , n
Feature 1
Control Cell
Death Cell
Feature 2
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
34 / 45
Current & Future Research
SVMs with Hard Margin Loss
n
min
w,b,z
subject to
X
1
kwk2 + C
zi
2
i=1
yi (hw, xi i + b) ≥ 1
if zi = 0, i = 1, . . . , n
zi ∈ {0, 1}
i = 1, . . . , n
Feature 1
Control Cell
Death Cell
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
Feature 2
March 30, 2011
35 / 45
Current & Future Research
SVMs with Hard Margin Loss
n
min
w,b,z
subject to
X
1
kwk2 + C
zi
2
i=1
yi (hw, xi i + b) ≥ 1
if zi = 0, i = 1, . . . , n
zi ∈ {0, 1}
i = 1, . . . , n
Feature 1
Control Cell
Death Cell
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
Feature 2
March 30, 2011
35 / 45
Current & Future Research
Multiple Instance Learning - Drug Activity Prediction
Drug
Conformation
c1
c2
c3
c4
c5
c6
c7
c8
c9
c10
c11
c12
c13
Naproxen
50
20
30
20
30
60
5
10
40
60
10
10
50
Sodium
50
30
20
40
30
45
5
40
10
50
10
50
10
Label
Effective
Effective
Effective
Ineffective
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
36 / 45
Current & Future Research
Multiple Instance Learning - Drug Activity Prediction
Drug
Conformation
c1
c2
c3
c4
c5
c6
c7
c8
c9
c10
c11
c12
c13
Naproxen
50
20
30
20
30
60
5
10
40
60
10
10
50
Sodium
50
30
20
40
30
45
5
40
10
50
10
50
10
Label
Effective
Effective
Effective
Ineffective
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
36 / 45
Current & Future Research
Multiple Instance Learning - Drug Activity Prediction
}
Effective (+)
Ineffective (−)
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
37 / 45
Current & Future Research
Multiple Instance Learning - Drug Activity Prediction
Naproxen
}
Effective (+)
Ineffective (−)
Sodium
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
37 / 45
Current & Future Research
Multiple Instance Learning - Drug Activity Prediction
Naproxen
Actual Positives
}
Effective (+)
Ineffective (−)
Sodium
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
37 / 45
Current & Future Research
Multiple Instance Learning - Drug Activity Prediction
Naproxen
}
Effective (+)
Ineffective (−)
Select at least one
from each positive bag
such that the margin
is maximized.
Sodium
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
37 / 45
Current & Future Research
Multiple Instance Learning - Drug Activity Prediction
Naproxen
Keep all negative instances
}
Effective (+)
Ineffective (−)
Select at least one
from each positive bag
such that the margin
is maximized.
Sodium
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
37 / 45
Current & Future Research
Multiple Instance Learning - Drug Activity Prediction
Naproxen
}
Effective (+)
Ineffective (−)
Sodium
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
37 / 45
Current & Future Research
Other Applications of Multiple Instance Learning
Drug/molecular activity prediction
Medical image annotation
Analysis of noisy time series data (e.g., local field potential, EEG)
Nanotoxicity
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
38 / 45
Current & Future Research
Hard Margin Multiple Instance Classification:
Formulation 1
ηi =
1
0
if instance i is selected
otherwise
min
X
1
kwk2 + C
vi
2
s.t.
− hw, xi i − b ≥ 1 − Mvi
∀i ∈ I −
hw, xi i + b ≥ 1 − Mvi − M(1 − ηi )
X
ηi = 1
∀i ∈ I +
i
∀j ∈ J +
i∈Ij
vi ∈ {0, 1}
∀i
ηi ∈ {0, 1}
∀i ∈ I +
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
39 / 45
Current & Future Research
Hard Margin Multiple Instance Classification:
Formulation 2
v̂j : Minimum misclassification for each positive bag.
min
X
X
1
kwk2 + C
vi + C
v̂j
2
+
−
i∈I
s.t.
j∈J
− (hw, xi i + b) ≥ 1 − Mvi ;
∀i ∈ I −
hw, xi i + b ≥ 1 − Mvi ;
X
v̂j =
ηi vi ;
∀i ∈ I +
∀j ∈ J +
i∈Ij
X
ηi = 1;
∀j ∈ J +
0 ≤ ηi ≤ 1
∀i ∈ I +
v̂j ∈ {0, 1}
∀j ∈ J +
vi ∈ {0, 1}
∀i
i∈Ij
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
40 / 45
Current & Future Research
Hard Margin Multiple Instance Classification:
Linearized Formulation 2
min
X
X
1
kwk2 + C
vi + C
v̂j
2
+
−
i∈I
s.t.
j∈J
− (hw, xi i + b) ≥ 1 − Mvi ;
∀i ∈ I −
hw, xi i + b ≥ 1 − Mvi ;
X
v̂j =
ẑi ;
∀i ∈ I +
∀j ∈ J +
i∈Ij
ẑi ≥ −1 + ηi + vi ;
∀i ∈ I +
ẑi ≤ ηi ;
∀i ∈ I +
ẑi ≤ vi ;
X
ηi = 1;
∀i ∈ I +
∀j ∈ J +
i∈Ij
0 ≤ ẑi ≤ 1
∀i ∈ I +
v̂j ∈ {0, 1}
∀j ∈ J +
Erhun Kundakcioglu (University of η
Houston)
1} for Toxic Evaluation of Nanoparticles
i ∈ {0,SVMs
∀i ∈March
I + 30, 2011
41 / 45
Current & Future Research
Hard Margin Multiple Instance Classification:
Formulation 3
min
X
X
1
kwk2 + C
vi + C
v̂j
2
+
−
i∈I
s.t.
j∈J
− (hw, xi i + b) ≥ 1 − Mvi ;
∀i ∈ I −
hw, xi i + b ≥ 1 − Mvi ;
X
v̂j ≥
vi − |Ij | + 1;
∀i ∈ I +
∀j ∈ J +
i∈Ij
vi ∈ {0, 1}
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
∀i
March 30, 2011
42 / 45
Current & Future Research
Results
Molecular activity prediction
Musk Data: A. Asuncion, D. Newman, UCI machine learning repository. http://mlearn.ics.uci.edu
# of feat.
10
20
40
80
166
10
20
40
80
166
10
20
40
80
166
# of ins.
40
40
40
40
40
80
80
80
80
80
160
160
160
160
160
F1
0.052
0.064
0.128
0.371
0.777
0.728
0.414
1.164
3.606
14.846
N/A
62.455
42.65
59.655
152.17
F2
0.092
0.091
0.157
0.328
0.681
0.744
0.722
1.618
4.648
9.142
127.44
5.51
18.76
66.14
114.505
F3
0.056
0.068
0.135
0.302
0.661
0.644
0.506
1.476
3.028
17.548
91.31
45.555
53.975
45.035
242.31
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
CP1
0.368
0.218
0.28
0.625
1.075
294.418
2.676
4.916
8.23
13.384
N/A
128.82
71.695
1767.635
583.665
CP3
0.074
0.057
0.107
0.283
0.786
1.412
0.788
0.454
1.586
4.444
N/A
10.555
9.825
8.025
20.395
March 30, 2011
43 / 45
Current & Future Research
Results
Molecular activity prediction
Musk Data: A. Asuncion, D. Newman, UCI machine learning repository. http://mlearn.ics.uci.edu
# of feat.
10
20
40
80
166
10
20
40
80
166
10
20
40
80
166
# of ins.
200
200
200
200
200
300
300
300
300
300
400
400
400
400
400
F1
N/A
65.14
30.035
191.87
677.84
N/A
95.54
808.54
N/A
N/A
N/A
N/A
N/A
N/A
N/A
F2
371.2
13.625
805.17
105.38
584.21
N/A
124.45
357.99
N/A
N/A
N/A
N/A
435.8633
N/A
N/A
F3
70.35
6.265
58.815
N/A
572.89
N/A
87.48
N/A
N/A
N/A
N/A
N/A
304.36
N/A
N/A
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
CP1
N/A
N/A
87.33
N/A
770.43
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
CP3
N/A
251.465
12.72
21.805
35.055
N/A
N/A
331.77
375.79
152
N/A
N/A
644.69
N/A
469.43
March 30, 2011
44 / 45
Acknowledgments
Georgios Pyrgiotakis
Particle Engineering Research Center, University of Florida
Mohammad H. Poursaeidi
Department of Industrial Engineering, University of Houston
Q&A
Erhun Kundakcioglu (University of Houston) SVMs for Toxic Evaluation of Nanoparticles
March 30, 2011
45 / 45