Computational Intelligence
Methods & Decision
Support Tools in Cultural
Materials
Anastasios Doulamis, Anastasia Kioussi, Maria Karoglou,
Klio Lakiotaki, Ekaterini Delegou,
Nikolaos Matsatsinis and Antonia Moropoulou
National Technical University of Athens
National Technical University of Athens,
School of Chemical Engineering
Computational Intelligence &
Decision Support Tools
• Assist experts to take solid decisions
• Reject non-preferable solutions
– Reduces the costs
• Identify hidden knowledge
– Image processing/analysis, computer vision
• Improve validation performances
• Results on optimal consolidation of cultural
material
National Technical University of Athens,
School of Chemical Engineering
Outlines
National Technical University of Athens,
School of Chemical Engineering
Optimal Consolidation of
Cultural Heritage Material
• Cultural heritage protection => targeted restoration
actions to increase monuments’ lifetime.
– conservation materials
• The performance of each material on the restoration
significantly differs with respect to its type, chemical
properties and the building substrate.
• Design phase: A decision support system which will
assist the engineer to extract optimal conclusions.
• Today, such section is expert-dependent process mainly exploiting
her/his experience.
National Technical University of Athens,
School of Chemical Engineering
Computational Intelligence in
Cultural Material Consolidation
•
We have applied different types of intelligent
tools for optimal selecting the most suitable
conservation materials
Linear
regression
Supervised
Neural
Intelligence
National Technical University of Athens,
School of Chemical Engineering
Fuzzy Kmeans &
Neural
Intelligence
AI & DSS for Conservation
Interventions
• Applications to two Cases of Conservation
Interventions
– Consolidation of Materials/Structures
– Cleaning of architectural surfaces
National Technical University of Athens,
School of Chemical Engineering
Consolidation interventions
How to support the decision making in choosing the most appropriate
consolidation material
The consolidation materials and interventions used intend to the :
 Modification of micro structural characteristics of the stone, leading to
lessening of stone susceptibility to salt decay
 Prevention of decay due to grains de-cohesion
 Amelioration of mechanical characteristics of the stone
Main categories of consolidation products are:
• Inorganic Materials
• Nano-limes
• Organic Materials
• Alkoxysilanes
National Technical University of Athens,
School of Chemical Engineering
Validation in Lab and in the
Monument Scale
National Technical University of Athens,
School of Chemical Engineering
DDS OUTPUT
Parameter
Climax
Availability in Greece
Parameter
Penetration depth
Unknown / Not satisfactory /
depend on any factors/
depends on the solvent / good
/ very good with the use of
specific solvent / very good/
Color change
No change /depend on the
surface / depend on the
excess of material at surface/
medium / high/unknown
Yes/No / Unknown
Irreversibility
0-10
Durability at environmental
loads
Chemical compatibility
Climax
0-10
Capillary absorption of
water
Low / medium / high/unknown
0-10
Change of hardness
Low / medium / high/unknown
Standards
Yes/No /
Unknown
Creation of film
0/1/2
National Technical University of Athens,
School of Chemical Engineering
Criteria Adopted
Criteria Adopted
Availability
Inversibility
Resistance
to Chemical
Environmental Compatibility
Standardized
Rules
Loads
Yes/No
Numerical
Numerical
Numerical
/Unknown
Values
Values
Values
Filming
Penetration
Discolored
Water
depth
Numerical
Quality Values
Values
National Technical University of Athens,
School of Chemical Engineering
Binary Values
Hardness
absorption
Quality Values
Quality Values
Quality Values
Experiments Set-Up
• Two scenarios for different application
substrate:
• The ranking is primarily based on chemical
composition of stones
1st Scenario: Calcareous Stone (35 samples)
2nd Scenario: Silicon-based Stone (34 samples)
In future, more parameters will be included like micro-structural
characteristics of material, mechanical properties etc,
National Technical University of Athens,
School of Chemical Engineering
A Feed-Forward Neural Network
w10,1
w11
w10,2
w10,q
w20,1
Input
Vector
..
.
w21
w20,2
w20,q
w 0J ,1
..
.
z
wq1
w 0J ,2
w 0J ,q

wi0  wi0,1 , wi0,2  wi0,q

w1  w11 , w21  wq1
National Technical University of Athens,
School of Chemical Engineering


T
T
i  1,2 J
Neural Networks Set-Up
• Three categories
(preferred, neutral
non-preferred)
• Two categories
(preferred, nonpreferred)
Continuous
Outputs
• Preference Order
Quantized
Outputs
National Technical University of Athens,
School of Chemical Engineering
• Two layers networks
• Constructive training
method
Different
Network Sizes
Results- Generic
2
1.4
Neural Output in Training Samples
Neural Output inTesting Samples
Desired output in Training Samples
Desired Output inTesting Samples
1.5
1.2
1
0.5
Selection Preference
Selection Preference
1
0
0.8
0.6
-0.5
0.4
-1
0.2
-1.5
0
5
10
15
20
25
Number of Samples
0
1
2
3
Average Error over 10 randomly
partitioned datasets when 20 hidden neurons are selected
Training error
Testing Error
1.5%
23.7%
National Technical University of Athens,
School of Chemical Engineering
4
5
Number of Samples
6
7
8
Effect of Network Size-Generic
55
50
Error in Testing Dataset (%)
45
40
35
30
25
20
2
4
6
8
10
12
Network Size
National Technical University of Athens,
School of Chemical Engineering
14
16
18
20
Results- Scenario 1
1.5
Neural Output inTesting Samples
Desired Output inTesting Samples
Selection Preference
1
0.5
0
-0.5
-1
1
test_output
2
3
4
5
Number of Samples
6
7
Average Error over 70 randomly
partitioned datasets when 20 hidden neurons are selected
National Technical University of Athens,
School of Chemical Engineering
Training error
Testing Error
1.2%
22.4%
Combined Fuzzy K-means &
Neural Networks
Testing on
Data
Neural
Networks
(supervise
d)
Fuzzy
K_Means
(unsupervised)
National Technical University of Athens,
School of Chemical Engineering
Results –Scenarios 1,2
Average Error over 70 randomly
partitioned datasets when 20 hidden neurons are selected
Average Error over 70 randomly
partitioned datasets when 20 hidden neurons are selected
SCENARIO 1
SCENARIO 2
Training error
Testing Error
0.4%
5.7%
National Technical University of Athens,
School of Chemical Engineering
Training error
Testing Error
0.7%
2.6%
The UTA* algorithm
AR
Alternative
Reference
set
Acronal 500 D
Conservare® H
100
Ludox HS30
Mowilith 30
Paraloid B72
g1
g2
g3
Reversibility
Availability
Hardness
6
yes
2
6
3
4
5
Ranking
order
1
wij  ui ( gij 1 )  ui ( gij )  0,
g4
low
Penetration
depth
good
k=1
no
low
satisfactory
k=2
7
2
no
yes
medium
large
good
dependent
k=3
k=4
2
i=1
yes
i=2
7large
i=3
Very good
i=4
k=5
i  1, 2,..., n



[  ( k )    ( k )]
[min] z 

 1

subject to
(a , a )  
if ak ak 1 


k k 1

 k

(
a
,
a
)

0
if
a
a

k
k

1
k
k

1


 n ai 1

wij  1

 i 1 j 1




 wij  0,  ( k )  0,  ( k )  0 i, j and k

and j  1, 2,..., ai  1
ui ( g1i )  0
i  1, 2,..., n


j 1

j
wit i  1, 2,..., n and j  2,3,..., ai 1
ui ( gi ) 

t 1


(ak , ak 1)  u[g(ak )]    ( )    ( )  u[g(a 1)]
  ( 1)    ( 1)

ai 1

*
National
Technical
University ofu Athens,
Post-optimality
analysis
i ( gi )  wij i  1,2,..., n
School of Chemical Engineering
j 1
m
[ (a )  (a )]  z  


k
k 1
Criteria
weights
*
k
19
Results-Scenario 1
Criteria Adopted
Availability
Inversibility Resistance
to
Chemical
Standardize
Compatibili d Rules
Environmen ty
tal Loads
Yes/No
Numerical
Numerical
Numerical
Binary
/Unknown
Values
Values
Values
Values
Filming
Penetration
Discolored
Water
Hardness
depth
absorption
Numerical
Quality
Quality
Quality
Quality
Values
Values
Values
Values
Values
National Technical University of Athens,
School of Chemical Engineering
Results-Scenario 2
Scenario 1
0.0085
0.120704544
0.054324815
0.096816889
0.097848487
availability in Greece
reversibility
0.120795954
0.067767522
Scenario 2
0.062514711
0.10829301
durability
0.09743482
chemical compatibility
durability
0.114843951
0.103792321 chemical compatibility
standards
film
film
penetration depth
0.030142594
color change
0.299685734
reversibility
standards
penetration depth
0.088442145
0.048352749
availability in Greece
water absorption
0.090789818
National Technical University of Athens,
School of Chemical Engineering
color change
0.059719141
0.323630796
water absorption
hardness
Porous Biocalcarenite
Pilot scale treatments for porous stone
consolidation in the Medieval City of Rhodes
Materials
LUDOX HS30 (PL)
Silbond HT20 (PH)
Rhodorsil RC70 (RP)
Acryl Siliconic Resin (EU)
National Technical University of Athens,
School of Chemical Engineering
Evaluation of the Compatibility of Conservation Interventions in lab
Water Absorption of Porous Stone & Consolidated Porous Stones
25
Imbibed Water, wgt. %
RPS
CSPH2
20
CSPL3
15
10
CSEU1
5
CSRP4
0
0
10
20
30
40
50
60
70
80
(Time, sec) 1/2
Changes
of water
absorption
curves (capillary) of consolidated porous
National Technical
University
of Athens,
stones
monitoring
by infrared thermography in the laboratory
School of and
Chemical
Engineering
IR Thermography
Investigation of Capillary Rise, Monument Scale
Investigated Surface: Gate
of St. Paul, Medieval
Fortifications of Rhodes
Evaluation of Pilot Consolidation Interventions, Monument Scale
Investigated
Surface:
Entrance of
Moat, Medieval
Fortifications
of Rhodes
15 months after the applications
28 months after the applications
Consolidation Materials: LUDOX HS30 (PL), Silbond HT20 (PH), Rhodorsil RC70 (RP)
acryl siliconic
resin
(EU) University of Athens,
National
Technical
School of Chemical Engineering
Validation of the results
-in laboratory (various analytical techniques
like capillary absorption test, mercury
intrusion porosimetry etc)
-in monument scale (non-destructive testing)
Feedback: Changes in
materials ranking
National Technical University of Athens,
School of Chemical Engineering
Computational Intelligence on
Cleaning Interventions Assessment
• We have applied the aforementioned methodology for
supporting the decision making on the assessment of
pilot cleaning interventions on marbles surfaces
• The application sites are located on the historic
buildings of National Library of Greece (NLG), and
National Archaeological Museum in Athens-Greece
(NAM)
• The diagnosed decay patterns are black grey crusts,
washed out surfaces, and fractured surfaces of marble.
National Technical University of Athens,
School of Chemical Engineering
Presentation of Applications Sites
Smooth Marble
Architectural Surface
NLG facade
Different protection degree from
rain wash
North Facade
National Technical University of Athens,
School of Chemical Engineering
Presentation of Applications Sites
Relief Marble
Architectural Surface
East Side, Full Protected
from rain Wash
NAM West Façade, North Column
Relief Marble Architectural Surface
Column Capital
East Side, Full Protected from rain
Wash
Relief Marble Architectural Surface
North Side, Different Protection
Degree from Rain Wash
National Technical University of Athens,
School of Chemical Engineering
North Column East Side
Decay Diagnosis - NLG
Depending on the protection
degree from the projected
horizontal geison
FOM
Friable
black-grey
crust
Cohesive
black-grey
crust
Inter-granular
fissured marble
National Technical University of Athens,
School of Chemical Engineering
SEM
FTIR
Decay Diagnosis - NAM
FOM
SEM
Black-grey
crust
East
orientation
FTIR
Washed out
surfaces
FOM
SEM
North
orientation
FTIR
National Technical University of Athens,
School of Chemical Engineering
Decay Diagnosis - NAM
Black-grey crust of great variety
FOM
regarding: the width, the presence or not of
FOM
SEM
Barite (patina), the location in the anthemia
relief, relief side
East orientation
Relief face
right part
FTIR
Relief side,
central part
Relief face
central part
FOM
FOM
SEM
FTIR
National Technical University of Athens,
School of Chemical Engineering
FTIR
SEM
In situ application of pilot cleaning interventions – Monument scale - NLG
Smooth marble architectural surfaces, black-grey crust, inter-granular fissured marble
Pab22,
P. ΑΒ57, 2h
Pnc22,
P. (NH4)2CO3, 2h
Pm22,
P. Mora, 2h
Ps32
P. Sepiolite,
3.5h
Pat2,
Atomized
water
Pm24,
P. Mora, 1.5h
Ps34
P. Sepiolite, 3h
Pab24,
Pnc24,
National P.
Technical
University
of Athens,
ΑΒ57, 1,5h
P. (NH4)2CO3, 1.5h
School of Chemical Engineering
In situ application of pilot cleaning interventions – Monument scale - NLG
Smooth marble architectural surfaces, black-grey crust, inter-granular fissured marble
Pnc12,
P. (NH4)2CO3, 1h
Pab12,
P. ΑΒ57, 1h
Pab14,
P. ΑΒ57, 1h
National Technical University
of Athens,
Pnc14,
P. (NH4)2CO3, 1h
School of Chemical Engineering
Ped12,
Π. EDTA, 1h
Ped14,
P. EDTA, 1h
In situ application of pilot cleaning interventions – Monument scale - NAM
Relief marble architectural surfaces, black-grey crust, east orientation
Ke
Ke
Ion
exchange
resin with
solution of
(NH4)2CO3,
40min
Ke2a
Ke3b
Π. ΑΒ57,
5min
Ion
exchange
resin with
deionised
water, 30min
Biological poultice
Ke
Ke1c
Wet micro-blasting method
•Spherical particles of CaCO3 d<80μm,
•Function pressure 0.5bar,
•Proportion of CaCO3/water: 1/3,
•d nozzle 12mm
• working distance
50 cm
University
of Athens,
National Technical
School of Chemical Engineering
KeG3
In situ application of pilot cleaning interventions – Monument scale - NAM
Relief marble architectural surfaces, washed out surfaces, north orientation
Kn
P. ΑΒ57, 5min
Kn3c
Ion exchange resin with
deionized water, 40min
Kn3b
Ion exchange resin with
deionized water, 10min
Kn2a
Ion exchange resin with
solution of (NH4)2CO3, 20min
Kn1a
Ion exchange resin with
solution of (NH4)2CO3, 10min
Kn1b
Double application of Ion
exchange resin with
solution of(NH4)2CO3,
2x10min
Kn1c
Kn2b
Ion exchange resin with
deionized water, 20min
Kn2c
Double application of Ion
exchange resin with
deionized water, 2x20min
Kn2d
National Technical University of Athens,
School of Chemical Engineering
Wet microblasting
method
In situ application of pilot cleaning interventions – Monument scale - NAM
Relief marble architectural surfaces of capital, east orientation
Kke
Ion exchange resin with
solution of (NH4)2CO3, 10min
Ion exchange resin with
solution of (NH4)2CO3, 40min
Ion exchange resin with
deionized water, 60min
Kke1a
Kke2a
Kke3a1
Kke3a2
Ion exchange resin with
deionized water, 10min
Ion exchange
resin with
deionized water,
20min
Wet micro-blasting method
kkeg51
Kke1b
Ion exchange
resin with of Athens,
National Technical University
deionized water,
School of Chemical Engineering
30min
Kke2b
Kke3b
Ion exchange resin
with solution of
(NH4)2CO3, 20min
kkee3ba
P. ΑΒ57, 5+ 15 min
Assessment of Cleaning Interventions:
Techniques & Parameters
Applying, after cleaning the same experimental techniques that
were applied before cleaning, a methodological approach for
cleaning assessment is compiled.
Comparison of the marble surfaces physico-chemical
characteristics before & after cleaning, along with recording the
variations of the corresponding critical parameters, makes
feasible the recommendation of the best cleaning according to
the examined case.
Digital Image Processing
of SEM images:
fracturing of the surface
Shape factor
(a roughness factor)
SEM-EDS: chemical &
mineralogical composition
stratification, total crust width, patina,
macro-crystalline gypsum layer width,
micro-crystalline gypsum layer width
Fracture Density
Patina preservation index
Friability index
Preservation index of gypsum layer
National Technical University of Athens,
School of Chemical Engineering
Assessment of Cleaning Interventions:
Techniques & Parameters
Laser Profilometry: texture &
roughness assessment
Roughness Rq (μm)
Surface area,
(ratio of actual to projected area)
Colorimetry
CIELab color space:
evaluation of color modifications
L, Luminosity
total colour difference
ΔE
difference in
difference in
red-green
blue-yellow
a* Technical University of Athens,
National
b*
School of Chemical Engineering
Assessment Criteria & Critical Assessment Parameters of Cleaning Interventions –
Experimental Techniques
Cleaning Assessment Criteria
Chemical-mineralogical
composition of the surfacesstratification
Preservation of Patina,
Preservation of Authentic
Material,
Scanning Electron
Microscopy with
Energy Dispersive Xray Spectroscopy
Color
Texture, Morphology &
Surface Cohesion - Surface
Microstructure
Removal of Black
Depositions
Surface Preservation State –
Decay Susceptibility –
Durability
Roughness, Rq,
Fracture Density
Ratio of actual to projected
area - Surface area
Colorimetry
Laser
Profilometry
Digital Image Analysis of
Scanning Electron
Microscopy Images
Experimental Techniques for Measuring Critical
Assessment Parameters of Cleaning
National Technical University of Athens,
Critical Assessment Parameters of Cleaning
School of Chemical Engineering
Total Color
Difference,
ΔΕ
Colorimetry
Monitoring of Surface Preservation State –
Durability of Marble
1.
2.
Decay patterns distribution on the building are mainly controlled by
material location, orientation, protection from rain-wash, atmospheric
conditions and pollution.
However, the long-term aesthetical and structural properties of marble are
closely related to the lateral and vertical distribution of particulate matter
and salts-gypsum, as well as to the bonding of the calcite grains in the
matrix; factors that are strongly affected by cleaning.
Therefore there is an urgent need for a tool to interrelate information – data,
between space and physical-chemical characteristics of building materialsmarble, taking into account their variation over time.
The suggested methodological tool is a GIS platform
National Technical University of Athens,
School of Chemical Engineering
Materials Mapping in GIS, Façade, National Archaeological Museum
Working scale: building’s facade
materials mapping performed using GIS based on the acquired data by NDT and inlab analytical techniques.
Acropolis
Athens
The area extend of each investigated material was calculated by the means
ofof GIS
 Historic plaster area was 248.56m2, whereas new plaster area was 13.63m2
National Technical University of Athens,
School of Chemical Engineering
Materials Mapping in GIS, Façade detail, National Archaeological Museum
Working scale: building’s facade
Digital decay mapping performed using GIS based on the acquired data by NDT and in-lab
analytical techniques.
Brown color depicts areas of coating total detachment and intense fracturing total area on west
façade: 25.56m2
Acropolis
Blue color represents the areas of coating loose interface to the substrate total area on
westof Athens
façade: 219.72m2
National Technical University of Athens,
School of Chemical Engineering
Façade detail - National Archaeological Museum
•material type
•applied cleaning method
•application details
•application area
•cost
National Technical University of Athens,
School of Chemical Engineering
Acropolis of Athens
Working scale: building’s facade
GIS thematic maps for decay & pilot conservation interventions
Recording & ascribing attributes to features
Attribute db,
(physical-chemical data,
indexes of building material
preservation state, before and
after conservation)
GIS db,
(topological characteristics
like area, perimeter,
adjacency, etc)
Relational
Data Base
National Technical University of Athens,
School of Chemical Engineering
Spatial Classification
of Decay.
Different Physical-chemical
characteristics & spatial
properties
Decay thematic map for the capital
surface, along with RDBs for both
front & side anthemia surfaces
RDB
RDB
Spatial
Classification
of
Conservation
Interventions
Pilot conservation interventions’
thematic map for the capital surface,
along with the RDB of the front
anthemia surface
RDB
GIS analysis using Boolean and logical
operations on decay thematic map for
the capital surface
Spatial entity in
compliance with the
combined
expression
central area of the
anthemia relief
Which is the entity that
comply with:
1) roughness ≥ 7,
2) fracture density ≥ 35.3
RDB
Suggested Information Management System
ANALYSIS &
OPERATIONAL
TOOLS
RELATIONAL
DATABASE
(ATTRIBUTES)
GIS
SPATIAL
DATA
Using the continuous process of GIS
platform datasets concerning building
pathology & conservation interventions
are recorded, correlated, distributed &
attributed to space in different working
scales during different time periods
Support on decision making for
cleaning assessment using
Computational Intelligence
Results-Crust
4.5
3.5
KMeans Output inTesting Samples
Desired Output inTesting Samples
4
3
3
Selection Preference
Selection Preference
3.5
2.5
2
2.5
2
1.5
1.5
1
0.5
0
2
4
6
8
10
Number of Samples
12
14
16
18
National Technical University of Athens,
School of Chemical Engineering
1
1
Neural Output inTesting Samples
Desired Output inTesting Samples
1.5
2
2.5
3
3.5
4
Number of Samples
4.5
5
5.5
6
Results-Washed-out
4.5
4.5
KMeans Output inTesting Samples
Desired Output inTesting Samples
3.5
3.5
3
3
2.5
2
2.5
2
1.5
1.5
1
1
0.5
1
Neural Output inTesting Samples
Desired Output inTesting Samples
4
Selection Preference
Selection Preference
4
2
3
4
5
Number of Samples
6
7
8
9
National Technical University of Athens,
School of Chemical Engineering
0.5
1
1.2
1.4
1.6
1.8
2
2.2
Number of Samples
2.4
2.6
2.8
3
Results-Fractured
4.5
4.5
KMeans Output inTesting Samples
Desired Output inTesting Samples
4
4
3.5
3.5
3
Selection Preference
Selection Preference
KMeans Output inTesting Samples
Desired Output inTesting Samples
2.5
2
3
2.5
2
1.5
1.5
1
1
0.5
1
2
3
4
Number of Samples
5
6
7
National Technical University of Athens,
School of Chemical Engineering
0.5
1
1.2
1.4
1.6
1.8
2
2.2
Number of Samples
2.4
2.6
2.8
3
Results-Overall
Average Error over 100 randomly
Average Error over 100 randomly
partitioned datasets when 20 hidden neurons are selected
partitioned datasets when 20 hidden neurons are selected
CRUST
FRACTURED
Training error
Testing Error
Training error
Testing Error
5.3%
18.5%
3.0%
11.2%
Average Error over 100 randomly
partitioned datasets when 20 hidden neurons are selected
WASHED OUT
Training error
Testing Error
4.2%
13.7%
National Technical University of Athens,
School of Chemical Engineering
Interoperable Description of
Cultural Material Content
• Cultural material should be stored according to standardized formats
•
•
Interoperability, Unified Accessibility, Portability, Exchangeability
Metadata: Description of data
• Extended Markup Language Schemes (XML’s)
• MPEG-7 (visual description schemes)
• MPEG-21 (resources and rights descriptions schemes)
• Database schemas (MySQL, Oracle)
• Tools for efficient Knowledge Search and Content Mining
•
Guidelines for new research efforts at European level
•
Great research effort, data alignment schemes, knowledge representation tools
National Technical University of Athens,
School of Chemical Engineering
Conclusions-Guidelines
•Artificial Intelligence can support automatic decisions
for
• Cultural material consolidation during the design
phase
• Finding the degree of importance for criteria used
•Supervised learning => Feedforward Neural Networks
•Unsupervised learning => Fuzzy k-means
•Linear Regression (UTA *) => degree of importance
•Results on Cleaning
•Validation Performances
•Issues on knowledge systems for cultural material
National Technical University of Athens,
School of Chemical Engineering
Thank for Your Attention!
National Technical University of Athens,
School of Chemical Engineering