Interactive Optimization by
Genetic Algorithms
Cases: Lighting Patterns and Image
Enhancement
Janne Koljonen
Electrical Engineering and Automation,
University of Vaasa
Outline
• Interactive Evolutionary Computation (IEC)
in general.
• Image enhancement.
• LED adaptive luminence lighting system
(LEDall).
• Project work (LEDall).
Interactive optimization by GA
• In Interactive Evolutionary Computation
(IEC), the computational fitness function is
replaced by a human evaluator.
– In other aspects the genetic algorithm may be
as usual.
Application domains of IEC
• In cases, where the favorable output can
be usually evaluated only subjectively, IEC
is applied.
– Such domains are e.g. music, graphics and
image enhancement.
Constraints
• The human intervention is a bottleneck of IEC
system: it is time consuming, difficult and even
boring to evaluate outputs of a system.
• The time constraint and patience of the user can
be overcome by limiting the number of fitness
evaluations.
– The attention has to be paid to the problem
complexity and the algorithm so that search strategy
can be guided gradually from a global phase into a
fine-tune search by the user.
IEC strategies
• A few strategies for the subjective fitness
evaluation have been reported: evaluation
score points with n levels, selection of
elite, which actually corresponds given
score points from 2 levels, and pair-wise
tournament.
• Others?
• How to compare n outputs in parallel/in
sequence?
IEC strategy suggestions
• In addition to a fitness function, the user
can be used to select the genetic
operators that should be applied
– Requires more expertise
• Alternatively, GA could have a pool of
different operators and a mechanism to
learn, which operators are efficient in
different cases.
Image enhancement
• People use image processing tools increasingly
as the costs of digital cameras have decreased.
• However, image processing tools contain
nowadays dozens of filters with a few
parameters each.
– An inexperienced user is barely capable of deciding,
which filters and parameters to use. Presumable the
method of trial and error is applied in such cases,
which is time consuming.
– Moreover, rarely one single filter is enough for the
desired output but a sequence of filters and
integrations of filtered images are required.
Objective
• Image enhancement and image restoration are usually
applied to improve the quality of the pictures or to
emphasize certain features and details.
• The result is another image that meets better the
requirements set for the image in a specific application.
• The difference, by definition, of image enhancement and
image restoration is in the output evaluation.
– While image enhancement is evaluated subjectively, the
objective of restoration is to recover the original image subjected
to e.g. noise or other degradation.
Objective
• Image enhancement and image restoration are usually
applied to improve the quality of the pictures or to
emphasize certain features and details.
• The result is another image that meets better the
requirements set for the image in a specific application.
• The difference, by definition, of image enhancement and
image restoration is in the output evaluation.
– While image enhancement is evaluated subjectively, the
objective of restoration is to recover the original image subjected
to e.g. noise or other degradation.
Objective
• The objective of image pre-processing may be
e.g. to remove noise from the image, to sharpen
the image, to adjust color/gray scale intensities,
or to highlight e.g. edges or other features that
can be used in segmentation and pattern
recognition stages of image analysis.
• Complex image enhancement and analysis
tasks are difficult even for experts. Hence, a
method to boost the search for an image
processing sequence would be advantageous
both for uninitiated and experts.
Applications
• Evolutionary computing or algorithms (EC/EA)
have be applied to partially automate image
enhancement, whose output may be subjected
to visual inspection or act as the input for further
image analysis and pattern recognition stages.
• Usually, the principle is to combine basic image
processing operations drawn from a finite set
and to optimize the relations between the
operations and the internal parameters of to the
operations.
Applications
• Interactive image enhancement optimization methods
have been applied e.g. to magnetic resonance (MR)
image pseudo-colorization using genetic programming.
• It has also been suggested that the user can be modeled
to decrease the need for human intervention.
• Visual image enhancement with a desired output image
has been studied by Nagao et al.
– the objective was to search for, with a GA, an approximation of
the transformation sequence leading to the given output.
• The desired output can also be defined by objective
criteria by the user.
• Pre-processing optimization as a part of pattern
recognition optimization has also been reported in the
literature
– experiments were done with radar signals.
LEDall
• Koljonen et al. (2004) have developed an
interactive LED lighting system to optimize
illumination pattern in close range optical
imaging.
• An I/O board with digital voltage outputs
controls 90 LEDs that are set around the
object to be imaged.
– Different lighting patterns can be searched for
to enhance different features of the image.
• Shadows, illumination levels, etc.
PWM control
• Since cameras (and human eye) have a
relative long expose time (time resolution),
pulse-width-modulation (PWM) can be
used to increase the number of luminance
levels of the LEDs.
– In LEDall 4 levels are used.
Totally 490 lighting combinations allowed!
– Q: How to optimize? A: With GA!
Interactive GA
• Initial population of 9 random lighting patters.
• Resulting images shown as a 3x3 grid of
images.
• User selects 0-8 images that contain favorable
features (parents).
• Illuminations of the parents are operated by
crossover and mutation to create offspring of
potentially more favorable illumination.
• Occationally, new random offspring are created
to retain diversity of the population.
Example
Random lightings
After 3 GA genetations
Applications
• LEDall or a similar device can be utilized
in many places and applications:
Application
Photography
Microscopy
Spectroscopy
Theater/exhibitions etc.
Offices
Architecture
Fitness function
Elitism/fitness points
Resolution
Calibration model fitness/prediction ability
Elitism/fitness points
Illumination power and smoothness
Room lighting design
Improvements/project work
•
•
•
•
•
•
More images, score points?
New (user controlled?) genetic operators?
More LEDs (LEDall2, PIC, Toni Harju)?
Better camera?
New applications?
Semi-automatic fitness funcition?
– Deterministic criteria.
• I/O and frame grapper routines exist
– Native Java functions.
Questions?