Browserbite: Accurate Cross-Browser
Testing via Machine Learning Over
Image Features
Nataliia Semenenko*, Tõnis Saar** and
Marlon Dumas*
*{nataliia,marlon.dumas}@ut.ee,
Institute of Computer Science,
University of Tartu, Estonia
**tonis.saar@stacc.ee,
Browsrbite and STACC, Tallinn, Estonia
Outline
•
•
•
•
Introduction
Visual cross-browser testing
Machine learning model
Results and future work
Cross-browser visual testing
Where’s that
button?
Internet Explorer 9
Internet Explorer 8
Goal
• Develop method for cross-browser visual
layout testing
• Replace human labor in visual testing
• Evaluate detected errors
Methods
• DOM (Document Object Model) based:
Mogotest (www.mogotest.com), Browsera
(www.browsera.com)
• Image processing – non-invasive black box
testing – Our current approach
Web page
Static image
Cross-Browser Visual testing
Web page visual segmentation
• Image segmentation into regions of interest
(ROI)
• ROI comparison
www.htcomp.ee
ROI comparison
•
•
•
•
Position
Size
Geometry
Correlation
VS
ROI from WIN7
Chrome
ROI from WIN7
IE8
Visual testing results
Web page
Static image
Image
segmentation
(into ROIs)
ROI
comparison
• Test set of 140 web pages from alexa.com
• 98% recall
• 66% precision
Example of true positive
Example of false positive
ROI comparison + ML
Web page
Static image
Image
segmentation
(into ROIs)
ROI
comparison
Classification
Machine learning
• 140 most popular websites of Estonia
according to www.alexa.com
• 1200 potential incompatibilities
• 40 subjects from 6 countries
• Two classes :False positive vs True postive
• Each ROI pair had 8 judgments
• Inter-rater reliability 0,94
ROI features
•
•
•
•
•
•
10 histogram bins
Correlation index
Horizontal and vertical position
Horizontal and vertical size
Configuration index
Mismatch Density
Machine learning
• Neural network
• Three layers
• 11 neurons in hidden layer
• Five-fold cross-validation
• Classification tree
Results and Conclusions
Measure
Neural network
0.75
Classification
tree
0.844
0.98
0.82
0.792
0.886
0.79
0.78
0.81
0.923
Plain Browserbite
Mogotest
Precision
0.66
Recall
F-score
0.964
Results and conclusions
Tool
Mogotest
CrossCheck [1] WebDiff [2]
BB+ML
Precision
75%
36%
96%
1.
2.
21%
Choudhary, S.R., Prasad, M.R., and Orso, A. (2012). CrossCheck: Combining Crawling and
Differencing to Better Detect Cross-browser Incompatibilities in Web Applications. (ICST),
2012 IEEE Fifth International Conference On, pp. 171–180.
Choudhary, S.R., Versee, H., and Orso, A. (2010). WEBDIFF: Automated identification of
cross-browser issues in web applications. (ICSM), pp. 1–10.
Future work
• Combination of image processing and DOM
methods
• Dynamic content suppression
Thank You!