Bridging the gap - state-of-the-art
testing research, Explanea, and why
you should care
Robert Feldt
Blekinge Institute of Technology &
Chalmers
All animations have been excluded in this pdf version!
onsdag 27 november 13
Bridging the gap - state-of-the-art
testing research, Explanea, and why
you should care
Robert Feldt
Blekinge Institute of Technology &
Chalmers
@drfeldt
All animations have been excluded in this pdf version!
onsdag 27 november 13
Bridging the gap - state-of-the-art
testing research, Explanea, and why
you should care
Dan Robert Feldt
Blekinge Institute of Technology &
Chalmers
@drfeldt
All animations have been excluded in this pdf version!
onsdag 27 november 13
onsdag 27 november 13
onsdag 27 november 13
We are also humans!
onsdag 27 november 13
Your Use of Research
• New/Better/Alternative Ideas (N)
• Strengthen/Confirm/Refine Beliefs (S)
• Contradict Beliefs (X)
• Point to Future/New/Missed Areas (F)
• Extend/Move Ideas to Your/Other Area (E)
• Attracting personnel
onsdag 27 november 13
onsdag 27 november 13
We’re dead
serious!
onsdag 27 november 13
No Fancy
Pics!
We’re dead
serious!
onsdag 27 november 13
No Fancy
Pics!
Intentionally left blank!
Mysterious...
We’re dead
serious!
onsdag 27 november 13
No Fancy
Pics!
Intentionally left blank!
Mysterious...
Short papers
are for
wimps!
We’re dead
serious!
onsdag 27 november 13
No Fancy
Pics!
Intentionally left blank!
Mysterious...
At least 20
pages!
Short papers
are for
wimps!
We’re dead
serious!
onsdag 27 november 13
onsdag 27 november 13
onsdag 27 november 13
onsdag 27 november 13
onsdag 27 november 13
Öka innovationshöjden
• Få ut forskningsresultat snabbare till näringslivet
• Arbeta närmare varandra
•
Lära av varandra: Tvåvägs utbyte
• Kvantifiera förändringsarbetet (statistik)
•
Räkna kronor/kvalitet - inte gå på magkänsla
• Hitta rätt personer som har mandat att förändra
onsdag 27 november 13
Exempel på ny forskning
• Fem enkla mallar räcker för att förbättra
kravarbetet
• Granskning är bättre än TDD (lite dyrare, men
hittar fler & viktigare problem)
• Kanban är bättre än Scrum (50% kortare ledtider
och 10% färre fel)
• Visuella GUI Tester kan automatisera systemtest
onsdag 27 november 13
Exempel på ny forskning
• Exploratory testing är i många fall bättre än
traditionell testning
• Metod för test design är generellt sett irrelevant –
testaren är viktigare…
• Enkla modeller predikterar vart fel kommer
upptäckas nästan lika bra som avancerade
modeller
• Enkla modeller kan appliceras nästan överallt
onsdag 27 november 13
onsdag 27 november 13
onsdag 27 november 13
onsdag 27 november 13
onsdag 27 november 13
Heatmaps shows “raw” data and
reveals patterns
onsdag 27 november 13
onsdag 27 november 13
Test failures
onsdag 27 november 13
Test failures
Lack of integration
traces!
onsdag 27 november 13
Testing only what is likely to fail
onsdag 27 november 13
Testing only what is likely to fail
onsdag 27 november 13
Testing only what is likely to fail
onsdag 27 november 13
Testing only what is likely to fail
onsdag 27 november 13
Testing only what is likely to fail
onsdag 27 november 13
Testing only what is likely to fail
onsdag 27 november 13
Testing only what is likely to fail
onsdag 27 november 13
Testing only what is likely to fail
Better than “80/20”!
onsdag 27 november 13
1.00
Activation rate
0.75
0.50
0.25
0.00
1 year
2 years
Test case age
onsdag 27 november 13
3 years
4 years
1.00
Failure rate
0.75
0.50
0.25
0.00
1 year
2 years
Test case age
onsdag 27 november 13
3 years
4 years
Results
80
80
60
60
Tree height
Tree height
Scatter plots show the distribution of tree sizes and
heights; target bias objectives are indicated by
crosses
40
20
40
20
100
200
Tree size
Boltzmann Sampler
onsdag 27 november 13
100
Tree size
QuickCheck
200
Results
80
80
60
60
Tree height
Tree height
Scatter plots show the distribution of tree sizes and
heights; target bias objectives are indicated by
crosses
40
20
20
100
200
Tree size
GödelTest
(Decay Distribution)
onsdag 27 november 13
40
100
Tree size
GödelTest
(Decay Histogram)
200
onsdag 27 november 13
We are looking for beta testers!
http://explanea.com
onsdag 27 november 13
Conclusions
•
Visualisations + group discussions lead to high-level
understanding of development patterns and
problems
•
Heatmaps excellent way to visualise large data sets
and see “distant” connections
•
•
•
•
Value is in the learning that happens, NOT in tools!
Post-mortem project analysis NOT enough!
Statistics and machine learning can optimize testing
We need to better bridge the gap between research
and practice. explanea.com is one, new approach.
onsdag 27 november 13
Thank you!
onsdag 27 november 13
Thank you!
Colleagues: R. Torkar, M. Staron
Steve Wittens, acko.net
explanea.com
onsdag 27 november 13
Download

Bridging the gap - state-of-the-art testing research