TDDD89
Lecture 2
fredag 6 november 15
First, some notes on submitting your thesis plans and the upcoming seminar.
Course instructions
Information search
fredag 6 november 15
Finding information
fredag 6 november 15
Learn about the subject area:
use Wikipedia, books and previous course material
Extract keywords that you can use when searching papers.
Use Google Scholar & Unisearch first, specific publications second
fredag 6 november 15
fredag 6 november 15
fredag 6 november 15
How to find
information
”HLA active probing
runtime performance
requirements in a
Wide Area Network”
fredag 6 november 15
{}
HLA
active probing
simulation
fault detection and localization
runtime performance requirements
in a Wide Area Network
fredag 6 november 15
latency, throughput
IP networks
Engineering information
vs Scientific information
fredag 6 november 15
Engineering
Science
Questions
How to solve a
problem
How to explain
something
Reliability
Working
solutions, proven
theories
Cited work
Sources
White papers,
software projects
Reviewed
publications
Iterative search
”On AI”
”On Planning”
”Fie Planner”
”Foo Heuristic”
”On Heuristic Search”
1
fredag 6 november 15
2
3
Scientific publishing
Primary studies
Secondary studies
”What”
fredag 6 november 15
Text books
Scientific publishing
Peer-reviewed
publications
Conference
Proceedings
Journal papers
Technical reports
Non-reviewed
publications
White papers
”How”
fredag 6 november 15
Publication types
P. Kruchten, H. Obbink, and J. Stafford. The past, present,
and future for software architecture. IEEE Software,
23(2):22–30, March–April 2006.
No empirical results. Shares experience on Software
Architecture research and development.
fredag 6 november 15
Publication types
T. K. Paul and M. F. Lau. A systematic literature review on
modified condition and decision coverage. In Proceedings
of the 29th Annual ACM Symposium on Applied Computing,
SAC ’14, pages 1301–1308, New York, NY, USA, 2014.
ACM.
Systematic Literature Review, secondary study. Published at a
conference
fredag 6 november 15
Publication types
C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell,
and A. Wesslén. Experimentation in Software Engineering.
Springer Berlin Heidelberg, 2012.
Guidelines textbook on empirical methods in Software
Engineering
fredag 6 november 15
Publication types
I. Maier, T. Rompf, and M. Odersky. Deprecating the
observer pattern. Technical report, École Polytechnique
Fédérale de Lausanne, 2010.
Technical report, non-reviewed publication. No empirical support
for claims, but suggestions of an architecture.
fredag 6 november 15
Publication types
A. Nilsson, J. Bosch, and C. Berger.Visualizing testing
activities to support continuous integration: A multiple case
study. In G. Cantone and M. Marchesi, editors, Agile Processes
in Software Engineering and Extreme Programming, volume
179 of Lecture Notes in Business Information Processing, pages
171–186. Springer International Publishing, 2014.
Case study, reviewed publication in journal
fredag 6 november 15
Publication types
J. Andrews, L. Briand, and Y. Labiche. Is mutation an
appropriate tool for testing experiments? In Proceedings of
the 27th International Conference on Software Engineering,
ICSE 2005, pages 402–411, May 2005. IEEE Computer
Society.
Experiment, reviewed publication presented at a conference and
published in proceedings from the conference.
fredag 6 november 15
What are results?
Type
Procedure/
technique
Descriptive
Models
Experience
reports
fredag 6 november 15
How?
Formal proofs,
experiments,
statistical
support,
Interviews,
observations,
usage data
Quality
Proper use of
statistics
Properly
accounting for
reality
Real systems &
people
What are strong results?
Table 6. Types of research validation represented in ICSE 2002 submissions and acceptances
Type of validation
Analysis
Evaluation
Experience
Example
Some example, can't tell whether it's toy or actual use
Persuasion
No mention of validation in abstract
TOTAL
Submitted
48 (16%)
21 (7%)
34 (11%)
82 (27%)
6 (2%)
25 (8%)
84(28%)
300(100.0%)
Accepted
11 (26%)
1 (2%)
8 (19%)
16 (37%)
1 (2%)
0 (0.0%)
6 (14%)
43 (100.0%)
Ratio Acc/Sub
23%
5%
24%
20%
17%
0%
7%
14%
M. Shaw. Writing goodValidation
software engineering research papers: Minitutorial.
In Proceedings of the 25th
Validation
International
pages 726–736, Washington, DC, USA,_,,2003.
100%300 Conference on Software Engineering, ICSE ’03,
IEEE Computer Society.
250
80% . . . . . . . . . . . . . . . .
~ :i~ ~
_
-
200
100
50
fredag 6 november 15
o
i
iiiiiiiiiii!
20%.
0% ~,.+~
~
fll .
~'~!!i
~
<,.+" ~
oe
.o"
n
Strong results
Real systems &&
proper analysis
fredag 6 november 15
How to evaluate papers
• Relevance = f(title, year, abstract, citations)
• The more specific the paper, the less
citations?
• Literature reviews: meta studies
• Publication types: journals, conferences, book
chapters
• Refer to the main results of the paper, not
that which is written in the introduction
fredag 6 november 15
What about white
papers/other stuff?
• Use to support existence: ”There are
several implementations of Flux
controllers”
• Not to support claims and propositions:
”Flux controllers are more user friendly
than Flax controllers”
fredag 6 november 15
Evaluation of paper
”Software product lines are related software
products that are customized to different
customers [1]”
[1] Kästner, C., Apel, S., and Kuhlemann, M. Granularity in software product lines. In Proceedings of the
30th International Conference on Software Engineering, ICSE ’08, pages 311–320, New York, USA, 2008.
Not the main result of [1]
[1] Pohl, K., Böckle, G., and van der Linden, F. J. (2005). Software product line engineering: foundations,
principles and techniques. Springer Science & Business Media.
Use above reference instead
fredag 6 november 15
Theory
1. Analysis: what is this? Classifications,
taxonomies, ontologies
2. Explanations: why does something
happen?
3. Predictions: what will happen?
fredag 6 november 15
Plagiarism & copyright
fredag 6 november 15
noplagiat.bibl.liu.se
fredag 6 november 15
Using image without
reference
Plagiarism +
copyright violation
Using image with
reference
Using own/CC image
with reference
Copyright violation
OK!
fredag 6 november 15
Using references
fredag 6 november 15
References
[1] has studied software design patterns
Odersky et al. have studied software design patterns [1].
Odersky et al. (2010) have studied software design
patterns.
fredag 6 november 15
There are a number of conventions of how to use references properly: use in-text references
or outside-text references consistently. IEEE has a good standard for this.
For a qualitative model
For an empirical model or tool
For a notation or technique
Paraphrasing
Example
... narrative
... data, usually statistical, on pra
... comparison o f systems in actu
Here's an example of how it works on
For a technique or procedure
...a "slice of life" example based
For a technique or procedure
...a system that I have been deve
For a technique or procedure
... a toy example, perhaps motiva
The "slice of life" example is most likely to be convincing, especially
explanation o f why the simplified example retains the essence of t
Toy or textbook examples often fail to provide persuasive validati
examples used as model problems by the field).
thought hard
about this,
and I believe passionately
Over a quarter
2002
abstracts
give that
no...
Persuasionof the I ICSE
For a technique
... if you do it the following way
For a system
... a system constructed
indication of how the paper's
results are validated,
if at like
allthis
For a model
... this example shows how my id
[1]. purely by persuasion is rarely sufficient for a research pap
Validation
original question was about feasibility, a working system, even wi
No serious attempt to evaluate result. This is highly unlikely to be ac
The most successful kind
4.2 Which of these are most common?
analysis and real-world expe
were also successful. Persu
Alas, well over a quarter of the ICSE 2002 abstracts
narrative evaluation was o
give no indication of how the paper's results are validated,
Table 6 gives the distribu
if at all. Even when the abstract mentions that the result
2002, based on reading the
was applied to an example, it was not always clear
followed by graphs of t
whether the example was a textbook example, or a report
Figures 5 and 6 show these
on use in the field, or something in between.
Blatant assertion
[1] M. Shaw. Writing good software engineering research papers: Minitutorial. In Proceedings of the 25th
International Conference on Software Engineering, ICSE ’03, pages 726–736, Washington, DC, USA, 2003.
IEEE Computer Society.
fredag 6 november 15
Do not copy verbatim from published papers.
732
$-*2&'-0' )- ":' <&!2$"7 !""#$%&"'8 !-+ ! +)4- !##)4
(75%)2 G" H $-+$0!"'( ":!" ":' +'($,- 1#)1'#"7 :!( ! -',!"$3'
$-*2&'-0' )- ":' <&!2$"7 !""#$%&"'.
Citations
/-0!1(&2!"$)- $( 3$'4'+ ") 1#)5)"'
!-+ &-+'#("!-+!%$2$"7. 9:$2' 2)4
,))+ *)# &-+'#("!-+!%$2$"78 '6"'-+;
78 :$,:'# 5'!(&#'( )* 0)&12$-, !#'
$-*2&'-0' ":'(' <&!2$"7 !""#$%&"'(.
") :!3' ! ($,-$*$0!-" '**'0" )- !
27 !-+ #'&(!%$2$"7.
"' %7 5'((!,' 1!(($-, !-+8 ":'#'*)#'8
*2&'-0'( *&-0"$)-!2$"7 !-+ '**'0"$3';
5)"'( #'&(!%$2$"7. 9:$2' ":' &(' )*
$-"'#-!2 #'&(!%$2$"78 *&-0"$)-!2$"78
'0"$3'-'((8 $" :!( ":' 1)"'-"$!2 ")
6$%$2$"7 !-+ &-+'#("!-+!%$2$"7. A$5$;
*&2 &(' )* )%@'0" 0)51)($"$)- 0!$-"'#-!2 #'&(!%$2$"78 *&-0"$)-!2$"78
'(($3' !-+ $-0)##'0" &(!,' 0!- 5!B'
'#("!-+. C:' &(' )* 0)51)($"$)- 5!7
-'(( !-+ '6"'-+$%$2$"7. 9:$2' ":' &('
3$'4'+ ") $-0#'!(' ":' *2'6$%$2$"78
-'((8 !-+ *&-0"$)-!2$"7 !""#$%&"'( )*
3$'4'+ ") 5!B' +'($,-( :!#+'# ")
"7 $( 3$'4'+ !( !- $-+$0!"$)- )* ":'
+'($,-. D- ,'-'#!28 ":' 5)#' 0)512'6
!"#"$ %&'()*'+( *)& ,'+-.(&/ 0123 4&/'(+ 5126&1*'&/ *2
78.9'*: ;**1'<8*&/
I!('+ &1)- ":' +'($,- 1#)1'#"7 ") <&!2$"7 !""#$%&"'
#'2!"$)-(:$1 5!"#$6 $- C!%2' E8 ":' #'2!"$3' ($,-$*$0!-0' )*
$-+$3$+&!2 +'($,- 1#)1'#"$'( ":!" $-*2&'-0' ! <&!2$"7
!""#$%&"' $( 4'$,:"'+ 1#)1)#"$)-!227 () ":!" ":' 0)51&"'+
3!2&'( )* !22 <&!2$"7 !""#$%&"'( :!3' ":' (!5' #!-,'. F #!-,'
)* ! ") #" 4!( ('2'0"'+ *)# ":' 0)51&"'+ 3!2&'( )* ":'
<&!2$"7 !""#$%&"'(.
J)# !- $-$"$!2 1!(( $- ('""$-, 5)+'2 1!#!5'"'#(8 !
4'$,:"'+ 3!2&' )* KL )# KM.E 4!( &('+ *)# ":' 1)($"$3'
$-*2&'-0'(8 !-+ ! 3!2&' )* ;L )# ;M.E 4!( &('+ *)# ":'
-',!"$3' $-*2&'-0'(. C:' $-$"$!2 4'$,:"'+ 3!2&'( )* +'($,1#)1'#"7 $-*2&'-0'( )- ! <&!2$"7 !""#$%&"' 4'#' ":'1#)1)#"$)-!227 0:!-,'+ ") '-(&#' ":!" ":' (&5 )* ":' -'4
4'$,:"'+ 3!2&'( )* !22 +'($,- 1#)1'#"7 $-*2&'-0'( )- !
<&!2$"7 !""#$%&"' !++'+ ") #"8 ":' ('2'0"'+ #!-,' *)# ":'
0)51&"'+ 3!2&'( )* <&!2$"7 !""#$%&"'. C:$( #'(&2"( $- ":'
#'2!"$)-(:$1( (:)4- $- C!%2' N. C:$( (0:'5' *)# 4'$,:"$-,
":' $-*2&'-0'( 4!( 0:)('- %'0!&(' $" 4!( ($512' !-+
("#!$,:"*)#4!#+ ") !1127.
Bansiya and Davis claim that the QMOOD model may
address ”different weightings, other perspectives, and new
goals and objectives” [1]
!"# $%&'('() *(+ ,+*-.'() ./% 01+%2
C:' OP??Q <&!2$"7 5)+'2 !22)4( 0:!-,'( ") %' '!($27
5!+' ") ":' 5)+'2 ") !++#'(( +$**'#'-" 4'$,:"$-,(8 )":'#
1'#(1'0"$3'(8 !-+ -'4 ,)!2( !-+ )%@'0"$3'(. F" ":' 2)4'("
2'3'28 ":' 5'"#$0( &('+ ") !(('(( +'($,- 1#)1'#"$'( 5!7 %'
0:!-,'+8 )# ! +$**'#'-" ('" )* +'($,- 1#)1'#"$'( 5!7 %' &('+
") !(('(( <&!2$"7 !""#$%&"'(8 )# '3'- ":' ('" )* <&!2$"7
!""#$%&"'( ") %' !(('(('+ 0!- %' 0:!-,'+. C:' $-*2&'-0'
#'2!"$)-(:$1( *#)5 +'($,- 1#)1'#"$'( ") <&!2$"7 !""#$%&"'(
[1] J. Bansiya and C. Davis.
A hierarchical model for object-oriented design quality assessment. IEEE
%'81$ H
Transactions
on Software Engineering, 28(1):4–17, Jan 2002.
IJKL=AM 'AAB=NJA;<O:;<=>?
FBEG;BAM &;LKA=E?<P=G<
fredag 6 november 15
Use proper citation if needed, but only cite if necessary.
Managing references
fredag 6 november 15
fredag 6 november 15
fredag 6 november 15
Writing about what
you’ve read
• Take notes of what you’ve read
• Consider what needs to be in your report.
Do not write everything you’ve read in
your report. Remember to have a strong
connection to your main method/results
fredag 6 november 15
Summary
• Start learning about the subject, then find
proper support for your claims. Use
different sources for learning and
references.
• There are different types of academic
publications and results.
• Do not plagiarize or copy images.
• Use proper reference management
software (check course web)
fredag 6 november 15