60-510 – Assignment#5 Due date: Midnight Friday 13th February 2015
Pick one of your 10 most important papers and write a one to one-and-half page
ANNOTATION. The assignment should have the following format
1) Your name.
2) The title of your survey.
3) Your annotation with the following format (include all of the headings which are in italic in
the following, with a few sentences under each heading
A complete bibliographic entry of the paper (just cut and paste from your list of papers or
typeset with the annotation). Followed by the following (including headings in italics).
The problem which the researchers/authors addressed. (Not what they created, but the overall
problem which they tried to solve – this may be the same for all of the exactly-on-topic papers.
Note: think about why they did the work, what benefits to society or to Computer Science arise
from solving the problem). (Note past tense). You must state what problem is addressed by
the researchers, why it is important to society or science. Do not mention anything about
the researchers work. Look at your topic title for the general problem being addressed and
then look at the paper to see what specific part of the problem the researchers are tackling.
Ask yourself the question: “why is this research important to me as an individual, to
society or to Science”. if the authors state why the problem is important, make sure you
state that.
Previous work by others referred to by the authors. Just list the papers referred to (Authors
[year] in ascending year order). Only list those papers referred to which are in your list of
exactly-on-topic papers. (Note present tense, also papers should be in ascending year order). If
the authors refer to any work that attempts to solve the same problem, then that paper
should be added to your list of exactly-on-topic papers and listed under this “Previous
work” section. If the authors refer to work that attempts to solve a related (but not exactly
the same) problem p, then you should state that “the authors refer to related work that
addresses the related problem of p”.
Shortcomings of previous work identified by the authors (state that no shortcomings of previous
work were mentioned by the authors if they did not do so). (Note present tense and in ascending
year order of papers discussed). However, if the authors did not refer to any shortcomings of
previous work described in exactly-on-topic papers, you MUST add a statement “Although
the authors do not refer to any shortcomings of previous work on exactly the same
problem, they do mention that existing work in the area fails to do ……(fill in with any
statements the authors make about the problem not being solved..”
The new idea, algorithm, architecture, protocol, etc. which the authors invented (past tense) and
which they describe (present tense) in the paper. Just a few lines to give the main idea. By a
“few lines”. This means 120 to 150 words. The description MUST be sufficient for the
reader to know what the new idea, algorithm, architecture or protocol was introduced by
the researcher(s).
Experiments and/or analysis conducted just a few lines. No need to copy diagrams, tables etc.,
unless they are very important and necessary to explain the contribution made by the authors.
(Note past tense). It is insufficient to state that “the authors conducted an experiment” or
“the authors implemented the system”, etc. You MUST give sufficient detail for the reader
to know what was done. For example, the authors conducted and experiment with 20
student subjects who tested the interface with 100 sample questions, such as “How many
countries have capitals that are south of the equator?’. The data source was DBpedia. The
response times of the system were recorded.”
There should be no mention in this section of the results obtained. For example “The
authors tested their algorithm with 3 data sets s1, s2, and s3”
Results that the authors claim to have achieved. (Note that you must state that the authors
“Claim to have …”. (Note present/past tense, e.g. The authors state that the analysis which they
conducted proves that their algorithm has polynomial time complexity). You must give
sufficient detail about the results of experiments (or mathematical analysis) so that the
reader can analyze the results (in order to compare with their own results, or the results of
others.) For example, the “The authors claim to have obtained the following results when
using the D query set: average query response time = 5ms, accuracy of query response =
86%, etc. (where the data set D is well known otherwise it would have to be described.”
You should state only what the raw results were, not what conclusions can be drawn from
them. For example, “table 1 shows the results of the experiments conducted by the
authors”.
Claims made by the authors with respect to the contribution that they have made. For example,
“The authors claim that their new algorithm has better time complexity than any known
algorithm at that time.” (Note present tense)
Another example: “The authors clam that the results obtained from the experiments show
that their new algorithm outperforms previous algorithms in terms of speed for all three
data sets s1, s2 and s3.” You should not state that “The results obtained from the
experiments show that their new algorithm outperforms previous algorithms in terms of
speed for all three data sets s1, s2 and s3.”
Citations to the paper by other researchers (you will only be able to do this after you have
completed all of the annotations) and other evidence of impact of the paper (for example, use of
the new algorithm in commercial software).
Notes on “Problem addressed”
Under the Problem Addressed heading most students tend to describe what the researchers were
trying to do, rather than why they did what they did. For example, suppose the paper were on a
new architecture for creating and deploying speech applications on the web (my project), then
the Problem addressed should be:
“The authors state that very few natural-language speech applications are deployed on
the web despite recent advances in speech recognition technology. They state that there should
be more NL speech applications to assist people with visual disabilities and for hands-free
interaction with computer applications.”
The researchers description of why this problem has not been solved should be
under the Shortcomings of Previous Work Identified by the Authors:
E.g.“The authors state that existing methods for building and deploying NL speech
applications require expertise in speech recognition technologies such as VXML, parsing theory,
and client-server programming. Therefore people with only a little programming knowledge
cannot build NL speech applications.” Or “Previous work by R simplifies the creation of speech
applications but not their deployment on the web”.
Anything to do with the new idea proposed by the researchers should be under the
heading New idea/algorithm/architecture/protocol, e.g.:
E.g.“The authors state that building NL speech applications and deploying them on the web
should be as easy as creating and deploying HTML web pages. The introduce, what they claim to
be a novel architecture called the LRRP architecture for Local Recognition Remote Processing,
in which speech recognition is carried out on the client device by a “speech browser” which
recognizes spoken input (according to a grammar which is downloaded from the remote
application) and which sends text to the remote application (which can be written in any
programming language) which accepts the text on its “standard input” and which returns the
result on its “standard output” . The results is then sent back to the speech browser on the client
device for output as synthesized voice”
Note that under the Problem Addressed, there should be no mention of what the authors
did. Nor is there any mention of what the authors did under the Shortcomings of Previous
Work Identified by the Authors.
It is sometimes difficult to identify the problem addressed. If the authors say nothing about the
problem, you should write “The authors did not say why they did the work that they did”. Papers
that do not discuss why the problem is important in a general sense are usually not very good
papers (with the exception of highly theoretical papers which sometimes state only that a
problem is an “open” problem (say) in graph theory.
Notes on English style
1) Try to make sure that there is “number agreement” in your English. For example. “The
authors states ..” should be “The authors state..” OR “The author states”.
2) Try to insert the words “the” or “a” or “an” when they required. Note that most uses of
nouns in English should be preceded by “the”, “a”, “an”, “any”, “one”, “two”…”several”
etc. For example the following is incorrect: “The authors introduce architecture called
A”. It should be: “The authors introduce an architecture called A”.
3) The word “the” in the phrase “the X” is used mostly when the X is known to the reader,
either through their expected knowledge or because X has been previously introduced in
the paper. The phrases “a X” or “an X” are used when X can be any X. For example
“John gave Mary an apple. Mary ate the apple”. The authors introduce a new
architecture, called A. The authors state that the architecture enables …”