1
>> M. C. Schraefel: I'm M. C. Schraefel, University of Southampton and a
visiting researcher here at Microsoft for the summer. Some of you have seen
me. The cool thing is that I haven't seen most of you sitting in this room, so
does that mean that the MS intern mailing list seems to work in terms of
distribution.
Thank you very much for making the time to come. And presumably you're all
here because you'd like to get your papers published and you're thinking about
paper publishing right now, yeah? And are you sitting here because you know
that you have a deadline before you wrap up your internship to get a paper
done? No?
Why are you here? Seriously, let's get some of that information out just to
make sure that this work that you guys are going to do, we're going to do
together, maps to what your hopes and expectations are. So you definitely said
no. So what's bringing you here, right there, right in front?
>>: Well, I'll have to write it afterwards. I mean, I don't have definite
deadlines. I have deadlines, but I just [inaudible].
>> M. C. Schraefel: Okay, cool. So you've got something in the offing and
you're really thinking about planning ahead.
>>: Well, there is a requirement for entering a qualification exam.
submit a paper.
I need to
>> M. C. Schraefel: So you've got a qualifying exam coming up and you need
something for that, okay. Does anybody have a prospective deadline in terms of
papers they want to submit for the work they're doing this summer? Okay. And
you said semi, kind of.
>>:
I already submitted one work.
We might target another one in a month.
>> M. C. Schraefel: Okay. So since you're sitting here, would you be so kind
as to share a couple of the questions that you have about paper writing that
made you think coming to this was a good idea? Like what brought you here,
basically?
>>:
To appeal to a lighter audience.
Sometimes you're writing [indiscernible]
2
papers, and you have these different types of bodies that you want to deal with
at the same time. Sell your concept better.
>> M. C. Schraefel: Okay. So let's keep track of some of these ideas. One is
audience, and how do I communicate to reach these folks who might be a little
bit outside of the normal domain that I write for, okay. Anything else? Yes?
>>: Sometimes, like I'm working on a project that's two components. I was
thinking that if I should publish two papers for different components or I
should concentrate on one smaller project to make a bigger contribution and a
better paper, maybe.
So like what is the main contribution of the project and how to write it down.
>> M. C. Schraefel: So it sounds like there's a couple of things going on in
there. Do you write a couple papers, or do you figure out also what's the
contribution and how's the best way to frame that so that you can get some good
results, yeah? Okay. Anything else? Yes, and please tell us who you are.
>>:
I'm Chris.
>> M. C. Schraefel:
paper?
>>:
Hi, Chris.
Are you one of the persons that sent around a
Yep.
>> M. C. Schraefel:
Excellent, okay.
>>: I've been publishing more in psychology-oriented venues and HCI. Given
that computer science was in the name, I figured this would be a good
opportunity to know what are more computer-sciencey people looking for in a
paper.
>> M. C. Schraefel: Okay.
audience thing again.
So this kind of comes back to this notion of
>>: Because I can't imagine that it's quite different from the other audiences
that I've been targeting. So I'm aware that it's different, and ->> M. C. Schraefel:
We'll find out about that.
Okay.
Yes?
3
>>: Another question I had about the writing process. Often when you're
preparing for a deadline, you have way to go about writing as a process. But
sometimes the way you write papers is really editing, getting feedback and so
on.
>> M. C. Schraefel:
Okay.
So is there a question there?
>>: So the question is, how do people approach, if you're writing a paper,
that would have a lot of impact, how do people approach that kind of writing.
>> M. C. Schraefel:
>>:
So how do people approach exactly --
The methodology, I guess, behind the writing process itself.
>> M. C. Schraefel: Methodology behind writing process, okay.
fair? Okay. Method process. Anything else? Yes.
Does that sound
>>: In a situation where your contributions are not exactly -- in a situation
where your contributions are not exactly measurable and you can't set up an
experiment with some numerical [indiscernible], how do you actually measure the
experiment or the results of the study, how to present what you've done and why
do you think it's important if you can't report numbers.
>> M. C. Schraefel: Can you give us an example? Sounds like you have a case
in mind. The question, if you didn't hear it, was what happens if you want to
report something, but you don't have numerical or quantifiable results. How do
you present something so that you can say this is still worthy of publication.
Is that the question?
>>: Yes, that's actually the question. Well, for example, there could be an
issue in HCI. If you have something and you have a theory that it should
simplify some typical user process, but you can't exactly measure whether it's
[indiscernible] or not, besides computer study.
>> M. C. Schraefel: Good grief, okay. So what are measures of success, and
how do you pick those. That's a broad one. Cool. Anything else?
>>:
On a related note --
4
>> M. C. Schraefel:
>>:
My name is [indiscernible] from the University of Washington.
>> M. C. Schraefel:
>>:
Tell us who you are.
And you are?
Alex, and I'm starting my --
>> M. C. Schraefel: You sent around a lovely paper that had lots of
quantitative stuff with it. Welcome. So go ahead.
>>: So on a related note to the last question, I've had some experience, I've
had some hard time publishing a qualitative paper, and [indiscernible] because
it's really hard to measure a contribution in a quantifiable way. So a pretty
common feedback from the reviewers are the results are very -- for the
[indiscernible] paper, it seems not so novel enough.
>> M. C. Schraefel:
>>:
Okay.
That's the common feedback.
>> M. C. Schraefel: Okay. So another related thing to measures is novelty in
terms of the contribution. Anything else you want to kick off with? Back
here, sure. And tell us who you are.
>>: I'm Pariya. So any paper, there are some questions that the authors have
to answer in terms of justifying their choices, some questions about the
problem itself, why they've seen this problem, so it's always a challenge for
me to where in the paper and how to address it. So is it way too early, we
don't have the context, but if we don't do that early enough, the reader might
be disappointed.
>> M. C. Schraefel:
So where do you put the problem?
>>: Where do you put the problem or the justification of this is an important
problem. Our approach is good.
>> M. C. Schraefel: Thank you for asking that. That one we'll actually be
able to tackle really intensely today. So I'm going to note that down so we've
got that, problem and justification.
5
And you guys were nodding. Is there a reason you were nodding vigorously to
that last comment? You were nodding too. Any reason?
>>:
Oh, it's just, it's something that when you write it --
>> M. C. Schraefel:
Thumbs up, okay.
>>: I've got a couple over here.
paper.
>> M. C. Schraefel:
And Erin, you had a comment or question?
How do you effectively write a position
A position paper, okay.
>>: A position paper. And we have one other that just came in, and that was
determining the order of authors.
>> M. C. Schraefel:
Okay.
>>: The last one was actually mine, and for me my think is shaping up an
introduction and discussion.
>> M. C. Schraefel:
Okay.
Intros and discussion.
>>: I have a question on the [indiscernible] justification would vary with
related work. Could it come at the end or ->> M. C. Schraefel: I have some very strong practice views about where related
work goes, especially as a reviewer. So I will share that. This whole thing,
so let me put that up here. This whole thing is not about absolutes. It's
about what I'm learning in the medical community is often referred to as
evidence-based practice.
So I'm going to share with you and we're going to work through some examples of
answering some of these questions or, let's say, beginning to address them as
opposed to assuming we'll have hard answers, that are based on experience from
the writing side, from the reviewing side, from putting conference and journal
programs together, whether special issues of journals or conference programs.
Again, based on that, you can then test any of the claims that I make by going
back and reading successful or influential papers.
6
And, of course, the way -- well, why don't you tell me some of the ways that
you can assess whether a paper is successful or influential. Yeah?
>>:
Citations.
>> M. C. Schraefel: Citation counts are certainly a great one. My favorite
type of papers to get cited right ones that are initial, off the blocks, and
everybody says but this was crap and then they have to go back still and cite
that paper. Those are awesome papers. I want one of those papers.
Anything else that tells you whether a paper is successful?
>>:
Conference on the journal that [indiscernible].
>> M. C. Schraefel: Okay. That's great. So we know now that things like a
particular conference or journal has a particular status. Get into that thing
that says something. You've had some exact there already by being invited in.
Is there anything else that might be ->>:
[inaudible].
>> M. C. Schraefel:
>>:
Which is harder to measure.
>> M. C. Schraefel:
>>:
Okay.
Field changing can be really important.
That's true.
Paper awards.
>> M. C. Schraefel: Paper awards, yes, awards can be another measure of some
kind of criteria of something about it was liked by somebody at some point for
sure.
>>:
It got its authors a job.
>> M. C. Schraefel: If it got its authors a job, yes, and that's interesting.
The other thing is, of course, if it helps somebody who is in an academic
position to make tenure or if it helps somebody else hang on to a job in terms
of these kinds of effects. So is there anything else you want to throw out
here before we get going? Yes, Alex.
7
>>:
Has some practical impact in terms of the product it's related to.
>> M. C. Schraefel: Right. You might actually have a paper that is not highly
cited by other academics, but might be pointed to as where something started or
was the first part of something that became a product in terms of tech transfer
and so on. So there are all sorts of measures of impact and influence around
papers by which we would measure their success.
But what we can come back to, pretty consistently, and you've also mentioned
about writing across disciplines or thinking about disciplinary audiences, what
we can pretty much consistently focus on is how core elements of a paper, of a
successful paper, something that we'd enjoy reading, are conveyed. So there's
a couple of things that we'll definitely be able to touch on today and get into
some gnarly details. So a little bit better than touching on.
But there's other things that we'll need to come back to in the follow-up, the
following week. And I'll let you know here's why.
So in terms of looking at the method, the process of writing, a little bit on
measures of success, problem justification, intro and discussion, so these
guys, a lot -- author order -- we'll do that right away. So these guys we'll
get to specifically today.
What will be sort of the outcome of this, in terms of practice between now and
next week so that you get some real practice around thinking through these
concepts and can test them, we'll come on to these questions about audience,
how to write the position paper, and scoping and even the related work issue
further, next time, because the question will be on you in terms of practicing
some of the principles we talk about today to see how they help begin to
address these questions.
So I would encourage you, over the next little while, to think about whatever
project it is you think you're going to have to write up next. We'll look at
doing a practice abstract, intro and conclusion for them. Even just roughing
them out to see what some of the challenges might be.
And a second ago, when I was talking about how do we know that a paper is
impactful, that comes back to this notion between now and next week while
you're developing your own work, of also validating any claims made here today.
8
Is that if you find from what we're going to go over today that you can then go
and look at existing papers that are successful against whatever criteria
you've just chosen, you can say, well, how many of the things that we talked
about in this workshop are actually showing up in the papers that are deemed to
be successful.
And not all of them will necessarily be there all the time, but my bet is that
you will start to see most of the things we go over in really good papers, and
we might also come up with a definition of what a really good paper is from a
slightly different criteria than what might be happening in your head right
now. So let's just check that out. We're going to get into the specifics in a
moment.
Now that we've kind of scoped the space out of where we're at today and where
we'll go next week, here's a general question for you. And the reason I come
on to this, I've got this term over here called targets and aim. There is a
guy by the name of Tony Blauer, who teaches combatives. Have any of you done
martial arts, combatives, anything where you've gotten to hit people for a
legitimate reason? I'd encourage you to consider that as part of your life
practice. It can be very satisfying, especially when trying to write a paper.
He says if you don't have a target, it's very difficult to aim. And part of
what we're trying to do when we're communicating about research and writing a
paper is we need a target to aim at, to figure out what is a successful paper.
So what would you say right now are some of the attributes of a great paper?
It might not be something you've actually thought about, to say yes, I could
tell you in two sentences what a great paper is. But if that's what we're
aiming for -- well, actually, the topic of the workshop was not necessarily to
write a great paper. It was to write a successful paper.
But if you want to think that the two probably go together a lot of the time,
what are some attributes?
>>: It attracts the audience through its title and abstract, because out of a
large [indiscernible] papers that gets them interested and gives them enough
details in the introduction that if they care about the problem, they'll go
through the entire paper.
>> M. C. Schraefel: Okay. So there's something in there, if I can summarize
that a little bit, about attraction. Something that draws in a reader and
9
makes some kind of connection.
reader somehow. Yeah.
>>:
It makes a contribution or it articulates a contribution.
[inaudible].
>> M. C. Schraefel:
>>:
So it's connecting with the
Contributions [inaudible].
>> M. C. Schraefel:
>>:
Anything else?
Sorry?
A significant contribution.
>> M. C. Schraefel:
Okay.
We want to have significant, do we?
Okay.
>>: It has nothing to do with writing the paper. It's like choosing your
research topic properly. I think when you write, the successful ingredient is
stating the problem clearly and be sure that you solve it.
>>:
It's more a list of why it's important.
>>: And making a contribution depends on when you started your research,
trying to prove a theorem nobody cares about.
>> M. C. Schraefel:
Okay.
>>:
We're all saying a good paper comes out of good research.
>>:
I know, but like --
>> M. C. Schraefel:
there.
>>:
So somebody has to care.
You sound like a real skeptic
I am not the most optimistic person in the world.
>> M. C. Schraefel: Okay. So we've got this notion of somehow, the framing of
the paper is attractive and we'll impact what attraction is in paper writing.
There's some notion of a significant contribution, and you've been arguing that
significance has something to do with the fact that somebody agrees that what
you're doing is worth doing in some way.
10
>>: I think it should be clear -- the [indiscernible] should be stated
clearly.
>> M. C. Schraefel: So that gets into the attraction thing. Clarity. Good
observation. Let's hang on to that. Anything else that you would say is about
a successful paper? Remember, this is science that we're usually talking about
so things ->>: My favorite papers, this is something I have to admit I don't know exactly
how they do it, but it's written in a way that seems timeless. Just by reading
it, it exudes experience. They describe the problem as I had a problem, I
solved it, but describe it in a way that the whole community identifies with
the problem and the whole community can accept the solution.
>> M. C. Schraefel: Are you thinking of any particular examples where that's
actually made it through a review process?
>>:
Nothing specific.
>> M. C. Schraefel: Definitely come up with some examples. If you've got that
belief in mind, there must be papers that go with it. As soon as you started
talking about that, I was thinking about Vannevar Bush's As We May Think. If
you haven't read it, it is now well-regarded as a classic in terms of being
both visionary, what some of you might even call a position paper, and was
game-changing repeatedly in a field. In fact, when Jim Gray gave his touring
award talk in the late '90s, early 2000s, I forget which exactly when he got
it, he referenced that work as inspirational. So I'd check that out.
Do you have your hand up back there as well? No. You're just resting
comfortably. Anything else? So again, find an exemplar. Especially a current
one would be fabulous. Anything else that goes into your ideas of a good
paper? Yes.
>>: So given that we want to get more to the writing type of stuff, so clear
language, well-written, it's clear what the goal is in every paragraph, where a
story is going. The writing is crisp.
>> M. C. Schraefel: So I've got a notion of clarity again. And you also said
something about a notion of where we're going, which is this idea of --
11
>>:
[indiscernible].
>> M. C. Schraefel:
Okay.
Yes, at the back.
>>: So I think if you're referring to something new or just some
[indiscernible] work. Like something you [inaudible].
>> M. C. Schraefel: So that's a successful paper, that you're providing
again -- I think that sort of relates to this notion of significance, a
contribution that you care about. If it's novelty, it's a value that you would
recognize. That will help. There's a value proposition there. Yes?
>>: So the question is a successful paper in points of view of the author or a
good researcher? Because I, like based on the answers, I feel like the answers
are one using good research. So if you have done something and you cannot
change the research, then it's a successful paper for an author is the paper
that gets accepted in a good conference, right?
>> M. C. Schraefel:
I'm not sure what your difference is that you're making.
>>: So like a successful research, when you want to start doing a research, or
if you've done a research and you want to write a paper about it, and then what
is a successful paper, it's a paper that is accepted, right?
>> M. C. Schraefel: The one that's accepted, right. I suppose. I've heard
friends of mine who are pilots talk about flights that way, that says the one
they can walk away from was a good flight.
>>: Related to that question, also, I feel that there's a notion of you can do
sort of bad research. Of course, you can do bad research, but most of the time
when you start with a research question, there is a reason why you're doing it,
and sometimes the results aren't what you expected it to be and it can feel
like you did a bad job, but you can still write a successful paper, because
there was a reason why you did it, right? And although that might not be
the ->> M. C. Schraefel:
>>:
Yes.
Your background is psychology, right?
12
>> M. C. Schraefel: I'm just listening
paper and doing the paper and it sounds
background, as opposed to an engineer's
that a successful paper is when you can
accept it.
to you talk about the motivation of the
very much influenced by a psychological
background who might just want to say
walk away from it, that the reviewers
>>: But it might be when you build something as an engineer without knowing
too much about it, it might just not work. But theoretically, it might have
worked. So it was, at the start, it seemed like a plausible thing you did.
But at the end, it seems like hey, I didn't get anywhere. But that could still
be a successful paper. That's what I'm trying to --.
>> M. C. Schraefel: The number of papers, sadly, that are accepted that talk
about negative results is a whole other area of discussion that we can get
into. But for right now, we're going to assume that you've done something and
you have a result that you can report that is not a negative result. Not
saying that you shouldn't be able to produce -- in fact, as I said, there's an
entire argument in computer science, engineering and so forth that says we
don't publish enough of our failures.
Be that as it may, not changing that today, we're going to focus on results
that where we can say, yes, we got this out of it. In fact, there's a certain
art to having a hypothesis where you start with, saying this is what we're
going to test and get, and you get something else at the end. And you have to
sort of reframe what you're trying to write.
That's okay. That's the process of discovery, as opposed to rewriting history
in terms of your work. But let's hold on that thought about the degree to
which one can report on success or not, okay? We can do that today? So we're
going to idealize the cases a little bit.
So again, some of the things that we've got here about the notion of a
successful paper, yes, agreed, if you're not that happy with the result but you
still get it into a good conference and people like it even better and it gets
cited, even better, great.
In terms of successful papers, I think where we want to focus is how do we get
that paper to that point where somebody will let it into a venue to be judged
by our peers, even further than the reviewing process.
13
So how do we get to that walk-away point, if you will. And then can begin to
judge it on these other categories of visionary and so forth and field changing
and so on. We have to get through the gate most of the times to do that.
You've all highlighted some really important attributes of where we're aiming
to get with this notion of successful paper. We've got notions that the
writing has to be clear. People have to be able to understand what we're
saying. That's good.
That's not easy if we don't practice writing. One of the things that I
mentioned, I think, in the blurb for this, and I'm just going to put it down
here, is the notion of practice and reps or repetitions, which why I'm
suggesting a writing assignment, an example to actually practice writing
intros, abstracts and conclusions over the coming week, whether they're ones
you keep or throw away.
Writing is a skill. Paper writing is a skill. The whole point of this
targeting exercise is to give you some components to focus on so you can
checklist your own writing to say have I got this component in my paper so that
I can practice making sure that it's there.
And these are actually checklists that the groups that I work with and the
folks that I write with, we go through as we iterate on a paper. Back here, we
were talking about method and process of writing a bit. Generally speaking,
when we start working on a new research problem, when I got here at the start
of the summer in June, in early June with the group that I'm working with, one
of our first questions was, based on the things that we're interested in doing,
how would you imagine writing up the paper for that.
If you assume that you're going to get some results from this, once we spec out
the idea a little bit, what does the paper look like? What could we possibly
say the contributions might be? Sol Greenberg, a researcher in Canada, has
this great thing in HCI about -- it was extremely influential for me, which is
if you imagine the study you're going to run, what are the optimal results that
could come out of that.
And if you had those optimal results from that study, what could you say with
them? That's a fabulous way to assess whether what you're doing is significant
or trivial. Run the experiment in your head, get to the results, imagine
14
making the statements about contribution, and see what you've got.
And if you're not happy with what you think is the contribution coming out of
that study or that work or whatever analysis you're trying to do, it might be a
great way to say, we have to change something, because this is not taking us
far enough down the road to be worthy.
Sometimes, however, I would encourage you to reality check your assumptions.
If you run the experiment in your head and you get to the contributions of what
you think that paper's going to produce or what that work will produce for
writing up in terms of results, check with a couple people you're working with.
Check with your line manager, your supervisor, and see if they're saying,
you're being too hard on yourself. There's actually a really good nugget in
there that is useful for pursuing.
And so one other sort of tip within this is once you've run your ideas, always
check them with somebody else that you respect, because we always tend to be
harder on our own ideas than might necessarily be the case. In fact, the
quality of some of the best folks I've had the opportunity to work with is that
they are very supportive of other people's ideas in trying to raise them up,
and they are extremely critical on their own in thinking, yeah, but that's not
really good. No, that's brilliant, you've just cured cancer. Oh, no, that's
not enough for today's work. You work with people like that.
Okay. So if we assume, then, that this takes practice, then it's because there
are skills that we can practice. And again, the connection of this meeting is
in order to get at some of these skills that will help actually take this,
which is still fairly fuzzy, you know. You want to be clear, but clear about
what? You want to draw the reader in, draw them into what? You've got a few
attributes here around novelty and value and the fact that you want a path,
fabulous. But what does that mean when it gets home and you actually have to
do it?
So let's take a sense of this in terms of this notion, potentially, of roles.
And by roles, it's like what is our job when we're writing and what is our job
when we're reading. So one of the key questions you might want to ask in terms
of developing this notion of what to focus on for skills here is when we're not
writing, when we're reading where do you -- okay. Let me rephrase that.
When do you start to go to your digital libraries to look for other people's
15
work?
>>:
Like when do you have to bother to read somebody else's paper?
Before you start.
>> M. C. Schraefel:
Before you start what?
>>: Running the experiment.
already.
>> M. C. Schraefel:
You need to know that someone hasn't done it
Yeah, someone hasn't done it already, okay.
Go on.
>>: When, my masters, basically it was a requirement if we started the
project, the very first step of the project was analyzing related work and
producing reports.
>> M. C. Schraefel: Now, how can you get to analyzing related work as the
first step of the project if you don't know what the project is.
>>:
You have a general idea what the problem you're --
>> M. C. Schraefel:
problem.
>>:
Right.
So the first step is really coming up with your
Someone else did that for us.
>> M. C. Schraefel: Okay. If you're inheriting a space, then, let's assume
you've done that, okay. But I do think you need to have a problem that you
want to think about, generally speaking. I mean, you could just go to a
digital library and start reading. Some people do that. Some people make time
to do that. That's actually one of the best rules of professional magazines,
like the ACM communications or IEEE compute is fabulous because it lets you dip
into many places in many fields that you wouldn't otherwise hit.
One of my other colleagues has a philosophy of setting aside an hour a week,
blocked off on his schedule, simply to read through those magazines to find out
what's going on not just in the space that he works in, but in related areas.
And the opportunities for serendipitous discovery of potential new colleagues
and related work from that practice is very worthwhile.
One of the key tips there, though, is make it a block in your schedule or it
16
won't happen.
Your reading block.
It's very, very useful.
Okay. So we have this notion that we read other people's papers when we're
trying to find out if the great idea that we've just come up with has already
been done. Any other reason for going to related work?
>>:
Trying to solve your problem.
>> M. C. Schraefel:
If not create the problem, solve the problem.
Right?
>>: Another interesting one is where I'm trying to write related work, I often
try to see how people have cited others' work. In what context they have done
that.
>> M. C. Schraefel: That's nice too is that a great thing about a good related
work article is you can find a lot more related work to your problem already
sussed out for you. But it all gets back to sort of the same thing. To what
degree has somebody else already worked in this space so that I can make my
work distinct from theirs.
So my question to you is, what do you look for or what do you do, read when
you're trying to figure that out. Where are you going?
>>:
[inaudible].
>> M. C. Schraefel: Yes, I'm assuming that you're able to find the papers that
you want. But how do you -- okay. Let's assume you've got this list of
papers. You put some search terms in, you've got the list of papers. What are
you looking at first?
>>:
[inaudible].
>> M. C. Schraefel:
>>:
Really?
Titles.
>> M. C. Schraefel: Yeah. So just in terms of thinking about your reader, you
being in this position, if somebody else is going to read your paper in the
future, they've gone to Google Scholar. They've seen it. The first thing
that's going to hit their eyes, probably, is a title.
17
So if you're thinking future thoughts about framing a successful paper, how
have a care about the title. So just real simple stuff is that if somebody's
trying to answer a question about their work, what cues might the title give
somebody, whether they want to click the link that is that paper?
>>:
They're having some keywords about the main contributions in the title.
>> M. C. Schraefel: Nice, nice. So some kind of description in the title that
maps to a concept. And again, this is the first thing that you can start to
check and validate. This is this evidence-based medicine, if you will, of
being able to say if you look for the well-cited papers in your area, take a
look at the title and see what kinds of descriptors or what's been used here is
of the notion of a keyword, actually comes up in the title that might encourage
somebody to look further at the work. So that's the first hook, if you will,
into creating a connection with a potential reader.
Now, that's, again, not all papers are going to have wonderful titles that you
would think of. They might simply start to get the balling rolling, somebody's
seeing it and referencing it. Great. What else do they see in that list of
Google Scholars?
>>:
Abstract.
>> M. C. Schraefel: Before you even get to the abstract, because you can't see
the whole thing in Google Scholar.
>>:
Authors and keywords.
>> M. C. Schraefel: Yes. There is another opportunity, and it's growing
hugely with papers in trying to find papers is the use of keywords.
Personally, I have been utterly negligent in filling in keywords in the time
that I've been writing papers, because I had no idea what they were actually
being used for by any human being.
It
So
of
So
to
turns out that increasingly, we are taking advantage of keywords in search.
keywords are another opportunity for helping your paper surface up in terms
what people you might want to have reading your work would be looking for.
this is another place to have a care is consider the keywords that you use
describe your work.
18
Things like the domain that it's in, if you're doing HCI, put HCI. If it's
Symantec web, put Symantec web. Why? Again, this is becoming useful in terms
of algorithms that are discovering papers and making associations of what comes
up in lists.
So don't be afraid of stating the obvious in your keywords and thinking about
how other people might want to discover this. In fact, something you might
want to do with your group is test out whether your keywords seem to describe
what's in the paper and connect with other domains. Go to Google, go to Bing,
test out those keywords and see what comes back in terms of academic search or
not.
And if it doesn't bring back what your expected results are, find the words
that will. This, again, is about helping the reader connect with your work.
So that gets down to a question we're going to be looking at about writers.
For whom are we trying to do this? Ourselves, yes, to get our publication out.
But people who are going to be making judgments initially about these are
readers.
So this is really looking at cues to help readers initially.
gotten to the introduction yet.
We haven't even
You also mentioned that you have authors. Now, you might think that there's
not a whole lot you can do about this, but this is just a slight philosophy
thing. I mean, you might be thinking, well, I've got -- these are the people
that I'm working with right now. Please feel encouraged to branch out to
people in your field to collaborate on work.
There is often a concern that collaboration might harm the opportunity to put
your foot down in the ground and say this is my space. For instance, there are
some well-known names in my field, in HCI, that if their name is somewhere in
the paper, you can kind of forget, if you're an author on that paper, even if
you were the first author. The person they remember is the big name person.
But you know what? Sometimes that's really okay. Especially if your paper
gets read and cited and the work starts to get known. There's nothing like a
good introduction at a cocktail party or a coffee social or anything else like
this. Being in good company is being in good company.
19
In the U.K., where I'm located, they have this notion of buying a round of
drinks that if you buy a round of drinks for everybody, the next one somebody
else will buy the round and so the cost of the evening's imbibing is shared in
that respect. Colleague of mine back there, Dave Drawer, suggests that paper
writing can often be like buying a round. If you're open to the possibilities
of collaborating and encourage co-authoring of work, what goes around comes
around.
And again, that's not about readers writers. That's about a community of
scholarly practice. But sometimes that author list can be extremely valuable
to your work being discovered. So something that you can actually think about
and think about influencing in your paper writing.
So before we've even gotten to the body of the paper, we've been thinking about
this notion of roles in terms of where are we actually putting this paper. The
paper exists, yes, as you pointed out, on our CV as a mark of where we're at
professionally. So it acts not just as translating knowledge from one place to
another, but also simply as, if you will, a credit check on us professionally.
Where have we published, with whom, how often, over what period of time. These
are all questions that are asked about us in various professional contexts.
But when we actually look at getting the paper that we want read, presumably
that's also why we're writing is not just to build up a career portfolio and
get a job, though that is completely worthy and understandable, because it is
good to eat. It's also, I find, good to eat in weather places like this, in a
dry environment. So being able to afford food and shelter is a very worthy
goal. And if you're doing this kind of research practice, part of sustaining
that is definitely having publications.
Assuming that, most of the time, the role that we actually need to focus our
skills practice for is for readers. What we've talked about here is the times
at which we become readers and look for other people's work and, therefore, as
writers, what we need to think about to help that work be discovered.
We've looked at just a couple of heuristics for being able to succeed there,
which is having a descriptive title. It's great to also have the name of what
everything you've invented and then, colon, distribute of title. And again,
this is evidence-based. You can check this. See how many of the papers that
you are looking at in your related work follow that model of some kind of cool
name or abbreviation, colon, followed by boring description.
20
But the boring description is really effective for these things that we've
talked about. Yes?
>>:
How long can the boring description be?
>> M. C. Schraefel: As long as you have lines for. I'll tell you what, one of
the main heuristics for figuring out the length of your title is how much space
you actually have left in the size of the paper you need to cut. If you find
you've got a ten-page paper and it's gone to ten pages and two lines, and you
know that frickin' title takes up this many points, time to shorten the title.
It one of the places ->>:
If you have too long title [inaudible].
>> M. C. Schraefel: It depends on the field. If you take a look through pub
med, especially in chemistry paper, you will see some amazing titles that are
six lines long.
>>: [indiscernible] I'm looking, let's say, for a paper on solving linear
equations and I find a paper that's called solving linear equations, that is
perfect. If you can find a paper using the interpolation of the
[indiscernible] to solving equations is very hard.
>> M. C. Schraefel: Well, again, it depends who you're writing that paper for.
Because if you want to focus down on a particular derivative or second order,
whatever that you want as your focus, and this is what you want people to find
and those are the readers you're going for, that amount of description can be
really valuable.
And in that case, you've succeeded, because if your paper is written in such a
way that it would actually be a waste of time for somebody who hasn't even
gotten past the solving -- the first part of this problem, who is not that
sophisticated, then you've helped them. That I'm skipping this paper is a
really good decision to be able to make.
I don't know if you guys have had this experience yet, where the title promised
so much and the paper delivered so little. Hate that. But that's what can
happen with a title that does not really map to the body of the work is that
again I have great sympathy for readers, because as you get on in this practice
21
and you're asked to review more, you become more often a reader than a writer.
And so anything that lies to you, where you feel betrayed in your trust as a
reader sets up a very poor relationship between you and that author anytime you
see their work again.
Seriously. We have memories of these kinds of things. We are wired to
remember in order to avoid negative experiences. And so if we have a title
that promised much and delivered little, and we look at who the authors are,
it's like suddenly our trust in that author goes down, and we remember that the
next time we see their papers when we're asked to review something. And in the
community, we certainly do.
So this notion, again, I don't know if I put the word down here, but roles is
going to come up. We're writing to establish trust as well with our audience.
We want our audience, our readers to be able to trust what we've given them.
So again, having the title in the keywords and so on map to what we're about to
deliver is kind of our first place that we make a promise. And we're going to
use this motion of promises quite a bit.
Now that you've figured out some titles, we can actually get to the notion of
what you've been wanting to talk about. After you read the title and you've
clicked only it and you want to see if this is a paper you want to read, what
do you do next?
>>:
[inaudible].
>> M. C. Schraefel: Yes, here we can talk about abstracts. So what is the
role of an abstract? Again, here's your thumbnail description. What's the
abstract supposed to do, in brief?
>>:
Summarize the paper.
>> M. C. Schraefel: Okay. That's very much an author-centric way of talking
about the role of the abstract.
>>: Let the reader know what you'll get out of the paper so he doesn't have to
read it to get that.
>> M. C. Schraefel:
That's definitely one role, is one thing.
But not all
22
readers are male, is that readers are looking for something about which they
can make a decision. And you've just said an interesting thing about abstracts
is you want to have enough information to know if you can cite this thing
without having to read the thing. That's fabulous.
That also assumes, therefore, that the abstract is accurate. I'm not sure how
many of you have gone through the mapping procedure to say, this abstract said
that it did this. But when you read the method, it's like really? Did it
actually do that? I don't know that it -- what it delivered actually maps to
its claims in the abstract. This might be more of an issue in the pub med
stuff than in computer science. But that's certainly been the experience in
pub med. So that's one thing about a decision point is, is this something that
I may want to cite without having to read it. What's the other decision?
>>: Excuse me. Does someone really cite actually other work, cite it for
their own paper without reading it in.
>> M. C. Schraefel:
>>:
Of course.
[indiscernible] didn't even have to [indiscernible].
>> M. C. Schraefel: Of course. Absolutely. I can't tell you how many times
that -- and let's not have some judgments up here. You might find yourself in
a position in the future where you are looking for making assessments about
whether your area of work has been discussed, and you want to cite. There are
252 papers that have covered this topic. And you've gone through the
experimental method expressed in the abstract of the piece to know sufficiently
that yes, this is one of those papers, without having to know it in detail.
So whatever judgment you want to make about that, yes, there are people who
will cite something they haven't read, and that can become very clear when
you're reviewing stuff as well. That they either didn't read the paper or they
so severely did not understand it, that both are really interesting problems.
So this is a decision point about even deciding whether you're going to read
it. The other decision is if you want to cite it, is do you want to read it to
begin with. So you're making a decision if you've gotten past the title and
now you're reading the abstract, presumably the decision point you're asking
is, do I need to read this paper. What pile does it go in. The emergency, I
must read this paper right now because it looks so good, or the I'll get to
23
this later when I'm really checking out stuff.
So if the decision on a reader's part
look like it's worth me spending time
those precious minutes. And when you
kind of important. What do you think
providing this evidence for people to
want to see?
>>:
The problem they are solving.
>> M. C. Schraefel:
Anything else?
>>:
is do I want to read this paper, does it
in my life that I will never get back,
think of it that way, the abstract is
needs to go into the abstract when you're
help make this decision? What do you
Beautiful.
What's the problem, how did they solve it.
And to what extent it's solved.
>> M. C. Schraefel: Results. Beautiful. This is the map of the universe.
Right here. What would you say is the difference between this map and an
introduction for your paper?
>>:
Length.
>> M. C. Schraefel: Okay. Length is definitely one measure. Besides length,
that would assume that just one's a compressed version of the other.
>>:
[inaudible].
>> M. C. Schraefel:
>>:
Sorry?
No related work.
>> M. C. Schraefel: That's interesting. So let's hang on to that. Usually,
in computer science, you do not see related work specifically cited in the
abstract. That's not true in all cases, in all fields. Anything else?
>>:
[inaudible] and the actual result.
>> M. C. Schraefel: Okay. So there's some notion that the intro is going to
spend a little more time on the problem, and the rest of the paper might map
out the solution results. We'll see if that's true. Anything else about
24
differences between intro and abstract besides length and focus on problem?
Yes?
>>: So [indiscernible] method and the solution and the other is how
[indiscernible] technique you're using.
>> M. C. Schraefel:
>>:
In the abstract, you mean.
[indiscernible].
>> M. C. Schraefel: Okay. Cool. So the abstract, again, is really helping
somebody understand what the main content is of the paper. The cut to the
chase, if you will, elevator version of the complete snapshot of the paper in
terms of what's the research problem that you're looking at, how did you solve
for that problem, and what are the specific results.
This heuristic of what is in an abstract, I challenge you to go and read,
especially in the ACM digital library if you're in computer science, read
abstracts against this heuristic and see how well computer science papers, in
particular, follow this model.
I'll give you a hint of my bias here is that we're terrible at writing good
abstracts. The sciences are brilliant at using this model for helping people
decide whether or not to read a paper because it gives enough of the meat of
the detail of the paper to make that decision.
So it's very well worth doing, because it helps somebody make that decision.
Is this a paper worth me spending my time on, is it not. And how we often get
that is also out of how did we solve the problem. What approach did we use in
solving this and what are the specific values that are significant. What did
you find.
Huge value in spending time doing an abstract. Do it at the end of the paper,
if you will, after you've finished everything else so you can get this nice
synthesis.
The next thing we're going to spend a little bit of time in the time we've got
left today is intro and a bit on the conclusion. If we've got time, we'll look
at the examples you sent around. If we do not get a chance to look at the
samples you sent around, I would encourage you to use these heuristics to read
25
the four papers that I sent around to everybody who has signed up here that
have been contributed by members here to say, oh, to what degree did this
follow these heuristics and is it okay not to.
Conclusions are another weak spot in our presentations usually that could use
some attention. We'll come back to why in an sec. Okay. So far, we've got a
nice map in terms of, again, our role of writing for readers. I've used this
notion of trust here is when we're communicating with readers, implicit in this
is that we want our readers to feel safe and believe what we're writing.
When we are acting as readers, it's not a comfortable place to be how the hell
did they do this? Or I don't believe that they were able to pull this off, or
you call that a sample size? Any of those kinds of responses that we have are
generally because something's been queued up in the paper that has caused us to
lose trust, question our trust, or, actually, not feel safe with the ground
work that the reader is creating.
Part of the role of the introduction is to make the path for what our great
invention is going to be safe for the reader. And the safety comes from some
of the things you've already articulated. So we're going to look at these in
terms of this other topic here, we've talked a little bit about notions of
papers, who they're written for. We're getting into some heuristics now around
we've talked about abstracts and intros a little bit. We're going to come back
to that. But the biggie you've cited on for the abstract is this notion of the
intro problem.
We said we've got the map of the world right here. Problem, solution, results.
It needs to be expected a little bit more and where I'm taking us now is to
suggest that the role of the intro is to build trust and safety for the reader
to be able to believe the meat of the paper, which is going to be where we talk
about exactly what it is we did and how we did it.
So the intro is crucial. In fact, Ravin Balakrishnan and somebody else I
worked with at the university of Toronto, who is a very well published, well
regarded HCI researcher has a heuristic about reviewing that he says the
introduction is where a paper is won or lost in the review process. The
quality of the introduction has that much of an influence on how the rest of
the paper is being read and whether it will be accepted.
One of his heuristics for papers, and you can go check this with any of the
26
papers he's published and any of the people he's published with is always have
a picture of your system on page 1. Why do you think that having a picture on
page 1 is a good idea?
>>:
Gives a nice overview of what you're about to show.
>> M. C. Schraefel: Can give a nice overview.
thinking about impact.
>>:
Anything else?
If you're
It sticks with the reader.
>> M. C. Schraefel: Yeah. If you've got a ton of papers that are just plain
pages, if you've got some kind of image, we're wired for vision. We lock on to
vision and we remember visual cues. So that's almost a way of reinforcing the
title and the keywords.
So thinking strategically about what that image is for the first page, along
with a good intro, can be extremely valuable.
Okay.
>>:
So if we want to build safety and trust, yes, is there a question?
So [indiscernible].
>> M. C. Schraefel: If you're not supposed to put any picture on the first
page, you better not. If your field wants something that looks bloody boring
and like everybody else's, then at least you're playing in that field.
I have, however, found sexy use of equations to be quite persuasive. Tables
are also high value items. Now, and I say that just generally. Tables are
great visuals because it's something that everybody will go look at in a paper
as a summary of what has been proposed.
I'll give you another quick one. Judicious use of headers, section headers.
If a person can effectively read your paper by looking at the headers, boy,
have you created value for that reader with your paper. It helps them follow
the map that we're going to build.
So let's do a little bit -- is that okay back there? It's not a judgment on
your field. Really. It's, again, conventions regard really important.
Finding out what the conventions are -- you've already asked about
27
cross-disciplinary writing. Finding out what the conventions of your field are
in terms of formatting, expectations, ordering, sometimes where the related
work goes if there's not a strong convention in your field, that's the only
time it's a question mark.
There might be a strong convention in your field about where related work goes.
But again, no matter what those principles are, the ones we're talking about
here will cross disciplinary boundaries, and I can prove that and we will look
at that.
But this notion, trust and safety, the first thing, intros generally,
especially for ten-page papers, are most of you write be about ten-page papers,
conferencing papers as opposed to the longer 40-page journal papers? The only
difference here is how long your intro needs to be. Like if you're writing a
chapter for your dissertation, intro size is somewhat different, especially
when the whole chapter is called introduction.
But even in that case, we need to create this map for safety and trust. So the
first thing, and again this is the first thing I encourage you to read other
people's introductions in successful papers for is exactly what you've already
mapped. What's the problem you're trying to solve. I would encourage you to
avoid generalities like, in today's world, we have a lot of use of the web. Or
numbers are really important.
Any type of generalization is taking up precious space that is not helping the
reader get to what you're trying to do. So biggest thing to do first, and this
is when you're doing your own writing of your abstracts and also of your fake
intros for next week, this is the challenge. How quickly can you express the
problem that you're trying to solve.
If you can't do it quickly, my guess is is that you're not entirely clear yet
on what exactly the problem is you're trying to solve. A way of getting at
that problem is to be able to identify two things. Well, actually, there's
about three ways you can do this. And depending on the problem, you will have
a choice. If you can hit all three, you've got a trifecta, great.
Problem, in terms of proof of it being a problem, who is it a problem for? If
you're looking in the research -- this is an insight from Patrick Winston at
M.I.T. who does a great seminar on how to give a job talk. And he has this
notion of the problem is something that you can cut in terms of time for how
28
long has it's been a problem.
We want to solve the traveling salesman problem. It's been a problem for a
while. And you can cite NP completeness and et cetera, et cetera. And this is
where you could potentially lose faith with your reader and lose trust because
that's not a solvable problem. So what are you saying you're trying to solve
this? What's going on here.
But let's assume that there has been some problem that has been an open
challenge for -- since, you know, Allen Touring said it was. Whatever.
can use some sense of time in a problem.
You
The other thing that you can't point to are experts who have tried to solve
this problem. So and so has tried to get at this solution for this problem by
doing this. Somebody else has tried to do that. And somebody else has tried
to do this.
So you can hit recognized authorities to show what the scope of this issue is.
Related to this is actually groups in terms of at Carnegie Mellon, they've been
trying to solve this problem for the past ten years. What do you think this
does in terms of our trust and safety map to be able to say the problem you
want to solve has these components to it? How would that help you as a reader,
if somebody's telling you all these places that they're trying to solve this
problem and this is how it's affected this community for the past 20 years.
What does that do for your trust?
>>:
It shows that the writer actually understands the problem.
>> M. C. Schraefel:
>>:
Yes.
And that they are able to express the problem.
>> M. C. Schraefel: Okay. So it shows one thing about trust is they seem to
know something about the related work in the area so the domain is safe. You
can kind of trust that they've got an understanding of the domain. Anything
else that this tells us? Yes, at the back.
>>:
The problem is important.
>> M. C. Schraefel:
It's important.
Lots of cool people are working on it.
29
Anything else?
>>:
It's showing they understand the complexity of the problem.
>> M. C. Schraefel: There's a sense of, yes, not only understanding where this
has been worked on, but some trust that they know the related work, they know
it's important. They have a sense that you, if you know that area, would agree
that yeah, it's a pretty good way of expressing the problem. So there's a
sense of complexity, the scope of the problem, why it's hard.
And, in fact, that's something that you can be explicit about within this is
helping somebody understand two things. Why is this problem worthy of
anybody's time, and what makes it challenging to solve. So why is it
nontrivial, why is it worthy.
And by being able to scope the problem in terms of for how long and for who
else, and you might say in what way has it been hard is part of it. And also,
you might want to be explicit about saying, running it down for the reader in
terms of why solve this? And it's heading towards potential contribution.
You're not being explicit about the contribution yet, but you're saying this is
useful.
So that is the first core piece of a paper is being able to articulate the
problem that's being solved and defend it in terms of the context, just as you
would expect to look at a problem and solve it as this is not going into great
detail on the related work. That will come in its own section or throughout
the paper, whatever the convention is for the culture.
But it is making it safe for the reader. So that's how you know how much
related work do I have to put into the intro? How do I have -- you were asking
earlier about how do I justify this problem. This is the solution to the
justification question.
If this doesn't exist, maybe your problem's not justifiable. But there's
nothing like reference to time or authority for being able to support that the
problem is worth doing and why the solution will be of benefit. Any questions
about that part of it?
Okay. So that's the first movement of your introduction. Again, if you're
trying to build up trust and safety for what's to come, now that you've
30
explained why the problem is being engaged in in this paper, what do you think
is the next thing you want to do in your introduction?
>>: Say how you're planning to tackle it.
[indiscernible].
What you think the solution
>> M. C. Schraefel: Okay. The next thing that comes up, let's call it the
approach. This is again, not to reiterate your methodology section, but simply
to make it reasonable for your reader to believe that the way that you are
approaching this is thought out and reasonable so that if you say that this is
a particularly difficult problem in these respects, are you saying -- and to
scope the problem, are you saying that you're going to solve the whole trying,
or that you're going to perhaps nudge the problem ahead.
We haven't found the whole solution, but we're contributing this component,
which is going to enable all these other cool things to happen. So you get a
chance to talk a little bit about the approach that you've taken. If it's in
HCI, it might be we ran a user study comparing the following three versions of
this interface, or we tested out these two models of articulating what our
design space was, and we used this user study, and using the user study would
give us these values back. And so from that, we're able to do whatever it is
we're able to do.
So sketching out the approach is, again, think of it as if you were reading it,
would you find this credible. So is the approach that you're going to take to
solve your problem credible. And again, I'm emphasizing over and over again
that the introduction is the map for your paper that is conveyed in such a way
as to make your reader excited, have confidence in your approach, and feel
safe.
Again, to come back to this notion of articulate the problem up front, if you
cut to the chase and describe the problem, you give the reader the chance to
feel empathy for the problem you're trying to solve.
If we don't do that, if we just describe -- something that happens a lot in
papers, it says this paper is about blah. We ran this study to show blah. The
reader might not care. If we say, this is a problem that seven thousand
million zillion people across the entire galactic universe are having right
now, and if we solve this, we will save the planets of the universe this much
energy, which will be used to keep the sun from dying before we've finished
31
this conference.
that.
Wow.
I'm motivated to want to read how you're going to do
Because really, again, we only have so much time in our lives to do things, and
why not help somebody feel as much passion and excitement for what we're doing
as we do. And a fast way to do that is to present the reader with an
opportunity to hook into the space of the paper, and the best way to do that is
to say this is the problem. This is why it's a horrible problem. This is why
it's been a horrible problem for ages, and this is why contributing even this
much of the solution is going to make life better.
Because really, what science is about, well, some might say science for science
sake is just to discover the truth of the universe. My sense of it is that's
very nice, but it's almost better when it helps make our lives better. So can
we excite people into thinking how the solution of this problem is actually
going to contribute anything to the world to make our lives better. Our days
better. Is there some contributory value.
So that's why, once we get to that, people are willing to go hock, how are you
going to do? How are you going to stop the sun from going out before the end
of this conference? How are you going to do that? Wow, that sounds really
cool. Are you building on somebody else's method that's another opportunity
for a little bit of related work to say, yeah, and we're using so and so's
method, but it only got this far so we found this really cool hook and it's
giving us a 72 percent improvement on the previous results.
The other thing that
notion of hard. If,
past work, help your
good. How would you
you've had is really
>>:
you want to show, again, is again I come back to the
for instance, you say you get a 72 percent improvement on
reader understand that that's actually really, really
help somebody understand that the level of success that
good as opposed to, eh? What could you do in the intro?
Make some comparisons.
>> M. C. Schraefel: Yes. Comparisons are wonderful things. Do not be afraid
to use them. This again is kind of getting at related work. Is to say the
closest advance other people's work in this space has made is 13 percent, which
has done whatever it's done. 75 percent means what when it's at home. What
does 75 percent mean? Does that mean that it will only take us two years to
translate the entire world's corpus of a particular language into another
32
language?
How can I get real about what that 72 percent means.
I mean, it's a big number, it sounds good, it's almost 100 percent. But of
what and why do I care? I got fascinated by a student's work in electronics
that was just talking about a particular kind of wrap on wire that was used in
houses. And initially, the introduction wasn't really doing much for me. I
said why is this such a good thing, this insulation that you've come up with in
homes.
He said, well, it's used in most of the houses in the U.K., and it tends to set
itself on fire, and so by changing this insulation around it, it's, A, cheaper.
And B, it means that houses aren't going to go poof. I was motivated to make
sure that not only was that research successful, but that that person would get
that work out there and table replaced in those people's houses and then to
check if I had that kind of wiring in the place I was living.
So that's how to make something real and meaningful to people. It's not
trivial to say that. Again, it's about making people's lives better, very
quickly. Especially, the other thing that you could say is that the process
was really cheap. Like the big insight there was success. We can do this as a
fraction of what it costs right now to do the state of the art. We can do it
much cheaper, faster, better, and it has the potential to save lives by fewer
people's houses getting set on fire. Sounds good to me.
So again, is it hard to do? Well, that's the nice thing about the solution is
that they can express that, yes, it was hard to do some of the problems -- some
of the things that they needed to understand about the polymers that they were
doing. That was kind of challenging. And so that was the real research
contribution that they had. But the social benefit contribution of how that
research could be translated was fabulous. So their contribution was not only
this neat polymer that they had built and why that was solving certain problems
in plastics, but also the possibility of a social contribution. Yay. Benefit.
Okay. We're nearly at the end of the introduction. The next part of the
introduction, after you've mapped out the introduction in terms of what your
problem is, why it's a worthy problem, how you're going to go about presenting
your solution for that problem, what do you think the next thing is in an
intro?
>>:
Lay out the rest of the paper.
33
>> M. C. Schraefel: Why do you do that?
the paper. Why do you do that?
>>:
The answer was lay out the rest of
So you can address the reader if you want [indiscernible].
>> M. C. Schraefel: That's an interesting courtesy. Let me help you decide if
you want to skip parts of my paper. But it's true. It's again this notion of
what are you doing in the intro to establish trust and safety. It's also in
this case again -- I said I'd use this word again -- promise.
The next part of the paper is the map of what's coming. This is the usually
thought to be quite boring part of the paper. You say in the following paper,
therefore, we will present the related work and then we'll talk about our
wonderful method and we'll -- fine. Doing that lets the reader know what's
coming, and they can be mentally prepared for how you're going to take them
from what you've promised here to delivering it.
It's this part about safety, comfort, familiarity. For instance, if that list
doesn't have related work in there and that culture wants lots of related work
and you don't have it in there, that reader is going to be freaking out, going
where is the related work? So if, for instance, in a case like that you don't
have a related work section, you can have a sentence that says, we will be
discussing the related work in context as it emerges through the methodology.
Okay. I can relax now. I know why there's not a related work section.
the reader relax, trust you and be willing to go on the journey.
Help
I've spent so much time on two things here, the relation of writers to readers,
as opposed to focusing on this is how to exactly write a paper. The reason
being is I've been trying to give us some heuristics around why writing a paper
this way is important. We've talked about the abstract in terms of its role
for a reader's decision making process. Do they want to read the paper? Do
they not. Do they think it's going to be important for citations even if they
just glance through it or not.
We've moved from the role of the abstract over to the role of the intro. If
you see this again in terms of reader decision points, the first decision point
we've talked about is when they get to the title you the description, the
keywords, the author list. They've gone to the abstract, deciding whether or
34
not they need to read your paper.
What decision are they making from reading the intro? If this one is about
having enough detail to figure out they want to read the paper, what's this
one? What decisions are happening in the intro?
>>:
Continuing to read the paper.
>> M. C. Schraefel: Yes, one could say globally it's about continuing to read
the paper, but what else is happening if we've been talking about trust and
safety. What is happening here?
>>: They identify with the problem and do they trust that you've solved it
with good weight.
>> M. C. Schraefel: So there's a couple things there. There is the notion of
I'm excited about this. I'm excited about the problem. And access. And also
confidence. So by setting up something explicitly like the problem, you are
giving readers access to your thinking quickly. You're not wasting their time.
You're letting them into the problem that you want to solve and they can decide
then and there, yes, this is exactly what is meaningful to me, and I'm
interested. This is written for me. This is not written for the author to
just spill whatever they're thinking about. They've actually cared enough
about me.
Doing the mini related work and method section here again, what's happening in
the introduction, and think about it yourself when you're reading intros, is do
I trust what this person's telling me. That's one. Two, am I or am I not
currently bored out of my mind. Boredom comes from two things. And in
psychology, this is well described by [indiscernible] flow. Boredom comes when
we're either looking at, yeah, yeah, this is all trivia like in today's world,
the web is an important computation device. Boring, because we know that.
It's nothing exciting.
Or you start in the deep end with such an obtuse formula that the person has no
way of accessing it. Those are the two extremes of boredom. No challenge. So
challenging you can't get any purchase.
The sweet spot in the middle is where you help the person find this was what
inspired us about our research, trying to solve this problem. This is why it's
35
important, and this is how we're going to get there together.
take you there by going through the following steps.
And I promise to
So the intro is where the reader is making a decision about your work. And
again, this is if you're thinking about readers as reviewers, and this is part
of the successful paper writing of respecting the reader enough to think from
the reader's perspective to engage them, to be excited about the problem, and
to give them a path to getting through your work.
I'm going to take two minutes to talk a little bit about the conclusion and
what we're going to do next time.
The conclusion, what do you think the role of the conclusion is? I'm going to
tell you just because we're running out of time. It's not a place to repeat,
in small, the paper. Hate conclusions that say in this section we did this and
then we did this and then we did this. You've already read the paper. If it
was so bad that you have to have it repeated for you in the conclusion, there's
a problem.
What's the role of the conclusion?
>>:
Future work.
>> M. C. Schraefel:
That's a future work section.
>>:
State the results you achieved and also some table --
>>:
The key take-away points.
>> M. C. Schraefel: Take-aways is a nice way of putting it. Yes, you should
have that table. As we've said, tables are very good to take away. So a very
fast summary of what the work has done is very useful.
But this is where you say this is the contribution of the work. Contribution
is your key take-away. What has what the reader has just done given them that
will help them advance their own research? How -- all this stuff you talked
about before, about the game changer, about how is this advancing the field.
What do we know now from having read the whole paper that we didn't know
before?
36
So if your intro sets up people to go on a journey with you, the conclusion is
sort of like on the game shows, where they say, tell them what they've won,
Bob. It's like you've gone through this whole paper. What did you get from
it. Well, by bothering to go through the work of this paper, you now have how
to construct a small moon launch system. From that system, you will be able to
make it to the sun and reset its clock work and be able to have the light
continue on until the end of this conference and that will be a good thing for
humanity because.
So this is where you have the opportunity to help the reader understand the
value of the intellectual exercise they've just taken on, and not just what
your paper did, the problem that it solved, but what bothering to solve that
problem is going to do for the field, for research. And as a platform for
future work because you've got these take-aways.
So again, heuristic-wise, what I would encourage you to do, should you decide
to take up this mission, is between now and next week, write up a fake abstract
for the work that you're thinking about, outline a fake intro, but definitely
write up that this is the problem this paper is solving. That first paragraph
that says this is the problem space, that's the thing that I find, working with
students and colleagues, we have to work on the most, how do you get that
problem statement really crisp and clear.
And imagine that you have perfect results. What's that concluding paragraph
going to be or two paragraphs. One that says in sum, this work did blah.
These are the contributions. Because you're going to imagine what those
contributions are right now while you're working on this stuff. Otherwise,
it's not worth doing.
So the other stuff that I had here for you was just to look at the abstract
intros and conclusions of the papers that were distributed. You can do that as
a personal exercise, again to apply the heuristics.
And our heuristics are, because we're right on time now, is to take a look,
heuristic one, take a look at the title and description of the paper. Is that
sufficient for you to make a decision about whether or not you'd want to read
that paper. Take a look at the authors. Do you recognize any of them or where
they're from. Do you have judgments about their institutions or their
colleagues that might help you decide whether you think the work is credible.
37
Does the abstract set out these three steps of problem, solution, and results.
And especially, when you are reading the introduction, do you have a clear,
clear sense of what the problem is that this work is about, how that problem
has been defended, why solving it is valuable, whether the solution approach is
credible, and how the paper is going to work. That's for the intro checklist.
The conclusion checklist which we invented
terms of contribution. Is there somewhere
tells you why you bothered to come here in
fact that you have bothered to engage with
ourselves is your take-aways in
in that paper's conclusion that
the first place, that celebrates the
that problem.
Those are the three core attributes of papers. When we look at some of your
examples, mail them to me. We'll go over them as a group with an eye towards
answering these questions about how is a position paper potentially different
than what we've talked about? How do we reach across different audiences. And
if we do these following heuristics, do we need to worry too much about that.
And what part around the little stuff around conventions. How do we get ahold
of that.
And also, we'll get a better sense from you doing these practice intros, of
being able to figure out how do I scope my contribution? Is it a one-pager?
Is it a two-pager? Should I think about a conference, a journal and a
workshop? How do I figure this out?
Here's another tidbit from Ravin Balakrishnan again. Great heuristic is always
aim your work to go into the best venue. So you need to know what the best
venues are. Do not think about second tier conferences or journals when you're
planning your research.
Always aim high, because that's where the best success is and the most impact
and the most meaning is. And if you're thinking about -- it's fine to think
about workshops and so on to stage the work, but always assume that you're
going for whatever the best venue is for your field and assess whether what
you're doing here will get you there. Because what else is worth spending time
on, because you're just, you know, you can do it. This is a map to how to do
it.
Do you have any fast closing questions before we wrap up for today? Do you
think you can take these heuristics and apply them to the papers you've been
looking at? Because one of the things that it would be great to come back with
38
as a discussion is do you start to read papers a little more critically now in
terms of how they're shaped and what you would expect to look at for good
papers.
As a reviewer, if I can't clearly see what the problem is for a paper within
that first couple paragraphs, it goes to the bottom of the pile. Seriously.
Not because I'm going, oh, I hate this work. This work sucks. Only partially
that might be going on in some part of my brain. It's happening because I have
a stack of 20 to go through. This helps papers survive to get to further
consideration. I guarantee it.
Be clear about this, cut to the chase on these things. Your paper will
magically rise -- it's not magic, but it will rise to the top of the stack in
terms of consideration, because you're making life easy for your reader, and
that's who you're writing for. Your CV, second. Your reader, first. Because
if it doesn't get through the reader, it don't end up on the CV.
Good? Coming back next week? All right. I hope this was useful for you. If
you have comments or feedback or anything, please just send me an email. You
know what it is now. It's in your notes. Thank you very much. See you next
week.