1
>> M.C. Schraefel: Once again, just by way of introduction, I'm M.C. I'm a visiting researcher here this summer, and a couple of other interns, of a we were talking about a paper, asked me if I'd like to do this seminar on how to put together papers. As we talked about last week, the notion is for anything, if you want to hit the target, it's a good idea to know where to aim. And so a couple of ideas were to look at possible heuristics that we could develop that would help us figure out how to aim the structure and organization of papers for success.
Were there any particular heuristics that stick out still for you guys this week that you remember from last week?
>>: I think [indiscernible] contribution was something that I saw was happening in papers that you remember.
>> M.C. Schraefel: Okay. Building trust, good one. We'll talk more about that in a second. Anything else?
>>: Explaining [indiscernible].
>> M.C. Schraefel: Explaining results in the intro? Did we talk about results at all in the intro?
>>: Only in the context of the state of art, is that what you mean? Putting them in context with state of art?
>>: Yes.
>> M.C. Schraefel: Okay, great. So in that case, we were talking about not so much about the results but why the approach you were using was going to be a good one and also why the problem was worthy. So this motion of context around both the problem and the approach to solving that problem, or at least pushing it a little further along, was one of the things we looked at for sticking in the intro.
Do you remember anything from last week?
>>: I wasn't here last week.

2
>> M.C. Schraefel: Well, that would be miraculous, then if you remembered anything from last week. Thanks for something this week. That's why we're kind of going over some stuff. So that's cool. Anything else at all.
>>: The conclusion, what does the reader now know.
>> M.C. Schraefel: That they didn't know before?
>>: Yes.
>> M.C. Schraefel: Awesome. So some thoughts on the conclusion is thinking again about what does the reader have they didn't have before? And if you were going to push that a little further, the role of the contributions, then, is what? Not only this is what you didn't have before. What's the next step?
>>: Where can you go from here.
>> M.C. Schraefel: What can you do with it, absolutely. These are very self-interested approaches. So what do I have that I didn't have before. And particularly if this is your area of interest, what can the reader do with that. So making those things super-clear. So that's great that you remembered those couple of things.
Oh, coming up to the front, yay. Thank you, Alex.
>>: Interesting thought was that --
>> M.C. Schraefel: Come on in and come on up. Way up. One sec.
>>: So you choose the title in the abstract for just the title and the author, you try to make sure you can show up as an attractive paper.
>> M.C. Schraefel: Okay, so that's something --
>>: Keywords.
>> M.C. Schraefel: Keywords. Is the other thing we talked about is the kind of stuff that you kind of think usually is important, but we found out that it's actually really important that when you think about the title, authors, and even keywords, again from a reader's perspective, what are we thinking

3 about? Discoverability.
So if we want somebody to find our paper, assuming we've gotten it published successfully, what was something that we thought about with the title that we could do? How to make it helpful to the reader?
>>: Nail the point that it was trying to make instead of being --
>> M.C. Schraefel: So nail the point is a good general heuristic. We can nail that down even further by saying if you've got -- remember the colon? Some zippy heuristic or abbreviation, rather, that you've got for the name of the thing, like this is my cookie monster is a great talk title I heard the other day. Cookie monster. But then what is it? A way to protect cookies that shield your authentication to a system. That's what it's about.
So that right in the title, we have the groovy name that we can remember easily, but also very much the domain. We get it's a sense about security. We get a sense it's about a particular kind of cookie, so we can tell right from that if we're interested in that. The authors also gives us some sense if we know the domain of who's doing this work, also where their organizations are tell us something about likely the credibility of the work, and the keywords that we choose, again, we talked a little bit last week about how sometimes keywords are the last things you think about and kind of throw a way, but these can be great ways to situate the paper in terms of the domain and of anything you couldn't get in the title.
So if you couldn't get security about cookies into the title, you can throw it into the keywords. Awesome.
So these are all great things that you've remembered and are getting sticky about. And just to put a line through this point, is that when we talk about discoverability over there and we talk about building trust and what does the reader have, the big thing here is that it's actually not about how to write a paper for us, but how to write a paper for who?
>>: Readers.
>> M.C. Schraefel: Just to state the obvious. Sometimes it's good to state the obvious to get it stuck in our head. The biggest part of the successful paper is actually writing with the reader in mind. And so all of the

4 heuristics that we're talking about here in this slide is how do we help the reader, you know. What do we want to know as a reader and what in the paper helps us? So how do we design our writing experience to actually support the readers.
Because we get readers at two levels, don't we? We get our readers from the people who are reviewing our papers to figure out whether that draft or that version is going to be successful in the place that we put it, and we also get readers after that. We don't know where they might be coming from. They might be coming from the domain of interest, like we've submitted a paper to a conference and people are coming to the conference and they're deciding to come to our session. Or maybe they've just accidentally stumbled into a session that has our paper in it, or they find that from a completely different domain, they're looking for stuff and they find it.
So we want to help them figure out some questions around looking at a paper.
So that's the general pitch. Let's go over some of the core attributes of a successful paper that we reviewed last week, and first when we talked about after we got through of the title, keywords and authors, next thing is the abstract.
So again, if we look at this list, this is really focused on thinking about the reader when we're writing the abstract, because this helps situate again the success of the discovery of the work.
Do you remember what we talked about that the role of the abstract was from the reader perspective?
>>: [inaudible].
>> M.C. Schraefel: Yeah, absolutely. So if somebody finds the paper in a search list and they click on the link before they download the paper, they have the abstract. So the abstract is the synopsis of the paper that's going to give them a sense of whether or not they need to read the whole thing or it's something to come back to. Is this an urgent paper to read, because it's so right what I need? Is it something that I'm going to need to know kind of for background but might not need to cite.
All these kinds of decisions, decisions, decisions are being supported by an abstract. So the question might be how does someone use this abstract? How

5 does it help them? So what are the attributes that should be stuck into an abstract to help readers make that decision? Even if you weren't here you can guess.
>>: Identify the problem?
>> M.C. Schraefel: Yep, problem is beautiful. So we want to figure out what is the problem exactly being solved. What else do you want to get to in an abstract?
>>: [inaudible].
>> M.C. Schraefel: So yeah, definitely you want to get at the results. How you get to those results is also important. So problem, method overview or approach, and then the results really, because that's it in a nutshell.
And as we talked about it a little bit last week, computer science, electronics, anybody else from any other discipline here? No. We're not very good at writing abstracts that do that. We kind of get very descriptive and narrativey in our abstracts. It's nice to have models of great abstracts.
Any paper you look at that's in pubmed.org, just do a random search on a topic.
Do a random search for protein use in athletes' muscles or cell division or genomics or anything at all. Science-science, as opposed to electronic-science and computer-science is very good at writing abstracts.
I would encourage all of us to just spend some time trolling through pubmed to see that those abstracts follow that pattern of, and I think I've got it here for us. Yes. Problem, what the solution and results were and the last little bit there that you can stick on is what does this give you, okay?
So if you can say what we know now that we didn't know before from this, you know, the results tell us this, that gives us a little hint of a recommendation. That's kind of nice. Which is more or less did we succeed, you know. We set out to develop this new plastic. We looked at these methods.
We didn't get that new plastic, but we now know this. These are valuable things to have. It's like ooh, okay. I should read this, because I want to know why you didn't quite get what you thought you got, but you got this other thing. Because these other guys say that they got this.

6
One of the other cool things about reading science papers as opposed to sometimes computer science papers is that there's a lot of stuff being done in similar areas. So it's interesting to be able to compare via the abstract. So these guys used this method, and they got this result. These guys look like they're testing the same thing, but they got a different result. Why is that?
Is it the method? Is it the population? What's different? What's the same?
How can we build on this?
If you want an example of an easy one to look up for that kind of comparison, go to pubmed and put chocolate milk into pubmed and there have been in the past several years a bunch of studies about whether chocolate milk is a better recovery drink for athletes than the expensive sugar drinks that are custom-designed for athletes. I won't spoil it for you. But I'll say that it's, again, because there have been different studies run and they all seem to come to the same sort of conclusion, but not always, it's a nice way to see how abstracts do the method section, because we all know about chocolate milk. And it's so cool to see chocolate milk in a science index.
And the other thing that's kind of cool to see is who funded the research for different outcomes? Okay. So again, the role of the abstract has value.
Again, where does it have value? In the reader's decision process about whether or not this paper is a must-read, a maybe-read, or a skip-it. And it's completely fine to be in the skip-it pile, as long as their decision is based on the right information. This is why abstract writing is really valuable, because if they skip our -- assuming we've given them a good title and they find the paper, if we give them a bad abstract, it doesn't give the person enough description about what's going on in the paper, and they're busy, the papers they're going to go to are the ones that are more clear in describing what's going on.
And those are the ones that will get cited. And part of the thing about being a successful paper, A, one of your colleagues last week said well, if it gets published, that's success. Yes, that's the first stage to success, absolutely.
But the future success of these papers is do they get used by anybody? So having them read is pretty good.
And sometimes, you can get an indication, well, how do you get an indication if a paper's being read these days?
>>: Citations.

7
>> M.C. Schraefel: Citations is if it's being used, which is even better than just being read, I think. Again, depends on your measure of success. If lots of people are reading your work, that's a lovely thing. Is there a way you can find out if people are reading your work?
>>: Hits on [indiscernible].
>> M.C. Schraefel: Hits, yes. So if you're emailed and somebody says I like your paper --
>>: Or asks questions?
>> M.C. Schraefel: Yes, asks questions is even better, because somebody could just be sucking up to you. And this way, if they actually ask questions, it does show that they're engaged with it.
But yes, you can go to many of the digital libraries now and find out how many times has it been downloaded. That doesn't guarantee that it's been read, but it does indicate that maybe something's going on with this paper. And that's kind of interesting to try to at least see that.
And again, as you're building your careers and these metrics become more important, being able to say, look, this was the top download from this site for the past four months is not knowing. I have colleagues that go to dinner on the fact that their paper has been downloaded X thousand times. Doesn't mean that it's been cited that many times, but downloads are not knowing. And it's becoming an increasingly valuable metric. And again, downloads happen because papers are discovered, et cetera, et cetera. So we've gotten to this point again with the notion of helping readers make decisions.
So again, let me emphasize that we're talking here about writing for readers.
This is the abstract part. Now, I'd like to suggest, but we're going to look at intros as well, but let's look at some examples. This paper is one of the links that Alex, I'm not sure if you sent, but one of your colleagues a week ago sent this link over. Was this one of yours? No. Papers. And let's just take a look at it against our heuristics to see if this paper actually maps to any of the things that we've said need to happen in a good abstract.
So give you a moment to read this, and again for those of you who might be

8 watching this before or after if you check the email from last week, before this seminar, I sent around the links from four papers and we're going to be looking at those here for their abstracts, intros and conclusions. So the links to this paper, Cross-Lingual Word Clusters For Direct Transfer of
Linguistic Structure, that's a very destrictive title, isn't it, and I think we went on title here. You can kind of tell what field it's in. Cross-Lingual
Word Structures For Direct Transfer of Linguistic Structures. You could practically write a Gilbert and Sullivan song about that.
So I'll let you read. Okay. What's the first thing we said that an abstract should have in it?
>>: Problem.
>> M.C. Schraefel: The problem. Where is the problem?
>>: [indiscernible] original structure is --
>> M.C. Schraefel: Okay. Is there any particular part of that first sentence that --
>>: Make sure the last two words --
>> M.C. Schraefel: The idea of significantly improving prediction of linguistic structures?
>>: [inaudible].
>>: Somehow just being in English was the problem.
>> M.C. Schraefel: Yeah. I think we know where we are with the prediction of linguistic structures as the space we're working in. What's the problem that these guys are claiming?
>>: That it's not easy to cross-lingual.
>> M.C. Schraefel: Well, there's a hole. Right? The work has been done in
English, mainly, and now we're going to extend it --
>>: [inaudible].

9
>> M.C. Schraefel: Right. And is that difficult. See, the one thing I don't get from this wonderful description, because the next thing we get is what?
What happens?
>>: What they do.
>> M.C. Schraefel: Lovely bit on method, wouldn't you say, how they did it?
So you're kind of going not so good on method?
>>: It's good. I guess it's just a little wordy. I think I see what you're saying about the problem, though. You're not actually saying why it's a problem, why do we care if we goes into different languages.
>> M.C. Schraefel: Yeah, there's kind of two things for me with the problem that, again, it's not horrible by any stretch. In fact, I think this was one of the student award winning papers. Or maybe, no, this one isn't. This is a real paper.
>>: That's actually a student paper.
>> M.C. Schraefel: A student paper, okay. So regardless, one of the things that we talked about a little bit last week is in specifying a problem space, and we do get that there's a problem like here we've got lots of work in
English. There's no other stuff in English. We're going to go do some work that's not just in English. And in order to do that stuff that's not just in
English, we're going to see if the way that it works in English can actually be applied to other languages, or do we need to build some new stuff. So that's kind of cool.
And second, and the authors tell us it's more interesting, we provide an algorithm for features derived from blah, blah, blah. We can do much better cross-lingual structure prediction. Now, one might say that people in this domain would know why that's important. But what I'm not an expert in this domain and I have no idea whether this is hard to do or easy to do, and the reason that nobody's bothered to do it outside of English is because it's trivial to do.
So that's -- remember, we talked about -- we will talk about it again, the notion of confidence in the introduction is that we want to help establish

10 readers to have confidence and why bother to read this. We've done this work.
Why is the important. Why should I care. Part of establishing that is to demonstrate that it's not easy. There's nothing in this abstract that said this was a particularly hard problem.
Now, the abstract doesn't have to do everything. The intro can do that, but
I'm just saying that's a question in my mind right now. Is this particularly difficult if it was the Olympics and it was a dive on a scale of one to ten, would this be rated a six, or is this a ten? No idea. Because when these folks then make claims, and I agree with you, I think it's a bit wordy, that when they make claims to say it was reduced up to 13 percent and there was improvements of 26 percent, you know, we've got a ten-ish percent reduction and a quarter on a hundred here improvement, again, is that a big deal? I have no scale against which to measure that. I have no comparison to say even in traditional English lexical structuring, the best you can do and blah and our new method has made it super-blah. I don't know.
So again, it has the structure of problem, method, results. I don't know where the contribution really is in terms of opening up this space. Has it done what it claimed to do? And just one other observation on this we might make, avoid telling your reader that some part of your work is more interesting than another part. Why do you think that might be a good idea?
>>: Because what's interesting to you might not be interesting to them.
>> M.C. Schraefel: Might not be interesting to the reader. It also sends up a red flag to have a reader automatically go oh, yeah? So now I'm going to read to see if it really is more interesting. No, it's not. You're boring me.
Or again, it's extra words that could be preciously saved, and are you saying that the first point is boring? Yeah, if it's not that interesting, why are you reporting on it? So do not send up flares to your reader to suggest that there is a weakness in your work. If these are the two cool finds, make it clear again by being able to say why it's important. You know, this kind of sounds interesting that we show that these results hold true for a number of languages across families. That sounds like it could be useful. If you know that the same approach can work in these different places, that means you don't have to -- they've done that work. You don't have to think about that.
That sounds, actually, like a really valuable result. And then there's this

11 algorithm for inducing cross-lingual clusters. Again, isn't there already an algorithm? Why is it -- what's the big deal? So I'm kind of repeating myself here, but this does map, we could say that it maps to our structure problem, method, results. Not so much on the -- there's kind of an implied contribution, but we'd like to impact that a little bit more.
So this one, pretty good. B-plus, A-minus. Yeah?
>>: Talking about the problem being hard or not, is it important to emphasize that the problem is hard? Is it enough to say that it hasn't been taught before?
>> M.C. Schraefel: Well, can you think of problems perhaps in your own research space that haven't been solved but, you know, that could be with five minutes of thought but nobody's bothered to write a paper about it because it's not that big a deal?
>>: Yeah.
>> M.C. Schraefel: So that's why. Again, it's to help somebody reading this understand that you understand where this fits in. So again, we have different types of papers for doing that. Sometimes, that's where we write a poster, you know, instead of a conference paper. We say this isn't a huge problem, but it's a nice little thing that now that it's solved, tah-dah. And it's great.
People can use it. But it wouldn't be considered big research contribution.
So again, it's scaling and scoping. Does that help? You agree, maybe? Yeah?
>>: A different point, actually. About the wordiness, some of the venues that have like very tight restrictions on abstracts, like 100 words, 150 words.
Some others are like well, you can have like 250 or they have you fill out the sections, like.
>> M.C. Schraefel: Right.
>>: So what does that tell us, then, about how they want their abstracts written?
>> M.C. Schraefel: Well, it sounds like you've answered your own question in terms of they say that it's a 250-word limit, what does that tell you they want

12 in an abstract?
>>: I guess more in the spirit of the paper, because it seems like you won't hit all the points you're talking about. Sometimes you need more than 150 words, unless it's just like very clipped sentences, like we did this, we did that.
>> M.C. Schraefel: Agreed, and you don't have to tell a story. You can imply the story. Because the abstract, the nice thing about the abstract -- and again, this is why I encourage looking at pubmed for science examples because they can be quite concise is if we had more time, we would as an exercise say okay, how would we get this down to 150 words based on just the information that we have here. How could you pull out the key pieces of this to get down
150 words.
So, for instance, first we showed that these results hold true for a number of languages across families. Where are things you could kill right there?
Seriously.
>>: I think the starting is really wordy, you could have talked about --
>>: First the sentences can be easily chopped into one, take two lines instead of one.
>> M.C. Schraefel: Passive structures like it has been established that, they take up a lot of words. They're lovely, but they're passive constructions.
They take up word space.
>>: I think they don't have a strong subject on words. Basically, who is doing the thing and then what's being done.
>>: I would like to see a citation here [indiscernible].
>> M.C. Schraefel: Again, abstracts if you want to use a citation, you would mention the author, potentially, as so-and-so's group showed blah.
>>: It has been established --
>> M.C. Schraefel: Yeah, I understand what you're saying, but again, if this is in this community they want to say we know na or whatever, we could just cut

13 to the chase and say were cluster features derived from -- you know, it might just be that there's a problem.
>>: You can start from the second sentence.
>> M.C. Schraefel: Yes, you can, good observation. And, in fact, we don't even need the while previous work has focused on. Just most of the stuff in this area is on English. We don't know how well it applies to doing cross-lingual things so we set out to -- well, we set out is a bit narrativey too.
But you can start to see where it can be chopped down. So again, the tighter an abstract is, the faster somebody can scan it and get to the meat of it, the better. So it's almost a nice exercise, even if you don't have a constraint for 150 words to 250 words is to see if you've got all the time and space in the world for an abstract, could you get it down 250? What would you lose if you did that, and is it important what you lose?
Because again, the shorter you can make an abstract for somebody, while keeping it nice and clear, how much more quickly are you helping that reader make a decision about the paper? And hopefully, they will fall in love with the fact that this is such a nice, concise abstract that the rest of the paper's going to be this clear too. And they will look forward to delightedly putting it on their queue. A little star beside it in the their [indiscernible] library and going to say come back to this one.
So the thing is that sometimes when we're writing stuff, this is often the example, when something's wordy, that we didn't have time to really carve at it, and this is part of why the encouragement for this particular workshop was start writing your abstracts and intros now if you have, like, a month until your paper, because being able to iterate on something like this to find out what's the important bit, what's the not so important bit, how can we carve it down, that's a luxury of time.
The weird thing is that we actually usually are longer in our first couple of iterations and the punchier, shorter, concise stuff comes after we've had a chance to sit with it.
So rather than thinking we have to get the abstract and intro right the first time, sketch sooner, edit more happier longer. Because you get to that reader

14 place. Because you have to write for yourself first. Core dump for yourself in your abstracts and intros. Get it all out on to the paper. That's completely fine and understandable. And then as you think you've got it there, thinking about the reader, start sanding. Start carving. Yeah in.
>>: How specific do you think the results should be in the abstract?
>> M.C. Schraefel: Sufficient. What is sufficient about the results. And that again is going to be a judgment call in terms of what your domain is, what the related work is, and what you have to prove. So again if you put yourself or put a colleague in the place of being a reviewer who knows the area especially to be able to say what's the important bit in my results. Do I need to give a whole lot more method to prove this, or is it just sufficient for me to say we showed this.
So getting that kind of feedback can be invaluable to helping you determine exactly what is sufficient for helping somebody make a decision about this.
>>: So that less like a book jacket, then, to get someone excited. Like oh,
I've got this coming up, but I don't really know what it is until I get to the findings section. Or you could say like --
>> M.C. Schraefel: Yeah, you know, you make a really great case. A lot of us in computer science tend to write our abstracts as if we're trying to, you know, create a suspense story and not really give people the punch line. It's sort of like we think of them as writing movie trailers and we want to give them exciting bits but not tell them how it ended.
A good abstract tells the reader how it ended. Seriously. And again, that is what's missing in a lot of computer science abstracts is we don't give them the ending, and it's the, what can I do with this -- imagine yourself reading papers from the abstract. Don't you want to know how it ended so that you can decide whether you want to read the whole story?
>>: I think a better way to put it is it's not the ending, but really what's the results and --
>> M.C. Schraefel: Sweety, I am using the word ending metaphorically. If papers were movies, we'd be talking about endings, but you're absolutely right.
Well, sometimes, again, you can't say what the impact of the results will be,

15 can you, because it hasn't been tested by others. You can assume that this is going to make life easier in some way for a bunch of researchers doing work in this space.
But can we claim impact before the work's been out for a while?
>>: No. But you can put it in some context saying you expect it to change things.
>> M.C. Schraefel: Yes. And that's completely reasonable. And that's what we'd be hoping for here isn't it? Exactly. Very, very good point. Absolutely awesome observation. Anything else on this one?
Okay. So again, success in terms of mapping to our heuristics for a successful abstract, but could it be tuned a bit more to help the reader get to the important bits a little faster? Yeah.
Okay. So moving along, this is another one. This one is definitely worth taking a look at. Peta-scale -- again, checking the title. Peta-scale phase-field simulation for dendritic solidification on the -- here we have something, that's probably an abbreviation of some kind of special supercomputer. Does anybody know what that supercomputer is? Oh, great, I don't feel like the only ignorant person here for not knowing the special flavor of the month supercomputer.
>>: It's Tokyo, so --
>> M.C. Schraefel: It's a Tokyo supercomputer, which is the inspiration for the whole e-science cyber thing in the NSF a few years ago was that Tokyo beat the U.S. in terms of supercomputers and that spawned an entire wave of funding in the U.S. to say, we must do better.
Okay. So I'll let you read this and then we'll talk this again against our heuristics for what makes a good abstract.
Do you hear music in the background, like the trumpets coming up about how successful and wonderful and amazing this has been?
Tell me about this abstract, how does it match the pattern?
>>: The problem context is very [indiscernible].

16
>> M.C. Schraefel: Yeah. What works here? How does it establish that great problem?
>>: It tells why it is important from the first time.
>> M.C. Schraefel: Yes.
>>: Really, I don't -- I'm not an expert in the field so I have no idea when they're talking about until you get to the very end.
>> M.C. Schraefel: Really? I have no idea in this space either, but I'm willing to suspend my disbelief and go with it, that it is important to develop engineering materials with the expected properties predicting patterns and solidified metals would be indispensable.
>>: I still thought they could kind of make the first three sentences concise.
After that, it's pretty solid. They talk about the space stimulation method and how it cannot be done today so they have to come up with something that can do it.
>> M.C. Schraefel: Anything else?
>>: [indiscernible] talking about the contents, but then when it gets to their own work, it gets very short and they don't really say what's the big contribution.
>>: [indiscernible].
>> M.C. Schraefel: Go ahead. What are you seeing?
>>: Says the supercomputer usages. So what did you do about it. So if you're saying, yes, this supercomputer is available, that doesn't mean much. But if you contributed to the development of the supercomputer or --
>> M.C. Schraefel: I don't think this is about the supercomputer. It's just --
>>: The [indiscernible] very good, which I have no proof what it means. And this single precision thing, I'm guessing that that's what they're trying to --

17
>>: What I'm saying is it's too abstract. Maybe too little.
>>: I noticed the same thing. [indiscernible] three quarters is basically describing the problem, which you normally only see one sentence. I'm not saying it's bad. I'm saying normally you only see one sentence, like this is the problem and then it goes into methodology or something. This is like three quarters of the abstract is describing it.
>> M.C. Schraefel: Right.
>>: Too much of the [indiscernible].
>>: They don't mention [indiscernible] in their approach. So the point is not whether supercomputer develop something. They should be [indiscernible] problem solving. Because they mention other techniques achieved nothing. They don't tell you how they achieved. Just read the paper. And I think they just wanted to set something about the problem just to say we solved the problem.
Here is our results. We don't want talk right now here in the paper. But you understand now that what did we do [indiscernible].
>>: My opinion is they were dealing with a different audience, so --
>> M.C. Schraefel: What we're trying to say, though, is that we should be writing abstracts to help audiences, period. And all of you have made this observation. I'm afraid I got very carried away with the compelling description of the problem because I'm not an expert in the field, and I enjoyed learning about this.
But what you've pointed out is that if this is the size of the abstract, there's an imbalance between the size of the problem and the amount of description around where the contribution is of the work is our new simulation technique, yeah? I'm thrilled to hear that they have a new simulation technique that is going to address this, but you guys are saying less problem, more -- a little more juice here, and why is this a big deal.
That's, again, the sort of thing about why is this hard or why is this good.
Like you've said, you're not sure about what weak scaling is. Even so, understanding perhaps the context in which, you know, what has previously been done, now we're able to do it. And why is the supercomputer in here? Is it

18 because that now you can access the supercomputer, you can get all these flops that are a really big deal? Because we've got peta-scale phase-field simulation. Are they trying to say that you have to get to peta-scale to do this, and now that we have, we've written the algorithms to let us do it?
>>: You have to do these things to actually [inaudible].
>> M.C. Schraefel: Yeah, so again, beautiful problem description. I'm looking forward to seeing that in the intro, actually. Could be a little -- very nice analysis of this in terms of you as a reader making a decision about whether or not to read this paper would have found it more useful to have a more compressed problem statement, a little bit more detail about what's involved in their simulation technique, and a little better understanding of why the results are crucial.
So in a sense, again, we've got the structure of problem, method, results, and we do have implied in here why this is a contribution, because it's supposed to be solving this initial problem that is greatly to be desired in the field. So it wraps that up, but now what you're focusing on is balance. This sounds really good. Let's hear a little bit more about it. Cool. Nice analysis.
Well done.
All right. Let's get one more example here. I guess when I was reading this,
I thought hm, okay. So toward a unified theory of multitasking continuum from concurrent performance to task switching, disruption and resumption. That is a big title. Was this the one you sent over, Alex? Okay. That is a long title, but it's okay. It's got a colon in it. Except I'm used to seeing an acronym and then the colon and then something to describe.
So take a look at this one. What are your thoughts? And that's not fair of me to put anything missing there. Reading it, it reads better the fifth time that
I've looked at it.
>>: It's a good size.
>>: Where is the actual resource? What's the contribution?
>> M.C. Schraefel: Don't you dare start talking about whether this is science or not? Okay. We're just going to not have that discussion, all right?
Doesn't matter. Sorry. Alex?

19
>>: It's a unified theory, they discuss how the theory accounts and how does it account. I don't see it.
>> M.C. Schraefel: So some question about the results.
>>: First one. Second one as well, they identified several theories and what
[indiscernible] theories previously.
>> M.C. Schraefel: I'm not sure I'm understanding the question.
>>: Once again, context around the problem.
>>: I thought they were saying that they tried to build some type of
[indiscernible] which empirical [indiscernible].
>>: They implied it. I can't say they said it.
>>: It's not a statement. Basically, they're trying to put a theoretical model for what would be costing millions and trying to fit that to what empirical equation is observed. And it mentioned some key [indiscernible] that they're trying to unify.
>> M.C. Schraefel: Any observations, you guys? Anything else? What's the problem?
>>: They don't tell you how important the problem is.
>> M.C. Schraefel: Yeah. What can't you do without this unified theory? My simple question. Sometimes, it just sounds good. Let's put them all together and see what we get out, because they say -- what do they claim? That it will help us better understand and predict multitasking behavior.
So are we to infer implicitly that that means that not having this unification means that we can't predict multitasking behavior and, therefore, we can't do something else. So again, this is nitpicking at this point, but what is the problem? Why should I care if there's a unified theory or not? Does this mean that if I don't use their unified theory, that somehow using somebody else's theory is incomplete? Where is it incomplete? What hasn't it been able to do?
Throw me a bone.

20
What can't you do with these other theories that if you put them together, better understand and predict multitasking behavior. I don't know. I think that's kind of general. Again --
>>: Who is the audience here?
>> M.C. Schraefel: I don't know.
>>: What journal?
>> M.C. Schraefel: Is that going to help you? Just a question. I mean, if you're trying to say maybe the audience understands it, I've done work in task switching and multi-focused attention. I'm having these questions. And again, they're nit-picky questions, but if I have a stack of papers to read as a reviewer, the ones that give me headaches right off the top of the stack are the ones where I have to spend time trying to figure out why this is important.
And I believe from this that the authors probably think that they've conveyed why it's important, because they're saying, this is going to help you. Predict multitasking behavior. And they believe that's a good thing to do.
To better understand and predict multitasking behavior, because we're looking at memory for goals theory of interruption resumption. So our theory unifies several theoretical effects to better understand and predict multitasking behavior.
I don't believe it. I'm not saying that they're wrong, but I'm saying that I, as somebody who has done something in this space, is going I didn't know there was a problem. So -- and I could be wrong entirely, this could be brilliant work. In fact, the person who sent this paper through said I use this paper a lot, it's great.
So for them, this is sliced bread. For me as a reader, getting into this whole thing at multitask, if I was starting fresh at this, well, I've learned something. There are a bunch of theories, supposedly around stuff, and these guys are trying to -- why do I need to know this? Do I need to spend time here? So yeah, problem with the problem. Is there a problem, really, or is this just something that's kind of cool to do?

21
Anything else that you'd like to see?
>>: This might be unrelated, but I think when you're discussing your problem, it's better to use more sentences, because this is so long that you lose track.
>> M.C. Schraefel: In this paper?
>>: Our theory applies to -- that sentence is four, five lines long.
>> M.C. Schraefel: Four or five lines. You're absolutely right. Shorter sentence, and I speak from a long run-on sentence person writing. I can create -- in fact, there's a rhetorical convention on how long you can string a sentence out, and in this 17th, early 18th century, writers were really big on -- you know what they're called? Explosive sentences. The more commas and semicolons you can get into the sentence before you can get to the period, you win. I can do it in my sleep.
So yes, I agree with you. Shorter sentences. But then, they take practice.
And I bet the person who wrote this thought this was a nice, concise abstract that said exactly what's in the paper. And this is what's pretty much in the paper. What we are hoping for is a little more help to understand why there's a problem that this work needed to be done that it actually is as helpful as they say it is.
Do they test their claims? Do we know anything about how this was assessed.
They put it together.
>>: They don't even give context on what is the first theory that's ever been built or have there been theories before this.
>> M.C. Schraefel: We get that there are theories. They're putting them together. But is there a methodology that we should understand about how they put it together? So again, we're really picking at this just as practice to think about -- because this is a published paper.
So success, right. We're doing this as an exercise about how we could make things even easier for readers to make decisions about the value of a paper.
Yeah?
>>: On a tangent, just the language that they use, when you read the

22 abstracts, you see the word theory, does that change how you perceive the work?
>> M.C. Schraefel: Does it?
>>: I've had -- I've worked with people, and they say don't use that word.
>> M.C. Schraefel: As soon as they see theory, it's like -- is that what you mean?
>>: Yeah, use other language. Don't write, like, my theory of.
>> M.C. Schraefel: Are they saying my theory of, though?
>>: They say our theory.
>> M.C. Schraefel: Well, the cool thing is that it's their theory of other people's theory. See, the context is that add cognitive architectural, actual cognitive theory of concurrent multitasking and this goals theory. So they're putting together -- but actually, they might not be putting together their theory. What they might have is what they say they are, which is -- well, I guess a unified theory is a theory, possibly. But it's of other people's theories.
So it's a gnarly one. It's true that most people can be very skeptical around as soon as the word theory goes up, it's like another little red flag about, oh, no, this means it's flaky. Is that what you're thinking?
>>: I've always, yeah, I'm kind of reluctant to call my work a theory of something. I've made this theory. Like technically, it's true, but it's just a language thing.
>> M.C. Schraefel: Theory is probably better than claiming paradigm shift. So if you have to choose between the two, but in this case, because they are -- are they? You know, is it a unified?
>>: You could say something like you come up with model or something. That's something that --
>> M.C. Schraefel: That pulls together these people's theories into a model that you can actually use. Here's how you can apply it. Yes. Nice call. But

23 that's right, because again it's sort of like, I guess what theory starts to feel like is, well, how do you use this? Because they're claiming that you can get something out of it, that it will be applicable. So actually, model might actually help calm the whole thing down to say we're taking three other groups or these three other theories that are used a lot in multitasking, and we're putting them together in a model that actually lets you look at multitasking behavior and do something with it.
Suddenly, I'd feel a lot happier. But again, one might say, well, that's not what our work is about. That's what your needs are. Yes, but I am the reader and I am determining your fate. Will I read this paper or not? I count as a download. I'm not clicking download.
So again, that kind of awareness and sensitivity to how words trigger things and again, just pushing a little bit on that, if this were a paper written by several people to sit around and have the luxury of time, because you started writing the intro and the abstract, the first day you started the project. And we're going to come up with a theory of other people's theories. It probably isn't a theory of other people's theories. It probably is a model that pulls together other people's theoretical work in a way that can be practical.
Suddenly, I want to read this paper, because I've already written the abstract into something that's useful, because this is what we care about.
What do I get out of it? This is very much what they're pleased with. And again, that sounds critical, and I guess it is in a way, but it's mildly so, because they've had success. This is published. This is not -- this abstract will not keep anybody from getting published. What it might do is keep them from getting read.
But again, not according to the person who put this paper through. So again, we're just looking at how to reduce barrier to reading entry.
I think this is the last abstract that we have and then we'll shovel on to the very, very critical intros. Okay. Automating String Processing in
Spreadsheets Using Input-Output Examples. Go for it. Read on. Also an accepted paper, published paper.
Any initial responses?
>>: The last line was cool.

24
>> M.C. Schraefel: Sorry?
>>: They love their paper.
>> M.C. Schraefel: I think they're very happy with their results, aren't they?
What triggers you to the thought that they -- what are the signs that they're pretty happy with their work or they seem to be happy with their work?
>>: They say a lot of good things. One thing that caught me was that they didn't say why we need this large.
>> M.C. Schraefel: It's kind of buried in there, isn't it? There's one little clue about why this might be a good thing. Can you find it?
>>: I think before just saying anything about it, saying what properties it has, I wonder why did you think to write this?
>> M.C. Schraefel: Beautiful. You know what you guys are bringing up? When you say these are reader questions. These are questions that readers will have about your paper. So if you find yourself asking why, try to address that in your own work. Because again, this wasn't -- this didn't stop this paper from getting published. But what's our criteria for success beyond publication?
>>: [inaudible].
>> M.C. Schraefel: Absolutely. So and perhaps if you're really interested in this particular kind of tool, you'll find this paper. But again, you ask a good question about can you find a clue to say what motivated them to do this?
>>: [inaudible].
>> M.C. Schraefel: Yes, we have a claim and a dangling preposition, with.
Don't end a sentence with, for, of, or any other prepositions. A wide variety of string manipulations tasks that end users struggle with. I don't know what string manipulation tasks are, but apparently I struggle with them. I'm looking forward to reading the paper to finding out what I'm struggling with and how this will solve that problem.
But let's assume that that's the motivation. You're right. Before they kind

25 of describe the problem, they're into the solution. So it might be actually helpful to show what is this string manipulation problem so that I, again, trust, belief. So that I can believe, oh, yes, that is a problem I recognize.
I could see why a solution to that would be a bit gnarly.
Or, you know, because there are different kinds of hard problems. They might be hard problems because it's just no one part of it is terribly difficult to solve, but the fact that it's just a messy knot of many little trivial problems that make, you know, life hell, if they were solved in an elegant way, could suddenly let sunshine come through.
So if that's what we're talking about here, I don't know. But it could be.
And that could be kind of cool.
Plainly, we get a lot of detail about what they've done or claim to have done.
So it's kind of resultsy/methody, all mixed up into one bit. So we know a lot about what they think they produced and we know something about how they evaluated it to claim success. But you said this was neat, this golden test.
Do you know what that means? You said you thought the last sentence was neat.
>>: This is basically saying they mention something that they have
[indiscernible] to excellence being used so they were really happy that it was actually implemented. It's not out there [indiscernible].
>> M.C. Schraefel: Did they actually say that?
>>: Yeah.
>> M.C. Schraefel: Where?
>>: So the algorithm is implemented has been implemented as an interactive add-in for Microsoft excel spreadsheet system.
>> M.C. Schraefel: Does that mean that it's in Microsoft excel?
>>: [indiscernible] year after writing this paper.
>> M.C. Schraefel: So it really is in Microsoft Excel?
>>: Yes.

26
>> M.C. Schraefel: Okay. That's cool.
>>: That's probably why they're saying it met the golden test, because it's now in the hands of actually people who can use it.
>> M.C. Schraefel: Tells us nothing about whether people actually use it. But that aside, okay. It's been implemented.
>>: [indiscernible] paper is wasn't in Microsoft Excel.
>> M.C. Schraefel: Okay. That's kind of how I took it. Well, we wrote it so it could be a plug-in for Microsoft Excel. Great. Super. But what is this golden test? Are you saying it's because it's -- what does synthesized part of itself mean? Do you know what that means? You guys are making me feel so much better, because I'm like what is it?
>>: I do because I know this paper.
>> M.C. Schraefel: Okay. So what is it.
>>: Synthesize parts of itself [indiscernible]. It synthesizes strings. And if it synthesizes parts of [indiscernible] its own system.
>> M.C. Schraefel: And that's a good thing, is it?
>>: It's interesting.
>> M.C. Schraefel: Okay.
>>: That's why it's called the golden test.
>> M.C. Schraefel: So that's a standard term, is it, golden test?
>>: No, it's not a standard term. It's a standard -- I don't know.
Literature. Others trying to --
>> M.C. Schraefel: Inspiring?
>>: Yes.

27
>> M.C. Schraefel: It's done something so wonderful, I couldn't imagine it.
It's being used beyond what I possibly imagined it for. Therefore, it's pretty frickin' awesome. Oh, yeah. The mind reels at the possibilities.
Okay. And you see, we have to be careful, because, you know, the author could be right around the corner for this. And again, plainly, they did some really cool stuff if it's been adopted in a tool. They succeeded. They've done something neat. We are being totally picky here, because we want to increase the chance of success for it being read by other people. So we're pointing at the little things that would help us believe in this paper.
We've already said the problem, if it came -- we're already asking why did you do this before we get past the first sentence. You're telling us about this string matching thing. Why, okay? The problem is giving short shrift. We had to dig for it to get a hint of it. We're not sure exactly what it is, and we're being told that they've got some novel concepts. We'd want to understand at least in the intro how they're novel, what did you come up with.
And the synthesis algorithm is very efficient, taking a fraction of a second.
Is that meaningful when we're talking about computer processing? Doesn't everything take a fraction of a second on a computer?
>>: Absolutely.
>> M.C. Schraefel: But in what sense?
>>: Hm? For lots of problems, it's not really fraction of seconds.
>> M.C. Schraefel: Again, you know that. When I look at search results and I can see how quickly a search result can be turned around, over an index of giga millions of items in fractions of seconds, my context is missing to demonstrate that for this, that's a big deal. So again, context is -- you see, when we write in our own domains, we forget that other people don't know this stuff.
This is why it is useful to have, again, the luxury of time to review these things. To have friends is also good, who are sharing your pain because they write papers too. And to ask them early on to check out your language, to check out your assumptions, to see if your context is there.
As you all know, if you're working on projects right now, it's really difficult

28 to separate yourself enough from the process for us to be able to say, oh, yeah, I've got the context here. I said, you know, fraction of a second here and we know that's a big deal. Who knows that's a big deal? Again, as I say, when we're talking about computers most of the time we're talking about fractions of seconds. A second is a long time. But you've just said in this context, not so much. Good to know.
Again, I was just at a talk yesterday, we were talking nths and nths and nths of seconds for process measurements. So it can detect noise in user input.
That sounds reasonable and do something nice with it. It supports an active interaction model wherein the user is prompted to provide outputs on inputs.
Okay. That sounds like some kind of feedback loop to make sure the computer knows what's going on and that's magic. Again, I'll believe that that's tricky, sort of. Don't know, but I'll go with it.
Again, what would help this is a little bit clearer on why, a little bit clearer than on the rationalization for why these processes were important to the solution and how it's been evaluated and, okay, the tech transfer thing, that's nice, but it's -- again, if you want to talk about contribution, is that as clear and effective as being able to tell us what kinds of problems it's actually solved and that it works and we've evaluated it on 6,000 different tests and it's been effective to this level. Yeah?
>>: Question on writing style. I've had abstracts that kind of turned out like this, where they're kind of like two times too long and I'm cutting out all the fluff and all like the connector words like however, things like that.
And then you get sentences like this, like the, the. It's just like a list.
>> M.C. Schraefel: Yeah.
>>: When you're reading, is it worth losing a little bit of information by having, like, an actual sentence?
>> M.C. Schraefel: I wouldn't call it fluff to use conjunctions. Again, it's about sense making. And if you have a bunch of sentences that are the dog went with the human. You know, that kind of see Jane run sentence structure, it's probably because you were cutting close to the deadline.
It takes time to think of creative options that maintain sense and continue flow but are concise. If you take nothing else away from this talk and these

29 workshops for the past two days or two weeks, two times, one is that you're writing for a reader and how to do that. And the other thing is if you want to write for a reader successfully, give yourself time to do these kinds of processes.
The best ways to give yourself time is as soon as you know what your project is, start writing a fake abstract, a fake intro, and a fake conclusion. And why I'm calling them fake at this point is because you don't exactly know what your results are. But you know what you're doing kind of with your project.
You know generally if you have related work to support it or not or if you're inventing something completely knew and you know ideally what you'd like your contributions to be if your work is successful.
If you don't know that by the first day, you'll know it by the first week if it's a three-month project. If you don't, there's a problem, because again, where we started, if you want to be successful, you've got to have a target to aim for so you have to know what the targets are. So you know early what your potential contributions are going to be.
When you start writing fake abstracts and intros and conclusions early, so the overview in the abstract, the intro that lays out the map and the conclusion that lays out the contributions, you give yourself the freedom of time to be able to think about how can I make this more concise. How can I make this tell the story without taking up a reader's time and making sure that I'm getting the problem, the context, et cetera, and that I have time to ship it out to other colleagues to take a look at to make sure this make sense.
I just wrote about the fifth version of an intro for a paper that a couple of us have been working on for months. I felt pretty good about it. One of their writers who hasn't looked at it in a while said it's kind of narrativey. It's kind of long on this bit before we get into this. I think we need to shorten it.
Of course, it was like -- I thought breathe, let it go. That's exactly what this process is about, because another part of it is to give yourself time to fall out of love with something you think is brilliant. It's really hard to do that if you don't have time to let it go, isn't it? It's sort of like code when you write it and you love it and someone goes this is crap. This is not very efficient.

30
But oh, I spent so much time. Don't you love that when somebody says yes, but
I worked so hard on it? Yes, but it's still poor. Yes, you worked hard. I recognize that and I love you for it, but it's not there. Same thing with papers. Give yourself time to fall out of love with something you've created.
Because it's when we're so attracted to it and so invested in it because it feels so right, yet it's still so wrong.
You want to have time because you want to love your colleague again too especially when they say that, and you just say no, it's good. It's good. The feedback is good. Because if they're thinking it and they're you're colleague, what's a reviewer going to think? So again, two messages. Writing for readers, give yourself time to write for the reader. So start early, start fake, start early, start often. Iterate.
So let's look at an introduction example. Again -- sorry?
>>: I have a question. For the --
>> M.C. Schraefel: Last one?
>>: Yeah. So seems to me that you didn't like the word fraction of a second.
>> M.C. Schraefel: Because I didn't understand what its value was.
>>: So you can you move back two slides?
>> M.C. Schraefel: Yeah.
>>: So also, we didn't know what [indiscernible] is, whether it's important.
>> M.C. Schraefel: Yes.
>>: So I think or I believe that the author has to put some messages according to the domain, and not everybody can understand these messages. So I don't know [indiscernible].
>> M.C. Schraefel: I bet it's peta flops.
>>: But they're essential, right?

31
>> M.C. Schraefel: Right.
>>: So that's why I believe that in the next two slides, seeing that this takes fraction of a second is good. It's not a bad thing to mention.
>> M.C. Schraefel: You're absolutely right. A domain-knowledgeable person will get it. Implicitly, I'm going to trust if it's published, it's credible.
I'm just saying that as a reader, I don't know why. I personally don't know why.
And I'm wondering, this is sort of a challenge to myself as a writer and to us considering writing for beyond the domain, how would you convey that? Could you queue that up in some way by again a little bit of historical context that this is the first time we've been able to get to a fraction of a second or that we know that these kind of -- can you do it in a concise way? Maybe you can't.
>>: But it could have been the domain with --
>> M.C. Schraefel: No, they won't laugh. They won't laugh because it's adding value. It's making it easier for them to -- I'm trying to think of examples in my own space where it wouldn't -- you wouldn't be stating the obvious to be able to reiterate that a measure has value. And again, I offer it as a challenge because one of the questions that came up last week is how do you write across domains.
And you've raised a good thing. I'm going to go back and look at some work and see if there are ways in my own domain where I've missed this and could start giving some value to measures that this would be useful, because I don't know that I could promise that everybody in this domain would understand how significant that is.
>>: I think a good way to practice with people is when you mention a number, then you say it's so much [indiscernible] state right of the art.
>> M.C. Schraefel: That's nice. That's a relatively easy way to do it.
Again, I don't think anybody would laugh by being able to make explicit the value add. And I'm not sure how we would do it in the fraction of a second thing, other than to say something about state of the art around these processes generally are not even implemented, because they go well over whatever the time is or something. I don't know. But there should be possibly

32 some way to queue up why this is such a win.
And again, I think rather than saying that this has met the golden test and blah, I would have rather had a little bit more benchmark. So again, it's not -- this is published. So plainly, it succeeded at that level. This is just, again, helping this be even more discoverable, useful, useable.
>>: This is probably done by some senior authors.
>> M.C. Schraefel: Actually, no. So let's take a look at intros really quickly so that we can get out to the shortcake social feeling ready for sugar.
So with the intro, we used the metaphor last time of the intro being the map of the world for a paper. And then its value-add is again especially about trust.
What is the most famous back seat question of any car road trip when there are kids?
>>: Are we there yet.
>> M.C. Schraefel: Yes! Are we there yet. This is what the intro has to address is that the reader is keenly desiring to understand why am I here, and is it going to be worth it to go through the next eight to 48 pages of this work. Help.
This is the writer's biggest opportunity to create that reader-written perspective of safety, confidence trust, and two other things that are wonderful to have in a paper. Joy and delight. Even if you think the paper --
Carol Goebel, this researcher in the U.K., and in grid-based computing, has this wonderful analogy of two types of papers. One she talks about as the glass of wine and bathtub paper, which you just can sit back in the tub with a glass of wine and enjoy reading the paper.
The other is the sit at the table with a cup of coffee. I have taken that to the next level of the double espresso paper. But again, that's still a joy, because it's like oh, this is really good and I need to pay attention to it.
Or this is really good and I'm just, you know, enjoying the ride on this, as opposed to I have to work really hard to get this.
Both are wonderful experiences. One is engaged in I want to spend my relaxing time with this, because this is so intellectually lovely. And the other is I

33 want to spend time working to get it, but I can tell there's something really cool going on here. Either of those is a wonderful state to be in. And you might want a time which one you have them for, but to have those kinds of papers, that's joy and delight. Different kinds, but both joy and delight.
The kinds of papers we don't want or that you might not enjoy are the ones that don't let you know are we there yet. Papers that work, whether they're delightful in the wine or coffee sense, are there because they provide necessary hooks for us to be engaged. And one of the hooks that we talked about in terms of the map is that it does solve problems for the reader, first of all, by giving us a problem that we can engage with.
So again, instead of describing this is our algorithm, it does this, it does this, what's the problem? Where is my house on fire that I need your paper to help put it out? I am motivated to read this because I'm engaged with your problem.
We also talked about the intro in terms of laying out what we've talked about, why your problem is not trivial, why it's valuable and we'll look again at some of these qualities about getting to this are we there yet in our road trip.
We talked about setting up the problem, establishing its significance in terms of proving it's significant, the approach that we're using in terms of our methodology and then what is coming in the paper. So giving the reader the map.
All of these things are about building trust and relaxation, because once we have the map, once we've set up the trust, actually, and that we say, oh, yeah,
I engaged with this problem. I can trust that they know what they're doing.
This method and approach seem interesting and I'm looking forward to seeing how they do it, and now I know how they're going to go through the next sections of the paper to get me there. I know what this trip is about and I am up for it.
As opposed to papers, again, from a reviewer's perspective of, I have no idea what these guys are trying to do. I'm trying to figure it out myself. I could be wrong about where we're going and so I get three pages into the paper and suddenly it's not what I expected, and it's, is it me? Is it them? I'm having to do too much of the work of figuring it out rather than the work of understanding what they're doing.

34
You want to write so that people are with you and going yes, that's an interesting problem. Yes, that makes sense how you solved it. Oh, that's a cool result. As opposed to, why are you telling me this now? What is this about? Those are not fun papers to read. And they really, they take -- we've all written them or we all will write them for whatever reason.
And again, why they don't succeed is, generally speaking, they don't make it easy for the reader to focus on what we're doing.
Here's a trivial example. Historically, I am a horrible speller. Spelling is not my forte. It has never been my forte. The spell corrector in Word is one of the best inventions on the planet for something like me. What happens when people who can spell see a spelling mistake? What do they do when they hit the word that is misspelled? Can you imagine? They're reading along, they see a word that's misspelled? Do they skip over it gracefully, or does their brain stop and say, that's wrong.
Same thing for the larger structures. If there are conventions in place about how to get a reader through a paper, and what they expect to be there like a problem statement and rationale why it's important and that, and it's not there, there's a cognitive dissonance that comes up that says that's wrong.
And so instead of focusing on what we're trying to say, get trapped or we get trapped in the structure and we're fighting against the issues and the conventions being ignored or just the clarity not being there to try to dive in to find the idea. That's wrong.
So again, part of this is trying to give us some heuristics, those targets to aim for to help us make assessments so that we can find out, do we have the pieces in place to help the reader move along.
Here's the intro, or at least the first page of the intro from our spreadsheet paper. See how this deals with problem, significance and proof of significance, method and map. So those are our heuristics that you might want to take away from this in terms of structure. So give you a chance to read this and then we'll move on.
Okay. I know you haven't had a chance to read the whole thing, or perhaps you have because you're speed readers or you've read it before. Bonus points for you. How do you think this sets up the problem?
>>: It's very good.

35
>> M.C. Schraefel: Yeah?
>>: [indiscernible] what other options are.
>> M.C. Schraefel: Yes, sets the problem very nicely.
>>: The problem is great, but I don't know how long the introduction is and how long the page limit is for this paper.
>> M.C. Schraefel: It goes on. This is the introduction. We're not going to do it. There's two pages of introduction, but this is just to focus on the problem statement. Because again, you can read it yourselves, and I would encourage you to do it.
Unlike the abstract, which again I think was written in a bit more of a rush, this really does give us the context of the problem. I'm willing to believe that a solution is necessary for what? What are some of the attributes that give us a sense of the significance of the problem?
>>: Numbers.
>> M.C. Schraefel: Numbers of what?
>>: Number of people and --
>>: Who needs the solutions.
>> M.C. Schraefel: Yes. Isn't it wonderful when you can say this thing affects all of these people. This is not some little thing that a niche bunch of scientists doing research on a problem that nobody cares about would like to have.
>>: It doesn't specify the problem soon enough?
>> M.C. Schraefel: Oh, I don't know. Okay. So yes, I suppose if you wanted to improve it, make it even more perfect, then sure, we could drill into the problem faster.
>>: Once you establish that all these people need it, then you should say that

36 these people will actually do it.
>> M.C. Schraefel: Fair point. But let's celebrate a few more things about what's working first. In terms of where they actually put in related work, they don't tell us exactly what the related work is, but they reference it.
And when I was reading it, they were putting in related work where I expected to see it.
So the fact that they say, these folks need to create off and one off applications to support business functions. If that reference wasn't there, I would be going oh, yeah? According to you and what Army. But they've put a reference there which I can then go check and say okay, somebody else has already proven the fact that this is a problem. There's no solution, and they're going to make the case by saying, not only is it not supported, but there's also all these features that do or do not accomplish the task -- that's another place where I would have said prove it. Related work.
Nicely done. It's not a ton of related work. This is the intro, after all.
And they suggest that people have to program and not everybody is a programmer and there's all these problems with programs. The nice thing here is that they then give us a little more context. This is the most popular spreadsheet program on the planet, probably, and we can do something maybe in here to fix it but end users don't do this. So what if we could something about this problem. It like what if you could. Could you? Is it possible? Turn to the next page.
Okay. So they on now and tell us a little bit about their method, about how they've gone into looking at this problem, how they could possibly solve it.
I'm not sure if I would use the expression it turns out that. But even if you go with that, that they've got the space, the specification that they need to work in and here's how they're going to solve the problem.
And again, this isn't the whole thing, because we haven't got down to the map yet, but they kind of lay out a believable problem space that says there's room to work in here. Other people have said that this is a problem space and that it's worth addressing. We can see a possible solution for it. We understand why normal people wouldn't bother with this solution so let's see how we can approach this problem and what the possible payoffs might be.

37
So this sort of is probably why the abstract didn't kill the paper because this really, I would say, is a pretty nice setup, especially of a problem. And starting to get into -- and yes, your point's well taken about what exactly is the solution and why is it maybe could get that in there sooner.
But just if we look at this in terms of setting up a problem that people are willing to -- you know, I don't do anything in spreadsheets, but I thought this is kind of interesting. I wouldn't mind understanding how you did this and why this is a big success and how you evaluated it.
And as soon as people talk about usability, it like does that mean you used real people to test this, or was this a machine learning approach, or how did you do this? What claims are you making?
So again, I'll just leave that as a quickie for us to look at. And let's look at -- what's the next one that we can take a look at? This is kind of getting into the conclusion, but we'll move over that a little bit.
So yeah, let's take an example of a conclusion. Just to see if we get, again, a check here on contribution is what we really want from the conclusion and what we can do with it. Do we see that here? And this is the same paper, by the way. So remember the question is for the conclusion, what does the reader have that they didn't have before, and what can they do with it.
>>: Take a second to scroll down.
>> M.C. Schraefel: Anytime I see somebody claiming that they've identified a killer application, that again is a red flag for me, because that's a pretty bold claim.
>>: It's okay. You can be bold in your paper if you are [indiscernible] if you are at MSR and you are working with product group.
>> M.C. Schraefel: I wouldn't ever dream of encouraging anybody to say that they've come up with a killer application in a research paper. A spreadsheet is a killer application. A web browser is a killer application. I'm not sure that a scripting plug-in is a killer application.
>>: It's automated string processing.

38
>> M.C. Schraefel: I'm not sure that's a killer application either. It's nice.
>>: But why do you have a red flag as a reader if someone is claiming --
>> M.C. Schraefel: Because it's not up to them to prove that. It's to society and their peers to give them the accolade of killer application. For somebody to say this is a killer app, it's like saying I'm great.
>>: Do people usually read conclusion after they read the paper or before that? Because in this specific case, some sections before the conclusion actually made some support for the claim that it's a killer application, support from different people from the Microsoft [indiscernible].
So if the reader has a [indiscernible], that might be support.
>>: I definitely should know a killer application is something so compelling that people like switch platforms for.
>> M.C. Schraefel: Right.
>>: That's where [indiscernible] the context at this point in time or not.
>> M.C. Schraefel: I, again, I --
>>: He's trying to attract attention. So you sometimes have to take the risk in terms of attracting attention.
>> M.C. Schraefel: It failed, okay? This is I would call an epic, arrogant frickin' fail. If it's that good, it will be celebrated as such and people will come back later and say, this is the paper that presented a killer app that, you know, we're willing to change platforms just to use this.
>>: By the way, the application here is not the word for actual piece of software. It's an application of a technique, methodology.
>> M.C. Schraefel: Nice point. Just so that we don't end before shortcake on the semantics here, my only caution to anybody writing a paper is actually humility in the face of success is better than arrogance 99 times out of 100.

39
It's why using a lot of adjectives, like this is very interesting about your own work, or our own work, is not appropriate. Let somebody else say this is really interesting. Present it in such a way that they are led to that conclusion. Don't make the conclusion for them.
In this case, you lose nothing by taking out the claim about killer application. You say what problem you've solved and you've made the case that it's a significant problem. Therefore, your solution is awesome. You've shown how it's been evaluated. That makes the case for you.
In other words, let your work speak for you. Here's another tip about language. Avoid the terms belief or hope in papers. Science is not about belief. Hope is not necessarily an appropriate expression in a paper. Like we hope that this has enabled the following. Has it or not?
Instead of belief, we set out to demonstrate. We establish that. Use facts where possible. And where you can't, why should somebody get behind you based on a belief? Unless you have that kind of credibility in the field that we had an intuition that this was -- if you're just starting out, who is going to trust your intuition? You haven't proven that your intuition is worth trusting. Maybe these kind of papers will help that, such that if somebody who's senior in your field says, well, if Jim Gray had said, well, I had this intuition about something about cubes with databases, you might go oh, okay, that sounds kind of cool.
Use facts.
So again, conclusion about what does the reader have that they didn't have before? I have no idea what this first paragraph is doing. Does that have anything to do with spreadsheets?
>>: Was there future work? Was there more?
>> M.C. Schraefel: It's fine to have a conclusion and future work together, especially if you're running out of work.
>>: Was there more?
>> M.C. Schraefel: No. This is it.

40
>>: So there is no future work here?
>> M.C. Schraefel: The future is robots.
>>: The first paragraph [indiscernible].
>> M.C. Schraefel: Exactly. So again, what the reviewers are obviously saying is there was enough good stuff in the paper that the conclusion -- and again, we tend to do this in computer science so much. Like the abstract, we kind of blow off the conclusion. The main weakness of conclusions is we often just say in this paper, we did the following. We just repeat the introduction and a few other words. No.
The strength of a good conclusion is one that actually does let the reader know what have we got that we didn't have before. So we know we've got a bunch of algorithms here that did some cool things with strings. What does that let the reader do with this?
How can you go further? What's the next steps in research because of this? So if you have a list of contributions that you have from your work, that should model what other people can build on this work. Here's another way of framing the conclusion. It shows that the work is not a dead-end.
So the conclusion provides an opportunity for the reader, who's gone through this with you, to be able to say look at this knowledge that I have that I didn't have before, and here are some things that I can do with it. And future work would be nice to, to be able to say we didn't test this following thing, but it shows us that you could probably do this. And this is how we're thinking about doing it. You might want to do it.
So again, the value of the conclusion is to let the reader see where the map could continue to if you had more time. Gosh, if we had more time to spend together, here's a few things we could do. For instance, I might say if we had more time to spend together now that we've gone through abstracts, intros and conclusion, we could spend time talking about how to structure related work, methodologies, and discussion sections. And what's the difference between an analysis section and a discussion section? We could do that together, but we have other pressing matters.
But it gives you a sense that, oh, we've got some stuff to do. So again,

41 conclusion is about what the reader -- you're not reiterating the paper.
You're reiterating what the gift is that your paper has given the reader.
Thank you so much for spending this time with me. Here's what you have. Tell us what they got, Bob. And how you can build further.
So here are some other examples on conclusions that we've talked about.
Where's the value. We won't go through these conclusions. But this is the core take-aways from this workshop.
It's really thinking about writing for readers. And when we say that you need to write for readers is time. And the way that we lose time is we usually start writing our papers when we're finished our work. What I'm suggesting is that a strategy for success to have papers read is to start writing your paper as soon as you've defined the project and you know there's going to be a paper outcome.
Why do we need to give ourselves time to write fake intro, abstracts and conclusions? What are some of the benefits of giving ourselves time that we've talked about?
>>: You'll be able to perceive [indiscernible] rethinking every line.
>> M.C. Schraefel: Okay. So give yourself a chance to see if you have the checklist of heuristics about the problem, the method, the results. Anything else that you want time to do?
>>: You can circulate it to a couple of colleagues.
>> M.C. Schraefel: Feedback. Anything else?
>>: Study design. You're like wow, I really need to write about this so I need to collect that data.
>> M.C. Schraefel: That too. Motivation for the work ahead. Remember, it gives you time to fall out of love with what you think is clever and make it more concise.
>>: Fill holes.
>> M.C. Schraefel: Fill holes that you might not notice the first time you're

42 writing. And again, this is really about perhaps a slightly different head space than we're used to thinking about writing. Usually, we think about writing, I must write up my results. What I'm hoping the take-away from this is, is I must write for other people to read what I have done.
The whole point of what we're doing is really to advance the state of the art.
And the only way we really advance the state of the art is when other people find value and use in what we've done. And so because you're putting so much energy into creating wonderful things, it's important to package it so that other people can appreciate that and be able to access it and make use of it.
So these questions are to help, can we answer these questions when we're looking at our paper. Can we put ourselves in the perspective of the reader and say yes, we've done our best to be able to answer why should I read this.
Why should I trust you? And what do I get from this?
If we can make sure that we are trying to answer these questions in our writing, we improve our chances for success. I guarantee it. And how I can guarantee that -- and again, we've looked at some heuristics for how to do each of these steps. Why should I read this engaging problem. The strategies we've looked at through the rest of the introductions and extracts. Why should I trust you. We've looked at, last time around, expertise. If you weren't here, check out the video from this about developing trust around how to refer to other experts in the space or how to create the problem so that we understand that it's valuable.
What do I get from this. We've talked about the value of the contribution statement and the conclusion. I've seen people get best papers because their conclusions were so clear and explicit in their contributions. It was like wow, this is awesome. This has made the whole case of why this was so valuable and I believe it and people are going to want this.
So again, writing is for other people to be able to read our paper, and we've talked about how to discover the paper, how to understand if we should -- from the title and keywords, how to understand if it's what we want to read from a good abstract, and how to let somebody have a good experience, that coffee or wine experience with our papers based on a really good intro that situates problem and all the rest of the things we talked about in the heuristics. I should be able to list them, eh?

43
The problem, the reason the problem is significant, how to trust that, our approach to it, and how we're going to describe that in the rest of the paper to lead to that conclusion.
Do you have any quick questions before we wrap up? Is this good? Some useful stuff? If you have questions coming up in the future, do not be afraid to ask.
Ask sooner. Practice writing. I'm going to leave you with practice, practice, practice, practice, practice and nobody likes doing this. Practice, practice, practice will make it a much better experience for you.
Thank you so much for coming. We now go and enjoy sugar in the sunshine. All right. Thank you very much.