- Declarative first sentences of the introduction, like "Analyzing Left-Handed 12-SAT is a key approach to solving the P versus NP question." Just because you say it doesn't make it true.
- "We use novel techniques that might be of independent interest." A double faux pas. You don't get to call your own techniques novel. "Might be of independent interest" is such a meaningless statement.
- Footnotes (and parenthetical statements) which interrupt the flow of the paper. If it's not worth mentioning in the text then don't mention it.
- Using citations as nouns like "[13] using techniques of [6] showed the main result of [4] follows easily from [18]." I hate having to keep flipping to and from the references to read these papers.
- Using the cliché "larger than the number of atoms in the known universe." It's big. We get it.
- Using the word "respectively" which says "I'm going to give you something hard to parse because I'm too lazy to write two sentences."
- Titles with symbols or complexity classes: If you can't describe your research with words you might consider becoming a mathematician.
Computational Complexity and other fun stuff in math and computer science from Lance Fortnow and Bill Gasarch
Tuesday, April 12, 2005
Paper Pet Peeves
Little things that annoy me in research papers.
Boy are you opinionated.
ReplyDeleteAh, yes, this is a very insightful post. If I may, I'd like to add a few of my pet peeves:
ReplyDelete1. Terrorists flying planes into buildings is one of my pet peeves. If you can't make a statement by publishing it in a newspaper or declaring it on television, you shouldn't just fly a plane into a building.
2. Presidential abuse of power. Just because the president of the United States doesn't agree with a particular party on an issue doesn't mean that he should sick the CIA on that party and have them 'silenced'. This, I feel, is unconstitutional.
There are a few others, but I will save them for your next 'pet peeves' post.
"We use novel techniques that might be of independent interest." A double faux pas. You don't get to call your own techniques novel. "Might be of independent interest" is such a meaningless statement.
ReplyDeleteNot sure I agree with this one. Say, someone proves a minor theorem on complexity classes using differential equations and advanced topology. I wouldn't hesitate to call the technique "novel" and of possible "independent interest" and highlight that in the introduction. Particularly if the paper is being submitted to a large conference where the reviewers have very little time to assess the relevance of the result.
Also:
ReplyDelete* Using numerical references instead of alpha references. [CS39] is much more meaningful than [6].
* Putting your abstract in the title (very common in FOCS/STOC).
* Putting "seminal", "novel", "clever idea", etc next to a refences, because you think those people are going to referee your paper.
I disagree with "may be of independent interest" - this is a way of saying that not only the bottom line is interesting, but also how you got there is interesting. In fact, it might be more interesting than the result itself.
I don't like it when papers repeat entire sentences from previous sections of the paper.
ReplyDeleteI understand that the intro, abstract, and a subsection might all mention one particular thing, but that doesn't mean you need to copy and paste.
As a new grad student, I found this helpful! Sure they're just opinions, but I think they're reasonable.
ReplyDeleteI don't like it when authors throw around adjectives like "Excellent" and "interesting." I'd like to be the judge of whether I think something is interesting.
* Using numerical references instead of alpha references. [CS39] is much more meaningful than [6].
ReplyDeleteVery true. Sadly many publishers require the latter form.
Alpha references are easier to remember should a second reference to the same paper occur later on.
Also if one is familiar with the field, an alpha reference might be all one needs to tell which paper the authors are talking about, thus saving a lookup at the bibliography section.
I share most of these peeves. (1)
ReplyDelete(1) Except I like footnotes.
What's wrong with titles containing symbols complexity classes? E.g. "IP = PSPACE" is totally clear. I feel that inside jokes and obscure references in titles are much more annoying.
ReplyDeleteI agree with all of your comments - but you didn't mention one which is used so often. The phrase "Without loss of generality ..." is typically used to narrow or reduce a problem to a more tractable form, without having to justify doing so.
ReplyDeleteI haven't yet written any paper. But seems like there is a world waiting to judge you. Not too inviting for youngsters.
ReplyDeleteWhat about Hastad's STOC 1986 paper?
ReplyDeleteHe had refered to the switching lemma being of "independent interest"
What's wrong with complexity classes in titles? And if one can get distracted by footnotes, one needs meditation or something!!! Parenthesis are a boon to mankind. (Just kidding). If one has a novel idea I don't see a point of being modest. Let me see, doesn't the 5th point contradict the 7th. You want a paper whose authors are off the point while writing the title but very very to the point while writing the article (No analogies please and no brackets either). Hmm, tough combination!!! I have been trying to google search for such a great paper. Atleast google doesn't give a positive result. Now the question is: Does such a paper exist?
ReplyDeleteI have one thing to add: could we imagine that once all citations are made as \cite[Theorem 1.2]{YZ99} (rather than "people say it was implicitely proved in some early paper of John Smith")?
ReplyDelete--Edward
P.S. I like footnotes and parentheses. They convert a paper into a kind of HTML.
...but one lesson that I learned from my own experience is that one should not put a crucial part of the main proof into a footnote! (For example, \cite[Theorem X]{YS99}\footnote{Clearly, their theorem applies to our case}.)
On the specific peeves...
ReplyDelete1) I'm not so upset about the novel techniques/independent interest thing. I admit it's a somewhat vacuous sentence, but it's easy to skip over, isn't it?
2) I don't mind using citations as nouns, either -- in fact I prefer it!
On a more general note, as I'm now reviewing for a conference, why oh why can't we just have theory submissions be in the same format as the final conference copy? Why do we go through this bizarre "10 pages in 11 point font single spaced, but you can add extra material in appendices" instead of just saying "the conference paper will be 10 double-column pages, so your submission will be also"? Hey, Lance, if you want a guest column on this issue, I'll be happy to write one...
Is "Primes is in P"
ReplyDeletea bad title compared to "Primes is in Polynomial time"?
I would second Michael Mitzenmacher's comment. I would guess that the "10 pages and appendix" submission form intends to ease the reviewers' job, but for papers that seem worthy I find that it makes things harder with me jumping back and forth between the body and the appendix. And for papers that are clearly not worthy, I think I would know how to skip some of the senseless drivel on my own :-)
ReplyDeleteI would second Michael Mitzenmacher's comment. I would guess that the "10 pages and appendix" submission form intends to ease the reviewers' job, but for papers that seem worthy I find that it makes things harder with me jumping back and forth between the body and the appendix. And for papers that are clearly not worthy, I think I would know how to skip some of the senseless drivel on my own :-)
ReplyDeleteI'll be the first to disagree with Michael's comment. I think having [the option of] appendices in a submission is important (e.g., so full proofs can be given and correctness verified) but such proofs do not need to be in the conference version if there is lack of space.
ReplyDeleteRegarding Michael's suggestion: I once had a paper rejected from a conference, and one of the referees' main complaint was that some of the proofs were omitted from the paper (we had all those proofs written, but didn't include them in the paper for the lack of space). Whether you like it or not, the presence of proofs in the appendix can make a difference in the acceptance of the paper. So, I don't think it's a good idea to eliminate the appendix, but it makes sense to use the same formatting for the submission and final version.
ReplyDeleteRe Lance's pet peeves: I don't agree with 2 (it's good that authors clarify what their contribution is; this could be the technique), 3 (I like footnotes and paranthesis, without them the text is too flat), and 6 (I like using respectively to avoid repetition). I don't mind complexity classes in the title, but symbols in the titles (or even subscripts/superscripts) make it harder to search for the title.
I think having two columns for conference submission is a horrible idea. It is a hard to read format, which is just long. Apendices are also bad but they are acceptable. What I currently do, is to submit a 10-12 pages single column submission, with no apendices, and put the full version on the web (i.e., arXiv). The idea that submitting more is better is unrealistic, since people can not read all you write. A submission to a conference is a proof of concept. If you can not convince the committer/referee after reading 10 pages, that your paper should be accepted, or that they should waste their energy and see the full version of your paper, then your paper should not be accepted.
ReplyDelete"I share most of these pet peeves, in particular 1, 4 and 7..."
ReplyDeleteUhmm... isn't that an example of let's see, which one was it...ah yes 4?
On a more serious note, I think many of these are matters of practicality.
-I think both declarative-sentences-in-intro and our-novel-techniques-rock-statements owe their origin to the terrible refereeing process in conferences. I recently sent what I think is a very good paper to a third-tier conference (for some practical reasons) and it got rejected. Apparently one of the referees did not bother to read past the abstract and wrote a negative report based on that. In fact some technical statements in the report are simply false and this is not a case of a bad intro. With referees like this, and PCs that listen to such referees, no wonder you do what you have to.
- Citations as nouns are often necessary because of space constraints. I also prefer alpha references but they look quite ugly.
- Respectively can be used in good and bad ways.
Just to be clear, I'm happy to have appendices, or arXiv version of papers available on-line, for the papers where the authors feel that 10 conference-style pages isn't enough for the details. It happens, I understand, and if reviewers might want to see the proofs, they should be available somehow. (I'd prefer arXiv or a pointer to a version on the authors' Web page to having to deal with 30 page submissions, though.)
ReplyDeleteBut still, why shouldn't the submission version be the same format as the final conference version? Wouldn't this save everyone a whole lot of time? Authors wouldn't have to make up multiple versions; reviewers would have a much better idea of what the paper would look like in the conference. (When there are 20 pages of appendices, does the reviewer have any idea what will make it in the final conference version?)
For those who complain that it's "hard to review such papers", I'd ask that you keep in mind that all I'm asking is that they're in the same format AS THE FINAL CONFERENCE VERSION! If you think that format is hard to read, shouldn't you also be complaining that the format for the conference version should be changed, so that everyone could read the conference papers more easily? People can argue what a good format is, but why should a good format be different for the submission and the final version we want people to read?
About Michael's suggestion:
ReplyDeletethe complexity conference
has been doing that for the
past two or three years.
The only difference is that
instead of asking for a
10-page submission in the
unreadable 2-column format,
we ask for a 20-pages
submission in the standard
format, no appendix.
About posting full papers
on the arxiv, or on ECCC,
simultaneously with the
submission: I would
encourage doing so, and
repealing the rule
that papers "should be
evaluated only on the basis
of the submission."
Luca
Conference versions are often quite unreadable, but it doesn't matter because you can just download the full version from the authors's webpage.
ReplyDeleteI thought the main point of the
standard two column conference version was to pack the maximum amount of information into ten pages, because in the old days conference proceedings were the main way to quickly disseminate results. Thus, it was more important to use the proceedings to give as much technical information as possible about a result than to create a readable, interesting, and clearly explained extended abstract.
Hi Lance,
ReplyDeleteHow about a pet post on "Reviewer Pet Peeves" ? In particular when a reviewer refuses to accept that the new result is better than his previous paper and makes up implausible reasons like "the new algorithms would not be practical" to reject a paper.
Conference versions are often quite unreadable, but it doesn't matter because you can just download the full version from the authors's webpage.
ReplyDeleteVery often, those unreadable papers do not have a full version, and the authors do not bother to produce one...
Moreover, if you see the phase "details will be given in the full version" in a conference version, then 9 out of 10 times the full version will never appear.
I'd say that if the full version of the paper has not been submitted to a journal within ten years or so, and there are substantial details missing in the conference version, then the result should be officially declared up for grabs again and journals should accept a full version of the paper from a different set of authors.
ReplyDeleteAlex
Refusing to mention the size of the universe in comparison with other large numbers puts you up against the big guns, I'm afraid.
ReplyDeleteLeonid Levin, in his 2004 'Kolmogorov lecture' at Royal Holloway, couldn't seem to stop using the analogy.
Upon the prospect of having your strong encryption scheme broken:
"You'd have to be the unluckiest particle in the whole universe...
...but then you'd have other problems!"