- If you offer a suggested correction then MAKE SURE IT IS CORRECT. The author may say Oh, I guess that's a mistake, I better make the correction having thought that you read it carefully. While that may well be the authors responsibility, be very careful. The first rule of proofreading is DO NO HARM.
- If the paper is a preprint then the author should be VERY GRATEFUL since they will be able to make the correction before it becomes official. But see next point.
- If the paper is already in a journal the author might want to correct the version on their own website. I can picture a day when the version on the authors website or arXiv are BETTER than the so-called official version. So the author should be grateful here as well.
- For arguments sake, lets say that in Karp's classic paper Reducibility Among Combinatorial Problems, where he proves 21 problems NP-compete, he made a mistake on 0-1 programming. The problem IS NP-complete and the correct proof is in many textbooks (also, anyone reading this blog can probably do it themselves). Is it worth wasting Karp's time with this?
- Most authors will be surprised and delighted that someone read their paper.
- Some authors won't care. Either they left the field or they don't care about the constant they got wrong or they don't want to be bothered. That is their right; however, what to do? You can't publish an erratum for them.
-
In High School while studying some Combinatorics
I came across the following passage.
The number of ways to arrange n distinct objects is n × (n-1) × ... × 1. For example, the number of ways to arrange 5 distinct objects is 5!.
I did not understand why they were so excited about the answer. Using an exclamation point seemed over the top. And CLEARLY there were two mistakes!- The answer of 5 is WRONG. The answer should be 5 × 4 × 3 × 2 × 1 = 120.
- There is a spurious period after the exclamation point.
Wednesday, October 05, 2011
If you find a mistake in someone elses paper you should....
What do you do if you read a paper or book and find mistakes in it?
My first impulse is to say:
Email the author. Always be polite and admit (which is true) that you
may be misunderstanding something.
A few thoughts:
Subscribe to:
Post Comments (Atom)
Regarding (6), I would not call it "right" in the case "they don't want to be bothered."
ReplyDeleteFrom http://www.wisdom.weizmann.ac.il/~oded/faq.html:
"if you have fundamental doubts regarding a central claim in anything that I have published (e.g., you believe there is an error), then it should be my duty to consider this seriously."
Regarding "what to do," why not set up a web-page where all conceptual "typos" with no immediate fix are listed? No harm is intended. It would just save some students a lot of time.
I think the "tone" matters a lot here.
ReplyDeleteThere is a difference between finding a bug (an error that cannot easily be fixed), a mistake (an error that can easily be fixed), and a typo (an error that is irrelevant to what is being proved).
If someone emails me to tell me they found a typo in a paper of mine, I thank them and correct the typo. What I don't like are people who write me to tell me about a "serious error" that turns out to be a typo.
Emanuele:
ReplyDelete"Regarding 'what to do,' why not set up a web-page where all conceptual "typos" with no immediate fix are listed?"
What do you think of the following? A forum, with one thread or subforum for each theory paper ever published. The threads could be auto-created by scraping dblp, or in some other automated way. People can ask questions about a paper in its thread, and anyone can answer. Authors who create a profile on this site can leave their email address, and any questions posted in their papers' threads could be automatically emailed to them -- however, since anyone can answer in the thread, the questioner doesn't have to wait on the author to respond.
Although questions about a paper can be raised on theory stackexchange, the above has the advantage that all questions related to a single paper are easily found in one place.
Is it worth wasting Karp's time with this?
ReplyDeleteIt depends. Are people likely to use the same erroneous reduction? then yes, it is worth making this known.
It might also help to give a heads up so that the authors pay more attention next time and don't carry on publishing flawed proofs.
I think Jelani has a great idea. In the past, there have been discussions of adding comments to the arxiv, for example.
ReplyDeleteIf there were a discussion thread for each paper published, maybe that would help get feedback from people who really care about your paper, rather than get conference referee reports from someone who clearly doesn't care and read (or didn't bother to read) the paper only at the last minute. A reviewer could even anonymously post questions and then future reviewers would be able to see the conversation.
One day I think we'll just have this rather than conference reviewing ..
one problem with the forum idea is that it will fill with incorrect 'corrections' and become useless quickly
ReplyDeleteJelani, I like the suggestion. But I think many people feel that pointing out errors in a public way is unnecessarily embarrassing for the author. For instance: http://meta.cstheory.stackexchange.com/questions/204/are-specific-questions-about-theorem-x-in-paper-y-fine/207#207
ReplyDeleteCheck this:
ReplyDeletehttp://www.scirate.com/
It's not just pointing out errors--it's also questions, clarifications, "have you considered this generalization", etc. There can be a civilized tone to discussions.
ReplyDeleteI don't see why this idea always gets dismissed because it will be embarrassing. The error is still there whether or not someone points it out on the forum and how efficient is it if 100 people all find the same error and wonder if they are correct?
I think it would help the overall understanding of papers. Each "incorrect" correction can be responded to by someone (not necessarily the author) and even if it is not a valid "correction", it could point out to the author a way to make their paper clearer.
Since its inception, ECCC has had a feature that allows the posting of comments on papers by others (unlike the Arxiv). I like it but it is underutilized.
ReplyDeleteYou can't publish an erratum for them.
ReplyDeleteYes, you can.
what a waste of post gasarch. i congratulate u for such a wasteful entry. there are gazillion and one things one could do and could not do. can u please for once bring up a topic that is less of bureaucratic non-sense and be a little original . Wat are u workin on ? what problems are hot ?
ReplyDeleteJelani, yes that is basically what I had in mind.
ReplyDeleteI like the idea of scraping DBLP.
I do not see the point of embarassment...
all we want is truth ;-)
No paper would be removed by any proceeding as a result
of this process.
I certainly wouldn't be shocked if some of my papers were active on the list ;-) On the other hand,
I would probably learn a lot by that kind of feedback.
I also think this type of feedback is especially
useful in a community such as ours, where many papers
are published in conferences only.
I am sure I am not the only one who has seen many
important or even seminal papers that contain mistakes
with no easy fix. And I may also not be the only one
who has contacted the authors about it in vain...
Indeed as Paul points out, this feature has always been available on ECCC. One problem with ECCC is that it is only used by a fraction of our community. (And certainly having this feedback system more active there might not encourage more people to post ;-) It also requires user's intervention to actually post a paper, while it would be good as Jelani suggested to have it populated automatically.
But maybe the biggest problem is that this feature on ECCC is hard
to judge, because despite visiting ECCC frequently (and liking it),
I don't think I have ever seen a single comment on it!
Thank for you an interesting post GASARCH. This is on-topic and relevant to the community and a discussion that should be had. Some interesting thoughts have been posted already (not including the poorly written/spelled crank message above).
ReplyDeleteBack in medieval times "stocks" were used for public humiliation. I would suggest that each year's STOC have a section for agreed-upon errors in previously-published theory papers.
ReplyDeleteI find Arnab's response embarrassing. We are more concerned with our reputations than if our work is actually correct or how this would affect others who use our work. That helps enforce our bad reputation in the math community as being unrigorous by often not providing complete or correct proofs.
ReplyDeleteAlso, a lot of the questions about a popular paper that is quite complicated and hastily written (think unique games paper) would help people in understanding parts of the paper that are unclear (often because we rush to write a paper for a conference deadline and then do not revise). Much of the discussion would not be about pointing out errors but for explanations.
Also, if an :error" turns out to be a typo, why would the authors be embarrassed?
I assume you're joking about 5 factorial (i.e., 5!) in #7, but it isn't really too clear. I sure hope it's a joke!
ReplyDeleteSame here. After reading the whole blog and thinking it's a thoughtful one, it's kind of hard to get serious any more when you read the last paragraph....
DeleteIf the paper/book is NOT already published, then it is the DUTY of the reader to tell a (even if potential) error he/she observed. Cf. our refereeing duty! If the paper IS already published, the situation is more subtle. It then depends on how heavily I want to use the result and how crucial the error is. If the result is important for me, but I cannot fix the error in 1-2 hours, I will definitely contact the author (this fortunately hasn't happened, as yet). But I would never point the error in my paper, without first contacting authors: I have seen several such papers, and this is the "peak of arrogance" for me.
ReplyDeleteThe main source of buggy profs are, however, "half cooked under time pressure" CONFERENCE publications. When writing my book on circuit complexity, one of the biggest problems was how to make a rigorous proof from often buggy "conference sketches".
@Bill, concerning your point (1): it is a DUTY of the author to be very careful when making a correction - for the commenter it is enough that he/she has DOUBTS. After all - this is author's paper/book. Unfortunately, the situation you described is not unrealistic, I also was in it quite recently, you know this :-)
@Paul, you are right: the possibility to comment papers at ECCC is indeed underutilized. And I guess, this is because of the too FORMAL commenting procedure itself: it is similar to that in journals. I would be happier to have a more flexible way to do this. Comments I've seen so far (except those by authors themselves) are, in fact, short papers (notes).
@Emanuele: I have seen several such "errata" for papers on author's home pages. This shows that the author is honest, but doesn't solve the problem. If a published, made PUBLIC paper has a crucial error, this fact must also be made PUBLIC. By sending an errata to that journal. Unfortunately, conference publications do not have such a possibility.
Start with "There is just one point where I have encountered a difficulty."?
ReplyDelete@Anonymous (5:13 PM, October 06, 2011): Being polite is NOT an issue. What we are aiming to produce is the TRUTH. And if somebody sees this "truth" is wrong (or not founded) -- then no pardon. Politeness in the truth search is even dangerous!
ReplyDeleteThank for you an interesting post GASARCH. This is on-topic and relevant to the community and a discussion that should be had. Some interesting thoughts have been posted already (not including the poorly written/spelled crank message above).
ReplyDelete---------------------------------------------------
I disagree. I agree with comment pertaining to crank/mispellod entry. Interesting thoughts have not really been provided here. Neither do I see anything new being posted here. Gassaraaaaachhhhh! PlEASE STICK TO SOMETHING MORE ORIGINAL. I would be happy to habitually peak in, if the quality of posts and discussions was lifted back to the standard when Lance was responsible.
I am not entirely discrediting your efforts; there were some great posts in the past, but please, get back to that standard.
@Anonymous(2x), Mr Standard + that of 3:44 AM, October 06, 2011: Are you SO impolite also when fighting for the truth? If so, half so bad. But why then instead of aggressing Bill (and other people here) you yourself do NOT write a mega interesting post?! (Bill and Lance will definitely publish it as a guest post.)
ReplyDeleteMr Standard, you should take Stasys advice and have a guest post. I am sure that Bill and Lance have other things on their mind and just keep on pooping out interesting yet time constrained thoughts which surmount to entries like this one. I am glad that even someone with your STANDARD keeps seeing effort has been spent (not as wisely as it should) but there's still hope left. (Unlike Obama where hope misrepresented efforts.)
ReplyDeleteExcluding theory papers, I have come across -numerous- papers (in ACM and IEEE) where I have asked the author(s) for their source code because I couldn't reproduce the results with the information provided in the papers themselves. However, so far, not a single author has come forward with their source code. In fact, for all my requests I have only received two replies, and they were along the lines of "Sorry, I'm too busy" and "I don't have the source code." In these cases, I feel that the authors are not confident of the results, or that they doctored them.
ReplyDeleteI do strongly prefer those authors who post their code online.
--Ateeq
I think pun is intended in point (7) .
ReplyDeleteAfter reading #7 it seems you don't understand #1 yourself. There are no mistakes in #7. 5! means 240 (not even 120). ! is not a exclamation - it is a factorial sign in this context. That's why there is a period after it.
ReplyDelete