I got a request to referee a paper that I really could not turn down since I'm one of the few people who is qualified This may be reason enough to reject--- if very few people could referee it then perhaps the topic is too obscure. However, obsurity of research is not todays topic. Today's posting is a rant!!
BEGIN RANT
The request to referee was an automatic email. I then had to do the following:
- Goto a website to accept.
- Receive an email telling me how to access the paper.
- Goto another website, click on something to receive another email telling me my password and login.
- Change my password to one that I could remember. This took a while since they didn't state their rules for passwords, nor did there error messages tell me what the rules were.
- Goto another website with that password and login and register by giving my name (gee, I think they would already have that), email (ditto), school address, areas of interest, key words of interest (that was really hard- it was a long list and nothing quite fit), my right thumb print, and my left eye retina scan.
- To get the paper itself the website kept on doing odd things. I called them (by telephone!) and the editor said that I was not being an idiot, the website had problems that day, and they would try to send me the paper, but they were not sure they could access it. I told them that if they did not get me the paper within 24 hours I would not referee it. They got it to me 23 hours 45 minutes later.
- I am looking forward to seeing if the website will be working when I fill in the referees report, which must be done on the web.
END RANT
I do not think I'm being a luddite to complain about this. Being a luddite (different link) is not the topic of todays post. I do not think that electronic refereeing systems using the web are a bad idea. But electronic refereeing systems are not todays posting. Todays posting was a rant!.
You don't mention which organization/publisher you were dealing with, but I think it's kind of ironic that a system presumably designed specifically for computer scientists to use would be so weak on the human-computer-interface elements.
ReplyDeleteSince you're only ranting today, would you mind if I (just slightly) spam your comments? I'll be hosting the 16th Carnival of Mathematics on 9/7 over at my blog (which is not on your blogrole, but I understand...). If any of your readers have any math or TCS posts they'd like to contribute, we'd love to have them.
This sounds like a typical encounter with one of the spawns of the behemoth Elsevier!
ReplyDeleteThe same thing happened to me recently (from your description, I suspect we were dealing with the same publishing company), and I almost turned down the request on those grounds. Ultimately I felt bad for the authors of the paper and agreed to do it after sending a nasty email to the editor (to forward to the publisher).
ReplyDeleteTalking about badly design online interfaces... Our company is listed on the Portuguese Stock Exchange. The regulatory entity (the equivalent of the SEC) requires we file every year a questionnaire. Here's the method:
ReplyDelete1. The questions come on a Word .doc, each question being given a numeric code.
2. A separate Excel .xls is provided, with the codes already in place.
3. The answers have to be written in the .xls file, which must be saved as a .csv file
4. The .csv file must be renamed .dat
5. This file is then uploaded on their extranet
They don't think this is byzantine enough, so they added the following kinks:
- The .csv file in question is required to use ";" instead of "," as the delimiter, due to the Portuguese standards.
- This file, when uploaded, if there are any errors, you get the first error on the file, with the format being "error on line x, upload aborted" (meaning that we had to upload the file around 50 times until we managed to fix all the errors...)
Now for the worst part - this story is not from 1996 - it's from 2006...
Bill: It would be a useful service to point out which journal or publisher this came from so that unsuspecting authors who don't want to subject their reviewers to this can avoid sending papers there.
ReplyDelete(It seems likely that the detailed questions are so that the publisher can do targeted marketing rather than because they need the information. I would refuse to answer such questions. If that disqualifies me as a reviewer then that's fine with me.)
(This is Bill G.)
ReplyDeleteThe Journal was
Transactions on Dependable
and Secure Computing.
It is an IEEE journal.
ALSO- Kurt, even though
you didn't ask, I WILL put
you on my blog roll. Your
blog is about complexity
theory, so its CERTAINLY
in scope.
So this paper must be on combinatorial recursion theory then?
ReplyDeleteMichael Mitzenmacher says:
ReplyDeleteOdd. I'm reviewing for Transactions on Information Theory, an IEEE journal. It's not a warm and fuzzy electronic system, but it's generally easier than what you describe. (I have to cut/paste their bizarre paper reference number and password, but that's about it.) It's probably on a different system, or maybe they really have all my info since I've reviewed for them before.
You want to hear me rant, let's talk about the painfully annoying style files IEEE and ACM require...
To Bill, Thanks!
ReplyDelete...let's talk about the painfully annoying style files IEEE and ACM require.
ReplyDeleteOh yes, let's. Why does IEEE think that it is a good idea to have text in Times yet equations in Computer Modern? It's not laziness. They actively changed my all-Times article into their hybrid house style. Does anybody have any idea?
Which publisher do you think have the nicest tex/latex style?
ReplyDeleteThank you for your rant. I so totally agree. I can't deal with automated emails asking me to check a website to find a paper (can't just email it?), but to access it I first have to ask for another email to reset my password (which email may or may not ever arrive).
ReplyDeleteI have a very simple policy. I will accept a copy of the paper, and I will write a report and send it to the editor by email. If they have a web site, it has to be as simple as that. If it isn't, I (a) don't do the mundane data-entry, and (b) complain. If they won't either do it for me or otherwise circumvent their system, then they are SOL.
ReplyDeleteWorks for me!
then they are SOL.
ReplyDeleteSOL?
I love rant. Rant is something which describes the pain and give an opportunity to cure it. Keep it comin'.
ReplyDeleteSOL?
ReplyDeleteSOL = Second Order Logic.
I agreed to review a paper for them in Sept., with a January target date. Personal events pushed that to early July, when...my listing as referee had vanished!
ReplyDeleteI wrote the report (finishing it on a plane) anyway, and sent it by e-mail with due apologies. After 3 automatic replies, I got a real one, saying the report was most timely and appreciated!
The disconnect between communication and expectation is the issue here. The system may save time and annoyance for editors, but leaves both editors and reviewers more in the dark about what the status is, and sometimes the personal touch is nicer even when you're the one being nagged...
The problem is not unique to journals. I've also become disgusted with easychair and their 'one size fits all' mentality about how program committees should work. Conferences with two-tier committees don't fit in the system. I also have a requirement for serving on program committees that I will only do it if I have the ability to see all papers that I do not have a conflict of interest on. That way I can compare the papers I am reviewing to the general mix of papers and get a global feel for the content of the program. Most program chairs end up ignoring that rule, so I'm getting reluctant to accept ANY program committee assignments.
ReplyDeleteThese tools are supposed to make science work more smoothly - not interfere with the quality of the editorial process. In many ways we were better off to have humans making decisions according to whatever conditions were appropriate.