What is the purpose of STOC? Initially it was to help spread knowledge of the latest results, through both talks and lunch. Even though we can now tweet the latest VDW numbers, STOC still serves this purpose. Another (likely unintended) purpose of STOC is to give researchers a quick yet prestigious way to publish. Hiring committees and Tenure committee's DO ask questions like How many STOC/FOCS publications does she have?. Some people think this is an awful system since these papers are not refereed carefully. I am not going to debate that here. My only concern is making STOC better at spreading knowledge.
What are some of the problems with STOC?
- People don't want to serve on the program committee since its a lot of time and they can't submit. The two-tiered system used for STOC 2013 seems like a good solution to this.
- Referees Reports (can we even call them that?) are often not very informative. The two-tiered system COULD help this since each committee member has less work and there is a small oversight committee. Another solution that some conferences use is to give the authors a chance to rebut a report and/or rewrite the paper. I'll discuss this more in the next point.
- Since the reviewing process is rushed there have been papers that are just plain WRONG. This can be confusing for someone coming to the literature. Also there are throw-away- comments like This can easily be extended to the case of weighted graphs. where this is not easy at all. How big a problem is this? How much worse than Journals is it? I DON"T KNOW. Would the Rebut/Rewrite help this? PRO: Referees don't have to decide RIGHT NOW what to do and can ask the authors things? CON: More back and fourth, more work. CAVEAT: This might make STOC more like a journal with fast turn-around time.
- Some of the papers never get into Journal Form. Again Rebut/Rewrite may help in that the STOC version is better, but this is more giving in to the problem rather than solving it. Demanding full versions of papers (now possible since with e-proceedings page limits are less of an issue) is a good idea (and I think IS being used now by STOC).
- Many good papers get turned down. Going to three parallel sessions would help this. There may be logistical problems here, but I think this is a good idea. Are there enough good papers to make this work? I think so- and the committee would have the freedom to NOT use all the sessions in case there aren't quite enough papers. I do not think this would make STOC's prestige decline.
- It has been said that only narrow technically hard stuff gets in and not simple short new ideas. Its hard to know if this is really true. But in any case the three-parallel sessions may help this since there would be room for diff types of papers.
- Personally I get more out of the workshops and invited talks then out of the refereed talks. Hence I would like more of those. Posters are good also. More to the point- I would like more VARIETY in whats at a conference since people get knowledge in different ways.
- Can you really communicate your latest and greatest result in a 20 minute time slot in a crowded room where the adjacent bathroom is out of order? Even though we've made great advances in technology (I call PowerPoint PROGRESS but some disagree) and in plumbing (in the old days STOC people had to use an outhouse- do young people even know what an outhouse is anymore?), is there a better way to do this? It was suggested that ALL talks be POSTER sessions (NIPS does this). This should NOT be viewed as inferior or demeaning so long as we still have published proceedings (whatever that means in the days of arXiv) and high standards. The only relevant question is: Would posters be a better way to convey results? I DO NOT KNOW, but I think it would be worth trying out.
So in summary I want to see (1) more workshops, invited talks, and student posters, (2) Full papers in the proceedings, (3) two-tiered program comm. (4) either go to three parallel sessions or have posters. Some of these could be combined-- like a workshop on max flows, and them posters on the max flow papers that got in. The rebut/rewrite I am more ambivalent on but that may also be a good idea. These ideas are NOT radical (and not even original) and it is NOT my purpose to drain STOC of its prestige. Whether that is a good idea is another debate.
Reviews of accepted and rejected papers should be publicly available in the style of a voting-based discussion board like cstheory.stackexchange. Reviews can be made anonymously, good reviews get voted up, and bad reviews can be commented on. It is such a huge waste of effort that rejected papers get reviewed a bunch of times at different venues, and then again for the journal submission.
ReplyDeleteThis is indeed a huge waste of our community resources, see http://windowsontheory.org/2012/07/16/community-brain-2/ for some thoughts.
ReplyDeleteI don't personally like your idea of posters. Whether or not posters should be regarded as inferior to talks, the plain fact is that people usually treat them as if they are inferior to talks and that is enough to make them a bad idea. The organizers usually display the posters in overly cramped conditions that make it hard to see the popular posters or the ones next to them due to the crowds. Wine and/or refreshments are often served during poster sessions, which definitely shows a lack of respect for them. Would you consider it appropriate to show your respect for a speaker by getting drunk or stuffing your face in the middle of their talk? Also, given the availability of this stuff, people often treat poster sessions as a time to socialize and chat in general, rather than actually looking at the posters.
ReplyDeletePoster sessions are also incredibly tiring for the poster presenters, who have to stand up and explain their stuff many times over for several hours rather than just doing it once for twenty minutes. They are also tiring for the few non presenters who take them seriously who have to spend a long time on their feet negotiating the crowds to see all the interesting posters. They are also discriminatory, in that if you have a disability that makes it hard for you to stand for a long time and negotiate crowds then you cannot do a poster session either as a presenter or a poster viewer. They are also discriminatory because you can't really video a poster session like you can with a talk, so you can't really make them available to those researchers who cannot attend the conference, e.g. due to lack of funds, illness, or looking after kids. These discriminatory issues apply to me, so I am strongly against poster sessions on these grounds alone.
Universities have rules and regulations about discrimination that they have to follow and they generally have committees to sort out just these sort of problems. Conferences are independent entities and they generally do not have this sort of oversight. Most of them are a mess when it comes to issues of discrimination, because they are only ever managed by those researchers who have no trouble showing up and negotiating the conference in the first place.
The NIPS conference does Posters instead of talks--- this is NOT an argument for doing it, but it does indicate people we could ask about to see how that goes.
ReplyDeleteI agree that food should not be served during it.
If there were posters INSTEAD of talks there would be more time for them
and similar ones would be in the same place, same room.
That might help the tiredness issues since it wouldn't be hours.
The Discrimination issue is a good counterargument.
- I hope the second tier PC will be larger in size and more diverse in terms of areas of expertise. Otherwise, consulting with experts outside the PC should be encouraged. Last time one of my paper get ALL reviews that are clearly from non-expert reviewers (something like "I am ok with accepting this paper if an expert in this area would say so"). I have heard similar complains from many other people. And, please have 3 reviewers for every paper.
ReplyDelete- I think having one round of rebuttal is a good idea. I have experienced such a system in some previous conferences and was pretty happy with it (even when my papers got rejected).
- I fully support the poster session. I have previously enjoyed poster sessions at STOC, SIGMOD, and VLDB as both a presenter and audience. If some people still prefer to have talks, we can even have BOTH poster sessions and talks (SIGMOD did this last year and I liked it). Of course, this requires more work for the authors's side but isn't it worth it given that you have more opportunity to advertise your work?
My take on the ideas:
ReplyDeleteWorkshops - This has been done before (at least with FOCS), on the day before the conference. Not a bad idea, but I don't think that more than a day or two would work, as together with the conference it would get too long. In particular more invited talks could take place inside the workshop day(s).
Full papers in the proceedings - I'm for it. The only reason not to have them was the publishing expense, but now this can be electronic only.
Two tiered PC - A good idea, maybe even a necessity. There are just too many submissions.
Having three parallel sessions is not a good idea, people should have a chance to hear more of the talks. Going all-poster is IMHO a really bad idea, and a radical one. A poster is in my eyes informationally equivalent to a 5-minute lecture, I'll take the 20-minute lectures. Student and open problem posters are Ok (without a "published proceedings" status as some may eventually become conference submissions).
but I don't think that more than a day or two would work, as together with the conference it would get too long.
DeleteI don't get this comment at all. If the conference is too long for you just skip some of the workshops. ALGO in Europe lasts an entire week and most people attend only for a subset of those days. As simple as that.
There are a number of reasonable options for the STOC PC structure, the 2-tier one among them. I don't see any one structure by itself as a panacea. We still are not at the level of SODA's 500+ submissions or AAAI's 1000+ submissions so there is more flexibility. As Joan said, optimizing the PC itself is the lesser issue (though there is the radical VLDB "conference as fast journal" option also - see below).
ReplyDeleteFor the conference itself, 3 parallel sessions rather than 2 parallel sessions would not radically change the atmosphere, provided that the conference retains some plenary sessions. SODA has done this for a while: In New Orleans it did not have a significantly different feeling from STOC (maybe because the number of attendees was similar) and there was a good distribution of attendees. If venues can accommodate this then it would be vastly preferable to extending the time, since extended time runs into loss of stamina from attendees and costs more.
For many years it seems to me that the sweet spot wrt STOC/FOCS has been in the 25-30% acceptance range and when they go out of that range (particularly going to the low 20-22% range but also when the range went significantly above 30%) I have noticed more consternation.
Workshops have been a very positive thing for STOC and FOCS. The more that the conference activities can make it a destination than merely an anointing of papers, the better.
I am quite leery of posters as a solution to the paper crowding problem. It seems to me that having fewer papers presented makes it more a matter of a anointing a very select few for presentation. I would rather have it be the case that presented papers are ALSO posters so that people get a chance to find out about the talks they missed in parallel session.
One radical option that Joan did not mention is the VLDB-style "conference as fast journal" format is being picked up by several other conferences. It resolves the problem of PC overload in a short 8-10 week stretch by making it a full year occupation (more likely half-year for us). For VLDB there are monthly deadlines. The same PC operates all year and accepts papers as they come. A paper can be sent back for revisions (and returned only once during that year). This allows the PC to do more full refereeing. The papers are presented at the conference as usual, though they appear in the PVLDB journal when they are accepted - even before the conference. I can certainly see potential pitfalls - and it may be less good in an area like TCS with many thriving journals - but the idea is intriguing.
Chiming in as a NIPS regular.
ReplyDelete-- NIPS papers that are presented as posters have the same "prestige" as a conference talk paper in the machine learning community. This is probably a cultural thing. I suspect that if poster sessions became a regular event at STOC, then there would be a cultural shift.
-- Food/drink are a NECESSITY for poster sessions. There will always be a healthy amount of socializing at any poster session. It's unclear to what extent having food/drink will create MORE socializing that there would be otherwise. But it is clear that a lack of food/drink at a poster session leads to a rather miserable experience. NIPS 2011 in Granada had very limited amounts of food/drink at the poster sessions, which was widely complained about.
-- Regarding whether having food/drink is a sign of lack of respect for the poster session, this is the first time such a thought has crossed my mind (although perhaps this is because I'm used to poster sessions). It could very well be the case that, compared to STOCS, it seems like people treat NIPS poster sessions with a disdainful lack of respect. But I haven't been bothered by this either as a poster peruser or presenter at NIPS.
-- One distinct advantage of NIPS posters over standard conference talks is the personal interaction with the poster presenter. You can back up the presenter to clear up confusing points or speed up the presentation if you understand the motivation, etc. This can often be a very big plus.
-- Poster sessions are VERY tiring for the presenters. The good news is that most papers have multiple co-authors, who can take shifts. Even single authors can take sizable breaks during the poster session to peruse other posters. It's OK to leave your poster un-attended for 20-30 minutes and just have people stare at it.
Maybe it's fair to point out that NIPS is not quite all-poster (although it almost is). Besides keynotes and invited talks, a small selection of papers are presented in a single track session. For our community this will be the equivalent of presenting only the papers that are now invited to a special journal issue, and moving everything else to posters.
ReplyDeleteI agree that this is a strange model that I do not prefer. I do like a poster session for ongoing work/work presented in other theory venues, etc.
If we're really serious about comparing PC methods, we should email submitters to these conferences after the reviews have been returned and ask them to fill out surveys concerning their submission experience.
ReplyDelete+1 to Jelani's comment
ReplyDelete-- Amit C