On relevance of work and best-paper awards:
We now have interesting tools to evaluate the success of work in the long term. Things like Citeseer and Google Scholar suddenly give us a view that we could not have had before. One thing that we discovered from these tools is that we are actually very poor in predicting the long term impact of work. There is very little correlation, for example, between the best paper awards that we give and the test-of-time awards that we give. Sometimes, miraculously, we have the same paper win the best paper and the test-of-time awards. But that is the exception rather than the rule. So I think people should focus on just doing the best work they can.
- On highly selective conferences and low acceptance rates:
I think the low acceptance rate is a terrible problem. I think that the idea that we are raising the quality is nonsense. I think that actually the quality goes down. I think we are very good at selecting about one third to one fourth of the papers; we do a pretty good job there. As we become more selective, the program committee gets larger, the whole discussion gets more balkanized, and the whole process gets more random. Nobody gets a global view.…Conferences are not scalable. They work nicely with up to roughly 300 submissions and a certain size of program committee. When you try to scale it up, very good papers get lost. It becomes more political. I think we are being held back by our own success.
- On conference publication vs journal publication (Here I did some
serious cut and pasting—Alex):
We are very unique among all the sciences in how we go about publication. We have these selective conferences. (People have stopped calling them refereed conferences. They are not really refereed. You don't get good referee reports.) Our conferences worked well in the world we used to inhabit…I think we had a model that worked successfully for about 30 years, but now we see cracks in the foundations…I don't have a good solution to this problem. We don't even have a good forum in which to discuss the problem… We ought to rethink how we do it. Right now, people try to fix things incrementally by having a larger conference with a bigger program committee, a two-level PC, a three-level PC. Maybe we need to rethink the way we do scholarly communication in our discipline…How can computer science go about changing its publication culture? Are there areas that move just as fast as we do, and have journal papers and conferences, but conferences are not the primary vehicle? I have questions about the basic model of scholarly publications. And I find it fascinating that it is difficult to have a conversation about this on a big scale, and make changes on a big scale. We are very conservative. It is interesting that computer science has been one of the slowest disciplines to move to open access publications. Other disciplines are way ahead of us in using online publications.
Sunday, March 19, 2006
An Interview with Vardi
Alex Lopez-Ortiz points out that this month's SIGMOD Record has an interview with Moshe Vardi from Rice University that touches on several topics of interest to the readers of this weblog. Lopez-Ortiz picked out some highlights of Vardi's views.