With that in mind, I list out some criteria I have heard played a role in a hiring decision. My point is NOT to help you game the system (note that some of the factors I've heard cut both ways) nor to argue that the system is good, bad, or ugly. My point is only that the process is more arbitrary-random-complex then you might think. While its not all merit, its also not all who-you-know or politics.
I invite you to give leave comments on factors you have heard of, but to keep it civil please do not mention the people or schools involved.
- Merit: This is itself ambiguous. More papers? Longer papers? Co-authors? How about take a sum where each summand is (number of citations)*(importance of paper)*(number of pages)/(number of coauthors). And then there is grant potential.
- Does a postdoc have a better chance then a fresh PhD? A postdoc has had more time to increase his weighted sum mentioned under Merit, but the school KNOWS he has had more time.
- Two-body-problems can be GOOD or BAD. Often a school does not have two positions.
- Being a women can be GOOD or BAD (for getting a job).
- I would like to say Being black can be GOOD or BAD (for getting a job) however since there are so few black PhD's in computer scientists applying for academic jobs, I have not heard any stories about this. (NOTE- I use the term black instead of African American since they need not be American.)
- Being socially inept can be GOOD or BAD. How could it be good? It plays to the stereotype. He's so socially inept, he must be a genius. I DO NOT recommend pretending you are socially inept.
- Speaking your mind can be GOOD or BAD. He'll be a leader in the community or He'll be a pain in the ass
- Having a Blog- the jury is still out on this one. I have heard of a case where having a blog helped someone get an interview. (It was a serious blog about the field he was in. I can't imagine that complexity blog would help me get a job if I was looking.)
- Spending too much time deciding what to have for lunch can be a negative. If he can't figure out what to have for lunch then how can he formulate a coherent research plan.
- Area can be a factor- does a school need or want someone doing area X? This is sometimes formalized, for example the school has money targeted to hire someone who does, a particular area. (Side topic- Targeted positions- Good or Bad? Good in that you are not going to have to compare people in diff areas. Bad in that there may be someone really good in another area who you want to hire but can't.)
- Subarea is a factor- for example, we want the kind of theorist who can talk to our people in systems.
- Having a champion in the dept who is helping push your case HELPS unless the person doing the pushing is a jerk or not good at pushing a case.
- Being an ex-convict probably hurts. If there was a theory genius who served 10 years in jail, that might be a negative. Might be mitigated if he could pull in some serious grant money. However, if the jail term was for embezzling grant money, then maybe not. Actually, the whole question might depend on what he was in jail for and is he now reformed.
- A very big factor is how good is the job market when you get out. This may be a much bigger factor than anything else on this list.
Subarea is a factor- for example, we want the kind of theorist who can talk to our people in systems.
ReplyDeleteThis mixes two items. The latter does not refer to a subarea (though I guess that you might have meant "whose research is applicable to problems in systems"). However, I would like to promote the kind of phrasing you give to a factor on its own.
"Can carry on a knowledgeable conversation with faculty from other areas about problems in those areas."
(This contrasts with "can only talk about subjects in his/her research area".)
It is worth getting on the right side of this factor.
The only way it could possibly be BAD would be if the conversation gives too strong contrary opinions about what those other faculty should be doing in their research.
--------------------
I presume that your imagined notion of numerical "merit" is deliberately goofy in ignoring both the quality of the work and the importance of the problems being tackled.
---------------------------
Your question about Postdocs vs fresh PhDs misses a key aspect. The volume of work is not the key. A postdoc is an opportunity to show greater independence and versatility, to show that one isn't just a one-trick pony, and to set research goals without the guiding hand of one's PhD supervisor.
In what ways can being a woman hurt you in the job search?
ReplyDeleteLast anon: being a woman can hurt you, because if you have any accomplishments, people can attribute it to the "affirmative action" that you got you this far. For example, if you did a good Phd Thesis from a good school, they may assume that you got into the school because you are a woman and that you got your hand held more because you are a woman. Thus, they may discount some of your achievements.
ReplyDeleteIn Europe, where people are more open about what they think and less PC, people on hiring committees often state that to hire a woman, she should be *better* than the competing man, because, after all, she had much more help along the way.
Does a postdoc have a better chance then a fresh PhD? A postdoc has had more time to increase his weighted sum mentioned under Merit, but the school KNOWS he has had more time.
ReplyDeleteI disagree that the second observation should be a factor (although maybe it is). A postdoc isn't just a chance to prove yourself, it is a chance to improve yourself. A person after a postdoc is likely a better researcher than they were before the postdoc. Therefore the expected next-6-year research quality/output level/grant money/etc are all higher for postdoc than a fresh graduate having the same "normalized research output per year".
I would agree with Paul Beame. I think it is quite useful for a CS theorist to be able to talk with people in other areas and also think about the real world, and be able to mathematically abstract out some of its elements, in order to say something fundamental.
ReplyDeleteIn Europe, where people are more open about what they think and less PC, people on hiring committees often state that to hire a woman, she should be *better* than the competing man, because, after all, she had much more help along the way.
ReplyDeleteDo they give specific examples of this extra help? Off the top of my head, I can imagine that maybe a graduate school accepts more women than it would otherwise due to such a program, and perhaps there are women-specific fellowships that she can apply for. But these only indirectly help someone get a job. The direct thing that gets you a job is good research, which is reflected in papers accepted, and there is no affirmative action program there.
As far as the indirect help, I doubt that many faculty committees take into account "extra help from funding", for instance. If they did, I would have a huge advantage over everyone who had RA's since my advisor never had money and I was a TA all through grad school. But they don't account for this, they compare my research output directly to that of someone with an RA. It follows that they should do the same with a woman who has a woman-specific fellowship and say, "Who cares where her money came from, the important question is what did she produce with it?"
I think it is a mistake to assume that sexism in Europe is the same as in the US. The cultures are very different.
ReplyDeleteHow do you evaluate a research output of a person, by "sum" or by "max"?
ReplyDeleteIf it is "max", then the resources used have decreasing return. I would believe one could prove a formal theorem around it.
If you eveluate the research output by "sum", then the resources used must also be accounted for
IMHO, the society interest is aligned with scientists optimizing for their "max" and not for "sum". The society would not tradeoff one Einstien to get 100 non-einstiens.
The purpose of a hiring committee should be to predict the "max" of a new hire in the next 5 or 10 years (besides other aspects such as teaching etc).
Kamal, I think that most researchers' max outputs are very similar, and so it should be the sum that is important. Only when there is a large gap in the max outputs, as in your Einstein example, then max is important. There is really a tradeoff.
ReplyDelete"max" output by definition can't be similar. They could be hard to compare or even uncomparable.
ReplyDeleteIf the relevant quality parameter is hard to compare or uncomparable, it does not mean you start comparing irrelevant parameters such as "sum". It is like, a Program Committee of a reputable conference say, "oh, the papers are really hard to compare or even uncomparable so we are going by the number of pages to decide acceptance/rejection".
One significant difference between good department and great department (or good PC and great PC) is exactly this. Good PCs sometime end up comparing the length or complexity of the proof, instead of the significance of the theorems, ideas, or approaches in the paper.
Should somebody like Perelman get a job along this system? I doubt ... (Perelman is just an "exotic" example.)
ReplyDelete"Max" or "sum" - where is a question? I think in counting. Counting papers, "good", "better", "excellent" papers. Not the potential.
B.t.w in Germany the situation is even worse. To get a job here you must "fit" in the current research done by the full profs. No "Perelmans" or the like are here welcome. Nor other "strangers". Nor those whose "max" or "sum" is larger than those of current profs.
The whole problem, I think, is in "irrevocable decisions". Zero error decisions are hard ... But what if profs (as other employees) should "renew" their positions every 5-7 years? Then, perhaps, also Perelman would have a chance ...
Should somebody like Perelman get a job along this system? I doubt ...
ReplyDeleteYears ago (before he proved Poincare), Perelman was offered jobs at many of the top US universities, and he turned them all down to return to his mother's apartment in Russia.
Perelman is many interesting things, but he's not an example of a "genius abused and maltreated by the system." The system keeps trying to give him stuff, and he keeps saying no!
In my experience (both as a faculty candidate and from serving on hiring committees), the most important factors are (1) the letters of recommendation and (2) having a strong supporter on the inside. More succinctly, who and what you know isn't as important as who knows you.
ReplyDeleteThe best letters are from experts in your narrow subsubfield, who are recognized as experts even outside your field, who know your work personally, but who are NOT your advisor, your co-author, or faculty in your home department. Good letters display intimate familiarity with technical details of your work, a broad perspective on the impact and importance of that work, and direct comparisons with other researchers at the same stage of their careers.
The best insider pull comes from people who could write a strong letter, although being an expert in your subfield isn't quite as important. Successfully advocating for a faculty candidate requires a lot of behind-the-scenes legwork. If nobody is willing (or able) to do that legwork on your behalf, you won't get the job.
Internal department politics also plays a very strong role, but that's completely out of your control, and therefore best ignored. Accept as axiomatic that you will be passed over for utterly stupid reasons, and get on with your life.
I've seen faculty candidates sabotage their own interviews in lots of interesting ways, but never by being indecisive over lunch. (Really, Bill? Really?) Normally, we figure out whether candidates can formulate a coherent research plan by asking them to describe their research plans.
I concur with Paul: Hiring committees like candidates who can talk coherently (if not productively) with a broad spectrum of the department.
Kamal: Definitely "sum" not "max" but of a function of reputation and impact (in that order), not number of pages or difficulty of proofs. One short readable paper may contribute more to your total impact than a deluge of longer, more technical results. Beyond a certain threshold, having a long CV is actually suspicious.
The postdoc/fresh-PhD question is interesting. One of the questions that people ask when interviewing systems candidates is whether their expertise is narrowly focused on their advisor's research project, or if they can formulate an independent research plan. Making this judgment from the candidates' statements and letters seems almost impossible, even for the top on-paper candidates, but it's usually easy to tell within the first hour of an interview. For theory candidates, on the other hand, this question never even comes up; the top theory candidates have already established themselves as independent experts, even as PhD students. Despite this apparent disparity, postdocs are much more common in theory than in systems.
This comment has been removed by the author.
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteStupid Blogger errors...
ReplyDeleteBeyond a certain threshold, having a long CV is actually suspicious.
ReplyDeleteNot that I am in any "danger" of exceeding this threshold, but could you elaborate on this point? I found your comment helpful, but this sentence surprised me.
Re: JeffE
ReplyDeleteI've asked people who were very familiar with my work (e.g. they wrote follow-up papers) for letters, since I had reason to believe that they thought highly of some part of it. However, the response I've always gotten is that they don't want to write the letters because we've never worked together. So I can't imagine actually getting a letter from someone who is not a co-author unless I solve some absolutely huge problem. But I guess that is what it takes to get hired these days.
What about spouses and children? I interviewed with a department where almost every faculty member (starting with the one on the drive from the hotel to the campus) seemed to ask me about my spouse (didn't have one) and plans for children (hadn't thought about it) and dinner conversation included much talk about the local public schools and parks and family-friendly places even though most of the people at the table had asked the previous questions and been given the stated answers. Clearly they wanted someone with a family or at least plans for a family to join the department.
ReplyDelete"Spending too much time deciding what to have for lunch can be a negative."
ReplyDeleteWhat about being asked where you want to go for dinner, telling them, and then the lead person hosting you saying where they want to take you even though you dislike that type of food and when you tell them that they say "oh, we'll find something you like, it's the best restaurant around" and then others agreeing that dinner should be there since it's an expensive place and the department is picking up the bill.
In short, the people in the department and their personalities and personal biases are probably far more an issue once you get an interview than anything else. If you weren't good on paper they would not bother to invite you. Now they want to know if they want to live with you for the next 40 years.
AnonNine:
ReplyDeleteWe ask about your family plans, hobbies, interests, etc. on recruiting visits not because they are factors in whether we'll give YOU a job, but are factors in whether YOU want to live WHERE WE ARE. If you have a spouse, we are interested in knowing that person's career requirements, so we can start looking into possible positions for him or her. If you have kids, you'll eventually want to know where the good school districts are, and how much it will influence housing costs.
It's true that universities don't want to waste offers on people unlikely to accept them, but it's also true that the departments are trying to look their most appealing to the candidates, not just evaluating the candidates. An interview is not a one-way street; the candidate is also evaluating us.
Russell Impagliazzo
AnonNine:
ReplyDeleteI hadn't read your posting carefully before I posted. It seems your point was that they continued to raise this kind of personal question even after you brushed it aside earlier. My guess is that they were desperately groping for a conversation topic, and failing to find one. That indicates to me that they had already failed to connect with you about technical matters. That might be why the interview was unsuccessful (if it was). I really don't think that anyone would care that you aren't planning to raise a family.
Anonymous responding to JeffE:
ReplyDeleteLetters from people fluent with and enthusiastic about your work who have no official connection to you ARE hard to get. But that is exactly why they are valuable!
Russell again
I think it quite strange (anti-social ?) to ask someone repeatedly about a spouse, kids, etc. First, it is illegal to ask about kids, etc. Second, if the person wants to volunteer that they have a spouse, they most likely will, especially if their spouse would need help relocating, etc. On the other hand, if the person is not married, they may feel embarrassed about it and not want to make a big deal about it.
ReplyDeleteRe: Russell
ReplyDeleteThat being said, many of the top candidates in years past have had letters only from people whom they've worked with or bin in the same department with. And often the advisor's letter was the strongest, deciding letter.
No need to scare everyone.
having a long CV is actually suspicious.
ReplyDeleteNever heard or seen this one. Frankly it sounds like what a mediocre department would say just to justify not hiring a person who would clearly outclass them.
Also I never seen number of authors come up as an issue. What I have seen is people who always seem to be the third wheel in a whole bunch of papers. However if you have published a series of co-authored papers on Quantum Belt Pocket calculators (2 and 4 bit models) but you are the only constant across them people will assume you are the driving force in your research and that you managed to assemble teams around you to effectively solve problems. This is generally considered a plus and if it isn't you don't want to work there.
Russell, I think you might be being a bit idealistic. They started the dinner conversation on family and stayed there. They did not want to talk about my teaching experiences or theirs, or my research (that I assumed was because we had discussed it quite a bit during the Q&A after my talk which went long with good questions) or theirs. I was briefly able to move the conversation to recreation interests of mine since they seemed to want to get to know the non-academic side of me, but they then moved that to talk of family-friendly parks and locations and stories about their children.
ReplyDeletemaybe once a place decides they don't want you they want to make it so that you don't want them either
ReplyDelete>In Europe, where people are more open about what they think and less PC, people on hiring committees often state that to hire a woman, she should be *better* than the competing man, because, after all, she had much more help along the way.
ReplyDeleteIt would be rather strange (and may I say wrong! and I am rally surprised!) that affirmative action for increasing the involvement of women in computer science would work against its objective. No amount of help (money/support/...) can make one a good researcher, so it is completely unfair in my opinion to downgrade the achievements of women by saying that they have benefited from affirmative action. Affirmative action can reduce the outside and cultural pressure, but they cannot improve a person's research considerably if she is not capable of carry the quality research herself. Should we also downgrade the achievements of researchers in US or those whose family are better off economically by the same argument?
GASARCH says: A very big factor is how good is the job market when you get out. This may be a much bigger factor than anything else on this list.
ReplyDeleteThat last point is IMHO very strong, because isn't it a corollary of the pigeon-hole theorem?
The point is that per-capita STEM undergraduate enrollment in North America has been steadily declining since the mid-1970s. This long-term trend both reflects and reinforces a declining technology sector of the economy, which in turn creates an increasingly tough academic job market in STEM overall.
Although undeniably important in individual cases, the first thirteen of GASARCH's career-related factors are local perturbations on STEM trends that are broadly adverse.
Isn't it the case, that unless and until the broad overall STEM decline is reversed, no amount of well-intended career advice (from any person or from any agency) can exert a net beneficial effect?
STEM enrollment may be declining but our CS enrollments are at record levels.
ReplyDelete"STEM enrollment may be declining but our CS enrollments are at record levels"
ReplyDeleteLOL ... still shorter is the British idiom "I've got mine, Jack!" :)
Responses belonging to the class of "I've got mine, Jack!" aren't intended to be taken seriously ... or are they?
I think there is a strong halo effect from the candidate's school, which is stronger than even a strong advisor from a lower-ranked school. If you want a job in academia, you have to be sure there are n+1 positions open, where n is the number of people currently graduating from Carnegie Mellon.
ReplyDeleteSubarea is a factor- for example, we want the kind of theorist who can talk to our people in systems.
ReplyDeleteIt may just be me, but why don't we ever hear the converse? "Will this systems candidate be able to talk to our people in theory?". Granted, theory people are fewer in number, but why is it always the expectation that theorists work with applied persons but never the other way around?
Bill: "having a long CV is actually suspicious."
ReplyDeleteAnon X: "Never heard or seen this one. Frankly it sounds like what a mediocre department would say just to justify not hiring a person who would clearly outclass them."
Actually, there are (at least) two cases here: (1) CVs that are overly long because they contain every lunch talk the person has ever given, etc, and (2) CVs that are strong but raise the question why the person would be interested or would even be on the market.
I have mainly run across (1) as an issue. When you are senior and established, there should be no reason to list all talks ever given, or all journals or conferences that you ever reviewed for, and all sessions you have chaired. This can send a message that the person is a little pompous and maybe not easy to deal with, and is also often associated with larger numbers of low quality pubs.
Concerning (2), yes, this can be a sign of insecurity, or based on experience. Or of mediocrity of the hiring department, but less likely IMO.
CVs that are overly long because they contain every lunch talk the person has ever given
ReplyDeleteBut in this case the problem is not a long CV, but a stuffed CV.
Yes, senior people should judiciously trim their CV, summarizing older minor accomplishments, e.g:
- University Professor (also recipient of five other intra-university awards)
Rather than
-University Professor
-Minor chair
-Best teaching award
-Junior hire of the year
-Starter grant supplement for distinguished candidates
Bill: "having a long CV is actually suspicious."
ReplyDeleteI've seen candidates publishing more than 15 papers a year and then a question was asked whether these 15+ papers are good or not. The feeling (for these candidates, of course there are exceptions where this is not true) was that majority of the papers were weak and then you question why the candidate is publishing weak papers rather than focusing on big research. Also, in these cases, it was always difficult to judge the quality of the research record, because if you pick a random paper out of 100+ publications, then the paper usually doesn't look very strong.
This is perhaps similar to what some other comments were saying about sum vs max.
In Europe, where people are more open about what they think and less PC, people on hiring committees often state that to hire a woman, she should be *better* than the competing man, because, after all, she had much more help along the way.
ReplyDeleteI was in several hiring committees in Europe (and also in the US) and never heard anything like that. And so I wouldn't generalize it.
But I'm quite sure such comments are flying around in some hiring committees in Europe, US, and other places. I just go hope this is very rare.
It may just be me, but why don't we ever hear the converse? "Will this systems candidate be able to talk to our people in theory?".
ReplyDeleteWhile you may not here this as often, I think it is true that a systems person who can speak with people in theory will (all other things being equal) have a better shot on the market.
I've seen candidates publishing more than 15 papers a year and then a question was asked whether these 15+ papers are good or not.
ReplyDeleteBut surely you ask the same question from candidates with 8 papers or 5 papers or 2 papers.
The feeling (for these candidates, of course there are exceptions where this is not true) was that majority of the papers were weak and then you question why the candidate is publishing weak papers rather than focusing on big research.
Feelings that I surely hope you also have when you have a five paper candidate none of which are relevant.
So where exactly does the long CV comes in as a negative, rather than the fact that the person does not have significant contributions?
Also, in these cases, it was always difficult to judge the quality of the research record, because if you pick a random paper out of 100+ publications, then the paper usually doesn't look very strong.
Is it really that hard to type author:candidates_name in Google scholar and look up the top five cited papers from the candidate? Do the letters of recommendation not highlight their most significant contribution?
Sounds like you guys are rationalizing a case of paper envy, rather than following sound academic decisions.
I know researchers with 15+ papers a year and weak research records and I know researchers with 15+ papers a year and outstanding research contributions, just like I know researchers with 8 papers and crappy records, and researchers with 8 papers a year and stellar records.
I think being a woman can definitely hurt. Women are more likely to need time off for family. If the family needs to move, a woman may be more likely to give up a position than a man would (because she is probably paid less than he is).
ReplyDeleteAnyway, I think the whole point of this is that for -any- factor, a hiring committee can decide to use it against you, no matter how irrelevant or positive it seems like it would be. I'm in the humanities and it is the same way. At the end of the day, you just have to be a good "fit" for the department and if they decide that you won't fit in with them or that you will just use them as a stepping stone to something "better" then you probably won't get hired, no matter how shiny you are. Yes, it is somewhat arbitrary. However, merit probably went a long way to getting you the interview.
I think the point is not that these are the RIGHT factors, it is that these are REAL factors that are used in making hiring decisions everyday whether or not they are right or wrong.
ReplyDeleteAnd in most cases there is absolutely nothing you can do about it.
There are some interesting political issues that can either work for or against you. For instance, I am a Canadian, and after I published a paper in a well-regarded international publication, for two years I was short-listed for every position I applied for outside of Canada, but have never been short-listed within Canada. A close friend of mine recently suggested that because my Ph.D. supervisor has a reputation for rejecting NSERC grant applications. his recent grad students pay a price. I am working in industry now.
ReplyDeleteI've seen candidates publishing more than 15 papers a year and then a question was asked whether these 15+ papers are good or not.
ReplyDelete...because more often than not, the answer is no. Most of the uncharacteristically long CVs I've seen in hiring committee meetings are chock full of crap. Or they're good in the aggregate, but appear to advance their research in too many incremental steps. Even if there are diamonds in the slurry, I have to ask—why did they publish all that other stuff? Why didn't they make one big splash instead of this long, meager dribble? Did they not notice that most of their papers were weak? Are they publishing so many papers to advance the state of the art, or only to make their CV longer? Are they going to value quantity over quality in their own PhD students? Or am I in fact witnessing a miracle candidate?
Despite what Dr. Anonymous two steps above me suggests, these are not irrelevant issues. I don't want my department associated with someone known to publish reams of crap, or who wastes my colleagues' time reviewing tons of incremental papers, or who thinks they're Thor's gift to computer science when they're not, or who tries to convince PhD students that their CVs need to look like the Manhattan Yellow Pages. Those aren't the people I want to work with; that's not the culture I want to work in; that's not (in my opinion) what's best for the Advancement of Knowledge™.
And notice I said “suspicious”, not “deadly”. Of course we read recommendation letters and search citation indices and even (gasp) read the actual papers. Sometimes that's enough to allay my suspicions; yes, there are miracle candidates. But just often enough for me to keep my suspicions, they're proven right.
So where exactly does the long CV comes in as a negative, rather than the fact that the person does not have significant contributions?
The problem isn't just that they may not have significant contributions, but also that they may have too many not-significant contributions. (See my earlier comment about "max" vs "sum".)
Sounds like you guys are rationalizing a case of paper envy, rather than following sound academic decisions.
What the hell is “paper envy”? Result envy or impact envy or reputation envy I could get behind, but paper envy? Really? Since when are papers something to be envious about?
Also: Not publicly taking credit for your opinions? Deeply suspicious.
Just wondering, what if someone publishes lots of papers of diverse quality, but only puts her top 10 on her CV?
ReplyDeleteJust wondering, what if someone publishes lots of papers of diverse quality, but only puts her top 10 on her CV?
ReplyDeleteit depends on the school and the length of time in the research world
there are departments that will drop you right in the 'no' stack if you don't have a high enough papers:years ratio for the position according to their own metric
Just wondering, what if someone publishes lots of papers of diverse quality, but only puts her top 10 on her CV?
ReplyDeletewill they hit Google Scholar and try to figure out why you are embarrassed to list the other papers
10 was an example. The question I was trying to ask is, if someone has enough good papers but also many papers which are of lower quality, and just lists those top ones as her main product of his research, would you still consider having many papers of lower quality as a negative?
ReplyDeletea question was asked whether these 15+ papers are good or not.
ReplyDelete...because more often than not, the answer is no.
Have a look at the list of most prolific theory authors in DBLP with ~15 papers a year
-Micha Sharir
-Noga Alon
-Moshe Vardi
-Joe Halpern
-Christos Papadimitrou
-Paul Spirakis
-Erik Demaine
-David Eppstein
-Leo Guibas
-Kurt Mehlhorn
-Oscar Ibarra
-Oded Goldreich
-Lane Hemaspaandra
-Pankaj Agarwal
-S. Muthukrishnan
-Avi Widgerson
I'll let the readers decide.
Even if there are diamonds in the slurry, I have to ask—why did they publish all that other stuff? Why didn't they make one big splash instead of this long, meager dribble? Did they not notice that most of their papers were weak? Are they publishing so many papers to advance the state of the art, or only to make their CV longer? Are they going to value quantity over quality in their own PhD students? Or am I in fact witnessing a miracle candidate?
Despite what Dr. Anonymous two steps above me suggests, these are not irrelevant issues. I don't want my department associated with someone known to publish reams of crap, or who wastes my colleagues' time reviewing tons of incremental papers, or who thinks they're Thor's gift to computer science when they're not, or who tries to convince PhD students that their CVs need to look like the Manhattan Yellow Pages. Those aren't the people I want to work with; that's not the culture I want to work in; that's not (in my opinion) what's best for the Advancement of Knowledge™.
The problem isn't just that they may not have significant contributions, but also that they may have too many not-significant contributions. (See my earlier comment about "max" vs "sum".)
This is circular reasoning: I don't like candidates that publish a lot because they publish a lot. Also their students tend to publish a lot and did I mention that I don't like candidates that publish a lot? Hence I don't like candidates that publish a lot as they poison the environment. QED.
Absent there are actual reasons why someone with substantial contributions plus a bunch of smaller papers is bad for science.
Facts are that different researchers have different styles. Some people publish three or four paper a year each of which is a gem while others, like those listed above are fountains of insight and ideas, big and small. There is simply no shortcut in evaluating candidates by length of CV.
It also strikes me that I've heard all of these criticisms before, in a slightly different context. The person being criticized for publishing too much "irrelevant" work was one P. Erdos.
a question was asked whether these 15+ papers are good or not.
ReplyDelete...because more often than not, the answer is no.
Have a look at the list of most prolific theory authors in DBLP with ~15 papers a year
-Micha Sharir
-Noga Alon
-Moshe Vardi
-Joe Halpern
-Christos Papadimitrou
-Paul Spirakis
-Erik Demaine
-David Eppstein
-Leo Guibas
-Kurt Mehlhorn
-Oscar Ibarra
-Oded Goldreich
-Lane Hemaspaandra
-Pankaj Agarwal
-S. Muthukrishnan
-Avi Widgerson
I'll let the readers decide.
Even if there are diamonds in the slurry, I have to ask—why did they publish all that other stuff? Why didn't they make one big splash instead of this long, meager dribble? Did they not notice that most of their papers were weak? Are they publishing so many papers to advance the state of the art, or only to make their CV longer? Are they going to value quantity over quality in their own PhD students? Or am I in fact witnessing a miracle candidate?
Despite what Dr. Anonymous two steps above me suggests, these are not irrelevant issues. I don't want my department associated with someone known to publish reams of crap, or who wastes my colleagues' time reviewing tons of incremental papers, or who thinks they're Thor's gift to computer science when they're not, or who tries to convince PhD students that their CVs need to look like the Manhattan Yellow Pages. Those aren't the people I want to work with; that's not the culture I want to work in; that's not (in my opinion) what's best for the Advancement of Knowledge™.
The problem isn't just that they may not have significant contributions, but also that they may have too many not-significant contributions. (See my earlier comment about "max" vs "sum".)
This is circular reasoning: I don't like candidates that publish a lot because they publish a lot. Also their students tend to publish a lot and did I mention that I don't like candidates that publish a lot? Hence I don't like candidates that publish a lot as they poison the environment. QED.
Absent there are actual reasons why someone with substantial contributions plus a bunch of smaller papers is bad for science.
Facts are that different researchers have different styles. Some people publish three or four paper a year each of which is a gem while others, like those listed above are fountains of insight and ideas, big and small. There is simply no shortcut in evaluating candidates by length of CV.
It also strikes me that I've heard all of these criticisms before, in a slightly different context. The person being criticized for publishing too much "irrelevant" work was one P. Erdos.
a question was asked whether these 15+ papers are good or not.
ReplyDelete...because more often than not, the answer is no.
Have a look at the list of most prolific theory authors in DBLP with ~15 papers a year
-Micha Sharir
-Noga Alon
-Moshe Vardi
-Joe Halpern
-Christos Papadimitrou
-Paul Spirakis
-Erik Demaine
-David Eppstein
-Leo Guibas
-Kurt Mehlhorn
-Oscar Ibarra
-Oded Goldreich
-Lane Hemaspaandra
-Pankaj Agarwal
-S. Muthukrishnan
-Avi Widgerson
I'll let the readers decide.
Even if there are diamonds in the slurry, I have to ask—why did they publish all that other stuff? Why didn't they make one big splash instead of this long, meager dribble? Did they not notice that most of their papers were weak? Are they publishing so many papers to advance the state of the art, or only to make their CV longer? Are they going to value quantity over quality in their own PhD students? Or am I in fact witnessing a miracle candidate?
Despite what Dr. Anonymous two steps above me suggests, these are not irrelevant issues. I don't want my department associated with someone known to publish reams of crap, or who wastes my colleagues' time reviewing tons of incremental papers, or who thinks they're Thor's gift to computer science when they're not, or who tries to convince PhD students that their CVs need to look like the Manhattan Yellow Pages. Those aren't the people I want to work with; that's not the culture I want to work in; that's not (in my opinion) what's best for the Advancement of Knowledge™.
The problem isn't just that they may not have significant contributions, but also that they may have too many not-significant contributions. (See my earlier comment about "max" vs "sum".)
This is circular reasoning: I don't like candidates that publish a lot because they publish a lot. Also their students tend to publish a lot and did I mention that I don't like candidates that publish a lot? Hence I don't like candidates that publish a lot as they poison the environment. QED.
Absent there are actual reasons why someone with substantial contributions plus a bunch of smaller papers is bad for science.
Different researchers have different styles. Some people publish three or four paper a year each of which is a gem while others, like those listed above, are fountains of insight and ideas, big and small. There is simply no shortcut in evaluating candidates by length of CV.
It also strikes me that I've heard all of these criticisms before, in a slightly different context. The person being criticized for publishing too much "irrelevant" work was one P. Erdos.
a question was asked whether these 15+ papers are good or not.
ReplyDelete...because more often than not, the answer is no.
Have a look at the list of most prolific theory authors in DBLP with ~15 papers a year
-Micha Sharir
-Noga Alon
-Moshe Vardi
-Joe Halpern
-Christos Papadimitrou
-Paul Spirakis
-Erik Demaine
-David Eppstein
-Leo Guibas
-Kurt Mehlhorn
-Oscar Ibarra
-Oded Goldreich
-Lane Hemaspaandra
-Pankaj Agarwal
-S. Muthukrishnan
-Avi Widgerson
I'll let the readers decide.
There is a bit of circular reasoning in your comments: I don't like candidates that publish a lot because they publish a lot. Also their students tend to publish a lot and did I mention that I don't like candidates that publish a lot? Hence I don't like candidates that publish a lot as they poison the environment. QED.
Some people publish three or four paper a year each of which is a gem while others, like those listed above, are fountains of insight and ideas, big and small. There is simply no shortcut in evaluating candidates by length of CV.
It also strikes me that I've heard all of these criticisms before, in a slightly different context. The person being criticized for publishing too much "irrelevant" work was one P. Erdos.
That just supports JeffE's comments. There is a *huge* variance among
ReplyDeletethose you listed. If you want to be quantitative, here is the same list
with total citations, top three cited papers (books and survey articles
excluded) in MathSciNet. The number of citations to the third most cited
paper kind of gives it away.
2141 | 165, 62, 61 -Christos Papadimitrou
3973 | 89, 82, 67 -Noga Alon
1278 | 88, 71, 60 -Oded Goldreich
1061 | 73, 67, 60 -Avi Wigderson
786 | 51, 51, 40 -Leo Guibas
1465 | 51, 40, 39 -Micha Sharir
629 | 109, 31, 30 -Moshe Vardi
1052 | 62, 33, 23 -Pankaj Agarwal
236 | 27, 25, 22 -Paul Spirakis
346 | 32, 29, 19 -Joe Halpern
632 | 35, 21, 19 -David Eppstein
622 | 24, 19, 15 -Kurt Mehlhorn
371 | 24, 17, 15 -Lane Hemaspaandra
244 | 16, 14, 13 -S. Muthukrishnan
339 | 15, 14, 12 -Erik Demaine
270 | 44, 25, 9 -Oscar Ibarra
Why is that the right metric? Why don't you use
ReplyDeletethe number of STOC/FOCS papers instead?
82 -Avi Wigderson
52 -Christos Papadimitrou
50 -Noga Alon
47 -Oded Goldreich
25 -Micha Sharir
20 -Moshe Vardi
19 -Leo Guibas
15 - Joe Halpern
14 -S. Muthukrishnan
11 -David Eppstein
8 -Paul Spirakis
8 -Oscar Ibarra
7 -Pankaj Agarwal
6 -Kurt Mehlhorn
6 -Erik Demaine
1 -Lane Hemaspaandra
total citations, top three cited papers (books and survey articles
ReplyDeleteexcluded) in MathSciNet.
If you are going to count citations you should use google scholar as MathSciNet is not very accurate when it comes to computer science. As it is the stats posted are pretty much useless.
You also need to adjust for age, as older papers are more cited than younger papers.
ISI Web of Knowledge,
ReplyDelete345, 268, 267 -Oded Goldreich
259, 214, 200 -Joe Halpern
523, 220, 195 -Christos Papadimitrou
354, 312, 187 -Leo Guibas
187, 167, 160 -Micha Sharir
235, 167, 150 -Noga Alon
240, 165, 133 -Oscar Ibarra
222, 221, 126 -Moshe Vardi
97, 93, 93 -Kurt Mehlhorn
270, 194, 86 -Avi Wigderson
64, 60, 56 -Pankaj Agarwal
196, 65, 55 -David Eppstein
56, 47, 39 -Paul Spirakis
50, 41, 39 -S. Muthukrishnan
28, 28, 28 -Erik Demaine
31, 24, 20 -Lane Hemaspaandra
> If you are going to count citations > you should use google scholar as
ReplyDelete> MathSciNet is not very accurate
> when it comes to computer science.
> As it is the stats posted are
> pretty much useless.
Google scholar would be useless as well, there is too much junk, self-citations, etc.
The fact that Mathscinet vs STOC/FOCS are pretty correlated wheras Mathscinet vs ISI is not correlated is surprising, why is that? Probably ISI collects citations from more junky papers.
Google scholar should give a page-rank style (with customizable formulas) ranking procedure.
ISI has such bad coverage of conferences that it couldn't be used to rank departments, let alone professors. Here's the same list, first by H-index and then by age weighted H-index:
ReplyDeleteRaw H-index:
Papadimitriou 88 61
Vardi 70 36
Sharir 67 29
Goldreich 66 39
Alon 61 39
Halpern 56 36
Wigderson 55 31
Muthukrishnan 52 35
Mehlhorn 51 27
Agarwal 49 27
Eppstein 46 25
Demaine 37 27
Ibarra 36 18
Spirakis 27 21
Hemaspaandra 24 15
Age weighted H-index:
Papadimitriou 88 61
Goldreich 66 39
Alon 61 39
Halpern 56 36
Vardi 70 36
Muthukrishnan 52 35
Wigderson 55 31
Sharir 67 29
Demaine 37 27
Mehlhorn 51 27
Agarwal 49 27
Eppstein 46 25
Spirakis 27 21
Ibarra 36 18
Hemaspaandra 24 15
Lastly, I don't see how any of the statistics given, even the flawed ones from MathSciNet and ISI, in any way supports Jeffe's comment.
His comment was that work from 15+ papers-a-year authors was "more often than not" not very good. The list above flat out contradicts that statement. The work of all or at least most of those above is very good, even if of varying quality.
On the one hand, I agree it is unfair to punish somebody just because they have a lot of papers, including (many but not all) weak papers. And your list is a good example why this is not necessarily the case.
ReplyDeleteBut the discussion here was about junior people looking for a job. And this is the key point you are missing, and why Jeffe's argument has a lot of truth to it. A senior person has lot of ideas, and collaborates a lot with junior people. Often, junior people seek such collaboration (e.g., students but also others faculty if the senior person well known). At the end, though, the junior people do most of the "boring" work - writing the result, polishing the paper, etc. It is not necessarily wrong, the senior person usually worked hard to achieve these "perks" - and also often has to do other tedious work, such as grant writing, to justify the fact that he/she only does the "fun" part of research.
The point being that a senior person - especially famous - can easily write 15+ very strong papers a year, by collaborating with willing junior people on the "fun" part, and letting them to do the rest. On the other hand, a junior person DOES NOT HAVE TIME to do a good job of being a major writer of so many good papers. So it is highly suspicious if a junior person has time to write so many good papers a year. This supports Jeffe's claim.
Still, an interesting question remains. Assume a junior person has enough high quality papers, but also padded the CV with a lot of "junk". Is this bad? The answer can go both ways, but is usually "yes". The point is that the junior person "wastes" his/her time on weak research, taking away from what he/she could have done. Even though he/she has enough good papers, it raises the question WHY not to have EVEN MORE. At best, the junk did not take too much time (e.g., collaboration with another weak junior student who did all the work on the junk paper). Still, it took some time, so why bother? There is no advantage for junk, only question marks, so why risk it?
In other words, if you are good, there is no upside to produce junk and pad your CV. If you are average, producing junk can fool some people, or can alienate some people - usually the latter (especially if applying to a "serious" place). If you are weak, padding the CV is your only chance to fool somebody. Given this, Jeffe's claim is correct: modulo some extremely rare "gems" (who are easy to recognize anyway by other factors), long CV for a JUNIOR person is highly suspicious.
On the one hand, I agree it is unfair to punish somebody just because they have a lot of papers, including (many but not all) weak papers. And your list is a good example why this is not necessarily the case.
ReplyDeleteBut the discussion here was about junior people looking for a job. And this is the key point you are missing, and why Jeffe's argument has a lot of truth to it. A senior person has lot of ideas, and collaborates a lot with junior people. Often, junior people seek such collaboration (e.g., students but also others faculty if the senior person well known). At the end, though, the junior people do most of the "boring" work - writing the result, polishing the paper, etc. It is not necessarily wrong, the senior person usually worked hard to achieve these "perks" - and also often has to do other tedious work, such as grant writing, to justify the fact that he/she only does the "fun" part of research.
The point being that a senior person - especially famous - can easily write 15+ very strong papers a year, by collaborating with willing junior people on the "fun" part, and letting them to do the rest. On the other hand, a junior person DOES NOT HAVE TIME to do a good job of being a major writer of so many good papers. So it is highly suspicious if a junior person has time to write so many good papers a year. This supports Jeffe's claim.
Still, an interesting question remains. Assume a junior person has enough high quality papers, but also padded the CV with a lot of "junk". Is this bad? The answer can go both ways, but is usually "yes". The point is that the junior person "wastes" his/her time on weak research, taking away from what he/she could have done. Even though he/she has enough good papers, it raises the question WHY not to have EVEN MORE. At best, the junk did not take too much time (e.g., collaboration with another weak junior student who did all the work on the junk paper). Still, it took some time, so why bother? There is no advantage for junk, only question marks, so why risk it?
In other words, if you are good, there is no upside to produce junk and pad your CV. If you are average, producing junk can fool some people, or can alienate some people - usually the latter (especially if applying to a "serious" place). If you are weak, padding the CV is your only chance to fool somebody. Given this, Jeffe's claim is correct: modulo some extremely rare "gems" (who are easy to recognize anyway by other factors), long CV for a JUNIOR person is highly suspicious.
Your list is a good example why long CV is not necessarily a problem. But the discussion here was about junior people looking for a job. And this is the key point you are missing, and why Jeffe's argument has a lot of truth to it. A senior person has lots of ideas, and collaborates a lot with junior people. At the end, though, the junior people do most of the "boring" work - writing the result, polishing the paper, etc. It is not necessarily wrong, the senior person usually worked hard to achieve these "perks" - and also often has to do other tedious work, such as grant writing, to justify the fact that he/she only does the "fun" part of research.
ReplyDeleteThe point being that a senior person - especially famous - can easily write 15+ very strong papers a year, by collaborating with willing junior people on the "fun" part, and letting them to do the rest. On the other hand, a junior person DOES NOT HAVE TIME to do a good job of being a major writer of so many good papers. So it is highly suspicious if a junior person has time to write so many good papers a year. This supports Jeffe's claim.
Still, an interesting question remains. Assume a junior person has enough high quality papers, but also padded the CV with a lot of "junk". Is this bad? The answer can go both ways, but is usually "yes". The point is that the junior person "wastes" his/her time on weak research, taking away from what he/she could have done. Even though he/she has enough good papers, it raises the question WHY not to have EVEN MORE. At best, the junk did not take too much time (e.g., collaboration with another weak junior student who did all the work on the junk paper). Still, it took some time, so why bother? There is no advantage for junk, only question marks, so why risk it?
In other words, if you are good, there is no upside to produce junk and pad your CV. If you are average, producing junk can fool some people, or can alienate some people - usually the latter (especially if applying to a "serious" place). If you are weak, padding the CV is your only chance to fool somebody. Given this, Jeffe's claim is correct: modulo some extremely rare "gems" (who are easy to recognize anyway by other factors), long CV for a JUNIOR person is highly suspicious.
Another reason why a long cv for a junior person is suspect because it raises the possibility that this person got on a lot of papers by making "just enough" contribution.
ReplyDelete"being a woman can hurt you, because if you have any accomplishments, people can attribute it to the "affirmative action" that you got you this far."
ReplyDeleteYeah, right!
http://www.mindingthecampus.com/originals/2010/04/on_women_stem_and_hidden_bias.html
Time to go now. Need to get back to worshiping at the diversity altar.
Anon63:
ReplyDeleteIf right-wingers say there's no sexual bias then it must be true! After all, if you can find a single person who doesn't think they were discriminated against, then clearly no one is. Proof by induction!
@Anon64
ReplyDeleteAnd if there's some under-representation of income class W/ethnicity X/race Y/gender Z or some combination of these, it must be "proof" in the liberal book of how "the Man" is keeping them down.
Vinod Vaikuntanathan -> Toronto (from 2011)
ReplyDelete