tag:blogger.com,1999:blog-3722233.post8965370414597652477..comments2024-03-18T17:27:11.613-05:00Comments on Computational Complexity: Sending out Job ApplicationsLance Fortnowhttp://www.blogger.com/profile/06752030912874378610noreply@blogger.comBlogger69125tag:blogger.com,1999:blog-3722233.post-82995187616144140612007-12-08T13:17:00.000-06:002007-12-08T13:17:00.000-06:00There is one positive aspect of companies like Mic...There is one positive aspect of companies like Microsoft and Google asking quiz questions of interviewees:<BR/><BR/>It has made undergraduates particularly attentive in their Data Structures and Algorithms classes since the subject matter and modes of thinking required in these classes are what seem to be most frequently tested. (Our data structures class has significant programming as well as theory.)<BR/><BR/>The respect of the material that this engenders is good for theory and a positive development for CS overall. <BR/><BR/>I really noticed a change in attitude in the early 90's as word spread about Microsoft's interview questioning and it has gotten even more so with Google interview questioning added to the mix. (For example, one of the early assignments in my algorithms course this fall had a question closely related to one that many got during their Google interviews.)<BR/><BR/>BTW: It is clear from the magazine inserts that Google produces that these quiz questions have a dual purpose of selecting smart people and advertising that only smart people get to work there, which helps the brand. They are a bit like initiation rites, too, making employees feel more like part of a select club.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-3722233.post-16560124073207593382007-12-07T21:57:00.000-06:002007-12-07T21:57:00.000-06:00This has to be one of the more interesting recent ...This has to be one of the more interesting recent discussions. Claire, please post more often. :)Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-3722233.post-21241138122082999012007-12-07T10:44:00.000-06:002007-12-07T10:44:00.000-06:00A repeated point in the comments seems to be that ...A repeated point in the comments seems to be that we professor-types are doing a poor job training our students for real-world jobs. Although I personally disagree, I'd like to hear more about this; please see my blog post at<BR/><BR/>http://mybiasedcoin.blogspot.com/2007/12/preparing-students-for-jobs.html<BR/><BR/>and feel free to discuss in the comments.Michael Mitzenmacherhttps://www.blogger.com/profile/02161161032642563814noreply@blogger.comtag:blogger.com,1999:blog-3722233.post-25609236181256125022007-12-07T06:00:00.000-06:002007-12-07T06:00:00.000-06:00I think the conclusion is clear: Industry needs t...I think the conclusion is clear: Industry needs to find out if people are good, does not trust the word of others that do not work within the company and has only a very short amount of time that they can allocate to make the decision. The choice is natural, interview the guy and see if he/she can pass your test. This, of course, weeds out really great people, but then again companies do indeed seem to be (in general) big moronic giants. Indeed, just as I would expect, the cogs that carry out the policy also defend the policy since, obviously, otherwise they must admit that they are powerless cogs within a giant machine simply spinning their wheels all day. They rationalize and delude themselves into thinking that their policy is powerful and smart - "high stress situations require people that can handle it," etc. All along entirely missing the possibility that a more cautious developer/researcher who does not spew garbage from his/her mouth without carefully verifying its validity may have caught the problem before it arose.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-3722233.post-53510063344590237342007-12-07T05:52:00.000-06:002007-12-07T05:52:00.000-06:00"The second point you are missing is that performi..."The second point you are missing is that performing well on a job also include performing well in high pressure situations."<BR/><BR/>Surely you are not trying to say that every software/research job includes handling high pressure situations in the job description? You make a good point for those that do, but I don't run into too many of those in software development.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-3722233.post-70266015044146291462007-12-07T03:09:00.000-06:002007-12-07T03:09:00.000-06:00Michael:I interviewed with Google and was not aske...Michael:<BR/><BR/>I interviewed with Google and was not asked a single question of the type described in the article you link to. All the questions were either math or algorithms.<BR/><BR/>My guess is that these questions are occasionally asked, and they are the only ones you hear about because CNN will not report that google asks about data structures.<BR/><BR/>For you to conclude from such articles that these types of questions are a major factor in hiring is very bad science. <BR/><BR/>That you rely on CNN to know what your students need in order to land a job goes a long way to explain why you (general you, as in universities) are often doing such a poor job at preparing them.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-3722233.post-5204178256473541442007-12-07T02:19:00.000-06:002007-12-07T02:19:00.000-06:00The dichotomy between "undergraduates" and "grad s...The dichotomy between "undergraduates" and "grad students" is funny and shows a lot about the mentality of the people it is coming from. <BR/><BR/>In "the real world", where the military-like titles of academia carry less weight, you can see - imagine - people with and without phds working on the same team in similar positions. And guess what? Often you can't even tell the difference!Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-3722233.post-17721224056912760812007-12-06T20:31:00.000-06:002007-12-06T20:31:00.000-06:00Most of the time grad students are not in high pre...Most of the time grad students are not in high pressure situations, so its more difficult to adapt to the high pressure demands of industry. But that doesn't rule out the possibility that recent PhD can't adjust -- especially if they previously have industry experience.<BR/><BR/>Perhaps the best way to prepare for a Google interview is prepare like it is an algorithms exam. In grad school you may not have to do many 3 hour exams, but there shouldn't be any excuse for preparing for such interviews especially when you know what to expect.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-3722233.post-77668035814608262242007-12-06T19:53:00.000-06:002007-12-06T19:53:00.000-06:00Anon 59: I obviously don't know anything about ho...Anon 59: I obviously don't know anything about how you personally interview. But I do have my own experience interviewing (on both sides), I do get feedback from students, and I do read articles about hiring online, all of which are consistent. See, for example<BR/><BR/>http://money.cnn.com/2007/08/29/technology/brain_teasers.biz2/index.htm<BR/><BR/>although you can find many similar and recent articles online detailing such interview tactics. Perhaps you interview using other techniques.<BR/><BR/>I have admitted in past comments that I'm sure there is some correlation between being able to answer such questions and some of the skills people might be hiring for. I just think such tactics ignore other skills that might be (more) valuable. For hiring massive number of undergraduates, as Kamal suggests, you might attempt to argue that this is as good a way as any, which is the direction Kamal was going. It appears that many of you doing the hiring feel this way, although you haven't provided what I feel is convincing evidence, or good answers to my objections. I'm happy to be pointed to some. <BR/><BR/>For hiring graduate students, I think this type of interview is much less informative, for reasons I've explained.Michael Mitzenmacherhttps://www.blogger.com/profile/02161161032642563814noreply@blogger.comtag:blogger.com,1999:blog-3722233.post-79646392166350515642007-12-06T19:42:00.000-06:002007-12-06T19:42:00.000-06:00Kamal,First, let me thank you for responding with ...Kamal,<BR/><BR/>First, let me thank you for responding with clear and cogent arguments. I may quibble with you, but I appreciate a reasoned response. <BR/><BR/>I agree with your point about numbers. Particularly for hiring undergraduates, I can see the benefits of the interview approach (quiz/puzzle style questions) suggested by various anonymous, and that the tradeoff between its strengths and weaknesses may differ from what I suggest in my comments. However, the original context of the post (and anonymous #12, which started this all) was in the context of hiring graduate students/recent PhDs. In this context, I think the tradeoff is much clearer, and stand by my comments.<BR/><BR/>While your point about "handling pressure" is also reasonable, I have some objections. It's not clear to me that these interviews test that. Handling brain-teasers or writing code at a board in front of an interviewer is a different kind of pressure than what you're talking about. It's also not clear to me that for most jobs this skill is really a priority. For example, wouldn't it better to hire someone (or at least some fraction of employees) who had the tendency to get projects done early, rather than someone who procrastinates until a deadline but then seems to do well under deadline pressure? I know I prefer working with the former rather than the latter, though I appreciate both skills.Michael Mitzenmacherhttps://www.blogger.com/profile/02161161032642563814noreply@blogger.comtag:blogger.com,1999:blog-3722233.post-77926745080154801152007-12-06T11:09:00.000-06:002007-12-06T11:09:00.000-06:00Michael,I don't think I *COMPLETELY* misunderstood...Michael,<BR/><BR/>I don't think I *COMPLETELY* misunderstood you. As you wrote, you don't do interviews of this type, you are not an expert on interviews (I'm guessing that you don't have a very good idea what kind of jobs we are interviewing for either, it's either "research" or "coding" for you), but you know that we are not interviewing well.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-3722233.post-5486476862725302132007-12-06T11:02:00.000-06:002007-12-06T11:02:00.000-06:00Michael, IMHO, you are mostly right. But I think t...Michael, IMHO, you are mostly right. But I think the other anonymous has a pinch of correctness too. I have an impression that Microsoft (non-research) is moving/has moved away from asking tricky questions. Still letters have much less weightage than CV. CV is really the primary indicator.<BR/><BR/>There are a couple of things you need to realize in your arguments. First, the number of available jobs matter a lot. Microsoft hires tens of employees a day. In tech industry in general there are more jobs than the number of qualified candidates. You can estimate this by the enormous salaries techies get compare to other professionals. So the employers do not need a precision in their evaluation. They need to identify candidates above a very low quality bars. Say whether a candidate is in the top 25% of the class (at large not only at a specific school). The letters mostly help distinguish between top 5% and top 10% of the class. I have never seen a letter distinguishing a candidate from bottom 75%, may be because in research we look for the toppers of the class. CV is used for this distinction. For an example, I have heard that Google has an explicit cut-off on GPA.<BR/><BR/>The second point you are missing is that performing well on a job also include performing well in high pressure situations. Sales jobs are mostly high pressure. Approaching deadlines for engineers is also a high pressure situation. In industry there are situations in which people could be called from their death bed. An open exploit in Windows which is being currently targetted, a data server not behaving properly and causing 404 error etc. I have friends in Amazon (engineering) who have to be on their toes all the time. A friend broke down once when Amazon servers were down for 30 minutes at night. The failure to act in a high pressure situation can cause more harm in a day than the benefit the company can accrue from the lifetime work of an employee.<BR/><BR/>So thinking fast and keep oneself intact in a high pressure situation is a part of the job duty which should also be tested appropriately. Your skills should not leave you alone in such situations, otherwise you are much less valuable than the other candidate. I think we do not see such situations often in universities and research labs. But they happen often in various degrees in the wider wild world.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-3722233.post-74110163228715768362007-12-06T07:23:00.000-06:002007-12-06T07:23:00.000-06:00You misstate my position completely, which is:1) I...You misstate my position completely, which is:<BR/><BR/>1) I probably have very useful information you should consider -- information of comparable value to an interview. (How comparable depends on the interview.)<BR/>2) Testing "basic skills" in a time-sensitive, pressured situation is not as simple as it sounds, and unless you're an expert (and to be clear, I'm not an expert in interviewing) you're probably not testing the basic skills you're really after.<BR/><BR/>I think if you read my comments again, you'll see that those are my points. You seem to be reading what others say my points are.Michael Mitzenmacherhttps://www.blogger.com/profile/02161161032642563814noreply@blogger.comtag:blogger.com,1999:blog-3722233.post-84876064353632087672007-12-06T02:11:00.000-06:002007-12-06T02:11:00.000-06:00Chuck: I interview for a wall st. firm. The only c...Chuck: <BR/><BR/>I interview for a wall st. firm. The only circumstance in which I would resort to stupid questions would be if the candidate doesn't know anything about what they claim to know about. In that case, I might politely pass the time with pirates and dungeons instead of walking out. Otherwise, if the candidate studies math I would ask which area of math they know about and ask questions about that. And so on.<BR/><BR/>Anonymous 55: <BR/>It is good to think aloud, as long as you are focused on the problem you are given. Don't babble about everything the problem reminds you of.<BR/><BR/>I think it is unfortunate that this discussion derailed into a debate of pro-and-con pirate questions. Some important issues were raised and brushed aside, namely the question of whether grad schools are doing a good job in improving (or at least preserving) the employability of their students.<BR/><BR/>Michael's position seems to be that (1) he knows better than us what we need and how to find what we need, and (2) there is an inverse correlation between a candidate's basic skills and overall quality.<BR/>Can anyone come up with anything more useful?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-3722233.post-34645641816302948812007-12-05T16:39:00.000-06:002007-12-05T16:39:00.000-06:00Anon. 52 & 53: Don't be afraid to work on the prob...Anon. 52 & 53: Don't be afraid to work on the problem aloud, they like that. Also, if you blank during the interview but realize the answer that night, drop them an email. It's always good to follow up anyway.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-3722233.post-15972080189238235572007-12-05T16:19:00.000-06:002007-12-05T16:19:00.000-06:00In response to comment number 43:> Who asks "a-ha!...In response to comment number 43:<BR/><BR/>> Who asks "a-ha!" quiz-questions anymore? <BR/><BR/>Most Wall Street firms still ask questions about pirates and marbles.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-3722233.post-82697737044544346972007-12-05T13:31:00.000-06:002007-12-05T13:31:00.000-06:00Just to add to comment 52, I have an extremely har...Just to add to comment 52, I have an extremely hard time sleeping the night before an interview because, again, I know that my performance 'tomorrow' will dictate whether or not my life moves in a positive direction. These are just biological traits of mine that I have been trying to overcome but meanwhile I have been rejected from various positions and the corresponding companies have been missing out on my skills and ability to contribute. Let's just assume, for the sake of argument, that I do have unique skills and I am particularly valuable as an employee (of course you have no idea who I am, but if you assume that I am run-of-the-mill then this argument becomes uninteresting).Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-3722233.post-72362443592694188782007-12-05T13:20:00.000-06:002007-12-05T13:20:00.000-06:00I have been interviewing lately and have found tha...I have been interviewing lately and have found that I have a difficult time working through problems that are new to me in such a high stress position. I find that my brain processes seem to be very different from those that occur when I'm working through a problem at work and I seem to have a difficult time confidently making mental leaps without questioning my reasoning very carefully. Of course, by the time I'm finished questioning my reasoning the interviewer has been twiddling his/her thumbs and has interrupted my thought processes five times trying to give me 'hints' which I ignore because they're distracting me from my current mental road. It is very frustrating to me at times because I have been told many times by many people (managers/professors) that I am an especially valuable employee/researcher; however, this is not going to come through in an interview process until I become more adept at this process. So perhaps I can be chalked up as a false negative but who's to say that I would not be a more valuable employee in the end than someone who passes the test? Not to mention that I have a personal stake in this because I want challenging jobs but have a difficult time 'thinking on my feet' amongst all of that stress (I think on my feet just fine in my current job where my next very important turn in life is not entirely hinged upon my answer). Just another perspective on the matter.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-3722233.post-11807173418901526972007-12-05T13:04:00.000-06:002007-12-05T13:04:00.000-06:00Let's return this thread to its original intent (a...Let's return this thread to its original intent (and probably of more interest to the audience of this blog)<BR/><BR/>re: anon 50<BR/><BR/>When have you seen an academic interview matter? What can a candidate do to come out of the interview a stronger candidate than before the interview? Can a savvy candidate use an interview to ease concerns that a person looking only at the CV might have?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-3722233.post-37301220115320956002007-12-05T12:43:00.000-06:002007-12-05T12:43:00.000-06:00Michael, I would have to say that for your average...Michael, I would have to say that for your average job (not academia and not research), a technical interview provides significantly more information than a CV and letters. A CV gives you very little useful info: a list of published papers means essentially nothing to someone who doesn't understand the titles or the relative rankings of conferences. Someone who is very bright, but not exceptional, may not have won many awards. Someone who worked in a hard area may not have published very much (and, of course, the converse too).<BR/><BR/>Letters, too, are mainly useful at the extremes: "this candidate is exceptional". Every one else falls in the middle. As an advisor, you may know the quality of your student's papers but do you really have a good sense for how smart they are, or how well they program? <BR/><BR/>On another note, interviews in academia, in my experience, are mostly a waste of time (I've been on both sides of them). Yes, the interviewer might learn something about the candidate (they have a good personality?), but typically this is irrelevant when the final hiring decision is made.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-3722233.post-17700022957432938502007-12-05T12:14:00.000-06:002007-12-05T12:14:00.000-06:00With this in mind, asking clever puzzles and 'Ah-a...<I>With this in mind, asking clever puzzles and 'Ah-a' questions helps to make the impression (especially for junior candidates) that the job itself will involve solving such puzzles, and thus makes the job more 'cool' and attractive to the candidate - I was explicitly told by a guy who interviews people for a software/algorithms company that this is a tactic that they use in order to attract good candidates.</I><BR/><BR/>So this is a form of deception? Or is the job really like that most of the time?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-3722233.post-4821260907774364902007-12-05T11:27:00.000-06:002007-12-05T11:27:00.000-06:00From the discussion one can get the impression tha...From the discussion one can get the impression that the only purpose of the job interview is for the employer to test the abilities of the candidate. We must remember that in many cases the candidate is also testing the employer. With this in mind, asking clever puzzles and 'Ah-a' questions helps to make the impression (especially for junior candidates) that the job itself will involve solving such puzzles, and thus makes the job more 'cool' and attractive to the candidate - I was explicitly told by a guy who interviews people for a software/algorithms company that this is a tactic that they use in order to attract good candidates. <BR/>Another issue - asking 'Aha' questions has a value even if the candidate does know the puzzle and its answer - it means that he is more likely to many such puzzles and be interested generally in solving math/cs riddles, which is a characteristic positively correlated with mathematical abilitiesAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-3722233.post-20627944621675964402007-12-05T11:22:00.000-06:002007-12-05T11:22:00.000-06:00Something that puzzles me about the corporate inte...Something that puzzles me about the corporate interview puzzle question ("Write a storage allocator in FORTH while rubbing your stomach counterclockwise"): what do such questions in an interview uncover, that a reasonably good standardized test would not? Why not simply have an application component that involves such an examination, either an existing one, or one composed by or for your company? Wouldn't that be more consistent and reliable, and less humiliating?<BR/><BR/>For at least some candidates, isn't the interview just as much about selling the job to them, as it is about testing if those ivory-tower, namby-pamby, hoity-toity "professors" with their "textbooks" and their fancy-pants "book learning" have "taught" them anything really *useful* out here in the *real world*, where the *rubber meets the road* and the *shit hits the fan*? Is it really the case, as implied by Spolsky's advice, that there is a vast sea of really good people just begging to work at your company?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-3722233.post-88700764610512727302007-12-05T11:10:00.000-06:002007-12-05T11:10:00.000-06:00People ask math and algorithms questions that anyo...<I> People ask math and algorithms questions that anyone who studied CS should be able to solve.</I><BR/><BR/>Sure they do, no one questions that. What we are arguing about is if they give you real insight into the candidate's ability.<BR/><BR/>Something like a linear time fibonacci suggested earlier is more of an aha-style question, even though on its face is a "programming question". A much better question is to give a piece of code that can be optimized in ten different ways. Good programmers will find at least six or seven of them, even if they don't have the Aha insight for the other three. Bad programmers will only see one or two. <BR/><BR/>By the same token, it would be ridiculous to ask this question to Michael if you were looking to hire a tornado-codes expert, yet you'll find companies out there who do.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-3722233.post-68240837092642166662007-12-05T11:08:00.000-06:002007-12-05T11:08:00.000-06:00You (and probably michael too) should go and get i...<I>You (and probably michael too) should go and get interviewed before you write nonsense on blogs. </I><BR/><BR/>I have, many times, on both sides of the table (hirer and potential employer), in industry and in academia.Anonymousnoreply@blogger.com