Thursday, January 24, 2013

The End of a Useless Test

First a word from our sponsor: Mihalis Yannakakis is celebrating his 60th birthday this year and you are invited to the party.

From the Educational Testing Service:
The last administration of the GRE Computer Science Test will be in April 2013. The test will be discontinued after the April 2013 administration. Scores will continue to be reportable for five years.
Why is the Computer Science Test being discontinued?
Over the last several years, the number of individuals taking the Computer Science Test has declined significantly. Test volume will soon reach a point where ETS can no longer support the test psychometrically. As a result, the GRE Program has decided to discontinue the Computer Science Test. The test will be offered for the last time in April 2013
"Pyschometrically" just refers to the ability to measure abilities, mental traits and processes. Seems redundant in this context.

Bill tells me Maryland CS used to require a subject GRE but no longer does. Georgia Tech CS does not require a GRE. We have a separate PhD program in Algorithms, Combinatorics and Optimization that recommends the math subject test.

Computer Science is a broad field and can't be easily tested, psychometrically or otherwise, and the score of the subject test does not help us determine who will be a good grad student.

The regular GRE exam is not that much better. The quantitative can only help weed the weak students. The analytic score should be a good predictor but isn't. The verbal score, at least among Americans, oddly enough may have the best correlation to success in grad school, but not reliable enough to put much weight on it.

So how do I judge PhD candidates? Grades in CS and math courses taking into account the quality of the university, any undergraduate research, and the recommendation letters.


  1. Do you have a department committee that processes applicants or do individual faculty choose ones they wish to admit (as their prospective students)? Do you have a quota on number of admits each year?

  2. It's so weird! What will happen to students who hadn't a chance to take this exam?! Many students including myself working hard to take this exam in 2013 (of course later that April!) and this plan completely destroy their future.
    There are many students who didn't gain good scores in their undergraduate program and CS subject is the only hope for them to improve their chance while applying for grad school!
    It's not fair at all!

    1. A student who does not have good grades as an ugrad and then
      suddenly does very well on the GRE's- not sure it will help much
      unless they've also done some research, have some good letters,
      or have some other criteria. When I see Bad Grades/Good GREs
      I think (perhaps unfairly) SMART BUT LAZY. ANd I also thing
      (perhaps unfairly) that people who are Smart but lazy become
      people who are Dumb but lazy over time.

      However, to be more positive- find a research project you are passionate
      about and that may compensate for some bad grades. Also, if
      grades INCREASE over time (e.g., we can forgive a bad
      first year if the rest is good) then that should also help.

    2. you are wrong. evidence indicates that iqs are stablle over time. for the record, gre and sat both correlate strongly with iq.

    3. @GASARCH
      Perhaps, and while your points are valid, I feel for the many students who work their way through college. Mediocre grades are not directly related to intelligence nor ability. There may be other factors and a GRE score could have helped make an argument for admission more sound. I completely agree with research (good research). If you can do your own then great, or volunteer your time to a professor who believes in you. You will not get paid (usually) but that isn't the point here. Also, make sure you have a portfolio. Your portfolio should not only contain creative works, but programs that are directly related to the foundations of computer science, discrete mathematics, and computational mathematics. For the latter, it really helps that any software/applications that you create have relevance to the type of research you wish to do in Graduate School. -Cheers

    4. As someone who has been working on programming on the side of an unrelated major for undergrad, I was strongly reliant upon the GRE subject test to show that I CAN compete with CS majors for a masters position. Otherwise, I get shoved to the back of the line for non-majors.

  3. In Europe we have always done perfectly well without graduate entrance exams and it has always seemed a little bizarre to me to put so much weight on yet another exam when all the students have been doing for the past four years is taking exams.

  4. There are many students who didn't gain good scores in their undergraduate program and CS subject is the only hope for them to improve their chance while applying for grad school! It's not fair at all!

    Oh, calm down. First, not getting into graduate school will not "completely destroy" your future. It may change your future, certainly, but that's not the same thing. Second, there are many other paths to graduate school than getting good GRE scores; for example, many departments (like mine) admit non-degree students into their classes, either online or an campus. Third, speaking from direct personal experience, earning poor grades as an undergraduate has consequences; take responsibility for them.

    (My department does not require GREs, either general or subject, but many applicants send them anyway. In practice, we use them primarily to filter out applicants who volunteer low scores.)

  5. --------
    Lance Fortnow posts: "So how do I judge PhD candidates? Grades in CS and math courses taking into account the quality of the university, any undergraduate research, and the recommendation letters."

    Lance's criteria are (essentially) identical to those by which we judge applicants for our surgical residency (typically 350 applicants for eight slots).

    At the end of their six year, we compare our admission rankings to our performance sixth-year performance rankings. The correlation is (essentially) zero.

    Conclusion  Far too few professional employment slots exist (in medicine? in CS? in all STEM disciplines?) relative to the number of qualified young aspirants.

    Implication  The failure to create new jobs, in sufficient quantity, that are professionally challenging, and that are family-supporting, is today's greatest STEM failing, and remediating this failure is the STEM community's most urgent challenge.

    1. The NYT's David Brooks spoke to the gate-keeper issue today:

      "Half of the jobs in university political science programs went to graduates of the top 11 schools. That is to say, if you have a Ph.D. from Harvard, Stanford, Princeton and so on, your odds of getting a job are very good. If you earned your degree from one of the other 100 degree-granting universities, your odds are not. These other 100 schools don’t even want to hire the sort of graduates they themselves produce."

      Carl Malamud, speaking at Aaron Swartz' memorial service, spoke to the gate-keeper issue too:

      "Sequestering knowledge behind pay walls — making scientific journals only available to a few kids fortunate enough to be at fancy universities and charging $20 an article for the remaining 99% of us — was a festering wound. It offended many people."

      The greatest disrespect we can show to these tough issues, is to imagine that our limited individual actions, and our minor committee decisions, have nothing to do with them.

    2. "Far too few professional employment slots exist (in medicine? in CS? in all STEM disciplines?) relative to the number of qualified young aspirants."

      I don't see how you draw this conclusion from your previous paragraph.

      In fact, it is the exact opposite in CS. I see companies begging to hire people, and there are not enough (qualified) people willing/available to take those positions. Getting a CS degree may be the most sure path to employment right now.

    3. More qualified people would take them if they paid well enough; otherwise they're not worth the pain of putting up with unqualified people.

  6. GRE should be replaced by random(100) . Please explain to me how learning obscure English vocab that 90% of native English speakers will never use in their entire lives is a measure of how good I am at CS. The actual English test,e.g. TOEFL, is 10 times easier than this GRE section. Plus, the amount of money they charge is ridiculous. We are not in the 1850's where I have to send my results to the royal courier to stamp them with Spanish wax so that the Abbey will admit me.

    I suggest that ETS takes a ride on a pinnace. (a pinnace is a small boat, that is not a life saving boat. I learned that for the GRE exam, which is obviously more important than discrete math or something).

  7. FWIW, I use almost all the GRE vocab words occasionally. It's nice to be able to communicate with people that understand you well.

    But the post is about the CS subject test, not the general GRE.

  8. If I had it to do all over again, I'd focus my time and energy on research from the getgo. Undergraduate research is fun stuff and it gives you (and the schools you're applying to) a much better perspective on how you'll enjoy grad school. Plus it gives you an idea of a research area and a foundation in that area.

    I'd even recommend some of the classic papers or surveys of classic papers as reading material over a general subject test. For instance, why not a situation where students are given something like a research paper that many deem well written and ask students to write a summary of that paper and the some of the open questions surrounding it. That would do a much better job at helping decipher the student's ability to do research, as this is an integral part of the research process.

  9. Do these tests (GRE, GMAT) apply to the transition from Undergrad to Grad only? What if a MSc plans to go for a PhD, do they still need to take these tests and - most importantly- why?

  10. So now do we have to write GRE or only GRE subject test (Math) or both if we want to join a grad college or they are all just moot now ?

  11. I know many with high GPA and low GRE and as many with low GPA and high GRE. The likelihood and ability of low GPA students improving their GRE/GMAT scores is far LESS than those with high GRE scores improving their GPA.........harnessing aptitude and ability (~ high GRE) requires motivation and desire (~high GPA) or sometimes a kick on the face (by parent, professor or recruiter)

  12. High I.Q. != smart. Smart = knowledge + wisdom. Sufficient intelligence + willingness to work hard. Research experience appears to be the best predictor. If you want to get into science, volunteer at a lab evenings/weekends or whenever you can. Gain the trust of the PI. A strong letter from a PI can turn things around fast.

  13. Awwww.... That was a fun test. I had to learn all sorts of BIZZARE stuff that I will NEVER, EVER have need for in the real world (Took it in 1995, scored high enough to get into MENSA and made my admission application to George Mason University's Graduate program pretty much a formality.