Tuesday, September 15, 2009

Fashionable Research

A student asks "How do you survive in the academic world if what you want to do is not fashionable?"

You shouldn't necessarily focus your research on the currently hot topic. Many people will flock to this area so you'll have considerable competition. And while today maybe you'll see many papers in area X being accepted to the big conferences this year, topics can get cold quickly and so will you.

But what about the other extreme, where you do research in an area of very small interest because you have a strong passion for it. Unless you get very lucky and this area goes hot in the future, you'll be giving up any chance of larger fame or fortune. But if you do good work in this area, you can use your passion to sell this area in a job talk and get strong letters from those few senior researchers in the field happy to promote people in their field. So often you can get a job at a reasonable university and spend your life working on what you love. Is that so bad?

Research always comes to passion and ability and you have love what you do and be good at it, no matter the topic. If you don't enjoy your research, you can usually make much more money doing something else you don't enjoy.

18 comments:

  1. With respect, Lance, some numbers would make your post stronger.

    (1) Each year, how many students with "passion and ability" for academic research enter college (nationally and globally)?

    (2) How many jobs "at reasonable universities" open up (nationally and globally)?

    Some order-of-magnitude Fermi estimates would do!

    IMHO, it's far better to grow (2) than shrink (1) ... perhaps we ought to be thinking about this goal as creatively as we know how?

    ReplyDelete
  2. Unless you are out in the front half of the field, doing what is very fashionable may not get you noticed.

    On the other hand you need to assess why something is unfashionable. Some fashions are merely "geographic"; something is fashionable in one community but not another. However, there are lots of reasons that research areas/problems go out of fashion:

    * The problems became too hard and the progress was too slow.

    This is risky but could have high payoff if you succeed in something significant.

    * The main problems were solved and the remaining open problems were viewed as peripheral

    If you can show why these "peripheral" problems really are interesting then you may have a winner but you need to make sure that there is enough scope there.

    * The only interest in the problems is as surrogates for attacking more difficult problems but the connection is based on a lot of indirection that is very tenuous. Someone expected these problems to be solved easily but they weren't.

    This could be a rathole. There may be no guarantee that solving them won't be harder than solving more significant problems. Ask around and get a variety of opinions about the value of the problems.

    * Technology changed and the issues are no longer viewed as relevant

    Sometimes subsequent technology changes can reinvigorate old ideas.

    * Something else related simply became hotter.

    There was no good reason to drop work in an area but other areas took the mindshare. This can be fine if you can solve significant problems.

    ReplyDelete
  3. Paul Beame's advice is IMHO excellent.

    Yet another good thing to do is read biographies---especially autobiographies---of those scientists/mathematicians/engineers whom you admire ... keeping Paul's advice in mind while you read them.

    You will find that not too many people focus their early research interests on "fashionable" areas---it is almost a liability to do so, because the competition is so fierce---but they *do* tend to work in areas that help them acquire a powerful mathematical and conceptual toolset.

    E.g., Schwinger's early work on (unfashionable) radar waveguides evolved into his mature work on Green function methods in quantum field theory.

    ReplyDelete
  4. Which one (out of Paul's list) would you say is the case for algorithms research, which is clearly considered unfashionable these days? Just curious.

    ReplyDelete
  5. With regard to algorithms research, perhaps the present situation best fits Paul Beame's category of "subsequent technology changes can reinvigorate old ideas."

    The idea being, that modern simulation-based science and engineering is evolving into "The art of solving practical problems that are in NP."

    Here the word "art" is inspired by Donald Knuth's (wonderful) forward to Petkovsek, Wilf, and Zeilberger's (wonderful) algorithms book A=B:

    "Science is what we understand well enough to explain to a computer. Art is everything else we do. ... Science advances whenever an Art becomes a Science. And the state of the Art advances too, because people always leap into new territory once they have understood more about the old."

    Obviously, a good strategy for young people is to focus on a discipline(s) that is in the early stages of the transition from art to science.

    Which discipline(s) would that be? Uhhhh ... that's one of them NP-hard questions! :)

    ReplyDelete
  6. Is there room for professors who do research into things they find interesting whether or not there is money or fame there? They would be active researchers and would be able to live up to their duties as a teacher and would be someone who could provide research opportunities and/or guidance for students.

    ReplyDelete
  7. Is there room for professors who do research into things they find interesting whether or not there is money or fame there?

    They are called "professors with tenure (who also don't care about grants)".

    Sorry, I couldn't resist.

    ReplyDelete
  8. When I was at PODC in August, someone asked me, "Are you the only person in the world who does what you do?" I had to answer, "Yes." So I think about the issue of this post sometimes.

    And here's something else I've been thinking about: my submission to SODA was just rejected, in part because the program committee was not sold on the importance of the paper's results to either self-assembly or distributed computing. And yet, I recently learned that same paper has been cited, even though I've only had it on the Arxiv since July. So... is it better to have a paper accepted to a "big" conference, but not generate citations, or is it better to have your work cited even if it is rejected from top venues?

    My answer to that question is that if I were looking for a job right now, I'd want the publication. Since I'm still building a resume and a future job talk, I suppose if I had to choose either/or, I'd rather have the citation, for now. Good thing too, cuz it's all I got!

    ReplyDelete
  9. That was a fine post by Aaron Sterling ... the best of luck to you, Aaron.

    In a similar vein---but in art rather than science---is this review from Arnold Schoenberg, which perhaps may inspire some young person:

    "There is a great man living in this country – a composer. He has solved the problem how to preserve one's self and to learn. He responds to negligence by contempt. He is not forced to accept praise or blame. His name is Ives."

    Anyone who wants to hear the voice of a person who creatively went their own way need only listen to the music of Charles Ives.

    Google Books lists more than a dozen biographies of Ives ... if you are a young person who is not familiar with Charles Ives' life and music ... why ... then today is a lucky day for you! :)

    I am not sure that any mathematician has lived a life like Ives ... perhaps someone can name one?

    ReplyDelete
  10. Ives was independently wealthy aside from his music; he didn't need to care about how it was received because he didn't need any successes to eat. Closest analogue today is probably Stephen Wolfram, who is just doing crazy, stupid shit but seems genuinely happy doing it.

    ReplyDelete
  11. There are many people who succeeded by all possible reasonable metrics, yet, they have never tried, nor never been working on fashionable things. Woody Allen comes to mind.

    (I also have some Computer Scientists in mind, but I'm just worried they may be upset if I compare them to Woody Allen... )

    ReplyDelete
  12. Could you please list the areas that are currently fashionable/not fashionable?

    ReplyDelete
  13. Very interesting post.
    For me the option is clear, I prefer doing what I love. The other option is a ridiculous choice, it is only because of money, which does not fullfill people needs once you have enough. I can understand going for the money if you live in a third world country where doing research you earn a very small income and it is imposible to live a decent life. But in north america you can leave a decent life doing what you want. I can not understand why people prefer to be rich because once you have enough money to live well having more does not change anything, and working in a job you do not like leads you to a lower quality of life.

    ReplyDelete
  14. Anonymous said: Closest analogue [to Charles Ives] today is probably Stephen Wolfram ...

    On reflection (and using "potential Nobelist" as a sorting criterion) I would rank David E. Shaw pretty high ... at FOMMS Dr. Shaw showed some millisecond-long dynamical protein simulations (a millisecond is a *long* time when the fluctuation time-scale is a femtosecond) ... these offered a glimpse into a world of conformational biology that is immensely more dynamical than anyone has dreamed.

    The point being, that David Shaw conceived and runs his groundbreaking research program entirely as a individual initiative.

    Off the top of my head, I can't think of any past Nobelist who was a scientific/mathematical "ronin" ... though Edwin Land surely was a contender ... perhaps this career arc will become more common in the future?

    ReplyDelete
  15. Which one (out of Paul's list) would you say is the case for algorithms research, which is clearly considered unfashionable these days?

    In its proper interpretation, "algorithms research" is very fashionable and this is simply a misplaced question. Most of what we do is algorithms, including algorithmic game theory, computational geometry, parametrized complexity etc. PRIMES in P was an algorithms paper. For years, though, I have heard people working on approximation algorithms for NP-hard problems say things like "back when people used to work on algorithms" (meaning "exact" algorithms) which has always seemed strange to me.

    If you take a very narrow interpretation which is "graph algorithms of the sort that appear in classic textbooks" then I would say that it is mostly the last item on my list: "Something else related simply became hotter". Other areas became hotter in part because the questions were interesting and less well explored. It was also because people could see that they could make quicker progress using some of the tools they knew, as well as some new cool tools (fancy LP rounding, SDPs, metric embedding) that were also ripe for exploration.

    While the number of people working on ordinary exact graph algorithms is much smaller than algorithmic game theory these days, there is regular progress and recent graduates making progress in the area have done fine on the job market.

    A couple of bits of advice, though:

    * As someone starting out, it is not good to narrow one's options too early. To use some cliches: it is good to have "multiple irons in the fire" and not "put all ones' eggs in the same basket". This can be a good strategy not only because progress on any one front is not guaranteed but also because an ability to work broadly will allow you to make more connections with people and be viewed more positively by hiring departments. Many a time I have seen a PhD graduate who has worked very narrowly during their dissertation suddenly blossom when exposed to new areas and problems as a postdoc.

    * This must be balanced. One should avoid going overboard and simply end up flitting from one topic to another. You need to focus enough to make progress and it will eventually be very useful to be known as an expert in some specific areas or techniques. However you shouldn't limit yourself to those.

    ReplyDelete
  16. On this topic, I found Freeman Dyson's talk Unfashionable Pursuits very inspirational. One of his examples, Hermann Grassmann made huge contributions to mathematics, pioneering exterior calculus and abstract algebra in 1844. The exact reasons why Grassmann's work was largely snubbed are not entirely agreed upon (to my knowledge). Regardless he did most of his work while teaching in gymnasium rather than university.

    Maybe Grassmann wasn't the perfect analogy to Ives (he certainly did seek recognition), but he was a dogged outsider who stuck to his guns.

    ReplyDelete
  17. The exact reasons why Grassmann's work was largely snubbed are not entirely agreed upon (to my knowledge).


    One reason is that he was a lousy writer. Another is that he never made a compelling case for the importance of exterior algebra. It turns out to play a fundamental role in geometry, but you'd never guess this from Grassmann's book. Partly he was just ahead of his time, but partly he was more interested in developing his theory for its own sake than in building connections or finding applications (so a lot of the potential audience never understood why they should care).


    Regardless he did most of his work while teaching in gymnasium rather than university.


    That's not so unusual: it was a far more respected position than a high school teacher in the US, or even a teacher at a modern gymnasium, plus there were fewer professorships back then. For example, Weierstrass taught for more than a decade in a gymnasium.

    he was a dogged outsider who stuck to his guns.

    Definitely!

    ReplyDelete
  18. Two comments about people:

    1. Charles Ives did not come from wealth. His father was a band conductor. Aftr graduating from Yale, Charles Ives went to work in business: he was VERY successful. He more or less invented estate planning, wrote a book about tax policies individuals could use to minimize their taxes, and eventually had his own firm. I believe he also did pioneering actuarial stuff.

    2. To reinforce the notion of high school teaching being more prestigious some time ago: Post (of Post Correspondence Problem, Post Tag Systems, simple sets, and Post's Problem) was a high school teacher, as was Bolyai (of noneuclidean geometry).
    I am not sure about today, but a decade ago teaching high school was quite prestigious in France.

    In fact, at elite US high schools teachers with PhDs have very respectable salaries--in the humanities it is often higher than the average university faculty salary.

    ReplyDelete