Thursday, January 31, 2013

Who do you write papers for?

Mitch Daniels, the former governor of Indiana and new president of Purdue, wrote an inaugural letter where he discusses many of the challenges of higher education. His lists several common attacks on universities and one caught my eye.
Too many professors are spending too much time "writing papers for each other," researching abstruse topics of no real utility and no real incremental contribution to human knowledge or understanding.
I write papers mainly for myself. I have my own opinions on what problems are important and where my interests and research strengths lie. One of the main draws of being a professor is having the freedom to choose our own research.

But we have to write for our peers as well. It's our peers that review and cite our papers and make decisions about grants and jobs. For the most part our peers are the only ones who read our papers.

In the end we write papers for society. Most of our papers taken individually add a small amount to human knowledge and are only of interest to fellow specialists. But taken together our research drives a field of inquiry allowing us to understand and take on new challenges. Even if we have trouble selling a specific theoretical computer science papers to a broader audience, taken as a whole theory helps us model and understand the power of computation and leads to smarter and faster algorithms on real world machines.

22 comments:

  1. Well, this is what you get when you let politicians and CEOs become presidents of universities. Nevertheless, academics seem happy being "ruled" by people who are cutting off the branch they're sitting on.

    ReplyDelete
  2. Just an observation of mine. It is probably not the norm. When I was last hiring programmers, every person I interviewed with a brand new B.S. in Computer Science all said something like:

    "If I can't find a job, or can't hack it I will go back to school and get my Masters and maybe PhD."

    I'm happy to say that most of my University professors were great at teaching me and knew their stuff. The previously mentioned interviews scare me. What about students publishing papers. They couldn't write a basic FizzBuzz, but they are pushing the boundaries of human knowledge. Things like running simulations for their professor's papers, and so on. :-/

    ReplyDelete
    Replies
    1. I think I've had these students in class. Many are the same ones who can't hack it in an MS or PhD program either, so I am not that worried.

      I hope the Purdue faculty sit this guy down fast and at least fix his rhetoric, if not his attitude. If not, they are in for a bad ride.

      Delete
  3. To be fair, some of the other criticisms he makes are valid. I extracted those, which I followe by a comment:

    1) an obsession with expensive capital projects have run up the cost to students without enhancing the value of the education they receive.

    We all know that a large donation today is likelier to end up as a building than as a hire of a world class scientist.


    2) The system lacks accountability for results. No one can tell if one school is performing any better than another.

    In any university there are over-performing schools and under-performing departments. Most universities do not have an internal system to divert resources to the former. In fact, if anything, the former end up subsidizing the later though common facilities paid by overhead on research funds.


    3) The mission of undergraduate instruction is increasingly subordinated to research and to work with graduate students.

    In most R1 universities is shocking how many courses are taught by grad students.

    ReplyDelete
  4. The "attack" listed in the letter makes a good case for the importance of expository work. It's not only our job to do research, but also to disseminate its results -- both within the community and beyond.

    As Rota pointed out, "You Are More Likely to Be Remembered by Your Expository Work".
    http://www.math.osu.edu/~nevai.1/MYMATH/rota_ams_notices_01_97.html

    (Not that I'm such a great writer myself...)

    ReplyDelete
    Replies
    1. In the areas I am familiar with I've never felt there's a lack of high quality expository work aimed at say undergraduates majoring in math or computer science.

      The upper bound on the amount of meaningful ideas that could be explained to someone with high school math or under is very small. That sort of thing can also be bad because all the technical caveats would have to be removed at which point you are essentially lying.

      Delete
  5. @Anonymous Fortunately, students in my department who go into research to escape industry tend not to produce much of any knowledge (or papers).

    ReplyDelete
  6. Too many lawyers and policymakers are spending too much time "writing briefs for each other," researching abstruse topics of no real utility and no real incremental contribution to human knowledge or understanding.

    ReplyDelete
  7. "In the end we write papers for society..." It's unfortunate that, for the most part, members of society can't read them.

    ReplyDelete
    Replies
    1. It doesn't matter that almost nobody on Twitter could understand the research that made and continues to make it possible. They benefit from that research.

      Delete
  8. --------------
    "Maybe the oldest quip (certainly the tiredest) in the higher ed lexicon is “the fights are so fierce because the stakes are so low.”
      — Mitchell E. Daniels Jr.
    --------------

    To: Mitchell E. Daniels Jr.
    From: Albert J. Dunlap

    re: "The oldest quip in higher education"

    Dear Mitch

    You will soon hear (from your faculty) the real oldest quip in academia, which is this "Deans can't read, but they can count."

    Here is some friendly advice: ignore that old saying. Instead, proceed full-steam with your metric-centric educational vision that "Performance, especially in the intellectual growth of students, will have to be measured, quantified, and documented. Why should our university [Purdue], already strong in the few indirect measurements that exist, not be a leader in taking accountability for the excellence to which it lays claim?"

    That is exactly the metric-centric management approach that I embraced as the CEO of the Sunbeam Corporation! And results were so terrific that I wrote a book all about the magnificent results of my metric-centric leadership: Mean Business: How I Save Bad Companies [and Universities too?] and Make Good Companies [and Universities too?] Great.

    You can do it, Mitch!

      — your admirer,
          Albert J. "Chainsaw Al" Dunlap

    ReplyDelete
    Replies
    1. well said Sidles!

      Delete
    2. Yes, without Daniels/Dunlap-style scientific performance metrics, we would be unable to properly appreciate the most stupifyingly fast-paced scientific advance of the past several decades: the announcement at QIP-2013 of the successful factoring of RSA-768 and N-20000 by quantum computing methods (manuscript: Smolin, Smith, and Vargo, arXiv:1301.7007).

      Well done, Smolin, Smith, and Vargo! :)

      Delete
  9. I wonder what would be the reaction if the same argument was uttered by a famous computer scientist.

    It seems unlikely that every paper you publish will have a noticeable impact on human advancement (e.g. you publish an algorithm, 6 months later someone publishes a different algorithm, with completely different ideas, that's more efficient). Still, perhaps the impact is more implicit and in any case, I believe most would agree that without a way of predicting the future, you acted in the best interests of society.

    On the other hand, look at the number of papers on social networks in recent years. Sure, many of them address real needs (privacy , etc.) but there is also a huge amount of papers which can be summarized to "we applied known technique to a graph which happens to be a social network and we found out a result that belongs in sociology rather than in mathematics or computer science". I would definitely call that a topic of "no utility" with no significant contribution.

    ReplyDelete
  10. DISCLAIMER: I have an MS in CS and work in industry, so my perspective may be a little different. I direct this only to researchers in CS. As an outsider, I have to say that I agree to a large extent with Daniels. From my vantage point, it seems like many researchers are trying to crank out papers as quickly as possible rather than working to promote what it is they are working on. This is certainly not true of all researchers, perhaps not even most. Maybe this has to do with incentives. Perhaps professors should be incentivized by other means such as work on open source projects or MOOCs instead. So many times I've read fascinating research papers, but find the author has not produced any code for public consumption. Or, they release some code that goes with their terse journal article, but supply no documentation, making it pretty much useless. I see this all the time. And the author of this blog post confirmed my suspicions that many researchers really are just writing papers for their own edification. Fortunately, there are many other researchers that champion their work and see that it does get put to real use outside of their office. These people get my highest respect for following their life's calling and using their talents to produce something of value for society as a whole. They are not simply playing in an academic sandbox.

    I completely understand the notion that people get their PhD's so they can work on whatever research they want, but why should anyone subsidize such a system? Again, I think most professors are doing things properly, but I agree with Daniels that there are large numbers producing no value simply because they choose not to. What a waste.

    ReplyDelete
    Replies
    1. Lots of what is used in the industry is based on those "useless" researches in academia. Is it the job of academics to prepare them and package them in even more easy to use form for those working in the industry. Sorry to shout this, but WE ARE NOT SOFTWARE DEVELOPERS, IT IS NOT OUR JOB TO DEVELOP, PREPARE, AND MAINTAIN DOCUMENTED EASY TO USE CODE. If you want that employ a good programmer with decent CS knowledge to write it for you.

      If we lived in a world where universities charged companies for the right of using their publicly available research the same way that companies charge each other for using ideas like "rectangular device with round corners" there won't be a need for any public subsidies. The partial public funding for academic research is justified on the bases that some of those publicly funded research is useful for industry and but they are not short term profitable outcomes that would justify companies (with the exception of big ones like Microsoft or Google or IBM) to invest in them.

      There is an oversight over those public funding by NFS and other organizations that provide them. You can argue that there can be a better allocation by focusing more on quick and ready to use research but it doesn't mean the current situation is bad and we can argue that an important part of funding for academic research is long term in area that companies are not ready to invest.

      The percentage of budget that is spent on funding research is rather small compared to several more objectable parts of it. The countries who spend a larger percentage of their budget on supporting scientific research position themselves better for economic developments and technological advances in the future.

      Delete
  11. The excerpt below is very important for innovators and revolutionary thinkers:

    "Diversity is prized except in the most important realm of all, diversity of thought. The academies that, through the unique system of tenure, once enshrined freedom of opinion and inquiry now frequently are home to the narrowest sort of closed‐ mindedness and the worst repression of dissident ideas."

    If possible, think about it please, and see live real dissident ideas about new generalized definitions and hidden assumptions on TCS foundations at http://arxiv.org/ftp/arxiv/papers/0907/0907.3965.pdf and http://www.andrebarbosa.eti.br/The_Cook-Levin_Theorem_is_False.pdf.

    ReplyDelete
    Replies
    1. André, the point is that Daniels' assertion "Diversity is prized except in the most important realm of all, diversity of thought" directly opposes Daniels' assertion "Performance, especially in the intellectual growth of students, will have to be measured, quantified, and documented."

      In academia, business, and military organizations alike, history shows us plainly that performance metrics all-too-naturally are administratively exploited Sauron-style:

      ---------------------
      The Dean-Ring's Inscription

      One Performance Metric to rule them all,
      One Performance Metric to find them.
      One Performance Metric to bring them all,
      and in Executive Steering Committees bind them
      in the Land of Administration where the Shadows lie.

        — translated from the tengwar
      ---------------------

      This is not to say the metrics are not useful (even essential) in academia and every other large enterprise. A crucial question for every academic dean is this: What span of research, teaching, and service diversity should academic metrics respect?

      Too many diversity-dimensions, and the (very real) administrative utility of metrics is lost. Too few diversity-dimensions, and the vital spirit of academic discourse is quenched Sauron-style/Dunlap-style. Is there a reasonable center-ground? That is what the students and faculty at Purdue — and every research university — are about to find out!

      Delete
    2. Dear Dr. Sidles,

      Creativity, courage and intelligence are non-measurable qualities, but “the intellectual growth of students” can be a possible metric, if it is cleverly treated into a university. TCS researchers and professors do not live at unreachable ivory towers. The real world can be sometimes ugly, dusty and dark, but it is where we all live together.

      I think we should write papers to the honor of Mathematics itself - sometimes against the powerful mainstream - instead of to our poor academic vanities. To write papers mainly for ourselves may be the best way to run in circles, look just for our navel and be blind about new ideas outside away from our glad sandboxes and beautiful aquariums.

      Hence, we have, for instance, entire theory about computers that only exist into our good dreams, whereas we deny formulating simple rules to measure our concrete work in the academy, for the tribute of Science itself. Why that paradox?

      Delete
    3. People do mathematics for many good reasons, and (as it seems to me and many) there is no metric that can encompass all of them. It is regrettable that fiscal pressures sometimes force the embrace of metrics that inadequately respect this vital diversity.

      Three time-tested, often-referenced essays that respect mathematical diversity are De Milo, Lipton, and Perlis "Social Processes and Proofs of Theorems and Programs" (1979), Richard Hamming's "You and Your Research" (1986), and Bill Thurston's "On proof and progress in mathematics" (1994). Google finds them all! :)

      Delete
  12. My chess-cheating report last month was a big case of this. Originally I was requested to write something brief and punchy in language chess players could understand. Then people in general, and especially some close to me, got more of a sense of depth of what happens when the only evidence is statistical---and not even about a "physical object" like a fingerprint or DNA sample. And concerns of human dignity and presumption of innocence and due-process. Hence after a painful week of "evolution" I turned around to writing carefully and drily for an audience of professional statisticians, or for a court case expecting cross-examination. That is, writing for the kind of people my cover letter says the chess world should seek for advice.

    ReplyDelete
  13. there is some confusion in the comments somewhat stemming from how this post was written. daniels was not criticizing education with the quote:

    "Too many professors are spending too much time "writing papers for each other," researching abstruse topics of no real utility and no real incremental contribution to human knowledge or understanding."

    it was in a list of common attacks he wrote out, ie "stuff that he's heard as feedback", and then he proceeds to a very thorough and impassioned defense of higher education and the college system on its merits, ie a rejection of the negative theses presented. the letter is quite remarkable in its scope esp as it considers the long sweep of education and the economic pressure that it may be coming under from new models eg online education etc.

    also, I think the respondents are missing a key element of the quote, and that is (OVER?) SPECIALIZATION. this is an increasing issue with all areas of research, not just CS, but it can definitely be seen in the TCS community, and how the community acknowledges it and addresses it is a measure of maturity of outlook etc. a long essay could be written on this subject alone (but cant think of anyone who recently addresses it).

    as in the quote "researchers are focusing more and more on less and less, so soon they will know everything about nothing!"

    I think I recall an old classic book/polemic on the subject by Morris Kline. (who a CS professor let me borrow over 2 decades ago as an undergraduate)...

    I urge everyone to take a look at the daniels letter if you are interested in the emerging issue of how the college system is working, or not working, or how it might change in the digital era, etc. (have seen some of lance's blogs on MOOCs etc, the letter has a lot to say around this subj).

    ReplyDelete