Chapter II.C.2.f(i)(c), Biographical Sketch(es), has been revised to rename the “Publications” section to “Products” and amend terminology and instructions accordingly. This change makes clear that products may include, but are not limited to, publications, data sets, software, patents, and copyrights.So you can list your patents or open-source software as a "product" right up there with the same status as an academic publication. This seems like a harmless change, us theorists can continue to just list our publications. But I worry about the slippery slope. Does this signal a future change to the NSF Mission that supports "basic scientific research"? We will be expected in the future to have "products" other than research publications?
Or is the NSF just saying that while they fund our research, it's not the research, but the manifestations of that research in whatever form they take, that gets judged for future grants?
You do have a product other than research publications. It is known as "a blog".
ReplyDeleteI think this is simply a recognition of the fact that publications are not the be all and end all of scientific communication that they once were, particularly in the age of the internet. In some research areas this may be more obvious than in others, but even in Theoretical Computer Science there are things like videos from conference talks, the Complexity Zoo, and research level discussions on quantum computing going on at Richard Lipton's blog. Admittedly, I do not know the precise definition of "product" that the NSF is using, but if it includes these things then I think it is good that researchers can claim credit for them. Of course, as a practical matter, I suspect that grant reviewers will continue to put far more weight on publications than anything else.
I choose to see that as a positive change. It is well known that the publication count is a poor measure of one's research. Making official that other types of output is to be considered is a confirmation of that fact
ReplyDeleteThe NSF Fastlane Awards Reporting system has had a "Publications/Products" section for years where one can enter websites, software etc. In some sense this is just the bio catching up. I am not sure why they didn't use the same "Publications/Products", though. (Anon: The NSF bio already gets around the "count" issue by allowing only 5 most closely related and 5 additional "publications" so that isn't something new.)
ReplyDeleteI also see this is mildly positive; however, I am bothered by
ReplyDeletethe ambiguity of the word ``product''. Does this blog count?
Does math-fiction count? If I wrote code for some obscure
math thing, posted it on my website, and got 0 downloads,
would that count? Do we list all these things and let
the reviewers decide what counts?
Anyone is free to choose to include any of these if they think that they will impress proposal reviewers more than their publications will. The award reporting system has an "Other Products" category that covers all of the kinds of things mentioned. One can ask: Is software that has 0 downloads more or less impressive than a publication that has 0 readers?
ReplyDeleteWell... It is a longstanding demand of some computer scientists, at least in France, that evaluation committees (be it for funding, recruitments, promotions) take into account more than publications, but also e.g. software implementations.
ReplyDeleteThe reason is that there is a huge difference in efforts between getting an algorithm correct on paper and publishing it, and doing so plus implementing it for real inside a system that can actually deal with non-toy problems. Such efforts should be rewarded.
I guess it is just always good that theories can be of practical use. It is so excited to see that some products are made which integrates good results of theoretists.
ReplyDeleteResearchers like McKay should be rewarded for NAUTY and the like.
ReplyDeleteI rather choose to be more productive.
ReplyDelete*Computer scientists everywhere have strokes at the prospect of suddenly having to think about actually, gasp, programming something!*
ReplyDeleteGasarch wrote: "If I wrote code for some obscure math thing, posted it on my website, and got 0 downloads, would that count?"
If that few people are interested in it, why are you even bothering to research it, in paper form OR program form? Stop wasting your intelligence on junk, contribute something to the world!
1) My example was hypothetical.
ReplyDelete2) Many people write papers that nobody will ever read. Is this a waste of time or intelligence? (1) you don't know which papers will never be read, and
(2) by working out stuff on papers that will never be read you may get
ideas for more important stuff. But any of these points is debatable.
Thanks for the response.
ReplyDelete"Many people write papers that nobody will ever read. Is this a waste of time or intelligence?"
If you argue that it's not, then the same argument applies to programs nobody ever downloads.
I should think computer scientists would be thrilled at the idea of NSF accepting products along with papers. Shouldn't that help you guys-- relative to other disciplines which aren't, you know, directly related (at least in some distant ancestral sense) to, you know, products?
Maybe the existing state of affairs (publish or perish) has allowed CS to become dominated by people who game the existing system (publish lots of papers, attend lots of conferences) while not actually programming anything. If a change of policy leads to more Donald Knuths, more LaTeXs, good. Right now the horrible crackpot Stephen Wolfram is absolutely dominating (with Wolfram Alpha) a huge portion of what average people see of computerized mathematics, and pretending otherwise won't make that go away; why don't you put down the PRPSPNQSQ vs. NPRPSPNQSQ taxonomy nonsense for awhile and solve something like THAT.
"Right now the horrible crackpot Stephen Wolfram is absolutely dominating (with Wolfram Alpha) a huge portion of what average people see of computerized mathematics, and pretending otherwise won't make that go away;"
ReplyDeleteI think that is inaccurate on many counts:
1) Mathematica really is a good piece of software. Wolfram's disastrous forays into other areas should not detract from his achievement of creating and then managing a very good piece of software.
2) Mathematica does not dominate computerized mathematics totally: off the top of my head I can count Maple, Magma and Singular as being important players (though admittedly Singular is a bit focused on one area). This again is about intended application domain: Mathematica and Maple give a good out-of-the-box set of tools that are useful to a large set of people. Software like Singular, GAP and Bertini are, on the other hand, focused on one job and do it well. Clearly, the more general system would be of more interest to "average people" while GAP would be more interesting to someone who mostly wants to do group theory calculations. Why is that a bad thing?