We've all seen it. Most of us have even written one. I'm talking
about that ``clearly marked paragraph'' in the summary page of each
and every NSF proposal:
Broader Impacts This proposal has far-reaching impacts. As part of my program, I will develop a new graduate course entitled My Research Area, that will introduce students to cutting-edge research in Proposal Topic X, Y, and Z. I will also incorporate these lectures into some of my Undergrad Courses. Special attention will be given to recruiting Women and Minorities. And I may even talk to A High School Student once in a while. Yada yada.As often written, these broader impact sections, like the one above, read like the teaching section of my job description. So then, what is a broader impact, really? How can we improve our impact? And, most importantly, why should we, as a community, care?
I am now on an airplane returning from an NSF summit organized by Tracy Camp, Juan Gilbert, Judy Goldsmith and Samir Khuller (kudos to you) that discussed just that. The summit consisted of about 100 members of the CISE research community, and the purpose was to collect input from us about what we want broader impacts to be and how they should work. My working group was tasked with fleshing out Broader Impact #4, ``Broad Dissemination and ...'' (who knew, there are in fact five types of broader impacts, contained in a bulleted list in some NSF document from 2007, and yes, they all have long unmemorable bureaucratic names). Here's what we had to say (disclaimer: all comments are colored by my own personal biases and are not intended to accurately reflect the opinions of the participants etc. etc.):
- What is a broader impact? All sorts of really cool things
count here. In our group on broader dissemination, we came up with:
blogging, YouTube video clips,
maintaining wiki pages,
writing a textbook, a
popular science book
, directing a play about science, designing a museum exhibit building computers with kids, with senior citizens, talking directly to the curious public in Scientific Cafe, with K-12 at National Lab Days, talking to the media, writing your representatives... Many of these have been done before, and I think there will be a link on the NSF website sometime soon giving pointers to some wonderful examples. More generally, a good broader impact is realistic and, ideally, measurable — points which ought to be discussed in the proposal.
- How can we as a community help our individual members improve their impact? Some people have an internal fire that is fueled by helping others, and for those we can enable their impacts by simply making it easier to give. For this, the NSF will provide lists of ideas, and several programs like National Lab Days and BPC further help by providing ``match-making services'' that give a searchable interface to existing outreach opportunities (looking around there, I found a local high school that wants someone to come talk about careers in science, for example). Then there are also carrots and sticks. The carrots are higher weight for broader impacts in the review process as well as the tenure process (ummmm, I'll believe it when I see it); and the sticks are holding PIs accountable for their proposed impacts through annual reports alongside a threat of withheld funding upon failure to attempt said impacts (spank spank).
- Why should we care? If you haven't figured it out by now, I am an incredibly cynical and suspicious individual. While I personally care about certain types of community service, I nonetheless felt that broader impacts in a proposal were simply a nod to Congress, a necessity that allows our elected representatives to justify giving us hundreds of millions of tax dollars, and of minimal importance in funding decisions and career success. I now see that, while there is a certain grain of truth in my snide remarks, the NSF is on a serious mission to change all this, and we should be too. We have a passion, and as privileged members of humanity, we have a duty to share our passion with those around us, thereby enriching their lives as ours were enriched for us by circumstance and chance and past generations of great givers.
Does theoryoverflow count as a means of broader impact?
ReplyDeleteIt is open to everyone and some participants there are people who do theory for fun, or need theory to help them in their jobs even though they are not theoreticians.
In other words, does helping CS people who are not in theory, for example, use algorithms count as broader impact?
Specialization, in general, was a key contribution to the industrial revolution and continues to work for the information age, promoting deep research and supporting reductionist directions. Apparently, though, Broader Impact is not something to be specialized: it is the duty to every researcher to excel on this count. What if some have more talents and interest in Broader Interest than others? Look, I'm not saying that we should be oblivious to how our work, however interesting to us, will make impact (apologies to Hardy); similarly, we should be cognizant of social responsibility for how our research might be applied. But is it the duty of every supportable researcher to come up with some "innovative" and exciting new proposal for Broader Impact?
ReplyDeleteanon1: that's an excellent point. I don't see why theoryoverflow should not be part of a broader impact portfolio if blogs are.
ReplyDeleteNow if only we get enough people to commit to it :).
http://area51.stackexchange.com/proposals/8766/theoretical-computer-science
I guess Nicole beat me to it - I had spoken to Bill Gasarch about doing a guest blog about this summit that just happened last week. I will add some remarks later on in any case to the excellent post by Nicole. (I am behind on several tasks, since I was the local organizer for the summit that took place last week -- it also makes me appreciate more what local arrangements people go through when organizing FOCS and STOC.)
ReplyDeleteMy *personal* feeling is that since this is important for NSF, PIs will respond if the panels take BI into account when making funding recommendations. For example, any PI who has had prior NSF funding should be asked to report on their BI activities. Panelists should be asked to report on the PIs BI activities and its impact. Only then will the situation begin to change.
While there were several statements made by speakers that I disagreed with, I did feel that a lot of ideas emerged (Nicole mentioned several of these) and people will find innovative ways about how their research and educational activities can have a broader impact.
Just as an example -- some people felt that them working with their Ph.D. students was "broader impact". I am not saying that I endorse this view -- its the viewpoint of some members of the research community, thats all.
Another point that is worth discussing is how much time should an Asst. Prof. devote to this activity? Will the time spent hurt them when it comes to their tenure decision. These are all good questions and perhaps when the chairs and deans are "on-board" with the program they will take such contributions into account.
In conclusion, I think BI is here to stay and with everyone doing their bit we may see a better informed society, more participation in our research community from under-represented groups, as well as an increasing enrollment in CS programs.
After organizing this meeting I also feel that the time is right for us to have a Dagstuhl type conference center here in the US. One small workshop type conference center on the east coast and a bigger conference center on the west coast for conferences that have 200 people.
ReplyDeleteThere are two ways of organizing conferences - have it on a Univ. campus in the summer, or go the route of using hotels. The cost of 5 star hotels is very high. Doing a conference on a Univ campus is a lot of work. We need a non-profit organization that runs weekly
meetings throughout the entire year. These can be CS workshops or workshops in other fields. I feel that this would be incredibly nice for having summer schools on advanced topics, as well as informal workshops.
"Another point that is worth discussing is how much time should an Asst. Prof. devote to this activity? Will the time spent hurt them when it comes to their tenure decision."
ReplyDeleteShould not broader impact take into account teaching and PhD students? That impact is much bigger than blogging or maintaining wikipedia pages. Why should "broader impact" only count things that are not part of a professor's job? Obviously, spending significant time on any of this stuff will hurt an assistant professor.
Outreach is important, but NSF is putting far too much emphasis on this. NSF is promoting this for publicity reasons, and it is hurting the research that should be NSF's focus. Maybe NSF should next require PIs help clean up the oil spill.
Samir, the Dagstuhl-style idea is excellent. How would it be funded ? If we expand to north america, then Banff provides some of this, I believe.
ReplyDeleteI don't want to sound too partisan about this, but I think that you're much better off establishing "theoryoverflow" as a section of MathOverflow than as a separate StackExchange site. I see very little difference between the social behavior of CS theorists and mathematicians, except which department happens to employ them. Don't be fooled by the fact that MathOverflow is a miraculous success; 95% of StackExchange sites are failures. The question is whether you want to be the 50th state or an independent Hawaii.
ReplyDeleteAs for "Broader Impacts": I understand the good intentions behind the Broader Impacts criterion I understand that it looks good from certain corners of NSF, and that Congress likes it even better and that that isn't a bad thing. Even so, "Broader Impacts criterion" continues to add confusion to the process of applying for an NSF grant. If there is a good way for the NSF to achieve these goals, what they have done isn't it.
ReplyDeleteBecause, we are already expected to explain how our work is important. Interdisciplinary work, technology, education, outreach, etc., are all already there, at least optionally. The NSF asks us to justify our work to the best of our abilities, and then with "Broader Impacts" to set that aside and provide another justification. It is like the scene in the Bugs Bunny cartoon Shishkabugs, in which the king declares "Every day the same thing --- variety! I want something different! Fix me hassenpfeffer!" Broader Impact is the "hassenpfeffer" of an NSF grant proposal.
Maybe the problem lies in just two letters of the slogan, "er". It might have been fine for the NSF to fund work with a broad impact. Under the current rules, however broad you said your work was, you also have to be broader. In practice the rules reward a certain kind of salesmanship in the grant proposal, aside from any effect on the actual research.
Here is a 10 minute recording of John von Neumann discussing the broader impacts of NORC (one of the first computers).
ReplyDeleteThe year is 1954. Von Neumann does a rather good job of making the case that these new-fangled "computers" will someday have a broad impact.
One reason is simple: von Neumann designed his research program to have broad impact.
It happens that Nicole participated in the "dissemination" part of Broader Impact and most of the comments have focused on that and the educational and participation components (which are the foci of the proposal caricature).
ReplyDeleteThese are important components but, it seems, not the only aspects of Broader Impact, at least as I understand the term in English. Societal impacts of the research itself and impacts on fields outside of the silo that is its funding target seem pretty important, though a lot harder to justify in the average proposal. (I notice that the impacts on other fields is NOT one of the criteria for Broader Impact in the NSF lingo, which has always been a bit puzzling to me.)
@Greg, you may very well be right. But mathoverflow is a huge site, and even with judicious tag filtering, I have to wade through lots of stuff to get to questions/answers that are of use to me.
ReplyDeleteMO has been very welcoming to algorithms questions, and I am very grateful for that. But for whatever reason, there are few theoryCS folk on MO, and the hope is that by creating a focused site, we get more participation. Already, I see a good number of people on the proposed site that I know from a research standpoint.
It's quite possible that I'll be wrong.
@Suresh: If these theorists devoted their pledges to MathOverflow, then it might well work. If you're worried about getting lost in the queue of questions, you could ask the MO admins to throw in the theory-flavored cs arXiv tags along with the default math arXiv tags. On the other hand, if you start a separate site, you may or may not reach critical mass.
ReplyDelete@Suresh: On the other hand, I don't mean to be too heavy-handed in my advice. If at first you don't succeed, you can try again with the other approach.
ReplyDeleteI think I agree with Greg. I've used mathoverflow once or twice, and often browse the questions and answers in several subject areas there. There is sufficient overlap between TCS and, say, combinatorics, graph theory, linear algebra, or asymptotic analysis, that it would be better to have TCS on the same site.
ReplyDeleteI would even suggest that there are several areas of mathematics covered on mathoverflow that are closer to TCS than they are to other areas of mathematics covered on the site.
However, if having a separate site meant more TCS people used it, then it is certainly worth giving it a shot.
There is a saying that "philosophical problems are never solved; they are dissolved/".
ReplyDeleteSimilarly, perhaps a good way to solve the "Broader Impacts" problem is to dissolve it by actually doing research that has broader impacts.
After all, given our planet's many urgent challenges, the history of the 21st century almost certainly is going to be mighty dismal, if we don't collectively succeed in doing research that has broad impacts.
Isn't this sobering 21st century reality the bottom-line reason that the NSF is focussing on "broader impacts"?
With the above motivation—and especially for that subset of Fortnow/GASARCH colleagues who take an interest in the history of math, science, and technology—I have borrowed from the 19th century literature, in particular the 1821 edition of Nathaniel Bowditch's New American Practical Navigator, to set forth the broadest and most optimistic 21st century "impact" scenario that I could reasonably conceive.
This "21st century broad impact" scenario is posted on Ian Durham's (highly recommended) blog Quantum Moxie, under Ian's topic The Sterilization of Science.
Of course, there are many other "21st century broad impact" scenarios that could be conceived. My main point is, that we had best start conceiving them, and making them happen.
And this is what the NSF (with prudent foresight) is now encouraging us to do ... fully recognizing that integrative "broad impact" research is among the hardest kinds of research to do effectively.
Thanks for all the comments folks.
ReplyDeleteAs I said there are many ways of achieving Broader Impact - some research naturally lends itself to accomplish this, for other research, sometimes it takes years for the researchers themselves to understand the Broader Impact the research can have. As long as we ask ourselves basic questions that we do not know the answers to, and enhance our understanding of CS, the
broader impact will follow (at least for some of the research). However, in proposals, we do have to be able to make the case effectively (more so in the future).
Suresh -- Bertinoro is a new conference center that has very quickly established itself -- however my impression is that it is not purely for CS meetings.
We need a place that is easily accessible by students, who can drive there and attend meetings relatively cheaply. For the NSF BI summit that we hosted the hotel negotiated bulk rate was 203$/night (+taxes etc). For some extra rooms we needed the rate hit 259$/night. Not to mention added fees that we paid the conference center for their facilities.
I heard from someone that NSF actually partially funds the Banff Research Station. If so, why not fund the opening of a center here?
I know some of us like to travel to more exotic locations, but really we need to put the center where the demand is.
One final point is that "Broad Impact" challenges often are messy ... not easily solved by proving a theorem, or inventing an algorithm.
ReplyDeleteA good example of a messy problem is being discussed today on The Oil Drum; a site that is emerging as "The Blog of Record" (in Lance's wonderfully memorable phrase) for energy professionals.
The Oil Drum's topic is 'The Renewables Gap: The Political Challenge of Affecting a Societal Transition to Renewable Sources of Energy Revisited'.
"Transition to Renewable Energy" is a good example of a "Broad Impact" challenge that (at the end of the day), the public looks to the math, science, and engineering community to solve.
As an aside, the traffic on The Oil Drum is about 100X the traffic on Computational Complexity ... and the level of mathematical and technical sophistication is approximately equal ... hmmmm.
What does computational complexity research have to say about the broad challenge of "Transition to Renewable Energy" (or any similarly urgent challenge)?
If the answer is "not much", or "it's not clear", or "we could explain, but it's too complicated for non-experts to understand" ... well ... whether it's fair or not ... those answers augur ill for the longer-term vitality of computational complexity as a key academic discipline.
One reason is that if we aren't pretty successful at meeting challenges like Renewable Energy, then in the long run—or perhaps even the dismayingly short run—there's not likely to be a viable profession called "complexity theorist", is there?
I posted (above) the link to John von Neumann's NORC lecture, as a good example of how to address these tough issues of public and professional outreach. It's true, though, that von Neumann's outreach efforts drew considerable criticism from his academic peers, for being excessively simplistic and/or conservative.
However John's brother Michael (with whom I used to talk when I was a graduate student) once told me that von Neumann's public persona was a distilled and carefully simplified construct, overlaid upon a more sophisticated set of internal values, whose deliberate simplicity enabled von Neumann to initiate and lead large enterprises.
So it is perhaps a good idea to ask: "How can the objectives of my discipline be distilled—even to the point of being over-simplified—in order to communicate more clearly, to both colleagues and the public, their relevance to urgent "broad impact" challenges?"
One final point is that "Broad Impact" challenges often are messy ... not easily solved by proving a theorem, or inventing an algorithm.
ReplyDeleteA good example of a messy problem is being discussed today on The Oil Drum; a site that is emerging as "The Blog of Record" (in Lance's wonderfully memorable phrase) for energy professionals.
The Oil Drum's topic is 'The Renewables Gap: The Political Challenge of Affecting a Societal Transition to Renewable Sources of Energy Revisited'
"Transition to Renewable Energy" is a good example of a "Broad Impact" challenge that (at the end of the day), the public looks to the math, science, and engineering community to solve.
What does computational complexity research have to say about the broad challenge of "Transition to Renewable Energy" (or any similarly urgent challenge)?
If the answer is "not much", or "it's not clear", or "we could explain, but it's too complicated for non-experts to understand" ... well ... whether it's fair or not ... those answers augur ill for the longer-term vitality of computational complexity as a key academic discipline.
One reason is that if we aren't pretty successful at meeting challenges like Renewable Energy, then in the long run—or perhaps even the dismayingly short run—there's not likely to be a viable profession called "complexity theorist", is there?
I posted (above) the link to John von Neumann's NORC lecture, as a good example of how to address these tough issues of public and professional outreach. It's true, though, that von Neumann's outreach efforts drew considerable criticism from his academic peers, for being excessively simplistic and/or conservative.
However John's brother Michael (with whom I used to talk when I was a graduate student) once told me that von Neumann's public persona was a distilled and carefully simplified construct, overlaid upon a more sophisticated set of internal values, whose deliberate simplicity enabled von Neumann to initiate and lead large enterprises.
So it is perhaps a good idea to ask: "How can the objectives of my discipline be distilled—even to the point of being over-simplified—in order to communicate more clearly, to both colleagues and the public, their relevance to urgent "broad impact" challenges?"
One final point is that "Broad Impact" challenges often are messy ... not easily solved by proving a theorem, or inventing an algorithm.
ReplyDeleteA good example of a messy problem is being discussed today on The Oil Drum; a site that is emerging as "The Blog of Record" (in Lance's wonderfully memorable phrase) for energy professionals.
The Oil Drum's topic is 'The Renewables Gap: The Political Challenge of Affecting a Societal Transition to Renewable Sources of Energy Revisited'
What does computational complexity research have to say about the broad challenge of "Transition to Renewable Energy" (or any similarly urgent challenge)?
If the answer is "not much", or "it's not clear", or "we could explain, but it's too complicated for non-experts to understand" ... well ... whether it's fair or not ... those answers augur ill for the longer-term vitality of computational complexity as a key academic discipline.
One reason is that if we aren't pretty successful at meeting challenges like Renewable Energy, then in the long run—or perhaps even the dismayingly short run—there's not likely to be a viable profession called "complexity theorist", is there?
I posted (above) the link to John von Neumann's NORC lecture, as a good example of how to address these tough issues of public and professional outreach. It's true, though, that von Neumann's outreach efforts drew considerable criticism from his academic peers, for being excessively simplistic and/or conservative.
However John's brother Michael (with whom I used to talk when I was a graduate student) once told me that von Neumann's public persona was a distilled and carefully simplified construct, overlaid upon a more sophisticated set of internal values, whose deliberate simplicity enabled von Neumann to initiate and lead large enterprises.
So it is perhaps a good idea to ask: "How can the objectives of my discipline be distilled—even to the point of being over-simplified—in order to communicate more clearly, to both colleagues and the public, their relevance to urgent "broad impact" challenges?"
Whoops! I'm very sorry for the repeated posts!
ReplyDeleteLance's site kept sending "invalid URI" messages, but it appears the posts were being processed anyway.
Lance/GASARCH, please delete the (inadvertent) duplicates as necessary.