My last post on the alleged P NE NP paper

(This is likely my last post on the alleged P NE NP paper unless more real news on it occurs. The only real news I can see happening at this point is a retraction.)

I have not commented much on the alleged P ≠ NP since I was waiting for better informed or more opinionated people to comment first. Now that the dust has somewhat settled here are some views of points of view and some pointers to points of interest.

1. The best places to read about the technical details of the problems with the proof are Lipton's Blog Entries: here, here, here, here, here, here and the Wikipedia entry here.
2. Some people are saying that Scott's post was nasty. Here is an excerpt from Scott's post:

   What's obvious from even a superficial reading is that Deolalikar's manuscript is well-written, and that it discusses the history, background, and difficulties of the P vs. NP question in a competent way. More importantly (and in contrast to 98% of claimed P ≠ NP proofs), even if this attempt fails, it seems to introduce some thought-provoking new ideas, particularly a connection between statistical physics and the first-order logic characterization of NP. I'll leave it to the commenters to debate whether Deolalikar's paper exhibits one or more of the Ten Signs A Claimed Mathematical Breakthrough Is Wrong.

   Scott, that is downright nasty! Calling a paper well-written!! Accusing it of having thought-provoking new ideas!! Scott, you are just one Nasty dude!!
   
   More seriously, Scotts offer of $200,000 if the proof is correct (more formally, if he wins the Mil prize) was considered nasty by some. Being skeptical and quantifing your skepticism is not nasty.
3. Other posts worth looking at:
   1. Written pre-Deo, Lance now looks like Nostradamus: So you think you've settled P vs NP. (NOTE- personal triumph- I got the spelling of Nostradamus right on my first try!)
   2. A pre-Deo post of Scott's on 10 signs a claimed mathematical breakthrough is wrong. Deolalikar passed most of these.
   3. A post-Deo post of Scott's that was likely inspired by Deo: 8 signs your proof of P ≠ NP is wrong. One of Scott's signs cuts both ways- he says that the use of Descriptive Complexity Theory is one of the signs. He claims (correctly) ... subtle differences in encoding can lead to HUGE differences in computational complexity. Indeed, this was one of the problems with the Deo-Proof. However, I think Descriptive complexity theory is one of the few techniques that has not been proven cannot work. So I take its use as a good sign.
4. Lance was on vacation when all of this happened so he posted on it when he got back. He was skeptical from the beginning and he says why. (See here.) People respond by saying that the paper inspired a lot of interesting discussion, and hence... I'm not sure- Lance shouldn't be skeptical? Lance shouldn't say he is skeptical? Lance should phrase his skepticism more politely? I'm not sure what the objection to his post really is.
5. Is the proof correct? It looks like there are enough serious flaws that the answer for now is no. By now this is old news.
6. I was originally skeptical (see here) since there has been no progress on P vs NP at all so its surprising that there is this much progress so fast. I do not know if this is valid reasoning. Has any serious hard math problem been solved all of a sudden after no progress on it? Has there truly been no progress? We might not know until we solve it in 3000 AD and look back and see which results from 1970-2010 ended up being relevant.
7. Deolalikar should, by Thursday Aug 26, 2010 (the first day of Barriers II) either (1) retract the claim, or (2) post a version that fixes ALL of the flaws found and explains the fixes. If he does neither of these two things then he will cease to be a serious researcher.
8. It would be impossible to have a version that fixes ALL of the flaws by Thursday Aug 26, 2010. Hence he needs to retract by then. (Some would say sooner. Much sooner. Like... now.)
9. Was all of this good for the community? It got some talking and the people who read the proof learned some complexity. If this gets more people interested in the problem, that's got to be a good thing. If some interesting ideas come out of this, even indirectly (e.g., Terry Tao is inspired to read up on finite model theory and comes up with a result of interest) then it will of course be a good thing. However, if more of these proofs come out and the community spends too much time on them, that will be bad.
10. Why did people take this paper so seriously?
    1. It was in LaTeX!
    2. It was long!
    3. It was by a respectable researcher. (More on that later.)
    4. It used familiar notation.
    5. It seemed to have some good ideas in. It may still. (More on that later.)
    6. It was on Slashdot and on Lipton's blog. But this begs the question: why was it on slashdot and why did Lipton take it seriously? Dick- I am not asking this rhetorically- if you read this please leave a comment about why you took it seriously. I am not asking this sarcastically or in any other nasty way. (I know YOU won't think so but other readers might.)
    7. Others took it seriously. Chicken and egg problem here.
    8. It had very few of the usual trademarks of papers written by cranks.
    9. It was a proof in the correct direction (a proof that P=NP would be taken less seriously, though the result, if true, would be more awesome!!!)
    10. In the age of the web, news travels fast (I know of 3 other serious people claiming to have shown P ≠ NP; however, that was pre-web so they were debunked before it got very far. I'm not quite sure which ones went public so I decline to mention their names.)
11. How respectable a researcher is Deolalikar? I do not know any other work that he is done. He is not someone you've heard about as having done excellent work. The fact that some in the community took his work seriously is an excellent sign that our community is not elitist. They took it seriously based on its merits. (The fact that some did not take it seriously is irrelevant.) ADDED LATER: The above was badly written and hence misunderstood. My point is that Deolalikar is not well known to the complexity community and hence the fact that many in the community took his work seriously speaks well of the community (or at least of those people) as not being elitist. Some of us WILL look at ideas from outside of the community.)
12. Are there interesting ideas in it? The answer from the latest comments by knowledgeable people on Lipton's blogs seems to be no. Oh well.
13. In the midst of all of this came the DUMBEST comment on a post of mine EVER!! In a recent post I noted that a recent (2007) article on number theory still referred to factoring as being easy. I meant to imply that this was bad- they should at least acknowledge the issues involved even if they don't care. One of the comments was

    Since when do mathematicians care about the notion of polynomial time?

    Note that this is the same week when Gowers and Tao, two Fields Medal winners, are carefully looking over an alleged proof that P ≠ NP. Other mathematicians were looking at it also. The notion that mathematicians do not care about polynomial time is just so 1990's.

But This One Is Different...

This summer I took a two part vacation: Touring Ireland July 26-August 5, with my wife to celebrate twenty years of marriage and a short trip to Santa Fe, Augut 9-12, to catch opera with my elder daughter. Let's talk about what happened in between.

On Friday, August 6, Vinay Deolalikar sent me and 21 of my closest frinds an email entitled "Proof announcement: P is not equal to NP". I took a quick look and white it did look better that most P/NP proof attempts (in the sense that Deolalikar doesn't make up his own notation), I spotted the following line in his write-up:

Distributions computed by LFP must satisfy this model.

Many P versus NP attempts work by claiming that a polynomial-time algorithm must act a certain way forgetting that an algorithm can completely ignore any semantic meaning in the problem. Since LFP is just a fancy way of saying "polynomial-time algorithm", it looked like Deolalikar fell into the same trap. So I filed the paper into my Why Me? folder as I have with many many others and figured that was the end of that.

As you readers all know, on Sunday the 8th the paper was slashdotted and Lipton, impressed that Deolalikar worked on infinite Turing machines (whatever they are), posted on the proof saying he was "certainly hopeful". People then flooded with me with emails, tweets and instant messages asking me about the paper.

I read over Lipton's post and took another look at the paper and still didn't see the semantic approach outlined in the paper as a viable attack on the P versus NP problem. At this point I knew people I trust would look over the paper carefully so I wouldn't have too. I couldn't keep completely quiet so I tweeted

Much ado about an unverified proof. The word "must" is troubling. I'll let others check it carefully.

In retrospect I should have been more negative. Kudos to Scott for taking a strong and risky stance.

The emails kept coming but I remembered my vacation message was still active and I didn't have to answer them. I went to Santa Fe and didn't check email until I returned.

There are two camps in our community on Deolalikar's paper. Those of us who saw Deolalikar's paper as just another in a long series of bad attempts at P v NP and wondering what all the fuss was about. And those who thought Deolalikar hit upon a new proof paradigm and despite the numerous problems, big and small, with the paper still hold hope something important will come out of it.

Deolalikar at this point should retract his P ≠ NP claim until he can address all these issues. In an email Deolalikar sent out to the gang of 22 on Friday the 13th, he restates his claim and overview of the proof and says he "fixed the issues in the finite model theory portion of the proof". He promises a revised version on his homepage in a couple of days.

Deolalikar is following the script, currently at stage 5 of the 12 stage process. Stage 6 will happen sooner than he expects.

P vs NP vs IEEE (Guest post by Paul Beame)

(Update on P vs NP: The proof uses Finite Model Theory which is sometimes called That stuff that Neil Immerman does.. Neil Immerman has found a flaw in it. See Lipton's Blog for more information. My prediction is still that something interesting will come out of this but NOT P ≠ NP. And for those of you who read my blog but do not follow Lance on Twitter (the empty set?) note that Lance tweeted Much ado about an unverified proof.. Of course, if we all felt that way then who would verify it? A PCP?)

Guest Post from Paul Beame!

While the complexity world is debating how much to invest in trying to extract useful information from the recent attempt at resolving P vs NP, there is another item that may be useful to complexity blog readers who also happen to be members of the IEEE Computer Society which sponsors the CCC, LICS, and FOCS conferences. Voting just opened in the election for IEEE CS officers including the President.

I'd like to alert voters to a write-in campaign for Joe Bumblis for president of the IEEE Computer Society.

As anyone who has run a CCC, FOCS, or LICS knows, the IEEE CS has had a lot of very bureaucratic rules for running conferences. Conferences are run by the IEEE CS Technical Committees (like the TC on Mathematical Foundations of Computing which does CCC, LICS, and FOCS and which I currently chair). These TCs are groups of volunteers who have little power in the current organization of the IEEE CS. These TC volunteers have long chafed against the bureaucracy, often with little response from the main organization. The main officers of the IEEE CS have been from the Publications and Standards part of the organization and these groups dominate the nominating committees. The TC side of the organization has been shut out. IEEE CS ran into financial difficulty for a variety of reasons unrelated to conferences and the result has been more bureaucracy based on the Standards mentality that the way to fix things is to set up more rules, which pit CS staff in opposition to TC and conference volunteers.

This year we finally succeeded in getting the IEEE CS to allow an option that should substantially reduce bureaucracy for conferences like CCC, LICS, and FOCS but the organization still resists anything like the flexibility for TCs that SIGs have within ACM and still is tied to bureaucratic rules. Many of the most active and thoughtful TC chairs have agreed that we need to push a write-in campaign to finally get someone from the TC side in a position of authority and Joe has agreed to serve. Joe has been vice-chair of the board of TC chairs and would bring a very different perspective that has a chance of shaking things up. By comparison one of the other candidates for president recently refused to authorize the LICS 2010 budget as part of the Federated Logic Conference (long after registrations had opened) because it was charging $10 too little for non-IEEE-members (or $8 too much for members, depending on how you count). It took my intervention to expose the ridiculousness of this position before he was overruled.

We need your vote now!

(NOTE FROM GASARCH- as someone who has run a CCC I can VERIFY what Paul is saying without pointing to Lipton's blog or using a PCP.)

Factoring in P ?

(Update on alleged P NE NP proof: There are some issues with it. See these posts on Lipton's blog: here and here and also see a Wikipedia site that (I think) Terry Tao set up here. )

I recently read the following: (Backdrop- the author had already defined p(n) to be the number of partitions of n.) NOTE- I use ki where the actual quote uses k_i. Looks better in html, though I am sure a better htmler than I could handle this.

For a positive integer n let d(n) be the sum of the divisors of n. For example

1. d(4) = 1 + 2 + 4 = 7
2. d(1000)=1+2+4+5+8+10+20+25+40+50+100+125+200+250+500+1000 = 2340
3. d(1001)=1+7+11+13+77+91+143+1001=1344

Unlike the numbers p(n), the numbers d(n) are easy to compute. There is a simple explicit formula for them. Namely, if n=2^k23^k3 ... p^kp then

d(n)=(2^k2+1-1)((3^k3+1-1)/2)...((p^kp+1-1)/(p-1))

They are assuming factoring is easy. Or perhaps they are assuming you are given the number already factored. There are no comments on what easy to compute might really mean, though they seem to think that having a simple explicit formula implies easy to compute.

To be fair, this book was written a while back before people in math really took these notions seriously. The book is Mathematical Omnibus: Thirty Lectures on Classic Mathematics by Fuchs and Tabachnikov. Copyright 2007.

Disclosure: I got my copy for free in my capacity as SIGACT NEWS book review editor.

That P ne NP proof- whats up with that?

Let me be he last on the block to tell you that an alleged proof of P ≠ NP is out there. NOT posting on it would be absurd; however, I cannot do any better than what Richard Lipton already posted so I point you to his post.

So what are my uninformed views? If I was a betting man I would bet that it won't pan out. I would bet that he proved something very interesting, but not P ≠ NP. Why do I think this? This is NOT based on the author who seems to be a person to be taken seriously. Its just that the problem is so hard and we've made no progress on it since... . Hmmm, when is the last time we made progress on it? Parity not in constant depth? Other bounds on weak models? Oracles? Natural Proofs? Something from Mulmuley's program? Fagin's theorem connecting the problem to finite model theory (which is used in the alleged proof)? I would have thought we would make slow progress at first. Not sure what type- Maybe SAT ∉ DTIME(n^k) for small values of k. Maybe something else.

Scott Aaronson apparently IS a betting man. See his post on the problem.

I was going to go to the Barriers in Computational Complexity II workshop. If the proof is correct I hope Amtrak gives refunds.

Actually- looking over the schedule, much of it will still be relevant. If the proof is CORRECT how will that change what we teach? What we work on? Lance's book-for-the-layperson on complexity theory? (I recently proofread a chapter on what the world would be like if P=NP. He may have to remove it. Too bad, it was awesome!)

The Solution to the Mark-Betty Game

RECALL from my last post the following game:

Let f(n) be a non-decreasing function from naturals to naturals. Consider the following game:

Let n be a large natural number. Mark and Betty alternate (Mark goes first) putting M's and B's on an n by n checkerboard. No square can have two markers on it. They play until all squares are covered; hence the game will end after n² moves. If at the end there is some row or column A where the number of M's in A is at least n/2 + f(n) then Mark wins! If not then Betty wins! (NOTE- the M's in A do not have to be adjacent.)

I asked for your intuitions on when Mark or Betty have a winning strategy. Here are the answers and a pointer to my writeup of it. NONE of this is my work- it is all the work of J.Beck (the reference is in the writeup). Here are the results:

1. If f(n) ≤ O(\sqrt(n)) then Mark has a winning strategy.
2. If f(n) ≥ Ω(\sqrt(n log n)) then Betty has a winning strategy.
3. My writeup is here. If I fill in more details it may change.

(NOTE- html's sqrt sign is pathetic. Here is what square root of n looks like: √ n. The sqrt sign has no top! That's why I use `sqrt')

My writeup of the first result is complete. I included more detail then you usually get in a math paper. I do not know if this is good or bad; however, I really wanted to check it all for myself. My writeup of the second result is much sketchier; however, it is essentially Beck's proof.

Beck has a book on games of this nature entitled Combinatorial Games: Tic Tac Toe Theory. Will children buy it thinking it is about tic-tac-toe? Maybe Bill Gates' kids: the book costs $146.00 new and $107.00 used.

I want your intuitions on this, so the less you know the more I want you to post

Let f(n) be a monotone increasing function from N to N. (CLARIFICATION ADDED LATER: N is the naturals., f is non-decreasing) Consider the following game:

Let n be a natural number which we thing of as being large. Mark and Betty alternate (Mark goes first) putting M's and B's on an n by n checkerboard. No square can have two markers on it; hence the game will end after n² moves. If at the end there is some row or column A where the number of M's in A is at least n/2 + f(n) then Mark wins! If not then Betty wins! (NOTE- the M's in A do not have to be adjacent.)

For which f(n) does Mark have a winning strategy? For which f(n) does Betty have a winning strategy? Much is known on this problem. I have written up some notes that I will post on Wednesday. (None of this is my work.)

What are your intuitions and why? If you already know the answers then please do not post. SO, paradoxically, the less you know the more I want you to post. I want to know what the intuitions of someone who does not know the problem are.

What is the complexity of these problems and metaproblems?

The following problem is from Doctor Eco's Cyberpuzzles. I have shortened and generalized it.

We are going to put numbers into boxes. If x,y,z are in a box then it CANNOT be the case that x+y=z. If you put the numbers 1,2,...,n into boxes, what is the smallest number of boxes you will need?

(CLARIFICATION ADDED LATER: I mean that x,y,z are ANY three numbers in a box. For example, 10,11, and 21 CANNOT all be in the same box. One of the comments thought they could by letting z=10, x=11, y=21. This is NOT what I intended.)

You can use a simple greedy algorithm to get some partition, but it might not be optimal. The following set is not NPC (by Mahaney's theorem) but also seems to not be in P: I suspect that the following set is not NPC and not in P:

{ (n,k) : there exists a way to partition {1,...,n} into at most k boxes so that no box has x,y,z with x+y=z }

There are many variants.

{ (n,k) : there exists a way to partition {1,...,n} into at most k boxes so that no box has x,y,z with x+y=2z }

This one I know is thought to be hard- it is asking for the min number of colors so that you can color {1,...,n} and not have any Monochromatic arithmetic sequences of length 3. This is an inverse van der Warden numbers; hence I am sure that it is not in P, and that there is no proof of this, and its not NPC.

Let E be a linear equation like x+y-z=0. We represent E by its coefficients., so E would be (1,1,-1). The statement E(a,b,c)=0 means a+b-c=0. Fix E on a fixed number of variables, say a. How hard is

{ (n,k) : there exists a way to partition {1,...,n} into at most k boxes so that no box has x₁, ..., x_a with E(x₁,...,x_a)=0 }

If we do not insist the set being partitioned was {1,..,n} we get more problems:

{ (a₁,...,a_n,k,E) : there exists a way to partition {a₁,...,a_n} into at most k boxes so that no box has x₁,...,x_m with E(x₁,...,x_m)=0 } I suspect this is NPC.

One can always use the Greedy Algorithm on the above problem, though it may not give the optimal answer. Consider the following meta problems: (``the problem'' can either refer to the version where we are partitioning {1,...,n} or a given set. Hence below there are eight problems, not four.)

1. { E : For the problem with E, Greedy gives optimal }. This is in co-NP.
2. { (E,a) : For the problem with E, Greedy gives within a*optimal }. This is in co-NP.
3. { E : the problem with E is in P }
4. { E : the problem with E is NPC }

For the last two I don't even know if they are decidable.

This Post is Quite Different from any you've ever read!!

I recently a letter from WETA (public TV) which I quote from:

This letter is quite different from any we've ever sent to you. For years we wrote to you about WETA's great programs and the need they filled in your life. Today, I must write to you about WETA's needs. And if friends like you don't respond to them, there will be far less programs to enjoy.

There is something wrong with this letter: I have gotten the exact same letter from them for about 4 years now. Hence the statement This letter is quite different from any we've send to you is not just false but verifiably false.

While I expect letters to exaggerate I do not expect to have easily verifiable lies that do not even help their cause. So why did they do this? I do not know. But whatever the reason, it is sheer incompetency. Hence I will not give to them. This raises the following question:

If a charity (or whatever Public TV is) asks for money they can exaggerate how much they need it. Should they?

1. Some readers will say GOSH, they really need the money! I better give!
2. Some readers will say They always need money. I am not going to bother.

We also have here a societal problem. Since many (legitimate) charities exaggerate about how dire their situation is or how serious their problem is, after a while it all gets tuned out. We have here a  Prisoners Dilemma problem- each one thinks (correctly?) that if they exaggerate their problems they will get more money. But if they all do it the public gets cynical and gives less money overall. (NOTE- I do not know this to be true, but I am curious. If anyone does know then let me know.)

How can they get out of this trap? I do not know. However, the least they can do is to not say things that are obviously false. There may be a well defined Game Theory or EC problem here. Or it may be a public policy problem. We won't know until its solved.

UPDATE IN May 2016: I keep getting these letters and got one today.

UPDATE IN July 2017: Got another letter today

UPDATE IN Sept 2017: Got another letter today

UPDATE IN March 2018: Got two in the month.

(I got some in between March 2018 and now but now I report it)

UPDATE IN JUNE 2022: Got one this month. 

A Seventh Mil. Problem

Richard Lipton had a wonderful post asking for a seventh Millennium Prize now that Poincare's conjecture has been solved. I posted a suggestion on his blog but got no comments. I'll expand on it and see if I get any comments here.

HISTORY: The original proof of VDW's theorem , in 1927, yields INSANE (not primitive recursive ) bounds on the VDW numbers. (Shelah (1988) later got primitive recursive bounds and Gowers (2001) got bounds you can actually write down!) Inspired by VDW's proof Erdos and Turan (1936), made two conjectures:

1. If A is a subset of N of positive upper density then A has arbitrarily long arithmetic sequences. Proven by Szemeredi in 1975 (see here for more.)
2. If &Sigma_{x ∈ A} 1/x diverges then A has arbitrarily long arithmetic sequences. (This conjecture implies the first one.)

A proof of either of these yields a proof of VDW theorem. The hope was that it would lead to a proof with better bounds. Szemeredi's proof of the first conjecture did not yield better bounds; however, Gowers proved the first conjecture a different way that did yield better bounds on the VDW numbers.

The second conjecture is still worthwhile since it may yield even better bounds and because it is interesting in its own right. So, I propose the second conjecture of Erdos-Turan as the 7th Millennium problem. (It might need a snazzier name. The Divergence Conjecture? The k-AP Conjecture? Suggestions are welcome!)

1. Greene and Tao have already shown that the primes have arbitrarily large arithmetic progressions.
2. The work that has gone into Szemeredi's theorem and the Greene-Tao theorem spanned many areas of mathematics. Hence this is not just an isolated problem.
3. The problem has been open since 1936. Hence it is a hard problem.
4. Will more connections to other parts of math be made? Is the problem too hard? A NO answer to both of these would make it not that good a problem.
5. The converse to the conjecture is not true. Note the following set:
   
   A = &cup_{k&isin N} {2^k + i : 0 &le i < k }
   
   The set A has arbitrarily long arithmetic sequences but If &Sigma_{x ∈ A} 1/x converges.
6. Is there a plausible condition that characterizes the sets that have arbitrarily long arithmetic sequences?
7. There is already (I think) a 3000 dollar bounty on the second conjecture. So the Clay Math Institute will have to just give 997,000 dollars.

CRA Snowbird Part II

Considerable discussion about funding at CRA Snowbird. Ken Gabriel, Deputy Director of DARPA, talked about how DARPA is restructuring its programs to become more university-friendly. They've made great progress though there are still some sticky issues of project-oriented proposals and security clearances. On a related note DARPA recently announced a new Crypto Program that may be of interest to the theory community.

Peter Harsha, the CRA director of public affairs, talked about NSF funding and how the renewal of the COMPETES act almost got derailed over pornography. NSF and CISE in particular did well in the administration's budget request but there is some uncertainty as we head into the fall elections.

The best part of the snowbird meeting is networking, talking to a number of CS leaders especially at the meals and breaks. The last session was small group meetings with current deans on how to deal with our own deans. Even though I'm not a chair I do find myself dealing with my dean and his staff quite often and we were able to get some good advice on quite a range of specific issues. Our group got lucky in matching up with  Dan Huttenlocher, Dean of Computing and Information Science at Cornell, and Martha Pollack, former Dean of the School of Information at Michigan. The best general advice: have a good working relationship with your dean and don't just ask or complain but really make the case on how the particular resource you need will benefit your school.

I'll be mostly on vacation and off the net for the next couple of weeks. Be nice to Bill while I'm gone.

CRA Snowbird Part I

I just returned home from my first trip to the CRA Snowbird Conference, the biennial meeting of CS chairs and other leaders in the CS community. I really enjoyed the short meeting and saw many old friends who are now chairs, some people I've only known by email and many I've talked to for the first time. There are a few theorists who have become chairs and deans, and, in the case of Jeff Vitter, provost at Kansas. Unlike theory conferences where I am usually one of the old people, most CS chairs are just about my age.

I, as I had to remind most people I met, am not a chair. I attended as a speaker on the Peer Review in Computing Research panel giving my usual spiel on how the current publication culture hurts the community-building aspects of conferences. Jeannette Wing made a great argument of how our deadline-driven research and conservative program committees may lead our field to lose its "vibrancy, excitement and relevance". 

A number of people talked about the big projects they work on which make me almost rethink having gone into theory. Almost. The need for better algorithms shows up in many of these talks. Yoky Matsuoka from U. Washington talked about the artificial hand her group developed that has the full range of motions of a human hand but they lack the algorithms to have the hand do even some simple natural tasks. Illah Nourbakhsh from CMU talked about building electric cars and his ideas of using a supercapacitor as an energy cache for batteries so the batteries become smaller and cheaper but hits challenging cache optimizing issues. The group is running a contest, best algorithm wins an electric car. 

Sally Fincher from the University of Kent gave a surprisingly strong talk Why Can't Teaching Be More Like Research? We get judged by our research based on how we compare to the global community but teaching is much more local and Sally talked about the implications of this distinction.

Most disappointing was the discussion on the NRC Rankings. Charlotte Kuh, who served on the NRC committee putting together the "soon" to be released report, said it will not give a specific ranking of each department but rather a range, like University of Southern North Dakota is ranked between 8th and 36th. And not just one range but five ranges based on different weights of the various criteria. And you can create other rankings based on your own choice of weights. All based on 2005-6 data. And they used citation data from the ISI which doesn't include most CS conferences. The CRA board talked them out of that but now the CS data and rankings will use no citation information at all. But even outside of CS, with multiple ranking ranges and old data, the NRC report will be of little value.

Do you want to review a book

I will be sending my next book review column for SIGACT NEWS off on July 28, 2010. It has LOTS of books on its BOOKS I WANT REVIEWED list. YOU get a chance to look at the book list and request one to review before the list goes out (if you do this then I will modify the list).

Here is a link to the list of books I want reviewed HERE. If you see a book you want then email me at gasarch@cs.umd.edu the name of the book and your postal address. We will then work out details over email- when the review is due, and whether me or the publisher sends it to you. I would like you to email me before July 28 so I can take those books off the list, but if you email after that date and the book you want is still available, that will be fine.

Before volunteering you should read my advice for reviewers.

Here is a LaTeX Template.

Factors for getting a job- Arbitrary, random, and complex

The Job Market in Theory (likely in all of academia) has more randomness and arbitrariness (are those the same?) then people may realize. Especially young PhD's who have never been to a faculty meeting where these things are discussed. The process is quite complex (is that the same as random and arbitrary?). I am NOT saying that merit plays no role. I am saying that its very hard to make a clean statement like Person X got a job an school Y because of Z.

With that in mind, I list out some criteria I have heard played a role in a hiring decision. My point is NOT to help you game the system (note that some of the factors I've heard cut both ways) nor to argue that the system is good, bad, or ugly. My point is only that the process is more arbitrary-random-complex then you might think. While its not all merit, its also not all who-you-know or politics.

I invite you to give leave comments on factors you have heard of, but to keep it civil please do not mention the people or schools involved.

1. Merit: This is itself ambiguous. More papers? Longer papers? Co-authors? How about take a sum where each summand is (number of citations)*(importance of paper)*(number of pages)/(number of coauthors). And then there is grant potential.
2. Does a postdoc have a better chance then a fresh PhD? A postdoc has had more time to increase his weighted sum mentioned under Merit, but the school KNOWS he has had more time.
3. Two-body-problems can be GOOD or BAD. Often a school does not have two positions.
4. Being a women can be GOOD or BAD (for getting a job).
5. I would like to say Being black can be GOOD or BAD (for getting a job) however since there are so few black PhD's in computer scientists applying for academic jobs, I have not heard any stories about this. (NOTE- I use the term black instead of African American since they need not be American.)
6. Being socially inept can be GOOD or BAD. How could it be good? It plays to the stereotype. He's so socially inept, he must be a genius. I DO NOT recommend pretending you are socially inept.
7. Speaking your mind can be GOOD or BAD. He'll be a leader in the community or He'll be a pain in the ass
8. Having a Blog- the jury is still out on this one. I have heard of a case where having a blog helped someone get an interview. (It was a serious blog about the field he was in. I can't imagine that complexity blog would help me get a job if I was looking.)
9. Spending too much time deciding what to have for lunch can be a negative. If he can't figure out what to have for lunch then how can he formulate a coherent research plan.
10. Area can be a factor- does a school need or want someone doing area X? This is sometimes formalized, for example the school has money targeted to hire someone who does, a particular area. (Side topic- Targeted positions- Good or Bad? Good in that you are not going to have to compare people in diff areas. Bad in that there may be someone really good in another area who you want to hire but can't.)
11. Subarea is a factor- for example, we want the kind of theorist who can talk to our people in systems.
12. Having a champion in the dept who is helping push your case HELPS unless the person doing the pushing is a jerk or not good at pushing a case.
13. Being an ex-convict probably hurts. If there was a theory genius who served 10 years in jail, that might be a negative. Might be mitigated if he could pull in some serious grant money. However, if the jail term was for embezzling grant money, then maybe not. Actually, the whole question might depend on what he was in jail for and is he now reformed.
14. A very big factor is how good is the job market when you get out. This may be a much bigger factor than anything else on this list.

How I Find Homework Problems

What do you get out of this paragraph (from Charlie Stross' The Atrocity Archives via Daniel Lemire)

The [Turing] theorem is a hack on discrete number theory that simultaneously disproves the Church-Turing hypothesis (wave if you understood that) and worse, permits NP-complete problems to be converted into P-complete ones. This has several consequences, starting with screwing over most cryptography algorithms—translation: all your bank account are belong to us—and ending with the ability to computationally generate a Dho-Nha geometry curve in real time.

I get a new homework question.

Show that NP-complete = P-complete if and only if NP = L.

Wave if you solve it.

Sparse problems in NP thought to not be in P

(This post is similar to this old post. I am posting this anyway since when I first posted I made fundamental mistake. I fixed it the point I was trying to make get lost.)

About once a semester I get asked

What are the natural problems that are in NP, not known to be in P, and not known to be NPC? What is known about them?

And I give the standard answers:

1. Graph Isomorphism. If its NPC then PH collapses so it is though to NOT be NPC. There is no real consensus about its status with respect to P.
2. Factoring. If its NPC then NP=coNP so it is thought to NOT be NPC. Most people think that it is NOT in P since people really want to solve this one and have not been able to. This is not a rigorous argument. Factoring is in QP. If quantum computers are ever practical then we may need to rethink how we think about these things.
3. Discrete Log. Actually, are there reasons to think this is NOT NPC?

The answer above are standard and I suspect most of my readers know them. However, there is a large source of problems that are in NP, likely not NPC, likely not in P, that people don't seem to talk about much: SPARSE PROBLEMS that are thought to be hard. The ones I know about are from Ramsey Theory. I give one example but there are many like it:

{(1ⁿ,1^m,c) : the n×m grid can be c-colored and not have any mono squares}

1. This is clearly in NP since the coloring itself is the witness.
2. Since this is a sparse set it is not NPC unless P=NP. (Actually it can be coded as a tally set.)
3. I want to say People think this is hard. You may ask Name Three of Them! Indeed, the number of people who work on Computational Ramsey Theory is fairly small. However, Ramsey Theory itself has many people working on it and nobody has anything close to a result indicating that this problem is in P. I personally think that it is hard.
4. For a fixed c there is a finite number of grids G1,...,Gp such that the n×m grid is c-colorable iff none of G1,...,Gp can fit inside the n×m grid. Hence, for fixed c, the problem is in P. (So the problem is Fixed Parameter Tractable.)
5. Everything said above holds if you replace squares with rectangles or other configurations in the above.

Questions:

1. Are there other sparse problems in NP that we think are NOT in P? (that is, ones that DO NOT come from Ramsey Theory).
2. Should we be teaching these in our complexity classes as examples of possible intermediary problems? The proof that they are likely not NPC is easy. However, to argue that these problems are likely not in P is hard. Also, these problems may appear unnatural to some students. (They may appear unnatural to some of my readers.)
3. It is easy to argue that Factoring is probably not in P since you can point to the fact that people REALLY want to solve it quickly and so far have not. This is not a rigorous argument but I do count it as evidence. Is there a similar argument for GI? Is there a similar argument for my Ramsey Problems?
4. Is there a mathematical reason to think that Factoring, GI, or my Ramsey Problems are not in P?

GCT Workshop (Guest Post by Josh Grochow)

Last week, the Center for Computational Intractability hosted a Geometric Complexity Theory Workshop. Geometric complexity theory (GCT) is an approach to P vs NP and related problems proposed by Ketan Mulmuley and Milind Sohoni, in which they were naturally led to using techniques in algebraic geometry and representation theory. This workshop was the most recent attempt to explicate GCT to both mathematicians and computer scientists, and discuss recent related results. Without looking at the attendance list, my impression was that the workshop was approximately 3/8 complexity theorists and 3/4 classical mathematicians (meaning that about 1/8 fell into both camps), and was about 1/3 students and 2/3 postdocs/professors. This turned out to be a pretty good mix.

The first two days were scheduled to be all Ketan all the time. Despite the fact that Ketan is an excellent presenter, talking for two days straight and keeping the audience interested would be a tough task for anyone. Not everyone made it to the end, but a significant fraction of the audience did. Ketan's lectures were interspersed with a lot of discussion and debate, including a couple short impromptu guest lectures from audience members on some background topics. Pointed questions came from both classical mathematicians and complexity theorists, and as often as not were answered by audience members from the other field. It was refreshing to see the humility of the giants of multiple fields: great classical mathematicians asked complexity theory questions that a complexity undergrad could answer, and vice versa. I think this is one of the few times I have ever heard of serious classical mathematicians mingling with complexity theorists, and I think it worked quite well. This type of interaction and more seems necessary for the GCT program, and can only be a Good Thing for both fields.

And now for two technical items from the workshop.

Avi Wigderson and others (sorry to the others, but Avi sticks out most in my memory, and I didn't learn everyone's names!) repeatedly suggested applying the techniques of GCT to easier problems than P vs NP and see if they can be pushed through to fruition for easier lower bounds. At the moment, GCT reduces P vs NP to certain hard conjectures in algebraic geometry and representation theory. The corresponding conjectures arising in easier lower bound problems would hopefully be easier, maybe even easy enough to solve in our lifetimes! Peter Burgisser has been looking at matrix multiplication using these techniques (an idea originally suggested by his adviser, Volker Strassen, long before GCT came along), and his student Christian Ikenmeyer gave a presentation on their results as one of the research talks on the third day.

Christian's talk was probably the most salient for complexity theorists not intimately familiar with GCT, so I'll mention a bit more about it to finish off the post. GCT introduces the idea of a "geometric obstruction" (=some type of representation-theoretic object) to "there are circuits of size n^c for SAT up to length n" (NP vs P/poly). If a geometric obstruction exists for all (or infinitely many) input lengths n, then P is not equal to NP. The conjectures arising in GCT imply that such obstructions exist. But even in smaller problems, such as matrix multiplication, no one had ever seen such geometric obstructions! Peter Burgisser and Christian Ikenmeyer did computer calculations and found geometric obstructions to the border-rank of 2x2 matrix multiplication. (The border-rank of 2x2 matrix multiplication roughly captures how many multiplications are needed to approximate 2x2 matrix multiplication, in a specific sense.) The geometric obstructions they found show that the border rank is at least 6 (it is known to be 7, by a significant algebro-geometric result of Joseph Landsberg). Although this only proved a weaker result than what was already known, for a comparatively easy lower bound (compared to P vs NP), this is an important proof-of-concept for the GCT approach of using geometric obstructions to prove complexity-theoretic lower bounds. This work also raises the intriguing possibility (also suggested in the GCT series of papers) that one might be able to "verify P neq NP up to length n" for larger and larger n, the same way people verify the Goldbach Conjecture or Riemann Hypothesis up to a certain numbers of integers/zeroes. Unfortunately, doing this in the GCT setting even up to n=10 seems to take too much time (whereas Goldbach and Riemann have been verified for millions of integers/zeroes).

Finally, I should mention even in the case of 2x2 matrix multiplication, the conjectures that arise seem almost as difficult as the ones arising in the P vs NP problem. Avi and others suggested we look at even easier problems.

UPDATE 7/20: Slides of the research talks are now available here, and videos will be added soon.

Can you ever be denied Full Prof? Can you ever really fail a PhD defense.?

1. Is it possible for someone to be denied Full Prof? Yes, but it is rare.
2. Is it possible for someone to fail a PhD defense? Yes, but it is rare.

These questions are similar. Why might either happen?

1. I suspect that the most common reason for someone to be denied Full Prof is that they went up without the Chairman's or the Departments approval. That is, they insisted on going up. I really cannot see a reason for a professor to do this except some sort of bad logic which I will discuss later.
2. I suspect that the most common reason for someone to fail a PhD defense is that they insisted on defending even though the advisor didn't think they were ready. While the advisor should have stopped it there may be other reasons (e.g., a job offer to the student has been made so he really needs the PhD NOW, or some deadline is coming up) that came into play. Could also be bad logic which I will discuss later.
3. Could also be because of politics. I've heard of this happening but never really saw a case I could verify myself. One problem: Everyone who is ever denied Tenure or Full Prof says that it was political. Hence its hard to tell when it really is.

BAD LOGIC:

1. A prof might think Everyone who I've ever seen go up for Full Prof has gotten it, so all I need to do is go up for it.
2. A student may think Everyone who I've ever seen defend their thesis has passed so all I need to do is get a defense scheduled. I have seen the following: nobody fails a defense for several years because the adviser don't put you up until you defend. Then the mentality sets in that all you need to do is defend and you'll pass. Then someone pushes this and ends up failing. THEN people are scared for the next few years, until its forgotten. That is why someone fails a PhD defense every 6 years or so.

WHY DENY SOMEONE FULL PROF? If you deny someone TENURE they LEAVE. Hence something is accomplished. But if you deny someone Full Prof they are still there, just annoyed. Hence it seems like a really bad idea to deny someone Full Prof.

WHY DO YOU WANT TO BE A FULL PROF? Salary increase? My school is now not giving raises. You get to write letters for more people. Is this really a good thing? You get to be on more committees. Is this really a good thing? You get more respect? This is questionable. People stop asking So, are you a full prof yet? This is a good thing.

HOW BAD IS IT: In both cases you can re-do. That is, you can come up for full prof later and you can fix your thesis and defend later. Even so, it can be a trauma.

FYI: I passed my PhD defense first try in May of 1985. I got Assistant prof in 1985, Associate Prof (Tenure) in 1991, and Full Prof in 1998.

Conflicts of Interest

a conflict-of-interest? Some thoughts.

Thought One

PROF: I can't vote on Professor X's Full Prof case since I have a conflict.

CHAIRMAN: (There are not that many Full Profs around so he is concerned about having a quorum.) Really? Whats your conflict?

PROF: My wife works as an F.R.A (Faculty Research Assistant) for Prof X.

CHAIRMAN: Is that really a conflict?

PROF: (Surprised) Uh--- I really think it is.

CHAIRMAN: Which way would it bias you?

PROF: (Even more surprised) I don't think that matters.

The odd thing is that it really is hard to say which direction it would bias PROF in. Does the wife like her job? Does she know that the prof is really good at what he does? Really bad at what he does? It could go in any direction.

Thought Two

I have reviewed two books that my advisor wrote in my SIGACT NEWS column. For his excellent books BLOWN TO BITS I have at the beginning of the review:

Disclaimer: Harry Lewis, one of the authors, was my PhD advisor.

That seems fair. However, the reader may wonder which direction the bias goes. In my case I thought Harry was an excellent advisor (Disclaimer: I also know that he reads this blog). But what if I thought he was a terrible advisor? I wonder if the reader has a right to know which direction the conflict goes in. Perhaps disclaimers should be of the following form.

Disclaimer: Harry Lewis, one of the authors, was my PhD advisor; however, I believe this review is unbiased.

Disclaimer: Harry Lewis, one of the authors, was my PhD advisor; He was an AWESOME advisor. Hence this review may be positively biased.

Disclaimer: Harry Lewis, one of the authors, was my PhD advisor; He was a TERRIBLE advisor. Hence this review may be negatively biased.

(The advantage of the last one is that if it is a positive review then the book must be really really good.)

Thought Three

Lets say I was having the NSF theory director over to my house for dinner on his birthday. Since he may give me a grant someday, should I charge him for it? How about a compromise- he pays for the dinner but I pay for desert. I think the rule may be that I can spend less than ten dollars over the course of the year. (Richard- I hope you enjoyed your birthday desert, since that's it for the year.)

Drowning in Data

At a CCC Council meeting last week, a common theme emerged. The Army is "swimming with sensors and drowning in data". Biologists are considering trashing some of the DNA sequences they have already acquired because it will likely be cheaper to resequence in the future maintain the current data. We've been collecting tons of information, what should we do with it and if we don't have the tools yet to deal with all this data, should we bother keep it?

This reminds me of one of my favorite homework problems in complexity: Given two log-space computable functions f and g (the read-only input tape and write-only output tape don't count for space), show that the composition f(g(x)) is also log-space computable. The direct approach doesn't work for you don't have the space to store g(x).

The solution is to recompute the bits of g(x) as you need them. The lesson: When storage is expensive, it is cheaper to recompute what you've already computed. And that's the world we now live in: Storage is pretty cheap but data acquisition and computation are even cheaper.

So who says complexity has nothing to say about the real world?

Theory Happenings

FOCS accepts, abstracts and PDFs. The conference itself will be held October 23-26 in Las Vegas.

There is a proposed TCS version of Math Overflow, a Q&A site to let the crowd help your research. The site needs your commitments or it won't happen. More on the Geomblog.

I plan a post about NSF goings on after some expected announcements of new programs and personnel but just a reminder that the CAREER deadline is fast approaching, July 20 for CS.

Is this solution cheating?

Consider the following problem:

A hole is drilled through the center of a sphere. The cylinder-with-caps is removed. The length of the removed cylinder (it also has caps on it which do not count for the length) is 6 inches. What is the volume of the remaining solid?

There are two ways to do this problem.

1. Here is the solution using calculus:
2. Here is a solution which you may consider cheating. The very asking of the question implies that the answer can be determined from the data given. Hence we can CHOOSE an instance of the problem and KNOW that our solution for this instance is always the solution. We choose to have a cylinder of radius 0 (so its just is a line of length 6). Hence the answer is the Volume of a Sphere that is 6 inches in diameter: (4/3)(π)3³=36π.

There are two ways this may be considered is cheating.

1. Minor one: Deriving the volume of a sphere itself requires calculus so I didn't really get around that issue. However, the Volume of a sphere is well known so I think this is a quibble. (Does anyone know a non-calculus proof for the formula for the the volume of a sphere?)
2. Major one: We used the fact that the answer can be determined from the data to find the answer. Is this appropriate?

How would you grade this if given as an answer on an exam? Here are some thoughts:

1. If you put this on an exam what would you do if a student had this solution? Reward them for thinking outside the box or penalize them for not showing they know calculus?
2. What if it was on a mathematics competition?
3. Best solution might be to make it a multiple choice question so they do not need to show how they did it. Those that think of the clever solution are rewarded by spending less time on it--- unless it took them a long time to think of the clever solution. Those that do it via calculus also get it right. You might want to make one of the choices Cannot be determined from the data given.

Broader Impacts

Nicole Immorlica reports on the NSF CISE Broader Impacts Summit held last week in DC.

We've all seen it. Most of us have even written one. I'm talking about that ``clearly marked paragraph'' in the summary page of each and every NSF proposal:

Broader Impacts This proposal has far-reaching impacts. As part of my program, I will develop a new graduate course entitled My Research Area, that will introduce students to cutting-edge research in Proposal Topic X, Y, and Z. I will also incorporate these lectures into some of my Undergrad Courses. Special attention will be given to recruiting Women and Minorities. And I may even talk to A High School Student once in a while. Yada yada.

As often written, these broader impact sections, like the one above, read like the teaching section of my job description. So then, what is a broader impact, really? How can we improve our impact? And, most importantly, why should we, as a community, care?
I am now on an airplane returning from an NSF summit organized by Tracy Camp, Juan Gilbert, Judy Goldsmith and Samir Khuller (kudos to you) that discussed just that. The summit consisted of about 100 members of the CISE research community, and the purpose was to collect input from us about what we want broader impacts to be and how they should work. My working group was tasked with fleshing out Broader Impact #4, ``Broad Dissemination and ...'' (who knew, there are in fact five types of broader impacts, contained in a bulleted list in some NSF document from 2007, and yes, they all have long unmemorable bureaucratic names). Here's what we had to say (disclaimer: all comments are colored by my own personal biases and are not intended to accurately reflect the opinions of the participants etc. etc.):

1. What is a broader impact? All sorts of really cool things count here. In our group on broader dissemination, we came up with: blogging, YouTube video clips, maintaining wiki pages, writing a textbook, a popular science book, directing a play about science, designing a museum exhibit building computers with kids, with senior citizens, talking directly to the curious public in Scientific Cafe, with K-12 at National Lab Days, talking to the media, writing your representatives... Many of these have been done before, and I think there will be a link on the NSF website sometime soon giving pointers to some wonderful examples. More generally, a good broader impact is realistic and, ideally, measurable — points which ought to be discussed in the proposal.
2. How can we as a community help our individual members improve their impact? Some people have an internal fire that is fueled by helping others, and for those we can enable their impacts by simply making it easier to give. For this, the NSF will provide lists of ideas, and several programs like National Lab Days and BPC further help by providing ``match-making services'' that give a searchable interface to existing outreach opportunities (looking around there, I found a local high school that wants someone to come talk about careers in science, for example). Then there are also carrots and sticks. The carrots are higher weight for broader impacts in the review process as well as the tenure process (ummmm, I'll believe it when I see it); and the sticks are holding PIs accountable for their proposed impacts through annual reports alongside a threat of withheld funding upon failure to attempt said impacts (spank spank).
3. Why should we care? If you haven't figured it out by now, I am an incredibly cynical and suspicious individual. While I personally care about certain types of community service, I nonetheless felt that broader impacts in a proposal were simply a nod to Congress, a necessity that allows our elected representatives to justify giving us hundreds of millions of tax dollars, and of minimal importance in funding decisions and career success. I now see that, while there is a certain grain of truth in my snide remarks, the NSF is on a serious mission to change all this, and we should be too. We have a passion, and as privileged members of humanity, we have a duty to share our passion with those around us, thereby enriching their lives as ours were enriched for us by circumstance and chance and past generations of great givers.

The P vs NP quiz Show. NP! NP! NP!

Some random thoughts about quiz shows.

THOUGHT ONE: There could be a quiz show based on P and NP. We all think that FINDING an answer is harder than VERIFYING an answer. We use this! The contestant gets a question (or perhaps a category of questions) and is asked

P or NP ?

NP means that they will try to ANSWER the question. P means they will get to hear a POSSIBLE answer and VERIFY IT. NP is worth more money, perhaps alot more. How to verify?--- a few ways are possible.

1. The host gives the correct answer and the contestant either says (1) OH, I knew that- here is how I knew it (and then gives an explanation of how they knew that). Thus the contestant must verify that he can verify. If so, they get some money. If not then perhaps a bit penalty like lose ALL of their money or get an electric shock. (2) OH, I didn't know that. No money but no penalty.
2. Hook the contestant to a lie-detector and if they say claim Oh, I knew that and they are not lying then they get some money. If they claim they knew it and are lying then they are already set up to get an electric shock, so do that.

Get the audience to yell NP! NP! NP! --- rooting for the contestant to GO FOR THE BIG MONEY!

Could there be quiz shows based on other complexity classes?

THOUGHT TWO: On the show Cash Cab after contestants have finished they can either WALK AWAY (with money that is usually between 400 and 1500 dollars) or risk it on a DOUBLE OR NOTHING video bonus question. Should you risk it? There are two parameters X and Y.

1. If you didn't do that well, so you have LESS THAN X, then you are off your game and shouldn't risk it.
2. If you won a lot of money, at least Y, then you should just be happy and walk away with it.
3. If you won between X and Y then do the video bonus question.

I am risk averse, so X ≥ Y and I would never do the video bonus question. How about you? It might matter how much 400 or 1500 dollars is worth to you--- so you may very well be a risk taker in Cash Cab but not in Deal or No Deal where the amounts of money are much larger and thus more likely to be life changing. So X and Y might be functions of, not just how risk averse you are, but also on how much money is at stake. Could this be modeled?

THOUGHT THREE: When you watch a quiz show where there is only one contestant (e.g., Who wants to be a Millionaire or Cash Cab) and you yell out an answer, and the contestants yell a different answer, who do you hope is right?

1. I hope that they are right.
2. Almost everyone I've asked disagrees: That is, if I ask Alice, Alice hopes that Alice is right and the contestant is wrong.
3. If I was watching with someone I wanted to impress then I might hope that I am right.
4. If I like (dislike) the contestant then I hope they get it right (wrong) ind of what I yelled out.
5. So- what do you hope for, and why?

I expect to see a paper at INNOVATIONS on any or all of these topics.

The Lefthanded Latina Lesbians in Algebraic Topology Workshop

There was a Women in Theory Workshop at Princeton From June 19-23. ***SORELLE***, who was there, has some intelligent and interesting things to say about it here.. I will, instead, offer up a question.

I was at a Ramsey Theory Workshop in Italy and it was noted that of the 27 people there, none were women. Is there a shortage of Women in Ramsey Theory? Is this a problem? I suspect there is a shortage, however Ramsey Theory is too small an area to worry about it. Hence I doubt there will ever be a Women in Ramsey Theory Workshop.

Should there be a African-Americans in Theory Workshop? There are so few that it would end up being a bunch of non-African-Americans discussing why there so few African-Americans in Theory. To have a workshop analogous to the Women in Theory Workshop you need a larger critical mass. How many African-Americans are in Theory? What if I drop the requirement they must be American? I honestly don't know but I suspect its not that many. (If I am wrong then please let me know in the comments.)

Let A be an area (e.g, Recursive Algebraic Topology) and P be a subset of people (e.g., Left handed Latina Lesbians). When does a on P in A Workshop make sense? If A is too small then it would not make sense. If there are very few P's in A then there really can't be a such a workshop unless its mostly non-P's discussing why there are so few P's in A. If P is too big then there may not be a need for such a workshop; however, there may still be if there are issues facing P's that are not facing non-P's.

1. There have been African Americans in Mathematics Workshop. Here A is big enough and P is above critical mass.
2. According to this 24% of all lawyers are female and 44% of all law students are female. There may still be a need for workshops for female lawyers as they may face issues that male lawyers do not. Here was one: A Transformational Workshop for Women Lawyers.
3. Male Nurses are few in number, but there are some and I suspect they have problems that women nurses do not have. I do not know if they have workshops, but they do have their own magazine here.

What other workshops or conferences or whatnot on these types of issue have there been?

Most conferences hope that they will last a long time (e.g., We just had the 25th CCC. I hope there is a 50th). I suspect that the participants in the Women in Theory Workshop hope that these workshops are eventually not needed.

Talking about your work with a layperson

How to best describe what we do to the layperson? It depends on what you mean by layperson.

I was in Austin Texas visiting my nephew Jason. I was also giving a talk at the University. My nephew fixes forklifts for a living and does not care about abstract things. (I'd be as bad at his job as he would be at mine.) He asked me what the talk was on. I think he asked just to be polite but did not really care. Even so, I tried to put it in as concrete terms as possible. My thought process: (1) Don't use n. Use an actual number, say 100. (2) Don't state a theorem. State an easily grasped corollary.

BILL: Picture that there is a 100 by 100 chessboard. Picture that you want to put queens on the diagonal so that every square of the board is either covered or under attack. What is the least number of queens you need for this? (NOTE: I leave it to my readers to prove that, for an n x n chessboard, this is IDENTICAL to finding large 3-free sets, that is, large sets that do not have arithmetic progressions of length 3.)

JASON: That is the dumbest problem I ever heard!. First off, chess is played on an 8x8 board. Secondly, only once while playing chess did I ever get my pawn to the end of the board to get a second queen, never mind getting n-o(n) queens (he didn't put it that way). And thirdly... who cares?

BILL: I don't suppose it would help to tell you that this relates to some deep mathematics of interest.

JASON: Of interest to who?

BILL: Uh... Never mind. (I thought about telling him that better bounds on the VDW numbers could get you $3000 dollars but decided not to.)

I usually do better than this with very concrete easily grasped statements. But he doesn't care about this problem. And actually, why should he? (I am sure some of my readers don't care about this problem either.)

The Future of STOC

I set up a new blog, Future of STOC to discuss role of our main conference and how to achieve it.

At STOC this year we had a relatively large attendance of 350 but why not 1000 or more? Why isn't our flagship conference bringing the theoretical computer science community together?

A couple of months ago I wrote up a proposal to address these issues and circulated them to a number of people in our community. You can read the proposal and the responses on the blog. Based on those responses the SIGACT Executive Committee decided to slowly increase the maximum number of acceptances but also open up the discussion at the STOC business meeting, which we did. Many people at STOC asked me to post the proposal and the responses and open up the discussion to the broader community. That's the purpose of the new blog.

We want to hear from you, the theory community. Tell us if you love STOC the way it is. Or how can we change STOC to make it more relevant to you? Even if you would never plan to come to STOC, tell us why.

Go to the Future of STOC and join the conversation.

Foundational ... or simply a curiosity (Guest Post by Vijay Vazirani)

(Guest Post by Vijay Vazirani)

Foundational ... or Simply a Curiosity?

Conventional wisdom has it that whereas linear programs have rational solutions, nonlinear programs have irrational ones. The discovery, in recent years, of increasing numbers of nonlinear convex programs that always admit rational solutions is challenging this long-held viewpoint. Furthermore, these convex programs capture the solutions to some fundamental problems in mathematical economics and game theory, e.g., market equilibrium under linear utilities.

As if that were not enough, these convex programs also admit combinatorial polynomial time algorithms for computing their (exact) optimal solutions; very recently, even strongly polynomial algorithms have been found. I am writing this guest post at the suggestion of several people who were not aware of these developments.

The main question that arises is: Is this phenomenon simply a curiosity or is it indicative of a grand, new chapter in the theory of algorithms? Let me put forth some reasons to believe that the latter may be the case.

Most of us are familiar with the notion of integral LP's -- LP's that behave like integer programs, in that they always admit integral optimal solutions -- and the key role they played in the birth and blossoming of combinatorial optimization. Indeed, some of the most elegant and foundational algorithms (e.g., for matching and max-flow) and algorithmic ideas (including the notion of polynomial time solvability) were discovered within this field.

Of course, not every combinatorial optimization problem admits such an LP; in particular, none of the NP-hard ones do. The latter were tackled via LP's that admit near-optimal integral solutions; their study lies at the core of the field of approximation algorithms. Again, it is remarkable that for many fundamental problems, their hardness of approximation is captured exactly as the integrality gap of such an LP (or SDP).

In view of this larger picture, and the clean, canonical and elaborate structure that was exploited in obtaining the combinatorial algorithms, the discovery of rational convex programs seems more than a curiosity, though only time will tell. However, rational convex programs are not likely to be found for more than a couple of handfuls of problems, as was the case for integral LP's. If so, where does this take us?

My best guess at present is that we need to seek combinatorial algorithms for finding near-optimal rational solutions to nonlinear convex programs; the motivation being that as always, we expect such algorithms to yield a wealth of valuable insights. In the past, such insights were crucially used for advancing the theories mentioned above. Once again, only time will tell if this is the right way of building on rational convex programs -- and of course, we should not forget that mathematics has its own agenda and its own ways of throwing surprises at us!

For further information, please see the introduction to the following paper here and references mentioned therein; the latter are available on authors' web sites.

CCC 2010

( Reminder:Deadline for submitting to special issue of Theory of Computing in honor of Rajeev Motwani is July 30. See here. )

CCC 2010!

1. Ran Raz gave an AWESOME invited talk on Parallel Repetition of two player games ( here are the slides). What was most striking to me is that this line of research lead to the solution of a math problem (the Foam Problem) which would seem to be completely unrelated to it. Has this happened before- research starting in TCS leads to the solution of a seemingly unrelated math problem? Of course, my question is ill defined since TCS, Math Problem and seemingly unrelated are not well defined.
2. The talks following Ran's were also on two player games so his talk was a great way to get you up to speed on what follows. I would recommend that the Program Committee try to do this in the future: have the invited talk be the first in a session on related material. (I also realize that this may be hard to pull off.)
3. Subhash Khot gave a good invited talk on The Unique Game Conjecture. (These Slides should soon be available soon on his website. There is already a written survey on the material on his website and also slides he gave at FOCS05 on UGC. Here is his webpage.) This conjecture has been a great breakthrough in that it gives us something reasonable to assume that gives approx lower bounds- some of which match the approx upper bounds. It has also had many other applications unforeseen even by Khot when he first conjectured it. (Over dinner Khot told me he is only 90% sure that it is true. Gee, I thought he would have more confidence than that!.)
4. The talks that followed Khot's talk were all on UGC, and that was a big win.
5. Oded Regev gave great invited talk on The Learning with Errors Problem. (Talk and paper are here.) This was a talk where I learned about a problem I had not heard of and its connections to other problems. One of the applications was crypto (not surprising- that is Regev's main area.) The talks that followed it did not connect to it as much as for the other invited talks. This does not take away from the fact that I learned stuff of interest that I did not know ANYTHING about before.
6. Jakob Nordstrom gave a nice talk On the relative strength of pebbling and resolution. I had not known that pebbling was used to give lower bounds on resolution and I look forward to reading his 200 page survey on this topic. (It is on his website.)
7. Eric Allender had a paper in the first CCC (called STRUCTURES then) on Auxiliary PDA's. Eric Allender had a paper in the 25th CCC on Auxiliary PDA's. An Auxiliary PDA is a PDA with an additional log space workspace. I look forward to an Eric Allender paper on Aux PDA's at the 50th CCC. Note that Eric has worked on many other things as well.
8. Hrbues, Wigderson, Yehudayoff had a paper on Relationless Completeness and Separations. The idea here is to take algebraic complexity and see if you can get lower bounds if you do not assume that the operations are associative or commutative. The good news is that they did! The bad news is that the bounds for the case where the operations are associative and communicative are as hard as they were when Valiant first proposed this model many years ago. Valiant was at the talk and told me later that he would have thought that there would be more progress by now.
9. Alexandra Kolla had a paper Spectral Algorithms for Unique games. She seems to think that the UGC is false!
10. Impagliazzo and Williams talk on Communication Complexity with Synchronized clocks was an interesting new model of Communication Complexity which was used to solve some problems in classical CC. Also easy to understand since it was the first talk on a model. I suspect that within 5 years people will be using hard math to study this model and the talks will be harder to follow.
11. Buhrman, Fortnow, Koucky, Loff's paper show us that random strings are powerful in Derandomization from Random Strings
12. The most depressing talk was Derandomizing Arthur-Merlin Games and approximate counting problems since it seemed to imply that we need lower bounds on circuits to get derandomization. Worth knowing but not good news.
13. Business Meetings and Spouses: I know of three people who brought their non-complexity spouses with them to Boston. (They went sightseeing during the conference.) There are three approaches to what to do the night of the business meeting. One went to the meeting and left the spouse at the Hotel. One skipped the meeting to be with their spouse. One brought the spouse to the meeting. What would you do?
14. Business meeting was uneventful. CCC11 will be in San Jose as part of FCRC, CCC12 will be in Portugal (this was already sort-of known). Next year we may be leaving the 10th century and rather than give out handwritten proceedings on parchment we will be doing something electronic. Valentine Kabanets and Dieter von Melkebeek will be on the CCC steering committee, replacing two people whose terms have expired.

László Lovász wins Kyoto Prize

László Lovász wins the Kyoto Prize for "Outstanding Contributions to Mathematical Sciences Based on Discrete Optimization Algorithms."

Alternative Careers for Logicians

(Will post on Complexity next week. I am waiting until the invited talks have their slides online so that I can point to them.)

Let's say you just got a PhD in Logic. (ASIDE- my spell check program flagged PhD, but it gets 54,000,000 hits on Google so it has to be correct.) The job market is not so good. That is, the academic job market is not so good. I have recently been alerted to two alternative career paths one could take.

1. Get a job on Wall Street! You are probably thinking Oh, they want people that are good at math and can program. While that is true, they actually want people who know about ordinals! Ordinals? Yes Ordinals! See here.
2. Peter Cholak is a logician at Notre Dame who has had seven PhD students finish. Two of them went then went to Catholic seminary and become Catholic priests.

What to make of this? In American there is a priest shortage (see here for an article about it that mentions the law of large numbers). Hence the priesthood is a good job market. Are there other logicians-turned-priests? In this day and age I suspect there are not that many. In an earlier time when neither field required the kind of training they do now, there may have been more.

The job market for Protestant Ministers and Rabbis is not very good (See Protestant Ministers and Rabbis.) I was unable to find out how the job market is for Imam's.

I suspect that the decision to go into any of these fields is more of a calling than having a keen eye on the job market. Peter Cholak's two logician-turned-priest students just got their calling a bit late. They were both at the Fifth Conference on Logic, Computability, and Randomness so they are both trying to keep up some with logic. That may be hard; however, they may have help from up above.

News from Cambridge

Now that teaching has ended I plan to focus again on my P v NP book. I won't go on a full blog sabbatical, instead I'll aim to post once a week as well as keep up my tweets.

As you readers know, STOC, Complexity and EC all met last week in Cambridge, Mass. Complexity closed its registration at 140, EC broke 200 and STOC had about 350, high numbers for those conferences helped by both location and co-location. All three conferences will be part of FCRC in San Jose next year. Salil Vadhan will be PC Chair for STOC, Omer Reingold for Complexity. STOC 2012 will be held in New York City, CCC 2012 in Porto, Portugal home of Port Wine and my favorite delicacy. EC doesn't think that far ahead. 

The 2010 Gödel Prize will be awarded at ICALP to Sanjeev Arora and Joseph S.B. Mitchell for their concurrent discovery of a polynomial-time approximation scheme (PTAS) for the Euclidean Travelling Salesman Problem. 

The STOC proceedings are on-line including best student paper and best papers. The EC proceedings is also on-line with their best paper and no best student paper this year. The Complexity proceedings (published by IEEE) not on-line yet but the best student paper was The Gaussian surface area and noise sensitivity of degree-d polynomial threshold functions by Daniel Kane of Harvard. CCC didn't pick any best paper winners.

The TCS Visions are now on the CCC site, and in many more formats on Theory Mattters. Please feel free to make use of these slides to sell theory to chairs, deans, students and the general public.

At the STOC business meeting I talked about addressing the problem that STOC still only attracts a small fraction of the theory community. More on that in future posts.

Michael Schapira has a nice guest post on the EC conference and Muthu on the EC workshops. Bill promises a post on CCC tomorrow and we may have some vidcasts for you in the future.

Having CCC and EC overlap made it difficult to be a close part of either meeting. I can't believe I skipped the EC Google-subsidized Charles River Lobster Dinner cruise for the CCC business meeting but at least I got some Facebook jelly beans. I also got the HiPPo below from Ronny Kohavi's invited EC talk. The EXP stands for experiments that one should use to make choices instead of the HiPPo (Highest Paid Person's Opinion). But EXP is also a complexity class and so the hippo captures both meetings for me.

Another post on Martin Gardner

(I will post about CCC 2010 later in the week.)

Several people have posted on the death of Martin Gardner:

1. Complexity Blog (Lance)
2. Shtetl Optimized (Scott).
3. Ars Mathematics

I did not because of travels, but now I can. (Full Disclosure- This Blog is part of the Scientific American Partner Network. Martin Gardner had a column in SA for many years.)

About a year and a half ago I reviewed some books of Martin Gardner for my SIGACT NEWS Book Review Column. (The review is in this column.) Before I send in a review I contact all of the authors of books, authors of reviews, and editors of the books and send them a copy of the review (by email). I was told that Martin Gardner didn't use email (he was 94 years old!). So I sent him the review by postal mail (you can ask your grandfather what that was).

I got a response! I posted it here though I blocked out his address and phone number for privacy reasons. As you can see the letter is coherent and cogent (I've gotten less coherent letters from people half his age). It is good to know that he was sharp till the end.

The first theorem I ever read on my own was in one of his books: it was that (in today's terms) a graph is Eulerian iff every vertex has even degree. (ADDED LATER: The correct theorem is that a graph is Eulerian (has a cycle that hits every edge exactly once) iff it is connected and every vertex has even degree.) I also learned about The Unexpected Hanging from one of his columns. These two stand out the most; however, I learned A LOT from his columns, plus I learned that there was more to math than what was taught in the classroom.

Whats your Game Mr. Bond- The sequel!

(I will post about CCC 2010 later in the week.)

The word Game is used in many different contexts within math and computer science. I list out all that a group of us at last years Dagstuhl thought of:

1. Duplicator-Spoiler Games are used in Descriptive Complexity theory and Logic. They are also called Ehrenfeucht-Fraisse games. (The pointer, a Wikipedia entry, has a pointer to EXCELLENT slides on these games.) Example result: wellfoundness for linear orders is not first order definable.
2. Every A &sub {0,1}^&omega gives rise to the following game: Alice picks c₁ &isin {0,1} Bob picks c₂ &isin {0,1} Alice picks c₃ &isin {0,1} Bob picks c₄ &isin {0,1} etc. If c₁c₂c₃ ... is in A then Alice wins. If not then Bob wins. The Axiom of Determinacy states that, for every A, one of the two players has a winning strategy. This has been proven for Borel sets A. Full AD contradicts Full AC. See here. I have posted on it before here.
3. Banach Games. Let A be a subset of R. All numbers picked are in R. Alice picks x₁. Bob picks x₂ < x₁. Alice picks x₃ < x₂. Bob picks x₄ < x₃. etc. If &sum x_i &isin A then Alice wins, otherwise Bob Wins. For more on these see here.
4. Banach Mazur Games. (I am not quite sure of this one since I could not find stuff on the web and it looks too much like the games for AD. If this is incorrect please comment.) Similar to the games under AD; however, instead of picking an element of {0,1} the players pick an element of {0,1}₊. This has been used to define when a set is meager and has been extended to poly-time settings. Also has been extended to graphs: see here
5. Martingales. Let A &sub {0,1}^&omega. x is in A, but the player does not know what it is. The player starts out with one dollar (wow!). When the player sees the first n bits of x he bids on the (n+1)st bit. Can he win? Depends on A. Has been used to define measure 0 and has been adapted to the poly time setting. Jack Lutz has worked a lot on this stuff.
6. Complexity Classes have been defined via games. Usually a prover wins if he can convince a verifier that x &isin A. Many parameters can be changed to get many classes (number of rounds, number of provers, strength of verifier, strength of prover(s), prob of error allowed). I would include the Unique Game Conjecture in this use of the word game.
7. Pebble Games have been used to get time-space trade offs on simple models of computation. More recently they have been used to get lower bounds on resolution theorem provers. For a survey see the first paper on this page. Warning- the survey is 200 pages!
8. Games have been used to prove theories decidable. Originally S2S (Second order theory with two successors, essentially the theory of infinite strings of 0's and 1's) was proven decidable by Michael Rabin. Later Gurevich and Harrington had a simpler proof using games. See this paper. Another excellent article on this, which alas is not on line, is Infinite Games played on Finite Graphs by Robert McNaughton, Annals of Pure and Applied logic, Volume 65, Issue 2, Pages 149-184.
9. Combinatorial Games like NIM have been well studied.
10. Game Theory needs no commentary here.
11. Richman Games are a way to combine Combinatorial Games with Game Theory. See this paper.
12. In adversary arguments in lower bounds we picture an adversary who is playing a game with the algorithm or with any possible algorithms.
13. Communication Complexity Games. Not sure these are really games, but the term is used.
14. Surreal numbers are actually games. See Conways book On Numbers and Games.
15. A lot of computer science and graphics go into video games.
16. There has been some study of strategies on real games like monopoly and chess. I gave one classification of games in this post. Note that GO and Chess are EXPTIME complete.

Note that most of these games are not fun.

Sims Complexity

I called home last night and my daughter Molly had an exciting story to tell. She was at her friend Danielle's house (the same Molly and Danielle from the video) and playing the Sims, a computer role-playing game.

One of the characters was using a computer which had an item "Solve the Unsolvable". Danielle clicked the phrase. A few minutes later a pop-up opens and says "Your Sim has made progress on the P = NP Problem".

Nothing like a video game to impress the little one.

STOC and EC and Complexity, Oh My

Tomorrow I'm off to Cambridge, MA for the 42nd Symposium on the Theory of Computing (STOC), the 25th Conference on Computational Complexity (CCC) and the 11th Conference on Electronic Commerce (EC). Looking like a big crowd. The hotels are full. There are more STOC tutorial registrants than seats in the tutorial room. CCC is threatening to cut off registration. Too many people is a good thing. We'll always find a way to squeeze people in.

These are the three summer conferences I go to regularly so I am registering and attending all three (anyone else?), even if it means racing back and forth across Oxford Street to catch people and talks at both the EC and CCC conferences. Bill will also be at STOC and Complexity. If you see us say hi.

These conferences also mark the beginning of summer for me as we just finished up spring quarter classes at Northwestern. 

Blogging will be light next week. Obviously a busy time and Bill never posts when traveling. I'll give updates via Twitter but don't expect live tweets from the STOC business meeting as I'm the MC this year. I'll do my best to keep it lively.

We do this all again next year as all three conferences are part of the FCRC meeting in San Jose.

Conference Acceptance Rates

In the latest CACM, Jilin Chen and Joseph Konstan analyze data from the ACM DL and conclude that low-acceptance rate conference have papers with higher impact (more citations). This shouldn't be too surprising. If Joe believes a conference A bestows more prestige on their paper than conference B, Joe might submit his paper to A instead of B if it Joe's paper had a higher probability of being accepted into conference B. So in equilibrium conferences with higher prestige papers should have a lower acceptance rate.

Chen and Konstan suggest that conferences could lower than acceptance rate to get higher prestige. I have two problems with that advice. First of course my general annoyance of treating CS conferences as "journals that happen be be held at a hotel". 

Chen and Konstan have the causation effect backwards. Prestigious conferences do have a lower acceptance rate with other factors kept the same. But you can't necessarily up the prestige by changing the acceptance rate. The one decision the PC can make is how many papers get accepted into the conference. But accept fewer papers and in the future less people will submit their papers since they no longer believe they have a decent chance of seeing their papers accepted. it's possible in the long run to increase the acceptance rate while lowering the number of papers accepted.

When STOC moved to double sessions and accepted more papers, did the acceptance rate dramatically go up? No, because we had more submissions to match. But why does STOC have a higher acceptance rate than similarly rated conferences in other subfields? Theoretical work by its nature is easier to self-judge and most people whose papers are well below the accept/reject border of quality don't bother to submit to STOC.

You have a prestigious conference by the people that come. You can't engineer prestige, it has to be nourished.

Advice for an Engineer

A reader from Mexico asks 

I am a near-to-be graduate student from Computational Engineering. However, it wasn't what I expected from my career. I want to be a mathematician.

Computational complexity seems incredibly interesting, and I really want to take part in this amazing field. But because I'm an engineer I lack the basics to do any formal or proper research. I feel like I made the wrong career decision, even though I'm not bad at it and have a good job.

So, even though I'm behind on the math schedule, I really want to participate on some form of research. I'm unsure on how to proceed, though. I'm not versed enough in Math to be of real use and I don't feel like going back to school just yet.

What would you recommend on my position?

There are a number of computer scientists and even complexity theorists who started out in "the real world," realized it didn't feel fulfilling and went back to graduate school and on to successful research careers. It's not an easy road but it can be done.

If you are not ready or able to go back to school right away you should start exploring complexity on its own. Start reading a textbook (like Arora-Barak), read some lecture notes and when you are ready, try reading some of the latest papers in the field on ECCC and ArXiv and try to tackle some of their open questions. If it still excites you definitely consider graduate school. Worst case you'll end up back in industry and twenty years from now you don't want to regret choices not made today.

Avner Magen (1968-2010)

Toronto Professor Avner Magen died in a climbing accident on Saturday. He's had a number of important results on a variety of algorithmic topics.

Avner was one of our postdocs at NEC and we were co-authors on a SODA paper. He was a good friend and colleague and he will be sorely missed.

The department set up a memorial site where you can read memories of Avner and add your own.