2009 Complexity Year in Review

We go all the way back to January for the paper of the year, Mark Braverman's Poly-logarithmic independence fools AC0 circuits. Runners up include the Moser-Tardos Constructive Proof of the Lovász Local Lemma (mostly for Robin Moser's great STOC talk) and Rahul Jain, Zhengfeng Ji, Sarvagya Upadhyay and John Watrous for QIP=PSPACE. Yet another great year for complexity.

We welcomed new bloggers Glencora Borradaile, Jon Katz, Harry Lewis, Richard Lipton, and Noam Nisan. And we have a new president who promises to restore science to its rightful place.

US Science funding in general and CS funding at the NSF in particular got a strong boost from the stimulus package and continues to be well funded in the new budget. Microsoft Research New England gobbles up Madhu Sudan and Boaz Barak. A new innovations conference starts next week and we explored barriers in August. Lots of postdoc jobs out there for theorists. If you are looking for a faculty job, well you might want to consider another postdoc. 

This year we started vidcasts and typecasts with special guests Molly Fortnow, Meena Mahajan, John Rogers, Rahul Santhanam, Chris Umans and Ryan Williams.

I had a fun year. My journal ACM Transactions on Computation Theory had its first two issues (submit your papers). I discovered Twitter. I became Theory Czar (SIGACT chair) with fellow blogger Michael Mitzenmacher as vice-chair. It's hard to complain about the powers-that-be when you are a power-that-is. I had two CACM articles Time for Computer Science to Grow Up and The Status of the P versus NP Problem, another year and the problem is still open. I just started a 3-month blog sabbatical to turn the latter CACM article into a book for the masses, though I can't resist the Year in Review post. Thanks to Bill for keeping the blog going.

We remember Rajeev Motwani, Amir Pnueli, Jack Schwartz, Imre Simon and Ray Solomonoff.

Thanks to our guest posters and contributors: Michele Budinich, Hans Courant, Dave Doty, Sorelle Friedler, Samir Khuller, Clyde Kruskal, Joe Kruskal, Bill Kahn, Michael Lucas, Lucy Moser, Ryan O'Donnell, Tal Rabin, Rahul Santhanam, Janos Simon, Aaron Sterling, Vijay Vazirani and Lenore Zuck.

Have a great New Years. Bill will be back next week and I'll be posting regularly again in March starting in Columbus. 

How to tell how good a TV show is

(This is my last blog of the year. Lance will interrupt his blog sabbatical to do an END OF THE YEAR blog later.)

The TV show MONK recently finished its 8th and final season. My wife and I are big fans and have seasons 1-7 on DVD (and we will get 8). But this post is not about Monk. Its about the question: How to determine how good a TV show is? I am sure that whatever I say here may apply to other problems.

First assign to each episode a number between 1 and 10 depending on how much you liked it. (This could be the hardest part of the method.) Let t be a parameter to be picked later. t stands for threshold. If your criteria is How likely is it that an episode is OUTSTANDING? then you would pick t large, perhaps 9. If your criteria is How likely is it that an episode DOES NOT SUCK? then you would pick t small, perhaps 2. Some of the methods use t, some do not.

There are many different ways to do this. We give a few of them:

1. The mean or median of all of the episodes.
2. The probability that a randomly chosen episode is rated above t. (Could also get into prob that it is within one standard deviation from t.)
3. The probability that a randomly chosen disc has an episode rated above t.
4. The probability that a randomly chosen disc has fraction f of its episodes rated above t.
5. Rate each disc in the DVD set for the entire season. The mean or median of all of these ratings.
6. The mean or median of the best season.
7. The mean or median of the worst season.

There are others as well. But the question really is, given a set of numbers grouped in a natural way (in this case roughly 8 sets of 16 numbers, and each set of 16 in groups of 4) how do you judge the quality?
For those who are fans of the show MONK here are my choices for OUTSTANDING and UNWATCHABLE episodes: here

What is an explicit Construction?

The Prob method (usually credited to Erdos) was once considered quite novel: You show something exists but you don't show how to construct it! An early example was lower bounds on the Ramsey Numbers: You can prove that there exists a 2-coloring of the edges of K_n with no monochromatic k-cliques where n=2^k/2, and the proof does not give a way to construct the coloring! When Joel Spencer wrote his first book on The Probabilistic Method (appeared in 1974) this kind of argument was still considered novel. Now they are quite common. Such arguments were (and still are) called Nonconstructive. However, at the time, the term was not defined rigorously.

Of course, once you know that such a coloring exists, it is easy to find one by just looking at all possibly colorings. Hence this use of nonconstructive proofs would not be an issue for logicians.

Many of the early prob method arguments actually showed that most colorings have the property you want. Hence they would lead to randomized polynomial time algorithms. With this in mind, here is one possible definition that seems to cover what Erdos and company were thinking informally:

Definition: A coloring of an object X is constructive if it can be obtained in time polynomial in the size of X.

Is this a good definition? Now that we think P=BPP, or at least that BPP is a good notion of feasible, perhaps we should call randomized algorithms constructive. Moser and Tardos think so since there paper entitled A constructive proof of the General Lovasz Local Lemma has a randomized algorithm.

Here is a history of the prob method as applied to lower bounds on VDW numbers (see here for definitions and some of the proofs). Let W(k) be the least number such that, no matter how you 2-color {1,...,W(k)}, there will be a monochromatic arithmetic sequence of length k.

1. W(k) &ge sqrt(k)2^(k-1)/2 is an easy application of the Prob Method. The proof is so easy that I could not find it written down anywhere, so I wrote it down in the paper pointed to above.
2. W(k) &ge sqrt(k)2^(k-1)/2 can be proved constructively by the method of conditional expectations. This is also so easy that I could not find it written down anywhere. So I wrote it down myself in the paper pointed to above.
3. Berlekamp showed that, if p is prime, then W(p) &ge p2^p constructively. (I prove a slightly weaker result in the paper linked to above.) (Berlekamp's original paper can be found on my website of VDW papers: here.)
4. Using the Local Lovasz Lemma one can show that W(p)&ge 2^k/ek. This proof is nonconstructive and does not yield and almost-all result, so it cannot be turned into a randomized algorithm.
5. Moser and Tardos (above link) (and later Beigel with a different proof) showed that the LLL can always be made to yield a prob algorithm. Hence they have a randomized algorithm to obtain a 2-coloring of {1,...,2^k/ek} with no mono k-AP's. This is not a constructive algorithm in the way that Erdos or Spencer might accept, though it does yield a feasible algorithm.
6. Chandrasekaran, Goyal, Haeupler in the paper Deterministic algorithms for the Lovasz Local Lemma have a deterministic version of the LLL with slightly worse bounds. From their paper one can obtain: W(k) &ge 2^{k/(1+&epsilon)} via a poly algorithm.

A hard problem inspired by an easy problem

The following problem was problem 1 (the easy one) on the Maryland Math Competition 2009 (I will later report on how the students did on it).

1. Show that for every set of three integers we can find two of them whose average is an integer.
2. Show that for every set of five integers we can find three of them whose average is an integer.

I am sure that most of my readers can do this problem. Here is a generalization. I do not know if it is true.

(Conjecture) For all k, every set of 2k-1 integers, there exists k of them whose average is an integer.

1. The UMCP competition asked for the k=2 and k=3 cases of the conjecture. They are true and easy.
2. I have done the k=4 case. It was tedious but not hard.
3. I think I have done the k=5 case but it was alot of cases so I may have missed one.

SO, is the conjecture true or not? I have asked around and people who should know don't seem to, so I throw it open to the blogosphere.

What I hope happens: Someone posts a nice proof.

What would also be okay: Someone posts a counterexample.

Guest Post- Women in Theory Workshop

(Tal Rabin requested to post this so I am doing so. This post is essentially her email, so call it a guest post.)

There will be a Women In Theory workshop for female graduate students in theoretical computer science in Princeton on June 19-23, 2010(see here) Tell all of the female students in your department about this workshop.

The workshop will have first-rate technical content and we also hope that it will help encourage female researchers and perhaps in the long run increase the unfortunately low number of women in our field. We will supply rooms and food for the participants, and will probably also be able to cover at least part if not all of their travel expenses - please let us know if that will be an issue. There may be a nominal registration fee.

The format will be similar to the WIT 2008 workshop. You can view information on that workshop at: here and view a video of WIT08 at: here.

For any questions, please email WomenInTheory2010@gmail.com . The deadline to apply is February 1st, 2010.

Mild Request for Guest Posters.

(Deadline to submit a paper to CCC is Dec 15. Depending on when you read this that could be today or in the past.)

As you all know from Lance's last post, Lance is taking a Blog Sabbatical. I will try to have 4-posts-a-week. But not holidays (I may define what a holiday is- like a week when I only have 3 posts I may declare Friday to be a holiday: no-post-day.)

This blog has always been generous about Guest Posts. Mostly people who wanted to guest post did so. More common is the following kind of exchange:

READER: You should do a post on Davenport-Schnitzel Sequences!

LANCE OR BILL: I've got a better idea, why don't you write one and we'll edit it and post it.

READER: Uh. Hmmm. Uh.... never mind.

Now that Lance is on Blog Sabbatical and I still WANT to get something up 4-times-a-week. I offer the following: IF you want to guest post then let me know and we'll see what makes sense. Rather than merely accepting Guest Posts, I am now ASKING for them. If I don't get any guest bloggers then I'll be singing this song:

CCC deadline Dec 15, 2009! (not factorial)

Submissions to 25th CCC are due TOMORROW! (Actually it could be TOMORROW, TODAY, or IN THE PAST depending on when you read this.) Should you submit?

1. If you have a paper all set to go, with proofs complete, that is in scope and interesting, then you certainly COULD submit. Whether you should depends on other factors.
2. If this is the first you've heard of the deadline then you probably shouldn't RUSH to get it done in time. However, some people work well that way. Not only am I not one of those people, I don't understand those people.
3. This year STOC and CCC are back-to-back in Boston. That's my favorite arrangement- the advantage of FCRC (STOC and CCC in the same place) without the disadvantage (too big!). Hence, submit, accept, or not, I think you should GOTO STOC and CCC if circumstances permit.
4. Should you look at who is on the committee and based on that decide if it will get in? I tend to ignore the committee. So many papers are subrefereed. Trying to figure out which conference will be better for you based on the committee is too hard to deduce. Better to go for SCOPE- is your paper a complexity paper or not? Also, if for some reason a STOC or FOCS paper is better for your resume you can submit there; however, you can then end up with NO conferences for a while and resubmit later and and and...
5. If you submit then make sure to get your submission spellchecked and proofread before you submit.

A Blog Sabbatical

With the end of the fall quarter I will take a break from the blog for a few months. This is not another End, just a chance to move my creative juices in another endeavor, to expand my CACM article into a book to spread the gospel of P v. NP to an even larger audience.

Bill will continue to post though not as often as we did together. I will post as needed for "really important stuff" and will continue to feed my Twitter with the less important but occasionally interesting/insightful/humorous stuff. 

I will return to blogging in mid-March with the start of Northwestern's spring quarter. I need the blog as I teach to keep my sanity. By then I should have written enough of the book to get past the point of no return. Maybe I'll even share some of it with you.

P.S. If any of you have good suggestions for publishers, let me know.

Whats your Game Mr. Bond? Nim?

BILL: Clyde is teaching a graduate course titled Games, Game Theory, and the Theory of Games. He tells me that there are basically eight kinds of games governed by three 2-valued parameters. (1) 2-player or multi-player, (2) complete information or incomplete information, (3) luck or no luck. For example Chess or Nim are 2-player, no hidden information, no luck (I won't quibble about the coin toss to see who goes first.) There are natural examples of most of the combinations. The one I have the most trouble with is multiplayer with no hidden information and no luck. (Clyde is NOT as dogmatic as the above indicates. One other aspect I've left out for this post is that some games are impartial and some are not.)

DARLING: Here is a multiplayer game with no luck and no hidden information: pick up sticks.

BILL: For Clyde's class that not a game.

DARLING: Why not? Its alot more fun than NIM!

Help me answer my darling. What makes something a game rather than a sport? Not sure pick-up-sticks is a sport either.

Lets try to get NATURAL examples of each category: One issue- the terms are not that well defined so you may disagree with some of these. We break it into two basic categories (for a better picture of the results goto here.)

1) Complete Information

1. 2-player, no luck. Go, Chess, Nim: Nim has had some nice math associated to it. Chess and Go have inspired very nice computer techniques (was alpha-beta pruning developed for chess originally?). There are some recent techniques for GO which I will talk about in a later posting.
2. 2-player, some luck: Backgammon, Can't Stop (a natural game but not that well known). Sorry! (that's not be apologizing, that's the game Sorry!). Parcheesi.
3. Multiplayer, no luck: Chinese Checkers. I do not know of ANY other examples.
4. Multiplayer, luck: Monopoly (and similar games).

2) Incomplete Information

1. 2-player, no luck. Stratego, Battleship: These are debatable. In fact, it may that that if defined carefully this combination is impossible.
2. 2-player, luck: Gin and most 2-player card game.
3. Multiplayer, no luck: Clue where each player only moves 5 squares per turn, so dice needed. One of Clyde's students families plays it this way, hence it is a natural game.
4. Multiplayer, luck: Poker. Most multiplayer card games.

QUESTION: Many games are hard via complexity theory. For example, GO and CHESS are EXPTIME complete (see here). Do these results tell us things about real games?

Is posting about 17x17 problem BAD FOR ACADEMIA?

(The 17x17 problem has gotten far wider attention than I imagined--- Brian Hayes posted it on his website: here, and its also here and here. The last website is odd in that it mentions my co-authors as also putting up the money, which is not true.)

(UPDATE- 17x17 and even 18x18 was solved. See my arXiv paper on grid colorng and/or Feb 8, 2012 post.)


(Reminder: NYU-IBM-COLUMBIA-THEORY DAY: see here.)

Anon 9 on comments on 17x17 post writes about my 17x17 post:

This is just like when teachers ask their students to model or code parts of a system that will be used in the teachers own research eventually. this is really bad for academia in general. Never again propose such things, please.

I will take this comment as the starting point in an intellectually honest discussion. I reply to it after this paragraph. I then URGE the poster to reply with either: (1) Yes, GASARCH you are correct, or (2) Yes, GASARCH you are correct, but its still bad for academia because ..., or (3) No, GASARCH you are wrong because ... . Doing either would be intellectually honest. Doing neither would be intellectually dishonest. You need not use capitals or use my phrases, but you get the idea.

1. When a teacher assigns students to write code for a system the students are forced to do it in order to get a good grade. I am not forcing anyone to work on the 17x17 challenge.
2. When a teacher assigns students to write code for a system and the students do not get a co-authorship then this could be bad (this might depend on the situation). I stated explicitly that whoever cracks 17x17 can publish it themselves, or, if they do enough other stuff, with me. In any case the terms are out in the open. Also note that whatever I do will get much scrutiny because I posted it on a public blog rather than a private classroom.
3. When a teacher assigns students to write code for a system the students might get nothing for his efforts. If a student cracks my problem they will get $289.00. If they try and fail they may learn things of interest (to be fair, this may be true for student-code-writer as well).
4. The 17x17 challenge has already stimulated discussion on three blog and might get some people interested on the math end (Ramsey Theory) or the computer end. How is this bad for academia?
5. As Michael pointed out what I am doing is similar to what Erdos did, though my problem is not deep mathematics. Do you think that when Erdos offered money for problems, this was bad?
6. If I had posted about the problem WITHOUT the cash prize would you still object to it? (One point: if I offered it without a cash prize I would have asked to RESOLVE the problem rather than to GIVE ME a verifiable coloring). If I had offered ALOT MORE money would you object? Is it a U-shaped curve: offer either less than 10 or more than 2000 and you are okay with it? I am not being funny--- I look forward to your intelligent to comment on this.)

Dequantification

After a talk on derandomization at Midwest Theory Day, someone asked if those techniques could also be used in quantum computing. 

In classical randomness under reasonable assumptions we can create a polynomial in m sized set S number of strings of length m such that every circuit of size m will accept with roughly the same probability whether we draw the input uniformly from S or from all strings of length m. Can we have an analog for quantum? 

There are many good parallels between probabilistic computing and quantum computing. Both seem to allow an exponential number of possible computations in parallel combined in a limited way. Probabilistic computation is essentially multiplying exponentially large stochastic matrices and quantum computing is multiplying exponentially large unitary (or just orthogonal) matrices.

If a quantum algorithm uses classical randomness, one can replace that randomness with measurements of appropriate qbits. So classical randomness does not give any additional power to quantum computation.

But could one use some hardness assumptions to remove the quantumness from a quantum computation leaving a deterministic algorithm? It's not clear we can pull out the quantumness and even if we could quantumness and randomness just work differently. Quantum algorithms get their power from interference, allowing adding positive and negative amplitudes causing cancellation, something very difficult to simulate probabilistically or with any form of psuedorandom generator.

In short no hope for dequantification. You'll just have to build that quantum computer if you want to run those quantum algorithms.

Complexity Vidcast 3

Quick announcement: If you are a student who wants to go to SODA but doesn't have the funds, click here.

I had forgotten we did this. Here's a video of my daughter Molly interviewing Bill and I about complexity our third Vidcast filmed last October before our Dagstuhl trip.

The Probability of P=NP

Dean Foster asked me for a probability that P=NP. Now P=NP is not a probabilistic event, either P=NP or P≠NP (if it's independent it's still equal or unequal in whatever model of set theory we happen to live in). So I responded that I was highly confident that Prob(P=NP)=0.

Not good enough for Dean, a professor in the Wharton statistics department, who said "But you aren't willing to give a number? Good Bayesians / Game Theorists have to bet on everything!"

A good Bayesian puts a probability p on every future event E where one would be indifferent to taking a small bet that pays off 1-p if the event is true and loses p if the event is false. 

As a computational complexity theorist we tend not to be Bayesians, rather look at worst-case performance under that assumption that everyone is working against us. But I've been talking with economists a bit recently so lets take the Bayesian approach.

Richard Lipton asked if one would bet their life that P≠NP. In some sense I have since the vast majority of my research becomes trivial or uninteresting if P=NP. But how much one bets isn't really the right question since that involves taking risk into account as well as beliefs. 

So what odds do I give? The problem is that I could only bet conditional on P v. NP being solved in some reasonable amount of time which would alter my beliefs since a proof that P=NP requires only an algorithm but a proof that P≠NP requires showing no algorithm can work. And the no trade theorem comes into play: If someone were to offer me a bet on P v. NP, I'd secretly suspect that they knew an algorithm or a proof I hadn't seen yet. But suppose that I could make a bet against a magical oracle would reveal the correct answer once a bet is made.

Still I can't in my heart give a positive probability to P=NP. So the probability of P=NP is zero, but in the measure theory sense that because an event has probability zero it doesn't mean it can't happen, only that it won't.

Congrats to new ACM fellows

Congrats to ALL of the ACM Fellows which were annouced here.

There are several theorists among them. I could try to list them or count them; however, the term theorist is not that well defined so I'll pass.

I wonder if any of them could use an additional $289.00...

17x17: Comments on your comments

One of the comments on my last post, the 17x17 post, inquired if I am also interested in the other unknown grids (NOW just 17x18, 18x18,21x12, 22x10). I AM. In fact, Brad Larson emailed me a 4-coloring of 21x10. Here it is. He does not get $289.00 but he does get my eternal gratitude. (ADDED LATER: BRAD LATER GOT A 4-COL OF 21x11. I ADDED IT TO THE LINK THAT GAVE THE 21x10 COLORING. I WILL SOON UPDATE MY LAST POST ABOUT POSSIBLE OBSTRUCTION SETS, AND MY CHART.) (ADDED EVEN LATER: BRAD GOT A 4-COL OF 22x10 ALSO.)

(UPDATE- 17x17 WAS solved, as was 18x18. See my arXiv paper on grid coloring and/or my post on Feb 8, 2012).


One of my former high school students emailed me because he was excited that my 17x17 posting was picked up by another blogger: here Is that worth being excited about? Will it make me a celebrity? Will it get me on THE COLBERT REPORT or THE DAILY SHOW? I doubt it.

Many readers commented on my last post. These comments have lead to two corrections- my chart had typos, and the paper had a typo in an equation. The other objections raised were (I think) incorrect; however, I could be wrong. In THIS post I will answer all of the questions in the last post. If there are remaining questions, please post.

1. The typos in the chart were fixed. THANKS for the corrections.
2. The equation that Aaron Sterling pointed out was incorrect had a typo. THANKS Aaron. That should be fixed soon.
3. The Obstruction set raised alot of questions. OBS_c is NOT the set of all non-c-colorable grids. It is the set of all MINIMAL non-c-colorable grids. More precise: if nxm is in OBS_c then nxm is NOT c-colorable but both (n-1)xm and nx(m-1) are c-colorable.
4. Sky thinks 9,10,11 are in conflict. I corrected the chart so that may no longer be an issue, but if it is, please comment.
5. Grid coloring is not related to coloring planar graphs. Grid coloring problems can be rephrased at hypergraph coloring problems. I doubt this is helpful, but I could be wrong.
6. Why $289.00? Because 17x17=289. I originally thought I would offer $2.89 but Lance pointed out that that might not be enough to generate interest. Then I thought $28.90, but Lance again thought that would not get people interested. $289.00 seems just about right to get people interested but not break the bank of Gasarch. The person who won't give away the answer fo $289.00--- will you take $2.89 instead? How about $28.90?
7. dut asks if the 74-sized Rect Free Subset is the smallest there is, or could there be one of sized 73. Actually we could just take out one element and get a 73-sized Rect Free Subset. I think dut meant to ask: in a 4-coloring do we know a number x such that no color appears less than x times? YES. Lets say (and this is true but probably can be improved) that there is no Rect Fres set of sized 80 of 17x17. Then the smallest possible number of times a color can appear can easily be lower bounded. li> One of the Anon posits that its impossible since this person thinks you can't have a 73-sized Rect Free Set. Alas, my original post had a 74-sized Rect Free Set. In fact, thats why I think you CAN 4-color 17x17. I look at it as a barrier result: All of the techniques to show a grid can't be c-colored used rect free sets. Hence 17x17 (and the others) will not be able to be proven not 4-colorable using our techniques. Hence new techniques are needed. Not quite as impressive as Oracles, Natural Proofs, or Algebraization.
8. Scott claims that my claims are riddled with errors. Scott- please either point out an error or post that you are mistaken. Clearly the CHART was riddled with errors, but has been corrected.
9. I agree with commenter after David Benson who would like to know what David Benson did. David Benson- please elaborate so we can see what you did and if its right.

Who Pays for Trips?

If Professor Alice at Faber College visits Dr. Bob at the University of Southern North Dakota, who should cover Alice's expenses? 

It depends on who does the asking. If Bob invites Alice to visit then Bob's school or grant should pay. If Alice invites herself she should offer to cover her expenses via her funds. Whether or not she gives a talk doesn't really matter though a distinguished lecturer should always get reimbursed.

If Alice goes to a conference, she should cover the expenses even if she gives a talk. Plenary lecturers often get free conference registration and/or travel reimbursement but this seems to vary dramatically by the conference and discipline.

What if Alice is going to a wedding in North Dakota and offers to stop by USND? Certainly Alice should not expect money from Bob. How much can Alice use of her grant money versus personal money for such trips. That's up to her own ethics (and various laws).

All of the above goes out the window if just one of Alice or Bob has travel funds.

Typically Alice books her own air travel. Local travel and lodging is either booked by Bob or Bob makes suggestions to Alice.

Sometimes to save money or just be friendly, Bob is willing to host Alice at his house (in the guest room). If Alice feels more comfortable in a hotel, she should carefully suggest it and offer to cover the extra cost.

Alice shouldn't expect an honorarium since part of Alice's job is to promote her research, and thus her university, to the broad community. But Alice need not turn down an honorarium if offered, though she needs to declare it as income on her tax returns. If Alice works for a company or the government there might be policies against receiving honorariums. 

Above all Alice and Bob should work out who pays ahead of time. Also one needs to know rules on funding that differ at universities and granting agencies (NSF requires use of US airlines for example) and what level of hotel one should use. And always best to avoid bad feelings if one has disagreements on reimbursements after the fact.

The 17x17 challenge. Worth $289.00. This is not a joke.

The 17x17 challenge: worth $289.00. I am not kidding.

Definition: The n x m grid is c-colorable if there is a way to c-color the vertices of the n x m grid so that there is no rectangle with all four corners the same color. (The rectangles I care about have the sides parallel to the x and y axis.)

The 17x17 challenge: The first person to email me a 4-coloring of 17x17 in LaTeX will win $289.00. (I have a LaTeX template below.)

UPDATE: THE RESULTS ABOUT OBS₄ HAVE CHANGED SINCE THE ORIGINAL POST SINCE BRAD LARSON EMAILED ME A 4-COL OF 21x10 and 21x11. UPDATE: BRAD LARSON EMAILED ME A 4-COL OF 22x10.

UPDATE- PROBLEM WAS SOLVED. See my arXiv paper on grid colorings OR
my Feb 8, 2012 post.

Why 17x17? Are there other grids I care about? We (We=Stephen Fenner and Charles Glover and Semmy Purewal) will soon be submitting a PAPER (see TALK for a superset of the slide-talk I've given on this paper) on coloring grids. Here are the results and comments:

1. For all c there is a finite set of grids a₁xb₁, ..., a_kxb_k such that a grid is c-colorable iff it does not contain any of the a_ixb_i (n x m contains a x b if a &le n and b &le m). We call this set of grids OBS_c (OBS for Obstruction Set).
2. OBS₂ = {3x7, 5x5, 7x3}
3. OBS₃ = {19x4, 16x5, 13x7, 11x10, 10x11, 7x13, 5x16, 4x19}
4. OBS₄ contains 41x5, 31x6, 29x7, 25x9, 23x10, 10x23, 9x25, 7x29, 6x31, 5x41
5. Exactly one of the following sets is in OBS₄: 21x13, 21x12.
6. Exactly one of the following sets is in OBS₄: 19x17, 18x17, 17x17.
7. If 19x17 is in OBS₄ then 18x18 might be in OBS₄ . If 19x17 is not in OBS₄ then 18x18 is not in OBS₄ .
8. A chart of what we do and do not know about 4-colorings of grids is here.
9. A Rectangle Free Subset of a x b is a subset of a x b that has no rectangles. Note that if a x b is 4-colorable then there must be a rectangle free subset of a x b of size Ceil(ab/4). We actually have a rectangle free subset of 17x17 of size Ceil(289/4)+1. Hence we think it is 4-colorable. For the rectangle-free subset see here. For the original LaTeX (which you can use for a template when you submit yours) see here. THIS IS WHY I AM FOCUSING ON 17x17--- BECAUSE I REALLY REALLY THINK THAT IT IS 4-COLORABLE. I could still be wrong.

What if 17x17 is not 4-colorable? Then nobody will collect the $289.00. Even so, if you find this out I would very much like to hear about it. I don't want to offer money for that since if your proof is a computer proof that I can't verify then its not clear how I can verify it. I also don't want to offer money for a reasonable proof since this may lead to having the Supreme Court decide what is a reasonable proof.

Can I get a paper out of this? FACT: If you get me the coloring and want to use it in a publication then that is fine. OPINION: It would not be worth publishing unless you get all of OBS₄. See next question.

What do you hope to get out of this? My most optimistic scenario is that someone solves the 17x17 problem and that the math or software that they use can be used to crack the entire OBS₄ problem. If this happens and my paper has not been accepted yet then we could talk about a merge, though this is tentative on both ends.

Has there been any work done on this problem that is not in your paper?

1. A Rutgers grad student named Beth Kupkin has worked on the problem: here.
2. A high school student (member of my VDW gang) Rohan Puttagunta has obtained a 4-coloring of 17x17 EXCEPT for one square: here.
3. SAT-solvers and IP-programs have been used but have not worked--- however, I don't think they were tried that seriously.
4. Here are some thoughts I have had on the problem lately: here.
5. There is a paper on the 3-d version of this problem: here.

Is Lance involved with this at all? When I gave a talk on grid colorings at Northwestern, Lance fell asleep.

Birthday Paradox Variance

First a message from David Johnson for proposals on locations for SODA 2012 both in and outside the US.

Here's an interesting approach to the birthday paradox using variances.

Suppose we have m people who have birthdays spread uniformly over n days. We want to bound m such that the probability that there are are at least two people with the same birthay is about one-half.

For 1 ≤ i < j ≤ m, let A_i,j be a random variable taking value 1 if the ith and jth person share the same birthday and zero otherwise. Let A be the sum of the A_i,j. At least two people have the same birthday if A ≥ 1.

E(A_i,j) = 1/n so by linearity of expectations, E(A) = m(m-1)/2n. By Markov's inequality, Prob(A ≥ 1) ≤ E(A) so if m(m-1)/2n ≤ 1/2 (approximately m ≤ n^1/2), the probability that two people have the same birthday is less than 1/2.

How about the other direction? For that we need to compute the variance. Var(A_i,j) = E(A_i,j²)-E²(A_i,j) = 1/n-1/n² = (n-1)/n².

A_i,j and A_u,v are independent random variables, obvious if {i,j}∩{u,v} = ∅ but still true even if they share an index: Prob(A_i,jA_i,v = 1) = Prob(The ith, jith and vth person all share the same birthday) = 1/n² = Prob(A_i,j=1)Prob(A_i,v=1).

The variance of a sum of pairwise independent random variables is the sum of the variances so we have Var(A) = m(m-1)(n-1)/2n².

Since A is integral we have Prob(A < 1) = Prob(A = 0) ≤ Prob(|A-E(A)| ≥ E(A)) ≤ Var(A)/E²(A) by Chebyshev's inequality. After simplifying we get Prob(A < 1) ≤ 2(n-1)/(m(m-1)) or approximately 2n/m². Setting that equal to 1/2 says that if m ≥ 2n^1/2 the probability that everyone has different birthdays is at most 1/2.

If m is the value that gives probability one-half that we have at least two people with the same birthday, we get n^1/2 ≤ m ≤ 2n^1/2, a factor of 2 difference. Not bad for a simple variance calculation.

Plugging in n = 365 into the exact formulas we get 19.612 ≤ m ≤ 38.661 where the real answer is about m = 23.

Enjoy the Thanksgiving holiday. We'll be back on Monday.

DIMACS at 20

Last Friday DIMACS celebrated its 20th anniversary. Muthu summarizes the event.

DIMACS has served the theoretical computer science community well over these two decades. They have hosted a number of postdocs and visitors usually around a Special Focus (originally Special years but one year is usually not enough). DIMACS runs a large number of educational and research activities but most importantly the great workshops over the years. DIMACS's reputation for having strong workshops allows it to continue to attract strong workshops and has helped make New Jersey (my home state) a major center of theory. 

The first DIMACS workshop I attended, Structural Complexity and Cryptography is where I first heard about about the first Gödel-prize winning research relating interactive proofs to hardness of approximation that was done primarily during that Special Year on Complexity Theory and Interactive Computation.

When I moved to the NEC Research Institute in 1999, I became quite active in DIMACS which had then started a Special Year on Computational Intractability including a great workshop on the Intrinsic Complexity of Computations with many great talks and discussions on the hardness of proving lower bounds. My talk on Diagonalization later became an article in the BEATCS complexity column.

I then joined the DIMACS executive board as the NEC representative just as the center was ending its 11-years of funding as an NSF Science and Technology Center. Amazing that DIMACS survived that transition and survived for these twenty years and beyond. Most of the credit goes to DIMACS director Fred Roberts who has often hustled for funding for specific special foci, projects and workshops, as well as finding people to run those foci and workshops.

The 2001 Workshop on Computational Issues in Game Theory and Mechanism Design truly established a new discipline in connecting computer science and economic theory. Based on the excitement from that workshop, DIMACS started a Special Focus on Computation and the Socio-Economic Sciences which Fred talked me into co-organizing with Rakesh Vohra, from Northwestern's Kellogg business school. After I moved back to Chicago in 2003, I met with Rakesh to plan the focus which led to collaboration and eventually my moving to Northwestern. 

The special focus had a number of exciting workshops particularly Information Markets which restarted that research area a couple years after the PAM disaster and our closing workshop on the Boundary between Economic Theory and Computer Science one of the few meetings that truly attracted both strong computer scientists and economists.

That's just a few of my DIMACS memories. Many others have similar stories for a center that helped shape the professional lives of myself and many other CS researchers. Congrats for 20 years and here's hoping for many more.

An undervalued Math Problem

As most of you know there are 7 problems worth $1,000,000 (see here). It may be just 6 since Poincare's conjecture has probably been solved. Why are these problems worth that much money? There are other open problems that are worth far less money. What determines how much money a problem is worth?

When Erdos offered money for a problem (from 10 to 3000 dollars) I suspect that the amount of money depended on (1) how hard Erdos thought the problem was, (2) how much Erdos cared about the problem, (3) how much money Erdos had when he offered the prize, and (4) inflation. (If anyone can find a pointer to the list of open Erdos Problems please comment and I'll add it here.)

Here is a problem that I have heard is hard and deep, yet it is only worth $3000 (Erdos proposed it). I think that it should be worth more.

BACKGROUND: Szemeredi's theorem: Let A&sube N. If the limit as n goes to infinity of size(A &cap {1,...,n})/n is bounded below by a positive constant then A has arbitrarily long arithmetic sequences. Intuition: if a set is large then it has arb long arith seqs. The CONJECTURE below uses a diff notion of large.

CONJECTURE: Let A&sube N. If &sum_{a&isin A} 1/a div

KNOWN: Its known that if A is the set of all primes (note that &sum_{a&isin A} 1/a diverges) then A has arbitrarily large arithmetic progressions. Nothing else is known! The conjecture for 3-AP's isn't even known!

Is this a good problem? If it is solved quickly (very unlikely) than NO. If absolutely no progress is made on it and no interesting mathematics comes out of the attempts than NO. It has to be just right.