MohammadTaghi HajiAghayi on David Johnson

More than a week ago, I heard the very sad news that David Johnson has passed away after one year fight with cancer. I felt that I should write a memorial note for him. Indeed I have done the same for Mihai Pătraşcu in the same blog and both Mihai and David were very similar to me from several aspects: both were my colleagues at AT&T and more importantly my dear friends, both they got their Ph.D. from MIT (the same place that I got my Ph.D. as well), they both were extraordinary researchers, and both passed away due to cancer after almost a year-long fight with it (and I was closely aware of their situations in that year). Indeed David read my memo for Mihai and he told me that he liked it. In addition, there is another reason that I feel respect for David; he was just a bit older than my father who also passed away very recently. So here I would like to put my thoughts into words for David (and this took me more time in this case since I wanted to mention some new thoughts given the comments already in this blog). To do so, I would like to mention some of David’s personal characteristics that I appreciated a lot and give some examples on them from my interactions with him. Indeed I have even mentioned some of these to him when he was alive and told him because of these (and other reasons), I am always proud to mention that I have him as my boss at some point in my career.

First of all, David was very humble and modest especially given his extraordinary CV: he won several awards especially Knuth prize, he is the co-author of one of the top most-cited books in CS, he was fellows of almost every community that he was involved with (e.g., ACM, SIAM, AT&T), he was a member and the chair of several prestigious award committees (like Gödel, Knuth, ACM Kanellakis, ACM Thesis Award) and indeed he was a founder of some of them (e.g., Kanellakis), and he was the founder of SODA, the best algorithms conference, among others. Despite all this he was a very humble and modest man and I think lots of people who interacted with him will fully agree on this. Just to give an example, in 1998, while I was still a second-year undergrad at Sharif University, I sent him an email asking whether he was aware of any book similar to Garey & Johnson but for parallel computing (indeed this was my first remote interaction with him); I was shocked how fast he answered my email just in a couple of hours with a relevant reference. This was especially very exciting and encouraging for me, since several other people never answered my emails at that time. More interestingly, later in 2012, I told him personally that I admired him for answering that email. He told me just wait a second and in a couple of minutes, he could find the exact same email from 1998 that I sent him; then we even discussed some English improvements for the email text as well.

Second he was a perfectionist from several aspects. Here are some examples. He was often the reviewer for P=NP or P!=NP papers for several journals. Probably lots of us even do not look into these papers unless written by a well-known fellow; however he was reading these papers very carefully to find the exact bugs and mention them to the authors. Indeed even when I sent him several referee requests for conferences for which I severed as a PC member, he always spent a lot of time to read the paper very carefully and often came with novel improvements and simplifications, sometime in a extend that authors of the paper under review wanted to have this anonymous referee as a co-author. All these happened despite he was a very busy man; however he still considered the task of refereeing a paper very seriously and respected the authors (and I think this is an example that lots of us can learn from it). He was a very good writer as well and spent a lot of time to improve the presentation of a paper, simplify it, and present it in a perfect way. I am proud to have one paper coauthored with David, a very long paper with several co-authors. On this paper David had the lead and indeed spent all the years that I was with AT&T (and even after than) to prepare the journal version of the paper. Indeed he was sending us the almost final version on Dec 2014 (and asked us for comments) just a month before he was diagnosed with cancer (I hope that still we can send the paper to a journal given the time that David spent on it). Another example of his perfectionism: he attended ALL SODA while he was alive and almost ALL STOC and FOCS (expect 1-2 years that AT&T had travel restrictions). Not only that, anytime that there was any talk in the conference, he attended at least one session. Yet another example: we had group lunches every day at AT&T.  That was David’s habit to ask everyone in the group to see whether they want to join. Now the interesting point was that he came exactly at noon EVERY DAY and you could even set your watch for 12pm when you saw him for lunch.

He was founder of SODA, the best algorithms conference. Indeed lots of us know David because he was the founder of SODA and he was handling SODA business meetings for lots of year as the chair of the steering committee. As a result, I often had lots of discussion with him regarding SODA and its future. We discussed what the protocol for selecting the chair of SODA should be, whether SODA should have an official Rebuttal Phase or not, etc. During discussion even some interesting topics came up which are good to discuss in the community as well. David believed since SODAs (and in general other major TCS conferences) are the main venues for publications but still we need full and correct mathematical proofs for our claims (despite the rest of CS), we should have a five-year period that any major claims and theorems for which the authors do not provide full proofs in a verifiable manner in arxiv or in a journal during these five years should be considered officially open for everyone to grab, prove formally, and get the full credit for that. Another discussion was that ideally SODA (and again other major TCS conferences) should go double-blind like lots of other major CS conferences in other fields. This will help to have much more fair selection in which the name of authors do not give advantage/disadvantage for acceptance (though PC chair still could see the author lists for some extreme cases).

I can probably write pages and pages of other memories on David’s excellent personal characteristics (e.g. he was a marathon runner, he held the annual barbecue for AT&T/Bell-labs theory interns, researchers, and alumni for more than two decades,  he served in Army between his Masters and Ph.D. and kept the same types of spirits and disciplines in the rest of his life, he always emphasized on putting his middle initial “S.” in his name especially due to Airport Security since his name is a very common name, etc), but I think I should stop at this point.

I hope that we have a great memorial event for him in the next SODA (SODA’17) the conference that he founded.

Rest in Peace David,

From Mohammad

Complexity versus Complexity

For those interested, I've started writing posts for the Predictwise Blog. Predictwise makes predictions of future events such as who will win the Republican Nomination (currently Trump with an 80% probability) based on prediction markets and other betting sites. This has been a fascinating election in terms of predictions, strategies, rules and game theory and I'm happy to try and makes sense out of it over at Predictwise without subjecting my readers here at Computational Complexity with too many political posts.

A reader had asked me to comment on a Slate article The Theory of Everything and Then Some, a book review of John Miller's A Crude Look at the Whole: The Science of Complex Systems in Business, Life, and Society. John Miller is a social scientist who works on the other "complexity theory" that studies that "simple local rules can have complex global implications". Often complex systems work quite well, like the invisible hand of the economy, but sometimes things can go wrong and the article often mentions the "flash crash" of trading programs reacting to each other causing a major drop in stock prices in May of 2010.

Our fields with the similar names are not as different as might appear. Much of what they study are inherently computational-like processes and we also look at emergent behavior from simple operations of Turing machine; read, write and move the tape. What they call non-linear we call computation. We do take very different approaches. The computational complexity theory community proves theorems where we can and helps understand the mathematical challenges of when we can't. The other complexity theorists try to explain by examples, simulations and simplified models.

The two communities often, but not always, seem to have disdain for one another and that's a shame. The tools of computational complexity can help understand the power and limitations of complex systems. These collaborations require them to understand how we can help them and for us to be willing to work on problems that may not yield difficult-to-prove theorems. That's what attracts me to prediction markets, a very simple kind of information aggregation system that still is very difficult to analyze as a computational mechanism.

What's missing from the article is how tools like machine learning can play in helping to predict the outcomes of many complex systems. The big deluge of data that starts off the article may add to the complexity but it almost paradoxically also makes it possible to learn from it.

Hilary Putnam passed away on March 13

Hilary Putnam passed away on March 13, 2016. Some of the obits say he  was a philosopher, mathematician, logician, and computer scientist.

He is probably best known to readers of this blog for his work on Hilbert's 10 problem and resolution.

HILBERT  TENTH:

Recall H10 stated in current terminology: Find an ALGORITHM that will, given a poly p(x1,,...,,xn) in many variables, with coefficients in the integers, determine if it has a diophantine solution.

Martin Davis, Hilary  Putnam, and Julia Robinson showed that if you also allow exponentation then the problem is  undecidable in the early 1960s. Yuri Matijasevich in 1970 showed how to express exps in terms of polynomials to complete the proof. The solution to Hilbert's 10th problem is often credited to all four of them which seems right to me.

One consequence of there proof: for any c.e. set A there is a poly p such that

A = { x | exists x1,...,xn p(x,x1,...,xn)=0}

Later work got the polynomial down to 13 variables.



RESOLUTION:

John Robinson (but see comments)  and later papers by  Davis-Putnam aad later Davis-Logemann-Loveland devised resolution theorem proven which is an early SAT-solver algorithm. Many modern algorithms are based on it. (Note- earlier version of this post had mistakes in it. I thank Paul Beame's comments below for clarifying the history.)

HOW TO CLASSIFY HIM:

 I suspect that Hilary Putnam would call himself a philosopher since that was his MOTIVATION.  That may be the best way to classify people (if we are inclined to do that), don't look at WHAT they do look at WHY they do it.

PHIL OF MATH- one problem with Philosophy, even Phil of Math, is that its hard to have well defined questions and therefore hard to answer them. I am NOT criticizing the field, just saying why I would have a hard time working in it.

The Value of Shapley

Nobel laureate Lloyd Shapley passed away Saturday. We best know Shapley for his stable matching algorithm with David Gale. Nicole Immorlica guest posted on stable matching shortly after Gale's passing in 2008.

I'd like to talk about another great innovation, the Shapley Value, a solution concept for cooperative games. For example, suppose no candidate has a majority of candidates heading into the Republican convention and there is no winner on the first ballot. Now we have many delegates that might group themselves into coalitions, and a union of coalitions that have enough delegates can determine the nominee. Larger coalitions have more power than smaller ones but even a single delegate coalition could tip the election. The Shapley value gives weights to the coalitions that measures their relative power with some nice linear and symmetric properties. In this scenario, the Shapley value of a coalition is the probability that adding that coalition will tip the election when coalitions are added in a random order.

Game Theorist Robert Aumann, another Nobel laureate, used the Shapley value to predict winning coalitions in Israeli elections.

The main challenge of the Shapley value is computational, in general it is #P-complete to compute but it can be approximated efficiently.

On Phillip Rogaway's The Moral Character of Cryptographic Work.

Some people have emailed me asking me to blog about the paper The Moral Character of Cryptographic Work by Phillip Rogaway  I urge you to read it, even if you disagree with it. Especially if you disagree with it. (Hmm- how will you know if you don't read it!)

There are so many issues raised in this paper that it could be (and might be) the topic of many blog posts. The first three paragraphs are today's topic:

Preamble. Most academic cryptographers seem to think that our field is a fun, deep, and politically neutral game—a set of puzzles involving communicating parties and notional adversaries. This vision of who we are animates a field whose work is intellectually impressive and rapidly produced, but also quite inbred and divorced from real-world concerns. Is this what cryptography should be like? Is it how we should expend the bulk of our intellectual capital? 

For me, these questions came to a head with the Snowden disclosures of 2013. If cryptography’s most basic aim is to enable secure communications, how could it not be a colossal failure of our field when ordinary people lack even a modicum of communication privacy when interacting electronically? Yet I soon realized that most cryptographers didn’t see it this way. Most seemed to feel that the disclosures didn’t even implicate us cryptographers. 

I think that they do. So I want to talk about the moral obligations of cryptographers, and my community as a whole. This is not a topic cryptographers routinely discuss. In this post-Snowden era, I think it needs to be. 

My thoughts:

1) I would add that the Target Breaking, the SONY hack, and the OPM breakin might also show that crypto has been a failure. He doesn't seem to mention those but I think they strengthen his case.

2) Might it be Security that is a colossal failure? Of course, crypto and security go together so it may be hard to disentangle whose failure it is.

3) Might it be that good crypto research has been done but is not being used- the tech transfer problem. He later claims that this would be relevant if crypto worked on the right problems in the
first place.

4) I tend to think he's right. Rather than me telling you why I think he's right, just read his paper.

David Johnson (1945-2016)

David Johnson, a leader and advocate for algorithms and all of theoretical computer science, passed away yesterday at the age of 70. A truly sad day for us all.

David's 1979 book with Michael Garey, Computers and Intractability: A Guide to the Theory of NP-Completeness, is still the best reference on the topic and perhaps the single most important resource in any computer scientist's library. David Johnson also wrote the NP-completeness column for the Journal on Algorithms and later the ACM Transactions on Algorithms, as well as "A Catalog of Complexity Classes" for the 1990 Handbook of Theoretical Computer Science. David founded the Symposium on Discrete Algorithms (SODA), a conference that is now often mentioned with STOC and FOCS as a top theory venue. He created the DIMACS algorithms challenges. He led SIGACT from 1987-1991, really transforming that organization, and served as its face for many years thereafter. I'm only scratching the surface of what he's done for the community, and can think of no one who put more effort into making the theoretical computer science as strong as it is.

Of course David was a great researchers as well, working on NP-completeness and approximation algorithms.

He received an ACM Fellow in 1995, the first SIGACT Distinguished Service prize in 1997 and the Knuth Prize in 2010. He used his Knuth prize lecture to push for practical applications for our algorithms. Just last month he was elected into the National Academy of Engineering.

I worked with David Johnson closely on various SIGACT activities. David never missed a STOC and we always invited him to the SIGACT Executive Committee dinners, not because he had an official role, but because he was David Johnson. I truly respected and admired David and glad I could call him a friend. We'll miss him deeply. STOC and SODA just won't be the same without him.

When do we care about the constants?

I've been reading two books recently: Asymptopia by Joel Spencer (He turns 70 soon!  Workshop for it!. My nephew things that celebrating your bday with a workshop would be... odd) and   The Joy of Factoring by Simon Wagtaff. In terms of content they are on two different topics. In terms of practicality they are different: Asymptopia is clearly a pure math book (there is one chapter on algorithms, but the rest is really pure math) whereas The Joy of Factoring is very practical in that it focuses on real algorithms for the important (for crytography) practical problem of factoring. However, there is one thing the books had in common: They both often care about multiplicative constants.

Example from Asymptopia: They gave better and better lower bounds on Ramsey numbers:

(1) R(k)  ≥  (1+o(1))(k/e sqrt(2)) 2^k/2  roughly (1+o(1))(0.26)2^k/2

(2) R(k)  ≥  (1+o(1))(k/e) 2^k/2 roughly (1+o(1))(1+o(1))(0.37)^k/2

(3) R(k)  ≥  (1+o(1))(k/sqrt(2)) 2^k/2 roughly (1+o(1))(0.71)^k/2

(It may be hard to read so I will clarify- the o(1) is little-o, a term that goes to 0 as k gets large.)

The first lower bound uses the prob method and you the reader has prob seen it or could prob derive it yourself. Prob. The second lower bound uses prob and a  clever way of coloring and then tossing out some vertices. The third lower bound uses the Local Lovasz Lemma.

Note that for this problem Joel Spencer cared about the constant.

Example from The Joy of Factoring: Since many (but not all!) factoring algorithms do not have rigorously proven run times (Number Theory is Hard!) it's harder to give clean examples here. The book often refers to tricks to get constants down and the notion that constants matters permeates the book. Here is one rigorous example of caring about constants:

Fermat's difference-of-squares algorithm goes as follows: We want to factor N. Let x=floor(sqrt(N)). Test each of the following numbers for being a square and stop when you get a square: x²-N, (x+1)²-N, (x+2)² - N, etc. When you find an r such that (x+r)²-N=y²  then you have (x+r-y)(x+u+y)=N. Almost surely this is a nontrivial factorization of N. (This algorithm is worse than the trivial sqrt(N) algorithm in some cases; however, it has some of the ideas needed for more sophisticated algorithms including the Quadratic Sieve.) Of course, one might be looking for the right r a long time. How long:

Let a be the largest divisor of N that is ≤ \sqrt(N). Let k=a/sqrt(N). Then the search will take

1+ (1-k)²sqrt(N)/(2k)

Again note that there are no hidden multiplicative constants.

So when do we care about constants and why?

1) If you are working on an algorithm for a problem people really want to solve then you need the constants to be small.

2) If you can get good bounds on the exact constants then you should.

3) If you have a technique and try it out you might end up just improving the constant. Even so, you have showed that the technique has merit.

4) Improving the constant may show progress which will later lead to more important improvements.

5) Chicken and Egg:  Here is an example from Asymptopia where he didn't care about the constant: Fix ε. Given three points in the unit square what is the prob that their area will be ≤ ε ?   He showed its Θ(ε).This proof is very nice. Tracking the constants used in his proof looks tedious. In order to care about the constants perhaps we need an interesting proof about them. To look for a proof technique that applies to them perhaps we need to care in the first place. Chicken and Egg?

Changing This Ancient Art Into a Science

The ACM announced yesterday that they will award the 2015 Turing Award to Whitfield Diffie and Martin Hellman for contributions to modern cryptography. The Turing award is the highest honor in all of computing. John Markoff in the New York Times also has the story.

Diffie and Hellman are best known for public-key cryptography, the brilliant idea that one could communicate secretly with someone you haven't communicated previously. Without public-key cryptography there would be no e-commerce. Equally important Diffie and Hellman brought computational complexity to bare, moving cryptography into its modern age. I strongly recommend reading their 1976 gem New Directions in Cryptography (PDF) particularly the introduction and chapter 6 where Diffie and Hellman connect cryptography to computational complexity and the P v NP problem itself defined only five years earlier. Here's the first paragraph:

We stand today on the brink of a revolution in cryptography. The development of cheap digital hardware has freed it from the design limitations of mechanical computing and brought the cost of high grade cryptographic devices down to where they can be used in such commercial applications as remote key cash dispensers and computer terminals. In turn, such applications create a need for new types of cryptographic systems which minimize the necessity of secure key distribution channels and supply the equivalent of a written signature. At the same time, theoretical developments in information theory and computer science show promise of providing provably secure cryptosystems, changing this ancient art into a science. 

One question for which I shall offer no opinion: Should Ralph Merkle have been a co-recipient of this award?