When do we care about small improvements?

A while back complexity blog,  Shtetl-optimized , and GLL all blogged about the improved matrix mult algorithms (Complexityblog: here, Shtetl-optimized: here, GLL here) of Stothers/Williams. It may have been on other theory blogs as well (if you know then let me know). We denote Matrix Mult Algorithm by MMA, and we use n^a instead of O(n^a).  All the papers we refer to can be found either  here or here.

1987: Cooper and Winograd get MMA in n^2.375477

2010: Stothers gets MMA in n^2.374

2011:  Williams gets MMA in  n^2.3728642

(Williams and  Stother were ind, though Williams used some of Stother's stuff to simplify her proofs for the final version.)

This Stothers/Williams results were a big deal! Williams paper got into STOC and, as mentioned above, three blogs reported on it AS SOON AS it was public.

Fast forward to

2014: Le Gall gets MMA in n^2.3728639. Wikipedia says that this MMA is a simplificaion of Williams algorithm.

The 2014 result may or may not be interesting (thats a tautology!). But what strikes me is that I came across it in 2015 and had not heard of it. I emailed Lance to ask if he had heard of it, he had not. I don't quite know if Lance and I not knowing about means its not that well known, but its at least on indicator.

ALL of which brings us back to the title of this blog: When do we care about small improvements?

1. We care about a small improvement if the result in question has been stuck where it was for a long time. This was the case for Stothers/Williams MMA and also for when the approx for metric TSP went from  approx of  3/2 to approx of (3/2 - c) (see here).
2. We care about small improvements if they illustrate an a new technique. The leaf-counting technique for lower bounds on number-of-comparisons for finding the ith largest gave clean proofs and (I think) small improvements to known results. (Leaf Counting technique in brief: Any Dec Tree for MAX has EVERY branch is of length at least n-1 hence 2^{n-2} leaves. Take a tree for ith largest. For all x_1,...,x_{i-1} prune the tree by having these elements beat anything else. How they compare to each other determine arbitrarily. This results in a MAX tree for n-i elements and hence has 2^{n-i} leaves.All such trees have disjoint sets of leaves. So original tree has at least (n choose i)2^{n-i} leaves, hence has a branch of length log_2( (n choose i)2^{n-i}) = n+ ilog n - i. Was this better than what was known at the time? Not sure but the proof was much simpler.)
3. We care about small improvements if they tell you that a natural upper or lower bound is NOT true (I can't seem to get the numbered lists right so regard the following items as subitems of this one.)

1. Bent/ John showed that Median required 2n -  2sqrt(n) - O(log n) comparisons and then later Dor and Zwick improved this to (2+ 1/2^50)n. (I can't seem to find a free online version of this- if you do please leave a comment.)
2.  Schonage, Paterson, Pippenger had median in 3n + o(n), and Dor and Zwick (not a typo- same people) improved this to 2.95n + o(n). (I can't seem to find a free online version of this- if you do please leave a comment.)
3. In both cases, especially (1), the improvement is really small and unimportant; however, knowing that 2n is NOT the lower bound and 3n is NOT the upper bound is worth knowing. Hence exact complexity is NOT going to be the nice number 2n or 3n. The only nice number between 2 and 3 is e, so lets hope its en. (I think I read that 2.5n is the actual conjecture. 2.5 is nice, but e is nicer.)

Changing STOC

At the recently completed STOC and the previous FOCS, much of the discussion revolved around reforming the conferences. You read the discussion and comments on Windows on Theory and I've also been cc'd on several very long email threads.

STOC, as the flagship conference of ACM SIGACT, should be the focal point of the community, the place where researchers circle their calendars and make sure they attend the event. You see that at SOSP for the systems community or SIGCOMM for networking. But not STOC, smaller than it was thirty years ago when the theory community had a fraction of the people we have today.

Instead STOC has become a conference for its authors, to give researchers a prestigious line in their CVs. While authors get to present their papers, STOC is no longer a primary place for dissemination, better served by arXiv and ECCC.

The problem is conference overload. We have two top theory conferences a year, STOC and FOCS, not to mention SODA, Computational Complexity and so many others. Conferences are expensive in both time and money and we can't afford to attend too many. People often choose more specialized conferences and workshops where they can focus on talking to people in their specialized research areas.

SOSP on the other hand meets only once every two years, accepts only thirty papers and gets 600 attendees.

The only true solution would merge and/or eliminate conferences. We don't need two major theory conferences a year. But that's not politically feasible.

So the talk is of a Theory Festival centered around STOC in 2017, to make an event that all theorists would want to attend. What that theory festival should or should not do is the topic of all the discussion. I'm not going to talk about the various proposals but I encourage strong experimentation to get us out of this bad equilibrium. Otherwise we end up with status quo and status quo does not bring our community together.

Learning from teaching a HS student Schur's theorem on change

(All the math this post refers to is in my manuscript which is here.)

Recall Schur's theorem on making change as stated in wikipedia and other source:

Let a1,...,aL be rel prime coin denominations. Then the number of ways to make n cents
change is n^L-1 /(L-1)!a1a2...aL  + Θ(n^L-2).

The proof I knew (from Wilfs book on generating functions)  was not difficult; however,it involved roots of unity, partial fractions, Taylor series, and Generating functions. I needed to present the proof to a HS students who was in precalc.  The writeup above is what I finally came up with. A few points.

1. HS students, or at least mine, knew complex numbers. Hence roots-of-unity was okay. The proof of Schur's theorem has another plus: he had asked me just recently how complex numbers could be used in the real world since they weren't... real. I said the are often used as an intermediary on the way to a real solution and gave him an example of a cubic equation where you spot a complex solution and use it to obtain the real solutions. Schur's theorem is a more sophisticated example of using complex numbers to get a result about reals (about naturals!) so that's a win.
2. Partial Fractions. If the student had had calculus then he would know what partial fractions were and believe me when I said they always work. But since he had not had calculus I prepared a proof that they worked. Then I realized--- I have never seen a proof that they work! This is a matter of timing- I saw them in High School Calculus in 1975 which was taught without proofs (just as well, analysis is a bad first-proof course) and I didn't quite realize that the techniques they taught us aren't quite a proof that it works. I came up with my own proof (I can't imagine its  original but I have not found a ref) in 2015. That's 40 years between seeing a concept and proving that it works. A personal record.
3. Taylor Series. I needed the Taylor series for 1/(1-x)^b   (just for b a natural). I came up with a proof that does not use calculus and that a HS student could follow. Very happy that I was forced to do do this. Actually uses a nice combinatorial identity!
4. The lemmas about partial fractions and about Taylor series are of course very very old. Are my proofs new? I doubt it though I have not been able to find a reference. If you know one please leave a polite comment.
5. Having gone through the proof so carefully I noticed something else the proof yields: Let M be the LCM of a1,...,aL.  For all 0\le r\le M-1 there is a poly p of degree L-1 such that if n\equiv r mod M then p(n) is the number of ways to make change of n cents.  I suspect this is known but could not find a ref (again- if you know one then please leave a polite comment.)

Moral of the story: By working with a HS student I was forced to find a proof for partial frac deomp, find a non-calc proof of a Taylor series, and obtain an interesting corollary. Hence this is already a win!

FCRC Complexity

On Wednesday at FCRC, the Complexity, STOC and EC (Economics and Computation) conferences all have sessions, a smorgasbord of talks, but tough decisions on what session to attend. Here's one you might have misses, the EC paper Why Prices Need Algorithms by Roughgarden and Talgam-Cohen that has a nice application of complexity to the existence of equilibrium, not whether the equilibrium is hard to compute but whether it exists.

Roughly Roughgarden and Talgam-Cohen show if a certain kind of a pricing equilibrium holds then one can get an efficient algorithm for a certain kind of reduction. Under reasonable complexity assumptions (like P <> NP) such reductions can't exist and so neither can the equilibrium.

Late Wednesday came the Complexity business meeting, the first open business meeting of the now unaffiliated Computational Complexity Conference. There were 84 attendees, a little bit down from last FCRC but higher than last year. There were 30 papers accepted out of 110 submissions. The 2016 conference will be held in Tokyo May 29-June 1.

There was much discussion on where Complexity will be in 2017 and on which journal will get the special issue for the next three years. Watch my twitter to see when they get set.

STOC Business Meeting

This week I'm at the Federated Computing Research Conference in Portland, a collection of many mostly ACM conferences. Last night was the STOC business meeting.

The meeting had beer but the beer had no alcohol. Not an auspicious start.

289 registrants, a bit lower than the past few STOCs. With EC and Complexity at the same conference you would think STOC would draw larger but it doesn't.

The conference had 93 accepted papers out of 347 papers. Best papers (copied from proceedings):
The papers “Exponential Separation of Information and Communication for Boolean Function”, by Anat Ganor, Gillat Kol and Ran Raz, “2-Server PIR with sub-polynomial communication” by Zeev Dvir and Sivakanth Gopi, and “Lower bounds on the size of semidefinite programming relaxations ” by James Lee, Prasad Raghavendra and David Steurer, were selected for the STOC Best Paper Award. The paper “Inapproximability of Nash Equilibrium”, by Aviad Rubinstein, was selected for the Danny Lewin Best Student Paper Award.

These and the 2015 STOC papers are publicly accessible via acm-stoc.org/stoc2015/toc.html forever. part of a new STOC website with a history of past conferences.

Babai and Spielman and Teng officially received the awards we've mentioned below. The distinguished service award went to Avi Wigderson.

As of July 1 we have a new SIGACT Executive Committee: Paul Beame (ex-Chair), Anna Karlin, Michael Mitzenmacher (Chair), Toni Pitassi, Eric Vigoda, Ryan Williams

Lots of useful information about funding and more in  Salil Vadhan's CATCS Report.

Upcoming conferences:

• FOCS 2015 October 17-20 in Berkeley
• SODA 2016 January 1012 in Arlington, VA.Abstract deadline July 1
• STOC 2016 - Cambridge, MA June 18-21 collocated with SoCG
• STOC 2017 - Potential "Theory Festival" - see below
• STOC 2018 - 50th STOC possibly near Marina del Ray, the home of the first STOC

The business meeting ended with a discussion about a "Theory Festival" for STOC 2017. The goal is to get STOC to be a "must attend" meeting the way SOSP is for systems and SIGCOMM for networking. Check out the many discussions on Boaz Barak's blog.

First issue of SIGACT news where I wasn't the editor. But...

I posted at some earlier time that I was resigning from the editorship of SIGACT NEWS book review column, and handing the reins over to Fred Green (who is older than me, so `handing over the reins' might not be quite right).

You can find the column on Fred's webpage here.

I wish him well of course. He's off to a great start:

1. The Cult of Pythagoras: Math and Myths by Alberto A. Martinez. Review by Bill Gasarch.
2. Infinitesimal: How a dangerous mathematical theory shaped the modern world, by Amir Alexander. Review by Bill Gasarch.
3. Martin Gardner in the Twenty-First Century, edited by Michael Henle and Brian Hopkins. Review by Bill Gasarch.
4. Algorithmic Barriers Falling: P=NP?, and The Essential Knuth, both by  Edgar Daylight. Review by Bill Gasarch.
5. Love and Math: The Heart of Hidden Reality by Edward Frenkel. Review by Bill Gasarch.
6. Structure and Randomness: Pages from Year One of a Mathematical Blog by Terence Tao. Review by Bill Gasarch. 

Why so many reviews by... me?!   Fred writes that its a tribute to my service which is of course true and appreciated. But I also note that when I was editor it would have been... odd? impolite? to have all of the columns by me in a single issue. But having Fred do it is fine. When Fred resigns 17 years from now (not to give away his age, but I think he'll retire before then), perhaps his successor will do that same.

There are still books to review! Here is a list of books I still have in my office. If you want to review them email both gasarch@cs.umd.edu, fgreen@clarku.edu.
Email the books and also your physical address to send it to.
 ALGORITHMS

ReCombinatorics: The algorithmics of ancestral recombination graphs and
phylogenic networks by Gusfield.


Tractability: Practical approach to Hard Problems.  Edited by Bordeaux, Hamadi, Kohli.

Recent progress in the Boolean Domain.  Edited by Bernd Steinbach


PROGRAMMNG LANGUAGES

Selected Papers on Computer Languages by Donald Knuth.

MISC COMP SCI

Introduction to reversible computing by Perumalla.


Digital Logic Design: A Rigorous Approach by Even and Medina

CoCo: The colorful history of Tandy's Underdog Computer by Boisy Pitre and
Bill Loguidice.

MATH AND HISTORY

Professor Stewart's Casebook of Mathematical Mysteries by Ian Stewart.

The Golden Ratio and Fibonacci Numbers by Richard Dunlap.



Mathematics Galore! The first five years of the St. Marks Institue of Mathematics by Tanton.

Mathematics Everywhere. Edited by Aigner and Behrends.

An Epsiodic History of Mathematics: Mathematical Culture Through Problem Solving by Krantz.

Proof Analysis: A Contribution to Hilbert's Last Problem by Negri and Von Plato.

A Metric Group Product

A guest post by Dylan McKay, recently graduated from Georgia Tech and soon to be PhD student at Stanford.

[Editor's Note: Turns out the given solution doesn't work and whether a metric group product over the non-negative reals exists remains open. Update 7/10/18: A non-constructive solution was posted on Mathoverflow back in 2010]

Here a cute puzzle motivated by a pair of undergrads and their poor understanding of what the phrase “Algebraic Geometry” really should mean:

Find a function f from the nonnegative reals to the nonnegative reals that satisfies the group axioms and the metric axioms, or prove that there is no such function. That is, find an f:RxR→R such that (R,f) is a group and a metric space. (I am using R to refer to the set of nonnegative real numbers).

As a quick reminder, the group axioms are:

- Closure: f(a,b) must be in R

- Identity: There must exist an element e such that for all a, f(e,a)=f(a,e)=a

- Associative: for all a,b,c, f(f(a,b),c) = f(a,f(b,c))

- Inverse: for all a, there must exist a b such that f(a,b)=e

And the metric axioms are:

- f(a,b) = 0 iff a=b

- f(a,b) <= f(a,c) + f(c,b)

- f(a,b) = f(b,a)

One really bizarre thing about this supposed function is that, what this metric does is essentially guarantee that, given some number x, every other number has a distance from x that is unique -- no other number is exactly that distance away! This is quite counterintuitive to how we think about distance. Each number is its own distance from 0, though, which is very much in line with intuition.

(If you want to solve the puzzle yourself, don't read any further until you do! Unless you want some hints, in which case, look at the next paragraph.)

We can, however, make some other observations that point us in the right direction. For instance, f(e,e) = 0 from the metric axioms and f(e,e) = e from the group axioms, so you get that e = 0. From this and the metric axioms, you get that every element must be it’s own inverse!

Here, we are reminded of a pretty familiar and trusty function, this exclusive-or (XOR) function! And indeed, this will lead us a to a solution.

Here is our candidate function:

Consider x and y in their binary representations. If they do not have the same number of bits to the left of the decimal point (or should it be called the binary point?), pad the one with fewer such bits with 0’s so they have the same number of such bits. Then f(x,y) will be the number represented in binary by the bit-wise exclusive or of these two strings (maintaining the same number of bits to the left of the decimal point, of course). And there we have it: our function!

Now of course, we still need to prove that it satisfies the axioms. But if you do not believe that it does, work it out for yourself! Each axiom is pretty simple to check, especially as we are (for the most part) familiar with this as an extension of an idea for some smaller groups. Well, all of them except our good friend the triangle inequality. This one actually isn’t too bad, but if you have trouble, I will include a hint at the bottom of the post.

Anyway, I hope you liked our puzzle!

Thanks!

-Dylan

Hint for triangle inequality: XOR and addition are really similar, but there is a key difference between them that make a XOR b <= a + b.

The city where the book publishers resides is Funkytown!

A while back  when I got back the galleys for a paper the publisher wanted to know the complete postal address of one of the co-authors and also the city where the publisher of a book in he bibliography was printed. The publisher didn't really have a city, so I wrote in FUNKYTOWN.

Is it important in a paper to have the city that the publisher is in? I am assuming that at one time it was (though I am not sure why even that is) but now it is
completely irrelevant.

Is it important to have an authors postal address? I can't imagine why nowadays in 2015, though this I can imagine as being important in an earlier era.


Question: What information do publishers ask for in galleys that are no longer relevant? Why do they? When will they stop?

Langs with provably bigger CFG's then CSG's

In a prior blog entry HERE I discussed very large differences in the size of machines. I didn't discuss CFG vs CSG so I'll do that now. We assume that the CFGs and CSGs are in Chomsky Normal Form (for CSG this means that RHS is any rule is of length 1 or 2).

The following are known:

1)  Let PERMn be the set of permutations of {1,...,n} (so  PERM3 is {123,132,213,231,312,321}). Then (1) there exists a CSG for PERMn of size O(n²), but (2) every CFG for PERMn requires size 2^{Ω(n)  .} The upper bound is easy, the lower bound is by Ellul, Krawetz, Shallit, Wang Here.

2) Let Wn = { ww : |w|=n}.  Then (1) there exists a CSG for Wn of size O(n), but (2) every CFG for Wn requires size  2^Ω(n). The upper bound we leave to the reader.    Filmus HERE for the lower bound.(ADDED LATER: In Comment Below Filmus (yes, the same one!) claims Wn has a CSG of size O(log n)).

3) Let Sn = { w : |w|=3n and the numbers of a's, b's, c's in w are all the same}. Then (1) there exists a CSG for Sn of size O(log n) but (2) every CFG for Wn requires size 2^Ω(n). The upper bound is  from Beigel and Gasarch (If you know of an earlier source leave a polite comment.)  The lower bound is from Filmus (the ref above).(ADDED LATER- In Comment Below Filmus (yes, the same one!) corrects me, his paper did not show Sn has exp lower bound, and in fact Sn DOES have a CFG of size O(n^3)).

4)  The languages above are (informally) natural. If one goes to unnatural languages then there is a result where the CFG is GINORMOUS:  For all f  ≤_T HALT, for all n, there is a lang Ln such that (1) there exists a CSG for Ln of size n, but (2) every CFG for Ln has  size ≥  f(n). First proven by Meyer, but a new proof by Beigel and Gasarch is in the paper pointed to above. (See that paper for the history.)

The results stated above are suitable for an ugrad formal lang theory course.

Are there natural langs with triple-exp or larger gap between CFG's and CSG's? There are very few techniques to get lower bounds for CFG's (the papers of Ellul et al, and Filmus, are the only ones I know) so  new techniques may be needed. Are  there even any good candidates for natural languages  with small CSG's and really really large CFG's?

Award Season

László Babai will receive the 2015 Knuth Prize and Daniel Spielman and Shang-Hua Teng will receive the 2015 Gödel Prize. ACM issued a press release for both awards which will be presented at the upcoming STOC at FCRC.

Babai did seminal research on interactive proofs, communication complexity, group algorithms and much more. One cannot count the number of PhD theses in mathematics and computer science that can trace themselves back to some initial work by Babai. I was lucky to have Laci Babai as a colleague, mentor and friend during my years at the University of Chicago.

Spielman and Teng, who received the 2008 Gödel Prize for smooth analysis, won again for three papers using nearly linear time Laplacian solvers for a series of graph problems.

• Spectral sparsification of graphs. SIAM J. Computing 40:981-1025, 2011.
• A local clustering algorithm for massive graphs and its application to nearly linear time graph partitioning. SIAM J. Computing 42:1-26, 2013.
• Nearly linear time algorithms for preconditioning and solving symmetric, diagonally dominant linear systems. SIAM J. Matrix Anal. Appl. 35:835-885, 2014. 

The ACM Awards ceremony later this month will have a number of theory related prizes.

• Dan Boneh is the recipient of the ACM-Infosys Foundation Award in the Computing Sciences for the development of pairing-based cryptography and its application in identity-based encryption.
• James Demmel is the recipient of the Paris Kanellakis Theory and Practice Award for his work on numerical linear algebra libraries, including LAPACK.
• Charles Leiserson will receive the ACM-IEEE CS Ken Kennedy award for his work on parallel computing.
• Jon Kleinberg is the 2014 recipient of the ACM – AAAI Allen Newell Award for social and information networks, information retrieval, and data science, and for bridging computing, economics and the social sciences.
• New ACM Fellows include Al Borodin, Faith Ellen, Michael Kearns, Valerie King, Yishay Mansour, Michael Mitzenmacher, Omer Reingold, Ronitt Rubinfeld and Aravind Srinivasan.

Who wins this bet?

Alice and Bob are at an auction and Alice wants to buy an encyclopedia set from 1980. Bob says don't buy that, you'll never use it. In this age of Wikipedia and Google and THE WEB.  Alice says you don't know that.  They agree that Alice will spend no more than $20.00 on it (and she does win it at $20.00) and that:

If Alice does not use the encyclopedia within 5 years then she owes Bob $10.00. If she does use it then Bob owes Alice $10.00.

3 years later its really cold outside. Alice's house is not that well insulated. So she takes carpets against the bottom cracks in the door and weights them down with the volumes of the encyclopedia. This helps her keep warm and cuts down on her heating bill.

Alice says I used the encyclopedia. Pay up! In your face! I used them! You were wrong!

 Bob says You didn't use them to look anything up. So that doesn't count.

Alice and Bob are asking YOU do decide. So leave comments either way and whoever has more votes before my next post wins! Feel free to leave reasons as well to persuade the other readers.

John Nash (1928-2015)

John Nash and his wife Alicia died in a taxi accident, returning from the airport after he received the Abel prize in Norway. The public knew John Nash as the "Beautiful Mind" of book and screen, but we knew him as one of the great geniuses of the 20th century. Rakesh Vohra captures Nash's life and work, including his amazing letters to the NSA.

I briefly met John Nash at some MIT alumni events in New Jersey when I lived there (even though neither of us were MIT undergrads). He would come with his wife and son, the son wearing a winter coat no matter the season. Nash just seemed like any other introverted scientist and was happy to talk though understating his research ("I did some work in game theory") and never revealing the challenging life he led.

Now that John and Alicia have found their final equilibrium, may we remember them and Nash's vision of using mathematics to understand the world we live in.

An Intentional and an Unintentional teaching experiment regarding proving the number of primes is infinite.

I taught Discrete Math Honors this semester. Two of the days were cancelled entirely because of snow (the entire school was closed) and four more I couldn't make because of health issues (I'm fine now). People DID sub for me those two and DID do what I would have done. I covered some crypto which I had not done in the past.

Because of all of this I ended up not covering the proof that the primes were infinite until the last week.

INTENTIONAL EXPERIMENT: Rather than phrase it as a proof by contradiction I phrased it, as I think Euclid did, as

Given primes p1,p2,...,pn you can find a prime NOT on the list. (From this it easily follows that the primes are infinite.)

Proof: the usual one, look at p1xp2x...xpn + 1 and either its prime or it has a prime factor not on the list.

The nice thing about doing it this way is that there are EASY examples where p1xp2x...xpn+1 is NOT prime

(e.g., the list is {2,5,11} yields 2x5x11 + 1 = 111 = 3 x 37, so 3 and 37 are both not in {2,5,11})


where as if you always use the the product of the first n primes then add 1, you don't get to a non-prime until 2x3x5x7x11x13 + 1 = 30031 = 59x 509.

They understood the proof better than prior classes had, even prior honors classes.

UNINTENTIONAL: Since I did the proof at the end of the semester they ALREADY had some proof maturity, more so than had I done it (as I usually do) about 1/3 of the way through the course.

They understood the proof better than prior classes had, even prior honors classes. Hence I should proof all of the theorems the last week! :-)

But seriously, they did understand it better, but I don't know which of the two factors, or what combination caused it. Oh well.

Theory Jobs 2015

In the fall we list theory jobs, in the spring we see who got them. Like last year, I created a fully editable Google Spreadsheet to crowd source who is going where. Ground rules:

• I set up separate sheets for faculty, industry and postdoc/visitors.
• People should be connected to theoretical computer science, broadly defined.
• Only add jobs that you are absolutely sure have been offered and accepted. This is not the place for speculation and rumors.
• You are welcome to add yourself, or people your department has hired.

This document will continue to grow as more jobs settle. So check it often.

Edit

Fiftieth Anniversary of the Publication of the seminal paper on Computational Complexity

Juris Hartmanis and Richard Stearns in a photo dated May 1963. The main theorem from their paper is on the board later improved by Hennie and Stearns. Photo courtesy of Richard Stearns.

The seminal paper of Juris Hartmanis and Richard Stearns, On the Computational Complexity of Algorithms, appeared in the Transactions of the American Mathematical Society in the May 1965 issue. This paper gave the name to the field of Computational Complexity which I took for the name of this blog. Hartmanis and Stearns received the Turing Award in 1993 for this work.

I've mentioned this paper several times in the blog before, including as a favorite theorem. Hartmanis and Stearns first formalized and truly popularized the idea of measuring time and other resources as a function the problem size, laying the foundation for virtually every paper in computational complexity and algorithms to follow.

Both Hartmanis and Stearns wrote about those early days. The main breakthroughs for their paper started in November 1962 and on December 31 Hartmanis wrote in his logbook "This was a good year," A good year indeed.

The law of the excluded middle of the road republicans

In the book Hail to the Chiefs about the presidents, when pointing to a race between two people who had no business being president (I think it was Franklin Pierce vs Winfield Scott) wrote something like That's the thing about elections, someone has to win. 

 Looking at the republicans running for the nomination I can (with the help of reading many of Nate Silver's Columns) tell you why, for each one, they can't win the nomination. Note that this is not a partisan thing. But again, someone has to win. Is it possible to have the statements

A1 can't win AND A2 can't win AND .... AND An can't win

and yet someone wins?

Here is a list of the candidates and why they can't win.

1. Jeb Bush is what passes for a front runner nowadays. Has the money, does not have the party (very few endorsements), and not doing well in polls in Iowa or New Hampshire, the first two states. Possibly because he does not hate Obama enough.  Its an interesting question of whether the party, the people, or the money pick the candidate. In the past its been the party and the money together, but that might be changing. CAN"T WIN: Viewed as too moderate by the people who go to Caucus's. Also some people may be put off by the family name. THOUGHT: If H CLINTON was not the likely Democratic candidate, thus making the family name thing a less of an issue in the general, I don't think he would have run at all. UPDATE- Dropped out after the South Carolina Primary. 
2. Marco Rubio. Running for president for the first time is hard. The republicans rarely nominate someone who hadn't run before  (W in 2000, Ford in 1976 which is an out lier since he was prez). Perhaps the voters/party/money like someone they are familiar with OR perhaps first timers make mistakes. CAN"T WIN: Will make some mistake and some might think its not his turn since he's so young. Also, Senators have it rough since they sometimes vote for bills in funny ways, TRIVIA: The electoral college reps from state X cannot cast both the Prez and Veep vote for people both from state X. So you won't see a Bush-Rubio or Rubio-Bush ticket  UPDATE- Dropped out in Mid March after losing his home state to Trump. Very odd in that various pundits pointed to him as being the hope to beat Trump even though he only won Minnesota and Puerto Rico. Looks like running for prez IS hard for first times- though Trump seems to be managing..
3. Rick Perry. He lost in 2012 because either he was too soft on immigrants (he supported some sort of Dream Act) or because of his Whoops moment in a debate. Frankly I have sympathy on that one--- I also sometimes  forget which cabinet positions I want to get rid of. He has tried to cure both of his problems by flip-flopping (or evolving) on immigration and by wearing glasses so at least he looks smart. CAN"T WIN:  His past failure makes him not look that serious this time around, and he will likely have another WHOOPS moment. UPDATE: He dropped out in September. His speech saying he was dropping out was pretty good (I am not being sarcastic).
4. Scott Walker. Like Rubio but might be in better shape because he's a governor. Still, people in his home state are beginning to turn on him, a bad sign. He may soon have the same problem that Gov Brownback of Kansas has--- you promise to cut taxes, close loopholes, and cut spending, and that might actually be a good idea if done right, but you cut taxes , don't close loopholes, cut spending in stupid places like education, run up a big debt, destroy your states economy, and ... get re-elected. CAN"T WIN: Having not run before Walker will   say something stupid. And as a gov of a moderate state I suspect some moderate things he did may be a problem for hard core republican voters.Plus his states current problems may be an issue. UPDATE: He dropped out in September. See Note on Bobby Jindal.
5. Ted Cruz. Which of these quotes did Ted Cruz really say and which did I hear in some satirical setting (as a fan of The Daily Show, The Colbert Report, John Oliver's show, The Capitol Steps, others, I lose track of where I heard what) (1) There was no ebola before Obamacare, or (2) Net neutrality is Obamacare for the internet. I'll answer at the end of the post. He is now using Obamacare himself. . CAN"T WIN: A niche candidate with a small but loyal set of supporters. Not enough CAVEAT:  Might be running to get some points of view out there like Adlai Stevenson and Barry Goldwater, though they got all the way to the nomination which I doubt he can.
6. Rand Paul. Interesting mathematically: an authenticity -  electability trade off.  Some people are attracted to his libertarianism, authenticity and consistency,  but not enough to get him anywhere close to the nomination. So he changes some of his positions to be more mainstream but less libertarian, authentic, and consistent. Alas, those who trade their integrity for electability end up with neither. CAN"T WIN: His evolving views might lose him his base but not gain him any establishment cred. Also he has the chicken-egg problem in that even people who like him don't think he can win the general election. (Ted Cruz and others on this list may also have this problem.) ADDED IN FEB- DROPPED OUT AFTER IOWA CAUCUS.
7. Mike Huckabee. Won't get much  support beyond his Social Conservative base. His stance on Same Sex Marriage will hurt him outside of the social conservatives, especially in 2016 (as opposed to his last run in 2008).  I'm surprised he's running- he got what he wanted from his last run, a show on FOX News. CAN"T WIN: Was a moderate on some economic and immigration issues (he may also `evolve' which won't work), a conservative on social issues, is just the wrong mix for current  republican primary voters. Note that many candidates are trying to avoid the Same Sex Marriage question as they know that being opposed to it will hurt them  in the general. Plus some don't want to be on the wrong side of history (or as the kids say WSOH) .  Politics- sometimes you're forced to have a public opinion that you disagree with and know will make you be on the WSOH , but you're stuck with it. I think Huckabee is sincere in his opposition to same sex marriage but he must surely know he's on the WSOH. He's hoping for a win in Iowa like he had in 2008. I predict that if he doesn't get it he'll drop out. ADDED IN FEB- DROPPED OUT AFTER IOWA CAUCUS
8. Ben Carson. I suspect he is actually running to be a FOX News commentator. At that he might succeed. CAN"T WIN:  First timer, never ran for anything, he'll be a curiosity not a candidate. Which of the following did he say:  (1) Obama is a sociopath (2)  Obamacare is like slavery. This may even hurt him with the Republican hard core who want someone who can win. CAVEAT: is being African-American going to help or hurt? I doubt his campaign will get far enough to tell. The other candidates and the debate panelists (in the sanctioned debates) will treat him with kid gloves to avoid being called racist. UPDATE- saying absurd things about Obama seems to not hurt any candidate. ADDED LATER- he dropped out in early March, prob because of his poor showing on Super Tuesday.
9. Carly Fiorina. She said that our founding fathers did not intend there to be a political class, what we now call politicians. They intended for ordinary people (like the president of HP), for the good of their community, to serve in office. She left out that the founding fathers also did not intend for women to be president. CAN"T WIN: First timer, never ran for anything.  (correction added later: She ran for Senator of California . She got the nomination but lost to Incumbent Barbara Boxer.) CAVEAT: is being female going to help or hurt? I doubt her campaign will get far enough to tell. The other candidates and debate panelists (in the sanctioned debates) will treat her with kid gloves to avoid being called sexist. HER HOOK: She claims that as a women she can neutralize H Clinton' women-advantage in the general. Interesting that she is making an electability argument instead of a policy argument, given that she has no chance of being elected. ADDED LATER- Dropped out after the NH primary.
10. Chris Christie. CAN"T WIN: Hated inside of New Jersey. Hated outside of New Jersey.ADDED LATER- Dropped out after the NH primary.
11. Bobby Jindal. Once said the Republicans have to stop being the stupid party. Later said some stupid things about Muslims in America. CAN"T WIN: If he ran as the moderate sane voice who will rescue the party from itself, he might get some traction. If he runs as anything else he has too much competition. Also a first-time-runner which is hard. UPDATE- He dropped out in November. He was in double digits (not his percent- the number of people who wanted him to be Prez- his wife, one of his kids, the other is in the Cruz-Camp.) He tried to be BOTH Tea-party AND establishment which was foreshadowed by saying Reps shouldn't be the stupid party (an establishment thing to say) and being racists towards Muslims (a Tea-Party thing). Was not liked by either. Scott Walker may have had the same problem. Rubio may have the same problem.
12. Lindsey Graham. CAN"T WIN:  Has  worked with democrats which should be a PRO but it's a CON. He's running to be the voice of more troops-on-the-ground in the mideast, not to win. Since Americans don't really want troops on the ground I doubt this will go anywhere; however, instead of troops the other candidates will show their machismo by deporting Muslims. UPDATE- He Dropped out.
13. John Kasich. Gov of Ohio. CAN"T WIN:  Not that well known. Democrats have nominated unknowns (B Clinton, B Obama) but republicans almost never do (W might count).
14. Donald Trump (ADDED LATER). Some people say that corporate America controls this country. If we make Donald Trump Prez we're just cutting out the middle-man. Won't get the nomination--- over half of republicans say they would never vote for him--- but his running is a farewell gift to Jon Stewart.
15. Rick Santorum (ADDED LATER). Against Birth Control? Really? Google him to find other reasons he can't win. Odd thing- usually the person who came in second the last time around has a pretty good shot at getting the nomination this time around, but the fact that he came in second last time may be a sign of how weak the field was last time. Is hoping for a win in Iowa like he had in 2012. I predict that if he doesn't do well in Iowa he'll drop out. UPDATE- DROPPED OUT AFTER THE IOWA CAUCUS.
16. George Pataki (ADDED LATER) Would be at home in the democratic party. I want to see him challenge Hillary from the right. Oh, he's a republican? But he's pro-choice, pro-gay, anti-gun, and doesn't seem to hate Obama. UPDATE- DROPPED OUT.
17. Rob Portman. (ADDED LATER) I saw him on a list of possible nominees. Not under `declared', not under `exploratory committee) (Chris Christe and Scott Walker are still exploring, which surprised me- I thought they already declared), but on a list of `No indication'. Not sure why he's on any list; however, as Rick Perry taught us last time (and this is not a joke) you need to start early.UPDATE- never got into the race.
18. Jim Gillmore (ADDED LATER).  Never made it into the adult debates nor even the kiddie table. Do not know why he is running since he does not have a chance and is not trying to push some sort of issue (that may be why Rand Paul is running- to promote a viewpoint) UPDATE: In the least important political story of the year, Jim Gillmore dropped out after the NH primary. 

UPDATE IN FEB: After NH primary its  down to 6. Maybe now they CAN all fit on a debate stage!

UPDATE IN FEB: After SC primary Jeb dropped so its down to 5. They can now all fit into a car!

UPDATE ON MAR 2. Ben Carson dropped out so its down to 4. Now they can have an intelligent debate.

UPDATE ON MAR 4. They spend some of the debate talking about the size of Donald Trump's... hands.

UPDATE ON MAR 16. Rubio Dropped out. Now its just Trump, Cruz, Kasich. They can all fit in
a Volkswagon.

There may be more (Donald Trump anyone? ADDED LATER- When I first posted this mentioning Donald Trump was supposed to be a joke. But political satire and reality may have finally merged with a reality-TV star running for Prez). But my point is that it seems like nobody can win, yet someone has to.  Do you know other examples where A OR B OR C has to be true, yet none of A,B,C look plausible?

I can phrase this another way:

novices can't win  (e.g., Ben Carson) AND first timers can't win (e.g., Mario Rubio) AND too moderate can't win (e.g., perception of Jeb) AND unknowns can't win (e.g., John Kasich).

They all can't be right, but looking at it   `first timers' is prob the weak link in my reasoning. I could replace it with `inexperienced politician'. Even so, sure looks like nobody can win.

Ted Cruz: Both of the quotes I attribute to him he really did say.
 
I don't know Ben Carson's original quote but he backtracked to Obama reminds you of a psychopath,
which is much better than saying Obama IS a psychopath . But he never said sociopath, so the quote I gave is NOT from Ben Carson.

On Obamacare he said its the worst thing to happen in America since slavery. But later
opposite-of-backtracking said it was in a way like slavery because it robs you of  your ability to control your own life.

The Virtuous Cycle

Last week we have a celebration of the new CS graduates at Georgia Tech where each graduating student says what their plans are for next year. Other than the few going to grad school, they all have great jobs but the difference I see this year is how many are staying in Atlanta. Any CS student who wants to stay in Atlanta can find a great job in Atlanta.

A large number of companies are moving their headquarters or established large facilities in the Atlanta area and the reasons they give almost always include the students and researchers at Georgia Tech. We're starting to grow a good start-up culture here as well. 

Companies in Atlanta want our students and our research. That helps add to the prestige of Georgia Tech which in turn draws more companies. A virtuous cycle. A similar story is playing out in several other cities across the country and I even saw it in tiny but growing Bozeman where I visited Montana State earlier this week. Computer Science these days plays a major role if not the largest role in many of these industries.

All this growth leads to challenges such as finding the people and resources to meet the growing demands. All told though a good problem to have.

We don't want the success of the STEM fields to come at the expense of the rest of academics. It shouldn't have to be CS vs Classics, Physics or Philosophy. How we make it all successful will be the great challenge in higher education in the years to come.

Sizes of DFAs, NFAs, DPDAs, UCFG, CFGs, CSLs.

If A is a decider (e.g, DFA)  or generator (e.g., CFG) then L(A) is the language  that it decides or generates.

The following are well known:

L(DFA) = L(NDFA) ⊂ L(DPDA) ⊂ L(PDA) ⊂ L(CSL).

We are concerned with the  size of these devices. For a DFA and NDFA the size is
the number of states, for a DPDA, PDA the size is the sum of the stack alphabet and the number of states, and for CSL its the number of nonterminals.

If D and E are two classes of devices (e.g., DFAs and DPDAs) then a bounding function for (D,E)  is a function f such that if L is recognized by both a D-device and an E-device, and L is recognized by an E-device of size n, then it is recognized by a D-device of size ≤ f(n). We abbreviate b-fun

Readers of this column likely know that f(n)=2^n is a b-fun for (DFA,NFA) and prob know that this is tight. Below are some results that I suspect many readers don't know. Some of them  may be suitable for inclusion in an undergrad theory class. In what is below ≤ means Turing-Less-Than-Or-Equal.

1.  Stearns showed that f(n) = n^{n^{n^{O(n)}}} is a b-fun for (DFA,DPDA).
2.  Valiant improved this to double exp for a b-fun for (DFA,DPDA).
3.  Meyer and Fischer showed the 2^n lower bound for (DFA,NDFA). They also showed a lower bound of 2^{2^{O(n)}} for (DFA,DPDA). I think the question of closing the gap between Valiant's result and the Meyer-Fischer result is still open; however, if you know a ref please leave a comment.
4. Valiant showed that the if f is a b-fun for (DPDA,UCFG) then HALT ≤  f.
5. Schmidt showed that if f is a b-fun for (UCFG,CFG) then HALT ≤  f. 
6. Hartmanis showed that if f is a b-fun for (DPDA,PDA) then HALT ≤ f
7. Hay  showed that if f is a b-fun for (DPDA,PDA) then f is NOT computable-in-HALT. 
8. Beigel and Gasarch prove a general theorem from which they obtain the following: (a) if f is a b-fun  for (DPDA,PDA) then f ≤_INF. (It is easy to show that there exists a b-fun f ≤ INF for (DPDA,PDA) so the Turing degree is now precise), and (b) if f is a b-fun for (PDA,CSL) then SAME AS PART (a).

Results 3,4,5,6,7 can be restated in the following way--- we'll do result 6 as an example:

 If f  ≤  HALT then there exists infinitely many n such that there exists L_n such that  (a) L_n is DPDA, (b) there is a PDA for L_n of size n, (c) any DPDA for L_n is of size  at least f(n).

 Are there any ``for almost all n '' type bounds? There are but they are much weaker.  The following theorems are from the Beigel-Gasarch paper pointed to above.

1. For almost all n there exists a  cofinite (hence DPDA) L_n that has a PDA of size O(n) but any DPDA for it has size 2^2^{n^{Ω(1)}}. 
2. Same as point 1 but for PDA,CSL.

Both results 1,2 above use natural languages  in that they are not created for the sole purpose of proving the theorem, no diagonalization (my spell check says thats spelled wrong but I think its spelled right). Using a construction Beigel-Gasarch obtained (Meyer probably had the result 40 years earlier with a diff proof, see the Beigel-Gasarch paper  for historical details) that if f  ≤  HALT then for almost all n there is a lang L_n such that L_n has a CSL of size n but any PDA for it is of size at least f(n).

Is Logarithmic Space Closed Under Kleene Star?

A Georgia Tech student asked the title question in an introductory theory course. The instructor asked his TA, the TA asked me and I asked the oracle of all things log space, Eric Allender. Eric didn't disappoint and pointed me to Burkhard Monien’s 1975 theorem

L is closed under Kleene star if and only if L = NL.

L here is the set of problems solved in deterministic O(log n) space and NL is the nondeterministic counterpart. For a set of strings A, Kleene star, denoted A^* is the set of all finite concatenations of strings of A. For example if A = {00,1} then A^* = {ε, 1, 00, 11, 001, 100, 111, 0000, 0011, 1100, …} where ε is the zero-length string.

Kleene star comes up in regular expressions but also makes for many a good homework problem.

1. Show that if A is in NL then A^* is also in NL.
2. Show that if A is in P then A^* is also in P.
3. Show that if A is c.e. (recognizable) then A^* is also c.e.

Problem 1 above is equivalent to saying NL is closed under Kleene star and implies the “if” part of Monien’s result. Here is a simple proof of the other direction that L closed under Kleene star implies L = NL.

Consider the following NL-complete problem: The set of triples (G,s,t) such that G is a directed graph with a restriction that all edges (i,j) have i<j and there is a path from node s to node t.

Define the following language

B = {G#i+1#G#i+2#...#G#j# | there is an edge (i,j) in G}

B is computable in log space and the string G#s+1#G#s+2#…#G#t# is in B^* if and only if there is a path from node s to node t. QED

Allender, Arvind and Mahajan give some generalizations to log-space counting classes and also notes that there are languages computable in AC⁰ (constant-depth circuits) whose Kleene star is NL-complete. B above is one such set.

Advice on running a theory day

Last semester I ran a THEORY DAY AT UMCP. Below I have ADVICE for people running theory days. Some I did, some I didn't do but wish I did, and some are just questions you need to ask yourself.

1) Picking the day- I had two external speakers (Avrim Blum and Sanjeev Arrora) so I was able to ask THEM what day was good for THEM. Another way is to pick the DAY first and then asking for speakers.

2) Number of external speakers- We had two, and the rest were internal. The external speakers had hour-long talks, the internal had 20-minute talks. This worked well; however, one can have more or even all external speakers.

3) Whoever is paying for it should be told of it towards the  beginning of the process.

4) Lunch- catered or out? I recommend catered if you can afford it  since good time for people to all talk. See next point.

5) If its catered you need a head count so you need people to register. The number you get may be off- you may want to ask when they register if they want lunch. Then add ten percent.

6) Tell the guest speakers what time is good for them to arrive before they make plans so you can coordinate their dinner the previous night.

7) If have the money and energy do name tags ahead of time. If not then just have some sticky tags and a magic marker.

8) Guest speakers- getting them FROM Amtrak or Airport to dinner/hotel --- give them a personal lift (they may be new to the city and a bit lost). Getting them from the event back TO the airport or amtrak, can call airline limo and taxi. (though if can give a ride, that's of course good.)

9) Pick a day early and stick with it. NO day is perfect, so if someone can't make it, or there is a faculty meeting that day, then don't worry about it.

10) Have website, speakers, all set at least a month ahead of time. Advertise on theory-local email lists, blogs you have access to, and analogs of theory-local for other places (I did NY, NJ, PA). Also email people  to spread the word.

11) Advertise to ugrads. Students are the future!

12) If you are the organizer you might not want to give a talk since you'll be too busy doing other things.

13) Well established regular theory days (e.g., NY theory day) can ignore some of the above as they already have things running pretty well.