What is Data Science?

The Simons Institute at Berkeley has two semester long programs this fall, Lower Bounds on Computational Complexity and Foundations of Data Science. The beginning of each program features a "boot camp" to get people up to speed in the field, complexity last week and data science this week. Check out the links for great videos on the current state of the art.

Data Science is one of those terms you see everywhere but not well understood. Is the the same as machine learning? Data analytics? Those pieces only play a part of the field.

Emmanuel Candès, a Stanford statistician, gave a great description during his keynote talk at the recent STOC theoryfest. I'll try to paraphrase.

The basic scientific method works as follows: You make an hypothesis consistent with the world as you know it. Design an experiment that would distinguish your hypothesis from the current models that we have. Run the experiment and accept, reject or refine your hypothesis as appropriate. Repeat.

The Higgs Boson followed this model as a recent example.

Technological Advances have given us a different paradigm.

1. Our ability to generate data has greatly increased whether it be from sensors, DNA, telescopes, computer simulations, social media and oh so many other sources.
2. Our ability to store, communicate and compress this data saves us from having to throw most of it away.
3. Our ability to analyze data through machine learning, streaming and other analysis tools has greatly increased with new algorithms, faster computers and specialized hardware.

All this data does not lend itself well to manually creating hypotheses to test. So we use the automated analysis tools, like ML, to create models of the data and use other data for testing those hypotheses. Data science is this process writ large.

We are in the very early stages of data science and face many challenges. Candès talked about one challenge: how to prevent false claims that arise from the data not unrelated to the current reproducibility crisis in science.

We have other scientific issues. How can we vouch for the data itself and what about errors in the data? Many of the tools remain adhoc, how can we get theoretical guarantees? Not to mention the various ethical, legal, security, privacy and fairness issues that vary in different disciplines and nations.

We sit at a time of exciting change in the very nature of research itself, but how can we get it right when we still don't know all the ways we get it wrong. 

Is Trivium (the Stream Cipher) used?

This Fall I am teaching the senior course in Crypto at UMCP. Its a nice change of pace for me since REAL people REALLY use this stuff! Contrast to last Spring when I taught

                   Ramsey Theory and its `Applications'

There is one topic in the Crypto course that LOOKS really useful but I can't tell if it IS being used, so I inquire of my readers. (I will probably come across others topics like that in the future.)

A Secure Stream Cipher is (informally) a way to, given a seed and optionally an Init Vector (IV), generate bits that look random. Alice and Bob communicate the seed either in person or over a private channel or perhaps by using RSA (or some other public key system) and they then both effectively have a very long string of random bits. They send the IV in the clear. They can then do one-time-pad (really a psuedo-one-time-pad). There are other uses for random-looking bits as well.

So what is needed is a Secure Stream Cipher.  Trivium seems to be one such. According to the Trivium wiki

It was submitted to the Profile II (hardware) of the eSTREAM compeition by its authors Christophe De Canniere and Bart Preneel, and has been selected as part of the portfolio for low area hardware ciphers (Profile 2) by the eSTREAM project. It is not patented.

According to these papers: here and here, and the Wikipedia entry, here the following are true:

1) Trivium takes an 80 bits seed and an 80 bit IV

2) The implementation is simple and is already in hardware. Around 3000 logic gates.

3) There are reasons to think its random-looking but no rigorous proof.

4) So far it has not been broken, though its not clear how many people have tried. Thats goes to my question-- how widely used it is it?

5) Trivium need 1152 steps in the init phase. If it only does 799 then The Cube Attack can break it in 2^68   which is better than the naive algorithm of trying every key and IV (2^160) but still not feasible.

6) Trivium is also An American Metal Band and a Medieval theory of education. Its a good name for a band. See my post What Rock Band Name Would you Choose? for fictional good names for bands with a math or theoretical cs connection.

OKAY, back to the main topic:

SO my questions:

Is Trivium used?

If so then by whom and for what (for the psuedo 1-time pad?) ?

If not then why not (e.g., some of of my points above are incorrect)? and should it be instead
of what is being used?

Katherine Johnson (1918-)

Katherine Johnson is celebrating her 100th birthday today. This is the first centenary post we've done for a living person.

The movie Hidden Figures made her story famous: In 1952, she joined NACA, the predecessor of NASA, in the all-black West Area Computing section of the Langley lab in Virginia. During the "space race" of the 50's and 60's she worked on trajectory analysis for the early human spaceflights. In 1960, she was the first woman to co-author a technical report for NASA on placing satellites over a specific latitude and longitude.

The West Area Computing section had human computers working on the critical calculations for air and space travel. Soon NASA started moving that work to IBM machines but much as we don't fully trust machine learning today, humans didn't initially trust these computers. John Glenn's first orbital mission required complex calculations to track his flight. He insisted on Katherine Johnson working out the computations herself, which she did. "If she says they're good then I'm ready to go".

In 2015, then President Obama awarded Katherine Johnson the highest US civilian honor, the Presidential Medal of Freedom.

The Zero-One Law for Random Oracles

A couple of years ago, Rossman, Servedio and Tan showed that the polynomial-time hierarchy is infinite relative to a random oracle. That is if you choose each string independently to be in or out of an oracle R with probability one, the polynomial-time hierarchy will be infinite relative to R with probability one. This is one in the measure theory sense, there are oracles where it is false, it is just that those oracles will occur with zero probability.

There are still a few open questions for random oracles, such as whether P = BQP, quantum and classical computing can solve the same problems efficiently.. We suspect that P is different than BQP relative to a random oracle because otherwise BQP would be the same as BPP unrelativized (and thus factoring is easy), but we have no proof. Could it be possible that this problem has no simple resolution, that P = BQP holds with probability 1/2 relative to a random oracle, or some other probability strictly between 0 and 1? As it turns out no.

Some statements do hold with intermediate probabilities. The sentence "0101 in R" holds with probability 1/2. Even for a fixed machine M, questions like "M^R accepts an infinite language" could hold with probability say 3/8.

But statements like P = BQP relative to R can't happen with intermediate probability. That's due to the Kolmogorov zero-one law. If you have a subset of oracles that are closed under finite differences, that set must occur with probability zero or one. Every statement about complexity classes has that property because we can hard wire finite differences of the oracle into the machine description without increasing the running time. It will change the machine but not the complexity class. So P = BQP holds with probability zero or one even though we can't tell which one yet.

The Kolmogorov zero-one law gives us a consistent look at complexity classes. Since the countable union of zero probability events still has probability zero, every finitely-described statement about complexity classes that hold with probability one, all simultaneously hold with probability one. While this random world does not match the unrelativized one, it does give us a relativized world where we can explore the different possible relationships between complexity classes.

Fractional Problems: 2.1-colorable, 2.8-SAT

Some graphs are 2-colorable, some graphs are 3-colorable, some graphs are...Does it make sense to say that a graph is 2.1-colorable? It does!(Source_ Factional Graph Theory by Schneinerman and Ullman-- I saw a talk on this by Jim Propp a long time ago.)

Def 1: A graph is (a,b)-colorable (with a \ge b) if you can assign to every vertex a set of b numbers from {1,...,a} such that if u and v are adjacent then the set of numbers are disjoint. Note that k-colorable is (k,1)-colorable. Let chi_b(G) be the least a such that G is (a,b)-col.
 The fractional chrom num of G is lim_{b-->infinity} chi_b(G)/b.

Def 3:  We restate the ordinary Chrom Number problem as an integer program (and NOT by using that
Chrom Num \le SAT \le IP).  In fact, our Int Prog will be LARGE. For every ind set I of G we have a 0-1 valued var x_I which will be 1 iff x_I is all one color. We want to minimize \Sum_I x_I with the constraint that, for every vertex v in the graph. sum_{v in I} x_I \ge 1, so every vertex is colored.
: Fractional Chrom number is what you get if you relax the above IP to an LP with x_I in [0,1] instead of {0,1}.

Defs 1 and 2  turn out to be equiv. The wikipedia entry on Fractional Chromatic Number (see here) is pretty good and has some applications to real world things.

QUESTION: 2-col is in P, 3-col is NPC. What about, say, 2.1-col. It turns out that, for every c>2, c-col is NPC.

Open question (which Jim Propp used to begin his lecture): Every planar graph is 5-col has an EASY proof. Every planar graph is 4-col has a HARD (or at least tedious) proof. Is there a nice proof that every planar graph is (say) 4.5-colorable? The answer is Yes, Every planar graph is 4.5 colorable.  I blogged on it here.

Are there other fractional problems related to NPC problems. YES- at a Dagstuhl there was a paper on (2+epsilon)-SAT. (by Austrin, Guruswami, Hastad) (see here).

What is fractional SAT? Lets recall ordinary k-SAT: every clause has k literals and you need to make at least one of them true. What if you wanted to make at least 2 of them true? (a/b)-SAT is if every clause has exactly b literals and you want an assignment that makes at least a in each clause true.

GOOD NEWS: for all epsilon, (2+epsilon) is NP-complete. Its not so much good that its true, but its good that its known.

BAD NEWS: The proof is hard, uses lots of tools.

ODD NEWS: The speaker said that they PROVED there was no easy proof.

I think its worth having your students try to DEFINE these terms on their own. The NPC proofs may be over their heads (they may be over my head), but the definitions are nice and the students might be able to derive them.

QUESTION: Do other NPC problems have Fractional versions? I would think yes. This could lead to a host of open problems OR perhaps they have already been asked. If you know of any, please comment.

How valuable is a Fields Medal?

(Johan Hastad won the Knuth Prize! The below post was written before I knew that but has a mild connection to it. See here for more info on the Hastad winning it, or see Lance's tweet, or see Boaz's blog post here. There will prob be other blogs about it as well. ADDED LATER: Lipton and Regan have a post on this here.)




The Fields Medal was recently awarded to

Caucher Birkar

Alessio Figalli

Peter Scholze

Akshay Venkatesh

I was going to try to give one sentence about what they did, but Wikipedia does a better job than I ever could so I point there: here. Terry Tao also has some comments on the Fields Medal here. So does Doron Zeilberger here.

How much is a Fields medal worth?

1) The winners get $15,000 each.

2) Winning a Fields medal gets one a higher salary and the ability to change schools, so the $15,000 might not be the main monetary part.  All Field Medalists are under 40 so the salary increases and such last for a much longer time then (say) a Nobel prize given for life achievements to someone much older. So you may rather win a Fields' medal when you are 39 than a Nobel when you are 70. The Abel prize is around 740,000 dollars and (I think) given for lifetime achievement so again, a Fields Prize may be better. (See here for more on the Abel Prize).  Which would I prefer to win? I would be delighted if that was my dilemma.

3) I am sure that none of the four winners went into math because of the allure of the $15,000 Fields Medal.

4) The title of this post is ambiguous. It can also be read as

how valuable is the actual medal?

The answer is $4000, much more than I would have thought. I only know this since it was recently stolen, see here.

This raises a linguistic question. The four people above can say

                                                          I WON a Fields Medal

The thief can say

                                                          I HAVE a Fields Medal

and hope that people don't quite realize that he didn't earn it.

(The article about the theft says the Fields medal is $11,500 dollars. Do they deduct the cost of the Medal itself? Or is the article wrong?)

While I Was Away

After the Oxford Workshop I enjoyed a two-week family vacation in Spain, where there was no rain in the plain, just very hot up to 106℉. The old Spanish cities knew how to optimize for shade and breeze, more than I can say for Oxford.

Meanwhile in a more moderate Brazilian climate, the International Congress of Mathematicians awarded their medals, including the Rolf Nevanlinna Prize to Constantinos Daskalakis in a year with several very strong candidates. The Nevanlinna prize gets awarded every four years to a researcher under 40 for contributions to mathematical aspects of information sciences. Costis was the then-student author of the 2004 Nash Equilbrium is PPAD-complete result and has gone on to be a leader in the algorithmic game theory community.

The ICM also distributes the Fields Medal, the highest honor in mathematics. Much ado is given to Peter Scholze who received the award this year at the age of thirty though remember that Alexander Razborov received his Nevanlinna prize at the age of 27 in 1990. Caucher Birkar also received the Fields Medal at the more standard age of 40 but had it for only a few minutes before it was literally stolen away.

I didn't realize how much I appreciate the convenience of Uber and Lyft until I had to get around cities where they don't exist. Meanwhile New York started to limit ride-sharing vehicles and I arrived in Madrid to a taxi strike protesting Uber in that city. The Yin and Yang of technology.

The Future of TCS Workshop, celebrating V Vazirani 60th, now online

On June 29, 2018, a workshop was held, in conjunction with STOC 2018, to celebrate the accomplishments of Vijay Vazirani on the  occasion of his 60th birthday, organized by his PhD students, Aranyak Mehta, Naveen Garg and Samir Khuller. The workshop was called "TCS: Looking into the Future" and true to the title, it was precisely that!  In front of a large, enthusiastic audience, left over from STOC, the star-studded lineup of speakers outlined some of the most avant-garde, far out ideas  on the future of computing.  Fortunately, this exciting and highly thought-provoking set of talks was recorded for posterity  and is available for all to view here
THE LAST WORD `here' IS THE LINK to the website which has links to the four talks.

The speakers were:

Len Adleman, Manuel Blum, Richard Karp, Leonard Schulman, Umesh Vazirani.

1) I URGE you to all WATCH those talks!

2) I really like it when talks are available on line after the fact so even if you didn't go (I didn't) you can still see the talks later.

3) So many talks to watch, so little time, alas!

4) Sorry for the white background for this post- that happens sometimes. NO comments on it please.

Three trick questions in Formal Lang Theory

There are three questions I ask in my Formal Lang Theory class that even the very best students get wrong. Two I knew were trick quesions, the other I was surprised by

1) If w is a string then SUBSEQ(w) is all strings you can form by replacing some symbols in w
with empty string. SUBSEQ(L) is defined in the obv way.

I ask the following in class (not on an exam). TRUE or FALSE and WHY and we'll discuss
If L is regular then SUBSEQ(L) is regular
If L is context free then SUBSEQ(L) is context free
If L is decidable then SUBSEQ(L) is decidable
If L is c.e. (used to be called r.e.) then SUBSEQ(L) is c.e.

The students pretty much get and prove that 1,2, and 4 are TRUE. They all think 3 is false.
But is true. For a strange reason

If L is ANY lang whatsoever then SUBSEQ(L) is regular. Comes from wqo theory. For more on this see a blog post I did when I was a guest blogger (it shows- the typeface is terrible) here

2) How many states does and NFA  need for { a^n : n \ne 1000} (or similar large numbers). ALL of the students think it takes about 1000 states. They are wrong: here

The two above I know people get wrong. The third one surprised me, yet every year the good students get it wrong

3) BILL: We showed that
a) 2-colorablility is in P, hence of course planar 2-colorability is in P
b) 3-colorability is NP-complete
c) 4-colorabilty of Planar graphs is in P

SO what about 3-colorability of planar graphs?

My very best student said the following last spring:

Planar 2-col is in P

Planar 4-col is in P

so I would guess that Planar 3-col is in P.

In prior years others made the same mistake.  My opinion of these students is NOT lowered, but I am surprised they make that guess. Of course, once you KNOW something you have a hard time recovering the state of mind of NOT knowing it, so my being surprised says more about my having drunk the Kool aid then their thought patterns.