Machine Learning and Complexity

 

Schloss Dagstuhl by Monet by Dall-E

At Dagstuhl earlier this month, I hung out for a little bit with the participants of the other seminar, Knowledge Graphs and their Role in the Knowledge Engineering of the 21st Century. Knowledge graphs are what you would expect them to be, nodes are objects like "Berlin" and "Germany" with directed edges with labels like "capital". Think of having knowledge graphs of hundreds of millions of nodes and how that could help answer queries about the world. These secondary workshops are shorter and focus on creating a new vision, in this case how to maximize the importance of knowledge graphs in an increasing ML-focused world.

Perhaps we need such a visioning seminar for complexity. While we often get lost in the mathematical questions and techniques in our field, computational complexity is designed to understand the difficulty of solving various problems. Machine learning and advances in optimization should be changing that conversation. If you imagine a world where P = NP (and I did exactly that in chapter 2 of my 2013 book) much of what you consider is starting to happen anyway. ML does fail to break cryptography but then again, isn't this the best of all possible worlds? 

Look at what Scott Aaronson said back in 2006.

If P=NP, then the world would be a profoundly different place than we usually assume it to be. There would be no special value in “creative leaps,” no fundamental gap between solving a problem and recognizing the solution once it’s found. Everyone who could appreciate a symphony would be Mozart; everyone who could follow a step-by-step argument would be Gauss; everyone who could recognize a good investment strategy would be Warren Buffett. 

If I can be a Monet, can Mozart be far behind? ML trading by some hedge funds are beating Warren Buffett but remember if everyone trades perfectly, no one beats the average. Gauss is going to be trickier but it's coming. There's a reason Scott is spending a year at OpenAI to understand "what, if anything, can computational complexity contribute to a principled understanding of how to get an AI to do what we want and not do what we don’t want".

Is the complexity of approximating Vertex Cover of degree 3 open? (ADDED LATER-NO)

 RECALL:

A max-problem f has a A P Time App Scheme (PTAS) if there is an algorithm ALG such that

 ALG(\epsilon) \ge (1-\epsilon)f(x).

A min-problem f has a A P Time App Scheme (PTAS) if there is an algorithm ALG such that

 ALG(\epsilon) \le (1+\epsilon)f(x).

(Note that the poly can depend on epsilon so it may be something like n^{1/epsilon}.)

MAX3SAT is, given a formula with \le 3 literals per clause, find an assignment

that maximized the number of clauses satisfied.

VCB-a is Vertex cover where graphs have degree \le a

The following are known:

0) MAX3SAT is in APX.

1) The PCP paper, here, showed that if MAX3SAT has a PTAS then P=NP.

2) Papadimitriou and Yannakakis (here)  had showed much earlier that MAX3SAT \le VCB-4 with an approx preserving reduction.

3) From (1) and (2) we have that VCB-4 has a PTAS then P=NP. (VC is in APX by an easy 2-approx).

4) Clearly VCB-2 is in P.

The following seems to be open, though if you know otherwise pleae leave a comment:

Is VCB-3 a) in P? b) NPC? (ADDED LATER- NPC- See comments.) 

Is the following true: if VCB-3 has a PTAS then P=NP. (ADDED LATER- NO PTAS-See Comments)

NOTE- all of the above is true for Ind Set-4 and Dom Set-4. So that leads to more open problems.

POSTED UPDATED VERSION OF Computers and Intractability: A guide to Algorithmic Lower Bounds posted (New title)

We have posted a revised version of 

Computational Intractability: A Guide to Algorithmic Lower Bounds

by Demaine-Gasarch-Hajiaghayi

The book is here.

(For the original post about it, edited it to use the new title (see below), see HERE.) 

We  changed the title (the title above is the new one) 

since the earlier title looked too much

like the title of Garey's and Johnson's classic. While that was intentional we 

later felt that it was too close to their title and might cause confusion. 

Of course changing the title might also cause confusion; however, 

this post (and we will email various people as well) will stem that confusion. 

We welcome corrections, suggestions and comments on the book. Email us at hardness-book@mit.edu

There are two different definitions of Integer Programming. Why?

Alice and Bob have the following conversation.

===============================

ALICE: In your book you define INT PROG as, given a matrix A and vectors b,c,

find the integer vector x such that Ax\le b and c DOT x is maximized.

This is not correct! You also need x\ge 0.

BOB: Really? I always heard it without that extra constraint, though I am

sure they are equivalent and both NP-complete (Alice nods).

Where did you see it defined with that extra constraint?

ALICE:

Wikipedia entry in IP

Chapter of a book at an MIT website

Something on Science Direct

A course at Duke

An article by Papadimitriou 

An article on arxiv

The book Graphs, Networks and Algorithms by Dieter Jungnickel

Bob, do you have examples where they do not use that extra constraint. 

BOB: 

Math Works

Lecture notes from UIUC

Lecture notes from Lehigh Univ.

The book Parameterized Complexity Theory by Flum and Grohe

The book Computers and Intractability : A Guide to the Theory of NP-Completeness by Garey and Johnson

ALICE: Both of our lists are impressive. So now what? 

--------------------------------------------------------------------

(This is Bill again.)

What indeed!

1) Why are there two definitions of Int Prog?

2) When is it good to use which one? 

Monarachy: A Problem with Definitions

 As I am sure you know, Queen Elizabeth II passed away at the age of 96 recently.  I am not a royal-watcher, but I am a royal-watcher-watcher. That is, the question of why people care about the lives of these people intrigues me. A few notes

1) Was she a good Queen? People tend to think so; however, since the job is somewhat ill-defined its hard to say. 

2) The Queen is supposed to be above politics (she does not vote- I was surprised to find out that legally she can, but she really can't). We know very few of Queen Elizabeth II's opinions on political events. But the notion of political is not well defined. One would think that if she did an appeal for people to take the COVID vax that would not be political, but somehow it is (I do not know if she did such an appeal). King Charles III believes in global warming and that we need to do something about it. This again should not be political but is. 

3) She is the second longest reigning Monarch. First is King Louis XIV who first became king at the age of 4. I had a blog complaining about this here. However, there is a more interesting point I want to make. From the first to the last day of King Louis XIV reign not much had changed. Technology, politics, other things just didn't change much. By contrast the world changed A LOT between Queen Elizabeth II first and last day:

a) The British were an important power in 1952. Less so now.

b) When her father died she was in Kenya and it took 4 hours to get the news to her. Now that would be immediate. 

c) Divorce was considered bad in 1952 and is why King Edmond VIII could not be king (he wanted to marry a twice-divorced woman whose ex-husbands were still alive). And now three of the Queen's children have been divorced.

d) Gay people.. enough said. There has even been a royal gay wedding, see here

Black people (can't call them African-Americans), Women,... you fill it in. 

e) When Charles wanted to get married it seemed to be important that he marry a virgin. We cannot imagine this mentality anymore. When Prince William and Kate got married they were already living together and this was NOT an issue for ANYONE. I looked up what the Church of England thought of it and all I got was some very bland comments like That's what young people do nowadays. 

3) Is the monarchy a good thing? As an American I feel I do not have a right to an opinion. If the citizens of the United Kingdom approve of the monarch (polls show they do) then who am I do tell them they are wrong? Even so, lets look at reasons for it

a) Tourism. It has been said that the Monarchy leads to MONEY from tourism. So it is worth the price? Nobody seems to know and it would be hard to tell. However, I don't think the citizens of the United Kingdom view  money as the reason for Monarchy. The American analog is giving Disneyland tax breaks to be in Florida which generates jobs. I doubt they think of the Monarchy in those mundane transactional terms. 

b) CS Lewis said 

Where men are forbidden to honour a king they honour millionaires, athletes, or film stars instead: even famous prostitutes and gangsters. For spiritual nature, like bodily nature, will be served; deny it food and it will gobble poison.

This is  bit odd- they must all pretend to like the monarchy to make it work. A long time ago when Charles and Dianna were both having affairs, 80% of the citizens the United Kingdom thought that was okay so long as they are discreet so the people don't find out. But- those ARE the people.

Also odd- CS Lewis was a theologian and a  believing Christian; however, his comment above can apply to God as well as to Kings. 

Thirty Years of Dagstuhl

 

Dagstuhl old-timers at the original castle

I'm back at Dagstuhl for the seminar on Algebraic and Analytic Methods in Computational Complexity. My first seminar at Dagstuhl was back in 1992. I've been coming for thirty years and have been here roughly thirty times. My last trip was pre-covid (virtual Dagstuhls don't count) and I really needed this chance to hang out and talk complexity with colleagues old and new.

Some changes since my last trip. The room doors have locks (there are rumors of an incident). You have to create your own keycard on a new machine logging into your Dagstuhl account. I had a long random password through a password manager and it was not so easy as process.

The main conference room has been updated with tech for hybrid meetings, and new led lights. Books were removed from the library to create a coffee breakout space.

No Bill this time so no typecasts. Still the best part of the week is talking and hearing about complexity. Today I learned about the orientations of Sperner's lemma, that there is one more triangle oriented according to the direction of the corner vertices than those oriented the other way. Christian Ikenmeyer used this fact to motivate a study of closure properties of #P-functions.