Sunday, September 19, 2021

The New Jeopardy Champion is a `A CS grad student from a school in New Haven'

 As of Sept 17, Matt Amodio has won 23 straight games in a row on Jeopardy and won over $800,000 in regular play. The following website is not quite up to date, but its close: here. Of course, that website will change. 

1) They refer to him as A CS grad student from a school in New Haven. My first thought was probably Yale, but whatever it is, they should just say it. I looked it up and it is Yale. So why aren't they just saying A CS grad student from Yale?  If someone works for an airline company they do not tell you which airline- prob to avoid giving that airline free publicity. But I would think a school is different. And I remember (perhaps incorrectly) that they DO say what school someone teaches at or is a student at. 

2) Longest streak: Ken Jennings: 74. Also most money in reg play: roughly 2.5 Mill

    2nd longest: James Holzhauer: 32. Also  2nd most money in reg play: roughly 2.4 Mill

    3rd longest: Matt Amodio: 23. Also  3rd most money in reg play: roughly 0.8 Mill

3) I do not think Matt will move into second place on any of these categories. He bets big on the daily doubles and it has paid off but either (a) he will miss and it will lead to a loss, or (b) he will just not get the daily double and be against a very good opponent. Item (b) happened to James H- and the person who beat him did have a good enough win streak to be in the Tournament of Champions.

4) Matt has played in front of (or will- counting tomorrow) six hosts: Robin Roberts, LeVar Burton, David Faber, Joe Buck, Mike Richards, and Mayim Bialik. Six is a record which I suspect won't be broken, except possibly by Matt himself if he also plays in front of Ken Jennings (the rest of 2021 will be Mayim B and Ken J as hosts). 

5) Matt works in AI. When he gets his PhD and is on the job market will his Jeopardy success help him, hurt him, or neither?  

6) James H and Matt A are both very good at calculating how much to bet. I think Ken J is not quite as good but still good. Generally the players on Jeop are not that good at that aspect. I had the chance to ask some a champions (not any of those three) why that was and she said that most people get into because of the trivia-aspect, not the betting aspect. I wonder if just as players now study lots of facts to prep, they will also learn how to bet better. 

7) Ken J as host is a bit odd in that, if he says (as Alex T did sometimes) That category looks hard I won't believe him. 

Sunday, September 12, 2021

Review of A Blog Book based on the Less Wrong Blog

There is a blog called lesswrong. Many people contribute to it (how many people must contribute to a website before it stops being called a blog and starts being called... Not sure what?). The theme is rationality. They (not sure who they are) have made a best-of collection from lesswrong which is named

A Map that Reflects the Territory (available here)

Actually, its not one book, but five mini-books. I quote the titles and paraphrase the first sentence of each:

Epistemology:  How we come to know the world.

Agency: The ability to take action in the world and control the future.

Coordination:  The ability of multiple agents to work together.

Curiosity: The desire to understand how the world works.

Alignment: The problem of aligning the thoughts and goals.

I have written a review of the the book. The book was my first exposure to the blog, except sometimes reading about the blog, probably on Scott's blog.

I am posting this to both complexity blog and to lesswrong, though with lesswrong I will have a different intro since they know lesswrong but might not know complextyblog. 

My review is here.

I would appreciate intelligent comments and suggestions, which I will use to improve the review.

Thursday, September 09, 2021

The Death of Expertise

Four years ago I tried to catch up with deep learning and this summer I aimed to try to catch up again. Who would've thought 2017 is ancient history.

I watched the lectures in the latest MIT course, played with GPT-3 and Codex, read the new Stanford manifesto on what they call foundation models, models trained that can preform on a wide range of tasks instead of a single goal. We've seen machine learning solve protein folding, detect cancer from x-rays better than radiologists, not to mention effectively solving many of the traditional AI problems (language translation, voice and face recognition, game playing, etc.) Watching Codex generate computer code from plain English is a game changer. Far from perfect but this technology is in its infancy.

From what I can tell, the main technological advances focus on learning when we don't have labelled data such as new techniques to transfer knowledge to new domains and using generative models to provide more data to train ML algorithms.

The trend to worry us all is that deep learning algorithms in the long run seem to do better if we limit or eliminate previous human knowledge from the equation. The game playing algorithms now train from just the rules alone (or even just the outcomes). We do use some knowledge: words come in some linear order, faces have hierarchical features, but not much more than that. Human expertise can help as we start solving a problem, even just to know that good solutions look like, but then it often gets in the way.

When we longer use a skill we tend to lose it, like my ability to navigate from a good map. If we eliminate expertise we may find it very difficult to get it back.

There's a more personal issue--people spend their entire careers creating their expertise in some area, and that expertise is often a source of pride and a source of income. If someone comes along and tells you that expertise is no longer needed, or even worse irrelevant or that it gets in the way, you might feel protective, and under guise of our expertise tear down the learning algorithm. That attitude will just make us more irrelevant--better to use our expertise to guide the machine learning models to overcome their limitations. 

You can't stop progress, but you can shape it.

Sunday, September 05, 2021

Guest Post on Solving (or trying to) Poly Diophantine Equations by Bogdan Grechuk

(Guest post by Bogdan Grechuk)


Motivated by Mathoverflow question here I have recently became interested in solving Polynomial Diophantine equations, that is, equations of the form

P(x1,...,xn)=0

for some polynomial P with integer coefficients. Because there are many such equations, I have decided to ask a computer to help me. Our conversation is presented here.


Highlights of the conversation: the height of a polynomial over Z in many variables is what you get when you make all of the coefficients positive and plug in x=2. For example, the height of

x3 - y4 + 5

is
23 + 24 + 5 = 8 + 16 + 5 = 29.

Note that, for all h, there are only a finite number of polynomials in many vars over Z with height h. With the help of my friend the computer we have looked at the equations with h=0,1,2,... and so on, and tried to  determine which ones have any integer solutions. As expected, the first equations were trivial, but at about h=22 we have started to meet quite interesting equations for which we needed help from  Mathoverflow to solve. The project is currently at h=29, with only one remaining open equation of this height.  Read the conversation with the computer, or my mathoverflow question

here
or my paper:

here
to find out more. I INVITE you to join me in this quest!

Thursday, September 02, 2021

The hierarchy and GapP

There is a great but little-known theorem from the early 90's by Seinosuke Toda and Mitsunori Ogihara (buried as Lemma 2.3 in their paper) that shows the polynomial-time hierarchy is randomly low for Gap-P.

Let M be a non-deterministic polynomial-time machine. #M(x) is the number of accepting paths of M on x. #P is the set of functions f such that f(x) = #M(x) for an NP machine M. 

Gap-M(x) is the difference between the number of accepting and rejecting paths. Unlike #M(x), Gap-M(x) could be negative. Gap-P is the set of functions such that f(x)=Gap-M(x) for an NP machine M. Equivalently Gap-P is the difference of two #P functions. Gap-P inherits all the nice closure properties of #P while being closed under negation and subtraction.

Theorem (Toda-Ogihara): Let q be a polynomial and f be a function in Gap-PPH. There is a function g in Gap-P and a polynomial p such that for all x, if you choose a binary string r randomly of length p(|x|), f(x) = g(x,r) with probability at least 1-2-q(|x|).

In other words, with randomness the PH as an oracle of Gap-P disappears.

Let me show you how the proof works for a specific f in GapPPH, namely the indicator function for SAT: f(φ) = 1 if φ is satisfiable and 0 otherwise.

Recall Valiant-Vazirani showed how to randomly reduce a formula φ to a set of formula ψ1,…,ψk such that

  1. If φ is not satisfiable then for all i, ψi will not be satisfiable
  2. If φ is satisfiable then with high probability for some i, ψi will have exactly one satisfying assignment.
Picking k > 4|x|q(|x|) will reduce the error in (2) to under 2-q(|x|). Let g(φ,r) use r to choose ψ1,…,ψk and output the Gap-P function 1-∏(1-#ψi) using the closure properties of Gap-P where #ψi is the number of satisfying assignments to ψi.
  • If φ is not satisfiable then for all i, #ψi=0 and g(φ,r) = 0.
  • If φ is satisfiable then with high probability for some i, #ψi=1 and thus g(φ,r) = 1.
Note that if r is a bad random sequence and φ is satisfiable, g(φ,r) might be an exponentially large integer, positive or negative. It's right most of the time but when it's wrong it's terribly wrong.

Tuesday, August 31, 2021

Since we will soon be back in the classroom, how was Zoom? Anything you want to maintain?

UMCP will have all classes on campus this Fall. There is a Mask Mandate. All students and faculty have to get vaccinated unless they have a health or religious exception. 92% are vaccinated, which I interpret as people are NOT abusing the exceptions (though I still wish it was higher, and it may go higher). (ADED LATER- right after I posted this I got an email saying that UMCP is now up to 97%).  Those NOT vaccinated have to get tested - I think twice a week. 

Now that we are back in the live-classroom, here are some thoughts about teaching on zoom. 

I taught on zoom:

Spring 2020: The last half of both Ramsey Theory and Automata Theory(Reg, CFG,P,NP,Dec,Undec)

Fall 2021:  Cryptography 

Spring 2021: Honors Discrete Math and Automata theory


a) I taught in the usual time slot but I recorded the lecture so those who could not make it (more common during the pandemic) could still see it. Attendance was low, verbal interaction was low, but chat-interaction was very good. Looking into if we can do a chat in an in-person class. I was recording lectures before the pandemic and will keep doing so.

b) My exams were open-notes, open-book, open-web. That cuts down on ways they can cheat, though they can still phone-a-friend. Or ask their cat.  Unusual-but-correct answers can happen, as I discussed in this blog.

c) I gave my muffin talk a few times on zoom. In person it goes very well as my enthusiasm is contagious.  On Zoom that affect is dampened so the audience was more sedate.  I gave it as a special lecture to High School students and to my REU students. Note that it was NOT part of a class so the usual motivation to learn it to do the HW is gone. Hence its more important they be excited about it. 

d) In person I carefully make sure that I wear a funny T-shirt every day, and its a diff one, and usually a math one to (if possible) match the topic. On Zoom I did not bother, though I sometimes used wallpaper to match the topic. 

e) I had to make up slides for Aut theory and for some of Discrete Math. For Crypto I already had slides. I like the slides I made up and will use them in the future. But see next point. 

f) In Discrete Math I went  faster than usual- perhaps because its on slides, perhaps because there were less questions since it was on zoom, perhaps because Emily my  TA was so awesome that they had less questions. (She is very interested in education and did a guest post about the pandemic and education here.) As a result I actually learned and presented the proofs that (1) the e is irrational (my slides are here) and that Liouville numbers are transcendental (my slides are here). While I enjoyed learning those theorems and I think the students understood them on some level, I will slow down next time.

g) Ramsey Theory: It is impossible to teach the Poly VDW theorem on slides, so I had to omit that part of the course. 

h) Bottom Line: Did the students learn more? less? the same? My impression is that the students learned about the same, but really really didn't like it. And thats legit- that is NOT just students complaining. 

Wednesday, August 25, 2021

The Long Road

Guest blogger Varsha Dani tells us why it's never too late.

This week, I am starting as an Assistant Professor at RIT and I am super excited about it. What's the big deal, you are probably thinking. Don't lots of people get hired in tenure track positions every year? Sure. But the difference in my case is that I got my Ph.D. in 2008. 

Why didn't I look for a position right away? There were a number of reasons. I was burned out. I was going to have a baby. The job market was not particularly good that year. I would have had a two-body problem. And most importantly, I thought it was just going to be a short break. I thought that there was no rush...

What did I do in those intervening years? A lot of things. I spent a lot of time with my kids, including part home-schooling them for some time. I found some new interests, both in Math and CS and outside. I found new collaborators and did some research in new areas, just for fun. Sometimes I was funded for it through grants, but mostly I wasn't. I wrote a lot of Computer Science and some math materials for Brilliant.org. I organized math clubs at my kids' elementary and middle schools. I hiked. I wrote poetry. I never intended nearly 13 years to go by, but somehow they did.  At some point I remembered that I had meant to take a short break, not to give up on being an academic altogether. But by then it seemed too late. At the beginning of my meant-to-be-short hiatus, I used to jokingly refer to myself as a "scholar at large" but by the time a decade had gone by I had started to feel extremely isolated and being a scholar for its own sake was not something to joke about anymore.  Each year I rolled the job-search dice, but with each passing year it seemed more and more futile to do so, and more and more of an imposition to ask people to write recommendations for me. And then, out of the blue, last year, I found a whole community of independent researchers who, like me, were pursuing their scholarly interests despite not being employed to do so, and who felt unapologetically unembarrassed, even proud of it.  And, even more out of the blue, this year I got an offer. And now, this Fall, I am actually doing it. I am actually an academic! To be honest, I feel more than a little trepidation about it, mixed in with the excitement. 

So why am I telling you this? Partly to celebrate. Partly to publicly thank my spouse, who has been extremely supportive of all my decisions (and all my actions that were born of indecision) over many years. Partly to give a shout-out to the wonderful folks at the Ronin Institute who helped me remember that we do science (and computer science) because we love it, not because it pays the bills. Ironically, I believe I needed to be reminded of that before I could get a job! But mostly, I'm writing this to reach out to anyone out there who thinks that their decisions have led to a one-way street they no longer want to be on. It may be hard, and there are, of course, no guarantees, but you won't know whether you can turn around, unless you try. 

Sunday, August 22, 2021

When Words Get Stretched Beyond Their Original meaning

STORY ONE:

 On a Jeopardy rerun with Alex Trebek the question (actually the answer, given the shows format) was (I paraphrase)


Who resigned his commision in the US Army Air Force in April 1941 after President Roosevelt publicly rebuked him for his views?

The answer (actually the question--Why does Jeopardy do this answer-question thing, drives me nuts!) was

Charles Lindbergh.

Alex Trebek then said  Charles Lindberg's views on WW II were not politically correct.

This really struck me since Politically correct means, to quote Wikipedia:

a term used to describe language, policies, or measures that are intended to avoid offense of disadvantage to members of particular groups in society. 

Wikipedia also adds that the term is generally used pejoratively with an implication that these policies are excessive or unwarranted. 

 But Alex Trebek is using the term to mean  incorrect or perhaps incorrect given what we know now or if you think history is written by the winners, then perhaps incorrect since Germany lost the war.  But my point is that I really don't think the term  politically incorrect  makes sense here.

STORY TWO

More recently I heard an anti-masker say

We should not let some woke school board take the right to not wear a mask away from parents and children.

Independent of if you are anti-mask-mandates or pro-mask-mandates, this seems like a strange use of the word  woke  which means, to paraphrase Wikipedia:

Having an awareness of racial prejudice, gender prejudice, sexual orientation prejudice, and the past and current discrimination they have and do cause. 

I've seen it both positively and negatively.

The anti-masker's using of the term seems odd in that mask wearing is not a woke issue. Perhaps he should have said 

We should not let some Nazi school board take the right to not wear a mask away from parents and children.

The term  Nazi while not actually correct, conveys that the school board is authoritarian. However, he really could not use the term that since he was was a neo-Nazi and proud of it. That raises a question: what pejorative  term can a Neo-Nazi use when they want to say someone is  Authoritarian? I ask non-rhetoically. 

But I am getting off topic here- my real point is that the word woke is being used to mean Authoritarian which is not even close to its original meaning. 


MY POINT

The above are examples of how a word in English may change its definition over time, which is not really news, but I found the examples interesting since I saw the origin of these words.

BILL, THIS IS A COMPLEXITY BLOG! SO TALK ABOUT COMPLEXITY. OR MATH!

In math do words change their meaning over time? Yes. Here are a few

Function: at one time `function' implicitly means a function that occurs in nature. So only continous and perhaps diff functions qualified. 

Sets: probably similar.

Efficient: At one time this was an informal notion (Joe Kruskal's paper on MST (see here) is an example of that), then it seemed to be P or perhaps BPP. For some its linear or O(n log n) with a small constant. Rather than say the notion changed, its more like it was never that well defined in the first place, and still isn't. 

Constructive: The many diff definitions of this word could be a blog post of its own. In fact, I thought it was, but I could not find it. I did find lots of blog posts that use the word constructive in diff ways. 

Elementary: Also has many definitions, though they are closer together than for Constructive. This one I did do a post on here