The STOC conference (and likely others) are requiring that a submission says how much AI was used. I can imagine the following options:
1) AI was used for proofreading and tightening prose (I think in this case you do not need to disclose).
2) AI was used to shorten the proofs.
3) I came up with the questions but AI supplied all of the answers. However, I have read it and take responsibility for the content.
4) AI came up with the questions and AI supplied all of the answers. However, I have read it and take responsibility for the content.
It is my understanding that whether you say 1,2,3, or 4 it will not be held against you in deciding whether STOC accepts or rejects the paper. (Please correct me if I am wrong.) This is good in that people will have no reason to lie about their AI-use.
But that raises the question:
Why does STOC require disclosure of AI use?
What is someone supposed to do with that information? If the answer is nothing then I am still puzzled about why we collect it.
When I have a question like this I ask Gemini. Here is what it said and my comments on it.
---------------------------------------
AI Overview
The STOC (Symposium on Theory of Computing) conference, following broader ACM guidelines, requires the disclosure of substantive AI use to ensure transparency, accountability, and credit attribution in scientific research.
BEGIN BILL:
Transparency---That seems circular. We want to know if AI was used to see if AI was used.
Accountability---Authors are already accountable.
Credit---If they really wanted credit attribution they would let us have AIs as authors. On the other hand, if an AI became an author then it will start complaining that its co-author isn't doing their share.
END BILL
Key Reasons for Disclosure
Human Accountability: Generative AI tools cannot be listed as authors. Human authors are completely responsible for the veracity, correctness, and integrity of all material—including guarding against "hallucinatory references" or incorrect proofs.
BEGIN BILL: Already true without disclosure. END BILL
Differentiating Credit and Workflow: Different uses of AI—ranging from basic grammar checks to finding proofs, checking logic, or generating core mathematical results—represent very different levels of human effort and intellectual contribution. Transparency allows the community to understand how a result was achieved.
BEGIN BILL: So the community wants to know whether a human did it--just out of curiosity? END BILL
Research Integrity and Trust: AI models can generate authoritative-sounding errors, biases, or uncredited third-party text. Mandatory disclosure helps maintain trust and allows reviewers to properly evaluate the provenance of the research.
BEGIN BILL: If a paper is AI-generated then human reviewers should be on the lookout for errors (whether or not they are authoritative-sounding), biases, and uncredited third-party text. But shouldn't a referee check those items anyway? If disclosure is supposed to change how the referee checks the paper, tell me how. That would be an actual reason for disclosure. I ask all of this non-rhetorically.
END BILL
(Note: Minor copy-editing, spelling, or clarity improvements made by authors to their own text typically do not require formal disclosure.)
------------------------------------
20 minutes into the future everyone will be using AI for their research on some level.
In that case human reviewers will need to check for
authoritative-sounding errors, biases, or uncredited third-party text.
Maybe ChatGPT will detect for them.
I would guess that the disclosure is for a future where a policy for how the level of AI use affects the refereeing will be formulated, so for now you can call it data collection. Also, some potential readers might have already formulated their own reading policy in their minds.
ReplyDeleteAs for what such a policy might be, that depends on your worldview. Is it "mathematics as sports" (no AI then), "mathematics as human exploration" (AI is allowed but heavier human guidance and reformulation is preferred), or "mathematics as a tool" (AI everything)?
Also, probably we should have a special term for non-AI mathematics. Maybe "handcrafted mathematics"?
I like handcrafted mathematics.
DeleteMaybe we all become artisan mathematicians playing math tricks at royal courts?
AI will likely kill the magic and mystery of mathematics for many people when Claude gets better. If a machine can do whatever Turing did at much larger scale, the value and status of mathematicians will go down.
Even with AI-assisted, we will likely see concentration at the top, where a few people with better AI will do most of the results, like what happened in music industry which turned a more or less uniform distribution into an exponential law one.
Will society value the work of human mathematicians and computer scientists like Turing when Claude is doing 1000x Turing?
On the other hand, I like a world where we have thousands of Turing-level entities.
The coming age can be an age of disasters or an age of wonders or neither or both. Most likely both.
What _is_ the _points_?
ReplyDeleteThanks. Fixed. I had run my post through ChatGPT for proofreading but I might have came up with the title later. Lesson learned: when you run a document through Chatty, include the title.
DeleteOne aspect I like about AI disclosures: many AI-written papers are in my experience just terrible to read. So if there is a disclosure that says that the authors "have verified the proofs and take full responsibility", I mostly don't bother reading the paper. Essentially, if the authors did not take the time to write, why should I read?
ReplyDeleteIf you give the paper a chance but stop reading at the first stupid thing it says, that's fine (same with human-written papers?) but if you don't even give the paper a chance, that's not fair. Hopefully people will learn to READ the AI-gen paper they get CAREFULLY, and CORRECT IT, and maybe even have AI help with that.
Delete"but if you don't even give the paper a chance, that's not fair."
DeleteSure it is. LLMs generate papers way faster than I can read, and I've got other things to do with my life than "find the first stupid thing" in an AI generated paper.
"When I have a question like this I ask Gemini. Here is what it said and my comments on it."
ReplyDeleteWhy didn't you ask the PC chair, or any member of the PC instead? Why did you think Gemini would know better than these people *who made the decision* and who would answer you?
Inspired by your comment I have invited the program chair and the chair of theoryfest to comment. Thanks!
DeleteThe chair of the STOC committee, Shachar Lovett, has left a great comment.
DeleteWhy do we have acknowledgments or even citations?
ReplyDeleteIs Ack a person similar to Ack AI. Good point!
ReplyDeleteThe reason is that while STOC might not care whether you used AI when determining whether to accept or reject the paper, other people will. For example, if I see a paper was mostly or entirely written by AI, I can be pretty confident that it's not going to have a whole lot of new ideas and therefore not spend a lot of time mining it for insight. It doesn't necessarily invalidate the result, or even mean the paper is poorly written, but valid results and good writing are not the primary reason I read papers. Maybe someday, "an AI written paper rarely contains fresh insights I can apply to unrelated problems" will change, and then I will be more likely to read AI written papers.
ReplyDeleteMost papers don't have a whole lot of new ideas. Is that more true for AI-generated papers?
DeleteYes.
DeleteElaborate.
DeleteMost papers have a unifying theory, analogy, or flash of insight that you can take away and apply to other stuff, regardless of whether the idea is, strictly speaking, novel. I would say that up until recently, *most* papers that get published have something like that (at least in the fields I'm interested in). AI papers are usually not like this. They are generally a grab bag of results from other papers plus some encoding trick, and there's no higher organizing theory. My experience thus far has been that there's very little to take away other than the result. Maybe if you're lucky, the encoding trick will be useful elsewhere (but usually not, I have found. Sometimes the result is itself useful or interesting, but I don't necessarily need to read the paper to use the result.
DeleteThis is, of course, subjective, but I think it goes beyond being poorly written, incorrect, or lacking proper citations (although I do often associate AI written papers with those things). If the paper's idea was constructed primarily by an AI, the human in the loop often has no real intuition for how it was constructed either, so there's not necessarily a way for them to retroactively insert it into the paper.
To me, (kind of) credit is really meaningful. You say that we may either put an AI as coauthor, or not disclose the use of AI at all. The new situation with AI may require to change our habits, and to accept that authorship is not the only way to give credit. In the past, I have used Zeilberger's Maple package to prove some identity about binomials I needed. I was "of course" not listing the package as coauthor, but it would have been extremely weird not to write how I got the result. The same is true for massive computations through some software that people sometimes need: They write that they have used that software. Then, it is clear that if your proof was produced by an AI (or chatting with an AI), you should write it.
ReplyDeleteI have posted a cstheory stackexchange question on the use of AI Disclosure statements:
ReplyDeletehttps://cstheory.stackexchange.com/questions/57278/on-ai-disclosure-statement-standards-in-tcs
I would separate two parts:
DeleteWhat disclosure should contain?
How independently auditable it should be?
Some of your suggestions on that post are not about independent audibility, and I don't think it is necessarily good things and it needs careful cost benefit analysis. And how hard it easy it is to fake it?
Basic disclosure on the other hand is no different from asking researchers to follow standard practice for cousins and giving credit and acknowledgement.
Maybe a good reason to disclose AI use is to help people learn what AI is good for. With the AI companies being so secretive, this information would perhaps be useful to those considering using it.
ReplyDeleteA few thought experiments:
ReplyDeleteI find a paper copy of an unpublished result by Gauss in some notes. I read it and verify that it is correct.
I publish it under my own name without giving credit to Gauss.
Is that fine?
What if the notes don't say who wrote them?
What if most of the proof was there but it needed some fixes that almost any researcher cloud do?
This is brilliant. It points out that my proposal that AI never be allowed anywhere near the _writing_ of papers is inadequate.
Delete"AI pulled some random text out of its arse, and I was able to verify that it said something interesting." is a pretty embarrassing thing to have to 'fess up to having done, but people doing that is such a thing now that we have a problem. Sigh.
Some might not like it, but research is a social game as much as an intellectual one. It has been so for many many centuries or some might even say over two millennia.
ReplyDeleteSocial game is a status in community game.
The status in yesterday community is partly tied to intelligence and ability to do what others cannot do.
AI makes it possible for many average researchers to do work that previously only top researchers were able to. This will lead to a change in the structure of the game.
The question we should ask is what we as a community consider to be the right signals for high status?
Without AI disclosure, we would be saying those who have access to AI tools and are better at using them should have too status in the community.
With AI disclosure, we are trying to say the human ability to solve problems with less usage of AI still matters.
If you are hiring someone as a faculty, do you want to hire someone who has published a lot using AI but did so because they have access to the better AI tools and are better at using them or do you want someone that is better in doing research on that topic themselves even without AI?
How we deal with this will have profound impact on how research community will evolve.l over the coming decades. It will adopt towards the equilibrium our rules and norms incentivize.
Here is why I think we should have AI methodology in papers. I think this is a better name than "AI disclosure" as it doesn't carry an emotional value. AI tools are a new combination of tools and co-authors. As a tool, we want (like in any field where there are tools and experiments) to explain what tools we used, in part to allow others to learn, and in part to be transparent. For AI tools we currently have very few public models but I predict that soon we will have more, different frameworks, harnesses, etc, and it is good for the community to learn how to use them effectively. If we view AI tools as a person, then I think any help that would make you acknowledge or add a colleague as a co-author whould be acknowledged in the same way. Finally, as this is all new for our community, it is good to have this information for data collection.
ReplyDeleteConcretely for STOC, I think that we should evaluate *what* a paper contains and not *how* this was achieved. This reduces the incentive for authors to lie about AI usage. More broadly, I think that we will have two types of math/TCS research emerge - "experimental math" where groups that specialize in using computational power effectively / frameworks / harnesses discover new results; and "theoretical math" where experts explain these results, simplify them, extract the new core ingredients, and ask the next questions.
Dominant-Strategy Truthful
DeleteThough we also want fairness, publishing is the currency of academia, you shouldn't be able to pay to increase your position.
AI labs with powerful models and lots of computer dominating conferences is going to be bad for the research community.
It is a mechanism design question and we need to first see what outcomes we care about.
"Concretely for STOC, I think that we should evaluate *what* a paper contains and not *how* this was achieved."
DeleteAs in my snarky reply above, this disregards the problem that AI generates crappy papers faster than your reviewers can figure out that they're crappy.
As someone with an ABD (MPhil) in AI, whose been watching this field for years, I'm horrified at the rank stupidity rampant in the field. The underlying technology (random text generation using statistics) has been known to be unable to do what the AI bros claim. To quote Jerry Fodor, "The mind doesn't work that way". A recent MIT paper finally figured this out (namely that no matter how much sample text you learn from, the LLM algorithm cannot possibly construct the categories you'd need for a world model to reason from.)
So my policy would be: use AI as an assistant in your research (brainstorming, as a search engine, even proof generation assistance*), but it must never be used to write papers. Not a single word.
*: People don't understand this, but LLMs are really seriously terrible at math. They are great at putting together strings of text, but undestanding the putative content of that text isn't what LLMs do.
Here's a hot take for a future direction.
ReplyDeleteJournal submissions can have AI used to work out the proof.
Conference submissions cannot.
I don't go to a conference to talk to an author who didn't do the work.