The problem is that eventually, there will be no humans who can follow the results AI will give. That’s the endgame for this tech: to produce knowledge at speeds and quality beyond what we can
Which asks the question of what knowledge is. Can knowledge be super human ? Or is knowledge a human matter ? If so (like i believe), then what those ai labs are doing is far from the end of the story. Because the goal of science is not to produce a certificate of something, but more to produce an explanation that can fit in a human brain, that can be reasoned on, and that can be retargeted. In this sense, producing a million lines proof is not really producing knowledge, even less so doing science.
That OpenAI setting helps, but there is a better way to do it. To completely opt-out of training, submit a request via the OpenAI privacy portal.
Visit this website https://privacy.openai.com/policies/en/ , click "Make a Privacy Request", choose "Do not train on my content", and complete the form. That submits a formal objection to training on your data, as required by GDPR/your local legislation.
Haha. I am 100% these companies will ignore this if they choose to. Just as they played fast and loose with copyright rules.
They would do it, the say “ah sorry chaps, impossible to extract it from the dataset by now, anyway we anonymized it so can’t tell what’s what, and we can’t risk losing to China. Oh look - did you see Superman fly outside?”.
This form is legally binding and has more legal weight than just clicking a toggle. If they still train on my data, they can get sued, and I'll get a payout.
Showing that a novel mathematical approach was in your prompts shortly before the model “proposes” that approach to relative amateurs would be ideal, if only we had a case like that…
It’s possible that they didn’t train on it, and the approach was derived by the LLM.
What’s a problem is that they haven’t outright denied it. That could be caution and them doing their due diligence first, it could be that it was intentional and they didn’t expect to get caught, or it could be because they have no way of knowing themselves.
This looks incorrect. OpenAI have flat out said that there's no need of doing this and both ways are equivalent
> We respect our users' choice whether to use their data to “improve our models for everyone” regardless of where they express that choice. Users can opt out in the in-app settings or indeed also in our privacy portal. They do not need to opt out in both places, and we will make this clearer in our Help Center.
Even if that's true right now, if one of the options is legally binding and the other is "we promise not to eat your data... for now", the first option is still better.
The main claim, that "they do not seem to have any mathematicians capable of understanding what they put out", was also corroborated by Sebastien (OpenAI) who explained they don't have any experts on Navier-Stokes.
This is speculation right now. The idea would be that if someone used the product and granted training rights, which is the default for many subscription levels, then some knowledge would have been imparted into the general weights of the new model.
oAI has made clear they did not specifically pull in any user data to context for this run.
Tristan Buckmaster’s post cited extensive use of LLMs in the process of his collaboration with Levent:
“We used several LLMs throughout: Anthropic’s Claude, OpenAI’s Codex, especially with GPT-5.6 Sol and, more recently, Astra.
The latter was only used for writeups and auditing our arguments. For most of the past year progress was slow. We worked through the literature and upgraded various preliminary results, up to obtaining finite time blow up for the Incompressible Porous Media equation (with smooth forcing).
This was until about a month ago, when we had real progress: on August 15th, we obtained the blow up results, with smooth forcing, for both Boussinesq and Euler. I can say the first LLM generated proof Levent sent me was the most horrendous I have ever read; we verified it on Lean on August 22nd. Since this point, we have been working around the clock to understand this proof and turn it into something readable.”
The OpenAI research post states they began training GPT-6 internally on August 28th, and that user chats are used to train models.
“We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem. While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models .”
For an incredibly niche topic like this, I believe it’s extremely likely that Buckmaster/Levant’s work would influence the direction of OpenAI’s agents’ work even as a de-identified drop in the overall bucket of training data.
It wouldn't surprise me that even if you opt-out they still train on 'de-identified' chat data. After all, facts cannot be copyrighted and one cannot be sued for unethical behaviour.
Can anyone explain to me why Tristan simply didn't go to settings page and turn off the thing? Especially when Levent was collaborating with him and Levent is completely aware of how the training data is used and the implications thereby?
If it is oversight, then that's ok and OpenAI can volunteer to make him the lead author which they did. But he's pissed that OpenAI is not letting Levent as well, who had access to internal Anthropic models. So this guy thinks
1. oh my bad i forgot to turn off the consent thing in settings page
2. also i'll collaborate with a literal Anthropic employee who has access to their internal models
3. i'll also reject OpenAI's deal to be the lead author because i want an employee of the competitor to be a part of it
Or is it just all PFM? (Pure, Fanciful Magic)
https://chatgpt.com/codex/cloud/settings/data#settings/DataC...
https://claude.ai/new#settings/data-privacy-controls
I just realized I've been happily "improving the model for everyone"...
Visit this website https://privacy.openai.com/policies/en/ , click "Make a Privacy Request", choose "Do not train on my content", and complete the form. That submits a formal objection to training on your data, as required by GDPR/your local legislation.
They would do it, the say “ah sorry chaps, impossible to extract it from the dataset by now, anyway we anonymized it so can’t tell what’s what, and we can’t risk losing to China. Oh look - did you see Superman fly outside?”.
What’s a problem is that they haven’t outright denied it. That could be caution and them doing their due diligence first, it could be that it was intentional and they didn’t expect to get caught, or it could be because they have no way of knowing themselves.
This looks incorrect. OpenAI have flat out said that there's no need of doing this and both ways are equivalent
> We respect our users' choice whether to use their data to “improve our models for everyone” regardless of where they express that choice. Users can opt out in the in-app settings or indeed also in our privacy portal. They do not need to opt out in both places, and we will make this clearer in our Help Center.
https://x.com/thsottiaux/status/2097746417012166816
How did it become part of their training data if it wasn't public? /confused
oAI has made clear they did not specifically pull in any user data to context for this run.
“We used several LLMs throughout: Anthropic’s Claude, OpenAI’s Codex, especially with GPT-5.6 Sol and, more recently, Astra.
The latter was only used for writeups and auditing our arguments. For most of the past year progress was slow. We worked through the literature and upgraded various preliminary results, up to obtaining finite time blow up for the Incompressible Porous Media equation (with smooth forcing).
This was until about a month ago, when we had real progress: on August 15th, we obtained the blow up results, with smooth forcing, for both Boussinesq and Euler. I can say the first LLM generated proof Levent sent me was the most horrendous I have ever read; we verified it on Lean on August 22nd. Since this point, we have been working around the clock to understand this proof and turn it into something readable.”
The OpenAI research post states they began training GPT-6 internally on August 28th, and that user chats are used to train models.
“We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem. While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models .”
For an incredibly niche topic like this, I believe it’s extremely likely that Buckmaster/Levant’s work would influence the direction of OpenAI’s agents’ work even as a de-identified drop in the overall bucket of training data.
So what might have been the incentive for OpenAI to do all this shenanigans? It might have to do with getting its models certified for AGI and getting out of lockin with Microsoft - https://deadneurons.substack.com/p/the-quiet-unwinding-of-mi...
If it is oversight, then that's ok and OpenAI can volunteer to make him the lead author which they did. But he's pissed that OpenAI is not letting Levent as well, who had access to internal Anthropic models. So this guy thinks
1. oh my bad i forgot to turn off the consent thing in settings page
2. also i'll collaborate with a literal Anthropic employee who has access to their internal models
3. i'll also reject OpenAI's deal to be the lead author because i want an employee of the competitor to be a part of it
I don't get the mindset.