AI training could be better
AI models come with "system cards" which describe each model. Some reach hundreds of pages of detail, but of the ones I've looked at, I've only seen at most a couple of generic sentences detailing the training data used.
A link to a model's system card that explains, in depth, the training data used, is something I'd be very interested to see.
## 1.1 Training data and process
Claude Opus 4.8 was trained on a proprietary mix of publicly available information from the internet, public and private datasets, and synthetic data generated by other models.
[Continues, explaining their filtering & crawling process.]
-- Claude Opus 4.8 System Card (https://www.anthropic.com/system-cards)
System cards look like research papers. Claude Open 4.8's is 246 pages long & the above quote is all the information we are given as to the sourcing of training data. One of the reasons for this lack of transparency is because these LLM developers are training on data they have admitted to illegally obtaining.
Anthropic, who develop Claude, have settled a class action lawsuit with authors with a payout of $1.5bn:
The company holds more than seven million pirated books in a central library, according to Judge Alsup's June decision, and faced up to $150,000 in damages per copyrighted work.
-- https://www.bbc.co.uk/news/articles/c5y4jpg922qo
Anthropic are not alone in lawsuits against them for possessing pirated books. Today there is nothing preventing a company from training on pirated works (except for fines, if they're caught), & we have only hope that they do the right thing when scraping the web.
Unfortunately:
Mark Zuckerberg approved Meta’s use of “pirated” versions of copyright-protected books to train the company’s artificial intelligence models, a group of authors has alleged in a US court filing.
Citing internal Meta communications, the filing claims that the social network company’s chief executive backed the use of the LibGen dataset, a vast online archive of books, despite warnings within the company’s AI executive team that it is a dataset “we know to be pirated”.
-- https://www.theguardian.com/technology/2025/jan/10/mark-zuckerberg-meta-books-ai-models-sarah-silverman
Meta employees even discussed the implications of using company hardware:
# Meta made efforts to avoid detection
Despite these concerns, Meta appears to have not only ploughed on and taken steps to avoid detection. In April 2023, an employee warned against using corporate IP addresses to access pirated content, while another said that “torrenting from a corporate laptop doesn’t feel right,” adding a laughing emoji.
-- https://www.techradar.com/pro/meta-purportedly-trained-its-ai-on-more-than-80tb-of-pirated-content-and-then-open-sourced-llama-for-the-greater-good
So I believe we should have more information about a model's training data, to hold model developers accountable. A number of technologies exist today that could be used to allow for this transparency.
Like I said in Privacy isn't a test (https://algamest.ghost.io/privacy-isnt-a-test/), I am sharing incomplete ideas with the intention of encouraging readers to have a mindset of questioning technology & demanding better from it.
Publishing a hash representation of the training dataset used, & having this verifiably tied to model code, would be a working solution.
The last step of having the two tied together, with the model being unusable if these don't match, is not something I know how would work, but I believe far more challenging problems have been solved today than this, & with demand this could be achieved.
Independently verifying a hash would require having access to the underlying training data, a measure of transparency in itself.
A step further is in Trusted Execution Environments (https://en.wikipedia.org/wiki/Trusted_execution_environment).
Nvidia already have Confidential Computing built into their modern hardware (https://www.nvidia.com/en-us/data-center/solutions/confidential-computing/), the same hardware all of the big tech companies are heavily invested in. This is used to verify that the hardware really is made by Nvidia and running exactly the software expected. I think with this in existence, we could add a hash, or anything else we don't want tampered with, as part of a process of verifying that a particular model, when run, truly was trained on the data it claims to have been.
It might be necessary to store the hash, or equivalent, at the point of manufacture. Since these devices have a limited lifespan, given how hard they're worked, I don't see this as a blocker.
This article (https://eco.com/support/en/articles/14796365-tees-for-ai-agents-verifiable-compute) explores this kind of idea much further, but doesn't mention using this on training data.
The next possibility is something Meta have an open source implementation of, I would guess for use with their own data (& ideally not for all of the author's books they talked about pirating in internal emails):
# Dataset watermarking
Watermarks embedded into training datasets to track data provenance and usage. This technique enables dataset creators to verify if their data was used to train specific models, providing accountability and attribution in the AI training pipeline.
-- from the "Dataset WM" tab: https://facebookresearch.github.io/meta-seal/
A watermark could be applied to an entire dataset.
I found an article explaining a three-step approach, focused on the personal data that is inevitably consumed as part of training LLMs, that goes even further.
To summarise, the author suggests:
1. Individuals have a "personal vault"
2. AI training reads from these vaults in their encrypted state, determining weightings & other information without needing to see personal data
3. Users maintain control over their vaults, can revoke access at any time & can specify compensation they want in exchange
These existing technologies show that the race to develop the most advance AI models (LLMs, etc.) have focused solely on the capability of their outputs, & little do with the methods of developing them.
This branch of technology could have had its start powered exclusively by renewable energy, but instead is driving demand for coal and gas:
https://fortune.com/2025/08/31/ai-data-center-boom-old-coal-plants/
It could have pioneered data transparency, but is only reaching new highs in lawsuit settlements for piracy.
It could have been a boost to the creative industry, with artists reaching wider audiences & compensated for their work.
But it's been none of these. Instead, AI has left a trail of problems in the race to the (moral) bottom. What we should now do is utilise the products of these acts of theft & pollution to improve how these technologies will operate in the future. We should use what we have, including the very tech that got us here, to climb back out of the problems that have been created, & go further.
Perhaps policies could be written quicker, ideas tested better & more proof gathered of wrong-doing than ever before.
