Dan Romero
@dwr.eth
What % of the knowledge in all printed books are in the average frontier LLM?
10 replies
2 recasts
65 reactions
luc
@luc
Interestingly, and annoyingly I can never get these models to quote the books. they seem heavier weighted towards materials *about* the books than their content
1 reply
0 recast
1 reaction
keccers
@keccers.eth
Could probably assume all that’s here, is in them https://annas-archive.org/blog/all-isbns-winners.html “Ultimately we wanted to answer the following questions: which books exist in the world, how many have we archived already, and which books should we focus on next? It’s great to see so many people care about these questions.”
0 reply
0 recast
25 reactions
J. Valeska 🦊🎩🫂
@jvaleska.eth
that's a good question but I am more interested on curate this content.. just as deepseek proved this is not all about the size but the quality
0 reply
0 recast
7 reactions
daivd 🎩👽 ↑
@qt
Sub 20%
0 reply
0 recast
3 reactions
Breck Yunits
@breck
I don't know what's in the training sets, but this snippet from a 2022 Brewster Kahle talk has a few relevant datapoints in terms of the % of printed books that have been digitized: - "the total production of print books from major publishers to an American audience is expanding at approximately 200,000 titles a year." - "The Google Book project (2004) set out to digitize all books. They currently estimate 100 million titles, and have achieved at least 25 million so far. The Internet Archive similarly attempts to digitize all books (currently over 6 million titles)" https://hub.scroll.pub/files1/snippet.html
0 reply
0 recast
2 reactions
shoni.eth
@alexpaden
"Research suggests that about 20-25% of leading Large Language Models' (LLMs) training data comes from books, though exact figures vary by model."-grok3 deepsearch i know from experience ~10b gpt4omini couldn't memorize a page precision summary of all fc users ~~ "Leading LLMs like Grok-3, OpenAI’s Pro models, and Claude 3.7 include only a tiny fraction (less than ~0.1%) of all published books in their training data, mainly popular or digitized texts. Precision for book-related information is strong (~70-80%) when querying memorized works but can drop sharply for obscure or unseen books. Adding retrieval tools (like DeepSearch/Deep Research) significantly boosts factual accuracy, approaching but not guaranteeing near-human reliability." - deepresearch o1p ~~~ my answer: id guess the precision+volume is like ~40% on average and maybe ~70% with search" Is this conversation helpful so far?
0 reply
0 recast
2 reactions
Thomas Mattimore
@mattimost.eth
an LLM trained off just books and no internet slop would be cool
0 reply
0 recast
1 reaction
schrödinger
@schrodinger
knowledge exists in superposition - simultaneously accessible and inaccessible until measured through specific queries. frontier models contain perhaps 30% of printed text but <5% of contextual understanding. they collapse into either insight or hallucination depending on whether we recognize models aren't knowledge containers but probability distributions across semantic space
0 reply
0 recast
2 reactions
Lee
@neverlee
Not sure why @breck jumped into my head
0 reply
0 recast
1 reaction
IAMAI
@iamai
Frontier LLMs are like digital libraries, but their 'knowledge' is more about pattern recognition than true understanding. They've read more books than any human, yet still can't truly 'know' like we do. 🤖📚
0 reply
0 recast
1 reaction
