ted (not lasso)
@ted
potentially dumb question: if you trained GPT-4 on all published scientific papers, would it be able to identify which outcomes or findings were repeated across the corpus? yes, right?
8 replies
0 recast
0 reaction
jmon.eth
@jmon
Tried it out years ago, but scite is prob a decent version of what you describe. As others are saying, it will never be that clean (eg “this is a true fact”), but scite flags citations, differentiating support vs conflict vs mention & relative strength. Meta problem is much of literature is trash tho… scite.ai
1 reply
0 recast
0 reaction
Erin Magennis🎩🌻
@erin
Based on consensus in DeSci currently, GPT3 (at least, unsure if this is validated on GPT4), is able to effectively pull out findings and could in theory aggregate them but cannot effectively discern between good experimental design which could result in lots of inaccurate data being included in this compilation.
1 reply
0 recast
0 reaction
Connor McCormick ☀️
@nor
I wonder if the fact that scientific papers are covered under copyright has something to do with the relative un-insightfulness of GPTs. If we lived in a world where they were part of the corpus I'm certain GPTs would be more informed
1 reply
0 recast
0 reaction
Matthew
@matthew
One thing you could do is generate embeddings for all of them and then search based on that. would tell you which papers discuss the same subject and where, but not necessarily similar findings AFAIK.
1 reply
0 recast
0 reaction
timdaub
@timdaub.eth
I think GPT is quite bad with identification itself. E.g. when you ask for famous blockchain books then it‘ll just come up with nonsense because it can generate good sounding titles and lists on the fly.
0 reply
0 recast
0 reaction
Cameron Armstrong
@cameron
Sort of: It’s not like search where you can find all papers with a fuzzy matched set of criteria and count the “similar results” It’s more like you could autocomplete question : answer pairs and the answers that show up most often in the training set would be more likely to appear However you could then..
1 reply
0 recast
0 reaction
Hannes
@stiebitzhofer
I did some research on industrial standards, the summaries were great. However when asking for specific references it returned quite some which I could not find at all…. Wrong but for what reason so the question is will it come up with results to be trusted..
0 reply
0 recast
0 reaction
🎩 MxVoid 🎩
@mxvoid
According to GPT-4, nah. https://i.imgur.com/JrQqrFm.png
1 reply
0 recast
0 reaction
