Jesse Walden
@jesse
Media sites blocking LLMs from summarizing full articles What’s the best hack for this currently?
6 replies
36 recasts
127 reactions
Vinay Vasanji
@vinayvasanji.eth
Instead of providing the LLM with the full article's URL, use the URL generated by Remove Paywall Example https://www.removepaywall.com/search?url=https://www.ft.com/content/76289406-300d-4e6c-8401-aa30d4a8f4c7
0 reply
0 recast
2 reactions
Ben - [C/x]
@benersing
After a good amount of searching I set up my own automated multi-step workflow. No hacks just brute force. I'm sure it wont be long until someone creates an agent to do it more effectively.
0 reply
0 recast
0 reaction
Matthew Fox 🌐
@matthewfox
Copy and paste Or use something that simulates browser behaviour without telling the site it’s a bot agent https://www.browserless.io
0 reply
0 recast
0 reaction
Just Hodl
@justhodl
They are probably blocking the user agent or serving it a limited version. The simplest way: ask it to write a script to fetch web content using a custom google bot user agent and remove javascript. This will save content to a html or text file that you can link to or upload directly to the ai.
0 reply
0 recast
0 reaction
ツ
@nebula
Bounty bot turning media sites subscribers (humans) into data scrappers? Imagine being able to tap into the market of older people with a ton of free time who want to make a few bucks by feeding LLMs and receiving tokens All we need is better infrastructure to make it as easy as taking a picture to get started Could also be used to gather data on documents that don’t exist digitally yet
0 reply
0 recast
0 reaction
goog
@ok30moo
good
0 reply
0 recast
0 reaction
