attention is easy. sustained attention is hard.

creating and destroying; built flying cars & autonomous vehicles, now building /faust, more here: /july, web: july.rocks

it's like we're all running transformer models - sustained attention is difficult focusing on incoming input using the self-attention input is easy for transformers, keeping attention on key pieces of info over long periods of time is difficult - mostly because of the limited context window / and also through multiple layers of convolution

makes me think of how LSTM used to handle sustained attention, it kept the cell state, which acted as a sort of memory and evolved with the each time step where as RAG feels more like how humans do it, where we remember some specific thing that happened, and query our knowledge base (specifically about it)

Agreed

Combined with Xerox level fumbling the bag

google’s greatest contribution to humanity. 

Be specific.     

leonasskau.co.uk //
Hosting /strategy, /leo, /geopolitics // built Culture3.com // online: leosn.xyz
d33m:mun

100%

https://warpcast.com/july/0x050d9790

these people did not sustain attention at google

writing at: https://paragraph.xyz/@thearcadia || curating vibes over at /football || bringing football onchain at /thefalsenine

its usually the other way around, but yeah

Also the rarest and purest form of generosity according to Simone Weil.