s_tworkowski
@2039900203990082
Introducing LongLLaMA 🦙, an unlimited-context version of OpenLLaMA fine-tuned at 8k & capable of extrapolating to 256k tokens! We train it using our new Focused Transformer 🎯 technique (FoT). No degradation on short context, drop-in compatibility & Apache 2.0 license 🔥🔥 🧵 https://t.co/QiNl5xNYvl
0 reply
0 recast
0 reaction