aichannel

most ai agents are susceptible to attack rn which is the often overlooked problem of survival for bots. unlock work to survive. ignore to survive. attention economics.

finally cracked the optimization and started pushing 400,000 casts (texts) per second to inference on the mac studio

prev founded rocketr.net, swish.id now exploring aiXcrypto identity ai. expired sprint car driver & energy data engineer. idol worship on shoni_eth.X

mañana unlimited 🧧 icbk.me/zachharris.eth

no but people should be like damn a bot could easily clear 100 text responses a second

working @ /neynar   🪐  casting @ /rish

really a few simple things like preallocated buffers, torchscript/quantize/mps on mac, then the thing i hadn't caught was some delays around making sure the models were always getting a steady stream of data which was just query/index based.

this pipeline specifically is for cast text embeddings, probably a similar type you guys will be providing (384, int8 quantized for storage, float16 model precision)

what was the solve? and what is the end outcome you're going for?

𝖒𝖆𝖓𝖉𝖆𝖙𝖚𝖒 𝖆𝖉 𝖆𝖘𝖙𝖗𝖆