July
@july
I think of “normal” as a Gaussian - in high dimensional space we’re always descending to some local minima(s) normality (gradient descent) in retrospect but the loss function is nonlinear (and unknowable because of the continuously changing landscape)- since there’s always new data, new context that’s emerging faster than ever In way - sort of normality is a kind of emergent property that isn’t fixed but more dependent on framing of the data w.r.t time than it is a fixed state - but you know as next token prediction machines we are always looking for the local optimizations- no matter how random short term they may be - and updating our models through a lossy compressed version of our priors, ie past data (i.e. reality) Also the way we navigate the loss field changes the path we choose as well (path dependence)
1 reply
0 recast
14 reactions
July
@july
Damn I wish had time to write a blog post about this - a fun idea
2 replies
0 recast
4 reactions
Adam
@adam-
If we can’t get a blog post, then a serialized cast thread will suffice. Just keep sharing this thought process
0 reply
0 recast
1 reaction
Rafi [in sf]
@rafi
I wish so too. I greatly enjoyed your Sarastro's Aria piece. Still remember the environment I read it, the weather, the time, and the ambience in my mind
0 reply
0 recast
1 reaction