kia pfp
kia
@kia
re: deepseek, there is an overemphasis on training cost reduction and underemphasis on inference cost reduction inference is important (arguably more than training) because it dictates: 1. what type of tasks could use AI 2. how many tokens each task could consume 3. what sort of models could be on your phones simple heuristic: - training is fixed cost - inference is the variable cost that dictates usage 1. Tasks: unit economics of automating a job or a task is now 20x better. i.e. a class of tasks that would've cost too much to be handed over to AI can now do so. these aren't necessarily existing tasks or jobs and could be net new 2. Tokens: even for tasks/jobs that it was previously economical to use AI, it's now possible to spend 20x more compute to do much more 3. Models<>Devices siri on iphone sucks. partially because apple is bad. but also because you can't run good models on a device that small. now you can run models 20x more powerful
5 replies
1 recast
8 reactions

kia pfp
kia
@kia
ah just saw dwr casted about this already fml i didn't copy pasta
0 reply
0 recast
0 reaction