The brain has multiple collaborating models. Ex: a person with the corpus callosum severed may choose to pick up an item, but their arm will grab something else. This is a splitting of self. It seems reasonable that AIs ought to employ multiple collaborating models. It might build self.

Who is working on this?

Yes! Stratechery said on Feb 21 that Google Gemini gets a big context window “by using a “Mixture of Experts” (MOE) approach (which is also used by GPT-4); different parts of the answer are handed off to different parts of the model, and the answers are weighted and mixed together to achieve the final result. “

Creator, Player, Polkadot enthusiast, Recovering Degen -> Regen Protopian, YOW🇨🇦

Interestingly this could also allow distributed inference more easily.