Quilibrium

in simple and eli5 terms:

what is a shard? why are they important? what do they mean for scalability?

father. working on /anky - new app coming out on the 20th of november

Shards are like splitting up a big job into smaller tasks. They're important because they let us handle huge amounts of data by breaking it into chunks that different computers can work on separately. 

For scaling up, shards are crucial. They mean we can keep growing our system to serve more users and store more info without everything grinding to a halt. It's like adding more workers to an assembly line - we can produce a lot more by dividing up the work.

Explorer of the Cryptoverse, Attention Trader, Serendipity Addict, Cat Worshiper. Butler @ Quilibrium.one

wowow

it is even hard for me to conceive or conceptualize or create a mental model of the notion of "different computers" working on the same thing separately

@lamat 's assembly line example is a good one. One fine detail:

Multiprocessing:  splitting some work up in the same computer e.g. each of your CPU cores zipping up part of a giant file

Sharding: splitting some work up across multiple computers e.g. computer 1 runs an analysis on all even numbered FIDs and computer 2 runs the same analysis but on all odd numbered FIDs, so that it takes half time of either computer doing the work alone