Data architecture - Concept
Sharding
Splitting a large dataset across several databases or machines
What's it for?
Spreads a very large set of data across several machines so one machine does not have to hold or process everything.
For example…
A customer database is sharded by customer ID so different groups of records live on different database servers and workload is spread out
As an application grows, new shards are added so one database machine does not have to hold every user's data