In the modern world of "big data" you'll encounter the word "shard". It's a technique for splitting NoSQL databases (like MongoDB, Redis, or Elasticsearch) across multiple machines, to handle the world in parallel. Where did this word come from?
The answer appears to be 1996's "Ultima Online". This was the first popular MMORPG -- the game genre now dominated by "World of Warcraft". At it's height, Ultimate Online had 250,000 subscribers. Unfortunately, the world "massive" in the MMORPG meant only a few hundred players at a time on a single server. In order to satisfy everyone, Ultima Online needed to create many, equal servers. When you logged into Ultima Online, you had to select the same server as your friends, or you wouldn't see them online.
Ultima Online included this as part of the storyline. The story is that that the High Wizard Mordaine tried to control the world by encapsulating it into a crystal, and when he was defeated, the crystal was shattered, breaking into in multiple parallel universes. This is all explained in the opening video (embedded below at the end of the this post).
Shards has been a colloquial term for multiple servers ever since. Other MMORPGs like EverQuest and WoW had "servers" and "realms", but players would still call them shards. When it came time to break the bounds of SQL, players who grew up with MMORPGs as kids applied the term to their jobs. Hence, sharding of databases was born.
In my massively online scanning project, "masscan", I use the term, even though I never played Ultima Online. My scanner uses the "--shard" parameter to specify one of many shards carrying out scanning. We are currently dropping the results into "MonogoDB", which can be automatically split across multiple shards. I'm also going to play with Redis and Elasticsearch, which all support shards.
All thanks to Mordaine.
Thanks to @gsuberland, @chort0, @ramblinpeck, and @jjarmoc for the pointers.
No comments:
Post a Comment