# Sharding your database

DevFeed: [Sharding your database](<https://devfeed.tech/articles/sharding-your-database-41123.md>)

Original publisher: [Read original article](<https://www.craigkerstiens.com/2012/11/30/Sharding-your-database/>)

Author: Map

Published: 2012-11-30T20:55:56Z

Content type: tutorial

Language: en

Sources: [Craig Kerstiens](<https://devfeed.tech/sources/craig-kerstiens.md>)

Topics: [sharding](<https://devfeed.tech/topics/sharding.md>), [Databases](<https://devfeed.tech/topics/databases.md>), [scaling](<https://devfeed.tech/topics/scaling.md>), [Heroku](<https://devfeed.tech/topics/heroku.md>)

Tags: [database](<https://devfeed.tech/tags/database.md>), [databases](<https://devfeed.tech/tags/databases.md>), [development](<https://devfeed.tech/tags/development.md>), [hash](<https://devfeed.tech/tags/hash.md>), [heroku](<https://devfeed.tech/tags/heroku.md>), [scaling](<https://devfeed.tech/tags/scaling.md>), [sharding](<https://devfeed.tech/tags/sharding.md>)

## AI overview

This guide explains database sharding for applications whose data outgrows the storage or performance limits of a single database. It covers logical and physical shards, choosing shard counts, determining a shard without querying the database, and primary-key considerations when distributing data across multiple databases.

## Source excerpt

I'm increasingly encountering users on Heroku that are encountering the need to [shard](http://en.wikipedia.org/wiki/Shard_(database_architecture)) their data. For most users this is something you delay as long as possible as you can generally go for sometime before you have to worry about it. Additionally scaling up your database is often a reasonable approach early on and something I encourage as a starting point as scaling up is easy to do with regards to databases. However, for the 1% of users that do need to shard when the time comes many are left wondering where to start, hence the following guide. What and Why Sharding is the process of splitting up your data so it resides in different tables or often different physical databases. Sharding is helpful when you have some specific set of data that outgrows either storage or reasonable performance within a single database. Logical Shards First when initially implementing sharding you'll want to create an arbitrary number of logical shards. This will allow you to change less code later when it comes to adding more shards. You'll also want to define your shards to the power of 2. Generally I'd recommend for most services 1024 can be a good number, I believe Instagram actually used 4096, either can really be an appropriate number. For simplicity sake lets start with an example of using 4 logical shards. First lets look at an example set of users: id | email | name ----+---------------------------+----------------- 1 | craig.kerstiens@gmail.com | Craig Kerstiens 2 | john.doe@gmail.com | John Doe 3 | jane.doe@gmail.com | Jane Doe 4 | user4@gmail.com | User 4 5 | user5@gmail.com | User 5 6 | user6@gmail.com | User 6 7 | user7@gmail.com | User 7 8 | user8@gmail.com | User 8 Dividing these up into logical shards we're going to have something that looks roughly like this: Its important when sharding that you find a mechanism that requires you to not hit the database. As the above example shows its using the ID of the row