1. What is sharding?
Sharding is a method of separating and storing a single logical set of data in the form of multiple databases. Another definition of sharding is horizontal data partitioning.
2. When and who invented sharding?
The concept of sharding has been used in the management of traditional centralized databases since the late 1990s. The term “shard” (fragment) gained traction through one of the first multiplayer online role-playing games, Ultima Online, in which the developers distributed players across different servers (different “worlds” in the game) to handle traffic.
A popular application scenario for sharding in business is to divide the user database into geographic locations. Users belonging to the same geographic location are combined into one group and hosted on a unique server.
3. What is sharding in the context of blockchain?
A blockchain is a database with nodes representing individual servers. In the context of blockchain, sharding refers to the division of the blockchain network into individual segments (shards). Each shard contains a unique set of smart contracts and account balances.
Each shard is assigned a node that verifies transactions and operations, as opposed to a scheme in which each node is responsible for verifying each transaction across the entire network.
Dividing the blockchain into more manageable segments increases transaction throughput and thus solves the scalability problem that most modern blockchains face.
4. How does sharding work?
An explanation using Ethereum as an example:
The Ethereum blockchain consists of thousands of computers or nodes, each of which “lends” a certain amount of hashrate to the network. It is this hashrate that allows the Ethereum Virtual Machine (EVM) to function – to execute smart contracts and manage decentralized applications (DApps).
Ethereum currently operates on a sequential execution basis, in which each of the nodes must calculate each transaction and process each transaction. Therefore, it takes a significant amount of time for a transaction to go through the verification process: Ethereum performs approximately 10 transactions per second, while Visa, for example, has the figure in the neighborhood of 24,000.
Adding computers to the network does not necessarily increase efficiency, since the entire registry is stored on each device, and the verification chain simply becomes longer.
The idea behind sharding is to abandon the model in which each node has to compute every operation, in favor of a parallel execution model in which the nodes process only certain computations. This allows multiple transactions to be processed in parallel.
The blockchain is divided into separate shards (subdomains or segments). Nodes manage only the part of the registry to which they are attached (executing processes and confirming transactions), rather than supporting the entire registry.
5. What problems does sharding solve?
Sharding is a potential solution to the scaling problem.
The more popular blockchain becomes, the more users initiate transactions, run decentralized applications and other processes on the network. As a result, transaction speeds drop, preventing blockchain from expanding in the long run. Increased transactional activity requires nodes to intensify the process of transaction verification. There is a threat that these blockchains could “clog up,” as happened with Ethereum during the CryptoKitties boom, when gaming accounted for 11% of the network’s transactions.
If groups of nodes are responsible for individual segments, then each node does not need to maintain the entire registry to perform each transaction. Therefore, transaction validation can be performed in parallel rather than in a linear fashion, which increases the speed of the network. In this way, the problem of scaling is solved.
6. What are the disadvantages of sharding?
The main problems of sharding are communication and security. If you divide the blockchain into isolated segments, each shard becomes a separate network. Users and applications in one subdomain will not be able to communicate with users and applications in another subdomain without using a special communication mechanism.
A segmented blockchain also poses a security problem because it is easier for hackers to hijack a single shard because of the lower hash rate required to control individual segments (the so-called 1% attack).
Once a segment is hijacked, attackers can route invalid transactions to the main network. Also, data in that particular segment can become invalid and be irretrievably lost. Ethereum offers a solution in the form of randomized sampling – shard protocols are randomly assigned to different sections to confirm block authentication.
7. What are the alternatives to sharding?
Developers have proposed two solutions to improve the performance and speed of blockchain transactions.
The first solution is to increase block size. The key idea is that the larger the block size, the more transactions can be placed in it and, therefore, the higher the number of transactions per second.
However, the larger the block, the more processing power needed to verify it. If the block size is increased significantly, only the most powerful computers will be able to manage the processing power needed to operate as nodes.
The high cost of such computer hardware means that pools of nodes will inevitably become smaller and more centralized, which increases the risk of a 51% attack. Increasing block size also requires a hardforward, which threatens to split the community: if not all users accept the upgrade, two different circuits using different coins will emerge. Increasing block size cannot be a long-term solution.
The second suggestion is to use altcoins so that different functions and applications are implemented on their own networks with their own coins.
This model would increase performance because a single blockchain would not be overloaded, but it would also increase security risks because processing power would be distributed across multiple blockchains. Again, the risk of hacking the network would also increase because the processing power required to execute a 51% attack would be much less.
8. Who uses sharding?
Zilliqa is the first platform to implement sharding. It managed to reach 2,828 transactions per second during the testnet phase.
The Near blockchain ecosystem allows developers to create and use decentralized applications. Near calls itself a “sharded blockchain on PoS” and claims that its sharding technology allows nodes to remain small enough to function on low-performance devices – potentially even cell phones.
Ethereum offers a blockchain ecosystem for the implementation of DApps based on smart contracts. The Ethereum Foundation plans to include sharding in an updated version of the Ethereum 2.0 protocol.
Other projects working with sharding include Cardano, QuarkChain and PChain.
9. What is the future of sharding?
Sharding technology appears in Libra’s digital currency white paper. Ahead of the launch, Facebook acquired Chainspace, whose development team specializes in sharding. Specific details are not yet known, but it can be assumed that a form of sharding will be introduced into Libra’s blockchain.
Sharding could theoretically be a solution to the so-called blockchain trilemma.
The blockchain trilemma, as Vitalik Buterin explained, is that only two of the three key features of blockchain – security, decentralization and scalability – can be preserved simultaneously. If the challenges of sharding can be overcome, it will be possible to scale distributed networks without sacrificing decentralization or security.