@dan recently posted an article about the problems with the blockchain databases in Hive and EOS, which we had highlighted 2 years ago in the "vertical scalability" section of the original Koinos whitepaper.
Our solution to this problem is something that tends to get lost in the conversation about Koinos which generally focuses more on our Mana system, proof-of-burn consensus algorithm, fork-less upgrades, and universal programming language support for smart contracts. So I'm glad Dan is finally bringing attention to this highly technical problem that very few people are familiar with.
StateDB
We call our solution "StateDB" and it started as an evolution on chainbase (the backing database used in Bitshares, Steem, Hive, and EOS), but has become its own beast entirely, replacing chainbase in the process. Chainbase has certain design flaws that have created both performance and scaling limitations. As Dan explains in his post, these limitations result from the use of what he calls "undo history" but we call “undo states."
Undo States
With undo states, the database tracks the most current state as the head block state plus pending transactions. This means the database never actually reflects the current state of the blockchain. In order to address that issue, when each block is applied the pending transaction state is “undone,” the old values are written back to the database, and then the block is applied. This is highly inefficient because it generally means performing the exact same calculations over again only to write the exact same state back to the database even though it was already there! StateDB has a fundamentally different design with a number of key benefits.
Key Benefits
When people hear "forks" they tend to think of "hard forks"; the coordinated rebooting of the blockchain which is required for upgrading system behaviors, assuming it does not have the modular upgradeability of Koinos. Forks are constantly occurring in a blockchain network as a natural consequence of being a decentralized system. When they a fork occurs the node has to undo all of the blocks that they have already applied back to the fork block and then replay them along with all the related calculations along with them. All of this building, deconstructing, and then rebuilding can dramatically increase the cost of running a node, especially since these actions often make extensive use of RAM. When the blockchain is near capacity this can also force database contention to dangerous levels.
StateDB takes a fundamentally different approach to managing this database that provides several key benefits:
- A new block can be applied while readers read from the state of a previous block which will make Koinos more responsive to API requests,
- Multiple blocks from multiple forks can be applied in parallel without any contention on shared resources,
- Fork resolution does not require writing to the database at all so Koinos will be able to run at a higher percentage of maximum capacity without suffering significant penalties from forks,
- Live snapshots can be taken of Koinos because a thread can write the state at a specific block to disk in the background while the node continues processing live blocks, and
- Node failures will be minor inconveniences as opposed to the catastrophe that it can be on other blockchains.
Positive State Deltas
StateDB accomplishes this, in part, through the use of what we call, “positive state deltas.” It has a single backing database that tracks irreversible state so that it never needs to be undone. Then, instead of storing old values of objects, it stores new values of objects, and projects those new values across multiple undo states (“state nodes” in StateDB vernacular) to present the consensus state at a given block. By storing positive deltas that are then collapsed during a read to provide access to the current state, undoing and redoing state is as simple as choosing which deltas to collapse which means that switching forks does not have implicit writes in the algorithm.
Furthermore, this can support processing a new block while reading state from the previous block in parallel.
RocksDB
In addition to the advantages listed above, this enables Koinos to minimize RAM usage in nodes by efficiently ferrying irreversible transactions out of RAM and into secondary memory. Since Koinos has a single database of irreversible state that never needs to be undone, that database can be stored entirely in disk and never brought back into RAM to resolve forks. Because StateDB is designed specifically to work with RocksDB, performance can be maintained even while commodity storage media is used.
This focus on optimizing the blockchain to use commodity hardware has the added benefit of making node operation accessible to ordinary people, not just those with access to the enterprise-grade hardware used in most other general purpose blockchain networks.
This feature alone will dramatically reduce the cost of running a Koinos node and make node operation accessible to far more people, thereby improving decentralization.
Proof-of-burn
Our innovative consensus algorithm, proof-of-burn, takes this egalitarianism to the next level by totally virtualizing the mining hardware. With proof-of-burn, the block producer’s incentive is not to perform as much work as possible, but to perform as little work as is necessary to attempt to produce on every block. PoB gives us the speed of DPoS, with the decentralization of PoW. To learn more about about proof-of-burn, check out this medium article.
Koinos Mainnet
The proof-of-burn smart contract is the only smart contract remaining to be written for the Koinos mainnet and we're excited to announce that work on that smart contract has already begun which is why now is the perfect time for developers to begin building their Koinos applications!
If you're interested in building on Koinos, stop on by our discord server, telegram chat, or schedule a free consultation by going to koinos.group.
To learn more about Koinos, head on over to koinos.io. Thanks for reading!