diff --git a/docs/consensus-spec.md b/docs/consensus-spec.md index e639b704..8aabf7f1 100644 --- a/docs/consensus-spec.md +++ b/docs/consensus-spec.md @@ -415,359 +415,28 @@ Each block is a byte string, which includes a header followed by a body. Block header is a JSON object that includes the following: -. `*BLOCK~ID~*` - integer id of the current block, starting from 0 and incremented by 1 -. `*BLOCK PROPOSER*` - integer id of the node that proposed the block. -. `*PREVIOUS BLOCK HASH*` - SHA-3 hash of the previous block -. `*CURRENT BLOCK HASH*` - the hash of the current block -. `*TRANSACTION COUNT*` - count of transactions in the current block -. `*TRANSACTION SIZES*` - an array of transaction sizes in the current block -. `*CURRENT BLOCK PROPOSER SIG*` - ECDSA signature of the proposer of the current block -. `*CURRENT BLOCK T~SIG~*` - BLS supermajority threshold signature of the current block +* `blockId*` - integer id of the current block, starting from 0 and incremented by 1 +* `blockProposer*` - integer id of the node that proposed the block. +* `previousBlockHash` - SHA-3 hash of the previous block +* `blockHash` - the hash of the current block +* `transactionCount` - count of transactions in the current block +* `transactionSizes` - an array of transaction sizes in the current block +* `proposerSig` - ECDSA signature of the proposer of this block +* `thresholdSig*` - BLS supermajority threshold signature of JSON object that includes winning `block proposer index` and `block_id`. +* `daProof` - `DA proof threshold signature of JSON object that includes `block_id`, `block proposer index` and `block_hash` Note: All integers in this spec are unsigned 64-bit integers unless specified otherwise. ## 5.2.2 Block body -`BLOCK BODY` is a concatenated transactions array of all transactions in the block. - -## 5.2.3 Blockhash - -Block hash is calculated by taking 256-bit Keccack hash of block header concatenated with block body, while omitting `CURRENT BLOCK HASH`, `CURRENT BLOCK SIG`, and `CURRENT BLOCK TSIG` from the header. The reason why these fields are omitted is because they are not known at the time block is hashed and signed. - -Note: Throughout this spec we use SHA-3 as a secure hash algorithm. - -## 5.2.4 Block verification - -A node or a third party can verify the block by verifying a threshold signature on it and also verifying the previous block hash stored in the block. Since the threshold signature is a supermajority threshold signature and since any honest node will only sign a single block at a particular block ID, no two blocks with the same block ID can get a threshold signature. This provides security against forks. - -## 5.3 Block proposal - -A block starts as a block proposal. A block proposal has the same structure as a block, but has the threshold signature element unset. - -Node concurrently make proposals for a given block ID. A node can only make one block proposal for a given block ID. - -Once a block proposal is selected to become a block by consensus, it is signed by a supermajority of nodes. A signed proposal is then committed to the end of the chain on each node. - - - -=== Compressed block proposal communication - -Typically pending queues of all nodes will have similar sets of messages, with small differences due to network propagation times. - -When node `*A*` needs to send to node `*B*` a block proposal `*P*`, `*A*` does need the send the actual transactions that compose `*P*`. `*A*` only needs to send transaction hashes, and then `*B*` will reconstruct the proposal from hashes by matching hashes to messages in its pending queue. - -In particular, for each transaction hash in the block proposal, the -receiving node will match the hash to a transaction in its pending -queue. Then, for transactions not found in the pending queue, the -receiving node will send a request to the sending node. The sending node -will then send the bodies of these transactions to the receiving node. -After that the receiving node will then reconstruct the block proposal. - -== Consensus data structures and operation - - - -=== Consensus rounds - -New blocks a created by running consensus rounds. Each round corresponds -to a particular `*BLOCK~ID~*`. - -At the beginning of a consensus round, each node makes a block proposal. - -When a consensus round completes for a particular block, one of block -proposals wins and is signed using a supermajority signature, becoming a -committed block. - -Due to a randomized nature of consensus, the is a small probability that -consensus will agree on an empty block instead of agreeing on any of the -proposed blocks. In this case, an empty block is pre-committed to a -blockchain. - -=== Catchup agent - -There are two ways, in which blockchain on a particular node grows and -`*TIP~ID~*` is incremented: - -Normal consensus operation: during normal consensus, a node constantly -participates in consensus rounds, making block proposals and then -committing the block after the consensus round commits. - -Catchup: a separate catchup agent is continuously running on a node. The -catchup engine is continuously making random sync connections to other -nodes. During a sync both nodes sync their blockchains and block -proposal databases. - -If during catchup, node `*A*` discovers that node `*B*` has a larger value -of `*TIP~ID~*`, `*A*` will download the missing blocks range from `*B*`, and -commit it to its chain after verifying supermajority threshold -signatures on the received blocks. - -Note that both normal and catchup operation append blocks to the -blockchain. The catchup procedure intended to catchup after hard -crashes. - -When the node comes online from a hard crash, it will immediately start -participating in the consensus for new blocks by accepting block -proposals and voting according to consensus mechanism, but without -issuing its own block proposals. Since a block proposal requires hash of -the previous block, a node will only issue its own block proposal for a -particular block id once it a catch up procedure moves the `*TIP~ID~*` to -a given block id. - -Liveliness property is guaranteed under hard crashes if the following is -true: normal consensus guarantees liveliness properly, catch-up -algorithm guarantees eventual catchup, and if the number of nodes in a -hard crashed state at a given time plus the number of Byzantine nodes is -less or equal `*N ⅓*`. - -Since the normal consensus algorithm is resilient to having latexmath:[\frac{N-1}{3}] -Byzantine nodes, normal consensus will still proceed if we count crashed -nodes as Byzantine nodes and guarantee that the total number of -Byzantine nodes is less than latexmath:[\frac{N-1}{3}]. When a node that crashed joins -the system back, it will immediately start participating in the new -consensus rounds. For the consensus rounds that it missed, it will use -the catchup procedure to download blocks from other nodes. - -== Normal consensus operation - -=== Block proposal creation trigger - -A node is required to create a block proposal directly after its -`*TIP~ID~*` moves to a new value. `*TIP~ID~*` will be incremented by $1$ -once a previous consensus round completes. `*TIP~ID~*` will also move, if -the catchup agent appends blocks to the blockchain. - -=== Block proposal creation algorithm - -To create a block a node will: - -. examine its pending queue, - -. if the total size of of transactions in the pending queue `TOTAL SIZE` is less or equal than `MAX BLOCK SIZE`, fill in a block proposal by taking all transactions from the queue, - -. otherwise, fill in a block proposal by of `MAX BLOCK SIZE` by taking transactions from oldest received to newest received, - -. assemble transactions into a block proposal, ordering transactions by sha-3 hash from smallest value to largest value, - -. in case the pending queue is empty, the node will wait for `BEACON TIME` and then, if the queue is still empty, make an empty block proposal containing no transactions. - -Note that the node does not remove transactions from the pending queue -at the time of proposal. The reason for this is that at the proposal -time there is no guarantee that the proposal will be accepted. - -`MAX BLOCK SIZE` is the maximum size of the block body in bytes. -Currently we use `MAX BLOCK SIZE = 8 MB`. FUTURE: We may consider -self-adjusting block size to target a particular average block commit -time, such as `1s`. - -`BEACON TIME` is time between empty block creation. If no-one is -submitting transactions to the blockchain, empty beacon blocks will be -created. Beacon blocks are used to detect normal operation of the -blockchain. The current value of `BEACON TIME` is `3s`. - -=== Block proposal reliable communication algorithm - -Once a node creates a block proposal it will communicate it to other -nodes using the data data availability protocol described below. - -The data availability protocol guarantees that if the the protocol -completes successfully, the message is transferred to the supermajority -of nodes. - -The five-step protocol is described below: - -1. Step 1: the sending node `*A*` sends the proposal `*P*` to all of its - peers - -2. Step 2: each peer on receipt of `*P*` adds the proposal to its - proposal storage database `PD` - -3. Step 3: the peer than sends a receipt to back to `*S*` that contains a - threshold signature share for `*P*` - -4. Step 4: `*A*` will wait until it collects signature shares from a - `supermajority` of nodes (including itself) `*A*` will then create a - supermajority signature `*S*`. This signature serves as a receipt that - a supermajority of nodes are in possession of `*P*` - -5. Step 5: `*A*` will send the supermajority signature to each of the - nodes. - -_Data Availability Receipt Requirement_ In further consensus steps, any -node voting for proposal `*P*` is required to include `*S*` in the vote. -Honest nodes will ignore all votes that do not include the supermajority -signature `*S*`. - -The protocol used above guarantees data availability, meaning that any -proposal `*P*` that wins consensus will be available to any honest nodes. -This is proven in steps below. - -Liveliness. If `*A*` is honest, than the five-step protocol above will -always complete. By completion of the protocol we mean that all honest -nodes will receive `*S*`. Byzantine nodes will not be able to stall the -protocol. - -By properties of the send operation discussed in Section 1.2 all sends -in Step 1-3 are performed in parallel. In step 4 node `*A*` waits to -receive signature shares for the supermajority of nodes. This step will -always take fine time, even if Byzantine nodes do not reply. This comes -from the fact that there is a supermajority of honest nodes. In step 5 -`*S*` will be added to outgoing message queues of all nodes. Since honest -nodes do accept messages, `*S*` will ultimately be delivered to all honest -nodes as described in Section 1.2. - -If a proposal has a supermajority signature, it is was communicated to -and stored on the simple majority of honest nodes. - -The proof directly follows from Lemma 3, and from the fact that an -honest node `*B*` only signs the proposal after `*B*` has received and -stored the proposal. - -If a proposal wins consensus and is to be committed to the blockchain, -then any honest node `*X*` that does not have the proposal can efficiently -retrieve it. - -First, a proposal will not pass consensus without having a supermajority signature. This comes from the fact that all nodes voting for the proposal will need to include `*S*` in the vote. - -By the properties of binary Byzantine agreement protocol of Mostéfaoui at al., a proposal can win consensus only if at least one honest node votes for the proposal. A proposal without a signature will never win consensus, since an honest node will never vote for it. - -Therefore, if a proposal won consensus, it is guaranteed to have a supermajority signature. - -Second by previous lemma, if a proposal has a supermajority signature, any honest node can retrieve it. This completes the proof. - -The protocol discussed above is important because it guarantees that if a proposal wins consensus, all honest nodes can get this proposal from other honest nodes and add it to the blockchain. - -=== Pluggable Binary Byzantine Agreement - -The consensus described above uses an Asynchronous Binary Byzantine Agreement (ABBA) protocol (ABBA). We currently use ABBA from Mostéfaoui et. all. Any other ABBA protocol `*P*` can be used, as long as it has the following properties - -. Network model: `*P*` assumes asynchronous network messaging model described in Section 1.2 - -. Byzantine nodes: `*P*` assumes less than one third of Byzantine nodes, as described by Equation (1). - -. Initial vote: `*P*` assumes, that each node makes an initial vote `yes(1)` or `no(0)`. - -. Consensus vote: `*P*` terminates with a consensus vote of either `yes` or `no`, where if the consensus vote is `yes`, its is guaranteed that at least one honest node voted yes. - -Note that, an ABBA protocol typically outputs a random number `*_COMMON COIN_*` as a byproduct of its operation. We use this `*_COMMON COIN_*` as a random number source. - -=== Consensus round - -A consensus round `*R*` is executed for each `*BLOCK~ID~*` and has the following properties: - -. For each `*R*` nodes will execute `*N*` instances of ABBA. - -. Each `*ABBA~i~*` corresponds to a vote on block proposal from the node `*i*` - -. Each `*ABBA~i~*` completes with a consensus vote of `yes` or `no` - -. Once all `*ABBA~i~*` complete, there is a vote vector `*v~i~*`, which - includes `yes` or `no` for each proposal. - -. If there is only one `yes` vote, the corresponding block proposal - `*P*` is committed to the blockchain - -. If there are multiple `yes` votes, `*P*` is pseudo-randomly picked from - the `yes`-voted proposals using pseudo-random number `*R*`. The - winning proposal index the remainder of division of `*R*` by - $n_~win~$, where $n_~win~$ is the total number of `yes` proposals. - -. The random number `*R*` is the sum of all ABBA `*_COMMON COIN_*`. - -. In the rare case when all votes are `no`, an empty block is - committed to the blockchain. The probability of an all-no vote is - very small and decreases when `*N*` increases. This is analyzed in - detail in the following sections. - -Liveliness: each consensus round `*R*` will always produce a block in a -finite time. - -The proof follows from the fact that each `*R*` runs `*N*` parallel versions -of `*ABBA*` binary consensus, and from the liveliness property of the -`*ABBA*` consensus - -Consistency: each consensus round will produce the same result `*P*` on -all nodes - -This follows from the consistency property of the ABBA consensus and -from the fact that the consensus round algorithm is deterministic and -does not depend on the node where it is executed. - -Data Availability: the winning proposal `*P*` is available to any honest -node. - -This follows from the fact, that ABBA will not return consensus `yes` -vote unless at least one honest node initially votes `yes`, and from the -fact that an honest node will not vote `yes` unless it has a data -availability proof (threshold signature `*S*`). - -== Consensus round vote trigger - -Each node `*A*` will vote for ABBAs in a consensus round `*R*` immediately -after proposal phase completes, meaning that two processes complete: - -1. `*A*` receives a supermajority of block proposals for this round, - including data availability signatures - -2. `*A*` transmits its block proposal to a supermajority of nodes - -Liveliness: the block proposal phase will complete in finite time, and -the node will proceed with voting - -Indeed, since a supermajority of nodes are honest, and since every -honest node sends its block proposal and data availability signature to -all other nodes, at some point in time `*A*` will receive proposals and -data availability signatures from a supermajority of nodes. - -Also, since a supermajority of destination nodes are honest, at some -point in time the node will transmit its block proposal to a -supermajority of nodes. - -It will vote `yes` for each block proposal that it received, and `no` -for each block proposal that it did not receive. - -Vote of each honest node will include latexmath:[\frac{2N+1}{3}] `yes` votes and -latexmath:[\frac{2N-1}{3}] `no` votes - -This simply follows from the fact, that node `*A*` votes immediately after -receiving a supermajority of block proposals, and from the fact that `*A*` -votes yes for each block proposal that it received - -== Finalizing Winning Block Proposal - -Once consensus completes on a particular node `*A*` and the winning block -proposal, the node will execute the following algorithm to finalize the -proposal and commit it to the chain. - -. `*A*` will check if it has received the winning proposal `*P*` - -. if `*A*` has not received the proposal, it will download it from its peer nodes using the algorithm described later in this document. It is possible to do it because of Lemma 11. - -. `*A*` will then sign a signature share for `*P*` and send it to all other nodes - -. `*A*` will then wait to receive signature shares from a supermajority of nodes, including itself - -. Once `*A*` has received a supermajority of signature shares, it will combine them into a threshold signature. - -. `*A*` will then commit the `*P*` to the blockchain together with the threshold signature of `*P*` - -The proposal download algorithm is specified below. The proposal assumes -that the proposal is split in $N-1$ chunks of equal size -latexmath:[\left\lceil \frac{size(P)}{N-1} \right\rceil], except the last chunk the size of -which will be the remainder of latexmath:[\frac{size(P)}{N-1}]. - -The purpose of the algorithm is to minimize network traffic. - -. `*A*` sends a message to each peer `*i*` , requesting for chunk `*i*` -. `*A*` waits until it receives a `supermajority - 1` of responses -. `*A*` then enumerates missing chunks -. `*A*` then randomly assigns each missing chunk to a servers, and empty chunks to each server that did not get a missing chunk assigned, and sends the corresponding requests to each server. -. `*A*` waits until receives `supermajority -1` of responses -. If `*A*` received all chunks, the algorithm is complete. Otherwise it goes back to step 3. - +`BLOCK BODY` is a concatenated byte string of all transactions in the block. +## 5.2.3 Block hash +Block proposal hash is calculated by taking hash of +* block header, while omitting `BLOCK HASH`, `BLOCK SIG`, and `TSIG` from the header. The reason why these fields are omitted is because they are not known at the time block proposal is hashed and signed. +* concatenated with `block body`