Because the introduction of headers-first synchronization in Bitcoin Core 0.10, the blockchain knowledge construction will be seen as having three elements:
- The block header tree: a tree knowledge construction, with the genesis block as root, containing all headers for recognized forks of the chain. This tree construction is all the time linked, i.e., it isn’t doable so as to add a header to the tree except its guardian is already within the tree as effectively.
- The block knowledge: for every entry within the block header tree, we might or might not have the corresponding block knowledge. Blocks will be obtained (and requested for) out of order, so long as the corresponding header exists within the header tree. There will be gaps, so it isn’t the case {that a} block can solely be saved if the guardian block can be saved. In fact, a block can solely develop into the lively chain tip if it is absolutely validated, which requires having seen all blocks earlier than it too. Because the introduction of block file pruning in Bitcoin Core 0.11, additionally it is doable to delete blocks once more after being validated; on this case, the corresponding headers are nonetheless stored within the tree.
- The lively chaintip (a reference to an entry in our block header tree) with its corresponding UTXO set (or chainstate), representing the fully-validated block that we at the moment think about lively (this was launched in Bitcoin 0.8).
Now to clarify how synchronization occurs, there are three separate processes that govern these knowledge buildings respectively:
- Header synchronization: the method of requesting, receiving, and storing, headers {that a} peer has however we don’t.
- Block synchronization: the method of requesting, receiving, and storing, the total block knowledge for headers we have already got, which friends have however we don’t.
- Block activation: the total validation of blocks we’ve got, and ensuing altering of the lively chain tip.
All of them occur in parallel in observe, but it surely’s simpler to consider them as a sequence.
1. Headers synchronization
Requesting headers. Step one of synchronization is studying about headers which our friends have. That is primarily executed by sending getheaders to every peer after we join or settle for a connection (by sending a locator which signifies which headers we have already got), to which they will reply with a headers message containing as much as 2000 new headers. If the total 2000 headers are obtained, we’ll ship one other getheaders to ask for extra. This course of continues till we’ve got all headers the peer has to supply. Once we’re far behind, we solely ask one peer (the headers sync peer) to reduce duplicating bandwidth utilization, however as soon as we get near the present timestamp, all friends are requested.
Non-connecting headers. If headers arrive that do not hook up with our current headers tree (i.e., the guardian of a obtained block is not in our tree but), we additionally ship a getheaders in response to first find out about all of the headers in between. In no way are headers processed after we do not know their mother and father but. This doesn’t suggest they’re rejected (as in: they don’t seem to be marked completely invalid); they’re simply ignored (as in the event that they have been by no means obtained in any respect). If all goes effectively, we’ll obtain them once more after their lacking guardian headers have arrived, at which level they will be processed usually.
Direct headers announcement. Because the introduction of the direct headers announcement mechanism (negotiated utilizing the BIP130 sendheaders message) added in Bitcoin Core 0.12.0, friends can ship us headers messages on to announce a brand new lively chain tip (see beneath). In case this mechanism shouldn’t be used (both as a result of the peer believes we’re far behind, or as a result of the peer would not help BIP130), they will additionally ship us an inv message with only a block hash. This too will set off us to ship a getheaders in response to find out about any headers we miss, as much as and together with the hash that was simply inved to us.
Header validation. When headers arrive, they’re validated to the extent doable. This contains syntactic correctness, proof-of-work, issue changes, guidelines about timestamps (bigger than the median of the previous 11 blocks, no more than 2 hours sooner or later), and guidelines about model numbers (BIP34, BIP66, and BIP65 put necessities on the model quantity). If a header fails any of those, it’s ignored, in addition to any headers that descend from it.
Header spam safety. Whereas block headers are tiny (81 bytes in a header message), we do want to guard towards storing tons of low-difficulty ones that an attacker may create. Proof-of-work inherently makes it very costly to supply good headers in mass, however an attacker can assemble a headers chain that forks off someplace early in historical past (e.g. simply after the genesis block) when the issue was low, with out ever amounting to a major quantity of labor. This might in the end result in reminiscence points for nodes if they might settle for these headers into their block header tree. Traditionally, checkpoints have been (amongst different issues) used to protect towards this, however since Bitcoin Core 24.0 one other, extra complete, mechanism is used: header pre-synchronization. In brief, it entails downloading the headers twice: a primary time to confirm the headers type a sequence with a major quantity of labor (however not including them to the block header tree but), and a second time the place they’re redownloaded and in the event that they match what was despatched within the first part, they’re added to the block header tree. You possibly can learn extra about this mechanism in this reply.
2. Block synchronization
Acceptable blocks. As soon as we all know concerning the headers, we will obtain blocks. On the whole, blocks are solely requested (and accepted) if they’re a part of a headers chain whose tip has not less than as a lot cumulative work as the present lively chain tip. Blocks that aren’t on our headers tree are by no means accepted (these could be orphan blocks), nor are blocks on forks whose header chain would not have a aggressive quantity of cumulative work.
Request mechanism. Blocks are requested utilizing getdata messages, to which the peer responds with block messages (getdatas are additionally used for transaction fetching, however I’m ignoring that right here). To resolve which friends these requests are despatched to, we preserve observe of the inv and headers bulletins friends have despatched us, so we’ve got an concept of what the final block header we’ve got in frequent with them is. We usually assume that friends who announce a block to us may really present the corresponding block knowledge (not less than after they set the NODE_NETWORK service flag, or NODE_NETWORK_LIMITED for sufficiently current blocks).
Scheduling of requests. The precise getdata requests are despatched for lacking blocks anyplace alongside the trail from the genesis block to the headers tree tip with the very best amassed work (which will be distinct from the lively chain tip) amongst all block header tree entries which aren’t marked as completely invalid. The requests are unfold over all friends for which we all know they’ve the respective block, in a round-robin vogue, with a restrict of 16 requested-but-not-yet-received blocks per peer. We don’t request blocks that are greater than 1024 blocks forward of the present lively chain tip, to restrict how out-of-order blocks will be obtained (this issues for block file pruning). So long as a block request is excellent, the identical block shouldn’t be requested once more from one other peer to keep away from duplicating bandwidth utilization. If a peer stalls too lengthy in responding to a block request, it’s disconnected, and the request will probably be despatched elsewhere.
Unrequested blocks. It’s doable {that a} block is obtained with out it being requested. Bitcoin Core is not going to ship unrequested blocks, however another P2P purchasers may. If this occurs, we’ll course of the header contained in the block as if it was obtained by way of a headers message, after which roughly if we’d need to request it after seeing the header, we additionally course of the block straight. In any other case, it’s ignored (i.e., we act as if solely the header was obtained).
Compact block bulletins. In Bitcoin Core 0.13.0, help for BIP152 compact blocks was added. In compact blocks, blocks are despatched with out their full transaction knowledge, however with brief (48-bit) salted hashes of the person transactions as a substitute. When receiving a compact block, the receiver tries to reconstruct the total block utilizing transactions they’ve in reminiscence, after which requests any lacking ones. The small print are out of scope right here, however compact blocks features a characteristic known as high-bandwidth mode. When enabled, we allow a restricted variety of friends to announce blocks to us straight utilizing a cmpctblock message, skipping a roundtrip utilizing headers/inv and getdata which saves latency. On this case, we once more course of the header embedded within the compact block first, after which course of the compact block if the block appears to be like like it is vitally near our lively chain tip (max 2 blocks forward).
Block validation. When a block is processed, all relevant consensus guidelines are carried out, which incorporates syntactic validity, recomputing the transaction Merkle root, transaction finality, BIP34 (“peak in coinbase”), segwit commitments, and the utmost block weight. If these checks fail, the block is ignored, or in some instances marked as completely invalid, relying on the kind of error. If the checks succeed, the block knowledge is saved on disk. Scripts, double-spending, and inflation can’t be verified but.
getblocks based mostly synchronization. Earlier than the introduction of headers-first synchronization a unique mechanism was used. As a substitute of getheaders to provoke synchronization, the same getblocks message was despatched, to which the peer would reply with an inv containing as much as 500 block hashes, which might then set off getdatas and additional getblocks to proceed. Bitcoin Core now not makes use of this mechanism since model 0.10, however it’s nonetheless supported for friends who need to synchronize this fashion (i.e., Bitcoin Core will reply to getblocks messages, however is not going to ship them).
3. Block activation
Lastly, when we’ve got the blocks, we will resolve what our lively chain tip needs to be, and absolutely validate the blocks concerned.
Lively chain tip choice. The rule is that at each cut-off date, we purpose to make the lively chain tip be:
- Amongst all blocks within the block header tree for which we’ve got the total knowledge and full knowledge for all its ancestors, and which aren’t (but) marked as completely invalid…
- contemplating solely those with the very best cumulative work…
- and amongst these, decide the one for which the total knowledge was obtained first.
Transferring in direction of the choice. If, at any level, the results of this choice shouldn’t be the identical as the present lively chain tip, we begin making progress in direction of the choice. First the blocks which are a part of the lively chain however not within the chosen chain are disconnected in reverse order, after which the blocks within the choice not within the lively chain are linked in ahead order. If disconnections are concerned within the activation course of, it’s known as a reorganization.
Full block validation. Connecting entails performing full validation, of all consensus guidelines, together with script validity, subsidy checks, and checking for double-spends, which want entry to the UTXO set. We solely keep the UTXO set on the lively chain tip, so connecting entails eradicating from the UTXO set any inputs within the block, and including any outputs. Disconnecting does the reverse, eradicating any outputs the disconnected block added, and re-adding any inputs it spent. If validation fails right here, the block is marked as completely invalid, which can probably trigger the choice logic above to vary its concept about what the lively chain tip needs to be.
Saying new chain ideas. At any time when the activation course of stabilizes, friends are despatched an announcement of our new lively chain tip. If just a few blocks have been linked, and the peer helps BIP130, a headers message is shipped with the headers of all linked blocks. In any other case we revert to sending an inv of simply the brand new tip. In case the peer chosen us as a high-bandwidth compact blocks peer, the announcement is shipped earlier than full validation completes in circumstances permitted by BIP152 (once more as a method to cut back latency).
