JSON-RPC: chainHead_v1_follow Subscription Sends newBlock events with unannounced parentBlockHash

[josepot]

The chainHead_v1_follow subscription is not compliant with the specification because it sometimes sends newBlock events where the parentBlockHash has not been previously announced. According to the specification:

parentBlockHash is guaranteed to be equal either to the current finalized block hash, or to a block reported in an earlier newBlock event.

Issue Details

While testing against the endpoint wss://sys.dotters.network/paseo, we observed that some blocks are missing from the newBlock events sequence. Specifically:

1. Block #3012893 with hash 0x2b8cbe6887a3e2869d462920d784b604a4683751aea1f2e10f02f97c9333abae is announced.
2. The next announced block is #3012896 with hash 0x0e95ce16ed5cbf2008bf00c5e1a65736a8cd00c6dcf08da17126d216f79605d9.

The two missing blocks are:

• Block #3012894 with hash 0x5c7ed91698a589a1b2ec120b6bea82db88ef0ecc4abc7c25d491d9a10b165841
• Block #3012895 with hash 0x84654e06c8a6d4f0793b377c84cd80d5e802e227636a6b7bc52f1088863447f7

As a result, the parentBlockHash of block #3012896 refers to an unannounced block, which violates the specification's guarantee.

This issue causes the PAPI client to crash.

Logs

I've attached the logs capturing this behavior: pre.log

Steps to Reproduce

1. Install Bun:

   curl -fsSL https://bun.sh/install | bash

2. Set up the project:

   mkdir paseo-test
   cd paseo-test
   bun init -y
   bun install polkadot-api
   bun papi add -w wss://sys.dotters.network/paseo pas

3. Create a file named paseo.ts with the following content:

   import { openSync, writeSync } from "node:fs";
   import { createClient } from "polkadot-api";
   import { getWsProvider } from "polkadot-api/ws-provider/web";
   import { pas } from "@polkadot-api/descriptors";
   import { withLogsRecorder } from "polkadot-api/logs-provider";
   import { fixUnorderedBlocks, parsed } from "polkadot-api/polkadot-sdk-compat";
   
   const pre = openSync("pre-enhancer.log", "w");
   const post = openSync("post-enhancer.log", "w");
   const rpcEnhancer = parsed(fixUnorderedBlocks);
   
   const client = createClient(
     withLogsRecorder(
       (log) => {
         writeSync(post, `${log}\n`);
       },
       rpcEnhancer(
         withLogsRecorder((log) => {
           writeSync(pre, `${log}\n`);
         }, getWsProvider("wss://sys.dotters.network/paseo")),
       ),
     ),
   );
   
   const api = client.getTypedApi(pas);
   
   api.query.System.Account.watchValue(
     "15roJ4ZrgrZam5BQWJgiGHpgp7ShFQBRNLq6qUfiNqXDZjMK",
   ).subscribe(
     (x) => {
       console.log("Account change", x.data);
     },
     console.error,
   );
   
   client.finalizedBlock$.subscribe(console.log, console.error);

4. Run the script:

   bun run paseo.ts

Impact

This issue disrupts the integrity of the block sequence received via the subscription, making it challenging to process data accurately. It hinders our ability to maintain a consistent view of the blockchain state, as we rely on the guarantee provided by the specification regarding parentBlockHash.

Expected Behavior

Each newBlock event's parentBlockHash should either be:

• Equal to the current finalized block hash, or
• A block hash that was reported in an earlier newBlock event.

Actual Behavior

The newBlock event for block #3012896 references a parentBlockHash that was neither finalized nor previously announced via a newBlock event.

Request

Please investigate why the chainHead_v1_follow subscription is emitting newBlock events with parentBlockHash values that have not been previously announced.

cc: @jsdw @lexnv

[josepot]

I was able to reproduce the same issue against wss://westend-rpc.polkadot.io. The logs:
pre-enhancer.log