Introduction

The current document is an extension to Polynomial PoRs for Subspace v2 describing reasoning, concrete choices, and trade-offs we make for Subspace verifiable archiving.

Reading the guide mentioned above first is strongly recommended. It provides the necessary background on mathematical primitives underlying KZG commitments and Reed-Solomon erasure codes.

The current document is structured as follows: first, we introduce several schemes, gradually building up complexity. Each scheme is provided with a high-level step-by-step description and a discussion of its pros and cons. Second, we consider in detail two constructions applicable directly to Subspace archiving. Last, we provide two schemes that we initially considered but later abandoned for the sake of keeping records.

Before and After

Archiving is conceptually a protocol for transforming blockchain history into Archiving History using some algorithm defined for all peers.

Specifically, in Subspace v1, archiving means a process of transforming blocks of the history at configured depth from the tip of the chain as follows:

1. blocks are SCALE-encoded and added to a buffer
2. once the buffer reaches the size of a full segment, it is sliced into records
3. records are erasure-coded
4. Merkle Tree is built over both source and parity records, and a witness is computed for each record
5. records are concatenated with corresponding witnesses to form pieces of Archiving History that farmers later use for plotting and farming

If you are familiar with Subspace v1, consider refreshing your memory of the v1 archiving process.

For Subspace v2, we are looking to replace the Merkle commitment in step 4 with KZG commitments while also adding the ability to prove step 3 was done correctly.

Archiving Constructions Overview

It would be crucial for Subspace archiving to leverage the KZG commitment relation to erasure coding to guarantee that the history pieces were erasure coded correctly and are recoverable on request.

One inherent challenge of using the KZG commitment scheme for Subspace archival is the incommensurability of the underlying data measurement units. Subspace archival, plotting, farming, and the Distributed Storage Network are designed around a notion of a piece as a unit of measurement. KZG, on the other hand, operates over field elements (see here for a more lengthy explanation). In practice, KZG commits to chunks of size up to 254 bits, and Subspace pieces are 1MiB. Every construction presented below aims to solve or circumvent this size disparity differently.

Flat commit-only

The most straightforward way to use the KZG commitment scheme would be to replace all usages of Merkle tree with it:

1. Collect $n$ pieces into a segment.
2. Erasure code them with rate 1/2 and obtain $2n$ pieces.
3. Hash each piece to chunk size.
4. Commit to piece hashes under KZG scheme and obtain segment commitment $C$.