Status quo

Right now for farming there are two implementations that are vertically integrated to varying degrees:

• subspace-farmer integrates farming, plotting, caching and serving data over the network
• Space Acres further vertically integrates node into the same package

While ^ were necessary to start and for ease of use, many larger farmers have a hard time scaling that architecture. The proposal is to introduce more specialization on farming side that will help to utilize resources more efficiently.

High-level cluster architecture

On high-level I propose to form a cluster consisting of following components:

• Controller that connects to the node and acts as a gateway for DSN
• Farmer that does farming and is able to extract pieces from plots if necessary, but don’t plot themselves
• Plotter that exposes plotting functionality to farmers, but don’t farm themselves
• Cache that provides caching functionality without any plotting or farming happening at all and can even use HDD

For convenience these roles might be combined if implementation of that isn’t too challenging (for example, farmers can also do plotting, for themselves or other farmers in case they are idling).

It will also be (at least eventually) to have redundancy in these components, such that it would be possible to do upgrades of components without significant downtime.

CLI would look something like subspace-farmer cluster <ROLE> <role-specific args...>.

This is mostly designed to work with LAN, but can technically work over public Internet as well.

Controller

Controller connects to the node via RPC and DSN. It handles notifications from node and orchestrates other components. It will send slot notifications to farmers, store and retrieve pieces from caches on requests from DSN, etc.

Initially there will be just one controller, but eventually there could be multiple controllers with leader election to reduce potential downtime.

Due to its features, controller may need modest amount of RAM depending on amount of space pledged for farming and caches and modest CPU will be sufficient. Disk requirements will be negligible and likely limited to network identity and list of known peers for networking purposes with infrequent writes. Network performance will be the most critical resource for controller.

Farmer