Skip to content

README.md

Latest commit

 

History

History
88 lines (61 loc) · 3.54 KB

README.md

File metadata and controls

88 lines (61 loc) · 3.54 KB
Raw

SP1 Cluster

Docker CI Lint CI Telegram Chat

The official multi-GPU proving service implementation for SP1, designed for the cloud and bare metal deployment.

Disclaimer. Use of SP1 Cluster is also subject to the disclaimers and limitations of liability described in the Legal / Disclaimer page.

Overview

This project implements a high performance SP1 proving cluster that can scale to thousands of GPUs. The system consists of several key components:

  • API: REST API for interfacing with the cluster backed by Postgres
  • Coordinator: Central service that manages task and worker coordination
  • CPU/GPU Node: Distributed worker nodes that execute proof generation tasks
  • Fulfiller: Service that syncs the cluster API with assigned proofs from the SP1 Prover Network
  • Bidder: Service that bids on proofs from the SP1 Prover Network
  • Network Gateway: SDK-compatible gRPC/HTTP gateway that terminates the sp1-sdk network contract against a self-hosted cluster (see bin/network-gateway/README.md)
  • Artifact Store: Redis or S3 store used for storing ephemeral proving data

For detailed information on the architecture and deployment instructions, see the docs.

Development

To get started, you will need to have the following installed:

git clone https://github.com/succinctlabs/sp1-cluster.git
cd sp1-cluster

# Build
cargo build --release

Benchmarking

Start gpu0. Compose also starts its API, coordinator, Redis, and Postgres dependencies:

docker compose -f infra/docker-compose.yml up -d gpu0

The CLI connects to the API service, not the coordinator. It must use the workers' artifact store:

export CLI_CLUSTER_RPC=http://127.0.0.1:50051
export CLI_REDIS_NODES=redis://:redispassword@127.0.0.1:6379/0

# One 20M-cycle Fibonacci proof.
./target/release/sp1-cluster-cli bench fibonacci 20

# Four at once, for aggregate cluster throughput.
./target/release/sp1-cluster-cli bench fibonacci 20 --count 4

Inside the Compose network, use service names: CLI_CLUSTER_RPC=http://api:50051 and CLI_REDIS_NODES=redis://:redispassword@redis:6379/0.

Deploying

To deploy to a Kubernetes cluster, you can use the provided Helm chart in the infra/charts/sp1-cluster directory.

cd infra/charts/sp1-cluster
cp values-example.yaml values-testnet.yaml

Edit the values-testnet.yaml file to configure your cluster.

helm install sp1-cluster . -f values-testnet.yaml