Running a Prover Network Client: Operational Notes for zkVM Teams in 2025

Short description: In 2025, decentralized prover networks matured from R&D curiosities into production infrastructure. This field guide gives decision‑makers concrete, up‑to‑date operational guidance to run prover clients reliably and cost‑effectively across Boundless (RISC Zero), Succinct’s SP1, and AVS‑based networks, with hardware blueprints, Kubernetes patterns, SLAs, and security controls.

Who this is for

Leaders at startups and enterprises evaluating ZK infrastructure who need precise, actionable notes to scope pilots, design budgets, and set SLAs for running zkVM prover clients.

---

1) The 2025 proving landscape in one page

• RISC Zero Boundless moved from an incentivized testnet into an active proof marketplace, with Proof of Verifiable Work (PoVW), staking-based collateral, and on‑chain settlement on Base/Ethereum. Provers stake ZKC, bid on jobs, and get slashed for missed deadlines; proofs settle as tiny receipts on host chains. (blockworks.co)
• Succinct’s SP1 zkVM hit real‑time Ethereum L1 block proving using 16 RTX 5090 GPUs (99.7% of blocks <12s). SP1 also shipped security hardening (“Turbo”), formally addressed vulnerabilities, and froze old verifiers on mainnets. (blog.succinct.xyz)
• StarkWare deployed S‑two (next‑gen STARK prover) on Starknet mainnet, replacing Stone and enabling much lower latencies—and setting the stage for decentralized proving. (starknet.io)
• AVS‑based prover markets emerged: Lagrange’s Prover Network runs on EigenLayer with >20+ institutional operators and integrations targeting ZK Stack provers (ZKsync) and coprocessors. ZKsync also piloted decentralized proving via Fermah’s universal proof market and AVSs. (lagrange.dev)
• Proof aggregation and cross‑stack interop advanced: Polygon’s AggLayer shipped “pessimistic proofs” on mainnet and uses SP1 to generate ZK proofs for its unified bridge model—relevant if your workloads aggregate or bridge proofs across stacks. (polygon.technology)
• Cost controls on verification: zkVerify (Horizen Labs) launched as a dedicated verification chain claiming 90%+ lower verification costs versus L1s, useful for large proof volumes. (prnewswire.com)

Bottom line: In 2025, you can realistically outsource proving to a decentralized market, bring your own GPUs/FPGAs to earn, or mix both—while pushing verification to a specialized chain to slash fees.

---

2) Choose your network(s) and client role

Most teams will operate in one of three roles. Pick based on your control/compliance needs and latency SLOs.

1. Market Prover (bring compute, earn rewards)

• Boundless: run Bento (their proving stack), stake ZKC on Ethereum, accept bids on Base, and publish work logs per epoch. Miss a deadline and 50% of the posted ZKC collateral burns, 50% becomes bounty. Rewards scale with stake and delivered “verifiable work.” (docs.beboundless.xyz)
• SP1 Network: enroll as an approved prover during the Stage 2.x testnet phases; production onboarding focuses on professional operators with datacenter GPUs (plus upcoming FPGA lanes). (blog.succinct.xyz)

1. Requester (buy proofs, integrate receipts)

• Boundless: submit jobs to the market, get a SNARK/STARK receipt verified on your chain (Base/Ethereum first), pay per job; Blockworks reported typical jobs targeted “< $30” for heavy zkVM compute during testnet. (blockworks.co)
• SP1: request proofs via the network explorer/API; real‑time block‑level proving is feasible with well‑tuned clusters. (blog.succinct.xyz)

1. AVS consumer (offload to restaked operators)

• Lagrange (EigenLayer AVS): decentralized prover network with institutional operators; used to decentralize ZK Stack provers or run coprocessor workloads. ZKsync’s roadmap calls for routing a large portion of proving to such external operators. (lagrange.dev)

If verification dominates your costs, plug in zkVerify and post attestations to multiple L1/L2s only as needed. (prnewswire.com)

---

3) Hardware blueprints that actually work in 2025

Use the right accelerators for your proof system and latency targets. Current “sweet spots”:

• Real‑time SP1 proving (Ethereum L1 blocks): 16× RTX 5090 (32 GB GDDR7 each) suffices for 99.7% blocks under 12s in Succinct’s benchmark. Useful p90/p95 latency SLOs for exchanges and bridges. (blog.succinct.xyz)
  • RTX 5090 key specs: 32 GB GDDR7, 21,760 CUDA cores, ~1.79 TB/s bandwidth, 575 W TGP. Plan for 1 kW PSU headroom per card in dense rigs. (techpowerup.com)
• High‑utilization datacenter nodes: H200 (141 GB HBM3e, 4.8 TB/s) for memory‑bound provers (large witnesses, FRI polynomials). MIG partitions help share GPUs across queues. (nvidia.com)
• FPGA lanes for SP1: AMD Alveo U55C via AntChain OpenLabs yields 15–20× over CPU on SP1 v4.1.0 programs; expect these to slot into the Succinct Prover Network for lower cost/latency classes. (blog.succinct.xyz)

Reference host specs we see working in production pilots:

• CPU: 32–64 cores (modern Xeon/EPYC), 256–512 GB RAM for heavy witness handling and multi‑GPU orchestration.
• Storage: 2–4× NVMe Gen4/5 in RAID0 (6–14 GB/s) for trace/witness temp; keep SRS and verifier keys on separate NVMe to avoid contention.
• Network: 25–100 GbE if you’re fetching state/witnesses at scale; low jitter matters for sub‑10s SLOs.
• Power & cooling: a 16×5090 rack can draw ~9–12 kW—budget for structured power and hot aisle containment.

Tip: For mixed fleets (H200 + 5090), schedule memory‑heavy circuits on H200 and hash/NTT‑heavy shards on gaming GPUs where feasible.

---

4) Kubernetes patterns for provers

Kubernetes is the easiest way to scale multiple provers, shard queues, and share big GPUs safely.

• Use NVIDIA’s device plugin (and GPU Operator) with MIG for H100/H200 to carve GPUs into right‑sized instances. Request resources like

  nvidia.com/mig-3g.40gb

  per Pod. (docs.nvidia.com)
• For non‑MIG GPUs (e.g., RTX 5090), enable time‑slicing to multiplex multiple proving tasks per card when latency budgets allow. (docs.nvidia.com)
• Affinity and anti‑affinity: spread recursive shards across fault domains (power phases, TORs). Pin Pods to GPU UUIDs when you must co‑locate recursive layers tightly; otherwise, keep scheduling flexible. (github.com)

Example DaemonSet snippet to expose MIG and stable UUIDs:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: nvidia-device-plugin-daemonset
spec:
  template:
    spec:
      containers:
      - name: nvidia-device-plugin
        image: nvcr.io/nvidia/k8s-device-plugin:stable
        args: ["--mig-strategy=mixed", "--device-id-strategy=uuid"]
        env:
        - name: FAIL_ON_INIT_ERROR
          value: "true"
        - name: PASS_DEVICE_SPECS
          value: "false"
        securityContext:
          allowPrivilegeEscalation: false

Docs and profiles: H100 MIG profiles, K8s GPU scheduling. (docs.nvidia.com)

---

5) Boundless (RISC Zero): what running a prover actually looks like

What to expect operationally:

• Stake and collateral:
  • Stake ZKC on Ethereum to unlock PoVW mining rewards.
  • Lock ZKC ~10× the job’s max fee as proving collateral on Base; miss the deadline and 50% burns, 50% becomes bounty. (docs.boundless.network)
• Software stack: Bento (multi‑tenant proving cluster) + RISC Zero toolchain on CUDA nodes. (docs.beboundless.xyz)
• Epoch cycle: submit aggregated “work logs” per epoch and claim rewards; rewards scale with stake and delivered compute. (docs.boundless.network)
• CLI essentials:

# Tooling
rzup install risc0-groth16

# Enable ZK mining in Bento
export REWARD_ADDRESS=0xYourRewardsWallet

# Stake on Ethereum (via official portal), then run Bento workers
bento start --cluster <name> --gpus all

# Periodically claim rewards after epoch finalization
boundless-cli claim-mining-rewards

(docs.boundless.network)

• Ethereum block proving with Zeth: Zeth proves L1 blocks inside the RISC Zero zkVM using Reth’s stateless execution; you’ll need an archival RPC to fetch block witnesses. Cache aggressively. (github.com)

SLA guidance: start with “job acceptance to proof settle” p95 ≤ 30s on multi‑GPU nodes for typical workloads; tighter if you specialize (e.g., Merkle/FRI offload pipelines).

---

6) Succinct SP1: hitting real‑time and staying secure

• Latency: SP1 Hypercube proved 99.7% of Ethereum L1 blocks under 12s on 16× RTX 5090. Target p95 ≤ 12s in pilots; budget a few outliers for “mega‑blocks.” (blog.succinct.xyz)
• Security: upgrade paths matter. Succinct disclosed and patched SP1 v3/Plonky3 issues; production use should be on SP1 Turbo+ and pinned verifiers (routers to older contracts were frozen). Require version attestation in your CI before admitting jobs. (blog.succinct.xyz)
• Hardware acceleration roadmap: FPGA (AMD Alveo U55C) lanes deliver 15–20× CPU speedups; expect network operators to add FPGA queues where latency/cost justify. (blog.succinct.xyz)
• Operational tip: isolate recursion layers into separate K8s queues and co‑locate with high‑bandwidth NVMe; use job annotations like

  recursion.depth

  to route to MIG slices with adequate memory.

Joining the network: Stage 2.x testnet prioritized professional operators; watch the explorer for onboarding windows and benchmarks. (blog.succinct.xyz)

---

7) AVS‑based networks (Lagrange, Fermah): decentralize proving via restaking

Why enterprises like this: you inherit operator diversity, slashing conditions, and baseline SLOs without contracting many vendors.

• Lagrange ZK Prover Network: launched on EigenLayer mainnet with Coinbase, OKX, Kraken’s Staked, Nethermind and more running provers; used to decentralize ZK Stack provers and zk coprocessor workloads. (prnewswire.com)
• ZKsync 2025 direction: route a large share of proving to external AVSs (Lagrange/Fermah), integrate EigenDA as DA. If you build on ZK Stack, plan for a mixed proving topology. (messari.io)
• Fermah: universal proof market backed by GPUs/FPGAs; seed round closed in late 2024; integrated by ZKsync for decentralized proof generation. (orrick.com)

Vendor diligence: require public slash records, attested hardware configs, and auditable queue fairness policies.

---

8) Data plumbing: SRS, witnesses, and caches are your P0 bottlenecks

• SRS sizes grow fast. For KZG ceremonies, SRS can be multi‑GB; Halo2 examples show SRS needs up to 2^25–2^26 for large circuits (tens to hundreds of GB RAM used by provers). Keep SRS on dedicated NVMe and pin versions with checksums. (zkresear.ch)
• Ethereum witnesses for block proving: Zeth requires archival RPC and pulls execution witnesses; add a prefetcher with content‑addressed cache (e.g., local RocksDB + S3 fallback) to avoid cold‑start misses. (github.com)
• Prover pipelines: pre‑proving and input prep are often the long pole; external teams reported cutting pre‑prove fetching/MPT building from minutes to seconds with optimized pipelines—plan for a prefetch stage per job class. (hozk.io)

---

9) Verification costs and where to put them

• If you verify on Ethereum, zk proof verification can cost up to hundreds of thousands of gas during congestion; zkVerify claims 90%+ reduction by offloading verification to its purpose‑built chain, then attesting to your target L1/L2s. Consider this for high‑volume applications. (prnewswire.com)

Example routing:

• Prove on Boundless/SP1 → Verify on zkVerify → Post attestations to Ethereum/Base/Arbitrum only when settlement is needed.

---

10) Security and compliance guardrails

• Version pinning and attestation:
  • SP1: enforce Turbo+; block V2/V3 provers; verify recursion circuits include completeness checks (Turbo patch). (blog.succinct.xyz)
  • Boundless: enforce minimum Bento build hash in your fleet; require deterministic guest builds (RISC Zero images) to avoid image drift. (github.com)
• Economic risk: understand slashing math on Boundless (≈10× collateral, 50% burn on miss). Treat wallet ops as production keys; separate staking and reward wallets. (docs.boundless.network)
• Isolation: MIG for H100/H200 or dedicated bare‑metal partitions for 5090s handling sensitive inputs. Use time‑slicing only for non‑sensitive jobs. (docs.nvidia.com)
• Formal verification posture: watch SP1’s ongoing formal work (e.g., Lean proof components) when moving compliance‑critical logic into the zkVM. (hozk.io)

---

11) Practical deployment examples

A) Boundless Bento on a 4×GPU box (Ubuntu 22.04, CUDA 12, RTX 5090)

# 1) Drivers & CUDA (omit here for brevity). Verify nvidia-smi shows 4x RTX 5090.
nvidia-smi

# 2) Install toolchains
curl -L https://rzup.risczero.com/install.sh | bash
rzup install risc0-groth16

# 3) Install Bento and set rewards wallet
export REWARD_ADDRESS=0xYourRewardsWallet
bento init --cluster=prod-a
bento start --cluster=prod-a --gpus all

# 4) Stake ZKC (once) on Ethereum via the official staking portal
# 5) Periodically submit work logs and claim rewards after epoch finalization
boundless-cli submit-worklog
boundless-cli claim-mining-rewards

Operational notes: post at least one aggregated work proof each epoch; monitor on‑chain job deltas; keep Base RPC quotas high to avoid settlement delays. (docs.boundless.network)

B) SP1 prover queue with MIG on H200 nodes (K8s)

apiVersion: batch/v1
kind: Job
metadata:
  name: sp1-core-shard
spec:
  template:
    spec:
      restartPolicy: OnFailure
      containers:
      - name: prover
        image: ghcr.io/succinctlabs/sp1-prover:turbo
        resources:
          limits:
            nvidia.com/mig-3g.40gb: "1"
        env:
        - name: SP1_MODE
          value: "hypercube"
        - name: SP1_PROVER_VERSION
          value: "4.1.0-turbo"
        - name: CACHE_DIR
          value: "/mnt/srs-cache"
        volumeMounts:
        - mountPath: /mnt/srs-cache
          name: srs
      volumes:
      - name: srs
        emptyDir: {}

Target p95 <12s only on high‑end 5090 clusters; for MIG’d H200, map shard sizes to MIG profiles and watch memory pressure. (blog.succinct.xyz)

C) Verifying off‑L1 to cut costs (zkVerify sidecar)

• Post proofs to zkVerify, retrieve verification result, then attest to target chains. Budget 90%+ savings when moving from direct L1 verification. (prnewswire.com)

---

12) SLAs and dashboards that matter

Track these, per queue:

• Ingest → proof receipt latency (p50/p95/p99), by job class (block proving vs. custom zkVM)
• Prover throughput in kHz (SP1 uses kHz metrics in FPGA studies) and cycles/sec per GPU/FPGA to spot regressions. (blog.succinct.xyz)
• Witness cache hit rates; SRS cache I/O latency; NVMe write amplification
• On‑chain settlement latency and failure modes (RPC timeouts, gas spikes)
• Economic health: posted collateral at risk (Boundless), slashing events, realized rewards, and GPU utilization

---

13) Cost model (worked example)

• Real‑time SP1 proving for L1 Ethereum blocks (16× 5090 cluster):
  • Capex: 16× 5090 at ~$2,000 each = $32k + hosts/rack/power. Opex: 9–12 kW continuous, plus datacenter fees. Throughput: p95 <12s for 99.7% of blocks in Succinct tests. (blog.succinct.xyz)
• Boundless proof jobs: marketplace pricing varied in testnet; public guidance suggested sub‑$30 for heavy zkVM tasks; expect tighter spreads as more GPUs join and PoVW stabilizes. (blockworks.co)
• Verification: move repeat verifications to zkVerify for 90%+ savings; reserve L1 verification for checkpoints. (prnewswire.com)

Tip: treat pre‑proving and data fetch as first‑class cost centers; a well‑built witness cache frequently “saves” more than a GPU upgrade.

---

14) Two 30‑day pilots we recommend

• Pilot A (market prover): Stand up Bento on a 4× 5090 node, stake a small ZKC amount, and accept low‑risk jobs to benchmark PoVW rewards vs. power costs. Add a second node and measure horizontal scaling. SLA: accept→settle p95 ≤ 45s. (docs.boundless.network)
• Pilot B (real‑time requester): Use SP1 network to prove live Ethereum blocks for an internal dashboard; aim for p95 <12s on a tuned 16× 5090 cluster or purchase capacity from a network operator; add zkVerify for verification cost reduction. (blog.succinct.xyz)

If you build on ZK Stack: test Lagrange/AVS routing for a subset of your chains to quantify decentralization benefits and operator diversity. (lagrange.dev)

---

15) Decision checklist for CTOs

• Governance and risk
  • Which networks’ slashing/penalty models can we accept (Boundless 50% burn on miss; AVS slashing via EigenLayer), and who signs the keys? (docs.boundless.network)
  • Do we require formal guarantees (SP1 Turbo and ongoing formal work), or is audit+testing sufficient? (blog.succinct.xyz)
• Latency budget
  • Target p95 <12s? Budget 16× 5090 or buy capacity; otherwise accept 30–120s windows and use cheaper fleets. (blog.succinct.xyz)
• Data plane
  • Do we control archival RPCs and witness caches (needed for Zeth/stateless execution) or rely on third‑parties? (github.com)
• Verification spend
  • Keep verification on L1 or shift to zkVerify for bulk? Model 90% savings scenarios. (prnewswire.com)
• Scale and portability
  • Use MIG/time‑slicing for mixed queues; keep SP1/Boundless images pinned and attest builds; separate staking vs. reward wallets. (docs.nvidia.com)

---

Appendix: Notes by ecosystem

• Starknet: S‑two is live on mainnet and materially faster than Stone; roadmap includes decentralized proving and validators voting on blocks by end‑2025—plan for client diversity. (starknet.io)
• Polygon AggLayer: pessimistic proofs on mainnet; SP1 is used in the pipeline; if you bridge across CDK/Sovereign stacks, align on proof formats and custody of aggregation keys. (polygon.technology)
• ZKsync: decentralized proving via Fermah/Lagrange and optional EigenDA; if you run a ZK Stack chain, prepare to route proving to external AVSs. (messari.io)

---

The takeaways

• Prover networks are production‑ready in 2025. You can earn with your GPUs/FPGAs, buy proofs with concrete SLAs, or do both. (blockworks.co)
• Hardware matters, but pipelines matter more. Invest in witness/SRS caching and Kubernetes scheduling before buying more GPUs. (github.com)
• Security is active work. Pin versions (SP1 Turbo), demand on‑chain slashing transparency, and audit your build determinism. (blog.succinct.xyz)

7Block Labs can help you blueprint these pilots, harden your GPU fleets, and connect you with trusted operators across Boundless, SP1, and AVS networks.