GraphQL Gateway Benchmarks

Performance benchmarks comparing GraphQL gateway implementations across Apollo Federation and GraphQL Federation.

Latest Results: Real-World | Burst | Synthetic

Test categories

Each benchmark runs a gateway against 4 subgraphs (Accounts, Inventory, Products, Reviews) and executes a heavy nested query. There are three test categories, each designed to measure different aspects of gateway performance:

• Real-World (constant-latency) — Shows how a gateway performs in real-world scenarios where downstream services make database calls and perform other I/O-bound work. Every downstream call adds 4 ms of latency, and the test runs with 50 concurrent VUs for 120 seconds.

• Burst (burst) — Shows how a gateway responds to rapidly increasing traffic. The benchmark holds a base load of 50 VUs for 10 seconds, ramps to 500 VUs over 40 seconds, and then ramps back to 50 VUs over 10 seconds.

• Synthetic (constant) — An artificial lab experiment that measures a gateway's theoretical throughput when downstream services make no database calls, perform no other I/O-bound work, and add no latency. This isolates gateway overhead from downstream latency. The test runs with 50 concurrent VUs for 120 seconds.

Benchmark setup

All benchmarks run against 4 HotChocolate subgraphs in both benchmark families (apollo-federation/* and composite-schema/*). The subgraphs are pinned to HotChocolate 16.6.8 (each subgraphs-net/build.sh verifies the pin and never rewrites package versions).

The committed result tables under benchmark-runs/ and the root RESULTS_*.md files are historical measurements: the constant and burst tables still contain Rust-subgraph (async-graphql + axum) rows from before the move to HotChocolate subgraphs. Those rows are preserved as measured and are not relabeled.

Gateways

Apollo Federation

• Apollo Gateway (Node.js)
• Apollo Router (Rust)
• Cosmo (Go)
• Grafbase (Rust)
• Hive Gateway (Node.js)
• Hive Gateway Router Runtime (Node.js)
• Hive Router (Rust)

GraphQL Federation

• Fusion (.NET) — benchmarked as latest stable (fusion, built and run on the .NET 10 SDK), plus, when a HotChocolate preview newer than the latest stable exists, the latest preview on two runtimes: fusion-nightly (built and run on the .NET 10 SDK) and fusion-nightly-net11 (built and run on the latest .NET 11 preview SDK bundled into its prebuilt artifact, with runtime-async). Running the same preview packages on both runtimes isolates framework effects from library changes.
• feddi (JVM) — built from source and bundled with a JDK 25 (feddi-dev/feddi-gateway). See its README for setup and a known heavy-query planner limitation.

Running benchmarks locally

Prerequisites

• k6
• jq - required by the monitor script
• python3 - used for JSON parsing
• curl - used for health checks
• bash, taskset (optional, Linux only for pinning)
• Gateway dependencies are handled by each gateway's install.sh

Run

./k6/benchmark.sh <gateway-path> [subgraphs-dir] [constant|constant-latency|burst|ramping]

Examples:

# composite-schema + HotChocolate subgraphs (default)
./k6/benchmark.sh composite-schema/gateways/fusion

# apollo-federation + HotChocolate subgraphs (default)
./k6/benchmark.sh apollo-federation/gateways/apollo-router

# burst mode
./k6/benchmark.sh apollo-federation/gateways/cosmo burst

# constant-latency mode (HotChocolate subgraphs with simulated 4ms IO delay)
./k6/benchmark.sh apollo-federation/gateways/grafbase constant-latency

What happens

1. Resolves subgraph directory (subgraphs-net)
2. Builds subgraphs (subgraphs-net/build.sh, which bundles the .NET SDK into the subgraphs directory so the prebuilt artifact carries its own runtime) unless USE_PREBUILT_SUBGRAPHS=1
3. Installs/builds gateway (gateway/install.sh) unless USE_PREBUILT_GATEWAY=1
4. Starts subgraphs and gateway (optionally CPU-pinned)
5. Executes one full-duration warmup run (discarded)
6. Executes measured runs (BENCH_RUNS, default 9)
7. Captures monitor + k6 summaries for each run
8. Prints run statistics (median/mean/CV and CPU usage medians)

Configuration

Environment variables:

Variable	Default	Description
MEASURE_SECONDS	120 (constant) / 60 (burst)	Benchmark measurement duration
BENCH_RUNS	9	Measured runs (plus one warmup run)
BENCH_VUS	50 constant / 500 burst	VU target passed to k6/k6.js
BENCH_DISPLAY_NAME	(auto)	Override report display name
BENCH_GATEWAY_NAME	(auto)	Override gateway identity in result metadata (used to split stable vs preview rows)
BENCH_GATEWAY_CHANNEL	stable	HotChocolate package channel for fusion's install.sh (stable = latest stable packages; preview = latest preview/nightly packages)
BENCH_GATEWAY_DOTNET	10	.NET runtime/target for fusion's install.sh (10 = bundled .NET 10 SDK, targets net10.0; 11 = bundled .NET 11 preview SDK, targets net11.0 with runtime-async)
BENCH_SUBGRAPH_TECH	(auto)	Subgraph tech label recorded in result metadata (.net)
USE_PREBUILT_SUBGRAPHS	0	Skip build.sh and expect prebuilt subgraph artifact (with bundled .NET runtime and apphosts; fails on benchmark machines without it — no local rebuild fallback)
USE_PREBUILT_GATEWAY	0	Skip install.sh and expect prebuilt gateway artifact

CPU pinning

On Linux with taskset, profiles.json is used to pin:

• system
• k6
• gateway
• subgraphs

Profile selection is based on logical core count (8 or 16 profile).

Output

Each run produces a timestamped results directory:

<gateway-dir>/results/<timestamp>/
  data.csv          # monitor time series
  metadata.json     # gateway/subgraph/machine metadata
  memory.json       # idle/peak/end RSS + avg core usage (k6/subgraphs)
  k6_summary.json   # full k6 metrics (machine-readable)
  k6_summary.txt    # k6 summary (human-readable)

GitHub Actions workflow

Workflow: benchmark.yml

• Triggers:
  • workflow_dispatch
  • scheduled daily at 21:00 UTC
• Dynamic matrix (k6/build-matrix.py):
  • Builds gateway matrix from selected input
  • Each gateway runs against the HotChocolate subgraphs (subgraphs-net)
• Build stages on ubuntu-latest:
  • build-subgraphs (prebuilt artifacts, each bundling its own .NET runtime)
  • build-gateways (prebuilt artifacts)
• Benchmark stages on the dedicated benchmark-runners runner group:
  • benchmark-constant-latency (4ms simulated IO delay)
  • benchmark-constant (runs after constant-latency completes)
  • benchmark-burst (runs after constant completes)
• Result generation:
  • generate-results-constant-latency
  • generate-results-constant
  • generate-results-burst
  • final benchmark-summary job publishes all result docs in the run summary
• Result documents are committed back to the branch by the generator jobs

Acknowledgments

This benchmark repository is based on the work by The Guild on graphql-gateways-benchmark.