---
url: /operate/performance.md
description: >-
  What Wow's benchmarks measure and on what, the latest reports' headline
  figures, how to read them, what they mean for sizing, and how to reproduce
  one.
---

# Performance

Wow is measured end to end: a load generator sends commands over HTTP to Wow nodes on PostgreSQL or MongoDB, with Kafka, and waits for the stage each scenario names. Each environment keeps one report of each kind, the latest, with its raw results. This page gives their headline figures, generated from the reports, and how to read and use them. For what a node does under load, see [Scaling and Capacity](./scaling); for the choice of store, see [Choose the Stores](./choosing-storage).

## What was measured {#what-was-measured}

The domain is the benchmark's own: accounts, and a transfer between two of them run by a saga ([`test/benchmarks/server`](https://github.com/Ahoo-Wang/Wow/tree/main/test/benchmarks/server)).

| Scenario | What one operation is | Answered when |
| --- | --- | --- |
| E1 write | A deposit to one account | Its commit is stored |
| E2 read your write | A deposit, then a read of the account's state | The read shows the deposit's version |
| E3 saga chain | A transfer prepared on one account | The saga's entry on the other account is stored |
| E4 read | One account's state by id, or a page of ten accounts | The answer |
| E5 cluster | E1 and E3 on three nodes, round-robin, steady and with one node stopped and restarted | As E1 and E3 |
| Mixed | Writes and reads together: half E1, the rest E4 | Each as its scenario |
| Large | E1 with five events of about 2 KB each | As E1 |

The keys are uniform over the accounts, Zipf (a few accounts take most commands), or one hot account. The data sets are 1,000 accounts, which every cache holds; 100,000, each touched once before the measurement; and a million, not touched (a cold cache).

Two environments:

* **GitHub Actions.** Each suite runs on a shared runner of its own, 4 cores and 16 GB, with the node, the store, Kafka and the load generator on it. A comparison runs the head and a released baseline in turn on one runner, so its ratios hold across runners while its absolute figures belong to that runner.
* **An Apple M5 Max** (18 cores, 64 GB). The stores run in containers on the same host as one node or three, and closed-loop sweeps go from 1 to 4,096 clients, further for E1 on one node.

## The figures {#figures}

No report quotes its figures here yet: the [reports](https://github.com/Ahoo-Wang/Wow/tree/main/test/benchmarks/results/reports) hold every measurement.

## How to read them {#how-to-read}

* **A peak** is the best throughput of a closed-loop step that failed at most 1% of its commands. It says what a node can do at all, not at what latency.
* **A sustainable rate** is the highest rate whose p99 stayed within 200 ms with at most 1% errors. It is the figure to size on. The gap between it and the peak is the latency a node trades for throughput.
* **A lower bound**, written `≥`, is a peak where the measurement could not go further: the load generator was the limit, or it was the sweep's last step.
* **One run is not a comparison.** A snapshot's figure is one run, and its difference from an earlier snapshot is given without a conclusion. Whether a change made Wow faster or slower is judged on pairs of the head and a baseline on one runner, against the suite's own run-to-run noise ([`noise.json`](https://github.com/Ahoo-Wang/Wow/blob/main/test/benchmarks/noise.json)).
* **The two environments are not compared with each other**, and neither is a deployment: on both, the store and the load generator share the machine with the nodes. Production sizing needs a run on the target hardware.

## What they mean for sizing {#sizing}

* **The database carries every node.** All the nodes' appends and reads land on one database, so once it is at its limit, neither a faster node nor more nodes raise the ceiling; a faster or less shared database does.
* **The tail sets the usable capacity.** Plan on the sustainable rate. Past it, latency grows faster than throughput, and past the peak a node refuses commands with `wow.command.busy` (429, with `Retry-After`) rather than queue them without end ([Limits a node enforces](./scaling#limits-a-node-enforces)).
* **A hot aggregate is a limit of its own.** One aggregate's commands run one after another on its owner, so the hot-account figures bound what any single aggregate can take, whatever the cluster's size. Spread the load over aggregates where the model allows.
* **More nodes split the partitions.** Throughput grows with nodes while the store keeps up ([Nodes and partitions](./scaling#nodes-and-partitions)). The three-node figures here come from nodes that share one host, or one runner, with the store and the load generator. They show what a cluster costs (forwarding, the hand-over of a stopped node's partitions) rather than what it gains on separate machines.

## Against Axon Framework 4 {#axon}

Before GA, Wow was compared with Axon Framework 4 on one PostgreSQL, both without a message bus, with the same domain, load generator and harness, on GitHub Actions ([actions-axon](https://github.com/Ahoo-Wang/Wow/blob/main/test/benchmarks/results/reports/actions-axon.md)) and on the M5 Max ([m5max-axon](https://github.com/Ahoo-Wang/Wow/blob/main/test/benchmarks/results/reports/m5max-axon.md)). Wow's E3 peak was above Axon's in both environments. Its E1 peak was above Axon's on the M5 Max; on Actions it was level with uniform keys and above with hot keys. The reports give the ratios and the conditions each holds under.

## Reproduce a figure {#reproduce}

On GitHub Actions, the Bench workflow (`.github/workflows/bench.yml`) runs a tier, or named suites, of the head against a baseline release:

```bash
gh workflow run bench.yml --ref main -f tier=L3 -f head_ref=main
```

L1 counts each operation's store round trips, L2 measures single points, L3 the grid of scenarios and data sets, and L4 the sustainable rates, the cliffs, the cluster and the cold cache. Locally, from the repository root, with Docker and a JDK 25:

```bash
test/benchmarks/harness/bench.sh build
test/benchmarks/harness/bench.sh up --backend=postgres --accounts=100000 --prime
test/benchmarks/harness/bench.sh saturate --reuse --backend=postgres --accounts=100000 --scenario=E1 --sweep
test/benchmarks/harness/bench.sh down
```

The [benchmark guide](https://github.com/Ahoo-Wang/Wow/blob/main/test/benchmarks/README.md) explains the harness, its options and every suite. Each report's own Reproduce section gives the exact commands behind its figures.

## The full reports {#reports}

[`test/benchmarks/results/reports/`](https://github.com/Ahoo-Wang/Wow/tree/main/test/benchmarks/results/reports) keeps the latest report of each kind on each environment: a snapshot and the comparison with Axon Framework 4, on GitHub Actions and on the M5 Max. A new report replaces the one before it. Each is a `.md` with a directory of the same name, holding the raw results its figures come from; its [README](https://github.com/Ahoo-Wang/Wow/blob/main/test/benchmarks/results/reports/README.md) lists them.