PartitionAppendQueue

The partition owner's append queue: the EventLog the kernel appends through on a node. Appends to one partition from any number of aggregates queue on the partition's lane, and one coroutine per lane commits what is queued with one EventLog.appendAll (group commit). On MongoDB, whose transactions abort each other when two writers bump one partition's counter, this turns contention into batching (about 100 appends a second with 32 concurrent writers on one partition, about 5.6 thousand queued).

Each caller gets its own stream's outcome, as EventLog.appendAll returns it: a conflict or a duplicate of one stream does not fail the others. The streams of one aggregate that one call hands over (a kernel's batch) queue as one unit and go into one batch together, so the log commits them as a prefix; a unit of an aggregate already in the batch waits for the next one, so appends of one aggregate keep their order. A batch whose EventLog.appendAll throws fails each of its callers with that exception, as a failing EventLog.append would; the lane goes on with the next batch. A caller cancelled while its append is still queued takes it out of the queue: it is never appended. Once taken into a batch, the append completes whatever the caller does.

When the log takes several partitions a call (EventLog.appendsAcrossPartitions), the lanes hand their batches to AppendQueueOptions.mergers committers instead of calling the log themselves: a committer takes the first batch handed and those handed meanwhile, at most AppendQueueOptions.mergeMax streams, and commits them with one call. The store's fixed cost of a call is then shared by the partitions, whose batches are small at moderate load (one or two streams). A lane waits for its batch before it takes the next, so each partition has one batch in flight, as without merging; a merged call that throws fails every caller of its batches.

The queue only serializes the appends of this node: a partition appended to from two nodes (Local fallback during a hand-off) still contends in the database, and the backend's own retry keeps it correct. The kernel stays unaware of it.

Each partition's lane holds at most capacity appends: the node's in-flight bound (me.ahoo.wow.runtime.InFlightLimit), which every append's command is counted in, so the lane never fills on its own; were it full, the caller would wait for room rather than queue without bound.

close stops taking appends and returns once the queued ones are committed.