Benchmark¶

Two measurements against the shipping 2.0 code: what the Magazine 1 → 2 dependency change does, and where time goes inside the consume path.

Method¶

Every point is 5 forked JVMs at each of 1, 4, 8, 16, 32 and 50 consumers, 10,000 messages per run, 32 shards, 256-byte payloads. Each run prefills the queue, then drains it; publishing is never inside the measurement. Variant order alternates between forks, and all variants share one container.

Two suites sit on the same harness:

• Dependency comparison — one worker per Magazine version. Reported as means.
• Consume path — ignisMQ's real MagazineQueue and MagazineConsumerTask. Reported as medians with the min-max fork spread.

They use different throughput definitions, so a msg/s figure from one is not comparable with the other.

attempts counts client-side calls through a proxy on the Aerospike client interface. Retries inside the driver are invisible to it, and a batch read counts as one attempt however many keys it carries.

Magazine 1 → 2¶

All 60 runs delivered 10,000 unique messages with zero duplicates and zero errors.

• Throughput improves 59–75% everywhere, and the gap widens rather than closes as consumers are added.
• The durable result is the flat line on the right. Magazine 2 holds at 4.00 Aerospike calls per message across a fifty-fold change in consumer count. Magazine 1 climbs to 7.1.
• That rise is contention. Magazine 1's read-then-increment fire path lets competing consumers act on a stale pointer read and retry a missing payload, so adding consumers adds wasted reads. Magazine 2's guarded increment claims only an available pointer.
• A throughput number is a property of this laptop. An attempt count that does not move with consumer count is a property of the algorithm — that is the part that transfers to production.

Tail latency falls by roughly a factor of three, and Magazine 1's p99 is already flattening near 60 ms by 16 consumers.

What the saving is made of¶

The ~2.0 attempts saved per message is not all Magazine:

The fire-timestamp column is exactly 1.000 everywhere — Magazine 1 records 10,000 put calls per run, Magazine 2 records zero. It is one write per message, so removing it saves one attempt per message however many consumers there are.

Everything that grows with consumer count is Magazine's. delete is exactly 1.000 in both, and operate is 2.00 in both to within 0.01. The whole difference sits in get: Magazine 1 rises from 2.05 per message to a peak of 3.10 at 32 consumers, while Magazine 2 holds at 1.00.

So at one consumer the saving is half fire-timestamp and half Magazine — and all of the additional saving as consumers scale is Magazine.

Inside the consume path¶

This half drives the real shipping classes — real metrics, real handler executor, real scheduler.

All 30 runs delivered 10,000 unique messages, with no sidelining, nothing unconsumed and no saturation refusals.

• Scaling is close to linear to 32 consumers, then flattens — 32 → 50 buys about 6%. That knee is the ten-core host saturating, not the queue: attempts per delivery move only from 4.007 to 4.013, so the extra consumers are waiting, not working harder.
• The handler is a rounding error. The green sliver in the right-hand chart is the handler; the rest is overwhelmingly Aerospike I/O. At 50 consumers the handler is 138 µs of a 3,658 µs path.

The four suspected costs¶

Three of the four were suspected of justifying a redesign. The chart settles it — two sit three orders of magnitude below the path they were supposed to be slowing.

The delete is the one worth acting on¶

Driving Magazine directly, single-threaded, timing fire() and delete() around each other:

The share is stable to under half a percentage point across forks even though the absolute microseconds move by 50% — the signature of a ratio between two costs that scale together.

Cross-checked by a second route: the drain scenario reports 4.007–4.013 attempts per delivery through the real consumer, this one reports 4.004 driving Magazine directly. Two independent instruments agreeing to within 0.01 attempts is the evidence that neither is miscounting.

A quarter of the storage cost of consuming a message is spent retiring it, one round trip at a time. The fix is a batched delete, and it has since been built and measured. Magazine 2.0 exposes deleteAll(Collection<MagazineData<T>>) — one round trip for the whole batch on the Aerospike backend — and the consumer now retires an accepted batch through it instead of deleting message by message. No delay is involved: the deletes were already issued back-to-back immediately after the handler returned for the whole batch, so collapsing them into a single call removes round trips from a burst that is already happening. The delete batch is the delivery batch; nothing accumulates across deliveries.

What batching the delete actually bought¶

Same instrument, same machine, re-run after the change. First, driving Magazine directly, varying only how the fired records are retired:

The attempt counts land exactly where arithmetic says they must: 3.004 + 1/N, because fire costs three calls and the batch adds one delete call per N messages. fire itself stays flat at 2,535–2,559 µs across all four, which is the control — only the retirement changed. At a batch of 10 the delete stops being a quarter of the storage path and becomes a twentieth.

Second, and the one that matters, through the real MagazineQueue + MagazineConsumerTask drain — 10,000 messages, 5 forks, 32 shards, batchSize=10 against no batching:

Between +22% and +49% more throughput, median +32.7%, for a change that removes no work other than round trips. Total Aerospike calls for a 10,000-message drain fall from ~40,083 to ~31,094, a 22.4% reduction, and every one of the 60 runs delivered all 10,000 messages exactly once with nothing sidelined.

Idle consumers already cost nothing in storage¶

Over a 15-second window on a fully drained queue:

The column that matters is client attempts, not polls. Aerospike traffic holds at a median of 2 attempts covering 64 batch keys per 15 seconds whether there is one consumer or fifty. Fifty idle consumers generate the same storage load as one.

The instrumentation is not measurable¶

The same drain, same JAR, same container, differing only by a flag:

A positive figure means the instrumented build measured faster — which it did at two of the three counts. At 50 consumers the fork-to-fork spread runs 3,112 to 4,211 msg/s, about 30% of the median, against a 0.2% difference under test.

The honest conclusion is not "metrics are cheap" but "this experiment cannot measure them".

Limitations¶

• One machine, one container. No network between client and server, no replication, no multi-node coordination.
• Absolute throughput is host-bound. The flattening beyond 32 consumers is the laptop saturating. The shape is meaningful; the ceiling is an artefact.
• attempts counts client calls, not server round trips. Driver-internal retries are invisible, and a batch read counts as one however many keys it carries.
• The two suites are not comparable to each other — different throughput windows, different aggregations.
• The Magazine comparison bundles three changes: the Magazine version, the fire-timestamp removal and each version's Aerospike driver. The first two are separated above; the driver is not.
• Five forks bound dispersion, they do not establish significance. Where the spread is wide — the 32-consumer rows especially — read the brackets, not the median.