← All comics

Twice the muscle

#131 2 images

Russ put a second machine behind the shop to get it through the sale. Every customer bought two sofas, paid for both, and had both delivered.

The mistake

A second instance of the order service is added to handle the load, but nothing splits the work between the two. Both instances read every incoming order from the same queue and each one handles it in full: taking payment, creating the order and sending it to the warehouse. There is no partitioning, locking or idempotency key to stop an order being processed twice. So every order is charged and fulfilled twice. The sales dashboard counts processed orders instead of unique customer orders, so it shows the duplicates as real growth, and removing the extra instance looks like a crash.

Share