On August 31, 2021 21:53 UTC, our monitoring informed us of an unusual 504 error code that originated from the Checkout API.

Upon further investigation, we discovered a timeout configuration error between our CDN and load balancer. This could have led to a situation where the request was still being processed by our API, but the CDN had already returned a 504 error message. These errors were hidden from our monitoring because the request was processed successfully, but the response never reached the customer.

The fix was introduced at 14:50 the following day. It was also including alerting enhancements.

All api.tipser.com endpoints were affected but only small portion of requests.