Difficult and impressive bugs I’ve experienced.

2021

Result returned wrong value

• Symptom: Result instance returned wrong value when it was used with coroutine.
• Related components: Kotlin (jvm) >= 1.5.0 < 1.5.30
• Root cause: Result was wrapped twice

Crisis of XID wraparound

• Symptom: XID was about to run out
• Related components: PostgreSQL 11
• Root cause: deleted too many tuples by backlog of GC batch
• Detail: presented at TreasureData Tech Talk 2022

2020

Rarely failed to send HTTP response by chunked transfer encoding

• Symptom: Rarely jersey server sent incomplete http response when ChunkedOutput was used.
• Related components: jersey < 2.32
• Root cause: Race condition of ChunkedOutput

2019

Sudden performance degradation of PostgreSQL

• Symptom: Suddenly DB response became very slow and didn’t respond at all in the end.
• Related components: PostgreSQL 9.4, AWS RDS
• Root cause: Exclusive lock on table extention
• Detail: slide