Increased error rate sending and receiving messages on email-based channels
Incident Report for Front
Resolved
This incident has been resolved.
Posted Sep 28, 2024 - 02:51 UTC
Update
Customers in us-west-2 and eu-west-1 regions have fully recovered. We are still working through the incoming message backlog on us-west-1.
Posted Sep 28, 2024 - 02:21 UTC
Monitoring
The fix has been deployed and we are seeing email delivery services have recovered. We're monitoring and catching up on backlog so there might be residual delays.
Posted Sep 27, 2024 - 21:47 UTC
Update
We have validated the fix and are in the process of doing a manual deploy to circumvent the current Docker outage.
Posted Sep 27, 2024 - 21:27 UTC
Update
We are validating the fix in our staging environment.
Posted Sep 27, 2024 - 21:02 UTC
Identified
We have identified the cause of the issue and are working on a fix. The issue affects some but not all channels.
Posted Sep 27, 2024 - 20:38 UTC
Update
We are continuing to investigate this issue.
Posted Sep 27, 2024 - 20:25 UTC
Investigating
We are currently investigating this issue.
Posted Sep 27, 2024 - 20:25 UTC
This incident affected: SMTP (non-Gmail / O365), Gmail, and O365.