Disruption in email service - Investigation & fix in progress ( US, EU & AUS datacenter )

Incident Report for SMS Magic

Postmortem

Issue:
Authentication failure in internal service due to missed configuration changes done in the recent release.

Impact:
Customers experienced message delivery failures, receiving error statuses and response ID 0 across multiple providers.

Root Cause:

There was an issue with authentication of one of the internal services due to missed configuration changes done in the recent release impacting the delivery of outgoing messages for few accounts.

Resolution:
We have identified the issue and reverted the changes. It is now working fine, and our engineers are monitoring the system.

Duration of the outage.Data Center: US
Started At: August 2, 2024, 8:30 AM UTC.
Restored At: August 2, 2024, 7:15 PM UTC.

Posted Aug 07, 2024 - 06:03 UTC

Resolved

The incident has been resolved

Posted Aug 06, 2024 - 12:19 UTC

Monitoring

A fix has been implemented and we are monitoring the results.

Posted Aug 06, 2024 - 12:08 UTC

Update

We have identified the fix and this has been resolved

Posted Aug 06, 2024 - 12:06 UTC

Identified

The issue has been identified and fix is being implemented

Posted Aug 06, 2024 - 09:18 UTC

Investigating

Issue:
The email notifications and alerts are not being delivered to our customers due to the email service being down.

Impact:
Email Notifications and alerts wouldn’t be delivered to customers

Resolution:
We are working on restoring the email service

Data Center: AUS, EU, US

Duration of Outage
Started At: 2024-08-05 08:11:16 - US UTC

Posted Aug 06, 2024 - 08:55 UTC

This incident affected: SMS Magic US Data Center (Email Service).