Resolved -
The issue has been resolved. We have put a fix in place, which will identify the repeated request from Zoho webhooks and ignore. SMS Service from Zoho webhooks are operational now.
Sep 25, 13:16 IST
Identified -
We have identified the issue as Zoho repeating the requests. After communicating with Zoho team, it was found that, there was a glitch on Zoho's end on Wednesday where the web-hooks configured in CRM (Associated with Time based action) haven't triggered to the records. Zoho had updated and fixed the same on Friday, however, the web-hooks got queued and executed continuously on Friday. That turned out to be the root-cause of the behavior experienced, as SMS Service endpoint was triggered repeatedly.
As a remedy we had disabled our endpoint used for Zoho Workflow Webhook. We are working on putting fixes to handle this. The service will be up by 25th Sept, 02:00PM IST.
Sep 24, 22:45 IST
Investigating -
The SMS Service, which is triggered from zoho workflow rules, will not be available due to some issue with zoho communication. Our team is investigating.
Sep 24, 14:18 IST
Some customers saw delay in incoming messages reaching their CRM, between Sept 19, 9:30 PM IST and Sept 20, 9:30 AM IST. This happened because few of our push workers failed due to network issues. The issue has been fixed and services are working as expected.
Sep 20, 12:52 IST
Resolved -
We have resolved the degraded performance of the server and servers are working fine at normal speed.
Issue identified at 11/Sep/2017:13:17 (GMT) and resolved by 11/Sep/2017:13:59 (GMT).
Incident Management Team
Sep 12, 12:05 IST
Monitoring -
Resolved and Monitoring - We have identified and resolved a degraded performance of our server due to higher load. We have resolved the issues and Servers are working at normal speed. Performance degradation was observed for a brief period of 42 minutes from 11/Sep/2017:13:17 pm to 13:59 pm GMT.