Voice Services Outage
Incident Report for Cytracom
Postmortem

On April 10 and 11, CPU, IO, and network contention within a shared tenancy environment led to service degradation. The interruption was in connection to a data center upgrade project that has been underway for several months and is now complete. In April, the final phase of migration away from legacy architecture was initiated. The disruptions on April 10 and 11 were the result of the temporary use of a third-party cloud provider—a maintenance step during the infrastructure upgrade that was required in the process of moving to new architecture. That provider experienced a service interruption that directly impacted Cytracom voice customers. The project is now complete, and all VoIP users have been migrated to the new data center architecture as intended.

Posted Apr 17, 2024 - 16:26 CDT

Resolved
This incident has been resolved.
Posted Apr 10, 2024 - 12:03 CDT
Update
A fix has been implemented and we are monitoring the results.
Posted Apr 10, 2024 - 11:45 CDT
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Apr 10, 2024 - 11:44 CDT
Investigating
We have identified an issue affecting call completion for a subset of our customers. Cytracom Engineering is working to resolve this and will provide updates here.
Posted Apr 10, 2024 - 11:39 CDT
This incident affected: Voice Services (Call Completion, Voicemail, Call Center).