This incident has been resolved. If you have any questions, please contact support: https://radiomast.io/support
Posted Jan 10, 2024 - 17:12 EST
Monitoring
All regions are operational and listener traffic levels are returning to normal.
Our engineering team has resolved the issue with ingest in Frankfurt.
We will continue to monitoring all services and performances across all regions to ensure there are no additional impacts. We apologize again for the impact this disruption had on your stations and we will be performing a post-mortem analysis to understand how we can more quickly mitigate broad networking outages in the future.
Posted Jan 10, 2024 - 17:06 EST
Update
Our Toronto datacenter connectivity is improving and traffic is returning to normal levels. Service to the Streaming Network, Cloud Transcoder, and Stream Monitoring services are broadly restored. Since our last update, we observed and mitigated impacts in our Singapore and Miami regions, and will continue to monitor and mitigate where necessary.
We are investigating new reports of issues our Frankfurt region and will provide another update when more information is available.
Thank you for your patience as we work to minimize the disruption to your stations.
Posted Jan 10, 2024 - 16:52 EST
Identified
Our engineering team has identified the root cause and are working with our datacenter partners to mitigate the underlying issue. We apologize for this disruption and will post updates as progress is made.
Posted Jan 10, 2024 - 16:21 EST
Investigating
We are currently investigating an issue impacting networking primarily across Eastern North America.
Impacted services include the website, control panel, Stream Monitoring, and Cloud Transcoder and Playout (beta) services in Toronto. Users may experience longer load times than usual on the website.
We will update this incident with more information as it becomes available. We are currently observing increased packet loss at our datacenters in Toronto and Montreal, but also unusual drops in broadcaster activity in Singapore and Miami, which possibly suggest that major residential or business ISPs may be affected.
Posted Jan 10, 2024 - 16:08 EST
This incident affected: Streaming Network, Monitoring, Cloud Transcoder, Website, Control Panel, and API.