Rackspace outage affects access to Safe4

Unfortunately on Thursday 29 June 2017 the Safe4 service was not available for just over two hours. The issue started at 11:14 BST, and service was resumed at 13:37. During this time it was not possible to connect to the Safe4 service through the normal web interface. We at Safe4 are very proud of our exceptional record of availability, and this was a significant outage. We appreciate that this has impacted our users and apologise for the inconvenience it will have caused.

As part of our routine operation we investigate all outages to determine the root cause and to see if there are any lessons that can be learnt in order to prevent a recurrence.

Our investigation showed that the issue was due to a global outage that had occurred at our hosting provider, Rackspace, affecting many of their customers. Rackspace have explained that the problem was caused by a licencing issue with one of their third party vendors used to provide their load balancing service. Safe4 makes use of load balancers in order to distribute web traffic across our servers to enable us to support our growing web traffic reliably. Since every access to our system uses the load balancers Safe4 was impacted by the problem at Rackspace.

The other parts of the Safe4 infrastructure continued to operate as normal, and at no stage was any customer information compromised.