Rackspace hosted Exchange suffered a devastating failure starting December 2, 2022 and is still continuous as of 12:37 AM December fourth. Initially referred to as connection and login problems, the guidance was eventually upgraded to reveal that they were handling a security incident.
Rackspace Hosted Exchange Issues
The Rackspace system went down in the early morning hours of December 2, 2022. Initially there was no word from Rackspace about what the problem was, much less an ETA of when it would be solved.
Consumers on Buy Twitter Verification reported that Rackspace was not reacting to support emails.
This has been quite the day with #Rackspace. Every hosted exchange customer has actually been down for 14 hours or two. Assistance isn’t reading/responding to tickets. Updates are unhelpful.
I am worried now that they fell victim to something bad like the ProxyNotShell PoC hack. https://t.co/jchKsAO3Z7
— Joe Sinkwitz (@CygnusSEO) December 2, 2022
A Rackspace client independently messaged me over social networks on Friday to relate their experience:
“All hosted Exchange clients down over the previous 16 hours.
Uncertain the number of companies that is, but it’s considerable.
They’re serving a 554 long delay bounce so people emailing in aren’t aware of the bounce for numerous hours.”
The official Rackspace status page provided a running upgrade of the interruption however the initial posts had no information aside from there was an outage and it was being investigated.
The first official update was on December second at 2:49 AM:
“We are investigating a concern that is affecting our Hosted Exchange environments. More details will be posted as they appear.”
Thirteen minutes later on Rackspace started calling it a “connection problem.”
“We are investigating reports of connectivity concerns to our Exchange environments.
Users may experience an error upon accessing the Outlook Web App (Webmail) and syncing their email customer(s).”
By 6:36 AM the Rackspace updates explained the continuous problem as “connectivity and login issues” then later on that afternoon at 1:54 PM Rackspace announced they were still in the “examination phase” of the blackout, still trying to find out what went wrong.
And they were still calling it “connectivity and login concerns” in their Cloud Workplace environments at 4:51 PM that afternoon.
Rackspace Recommends Migrating to Microsoft 365
4 hours later Rackspace referred to the scenario as a “significant failure”and started providing their customers complimentary Microsoft Exchange Plan 1 licenses on Microsoft 365 as a workaround till they comprehended the issue and might bring the system back online.
The official guidance specified:
“We experienced a substantial failure in our Hosted Exchange environment. We proactively closed down the environment to avoid any additional issues while we continue work to restore service. As we continue to resolve the source of the problem, we have an alternate option that will re-activate your capability to send out and get e-mails.
At no cost to you, we will be offering you access to Microsoft Exchange Plan 1 licenses on Microsoft 365 until further notice.”
Rackspace Hosted Exchange Security Occurrence
It was not till nearly 24 hours later at 1:57 AM on December 3rd that Rackspace officially announced that their hosted Exchange service was suffering from a security occurrence.
The announcement even more exposed that the Rackspace service technicians had powered down and detached the Exchange environment.
“After additional analysis, we have identified that this is a security occurrence.
The known effect is separated to a portion of our Hosted Exchange platform. We are taking required actions to examine and secure our environments.”
Twelve hours later on that afternoon they updated the status page with more info that their security team and outside experts were still working on fixing the failure.
Was Rackspace Service Affected by a Vulnerability?
Rackspace has not launched information of the security event.
A security event generally includes a vulnerability and there are two extreme vulnerabilities presently in the wile that were covered in November 2022.
These are the 2 most present vulnerabilities:
Microsoft Exchange Server Server-Side Request Forgery (SSRF) Vulnerability
A Server Side Demand Forgery (SSRF) attack enables a hacker to read and alter information on the server.
Microsoft Exchange Server Remote Code Execution Vulnerability
A Remote Code Execution Vulnerability is one in which an opponent has the ability to run harmful code on a server.
An advisory published in October 2022 explained the impact of the vulnerabilities:
“A validated remote enemy can perform SSRF attacks to intensify privileges and perform arbtirary PowerShell code on vulnerable Microsoft Exchange servers.
As the attack is targeted against Microsoft Exchange Mailbox server, the assaulter can possibly gain access to other resources through lateral motion into Exchange and Active Directory site environments.”
The Rackspace interruption updates have actually not shown what the particular issue was, just that it was a security incident.
The most present status upgrade since December fourth specified that the service is still down and clients are motivated to migrate to the Microsoft 365 service.
Rackspace posted the following on December 4, 2022 at 12:37 AM:
“We continue to make development in attending to the occurrence. The accessibility of your service and security of your data is of high significance.
We have actually committed substantial internal resources and engaged world-class external expertise in our efforts to decrease negative effects to customers.”
It’s possible that the above noted vulnerabilities belong to the security incident affecting the Rackspace Hosted Exchange service.
There has actually been no announcement of whether client information has been compromised. This occasion is still ongoing.
Featured image by Best SMM Panel/Orn Rin