Rackspace Hosted Exchange Failure Due to Security Incident

Posted by

Rackspace hosted Exchange suffered a disastrous interruption beginning December 2, 2022 and is still continuous since 12:37 AM December 4th. Initially referred to as connectivity and login problems, the assistance was eventually upgraded to reveal that they were dealing with a security event.

Rackspace Hosted Exchange Issues

The Rackspace system went down in the early morning hours of December 2, 2022. At first there was no word from Rackspace about what the issue was, much less an ETA of when it would be resolved.

Customers on Buy Twitter Verification reported that Rackspace was not responding to support emails.

A Rackspace client privately messaged me over social networks on Friday to relate their experience:

“All hosted Exchange clients down over the past 16 hours.

Unsure how many business that is, but it’s considerable.

They’re serving a 554 long delay bounce so individuals emailing in aren’t familiar with the bounce for numerous hours.”

The official Rackspace status page provided a running upgrade of the outage however the initial posts had no info aside from there was an outage and it was being examined.

The first official update was on December 2nd at 2:49 AM:

“We are examining a concern that is affecting our Hosted Exchange environments. More details will be posted as they appear.”

Thirteen minutes later on Rackspace started calling it a “connectivity concern.”

“We are investigating reports of connectivity problems to our Exchange environments.

Users might experience an error upon accessing the Outlook Web App (Webmail) and syncing their e-mail client(s).”

By 6:36 AM the Rackspace updates described the continuous issue as “connection and login issues” then later on that afternoon at 1:54 PM Rackspace revealed they were still in the “investigation phase” of the interruption, still trying to figure out what failed.

And they were still calling it “connection and login concerns” in their Cloud Office environments at 4:51 PM that afternoon.

Rackspace Recommends Moving to Microsoft 365

4 hours later on Rackspace described the circumstance as a “significant failure”and began using their clients free Microsoft Exchange Plan 1 licenses on Microsoft 365 as a workaround till they understood the problem and might bring the system back online.

The official assistance specified:

“We experienced a considerable failure in our Hosted Exchange environment. We proactively shut down the environment to avoid any additional problems while we continue work to bring back service. As we continue to work through the source of the problem, we have an alternate solution that will re-activate your capability to send out and receive e-mails.

At no cost to you, we will be supplying you access to Microsoft Exchange Strategy 1 licenses on Microsoft 365 up until more notice.”

Rackspace Hosted Exchange Security Occurrence

It was not up until almost 24 hr later on at 1:57 AM on December 3rd that Rackspace officially announced that their hosted Exchange service was struggling with a security incident.

The announcement even more revealed that the Rackspace professionals had actually powered down and detached the Exchange environment.

Rackspace posted:

“After more analysis, we have determined that this is a security occurrence.

The known effect is separated to a portion of our Hosted Exchange platform. We are taking needed actions to evaluate and safeguard our environments.”

Twelve hours later on that afternoon they updated the status page with more details that their security group and outdoors experts were still working on resolving the outage.

Was Rackspace Service Affected by a Vulnerability?

Rackspace has not released information of the security occasion.

A security event typically includes a vulnerability and there are 2 severe vulnerabilities presently in the wile that were covered in November 2022.

These are the two most current vulnerabilities:

  • CVE-2022-41040
    Microsoft Exchange Server Server-Side Demand Forgery (SSRF) Vulnerability
    A Server Side Request Forgery (SSRF) attack permits a hacker to check out and change data on the server.
  • CVE-2022-41082
    Microsoft Exchange Server Remote Code Execution Vulnerability
    A Remote Code Execution Vulnerability is one in which an aggressor is able to run malicious code on a server.

An advisory published in October 2022 described the effect of the vulnerabilities:

“A validated remote assaulter can perform SSRF attacks to escalate benefits and perform arbtirary PowerShell code on vulnerable Microsoft Exchange servers.

As the attack is targeted against Microsoft Exchange Mail box server, the attacker can possibly gain access to other resources by means of lateral movement into Exchange and Active Directory environments.”

The Rackspace outage updates have actually not indicated what the particular problem was, just that it was a security event.

The most present status upgrade as of December 4th specified that the service is still down and consumers are encouraged to move to the Microsoft 365 service.

Rackspace published the following on December 4, 2022 at 12:37 AM:

“We continue to make development in addressing the occurrence. The availability of your service and security of your information is of high significance.

We have committed substantial internal resources and engaged first-rate external proficiency in our efforts to reduce negative impacts to consumers.”

It’s possible that the above noted vulnerabilities belong to the security occurrence affecting the Rackspace Hosted Exchange service.

There has been no statement of whether customer info has actually been compromised. This occasion is still continuous.

Included image by Best SMM Panel/Orn Rin