Updated on 2023-01-06
Rackspace confirmed that the December 2022 cyberattack was conducted by the Play ransomware group that accessed some of its customers’ Personal Storage Table (PST) files containing lots of information.
Updated on 2023-01-05: Rackspace Says Attack was the Work of Play Ransomware Group
Managed cloud hosting provider Rackspace says that the December 2, 2022 attack that took down its hosted Microsoft Exchange service was conducted by the Play ransomware group. Rackspace is still working to recover email data. In a January 5, 2023 update, Rackspace explicitly states that the incident was not due to the ProxyNotShell exploit, as was being reported. Instead, the Play ransomware group used a zero-day vulnerability to gain access to the Rackspace Hosted Exchange email environment. Rackspace also writes that it does not plan to rebuild its hosted Microsoft Exchange service.
Note
- If you were using Rackspace hosted Exchange, recovered email (in the form of a PST file) will only be available for 30 days. Also note they have only recovered about 50% of their mailboxes so far. This raises the question of what do you do if your service provider decides to no longer offer your service? Do you have a data retrieval (recovery)/migration strategy? Do you have a sense of how long it would take you to qualify, secure and configure a replacement service? How often are you checking the health of your service providers? Nothing against Rackspace here, it’s due diligence. It wasn’t so long ago we learned the irony of “too big to fail.”
Read more in
- Hosted Exchange Disruption
- Microsoft Exchange Server Elevation of Privilege Vulnerability CVE-2022-41040
Updated on 2022-12-16: Rackspace ransomware’d
Customers affected by a ransomware attack nuking Rackspace’s hosted Exchange email are royally pissed at how the web hosting giant handled the incident. In an email to customers, Rackspace seemed to pat itself on the back, saying the incident “was quickly contained and limited solely” to customers’ Exchange emails, but made no reference of what it was doing to get their data back. With no sign of backups, customers look like they’re out in the cold. There’s also a scathing op-ed worth reading. Read more:
- On the 12th day of the Rackspace email disaster, it did not give to me …
- The death of Rackspace’s ‘Fanatical Support’
- Amid Outrage, Rackspace Sends Users Email Touting Its Incident Response
Updated on 2022-12-12: Rackspace faces three CALs
Cloud hosting provider Rackspace will have to defend at least three different class-action lawsuits related to a ransomware attack that hit a part of its server infrastructure and has left countless companies without access to their email servers. In an interview last week, Rackspace suggested they might not be able to recover all their customers’ data, which they referred to as “legacy data.” The company also appears to have given up on hosting Exchange email servers in its cloud and said it was migrating all its existing customers to Microsoft 365. Migrating its Exchange customers to a rival will cost the company $30 million, according to documents Rackspace filed with the SEC. Read more:
- Rackspace Says FBI Probing Outage Affecting Thousands of Small Businesses
- Another class-action lawsuit filed over Rackspace outage; says company failed to protect users’ data
Updated on 2022-12-08: Rackspace Acknowledges Outage Was Caused by Ransomware
Cloud services provider Rackspace has acknowledged that the outage that disrupted availability of its Hosted Exchange environments was due to a ransomware infection. Rackspace is making sure that all affected customers have access to Microsoft 365 and is providing guidance to help them migrate.
Note
- Rackspace has done a good thing in warning its customers that “In situations like these, it’s common for scammers and cybercriminals to try to take advantage.” Check your processes to be sure you’d be doing the same thing – once it is made public that Company X has been compromised and is notifying its customers, attackers pretend to be company X and try to scam passwords from customers. The important thing Rackspace has not yet divulged is *why* the attack succeeded – odds are high that reusable passwords were compromised somewhere. Ask your cloud service provider where they are in migrating privileged cloud infrastructure admin accounts to multi-factor authentication.
- Companies typically employ one of two approaches in breach notification: limit information made available on the attack; or be open and transparent about the incident. As Rackspace continues to investigate the cyber breach, let’s hope they fully share details of the event – to include what security applications were in place and operating. We all can learn from this unfortunate cyber incident.
- Be prepared to disclose both root cause and what you’ve done to prevent recurrence to both customers and regulators when you are breached. Transparency and honesty should be favored over spinning a story to make you look better.
Read more in
- Hosted Exchange Disruption
- Rackspace confirms ransomware attack behind days-long email meltdown
- Rackspace confirms email outage was from a ransomware attack
Updated on 2022-12-07
Hosted services firm Rackspace confirmed that the ongoing outages on its hosted Microsoft Exchange environments are caused by a ransomware attack. No other details have been released yet. Read more: Rackspace Confirms Exchange Outage Caused by Ransomware
Updated on 2022-12-06: It was ransomware
Rackspace has confirmed that the major outage of its Exchange email server infrastructure that took place over the weekend was caused by ransomware. Read more: Rackspace Technology Hosted Exchange Environment Update
Updated on 2022-12-03
Rackspace was forced to shut down its Hosted Exchange environment, owing to a cybersecurity incident. The cloud company did not share any other details on the incident. Read more: Rackspace Shuts Down Hosted Exchange Systems Due to Security Incident
Updated on 2022-12-02: Rackspace security incident
Cloud hosting platform Rackspace took down its hosted Microsoft Exchange email server infrastructure following what the company described as a “security incident.” The incident took place on Friday, December 2, and Rackspace was still working on restoring affected services at the time of this newsletter on Monday morning. No confirmation yet that this is a ransomware attack. British security researcher Kevin Beaumont believes Rackspace’s Exchange servers were most likely hacked using the ProxyNotShell vulnerability. Read more: Rackspace Cloud Office suffers destructive security breach
Overview: Rackspace Outage
Managed services provider Rackspace experienced a security incident that caused an outage of its hosted Exchange environment. As of early morning (EST) Monday, December 5, Rackspace says they “have successfully restored email services to thousands of customers on Microsoft 365 and continue to make progress on restoring email service to every affected customer. At this time, moving to Microsoft 365 is the best solution for customers who can now also implement temporary forwarding.
Note
- Last week I gave a lunchtime talk on Capitol Hill to Congressional aides who were involved in writing cybersecurity policy and one of the questions was “Wouldn’t all of this be solved if everything were run in the cloud, like Netflix and Amazon?” After all, a short “spinny circle of death” delay in video seems much better than what happened at the Colonial Gas Pipeline… Good to use example like Rackspace’s woes to make sure backup plans are in place and tested and that management understands that security issues really don’t change that much whether the computers are in our buildings or in the cloud’s buildings.
- Rackspace is providing support to either migrate to MS 365 or forward your email to another domain. Migrating to MS 365 is going to be the most familiar option, and Rackspace is providing archive copies of inboxes to customers for import into MS 365. Note that with either option there may be email “in flight” which may need to be resent as it is queued and waiting to be delivered. When migrating make sure you implement needed security settings such as MFA, ATP, leverage the Microsoft 365 Defender and Microsoft Purview compliance portals to make sure you aren’t missing anything.
- This may be one of the most interesting security incidents in a while. Rackspace’s business model is in reselling its hosted solutions. In this case, they have done what, in my opinion, is the right thing. They have started to request customers move over to the Microsoft 365 service. Rackspace has possibly a better change of rolling out patches quickly in their environments, but let’s face it, Microsoft is more in control of the source code of exchange than we are, and they may even start rolling out patches before anyone else
- Not a good day, week, or month to come for Rackspace. Hopefully, once systems have been restored and user operation back to normal, Rackspace will fully share details of the event – to include what security applications were in place and operating. We all can learn from this unfortunate cyber incident.
Read more in
