Active Directory & exponential complexity of domain recovery
Guido Grillenmeier, Chief Technologist, Semperis www.semperis.com
The cybersecurity landscape has drastically changed since the early days of Active Directory (AD).
A week doesn’t go by without an organisation’s on-premises Windows network being flattened by a ransomware or wiper attack. Indeed, January 2022 alone is a case in point.
On 9 January, doctors and nurses at Jackson Hospital in Florida, US were forced to track patient records on pen and paper for days after it shut down its computerised records system to avert a crisis-level ransomware attack.
Equally, payroll and staffing solutions specialist Kronos announced that it had recovered from a ransomware attack that left it unable to keep track of timekeeping which allows its customers to properly pay their employees.
Critically, these two incidents form only the very tip of the iceberg.
Accenture previously estimated that losses due to cybercrime could add up to $5.2 trillion between 2019 and 2024. Further, the International Data Corporation reports that 37% of organisations globally were the victim of a ransomware attack in 2021.
It is because of statistics such as these that Gartner identified the threat of new ransomware models as the single greatest emerging risk facing organisations in its latest Emerging Risks Monitor Report. Meanwhile, the European Union Agency for Cybersecurity (ENISA) also recently stated that we are witnessing the “golden era of ransomware” in its latest Threat Landscape report.
Given the intensity of the threat landscape today, the ability to recover your IT services quickly is key to your survival – and your Active Directory (AD) is a key component in this race against time! As such the recovery of your complete AD environment entirely from backup is no longer a nice thing to have – it is a business-critical requirement.
Domain recovery is a complex process
In years gone by, Microsoft has worked to improve Windows security substantially, adding features and capabilities to simplify AD object recovery and improve the behaviour of AD when running in a virtualised environment.
However, the fundamental problems of recovering an entire forest from backup haven’t changed. It is still an error prone, complex process that requires planning and practice for all but the most trivial AD deployments.
Recovering a domain entails many manual steps. These are described in Microsoft’s Active Directory Forest Recovery Guide, yet this is not just one simple article. It’s an extensive resource that directs to many other webpages that any individual would need to fully read and understand to be able to conduct a domain recovery with any degree of success.
A high-level overview of the steps involved in recovering an AD forest to a known-secure state can be summarised as follows:
1. Determine forest structure and available backups
2. Identify single DC for each domain with valid backup
3. Shut down all DCs in the forest
4. First recover Forest Root Domain
5. Then recover one DC of each child domain
6. Clean up and re-promote all other DCs in the forest
a. Ensure recovery of trust hierarchy and critical DNS resource records
b. Ensure recovery of parent domains prior to their child domains to maintain trust hierarchy
However, the reality of the situation is not so simple. Indeed, there are numerous sub steps that can be slotted in between those outlined.
Getting through the recovery process successfully requires coordination between AD engineers, recovery operations teams, and most likely virtualisation management teams as well. Everyone must execute their tasks flawlessly, in the right order, in probably the highest stress environment of their careers to date.
Further, the situation becomes increasingly complicated when the AD forest comprises multiple domains, creating a dependency chain which makes recovery even more difficult.
A company will always have to recover the main domain before they can recover any child domains. If you only have one domain, you are back online after you’ve recovered this – albeit after undertaking a complex recovery process. However, if you have an environment with many domains, or even subdomains, it becomes an administrative nightmare.
You cannot recover all domains in parallel. They must be recovered one at a time in a serial process that is lengthy, difficult and highly prone to error, creating a situation of exponential complexity in domain recovery.
The cost of ransomware and importance of backups
It is because of these difficulties that ransomware attacks can often cost organisations colossal sums.
A Sophos report reveals that the average cost to recover from a ransomware attack is $1.85 million. Yet this figure is not only attributed to the ransom demands involved – it also accounts for the downtime, people time, device costs, network costs and other lost opportunities associated with an attack.
It takes time and money to recover, especially with complex processes such as multi-domain recoveries involved. To avoid such a significant impact, firms therefore need to have appropriate and adequate recovery plans in place to get back online quickly in the event of an attack.
This begins with gaining a clear picture and full understanding of your AD forest structure so that you know where a recovery needs to begin should things head south.
Here, ensuring you have valid backups is critical.
All too often companies only realise they don’t have valid backups until it’s too late. To be completely safe, it is wise to regularly check backups and ensure that these are completely separated and disconnected from your environment.
Preparedness is critically important
For this reason, we need a proper backup of the AD domain controllers. But there are some equally significant considerations that need to be made here, too.
Companies may opt to turn to third party providers promising tools, but it is important to note that these also have their limitations.
Being able to back up AD domain controllers does not automatically mean that a tool can help you quickly recover your AD forest. Most of these solutions concentrating on OS-level backups might offer support in helping to recover individual servers and domain controllers, but they can’t coordinate the complex recovery process that is required to bring your AD forest back to life.
So, what do companies need to do?
There is simply no getting away from the fact that AD disaster recovery is a highly difficult undertaking. However, firms can prepare properly in a variety of ways.
Beyond having external backups, companies should look to practice a mock AD recovery process to provide some experience and insight into the challenges and process should an actual attack hit. In doing so, an action plan or playbook can be formulated, detailing the entire AD disaster recovery plan and clear responsibilities for executing it.
Equally, tools and solutions can be implemented that can help to prevent an AD disaster from happening in the first place, providing additional lines of defence which may stifle an attacker. Yet there is no 100% guarantee they will stop an attack. That’s why, regardless of how much you invest into prevention, you should still always anticipate an attack and prepare an adequate recovery plan.
This is more important now than ever before. AD didn’t used to be attacked all that often because it was difficult. However, today, you don’t need to be an expert to do so – with ransomware-as-a-service rampant, unsophisticated attackers are able to execute sophisticated attacks.
Further, there are always new vulnerabilities emerging. It’s only when Microsoft announces a new fix that these gaps are plugged, but before this is rolled out it is often the case that any new blind spot vulnerability has been leveraged by several hackers.
Therefore, more than ever companies need to prepare for it – if all hell breaks loose, you need a means of ensuring that your entire network isn’t lost.
To read more on this topic, Semperis’ AD disaster recovery white paper can be found here: https://www.semperis.com/resources/does-your-active-directory-disaster-recovery-plan-cover-cyberattacks/
Semperis is also announcing enhancements to its Active Directory Forest Recovery (ADFR) product to help organizations rapidly conduct post-attack forensics capabilities and recover Active Directory to a trusted, malware-free environment following a cyber disaster. More information can be found at (link to press release).
