IT Disaster Recovery: Top 10 Disaster Recovery Questions for CIO’s

Recovery point objective (RPO) and recovery time objective (RTO) are two important measurements in disaster recovery and downtime.

RPO is the maximum age of files that an organization must recover from backup storage for normal operations to resume after a disaster. The recovery point objective determines the minimum frequency of backups. For example, if an organization has an RPO of four hours, the system must back up at least every four hours.

RTO is the maximum amount of time, following a disaster, for an organization to recover files from backup storage and resume normal operations. In other words, the recovery time objective is the maximum amount of downtime an organization can handle. If an organization has an RTO of two hours, it cannot be down for longer than that.

The RPO and RTO help administrators choose optimal disaster recovery strategies, technologies and procedures.

Meeting tighter RTO windows requires positioning secondary data so that it can be accessed faster. Recovery-in-place is one method of restoring data more quickly. This technology moves backup data to a live state on the backup appliance, eliminating the need to move data across a network. It can protect against storage system and server failure. Before using recovery-in-place, an organization needs to consider the performance of the disk backup appliance, the time needed to move data from a backup state to a live state, and failback. Since recovery-in-place can take up to 15 minutes, an organization may need to perform replication if it wants a quicker recovery time.

Preparing for a disaster requires a comprehensive approach that encompasses hardware and software, networking equipment, power, connectivity and testing that ensures DR is achievable within RTO and RPO targets. While implementing a thorough DR plan isn’t a small task, the potential benefits are significant.

Disaster recovery planning and

and strategy

A disaster recovery plan provides a structured approach for responding to unplanned incidents that threaten a company’s IT infrastructure, including hardware and software, networks, procedures and people.

The plan provides step-by-step disaster recovery strategies for recovering disrupted systems and networks to minimize negative impacts to company operations. A risk assessment identifies potential threats to the IT infrastructure; the DR plan outlines how to recover the elements that are most important to the company.

According to independent consultant Paul Kirvan, the components needed in a DR plan include the following:

• A disaster recovery policy statement, plan overview and main goals of the plan.
• Key personnel and DR team contact information.
• Description of emergency response actions immediately following an incident.
• A diagram of the entire network and recovery site.
• Directions for how to reach the recovery site.
• A list of software and systems that will be used in the recovery.
• Sample templates for a variety of technology recoveries, including technical documentation from vendors.
• Tips for dealing with the media.
• Summary of insurance coverage.
• Proposed actions for dealing with financial and legal issues.
• Ready-to-use forms to help complete the plan.

According to Kirvan, the development team should include the following activities when creating their DR plan:

• Meet with the internal technology team and network administrator to establish the scope of the plan, and then brief senior management on the meeting.
• Gather all of the relevant network infrastructure documents.
• Identify the most serious threats and vulnerabilities to the infrastructure.
• Review the previous history of outages and disruptions and how the business handled them.
• Identify the most critical IT assets and determine their maximum outage time.
• Identify the emergency response team and its capabilities.
• Have management review the plan.
• Test the plan and update it if necessary.
• Schedule the next review/audit of disaster recovery capabilities.

An organization should consider its disaster recovery plan a living document. The DR plan needs scheduled reviews and updates to ensure it is accurate and will work if a recovery is required. The plan ahould also be updated whenever there are changes in the business that could affect disaster recovery.

Disaster recovery testing

Testing is critical to change management in DR planning, helping to identify gaps and providing a chance to rehearse actions in the event of a crisis. A DR plan has a lot of moving parts, so testing it can help the organization understand exactly what employees should be doing during disaster recovery scenarios.

An organization should have a schedule for testing its disaster recovery policy and be wary of how intrusive it is. DR testing too frequently can be draining on personnel, but organizations on a less frequent schedule often delay testing further. In addition, an organization should test its DR plan after any system changes.

One way of testing is to run in disaster mode for a period of time; for example, failing over to the recovery site, letting the systems run there for a week and then failing back.

Here are some ways to get the most out of disaster recovery testing:

• Secure management approval and funding for the test.
• Provide detailed information about the test.
• Make sure the entire test team is available on the planned test date.
• Ensure your test does not conflict with other scheduled tests or activities.
• Confirm test scripts are correct.
• Verify that the test environment is ready.
• Schedule a dry run of the test.
• Be ready to halt the test if needed.
• Have a scribe take notes.
• Complete an after-action report about what worked and what failed.
• Use the results from the test to update the DR plan.

While it is optimal to perform a comprehensive disaster recovery test, this may not always be possible because of a lack of funding, time or resources. In that case, an organization should still bring together the key participants, distribute all of the relevant documents and perform a walkthrough of the test. There are risks to this scaled-down DR testing approach, as technology that has not been thoroughly tested may not work properly when needed.

Cloud disaster recovery/Disaster recovery as a service

Disaster recovery as a service (DRaaS) is a cloud-based method of DR that has gained popularity in recent years.

Positives of DRaaS include lower cost, easier deployment and the ability to test plans regularly. Cloud services save a company money by running on a shared infrastructure. They are more flexible, as companies can sign up for just the services they need. DR tests can be completed by simply spinning up temporary instances.

But cloud-based disaster recovery may not be available after a large-scale disaster, as there may not be enough room at the DR site to run every DRaaS user’s applications. Cloud DR also increases bandwidth needs and could degrade network performance with more complex systems. Costs vary widely among vendors — some charge based on network bandwidth consumption or storage consumption — and can add up quickly.