Hyper Disater
Hyper Disater
Although backing up and recovering a server is often simple and straightforward, virtualization can bring an extra layer of complexity into the picture. In this article series, I will discuss your options for disaster recovery within a Hyper-V environment. Although there are many benefits to server virtualization, there is no denying that virtualization also adds an extra layer of complexity to server management. Perhaps nowhere is this more true than when it comes to backup and restore initiatives. What is even more frustrating is that there is a lot of misinformation on the Internet regarding backups and disaster recovery for virtual servers. In this article series, I am going to try to set the record straight by explaining the various disaster recovery options that are available to you in a Hyper-V environment.
Snapshots
The first subject that I want to address is snapshots. As you probably know, Hyper-V has a built in mechanism that allows you to take a point in time snap shot of your virtual machines. Although snapshots certainly have their place, a snapshot is not a substitute for a virtual server backup. Over the last several months, I have stumbled onto quite a few Web sites that have said that making a snapshot is the preferred method for backing up virtual machines in a Hyper-V environment. Although a snapshot is not a replacement for a backup, I think that I know where the confusion came from. Most of the backup applications on the market today use the Volume Shadow Copy Service (VSS). The VSS Writer creates a snapshot as a part of the backup process. It is important to understand though, that VSS snapshots and Hyper-V snapshots are not the same thing. Creating a snapshot in Hyper-V provides you with a quick and easy way of reverting a virtual machine to its previous state. For example, suppose that you were about to install a new version of an application onto a virtual machine. You could create a snapshot of the virtual machine before beginning the upgrade process. That way, if something were to go horribly wrong during the upgrade process, you could just restore the snapshot and your system would be back to the way that it existed before you attempted the upgrade. Even though this probably sounds an awful lot like a backup, there are some important differences between snapshots and backups. For one thing, snapshots are stored locally on the server. This means that if the server were to suffer a hardware failure, then you might end up losing your snapshots. In contrast, backups are usually either written to removable media or to a disk array on a dedicated backup server. Another very important distinction between Hyper-V snapshots and backups is that Hyper-V snapshots are not application aware. In fact, if you look at Microsofts support policy for virtualizing Exchange 2007, they specifically say that Making virtual machine snapshots of an Exchange guest virtual machine is not supported.
So why is that? Well, as I said, Hyper-V snapshots are not application aware. Exchange Server 2007 uses an extensible storage engine database to store Exchange Server data. Although a snapshot will include the database in its current state, data is initially written to database pages in memory, as a way of reducing Exchange Servers I/O requirements. A Hyper-V snapshot does not backup the contents of the systems memory, which in the case of Exchange Server could lead to data loss, an inconsistent database, or both. Of course thats just one example of why taking a hyper-v snapshot is not always a good idea. Whether your virtual server is running Exchange or something else, you must remember that if you restore a Hyper-V snapshot, the server will expect everything to be the same as it was at the time that the snapshot was taken. For example, if a virtual server is hosting a database, the database application may expect the same clients to be connected to the database as were connected at the time when the snapshot was made. This does not mean that Hyper-V snapshots are useless, quite the contrary. Snapshots are a great way of hedging your bets against a disaster when you are going to perform a risky procedure. The key to using snapshots successfully is planning for their use rather than using them recklessly. As I said earlier, Microsoft does not support the use of Hyper-V snapshots in an Exchange Server environment. Having said that though, you can safely make a Hyper-V snapshot if you first dismount the database and stop the Exchange related services. I am not advising anyone to do that, I am only pointing out that if you plan for a snapshot, and use snapshots responsibly, then they can be of benefit to you. I have previously mentioned some of the reasons why using a snapshot is not a good alternative to a more traditional backup method, but I never really took the time to explain what happens when you create a Hyper-V snapshot. As I am sure that you probably know, virtual machines running on Hyper-V use virtual hard drive files (.VHD files) instead of physical hard drive volumes. When you create a snapshot of a virtual machine, you are essentially freezing the virtual hard drive file so that you can go back to it at a later time if necessary. The problem with freezing your virtual hard drive files is that you would not be able to use your virtual machines as normal if the server was completely frozen. To get around this problem Microsoft has designed Hyper-V so that when you create a snapshot, the virtual hard drive is frozen, and a snapshot file is created. A lot of people assume that the snapshot file is a mirror image of the virtual hard drive file, but this is simply not the case. Taking a snapshot only usually takes a few seconds. With todays technology it would be physically impossible to duplicate a 500 GB virtual hard drive file in a few seconds time. What happens is that the snapshot consists of a .AVHD file, which is stored alongside the virtual machines other files. The virtual hard drive file is frozen, but that does not mean that it is not used. It only means that the virtual hard drive file becomes read only. From that point on, data may be read from the virtual hard drive file, but all writes occur within the snapshot file.
One thing to keep in mind is that having a snapshot means that there are now two possible places in which a file might exist. When Windows needs to read a file, it must first check to see if an updated version of the file exists within the snapshot. If not, then the file will be read from the virtual hard drive file. The implication is that having one or more snapshots of a virtual machine can have a major impact on the virtual machines performance! Fortunately, this performance impact does not have to be permanent. You can roll the system back to its previous state and then remove the snapshots, or you can merge the snapshot into the .VHD file. I will show you how to do that and more in Part 2.
Part -2
Creating a Snapshot
The procedure for creating a snapshot is actually very simple. If you look at Figure A, you can see that I have opened the Hyper-V Manager console, and selected one of my virtual machines that is presently running. If you look at the column on the right side of the console, you can see that it is divided into an upper and a lower section. The upper section contains action items that pertain to the server as a whole. The lower section contains items that are specific to the virtual machine that is currently selected. The third option from the bottom is Snapshot.
Figure A: The column on the right contains an option to create a snapshot of the currently selected virtual machine When you click the Snapshot button, Hyper-V will begin taking the snapshot. The process does not take very long at all. If you look at Figure B, you can see that by the time I was able to click the screen capture button, the snapshotting process was already 25% complete.
Figure B: It only takes a few seconds to make a snapshot After you create a snapshot, the snapshot will appear in the Snapshots pane beneath the list of virtual machines. If you look at Figure C, you will see that I have taken two snapshots, both of which are listed in a snapshot tree. The reason why the snapshots are listed in this way is because snapshots are cumulative.
Settings
As you can see in the figure above, Hyper-V tells you the date and time when each snapshot was recorded. Although this is helpful, it can be a little bit tough to remember the virtual machine state that is associated with each snapshot. Fortunately, Hyper-V allows you to make some notes regarding the purpose of each snapshot. To do so, right click on the snapshot that you want to annotate, and then choose the Settings command from the shortcut menu. This will cause Windows to display a screen that is very similar to the virtual machines Settings screen. The biggest difference between this screen and the normal Settings screen is that you can not change any of the hardware settings. Click on the Name option, and you will have the option of changing the snapshots name and entering some notes about the snapshot, as shown in Figure D. When you click OK, your notes will appear in Hyper-V Managers lower middle pane, as shown in Figure E.
Figure D: The Settings screen allows you to make notes about your virtual machine
Figure E: Your notes will appear in the consoles lower middle pane when you select the snapshot As you saw in Figure D, the Settings screen gives you the option of renaming the snapshot. You can also rename a snapshot by simply clicking on the Rename option located in the lower portion of the consoles Actions pane.
Applying a Snapshot
Suppose that you created a snapshot of a virtual machine prior to performing a risky operation. When the snapshot was complete, you performed the operation, and everything went horribly wrong. In such a situation, you would obviously want to restore the snapshot. To do so, just right click on the snapshot that you want to restore, and then choose the Apply command from the shortcut menu. Doing so will cause Hyper-V to display the warning message that is shown in Figure F.
Figure F As you can see from the warning message above, applying the snapshot causes the virtual machines current state to be lost, which is what you want if the virtual machine has been trashed. You will notice though, that you have the option of taking a snapshot before you apply the snapshot. That way, you can revert your machine to its original state at the time that the snapshot was made, but you have the option of rolling the virtual machine forward to the state that it is in right now. Keep in mind that simply applying the snapshot does not cause the snapshot to be deleted. The snapshot remains in place in case you want to have another go at the operation that caused the failure to occur. If you want to delete a snapshot, you will have to do so manually.
Deleting a Snapshot
Whether you decide to apply a snapshot or not, you are eventually going to want to delete your snapshots so that the virtual machines performance can be returned to normal. The most important thing to know about deleting snapshots is that doing so does not cause data loss. Deleting a snapshot simply removes your ability to apply that snapshot. Any data that is associated with the snapshot is merged with the virtual machine. Microsoft gives you a couple of different options for deleting snapshots. If you only want to delete a single snapshot, then you should just select the snapshot that you want to delete, and then click the Delete Snapshot option found in the lower portion of the Actions pane. If you want to delete all of the snapshots, then select the top level of the snapshot hierarchy, and then click the Delete Snapshot Subtree option.
When you delete snapshots, the snapshots will disappear almost immediately. Even though the Hyper-V Manager no longer shows the snapshots though, they still exist on disk. The .AVHD files will remain on disk until you either reboot the virtual machine or put it into a saved state. It is extremely important that you do not delete these snapshot files manually from outside of the Hyper-V Manager.
Part -3
Introduction
So far in this series, I have spoken about creating virtual machine snapshots, and about the advantages and disadvantages of doing so. As you may recall from my previous article, the biggest problem with virtual machine snapshots is that they do not offer the same amount of protection as a traditional backup, and they can really hurt a virtual machines performance. That being the case, I want to turn my attention to more traditional backup options for Hyper-V. When it comes to backing up a virtualized environment, you really have two choices. You can run a backup at the host server level, or you can run a backup within the guest operating system. Of course some organizations use a combination of the two methods.
The second requirement is that the guest operating systems must use volumes that are formatted as NTFS. FAT, FAT-32, and other file systems are not supported. Furthermore, the guest machines cannot be configured to use dynamic disks. The backup will only work if the guest machines use basic disks. Keep in mind that I am not referring to the volume that the .VHD file is stored on. The volume containing the .VHD file can be basic or dynamic. I am talking about whether the guest operating system sees the .VHD file as being a basic disk or a dynamic disk. Finally, the Volume Shadow Copy Service must be enabled for all volumes that contain VM components. Each volume must be configured to store its own shadow copy data. In other words, the shadow copies for C: must reside on C:. Therefore, each volume must have enough free space to comfortably accommodate shadow copy data.
compatible with this particular VSS writer. Although Windows Server Backup (Windows Server 2008s built in backup application that replaced NTBACKUP) is designed to create VSS backups, it is not designed to work with the Hyper-V VSS writer. If you really want to use Windows Server Backup, then you can register the Hyper-V VSS Writer with Windows Server Backup by creating a registry key. Please remember that editing the registry is dangerous, and that making a mistake can destroy Windows, your applications, or both. Normally I would tell you to create a full system backup before attempting a registry modification, but in this case all you can really do is just be really careful (unless you want to shut down your virtual machines and then make a full backup). To create the necessary registry key, open the Registry Editor in the host operating system, and then navigate through the registry tree to: HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT \CurrentVersion\WindowsServerBackup\Application Support\{66841CD4-6DED-4F4B-8F17FD23F8DDC3DE}. Keep in mind that you will have to manually create the Windows Server Backup container and all subsequent containers. Once the necessary structure is in place, create a new Reg_SZ key named Application Identifier, and assign it a value of Hyper-V. This will register the Hyper-V VSS writer for use with Windows Server Backup.
Conclusion
adverti sement
Although making host level backups of your Hyper-V server is a fairly good backup solution, there are some major limitations to using it that go beyond what I have already discussed. It is critical that you familiarize yourself with these issues, or you could find yourself facing some serious problems if you ever have to perform a restoration. I will explain these issues and much more in Part 4 of this series.
Part -4
Introduction
Although it is possible to create a Volume Shadowcopy Service backup of a Hyper-V server and all of its virtual machines, there are some major limitations that you need to be aware of. This article examines those limitations and discusses some workarounds. In my previous article in this series, I explained that there are some major limitations to making host level backups in a Hyper-V environment. That being the case, I want to take the opportunity to talk about these limitations and some workarounds.
Dynamic Disks
Assuming that you are using the Volume Shadowcopy Service to create an online backup of the virtual machines on your host server, then the first limitation that you need to be aware of is that your virtual machines cannot contain dynamic disks. It is perfectly acceptable for the host operating system to use virtual disks, but you have to make sure that the guest operating systems treat all of their associated virtual hard drive files as basic disks. The exact method of checking to see whether an operating system is using basic disks or dynamic disks varies a bit depending on the version of Windows that is being run. Generally speaking though, you should be able to open the Disk Management Console by entering the [Link] command at the Run prompt within a guest operating system. I have included a screen capture of the Disk Management Console in Figure A for the benefit of anyone who may not be familiar with it. If you look at Disk 0, you can see that it is a dynamic disk. In some cases it is possible to right click on the disk (not on the volumes on the disk), and choose the Convert to Basic command from the shortcut menu.
Figure A: By looking at the Disk Management Console within a guest machine, you can tell whether the guest OS is configured to treat virtual hard drives as basic disks or as dynamic disks A full blown discussion of the differences between basic and dynamic disks is beyond the scope of this article, but you do need to be aware that there are sometimes serious implications
involved in converting a disk. Therefore, make sure that you have a thorough understanding of how the conversion process will impact the server before you attempt to convert a disk.
Figure B: Windows Server Backup does not allow you to recover individual virtual machines
Keep in mind that this limitation is Windows Server Backup specific. It may not necessarily apply to every VSS based backup application, but you definitely need to check your backup application to determine whether this limitation applies or not. Another major limitation that you may encounter is that if a virtual machine contains multiple snapshots, then you would not have any trouble making a VSS backup, but you would not be able to restore the backup. Thankfully, there is a workaround to this issue. The trick to successfully restoring the virtual machine is to manually recover the individual snapshots first. Once you have recovered the snapshots, then you can restore the virtual machine. The exact method that you will use for doing so varies considerably depending on the backup software that you are using. Here are the general steps that you would perform:
1. If the virtual machine is running, turn it off and then delete the virtual machine 2. Perform a file level restoration on the snapshot folders. By default, snapshots are stored in C:\Program Data\Microsoft\Windows\Hyper-V\Snapshots 3. After you have recovered the individual snapshots, then perform an application recovery of Hyper-V. This will restore the virtual machines
A Saved State
One last limitation that you need to know about is that under certain conditions, virtual machines may be forced into a saved state while the VSS snapshot is taken. There are three specific conditions that can cause this to occur. First, a virtual machine will be forced into a saved state if the guest OS does not support VSS. This applies to older versions of Windows, such as Windows NT Server and Windows 2000 Server, as well as to non Windows operating systems. The second condition that can cause a virtual machine to be forced into a saved state during the VSS snapshot is the Hyper-V integration services not being installed. Again, this mostly effects older versions of Windows and guest machines running non Windows operating systems, but it can apply to any guest operating system if the integration services have not been installed. The third condition is that a guest machine will be forced into a saved state during the VSS snapshot process if the Backup (Volume Snapshot) component within the integration services has been disabled. If you have not really done much with the integration services beyond installing them in a default configuration, then this concept might seem foreign to you. It is really pretty simple though. Although the integration services are usually installed as a single entity, it is actually made up of several individual services. If you want to see these individual services, right click on a virtual machine and choose the Settings command from the shortcut menu. When the Settings dialog box appears, select the Integration Services option. As you can see in Figure C, doing so will cause Windows to display the individual integration services.
Figure C: The integration services are actually made up of several different services If you look at the figure above, you will notice that the last service on the list is the Backup (Volume Snapshot) service. This particular service must be enabled if you want to avoid putting the virtual machine into a saved state during the VSS snapshot process.
Conclusion
adverti sement
In this article, I have tried to explain many of the caveats associated with making a Volume Shadowcopy Service backup of Hyper-V and its guest machines. After reading about all of these issues, you may be wondering if creating this type of backup is even worth it. I am convinced however, that VSS backups for Hyper-V do have their place. In the next article in this series, I will explain what VSS backups are good for, and why these types of backups should only make up a portion of your overall backup strategy for virtual machines.
Part -5
Introduction
In my previous article in this series, I spoke about numerous limitations associated with performing a host level VSS backup of Hyper-V. Some of these limitations are specific to Windows Server Backup, but even more so, the backup process is restrictive enough that you may have been left wondering if perhaps there might be a better way to backup your virtual machines. I ended the previous article by saying that I was convinced by the advantages of using VSS backups on Hyper-V servers, but that VSS backups should only be a part of your overall backup strategy. Obviously, thats a pretty bold statement, so I want to use this article to elaborate on why I made it.
backup). That being the case, Hyper-V momentarily hibernates the virtual machine while the snapshot is taken. Unfortunately, there is a momentary interruption of service while the guest machine is hibernated, but the hibernation and snapshotting process usually happens fairly quickly. The reason why hibernation is necessary is because part of the hibernation process involves writing the contents of the virtual machines memory and in some cases even its virtual processor state to a file. This ensures that all of the data associated with the virtual machine is captured during the backup process and that transactions arent occurring during the backup, which would cause the backup to become corrupt.
of servers that you are backing up. In other words, if you are backing up five servers, then you are typically going to need five licenses. Many backup applications require you to install an agent onto the servers that you are going to be backing up. The agents not only facilitate communications between the backup application and the server, but they also typically provide the backup application with a handy way of counting how many servers are being backed up, and therefore how many licenses are being used. When you perform a host level backup of a Hyper-V server, you are backing up multiple virtual machines, but the only agent that is required is the one that is running on the host operating system. If you have a lot of virtual machines, then this can amount to substantial cost savings.
Conclusion
As you can see, there are definitely benefits to performing host level VSS backups of your Hyper-V servers. One thing to keep in mind though, is that in order to avoid writing an article that is backup application specific, I generalized most of the concepts that I wrote about. To the best of my knowledge, the information in this article is accurate for most of the major backup applications on the market, but it is possible that some backup applications may do things a bit differently. With that said, the next article in this series will talk about some of the limitations of VSS host level backups, and how you can get around those limitations.
Part -6
Introduction
In my previous article in this series, I talked about some of the advantages to performing VSS backups at the host level. Since I had already discussed the primary disadvantages of host level backups in Part 4 of this series, I want to turn my attention to guest level backups. In other words, I will take this opportunity to talk about the advantages and disadvantages of backing up individual virtual machines rather than backing up your Hyper-V server as a whole.
Granularity
If you have read some of my previous articles in this series, then you may already have a pretty good idea of what some of the advantages and disadvantages are of performing guest machine backups, but I want to discuss them anyway, just to make sure that we are all on the same page. By far the biggest advantage to guest machine level backups is the granularity that it provides you with. Unlike a host level backup, you are free to pick and choose exactly which files, folders, applications, etc. you would like to backup. Likewise, a guest level backup also provides
you with the ability to restore individual files and folders as opposed to forcing you to restore the server as a whole.
Simplicity
Another advantage to guest level backups is the simplicity involved in creating them. I will be the first to concede that some backup applications are anything but simple, but that is not really what I am talking about. What I mean is that from the standpoint of your backup application, performing a guest level backup of a virtual server is basically the same as backing up a physical server. There arent a lot of special considerations that you have to take into account just because you are backing up a virtual machine.
Compatibility
One possible disadvantage to guest machine backups is compatibility. I recently saw a situation in which an organization had set up a Hyper-V server with a few guest machines, and then connected a tape drive directly to the host server. The problem with this configuration was that the tape drive was only accessible to the servers parent partition. The virtual machines that were hosted on the server were not able to communicate with the tape drive. Likewise, the backup application that was running on the host operating system was not Hyper-V aware, so it had no way of reliably backing up the guest operating systems.
License Consumption
One possible disadvantage to performing a guest level backup is that you may consume a lot more licenses for your backup application than you would have had you just performed a host level backup. Most of the backup applications that I have worked with require you to purchase an agent license for every server that you are backing up. Performing a host level backup only requires a single agent (which typically requires one license), while backing up individual virtual machines requires a separate agent for each VM which typically requires (multiple licenses).
The problem with backing up individual virtual machines is that if you have to perform a restoration, your backup software would not recognize the fact that you are restoring a virtual server. While it is true that your backup still contains a copy of the Hyper-V Integration Services, there is more to a virtual machine than the Integration Services and the Windows operating system. If you think back to the time when you originally set up the virtual machine in question, you will recall that you had to start out by giving Hyper-V some basic information about it. For example, you had to tell Hyper-V how much memory to allocate to the virtual machine, which files would be used as virtual hard drives, where those files would be located at, and how the virtual machine would connect to your network. This (among other information) becomes a part of the virtual machine. What some administrators do not realize though, is that when you perform a backup of a guest operating system what you are really backing up is the contents of the virtual servers virtual hard drive files (the .VHD files). The low level configuration information that I mentioned above does not reside on a virtual hard drive. After all, the low level configuration information tells Hyper-V which virtual hard drives to use, so windows certainly cant embed the information within a virtual hard drive. So what does all of this mean to someone who has to recover a failed Hyper-V server? Well, it is not impossible to recover a virtual server using guest operating system backups. It is just that the recovery process is going to take longer to perform and it is going to involve a lot more work for the administrator. An administrator will have to manually recreate each virtual machine prior to restoring the individual backups. Of course this is only true in the event of a catastrophic failure. If you only need to recover some individual files then performing a virtual machine restoration using a guest backup is no different than performing a restoration on a physical server.
Conclusion
adverti sement
So now that I have discussed all of the pros and cons associated with host level and guest level backups, the big question becomes which backup method should you use. My recommendation is to use both methods, but dont do it blindly. What I mean is that most organizations have to make sure that their backups complete within a specific amount of time (known as a backup window). There is also a finite amount of space available on the backup media. The problem is that if you are performing both host level and guest level backups, then you are backing up most of the data on the server twice. That often means that backups are going to take twice as long to complete, and that they are going to consume twice as much space on your backup media. Believe it or not, there are some techniques that you may be able to use if you want to perform both types of backups. I will discuss these techniques in the next part of this series, as well as what you should do if you absolutely cant perform both types of backups.