Problem With Virtual Machine Backup–Part 2 of 2

In part 1, I mentioned that I was still having problem backing up the second and third virtual machines. 

This was the error message from Backup Exec.

Job ended: Thursday, November 03, 2011 at 10:15:11 PM
Completed status: Failed
Final error: 0xe000959b - An attempt to take a snapshot of a virtual machine failed because it was unable to quiesce a file system.
Final error category: Resource Errors

For additional information regarding this error refer to link V-79-57344-38299

This was the error message from vCenter.

Create virtual machine snapshot

Cannot create a quiesced snapshot because
the create snapshot operation exceeded the
time limit for holding off I/O in the frozen
virtual machine.

So I started troubleshooting the VSS since usually it is the culprit.  I checked to make sure SYNC driver and VMware snapshot provider were not installed.  Verified that VSS was working correctly using the VSSAdmin command and running the FREZEE.BAT.  I could not find anything wrong with the VSS.  Then I found the article Quiesced snapshots fail for Windows 2008 R2 guests and set disk.EnableUUID to false.  This will create crash consistent backup instead of application consistent backup.  However, this still does not work.

So I exported the ESXi host’s log to examine it and below were what I found.

In the  hostd log file, I saw Snapshot guest failed: The file is too big for the filesystem.

In the vmware.log file of the second and third virtual machines, I saw the following.

vmx| FILE: File_VMFSSupportsFileSize: Requested file size (1314259992576) larger than maximum supported filesystem file size (274877906944)
vmx| DiskLibCreateCustom: if your disk is on VMFS, you may consider increasing the block size.
….
vmx| SNAPSHOT: SnapshotBranch failed: The destination file system does not support large files (5).
vmx| [msg.checkpoint.save.fail2.std3] Error encountered while saving snapshot.
vmx| The destination file system does not support large files.

Bingo, that is the cause of the problem.  Both the virtual machines have virtual HDDs created in different datastores.  The problem was the datastore where the virtual machine’s vmx file (configuration file) resided was setup with a block size of 1MB.  By default the workingDir contains the vmx file and workingDir is the location where the snapshots are created.  With 1MB block size, the maximum virtual HDD size will be 256GB.  However, in the other datastores, there are virtual HDDs much bigger than 256GB and that caused the snapshot to fail.

According to the article Creating a snapshot for a virtual machine fails with the error: File is larger than maximum file size supported, I either change the workingDir to a datastore with enough block size or change the location of the virtual machine configuration files.  I chose the later and did a storage vMotion to migrate the configuration files to a new datastore with block size of 8MB.

Starting the storage vMotion while the virtual machine is still running.

Used the Advance interface to select what files to migrate and to which datastore.

By moving the configuration files to the new datatstore with the 8MB block size solved the backup issue.  The storage vMotion were not a smooth one and I should talk about it in my next post.

Pin It