Veeam Backup & Replication v7 Patch 4 released

Written by Marco on . Posted in Veeam

I noticed on twitter that Veeam released Patch 4 for Veeam B&R v7. So what is fixed?

New Features and Enhancements

VMware Virtual SAN (VSAN) 

  • In addition to adding basic support (as provided by other vendors), the intelligent load-balancing engine was enhanced to account for VSAN specifics. As the result, for each VM the job will pick backup proxy running on VSAN cluster node with most of the virtual disks’ data available locally. This significantly reduces backup traffic on VSAN cluster network, resulting in minimum possible impact on production environment from backup activities.

Microsoft SQL Server 2014

  • Added support for Microsoft SQL Server 2014 both as the protected guest workload (including application-aware processing functionality), and the back-end database for backup and Enterprise Manager servers.

License key auto update

  • Added automated license key update option to the License Information dialog. With auto-update enabled, the product will check Veeam licensing server periodically for an updated license key, and download and install the key automatically as soon as it becomes available. This feature is particularly useful to the Service Providers and subscription-based customers, and it removes the need to download and install the license key manually each time when the license extension is purchased.

Backup Copy

  • The maximum allowed amount of restore points in the Backup Copy job has been increased to 999.
  • Backup Copy will now resume the transfer after network connection drop to a Linux-based backup repository.
  • Backup Copy jobs should no longer report errors during the days when source backup jobs are not scheduled to run – for example, during the weekend.

Hyper-V

  • Added support for certain Hardware VSS Providers that previously could not be detected by the storage rescan process, and as such could not be used by the jobs.
  • Jobs will now retry failed snapshot creation when another shadow copy of the same volume is already in progress, instead of immediately failing to process a VM.

Resolved Issues

General 

  • Under certain circumstances, the target data mover may crash with the “Cannot allocate memory for an array” error while processing a backup job with WAN storage optimization setting.
  • Increased maximum amount of open files for Linux data mover to improve backup reliability in cases with a very large amount of concurrent tasks hitting the same repository.

vSphere

  • If the same ESX(i) host is added to Managed Servers both as a part of the vCenter, and as a standalone host, the jobs fail to remove VM snapshots correctly.
  • Replication job does not truncate Microsoft Exchange transaction logs when the corresponding setting is set to “Truncate logs on successful backup only”.
  • Setting EnableSameHostHotaddMode registry value (introduced in Patch #3) to 1 causes replication jobs to fail with the “Object reference not set to an instance of an object” error.
  • If you clone replica VM with vSphere, and start the replication job, the target VM will be selected randomly between the two VMs.

Hyper-V

  • Issues introduced with Windows Server 2012 R2 Update (April 2014 rollup), see KB1863 for more info.
  • Very large amount of parallel tasks may cause in significant delays (up to a few minutes) between assigning required backup infrastructure resources to the job, and the job actually starting to leverage those resources.
  • Replication jobs do not truncate Microsoft Exchange transaction logs regardless of the corresponding settings state.
  • Processing of virtual machines with long dash symbol in the file name fails with the “The system cannot find the path specified.\n\nFailed to open file [C:\ClusterStorage\FOLDER\GUID.xml] in readonly mode. (System.Runtime.InteropServices.COMException)” error.
  • Parallel snapshots of SMB3 share may fail with the “Failed to add volume to the VSS snapshot set. Another shadow copy creation is already in progress. Wait a few moments and try again” error.
  • Restoring a VM to localized Hyper-V cluster creates duplicate cluster resources remaining in the Starting state in Failover Cluster Manager.

Application-aware processing

  • During full VM restore and SureBackup job, Domain Controllers may spend longer than required in the DSRM mode (up to 10 minutes – until the corresponding timeout expires), thus unnecessarily delaying DC recovery.
  • Increased timeouts for network-less application aware processing of vSphere guests to improve the chance of processing completing successfully.

Backup Copy

  • Backup Copy configured with multiple backup jobs as the source does not include VMs which are excluded in one source backup job, but included in another.
  • Health Check process continuously fails with the “Timed out waiting for the backup repository resource to become available for backup integrity check” error during the periods when the Backup Copy job is restricted from using network bandwidth.
  • Backup Copy is incorrectly marked as failed despite completing successfully when source or target WAN accelerator becomes unavailable (for example, due to the network issue and reboot) during the period when Backup Copy job is waiting for its availability.

Backup from Storage Snapshots

  • Disabling failover to network processing mode on a backup proxy causes intelligent load balancing not to assign the backup proxy to jobs running with storage snapshots integration enabled.
  • Under certain circumstances, backup from 3PAR volumes may fail with the “LUN is already taken for set ESXs” error.

Configuration backup

  • The presence of null byte in tape changer serial number causes configuration backup to fail.

File Copy

  • Certain files are skipped with the File Copy job with the “Cannot copy a symbolic link of type Veeam.Backup.Core.CLocalFileItem into a directory of type Veeam.Backup.Core.CRemoteWinFolder” error, however the job reports success. For example, this may happen when copying files from volumes with Windows Server 2012 deduplication enabled.
  • File Copy job fails when attempting with the “Client error: boost::filesystem::remove: Access is denied” error when attempting to overwrite read-only file on the destination Windows server. The remaining files are not copied.

Native File Level Recovery

  • Starting file level recovery from the Restore wizard fails with the “Object reference not set to an instance of an object” error when SAN snapshots is picked as the recovery source. Starting the same process from SAN infrastructure tab does not have this issue.
  • In rare scenarios, file level recovery is unable to parse LDM contents of the virtual disk. As the result, Backup Browser opens empty, without any volumes mounted.
  • Restoring to original location for volumes which are mounted to a folder (as opposed to a drive letter) fails when such volumes reside on GPT partition or dynamic disk.

Multi-OS FLR

  • Folder permissions, group and owner IDs are not preserved when restoring a folder with “Preserve permissions and ownership” check box selected.
  • Multi-OS FLR fails with the “The file specified is not a virtual disk” error on replica VMs with VMDK larger than 2TB in size.

Tape

  • Depending on where the file is placed on the tape, restore process may not calculate the file size correctly, resulting in corrupted file restored from good backup.
  • File to Tape job fails to process files with full path of exactly 260 symbols long with the “The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters” error. Files with full path longer or shorter than 260 symbols are processed normally.
  • Restoring a very large file from tape may fail with the “Value was either too large or too small for an Int64″ error.
  • Restoring a VM from tape may fail with the “Unable to read tape metadata” error.

UI

  • VMs and Templates View of the virtual infrastructure Browse dialog, and Category View on the Virtual Machines tab do not show vSphere vApps.
  • Sending email notifications fails with the “Key not valid for use in specified state” error if SMTP credentials were entered by a user account other than Veeam Backup Service account.

Veeam Explorer for Microsoft Exchange

  • Opening mailbox database may occasionally fail with the “The process cannot access the file ‘C:\PATH\ese.dll’ because it is being used by another process” error.
  • Mounting mailbox database or transaction log volumes fails with the “Exchange logs cannot be replayed, block size mismatch” error from 512-byte emulation (512e) disks.
  • In rare scenarios, opening a mailbox from German localized Microsoft Exchange deployment fails with the “JetError -1404, JET_errIndexNotFound, No such index” error.

Veeam Explorer for Microsoft SharePoint

  • SharePoint restore wizard fails with the “Object reference not set to an instance of an object” error if actual SharePoint configuration does not match the one collected with application-aware processing logic during the backup job.

WAN Acceleration

  • Increased source WAN accelerator digests retention to 2 months to avoid digests recalculation every week.

Pilot Patch #4 (build 7.0.0.870)

The generally available version of Patch #4 (build 7.0.0.871) also addresses the following issue reported during pilot patch deployments:

  • Tape inventory fails on certain tape drives with the following error: “Tape fatal error. No more data is on the tape”

This patch also contains all fixes from Patch 1, R2 update and Patch 3

More Information:

Prior to installing this patch please reboot the Veeam server to clear any locks on the Veeam services and when the reboot is done, please stop all the Veeam services and apply the patch.

When the patch is installed manually, start the Veeam services and open Veeam, and allow it to update all the components.

To obtain the patch please download it from Download

Win a trip around the world with Veeam

Written by Marco on . Posted in Veeam

Veeam is about to get its 100,000th customer and is launching an interactive contest for a chance to win a trip around the world and other prizes (Google Glass, iPad and Microsoft Surface).

To participate, visitors need to register and predict the location of Veeam’s 100,000th customer on the interactive map. The closer you are to the right spot, the better chance you have to win the trip around the world and other prizes.

We currently have a live pre-registration page. The main contest will start next week.

Join in on the fun! We hope you and your readers will participate and celebrate this great achievement with us.

Guess the location here: http://world.veeam.com/

For more information: http://world.veeam.com/veeam_tc_2014.pdf

Veeam Backup & Replication v7 Patch 3 released

Written by Marco on . Posted in Veeam

I noticed on twitter that Veeam released Patch 3 for Veeam B&R v7. So what is fixed?

Engine

  • Significantly reduced load on SQL Server hosting the configuration database, and improved performance of various job activities and user interface operations.

vSphere

  • Improved vCenter Server hierarchy enumeration logic to speed up vCenter Server registration process in a few times.
  • Intelligent load balancing can now be configured to give preference to backup proxy located on the same host using the EnableSameHostHotaddMode (DWORD) registry value.
  • Replication jobs should now fail immediately upon encountering fatal error, rather than wasting time performing additional operations that are no longer relevant.

Hyper-V

  • Added timeouts on all snapshot operations, including snapshot creation, deletion, import to off-host proxy, and reading snapshot data to prevent conditions with stuck snapshots due to environmental issues.
  • Maximum volume snapshot life time is now customizable using the hvSnapshotLifeTimeHour (DWORD) registry value. Default value is 72 hours, but customization may be necessary to allow full backup of very large VMs on slow backup hardware.
  • Added ability to configure SCVMM communication timeouts using the SCVMMConnectionTimeoutMinutes (DWORD)

1-Click FLR

  • Support for downloading files larger than 4GB.

Built-in WAN acceleration

  • Data processing performance has been further improved by up to 25% comparing to R2 update.

Multi-OS File level recovery

  • Added support for file names with Unicode characters.

Tape

  • Support for backing up files with file names longer than 260 characters, and directory names longer than 248 characters.
  • Support for devices that do not implement sending error in response to attempt of writing to write protected tape.
  • Support for devices that do not implement sending the end of media event, such as certain IBM tape drives. Capacity for such drives is now controlled by the job.
  • Support for devices that do not implement drive enumeration in the drivers correctly. Such tape devices are now additionally queried for drives with SCSI commands.

User interface

  • Column sizes and sorting rules will now persist for each user in all grids and views.
  • Disabling GFS retention in Backup Copy job now offers to remove the existing full backup along.
  • Backup file repair process is now reported with the dedicated job status.

Rotated media

This update adds full support for rotated media with all job types and backup modes removes the need of using additional scripting. The new functionality is controlled by the following registry values:

ForceCreateMissingVBK (DWORD)

Enables support for rotated drives. With this functionality enabled, if any backup file from the latest full backup chain is missing (such as when the existing hard drive is replaced by another one), jobs will start the new backup chain and create the new full backup (instead of failing out).

Set to 1 to enable support for rotated drives.

ForceDeleteBackupFiles (DWORD)

Enables automatic cleanup of the backup repository from ALL existing files if any backup file from the latest full backup chain is missing (such as when the existing hard drive is replaced by the hard drive brought back from offsite location, and containing some older backup files).

Set to 3 to make the job delete the entire contents of the backup job’s folder only.

Set to 5 to make the job delete the entire contents of the root backup repository folder, thus potentially deleting any files belonging to other jobs along. Note that if backup repository points at the volume’s root folder, the entire volume’s contents is erased.

Resolved Issues

Engine

  • Job may fail VM processing if backup proxy has a very large amount of NICs with the “A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond” error. To work around this, increase the timeout by creating the newly introduced ConnectByIPsTimeoutSec (DWORD) registry value.
  • Network throttling rule does not apply if some job is already running at the time when the throttling period starts for the given rule.
  • In some scenarios, target data mover for CIFS-based backup repository starts on the backup server, rather than on the specified proxying server.
  • Summary.xml growing beyond 16385 bytes results in the following job runs fail with the “Read summary xml error” error.
  • Running too many parallel task on a single proxy or repository may cause the “Cannot find connections pool by id” error.
  • Stopping the job on transform day causes resource leak.

vSphere

  • Processing of VM with virtual disk larget than 2TB fails with the “Requested value ‘seSparse’ was not found. (System.ArgumentException)” error if there is an existing VM snapshot present.
  • Failback of VM with virtual disk larget than 2TB fails with the “Requested value ‘seSparse’ was not found. (System.ArgumentException)“.
  • Network remapping functionality of replication job does not function correctly if performed within a single DVS (Distributed Virtual Switch), or when source and target VM are connected to more than one DVS.
  • VeeamReplicaSummary growing beyond 16385 bytes results in the following job runs fail with the following error: “Failed to create replica vm disk“.
  • Full VM restore of a VM containing virtual disks with the same names creates a new VM with incorrect configuration if the original VM also had one or more snapshots present at the time of backup. Attempting to start the restored VM results in “Cannot open the disk or one of the snapshot disks it depends on. The system cannot find the file specified” vSphere error.
  • All jobs now detect and refuse to process VMs with virtual disks disk size not being a multiple of 1KB to prevent issues at restore (“Client error: Declared and actual sizes of the disk are different“) or Backup Copy (“Inline data validation failed“). Such disks cannot be created with vSphere user interfaces, and are usually a product of P2V conversions or similar activities.
  • Backing up VMs with non-default workingDir parameter from vCenter Server 4.X fails with the “Sequence contains no matching element” error.

Hyper-V

  • Changed Block Tracking fails on dynamic VHD disk aligned to 1024 bytes with the “Failed to update unaccounted changes for disk. Change tracking is disabled” error.
  • Replication job with seeding enabled fails with the “Client error: Client either not connected to server or connection is broken
  • Added workaround to prevent CSV cache from leaking during backup activities (Microsoft issue).
  • Under certain circumstances, replication to a cluster may fail with the “The process cannot access the file because it is being used by another process” error.
  • Stopping a job at certain “bad” time results in hardware VSS snapshot remaining on the storage with the “Failed to close current snapshot” error logged.
  • Volume rescan fails with the “Server ‘This server’ does not exists or access denied. Host discovery failed. The RPC server is unavailable” error when the product is installed on the Hyper-V host.
  • Selecting the Run server on this side option on the source host makes replication jobs fail.
  • Replication job statistics may show incorrect Read and Transferred values upon network disconnect at the end of incremental job run.
  • Swap exclusion notification is missing from the VM processing log.
  • Removed case sensitivity when obtaining host names from SCVMM server for better matching reliability with hosts already added to the configuration.

Application-aware processing

  • Application-aware processing breaks DFSR replication of SYSVOL when all domain controllers are backed up at the same time due to the parallel processing.
  • Transaction logs are truncated when the VM processing fails due to loss of connectivity with backup proxy even when the job is set to truncating logs on successful backup only.
  • Improved in-guest VSS snapshot handling to increase backup reliability for Windows Server 2008

Backup Copy

  • Health check process does not respect concurrent tasks limit and I/O throttling rules on backup repository.
  • After certain failures health check process may start logging failure sessions continuously.
  • Health check process fails with the “Failed to process method {Signature.FullRecheckBackup}: There is no FIB [10disk_1-flat.vmdk] in the specified restore point” error if the Backup Copy job is mapped on backup file containing VM backups with excluded disks.

File level recovery

  • Windows File Level Recovery fails to mount volumes with missing or incorrect volume lengths. The issue is typically observed on virtual disks created via the P2V process.
  • Restoring a file to the original location with Multi-OS File Level Recovery fails with the “Wasn’t able to find source file item [(/media/dm-0/root/test, /root/test)], skipping overwriting” error if the original file still exists.

Guest indexing

  • Catalog service may leak memory when processing very large amounts of guest indexes.

Built-in WAN Acceleration

  • Under rare circumstances, Backup Copy job to Linux-based repository may experience data mover process SegFault crash with Back-trace containing IP 0x848deff.
  • If target WAN accelerator runs out of disk space during digests generation, the job starts failing repeatedly with the following error: “An existing connection was forcibly closed by the remote host –tr: Cannot write data to the socket

Backup from Storage Snapshots

  • Processing of VM on assigned to backup proxy with multiple FC adapters may fail with “Server with WWN is not found on HP 3PAR host” if some FC adapters do not have access to production 3PAR storage.
  • In certain HP 3PAR storage configuration, VM processing may fail with the “Unable to find volume with Node WWN and LUN ID” error.
  • Processing of VM with workingDir parameter set to a different HP 3PAR or HP StoreVirtual datastore fails with the “Unable to find matching device for LVID” error.
  • HP StoreVirtual snapshots may fail to be deleted with the “Failed to connect to host” error due to reaching the limit of SSH connections to StoreVirtual storage after certain time.

Tape

  • Under certain circumstances, the tape backup job may fail with the “Violation of PRIMARY KEY constraint ‘PK_Tape.file_versions’. Cannot insert duplicate key in object ‘dbo.Tape.file_versions’” error.
  • Restoring a file split across multiple tapes may fail with the “Block offset in source file differs from the current write position in target file” error.
  • Very long read or write operation may fail on certain tape libraries (such as IBM TS3100) with the following error: “The requested resource is in use“.
  • Tape jobs using standalone drives hand on second task with the “Tape medium was used in backup session, NOT VALID” error.
  • Tape media is not automatically erased in standalone tape drives when “Do not protect” option is enabled on a media pool.
  • Returning tape mediums without barcode from Offline after re-inventorization results in “Tape requires return to original, but it is already filled” error if the original slot is occupied.
  • VBRCatalog\Tapes\Media\ folder is not properly cleaned up upon various tape media operations, resulting in thrash accumulation.
  • Changing a tape may result in the “Subquery returned more than 1 value” error.
  • Corrected statuses for a number of tape drive alerts according to their real severity.

Configuration backup

  • Configuration backup fails if a tape changer connected to the backup server has null byte in its serial number with the “‘.’, hexadecimal value 0×00, is an invalid character” error.
  • Configuration backup of very large configuration database fails with the “Compressed or Uncompressed size, or offset exceeds the maximum value. Consider setting the UseZip64WhenSaving property on the ZipFile” error.
  • Under certain circumstances, configuration backup restore fails with the “String or binary data would be truncated. The statement has been terminated” error

Enterprise Manager

  • The required SureBackup job does not start and stop immediately after Virtual Lab request is approved or dismissed in the Enterprise Manager.
  • Adding a domain user to portal roles results in the “The specified directory service attribute or value does not exist” warning.
  • “Keep only last” setting of the Event history accepts unsupported values, resulting in Options dialog on attached backup servers fail to with value not valid errors.

User Interface

  • One of the configuration database tables is never cleaned up, which may cause user interface sluggishness after using the product for extended time.
  • Sorting by Total Size and Free Space columns does not work correctly in Files > VMware vSphere > Datastores view.
  • Sorting by Type, Size or Modified columns in Files > VMware vSphere > Datastores makes the user interface hang.

This patch also contains all fixes from Patch 1 and R2 update

More Information:

Prior to installing this patch please reboot the Veeam server to clear any locks on the Veeam services and when the reboot is done, please stop all the Veeam services and apply the patch.

When the patch is installed manually, start the Veeam services and open Veeam, and allow it to update all the components.

To obtain the patch please download it from: Download


How to stop your Veeam backup job when it is stuck

Written by Marco on . Posted in Veeam

A couple of weeks ago I ran into an issue that one of my Veeam backup jobs was stuck in a stopping state. After waiting for a long time I started searching for a solution to kill this backup job.

I found a KB article at the Veeam Knowledgebase that is worth mentioning. This is KB1727 “How to forcibly stop jobs that are stuck in ‘stopping’ status”

Solution:

The problem may occur because a process or task that the job is waiting for has not been completed. Such a task can be snapshot removal. Therefore, as the first step it is recommended to check the vSphere Client to see if there is a snapshot removal process pending/working on the vSphere side.

Make sure no important jobs/restores are running because the following steps will force them to stop:

  1. Close the Veeam Backup & Replication console, and open services.msc on the Veeam Backup server,
  2. Stop the Veeam Backup Service.
  3. Open the Task Manager and kill all Veeam.Backup.Manager and VeeamAgent.exe processes.
  4. Restart the Veeam Backup Service to release file locks held by either SQL or the OS.
  5. Remove snapshots from VMs (if they have not been already removed).

Note. If you use the Virtual Appliance (HOTADD) mode, before removing the snapshots make sure there are no stuck disks on the Veeam Backup server or one of the backup proxies. Otherwise, the snapshots can be orphaned.

 

Nice to know…


Veeam Backup & Replication 7.0 R2 update

Written by Marco on . Posted in Veeam

 

Last week Veeam released there long awaited update to fix the support for VMware vSphere 5.5 and Windows 2012R2.

New Features

VMware 

  • vSphere 5.5 support, including support for 62TB virtual disks and virtual hardware v10 virtual machines.
  • vCloud Director 5.5 support.
  • Support for Windows Server 2012 R2 and Windows 8.1 as guest virtual machines (VMs).
  • Added ability to limit maximum amount of active VM snapshots per datastore to prevent it from being overfilled with snapshot deltas. The default value of 4 active snapshots can be controlled with MaxSnapshotsPerDatastore (REG_DWORD) registry key.

Microsoft

  • Windows Server 2012 R2 Hyper-V and free Hyper-V Server 2012 R2 support, including support for Generation 2 virtual machines.
  • System Center 2012 R2 Virtual Machine Manager (VMM) support.
  • Support for Windows Server 2012 R2 and Windows 8.1 as guest virtual machines (VMs).
  • Support for the installation of Veeam Backup & Replication and its components on Windows Server 2012 R2 and Windows 8.1.

Built-in WAN acceleration

  • Increased data processing performance up to 50% with hard drive based cache, and up to 3 times with SSD based cache. Multi-core CPU on source WAN accelerator is recommended to take full advantage of the enhanced data processing engine.

Replication

  • Added ability for source and target proxy servers to reconnect and resume replication when network connection between source and target site drops for a short period of time.

Tape

  • Added support for a number of enterprise-class tape libraries with partitioning functionality that allows presenting multiple tape library partitions to the same host.
  • Import/export slot interaction has been redesigned to add support for a number of IBM and Oracle tape libraries.

Application-aware processing 

  • Added ability for application-aware processing logic to detect passive Microsoft Exchange DAG database present on the VM, and process it accordingly.
  • Added support for Exchange CCR clusters.

User interface 

  • User interface should now remember size and positions off the main window, as well as all panels and columns.

Resolved Issues

Engine

  • Importing backup fails with the “Failed to load backup meta from file. Unexpected end of file has occurred” if VBM file size exceeds 10MB.
  • Deleted VM retention incorrectly applies to not deleted VMs when full backup transform duration exceeds the deleted VM retention period (14 days by default).
  • Removed unintentional compression (and further uncompression) of data transferred between source and target data movers when both are running on the same server, and compression is disabled in the job settings.

VMware 

  • Replication jobs should now automatically handle the situations when source VM disk size changes, instead of failing with the “Declared and actual sizes of the disk are different” error.
  • Replication jobs using Windows 2003 hot add source proxy fails with the “Client error: Failed to send command” error.
  • Very long running replication jobs may fail with the “Failed to process [srcSaveSignature]” error.
  • Re-IP addressing process may unintentionally update some reparse points found in the guest file system, which under certain circumstances may make them unusable.
  • vCenter Server version is not updated after vCenter update on the vSphere Web Plug-in configuration page of Enterprise Manager web UI.
  • vSphere Web Client plug-in shows “Access denied” error for all users that have never logged into Enterprise Manager.
  • Added ability to configure timeouts for networkless (VIX) operations via vixOperationTimeoutSec (REG_DWORD) and vixExecutionTimeoutSec (REG_DWORD) registry values in seconds.
  • Added ability to configure timeouts for Instant VM Recovery operations via IrMountLeaseTimeOut (REG_DWORD) and IrVcdMountLeaseTimeOut (REG_DWORD) registry values in minutes. The new, increased defaults are 30 and 45 minutes respectively.

Hyper-V 

  • Under certain scenarios of dynamic VHD virtual disk size increase, changed block tracking information can be reset, resulting in incremental job run having to read the entire VHD to determine the incremental changes.
  • Performing compact operation on VHD virtual disk can reset changed block tracking information.
  • Instant VM Recovery fails with the “Connection with Hyper-V mount agent was lost” error for VMs with dynamic VHD disk size not being a multiple of 256MB.
  • Failure to rescan any host makes rescan of the remaining cluster hosts to fail with the following error: “Failed to execute WMI Query
  • Instant VM Recovery fails with the “Failed to prepare VHD file for mounting” when target Hyper-V host’s storage uses native 4KB sector size disks.
  • Application-aware processing of Windows Server 2012 VM running on Windows Server 2008 R2 Hyper-V may hang for 10 minutes.
  • Removed the single VSS snapshot per cluster limit. As the results, all jobs should now correctly use the max concurrent snapshot per volume setting.
  • Replication job may fail during the free disk space check with the “Failed to extract volume mount point by the specified path” error, depending on how destination folder was selected in the replication job wizard.
  • Veeam jobs may cause non-VM owner Hyper-V nodes to log the following events to the system event log: “(eventid = 8194): “Unexpected error querying for the IVssWriterCallback interface. hr = 0×80070005, Access is denied
  • Changed block tracking functionality does not function on cluster disks.

Application-aware processing 

  • Exchange 2003 running in VM restored from Veeam backup may rename its mailbox databases after each subsequent reboot.

Backup Copy 

  • Retry cycles cause incorrect processing duration shown in the Backup Copy job report.
  • Backup Copy job reports show the inflated traffic savings ratios due to being calculated off the incorrect parameter.
  • Certain Backup Copy jobs failures result in Backup Copy job report not send immediately, but rather at the end of the copy interval.
  • A number of additional enhancements to the Backup Copy job report.

File level recovery 

  • Copy To fails on files with long paths with the following error: “The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248
  • Mount cache files should now be cleaned up from %AppData% folder automatically.

Guest indexing 

  • Guest file system catalog data processing performance was accelerated a few times in cases when the catalog contains very large amount of restore points.
  • Catalog service may take a very long time to start when there is a large data upload queue present.

1-Click VM Restore 

  • File restore operator scope settings should no longer affect ability to perform VM restore in certain scenarios with overlapping permissions.

Built-in WAN Acceleration 

  • Backup Copy job fails with the “Unable to initialize remote target session connection: all aliases have failed” after source VM disk has been resized.
  • Large Backup Copy jobs may fail with the “ERR |Invalid length of the binary data” error.
  • WAN accelerator resource is not released correctly if Backup Copy job did not copy any data during the copy interval, preventing the following copy intervals from obtaining access to WAN accelerators.
  • Under certain circumstances, the Backup Copy job health check may fail with the “Access is denied” error.

Backup from Storage Snapshots 

  • Added ability to perform multiple VM snapshot removal concurrently (according to the global concurrency setting) when multiple VMs are backed up from the same LUN by the same job.
  • HP StoreServ (3PAR) integrated jobs may fail with the “Invalid input: string length exceeds limit” error depending on the snapshot name length.

Tape 

  • Drive to library matching code does not process some symbols in the serial numbers, resulting in matching problems with certain tape libraries (for example, QUADStor VTL).
  • Added ability to disable tape device rescan completely with the DisableTapeSubsystem (REG_DWORD) registry key. This can help in situations when the same tape devices is shared between multiple backup solutions.
  • In some cases, stopping tape jobs can cause backup session not finalized correctly, resulting in future attempts to write to the same tape failing with the “Tape positioning” error.
  • The Do not protect data (cyclic overwrite tapes as required) option does not work correctly for standalone drives.
  • Tapes with multiple partitions cannot be correctly formatted.
  • Tape jobs fail with the “Tape changer alert: The library was unable to read the bar code on a cartridge” error upon receiving the corresponding alert from the tape library.
  • Certain tape device configurations cause “There is no media in the drive” error to be logged incorrectly when the media is in fact present in the drive.
  • Under rare circumstances, writing to the first tape in the media set may fail with the following error: “The INSERT statement conflicted with the FOREIGN KEY constraint “FK_Tape.file_parts_file_versions”

File Copy 

  • Linux symbolic links are not copied by file management operations and file copy jobs.

SureBackup 

  • SureBackup job may incorrectly use vCloud Director virtual machine backup when Application Group is configured by adding VM from vSphere infrastructure.
  • SureBackup does not update port group correctly when both production and virtual lab networks are located on the same Distributed Virtual Switch (DVS).

Veeam Explorer for Microsoft Exchange 

  • Launching Veeam Explorer for Exchange from Backup Browser may fail with the following error: “Error: JetError -546, JET_errLogSectorSizeMismatch, the log file sector size does not match the current volume’s sector size

Veeam Explorer for Microsoft SharePoint

  • Using localized SQL server for staging fails with the “File is a Directory” error.
  • Attachments lost during recovery when attachment path contains Unicode symbols, even though they can be seen in the user interface.

Known Issues

General 

  • Network traffic verification does not function when the Use multiple upload streams per job option is disabled.

vSphere 5.5 

  • E1000E vNIC adaptors has been found to crash ESXi host under heavy load. Avoid using them until the patch is available from VMware. For more information, please refer to VMware KB2059053.
  • Virtual appliance (hot add) processing mode does not support SATA disks. This is by design of VMware hot add functionality, which requires SCSI adaptors.
  • vSAN beta is not supported.

Hyper-V 2012 R2 

  • Offhost backup from deduplicated volume fails if Data Deduplication feature is disabled on backup proxy server, or incompatible with Windows Server deduplication version used on backed up volume.
  • Shared VHDX virtual disks backup is not supported.

Windows Server 2012 R2 

  • In Windows Server 2012 R2 environments with Windows Server deduplication feature in use, be sure to install Veeam Backup & Replication on Windows Server 2012 R2, and enable Data Deduplication feature. Windows Server 2012 deduplication driver is not compatible with Windows Server 2012 R2 deduplication, causing file level recovery errors with false data corruption reports.

R2 update is available for download starting November 13, 2013 (KB1831)

This update is cumulative and includes all fixes from Patch #1 below.

Veeam released Patch 1 for Veeam Backup & Replication v7.

Written by Marco on . Posted in Veeam

Veeam has released Patch 1 for Veeam Backup & Replication v7.

Mind this is NOT a patch to make Version 7 compatible with VMware vSphere 5.5! vSphere 5.5 is not yet supported by Veeam Backup & Replication v7.

For more information see Veeam KB1823.

So what is fixed in this patch.

Engine

  • Under certain circumstances, backup to CIFS target may fail with the following errors: “The process cannot access the file because it is being used by another process”, “Failed to create or open file \\server\share\file.vbk“.
  • In reversed incremental backup mode, retrying previously failed backup of newly added virtual disk makes the created, and the following restore points for this virtual disk invalid.
  • Linux version of standalone backup extraction tool fails to restore v7 backups with the following error: “Declared number of bank pages (XXXXX) is exceed limit (1024)”.
  • Network throttling rules are not applied if the “Use multiple upload streams option” is disabled.
  • Under certain circumstances, the reported processing rate may be much lower than actual with parallel processing enabled.
  • Added ability to manually configure more than 100 ports to be used for data movers. This may be needed when running more than 100 jobs concurrently.
  • Deleted VM retention option is now disabled by default for the newly created jobs, and its logic has been enhanced to better handle job failures.

VMware

  • Jobs fail to remove VM snapshots if vCenter server is added to Managed Servers in B&R user interface both as a Windows server, and as a vCenter server.
  • Replication jobs do not use multiple upload streams for the first processed disk of each VM. This leads to either reduced performance, or network throttling rules not applying during processing of the first disk.
  • Maximum concurrent snapshot removal settings are ignored, limiting VM snapshot removal to one at a time within a job with parallel processing enabled.
  • The source VM snapshot is now first to be removed during in replication jobs. This helps to reduce the time VM runs off snapshot.
  • Powered on source VMs are now shut down automatically to streamline self-service full VM restores.

Hyper-V

  • Instant VM Recovery fails when dynamic VHD disk size is not a multiple of 256 MB. Such disks cannot be created with Hyper-V management tools, but can appear as a result of P2V/V2V conversion, or produced by 3rd party tools.
  • Instant VM Recovery fails when target Hyper-V host’s storage uses native 4KB sector size disks.
  • Under certain circumstances, Instant VM Recovery may fail with the following error: “CVeeamFCTRegionDynamicVHD::RequestProcess: Failed to read needed portion of data, Details: IDriverRequest::DataSet: Too much data”
  • Adding a new virtual disk to the original VM makes the created, and the following replica restore point for the given virtual disk invalid.
  • Changing virtual disk exclusion settings on the existing replication job makes failover fail with the “Object reference not set to an instance of an object”.
  • Failure to rescan any host also fails the remaining hosts rescan with the following error: “Failed to execute WMI Query”
  • Restoring VHDX into the root of CSV folder makes the restored VM fail to start with the following error: “General access denied error”
  • SureBackup testing of VMs with SCVMM-specific network settings may fail with the following error: “Failed to import virtual machine VMNAME configuration”
  • Application-aware processing for guest OS without Microsoft VSS support is now skipped automatically regardless of the job’s settings.

Application-aware processing

  • Backing up all Active Directory domain controllers at the same time (which is something parallel processing made possible to achieve) may lead to DFS-R SYSVOL replication failures.
  • Changed Microsoft SharePoint farm topology collection failure event from warning to informational.

Backup Copy

  • Multiple enhancements to Backup Copy job’s email notifications and statistics.
  • Added ability to disable creating weekly full backups for GFS (Grandfather-Father-Son) retention policy.

Guest indexing

  • Temporary files produced by guest file system indexing process are not correctly removed from AppData\Local\Temp.

HP Storage Snapshots Integration

  • Initiating Active Full backup on the job with existing backups renders the created and the following backups in the backup chain unrecoverable.
  • Incremental replication runs fail with the “The method or operation is not implemented” error.
  • VM snapshots should now be created concurrently in scenarios when the job setup allows to backup multiple VMs from the same LUN. This decreases the time VMs will run off snapshot, and improves overall job performance.

SureBackup

  • Under certain circumstances, parsing route.exe output may fail. As a result, virtual lab creation process and SureBackup jobs may “hang” indefinitely.

Tape

  • Changer alerts may be incorrectly interpreted as drive alerts.
  • Tape drives appear online even when the corresponding tape library is offline.
  • When the “Continue media set” option is enabled, but the last tape is offline, the job fails with the following error: “Error Tape not exchanged”.
  • Restoring VM from tape fails with the “The directory name is invalid” error if the source VM backup job’s name contains invalid NTFS symbols (\|/*:?<>).
  • Eject media once the job finishes” option now applies to both full and incremental job runs.
  • Added support for ejecting media once the job finishes to standalone drives. Previously, attempting to use this option with standalone drives resulted in the “Automated library not found” error.
  • Tape job email report now includes barcodes and names of all tapes that were used in the corresponding job session.

User interface

  • Deleting a single imported backup that is located in the root folder of the backup repository deletes the whole repository instead.
  • Improved Microsoft Exchange server autodiscovery logic in restore to original location wizard of Veeam Explorer for Microsoft Exchange.
  • Veeam Explorer for Microsoft SharePoint crashes when desktop background changes (such as when switching to the new RDP session).

Other

  • PuTTY has been updated to the more recent version for security reasons.
  • Configuration backup fails with the “Value cannot be null” error if SMTP credentials in the email notification settings are not specified.

To obtain the patch please download it from: Download


Free NFR License for Veeam Backup Management Suite

Written by Marco on . Posted in Veeam

Veeam is giving away FREE 180-day NFR licenses for 2 sockets of Veeam® Backup Management Suite™ for VMware or Hyper-V for your home or work lab. Yes!!!
Veeam Backup Management Suite is built from Veeam Backup & Replication™ to deliver the data protection, visibility and control you need for your VMware or Hyper-V environment.

By leveraging the capabilities and benefits of virtualization, Veeam Backup Management Suite helps organizations modernize their data protection strategy and mitigate daily management risks.

Get a FREE NFR license for your lab to see the benefits of Veeam Backup Management Suite for yourself:

  • Complete visibility of your backup and virtual infrastructures
  • Advanced monitoring of backup jobs and resources
  • At-a-glance reporting and detailed documentation of backup status, performance, availability, utilization and more!

Link to the NFR request page.

And by returning a favor for this great gift, fill in the form at: http://www.surveymonkey.com/s/2013-CommChoice-WinITPro-FinalVoting

Veeam launched their new flagship, Veeam Backup & Replication v7

Written by Marco on . Posted in Veeam

Last week Veeam launched a new version of Veeam Backup & Replication v7. In this version there are a lot of new features and enhancements.


I already created a blog post about the 7 biggest features. See my blog post: http://blog.mrpol.nl/2013/05/23/veeam-announces-latest-features-v7/
For an overview of all the new things see the What’s new document.
These new Features are in my opinion the ones I like the most or looked out for the most.

Native tape support

Many organizations have invested in tape and are subject to policies requiring tape. Veeam Backup & Replication v7 offers you an easy way to copy files and backups on tape, with support for virtual tape libraries (VTLs), tape libraries and standalone drives. Files, VMs and restore points are tracked on tape, enabling you to easily locate items and making restores easy.

Built-in WAN Acceleration

Offsite storage of backups is a critical component of a good data protection strategy, and in v7, Veeam makes offsite storage easy with Built-in WAN Acceleration. Caching, variable block length data fingerprinting and TCP/IP protocol optimizations result in up to 50 times faster transfers of Veeam backups across the WAN while eliminating the need for additional backup jobs, copy scripts and capabilities like storage-based replication. Best of all, there are no VM- or host-based agents to install and no special network setup required.

High performance Backup from Storage Snapshots

Continuing what we started with the introduction of Veeam Explorer™ for Storage Snapshots in version 6.5, Veeam Backup & Replication v7 takes advantage of SAN snapshots again to offer new capabilities for powerful data protection. Exclusive innovations, including direct VMFS access and the ability to leverage VMware Changed Block Tracking, enable instant VMware snapshot commit and let you create backups and replicas from storage snapshots as often as you like and whenever you like. The result is dramatically improved RPOs for near-CDP that doesn’t impact your production environment or running VMs.

This VMware-only feature currently supports HP StoreVirtual VSA, HP StoreVirtual and HP StoreServ storage product lines.

Backup Copy jobs

Grandfather-father-son (GFS) policy. This data retention feature helps you meet data archival requirements by copying and storing full VM backups for specified intervals of time.

Engine enhancements

Parallel processing of multiple VMs. Enables better utilization and saturation of your storage infrastructure, eliminates dead time between backups and improves overall backup performance. 

Parallel processing of virtual disks within VMs. Significantly improves backup performance for VMs with multiple large disks and reduces the amount of time a VM runs off its snapshot, thus helping to reduce snapshot commit times.

Resume on disconnect. Data movers are now capable of transparently resuming communication after a network connection drops completely for a short period of time. This capability dramatically improves the reliability of remote backup and replication.

Hardware-accelerated compression. A new default compression level with a proprietary algorithm implementation leverages advanced CPU instruction sets (SSE extensions). This reduces backup proxy CPU usage up to 10 times when compared to the previous default compression level.

64-bit data movers. Backup repositories can now support appreciably larger backups. Please note versions of Windows prior to Microsoft Windows 2012 have a 16TB limit for maximum NTFS file size. This limit roughly equates to a maximum of 25TB of source data per backup job.

Backup repository overload protection. When a backup repository’s storage is overloaded with regard to I/O capacity, new tasks are not assigned to that backup repository even if task slots are available.

Reduced memory consumption. Memory consumption optimization allows 32-bit data movers to support larger backups.

Enhanced I/O pattern for data writes. Decreased fragmentation within backups files results in improved backup and restore performance, as well as reduced degradation from full backup file transformations over time.

Automatic termination of deduplication. When full backup files significantly exceed the maximum size recommended by Veeam Backup & Replication, deduplication is automatically disabled for those specific backup files. This measure is intended to increase the reliability of deployments that perhaps were not carefully planned or are not optimally configured.

Graceful job stop. Attempt to cleanly shut down all jobs when the Veeam Backup & Replication service is stopped (either manually or as a result of a computer shutdown).

Transparent backup of vCenter Server VM. The vCenter Server VM can now be backed up without having to resort to workarounds, such as adding the vCenter Server’s host by IP address and then configuring a separate job for it.

Concurrent snapshots commit operations. The maximum number of concurrent operations is now automatically limited to a preconfigured value (per standalone host or cluster). This reduces the impact of snapshot commits on your production environment. Maximum values are configured via updates to corresponding registry values.

Veeam Explorer for Microsoft Exchange

Microsoft Exchange 2013. Support for Microsoft Exchange 2013 is now available.

Restore to original location. Redesigned restores to original location are now faster and more reliable.

Restore to Office 365. Experimental support is available for restoring mailbox items to Microsoft Office 365.

Windows UI

Credentials Manager. All accounts used throughout Veeam Backup & Replication can be managed through one central dialog, allowing account passwords to be easily updated when they change.

Enterprise Manager

RESTful API. All Enterprise Manager functionality is now available through the RESTful API, enabling integration with existing web portals and other web applications.

Storage integration

HP StoreServ (3PAR). HP StoreServ is now supported by Veeam Explorer for Storage Snapshots (Fibre Channel connectivity only).

HP StoreVirtual VSA. Versions 10 and 10.5 are now supported by Veeam Explorer for Storage Snapshots.

HP StoreVirtual registration. The registration process has been revamped to avoid timeouts with large infrastructures.

Plug-in architecture. A new storage plug-in architecture is now in place to simplify the process of adding support for other storage vendors and models in future releases. This architecture is not meant to be leveraged by storage vendors directly; rather, implementation will be performed by Veeam.

Conclusion

Seeing all the new and enhanced features, definitely a good choice to upgrade to the newer version.
Links and other information.

Veeam announces latest features v7

Written by Marco on . Posted in Veeam

Last week Veeam announces the latest new big features in there new to release Veeam Backup & Replication v7 backup application.

This release is a big one with tons of new features and fixes. Veeam started in February with announcing the first new big feature and continued to do this every month. This is a summary of all the big new features with some extra information.

01. vCloud Director Support.

Veeam Reveals Enhanced Backup and Recovery for vCloud Director: Using the vCD API, Veeam will display the vCD infrastructure directly in Veeam Backup & Replication™, backup all vApp metadata and attributes, restore vApps and VMs directly to vCD, and support restore of fast-provisioned VMs. 

Solving Veeam Error: Host with uuid was not found

Written by Marco on . Posted in Veeam

Today I was upgrading an VMware ESX 4.1 host to VMware ESXi 5.1. I did the update with the VMware Update Manager. After a successful upgrade I tried to backup a VM with Veeam Backup & Replication 6.5. This resulted in an error. See screenshot.


How to solve this problem. I found this easy solution in the Veeam Support Forums.

Go to the license screen in Veeam Backup & Replication. Revoke the license of the ESXi host, and start the backup again. If all went well, the ESXi host gets a new Veeam license assigned and the Veeam backup is successful.

Problem solved!!

Extra information can be found: