Differences between IBM Spectrum Protect (TSM) and EMC Avamar backup tools

EMC AVAMAR Architecture Basic Introduction

• Avamar is a backup software tool acquired by EMC in November, 2006. This was EMC’s second major acquisition of enterprise level backup software, having acquired Legato in July of 2003.

• Avamar uses an incremental forever architecture supported by policy based management with versioning, expiration and reclamation.  It can simulate a traditional backup plan that appears to require full periodic backups. 

• It encrypts data in transit with 128 or 256 AES bit or proprietary encryption for data in transit and optionally at rest, compression, central administration and a dashboard. 

• The Avamar server runs on a single server for small environments or scales up by adding additional systems and redistributing workload with automatic load balancing.

• Avamar is based on a grid architecture that stores only a single instance of any unique data for all clients that use the grid servers.  The Avamar client (on the source server that hosts the application) scans its filesystem to identify new or changed files. The client then segments the data and uniquely identifies it using a SHA-1 hashing algorithm. If the client identifies that data segment is new the client sends the hash, called a segment ID, to the Avamar server where it checks if the data segment is new or already stored.  If the data is new to the Avamar server it asks the client to send the data.  
  

• The Avamar server grid stores the data on a node within the grid based on the unique ID.  Avamar is a distributed index based architecture and does not maintain a single meta database, rather it uses the self describing organization of the unique ID’s. The Avamar server takes checkpoints which are snapshots in case a roll back to a previous state is required. 

• Networker 7.4 SP 1 integrates with Avamar by offering a single agent that can be used for traditional backup using Networker or as an Avamar agent for backup with deduplication.  Networker also provides configuration and management of the Avamar client through the Networker Management Console.

• When an Avamar agent is used the data is backed up to an Avamar server, it is not backed up to a Networker server.  When the traditional Networker client is used to back data up to a Networker server Avamar is not used as a backend data store for Networker. Avamar integration with Networker is currently limited to agent integration and administration.     

Differences between IBM TSM and EMC AVAMAR Backup Software Tools

• Large, high transaction databases >1TB
• >10% daily change rates
• Encrypted, Compressed, or “Noisy” Data
• Image Files, Videos, X-rays 

• Avamar requires additional hardware and administrative expense for a second backup infrastructure assuming tape support is required either for lower cost disaster recovery or off site storage for long term archive. Whereas IBM TSM’s integrated tape support eliminates the need to create and maintain a second infrastructure to archive to tape, a capability used in most installations.

• Avamar requires a minimum of a three server configuration for full function.  Separate dedicated systems are required as “Access Nodes” for the database search function and also a separate dedicated system is required for the NDMP Accelerator for backup of Network Appliance filers and it appears an additional dedicated system is also required for backup of EMC Celerra. Whereas TSM only needs a single server to provide its services and can take advantage of servers with multi-core processors for higher levels of scalability and performance. 

• Scaling Avamar requires substantial additional systems.  The system that stores the de-duplicated data (“storage node”) is limited to storing 3.3TB of data and is limited to an 18 node system with 52.8TBs of data in standard form. One sophisticated and well supported Avamar user has reported that Amamar tops out at 12 to 13 TB in a cluster (Avamar ‘grid”).   

• TSM scalability of its disk storage pools has been demonstrated to 30TB using a single TSM server. Differences in the environment, such as average file size and dedupe chunk size, can vary scalability substantially but in this fairly typical TSM environment a TSM metadatabase of 368 GB db supported the 30TB of data in a deduplicated storage pool. In typical operational environments a single TSM server using deduplication is estimated to have scalability several times this size. 

• Avamar cannot directly write to tape. Avamar itself does not write data directly to tape, rather it uses its Data Transport function to send data to a third party backup product that can then write the data to tape. The Data Transport function is limited to the  use of either Networker or Symantec NetBackup running only on Solaris, Linux or Windows servers (no AIX or HP-UX support) to write the data to tape, requiring a second backup infrastructure to buy, plan, install, operate and maintain. Data Transport is limited to Avamar grids up to 22 TB, which reduces the theoretical scale down for an Avamar Grid from 52.8 TB by nearly 60%.

• TSM has robust tape support from its many server platforms and its use does not severely limit its scalability and, unlike Avamar, provides the function on TSM avoiding the need for a buy, plan, install, operate and maintain a second backup infrastructure.  

• Avamar has no HSM to offset its scalability limits. Avamar is a disk only backup/restore process and cannot minimize the amount of tier 1 disk used to store unique data. The capacity of the storage node could be stretched by limiting the number of versions of files kept, reducing the ability to recover further back in time. 

• TSM has built in hierarchical storage management and allows the use of multiple levels of disk, tape and optical storage to provide different levels of service at the minimum expense for storage media.    

• Avamar has no built-in Archive support, instead of providing built-in support for archiving Avamar has an extra cost option that allows Avamar to be the back end data store for EMC Centera, which is of limited value since Centera has native file level deduplication. 

• TSM has built in archive support and does not require an expensive additional piece of hardware, such as EMC Centera, to deliver archive support.

• Avamar lacks the ability to actively participate as part of an automated Information Lifecycle Management Process. Applications server and workstations often build up unneeded files over time  that consume disk space on both the primary and backup server.

• TSM can use automation to archive files identified by Tivoli Storage Productivity Center (TPC) so they don’t take up space on the application system.      

• Avamar has a less than robust core infrastructure, database checkpoints are taken twice a day and allow roll backs to a previous point in time but there is no  database transaction logging with a roll back/forward recovery process. Falling back to the last checkpoint, where only two checkpoints are taken per day, creates a potentially large recovery exposure where the latest backups are unavailable.

• TSM uses DB2 and provides the highest level of data integrity to maintain its meta data base with transaction logging and a roll back/forward recovery processes.    

• Vaulting of TSM database, recovery log, volume history information, device configuration information, DRP file (if using Disaster Recovery. Manager)  and copy pools for storage at an offsite location.

• Use of TSM server-to-server communications to enable enterprise configuration (multiple TSM servers), enterprise event logging and monitoring, and command routing. 

• TSM servers installed at multiple locations, optionally setup as peer to peer servers (that is, each server able to recover at the alternate site). 

• Use of TSM virtual volumes over TCP/IP connection to allow storage of TSM entities (TSM database backups, recovery log backups, and primary and copy storage pools, DRM plan files) on remote target servers. 

• Use of high bandwidth connections and data replication technology (such as IBM PPRC, EMC SRDF) to support asyschronous/sychronous data replication of TSM databases backups, recovery log backups, TSM database and recovery log mirrors, and storage pools. 

• Use of remote electronic tape vaulting of TSM database and recovery log backups, primary or copy storage pools. Extended distances can be achieved by using distance technologies, for example, extended SAN, DWDM, IP/WAN channel extenders.

• Avamar lacks application support for SAP and Informix. 

• TSM has wide application support including support for SAP and Informix.  

• Avamar lacks support for disk array copy services (EMC snap or IBM flashcopy).

• TSM supports IBM Flashcopy for Exchange, DB2 and MySAP on IBM San Volume Controller (SVC), IBM DS6000 and DS8000.    

• Avamar recommends their customers to use the newest API to backup VMware.  The use of the new VMware vStorage API for backup was needed by Avamar to overcome  problems with it prior options for VMware support.  Many customers have only recently just shifted to the use of the VMware Consolidated Backup (VCB) for backups and are again likely to take a conservative approach to adoption of the new technology, preferring instead to let bleeding-edge customers pilot the new vStorage API for Data Protection from VMware. Like other backup products Avamar is carrying forward the use of the popular VMware Consolidated Backup (VCB) API but it creates the unacceptable administrative burden of requiring that each virtual machine (VM) must have its own group  to schedule VCB backups which must be setup and maintained one at a time. At one point to bypass the VCB admin issue EMC was recommending the use of the Image backup based on an agent running in an ESX service console, even though this posed a serious performance issue for the service console. 

• TSM supports the new VMware vStorage API. TSM also continues to support the more popular VMware VCB without the administrative burden.    

• Avamar is a security exposure. The Avamar File System is not able to limit file access on a user-by-user basis. All files stored on a server are viewable by any user with access to Avamar File System.

• TSM has client level access for restores to provide data security.

• discover  client maintenance levels on the TSM FTP site
• download needed packages and store them on the TSM server
• manage packages stored on the TSM server (retention, etc.)
• select a maintenance level and distribute to a list of clients.  The code will then be distributed based on a predefined schedule.
• review the client distribution status

• The superficial integration that Avamar has with Networker is an attempt to cover up the inherent disadvantages of having two separate code bases that must be installed and maintained: a legacy product that couldn’t reasonably he enhanced to support built in deduplication and a deduplication product that does not scale well enough to even be used  as a back end data store to provide deduplication for the legacy enterprise data center product. The agents are packaged together and not integrated. Avamar needs its own management facilities and can’t run just using the Networker management function. The monitoring and reporting function Networker provides was so weak that EMC had to acquire Wysdm to support Avamar.  Contrast that with Tivoli’s built-in deduplication which is truly integrated with a multitude of advantages starting with eliminating the need to understand, deploy and manage two completely different infrastructures and ending with eliminating the need to bring in yet a third product to do centralized reporting. 

• Avamar deduplication to tape has many limitations. EMC recommends its use only for infrequent long term data retention use cases for performance reasons. It does not match the ability of TSM to mix the use of tape and disk in normal operational environments where restores are likely or frequent.

• Avamar itself does not store the data to tape, it requires the use of either Networker or Symantec NetBackup running only on Solaris, Linux or Windows servers (no AIX or HP-UX support) to write the data to tape after it is deduplicated by Avamar, requiring a second backup infrastructure to buy, plan, install, operate and maintain. 

• Avamar Lacks continuous data protection. Avamar has been positioned by EMC to backup frequent point in time copies by CLARiiON disk snapshot (like IBM Flashcopy) the solution is extremely expensive and still does not provide the frequency of data protection provided by TSM FastBack.

• Tivoli FastBack has (real time replication) continuous data protection which saves data throughout the day, greatly reducing the amount of data lost compared to the typical use case of nightly backup by Avamar.

• Avamar restore time is lengthy and is not appropriate for application that require higher levels of availability.

• Tivoli FastBack’s near instant restore does not require the restoration of data from the backup copy to a different disk accusable by the application server, rather with FastBack, the backup data becomes the production data, nearly instantly.

• Avamar lacks integrated bare machine recovery, requiring the use of  a separate product such as EMC HomeBase for restoring the Avamar server in case of disaster.

• TSM FastBack has integrated Bare Machine Recovery.