Source:NDMP Backup

From SEPsesam
Revision as of 15:38, 12 February 2019 by Sta (talk | contribs) (Fixed link.)
Other languages:


Template:Copyright SEP AG en

Docs latest icon.png Welcome to the latest SEP sesam documentation version 4.4.3 Grolar. For previous documentation version(s), check NDMP.


Overview

This article describes SEP sesam NDMP backup, introduced in version 4.4.3 Grolar. For details on backing up the NetApp NFS volumes via NFS, see NetApp Volume Backup.

SEP sesam enables you to protect and manage your storage file servers by providing support for Network Data Management Protocol (NDMP). NDMP is a common protocol for backing up and restoring data on storage devices that do not support installation of a backup agent. It ensures interoperability between NAS file servers and SEP sesam, and is used as an interface that enables SEP sesam to backup various NAS appliances and copy this data to a SEP sesam Server or Remote Device Server. SEP sesam supports version 4 of the NDMP protocol. For a list of supported NAS appliances, see the support matrix.

SEP sesam provides NDMP remote configuration, where the data from the NDMP host is backed up and transferred over a TCP/IP connection to SEP sesam Server's connected storage, e.g., disk storage or a virtual tape library that is used for data protection and archiving. SEP sesam provides the SEP sesam Data Management Application (SDMA) as a session master that is responsible for initiating and managing backup and restore operations of NDMP compliant systems. Each such operation is called an NDMP session. The SDMA cannot directly access source data; it serves as a remote NDMP client. Access to the source data and the destination of the backed up data is provided by NDMP data services. NDMP data service on NDMP host provides an interface to the source – the primary data storage, e.g., filers that store "live" data on an active file system. During backup, it converts the content of the file system into a data stream and sends it via the NDMP data connection to the SEP sesam Server or RDS. Note that the backup data format depends on the NDMP appliance and user-defined configuration of this format. During restore, it reads the data stream and recreates the file system. Another NDMP data service is located on the SEP sesam NDMP host (SNDMP) and provides the interface to the SEP sesam storage. During backup, it writes a data stream to the SEP sesam Server or RDS. During restore, it reads content from data storage and converts it into a data stream in an NDMP data connection.

The SDMA establishes and manages connection and communication between the source and the destination NDMP data services. SDMA uses NDMP control connection to issue commands to the servers while the data stream flows via the NDMP data connection, thus ensuring fast and efficient communication between the remote services.

The following figure shows the SEP sesam NDMP remote configuration.
NDMP.png
As of SEP sesam 4.4.3 Tigon V2, SEP sesam supports the NDMP CAB (Cluster Aware backup) extension to perform backup and restore operations for cluster nodes running ONTAP. The NDMP v4 protocol (supported as of ONTAP 8.2 and later) is modified with a NetApp Cluster Aware Backup (CAB) extension which allows mutual communication between a backup software (SDMA) and the NDMP server. Note that for now the CAB extension is supported only in the SVM-scoped NDMP mode, which allows you to perform backups and restores at the Storage Virtual Machine (SVM), if the NDMP data service is enabled on the SVM. You can back up and restore all volumes hosted across different nodes in a SVM of a cluster. The entire cluster is available for the backup, which means that all the volumes are exposed irrespective of where they are hosted. The SDMA notifies the NDMP server which volumes need be backed up even before a data connection has been established. It enables the NDMP server to identify the node on which the volume is hosted, so when the SDMA establishes the data connection, the NDMP server ensures that it is established from the appropriate node which hosting the volume.

The CAB advantage is that it enables SEP sesam to perform a local backup or restore by connecting to the cluster management interface IP address and accessing all the volumes belonging to all storage virtual machines of the cluster over this single interface.

Information sign.png Note
If CAB is not supported, then backup/restore can be performed only over the IP address of the cluster physical node on which backup source volume is located.

Key features

Supported backup formats

The NDMP protocol does not specify a backup data format, it depends on the used NDMP applliance. For a list of supported NAS appliances, see the support matrix. SDMA is implemented on all supported Linux and Windows systems.

The following backup formats are supported depending on appliance:

  • dump (default): Back up files and directories to tape. It is not deduplication- and compression aware and does not preserve deduplication when backing up data.
  • smtape: This format is only supported by NetApp. An image level backup that backs up blocks of data of the origin volume. It is deduplication aware and preserves dedup when backing up and restoring data. Does not support file history or direct access recovery (DAR), thus providing only recovery of a complete volume (including snapshots).
  • zfs: This format is only supported by Oracle Solaris ZFS. Share-based for both file systems and volumes. Does not support file history or direct access recovery (DAR), but may be faster for some datasets. It is only possible to restore the whole file system or volume (selective restore is not supported).

For example, with NetApp you can choose between the normal NDMP dump and smtape backup.

SEP sesam NDMP integration advantages

  • The NDMP protocol allows for backup and restore operations of heterogeneous filers and excludes the necessity of mapping a drive on an alternate server and then backing up.
  • SEP sesam provides centralised and automated backup and restore of data residing on remote NAS systems that do not have locally attached libraries.
  • NDMP backups can be stored to all SEP sesam supported storage devices. Si3 deduplication can also be used for NDMP data and can significantly reduce the size of NAS NDMP backups.
  • SEP sesam allows storage sharing between different NAS systems as well as combined NDMP and non-NDMP backup, thus optimizing storage space.
  • All backup levels (COPY, FULL, INCR, and DIFF) and selective restore from dump backups are supported.

NDMP restrictions

  • NDMP data can only be restored to the same vendor and NAS device running the same or higher OS version as the NAS device that was used for backup.
  • NAS deduplicated or compressed data is uncompressed before being sent across the network to a SEP sesam Server or RDS in uncompressed form.
  • NDMP backups might be time-consuming in large NAS configurations with large volumes.
SEP Tip.png Tip
Direct access recovery (DAR) is used during backup to enable restoring a single directory or file from a backup (selective restore). As of version 4.4.3 Grolar, DAR is enabled by default (no configuration is required). If required, it can be disabled. For details, see section dump backup.

Prerequisites

  • SEP sesam Server version 4.4.3 Grolar or higher. Note that backup and restore procedure are different for versions 4.4.3 and 4.4.3 Tigon. For details, see NDMP Backup 4.4.3 and NDMP Restore 4.4.3.
  • NDMP must be configured on your NAS appliance. Refer to your NAS appliance documentation for configuration instructions. Some basic information are provided below in the section Configuring NDMP host.
  • You must ensure the bidirectional communication between the SDMA and NAS. The NAS appliance will attempt to connect to SDMA, which needs access to port 10000 on the NDMP data server (NAS) and on the SDMA host system. Once the connection is established, unlimited ports must be opened to allow the transfer of the information to SEP sesam. Therefore, there must be no firewall between NAS and SDMA.
Information sign.png Note
NDMP uses port 10000 by default, but the server may listen on a different port. To specify a different port for the NDMP server, enter the following option in the backup task properties -> tab Options -> Save options field:
-a port=<value>
  • It is recommended that 10 gigabit Ethernet (10GbE) is available for larger NDMP backups.
  • The target media pool for backing up your NAS device must have sufficient space to contain the NAS data, which is always backed up to the pool uncompressed.
  • An additional NDMP license is required for SEP sesam with NDMP. For details, see List of Licenses.

Installing SEP sesam NDMP backup package

SEP NDMP module is already included in the installation package. SEP sesam NDMP (sbc_ndmp) must exist on the system which serves as your data mover. For details, see SEP sesam Quick Install Guide.

SEP sesam with NDMP requires an additional NDMP license. See List of Licenses. For more information, contact SEP sesam support at support@sep.de.

Configuring NDMP host

NDMP configuration involves several steps, including authentication with the NDMP server. Typically, you should use the root password or non-root user with a system-generated, NDMP-specific password. Refer to your NAS appliance documentation for configuration instructions!

NDMP configuration steps for NetApp are described in NetApp-specific NDMP configuration.

Adding NDMP host as a SEP sesam client

If you want to protect your NDMP-ready system, you must configure your NDMP host as a SEP sesam client.

  1. From Main Selection -> Components -> Topology, select the location of your choice, e.g., Storage systems, and click the New client button. The New client window opens.
  2. Enter the client Name. From the Platform drop-down list, select NDMP. From the Operating system drop-down list, select your system, for example NetApp. Then select PROXY as the Access mode for the SEP sesam server-client communication.
  3. NDMP new client 4.4.png
  4. Click the newly displayed NDMP Access tab. Enter the NDMP username and password. From the Data Mover drop-down list, select the system where the SDMA is running.
  5. NDMP access.png
  6. Click OK to add your NDMP host as a client.

Configuring NDMP backup

For NAS backup, the SEP sesam Server acts as a backup session master responsible for managing media, scheduling and monitoring NDMP backup and restore. To configure the NDMP backup, you must create a backup task (select the data to be backed up), then set up a backup schedule (specify when you want to back up your data) and link a backup event to it (specify how to back up the data and where to back it up to). For general details on how to configure a backup and its prerequisites, see Standard Backup Procedure. However, configuring an NDMP backup differs slightly from the standard procedure and requires some NDMP-specific steps, which are described below.

Depending on your appliance, you can configure a traditional NDMP dump backup or smtape backup (NetApp specific option). The backup task creation is similar for all NDMP backups, the main difference is that you must define additional arguments for smtape backup in the backup task properties -> tab Options -> Save options field.

Direct access recovery (DAR) index is mandatory to restore a directory or a single file from a backup (selective restore). As of version 4.4.3 Grolar, DAR is enabled by default (no configuration is required).

Information sign.png Note
You have to create a separate task for each volume you want to back up. With NDMP implementations which allow to set a directory as a backup source, you need a separate task for each directory that is selected as a source.

Dump backup

  1. From Main Selection -> Tasks -> By clients, select the NDMP host to be backed up and click New backup task. The New backup task window opens.
  2. Select the Source. Click the Browse button (big arrow) and select the volume you want to back up. NDMP bck source volume.jpg
  3. SEP Tip.png Tip
    Typically, NDMP implementations only allow to select/display volumes, therefore SEP sesam cannot expand the tree structure to display and select the underlying directories. However, some implementations allow to specify a directory as a backup source manually, while other allow to exclude files and folders from backup (see step 4). See examples below for details.

    In the Source field, you can manually enter the directory to be backed up for the following NDMP implementations:

    Clustered Data ONTAP (NetApp)
    <storage_virtual_machine_name>/<volume_name>:/<directory>
    
    7-Mode Data ONTAP
    /vol/<volume_name>:/<directory>
    
    EMC VNX
    EMC NDMP implementation does not support backing up individual files and folders. You can only exclude files and directories, as described in step 4.

    NDMP bck source directory Grolar.jpg

  4. If you have browsed the source, the task type and task name are set automatically. If you have entered the source manually, enter the name of the task in the Task Name field.
  5. Depending on NDMP implementation, you can manually exclude files and directories from backup. Switch to the Options tab and under additional call arguments (Expert options) in the Save options field specify the files and directories to be excluded by using the relevant syntax:
  6. Clustered Data ONTAP (NetApp) and 7-Mode Data ONTAP
    Specify the file names or patterns which you do not want to back up with the exact name. You must use integrated common parameter -x to work as exclude list, for example -x /dir1,/dir2. Note that a file or directory name must be preceded with a forward slash "/" and followed by a comma (without space!), if there are more items listed. The exclude list can contain up to 32 names. You can also use an asterisk (*) (a wildcard character) as the first or the last character of the string; each string can have up to two asterisks. For example, to exclude directories smallfiles1, smallfiles2, smallfiles3, smallfiles4 ... or 1data, 2data, 3data:
    -x /smallfiles* -x /*data
    To exclude dir1, dir2 and dir3, use:
    -x /dir1,/dir2,/dir3
    If your backup source is a directory (and not the whole volume), e.g., /vol/cifs1:/Home/Base, and you want to exclude one of its subdirectories, do NOT specify the whole path for the exclude, but only the directory that you want to exclude. For example, if you have specified a backup source /vol/cifs1:/Home/Base and you want to exclude the subdirectory /vol/cifs1:/Home/Base/subdir1, specify the exclude only with the subdirectory name and not with the whole path:
    -x /subdir1
    EMC VNX
    You have to specify the following variables to sbc_ndmp: EMC_EDIRxx and EMC_EFILExx (where xx needs to be replaced with increasing number, for example EMC_EDIR01=/cifs_3gb/images, EMC_EDIR02=/cifs_3gb/music etc.). If there is more than one file or directory to exclude, they must be enumerated from number 01; a max of 50 variables can be specified (EMC_EDIR50 and EMC_EFILE50 are the highest possible variables). You can also use the wildcard characters to specify the file or folder patterns. Example:
    -a env=EMC_EDIR01:/cifs_3gb/folder1,env=EMC_EDIR02:/cifs_3gb/folder2 NDMP backup exclude VNX Grolar.jpg
  7. By default, SEP sesam uses direct access recovery (DAR) to enable selective restore. While DAR can greatly reduce the time it takes to restore individual files, it might be unnecessary in case you want to backup and consequently restore only the whole volume. In such case, you may disable DAR to improve backup performance by entering the following option in the Options tab -> Save options field:
  8. -o list=none
  9. Save your task.
  10. Configure a schedule for your backup task, as explained in Creating a schedule. Once you have configured a schedule, you must create a new backup event for it.
    SEP Tip.png Tip
    You can also add your backup task to an existing schedule by double-clicking the backup task, selecting the tab Schedules and adding it to one or more schedules.
  11. Go to Main Selection -> Scheduling and select the schedule you created for NDMP backup, then click New Backup Event. Select one of the available backup levels (COPY, FULL, DIFF, or INC) from the Backup level drop-down list. For more information on configuring a backup event and available options, see Creating a backup event.

smtape backup

smtape (SnapMirror-To-Tape) is only supported by NetApp. It is an image-based high-performance disaster recovery solution from Data ONTAP that backs up blocks of data of the origin volume. It is deduplication aware and preserves dedup when backing up and restoring data.

  1. Create the task as described in the above procedure. Before saving the task, click the tab Options. In the Save options field, enter:
  2. -a ndmp_tape_format=smtape
  3. Save your task.
  4. Configure a schedule and create a new backup event for it: Select the schedule you created for NDMP backup, and then click New Backup Event.

You can view the status of your NDMP backup job by selecting Last backup state in the Main selection window. The backup status overview provides detailed information on the last run of backup jobs, including the task name, start and stop time of the last backup, backup level, data size, throughput, assigned media pool, etc.

If you have any problems with NDMP backup, check the Troubleshooting Guide.

What is next?

NDMP Restore