Source:HCL Domino Server Backup

From SEPsesam
Revision as of 12:01, 26 November 2020 by Sta (talk | contribs) (Minor correction.)
Other languages:

Copyright © SEP AG 1999-2024. All rights reserved.

Any form of reproduction of the contents or parts of this manual is allowed only with the express written permission from SEP AG. When compiling and designing user documentation SEP AG uses great diligence and attempts to deliver accurate and correct information. However, SEP AG cannot issue a guarantee for the contents of this manual.

Docs latest icon.png Welcome to the latest SEP sesam documentation version 4.4.3. Beefalo/Beefalo V2. For previous documentation version(s), check IBM Domino server archive.


Overview

Template:Domino rebrand en/en

SEP sesam Client SBCLN is a program library for backing up IBM Domino (previously Lotus Domino) databases. To perform IBM Domino database-level backups, you have to enable Transaction Logging in Archive mode on the IBM Domino server. Note that Domino databases can also be backed up with transaction logging disabled, however, this is not recommended as it only allows for FULL backups of databases and requires much more frequent full database backups. For details, see Configuring an IBM Domino server client as a SEP sesam Client.

Another important point in planning a backup strategy is considering a valid DBIID (Database Instance Identifier). Transactions only refer to information with an identical DBIID of a database. If tools, such as fixup or compact are used on databases, the database usually gets a new DBIID. New DBIID is assigned and the changes are logged to the database automatically, however, you have to run a full database backup after the console message Assigning New DBIID for DB xxxx appears. Neither planning nor operating these IBM Domino tools is part of SEP sesam.

For additional information on transaction logging and DBIID, see IBM FAQ about Transaction Logging, Transaction Logging on Domino Servers and Notes/Domino Best Practices: Transaction Logging.

Note that the Domino API does not offer direct backup or restore options for single emails or documents. It is designed more like a relational database and is set up for disaster recovery. Single mail recovery is possible but requires a profound knowledge of IBM Domino administration. For details, see Restoring single mails.

Backup level

Information sign.png Note
The INCR and DIFF behaviour has changed compared to previous SEP sesam versions < 4.4.3 Beefalo, where DIFF has backed up only new NSF files and INCR only relevant transaction logs; hence, INCR and DIFF were both required to save the changes. As of v. 4.4.3 Beefalo, DIFF and INCR backup levels are enhanced as explained below. If the database instance ID (DBIID) of a database has changed or there is a newly created NSF database file, then consequently the next DIFF or INCR automatically performs a backup of transaction log files and database files with the changed DBIID and/or newly introduced database files. This way your backups continue to remain in use without affecting the integrity of the backup and without having to perform a new full backup again, as was the case in previous versions of SEP sesam.
    FULL: The Domino databases selected in the backup source are backed up completely. The source all means that all files in the Domino data root, which are identified as Domino files, are backed up by using the Domino API. Full backup includes database files and all backup-relevant transaction logs (TXN-files).
  • DIFF: DIFF saves all IBM Domino files that have been changed from the last FULL or DIFF backup.
  • In case of a NSF (Notes Storage Facility) file, the following will be checked:
    • All newly created NSF database files which have not been backed up yet.
    • NSF files with changed database DBIID, which has changed as a result of some Domino database maintenance activities, such as using fixup (e.g., automatic fixup of corrupt databases) or compact.
    • NSF files with modified file time; in this case, NSF files are checked for any changes since the last DIFF or FULL backup (-T {since} argument).
  • INCR: INCR saves all relevant transaction logs, all new IBM Domino files and IBM Domino files with new DBIID. You can specify only all as a backup source for incremental backups.
  • COPY: Same as a full backup, without consideration of the transaction logs.

How does a Domino online backup work

When performing an IBM Domino online backup, files with Domino relevant filename extensions (e.g., .nsf, .ntf, .dic, .box, .dsk, .id, etc.) are backed up over the Domino API at the operating time of the Domino Server.

Information sign.png Note
notes.ini is not directly part of the IBM Domino (Notes) data files, however, it contains information and parameters about the Domino installation and configuration. SEP sesam online Domino backup tries to detect the location of notes.ini, first in the binary path of the Domino installation, then in the Domino data root. Once notes.ini is detected, it is backed up automatically with the Domino files. Otherwise a warning is issued in the backup protocol. In this case the administrator has to make sure that notes.ini is backed up with a path backup.

If a backup task with task type Lotus Notes, backup level Full and backup source all is started, SEP sesam Client connects with the Domino API and requests a backup of all files relevant to IBM Domino. In previous SEP sesam versions, only the databases which were open were backed up with temporary copy (and applying recent changes), however with version 4.4.3 Beefalo all databases are backed up with temporary copy which may affect backup performance. Additionally, processing backups on Linux differs slightly from processing on Windows where two concurrent threads are introduced with version 4.4.3 Beefalo.

  • On Linux, Domino takes the first database, copies it to the temporary SEP sesam path (gv_rw_tmp in PATHES section in the sm.ini) to the file system and then checks if any transactions are still in queue. If they are, first the transactions are executed and then the database is backed up by SEP sesam Client. This way all IBM Domino (Notes) databases are transferred from Domino to the file system and backed up sequentially by SEP sesam Client. To perform a successful backup, make sure that the temporary storage space is at least as big as the largest Domino database.
  • On Windows, up to 10 database files are copied to the local storage by using two threads to process these databases. The first thread is responsible for making a local copy of the database file (e.g., .nsf file). At the same time, the second thread is transferring the data and then removing the temporary file. While the use of multiple concurrent threads is more efficient and results in faster processing, it also requires more free space to store all database copies locally. To prevent backup from failing, make sure that required free space is available in the temporary storage, by default gv_rw_tmp in <SESAM_VAR>/ini/sm.ini. If there is insufficient temporary space, change the path in sm.ini to a directory with sufficient space.
  • SEP Tip.png Tip
    Windows only

    In case there is not enough free space available in the temporary storage, you can reduce the amount of databases to be copied by specifying -a threads=<number_of_databases> in the New Backup Task -> Options tab -> Backup options field. For example, if you want that only 5 databases are used, specify -a threads=5

    IBM reduce databases.jpg.


Information sign.png Note
If your IBM Domino databases use DAOS, check backup considerations for DAOS in section DAOS backup.

When restoring over the Domino API, the database that is being restored travels the same way back. The database file is initially put into the temporary SEP sesam space in the file system by SEP sesam and then transferred to the Domino API.

Configuring IBM Domino backup

Create two backup tasks to back up the IBM Domino databases. Note that backup tasks for IBM Domino are set as Lotus Notes task type in the GUI (except for the DAOS backup!).

First, you have to create a backup task for full backup. After the successful full backup, you have to create another backup task for incremental backup. If the Transaction Logging with the Archive mode is enabled (as recommended), the incremental backup will back up the transaction log files, new IBM Domino files and IBM Domino files with new DBIID. Note that it is only possible to specify all as a backup source for incremental backups.

Steps

  1. From Main Selection -> Tasks -> By Clients, select IBM Domino client and click New Backup Task. The New Backup Task window opens.
  2. Specify the Source. You can either browse for the databases that you want to include in the backup or enter their names followed by a comma. Note that by browsing the source (not entering it manually), the task name is set automatically. You can enter the following patterns as a backup source:
    • all: You have to specify all as a backup source for full and incremental backup. Note that you will set the desired backup level later, when creating a backup event. For details, see Creating a Backup Event.
      IBM Domino bck task.jpg
    • set directories: You can browse or enter specific directories that you want to back up. In our example, we will specify Domino file directory E:\notesdata for Windows and /srv/notesdata for Linux:
      • Enter the path manually to specify the directories that have to be backed up. In this case, all Domino relevant files and their subdirectories are backed up. Contrary to standard path backups of Linux clients, IBM Domino backups on Linux also follow the directories beyond file system boundaries and symbolic links.
        • Examples on Windows:
        • a) E:/notesdata/mail b) E:/notesdata/mail1,E:/notesdata/help
        • Examples on Linux:
        • a) /srv/notesdata/mail b) /srv/notesdata/mail1,/srv/notesdata/help
      • Backup with file pattern, e.g., all databases from the mail directory on Windows:
      • E:/notesdata/mail/*.nsf or all templates from the help directory on Linux: /srv/notesdata/help/*.ntf
      • Backup of single files:
        • Example on Windows:
        • E:/notesdata/mail/user1.nsf
        • Example on Linux:
        • /srv/notesdata/mail/user1.nsf

      IBM Domino selective bck.jpg

  3. Optionally, specify the files or patterns which you do not want to back up in the Exclude list field. Depending on your operating system, proceed as follows:
  4. Exclude list on Windows On Windows, specify the exclude list using a slash (/) as path delimiter, a backslash (\) is not supported. For example, if you want to exclude Domino mail from the backup, enter E:/DominoData/mail in the Exclude list field. Exclude Domino Windows.jpg
    Exclude list on Linux/Unix On Linux/Unix, specify the exclude list with the preceding backslash and a dot, e.g., \./srv/notesdata/help$'.
    Information sign.png Note
    In older SEP sesam versions < 4.4.3.45, the exclude list that is defined on Linux/Unix for the IBM Domino backup task, does not work if specified with \.. In this case, specify the exclude list without the preceding backslash and a dot, e.g., use /srv/notesdata/help$ instead of \./srv/notesdata/help$'.
    • Exclude list on Linux/Unix > v. 4.4.3.45:
    • Exclude list Linux2 Beefalo.jpg
    • Exclude list on Linux/Unix < v. 4.4.3.45:
    • Exclude list Linux1 Beefalo.jpg

    For details on various exclude options, see Creating Exclude List.

  5. Click OK to create the task.
  6. If you want to start the newly created task immediately, right-click the name of the task and click Immediate Start. Otherwise, you have to schedule the task by clicking New Schedule under Main Selection -> Scheduling -> Schedules. For details, see Creating a Schedule.
  7. Once you have configured a schedule, you must create a new backup event for it. For details, see Creating a Backup Event.
Information sign.png Note
  • You have to run a full database backup after the failure of an incremental job or if an incremental backup is corrupted or deleted, because this will break your backup chain. You need to start a new chain by performing a full backup.
  • Whenever a database receives a new DBIID automatically, you have to perform a full database backup after the console message Assigning New DBIID for DB xxxx appears.

Configuring access rights to the IBM Domino server on Linux

By default, the values for the user that runs the Domino service on Linux and the path of notes.ini server can be stored in the section [ENVIRONMENT] in sm.ini on the SEP sesam Client.

If the Domino user is not notes or differs from the standard value, it is important that an authorized user with access rights to connect internally to the IBM Domino server is specified for every backup task that is designated to back up data on the respective IBM Domino server. To specify the user, enter it in the New Backup Task -> Options tab, in the Backup options field (for the backup) and Restore options field:

 -a USER={notes-user}

If notes.ini is not located in /local/notesdata, enter the location of the notes.ini file in the backup task properties -> Options tab, in both the Backup options and Restore options field:

 -a NOTESINI={complete path the notes.ini file}

For example:

 -a USER=nadmin -a NOTESINI=/srv/notesdata/notes.ini

IBM Domino Linux options.jpg

Information sign.png Note
You have to enter this information for every backup task that is designated to back up files on the respective Domino server with the task type Lotus Notes.

DAOS backup

DAOS (Domino attachment and object service) was introduced in IBM Domino (previously Lotus Domino) v. 8.5.0. It allows the mail attachments to be stored outside the NSF file in Notes Large Object (.NLO) files. By transferring the attachments to a file system in your server and allowing to preserve a single copy rather than multiple copies of the same attachment, it preserves disk space. This means that the NSF database is no longer self-contained, therefore an additional backup of the .NLO data is required. For more details on DAOS, see IBM documentation, e.g., IBM Lotus Domino Attachment and Object Service (DAOS) deployment guide and The new IBM Lotus Domino attachment and object service.

Requirements

  • DAOS must be activated for the databases on an IBM Domino server that use DAOS, and the consolidation must be enabled in the server document (DAOS tab).
  • Additionally, every Domino database must have the Use Domino Attachment and Object Service advanced database property selected.

Backup considerations

SEP sesam automatically backs up DAOS configuration files daos.cfg and daoscat.nsf by regular Lotus Notes task type. However, you should be aware that IBM strongly advises not to back up the daos.cfg and daoscat.nsf files, because restoring the old copies of these index files (which would not match the existing NSF and NLO environment), can cause discrepancies and lead to data loss. The preferred method is to re-create these two files from the DAOS repository and the NSFs participating in DAOS, if required. For more details, see the IBM article DAOS Backup and Restore.

A separate backup is required to back up the NLO data. Consider the following for the .NLO backup:

  • If the DAOS path is set up under the Notes data path (e.g., E:\notesdata\DAOS), all DAOS files (.NLO) are backed up together with the other Notes files.
  • If the DAOS repository is configured outside of Domino's data visibility, you have to create an additional path backup and select the DAOS data location as a backup source. For details on creating a regular path backup, see Standard Backup Procedure.
Information sign.png Note
It is recommended that you set up a SEP sesam path backup to back up .NLO files in your DAOS repository regularly, e.g., together with FULL/DIFF/INCR Lotus Notes backup task. For details on creating a regular path backup, see Standard Backup Procedure. You should back up all the NSF data before backing up the .NLO files to ensure that the referenced .NLO files exist.

For details on backup recommendations, see IBM articles Backing up servers that participate in attachment consolidation (DAOS) and DAOS Backup and Restore.

Backing up IBM Domino partitioned server

A Domino partitioned server is a Domino server where several Notes instances are running. All running instances use the same (shared) program files, but every instance has its own data path and notes.ini. From the network view every Domino instance can run on a separate IP address; when they run on the same IP address, they can use different ports, depending on the Domino configuration. For further information on the installation and configuration of an IBM Domino partitioned server, see the relevant IBM documentation.

To back up a Domino partitioned server environment, you have to create a backup task for each partition. SEP sesam backup references the notes.ini of the corresponding partition directly.

Steps

  1. From Main Selection -> Tasks -> By Clients, select IBM Domino client and click New Backup Task. The New Backup Task window opens.
  2. Enter the Task name, for example notus05_notes_DataA.
  3. From the Task type drop-down list, select Lotus Notes.
  4. Specify all as the backup Source.
  5. Switch to the Options tab. In the Backup options field, enter the corresponding notes.ini, e.g.:
  6. -a NOTESINI="D:\IBM Domino\Data A\notes.ini" IBM Domino bck task options.jpg
  7. Click OK to create the task.
  8. If you want to start the newly created task immediately, right-click the name of the task and click Immediate Start. Otherwise, you have to schedule the task by clicking New Schedule under Main Selection -> Scheduling -> Schedules. For details, see Creating a Schedule.
  9. Once you have configured a schedule, you must create a new backup event for it. For details, see Creating a Backup Event.

For every partition you want to back up, you have to set the client name where the Domino program is running. Even though every Domino partition has its own IP address and DNS name, the backup job will reference the native program host. In the example above, the Domino Server itself has the DNS name notus05 and the partitions have the names notus05a and notus05b, but you always reference the notus05.

Using IBM Domino dbbackup utility

SEP sesam provides the Domino backup utility dbbackup which allows to back up a database by storing the database copy into a file.

The following dbbackup arguments are displayed with the dbbackup usage:

$Id: dbbackup.c,v 1.11 2017/11/13 10:44:35 miraculix Exp $
Usage:  dbbackup  <database filename> <output filename>
Hint:   To specify an .ini file, add an argument preceded by the equal sign ('='), e.g., 
'=../notes.ini'

Monitoring backup

You can view the status of your backup jobs by selecting Last Backup State in the Main Selection window. The backup status overview provides detailed information on the last run of backup jobs, including the task name, start and stop time of the last backup, backup level, data size, throughput, assigned media pool, etc.

As of 4.4.3 Beefalo V2, you can check the details of your backups online by using new Web UI. For details, see SEP sesam Web UI.