Archive:SEP sesam Extension for Zarafa
Introduction
The SEP sesam online module for Zarafa makes hot backups of Zarafa Groupware Servers possible. This includes saving and restoring single user mails, mailboxes and public folders.
System requirements
SEP sesam / Zarafa version
New functionality in SEP sesam 4.0.4: |
Starting with SEP sesam 4.0.4 enhanced functionality for Zarafa backups is available:
|
- Package zarafa-backup must be installed, otherwise backup and restore won't work
- Supported Versions
- SEP sesam Server
- SEP sesam Client
- Working path backup of the Zarafa server
Zarafa Community Edition
The Zarafa Community Edition does not include the Brick Level Backup API from Zarafa. Therefore the SEP sesam Zarafa Extension does not support the Zarafa Community Edition.
See: http://www.zarafa.com/content/editions
Of course, you can use SEP sesam to perform a filesystem backup of your Zarafa system.
Installation
Note |
If you are using the SEP Sesam Client Versions 3.6 or higher, you dont have to manually install the extension. It is already included in the Client Package |
Backup Configuration
The rest of the configuration can be accomplished using the SEP sesam GUI.
Configuration of mailbox backup
Attention |
Starting with SEPsesam version 4.0.4, differential and incremental backups are possible. This version has to be installed on Sesam Server and Client to do this. |
- Create a new backup task for the Zarafa server
- Choose Zarafa as task type
- Important! The size of the backup directory must have at least the volume of the largest mailbox
- Enter backup source in the following format:
- ALL
- All users and public folders
- ALL_USERS
- All users
- PUBLIC_FOLDER
- All public folders
- <User1>, <User2>, ...
- One or more user mailboxes
A final setup task for certain users should look like:
Exclude list:
The exclude list can be used to omit certain mailboxes from the backup. Exclude patterns are regular expressions which are compared against username.
Example:
^support$
will exclude just the user support and
support
will exclude users like support, sepsupport and support1.
This screenshot shows a backup task that saves all users but "mk" and "shettler":
Backup of multiple mailboxes simultaneously
Sesam client 4.0.5.x and higher allows a simultaneous backup of mailboxes. To activate that and set the number of simultaneous backup you have to set the value max_exports higher than zero (0). For instance, to configure 5 simultaneous backups for the task set the following setting in the task properties (tab Options 1
-a max_exports=5
This parameter forces the given count of zarafa-backup processes on the zarafa server. Keep in mind that your zarafa server system can handle so many processes at the same time.
Restore Configuration
Single mail restore
During the restore you can select single mails in the SEP sesam GUI. The only function you can select in the last register of the restore wizard is Auto recover after restore. If selected, SEP sesam will import the mail into the original mailbox. Otherwise the exported mail data and index files are restored into the file system to <SESAMVAR>/work/zarafa and can be imported manually with the command zarafa-restore.
Attention |
Please leave all other fields unchanged! |
The following example shows a restore of a single mail into the original mailfolder. SEP Sesam will import it automatically with the "Auto Recover and online after restore" option:
Troubleshooting
General information about the backup process
SEP Sesam uses the Zarafa Bricklevel Backup API in order to back up the user's mailboxes. This means the SEP Sesam client first receives a list of available users on the Zarafa system and then goes ahead and exports the mailboxes that are to be backed up by using the Zarafa tool "zarafa-backup".
The mailbox is exported to the <SESAM_ROOT>/var/work directory by default. Please ensure that the partition that is holding your work directory can hold as much space as the biggest mailbox in your Zarafa environment.
If the export of a certain mailbox fails for whatever reason (not enough disk space, internal Zarafa errors), it is always a good idea to look at the SEP sesam backup protocol.
The backup protocol
The SEP sesam backup protocol contains a detailed logging of what happened during the Zarafa backup. It also contains each call of the Zarafa Bricklevel Backup API.
This example shows a part of a Zarafa backup loggin with a successfull call of the Zarafa Bricklevel tool:
[..] 2009-11-06 10:15:43: sbc-3103: Info: DB Module: [Start export: [zarafa-backup -u ryan]] Fr 06 Nov 2009 10:15:43 CET: Creating folder and message list for user ryan Fr 06 Nov 2009 10:15:51 CET: Starting full backup for user ryan Fr 06 Nov 2009 10:16:16 CET: processed 264 of 264 messages in folder 'Inbox' Fr 06 Nov 2009 10:16:16 CET: processed 8 of 8 messages in folder 'Sent Items' Fr 06 Nov 2009 10:16:16 CET: processed 2 of 2 messages in folder 'Contacts' Fr 06 Nov 2009 10:16:16 CET: processed 8 of 8 messages in folder 'Calendar' Fr 06 Nov 2009 10:16:16 CET: Done backup for user ryan [..]
After exporting the mailbox SEP Sesam continues with backing up the exported files.
If the export fails for some reason the error is reported in the SEP Sesam logfile. This example shows a backup for a user that doesn't exist:
2009-11-06 15:36:36: sbc-3103: Info: DB Module: [Warning: User "userdoesnotexist" was not found on the server] 2009-11-06 15:36:36: sbc-3103: Info: DB Module: [BuildFinalUserList(-)] 2009-11-06 15:36:36: sbc-3103: Info: DB Module: [ProcessExcludeList(+)] 2009-11-06 15:36:36: sbc-3103: Info: DB Module: [ProcessExcludeList(-)] 2009-11-06 15:36:36: sbc-3103: Info: DB Module: [Found [0] mailboxes in mailnode] 2009-11-06 15:36:36: sbc-3103: Info: DB Module: [DB_InitOperation (-)] 2009-11-06 15:36:36: sbc-3103: Info: DB Module: [DB_GetItem(+)] 2009-11-06 15:36:36: sbc-3103: Info: DB Module: [DB_GetItem(): There are no users to backup] 2009-11-06 15:36:36: sbc-3103: Info: DB Module: [DB_GetItem(-), reason=DB_NUM_I_COMPLETE] 2009-11-06 15:36:36: sbc-1120: Error: Backup source could not be found. Exiting.
Testing the backup on the Zarafa server locally
In some cases it may be important to get a first impression of how long the backup takes. This can be done by backing up the Zarafa mailboxes directly on the Zarafa server itself without writing to some media. This is also a nice way to check if your firewall closes ports due to small timeouts.
This can be done by using the SEP Sesam backup client on the command line.
The following command executes a backup of all Zarafa mailboxes and writes its data to /dev/null :
root@zarafa_host#: /opt/sesam/bin/sesam/sbc -b -s - "ZARAFA:all" > /dev/null
Of course it's also possible to only back up certain users on the command line ("cl" is the username in this case):
root@zarafa_host#: /opt/sesam/bin/sesam/sbc -b -s - "ZARAFA:cl" > /dev/null
Testing the zarafa Bricklevel export with Zarafa tools
In some cases it makes sense to test the export of mailboxes by using only Zarafa tools. Mailboxes can be exported with the command "zarafa-backup". This example shows the export of the user "test" into the current directory:
root@zarafa_host#: zarafa-backup -u test -o .
After the export the directory should contain two files:
test.index.zbk test.data.zbk
The file "index.zbk" contains an indexed listing of the mail ID's that were in the mailbox. "data.zbk" holds the real data.
If this export fails with the Zarafa Bricklevel tools, please get in touch with your Zarafa support technician.
Special configuration settings for connection timeouts / firewalls
In any case the SEP Sesam Client opens a control connection to the SEP Sesam server. This control connection stays open during the whole backup process but does not transfer data. If there is no communication between the client and the SEP sesam server this connection may get dropped by some firewalls because of timeout values. Please make sure to set the timeout settings in your firewall to an adequate value.
Also the export of big mailboxes by the Zarafa Bricklevel tools can take some time. If you encounter logmessages like "XBSA Errors" and "Connection timouts" it is always a good idea to check your firewall loggings.
SMS timout value (SERVER SIDE) for big mailboxes
If you have big mailboxes and the export takes a long time we recommend to also increase the DATA_TIMEOUT setting for the STPD data transfers.
The following configuration variable in <SESAM_ROOT>/var/ini/stpd.ini can be set to a timeframe in seconds. The default value is "7200". If your biggest mailbox takes more than 2 hours to export this may become a problem.
[STPD_Thread] STPD_BUFSIZE=4 DATA_TIMEOUT=7200 AUTH_USERS=sms UPDATE_THROUGHPUT=30 STPD_ACCEPT=TRUE
SBC timeout value (CLIENT SIDE) for big mailboxes
On the client add the following configuration statement to <SESAM_ROOT>/var/ini/sm.ini
[Params] SBC_GEN_WAIT=10800
This increases the timeout the SEP sesam backup client waits for data to be transfered to 3 hours.