Troubleshooting Guide
Introduction
This guide is intended to help you quickly identify and resolve problems and errors during setup, installation and during normal operation of your SEP sesam system. SEP sesam often indicates that there has been a change or event that impacts overall system performance. Changes in SEP sesam backup performance are usually caused by system changes or failures.
First steps
To solve problems quickly and efficiently, make yourself familiar with the general troubleshooting information. Attempt to identify and resolve the problem by yourself. If you cannot find a solution to your problem, report it to SEP sesam support. The support team will then provide you with further instructions.
Identifying the problem
It is important to identify a problem as accurately as possible so that the best solution to the problem can be applied. Answer the following questions to identify the problem you are experiencing:
- What was the message or error?
- When did the error first appear?
- Does the error always occur after a specific action?
- Does the error always occur at the specific time, e.g., in the morning, in the evening, etc.?
- What steps are necessary to duplicate the error?
- Did you make any changes to your SEP sesam environment before the error occurred?
Common problems you can solve by yourself
Below is a list of the most common problems you can solve by yourself. Most of the errors you are likely to encounter can be solved by following these instructions.
- Check with sm_main status to see if all processes are running. If necessary, start the missing process again with sm_main reload.
- View the day log and search for errors.
- Check if the failure was caused by a Newday event. You can prevent Newday from aborting running activities, as described at Newday Event.
- Check if there is a time frame set in the schedule after which the scheduled event is cancelled (option Stop task if it runs longer...). See Creating a Schedule.
- Check specific protocols for backups, restores.
Before contacting support
Before contacting SEP sesam support, make sure that you have exhausted any options that might enable you to solve the problem by yourself. Perform these general checks:
- Check whether your problem is described in the Error Messages Guide or Troubleshooting Guide.
- Browse through the FAQ and check the forum.
- Go through the Common problems you can solve by yourself checklist above.
- Collect all relevant data about the problem and send it to SEP sesam support:
- error output
- message ID
- SEP sesam version (Server, RDS and Client)
- operating system environment
- LOGs (when there is a problem with a backup job, choose in the GUI 'JOB STATE'- 'BACKUPS' - Right click on the affected job and select the bottom menu item 'Download archive with all log files' -> Please send the created LOG archive too.) Proceed in the similar way for restore, migration or replication.
How to interpret SEP sesam's backup module's error messages?
SEP sesam backup modules are designed to produce extended error messages which may return information from 5 layers: SBC – XBSA – FTP – SMS – operating system. SEP sesam scans the protocol files for warnings and errors after backup and restore. In the event of a warning or an error, the first identified message is printed in the summary at the end of the protocol.
Every backup module uses the X/Open Backup Services API (XBSA) standard. SEP sesam XBSA is based on FTP implementation. The backup module connects to SEP sesam's FTPD daemon implementation – Sesam Transfer Protocol Daemon (STPD). Sesam Transfer Protocol Daemon (STPD) is a service that requests and delivers the backup data from or to the SMS Server and manages the data flow between the SEP sesam Server and a client. During a restore STPD receives the data from the SMS Server and sends it to the client, which then restores the data to the target system. Sesam Multiplex Stream (SMS) is a service that receives the backup data from STPD and writes the data to the backup media. During a restore, it reads the data from the backup media and sends it to STPD. Additionally, the SEP sesam backup client (SBC) module executes backup, migration and restore tasks. SBC collects and consolidates backup data on the client system and delivers it to STPD. A list of all SBC messages (C header file) can be found at SBC Messages.
An error message is composed of the messages from the triggering layer up to the upper layers. If an operating system returns an error, the error code and the operating system message are added to the SEP sesam error message. Because of this, error messages can also help troubleshoot problems that are not caused by SEP sesam (for example, OS problems).
Typical backup protocol
The following example shows a typical backup protocol. It is composed of 4 sections: about module, operational parameters, processing, and a summary.
2009-06-26 10:28:16: sbc-3036: Info: # SESAM BACKUP CLIENT FOR Windows NT FILE SYSTEMS, VERSION: 3.2A17 Build Revision: 1.257 (x64), Released: Jun 25 2009 # 2009-06-26 10:28:16: sbc-3063: Info: -------------------- Operation Parameters -------------------- 2009-06-26 10:28:16: sbc-3019: Info: OS info: Microsoft Windows Server 2008, Build: 6001 Service Pack 1 (x64) 2009-06-26 10:28:16: sbc-3100: Info: Program PID: 42900 2009-06-26 10:28:16: sbc-3030: Info: Operation: BACKUP, Level: COPY 2009-06-26 10:28:16: sbc-3031: Info: Storage Host: qsbox3:11001,0-0:SESAM_SECURE_AUTHENTICATION:**** 2009-06-26 10:28:16: sbc-3032: Info: Control Host: qsbox3:11001:SESAM_SECURE_AUTHENTICATION:* 2009-06-26 10:28:16: sbc-3040: Info: Device: SMS:disk1:SHARE:64 2009-06-26 10:28:16: sbc-3064: Info: --------------------- Operation Messages --------------------- 2009-06-26 10:28:16: sbc-3002: Info: Building file list from: [C:\SEPsesam\var\ini] 2009-06-26 10:28:16: sbc-3022: Info: Command line ["sbc" "-b" "-C" "qsbox3:11001" "-S" "qsbox3:11001" "-l" "copy" "-s" "SF20090626102812" "-d" "SMS:disk1" "-t" "weekly00001:1" "-j" "TEST_BACKUP" "-i" "job=TEST_BACKUP,nod=qsbox3,cmd=sbc,src=C/ /SEPsesam /var/ini,ptf=WNT,typ=Path,exc=" "C:/SEPsesam/var/ini" ] 2009-06-26 10:28:16: sbc-3003: Info: Opening saveset: SF20090626102812 2009-06-26 10:28:18: sbc-3104: Info: Saveset info: [SEGMENT=3] 2009-06-26 10:28:18: sbc-3004: Info: Begin writing to saveset... 2009-06-26 10:28:18: sbc-3074: Info: Backup start time [20090626102818] 2009-06-26 10:28:18: sbc-3143: Info: Starting with drive C: 2009-06-26 10:28:18: sbc-3006: Info: Saveset size: 98304 bytes. Throughput: 189.820 MB/Hour. 2009-06-26 10:28:18: sbc-3005: Info: Closing saveset. 2009-06-26 10:28:18: sbc-3052: Info: Items processed correctly: [25]. Not processed or incorrectly processed items: [0]. 2009-06-26 10:28:18: sbc-3007: Info: Operation successful. 2009-06-26 10:28:19: sbc-3001: Info: Exiting.
Backup error summary
The error message summary is prefixed by a short information string. The full error message is composed as follows:
{status}/{amount}/{saveset ID}/{SBCstart}/{message}
The components of this string have the following meanings:
{status} | {amount} | {saveset ID} | {SBCstart} | {message} |
0 - successful |
Amount of data stored on media | Automatically generated saveset ID | Starting time on the client | Message about the error |
The following example shows a backup error summary with all 5 layers prefixed by a short information string.
X/0/SF20060629233007/20060629232907/Error: XBSA Call BSAEndData (closing saveset) failed: System detected error, operation aborted. TRANSIENT or PERMANENT NEGATIVE reply: 553 STOR Failed. 1037: Writing data block on tape failed (23): Data error (cyclic redundancy check). 1039: Writing of Saveset Trailer failed.
The amount of details provided for backup or restore is defined by the log level.
How to set a log level
You can control the reporting (verbose) level to choose the details of messages that SEP sesam reports in various error logs. Running SEP sesam with a higher log level than default (0 for backup and restore) might be useful when you want to get more information about specific events or modules or when asked by support in the course of diagnosing your specific problem.
Note that increasing the log level increases the amount of information being logged and may negatively affect the performance of SEP sesam. Therefore you should only set a higher log level for your own analysis of a specific issue with backup or restore, or when asked by support to provide debugging information in case of troubleshooting. Once you have reproduced the issue, set the log level back to default if you have modified it in the debug.ini file.
You can set the log level in SEP sesam on the following levels:
- For specific backup or restore task in the GUI: Main selection -> Tasks -> By Clients -> double-click the backup or restore task to open the Properties. The log level is set under the Options tab for backup and Expert Options for the restore task. For details, see section Setting log level on a per-task basis in GUI.
- Globally for SEP sesam kernel modules by modifying
<SESAM_VAR>/ini/debug.ini
file on SEP sesam Server. For details, see section Setting log level globally for SEP sesam kernel modules in debug.ini.
The default SEP sesam directory for logs is <SESAM_VAR> (/var/opt/sesam/var
). For more information, see Directory Layout.
SEP sesam log levels
SEP sesam uses different log levels for different modules. The following list represents the reporting level of the backup and restore protocol, which is set in GUI on a per-task basis. Log levels are listed in order of increasing amount of information logged, from lowest to highest:
Setting | Log level details |
---|---|
0 | print only standard and error messages together with summary |
1 | display a line for every item when its processing starts: sbc-3008: Info: Processing item: [xxx]... |
2 | display a line for every item when its processing finishes: sbc-3108: Info: Item processed successfully: [xxx] |
3 | display backup module processing information (with DB_API modules) |
4 | display underlying module processing XBSA and detailed DB_API modules |
5 | display packing data (mtf, cpio, sidf) trace messages |
For details on how to set a log level for specific backup or restore task in the GUI or globally for SEP sesam kernel modules, see Setting Log Level.
Installation and configuration problems
Installation problems
SEP sesam Server installation on Windows failed
Problem
- The SEP sesam Server installation on Windows failed with an error.
Cause
- For SEP sesam Server installation a domain administrator account has been used.
⇒ Solution
- Before starting the SEP sesam installation on Windows, make sure that you are logged in as a local administrator (not domain administrator).
SEP sesam database problems
PostgreSQL startup failure after changing SEP sesam service user (Windows only)
Problem
By default, the SEP sesam service starts under the SYSTEM user account. At installation, postgreSQL database is set to start under this account. However, if the user name of the SEP sesam service is changed from SYSTEM (for example, to Administrator) after installation, PostgreSQL encounters startup issues and fails to start.
Cause
Changing the user name of the SEP sesam service affects the ownership and permissions of the PostgreSQL data directory, typically the db_pg folder. As a result, PostgreSQL is unable to access the necessary files and resources, causing failure to start up.
⇒ Solution
You can revert to the previous SEP sesam service account (SYSTEM), or change the ownership and permissions of the PostgreSQL data directory (db_pg folder) to the used SEP sesam service account user. In this case make sure you change ownership and permissions for subfolders and files as well (use Windows Explorer option Replace owner on subcontainers and objects). PostgreSQL regains the necessary access rights, enabling it to start successfully. Note that this workaround modifies the folder's ownership and permissions, so make sure that the used user account has appropriate privileges for managing the SEP sesam database.
Incorrect timezone setting in PostgreSQL
Problem
Exporting the system-wide environment variable TZ in the system-wide /etc/profile file can override the default system locales, leading to an incorrect timezone setting in the postgresql.conf file. As a result, the time values in the GUI may be incorrect. Mismatch between the system's default timezone and the TZ variable leads to inconsistent time values displayed in the GUI.
⇒ Solution
In the /etc/profile file update the TZ environment variable and set the timezone that matches the desired system default timezone. This will ensure that the timezone setting in the postgresql.conf file is aligned with the system default timezone.
License problems
SEP sesam Server license import fails
Problem
License problems after importing a newly created license
Cause
The license is not correctly imported
⇒ Solution
If you encounter problems after importing a newly created license (strange error messages; expiry message of the license, although the license file actually shows valid values; 'Valid until' shows a different value in the license info than in the license, etc.), switch to demo license and then reimport your new license. Try the following steps:
- From the SEP sesam GUI menu bar, select Help -> License information, and then click Import New License.
- Click File Manager, select the demo LIC file (sm_lic.ini) from
<SEPsesam>/skel
, and then click Apply. - In the License info dialog, click Import New License again.
- Use the File Manager to locate and import your newly issued license.
-> A final check (SEP sesam GUI menu bar -> Help -> License information) should show that the newly created license has been imported correctly and that all errors have disappeared.
License violation caused by exceeding the licensed volume
Problem
Exceeding the licensed volume of backups results in license violation.
Cause
The license violation occurs when there is too much data being considered for licensing purposes, often due to accumulated data that is no longer required.
⇒ Solution
If you encounter a license violation due to exceeding the volume, several steps can be taken:
- Check and expire the EOL of old backups that are no longer required, so they are purged and no longer count towards the license volume.
- Purge and clean up the datastore to remove any backups that should have been deleted but are still present.
- If backups are stored on tapes, they are no longer counted when the tape is reinitialized. In newer SEP sesam versions (≥5.0 Jaglion), backups are no longer counted even if the tape is deleted (including metadata deletion).
Perform a final check in the GUI under HELP - LICENSE INFO to ensure that the license error has been resolved.
Update problems
Client/RDS update does not start or does not work via GUI
Problem
- The direct update of Client/RDS via the GUI does not start.
Cause
- There is still old/wrong client information in the database, which hinders the update process.
⇒ Solution
- In the GUI, under Clients, right-click the client for which the update is not working, and then click the context menu option Reset Version Information.
- Then double-click the client to open its properties and click the trash icon next to the Last SEP sesam message field.
Now the update should start and run without any problems.
SEP sesam Server update failed
Problem
- The SEP sesam Server update failed with an error.
Cause
- SEP sesam services on the system that is being updated could not be started, causing SEP sesam Server update to fail.
⇒ Solution
- Check if SEP sesam Server services are running. If not, start the SEP sesam services manually and then use the same update for the repair installation. For details, see How to Start and Stop SEP sesam.
SEP sesam Windows Client update from v. 4.4.3.64 fails with error
Problem
- The SEP sesam Windows Client update from SEP sesam v. 4.4.3.64 Grolar to a newer version might fail with the following MSI installer error:
Unable to create a temp copy of transform
Cause
- This issue occurs if the transform path specified in the registry points to the wrong folder.
⇒ Solution
- To resolve this issue, you must rename the transform key in the Registry Editor.
Note | |
This procedure modifies the Windows registry; modifying Windows Registry incorrectly might crash your operating system and cause data loss. Ensure that you have a valid SEP sesam and operating system backup before proceeding! |
Proceed as follows:
- Open Registry Editor: use Start and type regedit.
- Navigate to the Products folder:
HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Installer\Products
- Depending on the installed SEP sesam Version find one of the following folders and click it:
SEP sesam Version 5.0.0.X:7737007073521AA4885C03C7AEE4954D
SEP sesam Version 5.1.0.X:7737007073521AA48A24533ED8EBE04E
- In the right pane with registry keys, right-click the Transforms key and rename it to Transforms_.
Automatic remote update from GUI fails with errors
Problem
- On Windows, automatic remote update failed with the following error:
Windows error: (0x800703FA) Illegal operation attempted on a registry key that has been marked for deletion
- The sm_msi.log log file on the client system shows the following:
InstallShield 11:08:04: Initializing Engine InstallShield 11:08:04: Marshalling the , error = 0x800703fa InstallShield 11:08:04: Open Script operation failed, error is 0x800703fa InstallShield 11:08:04: Failed to invoke __ISWIUnInit, error is 0x80070057 InstallShield 11:08:04: Failed to shutdown script engine for script C:\Users\sepshare\AppData\Local\Temp\{A2C12F7B-F8D3-4D99-8E34-61600D67DBCC}\setup.inx, error is 0x80070057 InstallShield 11:08:04: Initialize() Failure, Failed to Initialize script support, Error = 0x800703fa
Cause
- Typically, this error occurs when the sesam service is running under a specific user account instead of the default system account. An administrator uses this service user account to log on to the server (SEP sesam RDS or client) for the session and then logs off. This forcibly unloads the registry keys in the account profile and makes the keys unavailable for future use.
⇒ Solution
You have to enable the Windows User Profile Service which does not unload the registry keys after logging off:
- Open the Group Policy editor (Gpedit.msc) on the respective server.
- Open the UserProfiles folder: go to Computer Configuration -> Administrative Templates -> System -> UserProfiles.
- Look for the Do not forcefully unload the user registry at user logoff setting and select the option Enabled. For details, see Microsoft support article "800703fa Illegal operation attempted on a registry key" error .
Problems with interfaces
Alarm interface fails to send emails
The Alarm interface is designed to send backup logs, stored in the LIS file directory, to specified email recipients. In Apollon, changes have been made to the directory structure for LIS files to improve file management and optimize file system operations. LIS files are now stored in the <SESAM_ROOT>/var/lis directory, with a new sub-directory created for each Sesam day, which prevents accumulation of files in a single directory.
This change has inadvertently caused the Alarm interface to lose track of the required log files. The sm_alarm script is unable to locate the .not file, which leads to the NullPointerException error, and the Alarm interface cannot send emails.
⇒Solution
To resolve this issue, you have to manually modify the sm_alarm script. This will ensure the correct variables are used and the script can locate the required files. Note that the procedure for modifying the sm_alarm script differs slightly between Linux and Windows. For detailed instructions, refer to the sections below.
- On Linux
Open the sm_alarm.sh script and find the following line:
sm_smtp -A "sesam" -s "$mail_subject" -M "$( ls -tr ${gv_rw_lis}/$f*not|tail -1 )" ""
Replace that line with:
sm_smtp -A "sesam" -s "$mail_subject" -M "$3" ""
- On Windows
- Open the sm_alarm.ps1 script in a text editor and find the function send_backup_mail:
function send_backup_mail($type,$msg,$task) { ## Set mail subject $mail_subject = (write "Sesam ALARM: $type $task failed") ## Send Sesam not file as mail body $filename = ((Get-ChildItem "$env:lis\*$task*.not" | sort-object -property LastWriteTime | select-object -last 1).FullName) sm_smtp -A "sesam" -s "$mail_subject" -m "$msg" -a "$filename" }
Replace the function with:
function send_backup_mail($type,$message,$filename) { $task = ($message -replace ":.*","") ## Set mail subject $mail_subject = (write "Sesam ALARM: $type $task failed") ## Send Sesam not file as mail body sm_smtp -A "sesam" -s "$mail_subject" -m "$message" -a "$filename" }
- Find the following line:
{send_backup_mail $args[0] $args[1] $task
Replace that line with:
{send_backup_mail $args[0] $args[1] $args[2]
Alarm interface does not function as expected
The script that defines the functions of the Alarm interface can be modifed to customize the behaviour of the Alarm interface. In release 5.1.0.7, the script was rewritten to address an issue (see Alarm interface fails to send emails). As a result, any modifications made to the script prior to this release have been overwritten and the default behaviour of the interface is reestablished.
⇒Solution
To resolve this issue and restore the desired behavior of the Alarm interface, it is necessary to redo the customizations in the script that modified its behavior. Users should carefully review their previous modifications and apply them again to the updated script.
Uninstallation problems
For uninstallation problems, see Uninstalling SEP sesam.
System limits adjustment problem
Problem
- SEP sesam limits on the system have to be adjusted, however, the settings should not be overwritten by update.
⇒Solution
The best way to adjust SEP sesam limits on the system is to create the Systemd override file. By using this file the settings are not overwritten during the update.
To adjust limits on the system, proceed as follows:
Example
> > root@cefix:~# systemctl show sepsesam.service | grep ^LimitCORE= > LimitCORE=2147483648
- Create the override file and adjust the value:
- Reload the system by using the command:
- Then specify:
> mkdir -p /etc/systemd/system/sepsesam.service.d/ > echo "[Service]" > > /etc/systemd/system/sepsesam.service.d/override.conf echo > "LimitCORE=infinity" >> > /etc/systemd/system/sepsesam.service.d/override.conf
> root@cefix:~# systemctl daemon-reload
> root@cefix:~# systemctl show sepsesam.service | grep ^LimitCORE= > LimitCORE=infinity
Windows installer (MSI) installation error
Problem 1
- You receive the following warning: "An error occurred during the installation of assembly component... HRESULT: 0x80070bc9."
Possible causes
- MSI locks the database to install new software after Windows updates were installed.
⇒ Solution
- Reboot the host to unlock the database.
Note | |
The msi log file contains the line 'Transforming table Error' very often. This does not mean that an error occurred. It rather means that the table which is called Error has been transformed. Hence it is only an information not an error message. |
Problem 2
- Installation failed after uninstallation - you receive the following Popup:
- English: "The SEP sesam server was installed using a server installation package. Please retry the update using a server installation package instead of the server package you are using. Update will be aborted."
- German: "Der SEP sesam server wurde mit einem server Installationspaket installiert. Bitte wiederholen Sie den Update mit einem server Installationspaket statt des verwendeten server Pakets. Das Update wurde abgebrochen."
Possible causes
- The uninstallation did not remove HKLM\SOFTWARE\SEP Elektronik GmbH registry key.
⇒ Solution
- Open the MS Windows registry (e.g. call regedit) and remove HKLM\SOFTWARE\SEP Elektronik GmbH.
SLES
Problem
After upgrading the OS from SLES 11 to a newer version, SEP sesam server cannot read files in an XFS V4 datastore, whether connected directly or over iSCSI.
Cause
Starting from SLES 12, XFS no longer supports file systems in the V4 format. Newer versions, specifically SLES 12 or later, are incompatible with XFS V4, preventing SEP sesam from accessing the datastore. Note that this issue is caused by upgrading the operating system, and not by updates to the SEP sesam Server/RDS.
This may also affect other Linux distributions.
⇒ Solution
To solve this issue, create the file system XFS V5 on the partition containing the datastore:
- Migrate or back up your data from the affected datastore to another location. Make sure to retain the original file structure.
- Reformat the partition with the XFS V5 format, effectively deleting all data on that partition.
- Once the partition has been successfully reformatted with XFS V5, migrate or restore the previously backed-up data back to the datastore.
For more information, see SUSE Storage Administration Guide.
Red Hat Enterprise Linux
Problem
- The following warning occurs on a Windows operating system: "ERROR: SMS0004: The specified device does not exist."
⇒ Solution
- Reinstall the CBMR x.x SEP Sesam Version software package.
Powershell script not executed on a target machine
Problem
- Why is the Powershell script not executed on a target machine?
Possible causes
- By default, Microsoft installs Windows Powershell with the permission set to Restricted. This setting only allows the execution of commands in Powershell but no scripts.
⇒ Solution
- This can be changed with following command in Powershell:
Set-ExecutionPolicy RemoteSigned
For more information, see About Execution Policies.
IBM DB2
Problem
- There are problems with SEP sesam operations.
⇒ Solution
- Check contents of db user's
$HOME/sqllib/db2dump/db2diag.log
log file. - Check the messages on SEP sesam Server.
- Find more information in the sdb2 log. Log filename is set by XBSA LOGFILE=<Full pathname of sdb2.log> and log level by XBSA TRACE=1.
Note | |
All sesam XBSA messages have the prefix XBSA. You can set XBSA TRACE to 2 to show more details, but then the log files can become quite large. |
MacOS
Problem
- There are problems with SEP sesam operations.
⇒ Solution
The Mac installer has its own log messaging system. To view the log files, proceed as follows:
- Execute the installation/update of the client or GUI package step-by-step until you get to the last installer step Summary.
- Go to the Installer Menu and click Window -> Installer Log.
- Select among the following options: Show Errors Only, Show Errors and Progress or Show All Logs.
AIX
Problem
- After SEP sesam installation, an error occurs: STATUS=ERROR MSG=Some daemons offline
⇒ Solution
By default the sesam sshd daemon is not included in the AIX package. Therefore the command:
/opt/sesam/bin/sesam/sm_main status
will report the sshd as offline:
2016-08-27 16:20:05: sshd : offline
- Currently, this error can be ignored as only the CTRL access mode is supported for the AIX systems.
- To disable the sshd daemon, edit the file:
/opt/sesam/var/ini/sm.ini
and change the option:
sshd=on
to
sshd=off
- If any other services are not starting up, contact SEP sesam support.
Debian repository
Problem 1
- After running apt-get update, the following GPG error is shown:
W: GPG error: SEP Download Center jessie InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 68111EBBD273917B
Cause
- The SEP GPG key is missing in the system keyring.
⇒ Solution
- Install the key, as shown in the Debian Repository.
Problem 2
- During package installation the following warning (verification error) occurs:
WARNING: The following packages cannot be authenticated! sesam-<package> Install these packages without verification? [y/N]
Cause
- The SEP GPG key is missing in the system keyring.
⇒ Solution
- Install the key, as shown in the Debian Repository.
Problem 3
- While running apt-get update, the following message appears:
N: Loading the configured file »main/binary-i386/Packages« is skipped because the [...] depot does not support the »i386« architecture.
⇒ Solution
- Insert
[arch=amd64]
between the word deb and the URL of the repository, as follows:
deb [arch=amd64] https://download.sep.de/linux/repositories/debian/ stretch main
Problem 4
- While running the command apt-key add, the following message appears:
E: gnupg, gnupg2 and gnupg1 do not seem to be installed, but one of them is required for this operation
⇒ Solution
- Install the package gnupg to fulfill the dependency via package manger:
apt-get install gnupg
Problem 5
- SEP sesam provides signed Debian repositories for easier package installation, verification and updating of SEP sesam software. When installing SEP sesam Server via the Debian repository, the following message appears:
Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation: The following packages have unmet dependencies: sesam-srv : Depends: postgresql-9.5 but it is not installable or postgresql-9.6 but it is not installable or postgresql-10 but it is not installable E: Unable to correct problems, you have held broken packages.
Cause
- This problem typically occurs if you have used the wrong version of the Debian distribution in /etc/apt/sources.list.
⇒ Solution
- Correct the version of your Debian distribution, e.g., from stretch to buster, and then run apt-get update to refresh the repository information. For more details on the installation, see Debian Repository.
Troubleshooting SEP sesam authentication
After updating to SEP sesam version 5.0.0.x, no user other than the special user Administrator (Windows) or root (Linux) is elevated to Superuser
Problem
- After updating SEP sesam to version ≥ 5.0.0 Jaglion, the GUI complains that superuser rights are required, but only Administrator rights are listed for this GUI user. The Administrator user is not elevated to Superuser and access to the GUI is not possible without authentication.
Cause
- This is the normal behavior for Java policy authentication. After the initial installation of SEP sesam, no user other than the default Superuser is configured. As the name implies, the permissions of the Superuser account are unrestricted. As of 5.0.0 Jaglion, the Superuser is the only user type with full control over the SEP sesam environment. It is automatically assigned exclusively to the Administrator user when database-based authentication is enabled. If policy-based authentication is enabled, this user type with Superuser rights is assigned to Administrator (on Windows and Linux), root (Linux only) and sesam user. For more details, see User Roles and Permissions and About Authentication and Authorization.
⇒ Solution
If you use policy-based authentication (sm_java.policy) and log in from localhost, you can gain Superuser rights if your username is not listed in sm_java.policy. This is possible if the localFullAccess parameter is enabled (set to true in the <SESAM_ROOT>/var/ini/sm.ini
file). There is no such workaround for database-based authentication, as only Administrator (on Windows and Linux) and root (Linux only) can become Superuser.
Tips for backup troubleshooting
In case of a failed backup, you should follow the tips below:
- Find out when the problem occurred using the day log (.prt) and the status log (.status). The day log shows the causal progression of all SEP sesam activities of the backup day. The files with a file extension ending in .prt.err contain only the error messages from the day log.
- Display the directory files chronologically (with ls -lart on Linux).
- The log files should be read backwards from the end of the file. If a backup failed, you can usually find the indication of errors and their causes at the end of the respective log file.
- Compare failed backups with working backups:
- Check when the last successful backup of this task took place.
- Identify the differences between not and bck logs by comparing two different backups.
- Determine if there were any changes on the network or on the client.
- The values of the database calls in DB_ACCESS have the following explanations:
- result = 1: Database access is OK.
- msg > 0: Amount of result > 0.
Backup problems
Client backup error
Problem 1
Backup of a Windows or Linux client fails with the following error:
"E020-HOSTS [..] : Error: Network communication problem: SOCKET error: 10060
Possible causes
The cause of this problem is most likely:
- The active firewall on the SEP sesam backup Client. In this case, check the firewall settings and if ports 11322 or 11301 are blocked.
On Linux also check if the sesam SSHD daemon is running
ps axw | grep sm_sshd
and if port 11322 is open and in state LISTEN
lsof -i TCP | grep 11322
- An active antivirus program is blocking access or running its own firewall and blocking access.
- Third party firewall between the backup server and the backup client blocking access.
- The SEP sesam Client agent is not running on the system.**Other network related errors.
Problem 2
A client backup did not function properly. How can I determine where the problem is?
⇒ Solution
Run the following commands to determine the error:
Warning | |
The following commands produce a high network load. |
BACKUP SERVER UNIX, CLIENT WINDOWS
sm_ctrlc -l system {client name} sbc -b -s - f:/test >/dev/null
Data from the F:/ directory on Windows is written over the network to the directory /dev/null
on Unix. To display it, append -v 1 to the command above. Everything written to /dev/null
will be displayed.
sm_ctrlc -l system {client name} sbc -b -s - -v 1 f:/test >/dev/null
BACKUP SERVER UNIX, CLIENT UNIX
sm_ctrlc -l root {client name} sbc -b -s - /usr >/dev/null
To display the read data:
sm_ctrlc -l root {client name} sbc -b -s - -v 1 /usr >/dev/null
BACKUP SERVER WINDOWS, CLIENT UNIX
sm_ctrlc -l root {client name} sbc -b -s - /usr > NUL
With backup data logging:
sm_ctrlc -l root {client name} sbc -b -s - -v 1 /usr > NUL
If the test backup is to be run on the target backup client only, execute the following command:
In the Unix directory <sesam>/bin/sesam/
:
sbc -b -s - /usr >/dev/null
In the Windows directory <SESAM_ROOT>\bin\sesam\
:
sbc -b -s - f:/test > NUL
Enter -v 1 to show the backed up data on your monitor.
Failed backups will be automatically deleted
Problem
By default, SEP sesam deletes all failed backups after 3 days automatically to release the storage space. How can I keep such backups for a longer time? (< version 4.3.3)
⇒ Solution
If you want to keep your failed backups for more than 3 days, you may manually extend the backup EOL (expiration date) of this particular saveset. For details, see Manually extending EOL. Note that SEP sesam automatically retains the last successful backup saveset when the next backup fails. This means that SEP sesam extends the EOL of the previous 'successful' or 'with warnings' backup, thus ensuring that at least one successful backup is retained.
Failed to write the data to media during backup
Problem
During backup, operating system error "23 (ERROR_CRC)" is displayed.
Possible cause
The tape drive cannot write proper blocks onto the backup media.
⇒ Solution
Check the tape drive and backup media.
Backup to data store fails
Problem
Backup (or migration) to data store fails with the error message All available space of media is used, stop writing to media.
Possible cause
SEP sesam aborts a data transfer because less than 1 GB is available on the data store (disk storage). A disk failure can also cause this problem.
⇒ Solution The following points may help to resolve the problem:
- Check the system messages on Linux or the event log on Windows for any hardware problems.
- Remove orphaned savesets from the data stores by using the Clean up option, see Data Store.
- Delete some savesets or backups on this datastore or start a purge process for the data store to delete the obsolete (EOL-free) savesets and to get enough free space. For details, see Purging a Data Store.
- Increase the disk space.
- Run FSCK on the disk storage.
If you cannot solve this issue, report it to SEP sesam support. The support team will then provide you further instructions.
Incorrect login or password
Problem
During backup, the message "Login incorrect. Password incorrect" is displayed.
⇒ Solution
Check your name resolution (DNS or etc/hosts file). The SEP sesam Server and SEP sesam Client must be reachable with or without FQDN and should be able to resolve each other correctly, including the reverse lookup. If the resolution is correct, do the following:
- In the SEP sesam GUI, go to Main Selection -> Tasks -> By Clients, and select the client with the backup problem.
- Open the backup properties and click the Options tab.
- Type -v 4 at Save options.
- Start the backup again, then go to Main Selection -> Job state -> Backups and double-click the backup task to open its properties.
- Go to Logging -> Day log and search for the line Login incorrect. Password incorrect then correct the name resolution.
Data transfer via FTP fails with XBSA Initialisation error
Problem
At the beginning of a backup data transfer via FTP, the following error message appears:
- XBSA Initialisation error. Access to the requested object is not possible. NEGATIVE reply: 425 Can't open data connection. WINSOCK: Connection timed out. (0x274c,10060)
Possible causes
This problem can be caused by the following:
- Active firewall on the system that is supposed to transfer the backup data between SEP sesam Server and Client.
- Routing or Network Address Translation (NAT) between the systems involved.
⇒ Solution
The following steps can help to solve the problem:
- In the case of a firewall problem:
- Disable the firewall.
- Adjust the firewall application-based. This needs to be done on system, which should transfer the data. SEP sesam uses active FTP. When starting regular file backups on Windows add binary sbc.exe to the firewall exclude list. In case of BSR on Windows also add binary oodiag.exe to the firewall exclude list.
- Customize the firewall port-based. Make sure that all required ports are available on the system and are not blocked by a firewall. If a port range is open, it is important that you configure that port range in the STPD options for the client. For details, see Port numbers for SEP sesam Client.
- In case of a router/NAT, adjust the rules to allow SEP sesam RDS and Client to transfer data.
Network connection failure on physical or virtual systems
Problem
SEP sesam backup of physical or virtual systems fails due to network connection error (10054). The log files contain one of the following error messages:
10054 An existing connection was forcibly closed by the remote host
or
Error : Network communication problem: SOCKET error: 10054 - The virtual circuit which reset by the
remote side . recv () call failed.
For more details on Windows error codes, see Microsoft Developer Network documentation.
Possible causes
Cause 1: Virtualization solutions
Citrix XenTools and VMware Tools may create their own network card driver on a virtual machine, which will be used instead of the regular Windows system drivers.
The paravirtual network drivers may cause some problems:
- General CPU load of approx. 30% within a VM without any actions
- Very poor throughputs, even in disconnected networks or gigabit LANs
- Broken connections
According to Microsoft documentation, the above error message shows that the connection was reset by the client, i.e., the system to be backed up.
The problem is at the same time visible on the SEP sesam Server:
- The backup remains in the active status with a 0 GB/h performance.
- There are active sm_sms_backup processes.
- There are active sm_stpd processes.
SEP sesam has no feedback from the client that the backup was aborted. The connection has been hard reset.
Cause 2: Network/Port trunking
In several customer environments, port trunking was disabled either on the system to be backed up or on the backup server side. After the trunks were resolved, the backups could be performed without problems.
To resolve trunking issues, see Trunking configuration guide.
Cause 3: Firewall
If network disconnection occurs in less than 5 minutes, is most likely a TSO/trunking issue described above. If the network disconnects after 5, 60 or 120 minutes, there is a KeepAlive problem with the NAT/PAT router or a firewall.
Cause 4: Virus scanner
Virus scanners intervene actively in the data transmission and related processes and may thus be the cause of the problem. It is recommended to use the antivirus exclusions for SEP sesam as they can have adverse effects on backup and restore operations, as described in Antivirus Exclusions for SEP sesam.
Cause 5: Error on the SEP sesam Server side
If a problem with the process sm_stpd.exe or sm_stpd occurs is recorded in the event viewer or the dmesg output of a SEP sesam Server at the time the backup is aborted, the problem must be analysed from the SEP sesam Sever side. Note that incorrect routing can also be the cause of this problem.
For example, the SEP sesam Server and SEP sesam Client are on two different networks. Routing is not controlled by one central router but via static routes on the other network. A ping from the SEP sesam Server to the client works, but not vice versa. In such case, the permanent route to the client's network was set on the SEP sesam Server, but it was not part of the active routes.
To solve this problem, delete and re-add the route as follows:
route delete <client_network> route add -p <client_network> mask <network_mask_of_the_client_network> <SEP_sesam_Server_IP>
Cause 6: Disabling task offloading
For VMs
Disable the TSO (TCP Segment Offloading) feature in the VM.
Both Citrix XenServer and VMware ESXi use the existing TSO in the network cards. This feature allows you to swap out different operations that must be performed during the fragmentation of network packets to the network interface card (checksum calculation). The purpose of this is to reduce the CPU load. (The XenServer network card drivers (NIC) are more affected by this problem.)
On Windows
To disable this feature on Windows, the key DisableTaskOffload has to be set in the registry to prevent offloading to the network card, see Microsoft documentation. After setting this option on the backup client and then restarting, further backups run without any problems.
On Linux
On Linux, you can activate or deactivate the TSO settings during runtime using the ethtool tool as follows:
ethtool -K eth0 tso off
TCP retransmission
Fixing TCP retransmission errors at the network level fixes the root cause of the problem. To find TCP retransmission errors, the network analysis tool Wireshark is required.
You have to start a network analysis with Wireshark on the affected system (Capture -> Interfaces).
Note | |
Retransmission error appears as black lines in the Wireshark log. |
ON WINDOWS
Note | |
SEP sesam uses Microsoft’s Volume Shadow Copy Service (VSS) to perform backup for various task types. VSS failures are typically caused by system configuration and not by SEP sesam; this section may provide some instructions on how to troubleshoot such issues. If you cannot find your issue here, check also VSS Troubleshooting or refer to Microsoft's article Volume Shadow Copy Service for more detailed information on VSS. |
Path backup on Windows is not working
Problem
SEP sesam Server failed to execute a path backup of a Windows client (however, a path backup without VSS is working). The following error is displayed:
Problem while loading dynamic link library: [WIN32 API error: 1114 - A dynamic link library (DLL)
initialization routine failed. LoadLibrary() call failed for: [vss.dll]].
Possible causes
- Typically, when dll cannot be initialised, a running antivirus solution is preventing it.
- The client's VSS configuration is incorrect.
⇒ Solution
- Disable the antivirus software during backup.
- Check the client's VSS configuration by running the following commands:
- vssadmin list writers - check the status of all writers on a daily basis
- vssadmin list shadows - check existing VSS copies
- vssadmin list shadowstorage - check how much space is reserved and available space for VSS
- vssadmin resize shadowstorage - set the reserved space for VSS snapshots to unlimited
WIN32 API error: "1450"
Problem
- What should I do when a client backup fails with a WIN32 API error: "1450 - Not enough system resources to execute the requested service"?
- The backup of a client may end with the following error message in the backup log:
sbc-1148: Error: W2KSS Error: [WIN32 API error: 1450 - Not enough system resources to execute
the requested service. Cannot store registry key: [SOFTWARE]. RegSaveKey() call failed in BackupRegistry().].
Possible cause
Insufficient size of the registry/paged memory area. This problem affects SEP sesam as well as other backup tools, such as NTBackup.
⇒ Solution
The Microsoft article Backup program is unsuccessful when you back up a large system volume explains approaches for different versions of Microsoft Windows.
Unsuccessful backup of OneDrive files
Problem
SEP sesam is unable to access files in OneDrive remote storage. When attempting to back up files stored in OneDrive, SEP sesam log file displays the following warning:
Warning: Unable to open item - Access to the cloud file is denied.
Cause
OneDrive is a cloud-based service provided by Microsoft for storing, synchronizing, and sharing files. It has some unique characteristics that are specific to Microsoft's ecosystem and access to files is implemented differently than standard cloud solutions. SEP sesam may encounter difficulties in accessing and backing up the files that are stored in OneDrive cloud-like remote storage.
⇒ Solution
To resolve this issue, consider using the "offline mode" and syncing the files from OneDrive to local storage on the client device. This enables SEP sesam to bypass the issues with access and files are backed up from the local storage rather than the cloud. Ensure that you regularly update the local copy to reflect the changes in the cloud storage.
System state backup is backing up large amount of data due to DFS
Problem
When performing a Windows system state backup on the server that runs DFS (Distributed File System), large amount of data is backed up and the backup is slow.
Possible cause
If the DFS replication is enabled, the system state backup may include all the data from the DFS Replication service too.
⇒ Solution
Note | |
On a Domain Controller (DC) the DFS is used to replicate the SYSVOL, therefore it should be included in the system state backup. Use the following procedure only as a workaround if your system state backups are slow due to large amount of DFSR data. |
Exclude the DFS writer from the system state backup task in the SEP sesam GUI: from the Main Selection -> Tasks -> By clients, select your Windows client then double click on your system state backup task. In the properties window switch to the Options tab, and under Additional call arguments -> Backup options exclude the DFS writer as follows:
-x "VSS:/DFS Replication service writer"
Next, you have to back up the file system data, served by the DFS, separately by performing a regular Path backup (create a Path task type instead of a System_state backup task). For details, see Backing up System State.
System_State backup (RegLoadKey) error
Problem
- What does the warning "The system cannot find path. RegLoadKey()..." during System_State backup mean?
- You may see the following output in NOT-log:
C:\Program Files\SEPsesam\var\tmp\usr_wf_S-1-5-21-220523388-1123561945-839522115-1003]. 2020-04-13 02:04:20: sbc-2074: Warning: W2KSS Warning: [WIN32 API error: 3 - The system cannot find path. RegLoadKey() call failed for file: [C:\Documents and Settings\nn\ntuser.dat] in BackupUserProfiles().].
Possible cause
There are inconsistencies in the OS configuration. The reason is that a user profile has been deleted but the user account still exists. System_State backup is looking for files corresponding to the user in the file system but the files no longer exist.
⇒ Solution
- Delete the user in question or restore the profile date in the file system.
- Check the following in your registry to see whether it still includes references to usernames which no longer exist:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList
Access denied during Microsoft Windows backup via VSS
Problem
Why does a Microsoft Windows backup via VSS stop with the message "[CVssBaseObject::CreateVssBackupComponents] - Access denied"?
Possible cause
SEP sesam is not allowed to create a snapshot with the current user.
⇒ Solution
Check the user running the SEP sesam daemon and make sure that the user has all permissions to access the volume(s).
Stream data length exceeds buffer capacity
Problem
What does the warning "Stream data length bigger than buffer can accept. Input buffer length = [65536], Stream data size = (High part)[0] (Low part)[65564]" ?
Possible cause
SEP sesam uses 64 kB to back up Windows ACL files and folders and one object exceeds this buffer. You can use the Windows command icacls to display the ACL of a file or folder. The output looks like this:
C:\>icacls "C:\Documents and Settings\LocalService\Local Settings\Temp"
C:\Documents and Settings\LocalService\Local Settings\Temp NT AUTHORITY\LOCAL SERVICE:(I)(F) NT AUTHORITY\LOCAL SERVICE:(I)(OI)(CI)(IO)(F) NT AUTHORITY\SYSTEM:(I)(F) NT AUTHORITY\SYSTEM:(I)(OI)(CI)(IO)(F) BUILTIN\Administrators:(I)(F) BUILTIN\Administrators:(I)(OI)(CI)(IO)(F) Successfully processed 1 files; Failed processing 0 files
If you get several hundred or thousand lines, there is something wrong with the ACL.
⇒ Solution
Reset the permissions of the file's respective folder by using the command:
C:\>icacls "C:\Documents and Settings\LocalService\Local Settings\Temp" /reset
This command inherits the permissions of the parent object. You may have to adjust the permissions after running this command if manual settings have been applied for this object.
Backup on Windows 7 did not complete successfully
Problem
When you perform a Windows 7 backup, the backup might fail with one of the following errors.
sbc-1146: Error: DB Module: [WIN32 API error: 55 - The specified network resource or device is no longer available. sbc-2040: Warning: Cannot read item [\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy730\Windows\winsxs\x86_microsoft-windows-sort_31bf3856ad364e35_6.1.7600.16385_none_ab9479767ad67fd7\sort.exe: (2) WIN32 API error]:[ 2 - The system cannot find the file specified. ]. Padding remaining bytes... smk-3506: Info: Backup finished. Status: ERROR Error: Item generator returns [WIN32 API error: 2 - The system cannot find the file specified. ] for item [\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy136\Windows\winsxs\FileMaps\] smk-3506: Info: Backup finished. Status: ERROR Error: DB Module: [Not all items have been processed]
To see more detailed error, view the Event Log on your Windows client: Event Viewer -> Windows Logs -> System. The Event Id 33, volsnap shows the following message:
The oldest shadow copy of volume C: was deleted to keep disk space usage for shadow copies of volume C: below the user defined limit.
Possible causes
- There was not enough disk space for the Volume Shadow Service to take a snapshot; consequently, the operating system automatically deleted the snapshot due to the lack of space.
- This error occurs if the snapshot size exceeds the snapshot Maximum size: Use limit configured for the volume.
⇒ Solution
Check the Maximum size: Use limit on your Windows 7 client: Open a Windows Explorer on the client and right-click the drive letter to open the drive properties. Select the Shadow copies tab -> Settings -> check the Maximum size: Use limit:
- If you select No limit, Windows will not delete the snapshot regardless of how much space it consumes. Note, however, that in case of a large snapshot the process might be failing to create the snapshot because there is not enough disk space left on the respective drive.
- If you specify Use limit, change the maximum size limit to a large enough value to contain the snapshot with room to spare. According to Microsoft, it is recommended to set the maximum size limit to a value that equals 10% of the volume size. For example, if the C:\ drive has 100 GB of data, the limit should be set to 10 GB. Or, you can store the shadow copy data on another disk. For details, see Microsoft forum answers Shadow Copy.
ON LINUX
SEP sesam Linux Client error during backup
Problem
A SEP sesam Linux Client (SBC) issues an error or warning during backup.
Possible causes
Backup on Linux may finish with an error or warning if:
- the size of a file has changed during the backup
- a file is deleted during the backup (between 'find' and data processing)
- the 'find' function encountered an error
⇒ Solution
- To avoid these warnings and resolve the above errors, double-click the backup task to open its properties and under the Options tab in the Backup options field enter the following command:
-o ignore_finderr=<regex>|ALL
- If you want to avoid all such errors/warnings, specify:
-o ignore_finderr=ALL
GVFS error during Linux backup
Problem
- If a user is logged on to the gnome or kde session and makes use of the GVFS layer, the directory ~/.gvfs is created. This directory cannot be entered by any other user (even root).
- Additionally, the system call "stat" also errors out on this directory. The directory cannot be excluded, because while creating the file list and looking at the excludes, sbc_find once does a "stat()" call on the directory and receives an error.
⇒ Solution
- Create a new file named /etc/profile.local with the contents below /etc/profile.local:
GVFS_DISABLE_FUSE=1 export GVFS_DISABLE_FUSE
- Run the following commands for each affected folder:
fusermount -u /home/$USER/.gvfs
test with stat /home/$USER/.gvfs
Restore problems
Error mounting a saveset on SLES 15 SP4
Problem
- On SLES 15 SP4, when running guestfs tool to mount a saveset, the mount fails with the following error message:
error while loading shared libraries: libgcrypt.so.20: cannot open shared object file: No such file or directory
⇒ Solution
- Run the following commands on the affected SEP sesam Server:
echo '/usr/lib64/libgcrypt.so*' > /usr/lib64/guestfs/supermin.d/zz-libgcrypt
echo '/usr/lib64/libgpg-error.so*' > /usr/lib64/guestfs/supermin.d/zz-libgpg-error
Disaster recovery on Linux
Problem with a ReaR backup execution on SEP sesam Client
Problem
- ReaR backups on the SEP sesam Client are experiencing issues, but path backups are functioning correctly. The backup log may contain errors such as sbc_vadp requires additional libraries (fatal error). Example:
opt/sesam/bin/sesam/sbc_vadp requires additional libraries (fatal error) libpython2.7.so.1.0 => not found
Cause
- This issue is caused by the sbc_vadp binary, which was included in the 5.0.0.15 Service Pack 1 update, but is not required on the Client.
⇒ Solution
Delete the sbc_vadp binary. You can use the following command:
rm /opt/sesam/bin/sesam/sbc_vadp
The workflow mkrescue is not supported in the ReaR system
Problem
- The workflow mkrescue is not supported in the ReaR rescue/recovery system.
⇒ Solution
- Delete the file /etc/rear-release.
ReaR image hangs during bootup
Problem
- The system hangs during bootup like shown in the following image:
⇒ Solution
- Boot the system with the ACPI=OFF option (this option can be specified on the command line in the boot menu prompt, after the options BACKUP=SESAM OUTPUT=ISO).
The recovered system does not boot
Problem 1
- The system does not boot because /root/dev/console cannot be found.
Possible causes
- Certain distributions rely on the existence of the directory /dev/ while booting
- Certain static devices must exist before the udev daemon creates them.
⇒ Solution
- Include the /dev/ file system in your backup.
- If the restore cannot restore /dev/:
- Boot from the SEP sesam LIVE CD.
- Mount the ROOT partition of the restored system.
- Manually create the /dev/ directory.
- Manually create the /dev/console entry with:
mknod /path/to/target/mount//dev/console c 0 0
Problem 2
- The system does not boot because of missing libblkid.so.1.
Possible cause
- This is most likely caused by SELinux which is activated by default.
⇒ Solution
- Especially on RHEL6 or CentOS6 systems, follow these steps after rebooting from the ReaR recovery:
- Press a key when prompted by the boot loader (GRUB):
- Select the appropriate boot loader entry:
- Press e to modify the commands for the selected entry:
- Add selinux=0 to the commands:
- Press Enter to confirm the changes and b to boot up the machine with SELinux disabled.
- When having access to the system, change the option SELinux of /etc/selinux/config to the following: SELINUX=permissive
Afterwards, reboot the system and feel free to set the SELinux value back to enforcing if needed.
No bootable operating system can be found
Problem
- The system is not able to find a bootable OS instance after the restore.
Possible causes
- There may have been problems during the installation of the GRUB boot loader.
⇒ Solution
- The restore protocol includes a statement whether or not the installation of the boot loader was successful:
2009-12-14 14:48:27: sbc-3500: Info: Reinstall boot manager [/sesam/bin/sesam//sbc_grub_auto /mnt/disk/ AUTO]
- It is also possible to boot the system again from the live-CD, mount the target partitions and use grub-install to install the boot loader correctly.
The device does not have a corresponding BIOS drive
Problem
- During the restore, the following error occurs:
/dev/sda1 does not have any corresponding BIOS drive
Possible causes
- Check the file /boot/grub/device.map on the target system. If there are entries referring to the disk through /dev/by-disk/... as shown in the example below, the entry is most likely the reference to the hard disk partition of the broken system. GRUB will not find the proper device:
hd(0) /dev/disk/by-id/ata-SAMSUNG_SP2504C_S09QJ1GLA14263-part1
⇒ Solution
- Reboot from the live-CD
- Mount the root and boot partitions to /mnt/disk (and /mnt/disk/boot, if necessary)
- Restart grub-install with the following options:
grub-install --root-directory=/mnt/disk --recheck hd0
Output:
grub-probe: error: Cannot open `/boot/grub/device.map' /usr/sbin/grub-install: line 374: [: =: unary operator expected Installation finished. No error reported. This is the contents of the device map /mnt/disk/boot/grub/device.map. Check if this is correct or not. If any of the lines is incorrect, fix it and re-run the script `grub-install'. (hd0) /dev/hda (hd1) /dev/hdb
You can ignore the error line 374: [: =: unary operator expected.
More important is the result Installation finished. No error reported.
No corresponding BIOS drive for /dev/cciss/c0d0p2
Problem
- You receive the message: /dev/cciss/c0d0p2 does not have any corresponding BIOS drive in restore log.
⇒ Solution
- Please see: Novell support
fsck.ext3: File system has unsupported features
Problem
- During a restore of a system with kernel version 2.4 the system may not boot because the Live-CD creates a file system with features which are not supported by kernel 2.4.
Possible causes
- Most likely the file system options resize_inode,dir_index,large_file,ext_attr are causing the problem and making the system unbootable.
⇒ Solution
- Reboot from the Live-CD image, which includes the tool debugfs.
- Show the file system features with debugfs:
root@recover#: debugfs -w /dev/sda2 debugfs 1.41.1 (01-Sep-2008) debugfs: features Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file quit
Replace /dev/sda2 with the corresponding partition names on your system.
- To remove file system features:
root@recover#: debugfs -w /dev/sda2 debugfs: features -resize_inode -ext_attr -dir_index -large_file -needs_recovery -sparse_super Filesystem features: has_journal filetype quit
After removing the options, the system should boot correctly.
Incorrect inode size (256)
Problem
- After a successful restore the boot process stops with incorrect inode size (256).
Possible causes
- Older kernel versions (2.4) may use a different inode size than the one the file system's created through the Live-CD (which includes kernel 2.6). For example, this happens during the restore of SLES8 based systems which use an inode size of 128k.
⇒ Solution
- This can only be solved by formatting the devices manually from the Live-CD, using the proper mkfs options:
mkfs.ext3 -I 128 /dev/sda1
After this step, remount the partition to /mnt/disk and repeat the restore operations. Changing the inode size is only possible by reformatting the devices.
Missing root file system
Problem
- The restored system can't find a root file system and fails during resume.
Possible causes
- The /etc/fstab file was configured with the root file system as UUID.
⇒ Solution
- Specify the root file system device name in conventional device names if you are using a different physical disk. After booting, use YAST to reconfigure your boot loader or edit your /boot/grub/menu.lst manually:
root=/dev/sda2
Missing network cards
Problem
- The restored system does not find any network cards.
Possible causes
- If the restore was done to dissimilar hardware, SLES-based distributions may not configure the network devices correctly. SLES-based systems save their network configuration by using the system's MAC address. Most likely the system will not use eht0 as a device name, but eth1, as it has another MAC address.
⇒ Solution
- Use YaST and reconfigure your network interfaces.
Client does not start on the RHEL6/Debian9 recovery image
Problem
- The SEP sesam Client does not start automatically on RHEL6 and Debian9-based recovery images.
Cause
- The file /etc/init.d/functions is missing within the recovery image.
⇒ Solution
- The client can be started manually via:
/opt/sesam/bin/sesam/sm_main start
RHEL7 backup fails with an error
Error 1
- The RHEL backup fails with the following error:
ERROR: The LSB package is not installed.
⇒ Solution
- Install the lsb package as follows:
yum install redhat-lsb-core mkisofs syslinux
Error 2
- The RHEL backup fails with:
ERROR: Cannot find required programs: mingetty
For more details, see Rear dependencies on RHEL7.
⇒ Solution
To solve this problem, proceed as follows:
- Edit
- Run the backup again.
/var/opt/sesam/var/lib/rear/usr/share/rear/conf/default.conf
and from the line:
# required programs. Same as above, but if they are missing, we abort. REQUIRED_PROGS=( "$SCRIPT_FILE"
remove the line:
mingetty
ReaR error occurred during grub2-mkimage of bootx64.efi
Note | |
In order to be able to create an UEFI/EFI bootable ISO image, the additional tool ebiso has to be installed on the client system as described in the section Installing ebiso for creating UEFI aware ISO images. |
Problem
- The ReaR error occurrs during grub2-mkimage of bootx64.efi.
⇒ Solution
- To solve the problem, install the grub2-efi-x64-modules package.
SM_SSH does not work on SLES11 recovery image
⇒ Solution
- In this case, execute
mount -t tmpfs none /dev/shm/ -o rw,nosuid,nodev,noexec
- before starting the recovery process.
Client is unreachable after booting the rescue image
Problem
- After booting the rescue image the client is not reachable.
⇒ Solution
- Start the client manually using the following command on the rescue command line:
sh /etc/scripts/system-setup.d/59-start-sesam-client.sh
EFI bootable image cannot be created on SLES11
Problem
- EFI bootable image of GRUB2 cannot be created on SLES11.
Cause
- SEP sesam v. 4.4.3.64 Grolar is the last version that supports SLES11 with UEFI.
⇒ Solution
- To continue using SLES with UEFI, you should not upgrade to a later version of SEP sesam.
Installing ebiso for creating UEFI aware ISO images
In order to be able to create an UEFI/EFI bootable ISO image the, the additional tool ebiso has to be installed on the client system. This package is not part of a regular SLES12/SLES15 installation and can be downloaded at the following URL:
http://download.opensuse.org/repositories/Archiving:/Backup:/Rear/SLE_12/x86_64/
or
http://download.sep.de/utils/bsr-linux/
For other Linux distributions contact SEP support at support@sep.de for assistance.
Install ebiso as follows:
rpm -i ebiso-<version>.rpm
Note that in ReaR v. < 1.19, the generated ISO image mount migt be too small for storing all needed information and need to be adjusted.
In this case, under
/var/opt/sesam/var/lib/rear/usr/share/rear/lib/uefi-functions.sh (line 64)
change
(shim.efi|elilo.efi) size=128000 ;;
to
(shim.efi|elilo.efi) size=228000 ;;
Graphical User Interface (GUI) problems
After updating SEP sesam on Windows, GUI cannot be opened
Problem
After updating SEP sesam to version ≥ 4.4.3.82 Beefalo V2 on Windows, GUI cannot be opened when the group policy option Prevent access to registry editing tools is applied.
Possible cause
This issue seems to be related to the group policy (GPO) setting: if Prevent access to registry editing tools is applied for a specific user or group, opening the GUI does not work for the users which are prevented from accessing the Registry. In most cases, users with admin privileges will still be able to run the GUI as they are typically exempt for a GPO (and are able to access the Registry). If this policy is disabled (or not configured), the users may access the GUI regardless of their privileges.
⇒ Solution
To enable access to the GUI for the affected users (typically non-admin users), disable the group policy option Prevent access to registry editing tools.
Warning | |
Only experts should edit the registry; modifying Windows Registry incorrectly might crash your Windows operating system causing data loss. |
- Press the Windows key + R to open the Run window.
- Type gpedit.msc and press Enter.
- In the Group Policy Editor -> User Configuration -> Administrative Templates -> System, double-click on Prevent access to registry editing tools setting in the right panel and select Disable. Then click Apply.
GUI Server not accessible
Problem
SEP sesam cannot access the GUI Server.
Possible causes
- The network connection to SEP sesam is broken.
- The GUI Server process is not running.
⇒ Solutions
- Make sure that the computer is running.
- Start the GUI server process by running the command sm main reload rmi.
Database connection failure
Problem
SEP sesam cannot connect to the database.
Possible cause
The DB or RMI servers are not running.
⇒ Solution
Run the commands sm main reload db or sm main reload rmi to restart the servers.
GUI does not start
Problem
The Graphical User Interface (GUI) fails to start.
Possible causes
- There is a problem with Java rights.
- A problem with the screen resolution.
⇒ Solution
In case of the screen resolution problem, set a resolution to at least 1920x1080 (full HD).
Wrong time values in the GUI
Problem
In GUI, the displayed time values may not correspond with the system's default timezone.
Possible causes
- This can be caused by exporting the system-wide environment variable TZ in the system-wide /etc/profile file, which can override the default system locales, leading to an incorrect timezone setting in the postgresql.conf file.
- On Debian, the timezone settings are stored in the /etc/timezone file. If the timezone on the system is changed using the timedatectl set-timezone command, only the /etc/localtime file is updated. The /etc/timezone file remains unchanged.
⇒ Solution
- In the /etc/profile file update the TZ environment variable and set the timezone that matches the desired system default timezone. This will ensure that the timezone setting in the postgresql.conf file is aligned with the system default timezone.
- On Debian, update the settings in /etc/timezone file to set the correct timezone.
Online guide not available
Problem
The online guide cannot be accessed.
Possible causes
- Adobe Acrobat Reader is not installed.
- Adobe Acrobat Reader is installed but not configured in the GUI.
- The PDF file is not configured in the GUI.
⇒ Solution
Download Adobe Acrobat Reader from http://www.adobe.com.
Online help not available
Problem
The online help cannot be accessed.
Possible causes
- There is no browser installed on the system.
- The browser is installed but not configured in the GUI.
⇒ Solution
Install a browser.
Network problems
Network check
Problem
- Network connection is not working.
⇒ Solution
- Perform the network check. Use the commands ping, nslookup, address resolution etc. Check the connection with the corresponding SEP sesam access program (CTRL/SSH). The address resolution must be consistent, i.e., if for a TCP/IP name the resolution gives an IP-address, then the resolution for that IP-address must give the same TCP/IP name!
Example
# nslookup decunix Server: seplinux2.sep.de Address: 193.28.59.40
Name: decunix.sep.de Address: 193.28.59.94
# nslookup 193.28.59.94 Server: seplinux2.sep.de Address: 193.28.59.40
Name: decunix.sep.de Address: 193.28.59.94
Microsoft SQL Server
Incorrect login
Problem
- If a backup or restore ends with errors and you find the following information in the log files, an attempt has been made to address a SQL Server instance on a client that is not entered locally on this system. The selected Trusteed Connection only allows the registration on a SQL Server where the instance is locally active:
DB Module: [DB-Library: Login failed for user '(null)'. Reason: Not associated with a trusted SQL Server connection.] DB Module: [DB-Library message: Login incorrect.]
⇒ Solution
- The backup client or end node on the active SQL Server must be changed.
Microsoft SQL Server backup failure
Problem
- Backup fails with the message "The server principal "NT AUTHORITY\SYSTEM" is not able to access the database "<database>" under the current security context."
Possible causes
- The user SYSTEM is not allowed to connect to the MS-SQL database because the SEP sesam daemon runs as the user SYSTEM by default.
⇒ Solution
- Change the SEP sesam daemon to a user account that has permission to connect to the database(s). Open the Windows service management console on your MS-SQL server, edit the properties of the SEP sesam service, switch to Log on tab and use valid credentials, then restart the SEP sesam daemon.
MS SQL Server backup shows message "The Transaction Log For Database Is Full"
Problem
- During backup of MicroSoft SQL Server the backup log can show the message: "The transaction log for database '<DATABASE>' is full due to 'LOG_BACKUP'".
Possible causes
- This error message indicates that the LOG area is full. This can happen when no INC backup are made, only FULL backups.
⇒ Solution
- Increase the size of the LOG area and perform a FULL and INC backup so that the LOGs can be truncated correctly. Then, set up regular INC backups so that the LOG area does not fill up again.
For more information on resolving this error see here.
Restoring a database from AlwaysOn Availability Group fails with RESTORE cannot operate on database 'Test1' because it is configured for database mirroring or has joined an availability group
Note | |
MS AlwaysOn is only supported by SEP sesam with the assistance of SEP sesam support. |
Problem
- The MS SQL AlwaysOn Availability Group database restore fails with an error:
DB Module: [UNKNOWN error: 4294967295 -. Got error: [SQL Server]RESTORE cannot operate on database 'Test1' because it is configured for database mirroring or has joined an availability group. If you intend to restore the database, use ALTER DATABASE to remove mirroring or to remove the database from its availability group.]
Cause
- It is not possible to restore the MS SQL database which is part of AlwaysOn Availability Group.
⇒Solution
To restore a database which is part of AlwaysOn Availability Group successfully, proceed as follows:
- Remove the database from the AlwaysOn Availability Group.
- Restore the database to the primary replica in the AlwaysOn Availability Group.
- Add the restored database to the AlwaysOn Availability Group again by using Full as a data synchronization option.
Afterwards the database will be replicated to the secondary replicas.
Restore fails with The system cannot find the path specified
Problem
- If the original database was removed or the database should be restored to a different location, then the path for the database must be created before the restore. If the path for the database does not exist, the following message appears:
Directory lookup for the file "C:\Programme\Microsoft SQL Server\MSSQL$ZWEITE_DB\data\sesamdb_Data.MDF" failed with the operating system error 3(The system cannot find the path specified.). File 'sesamdb_Data' cannot be restored to 'C:\Programme\Microsoft SQL Server\MSSQL$ZWEITE_DB\data\sesamdb_Data.MDF'. Use WITH MOVE to identify a valid location for the file.
Cause
- The system cannot find the database path if the original database was removed or the database path wasn't created before restoring to a different location.
⇒Solution
- In this case you have to restore the database to a new location, as described in Restoring a database to a new location by using GUI relocation or move option.
Restore fails with Directory lookup for the file '...' failed...
Problem
- You receive the following warning:
DB Module: [DB-Library: Directory lookup for the file "e:\Database\SQL Server 2000 SE
\MSSQL\Data\sesam.mdf"failed with the operating system error 21(The device is not ready.).] DB Module: [DB-Library: File 'sesam_db' cannot be restored to 'e:\Database\SQL Server 2000 SE
\MSSQL\Data\sesam.mdf'. Use WITH MOVE to identify a valid location for the file.]
Possible causes
- The existing path where the database files were located is not on the target system or an entry was made in the "Move" option that used a non-existing path name.
⇒ Solution
- A path name must be created or the correct path must be entered in the Move option. Entering the correct path name is the easier solution. Using a long path name in the Move option may cause problems, in which case the input can be shortened.
Restore failure due to SQL Server connection
Problem
- The restore fails with the following error:
DB Module: [DB-Library message: Unable to connect: SQL Server is unavailable or does not exist. Unable to connect: SQL Server does not exist or network access denied.; Net-Library message: ConnectionOpen (Connect()).; ]
Possible causes
- The selected server does not exist. It is possible that an instance was incorrectly entered without the server name.
⇒ Solution
- Check the server name. If necessary, enter the restore target fully qualified as follows:
<HOSTNAME>/<Instance>/<DB Name>
Restore with Move option fails
Problem 1
- Restore using the Move option fails with the warning The physical file name '...' may be incorrect. The following error occurs:
DB Module: [DB-Library: A file activation error occurred. The physical file name
'c:/temp/sesam_log.ldf' may be incorrect. Diagnose and correct additional errors, and retry the operation.] DB Module: [DB-Library: File 'sesam_db_log' cannot be restored to 'c:/temp/sesam_log.ldf'. Use WITH MOVE to identify a valid location for the file.]
Possible causes
- The wrong syntax was used in the file name, e.g., / instead of \.
⇒ Solution
- Enter the path with the correct syntax.
Note | |
If the SEP sesam Server is using a Postgres database, e.g., with Linux x64, then the character '\' must be entered twice '\\', otherwise it will disappear. For example:
-a move=Mgmt_data:"e:\\SQL Server 2000 SE\\MSSQL\\Data\\Mgmt.mdf" -a move=Mgmt_log:"e:\\SQL Server 2000 SE\\MSSQL\\Data\\Mgmt_log.ldf" |
Problem 2
- Restore with Move option fails with Logical file '...' is not part of database. The following error occurs:
DB Module: [DB-Library: Logical file 'Mgmt_data' is not part of database 'sesam_db2'. Use RESTORE FILELISTONLY to list the logical file names.]
Possible causes
- The wrong logical name was entered in the Move option.
⇒ Solution
- Enter the logical filename correctly. This can be found in the backup log file (NOT-file). For example, in the backup log file, you see the following lines:
DB Module: [DB-Library: Processed 256 pages for database 'sesam_db', file 'sesam_db' on file 1.] DB Module: [DB-Library: Processed 1 pages for database 'sesam_db', file 'sesam_db_log' on file 1.]
The logical file names in this case are sesam_db and sesam_db_log. These should be entered in the Move option.
Restored database remains in the state Restoring
Problem
- A database remains in state Restoring... after the restore finishes.
Possible causes
- This happens if the option Auto recover was not selected in the restore wizard.
⇒ Solution
- Select option the option Auto recover in the restore wizard or execute sbc in the CLI with the option -a recover for the particular database. For example:
- Call the sbc in the command line with:
sbc -r -a recover sbcmsql:"/MIRACULIX/SECOND/msdb"
Note | |
During the restore of a database with additional transaction log files (generation restore), the database remains in the state "Restoring..." until the last restore (with the option -a recover) ends. |
Disaster recovery fails with ODBC SQL Server Driver][DBMSLPCN]SQL Server does not exist or access denied.
Problem
- The MS SQL disaster recovery fails with an error:
ODBC SQL Server Driver][DBMSLPCN]SQL Server does not exist or access denied.
Cause
- This error typically indicates either network or installation-related issues; the SQL Server does not exist, is not available or is not found.
⇒Solution 1:
- If possible, restore the system databases from the latest path backup. If no such backup exists, check solutions 2 and 3.
⇒Solution 2:
- Create a new MS SQL instance. For this, you will need the MS SQL install files. For details, see How to manually create the Microsoft SQL Server 2014 ACT7 Instance. Then restore the MS SQL databases to the new instance with the target path <server_name>/<instance_name>.
⇒Solution 3:
You can also rebulid MS SQL system databases. The procedure differs slightly if you rebuild the databases from CD or DVD drive (MS SQL Server 2005) or from the installation folder on your local computer in \Binn\Templates\ (MS SQL Server ≥ 2008), for example, C:\Program Files\Microsoft SQL Server\MSSQL10.MSSQLSERVER\MSSQL\Binn\Templates
. You have to run the setup.exe command from the command prompt, then proceed as follows:
- Click Start -> Run, type cmd, and click OK.
- Depending on your MS SQL version, run the relevant command to rebuild the system databases:
- To rebuild the system databases from CD or DVD drive (MS SQL Server 2005), run the following command:
start /wait <CD_or_DVD_drive>\setup.exe /qn INSTANCENAME=<instance_name> REINSTALL=SQL_Engine REBUILDDATABASE=1 SAPWD=<strong_password>
For example:
start /wait D:\setup.exe /qn INSTANCENAME=MSSQLSERVER REINSTALL=SQL_Engine REBUILDDATABASE=1 SAPWD=p@ssw0rd
For details, see How to Rebuild System Databases in SQL Server 2005.
setup /ACTION=REBUILDDATABASE /QUIET /INSTANCENAME=<instance_name> /SQLSYSADMINACCOUNTS=<accounts> /SAPWD=<strong_password>
For details and troubleshooting, see How to Rebuild System Databases in SQL Server 2008.
Microsoft Exchange Server
Exchange backup fails with VSS API error due to missing Microsoft Exchange VSS writer
Problem
When backing up Exchange, the backup fails because the Microsoft Exchange VSS writer required for backup is missing. The following error occurs:
sbc-1178: Error: VSS API error: CVssServer::CreateSnapshot: No volume could be determined. sbc-1146: Error: DB Module: [BackupProcessing: For the specified backup source no volumes could be determined. Sources - \Microsoft Exchange Writer] sbc-3052: Info: Items processed correctly: [0]. Not processed or incorrectly processed items: [0]. (SF20160503083743700@BLljg0gnWAa) sbc-1156: Error: Operation failed! sbc-3001: Info: Exiting.
This error occurs when one or more writers required for backup are missing or not available. Note that this is not the same as VSS error or failed VSS. In the latter case, check Common VSS problems.
A writer is consider missing, if running (as administrator) the Microsoft command-line tool vssadmin list writers
shows no available Microsoft Exchange VSS writer. If a VSS writer is missing, all backups that use that writer to perform VSS snapshots will fail. A missing writer is a failure of the Windows operating system. To fix this issue, Windows registry needs to be edited. SEP sesam cannot back up any data until the required VSS writers are available.
Solution
Note | |
SEP sesam is not responsible for any issue caused by Windows registry editing. Note that only experts should edit the registry, as using Windows registry editor incorrectly can cause serious problems, such as Windows to stop working. For more information on editing Windows registry, see Windows registry information for advanced users. It is advised to back up the registry before making any change to it. It is also recommended to perform the following procedure outside of working hours as it requires restarting the Microsoft Exchange Information Store service. |
To enable Microsoft Exchange VSS writer, on the Exchange server open and edit the registry with the following key value:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS\ParametersSystem\ "Disable Exchange Writer" DWORD=0
If the key already exists with a value of 1, change the value to 0. (Value 1 disables the Microsoft Exchange Writer.) For more information on enabling VSS Exchange writer, refer to How to turn on the Exchange writer for the Volume Shadow Copy service (it is marked as relevant for Windows Small Business Server 2003, but the procedure can also be applied to later versions).
Then restart the Microsoft Exchange Information Store service:
- Log in to the Exchange server and click Start.
- Enter services.msc in the search box, and press Enter. The Services window opens.
- From the list of services, right-click the Microsoft Exchange Information Store service (MSExchangeIS), and then click Restart. Note that during restart the users will be disconnected from Exchange.
After restarting Exchange, in the command prompt enter vssadmin list writers
and verify that the VSS writers are listed. Then run your Exchange backup again.
Exchange logs truncation errors
Problem
The Exchange logs are not truncated or the log files are missing.
Cause
Exchange database log truncation is part of Exchange services. SEP sesam notifies Exchange Writer of each successful backup of Exchange Server and Exchange DAG nodes. SEP sesam does not manage or delete Exchange logs and is therefore not responsible for truncating logs, Microsoft Exchange Writer is. SEP sesam only notifies Exchange of the backup status and then:
- In configurations without DAG, Exchange Writer truncates the transaction log files after a successful full or incremental backup.
- In DAG replicated configurations, log truncation is delayed by the replication service until all required log files have been replayed into all other copies. The replication service deletes the backed-up log files from both the active and the copy log file paths after verifying that the log files have been successfully applied to the copy database and that both the active database and the database copies checkpoint have passed the log files to be deleted.
⇒ Solution
To verify that Exchange log truncation has been called, check the Applications and Services Logs and look for errors related to the Exchange VSS or Replica writer.
Check ESE events with ID:
- 224: This event indicates that logs are being deleted and denotes the associated database.
- 225: This event indicates that there are no logs available for truncation (either ESE cannot determine whether the transaction log files have been committed to the Exchange Server database, there are not enough logs or Circular Logging is enabled).
- 299: The replication service truncates the log stream and removes transaction logs that are no longer needed (or reports that there is not a minimum amount of logs for truncation).
If the Exchange configuration uses DAG, the replication process between the different nodes can prevent log truncation in case of errors. These errors are indicated with the event source MSExchangeRepl.
Below are some entries from the application log in the Event Viewer (for a list of errors, see the Microsoft article MSExchangeRepl Errors and Events):
· Event ID 224 (ESE) MDB01 deleting log files C:\ExchVols\MDB01\Log Files\E0000000001.log to C:\ExchVols\MDB01\Log Files\E000000002B.log. · Event ID 225 (ESE) MDB01—no log files can be truncated; will be logged instead of Event ID 224 when circular logging is used. · Event ID 2046 (MSExchangeRepl) VSS writer has successfully completed the backup of database MDB01. · Event ID 2006 (ESE) MDB01 shadow copy completed successfully. · Event ID 2033 (MSExchangeRepl) VSS writer has successfully processed the backup completion event. · Event ID 2037 (MSExchangeRepl) VSS writer backup has been successfully shut down.
If errors occur in connection with the Exchange VSS or Replica Writer, check the Exchange configuration to determine the cause and resolve the errors. Such errors cannot be caused by SEP sesam, but until they are resolved, Exchange cannot truncate the log files, resulting in filling up the volume or disk where they reside.
Also check the current status of the VSS writer by running the following at an administrative prompt on the Exchange server:
vssadmin list writers
If the writer is missing or is in a failed state, reboot the Exchange server and verify that it is now listed and showing as stable. Then restart the SEP sesam Exchange backup task and check the log truncation.
Exchange Recovery Pro asks for License
If you have started Exchange Recovery Pro the first time you get prompted to install the license. If you use several different Windows users, you get this prompt for every user.
NetWare
Checking the reachability of the TSA services
Problem 1
- Determining all available TSAs of a NetWare/OES Server.
⇒ Solution
- On a NetWare Server (the sbc_smdr command must be executed on a OES Linux machine with the sesam-novell-client package installed (up to OES2015) or on a OES2018 server (the sesam-novell-client is already included in the SEP Sesam SLES12 Linux Client package):
#> /opt/sesam/bin/sesam/sbc_smdr -D -N "nw1:::admin.admins.mydomain:novell:0" "/NetWare" 2007-03-21 16:42:50: sbc-3500: Info: ::/NetWare 2007-03-21 16:42:50: sbc-3036: Info: # @(#)SESAM BACKUP CLIENT FOR NETWARE FILE SYSTEMS, VERSION: 1.8R3 Build: 1.161 20070320 17:31:00 Linux i386 abas # 2007-03-21 16:42:50: sbc-3074: Info: Backup start time [20070321164250] 2007-03-21 16:42:50: sbc-3500: Info: Starting Session "SESAM SBC_NLM Session" ... 2007-03-21 16:42:52: sbc-3500: Info: NW1.NetWare File System "NW1.NetWare File System" d_ 2000.01.01 00:00:00 2000.01.01 00:00:00 4096 - , 2007-03-21 16:42:52: sbc-3500: Info: NW1.Novell Directory "NW1.Novell Directory" d_ 2000.01.01 00:00:00 2000.01.01 00:00:00 4096 - ,
In this case, the TSAFS (NW1.NetWare File System) and TSANDS (NW1.Novell Directory) are loaded.
- On an OES(2)-Linux server:
#> /opt/sesam/bin/sesam/sbc_smdr -D -N "oesnix1::oesnix1:backup:novell:0" "/NetWare" 2007-03-21 15:55:01: sbc-3500: Info: ::/NetWare 2007-03-21 15:55:01: sbc-3036: Info: # @(#)SESAM BACKUP CLIENT FOR NETWARE FILE SYSTEMS, VERSION: 1.8R3 Build: 1.161 20070320 17:31:00 Linux i386 abas # 2007-03-21 15:55:01: sbc-3074: Info: Backup start time [20070321155501] 2007-03-21 15:55:01: sbc-3500: Info: Starting Session "SESAM SBC_NLM Session" ... 2007-03-21 15:55:01: sbc-3500: Info: OESNIX1.GroupWise System "OESNIX1.GroupWise System" d_ 2000.01.01 00:00:00 2000.01.01 00:00:00 4096 - , 2007-03-21 15:55:01: sbc-3500: Info: OESNIX1.Linux File System "OESNIX1.Linux File System" d_ 2000.01.01 00:00:00 2000.01.01 00:00:00 4096 - ,
The TSAFSGW (OESNIX1.GroupWise System) and TSAFS (OESNIX1.Linux File System) are loaded.
Problem 2
- Determining the sources (resources) of a Target Service File System.
⇒ Solution
- Depending on your installed services, these TSA's are visible:
- NetWare File System
- Linux File System
- GroupWise System
- Novell Directory
- iFolder Store
- Linux Cluster File System
- NetWare Cluster File System
Tip | |
To enable debug mode for TSA, see Enabling the Debug Options in TSAFS on OES Linux. |
Unable to browse or backup a NetWare Server
Problem
- It is not possible to browse or back up a NetWare Server.
Possible causes
- No TSA is loaded, or
- The wrong data mover was inserted.
⇒ Solution
- Restart the NetWare file system with unload TSAFS and load TSAFS.
- Edit the properties of the NetWare Server in SEP sesam and adjust the data mover in the NetWare Access tab.
When using OES 2018 SP2 with SMDR/TSA, the TSA backup may fail
Problem
- When using OES 2018 SP2 with SMDR/TSA, the following problem occurs: If two TSA modules (e.g., tsafs and tsands) are loaded via smdr.conf, one of them does not work. Consequently, the TSA backup may fail under certain circumstances.
⇒ Workaround
- Start only one TSA module via smdr.conf, the other module should be loaded manually later. As this is a Micro Focus bug, see the Micro Focus community issue smdr on OES2018SP2 not working for more details.
Sesam Error: Client returns no data
Problem
- The following error message appears in /var/log/messages on OES Linux client:
nds_nss_GetGroupsbyMember: failed to init socket, status = 0 nds_nss_GetPwdbyName: init sock returned 0 nds_nss_GetPwdbyName: init sock returned 0 nds_nss_GetPwdbyName: init sock returned 0
Possible causes
- Incomplete or wrong eDirectory LUM configuration.
- eDirectory Server certificate is invalid/expired.
⇒ Solution
- Fix the lum setup so that eDirectory authentication can complete.
- Use iManager and check and/or recreate the server certificates.
Error messages during a backup
Message 1
(0XFFFDFFD7) "Login denied".
Possible causes
- Invalid username or password.
- eDirectory Server certificate is invalid/expired.
⇒ Solution
- Edit the properties of the client in SEP sesam and adjust the login data in the NetWare Access and Novell SMS tabs.
- Use iManager and check and/or recreate the server certificates.
Message 2
(0XFFFDFFCD) "A data stream cannot be opened." "Unable to open a data stream."
Possible causes
- The user does not have permission to access all data.
⇒ Solution
- Change the user name or adjust the permissions.
Message 3
(0XFFFEFFCC) "Could not write an object to NDS or write to a stream"
or
(0XFFFEFFB1) "Connection to remote host is lost. Remote host might have disconnected."
Possible causes
- The TSA or the smdr deamon is not running.
⇒ Solution
- Check TSA with opt/novell/sms/bin/smsconfig -t.
Message 4
(0XFFFDFFDC) "An invalid path was used."
Possible causes
- A wrong backup path was used or the TSAFS was loaded in an incorrect mode.
⇒ Solution
- In the event of --tsamode=dual, it is possible to simply use a Linux path.
- The source NetWare server backs up all existing volumes respectively.
- "VOL-NAME:" backs up a specific volume when using --tsamode=netware.
- The option --tsamode=linux supports path to the volumes/volumename only.
- In the event of a restore, insufficient access rights could be a reason for this message.
Files are locked after backup, server freezes
Problem
- Files are locked for several minutes after a backup and the server freezes spontaneously, requiring a reset.
⇒ Solution
- Update the sesam-novell-client to a higher version to solve this problem.
File-based backup is faster than Novell volume backup
Problem
- The file-based backup is faster than the Novell volume backup.
⇒ Solution
- Disabling the TSA-cache will increase the performance of a Novell-based backup and also decrease the system load.
- Edit the file /etc/opt/novell/sms/tsafs.conf, change the line "cachingmode=enable" to "cachingmode=disable" and apply the settings with rcnovell-smdrd restart.
Checking the performance and availability of a NSS volume via tsatest on OES2
- Back up the resource '/' using the supplied credentials.
tsatest -u root -p unsecure
- Back up the resource /home using the supplied credentials.
tsatest --path=/home -u root -p unsecure
- Back up the NetWare target SYS:\SYSTEM on ACME_SERVER using the supplied credentials.
tsatest -s ACME_SERVER -v SYS: --path=SYSTEM -u root -p unsecure
- Back up the '/' resource using the supplied credentials and a buffer size of 131072 bytes.
tsatest -b 131072 -u root -p unsecure
- Back up the resource '/' on server ACME_SERVER using the supplied credentials. This will perform a remote backup if ACME_SERVER is not the server on which tsatest is loaded/executed.
tsatest -s ACME_SERVER -u root -p unsecure
To check the performance on a NetWare Server, use these instructions: http://support.novell.com/docs/Tids/Solutions/10092890.html
Micro Focus GroupWise
GroupWise configuration file gwha.conf does not exist
Problem 1
- GroupWise configuration file gwsep.conf does not exist in the default folder
/etc/opt/novell/groupwise/
.
⇒ Solution
- Copy the file gwsep.conf from
/opt/sesam/skel/templates/
to the folder/etc/opt/novell/groupwise/
. - Edit gwsep.conf and add the following line in the [MISC] section:
gwha=<path to gwha.conf>/gwha.conf
For example:
gwha=/etc/mygw/config/gwha.conf
Problem 2
- An issue with the libraries with external storage areas.
⇒ Solution
- Copy the file gwha.conf from
/opt/sesam/skel/templates/
to the folder/etc/opt/novell/groupwise/
on the GroupWise server. - Add one section per external storage area as follows:
[<Name of storage area>.<post office>.<domain>] home=<path to storage area>
For example:
[ablage1.po1.dom1] home=/var/grpwise/ablage1
Tip | |
The file gwsep.conf must be copied to all cluster nodes and edited. It must be identical on all nodes. |
Oracle
Testing the Oracle extension with sbttest on AIX does not work
Problem
- Testing the Oracle extension with sbttest on AIX will not work unless the full path to the library with argument -libname is specified.
⇒ Solution
- When running sbttest on AIX, specify the full path to the library with argument -libname, e.g.,
sbttest test1 -libname /opt/sesam/bin/sesam/libobk.so
or ...-libname $ORACLE_HOME/lib/libobk.so
.
Errors when attempting to perform a backup on AIX
Problem
- Running RMAN command on AIX ends with errors.
⇒ Solution
- RMAN command on AIX requires that the full path to the library is set in the script via PARMS SBT_LIBRARY={full_path_to_libobk.so}. For details, see RMAN specific parameters.
Rerunning the sbttest script ends with duplicate key error
Problem
- When rerunning the sbttest script with the same backup_file_name parameter, SEP sesam returns duplicate key error.
Cause
- This happens because the sbttest is using the same backup_file_name argument on the next run. SEP sesam interprets backup_file_name as the save_set_id and compares it with the IDs in the results table. When the same save_set_id is found, SEP sesam returns duplicate key error.
⇒ Solution
- When running sbttest, make sure that the backup_file_name argument is set to a different value for each run of the script.
Running bash script on AIX results in bad interpreter: No such file or directory error
Problem
- When running a bash script, a bad interpreter: No such file or directory error message is shown, for example:
-sh: ./sbc_oracle_rman.sh: /bin/bash: bad interpreter: No such file or directory
Cause
- Typically, on AIX bash is not included in the list of valid shells.
⇒ Solution
- Change the first line in sbc_oracle_rman.sh to
#!/bin/sh
.
ORACLE_HOME and ORACLE_SID variables are not set in the user environment
Problem
- If ORACLE_HOME and ORACLE_SID are not defined, SEP sesam script cannot connect to the target database.
⇒ Solution
Set ORACLE_HOME and ORACLE_SID variables using any of the following:
- Add the lines
export ORACLE_HOME=/u01/app/oracle/product/10gR2/db_1
andexport ORACLE_SID=PROD_DB
. - Use oraenv to set the appropriate environment, for example:
export ORACLE_SID=TEST export ORAENV_ASK=NO . oraenv
The RMAN file name is too long
Problem
- You receive the warning:
ORA-19506: failed to create sequential file, name="full_COMP1_1953897796_55938_1.bck", parms=""
Possible causes
- The file name of the backup set created by the RMAN is longer than 64 characters. For example, if the Oracle backup file format is set to format 'full_%d_%I_%s_%p.bck', when the parameter %s (backup set number) is integrated into the filename, the backup set number increases with every backup. As a consequence, the count of characters of that string will increase. If the filename length exceeds 64 characters, this will cause the backup to fail.
⇒ Solution
- Make sure that the character length of the backup set does not exceed 64 characters.
Informix
General troubleshooting
- Ensure that the following environment variables are correctly set:
- INFORMIXDIR
- INFORMIXSERVER
- ONCONFIG
- ROOTPATH
- Make sure to check the messages on the SEP sesam Server.
- Additional information can be found in the ONBAR and SIB log files.
- SEP recommends using a single log file for ONBAR and XBSA messages. You will then be able to see all calls from the database and SEP sesam in the correct order for this set environment variable:
XBSA_LOGFILE=<complete path to bar_act.log> and XBSA_TRACE=1.
- All SIB (sesam) messages have the prefix SIB.
- For more information, set XBSA_TRACE to 2, but then the log files can become quite large.
- To activate the ONBAR log, edit the following line in the onconfig file:
BAR_DEBUG 2 \# where 'num' = 0-9; 9 producing heaps output defaults to /tmp/bar_dbug.log
Specific error messages
Error message when executing oninit
Allocating and attaching to shared memory...FAILED oninit: Fatal error in shared memory creation
⇒ Solution
- For details, see IBM documentation.
Error during first backup attempt
Begin backup of critical file 'C:\PROGRA~1\IBM\Informix\11.70\etc\ixbar.0'. (-43078) Open or close failed on file 'C:\PROGRA~1\IBM\Informix\11.70\etc\ixbar.0', errno = 2 .
⇒ Solution
- Create the missing file in
C:\PROGRA~1\IBM\Informix\11.70\etc\ixbar.0
.
Error 131 during first logical logs backup attempt
Unable to start the logical log backup: Log backup to device 'nul' not allowed ... C:\PROGRA~1\IBM\Informix\11.70\bin\onbar_d complete, returning 131 (0x83)
⇒ Solution
- Set LTAPEDEV in ONCONFIG file to a value different to '/dev/null' or 'NUL', e.g. LTAPEDEV \\.\TAPE0 for Windows. For details, see IBM Informix Dynamic Server LTAPEDEV.
HCL (IBM) Domino Server
The Lotus Domino (previously Lotus Notes) corporate brand has been first acquired by IBM and then by HCL Technologies. The Lotus Domino product name is now HCL Domino server; however, SEP sesam documentation update (such as screenshots) to reflect these changes is being done in a phased manner, therefore HCL Domino may still appear under the name Lotus Notes, Lotus Domino or IBM Domino.
Failed update of SEP sesam Server with HCL Domino extension module
Problem
The SEP sesam update fails when attempting to update HCL Domino extension on a SEP sesam Server that also acts as the data-mover. An error may occur during the update process, related to the export of the $PATH variable from the sm.ini file. During the update, SEP sesam exports the variables from the sm.ini without replacing the real values of the PATH variable.
⇒ Solution
Comment out the PATH variable in the sm.ini file. Open the sm.ini file, typically located in the SESAM_BIN
directory, locate the line with the PATH variable and comment out this line by adding a hash symbol (#) at the beginning of the line.
After commenting out the PATH variable, attempt to update SEP sesam again. Once the update is successful, you can revert the changes in the sm.ini file if necessary.
HCL Domino Server backup reports recovery failure
Problem
- Backup reports "Recovery may fail" during backup of the SEP sesam extension for HCL Domino Server. During the backup, the following warning appears for several or all Notes databases:
sbc-2076: Warning: Item [D:\notus03data\mailboxes\mail1\cruoff.nsf] is not logged.
Recovering may fail.
Possible causes
- There may be two reasons for this:
- Transactional logging was not turned on or is not in Archived mode.
- Transactional logging is running in Archived mode but is explicitly turned off for this database in the options of the database.
⇒ Solution
- This message is a warning of the Notes Backup API and is only forwarded by SEP sesam. Turn on Transactional logging in general or just for this database.
Other than that, the message can be ignored. The warning is given during the backup for safety reasons because only the admin can determine if administrative intervention is necessary.
Insufficient temporary space
Problem
- Temporary space may not suffice because every database must first be copied to the client.
⇒ Solution
- Change the path gv_rw_tmp in <SESAM_VAR>/ini/sm.ini to a directory with sufficient space.
Transactional logging not active
Problem
- The INCR backups fail.
⇒ Solution
- Activate Transactional logging in Archived mode for Notes.
Transaction logs already in progress
Problem
- This message appears in the SEP sesam:
DB Module: [Archiving of transaction logs already in progress.]
Possible causes
- The HCL Domino API sends this message to show that the transactional logs are already in the state Backup. The state cannot be set twice. If a backup could not be completed successfully, this state mat appear.
⇒ Solution
- To finish the backup status, all logasio processes should be removed from the Notes system:
UNIX: killall -9 logasio WINDOWS: sm_kill logasio
- Alternatively, you can reboot the Notes server.
Backup reports incorrect transactional logging
Problem
- After the restore of Transactional logging, Notes reports incorrect Transactional logging.
⇒ Solution
- Delete the nlogctrl.* files in the log directory then restart the Notes server.
Server fails to restart after a Notes server crash
Problem
- After a crash of the HCL Domino server, the server cannot be started without rebooting.
⇒ Solution
- If a server crash occurs during a log backup ,logasio processes may remain active. These processes must be stopped. The server can be restarted after running the following command:
UNIX: killall -9 logasio WINDOWS: sm_kill logasio
Backup with missing options
Problem
- The restore of Notes on a Linux client ends with:
"RESTORE STATUS: Restore failed. 2007-05-02 08:52:32: sbc-1146: Error: DB Module: [Notes API NotesInitExtended() failure"
Possible causes
- The restore options are not identical to the backup options.
⇒ Solution
- Set the restore options according to the task. Additionally, the options can be set in the restore assistant in step Options under Advanced restore options. For example:
-v 3 -a USER=nadmin -a NOTESINI=/srv/notedata/notes.ini
Abnormal termination of the sbc.exe process
Problem
- HCL Domino Console repeats: "C:\..\SEPSesam\bin\sesam\sbc.exe Process has terminated abnormally".
Possible causes
- If there is a backup failure that causes the SEP sesam backup client to terminate abnormally, the failure will be noticed by the HCL Domino server API and repeat this error message every minute in its logging.
⇒ Solution
- Stop the Domino server forcibly by running nsd -kill.
Restoring Notes database files to different location doesn't work
Problem
- When doing a restore of database files as a Notes restore to a different file location under Notes_Data, all databases are skipped.
Possible causes
- Notes denies an access to an existing database with the same replica-id, e.g.:
Item [\JOBSCHED.NJF] not included. Skipped...
⇒ Solution
- In the SEP sesam restore wizard, select a file location outside of Notes_Data and store the database file there. This makes it possible to copy the file to the Notes data directory structure.
VMware vStorage API
Backup
Virtual machines residing on NFS storage are unresponsive during a backup
Problem
- When using NFSv3 to mount NFS data stores and performing VMware backup in HotAdd transport mode, the VM that is backed up is not reachable for approx. 30 seconds when the HotAdd data mover detaches the VMDK.
Cause
- This issue occurs when the target VM and the data mover are running on two different hosts (ESXi), and the NFSv3 protocol is used to mount NFS data stores.
⇒ Solution
- To avoid this problem, the VMware data mover and VM have to run on the same ESXi.
- Use the NFSv4 protocol to mount NFS data stores.
VMware vSphere backup on Linux fails due to VDDK error
Problem A VMware vSphere backup (on the SEP sesam Server or RDS) on Linux fails with the following error:
Load VDDK library failed: Cannot load: libvixDiskLib.so
Cause The VMware Virtual Disk Development Kit (VDDK) is not installed.
⇒ Solution Install the VMware Virtual Disk Development Kit (VDDK).
VMware backup or browsing of VMware vCenter fails with an error due to a Java VM security restriction
Problem
- VMware backup or browsing of VMware vCenter might fail with the following error:
Error: VM Exception: [VI SDK invoke exception:javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: Certificates does not conform to algorithm constraints].
Cause
- Due to advanced security settings, the Java virtual machine does not allow connection to VMware vCenter.
⇒ Solution
- To avoid the Java virtual machine security restriction, proceed as follows:
- On your SEP sesam Server, locate the following file in the Java installation directory:
- Change the line
jdk.certpath.disabledAlgorithms=MD2, RSA > 1024
to
jdk.certpath.disabledAlgorithms=
- Once done, restart the SEP sesam service.
JDK_HOME/jre/lib/security/java.security
Segmentation fault during VMware backup on Linux-based data movers
Problem
- VM backup on Linux-based data mover completed successfully, however a segfault event is recorded.
Cause
- This is the VMware library issue caused by the missing /sys/class/scsi_disk directory on the Linux system.
⇒ Solution
- Before executing the VMware backups on Linux-based data movers, use the modprobe command
modprobe sd_mod
to load the sd_mod kernel module and make the /sys/class/scsi_disk directory available.
VMware slow backup performance via NBD. OR: Backups are failing in SAN transport mode with VMs residing on VMFS 6.0
Problem
- Slow backup performance via NBD. Another problem is that backups might fail in SAN transport mode.
Cause
- vSphere 6.5 and 6.7 may have slow backup performance via NBD with VDDK 6.0.x due to the VMware policy concerning backward and forward compatibility for VDDK to support N-2 and N+1 releases.
- With vSphere 6.5 neither of VDDK 6.0.x versions provides SAN transport support when using datastores formatted with new VMFS 6.0 filesystem. Consequently, the backup will fail.
⇒ Solution
- On Windows, VDDK is installed automatically during the SEP sesam installation. However, if you use a new vSphere version you should check the VDDK Compatibility Matrix to find its corresponding VDDK version and install it manually, if required.
- On Linux, you have to install the required VDDK version manually. Note that every new release of vSphere has a corresponding VDDK version; typically, the version number of VDDK matches the version number of vSphere.
For details on the required version, see VDDK Compatibility Matrix.
VMware backup fails with error due to unrecognized extended LUN connected to a Linux data mover
Problem
- VMware backup fails with error, such as:
Error while reading data. Error: VixDiskLib: VixDiskLib_Read: Read 2048 sectors at 0 failed. Error 16000 (One of the parameters supplied is invalid) (DiskLib error 4096062: One of the parameters supplied is invalid) at 5024.
Cause
- On the Linux data mover in VMware environments, when you extend LUN while online (without restarting your Linux system), the extended LUN will not be instantly visible from the Linux operating system. The extended disk is automatically adjusted on Windows, while on Linux you need to rescan SCSI bus manually. Consequently, the backup will fail.
⇒ Solution
- To rescan SCSI bus on Linux without restarting it, use the following command when adding a new disc (X is the number of SCSI host to scan):
echo “- – -” > /sys/class/scsi_host/hostX/scan
- Use the following command when expanding an existing disc:
echo 1 > /sys/class/scsi_device/device/rescan
Single file restore
Mounting VMware saveset on Linux fails with error
Problem
- Mounting VMware saveset (on the SEP sesam Server or RDS) on Linux fails with an error, for example:
Client mount tools not installed
Cause
- The guestmount tool is not installed on the Linux host.
⇒ Solution
- To be able to access and mount the file system of an image on Linux, you have to use use the guestfs-tools. Install the guestfs-tools package as described in Installing guestfs-tools on Linux.
Single file restore is not working with VMware 6.0
Problem
- When restoring single files on VMware 6.0, the restore fails with: Cannot open Thin/TBZ disks in a multiwriter mode.
This log is also shown in the vCenter events.
Cause
- This error appears if the SCSI bus sharing type on the proxy machine is set to Physical.
⇒ Solution
- Shut down the proxy machine and change the type of SCSI bus sharing to None, so the virtual disks cannot be shared by other virtual machines.
When restoring a single file, SEP sesam cannot access storage on a VM that is configured with the SCSI LSI Logic SAS adapter
Problem
- In SEP sesam version ≥ 4.4.3.48 in combination with a Linux proxy VM, when trying to select the restore source in the restore wizard (step 4: Select Files), no items are displayed in the browser.
Cause
- This issue seems to be related to the SCSI controller of the proxy VM. When trying to restore a single file from a virtual machine, SEP sesam cannot access storage on a virtual machine that is configured with the SCSI LSI Logic SAS adapter.
⇒ Solution
- Open the properties of the proxy VM in your vCenter and change the controller on the VM from the LSI Logic SAS controller (on some VMs, this is selected by default) to an LSI Logic Parallel controller.
For more information, see VMware Docs article Change the SCSI Controller Type in the vSphere Client.
- Restart the restore wizard. If there are still no items displayed in the restore browser, reboot the proxy VM and then click the Update button.
After updating an existing SEP sesam structure with a new VDDK version, it is no longer possible to mount a VMDK
Problem
- After VDDK ≥ 6.5.x has been manually installed on Windows, it is no longer possible to mount a VMDK saveset on this host.
Cause
- If another VDDK version, for example VDDK ≥ 6.5.x, is manually installed on Windows, the host requires a reboot after the update if you want to mount a VMDK on this host.
⇒ Solution
- Reboot the host after the VDDK update, then run the following script from the command line:
vstor2install.bat in directory C:\Program Files\VMware\VMware Virtual Disk Development Kit\bin
Citrix XenServer
Error when using quiesced snapshots
Problem
- Using VM.snapshot_with_quiesce method fails in Citrix Hypervisor ≥ 8.1 or in the version ≥ 9.0.x.x drivers.
Cause
- VSS and quiesced snapshots of Windows VMs capability have been removed in Citrix Hypervisor ≥ 8.1 and in the version ≥ 9.0.x.x drivers. They are only supported in Citrix Hypervisor ≤ 8.0.
⇒ Solution
- If you want to continue using the quiesced snapshot feature with Windows VMs hosted on Citrix Hypervisor 8.0 and earlier, do not update to the 9.0.x.x drivers.
CBT backup fails because Hypervisor TLS certificate contains wrong IP address
Problem
- CBT backup fails with the NBD client connection error:
Failed to get NBD client for device '******: NBD client connection error: hostname '****' doesn't match u'******
Cause
- The TLS certificate of a Xen Server contains the wrong IP address. Therefore no NBD session can be established.
⇒ Solution
- Check all Citrix Hypervisors in the Pool whether the IP address in the certificate matches.
- On the Citrix Hypervisor where it does not match, execute the following commands:
- Create a backup of the existing certificate by using
- Create a new certificate
- Restart XenAPI service
- Restart NBD Service
mv /etc/xensource/xapi-ssl.pem /etc/xensource/xapi-ssl.pem.$(date -u +%Y%m%d)
/opt/xensource/libexec/generate_ssl_cert /etc/xensource/xapi-ssl.pem <ip-address or FQDN>
systemctl restart xapi
systemctl restart xapi-nbd
Message "Upload to URL ... failed" appears during restore, "insufficient space" error occurs during backup
Problems
- Restore fails with an error:
"Upload to URL ... failed."
SR_BACKEND_FAILURE_44: : There is insufficient space to create snapshot on Storage Repository.
Possible cause
- There is not enough space available on the default storage.
- If the "insufficient space" error occurs during the backup, it might be related to the snapshot process. XenServer was unable to create a snapshot for the VM because there was not enough space in your Storage Repository.
⇒ Solution
- Use XenCenter to select another default storage that has enough free disk capacity to store the virtual disk(s).
Other issues
- Some errors are not reported in detail by XEN API. More information can be found in XenServer in /var/log directory.
- In some pool configurations all pool members must have their management interface listed in DNS.
- If the settings below don't agree on asynchronous routing and connection drops and/or 3 second or 80 second delays, opening sockets may occour. This is very similar to having 2 network cards configured with ipv4 to the same subnet and connected to the same switch.
- If you use Cisco switches and you have configured (Citrix unsupported) lacp bonding 802.3ad, it is very important that both sides agree on the balancing type. The Linux default is layer2 and the Cisco default is layer3+4.
- Check the bonding mode on the host with cat /proc/net/bond/bond* for bonding mode:
IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0) or Transmit Hash Policy: layer3+4 (1).
- Users have reported that this command and a reboot solves the problem, but this cannot be formally recommended due to its untested and unsupported status:
xe pif-param-set uuid=${UUID_OF_BOND_PIF} other-config:bond-xmit_hash_policy=1
- In the common situation where 802.1ad with xmit_hash_policy=1 and STP are in use, multiple settings need to be adjusted on the Cisco side according to the article Considerations for Connecting XenServer to the Switch Ports.
- Most other network related performance and reliability issues can be resolved with Considerations for Connecting XenServer to the Switch Ports.
- In some situations when using thin storage repositories, an sr rescan is necessary to re-detect free space before a backup or restore.
SAP HANA
After initial configuration the first test backup out of the HANA Studio shows errors
Problem
- Once you configured SAP HANA to work with SEP sesam and start the first test backup, it finishes with errors.
Cause
- There might be an issue with case sensitive file paths, written in the
/var/opt/sesam/var/ini/backint_saphana.utl
files.
⇒ Solution
- Open the
/var/opt/sesam/var/ini/backint_saphana.utl
files and make sure that the directory names are listed in the appropriate case (lower case or upper case respectively).
SAP HANA backup fails as sbc_hana.sh tries to back up SAP HANA DB on a secondary system
Problem
- sbc_hana.sh does not recognize the secondary SAP HANA server as the secondary system and attempts to back up the SAP HANA database. Backup fails with error Connection refused.
Cause
- The secondary SAP HANA DB cannot be backed up as it permanently synchronizing data with the primary SAP HANA DB. Only the primary system is able to access the database when the database is running in SYNC mode. Therefore accessing the secondary system and running the backups on the replication target will fail.
⇒ Solution
- Manually switch to the primary host in the command event. For details, see Scheduling start of the SAP HANA script.
Low deduplication ratio for SAP HANA backups
Problem
The deduplication ratio for SAP HANA V2 backups is very low or 0%.
Cause
This issue occurs because SAP has enabled backup encryption as the default setting during new installations with HANA 2.0 SP7. When backup encryption is enabled, it significantly reduces the effectiveness of deduplication processes on the backup target.
⇒ Solution
To address this issue, you have two options:
- Disable backup encryption in SAP HANA Configuration
- In SAP HANA Studio, double-click an instance to open it and select the tab Configuration. Then in the global.ini locate the setting backup_encryption and set it to off.
- Note that SAP HANA backup encryption is a security feature. Disabling backup encryption will improve the deduplication ratio, but may reduce the security of your backups.
- Use a backup target without deduplication
- If you want to maintain encryption, you can switch to a backup target (SEP sesam datastore or other storage system) that doesn't perform deduplication.
For more information, see SAP HANA Configuration.
Useful SAP HANA commands
Note | |
The provided SAP HANA commands are intended to assist with troubleshooting low deduplication ratios. To ensure that the commands are appropriate for your specific environment, consult the official SAP HANA documentation before executing any commands. |
To check the encryption status of the SAP HANA databases, you can use the following command:
hdbsql -i <instance> -d SYSTEMDB -u SYSTEM -p <password> "SELECT DATABASE_NAME, SCOPE, IS_ENCRYPTION_ACTIVE, CONFIGURATION_CONTROL FROM SYS_DATABASES.M_ENCRYPTION_OVERVIEW"
In the SCOPE column, where the value is "BACKUP", if IS_ENCRYPTION_ACTIVE is "TRUE", the SAP HANA database will encrypt the data before creating the backup. The CONFIGURATION_CONTROL column shows whether the control over the encryption is managed by the SYSTEMDB or the local database itself.
To disable backup encryption, you can use the following commands:
- If CONFIGURATION_CONTROL is "SYSTEM DATABASE", connect to the SYSTEMDB and use:
hdbsql -i <instance> -d SYSTEMDB -u SYSTEM -p <password> "ALTER SYSTEM BACKUP ENCRYPTION OFF"
- If CONFIGURATION_CONTROL is "LOCAL DATABASE", connect to the specific tenant database and use:
hdbsql -i <instance> -d <database name> -u SYSTEM -p <password> "ALTER SYSTEM BACKUP ENCRYPTION OFF"
Not enough streams for SAP HANA external backups
Problem
- SAP HANA external backups can no longer be performed due to the following error:
2018-07-26 08:04:35 W007-SBC_COM[ 6664]: Timeout reached, drive group HANA has no free streams anymore
Cause
- There are not enough streams available to run SAP HANA external backups.
⇒ Solution
- In the data store properties, check the available streams per drive (by default, 5) and adjust them to 20.
SAP HANA backups are not started anymore
Problem
- After a while SAP HANA backups are not starting.
⇒ Solution
- Connect to the SAP HANA server and stop the client:
- /opt/sesam/bin/sesam/sm_main stop
- ps –ef | grep sm_sbc*
- Search for old sm_sbc* services and kill these services.
- Start the SEP sesam Client again with the command /opt/sesam/bin/sesam/sm_main start.
Backup finished with error [447]
Problem
- Backup failed with error.
YYYY-MM-DDTHH:MM:SS+00:00 P11252 1480e025c70 ERROR BACKUP SAVE DATA finished with error: [447] backup could not be completed, [110507] Backint exited with exit code 1, Traceback (most recent call last): File "/usr/local/sesam/lib/python2.7/site-packages/cx_Freeze/initscripts/Console.py", line 27, in <module> File "backint_saphana.py", line 124, in <module> File "sm_common.py", line 55, in __init__ IOError: [Errno 13] Permission denied: '/var/opt/sesam/var/log/lgc/sbc_backint_NOT DEFINED.log'
Cause
- The hdbbackint does not have the permissions to write to the logging directory.
⇒ Solution
- Execute
chmod 755 (directory)
to fix the issue.
SAP HANA backup failure – analyze log files for cause
Problem
- Backup was unsuccessful and is shown with a red dot in either the SAP HANA Studio or the SEP sesam GUI.
⇒ Solution
- Go to /var/opt/sesam/var/log/lgc and search for the hdbbackint_SID.log or hdbbackint_SID_log.log. These logs contain the logging information for the data connections to the SEP sesam Server and Remote Device Server (if used).
On the SEP sesam Server/RDS, <sesam_install>/var/log/sms will contain stpd_pid.log and stpd_pid.com.log files with the server-side logging information. Additional log information on the SEP sesam Server can also be found in <sesam_install>/var/log/lgc in sm_sbc_com*.log files.
Job status displays multiple running jobs
Problem
- Even though only a single backup or restore is running, multiple jobs are displayed in the Job Status view of the SEP sesam GUI.
⇒ Solution
- This is the usual behaviour. When performing a backup, HANA bundles the data to be saved into a few separate save sets and sends them to the SEP sesam device server. Therefore, multiple jobs will be displayed in the job status. The same in reverse applies to restores.
Log backups running too frequent
Problem
- Log backups are processed every 15 minutes. The time among log backups should be extended.
⇒ Solution
- Open the backup configuration in the SAP HANA Studio. On the right, there is a box where the interval can be adjusted. But be careful: The log area will grow bigger. If the log area is full, the database will stop performing transactions!
After recovery the hardware key changed
Problem
- After a bare metal recovery or full recovery, the hardware key is changed and the license is dropped.
⇒ Solution
- This is a known issue which can occur even if there is no change in the hardware and hostname with the same configuration: re-installation, restore, recovery, or rename will change the hardware key. Please contact SAP support. For details, see Hardware Key for the HANA Database.
SAP ASE
SAP ASE backup fails with Requested server name not found
Problem
- SAP ASE backup fails with CT-LIBRARY error: ... Requested server name not found.
Cause
- The server name hasn't been added to the interface file or the given server name does not exist.
Note
- If source is set with {server_name} then you probably have mistyped the server name or server's port (default 5000).
- If source is set without {server_name} then isql looks for the server specified by your DSQUERY environment variable.
⇒ Solution
- You have to extend the source with the correct server name and if necessary also the port.
Source format [/{server_name}[:{port}]]/{database}
For example, if the source is master, you have to add the server name as the first part, e.g. /SOL/master or /SYBASE16:5000/master.
- Test the connection to SAP ASE server by running the isql utility. For details, see SAP ASE Utility Guide: isql.
isql -U<username> -P<password> -S<server_name>
where <username> specifies a login name of a user with access to database (login names are case-sensitive!), <password> specifies your SAP ASE password, and <server_name> specifies the name of the SAP ASE server to which to connect. isql looks this name up in the interfaces file.
If this test fails, use the dsedit utility that allows you to view and edit server entries in the interfaces file using a GUI in Adaptive Server Enterprise. For details, see SAP ASE Utility Guide: dsedit.
SAP ASE backup fails with Failed to load library 'libsepsybase.so' error
Problem
- A backup fails with Failed to load library 'libsepsybase.so' or Failed to load library '%1' error.
Possible causes
- The library libsepsybase.so is not copied to the SAP ASE lib directory.
- A wrong binary library file is used.
- The LD_LIBRARY_PATH is not set correctly.
- The SAP ASE backup server is not running on the specified host.
⇒ Solution
- Copy the library libsepsybase.so to the SAP ASE lib directory.
- Check whether a necessary shared library is installed: file libsepsybase.so and how it is resolved: ldd libsepsybase.so.
sybase16:/opt/sap/ASE-16_0/lib # file libsepsybase.so libsepsybase.so: symbolic link to `/opt/sesam/bin/sesam/libsepsybase.so' sybase16:/opt/sap/ASE-16_0/lib # file /opt/sesam/bin/sesam/libsepsybase.so /opt/sesam/bin/sesam/libsepsybase.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped sybase16:/opt/sap/ASE-16_0/lib # ldd /opt/sesam/bin/sesam/libsepsybase.so linux-vdso.so.1 => (0x00007fffcd75c000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f65d0fa6000) librt.so.1 => /lib64/librt.so.1 (0x00007f65d0d9c000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f65d0b7f000) libc.so.6 => /lib64/libc.so.6 (0x00007f65d0803000) /lib64/ld-linux-x86-64.so.2 (0x00007f65d1cf3000)
- Check if the LD_LIBRARY_PATH variable is set correctly in the file SYBASE.env (The SAP shared libraries are located in the installed component’s lib directory).
sybase16:/opt/sap # grep LD_LIBRARY_PATH SYBASE.env [...] LD_LIBRARY_PATH=/opt/sap/ASE-16_0/lib:$LD_LIBRARY_PATH
- Check if the SAP ASE backup server is running on the specified host -> check for the process named backupserver.
sybase16:/opt/sap # ps -ef | grep backupserver root 15534 15533 0 Mar23 ? 00:00:00 /opt/sap/ASE-16_0/bin/backupserver -e/opt/sap/ASE-16_0/install/qsstor.log -N25 -C20 -I/opt/sap/interfaces -M/opt/sap/ASE-16_0/bin/sybmultbuf -Sqsstor
SAP ASE backup fails with ERROR: No Total: ... line found in LIS file
Problem
- SAP ASE backup fails with the following error: No Total: ... line found in LIS file.
Cause
- sm_backup tried to fetch the LIS file before the backup server has finished.
⇒ Solution
- Use the SEP sesam Client ≥ Beefalo V2 SP2 and set the following backup option in the SAP ASE backup task properties -> Options tab -> Backup options field:
-a action=sleep:20
Note | |
If the backup option -a action=sleep:xx does not work, use the FTP interface instead of HTTP. |
SAP ERP
Using XUSER MaxDB tool for authentication during backup fails with the error XUser not found!
Problem
- The SAP MaxDB XUSER command-line tool can be used to store user log-on data thus simplifying logon to databases. When using XUSER and the script runs as command event, the backup script reports the following error: XUser not found!
Cause
- The SAP MaxDB XUSER tool creates a file .XUSER.62 in the home drive of the corresponding user. If the script runs as command event, no environment is set and thus also not the HOME variable. If the HOME variable is not set, the DB cannot find the .XUSER.62 file belonging to the user and cannot map and verify the key to the user, therefore the command event fails.
⇒ Solution
- Export the HOME variable for the root user in the backup script (command event script) as follows:
#!/bin/bash HOME=/root export HOME /.../dbmcli .... -U <key> .... .....
SAP Oracle
SAP Oracle: "item from input file does not exist"
Problem
- brbackup backint log on Windows reports "item from input file does not exist". The following error messages appear:
In backint_<SID>.log
SSB:(3620): 121043: backint_back.c:( 331):: WARNING: item from input file does not exist: F:\ORACLE\T11\SAPDATA1\ERPUSR_1\ERPUSR.DATA1.
In sbc_<SID>.log
SSB:(3620): 121242: backint_func.c:(1928):: 2013-06-18 12:12:42: sbc-3008: Info: Processing item: [F:\ORACLE\T11\SAPDATA1\ERPUSR_1\ERPUSR.DATA1]... SSB:(3620): 121242: backint_func.c:(1928):: 2013-06-18 12:12:42: sbc-4000: Trace: GetFileSecurityInfo: Opening file [\\?\F:\ORACLE\T11\SAPDATA1\ERPUSR_1\ERPUSR.DATA1] with CreateFileW() function failed. Error code: 1314 SSB:(3620): 121242: backint_func.c:(1928):: 2013-06-18 12:12:42: sbc-2046: Warning: Cannot get item security data for [F:\ORACLE\T11\SAPDATA1\ERPUSR_1\ERPUSR.DATA1].
Possible causes
- The user starting the brbackup process does not have permission to access the security data (ACL) of the file F:\ORACLE\T11\SAPDATA1\ERPUSR_1\ERPUSR.DATA1 in Windows. Typically, only local group administrators have permission to read or write an object's SACL, which is controlled by the privilege or user right (SeSecurityPrivilege).
⇒ Solution
- Assign the user to the local administrator group of the SAP server.
- Ensure that the user right (SeSecurityPrivilege) is implicitly or explicitly given to the user.
- It may also be necessary to set the User Account Control Settings to Never notify through the Control Panel.
NetApp Volume Backup
FilerView http might be disabled
Problem
- FilerView might be disabled.
⇒ Solution
- Check if FilerView is enabled. Logon to your NetApp system via ssh and use the following command to check whether your FilerView http is enabled:
options httpd.admin.enable
In clustered environments or Ontap 8 Versions using Vserver:
vserver services web modify -vserver <NODE_NAME> -name ontapi -enabled true
Restore shows permission problems
Problem
- A restore fails with the following error message in the protocol:
2012-08-20 12:32:27: sbc-2044: Warning: Cannot create item [/var/opt/sesam/var/work/mnt/netapp/VOLUME_NAME/restore/]: Permission denied
⇒ Solution
- Check if the system acting as a data mover has a write access to the NetApp volume the data should be restored to.
Incorrect volume path specification
Problem
- Usually NFS shares on NetApp systems are exported beneath the
/vol/
directory. For example the nfs share test is mountable via:netapp:/vol/test/
This is not the case for all NetApp systems. Some systems export the volume directly via: netapp:/test/
⇒ Solution
- By default, SEP sesam mounts the volume via /vol/ specification. You can adjust the volume_path parameter in the advanced backup options by specifying the following:
-a volume_path=/
Backup fails with Permission denied - user mapping for NTFS style volumes is missing
Problem
- By default, SEP sesam is accessing the volumes for backup via NFS. If a desired volume for backup has security style set to NTFS, an appropriate user mapping has to be configured on the NetApp system. If not, backup may fail with Permission denied error message.
⇒ Solution
- Map the permission for the administrator user to root as shown in the screenshot below.
MySQL
MySQL backup fails with mysqldump: Error 2013: Lost connection to MySQL server during query
Problem
- MySQL backup fails with
mysqldump: Error 2013: Lost connection to MySQL server during query
Possible cause
- This error typically occurs when backing up MySQL databases to tape devices. The reason for this is the MySQL setting net_write_timeout, which defaults to a value of 60 seconds. It can take longer than 60 seconds to open the tape, but after 60 seconds MySQL database resets the connection while the backup is still in progress.
⇒ Solution
- Open the option file
/etc/my.cnf
and set a higher value for the variable net_write_timeout in section [mysqld] as follows:
[mysqld] ... other options .. net_write_timeout = 180
- Close the file and restart your mysql daemon for changes to take effect. If a restart is not an option, the setting can also be applied online, via the mysql command:
# mysql -u root -p -e "set global net_write_timeout=180;"
Keep in mind that this setting will reset to its original value during restart if it is not saved in the configuration file.
- If you have raised the value to 180 seconds, but backups are still failing with the same error message, try to use a higher timeout value. The timeout can get exceeded, for example, if the tape gets swapped during the MySQL backup. Note that some tape libraries require more than 180 seconds to handle the tapes.
The current active value can be determined as follows:
# mysql -u root -p -e "SHOW GLOBAL VARIABLES LIKE 'net_write_timeout'" +-------------------+-------+ | Variable_name | Value | +-------------------+-------+ | net_write_timeout | 60 | +-------------------+-------+
Problems during MySQL backup if user's password contains special characters – use configuration file to store the password
Problem
- There may be problems during backup if the password of the user, assigned for executing the backups, contains special characters. In general, the use of apostrophes as well as mutated vowels (umlauts) is suppressed in the GUI.
⇒ Solution
- Store the password (recommended for all MySQL backups) in a special configuration file rather than a task. If a password is stored separately in the configuration file, it will not appear in SEP sesam logging. See Using Option Files for a complete overview about MySQL configuration files on any supported platform. The following example is suitable for Unix and Linux:
- Create a configuration file named my.cnf. In case SEP sesam runs as user root and to ensure that the SEP sesam Client can read it correctly, store this file in
/root/.my.cnf
- Enter the following parameters into the file (the entry user is optional):
[client] user=root password=mysqlpw
- The backup tasks options must include the user name if not set:
-a user=username
The password will now be read from the configuration file during the backup.
Shell login of user root is permitted (i.e., on Ubuntu systems), but the variable $HOME does not exist
Problem
- If the variable $HOME does not exist, then the file my.cnf cannot be found.
⇒ Solution
- Edit the file <sesam-root>/var/ini/sm.ini and add the following lines at the bottom of the file:
[ENVIRONMENT] HOME=/root
Once you added the required variable, restart SEP sesam service for changes to take effect.
BSR Pro for Windows
BSR Pro error codes
When performing a BSR Pro backup or restore, the procedure might fail due to different reasons. The following list may be of help to understand the BSR Pro error codes.
- e0040002: There is a BSR Pro backup or restore currently running; simultaneous operation of more than one BSR Pro backup or restore process is not supported.
BSR Pro installation stops with a message: A new version was found. An update is not necessary.
Problem
- BSR Pro installation fails with a message, such as: A new version was found. An update is not necessary. This happens when the installation procedure has found another installed version of the BSR Pro (or the original program DiskImage from O&O).
Possible cause
- In the past, the BSR Pro or O&O DiskImage was already installed manually.
⇒ Solution
- If you want to keep the currently installed version, you have to perform the SEP sesam installation without the BSR Pro. Otherwise the currently installed BSR Pro version has to be uninstalled and then the SEP sesam installation together with BSR Pro has to be performed again.
BSR Pro fails to update during silent SEP sesam update
Problem
- BSR Pro is not updated when you update a SEP sesam Server, RDS or Client component including BSR in silent mode, e.g. via the powershell command line:
sesam-cli-5.1.0.3-windows.x64.exe -Wait -Verb runAs -ArgumentList '/s',/v"/quiet /lvoicewarmup \"C:\tmp\sm_msi_update.lgc\""
Cause
- BSR Pro is installed with own MSI installer and cannot be included automatically in a chained installation. During a manual update using the installer EXE, or during the remote update installation via the SEP sesam GUI (sm_update_client) BSR Pro will be updated normally. During the silent installation the update of BSR Pro is not performed.
⇒ Solution
To update the BSR Pro you can perform one of the following:
- Use the regular GUI-based update which also includes the updated version of BSR Pro.
- Use the SEP sesam GUI (sm_update_client) update which also includes BSR update.
BSR Pro backup fails due to snapshot backup issue
Problem
- BSR Pro backup fails with BSR Pro unknown return code (0X1).
Cause 1
- On some Windows OS (e.g., Windows Server 2012) you may receive a "volsnap" error in the Windows Event log: Event ID 25: The shadow copies of volume C: were deleted because the shadow copy storage could not grow in time. Consider reducing the IO load on the system or choose a shadow copy storage volume that is not being shadow copied.
This error indicates that the source drive is experiencing a high IO load, which caused VSS to fail and deleted the shadow copy snapshot, as a result causing a BSR Pro backup failure.
⇒ Solution 1
- You have to reduce the IO load on the drive being backed up or configure VSS to use a different drive with more available space (and less load) and ensure that there is sufficient storage space to create and maintain shadow copies. You may want to allocate a separate volume on a separate disk for storing shadow copies, as recommended by Microsoft, to improve performance and eliminate the possibility to delete the shadow copies due to a high I/O load.
For details, see Volume Shadow Copy Volsnap Event 25. See also the following Microsoft article: Event ID 25 — Diff Area Integrity
Cause 2
- If the BSR backup fails with error message 0X1 without a clear cause, it is often due to an inconsistency in the client's operating system.
⇒ Solution 2
- To verify this, check the Event Viewer on the client being backed up. The time of the error usually aligns with the errors logged in the BSRPro.log file. This log is located on the client in <SEPsesam>\ProgramData\SEPsesam\SEP sesam BSR Pro\.
- The easiest way to resolve the inconsistencies is to restart the client.
No savesets nor clients exist in SEP sesam after clicking Execute BSR Pro Quick-Start option
Problem
- The list of available save sets at the SEP sesam Server remains empty.
Possible cause
This error typically occurs when the connection to the SEP sesam Server cannot be established due to any of the following:
- Problem with name resolution: SEP sesam Server hostname is not known to client, name to IP or IP to name (reverse) resolution is not correct.
- The client's name (typically a new host is set up in case of disaster recovery with new IP) cannot be resolved at the SEP sesam Server.
- Wrong server name (a fully qualified domain name (FQDN) may be required).
- When booting from PE, it may take a little longer until the savesets appear (30-45 seconds).
⇒ Solution
- Check your name resolution on client and SEP sesam Server. Both names must be resolved, the client IP must be resolved to the client name, and SEP sesam Server must be reachable (check the route to it – FQDN may be required to reach it). For reference, see How to check DNS configuration. If the resolution in your network does not work as expected, there might be issues with IPv6 and IPv4.
- Add the client's name to the name resolver or add it to the /etc/hosts on SEP sesam Server, or use the GLBV gv_stpd_auth NONE to disable the IP name to IP address check on SEP sesam Server; in the latter case, open a shell on the SEP sesam Server and execute the command below. This command enables you to contact the SEP sesam Server even if the reverse name resolution does not work on the new client.
- Specify the SEP sesam Server with the FQDN. Open the BSR Pro Settings -> tab Security and in the Authentication table check the settings. Under the Network path/server, a valid sesam:{server} name or interface must be specified. Examples:
sm_glbv w gv_stpd_auth NONE
''Network path/server'' ''Port'' sesam:http://sep_server.mydomain.com 11000 sesam:sep_server2 11001
Note | |
Only the first entry is used to fill the list of available savesets! |
OODIAG crashes during the creation of the BootISO
Problem
- If you try to create a boot ISO with the BSR Pro on a system where the volumes are dynamic volumes, then the OODIAG service cored under certain circumstances.
Possible cause
- Problem is apparently the system volume. Possibly it is related to the dynamic volume.
⇒ Solution
- Install a newer SEP sesam version with BSR Pro.
- If possible, do not use dynamic volumes.
VSS snapshot for the system volume does not work
Problem
- Typically, VSS is used by default to log changes in data so that unchanged data is included in the backup. But on some systems the VSS snapshot for the system volume does not work.
⇒ Solution
- Start SEP sesam BSR Pro with the option -a snap=snapshot. This option creates a special snapshot that does not involve the Microsoft VSS Writer: data changes are logged using the installed filter driver to include the unchanged data in the backup.
Usage: -a snap=vss|snapshot where <vss> is default
It is not possible to get the error-log for a failed restore in a PE
Problem
- Disk full while accessing X: because dump is written to it.
Possible cause
- The LOG is getting too big because it is constantly being written on.
⇒ Solution
- Open the registry in the command shell by using regedit.
- Add the following reg-key:
[HKEY_LOCAL_MACHINE\SOFTWARE\O&O\O&O DiskImage\<version>] "DumpFilePath"="E:\\DUMPS\\"
- After the change the LOG is written to the path
E:\DUMPS
.
SEP sesam uninstallation of the BSR Pro component fails
Problem
- Uninstallation of the BSR Pro component fails and leaves invalid registry entries behind.
⇒ Solution
- After a failed uninstallation, the uninstaller information is probably corrupted and you have to get rid of any potentially harmful registry leftovers to ensure normal operation of SEP sesam. There are two possible ways to deal with failed uninstallation leftovers.
- The best way to uninstall these components is to use the installer; you should first reinstall BSR Pro in order to repair it, and only then uninstall it again.
- If the procedure above cannot be successfully applied, you have to manually remove the BSR Pro installation.
- In the Start menu/Search box, type regedit and click Enter. The Windows Registry Editor window opens.
- Delete the registry entries:
Tip | |
After a successful and complete uninstall of the BSR Pro component, the keys listed below (see registry entries) should no longer exist. If any of the listed keys is still present, it can be deleted manually as described in the next procedure. |
Note | |
The following steps describe how to modify the registry. If you modify the registry incorrectly, serious problems might occur. If you are not sure about what you are doing, we recommend that you contact SEP support at support@sep.de for assistance. |
HKEY_LOCAL_MACHINE\SOFTWARE\O&O\O&O DiskImage HKEY_LOCAL_MACHINE\SOFTWARE\O&O\O&O LiveUpdate\SEP sesam BSR Pro
BSR Pro backup fails with a connection error
Problem
- BSR Pro backup may fail with one of the following errors due to blocked network connection to SEP sesam Server:
script processing failed:
or
(Status: ERROR 425 Can't open data connection. WINSOCK: Connection timed out. (0x274c,10060)
or
Port negotiation failed. (10038)
Possible cause
- Windows Firewall is preventing or blocking your connections.
⇒ Solution
- Check the Windows Firewall rules on the SEP sesam Client with BSR Pro. If the firewall rule applied during SEP sesam installation is no longer valid, create an exception for oodiag.exe to allow connection through the firewall. For details on how to configure this firewall rule, check the Prerequisites in the SEP sesam BSR Pro – Backup Configuration.
NDMP
NDMP connection error during backup
Problem
- The NDMP server fails to connect to SEP sesam (sbc_ndmp) during an NDMP backup. Errors like the following are displayed and the backup fails.
NDMP error: ERR NDMP4_DATA_CONNECT NDMP4_CONNECT_ERR NDMP error: ERR NDMP4_DATA_ABORT NDMP4_ILLEGAL_STATE_ERR
or
NDMP error: NDMP_DATA_CONNECT
Possible causes
- In most cases, this is a firewall problem.
- Other incorrect network settings prevent the connection to sbc_ndmp.
- In the case of NetApp, multiple interfaces may prevent the connection to sbc_ndmp.
⇒ Solution
- Exclude sbc_ndmp.exe from the firewall scanner and run the backup again.
- Check the configuration of your network. Note that your network must be configured correctly to allow the connection to sbc_ndmp.
- In the case of NetApp, if multiple interfaces are suspected, set the preferred interface option on your NAS device to specify the interface which should be used for NDMP:
options ndmpd options ndmpd.preferred_interface <interface_name>
For more details, see NetApp-specific NDMP configuration.
NDMP restore to new restore target with non-existent subdirectory fails
Problem
- An NDMP restore to a new restore target where the intended restore target is a different, non-existent subdirectory fails. (In previous SEP sesam versions, the NDMP alternate restore to a non-existent subdirectory fails, but it is shown as successful. Although this is now fixed, you may want to verify that the previously restored NDMP data exists.)
Note that this is only the case if the new restore target is a non-existent subdirectory. For example, if you want to restore to a /<volume_name>/<sub_dir1>/<sub_dir2>
and <sub_dir1> does not exist, the restore will fail. You can still restore NDMP to the alternative target subdirectories if they already exist.
Cause
- If a subdirectory is specified as the new (non-existing) restore target, for example,
/<volume_name>/<sub_dir1>/<sub_dir2>
, the restore fails as non-existing subdirectories are not supported when restoring to another target. In this case, only a single level directory structure can be used as the restore target:/<volume_name>/<target_dir>
; so in the above example, the supported restore target path is/<volume_name>/<sub_dir1>
.
Workaround
- When performing an NDMP restore to a new restore target, make sure that you restore your data to the top-level directory (
/<volume_name>/<target_dir>
). Do not specify any non-existing subdirectories in the new target path, otherwise the restore will fail.
Selective NDMP restore does not restore empty directories and the restore fails
Problem
- In SEP sesam versions ≤ 4.4.3 Beefalo V2, when performing a selective NDMP restore, empty directories are not restored (created) on the target if:
- empty directories are present in the restore list along with normal directories and files (this is a known issue, but too minor to warrant a fix)
- at least one file is selected in another directory, for example, if the backup source is /vol/vol1 and the restore selection is /vol/vol1/dir_1/file_1 and /vol/vol1/empty_dir
Consequently, the restore fails.
Cause
- If only directories are selected for selective restore, the empty directories are added to the SEL file by sm_restore and the restore completes without error. However, if at least one file is selected together with an empty directory, the restore fails.
Workaround
- When performing a selective restore, try not to restore empty directories together with individual files, otherwise the restore will fail.
NDMP on NetApp
NDMP browsing of non-UTF-8 fails and restore to non-UTF-8 volume may not be possible
Problem
- When performing a restore, it is not possible to browse the contents of non-UTF-8 NetApp volumes. You may receive an error message similar to the following:
NDMP error: ERR NDMP4_SNAP_DIR_LIST ?0x20500106?
This can happen because:
- Non-UTF-8 volumes are not searchable if objects contain umlauts or special characters.
- Consequently, a restore to a non-UTF-8 volume is not possible if objects with special characters are included in the NDMP backup.
The following ONTAP CLI command displays the encoding language of all volumes:
volume show -vserver * -fields language
⇒ Solution
- NDMP backups of NetApp volumes without UTF-8 encoding language are basically possible, but the path in the backup task can only be specified manually.
- If a backup of a non-UTF-8 volume contains objects with umlauts or special characters, the restore can only be performed to a UTF-8 volume.
NDMP DAR restore of single files is too slow
Problem
- When using a Direct Access Recovery (DAR) which enables NDMP selective restore, restore is too slow, even though DAR should greatly reduce the time it takes to restore individual files.
Cause
- The general prediction is that DAR greatly increases restore speed, but this depends on the amount and size of data backed up. As stated by NetApp, the restore can take quite some time depending on the number of files in the directory, the position of the directory and file on the tape, and the number of tapes to be read. Refer to the NetApp documentation for details.
⇒ Solution
- Split the backup into smaller parts to reduce the amount of information that DAR has to manage during the restore.
Restore fails with: "Storing of nlist entries failed."
Problem
- Restore finishes with the error message: "Storing of nlist entries failed."
⇒ Workaround
- Instead of commonly used version 4 of the NDMP protocol, start the NDMP daemon in version 3 by entering the following commands:
ndmpd off ndmpd version 3 ndmpd on
- Then retry the NDMP restore.
Hyper-V
Hyper-V backup fails with error [ [NInternal::CVssAsyncDecorator::Wait] - VSS_S_ASYNC_PENDING]
Problem
- The Hyper-V backup fails with the following error:
Error: DB Module: [ [NInternal::CVssAsyncDecorator::Wait] - VSS_S_ASYNC_PENDING]
⇒ Solution
- You have to increase the VSS value in the configuration file <SESAM_VAR>/var/ini/sm.ini from 200
[SBC_OPTIONS]
VSS_WAIT_FOR_ASYNC_OP=200
to 600
[SBC_OPTIONS]
VSS_WAIT_FOR_ASYNC_OP=600
Hyper-V backup fails with time-out error
Problem
- The backup task fails with a time-out error in Windows:
Error: DB Module: [ [Failed at thaw] - VSS_E_WRITERERROR_TIMEOUT. [Microsoft Hyper-V VSS Writer] - VSS_E_WRITERERROR_TIMEOUT]
⇒ Solution
- Install the following Microsoft service pack on the Hyper-V Node and on each Microsoft VM running on that host.
Hyper-V backup fails with error VSS_E_SNAPSHOT_SET_IN_PROGRESS
Problem
- When backing up a Hyper-V VM in Windows Server 2012, the following error appears:
Error: VSS_E_SNAPSHOT_SET_IN_PROGRESS error and in the Windows Event logs VDS basic provider error
⇒ Solution
- Install the following Microsoft service pack on the Hyper-V Node and on each Microsoft VM running on that host.
Backup of a Linux VM fails
Problem
- When backing up a Linux VM without Guest Services support, backup job fails with error:
VSS_E_WRITERERROR_NONRETRYABLE
Backups of Linux systems with Guest Services support work normally.
⇒ Solution
- If backing up Linux VM fails with VSS_E_WRITERERROR_NONRETRYABLE error, open the VM Settings -> Integration Services and disable the option Backup (volume shadow copy).
Hyper-V restore fails
Problem
When attempting to restore a Hyper-V VM into a different Hyper-V environment, the restore might fail with one of the following errors:
Unable to import virtual machine due to configuration errors
or something like
The virtual machine <VM_name> is currently using an AMD processor, but physical computer <host_name> has an Intel processor. A running or saved virtual machine cannot be migrated to a physical computer that has a processor from a different vendor. However, you can move the virtual machine to this node if you shut down the virtual machine.
A configuration error may occur if a VM you want to restore is not compatible with the destination Hyper-V host. Restoring can fail for the following reasons:
- Hyper-V VMs are using advanced CPU features and fail if the CPUs are too different (different generation, type ...), when restoring VMs between servers with different processor types, e.g., Intel VT or AMD-V, or when the destination server is lacking virtualization extensions or they are disabled.
- Restoring a VM that is fully configured with virtual switches and network definitions into a different Hyper-V environment may result in the original switch definitions not being found.
⇒ Possible solutions
- Check VM and Hyper-V host compatibility
- Before restoring a VM to a different Hyper-V host, check whether the host can support the VM type you want to restore. Use the Compare-VM cmdlet to check a VM and a Hyper-V host compatibility. Run the following commands to first record the compatibility report in a PowerShell variable $Report and then display the compatibility report:
$report = Compare-VM -Path $Source $report.Incompatibilities | FT -AutoSize
Use the report to fix any compatibility or configuration issues. For details, see MS article Compare-VM.
- Enable processor compatibility mode for a virtual machine
- According to Microsoft: "Hyper-V performs pre-flight checks whenever a virtual machine live migration or save/restore operation is initiated. These checks compare the set of processor features that are available to the virtual machine on the source host against the set of features that are available on the target host. If these feature sets don’t match, the migration or restore operation is cancelled." For more information, see the whole MS article Processor Compatibility Mode in Hyper-V. To enable processor compatibility mode for a virtual machine, proceed as follows:
Note | |
The Processor Compatibility Mode is only applicable to processor types within the same vendor processor family. It does not enable migrations between AMD- and Intel-based hosts. For more information, check TechNet Magazine Tip: When to Use Processor Compatibility Mode to Migrate Virtual Machines. |
- Open Hyper-V Manager and shut down the VM which you want to configure for CPU compatibility mode.
- Right-click the powered off VM. In the Action pane, click Settings, and then click Processor.
- Expand Processor, and click Compatibility.
- Click Migrate to a physical computer with a different processor, and then click OK.
- Shut down VM, back up, and restore
- If you are restoring a VM from a Hyper-V host with one processor type (e.g., AMD) to a host with a different (e.g., Intel), you might try to move a Hyper-V VM by shutting it down, backing it up, and then restoring on the new host.
Hyper-V Manager 2016 VM fails to boot after restore
Problem
After successfully restoring a Hyper-V virtual machine (VM), the restored VM may be unable to boot. This issue specifically affects Hyper-V Manager 2016 or older versions. The cause of the boot failure is the Secure Boot option, which can create a mismatch between the VM's boot configuration and the restored environment.
⇒Solution
Disabling the Secure Boot option allows the VM to bypass the security checks that may be causing the boot problem. To disable the Secure Boot option:
- Shut down the restored VM in Hyper-V Manager 2016.
- Disable the Secure Boot option:
- In the Hyper-V Manager select the VM and go to Settings of the VM.
- Navigate to Hardware -> Security, and uncheck the option Enable Secure Boot.
- Click Apply and then OK to save the changes.
- Start the VM in the Hyper-V Manager.
During mount of a Hyper-V RCT backup, a Windows Explorer window is opened for every mounted VHDX
Problem
During mounting of a Hyper-V backup, Windows Explorer window opens for every mounted vhd(x). This problem occurs on Microsoft Windows workstation OS if Autoplay is enabled for removable drives and option Open folder to view files (File Explorer) is selected.
Possible solutions
For removable drives set the option Take no action as default, or disable Autoplay completely.
KVM/QEMU
Merging and deleting leftover snapshots
Problem
- The backup fails and the snapshot is left behind.
⇒ Solution
Use the virsh utility, as shown in the example:
- List the available snapshots for the domain:
- List the virtual disks for the domain:
- For each device that refers to SEP sesam snapshot, start a block commit to merge the snapshot:
- Confirm that the device is now switched to the original disk device:
- Delete the snapshot metadata information:
- Delete the snapshot file:
user@hypervisor:~$ virsh snapshot-list <domain_name> Name Creation Time State ------------------------------------------------------------ Sesam_SF20173828282@XXXX 2017-07-06 08:15:11 +0200 disk-snapshot
In this example, one leftover snapshot for this VM exists.
user@hypervisor:~$ virsh domblklist <domain_name> Target Source ------------------------------------------------ sda /path/to//Sesam_SF20173828282@XXXX.snapshot
user@hypervisor:~$ virsh blockcommit <domain_name> sda --active --verbose --pivot Block Commit: [100 %] Successfully pivoted
user@hypervisor:~$ virsh domblklist <domain_name> Target Source ------------------------------------------------ sda /my/original/base.img
user@hypervisor:~$ virsh snapshot-delete <domain_name> --metadata --snapshotname Sesam_SF20173828282@XXXX Domain snapshot sesam_snapshot deleted
user@hypervisor:~$ rm /path/to//Sesam_SF20173828282@XXXX.snapshot
Si3 Deduplication
Unable to establish connection to S3 data store
Problem
Si3 NG data store may be unable to establish secure connection to S3 storage with the following error:
Error: Could not access data store. Server Status: 2023-03-30 10:17:10: ERROR Not started due to error: S3 is not connected Server Status: 2023-03-30 10:17:10: ERROR Not started due to error: S3 is not connected
Cause
In case Si3 NG data store connects to a storage provider that uses a self-signed certificate, this certificate is not recognized as trustworthy by default because it is not issued by a trusted certificate authority. This can result in connection being denied and log files in /var/opt/sesam/var/log/sms
may contain a log message similar to this:
[...default-dispatcher-6] [1;31mERROR[0;39m [36mS3[0;39m - Unexpected error: {}, cause: {} software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: javax.net.ssl.SSLHandshakeException: General OpenSslEngine problem
⇒ Solution
To solve this problem use the keytool utility to import the public.crt certificate to the server certificate store. This will allow the Si3 server to recognize and trust the S3 storage provider's certificate, and establish a secure connection.
- Obtain the public certificate. Note that you can export it from the browser.
- Locate the cacerts file on your server. This is the location of your JVM certificate keystore.
- Import the public.crt certificate into the JVM's certificate keystore with the following command:
- on Linux:
keytool -import -trustcacerts -keystore /var/lib/ca-certificates/java-cacerts -storepass changeit -noprompt -alias <storage backend endpoint URL> -file /<path_to_certificate>/public.crt
- on Windows:
C:\Program Files\ojdkbuild\java-11-openjdk-11.0.15-1\bin>keytool -import -trustcacerts -keystore "C:\Program Files\ojdkbuild\java-11-openjdk-11.0.15-1\lib\security\cacerts" -storepass changeit -noprompt -alias <storage backend endpoint URL> -file <path_to_certificate>\public.crt
Issues with S3 or S3-compatible storage
Problem
- Si3 NG data store using S3 or S3-compatible storage can experience various issues, depending on cloud storage provider. These issues can affect backups, migrations, and replications. In addition, sanity state check of Si3 NG could report errors that have similar root cause.
Cause
- Some cloud storage providers (for example, Wasabi) have request rate restrictions (how many HTTP(S) requests are allowed per second). Also on local storage with S3 option enabled, when multiple RDSs access the same local S3 storage, this can generate a lot of IOPS (I/Os per second).
⇒ Solution
- You can adjust the settings on the affected Si3 NG data store:
- In the Main selection -> Components, click Data Stores to display the data store contents frame.
- Right-click the selected Si3 NG data store and then click Properties.
- Double-click a drive to open Drive Properties dialog, and then in Options field enter as follows:
dedup.s3.timeoutInSeconds=1200,dedup.s3.page.workers=2,dedup.maxAsyncRequests=50
- This will increase the timeout period, active page workers and request rate.
Si3 remains in "shutting down" state
Problem
- Manually stopping Garbage Collection (GC) fails and consequently Si3 remains in the "shutting down" state.
⇒ Solution
- Restart the Si3 daemon by using sm_main restart sds. For more details on stopping and starting the SEP sesam services, see How to Start and Stop SEP sesam.
Si3 deduplication may not work with NFSv4
Problem
- Si3 deduplication may not work with Network File System version 4 (NFSv4).
Cause
- SEP sesam operations, such as backup, restore and migration, may fail due to Java problems with NFSv4.
⇒ Solution
- To avoid this problem, connect your backup devices via NFSv3.
Repairing corrupted Si3 NG data store
You can repair the Si3 store when pages or objects get corrupted.
- First determine the scope of corruption:
- To get the list of corrupted objects use:
sm_dedup_interface -d <datastore> corruptedobjects
- To get the list of corrupted pages use:
sm_dedup_interface -d <datastore> corruptedpages
- To get the list of corrupted objects use:
- Use the following command to replace the page in /pages directory with an older version from /pages-trash directory:
sm_dedup_interface -d <datastore> repair pages
The pages in trash contain all chunks deleted on previous GC. The oldest version of a page takes priority. - Use the following command to search for and recover the missing chunks in /pages-trash directory:
sm_dedup_interface -d <datastore> repair start
During the repair process a new page is created, which contains all chunks from the current page (page affected by 'missing chunks' issue) and all chunks found in the trash.
Cleanup of unrecoverable Si3 store
Warning | |
You should use the commands described in this section only in case the corrupted store cannot be recovered. |
When corruptions in the Si3 store persist, the initial page version has already been purged from trash or there were fatal errors during backup or restore. In this case broken pages or missing chunks cannot be recovered.
Cleanup can be performed by deleting unrecoverable objects manually or by using the automatic cleanup function.
- Deleting objects
When there are only a few unrecoverable objects, delete each object with the following commands:
sm_dedup_interface -d <datastore> delete corruted_object_id_1 ... sm_dedup_interface -d <datastore> delete corruted_object_id_Nth
In case of many corruptions you can delete all corrupted objects using the following command:
sm_dedup_interface -d <datastore> fsck purge
- Garbage collection
When you have deleted all unrecoverable objects, run garbage collection (gc):
sm_dedup_interface -d <datastore> gc start
- Automatic cleanup function
To start an automatic cleanup function, use the following command:
sm_dedup_interface ... fsck purge auto
The automatic cleanup function runs the following sequence of commands: PCCK start -> OCCK start -> Delete all corrupted objects -> GC start.
Logging
The logging function uses a relatively powerful logback library. For more information, see Logback Project. Note that this information is intended for advanced users only.
- Logging info
- gv_rw_ini:sm_sds.xml (/var/opt/sesam/var/ini/sm_sds.xml)
- /var/opt/sesam/var/log/sms contains two log files:
- sm_dedup_server_info-<drive>.log: Log level INFO and higher.
- sm_dedup_server-<drive>.log: Log level DEBUG and higher. This file can become quite large.
- sm_dedup_gc-<drive>.log: garbage collection log.
- sm_dedup_fsck-<drive>.log: file system check log.
- Auto rotation if the log file size reaches 100 MB.
Files and directories
- Objects
For every SEP sesam saveset, three objects (files) are stored in the Si3 store:
- <ssid>.data
- <ssid>.info
- <ssid>.info2
The .data and .info files are identical to those of a normal data store. The .info2 file is required for the data to be appended to a Si3 object. All database information that is not available before a backup is completed is written to this file.
- Directories
Tape and tape devices issues
Problem with tape drives
Problem
There are problems with LTO tape drives which repeatedly request cleaning tape. Additionally, backups fail with I/O errors.
⇒ Solution
There are two possible causes for this:
- SEP sesam has problems communicating properly with tape devices if the hardware vendor's tape diagnostic tools are still installed after the tape read/write diagnostic test is successful. To prevent the tape drives from repeatedly issuing the cleaning request and ensure that backups complete successfully, remove the tape diagnostic tools immediately after completing the diagnostic tests. For example, there are some problems with the different tape drives LTO5 and LTO7. In our example, you will disable or remove the following components:
- Disable the HPE Version Control Agent.
- Disable all tape agents for HPE Insight Management Agents as shown below.
- Uninstall HP Library and Tape Tools (L&TT) from your server using the Add/Remove Program in the Window's Control Panel.
- Similar issues may be caused by common hardware-related problems. Refer to your hardware vendor documentation for details on hardware configuration and troubleshooting.
Restore fails due to a missing block that was not written to tape, but no error is generated during backup
Problem
(Applies only to Windows SEP sesam Server or RDS, ≤ SEP sesam Beefalo V2) When backing up to tape, a data block is not written to the tape when the segment size is reached at the End of Media (EOM), therefore restoring data referencing this block is not possible. A path archive can still be restored until EOM is reached, but a complete restore or migration fails.
⇒ Solution:
- Detect whether your tapes are affected by following the procedure How to detect affected savesets below.
- Update your Windows SEP sesam Server to a newer version. If your tape device is attached to a Windows RDS, you have to install the update to the respective RDS.
How to detect affected savesets
The following steps must be executed on your SEP sesam Server.
- Check the path behind the variable gv_rw_lis in sm.ini to see if your LIS folder's destination was changed. Example with modified LIS folder – set to
D:\SEPsesam\var\lis\
: - Open the command prompt ('Windows Command Processor' or Linux 'Terminal') and enter the following commands. Change to the directory of your LIS folder according to sm.ini gv_rw_lis=... :
- Windows (default):
C:\ProgramData\SEPsesam\var\lis
cd C:\ProgramData\SEPsesam\var\lis findstr /R /C:"^.*:.*:0:.*:1$" *.sgm
- Linux (default):
/var/opt/sesam/var/lis
cd /var/opt/sesam/var/lis grep "^.*:.*:0:.*:1$" *.sgm
- Windows (default):
- If the command retrieves some lines, check the related sgm files. In the following example, the third line (marked red) shows an occurrence of a lost block.
1:140479734912:1088:MP_File_Full00001:38956 1:143030262144:1089:MP_File_Full00001:48605 <span style="color:red">1:146212528704:0:MP_File_Full00010:1</span> 10:146212594176:1:MP_File_Full00010:55536
[PATHES] gv_rw_lis=D:\SEPsesam\var\lis\
The third line shows the following: On tape with ID 1: there are 146212528704 bytes written, but then it shows the segment 0 that is not a real segment number but caused by error, ending by :1 block that is actually lost.
Problems with LTO-9 tapes
Error loading new LTO-9 tape
Problem
SEP sesam is unable to load a new LTO-9 tape into a tape drive. When uncalibrated LTO-9 tapes are inserted into a library, they cannot be moved into the drive. During archive adjustment, SEP sesam identifies the slots containing such media and marks them with an orange indicator to signal that tape initialization is required.
Cause
Some tape libraries (for example, HPE MSL2024 up to G3, 1/8 G2 Autoloader, and similar) do not allow loading a non-calibrated LTO-9 tape media into a tape drive. This prevents accidental loading of non-calibrated tapes into the drive and helps prevent possible backup issues while a new LTO-9 tape initialization is in progress.
For more information, see HPE Support Center: LTO-9 Media initialization.
⇒ Solution:
For such libraries, use New Media Initialization Wizard to perform the new LTO-9 tape initialization.
Note that the initialization process for a tape can take between 20 minutes and up to 2 hours. In most cases it will complete within 30 minutes for a tape, however, in some cases it may take up to two hours. After start the initialization process cannot be aborted.
Limited or no access to new LTO-9 tapes
Problem
When introducing new LTO-9 tapes, SEP sesam may encounter access issues, time-outs or backup delays because of ongoing LTO-9 tape initialization.
Cause
Some tape libraries (for example, HPE StoreEver MSL6480, and similar) allow moving a non-calibrated LTO-9 tape media into a tape drive, but start the initialization process before the tape can be first used. In this case SEP sesam may encounter issues with backup because of ongoing LTO-9 tape initialization.
⇒ Solution:
LTO-9 tapes require one-time media initialization before read/write operations can first be started. In most cases this initialization process will complete within 30 minutes for a tape, however, in some cases it may take up to two hours. Note that after start the initialization process cannot be aborted.
To prevent issues with delayed or failed backups to new LTO-9 tapes, it is recommended to initialize the new LTO-9 tape using archive adjustment with the Automatic introduction option. This starts the media initialization before first use.
It is important that you do not use the option Adjustment by barcode only when running archive adjustment. This option collects barcodes for slots without moving the tape into the drive. Therefore, media initialization is not performed during archive adjustment. Instead, it will start the first time the tape is used.
Alternatively, you can use New Media Initialization Wizard to perform the new LTO-9 tape initialization before using it with SEP sesam.
Volume Shadow Copy Service (VSS)
SEP sesam uses Microsoft’s Volume Shadow Copy Service (VSS) to perform backups for various task types. It uses only the VSS components, provided by Microsoft. VSS failures are typically caused by the system configuration and not by SEP sesam. Note that VSS has some restrictions and limitations. For more detailed information on VSS, see the article Volume Shadow Copy Service from Microsoft.
This section provides only limited guidance for most common VSS issues. For more specific and difficult cases, refer to Microsoft support (KB).
Common VSS problems
Follow the instructions below if you encounter a VSS error. After each step, run a test to make sure that the problem has been fixed.
1. Backup is failing with a 10054 error. The event viewer shows the error: Cryptographic Services failed while processing the OnIdentity() call in the System Writer Object.
Problem
- The VSS backup on Windows 10 or 2012/R2 fails with an error. The backup log shows:
Info: Backup finished. Status: ERROR Data sending failed. (10054) An existing connection was forcibly closed by the remote host.
More detailed information about the cause of the error can be found in the application event log:
Cryptographic Services failed while processing the OnIdentity() call in the System Writer Object. Details: AddLegacyDriverFiles: Unable to back up image of binary Microsoft Link-Layer Discovery Protocol. System Error: Access is denied.
Cause
- The MSLLDP driver's security permissions do not allow NETWORK_SERVICE to access the driver record. This can cause the backup process to hang for a long time or fail.
⇒ Solution
- Open an administrative command prompt (NOT PowerShell) and execute the following command as one long command without carriage returns (do not move the cursor to the beginning of the line):
- If you get a successful result, e.g., [SC] SetServiceObjectSecurity SUCCESS, the problem is solved. Otherwise, reboot your Windows system.
sc sdset MSLLDP D:(D;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;BG)(A;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;SY)(A;;CCDCLCSWRPDTLOCRSDRCWDWO;;;BA)(A;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;SO)(A;;LCRPWP;;;S-1-5-80-3141615172-2057878085-1754447212-2405740020-3916490453)(A;;CCLCSWLOCRRC;;;SU)S:(AU;FA;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;WD)
2. Windows reboot
⇒ Solution
- Reboot your Windows. A Windows reboot often eliminates various VSS writer problems.
3. Disk space
Problem
- If there is insufficient disk space, VSS snapshots may fail.
⇒ Solution
- Make sure that sufficient disk space is available.
4. Multiple backup jobs with VSS
Problem
- If multiple VSS jobs are started, the operating system reports the following VSS error message:
VSS_E_SNAPSHOT_SET_IN_PROGRESS
Possible causes
- It is not possible to run multiple VSS snapshots at the same time. Windows can only start a VSS snapshot for each partition.
⇒ Solution
- Run the backup jobs from a client sequentially, for example:
- 22:00: Start System Recovery Backup
- 23:00: Start Custom Path Backup.
- If the temporal separation is not possible, try the following solution. These must be set in the following sm.ini settings on the client:
[SBC_OPTIONS] VSS_WAIT_FOR_ASYNC_OP=600 VSS_DELAY=120 VSS_MAX_RETRY=9
- With these settings, the VSS sesam client tries to execute operations sequentially and waits for an active VSS operation. The matching settings may vary depending on the system and backup jobs.
5. Third-party backup software
⇒ Solution
- Check if some other backup software from another vendor is running on the system, for example:
- Windows Backup, NTbackup
- Acronis, Backup Exec, etc.
Disable this backup software if necessary.
6. Remove old VSS snapshots
⇒ Solution
- Start the command prompt (CMD) as administrator and enter the following command:
vssadmin delete shadows /all
The command deletes all VSS snapshots of all volumes.
7. VSS writer status check
⇒ Solution
- Start the command prompt (CMD) as administrator and enter the following command:
vssadmin list writers
- If a VSS writer error appears, restart the following services:
- Volume Shadow Copy Service
- VSS application writer (Exchange, MS-SQL, Hyper-V)
- COM+ System Application Service
- Distributed Transaction Coordinator Service
- Run the command vssadmin list writers again and check if the problem is fixed.
8. Windows event log check
⇒ Solution
- Check the event logs of the Windows Event Viewer. Search for VolSnap errors corresponding to VSS at the time of the SEP sesam backup job.
- It often helps to search the internet for the event ID of the error to get an indication of the error cause.
9. Re-register VSS and COM+ components
Step 1:
- The following scripts re-register all VSS and COM+ components.
- Copy the commands into a new *.bat file. Start the batch script in the command line as administrator:
For 64-bit and 32-bit systems:
cd /d %windir%\system32 Net stop vss Net stop swprv regsvr32 ole32.dll regsvr32 vss_ps.dll Vssvc /Register regsvr32 /i swprv.dll regsvr32 /i eventcls.dll regsvr32 es.dll regsvr32 stdprov.dll regsvr32 vssui.dll regsvr32 msxml.dll regsvr32 msxml3.dll regsvr32 msxml4.dll regsvr32 Vssapi.dll regsvr32 Vssui.dll net start vss net start swprv
Step 2:
- Copy the commands into a new *.bat file. Start the batch script in the command line as administrator:
For 64-bit systems:
Net stop vss Net stop swprv regsvr32.exe /i %windir%\system32\eventcls.dll regsvr32.exe /i %windir%\system32\swprv.dll regsvr32.exe %windir%\system32\vssui.dll regsvr32.exe %windir%\SysWOW64\vss_ps.dll regsvr32.exe %windir%\SysWOW64\msxml.dll regsvr32.exe %windir%\SysWOW64\msxml2.dll regsvr32.exe %windir%\SysWOW64\msxml3.dll regsvr32.exe %windir%\SysWOW64\msxml4.dll regsvr32.exe %windir%\SysWOW64\ole32.dll regsvr32.exe %windir%\SysWOW64\oleaut32.dll regsvr32.exe %windir%\SysWOW64\es.dll regsvr32.exe %windir%\SysWOW64\comsvcs.dll vssvc /register net start swprv net start vss net stop winmgmt regsvr32 wmiutils.dll net start winmgmt
10. VM utility update
⇒ Solution
- If the system is a virtual machine, uninstall the installed utility version, start the VM, and then reinstall the latest VM utility version:
- VMware vSphere: VMware Tools
- Microsoft Hyper-V: Hyper-V Integration Services
- Citrix XenServer: XenServer Tools
11. Windows update
⇒ Solution
- Install the latest Windows updates, service packs and hot fixes from Microsoft.
- Run a test after the update.
VSS backup fails with VSS error
If a VSS error occurs when backing up your Windows client, you need to find out the cause. Open the Windows Event Viewer and check the relevant logs. These errors can be any of the following:
"VSS_ERROR_UNEXPECTED_ERRORCODE" "VSS_E_PROVIDER_VETO" "VSS_E_VOLUME_NOT_SUPPORTED"
Note | |
You may receive similar error messages and/or aborted backups if you try to perform a VSS backup on volumes larger than 64 TB, even if the volumes are excluded from the backup. Note that writable snapshots or snapshots larger than 64 TB produce the same error. |
Microsoft states the limitation of "64 TB" in the article Usability limit for Volume Shadow Copy Service (VSS) in Windows.
The above errors can be caused by any of the following:
Problem 1
- When trying to perform a VSS backup with volumes larger than 64 TB on a Windows server, you get the following error:
"DB Module: [ [NInternal::CVssAsyncDecorator::Wait] - S_OK]"
and the Windows event logs show a VSS event ID 12289
"VSS_ERROR_UNEXPECTED_ERRORCODE"
Cause
- Microsoft has a restriction of 64 TB as the maximum size for volumes when performing backups with VSS. It is not possible to use VSS for larger volumes, even if they are excluded from the backup.
⇒ Solution
- Enable the Volume Shadow Copy Service for all drives: Right-click on any of the listed Hard Disk Drives and click Configure Shadow Copies. In the Shadow Copies window, click Enable and then enable shadow copies for all drives.
Problem 2
- When trying to perform a VSS backup of a volume larger than 60 TB or when you create VSS backups of multiple volumes in Windows Server 2008/Windows Storage Server 2008, the VSS backup fails with the following message:
ERROR: [CVssServer::CreateSnapshot(): IVssBackupComponents::DoSnapshotSet: WaitForAsyncOperation] - VSS_E_PROVIDER_VETO
Possible causes
- As already explained (see above note), Microsoft has set a usability limit for volumes and snapshots with VSS.
- This error can also occur if a VSS provider blocks snapshot creation for the following reasons: Software has been installed with its own VSS provider or the existing VSS component is corrupted.
- HP Device Access Manager (FLCDLOCK service; part of the HP Protect Tools suite) prevents VSS operations from properly accessing drive partitions.
⇒ Solution
- Due to the limitation within volsnap.sys, do not use volumes larger than 60 TB as a backup source or destination for Windows backups.
- Remove volumes larger than 60 TB from the VSS snapshot action.
- Reconfigure all disks with a current size larger than 60 TB to 60 TB or less to prevent VSS from stopping when the snapshot is created.
- Also check the size of the shadow storage on the respective drives and increase it (if necessary). Use the Vssadmin command-line tool from an elevated command prompt (where c: should be replaced with the correct drive):
vssadmin Resize ShadowStorage /For=C: /On=C: /Maxsize=5%
- Fix the VSS problems and then re-register the VSS components. Additionally, check the size of the shadow storage on the drives in question and increase it (if necessary) with the Vssadmin command-line tool, as described in the previous solution.
- Uninstall HP Device Access Manager and restart your system. Note that incomplete uninstallation of HP Device Access Manager may cause problems.
You can use a third-party removal tool to correctly and completely uninstall Device Access Manager for HP ProtectTools. For details, see the UninstallHelps.com article How to uninstall Device Access Manager for HP ProtectTools?.
Note | |
The link provided will take you to a third party website. SEP provides such links for informational purposes only and is not responsible for the information, availability or accuracy of these external resources. |
Problem 3
- VSS Path backup from a Windows Storage Spaces volume fails with error:
Error: DB Module: [ [CVssServer::WaitForAsyncOperation: IVssAsync::QueryStatus] - VSS_E_PROVIDER_VETO]
Cause
- VSS snapshots are not supported for Windows Storage Spaces.
⇒ Solution
- To back up data from Windows Storage Spaces volumes, you have to disable the option Backup with VSS (enabled by default) in the backup task properties. For details, see Disabling/enabling VSS-based backups.
HPE StoreOnce Catalyst
HPE StoreOnce backups fail
If your backups fail, one of the reasons could be related to HPE StoreOnce sizing parameters. In this case, check the following:
- If you have defined a Physical or Logical Storage Quota for your Catalyst store, check if the quota limits have been reached. If so, increase the quota to a sufficient size. For details, see HPE StoreOnce Configuration.
- If you created an HPE Catalyst data store in SEP sesam and later changed the HPE Catalyst store size parameters, e.g., by changing or removing the storage quotas, check their values in SEP sesam GUI: Main selection -> Components -> Data Stores, double-click your StoreOnce store, and then click the HPE Catalyst Store State tab. If the data store status is set to "failed", you may need to adjust the StoreOnce data store (Size) Capacity and High watermark values to allow for correct calculation and make the data store functional again. For details, see Size and Disk space usage.
HPE StoreOnce backup failed with "553 STOR Failed. Error: Ctl_OpenObject failed, error: OSCLT_ERR_MAXIMUM_SESSIONS. (0)"
Problem
HPE StoreOnce backup failed with the following error message:
553 STOR Failed. Error: Ctl_OpenObject failed, error: OSCLT_ERR_MAXIMUM_SESSIONS.
This issue can occur when an HPE StoreOnce Catalyst data session fails to open because the maximum number of sessions on the HPE StoreOnce Catalyst server is reached, thus no other session can be started until resources are available. You can check the number of free data sessions in SEP sesam by opening your HPE StoreOnce data store properties and clicking the tab HPE Catalyst Store State, then checking the value of Free data sessions.
⇒ Solution
There are two possible solutions:
- The HPE Catalyst server supports a limited number of concurrent data sessions. Therefore, you need to ensure that the maximum number of data sessions for your HPE StoreOnce is not exceeded, or increase the maximum number of data sessions (including restores) that can be run simultaneously.
- Open SEP sesam and from the Main selection -> Components -> Data Stores double-click your HPE StoreOnce data store to open the properties. Then, under Drives click the relevant drive to view its properties. In the Max. channels drop-down list, decrease the number of available channels for concurrently running backup/migration streams. By default, the number of available channels is set according to your SEP sesam Server license (e.g., the standard license supports 10 concurrent streams, enabling 10 backup processes to run simultaneously).
HPE StoreOnce backup failed with error code "OSCLT_ERR_SERVER_OFFLINE"
Problem
A backup to a HPE StoreOnce datastore failed with the following error message:
553 STOR Failed. Command error with HPE StoreOnce server [<hostname>]: OSCLT_ERR_SERVER_OFFLINE. (0)
⇒ Solution
- Ensure that the SEP sesam Client can successfully connect to the HPE StoreOnce storage system over the network using the designated data transfer method.
- Check the DNS configuration on the SEP sesam Client to ensure it can resolve the storage system's hostname.
- Check network connectivity, correct routing, and that any required protocols or ports for communication are configured and accessible. Additionally, check the firewall settings along the network path to ensure no traffic is being blocked and that the correct routes are in place.
Proxmox VE
Proxmox VE backup does not work with the client name as plain IP address
Problem
- If the node added to the SEP sesam environment is not set up correctly and an IP address is used as the client name instead of the client's hostname matching the hostname returned by the Proxmox server, the backup fails.
⇒ Solution
- This problem can only be solved by correctly configuring the Proxmox clients in the SEP sesam environment. For more details how to correctly configure the client name, see Proxmox VE Backup: Adding the Proxmox server (and nodes) to the SEP sesam environment.
Proxmox VE backup failed with a connection error
Problem
- Proxmox VE backup failed with Error connecting to proxmox System: "('Connection aborted.', gaierror(-2, 'Name or service not known'))"]. This error occurs after a VM has been migrated to another cluster node. Consequently, the Proxmox VE backup fails.
⇒ Solution
- Ensure that every Proxmox VE cluster node can correctly resolve other cluster nodes using DNS or the Hosts file.
File backup of Proxmox hypervisor fails
Problem
- Backup of /etc/pve/ fails.
⇒ Solution
- Exclude the /etc/pve/ directory from backup. /etc/pve/ is just a file system representation of the cluster database, which is located under the path /var/lib/pve-cluster/config.db. For more information, see the Proxmox wiki article. For more information, see also Proxmox Cluster File System (pmxcfs).
See also
Analyzing SEP sesam Log Files – Interpreting Error Messages - Error Messages Guide – SEP sesam Exit Codes