Source:Troubleshooting Disaster Recovery

Disaster recovery on Linux

Problem with a ReaR backup execution on SEP sesam Client

Problem

ReaR backups on the SEP sesam Client are experiencing issues, but path backups are functioning correctly. The backup log may contain errors such as sbc_vadp requires additional libraries (fatal error).

opt/sesam/bin/sesam/sbc_vadp requires additional libraries (fatal error)
	libpython2.7.so.1.0 => not found

Cause

This issue is caused by the sbc_vadp binary, which was included in the 5.0.0.15 Service Pack 1 update, but is not required on the Client.

⇒ Solution

Delete the sbc_vadp binary. You can use the following command:

rm /opt/sesam/bin/sesam/sbc_vadp

The workflow mkrescue is not supported in the ReaR system

Problem

The workflow mkrescue is not supported in the ReaR rescue/recovery system.

⇒ Solution

Delete the file /etc/rear-release.

ReaR image hangs during bootup

Problem

The system hangs during bootup like shown in the following image:

⇒ Solution

Boot the system with the ACPI=OFF option (this option can be specified on the command line in the boot menu prompt, after the options BACKUP=SESAM OUTPUT=ISO).

The recovered system does not boot

Problem 1

The system does not boot because /root/dev/console cannot be found.

Possible causes

Certain distributions rely on the existence of the directory /dev/ while booting
Certain static devices must exist before the udev daemon creates them.

⇒ Solution

Include the /dev/ file system in your backup.
If the restore cannot restore /dev/:

Boot from the SEP sesam LIVE CD.
Mount the ROOT partition of the restored system.
Manually create the /dev/ directory.
Manually create the /dev/console entry with:

mknod /path/to/target/mount//dev/console c 0 0

Problem 2

The system does not boot because of missing libblkid.so.1.

Possible cause

This is most likely caused by SELinux which is activated by default.

⇒ Solution

Especially on RHEL6 or CentOS6 systems, follow these steps after rebooting from the ReaR recovery:

Press a key when prompted by the boot loader (GRUB):

Select the appropriate boot loader entry:

Press e to modify the commands for the selected entry:

Add selinux=0 to the commands:

Press Enter to confirm the changes and b to boot up the machine with SELinux disabled.
When having access to the system, change the option SELinux of /etc/selinux/config to the following:

Afterwards, reboot the system and feel free to set the SELinux value back to enforcing if needed.

No bootable operating system can be found

Problem

The system is not able to find a bootable OS instance after the restore.

Possible causes

There may have been problems during the installation of the GRUB boot loader.

⇒ Solution

The restore protocol includes a statement whether or not the installation of the boot loader was successful:

2009-12-14 14:48:27: sbc-3500: Info:     Reinstall boot manager
[/sesam/bin/sesam//sbc_grub_auto /mnt/disk/ AUTO]

It is also possible to boot the system again from the live-CD, mount the target partitions and use grub-install to install the boot loader correctly.

The device does not have a corresponding BIOS drive

Problem

During the restore, the following error occurs:

/dev/sda1 does not have any corresponding BIOS drive

Possible causes

Check the file /boot/grub/device.map on the target system. If there are entries referring to the disk through /dev/by-disk/... as shown in the example below, the entry is most likely the reference to the hard disk partition of the broken system. GRUB will not find the proper device:

hd(0) /dev/disk/by-id/ata-SAMSUNG_SP2504C_S09QJ1GLA14263-part1

⇒ Solution

Reboot from the live-CD
Mount the root and boot partitions to /mnt/disk (and /mnt/disk/boot, if necessary)
Restart grub-install with the following options:

grub-install --root-directory=/mnt/disk --recheck hd0

Output:

grub-probe: error: Cannot open `/boot/grub/device.map'
/usr/sbin/grub-install: line 374: [: =: unary operator expected
Installation finished. No error reported.
This is the contents of the device map /mnt/disk/boot/grub/device.map.
Check if this is correct or not. If any of the lines is incorrect,
fix it and re-run the script `grub-install'.

(hd0)   /dev/hda
(hd1)   /dev/hdb

You can ignore the error line 374: [: =: unary operator expected.
More important is the result Installation finished. No error reported.

No corresponding BIOS drive for /dev/cciss/c0d0p2

Problem

You receive the message: /dev/cciss/c0d0p2 does not have any corresponding BIOS drive in restore log.

⇒ Solution

Please see: Novell support

fsck.ext3: File system has unsupported features

Problem

During a restore of a system with kernel version 2.4 the system may not boot because the Live-CD creates a file system with features which are not supported by kernel 2.4.

Possible causes

Most likely the file system options resize_inode,dir_index,large_file,ext_attr are causing the problem and making the system unbootable.

⇒ Solution

Reboot from the Live-CD image, which includes the tool debugfs.
Show the file system features with debugfs:

root@recover#: debugfs -w /dev/sda2
debugfs 1.41.1 (01-Sep-2008)
debugfs:  features
Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file
quit

Replace /dev/sda2 with the corresponding partition names on your system.

To remove file system features:

root@recover#: debugfs -w /dev/sda2
debugfs:  features -resize_inode -ext_attr -dir_index -large_file -needs_recovery -sparse_super
Filesystem features: has_journal filetype
quit

After removing the options, the system should boot correctly.

Incorrect inode size (256)

Problem

After a successful restore the boot process stops with incorrect inode size (256).

Possible causes

Older kernel versions (2.4) may use a different inode size than the one the file system's created through the Live-CD (which includes kernel 2.6). For example, this happens during the restore of SLES8 based systems which use an inode size of 128k.

⇒ Solution

This can only be solved by formatting the devices manually from the Live-CD, using the proper mkfs options:

mkfs.ext3 -I 128 /dev/sda1

After this step, remount the partition to /mnt/disk and repeat the restore operations. Changing the inode size is only possible by reformatting the devices.

Missing root file system

Problem

The restored system can't find a root file system and fails during resume.

Possible causes

The /etc/fstab file was configured with the root file system as UUID.

⇒ Solution

Specify the root file system device name in conventional device names if you are using a different physical disk. After booting, use YAST to reconfigure your boot loader or edit your /boot/grub/menu.lst manually:

root=/dev/sda2

Missing network cards

Problem

The restored system does not find any network cards.

Possible causes

If the restore was done to dissimilar hardware, SLES-based distributions may not configure the network devices correctly. SLES-based systems save their network configuration by using the system's MAC address. Most likely the system will not use eht0 as a device name, but eth1, as it has another MAC address.

⇒ Solution

Use YaST and reconfigure your network interfaces.

Client does not start on the RHEL6/Debian9 recovery image

Problem

The SEP sesam Client does not start automatically on RHEL6 and Debian9-based recovery images.

Cause

The file /etc/init.d/functions is missing within the recovery image.

⇒ Solution

The client can be started manually via:

/opt/sesam/bin/sesam/sm_main start

RHEL7-related issues

RHEL7 backup fails with an error

Error 1

The RHEL backup fails with the following error:

ERROR: The LSB package is not installed.

⇒ Solution

Install the lsb package as follows:

yum install redhat-lsb-core mkisofs syslinux

Error 2

The RHEL backup fails with:

ERROR: Cannot find required programs: mingetty

For more details, see Rear dependencies on RHEL7.

⇒ Solution

To solve this problem, proceed as follows:

Edit

 /var/opt/sesam/var/lib/rear/usr/share/rear/conf/default.conf

and from the line:

# required programs. Same as above, but if they are missing, we abort.
 REQUIRED_PROGS=(
 "$SCRIPT_FILE"

remove the line:

 mingetty

Run the backup again.

ReaR error occurred during grub2-mkimage of bootx64.efi

	Note
	In order to be able to create an UEFI/EFI bootable ISO image, the additional tool ebiso has to be installed on the client system as described in the section Installing ebiso for creating UEFI aware ISO images.

Problem

The ReaR error occurrs during grub2-mkimage of bootx64.efi.

⇒ Solution

To solve the problem, install the grub2-efi-x64-modules package.

SLES-related issues

`SM_SSH` does not work on SLES11 recovery image

⇒ Solution

In this case, execute

mount -t tmpfs none /dev/shm/ -o rw,nosuid,nodev,noexec

before starting the recovery process.

Client is unreachable after booting the rescue image

Problem

After booting the rescue image the client is not reachable.

⇒ Solution

Start the client manually using the following command on the rescue command line:

sh /etc/scripts/system-setup.d/59-start-sesam-client.sh

EFI bootable image cannot be created on SLES11

Problem

EFI bootable image of GRUB2 cannot be created on SLES11.

Cause

SEP sesam v. 4.4.3.64 Grolar is the last version that supports SLES11 with UEFI.

⇒ Solution

To continue using SLES with UEFI, you should not upgrade to a later version of SEP sesam.

Installing ebiso for creating UEFI aware ISO images

In order to be able to create an UEFI/EFI bootable ISO image the, the additional tool ebiso has to be installed on the client system. This package is not part of a regular SLES12/SLES15 installation and can be downloaded at the following URL:

http://download.opensuse.org/repositories/Archiving:/Backup:/Rear/SLE_12/x86_64/

or

http://download.sep.de/utils/bsr-linux/

For other Linux distributions contact SEP support at support@sep.de for assistance.

Install ebiso as follows:

rpm -i ebiso-<version>.rpm

Note that in ReaR v. < 1.19, the generated ISO image mount migt be too small for storing all needed information and need to be adjusted.

In this case, under

/var/opt/sesam/var/lib/rear/usr/share/rear/lib/uefi-functions.sh (line 64)

change

(shim.efi|elilo.efi) size=128000 ;;

to

(shim.efi|elilo.efi) size=228000 ;;

Any form of reproduction of the contents or parts of this manual is allowed only with the express written permission from SEP AG. When compiling and designing user documentation SEP AG uses great diligence and attempts to deliver accurate and correct information. However, SEP AG cannot issue a guarantee for the contents of this manual.

Disaster recovery on Linux

Problem with a ReaR backup execution on SEP sesam Client

The workflow mkrescue is not supported in the ReaR system

ReaR image hangs during bootup

The recovered system does not boot

No bootable operating system can be found

The device does not have a corresponding BIOS drive

No corresponding BIOS drive for /dev/cciss/c0d0p2

fsck.ext3: File system has unsupported features

Incorrect inode size (256)

Missing root file system

Missing network cards

Client does not start on the RHEL6/Debian9 recovery image

RHEL7-related issues

RHEL7 backup fails with an error

ReaR error occurred during grub2-mkimage of bootx64.efi

SLES-related issues

SM_SSH does not work on SLES11 recovery image

Client is unreachable after booting the rescue image

EFI bootable image cannot be created on SLES11

Installing ebiso for creating UEFI aware ISO images

`SM_SSH` does not work on SLES11 recovery image