Disaster recovery on Linux
Problem with a ReaR backup execution
- SEP sesam Client package was successfully installed and path backups are working. If you have problems executing a ReaR backup check the backup log if there are some missing dependencies. Example:
There are binaries or libraries in the ReaR recovery system that need additional libraries /opt/sesam/bin/sesam/libvirtmod.so requires additional libraries (fatal error) libvirt.so.0 => not found
- In this case the package libvirt-client is missing.
- Depending on your distribution, install libvirt-client as follows:
- On RHEL/CentOS 7 use the command: yum install libvirt-client
- On SLES 12/15 use: zypper install libvirt-client
The workflow mkrescue is not supported in the ReaR system
- The workflow mkrescue is not supported in the ReaR rescue/recovery system.
- Delete the file /etc/rear-release.
ReaR image hangs during bootup
- The system hangs during bootup like shown in the following image:
- Boot the system with the ACPI=OFF option (this option can be specified on the command line in the boot menu prompt, after the options BACKUP=SESAM OUTPUT=ISO).
The recovered system does not boot
- The system does not boot because /root/dev/console cannot be found.
- Certain distributions rely on the existence of the directory /dev/ while booting
- Certain static devices must exist before the udev daemon creates them.
- Include the /dev/ file system in your backup.
- If the restore cannot restore /dev/:
- Boot from the SEP sesam LIVE CD.
- Mount the ROOT partition of the restored system.
- Manually create the /dev/ directory.
- Manually create the /dev/console entry with:
mknod /path/to/target/mount//dev/console c 0 0
- The system does not boot because of missing libblkid.so.1.
- This is most likely caused by SELinux which is activated by default.
- Especially on RHEL6 or CentOS6 systems, follow these steps after rebooting from the ReaR recovery:
- Press a key when prompted by the boot loader (GRUB):
- Select the appropriate boot loader entry:
- Press e to modify the commands for the selected entry:
- Add selinux=0 to the commands:
- Press Enter to confirm the changes and b to boot up the machine with SELinux disabled.
- When having access to the system, change the option SELinux of /etc/selinux/config to the following: SELINUX=permissive
Afterwards, reboot the system and feel free to set the SELinux value back to enforcing if needed.
No bootable operating system can be found
- The system is not able to find a bootable OS instance after the restore.
- There may have been problems during the installation of the GRUB boot loader.
- The restore protocol includes a statement whether or not the installation of the boot loader was successful:
2009-12-14 14:48:27: sbc-3500: Info: Reinstall boot manager [/sesam/bin/sesam//sbc_grub_auto /mnt/disk/ AUTO]
- It is also possible to boot the system again from the live-CD, mount the target partitions and use grub-install to install the boot loader correctly.
The device does not have a corresponding BIOS drive
- During the restore, the following error occurs:
/dev/sda1 does not have any corresponding BIOS drive
- Check the file /boot/grub/device.map on the target system. If there are entries referring to the disk through /dev/by-disk/... as shown in the example below, the entry is most likely the reference to the hard disk partition of the broken system. GRUB will not find the proper device:
- Reboot from the live-CD
- Mount the root and boot partitions to /mnt/disk (and /mnt/disk/boot, if necessary)
- Restart grub-install with the following options:
grub-install --root-directory=/mnt/disk --recheck hd0
grub-probe: error: Cannot open `/boot/grub/device.map' /usr/sbin/grub-install: line 374: [: =: unary operator expected Installation finished. No error reported. This is the contents of the device map /mnt/disk/boot/grub/device.map. Check if this is correct or not. If any of the lines is incorrect, fix it and re-run the script `grub-install'. (hd0) /dev/hda (hd1) /dev/hdb
You can ignore the error line 374: [: =: unary operator expected.
More important is the result Installation finished. No error reported.
No corresponding BIOS drive for /dev/cciss/c0d0p2
- You receive the message: /dev/cciss/c0d0p2 does not have any corresponding BIOS drive in restore log.
- Please see: Novell support
fsck.ext3: File system has unsupported features
- During a restore of a system with kernel version 2.4 the system may not boot because the Live-CD creates a file system with features which are not supported by kernel 2.4.
- Most likely the file system options resize_inode,dir_index,large_file,ext_attr are causing the problem and making the system unbootable.
- Reboot from the Live-CD image, which includes the tool debugfs.
- Show the file system features with debugfs:
root@recover#: debugfs -w /dev/sda2 debugfs 1.41.1 (01-Sep-2008) debugfs: features Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file quit
Replace /dev/sda2 with the corresponding partition names on your system.
- To remove file system features:
root@recover#: debugfs -w /dev/sda2 debugfs: features -resize_inode -ext_attr -dir_index -large_file -needs_recovery -sparse_super Filesystem features: has_journal filetype quit
After removing the options, the system should boot correctly.
Incorrect inode size (256)
- After a successful restore the boot process stops with incorrect inode size (256).
- Older kernel versions (2.4) may use a different inode size than the one the file system's created through the Live-CD (which includes kernel 2.6). For example, this happens during the restore of SLES8 based systems which use an inode size of 128k.
- This can only be solved by formatting the devices manually from the Live-CD, using the proper mkfs options:
mkfs.ext3 -I 128 /dev/sda1
After this step, remount the partition to /mnt/disk and repeat the restore operations. Changing the inode size is only possible by reformatting the devices.
Missing root file system
- The restored system can't find a root file system and fails during resume.
- The /etc/fstab file was configured with the root file system as UUID.
- Specify the root file system device name in conventional device names if you are using a different physical disk. After booting, use YAST to reconfigure your boot loader or edit your /boot/grub/menu.lst manually:
Missing network cards
- The restored system does not find any network cards.
- If the restore was done to dissimilar hardware, SLES-based distributions may not configure the network devices correctly. SLES-based systems save their network configuration by using the system's MAC address. Most likely the system will not use eht0 as a device name, but eth1, as it has another MAC address.
- Use YaST and reconfigure your network interfaces.
Client does not start on the RHEL6/Debian9 recovery image
- The SEP sesam Client does not start automatically on RHEL6 and Debian9-based recovery images.
- The file /etc/init.d/functions is missing within the recovery image.
- The client can be started manually via:
RHEL7 backup fails with an error
- The RHEL backup fails with the following error:
ERROR: The LSB package is not installed.
- Install the lsb package as follows:
yum install redhat-lsb-core mkisofs syslinux
- The RHEL backup fails with:
ERROR: Cannot find required programs: mingetty
For more details, see Rear dependencies on RHEL7.
To solve this problem, proceed as follows:
- Run the backup again.
and from the line:
# required programs. Same as above, but if they are missing, we abort. REQUIRED_PROGS=( "$SCRIPT_FILE"
remove the line:
ReaR error occurred during grub2-mkimage of bootx64.efi
|In order to be able to create an UEFI/EFI bootable ISO image, the additional tool ebiso has to be installed on the client system as described in the section Installing ebiso for creating UEFI aware ISO images.
- The ReaR error occurrs during grub2-mkimage of bootx64.efi.
- To solve the problem, install the grub2-efi-x64-modules package.
SM_SSH does not work on SLES11 recovery image
- In this case, execute
mount -t tmpfs none /dev/shm/ -o rw,nosuid,nodev,noexec
- before starting the recovery process.
Client is unreachable after booting the rescue image
- After booting the rescue image the client is not reachable.
- Start the client manually using the following command on the rescue command line:
EFI bootable image cannot be created on SLES11
- EFI bootable image of GRUB2 cannot be created on SLES11.
- SEP sesam v. 220.127.116.11 Grolar is the last version that supports SLES11 with UEFI.
- To continue using SLES with UEFI, you should not upgrade to a later version of SEP sesam.
Installing ebiso for creating UEFI aware ISO images
In order to be able to create an UEFI/EFI bootable ISO image the, the additional tool ebiso has to be installed on the client system. This package is not part of a regular SLES12/SLES15 installation and can be downloaded at the following URL:
For other Linux distributions contact SEP support at firstname.lastname@example.org for assistance.
Install ebiso as follows:
rpm -i ebiso-<version>.rpm
Note that in ReaR v. < 1.19, the generated ISO image mount migt be too small for storing all needed information and need to be adjusted.
In this case, under
/var/opt/sesam/var/lib/rear/usr/share/rear/lib/uefi-functions.sh (line 64)
(shim.efi|elilo.efi) size=128000 ;;
(shim.efi|elilo.efi) size=228000 ;;