Source:Troubleshooting Disaster Recovery

From SEPsesam
Revision as of 11:49, 22 March 2022 by Unknown (talk | contribs) (Updating to match new version of source page)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Other languages:


Template:Copyright SEP AG en

Docs latest icon.png Welcome to the latest SEP sesam documentation version 4.4.3 Beefalo/5.0.0 Jaglion. For previous documentation version(s), check documentation archive.


Disaster recovery on Linux

Problem with a ReaR backup execution

Problem

  • SEP sesam Client package was successfully installed and path backups are working. If you have problems executing a ReaR backup check the backup log if there are some missing dependencies.
  • Example:
    There are binaries or libraries in the ReaR recovery system that need additional libraries
    /opt/sesam/bin/sesam/libvirtmod.so requires additional libraries (fatal error)
    	libvirt.so.0 => not found
    

Cause

  • In this case the package libvirt-client is missing.

Solution

  • Depending on your distribution, install libvirt-client as follows:
    • On RHEL/CentOS 7 use the command: yum install libvirt-client
    • On SLES 12/15 use: zypper install libvirt-client

The workflow mkrescue is not supported in the ReaR system

Problem

  • The workflow mkrescue is not supported in the ReaR rescue/recovery system.

Solution

  • Delete the file /etc/rear-release.

ReaR image hangs during bootup

Problem

  • The system hangs during bootup like shown in the following image:
  • Rear-hang.jpg

Solution

  • Boot the system with the ACPI=OFF option (this option can be specified on the command line in the boot menu prompt, after the options BACKUP=SESAM OUTPUT=ISO).

The recovered system does not boot

Problem 1

  • The system does not boot because /root/dev/console cannot be found.

Possible causes

  • Certain distributions rely on the existence of the directory /dev/ while booting
  • Certain static devices must exist before the udev daemon creates them.

Solution

  • Include the /dev/ file system in your backup.
  • If the restore cannot restore /dev/:
  1. Boot from the SEP sesam LIVE CD.
  2. Mount the ROOT partition of the restored system.
  3. Manually create the /dev/ directory.
  4. Manually create the /dev/console entry with:
mknod /path/to/target/mount//dev/console c 0 0

Problem 2

  • The system does not boot because of missing libblkid.so.1.

Possible cause

  • This is most likely caused by SELinux which is activated by default.

Solution

  • Especially on RHEL6 or CentOS6 systems, follow these steps after rebooting from the ReaR recovery:
    1. Press a key when prompted by the boot loader (GRUB):
    2. Rhelcentos grub1.jpg
    3. Select the appropriate boot loader entry:
    4. Rhelcentos grub2.jpg
    5. Press e to modify the commands for the selected entry:
    6. Rhelcentos grub3.jpg
    7. Add selinux=0 to the commands:
    8. Rhelcentos grub4.jpg
    9. Press Enter to confirm the changes and b to boot up the machine with SELinux disabled.
    10. When having access to the system, change the option SELinux of /etc/selinux/config to the following:
    11. SELINUX=permissive

    Afterwards, reboot the system and feel free to set the SELinux value back to enforcing if needed.

No bootable operating system can be found

Problem

  • The system is not able to find a bootable OS instance after the restore.

Possible causes

  • There may have been problems during the installation of the GRUB boot loader.

Solution

  • The restore protocol includes a statement whether or not the installation of the boot loader was successful:
2009-12-14 14:48:27: sbc-3500: Info:     Reinstall boot manager
[/sesam/bin/sesam//sbc_grub_auto /mnt/disk/ AUTO]

  • It is also possible to boot the system again from the live-CD, mount the target partitions and use grub-install to install the boot loader correctly.

The device does not have a corresponding BIOS drive

Problem

  • During the restore, the following error occurs:
/dev/sda1 does not have any corresponding BIOS drive

Possible causes

  • Check the file /boot/grub/device.map on the target system. If there are entries referring to the disk through /dev/by-disk/... as shown in the example below, the entry is most likely the reference to the hard disk partition of the broken system. GRUB will not find the proper device:
hd(0) /dev/disk/by-id/ata-SAMSUNG_SP2504C_S09QJ1GLA14263-part1

Solution

  • Reboot from the live-CD
  • Mount the root and boot partitions to /mnt/disk (and /mnt/disk/boot, if necessary)
  • Restart grub-install with the following options:
grub-install --root-directory=/mnt/disk --recheck hd0

Output:

grub-probe: error: Cannot open `/boot/grub/device.map'
/usr/sbin/grub-install: line 374: [: =: unary operator expected
Installation finished. No error reported.
This is the contents of the device map /mnt/disk/boot/grub/device.map.
Check if this is correct or not. If any of the lines is incorrect,
fix it and re-run the script `grub-install'.

(hd0)   /dev/hda
(hd1)   /dev/hdb

You can ignore the error line 374: [: =: unary operator expected.
More important is the result Installation finished. No error reported.

No corresponding BIOS drive for /dev/cciss/c0d0p2

Problem

  • You receive the message: /dev/cciss/c0d0p2 does not have any corresponding BIOS drive in restore log.

Solution

fsck.ext3: File system has unsupported features

Problem

  • During a restore of a system with kernel version 2.4 the system may not boot because the Live-CD creates a file system with features which are not supported by kernel 2.4.

Possible causes

  • Most likely the file system options resize_inode,dir_index,large_file,ext_attr are causing the problem and making the system unbootable.

Solution

  • Reboot from the Live-CD image, which includes the tool debugfs.
  • Show the file system features with debugfs:
root@recover#: debugfs -w /dev/sda2
debugfs 1.41.1 (01-Sep-2008)
debugfs:  features
Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file
quit

Replace /dev/sda2 with the corresponding partition names on your system.

  • To remove file system features:
root@recover#: debugfs -w /dev/sda2
debugfs:  features -resize_inode -ext_attr -dir_index -large_file -needs_recovery -sparse_super
Filesystem features: has_journal filetype
quit

After removing the options, the system should boot correctly.

Incorrect inode size (256)

Problem

  • After a successful restore the boot process stops with incorrect inode size (256).

Possible causes

  • Older kernel versions (2.4) may use a different inode size than the one the file system's created through the Live-CD (which includes kernel 2.6). For example, this happens during the restore of SLES8 based systems which use an inode size of 128k.

Solution

  • This can only be solved by formatting the devices manually from the Live-CD, using the proper mkfs options:
mkfs.ext3 -I 128 /dev/sda1

After this step, remount the partition to /mnt/disk and repeat the restore operations. Changing the inode size is only possible by reformatting the devices.

Missing root file system

Problem

  • The restored system can't find a root file system and fails during resume.

Possible causes

  • The /etc/fstab file was configured with the root file system as UUID.

Solution

  • Specify the root file system device name in conventional device names if you are using a different physical disk. After booting, use YAST to reconfigure your boot loader or edit your /boot/grub/menu.lst manually:
root=/dev/sda2

Missing network cards

Problem

  • The restored system does not find any network cards.

Possible causes

  • If the restore was done to dissimilar hardware, SLES-based distributions may not configure the network devices correctly. SLES-based systems save their network configuration by using the system's MAC address. Most likely the system will not use eht0 as a device name, but eth1, as it has another MAC address.

Solution

  • Use YaST and reconfigure your network interfaces.

Client does not start on the RHEL6/Debian9 recovery image

Problem

  • The SEP sesam Client does not start automatically on RHEL6 and Debian9-based recovery images.

Cause

  • The file /etc/init.d/functions is missing within the recovery image.

Solution

  • The client can be started manually via:
  • /opt/sesam/bin/sesam/sm_main start

RHEL7-related issues

RHEL7 backup fails with an error

Error 1

  • The RHEL backup fails with the following error:
  • ERROR: The LSB package is not installed.

Solution

  • Install the lsb package as follows:
  • yum install redhat-lsb-core mkisofs syslinux

Error 2

Solution

  • To solve this problem, proceed as follows:
    1. Edit
    2. /var/opt/sesam/var/lib/rear/usr/share/rear/conf/default.conf and from the line: # required programs. Same as above, but if they are missing, we abort. REQUIRED_PROGS=( "$SCRIPT_FILE" remove the line: mingetty
    3. Run the backup again.

ReaR error occurred during grub2-mkimage of bootx64.efi

Information sign.png Note
In order to be able to create an UEFI/EFI bootable ISO image, the additional tool ebiso has to be installed on the client system as described in the section Installing ebiso for creating UEFI aware ISO images.

Problem

  • The ReaR error occurrs during grub2-mkimage of bootx64.efi.

Solution

  • To solve the problem, install the grub2-efi-x64-modules package.

SLES-related issues

SM_SSH does not work on SLES11 recovery image

Solution

  • In this case, execute
  • mount -t tmpfs none /dev/shm/ -o rw,nosuid,nodev,noexec before starting the recovery process.

Client is unreachable after booting the rescue image

Problem

  • After booting the rescue image the client is not reachable.

Solution

  • Start the client manually using the following command on the rescue command line:
  • sh /etc/scripts/system-setup.d/59-start-sesam-client.sh

EFI bootable image cannot be created on SLES11

Problem

  • EFI bootable image of GRUB2 cannot be created on SLES11.

Cause

  • SEP sesam v. 4.4.3.64 Grolar is the last version that supports SLES11 with UEFI.

Solution

  • To continue using SLES with UEFI, you should not upgrade to a later version of SEP sesam.

Installing ebiso for creating UEFI aware ISO images

In order to be able to create an UEFI/EFI bootable ISO image the, the additional tool ebiso has to be installed on the client system. This package is not part of a regular SLES12/SLES15 installation and can be downloaded at the following URL:

http://download.opensuse.org/repositories/Archiving:/Backup:/Rear/SLE_12/x86_64/

or

http://download.sep.de/utils/bsr-linux/

For other Linux distributions contact SEP support at support@sep.de for assistance.

Install ebiso as follows:

rpm -i ebiso-<version>.rpm

Note that in ReaR v. < 1.19, the generated ISO image mount migt be too small for storing all needed information and need to be adjusted.

In this case, under

/var/opt/sesam/var/lib/rear/usr/share/rear/lib/uefi-functions.sh (line 64)

change

(shim.efi|elilo.efi) size=128000 ;;

to

(shim.efi|elilo.efi) size=228000 ;;

See also

SEP sesam Disaster Recovery