Citrix XEN Server Troubleshooting

From SEPsesam
Jump to: navigation, search


Other languages:

Copyright © SEP AG 1999-2022. All rights reserved.

Any form of reproduction of the contents or parts of this manual is allowed only with the express written permission from SEP AG. When compiling and designing user documentation SEP AG uses great diligence and attempts to deliver accurate and correct information. However, SEP AG cannot issue a guarantee for the contents of this manual.

Docs latest icon.png Welcome to the latest SEP sesam documentation version 4.4.3 Beefalo/5.0.0 Jaglion. For previous documentation version(s), check documentation archive.


Citrix XEN Server

Error when using quiesced snapshots

Problem

  • Using VM.snapshot_with_quiesce method fails in Citrix Hypervisor ≥ 8.1 or in the version ≥ 9.0.x.x drivers.

Cause

  • VSS and quiesced snapshots of Windows VMs capability have been removed in Citrix Hypervisor ≥ 8.1 and in the version ≥ 9.0.x.x drivers. They are only supported in Citrix Hypervisor ≤ 8.0.

Solution

  • If you want to continue using the quiesced snapshot feature with Windows VMs hosted on Citrix Hypervisor 8.0 and earlier, do not update to the 9.0.x.x drivers.

CBT backup fails because Xen Server TLS certificate contains wrong IP address

Problem

  • CBT backup fails with the NBD client connection error:
  • Failed to get NBD client for device '******: NBD client connection error: hostname '****' doesn't match u'******
    

Cause

  • The TLS certificate of a Xen Server contains the wrong IP address. Therefore no NBD session can be established.

Solution

  • Check all Xen Servers in the Pool whether the IP address in the certificate matches.
  • On the Xen Server where it does not match, execute the following commands:
    1. Create a backup of the existing certificate by using
    2. mv /etc/xensource/xapi-ssl.pem /etc/xensource/xapi-ssl.pem.$(date -u +%Y%m%d)
      
    3. Create a new certificate
    4. /opt/xensource/libexec/generate_ssl_cert /etc/xensource/xapi-ssl.pem <ip-address or FQDN> 
      
    5. Restart XenAPI service
    6. systemctl restart xapi
      
    7. Restart NBD Service
    8. systemctl restart xapi-nbd 
      

Message "Upload to URL ... failed" appears during restore, "insufficient space" error occurs during backup

Problems

  • Restore fails with an error: "Upload to URL ... failed."
  • SR_BACKEND_FAILURE_44: : There is insufficient space to create snapshot on Storage Repository.

Possible cause

  • There is not enough space available on the default storage.
  • If the "insufficient space" error occurs during the backup, it might be related to the snapshot process. XenServer was unable to create a snapshot for the VM because there was not enough space in your Storage Repository.

Solution

  • Use XenCenter to select another default storage that has enough free disk capacity to store the virtual disk(s).

Other issues

  • Some errors are not reported in detail by XEN API. More information can be found in XenServer in /var/log directory.
  • In some pool configurations all pool members must have their management interface listed in DNS.
  • If the settings below don't agree on asynchronous routing and connection drops and/or 3 second or 80 second delays, opening sockets may occour. This is very similar to having 2 network cards configured with ipv4 to the same subnet and connected to the same switch.
  • If you use Cisco switches and you have configured (Citrix unsupported) lacp bonding 802.3ad, it is very important that both sides agree on the balancing type. The Linux default is layer2 and the Cisco default is layer3+4.
  • Check the bonding mode on the host with cat /proc/net/bond/bond* for bonding mode:
  • IEEE 802.3ad Dynamic link aggregation
    Transmit Hash Policy: layer2 (0) or Transmit Hash Policy: layer3+4 (1).

  • Users have reported that this command and a reboot solves the problem, but this cannot be formally recommended due to its untested and unsupported status:
  • xe pif-param-set uuid=${UUID_OF_BOND_PIF} other-config:bond-xmit_hash_policy=1 
    
  • In the common situation where 802.1ad with xmit_hash_policy=1 and STP are in use, multiple settings need to be adjusted on the Cisco side according to the article Considerations for Connecting XenServer to the Switch Ports.
  • Most other network related performance and reliability issues can be resolved with Considerations for Connecting XenServer to the Switch Ports.
  • In some situations when using thin storage repositories, an sr rescan is necessary to re-detect free space before a backup or restore.

See also

Citrix XEN Server