Xen318

Tuesday, January 24, 2017

Below is an alternate method that I have used to remove disks owned by control domain

Obtain the UUID of the VM in question

<UUID of VM> = 26708987-c6b2-3243-c49c-db20a96b15a8

From the command line interface run the following command:

xe vbd-list vm-uuid=26708987-c6b2-3243-c49c-db20a96b15a8
uuid ( RO) : 81ae5475-02fd-ba46-68f6-4a170102641a
vm-uuid ( RO): 26708987-c6b2-3243-c49c-db20a96b15a8
vm-name-label ( RO): FvWebProxy01
vdi-uuid ( RO): <not in database>
empty ( RO): true
device ( RO): xvdd

uuid ( RO) : 974dc129-48bf-eac2-ab6c-467ef0dfd620
vm-uuid ( RO): 26708987-c6b2-3243-c49c-db20a96b15a8
vm-name-label ( RO): FvWebProxy01
vdi-uuid ( RO): 8d673a64-26a2-4e8e-8c8d-0cd4efcf2520
empty ( RO): false
device ( RO): xvda

and then execute the following command. NOTE: I chose vdi-uuid 8d673a64 because the other VDI is listed as not in the database

xe vbd-list vdi-uuid=8d673a64-26a2-4e8e-8c8d-0cd4efcf2520
uuid ( RO) : 64cd3f2e-66c4-ae3f-7f53-d83d97d2601f
vm-uuid ( RO): 70ad4283-6d3e-47c6-9c3c-070f586f43c2
vm-name-label ( RO): Control domain on host: FsXENSVR152
vdi-uuid ( RO): 8d673a64-26a2-4e8e-8c8d-0cd4efcf2520
empty ( RO): false
device ( RO): sm/backend/2710d6e3-fb6b-0cdf-0230-845f87620eab/8d673a64-26a2-4e8e-8c8d-0cd4efcf2520

uuid ( RO) : 974dc129-48bf-eac2-ab6c-467ef0dfd620
vm-uuid ( RO): 26708987-c6b2-3243-c49c-db20a96b15a8
vm-name-label ( RO): FvWebProxy01
vdi-uuid ( RO): 8d673a64-26a2-4e8e-8c8d-0cd4efcf2520
empty ( RO): false
device ( RO): xvda

and then run the following command. NOTE: I chose UUID 64cd3f2 because as you can see the vm-name-label is Control domain on host

xe vbd-unplug uuid=64cd3f2e-66c4-ae3f-7f53-d83d97d2601f
The device is not currently attached
device: 64cd3f2e-66c4-ae3f-7f53-d83d97d2601f

xe vbd-destroy uuid=64cd3f2e-66c4-ae3f-7f53-d83d97d2601f

The VM disks that were owned by the control domain should now be gone.

Monday, January 23, 2017

VM disks Owned by the Control Domain

You tried to migrate a VM, move a disk, or take a snapshot, and you look on your SR to find that your VM’s disk is owned by “the control domain”.

1. Why does this happen?

Basically it boils down to disk operations on/from a VM failing. Operations such as move, copy, snapshot, export, etc.

2. How do I get around it?

One answer is to reboot.

Another way is to find the link between your virtual disk and DOM0 (the control domain) from your stand-alone XenServer (or primary server if you have a pool).

Use list_domains to get the UUID of DOM0 (the control domain):

The control domain UUID is 09dffafe-5bec-430d-bc80-6ddb2313beff

Now we have to find the VBD(s) that DOM0 (the control domain) owns:

xe vbd-list vm-uuid=<DOM0 UUID, such as 09dffafe-5bec-430d-bc80-6ddb2313beff id>

This will show information about any Virtual Block Device (VBD) that DOM0 (the control domain) may be hanging onto. What we are interested in is the UUID of the VBD(s).

Grab the VBD’s UUID and execute:

xe vbd-unplug uuid=<UUID of the VBD being held by DOM0>

Finally, run:

xe vbd-destroy uuid=<UUID of the VBD being held by DOM0>

And there – rescan your storage repository and your VDI (virtual disk interface) should be free for your use.

Switch Commands

Dell M8024-k switch

Enable SSH on switch

Connect to switch via CLI

Module A1(config)#crypto key generate dsa

Do you want to overwrite the existing DSA keys? [Y | N] :y

DSA key generation started, this may take a few

minutes..................................................................................................................................

.......................................................
DSA key generation complete.

Module A1(config)#crypto key generate rsa

Do you want to overwrite the existing RSA keys? [Y | N] :y

RSA key generation started, this may take a few minutes....................
RSA key generation complete.

Module A1(config)#ip ssh server

Module A1(config)#exit

Module A1#exit

Module A1>en
Password:***********

Module A1#show ip ssh

SSH Server enabled. Port: 22
Protocol Levels: Versions 1 and 2.
SSH Connections Currently in Use: ............. 0
Maximum number of SSH Sessions Allowed: ....... 5
SSH Session Timeout: .......................... 600
RSA key was generated.
DSA key was generated.
SSH Public Key Authentication is disabled.

Active Incoming Sessions.

Ip Address User Name Idle Time Session Time
--------------- --------------- ------------ ------------

Module A1#exit

Module A1>exit

=====================
Disable Telnet
=====================

config
ip telnet server disable
exit

test a telnet connection to see if it fails

========================================================================

Ran the following command from the command line interface :

config t
line vty 0 4
transport input ssh

line 5 15
transport input ssh

copy run start

I tested a telnet connection and it failed. SSH connection worked successfully.

========================================================================

============
Enable HTTPS
============

Module A2(config)#crypto certificate 1 generate

Module A2(config-crypto-cert)#?

common-name Specifies the common name.
country Specifies the country name.
do Run Privileged Exec mode commands.
duration Specifies number of days a self-signed certi fication
would be valid. If unspecified defaults to 3 65 day.
email Specifies the contact email address.
exit To exit from the mode.
key-generate Regenerate SSL RSA key.
location Specifies the location or city name.
organization-name Specifies the organization name
organization-unit Specifies the organization internal unit
show Show configured settings and operational sta tus.
state Specifies the state or province name.

Module A2(config-crypto-cert)#key-generate ?

<cr> Press enter to execute the command.
<length> Specifies the length of the SSL's RSA key. I f
unspecified, length defaults to 1024.

Module A2(config-crypto-cert)#key-generate

Module A2(config-crypto-cert)#exit

Certification Generation Successful..

Module A2(config)#ip http secure-certificate 1

Module A2(config)#ip http secure-server

Module A2(config)#exit

Module A2#show ip http server secure status

HTTPS Server is Enabled. Port : 443
DH Key exchange enabled.
Certificate 1 is active.
Issued by: self-signed
Valid from Mar 8 05:11:14 2006 GMTMar 8 05:11:14 2007 GMT0.0.0.0 to Mar 8 05:11:14 2007 GMT0.0.0.0
Subject: /CN=0.0.0.0
Fingerprint: DF1027F336CC450ED2AC1C740DF24921

Module A2#show ip telnet

Telnet Server is Disabled. Port : 23

Module A2#show ip http server status

HTTP Server is Disabled. Port : 80

Monday, May 9, 2016

XenServer Host Crashed - What to do?

Good info

http://www.gregorystrike.com/2012/04/16/xenserver-host-failed-what-now/

Courtesy of: http://citrixtechs.com/blog/help-my-citrix-xenserver-poolmaster-is-down-2/

It’s 3 O’Clock in the morning, do you know where your Xenserver Poolmaster is? Your client calls you frantic, and you start a GoToMeeting to see what’s wrong. If it’s down, this could have been the result of a few issues. Maybe there was a network glitch which resulted in the Citrix XenServer Poolmaster fencing itself from the rest of the farm. This can also result during a power outage, or other catastrophic failure. This is the normal defense mechanism built into Xenserver, and in the consulting world we see this type of scenario often. You can’t simply reboot the Poolmaster to bring it online. Restarting the toolstack will do you no good. There is a complex process that must be followed, so let’s discuss it –

If you’ve tried to connect to the pool from the Xencenter console, and it failed – your Poolmaster may be down. Verify this by dropping to a command prompt and issue a command like “Xe host-list” to see if you get a coherent response. If you get an error message like ““Cannot perform operation as the host is running in emergency mode” – then your Poolmaster is almost certainly down.

How do I get the Poolmaster back up?

This is easier said than done. First you’ve got to promote another server in the farm to become the Poolmaster, so that it can take over pool operations. From that servers CLI, type the command, “xe pool-emergency-transition-to-master” which will transition it to be the new Poolmaster. If the command runs successfully, you can recover the other pool servers by issuing the command, “xe pool-recover-slaves”. Now if pool management is working again, you should be able to successfully run the “xe host-list” command and get a valid response.

Now that the pool is back online, how do I fix the failed poolmaster?

1). First you have to figure out which server in the environment has failed. To do this, you’ll want to run the command, “xe host-list params=uuid,name-label,host-metrics-live”. Any servers that come back with “host-metrics-live = false” have failed. Take note of the UUID of any failed servers

2). Second, you must determine which VM’s were running on that failed server. You can do this by running the command, “xe vm-list is-control-domain=false resident-on=UUID_of_failed_server”. Once you’ve determined which VM’s were running, you need to reset their power state in order to get them to move onto another server. To do this, run the command, “xe vm-reset-powerstate resident-on=UUID_of_failed_server –force –multiple”. You should see the VM’s in question now show up as halted in the Xencenter console. Restart each of the VM’s, and they should now boot up onto surviving pool member servers.

For more information on various issues you can run into during this process, check out the official Citrix whitepaper here:

http://support.citrix.com/servlet/KbServlet/download/17140-102-18520/XenServer%20System%20Recovery%20Guide.pdf

What are some root causes as to why my Xenserver Poolmaster may have been down?

The usually suspects include your network, because if the poolmaster loses connectivity to some of the other Xenserver hosts in your environment, it could fence itself and go offline as a built in defense mechanism. Poolmaster fencing is a typical issue that can occur if there are network issues in your environment, so check with the network team before you pass go.

Monday, July 6, 2015

XenServer Host Failure

XenServer Host Failed

Find the UUID of failed server
xe host-list

Find out which VMs were running on failed server
xe vm-list resident-on= <UUID of XenServer>

Tell pool that the VMs are powered off
xe vm-reset-powerstate resident-on= <UUID of XenServer> --force --multiple

This will reset the power for the VMs that were running on this XenServer so that we can now see them in XenCenter

Friday, April 24, 2015

Steps to address a hung XenServer virtual machine:

Find the UUID of the hung virtual machines. This is done from the command line of the XenServer hosting the problem virtual machine. The command to execute is xe vm-list. Another way to find the UUID of the virtual machine is to click the ‘General’ tab of the virtual machine in XenCenter.

Once you have the UUID you need to find the domain ID of the hung virtual machine. This is done by executing the command list_domains from the XenServer hosting the problem virtual machine. You will need to match the UUID with domain ID number. The output of this command will look like this:

id | uuid | state

0 | 2fe455fe-3185-4abc-bff6-a3e9a04680b0 | R

47 | 267227f3-a59e-dafe-b183-82210cf51ec4 | B

59 | 298817fb-8a3e-7501-11e0-045a8aa860ff | B

(i.e. UUID 267227f3-a59e-dafe-b183-82210cf51ec4 has a domain ID of 47)

To free the virtual machine we need to execute destroy_domain command on the domain ID. This is done by executing the command:

/opt/xensource/debug/destroy_domain –domid 47

Note: 47 is the domain ID

If the Host reports the vm in a halted state after running xe vm-list, try to reset the power.

To reset the power state run the following command:

xe vm-reset-powerstate vm=VM NAME force=true where VM Name is the server name.

xe vm-shutdown vm=VMNAME force=true where VM NAME is the server name.

Once the VM is shutdown, restart on another XenServer

xe vm-start vm=VMNAME

Sometime running commands using the XenServer command line will yield you more information about a problem that otherwise XenCenter may not reveal to you.

For example: If the VM is having issues starting; executing the command xe vm-start name-label= <Name of VM> will likely get more information about what may be causing the issue.

Also, sometimes a server will not report to its host after being migrated. It is either stuck in powering off (staying in yellow) or powering on. Run this command on from the pool master for that pool.

xe-toolstack-restart

The VM should show back up. If not close out of XenCenter and reopen.

How to Setup XenServer 6.x to Auto-Start Virtual Machines

Setting the XenServer to allow Auto-Start

Gather the UUID’s of the pools you wish to auto-start.

To get the list of the pool’s on your XenServer type “xe pool-list”

Copy the UUID of the pool. If you have just one server, it will still have a pool UUID as noted in the following screenshot:

Then type the following command to set the pool or server to allow auto-start:
xe pool-param-set uuid=UUID other-config:auto_poweron=true

Note: Replacing UUID with the UUID of the XenServer or pool.

Setting the Virtual Machines to Auto-Start

Gather the UUID’s of the Virtual Machine you want to auto-start by typing:

xe vm-list

Note: This generates a list of Virtual Machines in your pool or server and their associated UUID’s.

Copy the UUID of the Virtual Machines you want to auto-start, and type the following command for each Virtual Machine to auto-start:

xe vm-param-set uuid=UUID other-config:auto_poweron=true

Note: Replace UUID with the UUID of the Virtual Machine to auto-start.