Posts Tagged VMWare
The resource “datastore_name” is in use while attempting to delete that data store
This weird behavior was observed on two data stores which we were planning to delete/unmounted from all our ESX/ESXi hosts.
The first error flagged “Can’t remove datastore ‘datastore_name‘ because Storage I/O Control is enabled on it. Correct it and re-try the operation”. So we tried disabling SIOC through vSphere client, the process never completed and got stuck at 4 – 14 %. This process didn’t flag any errors, but I had to increase the SIOC log to the maximum in order to find where the issue is.
To enable Storage I/O log, on each host > configuration > Advanced Settings, change Misc.SIOControlLogLevel to 7 and restart the SIOC on the host by using the following command through a SSH session
:
[For ESXi] /etc/init.d/storageRM restart
[For ESX] /etc/init.d/vmware-late restart
Error and warning messages can be traced in /var/log/messages
It became apparent that the SIOC wasn’t running on one of the hosts which caused the whole process of deleting a data store to fail.
Practical Powershell
Posted by Sam in Compellent, Poweshell, VMWare on 9 July, 2012
I have been trying to link the information I get from VMware vSphere and the Compellent SAN to be able to trace VM activity within the storage layer end. I found a good post on the above link which gives a good link between the two systems using Powershell.
Host cannot download files from VMware vCenter Update Manager patch store
After upgrading our vCentre server to the latest patch (vCentre 4.1.0-345042), our Update Manager failed to update or even attempt to scan any of our hosts with the error “Host cannot download files from VMware vCenter Update Manager patch store. Check the network connectivity and firewall setup, and check esxupdate logs for details.”
I have searched on the web for the solution to this error and people were mentioning double checking DNS entries on each of the hosts, but that never changed in my case.
I went back to our vCentre server and checked each vCentre server component version installed and it turned out that VUM wasn’t running the latest version. After upgrading and rebooting the machine, ESX host and VM scan and updates were working back again.
Want to become an ESXi expert? here is how…
VMWare has released a three hour training video plus a free ebook to take you through the new features of ESXi 4.1, it’s worth looking at for VMware and system administrators.
I hope you find this training useful.
Patch scan or remediation is not supported on “ServerName” because of unsupported or unknown OS
Coming from a WSUS era, VUM has provided flexibility and good patch/remediation management for virtual machines until you start seeing this error message “Patch scan or remediation is not supported on ServerName because of unsupported or unknown OS”, this has hit me hard especially moving forward to x64 versions of Windows 2008 R2 and Windows 7.
Vmware has released vSphere compatibility matrix found here. I have taken a snippet of this document shown below:
“Update Manager 4.1 and its subsequent update releases are the last releases of the product to support scanning and remediation of patches for Windows and Linux guest operating systems and applications running inside a virtual machine.” Supported document is found here. Make sure you rethink your patching management and deployment strategies for VM environment, I guess WSUS is still the way to go within Windows environment or purchasing MS System Centre Configuration Manager which makes software updates (not just windows updates) much easier with better control with other capabilities on top.
Vmware Snapshots and VM Performance
I am sure you have read or come across VMware best practices regarding snapshots. We use vDR in some of our backups in addition to Backup Exec to backup specific applications such as SQL and Exchange.
As a note, VMware snapshots are crash consistent and not application consistent. For that reason VMware do not recommend using snapshots on DC servers. To have an application consistent backup, Backup Exec and other third party backup application can do that for you.
Now back to our issue, running on multiple snapshots is bad. It is even worse when having multiple snapshots on one machine older than 3 days. Your machine would struggle reading from these multiple delta vmdk disks to recreate your data. These snapshots can get very large and can degrade the performance of the server immensely! We had this issue with an Exchange server that had 7 snapshots over 3 days old at various sizes, the server became unusable. The time it took to delete those snapshots were close to 10 hours and that wasn’t fun!
To avoid this crisis, setup alarms on your VCentre to cascade to all nodes in your cluster to alerts you for any VM running on snapshots.
There is a great KB article by VMware.
Extending RDM and VMFS LUN’s
You will find that mapped RDM datastores in virtual compatibility mode cannot be extended simply by a re-scan of the HBA and extending the drive. In this case you will need to shut down the server, make note of the scsi x:y postion, un-map that volume from the virtual machine and re-map it again. This would recreate the descriptor file with the added space.
A simple rescan within your VM OS would show the added space which you can extended using DISKPART or any other tool out there.
RDM’s in physcial compatibility mode can be extended easily without the above steps.
Changing QLogic FC HBA firmware configuration using vMA on ESXi
In order to manage an ESX or ESXi host using vMA they will need to be added to vMA first:
sudo vifp addserver <servername>
It would ask you for a username and password which would cache in order to run any future remote scripts.
vifptarget –s <servername>
This would let you target a specific ESX/ESXi host
For the list of available parameters for Qla2xxx and Qla4xxx have a look at (Changing HBA parameters in ESXi).
To apply a single or a set of parameters using vMA, use the following syntax:
Example: vicfg-module –s ‘qlport_down_retry=60 ql2xmaxqdepth=64’ qla2xxx
To display parameters set on an HBA use the command: vicfg-module –g qla2xxx
Make sure you reboot the host after making any changes to take effect.
Note: If you apply a single parameter change at a time it would reset all other parameters to default and only apply the last parameter change to the host.
Host vMotion fails with “source detected that destination failed to resume”
I have searched the internet high and low for some information on this error, the virtual machine fails at 82% during host migration.
No snapshots were attached to the server, however there were a large number of .vmsn files in the root folder. I have had problems backing up that server using vDR because it used to fail on creating the snapshot. Apparently these files are created even though the backup was failing which accumulated over time, files aren’t being marked for deletion.
Going back to vMotion, some of the information recorded on the web is about misconfiguration of VMFS ID’s depending on how you assigned the name to the datastore and how consistent is that name within your virtual environment. ESX assigns a hexadecimal value based on the name you assign to the datastore. For a successful vMotion, each shared datastore should have the same hexadecimal representation on each ESX host and that can be checked by running vdf –h on the console on each host.
The problem was solved by doing a storage vMotion to another shared datastore and then performing host vMotion which worked successfully. There is another way which entails disconnecting the shared datastore and re-mapping it again. I haven’t tried this myself by it worked for some people with similar issue. The latter will cause some downtime since you have to disconnect the LUN and reconnect it again which means shutting down all your effected VM’s.
First look at RVTools
As stated on the RVTOOL Site, it is a Windows .Net application which interacts with your virtual environment using the latest VI SDK to gather health and resource information about each of the ESX hosts and virtual machines. You can either connect directly to an ESX host or call out your environment via vCentre.
I have come across this tool while trying to find an easy way to examine and release a CD or Floppy Disk if mounted to any of the virtual machines, but this is only few of the many uses for this tool. The new version allows passthrough authentication, so you can use your domain username and password to login and use the tool. It also provides some new command line options.
I would recommend having a look at this tool.
SysCloud.dev 
