Archive for category VMWare
I will be running a webinar on behalf of UK Cloud Infrastructure User Group on the 27th of October @ 6 PM UK time. Please join me to take you through both Microsoft Cloud Adoption and Well Architected frameworks.
To register for the event: https://www.meetup.com/en-AU/UK-Cloud-Infrastructure-User-Group/events/273793787
I hope to see a lot of you there.
I found a nice script written to gather your environment Exchange environment RU level.
This is a message we have received while powering up our vMA template. The reason for this is that there isn’t an IP pool defined for the network adapter where vMA was plugged in to.
The solution is simple, click on your Cluster node within your vSphere client – click on IP Pools tab and then create an IP pool associating it with your physical NIC. This would assign an identity to your network adapter.
After creating the IP pool, you should be able to power on your appliance.
This weird behavior was observed on two data stores which we were planning to delete/unmounted from all our ESX/ESXi hosts.
The first error flagged “Can’t remove datastore ‘datastore_name‘ because Storage I/O Control is enabled on it. Correct it and re-try the operation”. So we tried disabling SIOC through vSphere client, the process never completed and got stuck at 4 – 14 %. This process didn’t flag any errors, but I had to increase the SIOC log to the maximum in order to find where the issue is.
To enable Storage I/O log, on each host > configuration > Advanced Settings, change Misc.SIOControlLogLevel to 7 and restart the SIOC on the host by using the following command through a SSH session
[For ESXi] /etc/init.d/storageRM restart
[For ESX] /etc/init.d/vmware-late restart
Error and warning messages can be traced in /var/log/messages
It became apparent that the SIOC wasn’t running on one of the hosts which caused the whole process of deleting a data store to fail.
Like many of you, I am frustrated with the information I get while a snapshot or several snapshots are being deleted. If you use the GUI to monitor this process then you won’t have much luck trying to determine at what stage the snapshot deletion process is at, the task would stick at 99% for minutes and maybe hours depending on the number of snapshots and their sizes.
There is no way to estimate time left, the only option is to SSH to the ESX or ESXi host, browse to the datastore where that specific machine files do reside. And run the following command
# ls –lhut
This should display files in the following format:
-rw——- 1 root root 40.0G Sep 6 08:27 ServerName-000001-delta.vmdk
This would arrange files according to their access time, starting from most recent. This method should give an indication of which .vmdk file the deletion process is active on. Each snapshot file has 6 digit number attached to it to indicate its position within the snapshot chain (e.g. ServerName-000001-delta.vmdk, ServerName-000002-delta.vmdk, ServerName-000003-delta.vmdk, etc). the deletion process goes through each file in sequence starting with ServerName-000001-delta.vmdk.
After upgrading our vCentre server to the latest patch (vCentre 4.1.0-345042), our Update Manager failed to update or even attempt to scan any of our hosts with the error “Host cannot download files from VMware vCenter Update Manager patch store. Check the network connectivity and firewall setup, and check esxupdate logs for details.”
I have searched on the web for the solution to this error and people were mentioning double checking DNS entries on each of the hosts, but that never changed in my case.
I went back to our vCentre server and checked each vCentre server component version installed and it turned out that VUM wasn’t running the latest version. After upgrading and rebooting the machine, ESX host and VM scan and updates were working back again.
I hope you find this training useful.
Coming from a WSUS era, VUM has provided flexibility and good patch/remediation management for virtual machines until you start seeing this error message “Patch scan or remediation is not supported on ServerName because of unsupported or unknown OS”, this has hit me hard especially moving forward to x64 versions of Windows 2008 R2 and Windows 7.
Vmware has released vSphere compatibility matrix found here. I have taken a snippet of this document shown below:
“Update Manager 4.1 and its subsequent update releases are the last releases of the product to support scanning and remediation of patches for Windows and Linux guest operating systems and applications running inside a virtual machine.” Supported document is found here. Make sure you rethink your patching management and deployment strategies for VM environment, I guess WSUS is still the way to go within Windows environment or purchasing MS System Centre Configuration Manager which makes software updates (not just windows updates) much easier with better control with other capabilities on top.
I am sure you have read or come across VMware best practices regarding snapshots. We use vDR in some of our backups in addition to Backup Exec to backup specific applications such as SQL and Exchange.
As a note, VMware snapshots are crash consistent and not application consistent. For that reason VMware do not recommend using snapshots on DC servers. To have an application consistent backup, Backup Exec and other third party backup application can do that for you.
Now back to our issue, running on multiple snapshots is bad. It is even worse when having multiple snapshots on one machine older than 3 days. Your machine would struggle reading from these multiple delta vmdk disks to recreate your data. These snapshots can get very large and can degrade the performance of the server immensely! We had this issue with an Exchange server that had 7 snapshots over 3 days old at various sizes, the server became unusable. The time it took to delete those snapshots were close to 10 hours and that wasn’t fun!
To avoid this crisis, setup alarms on your VCentre to cascade to all nodes in your cluster to alerts you for any VM running on snapshots.
There is a great KB article by VMware.