I think its fitting to begin this posting with a famous quote by Abraham Lincoln “If I had eight hours to chop down a tree, I’d spend six hours sharpening my ax”

I was in that very position a few days ago, I had to choose between recreating a VM, reinstalling Server 2008, running updates, then installing a key application with minimum down time due to the  accidental deletion of  some key VMware files. After analyzing my issue I decided to research the problem instead of just start chopping away with the obvious solution. In the end I can across the a very useful  KB article from vmware about recreating the vmdk descriptor file which did the trick for me.

Below is an image of the common files you would normally see when browsing a VMware guest data-store, now in my case everything was deleted except the “testsrvr2k8.vmdk” file.

VMware file extensions:

.VMDK — These files are the actual hard disk of the virtual machine itself, and tend to be the largest file within the folder. You can consider the size of this file to be roughly equivalent to the size of either the disk itself (if you’ve chosen to use preallocate disks) or the size of the data currently stored on that disk (if you use grow-able disks i.e thin provisioning).

.NVRAM – Consider this file the BIOS of the virtual machine.

.VMX – With typically one VMX file per folder, this file holds the configuration information for the virtual machine in a text format. Unlike almost all the other files you’ll see, these files can be edited using any text editing program, a process that is actually required for some functionality that is not exposed in the GUI.

.VMXF – This file, in XML format, includes additional information about the virtual machine if it has been added to a team. If a machine has been added to a team and then later removed, this file remains resident. This file can also be opened and read in a text editor.

.VMTM – For virtual machines actively participating in a team, this file stores information about that team membership.

.VMEM – These files, which contain a backup of the VMs paging file, are typically very small or non-existent when the virtual machine is powered off, but grow immediately to the size of configured RAM when the machine is powered on.

.VMSN and .VMSD – When snapshots are created for a virtual machine, these files are created to host the state of the virtual machine. The VMSN file stores the running state of the machine, what you could consider the “delta” between the VMDK at the point of the snapshot and what has been processed up until the present time. The VMSD stores information and metadata about the snapshot itself.

.VMSS – If you’ve suspended the state of your machine, this file contains the suspended state of that machine. These files typically only appear when virtual machines have been suspended.

1. Identify the size of the flat file in bytes.
2. Create a new blank virtual disk that is the same size as the original. This serves as a baseline example that is modified in later steps.

   Note: This step is critical to assure proper disk geometry.

3. Rename the descriptor file (also referred to as a header file) of the newly-created disk to match the name of the original virtual disk.
4. Modify the contents of the renamed descriptor file to reference the flat file.
5. Remove the leftover temporary flat file of the newly-created disk, as it is not required.

Detail Steps:

To create a virtual machine disk:

1. Log in to the terminal of the VMware ESX host.
   • For VMware ESX 4.1 and earlier, see Connecting to an ESX host using a SSH client (1019852). Alternatively, access the system directly and press Alt+F1 to begin the login process. Log in as root.
   • For VMware ESXi 4.1, see Using Tech Support Mode in ESXi 4.1 (1017910).
   • For VMware ESXi 4.0 and 3.5, see Tech Support Mode for Emergency Support (1003677).

2. Navigate to the directory that contains the virtual machine disk with the missing descriptor file using the command:

   cd /vmfs/volumes/myvmfsvolume/mydir
   
   Note:
   • If you are using a version of VMware ESXi, you can access and modify files and directories using the VMware vSphere client Datastore Browser or the vifs utility included with the vSphere CLI. For more information, see the section Performing File System Operations in vSphere CLI documentation.
   • If you are using VMware Fusion, the default location for the virtual machine files is <home>/Documents/Virtual Machines.localized/<virtual machine>/ folder where <home> is your home folder, and <virtual machine> is the name of the virtual machine.

3. Identify the type of SCSI controller the virtual disk is using. You can do this by examining the virtual machine configuration file (.vmx). The controller is identified by the line scsi#.virtualDev, where # is the controller number; there may be more than one controller and controller type attached to the virtual machine. This example uses lsilogic:

   scsi0.present = “true”
   scsi0.sharedBus = “none”
   scsi1.present = “true”
   scsi1.sharedBus = “virtual”
   scsi1.virtualDev = “lsilogic”
   
4. Identify and record the exact size of the -flat file using a command similar to:

   # ls -l vmdisk0-flat.vmdk
   -rw——- 1 root root 4294967296 Oct 11 12:30 vmdisk0-flat.vmdk
   

5. Use the vmkfstools command to create a new virtual disk:

   # vmkfstools -c 4294967296 -a lsilogic -d thin temp.vmdk This command uses these flags:

   -c <size>  (This is the size of the virtual disk).
   -a <virtual controller> (Whether the virtual disk was configured to work with BusLogic or LSILogic).
   -d thin  (This creates the disk in a thin-provisioned format).

   Note: To save disk space, we create the disk in a thin-provisioned format using the type thin. The resulting flat file then consumes minimal amounts of space (1MB) instead of immediately assuming the capacity specified with the -c switch. The only consequence, however, is the descriptor file contains an extra line that must be removed manually in a later step.

   The files temp.vmdk and temp-flat.vmdk are created as a result.
   

6. Delete temp-flat.vmdk, as it is not needed. Run the command:

   # rm temp-flat.vmdk
   
7. Rename temp.vmdk to the name that is required to match the orphaned .flat file (or vmdisk0.vmdk, in this example):

   # mv temp.vmdk vmdisk0.vmdk
   
8. Edit the descriptor file with a text editor:
   1. Under the Extent Description section, change the name of the .flat file to match the orphaned .flat file you have.
   2. Find and remove the line ddb.thinProvisioned = “1″ if the original .vmdk was not a thin disk. If it was, retain this line.
      # Disk DescriptorFile
      version=1
      CID=fb183c20
      parentCID=ffffffff
      createType=”vmfs”# Extent description
      RW 8388608 VMFS “vmdisk0-flat.vmdk”

      # The Disk Data Base

      #DDBddb.virtualHWVersion = “4″
      ddb.geometry.cylinders = “522″
      ddb.geometry.heads = “255″
      ddb.geometry.sectors = “63″
      ddb.adapterType = “lsilogic”

      ddb.thinProvisioned = “1″

      The virtual machine is now ready for power on. Verify your changes before starting the virtual machine.

• Although these steps are specific to the VMware ESX host’s Service Console (otherwise regarded as the Console OS), they may also be applied to platforms using VMware desktop products such as VMware Server or VMware Workstation. Instead of using the vmkfstools command, use the vmware-vdiskmanager utility. For more information on the Virtual Disk Manager, see the Virtual Disk Manager User’s Guide.
• When performing these operations on a different platform, you must use the equivalent commands for your platform. For example, move, copy, and del for Microsoft Windows platforms and mv, cp, and rm for Unix, Linux, and Mac OS X platforms.
• For more information about platform-specific commands, see: DOS versus Linux Commands from the Linux Documentation Project.

Once you are finish the above steps recreate you guest VM and instead of creating a new disk select “Use Existing Disk” and browse to the datastore and select your VMDK file from there. In my case that was testsrvr2k8.vmdk.

All worked well for me however when I turned on my VM it kept rebooting and blue-screening on me, the fix for that was to change the SCSI controller type from “LSI logic SAS” to “LSI Logic Parallel” rebooted and all was well.

Links:

http://kb.vmware.com/kb/1002511

I got the following message when I was attempting to change the DNS address on a few of  my VMware Servers today.

The IP address XXX.XXX.XXX.XXX you have entered for this network adapter is already assigned to another adapter Name of adapter. Name of adapter is hidden from the network and Dial-up Connections folder because it is not physically in the computer or is a legacy adapter that is not working. If the same address is assigned to both adapters and they become active, only one of them will use this address. This may result in incorrect system configuration. Do you want to enter a different IP address for this adapter in the list of IP addresses in the advanced dialog box?
In this message, XXX.XXX.XXX.XXX is an IP address that you are trying to set and Name of adapter is the name of a network adapter that is present in the registry but hidden in Device Manager.

I tried  Microsoft’s suggestion in this article “http://support.microsoft.com/?kbid=269155” however it didn’t work for me. I had to create a “system environment variable” and give it the value of “1″. Once that was completed I was able to follow the rest of the article. I removed all hidden adapters except for the “RAS Adapter” and I was then able to add the new IP address.

How to create a system variable:

On a Windows machine right click “My Computer” –> Properties –> Advanced tab–> Environment Variables–> System variables –>Click new –> Specify a “Variable name” (devmgr_show_nonpresent_devices)  and the “Variable value” which is 1

Enjoy now you can rest easy!

Reference Links:
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1179

http://support.microsoft.com/?kbid=269155

Phase one: Verification

1. Verify that the required network interface cards are up and the switch port is up and configure correctly.
ESX Command to check network cards:

 esxcfg-nics –l 

Once you confirmed that your NICS are up verify the duplex setting on both your network switch and your ESX host.

2. Verify that you have the  correct network addressing configuration:
less /etc/sysconfig/network

3.       Verify your vSwitch configuration.
esxcfg-vswitch –l
This command is also use to create and update a virtual machine (vswitch) network settings as you will see later one. Here you can see that NIC 0 and 2 are both assigned to vSwitch0

Verify your service console network setting.
esxcfg-vswif –l
This command is also used to create and update the service console network settings. This command is used if you cannot manage the ESX Server host through the VI Client because of network configuration issues

                   

      You can also use this command to create and update the virtual machine (vswitch) network settings.

If you are still unable to establish connectivity to the hosts, by this time, you can use the following commands to delete and recreate the vSwitch.

Phase two: ChangesAt this point if after verifying the following settings you don’t see anything unusual, you would take note of the above settings, delete them and recreate it.

esxcfg-vswitch  –d vSwitch0 –> To remove virtual switch
esxcfg-vswitch  –a vSwitch0 –> To re-add a virtual switch
esxcfg-vswitch -L vmnic0 vSwitch0 –> To assign physical nic0  (repeat and add additional nics)
esxcfg-vswif  -d vswif0 –> To remove the service console network.

esxcfg-vswif -a vswif0 -p “Service Console” -i 10.100.0.114 -n 255.255.255.128 –> Add a Service Console (vswif0)
esxcfg-vswitch -v 69 -p “Service Console” vSwitch0 –> Add Vlan 69 to service console on vSwitch0


After these steps its important to restart the network service with the “service network restart” command.

As stated in a post back in December I am working on rebuilding my home network, I looking to go “Green” and cut down on some of the excess energy. So what better way to do that than to virtualized my network. However with virtualization come decisions such as XenServer or VMware. So the purpose of this post is to go through my thought pattern as to why I want to leave WMware and migrate to XenServer.

Just for the record I am still using both ESXi and ESX enterprise at work, and soon going to be upgrading to Vsphere, so from time to time I will still be posting information about VMware. Now using the above chart you can clearly see lots of good reasons why you should give XenServer a try and not be too quick to choose ESXi. I will highlight some reasons for wanting to go the Xen side of things.

Reason one: Hardware

I have an HP DL 380 G4, and a DL360 G3 that I am planning on using for this project, now by default this Server‘s architecture does not natively support VMware’s new Vsphere. So instead of installing ESXi 3.5 and being locked into a product with a limited life cycle I started thinking of an alternative virtual solution hence Citrix XenServer.

Reason two: Ease of management

As stated above “centralized multi-server management”, currently at work we have several ESXi installations across the network and managing them at times can be a pain. VMware offers a solution for there paid product called “Virtual Center” however the free product does not have this function. So instead you have to log into each machine individually to apply patches or make system changes. After a while even if you have six ESXi servers this can easily become a hassle.

Reason three: Innovation

XenServer was the first solution on the market to employ paravirtualization, which allows a guest OS to be fully aware that it is being run on virtualized hardware. This collaboration between the OS and the virtualization platform enables the development of a simpler, leaner hypervisor, as well as highly optimized performance.

Reason four: System Maintenance

The VMware VI 3.5 updates site lists hundreds of patches that have been issued since the product was launched in December 2007 and the automated update process is touted by VMware as a way of managing the application of these updates. By contrast, Citrix has issued just five hotfixes for XenServer 4.0 since its release in August 2007 and includes automated update features in XenCenter for applying system updates.

Reason five: Budgetary Concerns

If you are a small to mid size business that’s now getting into virtualization your managment will want to first see how much return on invest (ROI) the can get on any solution before putting extra funding into it. With the enterprise level solution offered by Citrix you are able to do this at 3X less than the cost of using Vmware.

In conclusion I am not saying that VMware is any less superior than ZenServer  I am mainly tring to encourage others to not only lean towards the popular product in the marketplace but always try other options.

Reference links

http://www.citrix.com/site/resources/dynamic/salesdocs/XS_vs_VMware_comparison.pdf

I was recently faced with a VMware ESX issue and while researching a solution I found an useful article in the VMware knowledge base.

Issue:

I notice one of the Servers weren’t functioning correctly and after further investigation of the logs I notice this following error message “The Redo Log on Srvr01-0000001.vmdk has detected to be corrupt. The virtual machine needs to be powered down. If the problem still persists, you need to discard the redo log”. After attempting a few times to restart the guest OS and failing, I use VMware article which resolved this issues for me.

The article is well written so the only thing I would like to add is the followings steps for creating a local user account on the VMware host. This account is needed to ssh into the host and perform the steps necessary to resolve this issue. In addition Root login via SSH is disabled by default so these steps are necessary.



Steps to create a local user with shell access on a VMware ESX host:

Step One:

Launch your VMware infrastructure client and connect to the ESX server that’s hosting the problematic guest OS, if you connect to your virtual center server you wouldn’t get the option to create a local user. Login with your Root user and and password.

Step two:



Once you are logged in click on the “Users and Groups” tab, insure that you are in the “users view”, then right click a user account from the list select add and fill in requested information (username and password).

***note** Check off the box that says “Grant shell access to this user”

You can now ssh to the ESX server using your newly created credentials. Once you log in issue the “su -” command, you will be prompted for your root password enter it and you are ready to continue with the rest of the article.

Excerpt from the article:

To terminate the Master World and User Worlds for the virtual machine:

1. Run the following command to list the running virtual machines to determine the virtual machine ID for the affected virtual machine:

#cat /proc/vmware/vm/*/names

The output appears similar to:

vmid=1076 pid=-1 cfgFile=”/vmfs/volumes/50823edc-d9110dd9-8994-9ee0ad055a68/VMNAME/VMNAME.vmx” uuid=”50 28 4e 99 3d 2b 8d a0-a4 c0 87 c9 8a 60 d2 31″ displayName=”VMNAME-192.168.1.10″

Note: vmid=’1076′ is used as an example in this article.

2. Run the following command to identify the Master World ID:

# less -S /proc/vmware/vm/1076/cpu/status

Expand the terminal or scroll until you can see the right-most column labeled ‘group’. In this column you find the vm.####

In this example, ’1092′ is the ID of the Master World.

3. Run the following command to terminate the Master World and the virtual machine running in it:

/usr/lib/vmware/bin/vmkload_app -k9 1092

4. The virtual machine’s User Worlds and the virtual machine’s processes are stopped.

If the command is successful, you see output similar to:

# /usr/lib/vmware/bin/vmkload_app –kill 9 1070 Warning: Jul 12 07:24:06.303: Sending signal ’9′ to world 1070.

If the Master World ID is wrong, you see the error:

# /usr/lib/vmware/bin/vmkload_app –kill 9 1071

Warning: Jul 12 07:21:05.407: Sending signal ’9′ to world 1071.

Warning: Jul 12 07:21:05.407: Failed to forward signal 9 to cartel 1071: 0xbad0061

The virtual machine is now powered off. Power on the virtual machine. Verify that it is able to boot properly and that the message error no longer occurs.

Here is the link to the article –> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1006585.