The quest for the ideal virtualization strategy

Virtualization is steadily becoming more widespread and its use is redefining the standards for datacenter deployments throughout the industry. It allows for a true separation of hardware and software, and allows us to easily clone, snapshot, migrate, and redeploy instances of our machine images.

Virtualization is a hot topic, and there are certainly plenty of options. To name a few:

Datacenter deployments:

• VMWare vSphere (ESX, ESXi, vCenter)
• Citrix XenServer
• Kernel-based Virtual Machine (KVM)
• MS Hyper-V

Cloud deployments:

• Amazon EC2
• Google App Engine
• Rackspace

Depending on your feature and security requirements, you can narrow this list. Consider the following table:

	VMWare	Xen	KVM	Hyper-V	EC2	AppEngine
Own Hardware	√	√	√	√	X	X
Commercial Support	$	$	$	$	$	$
Paravirtualization	√	√	√	√	n/a	n/a
Live Migrations	$	?	?	?	?	?
Live Snapshots	$	√	√	√	?	?
HA / Failover	$	?	?	?	√	√
Centralized Management	$	$	?	$	√	√
Power Mgmt (iLOM)	$	?	?	?	n/a	n/a

There are many more features to list here, and I clearly have some research to do. I will try to revise this list often. The point to take away from this though, is that the more enterprise-class features you need, the more you spend. And the more likely you are to end up with a mature product like VMWare. But that doesn't mean that all options shouldn't be considered. When designing a solution, the most cost-effective option is going to be the winner.

Here is a pretty nice feature comparison of VMWare vs Citrix XenServer, but since it is a Citrix document, it is biased of course.

memory de-duplication with KSM for SLES 11 KVM guests

I have recently been working in a SLES shop, virtualizing multiple similar SLES guests via KVM. After doing some reading on memory deduplication, I found a new feature in Linux kernel 2.6.32 and later called KSM. Some documentation refers to this as Kernel Shared Memory. Others call it Kernel SamePage Merging. 

Enabling this feature will allow for KVM guests to share physical memory on the host device so that anything that is normally duplicated in RAM for all of your similar VMs, is condensed and shared.

This, along with other techniques such as using kvm base images, can allow for a higher VM/Hardware ratio.

To see if the kernel has the feature enabled, use the following command:

# grep KSM /boot/config-`uname -r`

You should get the following result:

CONFIG_KSM=y

Once I found that my kernel indeed had this feature turned on, I was curious if I was already using it without knowing. The following commands all returned 0 however, indicating that my qemu-kvm was not using the feature.

# cat /sys/kernel/mm/ksm/pages_sharing

0

# cat /sys/kernel/mm/ksm/pages_shared

0

# cat /sys/kernel/mm/ksm/pages_unshared

0

Additional reading indicated that I might need to recompile my qemu-kvm to enable the feature. After looking closely at the version shipped with my distro though, I began to doubt this. A little more time googling for answers and a beer or two later, I found an article about KSM on debian containing this little gem:

# echo 1 > /sys/kernel/mm/ksm/run

Initially, this didnt seem to take effect. However, after 30 minutes to an hour, I began to see results.

# cat /sys/kernel/mm/ksm/pages_sharing

1004274

# cat /sys/kernel/mm/ksm/pages_shared

191867

# cat /sys/kernel/mm/ksm/pages_unshared

1281385

I made this permanent by adding it to the end of my /etc/init.d/boot.local.

One blade with 16G ram, running 7 VMs, went from 90MB free RAM to 4447MB free RAM. Other hardware had similar results. Pretty awesome!

It seems Redhat and Fedora implement KSM differently, with a daemon process called ksmd and another for tuning. Any comments on their implementation would be appreciated.

~Ian