memory de-duplication with KSM for SLES 11 KVM guests

I have recently been working in a SLES shop, virtualizing multiple similar SLES guests via KVM. After doing some reading on memory deduplication, I found a new feature in Linux kernel 2.6.32 and later called KSM. Some documentation refers to this as Kernel Shared Memory. Others call it Kernel SamePage Merging. 

Enabling this feature will allow for KVM guests to share physical memory on the host device so that anything that is normally duplicated in RAM for all of your similar VMs, is condensed and shared.

This, along with other techniques such as using kvm base images, can allow for a higher VM/Hardware ratio.

To see if the kernel has the feature enabled, use the following command:

# grep KSM /boot/config-`uname -r`

You should get the following result:

CONFIG_KSM=y

Once I found that my kernel indeed had this feature turned on, I was curious if I was already using it without knowing. The following commands all returned 0 however, indicating that my qemu-kvm was not using the feature.

# cat /sys/kernel/mm/ksm/pages_sharing

0

# cat /sys/kernel/mm/ksm/pages_shared

0

# cat /sys/kernel/mm/ksm/pages_unshared

0

Additional reading indicated that I might need to recompile my qemu-kvm to enable the feature. After looking closely at the version shipped with my distro though, I began to doubt this. A little more time googling for answers and a beer or two later, I found an article about KSM on debian containing this little gem:

# echo 1 > /sys/kernel/mm/ksm/run

Initially, this didnt seem to take effect. However, after 30 minutes to an hour, I began to see results.

# cat /sys/kernel/mm/ksm/pages_sharing

1004274

# cat /sys/kernel/mm/ksm/pages_shared

191867

# cat /sys/kernel/mm/ksm/pages_unshared

1281385

I made this permanent by adding it to the end of my /etc/init.d/boot.local.

One blade with 16G ram, running 7 VMs, went from 90MB free RAM to 4447MB free RAM. Other hardware had similar results. Pretty awesome!

It seems Redhat and Fedora implement KSM differently, with a daemon process called ksmd and another for tuning. Any comments on their implementation would be appreciated.

~Ian