tag:blogger.com,1999:blog-49591522502943326062024-02-08T11:42:03.662-05:00Virtual DreamscapesDiscussions and notes from personal technical projects. <br>
Web design, scripting, high-availability, VMWare, Xen, KVM, cloud deployments, etc.Ian Pattonhttp://www.blogger.com/profile/07984763833112715564noreply@blogger.comBlogger2125tag:blogger.com,1999:blog-4959152250294332606.post-83621794097000574422011-05-08T14:43:00.001-05:002011-05-08T15:53:32.127-05:00The quest for the ideal virtualization strategyVirtualization is steadily becoming more widespread and its use is redefining the standards for datacenter deployments throughout the industry. It allows for a true separation of hardware and software, and allows us to easily clone, snapshot, migrate, and redeploy instances of our machine images.<br />
<br />
Virtualization is a hot topic, and there are certainly plenty of options. To name a few:<br />
<br />
Datacenter deployments:<br />
<ul><li>VMWare vSphere (ESX, ESXi, vCenter)</li>
<li>Citrix XenServer</li>
<li>Kernel-based Virtual Machine (KVM)</li>
<li>MS Hyper-V</li>
</ul><div>Cloud deployments:</div><ul><li>Amazon EC2</li>
<li>Google App Engine</li>
<li>Rackspace</li>
</ul><div><br />
</div><div>Depending on your feature and security requirements, you can narrow this list. Consider the following table:<br />
<br />
</div><div><br />
<table style="text-align: center; width: 100%;"><tbody>
<tr> <th></th> <th>VMWare</th> <th>Xen</th> <th>KVM</th> <th>Hyper-V</th> <th>EC2</th> <th>AppEngine</th> </tr>
<tr> <td><div style="text-align: left;">Own Hardware</div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">√</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">√</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">√</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">√</span></div></td><td><div style="text-align: center;">X</div></td><td><div style="text-align: center;">X</div></td> </tr>
<tr><td><div style="text-align: left;">Commercial Support</div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">$</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">$</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">$</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">$</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">$</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">$</span></div></td> </tr>
<tr><td><div style="text-align: left;">Paravirtualization</div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">√</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">√</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">√</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">√</span></div></td><td><div style="text-align: center;">n/a</div></td><td><div style="text-align: center;">n/a</div></td> </tr>
<tr><td><div style="text-align: left;">Live Migrations</div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">$</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">?</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">?</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">?</span></div></td><td><div style="text-align: center;">?</div></td><td><div style="text-align: center;">?</div></td> </tr>
<tr><td><div style="text-align: left;">Live Snapshots</div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">$</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">√</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">√</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">√</span></div></td><td><div style="text-align: center;">?</div></td><td><div style="text-align: center;">?</div></td> </tr>
<tr><td><div style="text-align: left;">HA / Failover</div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">$</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">?</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">?</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">?</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">√</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">√</span></div></td> </tr>
<tr><td><div style="text-align: left;">Centralized Management</div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">$</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">$</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">?</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">$</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">√</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">√</span></div></td> </tr>
<tr><td><div style="text-align: left;">Power Mgmt (iLOM)</div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">$</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">?</span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="font-family: Arial;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px;">?</span></span></div></td><td><div style="text-align: center;"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; font-family: Arial;">?</span></div></td><td><div style="text-align: center;">n/a</div></td><td><div style="text-align: center;">n/a</div></td> </tr>
</tbody></table><br />
<div>There are many more features to list here, and I clearly have some research to do. I will try to revise this list often. The point to take away from this though, is that the more enterprise-class features you need, the more you spend. And the more likely you are to end up with a mature product like VMWare. But that doesn't mean that all options shouldn't be considered. When designing a solution, the most cost-effective option is going to be the winner.<br />
<br />
Here is a pretty nice feature comparison of <a href="http://resources.idgenterprise.com/original/AST-0036435_Technicalandcommercialcomparison_Citrix_XenServer_VMware.pdf">VMWare vs Citrix XenServer</a>, but since it is a Citrix document, it is biased of course.</div><br />
</div>Ian Pattonhttp://www.blogger.com/profile/07984763833112715564noreply@blogger.com10tag:blogger.com,1999:blog-4959152250294332606.post-7773461147804074932011-05-08T13:02:00.001-05:002011-05-08T13:10:05.027-05:00memory de-duplication with KSM for SLES 11 KVM guests<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">I have recently been working in a SLES shop, virtualizing multiple similar SLES guests via KVM. After doing some reading on memory deduplication, I found a new feature in Linux kernel 2.6.32 and later called KSM. Some documentation refers to this as <a href="http://www.ibm.com/developerworks/linux/library/l-kernel-shared-memory/index.html">Kernel Shared Memory</a>. Others call it <a href="http://www.linux-kvm.com/content/using-ksm-kernel-samepage-merging-kvm">Kernel SamePage Merging</a>. </div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
</div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">Enabling this feature will allow for KVM guests to share physical memory on the host device so that anything that is normally duplicated in RAM for all of your similar VMs, is condensed and shared.</div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
</div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">This, along with other techniques such as using kvm base images, can allow for a higher VM/Hardware ratio.</div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
</div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">To see if the kernel has the feature enabled, use the following command:</div></div><div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"># grep KSM /boot/config-`uname -r`</span></div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
</div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">You should get the following result:</div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">CONFIG_KSM=y</span></div></div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
</div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">Once I found that my kernel indeed had this feature turned on, I was curious if I was already using it without knowing. The following commands all returned 0 however, indicating that my qemu-kvm was not using the feature.</div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
</div></div><div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"># cat /sys/kernel/mm/ksm/pages_sharing</span></div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">0</span></div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"># cat /sys/kernel/mm/ksm/pages_shared</span></div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">0</span></div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"># cat /sys/kernel/mm/ksm/pages_unshared</span></div></div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">0</span></div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
</div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">Additional reading indicated that I might need to recompile my qemu-kvm to enable the feature. After looking closely at the version shipped with my distro though, I began to doubt this. A little more time googling for answers and a beer or two later, I found an article about KSM on debian containing this little gem:</div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
</div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"># echo 1 > /sys/kerne<wbr></wbr>l/mm/ksm/r<wbr></wbr>un</span></div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
</div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">Initially, this didnt seem to take effect. However, after 30 minutes to an hour, I began to see results.</div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
</div></div><div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"># cat /sys/kernel/mm/ksm/pages_sharing</span></div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">1004274</span></div></div></div><div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"># cat /sys/kernel/mm/ksm/pages_shared</span></div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">191867</span></div></div></div><div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"># cat /sys/kernel/mm/ksm/pages_unshared</span></div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">1281385</span></div></div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
</div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">I made this permanent by adding it to the end of my /etc/init.d/boot.local.</div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
</div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">One blade with 16G ram, running 7 VMs, went from 90MB free RAM to 4447MB free RAM. Other hardware had similar results. Pretty awesome!</div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
</div></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">It seems Redhat and Fedora implement KSM differently, with a daemon process called ksmd and another for tuning. Any comments on their implementation would be appreciated.</div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
</div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><i>~Ian</i></div></div>Ian Pattonhttp://www.blogger.com/profile/07984763833112715564noreply@blogger.com0