Skip to content

VMware ESX CPU/MMU Virtualization Settings

October 8, 2012

ESX can determine whether a virtual machine should use hardware support for virtualization.  It makes this determination based on the physical processor type of the ESX host and the virtual machine.  To determine the Virtual Machine Monitor (VMM) mode, view the Properties of your virtual machine.

CPU_MMU_Virtualization

Overriding the default (Automatic) can provide better performance for some use cases, but what exactly are those use cases?  And more importantly, what are potential side effects to making a change?

Simply stated, hardware assistance can eliminate the overhead required to keep shadow page tables in synchronization with guest page tables.  But the technology we need to eliminate the memory overhead wasn’t available until 2009.  Learn more about shadow page tables in VMware Labs Memory Virtualization Presentation.

A Brief History

Prior to 2005 CPUs did not have support for any hardware assisted virtualization.

In 2005 Intel introduced their first generation hardware assisted virtualization technology known as Intel-VT in some Pentium 4 CPUs while AMD released support in 2006 with the introduction of the Athlon 64 series CPU.  This allows the Virtual Machine Monitor to make use of the hardware features for instruction-set virtualization. However, Memory Management Unit MMU virtualization was still done in software.

Keith Adams and Ole Agesen of VMware stated, “Surprisingly, we find that the first-generation hardware support rarely offers performance advantages over existing software techniques” in A Comparison of Software and Hardware Techniques for x86 Virtualization.

Fast forward to 2009, when Intel introduced its second level address translation in the Nehalem architecture known as Extended Page Table (EPT), and AMD followed with their third-generation Opteron processors and is offering Rapid Virtualization Indexing (RVI).  These technologies provide support for Memory Management Unit MMU virtualization.

For more information on Intel EPT I highly recommend reading Bhaskar Gowda’s informative bLOG post A peek into Extended Page Tables.

VMware Performance Testing reveals that significant gains in performance are achieved by leveraging these technologies.  See Performance Evaluation of AMD RVI Assist and Performance Evaluation of Intel EPT Hardware Assist.

Recommendations

Chances are you are probably already leveraging this technology, so the real question becomes when should I disable it?

First consider the fact that modifying the virtual machine monitor mode to support hardware MMU will force the guest operating system to use Large Memory Pages.  This eliminates the opportunity to share memory via Transparent Page Sharing.  For those of you using Windows XP in a VDI deployment this may reduce the overall number of virtual machines you can support per ESXi host.

The other key determining factor is whether or not the application(s) you are running are prone to TLB misses.  See VMware vSphere 5.1 Documentation Center for the definition of TLB, “The TLB (translation look-aside buffer) is a cache of translations maintained by the processor’s memory management unit (MMU) hardware. A TLB miss is a miss in this cache and the hardware needs to go to memory (possibly many times) to find the required translation. For a TLB miss to a certain guest virtual address, the hardware looks at both page tables to translate guest virtual address to host physical address.”

Applications that are sensitive to the latency of a TLB miss such as some Java based applications may benefit by disabling support for hardware MMU.  However you will have to determine which performance cost is higher, memory virtualization overhead or the cost of a TLB miss.

How do I modify CPU/MMU settings?

VMware provides Step-by-Step instructions for modifying your CPU/MMU settings in VMware KB: Changing the virtual machine monitor mode (1036775)

Alan Renouf posted the bLOG article Setting MMU for your VMs with a few PowerShell scripts that can help you make bulk changes.

Advertisements
No comments yet

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: