Author: Rajesh Dave
As Virtualization penetration continues to broaden, customers are expecting to deploy and run virtual machines across a diverse set of servers running different virtualization capable processors. When a running virtual machine is moved to a server running different processor, the virtualization platform needs to provide necessary support to ensure applications running inside virtual machines continue to run on the destination processor. Windows Server
2008 R2 Hyper-V introduces a capability called processor compatibility mode to allow moving a running virtual machine across Hyper-V supported processors. This document provides details on processor compatibility mode.
1.
I have a virtual machine running on a host A and would like to be able to move it to another host B in my data center for effective load balancing on the physical servers.
2.
I have built a cluster of hosts running identical processors. One of host fails due to hardware failure, so I buy another server and add to the cluster. The newly added server is most likely has newly added processor features than other hosts in the cluster. I would like to be able to move running virtual machines freely from any node of the cluster.
3.
I have a virtual machine running on host A, I save the VM with active memory state and would like to restore the same at a later date on any Hyper-V enabled host which may not be the same host.
Most of the well-behaved applications written use the recommended method of querying processor features available through calling the CPUID instruction. In the virtualization environment, when a VM is moved to another server the software applications running inside VM are completely unaware of the change. An application may try to execute CPU features that it discovered when it was started on source host but they may not be present on the destination host. The virtualization platform needs to ensure applications continue to run successfully. This is performed by the processor compatibility check. The processor compatibility check is performed before restoring a running VM on the destination to ensure destination host has identical or superset of source host’s CPU features.
As per the compatibility check if a VM is moved from host A to host B, depending upon processor features of host
A and host B, the VM migration might fail or succeed. See below table for details.
Host A processor features
Same as host B
(features: X, Y, Z)
Superset of host B
(features: X, Y,Z)
Subset of host B
(features: X, Y)
Host B processor features
Same as host A
(features: X, Y, Z)
Subset of host A
(features: X, Y)
Superset of host A
(features: X, Y,Z)
VM migration
Host A to Host B,
Host B to Host A
Host A to Host B
Host B to Host A
Host A to Host B
Host B to Host A
Results
Succeeds
Succeeds
Fails
Succeeds
Succeeds
Fails
1 | P a g e

Different set of host B*
(features: X, Y)
Different set of host A*
(features: Y, Z)
Host A to Host B
Host B to Host A
Fails
Fails
*There are no processors today that fall into this scenario but it is possible in future when processor vendors stop shipping some rarely used features.
While processor computability check provides robust VM migration, it also reduces flexibility, so starting Windows
Server 2008 R2 RC Hyper-V introduces Processor compatibility mode capability that will allow VM migration between hosts A and B successfully.
When a Virtual Machine (VM) is started on a host, the Hypervisor exposes the set of supported processor features available on the underlying hardware of that host to the VM. These sets of processor features are called the guest visible processor features. This set of processor features is available to the VM until the VM is restarted. When a running VM is migrated to another host, Hyper-V first compares verifies processor features currently available to the VM are also available on the destination host. If the destination processor does support all of the features available to the VM, the migration will fail.
With processor compatibility mode enabled, Hyper-V only exposes the guest VM to processor features that are available across all processors of the same processor architecture, i.e. AMD-to-AMD or Intel-to-Intel. This allows the VM to be migrated to any hardware platform of the same processor architecture. Processor features are
“hidden” by the Hypervisor by intercepting a VM’s CPUID instruction and clearing the returned bits corresponding to the hidden features.
When a VM in a processor compatibility mode is started following processor features are hidden from the VM:
Host running AMD based processor Host running Intel based processor
Below figure shows pictorial example of how VM processor compatibility mode works.
2 | P a g e

Windows Server 2008 R2 Hyper-V will not allow migrating a VM (with or without in processor compatibility mode) from AMD based hosts to Intel based hosts, and vice versa.
Let us revisit migration scenario table when a VM is in processor compatibility mode:
Host A processor features
Same as host B
(features: X, Y, Z)
Superset of host B
(features: X, Y,Z)
Subset of host B
(features: X, Y)
Different set of host B
(features: X, Y)
Host B processor features
Same as host A
(features: X, Y, Z)
Subset of host A
(features: X, Y)
Superset of host A
(features: X, Y,Z)
Different set of host A
(features: Y, Z)
VM migration
Host A to Host B,
Host B to Host A
Host A to Host B
Host B to Host A
Host A to Host B
Host B to Host A
Host A to Host B
Host B to Host A
Results
Succeeds
Succeeds
Succeeds
Succeeds
Succeeds
Succeeds
Succeeds
Succeeds
Processor features available to
VM on ether host
X,Y,Z
X,Y
X,Y
Y
VM’s processor compatibility mode is OFF by default. It can be turned on by one of the two ways:
 Hyper-V Manager: from Hyper-V Manager, select CPU setting wizard of the VM. Check a check-box labeled “Migrate to physical computer with a different processor version”, see below image for details:
You can change this setting when VM is not running.
 System Center Virtual Machine Manager 2008 R2
Q: What Hyper-V features are impacted by VM processor compatibility mode?
A: Live Migration, Quick Migration and Save/Restore features of Windows Server 2008 R2 Hyper-V will benefit from VM processor compatibility mode.
Q: What are the hardware requirements for processor compatibility mode to work?
A: The CPUs must support hardware assisted virtualization (e.g. Intel’s IVT or AMD’s AMD-V)
3 | P a g e

Q: Do I need to put VM in processor compatibility mode if I want to move or restore VM that was booted on an older processor to a newer processor?
A: VMs get CPU features when it is started so as long as running VM is moved or restored on a machine that is running the same processor or processor that has superset of features move or restore will succeed without VM in processor compatibility mode. As of today, none of the processor vendors have deprecated any features in the newer processors so all newer processors provide same or superset of features.
Q: What should I consider before turning on processor compatibility mode?
A: Before turning on processor compatibility mode, please consider following:
 If moving down to older processer versions consider potential performance reduction due to loss of processor features which might accelerate applications performance. It’s recommended that you properly test these scenarios.
 An application may use non-recommended methods for processor feature detection, such as instruction probing, i.e. using an exception handler and attempting to execute a feature’s instruction to determine the presence of a feature. Or, basing feature presence on the type of processor, i.e. family, model and stepping.
 Applications may fail to launch because of the lack of a specific a processor feature
It is recommended that before deployment, any application that may run within a VM in a compatibility mode should be started, exercised and migrated on every node within the cluster.
Q: What is processor compatibility check?
A: Processor compatibility check is performed by Hyper-V when a running VM is moved from host A to host B. The compatibility check is performed to ensure a VM’s applications will continue to run successfully on the host B after the move.
Q: What are the application development requirements for taking advantage from processor compatibility mode?
A: Application developers should use CPUID instruction to detect processor features; instruction probing method is not recommended and may result in application and/or VM crash.
Q: What if I try to live migrate a VM with processor compatibility mode from a host A (Intel based) to host B (AMD based)?
A: Live migration attempt will fail since the processor compatibility check will not succeed.
Q: Can I individually select which features should be hidden from VM when put in processor compatibility mode?
A: No, once you put VM in a processor compatibility mode, Hyper-V will hide the necessary set of features from the VM to ensure a successful migration between Hyper-V enabled processors.
Q: What if I try to add a host (AMD based) to the cluster of all hosts running Intel based processors?
4 | P a g e

A: Failover cluster supports cluster of hosts with different architectures so you will be allowed to build such a cluster. However, planned downtime (live migration or quick migration) operations between hosts running different processor architecture will not succeed.
Q: What happens if in an unplanned failover of a cluster node if my cluster has some Intel based hosts and the rest are AMD based?
A: There is no impact regardless of VM’s processor compatibility mode in unplanned failover scenario. During unplanned failover VM is essentially restarted in which case VM will relearn about processor features available from the host.
5 | P a g e