ESXI VM CPU Ready



https://kb.vmware.com/s/article/2002181
https://kb.vmware.com/s/article/2001003
http://www.joshodgers.com/2013/01/05/how-much-cpu-ready-is-ok/
http://www.vmcalc.com/


vCPU to Core RatioThe number of VM vCPUs allocated compared to the number of physical CPU cores available.

If workloads are CPU-intensive, the vCPU-to-core ratio will need to be smaller; if workloads are not CPU-intensive, the vCPU-to-core ratio can be larger. A typical vCPU-to-core ratio for server workloads is about 4:1—four vCPUs allocated for each available physical core. However, this can be much higher if workloads are not CPU-intensive.


 %READY - the percentage of time that the virtual machine was ready but could not be scheduled to run on a physical CPU.

Note: The result of above calculations are a sum of each virtual CPUs %RDY time. Individual core Ready time can be roughly estimated by dividing by the number of cores. However, it is more accurate to go by the vCPU metrics themselves. vCPU metrics are only collected in realtime by default.


Performance Threshold: 
10% per vCPU based on the following guide:
Good: 1% to 5%
Moderate: 6% to 9%
Bad: 10%+


If the load average is too high, and the ready time is not caused by CPU limiting, adjust the CPU load on the host. 

To adjust the CPU load on the host, either:
1) Increase the number of physical CPUs on the host
2) Decrease the number of virtual CPUs allocated to the host. To decrease the number of virtual CPUs allocated to the host, either:
 - Reduce the total number of CPUs allocated to all of the virtual machines running on the ESX host. For more information, see Determining if multiple virtual CPUs are causing performance issues (1005362).
 - Reduce the number of virtual machines running on the host.




For Production server workloads
<2.5% CPU Ready Generally No Problem!
2.5%-5% CPU Ready Minimal contention that should be monitored during peak times
5%-10% CPU Ready Significant Contention that should be investigated & addressed
>10% CPU Ready Serious Contention to be investigated & addressed ASAP!
However, applications which are latency sensitive may be severely impacted even with low levels of CPU ready, these types of VMs should be on clusters with lower CPU overcommitment, leverage DRS rules to separate the contending workloads or in extreme cases, dedicated clusters.

On the flip side, Some servers are much more tolerant to CPU ready, and 5%-10% CPU ready or higher may not noticeably impact performance.


For Virtual Desktop workloads, what level of CPU ready is acceptable will largely depend on the individual user (ie: Power User verses Task Worker). Keep in mind virtual desktop deployments generally have high CPU consolidation ratios of  around 6:1 all the way to >12:1.
I would suggest the following , again as a rule of thumb
<5% CPU Ready Generally No Problem!
5%-10% CPU Ready Minimal contention that should be monitored during peak times
>10% CPU Ready Contention to be investigated & addressed where the end user experience is being impacted.
Ready time greater than 10 percent could be a performance concern. However, some less CPU-sensitive applications and VMs can have much higher values of ready time and still perform satisfactorily. 
EXAMPLE:
HOST Dell R710  
CPU Xeon L5520 @ 2.2GHz
CPUs 2 sock x 4 cores
CPU HT enabled yes
vCPUs 16
ESXI 6.0, 4192238

http://www.four2.org/cpubusy/cpubusy.zip


HOST CPU Ready


Calculating CPU cycles and contention




Results:


Different configurations




HOST VM1 VM2 VM3 VM4
VMs vCPU
on Host
CPU Stress VM1
cpubusy-result
VM1 CPU VM2 CPU VM3 CPU VM4 CPU CPU
usage
Summation
Ready, ms
CPU
ready %
Summation
Ready, ms
CPU
ready %
Summation
Ready, ms
CPU
ready %
Summation
Ready, ms
CPU
ready %
Summation
Ready, ms
1 16 vCPU cpubusy 0.546 12 na na na ~18 500 0.2 600 na na na na na na
1 16 vCPU cpuz+cpubusy 0.905 100 na na na ~51 1330 0.4 1400 na na na na na na
2 32 vCPU cpuz+cpubusy 0.921 100 1 na na ~54 4800 1.2 3700 5.0 1100 na na na na
3 48 vCPU cpuz+cpubusy 0.951 100 1 1 na ~55 7300 1.6 5100 4.0 1260 4.0 1260 na na
4 64 vCPU cpuz+cpubusy 0.811 100 1 1 1 68 10700 0.6 1900 0.8 2400 0.8 2415 1.5 4800
4 64 vCPU 2 cpuz+cpubusy 1.1-1.51 100 1 100 1 80 61900 8.2 26300 0.8 2500 9.9 31800 0.8 2700
4 64 vCPU 4 cpuz+cpubusy 2.4-3.1 100 100 100 100 87 128000 10.5 33500 10.8 34500 9.2 29500 10.0 32000
4 64 vCPU cpubusy 0.561 8 2 2 2 30 7100 0.9 3000 0.4 1400 0.4 1400 0.5 1660
6 96 vCPU cpubusy 0.561-0.577 8 2 2 2 20 13830 0.8 2700 0.7 2300 0.7 2300 0.7 2300


Results in Xorux