VMWare vSphere ESXi Management with FreeBSD guests

VMWare troubleshoot performance
http://vmind.ru/2013/07/11/performance-troubleshooting-vmware-vsphere/

Install Vmware vSphere
http://www.derekseaman.com/vsphere-5-1-resources

ESXi host enable SNMP
esxcli system snmp set --communities xxxx
esxcli system snmp set --enable true
esxcli network firewall ruleset set --ruleset-id snmp --allowed-all true
esxcli network firewall ruleset set --ruleset-id snmp --enabled true
/etc/init.d/snmpd restart

ESXi SSH commands

~ # esxcfg-route -l
VMkernel Routes:
Network          Netmask          Gateway          Interface
10.1.63.0      255.255.255.0    Local Subnet     vmk0
10.2.200.0     255.255.255.0    10.1.63.200    vmk0
192.168.151.0    255.255.255.0    192.168.155.1    vmk2
192.168.152.0    255.255.255.0    192.168.156.1    vmk1
192.168.155.0    255.255.255.0    Local Subnet     vmk2
192.168.156.0    255.255.255.0    Local Subnet     vmk1
default          0.0.0.0          10.1.63.1      vmk0

~ # esxcli network ip interface ipv4 get
Name IPv4 Address     IPv4 Netmask   IPv4 Broadcast   Address Type DHCP DNS
---- --------------- ------------- --------------- ------------ --------
vmk0 10.1.63.100    255.255.255.0 10.1.63.255    STATIC           false
vmk2 192.168.155.100 255.255.255.0 192.168.155.255 STATIC           false
vmk1 192.168.156.100 255.255.255.0 192.168.156.255 STATIC           false
~ #

~ # esxcli network ip interface ipv4 get
Name IPv4 Address    IPv4 Netmask   IPv4 Broadcast   Address Type DHCP DNS
---- -------------- ------------- --------------- ------------ --------
vmk0 10.1.63.50    255.255.255.0 10.1.63.255    STATIC           false
vmk1 192.168.155.50 255.255.255.0 192.168.155.255 STATIC           false
vmk2 192.168.156.50 255.255.255.0 192.168.156.255 STATIC           false
~ #

~ # esxcfg-route -l
VMkernel Routes:
Network          Netmask          Gateway          Interface
10.1.63.0      255.255.255.0    Local Subnet     vmk0
10.2.200.0     255.255.255.0    10.1.63.200    vmk0
192.168.151.0    255.255.255.0    192.168.155.1    vmk1
192.168.152.0    255.255.255.0    192.168.156.1    vmk2
192.168.155.0    255.255.255.0    Local Subnet     vmk1
192.168.156.0    255.255.255.0    Local Subnet     vmk2
default          0.0.0.0          10.1.63.1      vmk0
~ #

0) VMWare ESXi Utilities
- I/O Analyzer    https://labs.vmware.com/flings/io-analyzer
- I/O Benchmark    http://greg.porter.name/wiki/HowTo:iometer
- IOPs info         http://flashdba.com/2013/06/27/storage-myths-iops-matter/

- RVTools    http://www.robware.net/

VMware Tools for FreeBSD Guests
https://www.vmware.com/support/ws55/doc/ws_newguest_tools_freebsd.html

1) Install VMWare Tools on FreeBSD 9.0 host
Before you install VMWare tools on FreeBSD 9, you need the modules to work with FreeBSD 9.
VMWare is slow to update the vmware tools for the FreeBSD guest. So you are just going to have to update them yourself.

In vSphere Client, choose VM | Install VMWare Tools.

freebsd-update fetch install
freebsd-update install
portsnap fetch extract
cd /usr/ports/misc/compat6x/; make config-recursive install clean
mkdir /cdrom; mount -t cd9660 /dev/cd0 /cdrom
cp /cdrom/vmware-freebsd-tools.tar.gz /tmp ; cd /tmp ; tar -xzf vmware-freebsd-tools.tar.gz

echo 'MK_CLANG_IS_CC=no' >> /etc/make.conf#not needed for ESXi 5.1

cd /tmp/vmware-tools-distrib/lib/modules/source/ ; tar -xf vmmemctl.tar; cd vmmemctl-only; make; make install
cd /tmp/vmware-tools-distrib/lib/modules/source/ ; tar -xf vmblock.tar; cd vmblock-only; make; make install

/tmp/vmware-tools-distrib/vmware-install.pl -d
# with "-d" accepts all defaults

2) Install VMware Tools on FreeBSD 9.0 with VMXNET/VMXNET3 network drivers

Select VM: Guest->Install/Upgrade VMware Tools

cd /tmp; fetch http://ogris.de/vmware/vmxnet.diff && fetch http://ogris.de/vmware/vmxnet3.diff ;
mount -t cd9660 /dev/cd0 /mnt && tar xzf /mnt/vmware-freebsd-tools.tar.gz ;
cd vmware-tools-distrib/lib/modules/source/ ;
tar -xvf vmblock.tar && tar -xvf vmmemctl.tar && tar -xvf vmxnet.tar && tar -xvf vmxnet3.tar ;
cd vmblock-only && make && make install ;
cd ../vmmemctl-only && make && make install ;
cd ../vmxnet-only && patch -p1 < /tmp/vmxnet.diff && make && make install ;
cd ../vmxnet3-only && patch -p1 < /tmp/vmxnet3.diff && make && make install ;
cd ../../../.. && perl ./vmware-install.pl ;

Press ENTER as many time is needed.
- Update /etc/rc.conf, e.g. change ifconfig_em0 to ifconfig_vmx3f0 (sic!) in case of vmxnet 3
- Update Firewall rules (em0 -> vmx3f0)
- Shutdown the FreeBSD VM with [ poweroff ] command
- Change network interfaces @ VM
- Start VM is it should be with new network interfaces.

Mirrors to Diff files:
- vmxnet.diff http://pastebin.com/raw.php?i=XQpdsYMD
- vmxnet3.diff http://pastebin.com/raw.php?i=LM3knk0H

Problem
As of May 2012, VMware Tools that come with ESXi 5 don't work on FreeBSD 9. I found this article in which Jared Barneck reports that he just had to recompile vmmemctl and vmblock. The source code for both of these modules (and vmxnet/vmxnet3 as well) is included in vmware-freebsd-tools.tar.gz. However, you can compile neither vmxnet nor vmxnet3 without some minor modifications.
Make sure that you have at least the kernel source tree in /usr/src/sys.
Solution

Download vmxnet.diff and vmxnet3.diff
In the vSphere Client, select your FreeBSD virtual machine and (right-)click Guest->Install/Upgrade VMware Tools
In your FreeBSD machine, run mount -t cd9660 /dev/cd0 /mnt and tar xzf /mnt/vmware-freebsd-tools.tar.gz
    cd vmware-tools-distrib/lib/modules/source/
Untar vmblock.tar, vmmemctl.tar, vmxnet.tar, and vmxnet3.tar
Now apply my patches:
        cd vmxnet-only
        patch -p1 < /tmp/vmxnet.diff
        cd ../vmxnet3-only
        patch -p0 < /tmp/vmxnet3.diff
Change into vmblock-only and run make && make install
Change into vmmemctl-only and run make && make install
Change into vmxnet-only and run make && make install
Change into vmxnet3-only and run make && make install
Switch back to the top-level directory (vmware-tools-distrib) and run vmware-install.pl as usual
Update /etc/rc.conf, e.g. change ifconfig_em0 to ifconfig_vmx3f0 (sic!) in case of vmxnet 3
http://ogris.de/vmware/

2.a) Install VMware Tools on FreeBSD 10.0 REL (working solution)
First of all

cd /usr/ports/misc/compat6x
make install clean
ln -s /usr/local/bin/perl /usr/bin/perl

Info http://ogris.de/vmware/freebsd10.html

In case of Error

make: "/usr/share/mk/bsd.kmod.mk" line 12: Unable to locate the kernel source tree. Set SYSDIR to override.
rm -rf /usr/src
svnlite checkout https://svn0.us-west.freebsd.org/base/releng/10.1 /usr/src

cd /tmp/vmware-tools-distrib/vmware-tools-distrib
./vmware-install.pl -d

2.a) Install VMware Tools on FreeBSD 10.0 REL

- No GCC in the default install of FreeBSD 10 - It's CLANG only and GCC from ports cannot be used to build the modules.
- The vmx(4) driver has been added. vmx(4) is a VMware VMXNET3 ethernet driver ported from OpenBSD. [r254738]
http://www.freebsd.org/releases/10.0R/relnotes.html

# cd /tmp/vmware-tools-distrib/lib/modules/source/ ; tar -xf vmmemctl.tar; cd vmmemctl-only; make; make install
os.c:267:33: error: implicit declaration of function 'kmem_alloc' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
   p->bitmap = (unsigned long *)kmem_alloc(kernel_map, p->size);
                                ^
os.c:267:33: note: did you mean 'kmem_malloc'?
   p->bitmap = (unsigned long *)kmem_alloc(kernel_map, p->size);
                                ^~~~~~~~~~
                                kmem_malloc
@/vm/vm_extern.h:58:13: note: 'kmem_malloc' declared here
vm_offset_t kmem_malloc(struct vmem *, vm_size_t size, int flags);
            ^
os.c:267:16: error: cast to 'unsigned long *' from smaller integer type 'int' [-Werror,-Wint-to-pointer-cast]
   p->bitmap = (unsigned long *)kmem_alloc(kernel_map, p->size);
               ^
os.c:274:14: error: incompatible pointer types passing 'vm_map_t' (aka 'struct vm_map *') to parameter of type 'struct vmem *'
      [-Werror,-Wincompatible-pointer-types]
   kmem_free(kernel_map, (vm_offset_t)p->bitmap, p->size);
             ^~~~~~~~~~
@/vm/vm_extern.h:59:29: note: passing argument to parameter here
void kmem_free(struct vmem *, vm_offset_t, vm_size_t);
                            ^
3 errors generated.
*** Error code 1

Stop.
make: stopped in /tmp/vmware-tools-distrib/lib/modules/source/vmmemctl-only
install: ../vmmemctl.ko: No such file or directory
*** Error code 71

Stop.
make: stopped in /tmp/vmware-tools-distrib/lib/modules/source/vmmemctl-only

Install GCC http://www.freebsd.org/doc/en/articles/custom-gcc/article.html
cd /usr/ports/devel/binutils &&make config-recursive install clean
cd /usr/ports/lang/gcc && make config-recursive install clean

$ cat make.conf
# added by use.perl 2012-06-07 03:03:21
PERL_VERSION=5.10.1
.if !empty(.CURDIR:M/usr/ports/*) && exists(/usr/local/bin/gcc46)
CC=gcc46
USE_GCC=4.6+
CXX=g++46
CPP=cpp46
FC=gfortran46
.endif

FFLAGS=-O2 -mtune=athlon64
CC=gcc46
CXX=g++46
CPP=cpp46
FC=gfortran46

3) Enable SNMP on ESXi 5.0 hosts
1) Enable temporarly SSH on ESXi host

2) Login to ESXi host through SSH

vi /etc/vmware/snmp.xml

press i to enter Insert mode
enable -> true
communities -> mycommunity

<config><snmpSettings><enable>true</enable><communities>mycommunity</communities><targets></targets></snmpSettings></config>..

Press ESC to exit insert mode
:wq to save and exit

3) Restart services

/etc/vmware # /sbin/services.sh restart

4) Check SNMP

# snmpwalk -v 2c -c mycommunity 10.10.10.1
SNMPv2-MIB::sysDescr.0 = STRING: VMware ESX 5.0.0 build-1117897 VMware, Inc. x86_64
SNMPv2-MIB::sysObjectID.0 = OID: VMWARE-PRODUCTS-MIB::vmwESX
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (262003) 0:43:40.03
SNMPv2-MIB::sysContact.0 = STRING: not set
SNMPv2-MIB::sysName.0 = STRING: localhost
SNMPv2-MIB::sysLocation.0 = STRING: not set
SNMPv2-MIB::sysServices.0 = INTEGER: 72

5) Do not forget to disable SSH at ESXi host

4) Enable additional network card drivers on ESXi 5.0 hosts
1) Enable ESXi shell
2) Check network drivers

esxcfg-nics -l
Name PCI Driver Link Speed Duplex MAC Address MTU Description
vmnic0 0000:03:00.00 r8168 Up 100Mbps Full 00:19:99:aa:ec:ab 1500 Realtek Realtek 8168 Gigabit Ethernet

lspci -v | grep Ethernet
000:003:00.0 Ethernet controller Network controller: Realtek Realtek 8168 Gigabit Ethernet [vmnic0]
000:004:07.0 Ethernet controller Network controller: Intel Corporation 82558B PRO/100+ PCI (TP)

http://pibytes.wordpress.com/2013/02/17/vmware-esxi-network-driver-install-and-upgrade/

3) Download driver from
https://my.vmware.com/group/vmware/details?downloadGroup=DT-ESXI5X-INTEL-IGB-5051&productId=229
4) Enter maintenance mode
5) Unzip downloaded file, scp to /tmp on ESXi host
6) Install driver

# esxcli software vib install -v /tmp/igb-5.0.5.1-1290180/Intel_bootbank_net-igb_5.0.5.1-1OEM.500.0.0.472560.vib
Installation Result
   Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
   Reboot Required: true
   VIBs Installed: Intel_bootbank_net-igb_5.0.5.1-1OEM.500.0.0.472560
   VIBs Removed: VMware_bootbank_net-igb_2.1.11.1-3vmw.500.2.26.914586
   VIBs Skipped:
~ #

7) Exit from maintenance mode
8) Reboot

5) Install VMware Tools on RHEL 6 & Centos 6
ESXi 5.1 (v. 1065491)
CentOS 6.3 64bits

yum install make gcc kernel-devel kernel-headers glibc-headers perl
! mount VMWareTools CD in VM
mkdir /mnt/cdrom
mount /dev/cdrom /mnt/cdrom
cp /mnt/cdrom/VMwareTools-9.0.5-1065307.tar.gz /tmp/
cd /tmp/
tar xvfz VMwareTools-9.0.5-1065307.tar.gz
/tmp/vmware-tools-distrib/vmware-install.pl -d
! add -d to auto accept defaults

- Install VMware Tools on SUSE SLED 11.0
ESXi 5.1 (v. 1065491)

! mount VMWareTools CD in VM
mkdir /mnt/cdrom
mount /dev/cdrom1 /mnt/cdrom
cp /mnt/cdrom/VMwareTools-9.0.15-2323214.tar.gz /tmp/
cd /tmp/
tar xvfz VMwareTools-9.0.15-2323214.tar.gz
/tmp/vmware-tools-distrib/vmware-install.pl -d
! add -d to auto accept defaults

- How to install VMWare tools on SUSE Linux Enterprise Server 11

1 Launch YaST | Software | Software Management
2 Change Filter to Patterns, and make sure c++ compiler is installed.
If the c++ compiler pattern is not available, add the SLES DVD/ISO as a repository
http://www.novell.com/support/kb/doc.php?id=7014489
If the c++ pattern is not installed, select it in the list of available patterns and install c++ compiler
3 Right-click the VMWare guest in the VMWare client, and click Install VMWare tools
This may also be under a Guest tab after right-clicking on the guest VMWare
4 Close YaST
5 mount /dev/cdrom /media; cp /media/*.tar.gz /tmp
6 cd /tmp; tar -zxvf VM*.tar.gz; perl /tmp/vmware-tools-distrib/vmware-install.pl -d
7 init 6 to restart the server
8 /etc/init.d/vmware-tools status to make sure it is running

6) P2V Converting Physical Machine to ESXi VM
SOFTWARE USED:
- FileZilla Server 0.9.42 beta (FTP SERVER) (with other FTP server appears errors like E0004000a)
- Acronis True Image 11 Home (doesnt work with newer versions info )
- VMware vCenter Converter Standalone 5.1.0

1) Acronis True Image is LiveCD which will transfer using network (FTP) disk clone of physical machine creating *.tib files
2) FTP Server will receive all *.tib files (usually backup is splitted by 2GB files)
3) VMware vCenter Converter Standalone will convert and migrate VM directly to vSphere Infrastructure to ESXi host.
4) After VM is migrated, OS (is this case FreeBSD) will not load because mount point are set to IDE HDD, but ESXi is set to emulate SCSI drive.
To correct that:

mountroot> ufs:/dev/da0s1a
Trying to mount root from ufs:/dev/da0s1a
.... will mount /
.... also will appear info about all mount point
....
# mount -o rw /dev/da0s1a /
# mount -o rw /dev/da0s1f /usr
# mount -o rw /dev/da0s1d /var
# vi /etc/fstab
... correct fstab file
# reboot

8) Connect iSCSI datastore

iSCSI connection will be used over existing Ether-channel ethernet ports (managemendt and data).

1) Configure Storage (in our case is IBM V3700) to accept new host and map some Volume to the host.