Pages

VMWare vSphere ESXi Management with FreeBSD guests

VMWare troubleshoot performance
http://vmind.ru/2013/07/11/performance-troubleshooting-vmware-vsphere/

Install Vmware vSphere
http://www.derekseaman.com/vsphere-5-1-resources




ESXi host enable SNMP
 esxcli system snmp set --communities xxxx
 esxcli system snmp set --enable true
 esxcli network firewall ruleset set --ruleset-id snmp --allowed-all true
 esxcli network firewall ruleset set --ruleset-id snmp --enabled true
 /etc/init.d/snmpd restart



ESXi SSH commands
~ # esxcfg-route -l         
VMkernel Routes:
Network          Netmask          Gateway          Interface     
10.1.63.0      255.255.255.0    Local Subnet     vmk0          
10.2.200.0     255.255.255.0    10.1.63.200    vmk0          
192.168.151.0    255.255.255.0    192.168.155.1    vmk2          
192.168.152.0    255.255.255.0    192.168.156.1    vmk1          
192.168.155.0    255.255.255.0    Local Subnet     vmk2          
192.168.156.0    255.255.255.0    Local Subnet     vmk1          
default          0.0.0.0          10.1.63.1      vmk0        


~ # esxcli network ip interface  ipv4 get
Name  IPv4 Address     IPv4 Netmask   IPv4 Broadcast   Address Type  DHCP DNS
----  ---------------  -------------  ---------------  ------------  --------
vmk0  10.1.63.100    255.255.255.0  10.1.63.255    STATIC           false
vmk2  192.168.155.100  255.255.255.0  192.168.155.255  STATIC           false
vmk1  192.168.156.100  255.255.255.0  192.168.156.255  STATIC           false
~ #

~ # esxcli network ip interface  ipv4 get
Name  IPv4 Address    IPv4 Netmask   IPv4 Broadcast   Address Type  DHCP DNS
----  --------------  -------------  ---------------  ------------  --------
vmk0  10.1.63.50    255.255.255.0  10.1.63.255    STATIC           false
vmk1  192.168.155.50  255.255.255.0  192.168.155.255  STATIC           false
vmk2  192.168.156.50  255.255.255.0  192.168.156.255  STATIC           false
~ #


~ #  esxcfg-route -l 
VMkernel Routes:
Network          Netmask          Gateway          Interface     
10.1.63.0      255.255.255.0    Local Subnet     vmk0          
10.2.200.0     255.255.255.0    10.1.63.200    vmk0          
192.168.151.0    255.255.255.0    192.168.155.1    vmk1          
192.168.152.0    255.255.255.0    192.168.156.1    vmk2          
192.168.155.0    255.255.255.0    Local Subnet     vmk1          
192.168.156.0    255.255.255.0    Local Subnet     vmk2          
default          0.0.0.0          10.1.63.1      vmk0          
~ #





0) VMWare ESXi Utilities
 - I/O Analyzer        https://labs.vmware.com/flings/io-analyzer
 - I/O Benchmark    http://greg.porter.name/wiki/HowTo:iometer
 - IOPs info             http://flashdba.com/2013/06/27/storage-myths-iops-matter/

 - RVTools              http://www.robware.net/

VMware Tools for FreeBSD Guests
https://www.vmware.com/support/ws55/doc/ws_newguest_tools_freebsd.html

1) Install VMWare Tools on FreeBSD 9.0 host
Before you install VMWare tools on FreeBSD 9, you need the modules to work with FreeBSD 9.
VMWare is slow to update the vmware tools for the FreeBSD guest. So you are just going to have to update them yourself.

In vSphere Client, choose VM | Install VMWare Tools.
freebsd-update fetch install
freebsd-update install
portsnap fetch extract
cd /usr/ports/misc/compat6x/; make config-recursive install clean
mkdir /cdrom; mount -t cd9660 /dev/cd0 /cdrom
cp /cdrom/vmware-freebsd-tools.tar.gz /tmp ; cd /tmp ; tar -xzf vmware-freebsd-tools.tar.gz

echo 'MK_CLANG_IS_CC=no' >> /etc/make.conf
#not needed for ESXi 5.1

cd /tmp/vmware-tools-distrib/lib/modules/source/ ; tar -xf vmmemctl.tar; cd vmmemctl-only; make;  make install
cd /tmp/vmware-tools-distrib/lib/modules/source/ ; tar -xf vmblock.tar;  cd vmblock-only; make;  make install

/tmp/vmware-tools-distrib/vmware-install.pl -d
# with "-d" accepts all defaults

2) Install VMware Tools on FreeBSD 9.0 with VMXNET/VMXNET3 network drivers

Select VM: Guest->Install/Upgrade VMware Tools
cd /tmp; fetch http://ogris.de/vmware/vmxnet.diff && fetch http://ogris.de/vmware/vmxnet3.diff ;
mount -t cd9660 /dev/cd0 /mnt && tar xzf /mnt/vmware-freebsd-tools.tar.gz ;
cd vmware-tools-distrib/lib/modules/source/ ;
tar -xvf vmblock.tar && tar -xvf vmmemctl.tar && tar -xvf vmxnet.tar && tar -xvf vmxnet3.tar ;
cd vmblock-only && make && make install ;
cd ../vmmemctl-only && make && make install ;
cd ../vmxnet-only  && patch -p1 < /tmp/vmxnet.diff  && make && make install ;
cd ../vmxnet3-only && patch -p1 < /tmp/vmxnet3.diff && make && make install ;
cd ../../../.. && perl ./vmware-install.pl ;
Press ENTER as many time is needed.
 - Update /etc/rc.conf, e.g. change ifconfig_em0 to ifconfig_vmx3f0 (sic!) in case of vmxnet 3
 - Update Firewall rules (em0 -> vmx3f0)

 - Shutdown the FreeBSD VM with [ poweroff ] command
 - Change network interfaces @ VM
 - Start VM is it should be with new network interfaces.

Mirrors to Diff files:
 - vmxnet.diff  http://pastebin.com/raw.php?i=XQpdsYMD
 - vmxnet3.diff  http://pastebin.com/raw.php?i=LM3knk0H

Problem
As of May 2012, VMware Tools that come with ESXi 5 don't work on FreeBSD 9. I found this article in which Jared Barneck reports that he just had to recompile vmmemctl and vmblock. The source code for both of these modules (and vmxnet/vmxnet3 as well) is included in vmware-freebsd-tools.tar.gz. However, you can compile neither vmxnet nor vmxnet3 without some minor modifications.
Make sure that you have at least the kernel source tree in /usr/src/sys.
Solution

Download vmxnet.diff and vmxnet3.diff
In the vSphere Client, select your FreeBSD virtual machine and (right-)click Guest->Install/Upgrade VMware Tools
In your FreeBSD machine, run mount -t cd9660 /dev/cd0 /mnt and tar xzf /mnt/vmware-freebsd-tools.tar.gz
    cd vmware-tools-distrib/lib/modules/source/
Untar vmblock.tar, vmmemctl.tar, vmxnet.tar, and vmxnet3.tar
Now apply my patches:
        cd vmxnet-only
        patch -p1 < /tmp/vmxnet.diff
        cd ../vmxnet3-only
        patch -p0 < /tmp/vmxnet3.diff
Change into vmblock-only and run make && make install
Change into vmmemctl-only and run make && make install
Change into vmxnet-only and run make && make install
Change into vmxnet3-only and run make && make install
Switch back to the top-level directory (vmware-tools-distrib) and run vmware-install.pl as usual
Update /etc/rc.conf, e.g. change ifconfig_em0 to ifconfig_vmx3f0 (sic!) in case of vmxnet 3
http://ogris.de/vmware/


2.a) Install VMware Tools on FreeBSD 10.0 REL (working solution)
First of all
cd /usr/ports/misc/compat6x
make install clean
ln -s /usr/local/bin/perl /usr/bin/perl
Info http://ogris.de/vmware/freebsd10.html

In case of Error
make: "/usr/share/mk/bsd.kmod.mk" line 12: Unable to locate the kernel source tree. Set SYSDIR to override.
rm -rf /usr/src
svnlite checkout https://svn0.us-west.freebsd.org/base/releng/10.1 /usr/src
cd /tmp/vmware-tools-distrib/vmware-tools-distrib
./vmware-install.pl -d   

2.a) Install VMware Tools on FreeBSD 10.0 REL

- No GCC in the default install of FreeBSD 10 - It's CLANG only and GCC from ports cannot be used to build the modules.
-  The vmx(4) driver has been added. vmx(4) is a VMware VMXNET3 ethernet driver ported from OpenBSD. [r254738]
http://www.freebsd.org/releases/10.0R/relnotes.html

# cd /tmp/vmware-tools-distrib/lib/modules/source/ ; tar -xf vmmemctl.tar; cd vmmemctl-only; make;  make install
os.c:267:33: error: implicit declaration of function 'kmem_alloc' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
   p->bitmap = (unsigned long *)kmem_alloc(kernel_map, p->size);
                                ^
os.c:267:33: note: did you mean 'kmem_malloc'?
   p->bitmap = (unsigned long *)kmem_alloc(kernel_map, p->size);
                                ^~~~~~~~~~
                                kmem_malloc
@/vm/vm_extern.h:58:13: note: 'kmem_malloc' declared here
vm_offset_t kmem_malloc(struct vmem *, vm_size_t size, int flags);
            ^
os.c:267:16: error: cast to 'unsigned long *' from smaller integer type 'int' [-Werror,-Wint-to-pointer-cast]
   p->bitmap = (unsigned long *)kmem_alloc(kernel_map, p->size);
               ^
os.c:274:14: error: incompatible pointer types passing 'vm_map_t' (aka 'struct vm_map *') to parameter of type 'struct vmem *'
      [-Werror,-Wincompatible-pointer-types]
   kmem_free(kernel_map, (vm_offset_t)p->bitmap, p->size);
             ^~~~~~~~~~
@/vm/vm_extern.h:59:29: note: passing argument to parameter here
void kmem_free(struct vmem *, vm_offset_t, vm_size_t);
                            ^
3 errors generated.
*** Error code 1

Stop.
make: stopped in /tmp/vmware-tools-distrib/lib/modules/source/vmmemctl-only
install: ../vmmemctl.ko: No such file or directory
*** Error code 71

Stop.
make: stopped in /tmp/vmware-tools-distrib/lib/modules/source/vmmemctl-only
Install GCC http://www.freebsd.org/doc/en/articles/custom-gcc/article.html
cd /usr/ports/devel/binutils &&make config-recursive install clean
cd /usr/ports/lang/gcc &&  make config-recursive install clean

$ cat make.conf
# added by use.perl 2012-06-07 03:03:21
PERL_VERSION=5.10.1
.if !empty(.CURDIR:M/usr/ports/*) && exists(/usr/local/bin/gcc46)
CC=gcc46
USE_GCC=4.6+
CXX=g++46
CPP=cpp46
FC=gfortran46
.endif

FFLAGS=-O2 -mtune=athlon64
CC=gcc46
CXX=g++46
CPP=cpp46
FC=gfortran46


3) Enable SNMP on ESXi 5.0 hosts
1) Enable temporarly SSH on ESXi host



2) Login to ESXi host through SSH
vi /etc/vmware/snmp.xml
press i to enter Insert mode
enable -> true
communities -> mycommunity
<config><snmpSettings><enable>true</enable><communities>mycommunity</communities><targets></targets></snmpSettings></config>..

Press ESC to exit insert mode
:wq  to save and exit

3) Restart services
/etc/vmware # /sbin/services.sh restart

4) Check SNMP
# snmpwalk -v 2c -c mycommunity 10.10.10.1
SNMPv2-MIB::sysDescr.0 = STRING: VMware ESX 5.0.0 build-1117897 VMware, Inc. x86_64
SNMPv2-MIB::sysObjectID.0 = OID: VMWARE-PRODUCTS-MIB::vmwESX
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (262003) 0:43:40.03
SNMPv2-MIB::sysContact.0 = STRING: not set
SNMPv2-MIB::sysName.0 = STRING: localhost
SNMPv2-MIB::sysLocation.0 = STRING: not set
SNMPv2-MIB::sysServices.0 = INTEGER: 72
5) Do not forget to disable SSH at ESXi host

4) Enable additional network card drivers on ESXi 5.0 hosts
1) Enable ESXi shell
2) Check network drivers
esxcfg-nics -l
Name    PCI           Driver      Link Speed     Duplex MAC Address       MTU    Description
vmnic0  0000:03:00.00 r8168       Up   100Mbps   Full   00:19:99:aa:ec:ab 1500   Realtek Realtek 8168 Gigabit Ethernet

lspci -v | grep Ethernet
000:003:00.0 Ethernet controller Network controller: Realtek Realtek 8168 Gigabit Ethernet [vmnic0]
000:004:07.0 Ethernet controller Network controller: Intel Corporation 82558B PRO/100+ PCI (TP)
http://pibytes.wordpress.com/2013/02/17/vmware-esxi-network-driver-install-and-upgrade/

3) Download driver from
https://my.vmware.com/group/vmware/details?downloadGroup=DT-ESXI5X-INTEL-IGB-5051&productId=229
4) Enter maintenance mode
5) Unzip downloaded file, scp to /tmp on ESXi host
6) Install driver
 # esxcli software vib install -v /tmp/igb-5.0.5.1-1290180/Intel_bootbank_net-igb_5.0.5.1-1OEM.500.0.0.472560.vib
Installation Result
   Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
   Reboot Required: true
   VIBs Installed: Intel_bootbank_net-igb_5.0.5.1-1OEM.500.0.0.472560
   VIBs Removed: VMware_bootbank_net-igb_2.1.11.1-3vmw.500.2.26.914586
   VIBs Skipped:
~ #
7) Exit from maintenance mode
8) Reboot

5) Install VMware Tools on RHEL 6 & Centos 6
ESXi 5.1  (v. 1065491)
CentOS 6.3 64bits

yum install make gcc kernel-devel kernel-headers glibc-headers perl
! mount VMWareTools CD in VM
mkdir /mnt/cdrom
mount /dev/cdrom /mnt/cdrom
cp /mnt/cdrom/VMwareTools-9.0.5-1065307.tar.gz /tmp/
cd /tmp/
tar xvfz VMwareTools-9.0.5-1065307.tar.gz
/tmp/vmware-tools-distrib/vmware-install.pl -d
! add -d to auto accept defaults

- Install VMware Tools on SUSE SLED 11.0
ESXi 5.1  (v. 1065491)

! mount VMWareTools CD in VM
mkdir /mnt/cdrom
mount /dev/cdrom1 /mnt/cdrom
cp /mnt/cdrom/VMwareTools-9.0.15-2323214.tar.gz  /tmp/
cd /tmp/
tar xvfz VMwareTools-9.0.15-2323214.tar.gz  
/tmp/vmware-tools-distrib/vmware-install.pl -d
! add -d to auto accept defaults

How to install VMWare tools on SUSE Linux Enterprise Server 11
1 Launch YaST | Software | Software Management
2 Change Filter to Patterns, and make sure c++ compiler is installed.
If the c++ compiler pattern is not available, add the SLES DVD/ISO as a repository
http://www.novell.com/support/kb/doc.php?id=7014489
If the c++ pattern is not installed, select it in the list of available patterns and install c++ compiler
3 Right-click the VMWare guest in the VMWare client, and click Install VMWare tools
This may also be under a Guest tab after right-clicking on the guest VMWare
4 Close YaST
5 mount /dev/cdrom /media; cp /media/*.tar.gz /tmp
6 cd /tmp; tar -zxvf VM*.tar.gz; perl /tmp/vmware-tools-distrib/vmware-install.pl -d
7 init 6 to restart the server
8 /etc/init.d/vmware-tools status to make sure it is running

6) P2V Converting Physical Machine to ESXi VM
SOFTWARE USED:
 - FileZilla Server 0.9.42 beta  (FTP SERVER) (with other FTP server appears errors like E0004000a)
 - Acronis True Image 11 Home (doesnt work with newer versions info )
 - VMware vCenter Converter Standalone 5.1.0

1) Acronis True Image is LiveCD which will transfer using network (FTP) disk clone of physical machine creating *.tib files
2) FTP Server will receive all *.tib files (usually backup is splitted by 2GB files)
3) VMware vCenter Converter Standalone will convert and migrate VM directly to vSphere Infrastructure to ESXi host.
4) After VM is migrated, OS (is this case FreeBSD) will not load because mount point are set to IDE HDD, but ESXi is set to emulate SCSI drive.
To correct that:
mountroot> ufs:/dev/da0s1a
Trying to mount root from ufs:/dev/da0s1a
.... will mount /
.... also will appear info about all mount point

....
# mount -o rw /dev/da0s1a /
# mount -o rw /dev/da0s1f /usr
# mount -o rw /dev/da0s1d /var
# vi /etc/fstab
... correct fstab file
# reboot

8) Connect iSCSI datastore

iSCSI connection will be used over existing Ether-channel ethernet ports (managemendt and data).

1) Configure Storage (in our case is IBM V3700) to accept new host and map some Volume to the host.




2) Configure ESXi host
Select ESXi Host -> Configuration Tab -> Storage Adapter -> Add

Will appear new Adapter like vmhba34, select Properties
 


 If all settings are correctly done, iSCSI device and paths will appear.


7) next
There was an error connecting to VMware vSphere Update Manager
http://www.vmadmin.co.uk/resources/41-vum/356-vumservernameerror