Zabbix monitoring

Zabbix 2.2 install

portinstall openipmi
cd /usr/home/sc/zabbix221/zabbix-2.2.2rc1
./configure --enable-server --enable-agent --with-mysql --enable-ipv6 --with-net-snmp --with-libcurl --with-libxml2 --with-ssh2 --enable-proxy --with-openipmi
make install
Openipmi not compiling from ports
===>  Building package for openipmi-2.0.19_3
Creating package /usr/ports/sysutils/openipmi/work/pkg/openipmi-2.0.19_3.tbz
Registering depends: python27-2.7.8_4 popt-1.16 gettext- libiconv-1.14_3 libexecinfo-1.1_3 net-snmp-5.7.2_16 perl5-5.16.3_11 pkg-1.3.6 indexinfo-0.2.
Creating bzip'd tar ball in '/usr/ports/sysutils/openipmi/work/pkg/openipmi-2.0.19_3.tbz'
tar: lib/python2.7/site-packages/ Cannot stat: No such file or directory
tar: lib/python2.7/site-packages/OpenIPMI.pyc: Cannot stat: No such file or directory
tar: lib/python2.7/site-packages/OpenIPMI.pyo: Cannot stat: No such file or directory
tar: lib/python2.7/site-packages/_OpenIPMI.a: Cannot stat: No such file or directory
tar: lib/python2.7/site-packages/ Cannot stat: No such file or directory
tar: Error exit delayed from previous errors.
pkg_create: make_dist: tar command failed with code 256
*** [do-package] Error code 1

Stop in /usr/ports/sysutils/openipmi.
*** [install] Error code 1

Stop in /usr/ports/sysutils/openipmi.
workaround: add to Makefile:

Make default font a little bigger
[root@zabbix2]# diff ./default.css  /usr/local/www/apache22/data/styles/default.css 
<       font-family: verdana, arial, helvetica, sans-serif; font-size: 12px;
>       font-family: verdana, arial, helvetica, sans-serif; font-size: 10px;
<       font-size: 12px; font-family: Verdana, arial, helvetica, sans-serif;
<       border: 1px solid #ACA899; background-color: white; margin: 0 2px 0 0; height: 14px;
>       font-size: 10px; font-family: Verdana, arial, helvetica, sans-serif;
>       border: 1px solid #ACA899; background-color: white; margin: 0 2px 0 0; height: 12px;
<       margin: 0 2px 0 0; height: 20px; line-height: 14px; max-width: 314px;
>       margin: 0 2px 0 0; height: 14px; line-height: 14px; max-width: 314px;

Zabbix 2.2 Traps configuration

cat ./zabbix_server.conf  | grep -i Trap

# TrapperTimeout=300
disableAuthorization yes
ignoreauthfailure no
donotlogtraps no
pidfile /var/run/
authCommunity log,execute,net public
perl do "/usr/local/etc/zabbix/"
In zabbix Web interface:
 - add host (eg TrapTests) with IP
 - add SNMP interface
 - create Itemwuth Type: SNMP_Trap and key:snmptrap.fallback, Type_of_info:Text

Send SNMP trap to
snmptrap -v 1 -c public '' '' 6 99 '55'  s "teststring"

Zabbix 2.2 VMWare monitoring

Zabbix server config
     cat ./zabbix_server.conf  | grep -i vmwa
Zabbix Web
In case of using vCenter
Add Host with vCenter IP  -> Add template Template Virt VMware
in Tab Macros
{$PASSWORD}  =  zabbixesx
{$URL}  =
{$USERNAME}  =  zabbix

Zabbix will low-level discover VMs

In case of NOT using vCenter
Add Host with ESXi IP -> Template Virt VMware Hypervisor

Zabbix upgrade to version 2 from 1.8.x with corrupted database

ERROR 1062 (23000) at line 1068: Duplicate entry '47-17412' for key 'items_applications_1'
Failed to patch Zabbix database. Restore from backup
Press any key to continue...
1) create database zabbix2 character set utf8;
2) install clean zabbix 1.8.5 cu UTF8 DB;
3) mysql -u root --default-character-set=utf8 -p zabbix2 < ./1815_clear_utf8.sql
4) mysql -u root --default-character-set=utf8 -p zabbix2 < ./zabbix42_only_data_cu_history_fara-app.sql
5) upgrade to 2.0: ./upgrade -u root -p zabbix2

From Official Site
 ./configure --enable-server --enable-agent --with-mysql --enable-ipv6 --with-net-snmp --with-libcurl --with-libxml2
To configure the sources for a Zabbix server and agent, you may run something like:
./configure --enable-server --enable-agent --with-mysql --enable-ipv6 --with-net-snmp --with-libcurl --with-jabber  --enable-proxy

  Detected OS:           freebsd9.0
  Install path:          /usr/local
  Compilation arch:      freebsd
  Enable server:         yes
  Server details:
    With database:         MySQL
    WEB Monitoring via:    cURL
    Native Jabber:         yes
    SNMP:                  net-snmp
    IPMI:                  no
    SSH:                   no
    ODBC:                  no
  Enable proxy:          yes
  Proxy details:
    With database:         MySQL
    SNMP:                  net-snmp
    IPMI:                  no
    SSH:                   no
  Enable agent:          yes
  Enable Java gateway:   no
  LDAP support:          no
  IPv6 support:          yes
*            Now run 'make install'                       *
*                                                         *
*            Thank you for using Zabbix!                  *
*              <>                    *
# make install

/usr/local/sbin/zabbix_server -V
Zabbix server v2.0.3 (revision 30485) (03 October 2012)
Compilation time: Oct 12 2012 13:41:35

/usr/local/sbin/zabbix_agent -V
Zabbix agent v2.0.3 (revision 30485) (03 October 2012)
Compilation time: Oct 12 2012 13:41:35

/usr/local/sbin/zabbix_server -c  /usr/local/etc/zabbix/zabbix_server.conf
zabbix_server [47707]: wrong value of [Timeout] in config file [/usr/local/etc/zabbix/zabbix_server.conf], line 261

mysqldump -u root -p  --ignore-table=zabbix.history --ignore-table=zabbix.history_uint  zabbix > zabbix42_fara_history.sql
mysqldump -u root -p --no-create-db --no-create-info  --replace  --ignore-table=zabbix.history --ignore-table=zabbix.history_uint  zabbix > zabbix42_only_data_fara_history.sql
mysqldump -u root -p --no-create-db --no-create-info  --replace  --ignore-table=zabbix.items_applications  zabbix > zabbix42_only_data_cu_history_fara-app.sql

mysqldump -u root -p  -S /tmp/mysql.sock zabbix1815
./upgrade --user=root --password=xxx -S /tmp/mysql.sock zabbix1815

Zabbix General Information
Number of processed values per second
This is the average number of new values Zabbix server receives every second.
For example, if we have 3000 items for monitoring with refresh rate of 60 seconds, the number of values per second is calculated as 3000/60 = 50
It means that 50 new values are added to Zabbix database every second.

Zabbix keeps a 1-hour max/min/avg/count set of values for each item in the table trends.
The data is used for trending and long period graphs. The one hour period can not be customised.

The Housekeeping is a task run by Zabbix which takes care of removing the unnecessary data of the history, alerts and alarms tables. Taking a look at the zabbix log, you can find out how many records are deleted from the database.
This procedure is configured by means of different parameters into the zabbix_server.conf file.

Zabbix requires a database.  If you are using SQLite, one will be
created automatically.  Otherwise you can install one as follows (<db>
is your database engine and <db_client> is its client program):

% cd /usr/local/share/zabbix/server/create
% cat schema/<db>.sql data/data.sql data/images_<db>.sql | <db_client>

To upgrade an existing database, see the scripts in
===> Correct pkg-plist sequence to create group(s) and user(s)
===>   Compressing manual pages for zabbix-server-1.8.13,2

PHP settings
max_execution_time = 600
memory_limit = 256M
post_max_size = 32M
max_input_time = 600

UPDATE  `zabbix`.`items` SET  `history` =  '8', `trends` =  '366';

If you have a large environment, I recommend you to disable housekeeping, the explanation in this post.

Mysql Database Grow (36 Gb) | How Optimize parameters

  1. Consider not more than 7 days for history (3 days for medium environments)
  2. Avoid intervals less than 60 seconds
  3. Keep your eyes on disksize from history/trend tables

UPDATE  `zabbix`.`users` SET  `refresh` =  FLOOR(180 + (RAND() * 60)) WHERE  `users`.`userid` >0;

    Before acceleration
    [root@zabbix /usr/local/etc/zabbix/]#  ab -c 50 -n 500 http://localhost/overview.php      
    This is ApacheBench, Version 2.3 <$Revision: 655654 $>
    Copyright 1996 Adam Twiss, Zeus Technology Ltd,
    Licensed to The Apache Software Foundation,

    Benchmarking localhost (be patient)
    Completed 100 requests
    Finished 500 requests

    Server Software:        lighttpd/1.4.30
    Server Hostname:        localhost
    Server Port:            80

    Document Path:          /overview.php
    Document Length:        12471 bytes

    Concurrency Level:      50
    Time taken for tests:   43.775 seconds
    Complete requests:      500
    Failed requests:        51
       (Connect: 0, Receive: 0, Length: 51, Exceptions: 0)
    Write errors:           0
    Total transferred:      6378445 bytes
    HTML transferred:       6235445 bytes
    Requests per second:    11.42 [#/sec] (mean)
    Time per request:       4377.529 [ms] (mean)
    Time per request:       87.551 [ms] (mean, across all concurrent requests)
    Transfer rate:          142.29 [Kbytes/sec] received

    Connection Times (ms)
                  min  mean[+/-sd] median   max
    Connect:        0    4  19.5      0     309
    Processing:  1923 4178 721.1   4214    7223
    Waiting:     1842 3924 719.2   3995    6479
    Total:       1925 4182 719.9   4218    7225

    Percentage of the requests served within a certain time (ms)
      50%   4218
      66%   4463
      75%   4597
      80%   4689
      90%   4958
      95%   5269
      98%   5992
      99%   6325
     100%   7225 (longest request)
    [root@zabbix /usr/local/etc/zabbix/]#

    After install


    1) Problem starting zabbix_agentd
    a) issue #1
    zabbix_agentd [84972]: cannot create Semaphore: [28] No space left on device
    zabbix_agentd [84972]: unable to create mutex for log file
    Increase semophores
    # cat /boot/loader.conf
    New settings will be available after restart.

    b) issue #2
    8412:20110908:081804.268 Zabbix Agent started. Zabbix 1.8.3 (revision 13928).
    8412:20110908:081809.273 Can't recreate shared memory for collector. [too many attempts]
    # ipcs -a | egrep -i 'zabbix|Memory|sem'
    Shared Memory:
    m        65536   1817297945 --rw------- zabbix   zabbix   zabbix   zabbix 6 493428 1225 1225 11:08:38 no-entry 11:08:38
    T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP          NSEMS OTIME    CTIME 
    s        65536   2052178969 --rw------- zabbix   zabbix   zabbix   zabbix 9 11:56:38 11:08:38
    Remove  Shared Memory Identifiers & Semaphore Array :
    # ipcrm -M 1817297945
    # ipcrm -S 2052178969

    2)User scripts for zabbix

    Backup zabbix2 DB with config only (without history, events)

    1 comment :

    1. in pre 2 ver was
      cd /usr/local/share/zabbix/server/create
      cat schema/mysql.sql data/data.sql data/images_mysql.sql | mysql -u zabbix -p zabbix

      in v2 we dont have that path