Cacti (ver 0.8.8a) installation,customization and optimization on FreeBSD 9.0

The goal of the article: build fast system for collecting snmp data with cacti 0.8.8a with following configuration:
- detailed stats for every minute - last 10 days, 
- averaged stats - last 3 years.

last update: 5 december.2012

Table of content:
0) Preparing System  (hardware/software)
1) OS configuration

2) Cacti Install
3) Cacti Optimize (make more fast)
4) Cacti Customize (change how will look)
5) Plugins
6) Cacti FAQ (errors appeared in working process)
7) Results
8) Tools used
another nice instruction http://forums.cacti.net/viewtopic.php?f=6&t=48159

0) General info about RRDTool, OS and Hardware used

Will be needed some decent hardware (CPU, RAM, HDD)  and recent software.
Hardware:
stats# sysctl -a | egrep -i 'hw.machine|hw.model|hw.ncpu|hw.physmem'
hw.machine: amd64
hw.model: Intel(R) Xeon(TM) CPU 2.80GHz
hw.ncpu: 4
hw.physmem: 2131570688
hw.machine_arch: amd64
stats# mptutil show volumes
mpt0 Volumes:
  Id     Size    Level   Stripe  State  Write-Cache  Name
     0 (   68G) RAID-1          OPTIMAL   Disabled 
stats# mptutil show drives
mpt0 Physical Drives:
   0 (   68G) ONLINE <SEAGATE ST373207LC 7808> SAS bus 0 id 0
   1 (   68G) ONLINE <SEAGATE ST373207LC 7808> SAS bus 0 id 1
OS:
# uname -a
FreeBSD stats 9.0-RELEASE FreeBSD 9.0-RELEASE #1: Mon Mar  5 22:28:22 EET 2012     sc@stats:/usr/obj/usr/src/sys/MYKERN amd64
Basic info to know:
a) Cacti Manual 0.8.8
b) RRDTool Basic Information
c) RRDTool Advanced Graphing (cool stuff)
d) http://www.vandenbogaerdt.nl/rrdtool/tutorial/rrdcreate.php

RRD Structure and RRA definition sample with 5 min polling (70 days no avg, 3 years averaged):
Round Robin Archives
Name                              Steps   Rows    Timespan
daily (no avg) 70 days            1       20000   90000   # RRD file size depends mostly on this Rows value
weekly (30 min avg) 14 days       6       700     610000
monthly (2 hour avg) 62 days      24      750     2678400
yearly (24 hour avg) 400 days     288     400     34053184
3 years (24 hour avg) 1200 days   288     1200    95100000
RRD File structure in my vision :)

1) OS Configuration

- FIREWALL  /etc/rc.firewall
- NTP   /etc/ntp.conf 
   ( http://sclabs.blogspot.com/2011/02/freebsd-system-monitoring.html)
- DNS  /etc/resolv.conf

nameserver    8.8.8.8
nameserver    208.67.222.222
nameserver    8.8.4.4
nameserver    208.67.220.220

- FSTAB

# Device    Mountpoint    FStype    Options Dump    Pass#
/dev/da0p2    /       ufs    rw    1    1
/dev/da0p3    none        swap    sw    0    0
/dev/da0p4    /var        ufs    rw,noatime    2    2
/dev/da0p5    /tmp        ufs    rw,noatime    2    2
/dev/da0p6    /usr        ufs    rw,noatime    2    2
/dev/da1s1d /rrd        ufs    rw,noatime    2    2
- CRONTAB  /etc/crontab
## Cacti stuff
# original     */5 * * * * /usr/local/bin/php /usr/local/share/cacti/poller.php > /dev/null 2>&1
*/5  *  *  *   *  www   /usr/local/bin/php -q /usr/local/www/apache22/data/stats/poller.php > /dev/null 2>&1
27  3  *  *  *  www   /usr/local/bin/php -q /usr/local/www/apache22/data/stats/cli/rebuild_poller_cache.php  > /dev/null 2>&1
52  3  *  *  *  www   /usr/local/bin/php -q /usr/local/www/apache22/data/stats/cli/poller_reindex_hosts.php -id=All  > /dev/null 2>&1

2) CACTI  INSTALLATION

1) Update FreeBSD port tree,
2) Install FAMP (Apache+MySQL) server,
3) Install Cacti and some specific tools Spine, Memcached template, Better MySQL Cacti Template on FreeBSD 7.2

Always check latest stable available version.
http://svn.cacti.net/viewvc/
stats# locate cacti | grep descr    
/usr/ports/net-mgmt/better-cacti-templates/pkg-descr
/usr/ports/net-mgmt/cacti/pkg-descr
/usr/ports/net-mgmt/cacti-spine/pkg-descr

a) Install Cacti base
stats# cd /usr/ports/net-mgmt/cacti/
stats# make config-recursive install clean

1) mysql
stats# mysqladmin --user=root -p create cacti
Enter password:
stats# mysql --user=root -p    
Enter password:
mysql>
mysql>GRANT ALL ON cacti.* TO cactiuser@localhost IDENTIFIED BY 'cactiuser'; FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.01 sec)
Query OK, 0 rows affected (0.00 sec)
mysql> quit
stats# mysql --user=root -p cacti < /usr/local/share/cacti/cacti.sql

2) apache
add mysql info# ee /usr/local/share/cacti/include/config.php
edit apache config# Alias /cacti "/usr/local/share/cacti/"
Be sure apache gives an access to the directory ('Allow from' keywords).

3) crontab
# add to crontab
*/1 * * * * cacti /usr/local/bin/php /usr/local/share/cacti/poller.php > /dev/null 2>&1

4) web interface
Open a cacti login page in your web browser and login with admin/admin
b) Install Cacti Spine under FreeBSD 9.0 with ports.
stats# locate spine | grep descr
/usr/ports/net-mgmt/cacti-spine/pkg-descr
stats# cat /usr/ports/net-mgmt/cacti-spine/Makefile | grep "PORTVERSION="
PORTVERSION=    0.8.8
stats# cd /usr/ports/net-mgmt/cacti-spine/ ; make install clean
...
===> Registering installation for spine-0.8.8
===> SECURITY REPORT:
This port has installed the following files which may act as network
servers and may therefore pose a remote security risk to the system.
/usr/local/bin/spine
===> Cleaning for spine-0.8.8
stats# spine --help
SPINE 0.8.8 Copyright 2002-2012 by The Cacti Group
c) Manual install ( if last spine 0.8.8 is not in ports yet)
Download tar from http://www.cacti.net/spine_download.php, and extract.
1) cd' to the directory containing the package's source code
2) Edit ./configure file
ac_default_prefix=/usr/local/spine    ->    ac_default_prefix=/usr/local
3) ./configure
4) make install clean
# spine -v
SPINE 0.8.8  Copyright 2002-2012 by The Cacti Group
d) Spine configure
stat# mcedit /usr/local/etc/spine.conf
DB_Host localhost
DB_Database cacti
DB_User cactiuser
DB_Pass cactipassword
DB_Port 3306
e) Spine setup in Cacti web

3) CACTI  OPTIMIZATIONS

http://forums.cacti.net/viewtopic.php?t=11747
Alright, so you've gotten Cacti installed and you've got a problem. Here are a few things you should check and read before posting your question:

[HOWTO] Cacti's setup for really BIG environments
http://forums.cacti.net/about29707.html

1) Check official patches at http://www.cacti.net/download_patches.php
Sample patching for older versions:
stats# wget http://www.cacti.net/downloads/patches/0.8.7i/settings_checkbox.patch
stats# patch -p1 -N < settings_checkbox.patch
....
Hmm...  The next patch looks like a unified diff to me...
The text leading up to this was:
--------------------------
|--- 0.8.7i/settings.php        2011-10-12 15:43:42.356672000 -0400
|+++ 0.8.7i-patched/settings.php        2011-12-21 13:50:23.868244800 -0500
--------------------------
Patching file settings.php using Plan A...
Hunk #1 succeeded at 31.
done
2) PHP options ( edit /usr/local/etc/php.ini )
max_execution_time = 180     ; Maximum execution time of each script, in seconds
max_input_time = 180    ; Maximum amount of time each script may spend parsing request data;
max_input_nesting_level = 64 ; Maximum input variable nesting level
memory_limit = 512M      ; Maximum amount of memory a script may consume
date.timezone = "Europe/Chisinau"
short_open_tag = On
upload_max_filesize = 24M
post_max_size = 24M

3) Exclude Deprecated warnings in PHP >5.3:
PHP Deprecated:  Assigning the return value of new by reference is deprecated in
/usr/share/cacti/lib/adodb/adodb.inc.php on line 866
As these have been fixed in Cacti 0.8.8, simply following the recommended procedure and do not show these warnings in production systems.  and set the following values:
; error_reporting
;   Default Value: E_ALL & ~E_NOTICE
;   Development Value: E_ALL | E_STRICT
;   Production Value: E_ALL & ~E_DEPRECATED
error_reporting = E_ALL & ~E_DEPRECATED
4) Restart apache after adjusting the settings.
If for some reason you have no access to php.ini or cannot restart apache, add the following line of code to include/config.php
error_reporting(~E_DEPRECATED);
5) MySQL 5.5
- disable binlogs
my.cnf
[mysqld]
# MOST IMPORTANT FOR SPINE/BOOST PERFORMANCE
tmp_table_size = 512M
max_heap_table_size = 512M

# SOME TUNING
key_buffer = 256M
max_allowed_packet = 16M
myisam_sort_buffer_size = 32M
net_buffer_length = 8K
query_cache_size = 128M
read_buffer_size = 1M
read_rnd_buffer_size = 32M
skip-external-locking
sort_buffer_size = 128M
table_open_cache = 1024
table_definition_cache = 1024

# ENABLE LOGGING SLOW QUERIES
log_slow_queries
long_query_time = 2
log_queries_not_using_indexes = 1

# DISABLE BIN LOG
#log-bin=mysql-bin
#binlog_format=mixed
Restart MySQL
# /usr/local/etc/rc.d/mysql-server restart
Stopping mysql.
Waiting for PIDS: 45205.
Starting mysql.

Verify:
1) Console -> Utilities -> Boost Status
Current Boost Status
Boost On Demand Updating:    Idle
Total Data Sources:    8513
Total Boost Records:    50863
Boost Storage Statistics
Database Engine:    MEMORY
Current Boost Table Size:    36 MBytes
Avg Bytes/Record:    545 Bytes
Max Record Length:    111 Bytes
Max Allowed Boost Table Size:    108 MBytes
Estimated Maximum Records:    207767 Records

Gaps appear if max_heap_table_size is not sufficient (not correctly calculated), even polling time is small and there are not time outs
Last Run Statistics Time:12.3577 Method:spine Processes:1 Threads:40 Hosts:688 HostsPerProcess:688 DataSources:8513 RRDsProcessed:0

2) Table cache/ Open table:
SELECT * , (Open_tables * Uptime / Opened_Tables)OpenTableFactor
FROM (SELECT variable_value Uptime FROM information_schema.global_status WHERE variable_name = 'Uptime_since_flush_status' )up,
(SELECT variable_value Open_tables FROM information_schema.global_status WHERE variable_name = 'Open_tables' )opn,
(SELECT IF( variable_value =0, 1, variable_value ) Opened_tables FROM information_schema.global_status
WHERE variable_name = 'Opened_tables' ) opnd

Uptime     Open_tables     Opened_tables     OpenTableFactor
51717     256     29131    454.48

4) CACTI  CUSTOMIZATION

4.1) One minute settings
First of all some info : http://docs.cacti.net/manual:087:8_rrdtool.03_big_rra

Cacti by default is configured for 5-minute polling, so some changes needed to be made:

4.1.1) Default Cacti RRAs
5 minute polling interval (12 polls in 1 hour) and looks like http://www.netnod.se/ix-stats/sums/All.html:

Round Robin Archives
Name Steps Rows Timespan**
Hourly (1 Minute Average) 1 500 14400
Daily (5 Minute Average) 1 600 86400
Weekly (30 Minute Average) 6 700 604800
Monthly (2 Hour Average) 24 775 2678400
Yearly (1 Day Average) 288 797 33053184

Name: just suggestive name
Steps (archiving ratio): how many data points (poll values) are needed to put as one value of the RRA.
For RRA "Monthly" 24 poll values (24 polls * 5 minute between poll = 120 min= 2 hour) will be consolidates as 1 value.
By default one step represents 300 seconds (5 minutes), which is how often data is entered into the rrd file.
Rows (Hours to store values): how many generations data is kept in the RRA.  

Storage (calculated) default RRAs , 1 RRD = 94 KB (94824 Bytes)
Timespan (used for graphs only): how many seconds to display in graph for this RRA.
X-Files Factor: The amount of UNKNOWN data that can still be regarded as known (= 0.5 always)
http://docs.cacti.net/manual:087:8_rrdtool#rrdtool

4.1.2) RRA's for 1 minute

Will need some calculations: RRD Calculator http://db.tt/XRHohxld  or file

Time table numbers for 1 minute and 5 minute polling:
    Period       1sec        1min        5min
    ------------------------------------------------
    1 min        60          1           -
    5 min        300         5           1
    1 hour       3600        60          12
    1 day        86400       1440        288
    1 week       604800      10080       2016
    1 month      2'628000    43800       8760
    3 month      8'035200    133920      26784
    1 year       31'536000   525600      105120
    3 years      94'608000   1'576800    315360
    5 years      157'680000  2'628000    525600

You may compute the timespan (that specific RRA will store data) by multiplying Steps * Rows (if polling interval is 5 minutes - multipy by 5*60, for 1 minut -  multiply by 60).

Example: 
Monthly (5 minute poll, 2hr avg) = 24*775*5*60 sec = 5580000 sec = 5580000 / 86400 =  64.5 days.
Monthly (1 minute poll, 2hr avg) = 120*775*1*60 sec = 5580000 sec = 5580000 / 86400 =  64.5 days.

Round Robin Archives
Name                                       Steps    Rows    Timespan
Daily (1 min no avg - last 10 days)        1        90000   90000
Weekly (30 min avg - last 14 days)         30       700     610000
Monthly (2 hour avg - last 2 months)       120      750     2678400
Yearly (1 day avg - last year)             1440     400     33053184
3 Years (1 day avg - last 3 years)         1440     1200    95100000
4.1.3) Modify Data Templates

4.1.4) Modify Graph Template

4.1.5) Modify Data Query to add Graph Template

b) RRA Tuning for 1 minute polling

c) Cacti Data Templates Update
By default cacti installation includes step=300, heartbeat=600.
Step=300 (once in 5 minute),Heartbeat=600 (2 timeout for insert UNKNOWN in rrd) = 2*step.

Massive MySQL update
mysql>
update data_template_data set rrd_step='60';# 27 rows affected.
update data_template_rrd set rrd_heartbeat='120';# 40 rows affected.
d) Crontab adjustment
d-a) OS crontab
*/1 * * * * root /usr/local/bin/php /www/cacti/poller.php > /dev/null 2>&1
# cacti
*/1     *       *       *       *       root    /usr/local/bin/php -q /usr/local/share/cacti/poller.php > /tmp/poller.php  > /dev/null 2>&1
30      3       *       *       *       root    /usr/local/bin/php -q /usr/local/share/cacti/cli/rebuild_poller_cache.php  > /dev/null 2>&1
44      3       *       *       *       root    /usr/local/bin/php -q /usr/local/share/cacti/cli/poller_reindex_hosts.php -id=All  > /dev/null 2>&1

d-b) Cacti Polling Settings


d) Storage

Size of 1 RRD = 94'668 bytes


e) Delete old (5minute polling) RRA
That changes only apply to fresh rrd files.
Existing ones have to be tweaked to mimic this behaviour.
We are aware of the fact, that currently the support is at basic level only.
It is fine to adjust all RRA's as mentioned above when doing a fresh start, deleting all existing rrd files.
Using official stuff only, it is NOT POSSIBLE to convert a 5 min absed rrd file to 1 min based.

4.2) Adding a 5-year graph
data-sources -> RRA is where you have to go. Create a new RRA by pressing ADD and enteriung the following data:
  • steps : 2016 - this means that assuming one step every 5 minutes (thats the cacti-poller) then 2016 steps is exactely one week and this is how often a new point is added to the graph
  • rows : 1000 - that many values are kept in the RRA-database.
  • timespan : 158112000 - this is 5 Years in seconds
  • name : Five Years (7 day - average)
Then you go to the data-templates and for each template you can add you new RRA in the multi-select-field of. From this second on you will see the new "5-year-graph" in every graph using this datasource.

4.3) HTML Select Size
add to /lib/html_form.php twice 
change
print "<select id='$form_name' name='$form_name" . "[]'" . $class . " multiple>\n";
with
print "<select size='8' id='$form_name' name='$form_name" . "[]'" . $class . " multiple>\n";

/stats/lib]# diff ./html_form.php ./html_form.php088a
467c467
< print "<select size='8' id='" . htmlspecialchars($form_name) . "' name='" . htmlspecialchars($form_name) . "'" . $class . $on_change . ">";
--
> print "<select id='" . htmlspecialchars($form_name) . "' name='" . htmlspecialchars($form_name) . "'" . $class . $on_change . ">";
616c616
< print "<select size='8' id='$form_name' name='$form_name" . "[]'" . $class . " multiple>\n";
---
> print "<select id='$form_name' name='$form_name" . "[]'" . $class . " multiple>\n";
4.4) Change graphs font
OS: FreeBSD 8.2
a) OS font list can be found at
# fc-list   

b) Get and install new fonts
# cd /usr/ports/x11-fonts/webfonts/ && make install clean
# cd /usr/local/lib/X11/fonts ; wget http://download.damieng.com/fonts/redistributed/DroidFamily.zip; unzip DroidFamily.zip

c) Make new fonts usable
# fc-cache -v

CACTI SETTINGS
default     Lucida Sans Typewriter,10
/settings.php?tab=path
RRDTool Default Font    ->  Droid Sans Mono,10
OR
RRDTool Default Font    ->  Lucida Sans Typewriter,10

/settings.php?tab=visual
Title Font Size            ->  9
Title Font File            ->  empty
Legend Font Size        ->    8
Legend Font File        ->    empty
Axis Font Size            ->    9
Axis Font File            ->    empty
Unit Font Size            ->    8
Unit Font File            ->    empty


4.5) Change Graph Title Description
From Cacti Menu
step1)
Settings:
Visual -> Graph Management -> Maximum Title Length = 100
Visual -> Data Queries -> Maximum Field Length =50
Visual -> Data Sources -> Maximum Title Length = 100


step 2)
Data Queries -> SNMP - Interface Statistics-> In/Out Bits and In/Out Bits (64-bit Counters)  add:

Data Template - Interface - Traffic:
name
    |host_description| - |query_ifName| to |query_ifAlias|

Graph Template - Interface - Traffic (bits/sec)
title |host_description| - |query_ifName| to |query_ifAlias|

4.6) Graph View
By default graph_image.php display images without RRD legend.
/* override: skip drawing the legend? */
if (!empty($_GET["graph_nolegend"])) {
to
/* override: skip drawing the legend? */
if (empty($_GET["graph_nolegend"])) {
4.7) Cacti TAB images (CONSOLE, GRAPHS gifs)
Original photoshop psd files
http://dl.dropbox.com/u/61785248/cacti%20tab%20image.zip

4.8) Cutomize RRDTool X-Grid

Basics http://www.vandenbogaerdt.nl/rrdtool/tutorial/x-grid.php
http://oss.oetiker.ch/rrdtool/doc/rrdgraph.en.html

Modified file ./lib/rrd.php
$myxgrid="";
$z =  $graph_start - $graph_end;
// 1 - 8 hours
if (($z > -28800) && ($z <= -7200))         { $myxgrid = "--x-grid MINUTE:10:MINUTE:30:MINUTE:30:0:%H:%M   "; }
// 8 - 18 hours
elseif (($z > -64800) && ($z <= -28800))     { $myxgrid = "--x-grid MINUTE:30:HOUR:1:HOUR:1:0:%H   "; }                        
// 18 - 36 hours
if (($z > -130000) && ($z <= -64800))        { $myxgrid = "--x-grid MINUTE:30:HOUR:2:HOUR:2:0:%H   "; }
// 36 hours - 17 days
elseif (($z >= -1500000) && ($z < -130000)){ $myxgrid = "--x-grid HOUR:6:DAY:1:DAY:1:0:%d  "; }
// 17 - 69 days  (3rd graph - monthly)
elseif ( ($z >= -6000000) && ($z < -1500000) )        { $myxgrid = "--x-grid DAY:1:WEEK:1:WEEK:1:0:%e/%b  "; }
// 69 - 347 days
elseif ( ($z >= -30000000) && ($z < -6000000) )        { $myxgrid = "--x-grid WEEK:1:MONTH:1:MONTH:1:0:%b  "; }
// 347 - 500 days  (4th graph - yearly)
elseif ( ($z >= -43200000) && ($z < -30000000))        { $myxgrid = "--x-grid MONTH:1:YEAR:1:MONTH:1:2278400:%b"; }
// more than 500 days  (5th graph 3 years)
elseif (($z < -43200000))                            { $myxgrid = "--x-grid MONTH:1:MONTH:12:MONTH:12:910000:%Y  "; } 

Cacti RRDTool X-grid cutomized
$graph_opts .=
    "--imgformat=" . $image_types{$graph["image_format_id"]} . RRD_NL .
    "--start=$graph_start" . RRD_NL .
    "--end=$graph_end" . RRD_NL .
...  
    "$myxgrid" . RRD_NL .
...
Another options to modify
// "--border 4"
// "--color=MGRID#0000FF"
// " --grid-dash 1:0"
4.9) Display graph on merge cells


Graph ID = 5407
[root@/stats088/stats/lib]# diff ./html.php ./html.php088
269,272c269
<
< <td align='center' width='<?php if ( ($graph["local_graph_id"] != 5407) && ($graph["local_graph_id"] != 4987) )
<                         { print ceil(100 / read_graph_config_option("num_columns")); print '%';} else {print '100%\''; print ' colspan=2'; $i++; }?>
<                                                 '>
---
>                       <td align='center' width='<?php print ceil(100 / read_graph_config_option("num_columns"));?>%'>

5) CACTI Customization with plugins

a)
b) RRDCleaner
Once again check r/w permission on RRD files, what user can read/write (or delete) files.


6) CACTI FAQ

0) PHP Notice:  Undefined index:  in /usr/local/www/apache22/data/lib/rrd.php on line 289
PHP Notice:  Undefined index:  in /usr/local/www/apache22/data/lib/rrd.php on line 289
ERROR: Unrecognized consolidation function

When creating new RRA just not forget to add some consolitation function.
Error appear when configuration is like in picture:
Need to select one or more consolidation funtion

1) 05/19/2011 11:24:05 AM - SPINE: Poller[0] ERROR: SQL Failed! Error:'1062', Message:'Duplicate entry '1957-voiporig-2011-05-19 11:24:01' for key 'PRIMARY'', SQL Fragment:'INSERT INTO poller_output_boost (local_data_id, rrd_name, time, output) VALUES (1957,'voiporig','2011-05-19 11:24:01','27'),(1958,'voipterm','2011-05-19 11:24:01','163')'

Causes:
- multiple cron jobs running cacti,
- multiple spine processes (Maximum Concurrent Poller Processes = 1 is OK),
- manually running the poller.

Solution:
$mysql
mysql> TRUNCATE TABLE `poller_output` 

2) Recache Event
In case you are using Data Queries, Cacti is subject to "recache" in case the recognized Data Query entries change. E.g., a new interface may show up and you may want to see that.
There are various tests to trigger a recache, default is to recache in case the target device rebooted (Uptime goes backwards).

05/19/2011 11:43:20 AM - CMDPHP: Poller[0] Host[417] DS[4963] WARNING: Result from SNMP not valid. Partial Result: No Such Instance cur

Problem with Data Source ( interface changed/removed)
Solution: Reload Data Query for Host[417] and creat new graphs with new Data Sources.

PCOMMAND: Poller[0] Host[15] WARNING: Recache Event Detected for Host
Solution:
host.php?action=edit&id=15
delete and readd
Data Query Name: SNMP - Interface Statistics with
Re-Index Method= Uptime Goes Backwards


It means poller is marking hosts for reindexing.
It means that it has detected some SNMP fields are not matching the indexes.
[root@reg /www/statis/cli]# /usr/local/bin/php ./poller_reindex_hosts.php --id=All
WARNING: Do not interrupt this script.  Reindexing can take quite some time
.................................................................

[root@reg /www/statis/cli]#/usr/local/bin/php ./rebuild_poller_cache.php
WARNING: Do not interrupt this script.  Rebuilding the Poller Cache can take quite some tie
......................................................................................................
Modify your php.ini file and increase the Memory and Max Execution time.
Please increase to 64 MB and then restart apache.

Truncate your poller_reindex table and then run the reindex hosts script on all you host and let me know what becomes of it.

3)  05/19/2011 11:22:05 AM - CACTID: Poller[0] ERROR: Problem with MySQL: 'Duplicate entry '0-1-5:12' for key 1'
   1] mysql -u root -p cacti
>truncate table poller_output;
>quit
   2] mysqlcheck --auto-repair --databases cacti


4) 05/19/2011 11:24:05 AM - POLLER: Poller[0] WARNING: Poller[0] Host[1] SNMP Time was not numeric
Problem: Availability settings for this host (SNMP ping time incorrect format)
Solution: Only ICMP ping

5) Changing device ID number (cacti forum thread)
(old host id =1121, new host id = 4)
mysql query in :
update data_local set data_local.host_id=4 where data_local.host_id=1121;
update graph_local set graph_local.host_id=4 where graph_local.host_id=1121;
update graph_tree_items set graph_tree_items.host_id=4 where graph_tree_items.host_id=1121;
update host set host.id=4 where host.id=1121;
update host_graph set host_graph.host_id=4 where host_graph.host_id=1121;
update host_snmp_cache set host_snmp_cache.host_id=4 where host_snmp_cache.host_id=1121;
update host_snmp_query set host_snmp_query.host_id=4 where host_snmp_query.host_id=1121;
update poller_item set poller_item.host_id=4 where poller_item.host_id=1121;
update poller_reindex set poller_reindex.host_id=4 where poller_reindex.host_id=1121;
6) Cisco NBAR
File http://forums.cacti.net/viewtopic.php?f=12&t=8050&hilit=nbar

Code:
php -q ss_nbar_all.php x.x.x.x commstring 2 query index

I would get
Code:
Fatal error: Call to undefined function read_config_option() in /var/www/xxxx/cacti/lib/snmp.php on line 233

I added the following line into ss_nbar_all.php:
Code:
include_once(dirname(__FILE__) . "/../include/global.php");

It now runs like a charm.
I just wanted to let anyone else having the same issues know the fix.

7)  Importing template Error: XML: Cacti version does not exist
When importing poller_stats template, XML Error appears

Solving add "0.8.8"  => "0024" in /include/global_arrays.php
$hash_version_codes = array(
    "0.8.4"  => "0000",
...
    "0.8.7e" => "0019",
    "0.8.7f" => "0020",
    "0.8.7g" => "0021",
    "0.8.7h" => "0022",
    "0.8.7i" => "0023",
    "0.8.8"  => "0024"
    );

8) RRD step mismatch
in Config step is 300, but RRD file wass created with step=60;
file = /zzz/cacti-www/stats/rra/438_traffic_in_7601.rrd
step = 60

7) CACTI  TROUBLESHOOTING  AND  RESULTS


Check Cacti performance and host that could create time-out
# spine  -R -S -V 2 -C /usr/local/etc/spine.conf
SPINE: Using spine config file [/usr/local/etc/spine.conf]
SPINE: Version 0.8.8a starting
Host[931] ERROR: Problems parsing Multi SNMP OID! (oid: 10.6.11.1 1.3.6.1.4.1.2011.6.3.4.1 ), Set MAX_OIDS to 1 for this host to isolate bad OID
SPINE: Time: 26.8307 s, Threads: 40, Hosts: 877

# ss_poller.php ss_poller
output: Time:101.9749 Method:spine Processes:1 Threads:40 Hosts:877 HostsPerProcess:877 DataSources:11141 RRDsProcessed:0
For verbose output use
# spine  -R -S -V 5 -C /usr/local/etc/spine.conf
Check RRDTool, NET-SNMP version
# rrdtool -v
RRDtool 1.4.7  Copyright 1997-2012 by Tobias Oetiker <tobi@oetiker.ch>
               Compiled Dec  5 2012 11:36:16

RRDtool 1.4.5  Copyright 1997-2010 by Tobias Oetiker <tobi@oetiker.ch> (bug)

# snmpwalk -V              
NET-SNMP version: 5.7.1

Cacti poller comparison:
Hosts:668 HostsPerProcess:67 DataSources:7374

N  proc/thr seconds
10    1/10    148-150
20    1/20    77-80
30    3/10    92-93
36    6/6        83
100    10/10    29-30
100 5/20    29
100    4/25    30-31
100    3/33    31
100    2/50    26
100    1/100    20-21
75    1/75    23-27
125 1/125    20.7
>100
120 6/20    298.8597 Method:spine Processes:6 Threads:20

Cacti stats:
Poller Type = SPINE / 1 minute polling
Maximum Concurrent Poller Processes = 1
Maximum Threads per Process = 6

Boost
Boost Memory Maximum 390 MB
RRD Updates 330k
Enable direct population of poller_output_boost table by spine
05/19/2011 11:08:06 AM - SYSTEM DSSTATS STATS: Type:HOURLY, Time:0.2949
05/19/2011 11:08:06 AM - SYSTEM STATS: Time:6.3456 Method:spine Processes:1 Threads:6 Hosts:23 HostsPerProcess:23 DataSources:2575 RRDsProcessed:0

8) Other things used

1) Edit one line in multiple files
perl -pi -w -e 's/FIND/REPLACE/g;' *.php
perl -pi -w -e 's/\/usr\/local\/www\/apache22\/data\/statis\/rra/\/usr\/local\/www\/apache22\/data\/stats\/rra/g;' ./*
2) Add RRA Association if RRD have not. (need to recreate RRD)
INSERT INTO `cacti088`.`data_template_data_rra` ( `data_template_data_id` , `rra_id` ) VALUES ( '3874', '7');

'7' - is RRA for 3 years.
SELECT data_template_data_id FROM `data_template_data_rra` WHERE `rra_id` =1;
3) Graphs width DB mass update
UPDATE `cacti088a`.`graph_templates_graph` SET `width` = '500' WHERE `graph_templates_graph`.`id` >0;
4) Host settings mass updates
Downed Device Detection
The method Cacti will use to determine if a host is available for polling.
NOTE: It is recommended that, at a minimum, SNMP always be selected.
Ping (ICMP) or SNMP Uptime     is    availability_method = 4
UPDATE `cacti088a`.`host` SET `availability_method` = '4', `ping_timeout` = '500', `ping_retries` = '3' WHERE `host`.`id` >0;
) reserved


Links:
http://www.jpnap.net/english/jpnap-tokyo-i/traffic.html
Lighttpd Stats for cacti - http://habrahabr.ru/post/66602/

No comments :

Post a Comment