For long term monitoring = SNMP with storage MRTG/RRDTool (e.g. Cacti).
Realtime monitoring with history of any kind of information = zabbix.
A computer has four basic resources:
1) CPU
2) Memory (RAM)
3) I/O (input/output) - HDD
4) Network
If any one of them is filled to capacity, the others cannot be used to their maximum.
Performance problems are usually caused by running more tasks than the computer can handle.
Key Steps:
1) Know your hardware and resources limitation
CPU, Memory (RAM), Disk (HDD), Network, Processes
tools: top, vmstat, iostat, netstat, ps, ifconfig and other detailed (sar, mpstat, dstat)
2) Know your software and configuration limitation
OS (kernel), packages
3) know your application (MySQL)
MySQL (error, slow query logs, mem usage)
0) Configure SNMP for FreeBSD server
1) Install bsnmp-ucd, which is a module for bsnmpd (FreeBSD snmp agent) which allows you to get memory, load average, cpu utilization and other system statistics. It implements parts of UCD-SNMP-MIB for this.
# cd /usr/ports/net-mgmt/bsnmp-ucd && make all install clean2) Edit /etc/snmpd.config to add the line
begemotSnmpdModulePath."ucd" = "/usr/local/lib/snmp_ucd.so"3) Edit /etc/snmpd.config to change the read variable to set your snmp community string
Read the manual pages bsnmpd(1) and bsnmp-ucd(8) for more info.
4) Add to /etc/rc.conf
bsnmpd_enable="YES"5) Start bsnmp by issuing
# /etc/rc.d/bsnmpd start
1) System (hardware) related information
dmesg -- display the system message buffer, startup info
$ dmesg -a | moresysctl -- get or set system information
# sysctl -a # all parameterspciconf -l -cv = diagnostic utility for the PCI bus (Show PCI devices)
# sysctl -a | grep hw.model
# sysctl -ad # description
sysctl -a | egrep -i 'hw.machine|hw.model|hw.ncpu'
hw.machine: amd64
hw.model: Intel(R) Xeon(R) CPU X5650 @ 2.67GHz
hw.ncpu: 24
hw.machine_arch: amd64
sysctl -a | egrep -i "hw.*mem:"
hw.physmem: 8569384960
hw.usermem: 7562637312
hw.realmem: 9126801408
sysinfo -- utility used to gather system configuration information
# cd /usr/ports/sysutils/sysinfodmidecode - System Information
# make install clean
# sysinfo -a
#dmidecode -t systemTemperatures
dmidecode 2.10
SMBIOS 2.5 present.
Handle 0x0100, DMI type 1, 27 bytes
System Information
Manufacturer: Dell Inc.
Product Name: PowerEdge 2950
...
CPU
# kldload coretempHDD
# echo 'coretemp_load="YES"' >> /boot/loader.conf
# sysctl -a |grep temper
a) diskinfo -ctv /dev/ad0 = utility prints out information about a disk device, and optionally runs a naive performance test on the device.
b) smartd is a daemon that monitors the Self-Monitoring, Analysis and Reporting Technology (SMART) system built into many ATA-3 and later ATA, IDE and SCSI-3 hard drives. The purpose of SMART is to monitor the reliability of the hard drive and predict drive failures, and to carry out different types of drive self-tests.
c) RAID Info
stats# dmesg -a | grep -i "lsi"
mpt0: <LSILogic 1030 Ultra4 Adapter> port 0x2000-0x20ff mem 0xdd210000-0xdd21ffff,0xdd200000-0xdd20ffff irq 18 at device 5.0 on pci3
da0: <LSILOGIC 1030 IM 1000> Fixed Direct Access SCSI-2 device
RAID LSI Info from /var/log/meassages:
Mar 5 17:54:33 stats kernel: mpt0:vol0(mpt0:0:0): Status ( Enabled Re-Syncing )
Mar 5 17:54:33 stats kernel: mpt0:vol0(mpt0:0:0): Low Priority Re-Sync
Mar 5 17:54:33 stats kernel: mpt0:vol0(mpt0:0:0): 1967431 of 143347995 blocks remaining
Mar 5 18:02:51 stats kernel: (mpt0:vol0:1): Physical Disk Status Changed
Mar 5 18:02:51 stats kernel: (mpt0:vol0:1): Volume Status Changed
Mar 5 18:02:51 stats kernel: mpt0:vol0(mpt0:0:0): RAID-1 - Optimal
Mar 5 18:02:51 stats kernel: mpt0:vol0(mpt0:0:0): Status ( Enabled )
Mar 5 18:02:51 stats kernel: (mpt0:vol0:1): Online
Mar 5 18:02:51 stats kernel: (mpt0:vol0:1): Status ( )
stats# mptutil show drives
mpt0 Physical Drives:
0 ( 68G) ONLINE <SEAGATE ST373207LC 7808> SAS bus 0 id 0
1 ( 68G) ONLINE <SEAGATE ST373207LC 7808> SAS bus 0 id 1
stats# mptutil show volumes
mpt0 Volumes:
Id Size Level Stripe State Write-Cache Name
0 ( 68G) RAID-1 OPTIMAL Disabled
mptutil show volumes
cd /usr/ports/sysutils/smartmontools
smartctl -i /dev/ad0
smartctl -d ata -A /dev/ad0 | grep -i temperature
/usr/ports/sysutils/healthd
mbmon
2) OS info
uname = get the kernel version (and BSD version)
$ uname -arcorder -- print a dependency ordering of interdependent files
FreeBSD myhost.servers.eu 7.1-RELEASE FreeBSD 7.1-RELEASE #0: Thu Jan 1 14:37:25 UTC 2009 root@logan.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386
rcorder /etc/rc.d/* /usr/local/etc/rc.d/*
uptime = Uptime + load average
$ uptime
1:06PM up 42 days, 2:39, 1 user, load averages: 0.05, 0.06, 0.02
date = display or set date and time
last = indicate last logins of users and ttys
shutdown -r now = restart system (#shutdown -r 09:30 "Reboot for maintenance")
vmstat -z = system limit, what is necessary to increase
Check if IPFW is enabled
w = display who is logged in and what they are doing
Drop SSH session
Parameters:
netstat -m = mbuf clusters in use (need to be min 10% free)
sysctl kern.maxfiles kern.openfiles = Open files
sysctl kern.ipc.maxsockets kern.ipc.numopensockets = Open sockets
sysctl vm.kvm_size vm.kvm_free = KVM Size
sysctl kern.ipc.maxpipekva kern.ipc.pipekva = Pipe KVA usage
sysctl kern.ipc.nsfbufs kern.ipc.nsfbufsused = Number of sendfile(2) sf_bufs in use
Configure NTP
Network Time Protocol (NTP) is a networking protocol for clock synchronization between computer systems over packet-switched, variable-latency data networks.
NTP uses a hierarchical, semi-layered system of levels of clock sources. Each level of this hierarchy is termed a stratum and is assigned a layer number starting with 0 (zero) at the top.
Stratum 0 - as atomic (caesium, rubidium) clocks, GPS clocks or other radio clocks.
Stratum 1 - serveres attached to Stratum 0 devices. Normally they act as servers for timing requests from Stratum 2 servers via NTP.
Stratum 2 - These are computers that send NTP requests to Stratum 1 servers.
Stratum 3 - These computers employ exactly the same NTP functions of peering and data sampling as Stratum 2, and can themselves act as servers for lower strata.
1) /etc/ntp.conf
7) Manual NTP update
ntpdate <ntp server ip> = update system time
last = indicate last logins of users and ttys
shutdown -r now = restart system (#shutdown -r 09:30 "Reboot for maintenance")
vmstat -z = system limit, what is necessary to increase
Check if IPFW is enabled
# sysctl -a | grep ip.fw.enableSSH Sessions
net.inet.ip.fw.enable: 0
w = display who is logged in and what they are doing
[root@zabbix /usr/]# w
10:19AM up 153 days, 19:12, 3 users, load averages: 1.42, 1.04, 0.82
USER TTY FROM LOGIN@ IDLE WHAT
user1 pts/0 10.11.12.51 Tue04PM 5days su (bash)
user2 pts/3 10.10.23.43 10:05AM - mc
user3 pts/2 10.11.2.6 28Nov13 11days -bash (bash)
Drop SSH session
pkill -9 -t pts/0
Parameters:
netstat -m = mbuf clusters in use (need to be min 10% free)
sysctl kern.maxfiles kern.openfiles = Open files
sysctl kern.ipc.maxsockets kern.ipc.numopensockets = Open sockets
sysctl vm.kvm_size vm.kvm_free = KVM Size
sysctl kern.ipc.maxpipekva kern.ipc.pipekva = Pipe KVA usage
sysctl kern.ipc.nsfbufs kern.ipc.nsfbufsused = Number of sendfile(2) sf_bufs in use
Configure NTP
Network Time Protocol (NTP) is a networking protocol for clock synchronization between computer systems over packet-switched, variable-latency data networks.
NTP uses a hierarchical, semi-layered system of levels of clock sources. Each level of this hierarchy is termed a stratum and is assigned a layer number starting with 0 (zero) at the top.
Stratum 0 - as atomic (caesium, rubidium) clocks, GPS clocks or other radio clocks.
Stratum 1 - serveres attached to Stratum 0 devices. Normally they act as servers for timing requests from Stratum 2 servers via NTP.
Stratum 2 - These are computers that send NTP requests to Stratum 1 servers.
Stratum 3 - These computers employ exactly the same NTP functions of peering and data sampling as Stratum 2, and can themselves act as servers for lower strata.
1) /etc/ntp.conf
server 0.europe.pool.ntp.org2) /etc/rc.conf
server 1.europe.pool.ntp.org
server 2.europe.pool.ntp.org
server 3.europe.pool.ntp.org
server 0.ro.pool.ntp.org
server 1.ro.pool.ntp.org
server 2.ro.pool.ntp.org
server 3.ro.pool.ntp.org
server 0.freebsd.pool.ntp.org iburst maxpoll 9
server 1.freebsd.pool.ntp.org iburst maxpoll 9
server 2.freebsd.pool.ntp.org iburst maxpoll 9
logfile /var/log/ntp.log
ntpd_enable="YES"3) /etc/newsyslog.conf
ntpd_sync_on_start="YES"
/var/log/ntp.log 644 3 100 * J4) /etc/rc.firewall
# allow ntp updates5) Restart syslog, ntp, ipfw demons
$cmd 00307 allow udp from any 123 to external_ip in via external_interface
$cmd 00308 allow udp from external_ip to any 123 out via external_interface
/etc/rc.d/ipfw restart6) Verify
/etc/rc.d/syslogd restart && /etc/rc.d/ntpd restart
# ntpq -p# cat /var/log/ntp.log
remote refid st t when poll reach delay offset jitter
==============================================================================
+svp.provu.co.uk 213.2.4.80 4 u 14 128 377 85.478 9.648 18.087
-procyon.promode 147.231.100.5 3 u 9 128 377 41.975 -12.456 26.108
-sysprod.libfy.c 192.93.2.20 2 u 90 128 377 63.729 44.592 31.613
+ns1.luns.net.uk 157.44.176.4 2 u 16 128 377 68.927 -9.027 30.874
+89.149.54.30 131.107.13.100 2 u 13 128 377 12.901 -9.513 31.644
+birlic.alsys.ro 80.96.120.252 2 u 17 128 377 47.640 10.853 26.184
+ups.alsys.ro 192.53.103.108 2 u 7 128 377 47.632 19.362 22.653
+cache.alsys.ro 192.53.103.108 2 u 15 128 377 12.864 36.949 32.341
+chomper.fedora. 193.226.65.36 2 u 66 128 377 0.811 28.456 28.943
+jekader.fedora. 129.69.1.153 2 u 12 128 377 0.404 -0.122 22.713
*95-65-29-212.st 80.96.120.253 2 u 125 128 377 0.989 -1.793 20.938
* - current NTP server selected
25 Jul 08:41:29 ntpd[12004]: synchronized to 212.121.88.250, stratum 2
25 Jul 11:47:34 ntpd[12004]: time reset +11165.244720 s
25 Jul 11:50:53 ntpd[12004]: synchronized to 80.68.242.188, stratum 3
7) Manual NTP update
ntpdate <ntp server ip> = update system time
# ntpdate 0.europe.pool.ntp.org
11 May 11:52:59 ntpdate[29677]: adjust time server 213.198.55.2 offset 0.000356 sec
3) Basic OS health overview
http://freebsdguide.ru/_19/_3/
Terms used by unix systems for Memory management:
Virtual Memory - The entire memory space available to the Unix Kernel. This consists of both the physical memory and all swap space.
Physical memory - is the RAM attached to the motherboard. This is the stuff you go out and buy and fit to your computer. It does NOT include CPU cache, video memory or other memory attached to controller cards etc.
Swap space (a.k.a Paging space) - The space on disk (HDD) used to extend the physical memory of a system. Swap space is a how a system with 64MB Ram can run a 100MB program.With rare exceptions all Unix systems should have at least one Swap space
Paging - The action of transferring one Page of physical memory from, or to, the swap space. A page is typically in the order of 4KB.
Swapping - The action of transferring an entire program (code & data) between Physical memory and the Swap space.Note that very few Unix systems ever swap in practice.
vmstat - report virtual memory statistics
vmstat 3 = Refresh stats every "3" seconds
vmstat -z = Report on memory used by the kernel zone allocator
[sc@core ~]$ vmstat 1
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr mf0 mf1 in sy cs us sy id
1 40 0 2696M 285M 2498 31 74 6 299 116 0 0 1038 130 194 12 5 84
16 47 0 2824M 403M 2527 0 0 0 603 556 10 55 1090 9445 18865 29 10 61
procs
r - The number of processes that are blocked waiting for CPU time. If this number is high, your CPU is the bottleneck.b - The number of processes that are blocked waiting for system input/output (generally, waiting for disk access). If this number is high, your disk is the bottleneck.
w - The number of processes that are runnable but are entirely swapped out.If you regularly have processes swapped out, your memory is inadequate for the work you are doing on the system.
memory
avm - The average number of pages of virtual memory that are in use. If this value is abnormally high or increasing, your system is actively consuming swap space.
fre - The number of memory pages available for use. If this value is abnormally low, you have a memory shortage.
page
flt - The number of page faults, where information needed was not in real memory and had to be fetched from swap space or disk.
re - The number of pages that have been reclaimed or reused from cache.
pi - Short for pages in; this is the number of pages being moved from real memory to swap.
po - Short for pages out; this is the number of pages being moved from swap to real memory.
fr - How many pages are freed per second.
sr - How many pages are scanned per second.
disks
The number shown is the number of disk operations per second, a valuable clue to determining how well your disks are handling their load.
faults
Faults are not bad; they’re just received system traps and interrupts.in - The number of system interrupts (IRQ requests) received in the last five seconds.
sy - The number of system calls in the last five seconds.
cs - The number of context switches in the last second, or a per-second average since the last update.
cpu
Breakdown of percentage usage of CPU time.us - user time for normal and low priority processes
sy - system tasks
id - cpu idle
systat = display system statistics
systat [-display] [refresh-interval]RAM memory
systat -vmstat 1 = Display general system load
systat -netstat 1 = Display network connections.
systat -ifstat 1 = Display the network traffic going through active interfaces
systat -tcp 1 = TCP statistics (also: icmp, ip, icmp6, ip6 )
systat -iostat 1 = Display statistics about processor use and disk throughput
systat -swap 1 = Show information about swap space usage on all the swap areas compiled into the kernel
fetch http://pastebin.com/download.php?i=0mCKp5gn; tr -d '\015' < download.php\?i=0mCKp5gn > free; mv free /usr/local/bin/free; chmod +x /usr/local/bin/free
$ free
SYSTEM MEMORY INFORMATION:
mem_wire: 484429824 ( 461MB) [ 15%] Wired: disabled for paging out
mem_active: + 217874432 ( 207MB) [ 7%] Active: recently referenced
mem_inactive:+ 1957052416 ( 1866MB) [ 62%] Inactive: recently not referenced
mem_cache: + 34160640 ( 32MB) [ 1%] Cached: almost avail. for allocation
mem_free: + 412622848 ( 393MB) [ 13%] Free: fully available for allocation
mem_gap_vm: + 835584 ( 0MB) [ 0%] Memory gap: UNKNOWN
-------------- ------------ ----------- ------
mem_all: = 3106975744 ( 2963MB) [100%] Total real memory managed
mem_gap_sys: + 94896128 ( 90MB) Memory gap: Kernel?!
-------------- ------------ -----------
mem_phys: = 3201871872 ( 3053MB) Total real memory available
mem_gap_hw: + 19353600 ( 18MB) Memory gap: Segment Mappings?!
-------------- ------------ -----------
mem_hw: = 3221225472 ( 3072MB) Total real memory installed
SYSTEM MEMORY SUMMARY:
mem_used: 817389568 ( 779MB) [ 25%] Logically used memory
mem_avail: + 2403835904 ( 2292MB) [ 74%] Logically available memory
-------------- ------------ ----------- ------
mem_total: = 3221225472 ( 3072MB) [100%] Logically total memory
Swap Information
pstat, swapinfo -- display system data structures
See how large system tables have become if the system is under heavy load
swapinfo = swap information
# swapinfo -h
Device 1K-blocks Used Avail Capacity
/dev/ad0s1b 1013632 116K 990M 0%
#pstat -T
332/7880 files
0M/1485M swap space
# sysctl vm.stats.vm.v_swappgsout
vm.stats.vm.v_swappgsout: 12
# sysctl vm.stats.vm.v_swappgsin
vm.stats.vm.v_swappgsin: 11
4) CPU and Input/Output monitoring
Performance metrics for CPU:
1) Cpu Utilization
- This indicates how much of the CPU is currently getting used.
- This is fairly straight forward, and you can view the CPU utilization from the top command.
- 100% CPU utilization means the system is fully loaded.
- So, a higher %age of CPU utilization will cause performance issues.
2) Context Switch
- When CPU switches from one process (or thread) to another, it is called as context switch.
- Context switching is very essential for multitasking of the CPU.
- However, a higher level of context switching can cause performance issues.
3) Run Queue
- Run queue indicates the total number of active processes in the current queue for CPU.
- When CPU is ready to execute a process, it picks it up from the run queue based on the priority of the process.
- Please note that processes that are in sleep state, or i/o wait state are not in the run queue.
- So, a higher number of processes in the run queue can cause performance issues.
4) Load Average
- This indicates the average CPU load during the last one-, five-, and fifteen-minute periods.
- Metric = 1, CPU's run queue is full.
- Max metric = number of CPU/cores (4 cores = load avg 4 shows system is 100% load)
- Calculated by combining both the total number of process in the queue, and the total number of processes in the uninterruptable task status.
top = realtime display information about the top cpu processes and of what are they doing:
- paging to/from swap,
- spending lots of time in the kernel, or processing interrupts,
- processes/threads are using CPU (default view)
- processes/threads are using I/O (top -m io -o total) = Linux iotop analog for FreeBSD,
- syscalls (STATES):
RUN, CPU0, CPU1... - proccess is executed
ucond/umtx: waiting on an application thread lock(Это может обозначать, что поток ждёт выполнения другого занятого потока).
biord/biowr/wdrain - disk I/Oactivity
sbwait (Wait for data to arrive at/drain from a socket buffer) - perfectly normal, legal, state. Daemons get into it because they're waiting for anything to happen on the socket(s) they are listening to. Process is waiting for a socket buffer to be filled or emptied.
nanslp - Process is blocked in the nanosleep(2) syscall, which is an explicit request to pause for a certain amount of time. Hard to generalize this one. It should be infrequent,
or it might be used by something implementing its own polling.
select - Process is blocked in the select(2) syscall. Обычно это значит, что процесс ждёт внешнего ввода (сеть, терминал, изменение файла, и т.п.). select поддерживает только ожидание чего-то, что связано с файловым дескриптором (например socket, tty, vnode).
lockf - Process is blocked waiting for a file lock to be released. Could be an flock(2) lock or an fcntl(2)/F_SETLK record lock.
piperd - Process is waiting for data to arrive on a pipe ("rd" is short for read).
I/O View:
voluntary context switch - process blocks waiting for a resource
involuntary context switch - kernel decides that the process should stop running for now
Processes:
ps = process status
-a Display information about other users' processes as well as your own.
-x When displaying processes matched by other options, include processes which do not have a controlling terminal.
-w Use 132 columns to display information, instead of the default which is your window size.
If the -w option is specified more than once, ps will use as many columns as necessary without regard for your window size.
-u Display information associated with the following keywords: user, pid, %cpu, %mem, vsz, rss, tt, state, start, time, and command.
The -u option implies the -r option.
-m Sort by memory usage, instead of the combination of controlling terminal and process ID.
-r Sort by current CPU usage, instead of the combination of controlling terminal and process ID.
-t Display information about processes attached to the specified terminal devices. ("w" command will help)
sample:
# ps -auxwww | grep mysqlanother method:
mysqld1 86615 0.0 10.3 119828 52748 p8 I 11:00AM 1:20.02 [mysqld]
Top CPU processes: ps auxwww|sort -nr +2| grep -v idle| head -15
Top Memory processes: ps auxwww|sort -nr +3| head -15
pstree = processes tree
dtpstree = display a tree of processes
# dtpstree -t
/sbin/init-+-/usr/sbin/cron
+-/sbin/devd
+-/usr/libexec/ftpd
+-8*[/usr/libexec/getty]
+-healthd
+-/usr/local/sbin/httpd---8*[/usr/local/sbin/httpd]
+-/usr/sbin/inetd
+-mysqld
+-ntpd: ntp engine---ntpd: dns engine
+-ntpd: [priv]
+-/usr/local/sbin/munin-node
+-sendmail: accepting connections
+-sendmail: Queue runner@00:30:00 for /var/spool/clientmqueue
+-/usr/sbin/sshd-+-sshd: sc [priv]---sshd: sc@ttyp7----bash---su----su---mc---bash---dtpstree
| +-sshd: sc [priv]---sshd: sc@ttyp0----bash---su----su---mc---bash
+-/home/radiusd1/sbin/startd---/home/radiusd1/sbin/radiusd
+-/usr/sbin/syslogd
+-zabbix_agentd: main process-+-zabbix_agentd: main process
+-3*[zabbix_agentd: waiting for connection]
+-zabbix_agentd: poller [sleeping for 1 seconds]
# start ATOP fixed with 1 sec interval, per-second specific
atop -f1 1
(interactive 'l' key , CPU->0)
Disk info
gstat -- print statistics about GEOM disks
# gstat
dT: 1.001s w: 1.000s
L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name
0 14 12 173 5.0 2 40 0.1 6.0| mfid0
0 14 12 173 5.0 2 40 0.1 6.0| mfid0s1
0 69 0 0 0.0 69 8711 0.5 1.7| mfid1
0 0 0 0 0.0 0 0 0.0 0.0| cd0
0 0 0 0 0.0 0 0 0.0 0.0| mfid0s1a
0 0 0 0 0.0 0 0 0.0 0.0| mfid0s1b
0 0 0 0 0.0 0 0 0.0 0.0| mfid0s1d
0 12 12 173 5.0 0 0 0.0 6.0| mfid0s1e
0 2 0 0 0.0 2 40 0.1 0.0| mfid0s1f
0 69 0 0 0.0 69 8711 0.5 1.7| mfid1s1
0 0 0 0 0.0 0 0 0.0 0.0| md0
0 69 0 0 0.0 69 8711 0.6 1.7| mfid1s1d
top -m io -o vcsw
mount = shows mounted file systems with flags
df -h = display free disk space in `Human-readable' output
du -h -d 1 /usr = disk usage (folder) statistics, in `Human-readable' output, 1 depth /usr4) ncdu = NCurses Disk Usage
fdisk /dev/ad0 = PC slice table maintenance utility
disklabel /dev/ad0s1 = read and write BSD label
/stand/sysinstall = can ciew and modify disk mounting points
cat /etc/fstab = static information about the file systems
fstat = identify active files
pstat -f = display system data structures
iostat = report I/O statistics
iostat -d -w 5 mfid0
# iostat -dxz -w 10 -c 2 (count=2)
extended device statistics
device r/s w/s kr/s kw/s qlen svc_t %b
mfid0 23.0 24.7 823.1 811.9 0 18.9 8
mfid1 2.4 32.9 32.0 3864.6 0 9.8 2
extended device statistics
device r/s w/s kr/s kw/s qlen svc_t %b
mfid0 0.3 3.9 3.2 108.2 0 0.7 0
mfid1 0.3 22.6 4.8 2725.2 0 1.0 1
# iostat -d 5 (update every 5 seconds)
mfid0 mfid1 md0 cd0
KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s
34.26 48 1.60 110.41 35 3.81 0.00 0 0.00 0.00 0 0.00
13.96 3 0.04 115.00 13 1.44 0.00 0 0.00 0.00 0 0.00
20.07 6 0.11 124.34 33 4.03 0.00 0 0.00 0.00 0 0.00
# iostat -x 5
extended device statistics
device r/s w/s kr/s kw/s qlen svc_t %b
mfid0 23.0 24.7 823.1 811.9 0 18.9 8
mfid1 2.4 32.9 31.9 3864.6 0 9.8 2
md0 0.0 0.0 0.0 0.0 0 0.4 0
cd0 0.0 0.0 0.0 0.0 0 0.0 0
pass0 0.0 0.0 0.0 0.0 0 0.0 0
extended device statistics
device r/s w/s kr/s kw/s qlen svc_t %b
mfid0 1.0 3.0 36.8 44.5 0 1.7 1
mfid1 0.0 8.0 0.0 849.9 0 0.5 0
md0 0.0 0.0 0.0 0.0 0 0.0 0
cd0 0.0 0.0 0.0 0.0 0 0.0 0
pass0 0.0 0.0 0.0 0.0 0 0.0 0
^C
systat -vmstat
Process debbuging : ktrace, truss
ktrace -i -p 5349
ktrace -C
kdump -Hs
5) Network Information
For Network Troubleshooting read.
ifconfig = view and configure network interface parameters (IP, mask, MAC, status)
systat -ifstat 1 = trafic statistic for "1" seconds (OR use ifstat)
netstat - вывод активных сетевых соединений (сокетов)
systat -netstat n = realtime netstat
sockstat = list open sockets
sockstat -4
trafshow -i rl0 = traffic related to interface "rl0"
iftop = display bandwidth usage on an interface by host
nettop = показывает сетевую активность по категориям
nstrems = показывает текущие потоки на интерфейсе
tcptrack = показывает текущие соединения
/usr/ports/net/bmon
netstat -r = routing table
tcpflow
arp -a = ARP table
Packet sniffers like tcpdump, netflow:
tcpdump -i rl0 host 192.168.61.20 and port 80
Tcpdump packet content
tcpdump -i em1 -nnXSs 0 'port 80'netstat -s = errors or places where you’re out of memory or buffers.
# ifconfig network-interface down
# ifconfig network-interface up
# /etc/rc.d/netif start
# /etc/rc.d/netif stop
# /etc/rc.d/netif restart
# /etc/rc.d/routing restart
# /etc/rc.d/networking restart && /etc/rc.d/routing restart
6) Other information
To see the last 10 lines of a long file, use “tail filename”. To see the first 10 lines, use “head filename”.
/var/mail/root
/var/spool/mail/root сыплются сообщения системы об ошибках и других событиях
tmux vs screen - terminal
http://www.dayid.org/os/notes/tm.html
time = The time utility executes and times the specified command.
# time ntpdate 0.europe.pool.ntp.org
11 May 11:54:21 ntpdate[30324]: adjust time server 213.198.55.2 offset 0.002878 sec
real 0m0.823s
user 0m0.000s
sys 0m0.002s
6) MySQL top
cd /usr/ports/databases/mtopsetup:
make install clean
For mysql 4.0.2 and greater:
mysql> grant super, reload, process on *.* to mysqltop;
mysql> grant super, reload, process on *.* to mysqltop@localhost;
mysql> flush privileges;
7)
Compare
diff - outputs the differences between two files
8) pdmenu - /usr/local/etc/pdmenurc
#!/usr/local/bin/pdmenu
# Sample menus for Pdmenu.
# Define the main menu.
menu:main:Main Menu
show:_Updates::updates
show:_Restart Services ::restart
show:_Monitoring ::monitoring
nop
show:_System Info (CPU, Mem) ::sysinfo
nop
exec:_Who's online?:truncate:echo "These users are online:";w -n
exec:_Change current user's password:pause:passwd
nop
exec:_Pdmenu edit menu::mcedit /usr/local/etc/pdmenurc
nop
exit:_Exit
menu:updates:Updates:Some regular updates
exec:_Ports Update::portsnap fetch update
exec:Ports First Time (_extract)::portsnap fetch extract
nop
exec:_FreeBSD fetch updates ::freebsd-update fetch
exec:_FreeBSD fetch install ::freebsd-update install
exec:_FreeBSD fetch rollback ::freebsd-update rollback
nop
exec:_NTPdate:pause:ntpdate 2.md.pool.ntp.org
nop
exit:_Main menu..
menu:restart:Restart Services:Restart some services
exec:_IPFW::/etc/rc.d/ipfw restart
exec:_Netif and routing::/etc/rc.d/netif restart && /etc/rc.d/routing restart
nop
exec:_FreeBSD fetch updates ::freebsd-update fetch
exec:_FreeBSD fetch install ::freebsd-update install
exec:_FreeBSD fetch rollback ::freebsd-update rollback
nop
nop
exit:_Main menu..
menu:monitoring:Monitoring:Some basic monitoring
exec:_Top::top
exec:_vmstat 1::vmstat 1
nop
nop
nop
exit:_Main menu..
menu:sysinfo:Sysinfo:Hardware system information
exec:_OS version :display,pause:sysinfo -c os
exec:_CPU :display,pause:sysinfo -c cpu
exec:_RAM :display,pause:sysinfo -c mem
exec:_Network :display,pause:sysinfo -c network
nop
exec:_All :display,pause:sysinfo -c -a
nop
nop
exit:_Main menu..
Info from:
http://216.147.18.102/unixfaq/explain_vm.shtml
http://www.thegeekstuff.com/2011/03/linux-performance-monitoring-intro/
Good blog!
ReplyDeletefreebsdguide.ru