CCNP Tshoot - Basic Cisco Catalyst Switch Troubleshooting

Understanding of Ethernet switch operation at Layer 2 is critical to troubleshooting many issues.

 - To improve availability, many enterprise networks interconnect Layer 2 switches with redundant connections.
 - A redundant topology, however, can result in Layer 2 loops, which can cause frames to endlessly circle a LAN
 - STP is frequently used to logically break these Layer 2 topological loops by strategically blocking ports, while being able to detect a link failure and bring up a previously blocked switch port to restore connectivity.

VLAN Troubleshooting
To effectively troubleshoot Ethernet-based LAN environments, you should be familiar with the basics of Layer 2 switch operation.
 - Ethernet hubs, which take bits in one port and send those same bits out all other ports.
 - Ethernet switches learn the devices connected to their ports (more efficient bandwidth utilization on a LAN and eliminates the concern of collisions).

Troubleshooting an issue wheere communication is limited to a common VLAN, you might consider possibilities such as the following:
■ Hardware issues:Potential hardware problems include the cabling interconnecting devices and the devices themselves.
■ VLAN configuration:For traffic to move from one VLAN to another, that traffic must be routed.
■ Trunk configuration:Switches are often interconnected via an Ethernet trunk (for example, an IEEE 802.1Q trunk). 

 - The Address Resolution Protocol (ARP) is a protocol used for resolution of network layer addresses (IP) into link layer addresses (MAC).
 - ARP communicated within the boundaries of a single network (broadcast), never routed across internetwork nodes.
 - This property places ARP into the Link Layer of the Internet Protocol Suite,while in the Open Systems Interconnection (OSI) model, it is often described as residing between Layers 2 and 3, being encapsulated by Layer 2 protocols.
 - Cisco: Entries remain in the ARP cache for 14,400 seconds (4 hours).
 - ARP request is a broadcast, its destination MAC address is FFFF.FFFF.FFFF.
 - The EtherType for ARP is 0x0806.

ARP request
 - ARP probe - an ARP request constructed with an all-zero sender IP address and used in the IPv4 Address Conflict Detection specification (RFC 5227). Before beginning to use an IPv4 address (whether received from manual configuration, DHCP, or some other means), a host implementing this specification must test to see if the address is already in use, by broadcasting ARP probe packets.
 - ARP announcements (also called a gratuitous ARP message) - useful for updating other hosts mapping of a hardware address when the senders IP address or MAC address has changed, broadcasting as an ARP request containing the senders protocol address (SPA) in the target field (TPA=SPA), with the target hardware address (THA) set to zero. An ARP announcement is not intended to solicit a reply; instead it updates any cached entries in the ARP tables of other hosts that receive the packet. Many operating systems perform gratuitous ARP during startup.
 - An ARP proxy is a system which answers the ARP request on behalf of another system for which it will forward traffic, normally as a part of the network's design, such as for a dialup internet service.
 - ARP spoofing the answering system, or spoofer, replies to a request for another system's address with the aim of intercepting data bound for that system.

Gratious ARP
Layer 2 Troubleshooting Techniques
 - Examine the first switch along the path of a frame. If that switch has not learned the MAC address of the sender, you might need to check hardware and software upstream from the switch.
 - Perhaps the first-hop switch did learn the MAC address of the sender, but that MAC address was learned on an inappropriate VLAN (check the VLAN configuration of the switch).
 -  Alternatively, the switch might have learned the sender’s MAC address but not on the port that connects to the sender (duplicate MAC address).

Cisco Catalyst Switch Troubleshooting Commands
clear mac address-table dynamic (older version:  clear mac-address-table)
Clears dynamically learned MAC addresses from a switch’s MAC address table; this can help a troubleshooter determine if a previously learned MAC address is relearned.

show mac address-table (on some Catalysts: show mac-address-table)
Displays MAC addresses learned by a switch along with each associated port and VLAN of the MAC address

show vlan
Shows to which VLANs the ports of a switch belong

show interfaces trunk
Displays which VLANs are permitted on the trunk ports of a switch and which switch ports are configured as trunks

show interfaces switchport 
Displays summary information for the ports on a switch, including VLAN and trunk configuration information

traceroute mac <source_MAC_address>  <destination_MAC_address>
Uses CDP information to produce a list of switches to be transited for traffic traveling from a specified source MAC address to a specified destination MAC address.

Spanning Tree Protocol Troubleshooting
Administrators of corporate telephone networks often boast about their telephone system (that is, a PBX system) having the five nines (99.999) of availability - 99.999 percent of the time, which translates to about five minutes of downtime per year.

BPDU packets contain information on ports, addresses, priorities, and costs and ensure that the data ends up where it was intended to go.
BPDU messages are exchanged across bridges to detect loops in a network topology.
The loops are then removed by shutting down selected bridge interfaces and placing redundant switch ports in a backup, or blocked, state.

Switches in an STP topology
 - Root bridge:The root bridge is a switch elected to act as a reference point for a spanning tree. The switch with the lowest bridge ID (BID) is elected as the root
bridge. The BID is made up of a priority value and a MAC address.
 - Nonroot bridge:All other switches in the STP topology are considered nonroot bridges.

STP Port Types
 - Root port  - Every nonroot bridge has a single root port, which is the port on that switch that is closest to the root bridge, in terms of cost.
 - Designated port - Every network segment has a single designated port, which is the port on that segment that is closest to the root bridge, in terms of cost.
Therefore, all ports on a root bridge are designated ports.
 - Nondesignated port - block traffic to create a loop-free topology.

Port states
 - Blocking: The port remains in the blocking state for 20 seconds by default. During this time the nondesignated port evaluates BPDUs in an attempt to determine its role in the spanning tree.
 - Listening: The port moves from the blocking state to the listening state and remains in this state for 15 seconds by default. During this time, the port sources BPDUs, which inform adjacent switches of the port’s intent to forward data.
 - Learning: The port moves from the listening state to the learning state and remains in this state for 15 seconds by default. During this time, the port begins to add entries to its MAC address table.
 - Forwarding: The port moves from the learning state to the forwarding state and begins to forward frames.

Display information about the STP state of a switch
show spanning-tree [vlan <vlan_id>]
Show information contained in BPDUs
Switch#sh spanning-tree interface FastEthernet0/3 detail
 Port 3 (FastEthernet0/3) of VLAN0001 is designated forwarding
   Port path cost 19, Port priority 128, Port Identifier 128.3.
   Designated root has priority 32769, address 000f.233b.8a80
   Designated bridge has priority 32769, address 0016.c831.9000
   Designated port id is 128.3, designated path cost 19
   Timers: message age 0, forward delay 0, hold 0
   Number of transitions to forwarding state: 1
   Link type is point-to-point by default
   BPDU: sent 416, received 1
STP Troubleshooting Issues
If STP fails to operate correctly, Layer 2 frames can endlessly circulate through a network.

Corruption of a Switch’s MAC Address Table - in the event of an STP failure, the MAC address table of a switch can become corrupted
Broadcast Storms -  broadcast frame copies continue to loop through the network.

Troubleshooting EtherChannel
- An exception to STP operation can be made if two switches are interconnected via multiple physical links and those links are configured as an EtherChannel.
- An EtherChannel logically combines the bandwidth of multiple physical interfaces into a logical connection between switches

Troubleshooting an EtherChannel issue:
 - Mismatched port configurations: The configurations of all ports making up an EtherChannel, on both switches, should be identical. For example, all ports should
have the same speed, duplex, trunk mode, and native VLAN configurations.
 - Mismatched EtherChannel configuration: Both switches forming the EtherChannel should be configured for the same EtherChannel negotiation protocol (LACP or PAgP).
- Inappropriate EtherChannel distribution algorithm: EtherChannel determines which physical link to use to transmit frames based on a hash calculation. The hashing approach selected should distribute the load fairly evenly across all physical links (load distribution could be uneven).

Trouble Ticket: STP
 - All the trouble tickets are based on the same basic network topology, although addressing and links might vary for some trouble tickets.
 - All trouble tickets begin with a problem report and a network topology diagram. 
 - Some of TTs provide you with baseline data, and all the trouble tickets offer output from appropriate verification commands (show/debug).

Example commands checked"
SW1# show spanning-tree vlan 1
Gi0/8 Desg FWD 19 128.8 P2p
Gi0/9 Root FWD 19 128.9 P2p
Gi0/10 Altn BLK 100 128.10 Shr

SW1# show spanning-tree summary
Switch is in pvst mode
SW1# show spanning-tree interface gig 0/10 detail
Port 10 (GigabitEthernet0/10) of VLAN0001 is alternate blocking

00:15:45: %SW_MATM-4-MACFLAP_NOTIF: Host 0009.b7fa.d1e1 in vlan 1 is flapping between port Gi0/8 and port Gi0/9
%SW_MATM-4-MACFLAP_NOTIF console message appearing on switch SW1 indicates that the port of a MAC address in the MAC address table of switch SW1 is flapping between a couple of ports.

Solution - Enabling STP for VLAN 1 on Switch SW1/SW2
SW1(config)# spanning-tree vlan 1
SW2(config)# spanning-tree vlan 1