- configuration changes made on port-channel interface apply to ALL physical interfaces from port-channel,
- before being bundled, each of the physical ports from channel should have the same SPEED/DUPLEX/NATIVE-VLAN/VLAN- range/TRUNK status/TYPE.
- up to eight physical ports of the same Ethernet media type and speed,
- Generally, all bundled ports first must belong to the same VLAN
- If used as a trunk, bundled ports must be in trunking mode, have the same native VLAN, and pass the same set of VLANs.
- must be configured with identical STP settings.
- LACP does not support half-duplex. Half-duplex ports in an LACP EtherChannel are put in the suspended state.
- Configure all LAN ports in an EtherChannel to use the same EtherChannel protocol; you cannot run two EtherChannel protocols in one EtherChannel.
L2L3 etherchannel
- Layer 3 EtherChannels, assign Layer 3 addresses to the port channel logical interface, not to the LAN ports in the channel.
- Layer 2 EtherChannels, assign all LAN ports in the EtherChannel to the same VLAN or configure them as trunks.
Layer 3 EtherChannels, you must manually create the port-channel logical interface first and then put the Layer 3 LAN ports in to the channel group
Layer 2 EtherChannels, configure the LAN ports with the channel-group command, which automatically creates the port channel logical interface.
- You cannot put Layer 2 LAN ports into a manually created port channel interface.
- For Cisco IOS to create port channel interfaces for Layer 2 EtherChannels, the Layer 2 LAN ports must be connected and functioning.
Port channel Configuration
- After you configure an EtherChannel, configuration changes applied to the port-channel interface apply to all the physical ports assigned to the port-channel interface.
- Configuration changes applied to the physical port affect only the port where you apply the configuration.
- To change the parameters of all ports in an EtherChannel, apply configuration commands to the port-channel interface, for example, STP commands or commands to configure a Layer 2 EtherChannel as a trunk.
info: http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst3750/software/release/12-1_19_ea1/configuration/guide/3750scg/swethchl.pdf
Default EtherChannel Configuration
Feature Default Setting
Channel groups None assigned.
Port-channel logical interface None defined.
PAgP mode No default.
PAgP learn method Aggregate-port learning on all ports.
PAgP priority 128 on all ports.
LACP mode No default.
LACP learn method Aggregate-port learning on all ports.
LACP port priority 32768 on all ports.
LACP system priority 32768.
LACP system ID LACP system priority and the switch MAC address.
Load balancing Source-MAC address of the incoming packet.
L2 etherchannel config (catalyst 4500)
!
interface Port-channel45
switchport
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1,4,9
switchport mode trunk
switchport nonegotiate
!
interface GigabitEthernet7/6
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1,4,9
switchport mode trunk
switchport nonegotiate
speed 1000
duplex full
no cdp enable
channel-group 45 mode on
!
interface GigabitEthernet6/29
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1,4,9
switchport mode trunk
switchport nonegotiate
speed 1000
duplex full
no cdp enable
channel-group 45 mode on
!
L3 etherchannel config (Cisco 7600)
!
interface Port-channel46
ip address 10.1.1.1 255.255.255.252
!
interface TenGigabitEthernet7/1
no ip address
load-interval 30
no cdp enable
channel-group 12 mode on
!
interface TenGigabitEthernet7/2
no ip address
shutdown
no cdp enable
channel-group 12 mode on
!
Troubleshooting commands
show etherchannel summary EtherChannel status of each member port
show etherchannel port
show etherchannel port-channel Time stamps of EtherChannel changes
show etherchannel detail Detailed status about each EtherChannel component
show etherchannel load-balance Load-balancing hashing algorithm
show etherchannel port-channel Load-balancing port index used by hashing algorithm
show {pagp | lacp} neighbor EtherChannel neighbors on each port
show lacp sys-id LACP system ID
Cisco offers a method of scaling link bandwidth by aggregating, or bundling, parallel links, termed the EtherChannel technology.
Two to eight links of either are bundled as one logical link of Fast EtherChannel (FEC), Gigabit EtherChannel (GEC), or 10-Gigabit Etherchannel (10GEC):
- Fast Ethernet (FE) full-duplex bandwidth of up to 1600 Mbps (8x100Mbps down + 8x100Mbps up),
- Gigabit Ethernet (GE) max 16 Gbps,
- 10-Gigabit Ethernet (10GE) max 160 Gbps.
Ordinarily, having multiple or parallel links between switches creates the possibility of bridging loops, an undesirable condition.
EtherChannel avoids this situation by bundling parallel links into a single, logical link, which can act as either an access or a trunk link.
Although an EtherChannel link is seen as a single logical link, the link doesn’t necessarily have an inherent total bandwidth equal to the sum of its component physical links. (FEC max is 800 Mbps) the single resulting FEC bundle does not operate at this speed.
Instead, traffic is distributed across the individual links within the EtherChannel.
If one link within the bundle is favored by the load-distribution algorithm, that link will carry a disproportionate amount of traffic. In other words, the load isn’t always distributed equally among the individual links.
EtherChannel also provides redundancy with several bundled physical links. If one of the links within the bundle fails, traffic sent through that link automatically is moved to an adjacent link. Failover occurs in less than a few milliseconds and is transparent to the end user.
Distributing Traffic in EtherChannel
Load is not necessarily balanced equally across all the links
Frames are forwarded on a specific link as a result of a hashing algorithm.
The hash algorithm computes a binary pattern that selects a link number in the bundle to carry each frame.
The algorithm can use:
- src/dst IP, or a combination of both
- src/dst MAC,
- TCP/UDP port numbers.
If two addresses or port numbers are hashed, a switch performs an exclusive-OR (XOR) operation on one or more low-order bits of the addresses or TCP/UDP port numbers as an index into the bundled links.
EtherChannel consisting of
- two links bundled together requires a 1-bit index (2^1=2 links). If the index is 0, link 0 is selected; if the index is 1, link 1 is used.
- four-link bundle uses a hash of the last 2 bits (2^2=4).
- eight-link bundle uses a hash of the last 3 bits (2^3=8).
Example
As an example, consider a packet being sent from IP address 192.168.1.1 to 172.31.67.46.
- Because EtherChannels can be built from two to eight individual links, only the rightmost (least-significant) 3 bits are needed as a link index.
- From the source and destination addresses, these bits are 001 (1) and 110 (6)
- a two-link EtherChannel, a 1-bit XOR is performed on the rightmost address bit: 1 XOR 0 = 1, causing Link 1 in the bundle to be used.
- a four-link EtherChannel produces a 2-bit XOR: 01 XOR 10 = 11, causing Link 3 in the bundle to be used.
- an eight-link EtherChannel requires a 3-bit XOR: 001 XOR 110 = 111, where Link 7 in the bundle is selected.
A conversation between two devices always is sent through the same EtherChannel link because the two endpoint addresses stay the same.
However, when a device talks to several other devices, chances are that the destination addresses are distributed equally with 0s and 1s in the last bit (even and odd address values).
Load imbalance - if one pair of hosts has a much greater volume of traffic than the other pair, one link in the channel will be used much more than the other.
To remedy this condition, you should consider other methods of hashing algorithms for the channel. For example, a method that combines the source and destination addresses along with UDP or TCP port numbers in a single XOR operation can distribute traffic much differently.
Switch(config)# port-channel load-balance <method>
method Hash Switch ModelThe default configuration is to use source XOR destination IP addresses, or the src-dst-ip method.
src-ip bits All models
dst-ip bits All models
src-dst-ip XOR All models
src-mac bits All models
dst-mac bits All models
src-dst-mac XOR All models
src-port bits 6500, 4500
dst-port bits 6500, 4500
src-dst-port XOR 6500, 4500
The default for the Catalyst 2970 and 3560 is src-mac for Layer 2 switching.
If Layer 3 switching is used on the EtherChannel, the src-dst-ip method will always be used, even though it is not configurable.
SW-4500#show etherchannel port-channel* If most of the traffic is IP, it might make sense to load balance according to IP addresses or TCP/UDP port numbers.
Channel-group listing:
----------------------
Group: 45
----------
Port-channels in the group:
---------------------------
Port-channel: Po45
------------
Age of the Port-channel = 49d:17h:02m:47s
Logical slot/port = 11/45 Number of ports = 2
GC = 0x00000000
Port state = Port-channel Ag-Inuse
Protocol = -
Ports in the Port-channel:
Index Load Port EC state No of bits
------+------+------+------------------+-----------
0 00 Gi6/29 On/FEC 0
1 00 Gi7/6 On/FEC 0 <---- Although this information is not intuitive, you can use the hex values to get an idea of each link’s traffic loads relative to the others.
Time since last port bundled: 49d:17h:02m:47s Gi7/6
Time since last port Un-bundled: 49d:17h:02m:47s Gi7/6
For non-IP frames: If a frame can’t meet the load-balancing criteria, the switch automatically falls back to the “next lowest” method. With Ethernet, MAC addresses must always be present, so the switch distributes those frames according to their MAC addresses.
A switch also provides some inherent protection against bridging loops with EtherChannels.
When ports are bundled into an EtherChannel, no inbound (received) broadcasts and multicasts are sent back out over any of the remaining ports in the channel.
Outbound broadcast and multicast frames are load-balanced like any other.
EtherChannel Negotiation Protocols
EtherChannels can be negotiated between two switches to provide some dynamic link configuration. (PAgP and LACP)
Mode Negotiation Packets Sent? CharacteristicsBoth the active and passive LACP modes enable ports to negotiate with partner ports to an EtherChannel based on criteria such as port speed and, for Layer 2 EtherChannels, trunking state and VLAN numbers.
PAgP LACP
On On No All ports channeling
Auto Passive Yes Waits to channel until asked
Desirable Active Yes Actively asks to form a channel
Port Aggregation Protocol
The Port Aggregation Protocol (PAgP) is a Cisco-proprietary protocol that can be run only on Cisco switches and on those switches licensed by vendors to support PAgP.
PAgP can be configured in active mode (desirable), in which a switch actively asks a far end switch to negotiate an EtherChannel, or in passive mode (auto, the default), in which a switch negotiates an EtherChannel only if the far end initiates it.
PAgP (Port Aggregation Protocol) - Cisco proprietary protocol for automatic EtherChannel configuration and negotiation between switches.
PAgP forms an EtherChannel only on ports that are configured for either identical static VLANs or trunking.
PAgP also dynamically modifies parameters of the EtherChannel if one of the bundled ports is modified.
PAgP Interaction with Other Features
DTP and CDP send and receive packets over the physical ports in the EtherChannel.
Trunk ports send and receive PAgP protocol data units (PDUs) on the lowest numbered VLAN.
- In Layer 2 EtherChannels, the first port in the channel that comes up provides its MAC address to the EtherChannel. If this port is removed from the bundle, one of the remaining ports in the bundle provides its MAC address to the EtherChannel.
- For Layer 3 EtherChannels, the MAC address is allocated by the stack master as soon as the interface is created (through the interface port-channelglobal configuration command).
- PAgP sends and receives PAgP PDUs only from ports that are up and have PAgP enabled for the auto or desirable mode.
Link Aggregation Control Protocol
LACP is a standards-based alternative to PAgP, defined in IEEE 802.3ad.
LACP packets are exchanged between switches over EtherChannel-capable ports. LACP also assigns roles to the EtherChannel’s endpoints.
The switch with the lowest system priority (a 2-byte priority value followed by a 6-byte switch MAC address) is allowed to make decisions about what ports actively are participating in the EtherChannel at a given time.
Ports are selected and become active according to their port priority value (a 2-byte priority followed by a 2-byte port number), where a low value indicates a higher priority.
A set of up to 16 potential links can be defined for each EtherChannel. Through LACP, a switch selects up to eight of these having the lowest port priorities as active EtherChannel links at any given time. The other links are placed in a standby state and will be enabled in the EtherChannel if one of the active links goes down.
Like PAgP, LACP can be configured in active mode (active), in which a switch actively asks a far-end switch to negotiate an EtherChannel, or in passive mode (passive), in which a switch negotiates an EtherChannel only if the far end initiates it.
LACP Interaction with Other Features
The DTP and the CDP send and receive packets over the physical ports in the EtherChannel.
Trunk ports send and receive LACP PDUs on the lowest numbered VLAN.
- In Layer 2 EtherChannels, the first port in the channel that comes up provides its MAC address to the EtherChannel. If this port is removed from the bundle, one of the remaining ports in the bundle provides its MAC address to the EtherChannel.
- For Layer 3 EtherChannels, the MAC address is allocated by the stack master as soon as the interface is created through the interface port-channelglobal configuration command.
- LACP sends and receives LACP PDUsonly from ports that are up and have LACP enabled for the active or passive mode.
EtherChannel Configuration
PAgP
To configure switch ports for PAgP negotiation (the default)
Switch(config)# interface <type mod/num>On all Cisco IOS–based Catalyst models, you can select between PAgP and LACP as a channel-negotiation protocol.
Switch(config-if)# channel-protocol pagp
Switch(config-if)# channel-group <number> mode {on | {{auto | desirable} [non-silent]}}
Some older models, however, offer only PAgP, so the channel-protocol command is not available.
The interfaces are not assigned to channel groups until you configure them manually.
By default, PAgP operates in silent submode with the desirable and auto modes, and allows ports to be added to an EtherChannel even if the other end of the link is silent and never transmits PAgP packets.
- If you expect a PAgP-capable switch to be on the far end, you should add the non-silent keyword to the desirable or auto mode.
- The non-silent keyword is always used with the auto or desirable mode.
- This requires each port to receive PAgP packets before adding them to a channel. If PAgP isn’t heard on an active port, the port remains in
the up state, but PAgP reports to the Spanning Tree Protocol (STP) that the port is down.
- If you do not specify non-silent with the auto or desirable mode, silent is assumed.
- The silent setting is for connections to file servers or packet analyzers; this setting enables PAgP to operate, to attach the interface to a channel group, and to use the interface for transmission.
Tip: In practice, you might notice a (15-second) delay from the time the links in a channel group are connected until the time the channel is formed and data can pass over it.
Even if the two interfaces are using PAgP auto mode, the link will still eventually come up, although not as a channel. You might notice that the total delay before data can pass over the link is actually approximately 45 or 50 seconds.
The first 15 seconds are the result of PAgP silent mode waiting to hear inbound PAgP messages, and the final 30 seconds are the result of the STP moving through the listening and learning stages.
example: EtherChannel load-balancing hash of both source and destination port numbers, switch actively negotiating a channel
Switch(config)# port-channel load-balance src-dst-port
Switch(config)# interface range gig 3/1 – 4
Switch(config-if)# channel-protocol pagp
Switch(config-if)# channel-group 1 mode desirable non-silent
LACP
To configure switch ports for LACP negotiation
Switch(config)# lacp system-priority <priority>- First, the switch should have its LACP system priority defined (1 to 65,535; default 32,768).
Switch(config)# interface <type mod/num>
Switch(config-if)# channel-protocol lacp
Switch(config-if)# channel-group <number> mode {on | passive | active}
Switch(config-if)# lacp port-priority <priority> on Enable Etherchannel only
active Enable LACP unconditionally (it will start to send LACP packets)
passive Enable LACP only if a LACP device is detected (it will wait until a LACP packet will arrive)
desirable Enable PAgP unconditionally (it will start to send PAgP packets)
auto Enable PAgP only if a PAgP device is detected (it will wait until a PAgP packet will arrive)
If not changed both switches will have the same system priority (32,768), and the one with the lower MAC address will become the decision maker.
- lacp port-priority - configure a lower port priority (1 to 65,535; default32,768) for any interfaces that must be active, and a higher priority for interfaces that
might be held in the standby state. This prepares extra standby interfaces to replace failed active ones.
Example configuration
Switch(config)# lacp system-priority 100Troubleshooting an EtherChannel
Switch(config)# interface range gig 2/1 – 4 , gig 3/1 – 4
Switch(config-if)# channel-protocol lacp
Switch(config-if)# channel-group 1 mode active
Switch(config-if)# lacp port-priority 100
Switch(config-if)# exit
Switch(config)# interface range gig 2/5 – 8 , gig 3/5 – 8 <--- default 32768 lacp port-priority, so this ports will be held as stanby interfaces
Switch(config-if)# channel-protocol lacp
Switch(config-if)# channel-group 1 mode active
show etherchannel summary
show etherchannel port
show running-config interface <type mod/ num>
show interface <type mod/num> etherchannel
show etherchannel load-balance
The whole concept is based on consistent configurations on both ends of the channel.
- EtherChannel on mode does not send or receive PAgP or LACP packets.
- EtherChannel desirable (PAgP) or active (LACP) mode attempts to ask the far end to bring up a channel. Therefore, the other end must be set to either desirable or
auto mode.
- EtherChannel auto (PAgP) or passive (LACP) mode participates in the channel protocol, but only if the far end asks for participation. Therefore, two switches in the auto or passive mode will not form an EtherChannel.
- PAgP desirable and auto modes default to the silent submode, in which no PAgP packets are expected from the far end. If ports are set to nonsilent submode, PAgP
packets must be received before a channel will form.
Verify the EtherChannel state:
Cisco7600#show etherchannel summaryVerify the channel negotiation mode:
Flags: D - down P - bundled in port-channel
I - stand-alone s - suspended
H - Hot-standby (LACP only)
R - Layer3 S - Layer2
U - in use f - failed to allocate aggregator
M - not in use, minimum links not met
u - unsuitable for bundling
w - waiting to be aggregated
d - default port
Number of channel-groups in use: 14
Number of aggregators: 14
Group Port-channel Protocol Ports
------+-------------+-----------+-----------------------------------------------
11 Po11(RU) - Te4/2(P) Te7/4(P)
12 Po12(RU) - Te4/1(P) Te7/1(P)
Cisco7600# show etherchannel port
Channel-group listing:
-----------------------
Group: 11
----------
Ports in the group:
-------------------
Port: Te4/2
------------
Port state = Up Mstr In-Bndl
Channel group = 11 Mode = On Gcchange = -
Port-channel = Po11 GC = - Pseudo port-channel = Po11
Port index = 0 Load = 0x55 Protocol = -
Mode = LACP
Age of the port in the current state: 170d:20h:08m:43s
Port: Te7/4
------------
Port state = Up Mstr In-Bndl
Channel group = 11 Mode = On Gcchange = -
Port-channel = Po11 GC = - Pseudo port-channel = Po11
Port index = 1 Load = 0xAA Protocol = -
Mode = LACP
Age of the port in the current state: 157d:20h:46m:30s
Cisco7600#sh run int Te4/2
interface TenGigabitEthernet4/2
no ip address
load-interval 30
channel-group 11 mode on
Cisco7600#sh int Te4/2 etherchannel
Port state = Up Mstr In-Bndl
Channel group = 11 Mode = On Gcchange = -
Port-channel = Po11 GC = - Pseudo port-channel = Po11
Port index = 0 Load = 0x55 Protocol = -
Mode = LACP
Age of the port in the current state: 170d:20h:12m:18s
Cisco7600# show etherchannel load-balance
EtherChannel Load-Balancing Configuration:
dst-ip
mpls label-ip
EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Destination MAC address
IPv4: Destination IP address
IPv6: Destination IP address
MPLS: Label or IP
Some messages from the switch might look like errors but are part of the normal EtherChannel process. For example, as a new port is configured as a member of an existing EtherChannel, you might see this message:
4d00h: %EC-5-L3DONTBNDL2: FastEthernet0/2 suspended: incompatible partner port with FastEthernet0/1When the port first is added to the EtherChannel, it is incompatible because the STP runs on the channel and the new port. After STP takes the new port through its progression of states, the port is automatically added into the EtherChannel.
Other messages do indicate a port-compatibility error (different duplex mode than the other ports in the EtherChannel)
4d00h: %EC-5-CANNOT_BUNDLE2: FastEthernet0/3 is not compatible with FastEthernet0/1 and will be suspended (duplex of Fa0/3 is full, Fa0/1 is half)