CCNP Route - Path control, part 3: PBR - Policy-Based Routing and IP Service Level Agreement

route-map blah permit 10
 match ip address 100
 set ip default next-hop

ip next-hop
If next-hop exist in RIB => PBR, via specified next-hop
 if next-hop does not exist in RIB => via RIB

ip default next-hop
If the destination exist in RIB => not PBR, via RIB
 if destination does not exist in RIB => via specified next-hop

The difference between adding the default keyword is the order of operations that is used.
Without default, the router first attempts to route using PBR.  If there is no match or a match on a deny statement the router then uses the normal forwarding process.
With the default keyword, the router instead attempts to use the routing table first and if no route can be found then tries PBR.

set ip next-hop recursive
set interface Serial0/0.1 FastEthernet0/0 Serial0/0.2
set ip next-hop recursive allow multiple interfaces or next-hop addresses to be listed in a single set statement.  If the 1st interface (or interface associated with the next-hop address, if a next-hop is used) is down or L2 address is unknown, the rest of the interfaces/addresses in the list are tried in order.

The term path control has many fine shades of meaning, depending on the context.
The broadest typical use of the term refers to any and every function that influences where a router forwards a packet.
In other cases, the term path control refers to tools that influence the contents of the routing table, most typically referring to routing protocols.

Policy-Based Routing
When a packet arrives at the incoming interface of a router, the router’s data plane processing logic takes several steps to process the packet
 1) the router must check the incoming frame’s Frame Check Sequence (FCS) and discard the frame if errors occurred in transmission,
 2) If the FCS check passes, the router discards the incoming frame’s data link header and trailer, leaving the Layer 3 packet.
 3) the router does the equivalent of comparing the destination IP address of the packet with the IP routing table, matching the longest prefix route.

Note: Most routers today default to use Cisco Express Forwarding (CEF), which causes the router to match the destination of IP packets with the CEF table, instead of the IP routing table. IOS derives the CEF table from the information in the IP routing table, with much faster table lookup as compared with using the routing table directly.

Policy-Based Routing (PBR) overrides the router’s natural destination-based forwarding logic. PBR intercepts the packet after de-encapsulation on the incoming interface, before the router performs the CEF table lookup.
By default PBR is processed-switch( not CEF-switched) but you can fast-switch the PBR'd traffic which will give you better performance than process-switching.
PBR and IP-CEF doesnt work together in older IOS  (even applied correct PBR, packets matched with PBR are routed via RIB/CEF)
CEF PBR is enabled automatically
Older command can be used to manually allow fast-switched PBR (less efficient, not recommended)

PBR chooses how to forward the packet by using matching logic defined through a route map, which in turn typically refers an IP ACL and to the specific next-hop IP address or outgoing interface–for packets matched by the route map.

Route maps are complex access lists that allow some conditions to be tested against:
 - the packet  in PBR (to routed or through router) or 
 - the prefix in route-filtering (route to destination) in question using match commands.
Route-map configuration for packet and route will be different.

If the conditions match, some actions can be taken to modify attributes of the packet or route.
These actions are specified by set commands.

Basic config :
1. Define an ACL (whatever is permitted by ACL is what will be matched)
1. Create a route map with the logic to match packets, and choose the route.
2. Enable the route map for use with PBR, on an interface, for packets entering the interface (where the traffic is coming in.).

PBR work only on INCOMING DIRECTION, not outgoing.
All route maps have an implicit deny clause at the end that matches all packets not already matched by the route map. 
For packets matched by a deny clause, PBR lets the packet go through to the normal IP routing process.
Matching the Packet and Setting the Route

Route maps have the following common characteristics:
 - The default for the route-map command is permit, with a sequence-number of 10.
 - Route maps are executed in the order of the lowest sequence number to the highest. You can edit or modify maps by using the sequence number.
 - If a match is found within a route map instance, execution of further route map instances stops.
 - You can use route maps to permit or deny the information found true by the match statements.
 - If multiple match statements are called within a single route map instance, all match statements must match for the route map instance to yield a true result.
 - If route maps are applied in a policy-routing environment, packets that do not meet the match criteria are then forwarded according to the route table.
 - A route-map statement without any match statements will be considered matched.
   If there is no match statement in the route map instance, all routes and packets are matched. The set statement will apply to all routes or packets.
 - If there is not a corresponding ACL to the match statement in the route map instance, all routes are matched. The set statement, in turn, applies to all routes.
 - As with ACLs, an implicit deny is included at the end of the route map policy.

To match packets with a route map enabled for PBR:
! reference standard and extended ACLs
match ip address

! allows you to specify a range of lengths, in bytes
match length min max
Route-map logic example
route-map demo permit 10
    match x y z
    match a
    set b
    set c
route-map demo permit 20
    match q
    set r
route-map demo permit 30  

If {(x or y or z) and (a) match} then {set b and c}
If q matches then set r
Set nothing

When a route map clause (with a permit action) matches a packet, the set command defines the action to take regarding how to forward the packet.
! Next-hop addresses must be in a connected subnet; forwards to the first address in the list for which the associated interface is up.
set ip next-hop ip-address [. . . ip-address]
! Forwards packets using the first interface in the list that is up.
set interface interface-type interface-number[. . . interface-type interface-number]
! policy routing first attempts to route based on the routing table.
set ip default next-hop ip-address [. . . ip-address]
set default interface interface-type interface-number[. . . interface-type interface-number]
If the PBR route (as defined in the set command) is not working–because the outgoing interface is down or the next hop is unreachable using a connected route–then IOS next tries to route the packet using the normal destination-based IP routing process.

With default parameter: IOS matches the associated PBR route map permit clause, IOS applies the normal destination-based routing logic first, with one small exception: it ignores any default routes.

The set ip default next-hop command verifies the existence of the destination IP address in the routing table, and…
 - if the destination IP address exists, the command does not policy route the packet, but forwards the packet based on the routing table.
 - if the destination IP address does not exist, (instead of use default route or drop packet) the command policy routes the packet by sending it to the specified next hop.

The set ip next-hop command verifies the existence of the next hop specified, and…
 - if the next hop exists in the routing table, then the command policy routes the packet to the next hop.
 - if the next hop does not exist in the routing table, the command uses the normal routing table to forward the packet.

Policy-Based Routing Using the set ip default next-hop and set ip next-hop

PBR example: packets from PC2 ( are going to dest through S0/0/1, and not as routing table best through S0/0/0

interface Fastethernet 0/0
ip address
ip policy route-map PC2-over-low-route
route-map PC2-over-low-route permit
match ip address 101
set ip next-hop
access-list 101 permit ip host
show route map
show ip policy
debug ip policy
R1# show ip policy
Interface Route map
Fa0/0 PC2-over-low-route
R1# show route-map
route-map PC2-over-low-route, permit, sequence 10
Match clauses:
ip address (access-lists): 101
Set clauses:
ip next-hop
Policy routing matches: 12 packets, 720 bytes
R1# debug ip policy
*Sep 14 16:57:51.675: IP: s= (FastEthernet0/0), d=, len 28, policy match
*Sep 14 16:57:51.675: IP: route map PC2-over-low-route, item 10, permit
*Sep 14 16:57:51.675: IP: s= (FastEthernet0/0), d= (Serial0/0/1), len 28, policy routed
*Sep 14 16:57:51.675: IP: FastEthernet0/0 to Serial0/0/1

PBR Recursive Next Hop
The PBR Recursive Next Hop feature enhances route maps to enable configuration of a recursive next-hop IP address that is used by policy-based routing (PBR). The recursive next-hop IP address is installed in the routing table and can be a subnet that is not directly connected. If the recursive next-hop IP address is not available, packets are routed using a default route.
Because Cisco Express Forwarding (CEF) or process switching provides the infrastructure, the benefit of this feature is the CEF load sharing.
This feature was integrated into Cisco IOS Release 12.3(14)T.

Router# show running-config | beg abccomp
route-map abccomp permit 10
 match ip address 101 ! Defines the match criteria for an access list.
  ! If the match criteria is met, the recursive IP address is set.
 set ip next-hop recursive
 set ip next-hop

Setting the Recursive Next-Hop IP Address
The infrastructure provided by CEF or process switching performs the recursion to the next-hop IP address. The configuration sequence, which affects routing, is as follows:
1. Next-hop
2. Next-hop recursive
3. Interface
4. Default next-hop
5. Default interface
If both a next-hop and a recursive next-hop IP address are present in the same route-map entry, the next-hop is used. If the next-hop is not available, the recursive next-hop is used. If the recursive next-hop is not available and no other IP address is present, the packet is routed using the default routing table; it is not dropped. If the packet is supposed to be dropped, use the set next-hop recursive command followed by a set interface null0 configuration.
Perform this task to set the IP address for the recursive next-hop router.

Enabling Fast-Switched PBR
PBR can now be fast-switched. Prior to Cisco IOS Release 12.0, PBR could only be process-switched, which meant that on most platforms the switching rate was approximately 1000 to 10,000 packets per second.
This speed was not fast enough for many applications.
Users that need PBR to occur at faster speeds can now implement PBR without slowing down the router.

PBR must be configured before you configure fast-switched PBR. Fast switching of PBR is disabled by default.
To enable fast-switched PBR, use the following command in interface configuration mode:
Router(config-if)# ip route-cache policy

Enabling Local PBR
Packets that are generated by the router are not normally policy-routed.
To enable local PBR for such packets, indicate which route map the router should use by using the following command in global configuration mode:
Router(config)# ip local policy route-map map-tag

Display the route map used for local PBR, if one exists.
Router#  show ip local policy

Enabling CEF-Switched PBR
Beginning in Cisco IOS Release 12.0, PBR is supported in the Cisco Express Forwarding (CEF) switching path.
CEF-switched PBR has better performance than fast-switched PBR and, therefore, is the optimal way to perform PBR on a router.
No special configuration is required to enable CEF-switched PBR. It is on by default as soon as you enable CEF and PBR on the router.
Note The ip route-cache policy command is strictly for fast-switched PBR and, therefore, not required for CEF-switched PBR.

PBR Function for Routes manipulations
router bgp 64516
 neighbour x.x.x.x route-map Prepend out
route-map Prepend permit 10
  match ip address ACL (
  set as-path prepend 64516
route-map Prepend permit 20
 - route-maps used for PBR do not need a last empty clause because the implicit action in this case is normal destination based routing,
 - route-map used on a BGP session both inbound or outbound is a route filter that decides what routes can be advertised/received to/from the neighbor and allows also to perform some BGP attributes manipulation.
   In this case a last empty clause matching anything else is needed if all other routes have to be advertised/received,  because otherwise the filtering action blocks all routes not matching the previous route-map clauses.
   Also route-maps used for redistribution are route filters and have to be terminated accordingly.
When route-map is used for PBR: We don’t need the last command route-map pbr permit 20 to permit other traffic according to Cisco:

“If the packets do not meet any of the defined match criteria (that is, if the packets fall off the end of a route map), then those packets are routed through the normal destination-based routing process.
If it is desired not to revert to normal forwarding and to drop the packets that do not match the specified criteria, then interface Null 0 should be specified as the last interface in the list by using the set clause.”

Additional PBR Functions
Applying PBR to Locally Created Packets
In some cases, it may be useful to use PBR to process packets generated by the router itself.

ip local policy route-map name

Setting IP Precedence
Quality of service (QoS) refers to the entire process of how the network infrastructure can choose to apply different levels of service to different packets.
For example, a router may need to keep delay and jitter (delay variation) low for VoIP and Video over IP packets, because these interactive voice and video calls only work well when the delay and jitter are held very low. So, the router might let VoIP packets bypass a long queue of data packets that might be waiting to exit an interface, giving the voice packet better (lower) delay and jitter.

Most QoS designs mark each packet with a different value inside the IP header ( all VoIP packets could be marked with a particular value so that
the router can then find those marked bits, know that the packet is a VoIP packet due to that marking, and apply QoS accordingly).

Although the most commonly used QoS marking tool today is Class-Based Marking, in the past, PBR was one of the few tools that could be used for this important QoS function of marking packets. PBR still supports marking; however, most modern QoS designs ignore PBR’s marking capabilities.
The IP header originally defined a ToS.
Back in the 1990s, the ToS byte was redefined as the Differentiated Services (DS) byte, with the first six bits defined as the Differentiated Service Code Point (DSCP). Most QoS implementations today revolve around setting the DSCP field.
set ip precedence value
set ip tos value

PBR can also react to some dynamic measurements of the health of the IP network. To do so, PBR relies on the IP Service-Level Agreement (IP SLA) tool

IP Service-Level Agreement

IP SLA primarily acts as a tool to test and gather data about the network.
Many network management tools support the ability to configure IP SLA from the management tools’ graphical interfaces (using IOS RTTMON MIB).
Cisco Works Internetwork Performance Monitor (IPM) - can be used to configure and monitor the data collected using IP SLA operations.

The measurement can be as simple as using the equivalent of a ping to determine if an IP address responds, or as sophisticated as measuring the jitter (delay variation) of VoIP packets that flow over a particular path.

Running IP SLA can cause high CPU and memory utilization on the Routers.
IP SLA can give inaccurate results if the CPU utilization is very high on the IP SLA source device.
The Clocks needs to be Sync when running IP SLA on the Source and Responder devices.
You can deploy a shadow router in the network whose purpose is just to do the IP SLA so that the production routers do no get overloaded with the SLA operations.

Understanding IP SLA Concepts
IP SLA uses the concept of an operation - defines a type of packet that the router will generate, the destination and source address, and other characteristics of
the packet.
A single IP SLA operation could define the following:
■ Use ICMP echo packets.
■ Measure the end-to-end round trip response time (ICMP echo).
■ Send the packets every 5 minutes, all day long.

Cisco IOS IP SLAs collects a unique subset of the following performance metrics:
 - Delay (both round-trip and one-way)
 - Jitter (directional)
 - Packet loss (directional)
 - Packet sequencing (packet ordering)
 - Path (per hop)
 - Connectivity (directional)
 - Server or website download time
 - Voice quality scores

IP SLA has origins in earlier IOS features, including the Response Time Reporter (RTR)feature. The RTR feature uses the rtr command and uses the term probe to refer to what IP SLA refers to as an operation.

1) SLA operation sends packets to a host, the router can only use operation types that send packets that the host understands (ICMP echo requests, TCP connection requests, or even HTTP GET requests to a web server)
2) The operation can also send packets to another router, which gives IP SLA a wider range of possible operation types (configuring remote router as an IP SLA responder).
The following list summarizes the majority of the available operation types, just for perspective:
■ ICMP (echo, jitter)
■ RTP (VoIP)
■ TCP connection (establishes TCP connections)
■ UDP (echo, jitter)

Configuring and Verifying IP SLA
! Create IP SLA Operation
ip sla sla-ops-number
! Define the operation type and the parameters for that operation type
icmp-echo {destination-ipaddress | destination-hostname} [source-ip {ip-address | hostname} | source-interface interface-name]
! (Optional) Define a (nondefault) frequency that the operation should send the packets, in seconds
frequency seconds
! Schedule when the SLA will run (global command)
 ip sla schedule sla-ops-number [life{forever| seconds}] [start-time{hh:mm[:ss] [month day | day month] | pending| now| afterhh:mm:ss}] [ageoutseconds] [recurring]
! disable and delete IP SLA Operation
no ip sla sla-ops-number
show ip sla configuration
show ip sla statistics

R1 PBR tells - process packets generated by R1, including the IP SLA operation packets
R1# conf t
Enter configuration commands, one per line. End with CNTL/Z.
R1(config)# ip sla 11
R1(config-ip-sla)# icmp-echo source-ip
R1(config-ip-sla)# frequency 60
R1(config-ip-sla)# exit
R1(config)# ip sla schedule 11 start-time now life forever
! Changes to the PBR configuration below
R1(config)# access-list 101 permit ip host host
R1(config)# ip local policy PC2-over-low-route
R1(config)# end
Tracking SLA Operations to Influence Routing

You can configure both static routes and PBR to be used only when an SLA operation remains successful.
The configuration to achieve this logic requires the configuration of a tracking object and cross references between the static route, PBR, and IP SLA.
STATIC ROUTE   --->   Tracking OBJECT    <------  PBR
                      (SLA Operation)
The tracking object looks at the IP SLA operation’s most recent return code to then determine the tracking state as either “up” or “down.”
Depending on the type of SLA operation, the return code may be a simple toggle, with “OK” meaning that the last operation worked.
One of the main reasons that IOS requires the use of this tracking object is so that the routes do not flap.
Route flapping - when a router adds a route to the routing table, then quickly removes it; conditions change, so the router soon adds the route to the table
again; and so on.

Configuring Enhanced Object Tracking
You can configure a tracked list of objects with a Boolean expression, a weight threshold, or a percentage threshold. A tracked list contains one or more objects. An object must exist before it can be added to the tracked list.
• You configure a Boolean expression to specify calculation by using either “AND” or “OR” operators.
• When you measure the tracked list state by a weight threshold, you assign a weight number to each
object in the tracked list. The state of the tracked list is determined by whether or not the threshold
was met. The state of each object is determined by comparing the total weight of all objects against
a threshold weight for each object.
• When you measure the tracked list by a percentage threshold, you assign a percentage threshold to
all objects in the tracked list. The state of each object is determined by comparing the assigned
percentages of each object to the list.

Configuring a Static Route to Track an IP SLA Operation
track <object-number> ip sla <sla-ops-number> [state | reachability]
! (Optional)  Configure the delay to regulate flapping of the tracking state
delay {down seconds| up seconds}
ip route destination mask{interface | next-hop} track <object-number>
show track
track 2 ip sla 11 state
delay up 90 down 90
ip route s0/0/1 track 2

Configuring PBR to Track an IP SLA
When the tracking object is up, PBR works as configured. When the tracking object is down, PBR acts as if the set command does not exist.
That means that the router will still attempt to route the packet per the normal destination-based routing process.
set ip next-hop
set ip next-hop verify-availability 1 track 2

IP SLAs Threshold Monitoring and Notifications (link@cisco)

IP SLAs includes the capability for triggering SNMP notifications based on defined thresholds. This allows for proactive monitoring in an environment where IT departments can be alerted to potential network problems, rather than having to manually examine data.

IP SLAs supports threshold monitoring for performance parameters such as average jitter, unidirectional latency and bidirectional round trip time and connectivity. This proactive monitoring capability provides options for configuring reaction thresholds for important VoIP related parameters including unidirectional jitter, unidirectional packet loss, and unidirectional VoIP voice quality scoring (MOS scores).

IP SLAs can generate system logging (syslog) messages when the reaction threshold increases or decreases beyond the configured values for packet loss, average jitter, or MOS. These system logging messages can then be sent as SNMP notifications (traps) using the CISCO-SYSLOG-MIB.

In the following example, IP SLAs operation 10 (a UDP Jitter operation) is configured to send an SNMP logging trap when the MOS value exceeds 4.9 (best quality) of falls below 2.5 (poor quality):

Router(config)# ip sla monitor reaction-configuration 10 react mos threshold-type immediate threshold-value 490 250 action-type trapOnly 
IP SLAs ICMP Path Echo Operations

 - IP Service Level Agreements (SLAs) Internet Control Message Protocol (ICMP)Path Echo operation to monitor end-to-end and hop-by-hop response time between a Cisco device and other devices using IP.
 - ICMP Path Echo is useful for determining network availability and for troubleshooting network connectivity issues. The results of the ICMP Path Echo operation can be displayed and analyzed to determine how ICMP is performing.
 - Devices with supports RFC 862, Echo protocol, can be used.
 - Use the debug ip sla trace and debug ip sla error commands to help troubleshoot issues with an IP SLAs operation.

! configuration of the IP SLAs ICMP Path Echo operation number 7 that will start in 30 seconds and run for 5 minutes.
ip sla 7
 frequency 30
ip sla schedule 7 start-time after 00:00:30 life 300
show ip sla configuration
show ip sla summary

IP SLA Options
IP SLA Components
1. IP SLA Source (mandatory) , this is the source of the SLA traffic.
2. IP SLA Responder (optional)

IP SLA Responder is optional, it is a software component that runs on the Cisco router or switch. The responder will communicate with the IP SLA source using a separate control protocol. The IP SLA control protocol allows the source and destination to give you clean and most accurate result of the IP SLA test.

IP SLA Types
IP SLA Probes  can be generated  for testing  and analyzing the following

ICMP Path Jitter
IP SLA generates ICMP traffic and measures
a. Hop by Hop Jitter
b. Packet Loss
c. Total Delay

IP SLA generates ICMP traffic and measures Round Trip Delay for entire path

ICMP Path Echo
IP SLA generates ICMP traffic and measures Round Trip Delay by breaking it down on a hop by hop basis

UDP Jitter
IP SLA generates UDP traffic and measures
a. Round Trip Delay
b. One Way Delay
c. One Way Jitter
d. One Way packet loss

UDP Jitter For VoIP
IP SLA generates UDP traffic and measures
a. Round Trip Delay
b. One Way Delay
c. One Way Packet loss
d. One Way Jitter
e. Has the capability to simulate various codecs
f. Has the capability to do Voice quality Scoring

UDP Echo
IP SLA generates UDP traffic and measures Round Trip Delay for UDP Traffic.

IP SLA  measures the round trip time for the web page

IP SLA measures the round trip time for the file transfers

IP SLA measures the total time for the DHCP address acquisition

TCP Connect
IP SLA measures the time to connect to a destination using TCP

Example creating a TCP operation that does not require Cisco IOS IP SLAs responder, since the operation tries to perform a TCP connection to a well-defined port (http server).
The control disable command allows the TCP operation to be used without the IP SLAs responder.
control disable - Use only if you disable the IP SLAs control protocol on both the source and target routers.
By default IP SLA uses control protocol to setup the connection and the responder adds timestamps etc for increase accuracy of IP SLA. If the other side is not a responder then the router will try to resend control packets and will eventually fail.
(config)# ip sla 1
(config-ipsla)# tcp-connect 80 control disable
(config)# ip sla schedule 1 start-time now

Creating a TCP operation that requires a responder. The DSCP bits are also specified to measure QOS:
(config)# ip-sla 1
(config-ip-sla)# tcp-connect 8008
(config-ip-sla)# tos 4
(config)# ip-sla schedule 1 start-time now

IP SLA Reset
To perform a shutdown and restart of the Cisco IOS IP Service Level Agreements (SLAs) engine
ip sla monitor reset  ! up to Cisco IOS Release 12.4(4)T, 12.2(33)SB, and 12.2(33)SXI
ip sla reset

IP SLA Options
start-time {hh:mm[:ss] [month day | day month] | pending | now | after hh:mm:ss}
threshold <milliseconds>

start-time {hh:mm[:ss]
hh:mm[:ss] - Absolute start time, in 24-hour clock format with hours (hh), minutes (mm), and seconds (ss)
after hh:mm:ss - Specifies that start time is up to one 24-hour day after this command is configured

threshold <milliseconds>
Length of time required for a rising threshold to be declared, in milliseconds (ms).
Range is 0 to 60000. Default is 5000.

timeout (IP SLA) - <milliseconds>
Length of time the operation waits to receive a response from its request packet, in milliseconds (ms).
Range is 0 to 604800000.
We recommend that the value of the milliseconds argument be based on the sum of both the maximum round-trip time (RTT) value for the packets and the processing time of the IP SLAs operation.

For the IP SLAs User Datagram Protocol (UDP) jitter operation, the following guidelines are recommended:
 - (frequency seconds) > ((timeout milliseconds) + N)
 - (timeout milliseconds) > (threshold milliseconds)
where N = (num-packets number-of-packets) * (interval interpacket-interval)

For all other IP SLAs operations, the following configuration guideline is recommended:
(frequency seconds) > (timeout milliseconds) > (threshold milliseconds)