wdiff draft-ietf-grow-bgp-gshut-01.txt draft-ietf-grow-bgp-gshut-02.txt

Network Working Group Pierre Francois
Internet-Draft Universite catholique de Louvain
Intended status: Informational Bruno Decraene
Expires: April 29, 2010 28, 2011 France Telecom
Cristel Pelsser
Internet Initiative Japan
Keyur Patel
Clarence Filsfils
Cisco Systems
October 26, 2009 25, 2010

Graceful BGP session shutdown
draft-ietf-grow-bgp-gshut-01
draft-ietf-grow-bgp-gshut-02

Abstract

This draft describes operational procedures aimed at reducing the
amount of traffic lost during planned maintenances of routers,
involving the shutdown of BGP peering sessions.

Status of this Memo

This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79. This document may contain material
from IETF Documents or IETF Contributions published or made publicly
available before November 10, 2008. The person(s) controlling the
copyright in some of this material may not have granted the IETF
Trust the right to allow modifications of such material outside the
IETF Standards Process. Without obtaining an adequate license from
the person(s) controlling the copyright in such materials, this
document may not be modified outside the IETF Standards Process, and
derivative works of it may not be created outside the IETF Standards
Process, except to format it for publication as an RFC or to
translate it into languages other than English.

Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.

Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."

The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.

The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.

This Internet-Draft will expire on April 29, 2010. 28, 2011.

This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document (http://trustee.ietf.org/license-info). document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document.

Abstract Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the BSD License.

This draft describes operational procedures aimed at reducing document may contain material from IETF Documents or IETF
Contributions published or made publicly available before November
10, 2008. The person(s) controlling the
amount of traffic lost during planned maintenances copyright in some of routers,
involving this
material may not have granted the shutdown IETF Trust the right to allow
modifications of BGP peering sessions.

Table such material outside the IETF Standards Process.
Without obtaining an adequate license from the person(s) controlling
the copyright in such materials, this document may not be modified
outside the IETF Standards Process, and derivative works of Contents

1. it may
not be created outside the IETF Standards Process, except to format
it for publication as an RFC or to translate it into languages other
than English.

Table of Contents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 3
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 3
3. Packet loss upon manual eBGP session shutdown . . . . . . . . 5 4
4. Practices to avoid packet losses . . . . . . . . . . . . . . . 5 4
4.1. Improving availability of alternate paths . . . . . . . . 6 5
4.2. Graceful shutdown procedures for eBGP sessions . . . . . . 6 5
4.2.1. Outbound traffic . . . . . . . . . . . . . . . . . . . 6 5
4.2.2. Inbound traffic . . . . . . . . . . . . . . . . . . . 7 6
4.2.3. Summary of operations . . . . . . . . . . . . . . . . 8
4.2.4. BGP implementation support for G-Shut . . . . . . . . 9
4.3. Graceful shutdown procedures for iBGP sessions . . . . . . 9
5. Forwarding modes and forwarding loops . . . . . . . . . . . . 10
6. Dealing with Internet policies . . . . . . . . . . . . . . . . 10
7. Link Up cases . . . . . . . . . . . . . . . . . . . . . . . . 11
7.1. Unreachability local to the ASBR . . . . . . . . . . . . . 11
7.2. iBGP convergence . . . . . . . . . . . . . . . . . . . . . 11
8. IANA considerations . . . . . . . . . . . . . . . . . . . . . 12
9. Security Considerations . . . . . . . . . . . . . . . . . . . 12
10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 13
11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Appendix A. Summary of operations . . . . . . . . . . . . . . . . 13
A.1. Pre-configuration . . . . . . . . . . . . . . . . . . . . 13
A.2. Operations at maintenance time . . . . . . . . . . . . . . 14
Appendix B. Alternative techniques with limited applicability . . 14
B.1.
A.1. In-filter reconfiguration . . . . . . . . . . . . . . . . 14
B.2.
A.2. Multi Exit Discriminator tweaking . . . . . . . . . . . . 15
B.3.
A.3. IGP distance Poisoning . . . . . . . . . . . . . . . . . . 16
Appendix C. Effect of the g-shut procedure on the convergence . . 16
C.1. Maintenance of an eBGP session . . . . . . . . . 15
Authors' Addresses . . . . . 16
C.1.1. Propagation on the other eBGP sessions of the
g-shut initiator . . . . . . . . . . . . . . . . . . . 16
C.1.2. Propagation on 15

1. Introduction

Routing changes in BGP can be caused by planned, manual, maintenance
operations. This document discusses operational procedures to be
applied in order to reduce or eliminate losses of packets during the other iBGP sessions
maintenance. These losses come from the transient lack of
reachability during the
g-shut initiator . . . . . . . . . . . . . . . . . . . 16
C.1.3. Propagation of updates in an iBGP full-mesh . . . . . 17
C.1.4. Propagation of updates from iBGP to iBGP in a RR
hierarchy . . . . . . . . . . . . . . . . . . . . . . 17
C.2. Maintenance of an iBGP session . . . . . . . . . . . . . . 18
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 18

1. Introduction

This document presents procedures for the cases where the forwarding
plane is impacted by the maintenance, hence when the use of Graceful
Restart does not apply.

The procedures described in this document can be applied to reduce or
avoid packet loss for outbound and inbound traffic flows initially
forwarded along the peering link to be shut down. These procedures
allow routers to keep using old paths until alternate ones are
learned, ensuring that routers always have a valid route available
during the convergence process.

The goal of the document is to meet the requirements described in
[REQS] at best, without changing the BGP protocol or BGP
implementations.

Still, it explains why reserving a community value for the purpose of
BGP session graceful shutdown would reduce the management overhead
bound with the solution. It would also allow vendors to provide an
automatic graceful shutdown mechanism that does not require any
router reconfiguration at maintenance time.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].

2. Terminology

g-shut initiator : a router on which the session shutdown is
performed for the maintenance.

g-shut neighbor : a router that peers with the g-shut initiator via
(one of) the session(s) to be shut down.

Note that for the link-up case, we will refer to these nodes as g-no-
shut initiator, and g-no-shut neighbor.

Initiator AS : the Autonomous System of the g-shut initiator.

Neighbor AS : the Autonomous System of the g-shut neighbor.

Affected path / Nominal / pre-convergence path : a BGP path via the
peering link(s) undergoing the maintenance. This path will no longer
exist after the shutdown.

Affected prefix : a prefix initially reached via an affected path.

Affected router : a router having an affected prefix.

Backup / alternate / post-convergence path : a path towards an
affected prefix that will be selected as the best path by an affected
router, when the link is shut down and the BGP convergence is
completed.

Transient alternate path : a path towards an affected prefix that may
be transiently selected as best by an affected router during the
convergence process but that is not a post-convergence path.

Loss of Connectivity (LoC) : the state when a router has no path
towards an affected prefix.

3. Packet loss upon manual eBGP session shutdown

Packets can be lost during a manual shutdown of an eBGP session for
two reasons.

First, routers involved in the convergence process can transiently
lack of paths towards an affected prefix, and drop traffic destined
to this prefix. This is because alternate paths can be hidden by
nodes of an AS. This happens when the paths are not selected as best
by the ASBR that receive them on an eBGP session, or by Route
Reflectors that do not propagate them further in the iBGP topology
because they do not select them as best.

Second, within the AS, routers' the FIB of routers can be transiently
inconsistent during the BGP convergence and packets towards affected
prefixes can loop and be dropped. Note that these loops only happen
when ASBR-to-
ASBR ASBR-to-ASBR encapsulation is not used within the AS.

This document only addresses the first reason.

4. Practices to avoid packet losses

This section describes means for an ISP to reduce the transient loss
of packets upon a manual shutdown of a BGP session.

4.1. Improving availability of alternate paths

All solutions that increase the availability of alternate BGP paths
in
at routers performing packet lookups in BGP tables [BestExternal]
[AddPath] help in reducing the LoC bound with manual shutdown of eBGP
sessions.

One of such solutions increasing diversity in such a way that, at any
single step of the convergence process following the eBGP session
shutdown, a BGP router does not receive a message withdrawing the
only path it currently knows for a given NLRI, allows for a
simplified g-shut procedure. This simplified procedure would only
tackle potential LoC for

Increasing diversity with [AddPath] might lead to the inbound traffic. respect of this
property, depending on the path propagation decision process that
add-path compliant routers would use.

Using advertise-best-external [BestExternal] on ASBRs and RRs helps
in avoiding lack of alternate paths in route reflectors upon a
convergence. Hence it reduces the LoC duration for the outbound
traffic of the ISP upon an eBGP Session shutdown by reducing the iBGP
path hunting.

Still it does not ensure that BGP routers will always have at least
one path towards affected prefixes during the convergence following
the event. This property may be verified in future revisions of
[BestExternal], notably of its Section 3, hence the current proposal
will be updated accordingly.

Increasing diversity with [AddPath] might lead to the respect of this
property, depending on the path propagation decision process that
add-path compliant routers would use.

Note that the LoC for the inbound traffic of the maintained router,
induced by a lack of alternate path propagation within the iBGP
topology of a neighboring AS is not under the control of the operator
performing the maintenance, hence the maintenance. The procedure described in Section 4.2.2
should thus be applied upon the maintenance, even if not
required for the outbound traffic. procedure
described in Section 4.2.1 is not applied.

4.2. Graceful shutdown procedures for eBGP sessions

This section aims at describing a procedure to be applied to reduce
the LoC with readily available BGP features, and without assuming a
particular iBGP design in the Initiator and Neighbor ASes.

4.2.1. Outbound traffic

This section discusses a mean to render the affected paths less
desirable by the BGP decision process of affected routers, still
allowing these to be used during the convergence convergence, while alternate
paths are propagated to the affected routers.

A decrease of the local-pref value of the affected paths can be
issued in order to render the affected paths less preferable, at the
highest possible level of the BGP Decision Process.

This operation can be performed by reconfiguring the out-filters
associated with the iBGP sessions established by the g-shut
initiator.

The modification of the filters MUST supplant any other rule
affecting the local-pref value of the old paths.

Compared to using an in-filter of the eBGP session to be shut down,
the modification of the out-filters will not let the g-shut initiator
switch to another path, as the input to the BGP decision process of
that router does not change. As a consequence, the g-shut initiator
will not send a withdraw message over its iBGP modify the state of its dataplane, and will not withdraw the
affected paths over its iBGP sessions when it receives an alternate path over an iBGP session.
paths. It will however modify the local-pref of the affected paths
so that upstream routers will switch to alternate ones.

When the actual shutdown of the session is performed, the g-shut
initiator will itself switch to the alternate paths.

In cases some BGP speakers in the AS override the local-pref
attribute of paths received over iBGP sessions, the procedure
described above will not work. In such cases, the recommended
procedure is to tag the paths sent over the iBGP sessions of the
g-shut initiator with an AS specific community. This AS specific
community should lead to the setting of a low the lowest local-pref value.
To be effective, the configuration related to this community MUST
supplant or be applied after the already configured local-pref
overriding.

An operator may decide to follow a simplified procedure and directly
apply an in-filter reducing the local preference of the paths
received over the eBGP session being brought down. While this
procedure will be effective in many cases, corner cases as described
in Appendix A.1 may happen, which may lead to some LoC for some
affected destinations. The use of this simplified procedure does not
lead to LoC when used in conjunction with [BestExternal].

4.2.2. Inbound traffic

The solution described for the outbound traffic can be applied at the
neighbor AS. This can be done either "manually" or by using a
community value dedicated to this task.

4.2.2.1. Phone call

The operator performing the maintenance of the eBGP session can
contact the operator at the other side of the peering link, and let
him apply the procedure described above for its own outbound traffic.

4.2.2.2. Community tagging

A community value (referred to as GSHUT community in this document)
can be agreed upon by neighboring ASes. A path tagged with this
community must be considered as soon ASes and used to be affected by a maintenance
operation. trigger the g-shut
behavior at the g-shut neighbor.

4.2.2.2.1. Pre-Configuration

A g-shut neighbor is pre-configured to set a low local-pref value for
the paths received over eBGP sessions which are tagged with the GSHUT
community.

This rule must supplant any other rule affecting the local-pref value
of the paths.

This local-pref reconfiguration SHOULD be performed at the out-
filters of the iBGP sessions of the g-shut neighbor. That is, the
g-shut neighbor does not take into account this low local-pref in its
own BGP best path selection. As described in Section 4.2.1 this
approach avoids sending the withdraw messages that can lead to LoC. LoC in
some cases.

4.2.2.2.2. Operational action upon maintenance

Upon the manual shutdown, the output filter associated with the
maintained eBGP session will be modified on the g-shut initiator so
as to tag all the paths advertised over the session with the GSHUT
community.

4.2.2.2.3. Transitivity of the community

If the GSHUT community is an extended community, it SHOULD be chosen
non-transitive. In that case, the clarification described in
[Clarification4360] is required.

If a regular community is used, this community SHOULD be removed from
the path when the path is propagated over eBGP sessions.

Not propagating the community further in the Internet reduces the
amount of BGP churn and avoids rerouting in distant ASes that would
also recognize this community value. In other words, from a routing
stability perspective, it helps concealing the convergence at the
maintenance location. From a security policy perspective, it prevents
malignant ASes from using the community over paths propagated through
intermediate ASes that do not support the feature, in order to
perform inbound traffic engineering at the first AS recognizing the
community.

ASes which support the g-shut procedure SHOULD remove the community
value(s) that they use for g-shut from the paths received from
neighboring ASes that do not support the procedure or to whom the
service is not provided.

There are cases where an interdomain exploration is to be performed
to recover the reachability, e.g., in the case of a shutdown in
confederations where the alternate paths will be found in another AS
of the confederation. In such scenarios, the community value SHOULD
be allowed to transit through the confederation but SHOULD be removed
from the paths advertised outside of the confederation.

When the local-pref value of a path is conserved upon its propagation
from one AS of the confederation to the other, there is no need to
have the GSHUT community be propagated throughout that confederation.

4.2.2.2.4. Easing the configuration for G-SHUT

From a configuration burden viewpoint, it is much easier to use a
single dedicated value for the GSHUT community.

First, on the g-shut initiator, an operator would have a single
configuration rule to be applied at the maintenance time, which would
not depend on the identity of its peer. This would make the
maintenance operations less error prone.

Second, on the g-shut neighbor, a simple filter related to g-shut can
be applied to all iBGP sessions. Additionnaly, this filter does not
need to be updated each time neighboring ASes are added or removed.

The FCFS community value 0xFFFF0000 has been reserved for this
purpose [BGPWKC].

4.3. Graceful shutdown procedures for iBGP sessions

If the iBGP topology is viable after the maintenance

4.2.3. Summary of operations

This section summarizes the session,
i.e, if all BGP speakers of configurations and actions to be
performed to support the AS have an iBGP signaling path for
all prefixes advertised on this g-shut iBGP session, then procedure for eBGP peering links.

4.2.3.1. Pre-configuration

On each ASBR supporting the
shutdown of g-shut procedure, set-up an out-filter
applied on all iBGP session does not lead to transient
unreachability.

However, in the case of a shutdown of a router, a reconfiguration sessions of the out-filters ASBR, that :
o sets the local-pref of the paths tagged with the g-shut initiator MAY be performed
community to set a low
local-pref value for the paths originated by
o removes the g-shut initiator
(e.g, BGP aggregates redistributed community from other protocols, including
static routes).

This behavior is equivalent to the recommended behavior for paths.
o optionally, adds an AS specific g-shut community on these paths
"redistributed" from eBGP sessions
to iBGP sessions in the case of
the shutdown of an ASBR.

5. Forwarding modes and forwarding loops indicate that these are to be withdrawn soon. If some
ingress ASBRs reset the local preference attribute, this AS applying
specific g-shut community will be used to override other local
preference changes.

4.2.3.2. Operations at maintenance time

On the solution does not rely g-shut initiator :
o Apply an out-filter on encapsulation to
forward packets from the Ingress Border Router maintained eBGP session to tag the Egress Border
Router, then transient forwarding loops and consequent packet losses
can occur during the convergence process, even if
paths propagated over the procedure
described above is applied. Hence if zero LoC is required,
encapsulation is required between ASBRs of session with the AS.

Using g-shut community.
o Apply an in-filter on the out-filter reconfiguration avoids maintained eBGP session to tag the forwarding loops
between paths
received over the g-shut initiator and its directly connected upstream
neighboring routers. Indeed, when this reconfiguration is applied, session with the g-shut initiator keeps using its own external path and lets the
upstream routers converge community.
o Wait for convergence to happen.
o Perform a BGP session shutdown.

4.2.4. BGP implementation support for G-Shut

A BGP router implementation MAY provide features aimed at automating
the alternate ones. During this phase,
no forwarding loops can occur between application of the g-shut initiator and its
upstream neighbors graceful shutdown procedures described above.

Upon a session shutdown specified as to be graceful by the operator,
a BGP implementation supporting a g-shut initiator keeps using feature would

1. Update all the affected paths via its propagated over the corresponding eBGP peering links. When all
session, tagging the upstream routers have
switched GSHUT community to them. Any subsequent
update sent to alternate paths, the transition performed by session being gracefully shut down would be
tagged with the g-shut
initiator when GSHUT community.
2. Lower the local preference value of the paths received over the
eBGP session is actually being shut down, will be loopfree.
Transient forwarding loops between other routers will not be avoided
with this procedure.

6. Dealing upon their propagation over iBGP
sessions. Optionally, also tag these paths with Internet policies

A side gain an AS specific
g-shut community. Note that alternatively, the local preference
of the maintenance solution is that it paths received over the eBGP session can also reduce be lowered on
the churn implied by a shutdown g-shut initiator itself, instead of an eBGP session.

For this, it is recommended only when propagating
over its iBGP sessions. This simplified behavior can lead to apply the filters modifying
some LoC, as described in Appendix A.1, if not used in
conjunction with [BestExternal].
3. Optionally shut down the local-
pref value of session after a configured time.
4. Prevent the GSHUT community from being inherited by a path that
would aggregate some paths tagged with the GSHUT community.
This behavior avoids the GSHUT procedure to values strictly lower but as close as
possible be applied to the local-pref values of
aggregate upon the post-convergence paths.

For example, if an eBGP link is shut down between a provider and graceful shutdown of one of its customers, and another link with this customer remains active,
then covered
prefixes.

4.3. Graceful shutdown procedures for iBGP sessions

If the value of iBGP topology is viable after the local-pref maintenance of the old paths SHOULD be decreased
to session,
i.e, if all BGP speakers of the smallest possible value of AS have an iBGP signaling path for
all prefixes advertised on this g-shut iBGP session, then the 'customer' local_pref range,
minus 1. Thus, routers will
shutdown of an iBGP session does not transiently switch lead to transient
unreachability.

However, in the case of a shutdown of a router, a reconfiguration of
the out-filters of the g-shut initiator MAY be performed to set a low
local-pref value for the paths received originated by the g-shut initiator
(e.g, BGP aggregates redistributed from shared-cost peers or providers, which could lead other protocols, including
static routes).

This behavior is equivalent to the
propagation of withdraw messages over recommended behavior for paths
"redistributed" from eBGP sessions with shared-cost
peers and providers.

Proceeding like this reduces both BGP churn and traffic shifting as
routers will less likely switch to transient paths.

In the above example, it also prevents transient unreachabilities iBGP sessions in the neighboring case of
the shutdown of an ASBR.

5. Forwarding modes and forwarding loops

If the AS that are due applying the solution does not rely on encapsulation to
forward packets from the sending of "abrupt" withdraw
messages Ingress Border Router to shared-cost peers and providers.

7. Link Up cases

We identify two potential causes for the Egress Border
Router, then transient forwarding loops and consequent packet losses upon an
eBGP link up event. The first one is local to
can occur during the g-no-shut
initiator, convergence process, even if the second one procedure
described above is due to applied. Hence if zero LoC is required,
encapsulation is required between ASBRs of the BGP convergence following AS.

Using the
injection of new best paths within out-filter reconfiguration avoids the iBGP topology.

7.1. Unreachability local to forwarding loops
between the ASBR

An ASBR that selects as best a path received over a newly brought up
eBGP session may transiently drop traffic. This can typically happen g-shut initiator and its directly connected upstream
neighboring routers. Indeed, when this reconfiguration is applied,
the nexthop attribute differs from g-shut initiator keeps using its own external path and lets the IP address of
upstream routers converge to the eBGP
peer, alternate ones. During this phase,
no forwarding loops can occur between the g-shut initiator and its
upstream neighbors as the receiving ASBR has not yet resolved g-shut initiator keeps using the MAC address
associated with affected
paths via its eBGP peering links. When all the IP address of that "third party" nexthop.

A BGP speaker implementation could avoid such losses upstream routers have
switched to alternate paths, the transition performed by ensuring that
"third party" nexthops are resolved before installing paths using
these in the RIB.

If g-shut
initiator when the link up event corresponds to an eBGP session that is being
manually brought up, over an already up multi-access link, then actually shut down, will be loopfree.
Transient forwarding loops between other routers will not be avoided
with this procedure.

6. Dealing with Internet policies

A side gain of the
operator can ping third party nexthops maintenance solution is that are expected it can also reduce
the churn implied by a shutdown of an eBGP session.

For this, it is recommended to be used
before actually bringing apply the session up, or ping directed broadcast filters modifying the subnet IP address local-
pref value of the link. By proceeding like this, the MAC
addresses associated with these third party nexthops will be resolved
by the g-no-shut initiator.

7.2. iBGP convergence

Similar corner cases paths to values strictly lower but as described in Appendix C.1.4 for close as
possible to the link down
case, can occur during local-pref values of the post-convergence paths.

For example, if an eBGP link up event.

A typical example for such transient unreachability for a given
prefix is the following :

1. A Route Reflector, RR1, is initially advertising the current
best path to the members of its iBGP RR full-mesh. It
propagated that path within its RR full-mesh. Another route
reflector of the full-mesh, RR2, knows only that path towards
the prefix.
2. A third Route Reflector of the RR full-mesh, RR3 receives shut down between a
new best path orginated by the "g-no-shut" initiator, being provider and one
of its RR clients. RR3 selects it as best, and propagates an
UPDATE within its RR full-mesh, i.e., to RR1 and RR2.
3. RR1 receives that path, reruns its decision process, customers, and
picks another link with this new path as best. As a result, RR1 withdraws its
previously announced best-path on customer remains active,
then the iBGP sessions value of its RR
full-mesh.
4. If, for any reason, RR3 processes the withdraw generated in
step 3, before processing local-pref of the update generated in step 2, RR3 old paths SHOULD be decreased
to the smallest possible value of the 'customer' local_pref range,
minus 1. Thus, routers will not transiently suffers switch to paths received
from unreachability for shared-cost peers or providers, which could lead to the affected prefix.

The use
propagation of [BestExternal] among withdraw messages over eBGP sessions with shared-cost
peers and providers.

Proceeding like this reduces both BGP churn and traffic shifting as
routers will less likely switch to transient paths.

In the RR of above example, it also prevents transient unreachabilities in
the iBGP full-mesh can
solve these corner cases by ensuring neighboring AS that within an AS, are due to the
advertisement sending of a new route "abrupt" withdraw
messages to shared-cost peers and providers.

7. Link Up cases

We identify two potential causes for transient packet losses upon an
eBGP link up event. The first one is not translated into local to the withdraw g-no-shut
initiator, the second one is due to the BGP convergence following the
injection of a
former route.

Indeed, "best-external" ensures that an ASBR does not withdraw a
previously advertised (eBGP) path when it receives an additional,
preferred path over an new best paths within the iBGP session. Also, "best-intra-cluster"
ensures topology.

7.1. Unreachability local to the ASBR

An ASBR that selects as best a RR does not withdraw a previously advertised (iBGP) path to its non clients (e.g. other RRs in received over a mesh of RR) newly brought up
eBGP session may transiently drop traffic. This can typically happen
when it
receives a new, preferred path over an iBGP session.

8. IANA considerations

Applying the g-shut procedure is rendered much easier with a reserved
g-shut community value. The community value 0xFFFF0000 has been
reserved nexthop attribute differs from the FCFS community pool for this purpose.

9. Security Considerations

By providing IP address of the g-shut service to a neighboring AS, an ISP provides
means to this neighbor to lower eBGP
peer, and the local-pref value assigned to receiving ASBR has not yet resolved the
paths received from this neighbor.

The neighbor could abuse MAC address
associated with the technique and do inbound traffic
engineering IP address of that "third party" nexthop.

A BGP speaker implementation could avoid such losses by declaring some prefixes as undergoing a maintenance so
as to switch traffic to another peering link. ensuring that
"third party" nexthops are resolved before installing paths using
these in the RIB.

If this behavior the link up event corresponds to an eBGP session that is not tolerated by being
manually brought up, over an already up multi-access link, then the ISP, it SHOULD monitor
operator can ping third party nexthops that are expected to be used
before actually bringing the
use session up, or ping directed broadcast
the subnet IP address of the g-shut community link. By proceeding like this, the MAC
addresses associated with these third party nexthops will be resolved
by this neighbor.

ASes which support the g-shut procedure SHOULD remove g-no-shut initiator.

7.2. iBGP convergence

Similar corner cases as described in Appendix A.1 for the community
value(s) that they use link down
case, can occur during an eBGP link up event.

A typical example for g-shut from such transient unreachability for a given
prefix is the paths received from
neighboring ASes that do not support following :

1. A Route Reflector, RR1, is initially advertising the procedure or current
best path to whom the
service is not provided. Doing so prevents malignant ASes from using the community through intermediate ASes members of its iBGP RR full-mesh. It
propagated that do not support the
feature, in order to perform inbound traffic engineering.

10. Acknowledgments

The authors wish to thank Olivier Bonaventure and Pradosh Mohapatra
for their useful comments on this work.

11. References

[AddPath] D. Walton, A. Retana, and E. Chen, "Advertisement of
Multiple Paths in BGP", draft-walton-bgp-add-paths-06.txt
(work in progress).

[BestExternal]
Marques, P., Fernando, R., Chen, E., and P. Mohapatra,
"Advertisement of the best-external route to IBGP",
draft-ietf-idr-best-external-00.txt, May 2009.

[REQS] Decraene, B., Francois, P., Pelsser, C., Ahmad, Z., and T.
Takeda, "Requirements for the graceful shutdown of BGP
sessions",
draft-ietf-grow-bgp-graceful-shutdown-requirements-
01.txt, October 2009.

[RFC4360] Sangli, S., Tappan, D., and Y. Rekhter, "BGP Extended
Communities Attribute", RFC 4360, February 2006.

[Clarification4360]
Decraene, B., Vanbever, L., and P. Francois, "RFC 4360
Clarification Request",
draft-decraene-idr-rfc4360-clarification-00,
October 2009.

[BGPWKC] "http://www.iana.org/assignments/
bgp-well-known-communities".

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.

Appendix A. Summary of operations

This section summarizes the configurations and actions to be
performed to support the g-shut procedure for eBGP peering links.

A.1. Pre-configuration

On each ASBR supporting the g-shut procedure, set-up an out-filter
applied on all iBGP sessions of the ASBR, that :

. sets the local-pref of the paths tagged with the g-shut community
to a low value

. removes the g-shut community from the path.

Optionally, add an AS specific g-shut community on the path to
indicate that this path is to be shutdown. If some ingress ASBRs
reset the local preference attribute, this AS specific g-shut
community will be used to override other local preference changes.

A.2. Operations at maintenance time

On the g-shut initiator :

. Apply an in-filter on the maintained eBGP session to tag the paths
received over the session with the g-shut community.

. Apply an out-filter on the maintained eBGP session to tag the
paths propagated over the session with the g-shut community.

. Wait for convergence to happen.

. Perform a BGP session shutdown.

Appendix B. Alternative techniques with limited applicability

A few alternative techniques have been considered to provide g-shut
capabilities but have been rejected due to their limited
applicability. This section describe them for possible reference.

B.1. In-filter reconfiguration

An In-filter reconfiguration on the eBGP session undergoing the
maintenance could be performed instead of out-filter reconfigurations
on the iBGP sessions of the g-shut initiator.

Upon the application of the maintenance procedure, if the g-shut
initiator has an alternate path in its Adj-Rib-In, it will switch to
it directly.

If this new path was advertised by an eBGP neighbor of the g-shut
initiator, the g-shut initiator will send a BGP Path Update message
advertising the new path over its iBGP and eBGP sessions.

If this new path was received over an iBGP session, the g-shut
initiator will select that path and withdraw the previously
advertised path over its non-client iBGP sessions. There can be iBGP
topologies where the iBGP peers of the g-shut initiator do not know
an alternate path, and hence may drop traffic.

Also, applying an In-filter reconfiguration on the eBGP session
undergoing the maintenance may lead to transient LoC, in full-mesh
iBGP topologies if

a. An ASBR of the initiator AS, ASBR1 did not initially select within its own external path as best, and

b. An ASBR RR full-mesh. Another route
reflector of the initiator AS, ASBR2 advertises a new full-mesh, RR2, knows only that path
along its iBGP sessions upon towards
the reception prefix.
2. A third Route Reflector of ASBR1's update
following the in-filter reconfiguration on RR full-mesh, RR3 receives a
new best path orginated by the g-shut "g-no-shut" initiator, being one
of its RR clients. RR3 selects it as best, and

c. ASBR1 receives the update message, runs propagates an
UPDATE within its Decision Process RR full-mesh, i.e., to RR1 and hence withdraws RR2.
3. RR1 receives that path, reruns its external path after having selected
ASBR2's decision process, and
picks this new path as best, and

d. An impacted router of best. As a result, RR1 withdraws its
previously announced best-path on the AS iBGP sessions of its RR
full-mesh.
4. If, for any reason, RR3 processes the withdraw of ASBR1 generated in
step 3, before processing the update generated in step 2, RR3
transiently suffers from ASBR2.

Applying a reconfiguration unreachability for the affected prefix.

The use of [BestExternal] among the out-filters prevents such transient
unreachabilities.

Indeed, when RR of the g-shut initiator propagates iBGP full-mesh can
solve these corner cases by ensuring that within an update of AS, the old
path first,
advertisement of a new route is not translated into the withdraw from ASBR2 of a
former route.

Indeed, "best-external" ensures that an ASBR does not trigger unreachability
in other nodes, as the old withdraw a
previously advertised (eBGP) path is still available. Indeed, even
though when it receives an additional,
preferred path over an iBGP session. Also, "best-intra-cluster"
ensures that a RR does not withdraw a previously advertised (iBGP)
path to its non clients (e.g. other RRs in a mesh of RR) when it
receives alternate paths, the g-shut initiator keeps using
its old a new, preferred path as best as the in-filter of the maintained eBGP session
has not been modified yet. over an iBGP session.

8. IANA considerations

Applying the out-filter reconfiguration also prevents packet loops
between the g-shut initiator and its direct neighbors when
encapsulation procedure is not used between the ASBRs of the AS.

B.2. Multi Exit Discriminator tweaking rendered much easier with a reserved
g-shut community value. The MED attribute of community value 0xFFFF0000 has been
reserved from the paths FCFS community pool for this purpose.

9. Security Considerations

By providing the g-shut service to be avoided can be increased so as a neighboring AS, an ISP provides
means to force the routers in this neighbor to lower the neighboring AS local-pref value assigned to select other paths.

The solution only works if the alternate
paths are received from this neighbor.

The neighbor could abuse the technique and do inbound traffic
engineering by declaring some prefixes as good undergoing a maintenance so
as the
initial ones with respect to the Local-Pref value and the AS Path
Length value. In the other cases, increasing the MED value will switch traffic to another peering link.

If this behavior is not
have an impact on tolerated by the decision process of ISP, it SHOULD monitor the routers in
use of the
neighboring AS.

B.3. IGP distance Poisoning

The distance to g-shut community by this neighbor.

ASes which support the BGP nexthop corresponding to g-shut procedure SHOULD remove the maintained
session can be increased in community
value(s) that they use for g-shut from the IGP so paths received from
neighboring ASes that do not support the old paths will be
less preferred during the application of procedure or to whom the IGP distance tie-break
rule. However, this solution only works for
service is not provided. Doing so prevents malignant ASes from using
the paths whose
alternates are as good as community through intermediate ASes that do not support the old paths with respect
feature, in order to their Local-
Pref value, their AS Path length, perform inbound traffic engineering.

10. Acknowledgments

The authors wish to thank Olivier Bonaventure and Pradosh Mohapatra
for their MED value.

Also, useful comments on this poisoning cannot be applied when nexthop self is used as
there is no nexthop specific to the maintained session to poison work.

11. References

[AddPath] D. Walton, A. Retana, and E. Chen, "Advertisement of
Multiple Paths in
the IGP.

Appendix C. Effect BGP", draft-walton-bgp-add-paths-06.txt
(work in progress).

[BestExternal]
Marques, P., Fernando, R., Chen, E., and P. Mohapatra,
"Advertisement of the g-shut procedure on the convergence

This section describes the effect of applying best-external route to IBGP",
draft-ietf-idr-best-external-00.txt, May 2009.

[REQS] Decraene, B., Francois, P., Pelsser, C., Ahmad, Z.,
Armengol, A., and T. Takeda, "Requirements for the solution.

C.1. Maintenance
graceful shutdown of an eBGP session BGP sessions",
draft-ietf-grow-bgp-graceful-shutdown-requirements-
06.txt, October 2010.

[RFC4360] Sangli, S., Tappan, D., and Y. Rekhter, "BGP Extended
Communities Attribute", RFC 4360, February 2006.

[Clarification4360]
Decraene, B., Vanbever, L., and P. Francois, "RFC 4360
Clarification Request",
draft-decraene-idr-rfc4360-clarification-00,
October 2009.

[BGPWKC] "http://www.iana.org/assignments/
bgp-well-known-communities".

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.

Appendix A. Alternative techniques with limited applicability

A few alternative techniques have been considered to provide g-shut
capabilities but have been rejected due to their limited
applicability. This section describes the effect of applying the solution describe them for the
shutdown of an eBGP session.

C.1.1. Propagation on the other eBGP sessions of the g-shut initiator

Nothing is propagated on the other eBGP sessions when the out-filters
reconfiguration step is applied. The possible reference.

A.1. In-filter reconfiguration is indeed only
defined for its iBGP sessions.

The

An In-filter reconfiguration of the iBGP out-filters will trigger the
reception of alternate paths at the g-shut initiator. As on the eBGP
in-filters have not been modified at that step, the old paths are
still preferred by the g-shut initiator.

C.1.2. Propagation on session undergoing the other iBGP sessions
maintenance could be performed instead of the g-shut initiator

During the out-filter reconfiguration, path updates are propagated
with a reduced local-pref value for the affected paths. As a
consequence, Route Reflectors and distant ASBRs select and propagate
alternate paths through the iBGP topology as they no longer select
the old paths as best.

When the shut-down is performed, for each affected prefix, the g-shut
initiator propagates reconfigurations
on its iBGP sessions:

. The alternate path, if the best path was received over an eBGP
sessions.

. A withdraw, if the best path was received over an iBGP sessions.

C.1.3. Propagation of updates in an iBGP full-mesh

No transient LoC can occur if a reconfiguration sessions of the iBGP out-
filters on the g-shut initiator is performed.

C.1.4. Propagation of updates from iBGP to iBGP in a RR hierarchy initiator.

Upon the reception application of the update of a primary path with a lower
local-pref value from a client, a Route Reflector RR1 will either
propagate maintenance procedure, if the update, or select g-shut
initiator has an alternate path, depending on the
fact that the updated primary path is still in its Adj-Rib-In, it will switch to
it directly.

If this new path was advertised by an eBGP neighbor of the best one w.r.t. g-shut
initiator, the
state of g-shut initiator will send a BGP Path Update message
advertising the Adj-Rib-In of RR1. new path over its iBGP and eBGP sessions.

If the updated primary this new path is still the best, then was received over an iBGP session, the RR g-shut
initiator will
propagate an update for this select that path to and withdraw the iBGP neighbors to which it previously
advertised the path. Hence it cannot cause transient lack
of path in the Adj-Rib-In of over its non-client iBGP neighbors.

If sessions. There can be iBGP
topologies where the iBGP peers of the g-shut initiator do not know
an alternate path is picked, path, and this path was also originated by
a client of RR1, hence may drop traffic.

Also, applying an update will also be propagated to In-filter reconfiguration on the same
neighbors as eBGP session
undergoing the one maintenance may lead to which the primary path was initially
propagated. Hence it cannot cause transient lack of path LoC, in the Adj-
Rib-In full-mesh
iBGP topologies if

a. An ASBR of the initiator AS, ASBR1 did not initially select
its iBGP neighbors.

If an alternate own external path is picked, as best, and this path was received from a
member

b. An ASBR of the initiator AS, ASBR2 advertises a new path
along its Route-Reflector iBGP full-mesh, then a withdraw message
is sent. As sessions upon the alternate reception of ASBR1's update
following the in-filter reconfiguration on the g-shut initiator,
and

c. ASBR1 receives the update message, runs its Decision Process
and hence withdraws its external path has been sent over each session after having selected
ASBR2's path as best, and

d. An impacted router of the iBGP full-mesh, AS processes the propagation of a withdraw for the primary
path of RR1 is done to routers that are expected to know ASBR1
before processing the
alternate path picked by RR1.

The following example describes a situation where some corner case
timings could lead to transient unreachability update from some members ASBR2.

Applying a reconfiguration of the iBGP full-mesh.

1. A Route Reflector RR1 only knew about the primary path upon out-filters prevents such transient
unreachabilities.

Indeed, when the shutdown.

2. A member of its RR full-mesh, RR2, g-shut initiator propagates an update of the old
path with a lower local-pref.

3. Another member of its RR full-mesh, RR3 processes first, the
update, selects an alternate path, and propagates an update withdraw from ASBR2 does not trigger unreachability
in other nodes, as the mesh.

4. RR2 old path is still available. Indeed, even
though it receives the alternate path, selects it as best, and
hence withdraws paths, the updated g-shut initiator keeps using
its old path on as best as the iBGP sessions in-filter of the
mesh.

5. If for any reason, RR1 receives maintained eBGP session
has not been modified yet.

Applying the out-filter reconfiguration also prevents packet loops
between the g-shut initiator and processes its direct neighbors when
encapsulation is not used between the withdraw
generated ASBRs of the AS.

Note that applying this simplified procedure in step 4 before processing conjunction with
[BestExternal] does not lead to LoC.

A.2. Multi Exit Discriminator tweaking

The MED attribute of the update generated paths to be avoided can be increased so as
to force the routers in
step 3, RR1 transiently suffers from unreachability for the
affected prefix. neighboring AS to select other paths.

The solution only works if the alternate paths are as good as the
initial ones with respect to the Local-Pref value and the AS Path
Length value. In such corner cases, the solution improves other cases, increasing the iBGP convergence
behavior/LoC but does MED value will not ensure 0 packet loss, as we cannot define a
simple solution relying only
have an impact on a reconfiguration of the filters decision process of the g-shut initiator. Improving the availability of alternate paths routers in Route Reflectors, using [BestExternal], or [AddPath], seems the
neighboring AS.

A.3. IGP distance Poisoning

The distance to be the most pragmatic solution BGP nexthop corresponding to these corner cases.

The use of [BestExternal] in the iBGP full-mesh between RRs maintained
session can solve
these corner cases by ensuring that within an AS, the advertisement
of a new path is not translated into be increased in the withdraw of a former path.

Indeed, "best-external" ensures that an ASBR does not withdraw a
previously advertised (eBGP) path when it receives an additional,
preferred path over an iBGP session. Also, "best-intra-cluster"
ensures IGP so that a RR does not withdraw a previously advertised (iBGP)
path to its non clients (e.g. other RRs in a mesh of RR) when it
receives a new, preferred path over an iBGP session.

C.2. Maintenance of an iBGP session

If the shutdown does not temper with old paths will be
less preferred during the viability application of the iBGP
topology, IGP distance tie-break
rule. However, this solution only works for the described procedure paths whose
alternates are as good as the old paths with respect to their Local-
Pref value, their AS Path length, and their MED value.

Also, this poisoning cannot be applied when nexthop self is used as
there is sufficient no nexthop specific to avoid LoC. the maintained session to poison in
the IGP.

Authors' Addresses

Pierre Francois
Universite catholique de Louvain
Place Ste Barbe, 2
Louvain-la-Neuve 1348
BE

Email: pierre.francois@uclouvain.be
URI: http://inl.info.ucl.ac.be/pfr

Bruno Decraene
France Telecom
38-40 rue du General Leclerc
92794 Issi Moulineaux cedex 9
FR

Email: bruno.decraene@orange-ftgroup.com

Cristel Pelsser
Internet Initiative Japan
Jinbocho Mitsui Bldg.
1-105 Kanda Jinbo-cho
Tokyo 101-0051
JP

Email: pelsser.cristel@iij.ad.jp

Keyur Patel
Cisco Systems
170 West Tasman Dr
San Jose, CA 95134
US

Email: keyupate@cisco.com

Clarence Filsfils
Cisco Systems
De kleetlaan 6a
Diegem 1831
BE

Email: cfilsfil@cisco.com