INTERNET-DRAFT D. Meyer (Editor) draft-ietf-grow-rift-01.txt Category Informational Expires:JulyAugust 2004JanuaryFebruary 2004 Operational Concerns and Considerations for Routing Protocol Design -- Risk, Interference, and Fit (RIFT)<draft-ietf-grow-rift-00.txt><draft-ietf-grow-rift-01.txt> Status of this Document This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. The key words "MUST"", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC 2119]. This document is a product of the RIFT Design Team. Comments should be addressed to the authors, or the mailing list at grow@lists.uoregon.edu. Copyright Notice Copyright (C) The Internet Society (2004). All Rights Reserved. Abstract The Risk, Interference, and Fit (RIFT) design team was formed to document the concerns and considerations surrounding the use of Internet routing protocols for functions not directly related to routing of IP packets within the Internet and IP networks. This document is the output of that activity. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5 2. Scope of this Work . . . . . . . . . . . . . . . . . . . . . . 5 3. Problem Statement. . . . . . . . . . . . . . . . . . . . . . . 6 3.1. Risk, Interference, and Application Fit (RIFT) . . . . . . 6 3.1.1. Risk: Software Engineering . . . . . . . . . . . . . . . 7 3.1.2. Interference: Protocol Specification/Dynamic Behavior . 7 3.1.3. Application Fit: Distribution Topology . . . . . . . . . 7 4. Definitions. . . . . . . . . . . . . . . . . . . . . . . . . . 8 4.1. Reachability Information. . . . . . . . . . . . . . . . . . 8 4.2. Layer 3 Routing Information . . . . . . . . . . . . . . . . 8 4.2.1. Standard Routing Information . . . . . . . . . . . . . . 9 4.3. Auxiliary (non-routing) Information . . . . . . . . . . . . 9 4.4. Address Family Identifier (AFI) . . . . . . . . . . . . . . 9 4.5. Subsequent Address Family Identifier (SAFI) . . . . . . . .910 4.6. Network Layer Reachability. . . . . . . . . . . . . . . . .910 4.7. Application . . . . . . . . . . . . . . . . . . . . . . . . 10 4.8. Routing Protocol. . . . . . . . . . . . . . . . . . . . . . 10 4.9. Fate Sharing. . . . . . . . . . . . . . . . . . . . . . . .1011 5. Architectural Models . . . . . . . . . . . . . . . . . . . . . 11 5.1. General Purpose Transport Infrastructure (GPT) Model. . . .1112 5.2. Special Purpose Transport Infrastructure (SPT) Model. . . . 12 6. Analyzing Risk and Interference. . . . . . . . . . . . . . . .1213 6.1. Risk: Code Impact, and Resource Sharing . . . . . . . . . . 13 6.1.1. Code Impact. . . . . . . . . . . . . . . . . . . . . . . 13 6.1.2. Resource Sharing . . . . . . . . . . . . . . . . . . . .1314 6.1.2.1. Resource Sharing and Operating System Level Issues . 14 6.2. Interference. . . . . . . . . . . . . . . . . . . . . . . .1415 7. GTP and SPT Models: Risk and Interference. . . . . . . . . . . 15 7.1. Risk. . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 7.1.1. Code Impact. . . . . . . . . . . . . . . . . . . . . . .1516 7.1.2. Resource Sharing . . . . . . . . . . . . . . . . . . . .1617 7.1.3. Multisession BGP . . . . . . . . . . . . . . . . . . . . 17 7.2. Interference. . . . . . . . . . . . . . . . . . . . . . . .1819 7.2.1. Multisession BGP . . . . . . . . . . . . . . . . . . . . 19 8. Application Fit. . . . . . . . . . . . . . . . . . . . . . . . 19 8.1. RFC 2547 Style VPNs . . . . . . . . . . . . . . . . . . . .1920 8.1.1. RFC 2547 and Label Distribution. . . . . . . . . . . . . 21 8.2. VPWS. . . . . . . . . . . . . . . . . . . . . . . . . . . .20 8.3. VPLS.21 8.2.1. Assertion #1 . . . . . . . . . . . . . . . . . . . . . . 22 8.2.2. Counter-Assertion #1 . . . . .21 9. Operational Implications. . . . . . . . . . . . . 22 8.2.3. Assertion #2 . . . . . . .22 10. Other Models.. . . . . . . . . . . . . . . 23 8.2.4. Counter-Assertion #2 . . . . . . . . .22 11. Conclusions. . . . . . . . . 23 8.2.4.1. Assertion #2a . . . . . . . . . . . . . . . . . . . . 23 8.2.4.2. Counter-Assertion #2a . . . . . . . . . . . . . . . . 23 8.2.5. Assertion #3 . . . . . . . . . . . . . . . . . . . . . . 24 8.2.6. Counter-Assertion #3 . . . . . . . . . . . . . . . . . . 25 8.3. VPWS andRecommendationsPer-Wire Attributes. . . . . . . . . . . . . . . .22 12. Intellectual Property27 8.3.1. Assertion #4 . . . . . . . . . . . . . . . . . . . .22 13. Design Team. . 27 8.3.2. Counter-Assertion #4:. . . . . . . . . . . . . . . . . . 27 8.3.3. Assertion #5 . . . . . .22 14. Acknowledgments. . . . . . . . . . . . . . . . 27 8.3.4. Counter-Assertion #5 . . . . . . .23 15. Security Considerations. . . . . . . . . . . 27 8.3.5. Assertion #6 . . . . . . . .24 16. IANA Considerations. . . . . . . . . . . . . . 28 8.3.6. Counter-Assertion #6 . . . . . . .24 17. References.. . . . . . . . . . . 28 8.3.7. Assertion #7:. . . . . . . . . . . . . . .25 17.1. Normative References. . . . . . . 28 8.3.8. Counter-Assertion #7:. . . . . . . . . . . . . . .25 17.2. Informative References. . . 29 8.4. VPLS. . . . . . . . . . . . . . . . . .27 18. Editor's Address.. . . . . . . . . . 29 8.4.1. Assertion #8 . . . . . . . . . . . . . . . . . . . . . . 2919. Full Copyright Statement.8.4.2. Counter-Assertion #8 . . . . . . . . . . . . . . . . . . 291. Introduction The stability of the global Internet routing system has been the subject of much research (see e.g., [RVBIB]) and discussion on various IETF mailing lists [IETFOL]. Much of the research into the8.4.3. Assertion #9 . . . . . . . . . . . . . . . . . . . . . . 30 8.4.4. Counter-Assertion #9 . . . . . . . . . . . . . . . . . . 30 9. Operational Implications . . . . . . . . . . . . . . . . . . . 30 9.1. OAM Functionality . . . . . . . . . . . . . . . . . . . . . 30 9.1.1. Assertion #10: . . . . . . . . . . . . . . . . . . . . . 30 9.1.2. Counter-Assertion #10: . . . . . . . . . . . . . . . . . 31 9.2. Full-Mesh Issues. . . . . . . . . . . . . . . . . . . . . . 31 10. Conclusions and Recommendations . . . . . . . . . . . . . . . 31 11. Intellectual Property . . . . . . . . . . . . . . . . . . . . 31 12. Design Team . . . . . . . . . . . . . . . . . . . . . . . . . 31 13. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 32 14. Security Considerations . . . . . . . . . . . . . . . . . . . 33 15. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 33 16. References. . . . . . . . . . . . . . . . . . . . . . . . . . 34 16.1. Normative References . . . . . . . . . . . . . . . . . . . 34 16.2. Informative References . . . . . . . . . . . . . . . . . . 37 17. Editor's Address. . . . . . . . . . . . . . . . . . . . . . . 38 18. Full Copyright Statement. . . . . . . . . . . . . . . . . . . 38 1. Introduction The stability of the global Internet routing system has been the subject of much research (see e.g., [RVBIB]) and discussion on various IETF mailing lists [IETFOL]. Much of the research into the routing system has centered around the analysis of the dynamics and stability of the Border Gateway Protocol Version 4 [BGP] (hereafter referred to as BGP). However, while the theoretical properties of BGP remains a topic of great interest, a more recent discussion has focused on effects of the addition of new types of Network Layer Reachability Information, or NLRI to BGP. In particular, the advent of two BGP attributes, Multiprotocol Reachable NLRI (MP_REACH_NLRI), and Multiprotocol Unreachable NLRI (MP_UNREACH_NLRI) [RFC2858], have made it possible to encode and transport a wide variety of features and their associated signaling using the BGP transport infrastructure. Examples include include IPv6 [RFC2460], flow specification rules [FLOW], IP VPNs [RFC2547BIS], Virtual Private LAN services [VPLS], Virtual Private Wire Service [VPWS], and auto-discovery mechanisms for VPNs in general [BGPVPN], This document outlines the concerns and issues surrounding using the BGP infrastructure as a generic feature and signaling transport. However, the similar concerns apply to the Interior Gateway Protocols (IGPs) in common use (e.g., ISIS [RFC1142] or OSPF [RFC2328]). The rest of this document is organized as follows: Section 2 outlines the scope of this work. Section 3 introduces the problem statement which is the focus of this document, section 4 provides definitions, and section 5 outlines the main architectural models that are discussed. The remaining sections discuss the the implications of those models. 2. Scope of this Work It is the intention of the RIFT design team that this document serve as a guide for both protocol designers and network operators. The goal is to outline the implications associated with employing existing routing protocols to enable additional feature sets and functionality, as contrasted with designing new mechanisms to carry those feature sets and functionalities. The issues, concerns and considerations discussed in this document focus on the implications for BGP [BGP,RFC1771]. It is important to note that similar issues will arise when considering generalizations to the information that the IGPs carry. 3. Problem Statement The advent of the MP_REACH_NLRI and MP_UNREACH_NLRI attributes, combined with the resulting generalization to the BGP infrastructure, have created the opportunity to use BGP to transport a wide variety of data types and their associated signaling. The combination of a BGP data type and its associated signaling is frequently called an "application"; example applications include the IPv4 and IPv6 [RFC2460] routing systems, flow specification rules [FLOW], auto- discovery mechanisms for Layer 3 VPNs [BGPVPN], virtual private LAN services [VPLS], and virtual private Wire Service [VPWS]. More recently, the discussion in the IETF community has focused on the use of the BGP as a generalized feature transport infrastructure [IETFOL]. The debate has recently intensified due to the emergence of a new class of application that uses the BGP infrastructure to distribute information that is not directly related to inter-domain routing. Examples of such applications include the use of the BGP transport infrastructure to provide auto-discovery for IP VPNs [RFC2547BIS], the virtual private LAN services mentioned above [VPLS] and VPNs in general [BGPVPN]. 3.1. Risk, Interference, and Application Fit (RIFT) As mentioned above, much of the debate surrounding these new uses of the BGP transport infrastructure has focused on the potential tradeoffs between the stability of the Internet routing system, as effected by the deployment of new applications, and the desire on the part of service providers to rapidly deploy these new applications, and to reduce the operational cost by re-using existing protocols. These tradeoffs have at times been described in terms of risk, interference, and application fit. Risk models the software engineering impact of new applications on a generic implementation, while interference models the impact of new applications on protocol definition and behavior. Finally, application fit models the similarity between an application's data and signaling requirements and a specific distribution algorithm. Each is described below. 3.1.1. Risk: Software Engineering Risk attempts to assess the robustness tradeoffs inherent in the addition of new applications to a given implementation. That is, risk models the impact of generic software engineering issues on a given implementation. These issues include the impact of new applications on existing implementations and on the fate sharing properties of those implementations. A second aspect of risk lies in the trade-off of extending an existing protocol versus designing, implementing, and deploying a new protocol. 3.1.2. Interference: Protocol Specification/Dynamic Behavior Interference models the potential for a new application to adversely effect the operation of an existing implementation at the protocol level, by inadvertently introducing a detrimental dependency of some kind. That is, an application is said to "interfere" with an existing application if, by virtue of the application's protocol extension(s), one or more fundamental properties of the protocol's operation are detrimentally altered. For example, could we create a new state which introduces an unanticipated deadlock situation to occur? Or could we destabilize the distributed behavior of the protocol? Or might we simply run out of the attributes or bits available (as happened, for example, with RADIUS [RFC2138])? 3.1.3. Application Fit: Distribution Topology Application fit refers to how closely the requirements of the data to be distributed match the underlying capabilities of a distribution mechanism. For example, it is clearly inefficient to broadcast data to all peers that is only required between two peers, just as it is inefficient to unicast (replicate) data that is required by all peers when a single broadcast would do. 4. Definitions 4.1. Reachability Information Reachability information refers to information describing some part of a network, along with how one can reach it, and perhaps also containing attributes of the implied path to the network locale. Typically, this information pertains to IP routing information; an example of non-IP reachability is VPLS information [VPLS]. 4.2. Layer 3 Routing Information Layer 3 routing information represents either link state information or network reachability information. Link state information represents Layer 3 adjacencies and topology. Link state routing protocols, such as OSPF [RFC2328] and ISIS [RFC1142], flood link state information throughout an IGP domain, so that each participating router maintains an identical copy of a database that is computed to reflect the complete Layer 3 topology. Layer 3 reachability information expressed as an IP address prefix represents the set of destinations (systems) whose IP addresses are contained in the IP address prefix. Distance/path vector routing protocols, such as BGP, distribute Layer 3 reachability information among routing domains. Routers use both types of Layer 3 routing information (link state and reachability) to produce IP forwarding tables. That, is, for purposes of this discussion, "routing information" relates to the Layer 3 inter-domain routing data traditionally carried by BGP. Finally, if one defines routing information as "information used to forward packets", combined with the above definition of reachability information, then we can consider information such as described in [FLOW] (for example) to be routing information (since it is attempting to add a level of granularity to how an 'aggregate' is defined). That is, [FLOW] intends to complement to the existing routing information, and the flow information is dependent on IP4 unicast reachability advertised by the same neighbor. 4.2.1. Standard Routing Information In the most general terms, then, a routing protocol distributes data to accomplish the following three functionalities: (i). To govern the routing decision process (e.g., the standard BGP decision process) (ii). To constrain the flow of information (for example, with BGP communities) (iii). To tell the recipient how to get packets to the next hop We will refer to information that falls into this class as "standard routing information". 4.3. Auxiliary (non-routing) Information Auxiliary Information is any information that is exchanged by routers which is neither Layer 3 routingsystem has centered around the analysisinformation, nor reachability information. IS-IS hostname TLVs are an example ofthe dynamicsauxiliary information [RFC1142]. 4.4. Address Family Identifier (AFI) An Address Family contains addresses that share common structure andstability ofsemantics. An Address Family Identifier (AFI) uniquely identifies each address family. Several routing protocol messages contain a field that represents theBorder GatewayAFI. The AFI identifies the address type used by another data item contained in that message. The Routing Information ProtocolVersion 4 [BGP] (hereafter referred to as BGP). However, while(RIP) [RFC2453], Distance Vector Multicast Routing Protocol (DVMRP) [RFC1075], and BGP all employ the AFI field. For example, thetheoretical properties ofBGPremainsMP_REACH_NLRI and MP_UNREACH_NLRI attributes contain an AFI field. These BGP attributes also contain atopicNLRI field that enumerates reachable or unreachable subnetworks corresponding to the associated address family. The AFI field indicates the address type by which reachable subnetworks are identified. When BGP is used to distribute Layer 3 routing information, AFIs can indicate the following address types: IPv4, IPv6, VPNv4 [RFC2547BIS]. When BGP is used to distribute auxiliary information, AFIs can indicate other address families. 4.5. Subsequent Address Family Identifier (SAFI) A Subsequent Address Family Identifier (SAFI) is part ofgreat interest,the BGP MP_REACH_NLRI and MP_UNREACH_NLRI attributes. These BGP attributes also contain amore recent discussion has focused on effects ofNLRI field that enumerates reachable or unreachable subnetworks. The SAFI augments theaddition of new types ofAFI, carrying additional information regarding networks enumerated in the NLRI field. 4.6. Network Layer Reachability Network Layer Reachability Information, or NLRIto BGP. In particular,is theadventdata described by the AFI/SAFI fields [AFI,SAFI]. While these concepts were originally described for protocols such as DVMRP [RFC1075], the bulk oftwo BGP attributes, Multiprotocol Reachable NLRI (MP_REACH_NLRI), and Multiprotocol Unreachable NLRI (MP_UNREACH_NLRI) [RFC2858], have made it possible to encode and transport a wide varietythe generalization offeatures and their associated signaling usingtheBGP transport infrastructure. Examples include include IPv6 [RFC2460], flow specification rules [FLOW], IP VPNs [RFC2547BIS], Virtual Private LAN services [VPLS], Virtual Private Wire Service [VPWS], and auto-discovery mechanisms for VPNsNLRI described ingeneral [BGPVPN], Thisthis documentoutlinesderives from theconcerns and issues surrounding usingintroduction of the MP_REACH_NLRI and MP_UNREACH_NLRI attributes to BGPinfrastructure as[RFC2858]. 4.7. Application The term application is used in this document to refer to the combination of ageneric featureBGP data type and any signalingtransport. However,data that is carried by BGP in support of thesimilar concerns apply toservice theInterior Gateway Protocols (IGPs) in common use (e.g., ISIS [RFC1142] or OSPF [RFC2328]).data type carries. Therest of this documentdata type isorganized as follows: Section 2 outlinestypically described in an AFI/SAFI, while thescopeactual data is frequently contained in both NLRI and BGP community attributes [RFC1997]. 4.8. Routing Protocol A routing protocol is composed ofthis work. Sectiontwo basic components: a data distribution algorithm and a decision algorithm. A router typically obtains Layer 3introducesrouting information via its data distribution algorithm, and it uses this information to produce an IP forwarding table (by applying theproblem statement whichprotocol's decision algorithm to the received routing data). Note that it is thefocususe ofthis document, section 4 provides definitions, and section 5 outlines the main architectural modelsBGP's data distribution algorithm thatare discussed. The remaining sections discuss theis theimplications of those models. 2. Scopefocus of thisWork It isdocument. However, when judging application fit, one may also consider whether theintention ofdecision algorithms suit theRIFT design team that this document serve as a guide for both protocol designers and network operators.application. 4.9. Fate Sharing Thegoal isfate sharing principle for end tooutline the implications associated with employing existing routingend network protocols was first enunciated by Dave Clark [CLARK]. As applied toenable additional feature sets and functionality, as contrasted with designing new mechanisms to carry those feature sets and functionalities. The issues, concerns and considerations discussed in this document focus on the implications for BGP [BGP,RFC1771]. It is important to note that similar issues will arise when considering generalizationssoftware systems, fate sharing refers to theinformation that the IGPs carry. 3. Problem Statement The adventsharing of common resources among a group of applications. In our case, theMP_REACH_NLRI and MP_UNREACH_NLRI attributes, combined withparticular "fate" of most interest is theresulting generalizationability of one application, call it application A, tothe BGP infrastructure, have created the opportunitycause an application with which it is fate sharing, call it application B, touse BGPexperience one or more faults due totransport a wide variety of data types and their associated signaling. The combinationfaults in application A. Fate- sharing can exist at many levels, including between modules on a system, between routing protocols, between sessions of aBGP data type and its associated signaling is frequently called an "application"; examplerouting protocols such as BGP, or between applicationsinclude the IPv4 and IPv6 [RFC2460]within a routingsystems, flow specification rules [FLOW], auto- discovery mechanisms for Layer 3 VPNs [BGPVPN], virtual private LAN services [VPLS], and virtual private Wire Service [VPWS]. More recently,protocol. 5. Architectural Models In this section, we consider thediscussiontwo architectural models which are motivated by salient questions considered in this document, namely: (i). Does theIETF community has focusedBGP distribution protocol suit a particular application (i.e., does an application fit the BGP distribution protocol)? (ii). What are the effects on theuseglobal routing system (if any) of carrying that application using the BGPas a generalized feature transport infrastructure [IETFOL]. The debate has recently intensified due todistribution protocol? These questions must be analyzed in terms of theemergencecost ofa new classprotocol and code development, as well as in terms ofapplication that usestheBGP infrastructure to distribute informationoperational expense thatismay be incurred by utilizing (or notdirectly related to inter-domain routing. Examples of such applications includeutilizing) theuse ofmechanisms already present in BGP. Two models, describing alternate viewpoints, are examined in the following sections. 5.1. General Purpose Transport Infrastructure (GPT) Model The GPT model models BGPtransportdata distribution infrastructureto provide auto-discovery for IP VPNs [RFC2547BIS],as a generic application transport mechanism. As such, it focuses on application fit, and assumes that thevirtual private LAN services mentioned above [VPLS]tradeoffs, both in terms of risk andVPNsinterference can be managed ingeneral [BGPVPN]. 3.1. Risk, Interference, and Application Fit (RIFT)an efficient manner. Asmentioned above, much ofa result, thedebate surroundingGTP models thesenew uses of the BGP transport infrastructure has focused on the potential tradeoffs between the stabilityissues not in terms of whether theInternet routing system, as effected by the deployment of new applications,application andthe desire on thesignaling data that need to be distributed are part ofservice providers to rapidly deploysome particular class (routing, in this case), but rather whether the requirements for the distribution thesenew applications, andattributes are similar enough toreduce the operational cost by re-using existing protocols. These tradeoffs have at times been described in terms of risk, interference, and application fit. Risk modelsthesoftware engineering impactdistribution mechanisms ofnew applications onBGP. In those cases when distribution requirements are sufficiently similar, BGP can be ageneric implementation, while interference modelslogical candidate for a transport infrastructure. Note that this is not because of theimpactnature ofnew applications on protocol definition and behavior. Finally, application fit modelsinformation distributed, but rather due to the similaritybetween an application's data and signaling requirements andin the transport requirements. There are of course other operational considerations that make BGP aspecific distribution algorithm. Each is described below. 3.1.1. Risk: Software Engineering Risk attemptslogical candidate, including its close toassessubiquitous deployment in therobustness tradeoffs inherentInternet (as well as in intra-nets), its policy capabilities, and operator comfort levels with the technology. 5.2. Special Purpose Transport Infrastructure (SPT) Model The SPT model, on theaddition of new applications to a given implementation. That is, riskother hand, models theimpact of generic software engineering issues onBGP infrastructure as agiven implementation. These issues include the impact of new applications on existing implementationsspecial purpose transport designed specifically to transport inter- domain routing information. As such, it is more sensitive to risk and interference than to application fit. There are two basic arguments supporting the SPT model: The first is based on thefate sharing properties of those implementations. A second aspect ofperceived riskliesprofile involved inthe trade-off of extending an existing protocol versus designing, implementing, and deploying aadding newprotocol. 3.1.2. Interference: Protocol Specification/Dynamic Behavior Interference modelsapplications to thepotential for aBGP transport infrastructure or newapplicationfeatures toadversely effect the operation of anexistingimplementation at the protocol level, by inadvertently introducing a detrimental dependency of some kind. That is, an applicationBGP applications. The concern here issaidthat changes to"interfere" with an existing application if, by virtue of the application's protocol extension(s), one or more fundamental properties of the protocol's operation are detrimentally altered. For example, could we create a new state which introduces an unanticipated deadlock situationBGP implementations will cause software quality tooccur? Or could wedegrade, and hence destabilize thedistributed behavior ofglobal routing system. This position is based upon well understood software engineering principles, and is strengthened by long-standing experience that there is a direct correlation between software features and software stability [MULLER1999]. This concern is augmented by theprotocol? Or might we simply run outfact that in many cases, the existence of theattributes or bits available (as happened,code forexample, with RADIUS [RFC2138])? 3.1.3. Application Fit: Distribution Topology Application fit refers to how closely the requirements ofthese features, even if unused, can also cause destabilization in thedata torouting system, since in many cases software faults cannot bedistributed matchisolated. A second concern is based on interference arguments, notably that theunderlying capabilitiesincrease in complexity ofa distribution mechanism. For example, it is clearly inefficientBGP due tobroadcastthe number of datato all peerstypes thatis only required between two peers, just asitis inefficient to unicast (replicate) data that is required by all peers when a single broadcast would do. 4. Definitions 4.1. Reachability Information Reachability information refers to information describing some part of a network, along with how onecarries canreach it, and perhapsalsocontaining attributespotentially destabilize the global routing system. This concern is based on a wide range of concerns, including theimplied path tofact that thenetwork locale. Typically, this information pertains to IP routing information; an exampleinteraction ofnon-IP reachability is VPLS information [VPLS]. 4.2. Layer 3 Routing Information Layer 3 routing information represents either link state information or network reachability information. Link state information represents Layer 3 adjacenciesBGP dynamics andtopology. Link statecurrent deployment practices are poorly understood, and that the addition of non-routing data types may adversely effect convergence and other scaling properties of the global routingprotocols, such as OSPF [RFC2328]system. 6. Analyzing Risk andISIS [RFC1142], flood link state information throughoutInterference One way to frame the tradeoffs involved in a model's risk profile is in terms of the software engineering issues surrounding where anIGP domain, soimplementation might demultiplex among applications. The important point here is thateach participating router maintainsanidentical copyimplementation's choice ofa database that is computed to reflectdemultiplexing point directly affects thecomplete Layer 3 topology. Layer 3 reachability information expressed as an IP address prefix representsimplementation's risk profile due to its effects on existing code, and on thesetsystem resources it requires to be shared among those applications. 6.1. Risk: Code Impact, and Resource Sharing For purposes ofdestinations (systems) whose IP addresses are contained inthis discussion, then, we consider theIP address prefix. Distance/path vector routing protocols, such as BGP, distribute Layer 3 reachability information among routing domains. Routers use both typesrisk profile ofLayer 3 routing information (link statethe SPT andreachability)GPT models with respect toproduce IP forwarding tables. That,their application demultiplexing point. The GPT model typically provides a single point for demultiplexing all applications (i.e., the AFI/SAFI). On the other hand, the SPT model, provides an application demultiplexing point above BGP (typically at the TCP port level). That is, in the GPT model, applications typically share a common transport session, while the SPT model generally envisions one or more applications per transport session (see section 7.1.3 forpurposesa discussion of the impact of multisession BGP [MULTISESSION,SOFTNOTIFY] on thisdiscussion, "routing information" relatestaxonomy). Finally, note that these models can have very different risk profiles with respect to code impact and resource sharing. Some of the questions relating to risk assessment are considered below. 6.1.1. Code Impact In this section, we outline theLayer 3 inter-domain routing data traditionally carried by BGP. Finally, ifhigh-level questions onedefines routing information as "information used to forward packets", combined withmight ask in assessing theabove definition of reachability information, then we can consider information such as describeddifference in[FLOW] (for example)risk between GPT model and the SPT model based on their effect on an existing code base. o Does the code below the demultiplexing point need to berouting information (since itchanged when a new application isattemptingadded? o Does the code in existing applications have toaddbe changed when alevel of granularity to how an 'aggregate'new application isdefined). Thatadded (that is,[FLOW] intends to complementto what extent are theexisting routing information, and the flow information is dependent on IP4 unicast reachability advertised byapplications decoupled)? o Can thesame neighbor. 4.3. Auxiliary (non-routing) Information Auxiliary Information is any information that is exchanged by routers which is neither Layer 3 routing information, nor reachability information. IS-IS hostname TLVs are an example of Axillary information [RFC1142]. 4.4. Address Family Identifier (AFI) An Address Family contains addresses that share common structurecode in separate applications be developed, tested, released, debugged andsemantics. An Address Family Identifier (AFI) uniquely identifies each address family. Several routing protocol messages contain a field that representspackaged independently from other applications? o Is there significant code below theAFI. The AFI identifiesdemultiplexing point that can be shared among all applications? 6.1.2. Resource Sharing In this section, we outline theaddress type used by another data item containedhigh-level questions one might ask inthat message. The Routing Information Protocol (RIP) [RFC2453], Distance Vector Multicast Routing Protocol (DVMRP) [RFC1075],assessing the difference in risk between GPT model andBGP all employtheAFI field. For example,SPT model with respect to theBGP MP_REACH_NLRIrequirements andMP_UNREACH_NLRI attributes contain an AFI field. These BGP attributes also contain a NLRI field that enumerates reachableproperties of the system resource sharing they require. In particular: o Do applications have to compete for socket buffers, and hence have the potential to block orunreachable subnetworks correspondingstarve each other (at the TCP port level)? o Do applications have to compete for possible protocol-level transport-related buffers and queues, and hence have theassociated address family. The AFI field indicatespotential to starve or block each other at theaddress type by which reachable subnetworks are identified. When BGP is usedprotocol send/receive level? o Do applications have todistribute Layer 3 routing information, AFIs can indicatecompete for a possible per-connection processing time budget, hence have thefollowing address types: IPv4, IPv6, VPNv4 [RFC2547BIS]. When BGP is usedpotential todistribute auxiliary information, AFIs can indicatestarve each otheraddress families. 4.5. Subsequent Address Family Identifier (SAFI) A Subsequent Address Family Identifier (SAFI) is part ofat theBGP MP_REACH_NLRIintra-process scheduling level? 6.1.2.1. Resource Sharing andMP_UNREACH_NLRI attributes. These BGP attributes also contain a NLRI field that enumerates reachable or unreachable subnetworks. The SAFI augmentsOperating System Level Issues In this section, we outline theAFI, carrying additional information regarding networks enumeratedhigh-level questions one might ask in assessing theNLRI field. 4.6. Network Layer Reachability Network Layer Reachability Information, or NLRI isdifference in risk between GPT model and thedata described bySPT model based on theAFI/SAFI fields [AFI,SAFI]. While these concepts were originally described for protocols such as DVMRP [RFC1075],affect on resource sharing at thebulk ofoperating system level. In particular: o Do applications share a common scheduling context? That is, do applications have to compete for per-process scheduling budgets? o What is thegeneralizationdegree of fate sharing between applications? 6.2. Interference Interference models theNLRI described in this document derives frompotential for an application to affect theintroductionbehavior ofthe MP_REACH_NLRI and MP_UNREACH_NLRI attributes to BGP [RFC2858]. 4.7. Application The terman existing applicationis usedor applications. For example, inthis document to refer tothecombinationcase of the Internet routing system, one might ask if aBGP data type and any signaling data that is carriedcertain application "interferes" with IPv4 Unicast routing byBGP in supportaffecting some aspect ofthe service the data type carries. The data type is typically describedits protocol operation (e.g., convergence time). Interference inan AFI/SAFI, whiletheactual data is frequently contained in both NLRI and BGP community attributes [RFC1997]. 4.8. Routing Protocol A routing protocol is composed of two basic components: a data distribution algorithm and a decision algorithm. A router typically obtains Layer 3Internet routinginformation viasystem has itsdata distribution algorithm,roots in the observation that the routing system itself can be described as highly self-dissimilar, with extremely different scales andit useslevels of abstraction. Complex systems with thisinformationproperty are susceptible toproduce an IP forwarding table (by applying"coupling", which RFC 3439 [RFC3439] defines as follows: The Coupling Principle states that as things get larger, they often exhibit increased interdependence between components. COROLLARY: The more events that simultaneously occur, theprotocol's decision algorithm tolarger thereceived routing data). Notelikelihood that two or more will interact. This phenomenon has also been termed "unforeseen feature interaction" [WILLINGER2002]. That is, interference, if and where it occurs, has its roots in complexity and is frequently theuse of BGP's data distribution algorithm that is the focusresult ofthis document. However, when judgingapplicationfit, one may also consider whethercoupling. 7. GTP and SPT Models: Risk and Interference In this section, we analyze thedecision algorithms suitrisk and interference profiles of theapplication. 4.9. Fate Sharing The fate sharing principle for end to end network protocols was first enunciated by Dave Clark [CLARK].SPT and GPT models. 7.1. Risk Asapplied to software systems, fate sharing refers tomentioned above, risk models thesharing of common resources among a group of applications. In our case,robustness tradeoffs around generic software architecture and engineering associated with protocol implementations, including theparticular "fate" of most interest isimpact on existing protocol implementations, and on theability of one application, call it application A, to cause an application with which it isfatesharing, call it application B, to experience one or more faults due to faults in application A. Fate-sharingcan exist at many levels, including between modules on a system, between routing protocols, between sessionsproperties of those implementations. In the following sections we consider these components ofa routing protocols such as BGP, or between applications within a routing protocol. 5. Architectural Modelsrisk for both the GPT and SPT models. 7.1.1. Code Impact In this section, weconsideroutline the answers to thetwo architectural models which are motivated by salientquestionsconsidered in this document, namely: (i).posed above. o Does theBGP distribution protocol suitcode below the demultiplexing point need to be changed when aparticular application (i.e., does annew applicationfit the BGP distribution protocol)? (ii). Whatis added? In theory, such code changes are unlikely to be required in theeffects onSPT model, as theglobal routing system (if any) of carryingSPT model envisions that a new applicationusingwill have a new demultiplexing point (port). The GPT model does not by definition require new code below theBGP distribution protocol? These questions must be analyzeddemultiplexing point either. Specifically, it should interms oftheory be possible to isolate code below thecost of protocoldemultiplexing point with suitable abstraction andcode development, as wellconstructs such asin terms of the operational expense that may be incurred by utilizing (or not utilizing)AFI/SAFI API registries. o Does themechanisms already presentcode inBGP. Two models, describing alternate viewpoints,existing applications have to be changed when a new application is added (that is, to what extent areexamined inthefollowing sections. 5.1. General Purpose Transport Infrastructure (GPT) Modelapplications decoupled)? TheGPTSPT modelmodels BGP data distribution infrastructure as a genericenvisions applicationtransport mechanism.independence with respect to demultiplexing point. As such, itfocuses on application fit, and assumesis unlikely to require such changes. However, it is important to note thatthe tradeoffs, both in terms of riskgood software engineering practices encourage code reuse andinterference can be managed in an efficient manner.construction of general purpose libraries. As a result, if applications share libraries and/or other code, theGTP models these issues not in terms of whether the applicationpractical independence decreases, andsignaling data that need toconsequently risk increases. The same analysis can bedistributed are part of some particular class (routing, in this case), but rather whether the requirementsmade for thedistribution these attributesGPT model, since in this case we aresimilar enough toalready demultiplexing on thedistribution mechanisms of BGP. In those cases when distribution requirements are sufficiently similar, BGP canAFI/SAFI fields. o Can the code in separate applications bea logical candidate for a transport infrastructure. Note thatdeveloped, tested, released, debugged and packaged independently from other applications? While this isnot because of the nature of information distributed, but rather due to the similaritytheoretically possible in thetransport requirements. There are of course other operational considerations that make BGP a logical candidate, including its close to ubiquitous deploymentSPT model (and possibly more difficult in theInternet (as well as in intra-nets), its policy capabilities,GPT model) practice andoperator comfort levels with the technology. 5.2. Special Purpose Transport Infrastructure (SPT) Model The SPT model, on the other hand, models the BGP infrastructure as a special purpose transport designed specifically to transport inter- domain routing information. As such, it is more sensitiveexperience has shown that achieving this type of independence is difficult in either model. 7.1.2. Resource Sharing In this section, we address the questions raised above to assess the difference in risk between GPT model andinterference than to application fit. There are two basic arguments supportingthe SPTmodel: The first ismodel based on theperceived risk profile involved in adding neweffect on resource sharing considerations. o Do applications have tothe BGP transport infrastructure or new features to existing BGP applications. The concern here is that changes to BGP implementations will cause software quality to degrade,compete for socket buffers, and hencedestabilize the global routing system. This position is based upon well understood software engineering principles, and is strengthened by long-standing experience that there is a direct correlation between software features and software stability [MULLER1999]. This concern is augmented byhave thefact that in many cases,potential theexistence ofto block or starve each other (at thecodeGPT level)? The SPT model does not require applications to compete forthese features, even if unused, cansocket level resources. It should alsocause destabilization in the routing system, since in many cases software faults cannotbeisolated. A second concern is based on interference arguments, notably that the increase in complexity of BGP duepossible tothe number of data types that it carries can also potentially destabilize the global routing system. This concern is based on a wide rangeachieve this type ofconcerns, including the fact thatapplication independence in theinteraction of BGP dynamicsGPT model with multisession BGP. o Do applications have to compete for possible protocol-level transport-related buffers andcurrent deployment practices are poorly understood,queues, andthathence have theaddition of non-routing data types may adversely effect convergence andpotential to starve or block each otherscaling properties ofat theglobal routing system. 6. Analyzing Risk and Interference One way to frameprotocol send/receive level? Again, while thetradeoffs involved inSPT model does not require competition for transport-level resources, it should be possible to achieve similar behavior with multisession BGP. o Do applications have to compete for amodel's risk profile is in terms of the software engineering issues surrounding where an implementation might demultiplex among applications. The important point here is that an implementation's choice of demultiplexing point directly affectspossible per-connection processing time budget, hence have theimplementation's risk profile duepotential toits effects on existing code, and onstarve each other at thesystem resources it requiresintra-process scheduling level? Applications written tobe shared among those applications. 6.1. Risk: Code Impact, and Resource Sharing For purposes of this discussion, then, we considertherisk profile ofthe SPTand GPT modelsmodel should not require this type of resource competition. It should also be possible to reduce this type of resource competition withrespectmultisession BGP. o Do applications have totheir application demultiplexing point. The GPT model typically provides a single pointcompete fordemultiplexing all applications (i.e.,resources within theAFI/SAFI). Onnetwork (e.g., bandwidth), when theother hand,protocol session spans multiple hops ? Neither the SPTmodel, provides an application demultiplexing point above BGP (typically atmodel nor theTCP port level). That is,GPT model (again, with multisession BGP) should require competition for network resources in this case. 7.1.3. Multisession BGP Suppose that one makes theGPT model, applications typically sharesimplifying assumption that acommon transport session, whileGPT implementation's risk profile is dominated by theSPT model generally envisionsprobability that an error in oneor more applications per transport session (see section 7.1.3 for a discussionAFI/SAFI stream will cause some subset of theimpact of multisession BGP [MULTISESSION,SOFTNOTIFY] onother AFI/SAFI streams to malfunction (e.g., reset). In thistaxonomy). Finally, note that these models can have very differentcase, riskprofiles with respect to code impactmight be characterized as a function of the model andresource sharing. Somethe number of AFI/SAFI carried. Given this simplification, thequestions relating toriskassessment are considered below. 6.1.1. Code Impact In this section,profile looks loosely like Risk = f(Model, |{AFI,SAFI}|) where f:{GPT, SPT} X |{AFI, SAFI}| -> N Note that weoutline the high-level questions one might ask in assessingassume that f(SPT,n) = O(f(GPT,n)) where O(f) = {g:N->R | there exists c > 0 and n such that g(n) < c*f(n)} That is, that thedifference inSPT riskbetweenprofile is bounded by the GPTmodel andrisk profile. Clearly, theSPT model based on their effect onexistence of such anexisting code base. o Doesupper bound is an integral aspect of any argument favoring thecode belowSPT model. Note that for thedemultiplexing point need to be changed whenSPT model, we can think of the number of AFI/SAFI that anew application is added? o Doessingle session carries as a small constant, call it k. k will typically be small (close to 1), since by definition thecodeSPT model envisions a small number of AFI/SAFI per session (e.g., for AFI/SAFI IPv4/unicast and IPv6/unicast, k = 2). When formulated inexisting applications havethis way, one can see that one objective of multisession BGP is tobe changed whenfind anew application is added (thatvalue, call it g, such that f(GPT, g) ~ f(SPT,k), for small values of k (i.e., k close to 1) where A(n) ~ B(k) ==> A(n) = B(k) + h(n), h(n) >= 0 That is, A(n) is approximately equal towhat extent are the applications decoupled)? o Can the code in separate applications be developed, tested, released, debugged and packaged independently from other applications? o Is there significant code below the demultiplexing point that can be shared among all applications? 6.1.2. Resource SharingB(k) In thissection, we outline the high-level questions one might ask in assessing the difference in risk between GPT model andcase, g is theSPT model with respect tosize of therequirementsmultisession AFI/SAFI grouping, andpropertiesfor small values of g, multisession BGP can have a risk profile that looks very much like thesystem resource sharing they require.SPT risk profile. Inparticular: o Do applications have to competeparticular, forsocket buffers, and henceg = 1, both models would havethe potential to block or starve eachsimilar risk profiles. Of course, there are many other(atcomponents of risk that that are not considered by this analysis, such as collateral issues resulting from theTCP port level)? o Do applications have to compete for possible protocol-level transport-related buffers and queues,existence of faulty shared code, operating system process andhence have the potential to starve or block each other atmemory structure, etc. 7.2. Interference Interference concerns stem from theprotocol send/receive level? o Do applications havepossibility that application coupling can lead tocompete for a possible per-connection processing time budget, hence havethepotential to starve each other atdestabilization of theintra-process scheduling level? 6.1.2.1. Resource SharingInternet routing system in unanticipated andOperating System Level Issuesunexpected ways. In thissection,section weoutlineconsider interference properties of thehigh-level questionsGPT and SPT models. 7.2.1. Multisession BGP Multisession BGP also seeks to reduce the interference profile of the GPT model by eliminating onemight askpotential source of interference, namely, the potential interference due to presence of multiple AFI/SAFIs inassessinga single BGP session. Following thedifferenceanalysis presented inrisk between GPT modelsection 7.1.3, we can see that for small groupings (described as small values of g in section 7.1.3), the interference profiles of both models converge. 8. Application Fit In the following sub-sections, application fit is examined from the perspective of analyzing the signaling and data distribution needs of three representative applications, namely: RFC 2547 Style VPNs VPWS VPLS However, before investigating how theSPT model based onBGP data distribution mechanism (and its extensions) fit theaffect on resource sharing atrequirements of these applications, it is useful to briefly review theoperating system level.gross characteristics of the BGP data distribution infrastructure. Inparticular: o Do applications shareparticular, we examine which distribution topologies can be naturally built using internal BGP (or iBGP). iBGP has been described loosely as a broadcast mechanism since an iBGP speaker sends information to all its peers. This is typically achieved by means of one or more route reflectors (or RRs); acommon scheduling context? That is, do applicationsmore direct but less scalable means is for each iBGP speaker to have a BGP session with each iBGP peer. It may, however, be more accurate tocompete for per-process scheduling budgets? o Whatcharacterize iBGP as a constrained broadcast mechanism. This is because thedegreeuse offate sharing between applications? 6.2. Interference Interference models the potential forcommunities in conjunction with import and export policies allows anapplicationiBGP speaker toaffect the behavioreffectively limit its communication to a subset ofan existing application or applications. For example, inthecasefull set of iBGP peers; theInternet routing system, one might ask if a certain application "interferes" with IPv4 Unicast routing by affecting some aspectefficiency ofits protocol operation (e.g., convergence time). Interference in the Internet routing system has its roots in the observation that the routing system itselfconstrained broadcast can bedescribedimproved by techniques such ashighly self-dissimilar, with extremely different scalesdescribed in [ORF] andlevels of abstraction. Complex systems with this property[RTCONST]. 8.1. RFC 2547 Style VPNs There aresusceptiblefive classes of information that need to"coupling", whichbe distributed for RFC3439 [RFC3439] defines as follows: The Coupling Principle states that as things get larger, they often exhibit increased interdependence between components. COROLLARY: The more events that simultaneously occur, the larger the likelihood that two or more will interact. This phenomenon has also been termed "unforeseen feature interaction" [WILLINGER2002]. That is, interference, if and where it occurs, has its roots in complexity2547 style VPNs: (a). Membership (auto-discovery) (b). Prefixes (c). Labels (d). BGP nexthop, and (e). Path selection attributes The first of these, membership or auto-discovery, must be sent to all peers, as a BGP speaker does not know a priori which of its peers are members of a given VPN. Membership of a given VPN isfrequentlyrecognized by theresultuse ofapplication coupling. 7. GTP and SPT Models: Risk and Interference Inextended communities called Route Targets. BGP is well- suited for thissection, we analyze the risk and interference profilesmode of distribution. The next three of these constitute theSPT and GPT models. 7.1. Risk As mentioned above, risk models the robustness tradeoffs around generic software architecturereachability information. They say what part of a given VPN (b) is reachable, andengineering associated with protocol implementations, including the impact on existing protocol implementations,how it is to be reached (c andon the fate sharing propertiesd). The final piece ofthose implementations. Ininformation is used for selection if there are multiple paths to a given prefix of a VPN, as in thefollowing sections we considercase of multi-homing. All of thesecomponentspieces of information need only be distributed to members ofrisk for boththeGPT and SPT models. 7.1.1. Code Impact InVPN, i.e., they require a constrained broadcast mechanism. BGP is reasonably well-suited for thissection, we outlinemode of distribution using import and export NLRI filtering. The addition of theanswersmechanism in [RTCONST] makes BGP even better suited to this. The encoding of this information as defined in [RFC2547BIS] puts all of this information in a single NLRI. This seems to imply that a broadcast mechanism has to be used for thequestions posed above. o Does the code belowdistribution of RFC 2547 VPN information. However, thedemultiplexing point needcombination of [RTCONST] and [RFC2918] allow BGP tobe changed when a new application is added?distribute this information correctly yet efficiently. Intheory, such code changes are unlikelysummary, there seems to berequired in the SPT model, as the SPT model envisionslittle argument thata newthe RFC 2547 applicationwill haveis anew demultiplexing point (port). The GPT model does not by definition require new code belowrouting application. This is because thedemultiplexing point either. Specifically, it shouldinformation that gets sent via BGP intheory be possibleRFC 2547 is generally considered toisolate code belowbe "routing information". That is, thedemultiplexing pointprotocol distributes address prefixes, along withsuitable abstraction and constructs such as AFI/SAFI API registries. o Does the code in existing applications havetheir next hops (and of course, some additional attributes). Finally (and perhaps most importantly), there seems to bechanged when a new application is added (that is, to what extent arelittle argument that theapplications decoupled)? The SPT model envisionsinformation distributed by the RFC 2547 applicationindependenceis standard routing information. 8.1.1. RFC 2547 and Label Distribution One issue that is frequently raised with respect todemultiplexing point. As such, it is unlikely to require such changes. However, itwhether or not the RFC 2547 VPN application isimportant to note that good software engineering practices encourage code reuse and construction of general purpose libraries. Asaresult, if applications share libraries and/or other code,routing application surrounds the fact that, in the 2547 application, BGP distributes MPLS labels along with thepractical independence decreases, and consequently risk increases.routes. Thesame analysis can be made forcontention then, is that theGPT model, sinceRFC 2547 application represents more than just a routing application. However, in this casewe are already demultiplexing ontheAFI/SAFI fields. o CanMPLS label is just a shorthand way of representing one or more address prefixes. That is, thecodeassertion is that inseparate applications be developed, tested, released, debugged and packaged independently from other applications? Whilethis case, the label represents "standard routing information". 8.2. VPWS The question of whether VPWS istheoretically possible ina "good fit" for theSPT modelBGP transport infrastructure is the source of much discussion (andpossiblycontroversy). In this section, we will review both positions and their supporting arguments as a series of assertions and counter-assertions (we will use this format throughout the rest of this section). The key debate with respect to VPWS centers around what set of services are being defined, and how they are to be signaled. One way to analyze the VPWS application, then is in terms of two of its moredifficultcontentious functionalities, namely: (a). Auto-discovery Auto-discovery refers to discovery of the set of nodes that belong in a common L2VPN, and (b). Signaling Signaling refers to theGPT model) practicesetup andexperience has shownmaintenance of the point-to-point pseudo-wires thatachieving this typecarry the traffic ofindependencethe L2VPN. The next sections examine the various assertions and counter- assertions around auto-discovery and signaling for VPLS. 8.2.1. Assertion #1 Assertion #1 states VPWS isdifficult in either model. 7.1.2. Resource Sharing Innot a routing application. Those supporting thissection,assertion argue that in the case of VPWS, we are not distributing addressthe questions raised above to assess the difference in risk between GPT modelprefixes, and (importantly) unlike theSPT model based oncase of RFC 2547 style VPNs, the BGP decision process is not used (or at least it is not used in theeffect on resource sharing considerations. o Do applications havesame way). Further, proponents argue that what we are distributing is state information that corresponds tocompete for socket buffers,point-to-point entities, i.e., pseudo-wires, andhence have the potentialthus argues that that the VPWS application is completely different. 8.2.2. Counter-Assertion #1 Counter-Assertion #1 states that VPWS is a routing application. More specifically, this position is outlined in [VPLS] (section 3.4), namely: "It is often desired toblock or starve each other (at the GPT level)? The SPT model does not require applicationsmulti-home a VPLS site, i.e., tocompete for socket level resources. It should also be possibleconnect it toachieve this type of application independencemultiple PEs, perhaps even in different ASes. In such a case, theGPT model with multisession BGP. o Do applications havePEs connected tocompete for possible protocol-level transport-related buffers and queues, and hence havethepotential to starve or block each other atsame site can either be configured with theprotocol send/receive level? Again, whilesame VE ID or with different VE IDs. In theSPT model does not require competition for transport-level resources,latter case, itshould be possibleis mandatory toachieve similar behavior with multisession BGP. o Do applications haverun STP on the CE device, and possibly on the PEs, tocompete forconstruct apossible per-connection processing time budget, hence haveloop-free VPLS topology. In thepotential to starve each other atcase where theintra-process scheduling level? Applications writtenPEs connected to the same site are assigned theSPT model should not require this type of resource competition. It should also be possible to reduce this type of resource competition with multisession BGP. o Do applications havesame VE ID, a loop-free topology is constructed by routing mechanisms, in particular, by BGP path selection. When a BGP speaker receives two equivalent NLRIs (see below for the definition), it applies standard path selection criteria such as Local Preference and AS Path Length tocompete for resources withindetermine which NLRI to choose; it MUST pick only one. If thenetwork (e.g., bandwidth), whenchosen NLRI is subsequently withdrawn, theprotocol session spans multiple hops ? NeitherBGP speaker applies path selection to theSPT model norremaining equivalent VPLS NLRIs to pick another; if none remain, theGPT model (again,forwarding information associated withmultisession BGP) should require competition for network resources in this case. 7.1.3. Multisession BGP Suppose that one makes the simplifying assumptionthata GPT implementation's risk profileNLRI isdominated by the probabilityremoved." 8.2.3. Assertion #2 Assertion #2 states thatan error in one AFI/SAFI stream will causeauto-discovery for VPWS requires somesubsetform ofthe other AFI/SAFI streamsconstrained broadcast. There doesn't seem tomalfunction (e.g., reset).be much controversy that auto-discovery does require some sort of constrained broadcast mechanism (which we don't want to be limited to a single AS), and we may want to be able to optimize it by using a RP (rendezvous point) like mechanism. BGP route reflectors (RR) provide a convenient and ubiquitously deployed candidate RP. In thiscase, risk might be characterizedcase (RR as RP), the fit is good since auto-discovery, like routing, requires an n-party protocol where each party has no afunctionpriori knowledge of themodel and the numberexistence or identity ofAFI/SAFI carried. Given this simplification,therisk profile looks loosely like Risk = f(Model, |{AFI,SAFI}|) where f:{GPT, SPT} X |{AFI, SAFI}| -> N Note that we assumeother n-1 parties. 8.2.4. Counter-Assertion #2 There is no real counter-position to Assertion #2, as it simply states thatf(SPT,n) = O(f(GPT,n)) where O(f) = {g:N->R |VPWS auto-discovery requires some form of constrained broadcast (about which thereexists c > 0 and n such that g(n) < c*f(n)} That is,is some controversy; see Assertion #2a below). 8.2.4.1. Assertion #2a Assertion #2a states that auto-discovery is not needed for VPWS. Further, theSPT risk profileAssertion #2a states that there is not a validated need for VPWS auto-discovery, since auto-discovery is useful only when creating full mesh layer 2 topologies, which are undesirable due to their (well-understood) poor scaling properties; hence auto-discovery for VPWS is not useful. 8.2.4.2. Counter-Assertion #2a <what isbounded bytheGPT risk profile. Clearly,counter-assertion here? auto-discovery is useful for partial mesh? cite?> In summary, with theexistenceexception ofsuch an upper boundAssertion #2a, the major controversy surrounding VPWS isan integral aspectin signaling piece ofany argument favoringtheSPT model. Noteapplication. The "VPWS is not a routing application" camp argues thatfortheSPT model, we can think ofVPWS signaling requirements do not fit thenumber of AFI/SAFIBGP distribution infrastructure, while the "VPWS is a routing application" camp believes that BGP is asingle session carries asgood fit. The next sections examine these assertions. 8.2.5. Assertion #3 Assertion #3 states that VPWS applications are not asmall constant, call it k. k will typicallygood fit for BGP. This argument is based on the assertion that BGP is poorly suited to the VPWS signaling requirements because pseudo-wires are inherently point-to-point (see, for example [L2VPNSIG]). Further, the assertion is that VPWS signaling is qualitatively different than in routing or auto-discovery, in which each piece of information must besmall (closedistributed to1), since by definitiontheSPT model envisionsn participants. The conclusion here is that BGP's distribution mechanisms are asmall number of AFI/SAFI per session (e.g.,poor match forAFI/SAFI IPv4/unicast and IPv6/unicast, k = 2). When formulated inVPWS signaling. Another way to think about thisway, one can seeis thatone objective of multisessionBGP generally works from a single database, and then applies some filtering on a per-connection basis; this only makes sense if most of the information is going to go tofindavalue, call it g, such that f(GPT, g) ~ f(SPT,k), for small valueslot ofk (i.e., k close to 1) where A(n) ~ B(k) ==> A(n) = B(k) + h(n), h(n) >= 0 That is, A(n)places. For example, suppose that a RR isapproaches B(k)used for VPWS signaling, and there is the need to set up n pseudo-wires. In this case,g isinstead of sending n setup messages, one sends one large "meta-setup" message with all the info that would have been in the n setup messages. That is, let n = number of pseudo-wires l = the size of themultisession AFI/SAFI grouping, and for small valuesper-wire label information k = the size ofg, multisession BGP canthe per-wire information In this case, the meta-setup message will be of size O((l + k) * n). After receiving the setup message, the RR then must send the n messages that could havea risk profilebeen sent by the endpoint (note thatlooks very much likethis is almost true; theSPT risk profile. In particular, for g = 1, both modelsendpoint would havesimilar risk profiles. Of course, there are many other componentsto send n messages ofrisk thatsize (l + k), but the RR will have to send n copies of the larger setup message). 8.2.6. Counter-Assertion #3 Counter-Assertion #3 states thatare not considered bythe VPWS application is a good fit for BGP (see, for example [L2VPNT]). In particular, thisanalysis, suchcamp suggests that a RR really only needs to distribute the label-range [LABELRANGE], so the setup message isn't really n times ascollateral issues resulting fromlarge, but rather is analyzed as follows: Let n = number of pseudo-wires m = theexistencesize offaulty shared code, operating system processthe label-range data k = the size of the per-wire information Then the messages will be of size O(m + (k * n)), andmemory structure, etc. 7.2. Interference Interference concerns stem frommost importantly for thepossibilitylabel-range argument: O(m + (k * n)) < O((l + k) * n) That is, the label-range concept reduces the size of the messages thatapplication coupling can leadneed to be sent to and by thedestabilization ofRR. However, some will argue that the label-range concept is efficient if and only if: (a). A large enough label range is preallocated to accommodate all the systems you might ever want to add to theInternet routing system in unanticipatedVPLS/VPWS (assuming that service interruption is not acceptable), andunexpected ways.(b). There is no per-wire information other than labels that needs to distributed Inthis section we consider interference properties ofthese cases, theGPT and SPT models. 7.2.1. Multisession BGP Multisession BGP also seeks tolabel range approach can reduce theinterference profile of the GPT model by eliminating one potential sourcesize ofinterference, namely,thepotential interference due to presence of multiple AFI/SAFIs in a single BGP session. Followingsetup messages as analyzed above. However, theanalysis presented in section 7.1.3, we can seecounter argument is thatfor small groupings (describedany such reduction will become a second-order effect assmall values of g in section 7.1.3), the interference profilessoon as some other piece ofboth models converge. 8. Application Fitper-wire status or configuration (e.g., MTU) information must be distributed. In addition, thefollowing sub-sections, application fit is examined from the perspective of analyzing the data distribution needs of three representative classesidea ofapplication, namely: RFC 2547 Style VPNs VPWS VPLS 8.1. RFC 2547 Style VPNs First, it is usefulpre- allocating a large enough label range toreview the distribution mechanisms available in BGP, in particular,accommodate future expansion, while saving bits ini-BGP. i-BGPthe setup messages, hasbeen described loosely asother costs which may be large. In particular: (a). Until the future expansion takes place (if it ever does), one may be wasting quite abroadcast mechanism since an i-BGP speaker sends informationlot of labels (noting that that each label you distribute requires you toall its peers. This is typically achieved by meansallocate a piece of high-speed memory in your forwarding engine; putting some of it aside for possible later use seems very costly. Each one you put aside is, e.g., oneor more route reflectors; a more direct butlessscalable means isRFC2547 route you can support). However, if you don't preallocate enough contiguous label space foreach i-BGP speaker tofuture expansion, then if the expansion occurs you must start adding additional labels or label ranges, and your setup messages start getting longer anyway (in theory, you could just carve a new set of label ranges, instead of adding new ones; counter-position: if you did that, you'd have to bring down your whole VPWS (and possibly VPLS) every time you add aBGP session with each i-BGP peer. However,new endpoint). (b). Fragmentation of the label space, which can result from this preallocation, has real impact on label switching implementations (as the MPLS architecture explicitly leaves itis more accuratetocharacterize i-BGP as a constrained broadcast mechanism. This is becausetheuse of communities in conjunction with import and export policies allows an i-BGP speakerimplementation toeffectively limitdevelop itscommunication toown label assignment strategies). So if, for example, asubset of the full set of i-BGP peers; the efficiency of constrained broadcasthardware designer thinks s/he canbe improvedimprove performance bytechniques such as describedusing, say, prime numbered labels first, s/he should have the ability to design her/his system in[ORF]this way. If an application is going to come along and[RTCONST]. There are five classes of informationdemand thatneedlabels be assigned in contiguous groups, implementations which are perfectly conformant to the architecture may not bedistributed for RFC 2547 style VPNs: (a). Membership (auto-discovery) (b). Prefixesable to support that application. (c).Labels (d). BGP nexthop, and (e). Path selection attributes The firstFor diagnosis ofthese, membership or auto-discovery, must be sent to all peers, as a BGP speaker doesnetwork problems, the label-range approach may have the additional issue that the operator may not knowa priori(a priori) whichof itslabel(s) were assigned to which endpoint(s). (d). Finally, one may argue that label-range allocation is sub-optimal for non-full mesh topologies, since all peersare membersof the VPN must hear about the agiven VPN. Membership oflabel-range withdraw, and (in agiven VPN is recognized bynon-full mesh topology), not all peers need to know about it. In any event, one may argue that theusescaling benefits ofcertain extended communities called Route Targets. BGPusing a RR in routing isclearly eminently well-suited for this mode of distribution. The next threethat the RR pre-digests all the received info; it runs the (BGP) decision process, and only forwards the results ofthese constitutethereachability information. They say what partdecision process, rather than forwarding all the raw data. In the case ofa given VPN (b)VPWS (and possibly VPLS), the argument isreachable,that this advantage is absent (i.e., we don't run BGP path selection), andhowas a result, the RR doesn't help with scaling in the same way it does with routing. Of course, the counter position isto be reached (c and d). The final piecethat some form ofinformationBGP path selection isusedused; see discussion above). Finally, one may argue that using the RR will introduce some latency into the label withdraw procedure. 8.3. VPWS and Per-Wire Attributes While several per-wire attributes have been defined (see [L2TPV3], forselection if thereexample), the need for per-wire attributes for VPWS remains controversial. The following sections examine those controversies. 8.3.1. Assertion #4 Assertion #4 is that VPWS requires various per-wire parameters. These may include (but aremultiple pathsnot limited to) MTU, whether toa given prefixuse sequencing capabilities, bandwidth capabilities, and QoS. In addition, during the lifetime of aVPN,pseudo-wire, there are per-wire status indications that may need to be passed to the other endpoint. 8.3.2. Counter-Assertion #4: Counter-Assertion #4 states that it has not been demonstrated that VPWS needs per-wire attributes as few (per-wire attributes) have as yet been defined (see, e.g., [MARTINI]). 8.3.3. Assertion #5 Assertion #5 states that passing per-wire attributes through an RR will likely be inefficient. The argument here is that in thecase of multi-homing. All ofevent that per-wire attributes are required, passing thesepieces of information need only(per-wire) attributes through a RR will bedistributed to members ofsub-optimal as theVPN, i.e., they require a constrained broadcast mechanism. BGP is reasonably well-suited for this mode of distribution using import and export NLRI filtering. The addition ofRR will forward themechanism in [RTCONST] makes BGP even better suitedstatus tothis. The encoding of this information as defined in [RFC2547BIS] putsallof this information in a single NLRI. This seemsthe VPWS members, not just toimplythe one endpoint thata broadcast mechanismis interested in it. For attributes like sequence numbers, it may even more difficult as one has tobe used formake sure thedistribution of RFC 2547 VPN information. However,sequence numbers resynchronize properly when thecombination of [RTCONST] and [RFC2918] allow BGPpseudo-wire flaps. This seems somewhat difficult todistribute this information correctly yet efficiently. Finally,achieve through a BGP RR. 8.3.4. Counter-Assertion #5 The counter assertion here is that, since few (or no) per-wire attributes have been defined (counter-assertion #4), the fact that it isusefulinefficient toobserve that standarduse a RR for distribution is irrelevant. 8.3.5. Assertion #6 Assertion #6 states that, while still an open issue, pseudo-wire congestion control may require regular point-to-point control message exchanges, something which BGPpath selection mechanisms (local pref, MED, AS path length, etc.) can be appliedwould seem ill-equipped to handle. 8.3.6. Counter-Assertion #6 In this case, theinformation in (e). The conclusioncounter assertion is thatBGPsince few (or no) per- wire attributes have been defined (see counter-assertion #4), and further, since congestion control for pseudo-wires isquite well-suitedstill an open issue, arguing fit is premature. 8.3.7. Assertion #7: Assertion #7 states that the primary motivation for VPWS is to deliver existing service models (i.e., Frame Relay and ATM) over a packet infrastructure (this is as opposed to some new service). In thisapplication, and,case, common deployments involve partial mesh topologies (more specifically multiple hub and spoke connections, with some hub to hub connectivity that makes sense for theadditionenterprise traffic profile). In addition, some ofmechanismsthe connections in suchas [RTCONST] and [RFC2918],deployments require per- wire characteristics (e.g., guaranteed throughput for voice, etc). In other words, thefitargument here iseven closer. 8.2.that a VPWSA VPN based onservice designed to support so-called legacy services (Frame Relay and ATM) will require point-to-point signaling due to existing topologies and the need for per-wire attributes. Further, for new VPWS services that require full-mesh auto-provisioning, the "Colored Pools PW Provisioning Model" [L2VPN] suggests aVirtual Private Wire Service [VPWS]method to support such provisioning while retaining the point-to-point signaling required to support per-wire attributes. 8.3.8. Counter-Assertion #7: <what is the counter-assertion here?> 8.4. VPLS A VPLS service connects anumbernumber of sites by an emulated LAN segment. In the next sections, we examine whether VPLS maybe be considered to be a routing application, and hence whether BGP is a good fit for its distribution requirements. 8.4.1. Assertion #8 Assertion #8 states that VPLS is a routing application, since the notion ofsites by virtual wires (or pseudo-wires). The information needed"VPLS site identification" is analogous tocreate sucha VPNcomprises: (a). Membership (auto-discovery) (b). VPNsiteidentification (c). Labels (d). BGP nexthop (e). Path selection attributes, and (f). Per-wire information Theidentifier for VPWS (which this camp also views as a routing application). As a result, the analysis of thefirstdistribution needs of these five items is exactly as for RFC 2547 VPNs,with the slight change thatand thedefinition of a 'part of a VPN'conclusion isno longer an IP prefix, butthat BGP isa VPN site identifier, which can be viewed as the VPWS prefix. The distribution requirementsreasonably well-suited for this application, andthe fitwithBGP distribution mechanisms is identical to RFC 2547. The one major change isthepotential for 'per-wire' attributes, such as bandwidth for a given site-to-site connection. This information should be distributed on a point-to-point basis. BGP mechanisms are not efficient for point-to-point distribution. However, it is an open question whether such 'per-wire' attributes really need to be exchanged, as evidenced byaddition of [RTCONST] and [REFRESH], thefact that LDP signaling for pseudo- wires [MARTINI] has not defined any such attributes. If per-wire informationfit isindeed not necessary, BGP distribution mechanisms are as well-suited for VPWS VPNs as for RFC 2547 VPNs. Noteeven better. Finally, note that existing BGP path selection mechanisms can be used as is forVPWS,VPLS, and can prove useful for multi-homed sites.8.3. VPLS A8.4.2. Counter-Assertion #8 Counter-Assertion #8 states that VPLSconnectsis not anumber of sites by an emulated LAN segment. The information neededrouting application. In particular, the contention here is that while the VPLS NLRI are used tocreateidentify that aVPLS consists of: (a). Membership (auto-discovery) (b). VPLS site identification (c). Labels (d). BGP nexthop, and (e). Path selection attributes The notion of 'VPLS site identification' is analogousparticular PE belongs to aVPN site identifier for VPWS. The analysis ofparticular VPLS instance (as described in Assertion #8),the path which data traffic follows will depend on thedistribution needs of these five items is exactly as for RFC 2547 VPNs,route to that PE, and that route is determined by theconclusionordinary IP routing. As a result, it is not relevant which neighbor a VPLS NLRI was received from, and hence is not routing. 8.4.3. Assertion #9 Assertion #9 is thatBGPconstrained or true broadcast isreasonably well-suitednot valuable forthis application, and withVPLS, since theaddition of [RTCONST] and [REFRESH],same label can not be used by all peers. In particular, thefit is even better. Note that existing BGP path selection mechanismssame label can not be usedasby all peers since MAC address learning isfor VPLS, and can prove useful for multi-homed sites.performed in the data plane. 8.4.4. Counter-Assertion #9 <what is the counter-assertion here? label ranges?> 9. Operational Implications In this section we examine the operational implications of the various choices in the design spaces described in this document. 9.1. OAM Functionality A service provider (SP) may want to know exactly where a particular pseudo-wire leaves its domain, and in addition may want to keep various counts and bits of status at that point. Further, the SP may want to be able to do data path testing to that point. That is, a SP may want point-to-point pseudo-wire state to be maintained at its border routers. 9.1.1. Assertion #10: Assertion #10 states that it may be difficult for service providers to maintain point-to-point pseudo-wire state at their border routers with the proposed BGP signaling mechanisms. This is because those mechanisms provide no way to ensure that a pseudo-wire data path will leave the network at a node which has state information for that pseudo-wire. 9.1.2. Counter-Assertion #10: <counter assertion?> 9.2. Full-Mesh Issues 10.Other Models 11.Conclusions and Recommendations12.11. Intellectual Property The IETF takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Information on the IETF's procedures with respect to rights in standards-track and standards-related documentation can be found in BCP-11 [RFC2028]. Copies of claims of rights made available for publication and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementors or users of this specification can be obtained from the IETF Secretariat. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights which may cover technology that may be required to practice this standard. Please address the information to the IETF Executive Director.13.12. Design Team The design team that produced this document consisted of Daniel Awduche (awduche@awduche.com), Ron Bonica (Ronald.P.Bonica@mci.com), Hank Kilmer (hank@rem.com), Kireeti Kompella (kireeti@juniper.net), Chris Lewis (chrlewis@cisco.com), Danny McPherson (danny@tcb.net), David Meyer (dmm@1-4-5.net) and Peter Whiting (pwhiting@vericenter.com).14.13. Acknowledgments David Ball, Peter Gutierrez, Susan Harris, Pedro Marques, Eric Rosen, Pekka Savola, and Mark Townsley have all made many insightful comments on earlier versions of this document.15.14. Security Considerations This document specifies neither a protocol nor an operational practice, and as such, it creates no new security considerations.16.15. IANA Considerations This document creates a no new requirements on IANA namespaces [RFC2434].17.16. References17.1.16.1. Normative References [AFI] http://www.iana.org/assignments/address-family-numbers [BGP] Rekhter, Y, T.Li, and S. Hares, "A Border Gateway Protocol 4 (BGP-4)", draft-ietf-idr-bgp4-23.txt. Work in progress. [BGPVPN] Ould-Brahim, H., E. Rosen, and Y. Rekhter, "Using BGP as an Auto-Discovery Mechanism for Provider-provisioned VPNs", draft-ietf-l3vpn-bgpvpn-auto-00.txt. Work in progress. [CLARK] Clark, D., "Design Philosophy of the DARPA Internet Protocols", Computer Communication Review, volume 25, number 1, January 1995. ISSN # 0146-4833. [EXTCOMM] Sangali, S., D. Tappan, and Y. Rekhter, "BGP Extended Communities Attribute", draft-ietf-idr-bgp-ext-communities-06.txt. Work in progress. [FLOW] Marques, P, et. al., "Dissemination of flow specification rules", draft-marques-idr-flow-spec-00.txt. Work in progress. [LABELRANGE] What is the cite here? [L2VPN] Andersson, L. and E. Rosen, "L2VPN Framework", draft-ietf-l2vpn-l2-framework-03.txt. Work in Progress. [L2VPNSIG] Rosen, E. and V. Rodoaca, "Provisioning Models and Endpoint Identifiers in L2VPN Signaling", draft-ietf-l2vpn-signaling-00.txt. Work in Progress. [L2VPNT] Kompella, K. (Editor), "Layer 2 VPNs Over Tunnels", draft-kompella-l2vpn-l2vpn-00.txt. Work in Progress. [L2TPv3] Lau, J., M. Townsley and I. Goyret (Editors), "Layer Two Tunneling Protocol (Version 3)", draft-ietf-l2tpext-l2tp-base-11.txt. Work inprogress.Progress. [MARTINI] Martini, L., E.Rosen, and T. Smith, "Pseudowire Setup and Maintenance using LDP", draft-ietf-pwe3-control-protocol-05.txt. Work in progress. [MULLER1999] Muller, R. et. al., "Control System Reliability Requires Careful Software Installation Procedures", International Conference on Accelerator and Largeand Large Experimental Physics Systems, 1999, Trieste, Italy. [MULTISESSION] Scudder, J. and C. Appanna, "Multisession BGP, draft-scudder-bgp-multisession-00.txt. Work in progress. [ORF] Chen, E., and Rekhter, Y., "Cooperative Route Filtering Capability for BGP-4", draft-ietf-idr-route-filter-09.txt. Work in progress. [RTCONST] Bonica, R. et al, "Constrained VPN route distribution", draft-marques-ppvpn-rt-constrain-01.txt. Work in progress. [SOFTNOTIFY} Nalawade, G., K. Patel, J. Scudder, and D. Ward, "BGPv4 Soft-Notification Message", draft-nalawade-bgp-soft-notify-00.txt., Work in progress. [RFC1075] Waitzman, D., C. Partridge, and S. Deering, "Distance Vector Multicast Routing Protocol", RFC 1075, November, 1988. [RFC1142] Oran, D. Editor, "OSI IS-IS Intra-domain Routing Protocol", RFC 1142, February, 1990. [RFC1771] Rekhter, Y., and T. Li, "A Border Gateway Protocol 4 (BGP-4)", RFC 1771, March 1995. [RFC1958] Carpenter, B., "Architectural principles of the Internet", Editor. RFC 1958, June 1996. [RFC1997] Chandra, R., P. Traina, and T. Li, "BGP Communities Attribute", RFC 1997, August, 1996. [RFC2138] Rigney, C., et. al., "Remote Authentication Dial In User Service (RADIUS)", RFC 2138, April, 1997. [RFC2328] Moy, J., "OSPF Version 2", RFC 2328, April, 1998. [RFC2453] Malkin, G., "RIP Version 2", RFC 2453, November, 1998. [RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6 (IPv6) Specification", RFC 2460, December, 1998. [RFC2547BIS] Rosen, E., et. al., "BGP/MPLS IP VPNs", draft-ietf-l3vpn-rfc2547bis-00.txt. Work in progress. [RFC2858] Bates, T., et. al., "Multiprotocol Extensions for BGP-4", RFC 2858, June 2000. [RFC2918] Chen, E., "Route Refresh Capability for BGP-4", RFC 2918, September 2000. [RFC3036] Anderson, L., et. al., "LDP Specification", RFC 3036, January 2001. [RFC3439] Bush, R. and D. Meyer, "Some Internet Architectural Guidelines and Philosophy", RFC 3439, December, 2002. [SAFI] http://www.iana.org/assignments/safi-namespace[VLPS][VPLS] Kompella, K., et. al. "Virtual Private LAN Service",draft-ietf-l2vpn-vpls-bgp-02.txt.draft-ietf-l2vpn-vpls-bgp-01.txt. Work in progress. [VPWS] Kompella, K. et.al. "Layer 2 VPNs Over Tunnels", draft-kompella-ppvpn-l2vpn-04.txt. Work in progress.17.2.16.2. Informative References [IETFOL] https://www1.ietf.org/mailman/listinfo/routing-discussion [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", RFC 2119, March, 1997. [RFC2026] Bradner, S., "The Internet Standards Process -- Revision 3", RFC 2026/BCP 9, October, 1996. [RFC2028] Hovey, R. and S. Bradner, "The Organizations Involved in the IETF Standards Process", RFC 2028/BCP 11, October, 1996. [RFC2434] Narten, T., and H. Alvestrand, "Guidelines for Writing an IANA Considerations Section in RFCs", RFC 2434/BCP 26, October 1998. [RVBIB] http://www.routeviews.org/papers [WILLINGER2002] Willinger, W., and J. Doyle, "Robustness and the Internet: Design and evolution", 2002.18.17. Editor's Address David Meyer Email: dmm@1-4-5.net19.18. Full Copyright Statement Copyright (C) The Internet Society (2004). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.Meyer, et. al.