MBONED Working Group P. Tarapore, Ed. Internet-Draft R. Sayko Intended status: Best Current Practice AT&T Expires: April 30, 2018 G. Shepherd Cisco T. Eckert, Ed.Futurewei TechnologiesHuawei R. Krishnan SupportVectors October 27, 2017 Use of Multicast Across Inter-Domain Peering Pointsdraft-ietf-mboned-interdomain-peering-bcp-12draft-ietf-mboned-interdomain-peering-bcp-13 Abstract This document examines the use of Source Specific Multicast (SSM) across inter-domain peering points for a specified set of deployment scenarios. The objective is to describe the setup process for multicast-based delivery across administrative domains for these scenarios and document supporting functionality to enable this process. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on April 30, 2018. Copyright Notice Copyright (c) 2017 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling the copyright in such materials, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English.Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Overview of Inter-domain Multicast Application Transport . . 5 3. Inter-domain Peering Point Requirements for Multicast . . . . 6 3.1. Native Multicast . . . . . . . . . . . . . . . . . . . .67 3.2. Peering Point Enabled with GRE Tunnel . . . . . . . . . .78 3.3. Peering Point Enabled with an AMT - Both Domains Multicast Enabled . . . . . . . . . . . . . . . . . . . .910 3.4. Peering Point Enabled with an AMT - AD-2 Not Multicast Enabled . . . . . . . . . . . . . . . . . . . . . . . . .1012 3.5. AD-2 Not Multicast Enabled - Multiple AMT Tunnels Through AD-2 . . . . . . . . . . . . . . . . . . . . . . . . . .1114 4. Functional Guidelines . . . . . . . . . . . . . . . . . . . .1316 4.1. Network Interconnection Transportand SecurityGuidelines13. . . . . . 16 4.1.1. Bandwidth Management . . . . . . . . . . . . . . . . 16 4.2. Routing Aspects and Related Guidelines . . . . . . . . .1418 4.2.1. Native Multicast Routing Aspects . . . . . . . . . .1519 4.2.2. GRE Tunnel over Interconnecting Peering Point . . . .1519 4.2.3. Routing Aspects with AMT Tunnels . . . . . . . . . .1620 4.2.4. Public Peering Routing Aspects . . . . . . . . . . . 22 4.3. Back Office Functions - Provisioning and Logging Guidelines . . . . . . . . . . . . . . . . . . . . . . .1823 4.3.1. Provisioning Guidelines . . . . . . . . . . . . . . .1824 4.3.2.Application AccountingInterdomain Authentication Guidelines . . . . . . . .. . 2025 4.3.3. Log Management Guidelines . . . . . . . . . . . . . .2026 4.4. Operations - Service Performance and Monitoring Guidelines . . . . . . . . . . . . . . . . . . . . . . .2127 4.5. Client Reliability Models/Service Assurance Guidelines .2329 4.6. Application Accounting Guidelines . . . . . . . . . . . . 29 5. Troubleshooting and Diagnostics . . . . . . . . . . . . . . .2329 6. Security Considerations . . . . . . . . . . . . . . . . . . .24 7. IANA Considerations30 6.1. DoS attacks (against state and bandwidth) . . . . . . . . 30 6.2. Content Security . . . . . . . . . . . . . . .25 8. Conclusions. . . . . 32 6.3. Peering Encryption . . . . . . . . . . . . . . . . . . . 34 6.4. Operational Aspects .25. . . . . . . . . . . . . . . . . . 34 7. Privacy Considerations . . . . . . . . . . . . . . . . . . . 35 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 37 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . .2537 10. Change log [RFC Editor: Please remove] . . . . . . . . . . .2637 11. References . . . . . . . . . . . . . . . . . . . . . . . . .2639 11.1. Normative References . . . . . . . . . . . . . . . . . .2639 11.2. Informative References . . . . . . . . . . . . . . . . .2740 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . .2841 1. Introduction Content and data from several types of applications (e.g., live video streaming, software downloads) are well suited for delivery via multicast means. The use of multicast for delivering suchcontent/content or other data offers significant savings of utilization of resources in any given administrative domain. End user demand for suchcontent/datacontent or other data is growing. Often, this requires transporting thecontent/datacontent or other data across administrative domains via inter-domain peering points. The objective of this Best Current Practices document is twofold: o Describe the technical process and establish guidelines for setting up multicast-based delivery of applicationcontent/data acrosscontent or other data across inter-domain peering points via a set of use cases. o Catalog all required information exchange between the administrative domains to support multicast-based delivery. This enables operators to initiate necessary processes to support inter-domain peering with multicast. The scope and assumptions for this document arestatedas follows: o Administrative Domain 1 (AD-1) sources content to one or more End Users (EUs) in one or more Administrative Domain 2 (AD-2). AD-1 and AD-2 want to use IP multicast to allow supporting large and growing EU populations with minimum amount of duplicated traffic to send across network links. o This document does not detail the case where EUs are originating content. To support that additional service, it is recommended to use some method (outside the scope of this document) by which the content from EUs is transmitted to the application in AD-1 that this document refers to as the multicast source and let it send out the traffic as IP multicast. From that point on, the descriptions in this document apply, except that they are not complete because they do not cover the transport or operational aspects of the leg from EU to AD-1. o This document does not detail the case where AD-1 and AD-2 are not directly connected to each other but only via one or more AD-3 (transit providers). The cases described in this document where tunnels are used between AD-1 and AD-2 can be applied to such scenarios, but SLA ("Service Level Agreement") control for example would be different. Other additional issues will likely exist as well in such scenarios. This is for further study. o For the purpose of this document, the term "peering point" refers toan interfacea network connection ("link") between twonetworks/administrativeadministrative network domains over which traffic is exchanged between them.A Network-NetworkThis is also referred to as a Network-to-Network Interface(NNI)(NNI). Unless otherwise noted, the peering point isan exampleassumed to be a private peering point, where the network connection is a physically or virtually isolated network connection solely between AD-1 and AD-2. The other case is that of a broadcast peeringpoint.point which is a common option in public Internet Exchange Points (IXP). See Section 4.2.2 for more details about that option. o Administrative Domain 1 (AD-1) is enabled with native multicast. A peering point exists between AD-1 and AD-2. o It is understood that several protocols are available for this purpose including PIM-SM and Protocol Independent Multicast - Source Specific Multicast (PIM-SSM) [RFC7761], Internet Group Management Protocol (IGMP) [RFC3376], and Multicast Listener Discovery (MLD) [RFC3810]. o As described in Section 2, the source IP address of the multicast stream in the originating AD (AD-1) is known. Under this condition, PIM-SSM use is beneficial as it allows the receiver's upstream router to directly send a JOIN message to the source without the need of invoking an intermediate Rendezvous Point (RP). Use of SSM also presents an improved threat mitigation profile against attack, as described in [RFC4609]. Hence, in the case of inter-domain peering, it is recommended to use only SSM protocols; the setup of inter- domain peering for ASM (Any-Source Multicast) is not in scope for this document. oAD-1The rest of the document assumes that PIM-SSM andAD-2BGP areassumedused across the peering point plus AMT and/or GRE according toadopt compatible protocols.scenario. The use ofdifferentother protocols is beyond the scope of this document. o An Automatic Multicast Tunnel (AMT) [RFC7450] is setup at the peering point if either the peering point or AD-2 is not multicast enabled. It is assumed that an AMT Relay will be available to a client for multicast delivery. The selection of an optimal AMT relay by a client is out of scope for this document. Note that AMT use is necessary only when native multicast is unavailable in the peering point (Use Case 3.3) or in the downstream administrative domain (Use Cases 3.4, and 3.5). o The collection of billing data is assumed to be done at the application level and is not considered to be a networking issue. The settlements process for end user billing and/or inter-provider billing is out of scope for this document. o Inter-domain network connectivity troubleshooting is only considered within the context of a cooperative process between the two domains.Thus, the primary purpose of this document is to describe a scenario where two AD's interconnect via a a peering point with each other. Security and operational aspects for exchanging traffic on a public Internet Exchange Point (IXP) with a large shared broadcast domain between many operators, is not in scope for this document. It may be possible to have a configuration whereby a transit domain (AD-3) interconnects AD-1 and AD-2. Such a configuration adds complexity and may require manual provisioning if, for example, AD-3 is not multicast enabled. This configuration is out of cope for this document; it is for further study.This document also attempts to identify ways by which the peering process can be improved. Development of new methods for improvement is beyond the scope of this document. 2. Overview of Inter-domain Multicast Application Transport A multicast-based application delivery scenario is as follows: o Two independent administrative domains are interconnected via a peering point. o The peering point is either multicast enabled (end-to-end native multicast across the two domains) or it is connected by one of two possible tunnel types: o A Generic Routing Encapsulation (GRE) Tunnel [RFC2784] allowing multicast tunneling across the peering point, or o An Automatic Multicast Tunnel (AMT) [RFC7450]. o A service provider controls one or more application sources in AD-1 which will send multicast IP packetsforvia one or more(S,G)s.(S,G)s (multicast traffic flows, see Section 4.2.1 if you are unfamiliar with IP multicast). It is assumed that the service being provided is suitable for delivery via multicast (e.g. live video streaming of popular events, software downloads to many devices, etc.), and that the packet streams willbe part ofcarried by a suitable multicast transport protocol. o An End User (EU) controls a device connected to AD-2, which runs an application client compatible with the service provider's application source. o The application client joins appropriate (S,G)s in order to receive the data necessary to provide the service to the EU. The mechanisms by which the application client learns the appropriate (S,G)s are an implementation detail of the application, and are out of scope for this document. The assumption here is that AD-1 has ultimate responsibility for delivering the multicast based service on behalf of the content source(s). All relevant interactions between the two domains described in this document are based on this assumption. Note that domain 2 may be an independent network domain(e.g.,(e.g.: Tier 1 network operator domain). Alternately, domain 2 could also be an Enterprise network domain operated by a singlecustomer.customer of AD-1. The peering point architecture and requirements may have some unique aspects associated with the Enterprise case. The Use Cases describing various architectural configurations for the multicast distribution along with associated requirements is described in section 3. Unique aspects related to the Enterprise network possibility will be described in this section. Section 4 contains a comprehensive list of pertinent information that needs to be exchanged between the two domains in order to support functions to enable the application transport. Note that domain 2 may be an independent network domain (e.g., Tier 1 network operator domain). Alternately, domain 2 could also be an Enterprise network domain operated by a single customer. The Use Cases describing various architectural configurations for the multicast distribution along with associated requirements is described in Section 3. The peering point architecture and requirements may have some unique aspects associated with the Enterprise case. These unique aspects will also be described in Section 3. Section 4 contains a comprehensive list of pertinent information that needs to be exchanged between the two domains in order to support functions to enable the application transport. 3. Inter-domain Peering Point Requirements for Multicast The transport of applications using multicast requires that the inter-domain peering point is enabled to support such a process. There are five Use Cases for consideration in this document. 3.1. Native Multicast This Use Case involves end-to-end Native Multicast between the two administrative domains and the peering point is also native multicast enabled - see Figure 1. ------------------- ------------------- / AD-1 \ / AD-2 \ / (Multicast Enabled) \ / (Multicast Enabled) \ / \ / \ | +----+ | | | | | | +------+ | | +------+ | +----+ | | AS |------>| BR |-|---------|->| BR |-------------|-->| EU | | | | +------+ | I1 | +------+ |I2 +----+ \ +----+ / \ / \ / \ / \ / \ / ------------------- ------------------- AD = Administrative Domain (Independent Autonomous System) AS = Application (e.g., Content) Multicast Source BR = Border Router I1 = AD-1 and AD-2 Multicast Interconnection (e.g., MBGP) I2 = AD-2 and EU Multicast Connection Figure 1:-Content Distribution via End to End Native Multicast Advantages of this configuration are: o Most efficient use of bandwidth in both domains. o Fewer devices in the path traversed by the multicast stream when compared to an AMT enabled peering point. From the perspective of AD-1, the one disadvantage associated with native multicast into AD-2 instead of individual unicast to every EU in AD-2 is that it does not have the ability to count the number of End Users as well as the transmitted bytes delivered to them. This information is relevant from the perspective of customer billing and operational logs. It is assumed that such data will be collected by the application layer. The application layer mechanisms for generating this information need to be robust enough such that all pertinent requirements for the source provider and the AD operator are satisfactorily met. The specifics of these methods are beyond the scope of this document. Architectural guidelines for this configuration are as follows: a. Dual homing for peering points between domains is recommended as a way to ensure reliability with full BGP table visibility. b. If the peering point between AD-1 and AD-2 is a controlled network environment, then bandwidth can be allocated accordingly by the two domains to permit the transit of non- rate adaptive multicast traffic. If this is not the case, then it is recommended that the multicast traffic should support rate- adaption. c. The sending and receiving of multicast traffic between two domains is typically determined by local policies associated with each domain. For example, if AD-1 is a service provider and AD-2 is an enterprise, then AD-1 may support local policies for traffic delivery to, but not traffic reception from, AD-2. Another example is the use of a policy by which AD-1 delivers specified content to AD-2 only if such delivery has been accepted by contract. d. Relevant information on multicast streams delivered to End Users in AD-2 is assumed to be collected by available capabilities in the application layer. The precise nature and formats of the collected information will be determined by directives from the source owner and the domain operators.e. The interconnection of AD-1 and AD-2 should, at a minimum, follow guidelines for traffic filtering between autonomous systems [BCP38]. Filtering guidelines specific to the multicast control- plane and data-plane are described in section 6. 3.2. Peering Point Enabled with GRE Tunnel3.2. Peering Point Enabled with GRE Tunnel The peering point is not native multicast enabled in this Use Case. There is a Generic Routing Encapsulation Tunnel provisioned over the peering point. See Figure 2. ------------------- ------------------- / AD-1 \ / AD-2 \ / (Multicast Enabled) \ / (Multicast Enabled) \ / \ / \ | +----+ +---+ | (I1) | +---+ | | | | +--+ |uBR|-|--------|-|uBR| +--+ | +----+ | | AS |-->|BR| +---+-| | +---+ |BR| -------->|-->| EU | | | | +--+ <.......|........|........>+--+ |I2 +----+ \ +----+ / I1 \ / \ / GRE \ / \ / Tunnel \ / ------------------- ------------------- AD = Administrative Domain (Independent Autonomous System) AS = Application (e.g., Content) Multicast Source uBR = unicast Border Router - not necessarily multicast enabled may be the same router as BR BR = Border Router - for multicast I1 = AD-1 and AD-2 Multicast Interconnection (e.g., MBGP) I2 = AD-2 and EU Multicast Connection Figure 2: Content Distribution via GRE Tunnel In this case, the interconnection I1 between AD-1 and AD-2 in Figure12 is multicast enabled via a Generic Routing Encapsulation Tunnel (GRE) [RFC2784] between the two BR and encapsulating the multicast protocols across it. Normally, this approach is choosen if theinterface.uBR physcially connected to the peering link can or should not be enabled for IP multicast. This approach may also be beneficial if BR and uBR are the same device, but the peering link is a broadcast domain (IXP), see Figure 6. The routing configuration is basically unchanged: Instead of BGP (SAFI2) across the native IP multicast link between AD-1 and AD-2, BGP (SAFI2) is now run across the GRE tunnel. Advantages of this configuration: o Highly efficient use of bandwidth in both domains, although not as efficient as the fully native multicast Use Case. o Fewer devices in the path traversed by the multicast stream when compared to an AMT enabled peering point. o Ability to supportonlypartial and/or incremental IP multicast deployments in AD- 1 and/orAD-2 (the two Border Routers in Figure 1 do notAD-2: Only the path(s) between AS/BR (AD-1) and BR/EU (AD-2) need to bethe two "unicast" domain border routers; instead theymulticast enabled. The uBRs may not support IP multicast or enabling it could be seen as operationally risky on that important edge node whereas dedicated BR nodes for IP multicast may be more acceptable at least initially. BR can also beanywhere in AD-1 and AD-2). o GRE is anlocated such that only parts of the domain may need to support native IP multicast (e.g.: only the core in AD-1 but not edge networks towards uBR). o GRE is an existing technology and is relatively simple to implement. Disadvantages of this configuration: o Per Use Case 3.1, current router technology cannot count the number of end users or the number bytes transmitted. o GRE tunnel requires manual configuration. o The GRE must be established prior to stream starting. o The GRE tunnel is often left pinned up. Architectural guidelines for this configuration include the following: Guidelines (a) through (d) are the same as those described in Use Case 3.1. Two additional guidelines are as follows: e. GRE tunnels are typically configured manually between peering points to support multicast delivery between domains. f. It is recommended that the GRE tunnel (tunnel server) configuration in the source network is such that it only advertises the routes to the application sources and not to the entire network. This practice will prevent unauthorized delivery of applications through the tunnel (e.g., if application - e.g., content - is not part of an agreed inter-domain partnership). 3.3. Peering Point Enabled with an AMT - Both Domains Multicast Enabled Both administrative domains in this Use Case are assumed to be native multicast enabled here; however, the peering point is not. The peering point is enabled with an Automatic Multicast Tunnel. The basic configuration is depicted in Figure 2. ------------------- ------------------- / AD-1 \ / AD-2 \ / (Multicast Enabled) \ / (Multicast Enabled) \ / \ / \ | +----+ +---+ | I1 | +---+ | | | |+------+ | | +------++--+ |uBR|-|--------|-|uBR| +--+ | +----+ | | AS|------>| AR |-|---------|->| AG |-------------|-->| EU ||-->|AR| +---+-| | +---+ |AG| -------->|-->| EU | |+------+|I1|+------++--+ <.......|........|........>+--+ |I2 +----+ \ +----+ / AMT \ / \ / Tunnel \ / \ / \ / ------------------- ------------------- AD = Administrative Domain (Independent Autonomous System) AS = Application (e.g., Content) Multicast Source AR = AMT Relay AG = AMT Gateway uBR = unicast Border Router - not multicast enabled otherwise AR=uBR (AD-1), uBR=AG (AD-2) I1 = AMT Interconnection between AD-1 and AD-2 I2 = AD-2 and EU Multicast Connection Figure2:3: - AMT Interconnection between AD-1 and AD-2 Advantages of this configuration: o Highly efficient use of bandwidth in AD-1. o AMT is an existing technology and is relatively simple to implement. Attractive properties of AMT include the following: o Dynamic interconnection between Gateway-Relay pair across the peering point. o Ability to serve clients and servers with differing policies. Disadvantages of this configuration: o Per Use Case 3.1 (AD-2 is native multicast), current router technology cannot count the number of end users or the number of bytes transmitted to all end users. o Additional devices (AMT Gateway and Relay pairs) may be introduced into the path if these services are not incorporated in the existing routing nodes. o Currently undefined mechanisms for the AG to automatically select the optimal AR. Architectural guidelines for this configuration are as follows: Guidelines (a) through (d) are the same as those described in Use Case 3.1. In addition, e. It is recommended that AMT Relay and Gateway pairs be configured at the peering points to support multicast delivery between domains. AMT tunnels will then configure dynamically across the peering points once the Gateway in AD-2 receives the (S, G) information from the EU. 3.4. Peering Point Enabled with an AMT - AD-2 Not Multicast Enabled In this AMT Use Case, the second administrative domain AD-2 is not multicast enabled. Hence, the interconnection between AD-2 and the End User is also not multicast enabled. This Use Case is depicted in Figure 3. ------------------- ------------------- / AD-1 \ / AD-2 \ / (Multicast Enabled) \ /(Non-Multicast(Non Multicast \ / \ / Enabled) \ N(large) | +----+ +---+ | | +---+ | #EU | | |+------+ | |+--+ |uBR|-|--------|-|uBR| | +----+ | | AS|------>| AR |-|---------|-----------------------|-->|EU/G||-->|AR| +---+-| | +---+ ................>|EU/G| | |+------+ || +--+ <.......|........|........... |I2 +----+ \ +----+ /\N x AMT\ / \ / Tunnel \ / \ / \ / ------------------- ------------------- AS = Application Multicast Source uBR = unicast Border Router - not multicast enabled, otherwise AR = uBR (in AD-1). AR = AMT Relay EU/G = Gateway client embedded in EU device I2 = AMT Tunnel Connecting EU/G to AR in AD-1 through Non-Multicast Enabled AD-2. Figure3: -4: AMT Tunnel Connecting AD-1 AMT Relay and EU Gateway This Use Case is equivalent to having unicast distribution of the application through AD-2. The total number of AMT tunnels would be equal to the total number of End Users requesting the application. The peering point thus needs to accommodate the total number of AMT tunnels between the two domains. Each AMT tunnel can provide the data usage associated with each End User. Advantages of this configuration: oHighly efficientEfficient use of bandwidth inAD-1.AD-1 (The closer AR is to uBR, the more efficient). o Ability for AD-1 to introduce IP multicast based content delivery without any support by network devices in AD-2: Only application side in the EU device needs to perform AMT gateway library functionality to receive traffic from AMT relay. o Allows for AD-2 to "upgrade" to Use Case 3.5 (see below) at a later time without any change in AD-1 at that time. o AMT is an existing technology and is relatively simple to implement. Attractive properties of AMT include the following: o Dynamic interconnection between Gateway-Relay pair across the peering point. o Ability to serve clients and servers with differing policies. o Each AMT tunnel serves as a count for each End User and is also able to track data usage (bytes) delivered to the EU. Disadvantages of this configuration: o Additional devices (AMT Gateway and Relay pairs) are introduced into the transport path. o Assuming multiple peering points between the domains, the EU Gateway needs to be able to find the "correct" AMT Relay in AD-1. Architectural guidelines for this configuration are as follows: Guidelines (a) through (c) are the same as those described in Use Case 3.1. d. It isrecommendednecessary that proper procedures are implemented such that the AMT Gateway at the End User device is able to find the correct AMT Relayin AD-1 acrossfor each (S,G) content stream. Standard mechanisms for that selection are still subject to ongoing work. This includes use of anycast gateway addresses, anycast DNS names, explicit configuration that is mapping (S,G) to a relay address or letting thepeering points. Theapplicationclientin theEU device is expected to supplyEU/G provide the(S, G) informationrelay address to theGateway for this purpose.embedded AMT gateway function. e. The AMT tunnel capabilities are expected to be sufficient for the purpose of collecting relevant information on the multicast streams delivered to End Users in AD-2. 3.5. AD-2 Not Multicast Enabled - Multiple AMT Tunnels Through AD-2 This is a variation of Use Case 3.4 as follows: ------------------- ------------------- / AD-1 \ / AD-2 \ / (Multicast Enabled) \ /(Non-Multicast(Non Multicast \ / +---+ \ (I1) / +---+ Enabled) \ | +----+ |uBR|-|--------|-|uBR| ||+--+ +--+| | | +--+ +---+ |+------+|||AG| |AG|+---+ +---+ | +----+ | | AS|------>| AR |-|-------->||AR|------------->|AR|-|-->|EU/G||-->|AR|<........|.... | +---+ |AG/|....>|EU/G| | |+------+ | I1 ||1|I2 |2+--+ | ......|.|AG/|..........>|AR2| |I3 +----+ \ +----+ /\+--+ +--+ /I1 \ |AR1| I2 +---+ / \ / single \+---+ / \ / AMT Tunnel \ / ------------------- ------------------- uBR = unicast Border Router - not multicast enabled otherwise AR=uBR (AD-1) or ubr=AGAR1 (AD-2) AS = Application Source AR = AMT Relay in AD-1 AGAR1 = AMT Gateway/Relay node in AD-2 across Peering Point I1 = AMT Tunnel Connecting AR in AD-1 to GW in AGAR1 in AD-2 AGAR2 = AMT Gateway/Relay node at AD-2 Network Edge I2 = AMT Tunnel Connecting Relay in AGAR1 to GW in AGAR2 EU/G = Gateway client embedded in EU device I3 = AMT Tunnel Connecting EU/G to AR in AGAR2 Figure4: -5: AMT Tunnel Connecting AMT Relay and Relays Use Case 3.4 results in several long AMT tunnels crossing the entire network of AD-2 linking the EU device and the AMT Relay in AD-1 through the peering point. Depending on the number of End Users, there is a likelihood of an unacceptably high amount of traffic due to the large number of AMT tunnels - and unicast streams - through the peering point. This situation can be alleviated as follows: o Provisioning of strategically located AMT nodesat the edges ofin AD-2 AD-2. An AMT node comprises co-location of an AMT Gateway and an AMT Relay. No change is required by AD-1 compared to 3.4. This can be done whenever AD-2 seems fit (too much traffic across peering point. o One such node is at the AD-2 side of the peering point (node AGAR1 inFigure 4).above Figure). o Single AMT tunnel established across peering point linking AMT Relay in AD-1 to the AMT Gateway in the AMT node AGAR1 in AD-2. o AMT tunnels linking AMT node AGAR1 at peering point in AD-2 to other AMT nodes located at the edges of AD-2: e.g., AMT tunnel I2 linking AMT Relay in AGAR1 to AMT Gateway in AMT node AGAR2 in Figure 4. o AMT tunnels linking EU device (via Gateway client embedded in device) and AMT Relay in appropriate AMT node at edge of AD-2: e.g., I3 linking EU Gateway in device to AMT Relay in AMT node AGAR2. o In the most simple option (not shown), AD-2 only deploys a single AGAR1 and lets EU/G build AMT tunnels directly to it. This setup already solves the problem of replicated traffic across the peering point. As soon as there is need to support more AMT tunnels to EU/G, then additional AGAR2 nodes can be deployed by AD-2. The advantage for such a chained set of AMT tunnels is that the total number of unicast streams across AD-2 is significantly reduced, thus freeing up bandwidth. Additionally, there will be a single unicast stream across the peering point instead of possibly, an unacceptably large number of such streams per Use Case 3.4. However, this implies that several AMT tunnels will need to be dynamically configured by the various AMT Gateways based solely on the (S,G) information received from the application client at the EU device. A suitable mechanism for such dynamic configurations is therefore critical. Architectural guidelines for this configuration are as follows: Guidelines (a) through (c) are the same as those described in Use Case 3.1. d. It isrecommendednecessary that proper procedures are implemented such that the various AMT Gateways (at the End User devices and the AMT nodes in AD-2) are able to find the correct AMT Relay in other AMT nodes as appropriate.The application client in the EU deviceStandard mechanisms for that selection are still subject to ongoing work. This includes use of anycast gateway addresses, anycast DNS names, or explicit configuration that isexpectedmapping (S,G) tosupplya relay address. On the(S, G)EU/G, this mapping informationtomay come from theGateway for this purpose.application. e. The AMT tunnel capabilities are expected to be sufficient for the purpose of collecting relevant information on the multicast streams delivered to End Users in AD-2. 4. Functional Guidelines Supporting functions and related interfaces over the peering point that enable the multicast transport of the application are listed in this section. Critical information parameters that need to be exchanged in support of these functions are enumerated, along with guidelines as appropriate. Specific interface functions for consideration are as follows. 4.1. Network Interconnection Transportand SecurityGuidelines The term "Network Interconnection Transport" refers to the interconnection points between the two Administrative Domains. The following is a representative set of attributes that will need to be agreed to between the two administrative domains to support multicast delivery. o Number of Peering Points. o Peering Point Addresses and Locations. o Connection Type - Dedicated for Multicast delivery or shared with other services. o Connection Mode - Direct connectivity between the two AD's or via another ISP. o Peering Point Protocol Support - Multicast protocols that will be used for multicast delivery will need to be supported at these points. Examples of protocols include eBGP [RFC4760] and MBGP [RFC4760]. o Bandwidth Allocation - If shared with other services, then there needs to be a determination of the share of bandwidth reserved for multicast delivery.When determining the appropriate bandwidth allocation, parties should consider use of a multicast protocol suitableSee section 4.1.1 below forlive video streaming that is consistent with Congestion Control Principles [BCP41].more details. o QoS Requirements -Delay/latencyDelay and/or latency specifications that need to be specified in an SLA. o AD Roles and Responsibilities - the role played by each AD for provisioning and maintaining the set of peering points to support multicast delivery.4.2. Routing Aspects4.1.1. Bandwidth Management Like IP unicast traffic, IP multicast traffic carried across non- controlled networks must comply to Congestion Control Principles as described in [BCP41] andRelated Guidelines The main objectiveexplained in detail for UDP IP multicastdelivery routingin [BCP145]. Non-controlled networks (such as the Internet) are those where there is no policy for managing bandwidth other than best effort with fair share of bandwidth under congestion. As a simplified rule of thumb, complying toensurecongestion control principles means to reduce bandwidth under congestion in a way thatthe End User receives theis fair to competing competing (typically TCP) flow ("rate adaptive"). In many instances, multicaststreamcontent delivery evolves fromthe "most optimal" source [INF_ATIS_10] which typically: o Maximizes the multicast portion of the transportintra- domain deployments where it is handled as a controlled network service andminimizes any unicast portionofthe delivery, and o Minimizes the overall combined network(s) route distance. This routing objective appliesnot complyng toboth Native and AMT; the actual methodologycongestion control principles. It was given a reserved amount of bandwidth and admitted to thesolution will be different for each. Regardless,network so that congestion never occurs. Therefore therouting solution is expected: o To be scalable, o To avoid/minimize new protocol development or modifications, and o Tocongestion control issue should berobust enough to achieve high reliability and automatically adjustgiven specific attention when evolving tochanges/problems inan interdomain peering deployment. In the case where end-to-end IP multicastinfrastructure. Fortraffic passes across the network of two ADs (and their subsidiaries/customers), bothNative and AMT environments, havingADs must agree on asource as closeconsistent traffic management policy. If for example AD-1 sources non congestion aware IP multicast traffic and AD-2 carries it aspossiblebest effort traffic across links shared with other Internet traffic and subject tothe EU network is most desirable; therefore, incongestion, this will not work: Under congestion, somecases, an AD may prefer to have multiple sources near different peering points. However, that is entirely an implementation issue. 4.2.1. Native Multicast Routing Aspects Native multicast simply requiresamount of that traffic will be dropped, rendering theAdministrative Domains coordinateremaining packets often as undecodeable garbage clogging up the network in AD-2 andadvertisebecause this is not congestion aware, thecorrect source address(es) atloss does not reduce this rate. Competing traffic will not get theirnetwork interconnection peering points(i.e., border routers). An examplefair share under congestion, and EUs will be frusted by extremely bad quality of both their IP multicastdelivery via a Native Multicast process across two Administrative Domains is as follows assumingand other (e.g.: TCP) traffic. Note thatthe interconnecting peering points are also multicast enabled: o Appropriate information is obtained by the EU client whothis is not an IP multicast technology issue, but solely asubscriber to AD-2 (see Use Case 3.1). This informationtransport/application layer issue: The problem would equally happen if AD-1 would send non-rate adaptive unicast traffic,, for example legacy IPTV video-on-demand traffic which typically is also non congestion aware. Because rate adaption inthe formIP unicast video is commonplace today because ofmetadata andABR (Adaptive Bitrate Video), itcontains instructions directing the EU client to launch an appropriate application if necessary, as well as additional informationis very unlikely forthe application about the source location and the group (or stream) idthis to happen though in reality with IP unicast. While theform of the "S,G" data. The "S" portion provides the namerules for traffic management apply whether or not IPaddress ofmulticast is tunneled or not, thesourceone feature that can make AMT tunnels more difficult is the unpredictability of bandwidth requirements across underlying links because of the way they can be used: With native IP multicaststream. The metadata may also contain alternate delivery information such as specifyingor GRE tunnels, theunicast addressamount of bandwidth depends on thestream. o The client usesamount of content, not thejoin message with S,Gnumber of EUs - and is therefore easier tojoin the multicast stream [RFC4604]. To facilitate this process,plan for. AMT tunnels terminating in EU/G on thetwo AD's need to doother hand scale with thefollowing: o Advertisenumber of EUs. In thesource id(s) overvicinity of thePeering Points. o Exchange relevant Peering Point information such as CapacityAMT relay they can introduce very large amount of replicated traffic andUtilization. o Implement compatible multicast protocols to ensure proper multicast delivery across the peering points. 4.2.2. GRE Tunnel over Interconnecting Peering Point If the interconnecting peering pointit is notmulticast enabled and both AD's are multicast enabled, then a simple solution isalways feasible to provisiona GRE tunnel betweenenough bandwidth for all possible EU to get thetwo AD'shighest quality for all their content during peak utilization in such setups -see Use Case 3.2.2. The termination points of the tunnel will usually be a network engineering decision, but generally will be between the border routers or even between the AD 2 border router andunless theAD 1 source (or source access router). The GRE tunnel would allow end-to-end native multicast orAMTmulticastrelays are very close totraverse the interface. Coordination and advertisement ofthesource IPEU edge. Therefore it isstill required. The two AD's need to follow the same process as described in 4.2.1also recommended tofacilitateuse IP multicastdelivery across the Peering Points. 4.2.3. Routing Aspects with AMT Tunnels Unlike Native Multicast (with or without GRE), an AMT Multicast environment is more complex. It presents a dual layered problem because there are two criteria that should be simultaneously met: o Find the closestrate adaptation even inside controlled networks when using AMTrelaytunnels directly tothe end-userEU/G. Note thatalso hasrate-adaptive IP multicastconnectivity totraffic in general does not mean that thecontent source, and o Minimizesender is reducing theAMT unicast tunnel distance. Therebitrate, but rather that the EUs that experience congestion areessentially two componentsjoining tothe AMT specification AMT Relays: These serve the purposea lower bitrate (S,G) stream oftunneling UDP multicast traffic to the receivers (i.e., End-Points). The AMT Relay will receivethetraffic nativelycontent, similar to adaptive bitrate streaming over TCP. Migration fromthenon rate-adaptive to rate adaptive bitrate in IP multicastmedia source and will replicate the stream on behalf ofdoes therefore also change thedownstream AMT Gateways, encapsulatingdynamic (S,G) join behavior in the network resulting in potentially higher performance requirement for IP multicastpackets into unicast packets and sending them over the tunnel towardprotocols (IGMP/PIM), especially on the last hops where dynamic changes occur (including AMTGateway.gateway/relays): Inaddition, the AMT Relay may perform various usage and activity statistics collection. This resultsnon rate-adaptive IP multicast, only "channel change" causes state change, inmoving the replication point closer to the end user, and cuts down on traffic across the network. Thus, the linear costs of adding unicast subscribers can be avoided. However, unicast replication is still required for each requesting End-Point withinrate-adaptive also theunicast-only network. AMT Gateway (GW): The Gateway will reside on an End-Point -congestion situation causes state change. Even though not fully specified in this document, peerings that rely on GRE/AMT tunnels may bea Personal Computer (PC)across one ora Set Top Box (STB). The AMT Gateway receives join and leave requests from the Application viamore transit ADs instead of anApplication Programming Interface (API). In this manner,exclusive (non-shared, L1/L2) path. Unless those transit ADs are explicitly contracted to provide other than "best effort" transit for theGateway allowstunneled traffic, theEnd-PointIP multicast traffic tunneled must be rate adaptive toconduct itself as a true Multicast End-Point. The AMT Gateway will encapsulate AMT messages into UDP packetsnot violate BCP41 across those transit ADs. 4.2. Routing Aspects andsend them through a tunnel (across the unicast-only infrastructure) to the AMT Relay.Related Guidelines Thesimplest AMT Use Case (section 3.3) involves peering points that are not multicast enabled between twomain objective for multicastenabled AD's. An AMT tunneldelivery routing isdeployed between an AMT Relay onto ensure that theAD 1 sideEnd User receives the multicast stream from the "most optimal" source [INF_ATIS_10] which typically: o Maximizes the multicast portion of thepeering pointtransport andan AMT Gateway on the AD 2 sideminimizes any unicast portion of thepeering point. One advantage to this arrangement is thatdelivery, and o Minimizes thetunneloverall combined network(s) route distance. This routing objective applies to both Native and AMT; the actual methodology of the solution will be different for each. Regardless, the routing solution isestablished on an as needed basisexpected: o To be scalable, o To avoid or minimize new protocol development or modifications, andneed noto To bea provisioned element. The two AD's can coordinaterobust enough to achieve high reliability andadvertise special AMT Relay Anycast addresses with each other. Alternately, they may decideautomatically adjust tosimply provision Relay addresses, though this would not be an optimal solutionchanges and problems interms of scalability. Use Cases 3.4the multicast infrastructure. For both Native and3.5 describe more complicatedAMTsituationsenvironments, having a source asAD-2 is not multicast enabled. For these cases, the End User device needs to be ableclose as possible tosetup an AMT tunnel inthe EU network is mostoptimal manner. There are many methods by which relay selection can be done including the use of DNS based queries and static lookup tables [RFC7450]. The choice of the methoddesirable; therefore, in some cases, an AD may prefer to have multiple sources near different peering points. However, that is entirely an implementationdependentissue. 4.2.1. Native Multicast Routing Aspects Native multicast simply requires that the Administrative Domains coordinate andis up toadvertise the correct source address(es) at their networkoperators. Comparison of various methods is out of scope for this document; it is for further study.interconnection peering points(i.e., border routers). Anillustrativeexample of multicast delivery via arelay selection based on DNS queries and Anycast IP addressesNative Multicast processfor Use Cases 3.4 and 3.5across two Administrative Domains isdescribed here. Using an Anycast IP address for AMT Relays allows for all AMT Gateways to find the "closest" AMT Relay - the nearest edge of the multicast topology of the source. Noteas follows assuming thatthis is strictly illustrative; the choice ofthemethod is up to the network operators. The basic process is as follows:interconnecting peering points are also multicast enabled: o Appropriatemetadatainformation is obtained by the EU clientapplication. Thewho is a subscriber to AD-2 (see Use Case 3.1). This information is in the form of metadata and it contains instructions directing the EU client to launch anordered list of particular destinations to seek the requested stream and,appropriate application if necessary, as well as additional information formulticast, specifiesthe application about the source location and the group (or stream)IDid in the form of the "S,G" data. The "S" portion provides theURI (namename or IPaddress)address of the source of the multicaststream and the "G" identifies the particular stream originated by that source.stream. The metadata may also contain alternate delivery information such as specifying the unicast address of theunicast form ofstream. o The client uses thecontentjoin message with S,G tobe used, for example, ifjoin the multicast streambecomes unavailable.[RFC4604]. To facilitate this process, the two AD's need to do the following: oUsingAdvertise theinformation fromsource id(s) over themetadata, and possiblyPeering Points. o Exchange relevant Peering Point informationprovisioned directly insuch as Capacity and Utilization. o Implement compatible multicast protocols to ensure proper multicast delivery across theEU client,peering points. 4.2.2. GRE Tunnel over Interconnecting Peering Point If the interconnecting peering point is not multicast enabled and both AD's are multicast enabled, then aDNS querysimple solution isinitiated in ordertoconnect the EU client/AMT Gateway to an AMT Relay. o Query results are obtained, and may return an Anycast address orprovision aspecific unicast addressGRE tunnel between the two AD's - see Use Case 3.2.2. The termination points ofa relay. Multiple relaysthe tunnel willtypically exist. The Anycast address isusually be aroutable "pseudo- address" shared amongnetwork engineering decision, but generally will be between therelays that can gain multicastborder routers or even between the AD 2 border router and the AD 1 source (or source access router). The GRE tunnel would allow end-to-end native multicast or AMT multicast to traverse thesource. o If a specificinterface. Coordination and advertisement of the source IPaddress uniqueis still required. The two AD's need toa relay was not obtained, the AMT Gateway then sends a message (e.g.,follow thediscovery message)same process as described in 4.2.1 to facilitate multicast delivery across theAnycast address such that the networkPeering Points. 4.2.3. Routing Aspects with AMT Tunnels Unlike Native Multicast (with or without GRE), an AMT Multicast environment ismakingmore complex. It presents a dual layered problem because there are two criteria that should be simultaneously met: o Find therouting choice of particular relay - e.g.,closest AMT relay to theEU. (Noteend-user thatin IPv6 there is a specific Anycast format and Anycast is inherent in IPv6 routing, whereas in IPv4 Anycast is handled via provisioning inalso has multicast connectivity to thenetwork. Details are out of scope for this document.)content source, and oThe contactedMinimize the AMTRelay then returns its specificunicastIP address (after which the Anycast address is no longer required). Variations may exist as well. o The AMT Gateway uses that unicast IP address to initiate a three- way handshake withtunnel distance. There are essentially two components to the AMTRelay. ospecification AMTGateway provides "S,G"Relays: These serve the purpose of tunneling UDP multicast traffic to the receivers (i.e., End-Points). The AMT Relay(embedded in AMT protocol messages). o AMT Relay receiveswill receive the"S,G" informationtraffic natively from the multicast media source anduseswill replicate theS,G to joinstream on behalf of theappropriate multicast stream, if it has not already subscribed to that stream. odownstream AMTRelay encapsulatesGateways, encapsulating the multicaststreampackets into unicast packets and sending them over the tunnelbetweentoward the AMT Gateway. In addition, the AMT Relay may perform various usage and activity statistics collection. This results in moving theGateway, providing the requested contentreplication point closer to theEU. 4.3. Back Office Functions - Provisioningend user, andLogging Guidelines Back Office refers tocuts down on traffic across thefollowing: o Servers and Content Management systems that supportnetwork. Thus, thedeliverylinear costs ofapplications via multicast and interactions between AD's. o Functionality associated with logging, reporting, ordering, provisioning, maintenance, service assurance, settlement, etc. 4.3.1. Provisioning Guidelines Resources for basic connectivity between AD's Providers need to be provisioned as follows: o Sufficient capacity must be provisioned to support multicast-based delivery across AD's. o Sufficient capacity mustadding unicast subscribers can beprovisionedavoided. However, unicast replication is still required forconnectivity between all supporting back-offices ofeach requesting End-Point within theAD's as appropriate. This includes activating proper security treatment for these back- office connections (gateways, firewalls, etc) as appropriate. o Routing protocolsunicast-only network. AMT Gateway (GW): The Gateway will reside on an End-Point - this could be any type of IP host such asneeded, e.g. configuring routers to support these. Provisioning aspects relateda Personal Computer (PC), mobile phone, Set Top Box (STB) or appliances. The AMT Gateway receives join and leave requests from the Application via an Application Programming Interface (API). In this manner, the Gateway allows the End-Point toMulticast-Based inter-domain delivery areconduct itself asfollows.a true Multicast End-Point. TheabilityAMT Gateway will encapsulate AMT messages into UDP packets and send them through a tunnel (across the unicast-only infrastructure) toreceive requested application viathe AMT Relay. The simplest AMT Use Case (section 3.3) involves peering points that are not multicast enabled between two multicast enabled AD's. An AMT tunnel istriggered via receipt ofdeployed between an AMT Relay on thenecessary metadata. Hence, this metadata must be provided toAD 1 side of theEU regarding multicast URL -peering point andunicast fallback if applicable. AD-2 must enablean AMT Gateway on thedeliveryAD 2 side ofthis metadata totheEU and provision appropriate resources forpeering point. One advantage to thispurpose. Native multicast functionalityarrangement isassumed to be available across many ISP backbones, peering and access networks. If, however, native multicastthat the tunnel isnotestablished on anoption (Use Cases 3.4as needed basis and3.5), then: o EU must have multicast client to useneed not be a provisioned element. The two AD's can coordinate and advertise special AMTmulticast obtained either from Application Source (per agreementRelay Anycast addresses withAD-1) or from AD-1 or AD-2 (if delegated by the Application Source). o If provided by AD-1/AD-2, then the EU could be redirectedeach other. Alternately, they may decide toa client download site (note:simply provision Relay addresses, though thiscouldwould not be anApplication Source site). If provided byoptimal solution in terms of scalability. Use Cases 3.4 and 3.5 describe more complicated AMT situations as AD-2 is not multicast enabled. For these cases, theApplication Source, then this Source would haveEnd User device needs tocoordinate with AD-1be able toensuresetup an AMT tunnel in theproper clientmost optimal manner. There are many methods by which relay selection can be done including the use of DNS based queries and static lookup tables [RFC7450]. The choice of the method isprovided (assuming multiple possible clients). o Where AMT Gateways support different application sets, all AD-2 AMT Relays needimplementation dependent and is up tobe provisioned with all source & group addressesthe network operators. Comparison of various methods is out of scope forstreamsthis document; it isallowed to join. o DNS across each AD must be provisioned to enablefor further study. An illustrative example of aclient GWrelay selection based on DNS queries and Anycast IP addresses process for Use Cases 3.4 and 3.5 is described here. Using an Anycast IP address for AMT Relays allows for all AMT Gateways tolocatefind theoptimal"closest" AMT Relay(i.e. longest multicast path and shortest unicast tunnel) with connectivity to- the nearest edge of thecontent'smulticast topology of the source.Provisioning Aspects Related to Operations and Customer Care are stated as follows. Each AD providerNote that this isassumed to provision operations and customer care access to their own systems. AD-1's operations and customer care functions must have visibility to whatstrictly illustrative; the choice of the method ishappening in AD-2's network orup to theservice providednetwork operators. The basic process is as follows: o Appropriate metadata is obtained byAD- 2, sufficientthe EU client application. The metadata contains instructions directing the EU client toverify their mutual goals and operations, e.g.an ordered list of particular destinations toknow howseek theEU's are being served. This can be donerequested stream and, for multicast, specifies the source location and the group (or stream) ID intwo ways: o Automated interfaces are built between AD-1the form of the "S,G" data. The "S" portion provides the URI (name or IP address) of the source of the multicast stream andAD-2 suchthe "G" identifies the particular stream originated by thatoperations and customer care continue using their own systems. This requires coordination betweensource. The metadata may also contain alternate delivery information such as thetwo AD's with appropriate provisioningaddress ofnecessary resources.the unicast form of the content to be used, for example, if the multicast stream becomes unavailable. oAD-1's operationsUsing the information from the metadata, andcustomer care personnel are provided accesspossibly information provisioned directlyto AD-2's system. In this scenario, additional provisioninginthese systems will be needed to provide necessary access. Additional provisioning must be agreed to bythetwo AD's to support this option. 4.3.2. Application Accounting Guidelines All interactionsEU client, a DNS query is initiated in order to connect the EU client/AMT Gateway to an AMT Relay. o Query results are obtained, and may return an Anycast address or a specific unicast address of a relay. Multiple relays will typically exist. The Anycast address is a routable "pseudo- address" shared among the relays that can gain multicast access to the source. o If a specific IP address unique to a relay was not obtained, the AMT Gateway then sends a message (e.g., the discovery message) to the Anycast address such that the network is making the routing choice of particular relay - e.g., closest relay to the EU. Details are outside the scope for this document. See [RFC4786]. o The contacted AMT Relay then returns its specific unicast IP address (after which the Anycast address is no longer required). Variations may exist as well. o The AMT Gateway uses that unicast IP address to initiate a three- way handshake with the AMT Relay. o AMT Gateway provides "S,G" to the AMT Relay (embedded in AMT protocol messages). o AMT Relay receives the "S,G" information and uses the S,G to join the appropriate multicast stream, if it has not already subscribed to that stream. o AMT Relay encapsulates the multicast stream into the tunnel betweenpairsthe Relay and the Gateway, providing the requested content to the EU. 4.2.4. Public Peering Routing Aspects AD-1a AD-1b BR BR | | --+-+---------------+-+-- broadcast peering point LAN | | BR BR AD-2a AD-2b Figure 6: Broadcast Peering Point A broadcast peering point is an L2 subnet connecting 3 or more ADs. It is common in IXPs and usually consists of ethernet switch(es) operated by the IXP connecting to BRs operated by the ADs. In an example setup domain AD-2a peers with AD-1a and wants to receive IP multicast from it. Likewise AD-2b peers with AD-1b and wants to receive IP multicast from it. Assume one or more IP multicast (S,G) traffic streams can be served by both AD-1a and AD-1b, for example because both AD-1a and AD-1b do contract this content from the same content source. In this case, AD-2a and AD-2b can not control anymore which upstream domain, AD-1a or AD-1b will forward this (S,G) into the LAN. AD-2a BR requests the (S,G) from AD-1a BR and AD-2b BR requests the same (S,G) from AD-1b BR. To avoid duplicate packets, an (S,G) can be forwarded by only one router onto the LAN, and PIM-SM/PIM-SSM detects requests for duplicate transmission and resolve it via the so-called "assert" protocol operation which results in only one BR forwarding the traffic. Assume this is AD-1a BR. AD-2b will then receive the multicast traffic unexpectedly from a provider with whom it does not have a mutual agreement for the traffic. Quality issues in EUs behind AD-2b caused by AD-1a will cause a lot of responsiblity and troubleshooting issues. In face of this technical issues, we describe the following options how IP multicast can be carried across broadcast peering point LANs: 1. IP multicast is tunneled across the LAN. Any of the GRE/AMT tunneling solutions mentioned in this document are applicable. This is the one case where specifically a GRE tunnel between the upstream BR (e.g.: AD-1a) and downstream BR (e.g.: AD-2a) is recommended as opposed to tunneling across uBRs which are not the actual BRs. 2. The LAN has only one upstream AD that is sourcing IP multicast and native IP multicast is used. This is an efficient way to distribute the same IP multicast content to multiple downstream ADs. Misbehaving downstream BRs can still disrupt the delivery of IP multicast from the upstream BR to other downstream BRs, therefore strict rules must be follow to prohibit that case. The downstream BRs must ensure that they will always consider only the upstream BR as a source for multicast traffic: e.g.: no BGP SAFI-2 peerings between the downstream ADs across the peering point LAN, so that only the upstream BR is the only possible next-hop reachable across this LAN. And routing policies configured to avoid fall back to the use of SAFI-1 (unicast) routes for IP multicast if unicast BGP peering is not limited in the same way. 3. The LAN has multiple upstreams, but they are federated and agree on a consistent policy for IP multicast traffic across the LAN. One policy is that each possible source is only announced by one upstream BR. Another policy is that sources are redundantly announced (problematic case mentioned in above example), but the upstream domains also provide mutual operational insight to help troubleshooting (outside the scope of this document). 4.3. Back Office Functions - Provisioning and Logging Guidelines Back Office refers to the following: o Servers and Content Management systems that support the delivery of applications via multicast and interactions between AD's. o Functionality associated with logging, reporting, ordering, provisioning, maintenance, service assurance, settlement, etc. 4.3.1. Provisioning Guidelines Resources for basic connectivity between AD's Providers need to be provisioned as follows: o Sufficient capacity must be provisioned to support multicast-based delivery across AD's. o Sufficient capacity must be provisioned for connectivity between all supporting back-offices of the AD's as appropriate. This includes activating proper security treatment for these back- office connections (gateways, firewalls, etc) as appropriate. o Routing protocols as needed, e.g. configuring routers to support these. Provisioning aspects related to Multicast-Based inter-domain delivery are as follows. The ability to receive requested application via multicast is triggered via receipt of the necessary metadata. Hence, this metadata must be provided to the EU regarding multicast URL - and unicast fallback if applicable. AD-2 must enable the delivery of this metadata to the EU and provision appropriate resources for this purpose. Native multicast functionality is assumed to be available across many ISP backbones, peering and access networks. If, however, native multicast is not an option (Use Cases 3.4 and 3.5), then: o EU must have multicast client to use AMT multicast obtained either from Application Source (per agreement with AD-1) or from AD-1 or AD-2 (if delegated by the Application Source). o If provided by AD-1/AD-2, then the EU could be redirected to a client download site (note: this could be an Application Source site). If provided by the Application Source, then this Source would have to coordinate with AD-1 to ensure the proper client is provided (assuming multiple possible clients). o Where AMT Gateways support different application sets, all AD-2 AMT Relays need to be provisioned with all source & group addresses for streams it is allowed to join. o DNS across each AD must be provisioned to enable a client GW to locate the optimal AMT Relay (i.e. longest multicast path and shortest unicast tunnel) with connectivity to the content's multicast source. Provisioning Aspects Related to Operations and Customer Care are stated as follows. Each AD provider is assumed to provision operations and customer care access to their own systems. AD-1's operations and customer care functions must have visibility to what is happening in AD-2's network or to the service provided by AD- 2, sufficient to verify their mutual goals and operations, e.g. to know how the EU's are being served. This can be done in two ways: o Automated interfaces are built between AD-1 and AD-2 such that operations and customer care continue using their own systems. This requires coordination between the two AD's with appropriate provisioning of necessary resources. o AD-1's operations and customer care personnel are provided access directly to AD-2's system. In this scenario, additional provisioning in these systems will be needed to provide necessary access. Additional provisioning must be agreed to by the two AD's to support this option. 4.3.2. Interdomain Authentication Guidelines All interactions between pairs of AD's can be discovered and/or be associated with the account(s) utilized for delivered applications. Supporting guidelines are as follows: o A unique identifier is recommended to designate each master account. o AD-2 is expected to set up "accounts" (logical facility generally protected by credentials such as login passwords) for use by AD-1. Multiple accounts and multiple types or partitions of accounts can apply, e.g. customer accounts, security accounts, etc. The reason to specifically mention the need for AD-1 to initiate interactions with AD-2 (and use some account for that), as opposed to the opposite direction is based on the recommended workflow initiated by customers (see Section 4.4): The customer contacts content source (part of AD-1), when AD-1 sees the need to propagate the issue, it will interact with AD-2 using the aforementioned guidelines. 4.3.3. Log Management Guidelines Successful delivery (in terms of user experience) of applications or content via multicast between pairs of interconnecting AD's can be improved through the ability to exchange appropriate logs for various workflows - troubleshooting, accounting and billing, traffic and content transmission optimization, content and application development optimization and so on. The basic model as explained in before is that the content source and on its behalf AD-1 take over primary responsibility for customer experience and the AD-2's support this. The application/content owner is the only participant who has and needs full insight into the application level and can map the customer application experience to the network traffic flows - which it then with the help of AD-2 or logs from AD-2 can analyze and interpret. The main difference between unicast delivery and multicast delivery is that the content source can infer a lot more about downstream network problems from a unicasted stream than from a multicasted stream: The multicasted stream is not per-EU except after the last replication, which is in most cases not in AD-1. Logs from the application, including the receiver side at the EU, can provide insight, but can not help to fully isolate network problems because of the IP multicast per-application operational state built across AD-1 and AD-2 (aka: the (S,G) state and any other feature operational state such as DiffServ QoS). See Section 7 for more discussions about the privacy considerations of the model described here. Different type of logs are known to help support operations in AD-1 when provided by AD-2. This could be done as part of AD-1/AD-2 contracts. Note that except for implied multicast specific elements, the options listed here are not unique or novel for IP multicast, but they are more important for services novel to the operators than for operationally well established services (such as unicast). Therefore we detail them as follows: o Usage information logs at aggregate level. o Usage failure instances at an aggregate level. o Grouped or sequenced application access. performance, behavior and failure at an aggregate level to support potential Application Provider-driven strategies. Examples of aggregate levels include grouped video clips, web pages, and sets of software download. o Security logs, aggregated or summarized according to agreement (with additional detail potentially provided during security events, by agreement). o Access logs (EU), when needed for troubleshooting. o Application logs (what is the application doing), when needed for shared troubleshooting. o Syslogs (network management), when needed for shared troubleshooting. The two AD's may supply additional security logs to each other as agreed to by contract(s). Examples include the following: o Information related to general security-relevant activity which may be of use from a protective or response perspective, such as types and counts of attacks detected, related source information, related target information, etc. o Aggregated or summarized logs according to agreement (with additional detail potentially provided during security events, by agreement). 4.4. Operations - Service Performance and Monitoring Guidelines Service Performance refers to monitoring metrics related to multicast delivery via probes. The focus is on the service provided by AD-2 to AD-1 on behalf of all multicast application sources (metrics may be specified for SLA use or otherwise). Associated guidelines are as follows: o Both AD's are expected to monitor, collect, and analyze service performance metrics for multicast applications. AD-2 provides relevant performance information to AD-1; this enables AD-1 to create an end-to-end performance view on behalf of the multicast application source. o Both AD's are expected to agree on the type of probes to be used to monitor multicast delivery performance. For example, AD-2 may permit AD-1's probes to be utilized in the AD-2 multicast service footprint. Alternately, AD-2 may deploy its own probes and relay performance information back to AD-1. Service Monitoring generally refers to a service (as a whole) provided on behalf of a particular multicast application source provider. It thus involves complaints from End Users when service problems occur. EUs direct their complaints to the source provider; in turn the source provider submits these complaints to AD-1. The responsibility for service delivery lies with AD-1; as such AD-1 will need to determine where the service problem is occurring - its own network or in AD-2. It is expected that each AD will have tools to monitor multicast service status in its own network. o Both AD's will determine how best to deploy multicast service monitoring tools. Typically, each AD will deploy its own set of monitoring tools; in which case, both AD's are expected to inform each other when multicast delivery problems are detected. o AD-2 may experience some problems in its network. For example, for the AMT Use Cases, one or more AMT Relays may be experiencing difficulties. AD-2 may be able to fix the problem by rerouting the multicast streams via alternate AMT Relays. If the fix is not successful and multicast service delivery degrades, then AD-2 needs to report the issue to AD-1. o When problem notification is received from a multicast application source, AD-1 determines whether the cause of the problem is within its own network or within the AD-2 domain. If the cause is within the AD-2 domain, then AD-1 supplies all necessary information to AD-2. Examples of supporting information include the following: o Kind of problem(s). o Starting point & duration of problem(s). o Conditions in which problem(s) occur. o IP address blocks of affected users. o ISPs of affected users. o Type of access e.g., mobile versus desktop. o Network locations of affected EUs. o Both AD's conduct some form of root cause analysis for multicast service delivery problems. Examples of various factors for consideration include: o Verification that the service configuration matches the product features. o Correlation and consolidation of the various customer problems and resource troubles into a single root service problem. o Prioritization of currently open service problems, giving consideration to problem impact, service level agreement, etc. o Conduction of service tests, including one time tests or a series of tests over a period of time. o Analysis of test results. o Analysis of relevant network fault or performance data. o Analysis of the problem information provided by the customer (CP). o Once the cause of the problem has been determined and the problem has been fixed, both AD's need to work jointly to verify and validate the success of the fix. 4.5. Client Reliability Models/Service Assurance Guidelines There are multiple options for instituting reliability architectures, most are at the application level. Both AD's should work those out with their contract or agreement and with the multicast application source providers. Network reliability can also be enhanced by the two AD's by provisioning alternate delivery mechanisms via unicast means. 4.6. Application Accounting Guidelines Application level accounting needs to be handled differently in the application than in IP unicast because the source side does not directly deliver packets to individual receivers. Instead, this needs to be signalled back by the receiver to the source. For network transport diagnostics, AD-1 and AD-2 should have mechanisms in place to ensure proper accounting for the volume of bytes delivered through the peering point and separately the number of bytes delivered to EUs. 5. Troubleshooting and Diagnostics Any service provider supporting multicast delivery of content should have the capability to collect diagnostics as part of multicast troubleshooting practices and resolve network issues accordingly. Issues may become apparent or identified either through network monitoring functions or by customer reported problems as described in section 4.4. It is recommended that multicast diagnostics will be performed leveraging established operational practices such as those documented in [MDH-04]. However, given that inter-domain multicast creates a significant interdependence of proper networking functionality between providers there does exist a need for providers to be able to signal (or otherwise alert) each other if there are any issues noted by either one. Service providers may also wish to allow limited read-only administrative access to their routers to their AD peers for troubleshooting. Of specific interest are access to active troubleshooting tools especially [Traceroute] and [I-D.ietf-mboned-mtrace-v2]. Another option is to include this functionality into the IP multicast receiver application on the EU device and allow for these diagnostics to be remotely used by support operations. Note though that AMT does not allow to pass traceroute or mtrace requests, therefore troubleshooting in the presence of AMT does not work as well end-to- end as it can with native (or even GRE encapsulated) IP multicast, especially wrt. to traceroute and mtrace. Instead, troubleshooting directly on the actual network devices is then more likely necessary. The specifics of the notification and alerts are beyond the scope of this document, but general guidelines are similar to those described in section 4.4 (Service Performance and Monitoring). Some general communications issues are stated as follows. o Appropriate communications channels will be established between the customer service and operations groups from both AD's to facilitate information sharing related to diagnostic troubleshooting. o A default resolution period may be considered to resolve open issues. Alternately, mutually acceptable resolution periods could be established depending on the severity of the identified trouble. 6. Security Considerations 6.1. DoS attacks (against state and bandwidth) Reliable operations of IP multicast requires some basic protection against DoS (Denial of Service) attacks. SSM IP multicast is self protecting against attacks from illicit sources. Their traffic will not be forwarded beyond the first hop router because that would require (S,G) memership reports from receiver. Traffic from sources will only be forwarded from the valid source because RPF ("Reverse Path Forwarding") is part of the protocols. One can say that [BCP38] style protection against spoofed source traffic is therefore built into PIM-SM/PIM-SSM. Receivers can attack SSM IP multicast by originating such (S,G) membership reports. This can result in a DoS attack against state through the creation of a large number of (S,G) states that create high control plane load or even inhibit later creation of valid (S,G). In conjunction with collaborating illicit sources it can also result in illicit sources traffic being forwarded. Today, these type of attacks are usually mitigated by explicitly defining the set of permissible (S,G) on e.g.: the last hop routers in replicating IP multicast to EUs; For example via (S,G) Access Control Lists applied to IGMP/MLD membership state creation. Each AD is expected to prohibit (S,G) state creation for invalid sources inside their own AD. In the peering case, AD-2 is without further information not aware of the set of valid (S,G) from AD-1, so this set needs to be communicated via operational procedures from AD-1 to AD-2 to provide protection against this type of DoS attacks. Future work could signal this information in an automated way: BGP extensions, DNS Resource Records or backend automation between AD-1 and AD-2. Backend automation is the short term most viable solution because it does not require router software extensions like the other two. Observation of traffic flowing via (S,G) state could also be used to automate recognition of invalid (S,G) state created by receivers in the absence of explicit information from AD-1. The second DoS attack through (S,G) membership reports is when receivers create too much valid (S,G) state to attack bandwidth available to other EU. Consider the uplink into a last-hop-router connecting to 100 EU. If one EU joins to more multicast content than what fits into this link, then this would impact also the quality of the same content for the other 99 EU. If traffic is not rate adaptive, the effects are even worse. The mitigation is the same as what is often employed for unicast: Policing of per-EU total amont of traffic. Unlike unicast though, this can not be done anywhere along the path (e.g.: on an arbitrary bottleneck link), but it has to happen at the point of last replication to the different EU. Simple solutions such as limiting the maximum number of joined (S,G) per EU are readily available, solutions that consider bandwidth consumed exist as vendor specific feature in routers. Note that this is primarily a non-peering issue in AD-2, it only becomes a peering issue if the peering-link itself is not big enough to carry all possible content from AD-1 or in case 3.4 where the AMT relay in AD-1 is that last replication point. Limiting the amount of (S,G) state per EU is also a good first measure to prohibit too much undesired "empty" state to be built (state not carrying traffic), but it would not suffice in case of DDoS attack - viruses that impact a large number of EU devices. 6.2. Content Security Content confidentiality, DRM (Digital Restrictions Management), authentication and authorization are optional based on the content delivered. For content that is "FTA" (Free To Air), the following considerations can be ignored and content can be sent unencrypted and without EU authentication and authorization. Note though that the mechanisms described here may also be desireable by the application source to better track users even if the content itself would not require it. For interdomain content, there are at least two models for content confidentiality, DRM and end-user authentication and authorization: In the classical (IP)TV model, responsibility is per-domain and content is and can be passed on unencrypted. AD-1 delivers content to AD-2, AD-2 can further process the content including features like ad-insertion and AD-2 is the sole point of contact regarding the contact for its EUs. In this document, we do not consider this case because it typically involves higher than network layer service aspects operated by AD-2 and this document focusses on the network layer AD-1/AD-2 peering case, but not the application layer peering case. Nevertheless, this model can be derived through additional work from what is describe here. The other case is the one in which content confidentiality, DRM, end- user authentication and authorization are end-to-end: responsibilities ofAD'sthe multicast application source provider and receiver application. This is the model assumed here. It is also the model used in Internet OTT video delivery. We discuss the threads incurred in this model due to the use of IP multicast in AD- 1/AD-2 and across the peering. End-to-end encryption enables end-to-end EU authentication and authorization: The EU may be able to IGMP/MLD join and receive the content, but it can only decrypt it when it receives the decryption key from the content source in AD-1. The key is the authorization. Keeping that key to itself and prohibiting playout of the decrypted content to non-copy-protected interfaces are typical DRM features in that receiver application or EU device operating system. End-to-end ecnryption is continuously attacked. Keys may bediscovered and/orsubject to brute force attack so that content can beassociateddecrypted potentially later, or keys are extracted from the EU application/device and shared with other unauthenticated receivers. One important class of content is where theaccount(s) utilized for delivered applications. Supporting guidelines arevalue is in live consumption, such asfollows: o A unique identifiersports or other event (concert) streaming. Extraction of keying material from compromised authenticated EU and sharing with unauthenticated EU isrecommended to designate each master account. o AD-2not sufficient. It isexpected to set up "accounts" (logical facility generally protected by login/password/credentials)also necessary foruse by AD-1. Multiple accounts and multiple types/partitionsthose unauthenticated EUs to get a streaming copy ofaccountsthe content itself. In unicast streaming, they canapply, e.g. customer accounts, security accounts, etc. 4.3.3. Log Management Guidelines Successful deliverynot get such a copy from the content source (because they can not authenticate) and because ofapplications via multicast between pairsasymmetric bandwidths, it is often impossible to get the content from compromised EUs to large number ofinterconnecting AD's requires that appropriate logsunauthenticated EUs. EUs behind classical 16 Mbps down, 1 Mbps up ADSL links are the best example. With increasing broadband access speeds unicast peer-to-peer copying of content becomes easier, but it likely will always beexchanged between themeasily detectable by the ADs because of its traffic patterns and volume. When IP multicast is being used without additionals security, AD-2 is not aware which EU is authenticated for which content. Any unauthenticated EU insupport. Associated guidelines are as follows.AD-2needs to supply logs to AD-1 per existing contract(s). Examplescould therefore get a copy oflog types includethefollowing: o Usage information logs at aggregate level. o Usage failure instances at an aggregate level. o Groupedencrypted content without suspicion by AD-2 orsequenced application access. performance/behavior/ failure at an aggregate level to support potential Application Provider-driven strategies. Examples of aggregate levels include grouped video clips, web pages,AD-1 andsetseither live- deode it in the presence ofsoftware download. o Security logs, aggregatedcompromised authenticated EU and key sharing, orsummarized according to agreement (with additional detail potentially provided during security events, by agreement). o Access logs (EU), when needed for troubleshooting. o Application logs (what islater decrypt it in theapplication doing), when needed for shared troubleshooting. o Syslogs (network management), when needed for shared troubleshooting. The two AD's may supply additional security logspresence of federated brute force key cracking. To mitigate this issue, the last replication point that is creating (S,G) copies toeach other as agreedEUs would need toby contract(s). Examples includepermit those copies only after authentication of EUs. This would establish thefollowing: o Information related to general security-relevant activity which may besame authenticated EU only copy deliver thast is used in unicast. Schemes for per EU IP multicast authentication/authorization (and in result non-delivery/copying ofuse from a protective or response perspective, such as typesper-content IP multicast traffic) have been built in the past andcountsare deployed in service providers for intradomain IPTV services, but no standard exist for this. For example, there is no standardized radius attribute for authenticating the IGMP/MLD filter set, but implementations of this exist. The authors are specifically also not aware ofattacks detected, relatedschemes where the same authentication credentials used to get the encryption key from the content sourceinformation, related target information, etc. o Aggregated or summarized logs accordingcould also be used toagreement (with additional detail potentially provided during security events, by agreement). 4.4. Operations - Service Performanceauthenticate andMonitoring Guidelines Service Performance refers to monitoring metrics related to multicast delivery via probes. The focus is onauthorize theservice provided by AD-2 to AD-1 on behalf of allnetwork layer IP multicastapplication sources (metrics may be specifiedreplication forSLA use or otherwise). Associated guidelines are as follows: o Both AD'sthe content. Such schemes areexpectedtechnically not difficult tomonitor, collect,build andanalyze service performance metrics for multicast applications. AD-2 provides relevant performance information to AD-1; this enables AD-1 to create anwould avoid creating and maintaining a separate network forwarding authentication/ authorization scheme decoupled from the end-to-endperformance view on behalfauthentication/ authorization system of themulticast application source. o Both AD'sapplication. If delivery of such high value content in conjunction with the peering described here is desired, the short term recommendations areexpectedfor sources toagree onclearly isolate thetype of probes to besource and group addresses used for different content bundles, communicate those (S,G) patterns from AD-1 tomonitor multicast delivery performance. For example, AD-2 may permit AD-1's probes to be utilized inthe AD-2multicast service footprint. Alternately, AD-2 may deploy its own probesandrelay performance information backlet AD-2 leverage existing per-EU authentication/ authorization mechanisms in network devices toAD-1. oestablish filters for (S,G) sets to each EU. 6.3. Peering Encryption Encryption at peering points for multicast delivery may be used per agreement between AD-1/AD-2. In theeventcase of a private peering link, IP multicast does not have attack vectors on a peering link different from those ofperformance degradation (SLA violation), AD-1 may have to compensateIP unicast, but themulticast application source per SLA agreement. As appropriate, AD-1content owner mayseek compensation from AD-2 if the causehave defined high bars against unauthenticated copying of even thedegradation isend-to-end encrypted content, and inAD-2's network. Service Monitoring generally refers to a service (as a whole) providedthis case AD-1/AD-2 can agree onbehalfadditional transport encryption across that peering link. In the case of aparticular multicast application source provider. It thus involves complaints from End Users when service problems occur. EUs direct their complaints to the source provider; in turnbroadcast peering connection (e.g.: IXP), transport encryption is also thesource provider submits these complaints to AD-1. The responsibility for service delivery lies with AD-1; as such AD-1 will needeasiest way todetermine whereprohibit unauthenticated copies by other ADs on theservice problemsame peering point. If peering isoccurring - its own network oracross a tunnel going across intermittent transit ADs (not discused inAD-2. It is expected that each AD will have tools to monitor multicast service statusdetail inits own network. o Both AD's will determine how best to deploy multicast service monitoring tools. Typically, each AD will deploy its own setthis document), then encryption ofmonitoring tools; in which case, both AD's are expectedthat tunnel traffic is recommended. It not only prohibits possible "leakage" of content, but also toinform each other when multicast delivery problems are detected. o AD-2 may experience some problemsprotects the the information what content is being consumed inits network. For example,AD-2 (aggregated privacy protection). See the following subsection for reasons why theAMT Use Cases, one or more AMT Relays may be experiencing difficulties. AD-2peering point maybe ablealso need tofix the problembe encrypted for operational reasons. 6.4. Operational Aspects Section 4.3.3 discusses exchange of log information, this section discussed exchange of (S,G) information and Section 7 discusses exhange of program information. All these operational pieces of data should byrerouting the multicast streamsdefault be exchanged viaalternate AMT Relays. If the fix is not successfulauthenticated and encrypted peer- to-peer communication protocols between AD-1 andmulticast service delivery degrades, thenAD-2needsso that only the intended recipient in the peers AD have access toreportit. Even exposure of theissueleast sensitive information toAD-1. o When problem notification is receivedthird parties opens up attack vectors. Putting for example valid (S,G) information into DNS (as opposed to passing it via secured channels froma multicast application source,AD-1determines whetherto AD-2) to allow easier filtering of invalid (S,G) would also allow attackers to easier identify valid (S,G) and change their attack vector. From thecauseperspective of theproblemADs, security iswithin its own network or within the AD-2 domain. Ifmost critical for thecause is withinlog information as it provides operational insight into the originating AD, but it also contains sensitive user data: Sensitive user data exported from AD-2domain, then AD-1 supplies all necessary informationtoAD-2. ExamplesAD-1 as part ofsupporting information includelogs could be as much as thefollowing: o Kind of problem(s). o Starting point & durationequivalent ofproblem(s). o Conditions5-tuple unicast traffic flow accounting (but not more, e.g.: no application level information). As mentioned inwhich problem(s) occur. o IP address blocks of affected users. o ISPs of affected users. o Type of access e.g., mobile versus desktop. o Locations of affected EUs. o Both AD's conduct some form of root cause analysis for multicast service delivery problems. Examples of various factors for consideration include: o Verification that the service configuration matches the product features. o CorrelationSection 7, in unicast, AD-1 could capture these traffic statistics itself because this is all about AD-1 originated traffic flows to EU receivers in AD-2, andconsolidationoperationally passing it from AD-2 to AD-1 may be necessary when IP multicast is used because of thevarious customer problems and resource troubles intoreplication happening in AD-2. Nevertheless, passing such traffic statistics inside AD-1 from asingle root service problem. o Prioritization of currently open service problems, giving considerationcapturing router toproblem impact, service level agreement, etc. o Conduction of service tests, including one time tests or a series of tests overaperiod of time. o Analysis of test results. o Analysis of relevant network fault or performance data. o Analysis ofbackend system is likely less subject to third party attacks then passing it interdomain from AD-2 to AD-1, so more diligence needs to be applied to secure it. If any protocols used for theproblemoperational informationprovided by the customer (CP). o Onceexchange are not easily secured at transport layer or higher (because of thecauseuse of legacy products or protocols in theproblem has been determinednetwork), then AD-1 andthe problem has been fixed, both AD's need to work jointlyAD-2 can also consider toverify and validateensure that all operational data exchange goes across thesuccesssame peering point as the traffic and use network layer encryption of thefix. o Faultspeering point as discussed inservice could leadbefore toSLA violation for whichprotect it. End-to-end authentication and authorization of EU may involve some kind of token authentication and is done at themulticastapplicationsource provider may have to be compensated by AD-1. Subsequently, AD-1 may have to be compensated by AD-2 based onlayer independently of thecontract. 4.5. Client Reliability Models/Service Assurance Guidelines Theretwo AD's. If there aremultiple options for instituting reliability architectures, mostproblems related to failure of token authentication when end-users areatsupported by AD-2, then some means of validating proper working of theapplication level. Both AD's should work those out with their contract/agreement and withtoken authentication process (e.g., back-end servers querying the multicast application sourceproviders. Network reliability can alsoprovider's token authentication server are communicating properly) should beenhanced byconsidered. Implementation details are beyond the scope of this document. Security Breach Mitigation Plan - In the event of a security breach, the two AD'sby provisioning alternate delivery mechanisms via unicast means. 5. Troubleshooting and Diagnostics Any service provider supporting multicast delivery of content shouldare expected to have a mitigation plan for shutting down thecapability to collect diagnostics as part of multicast troubleshooting practicespeering point andresolve network issues accordingly. Issues may become apparent or identified either through network monitoring functions or by customer reported problems as described in section 4.4.directing multicast traffic over alternative peering points. It is also expected thatmulticast diagnosticsappropriate information will becollected according to currently established practices [MDH-04]. However, given that inter-domain multicast creates a significant interdependence of proper networking functionality between providers there does exist a needshared forproviders to be able to signal/alert each other if there are any issues noted by either one. Service providers may also wish to allow limited read-only administrative access to their routers via a looking-glass style router proxy to facilitatethedebuggingpurpose ofmulticast control statesecuring the identified breach. 7. Privacy Considerations The described flow of information about content andpeering status. Software implementations forthe end-user described in thispurposedocument aims to maintain privacy: AD-1 isreadily available [Traceroute], [I-D.ietf-mboned-mtrace-v2]operating on behalf (or owns) the content source andcan be easily extended to provide access to commonly-used multicast troubleshooting commands in a secure manner. The specificsis therefore part of thenotification and alerts are beyondcontent-consumption relationship with thescope of this document, but general guidelinesend- user. The privacy considerations between the EU and AD-1 aresimilar to those describedtherefore insection 4.4 (Service Performance and Monitoring). Somegeneralcommunications issues are stated(exception see below) the same asfollows. o Appropriate communications channels willif no IP multicast was used, especially because for any privacy conscious content, end-to-end encryption can and should beestablished betweenused. Interdomain multicast transport service related information is provided by the AD-2 operators to AD-1. AD-2 is not required to gain additional insight into the user behavior through this process that it would not already have without thecustomerservice collaboration with AD-1 - unless AD-1 andoperations groupsAD-2 agree on it and get approval fromboth AD's to facilitate information sharing relatedthe EU. For example, if it is deemed beneficial for EU todiagnostic troubleshooting. o A default resolution period maydirectly get support from AD-2 then it would in general beconsiderednecessary for AD-2 toresolve open issues. Alternately, mutually acceptable resolution periods couldbeestablished depending on the severityaware of theidentified trouble. 6. Security Considerations Frommapping between content and network (S,G) state so that AD-2 knows which (S,G) to troubleshoot when the EU complains about problems with asecurity perspective, normal security procedures are expectedspecific content. The degree tobe followedwhich this dissemination is done byeach ADAD-1 explicitly tofacilitate multicast deliverymeet privacy expectations of EUs is typically easy toregistered and authenticated end users. Additionally: o Encryption - Peering point links may be encrypted per agreementassess by AD-1. Two simple examples: For a sports content bundle, every EU will happily click on the "i approve that the content program information is shared with your service provider" button, to ensure best service reliability because service conscious AD-2 would likely also try to ensure that high value content, such as the (S,G) formulticast delivery. o Security Breach Mitigation Plan - InSuperBowl like content would be theeventfirst to receive care in case ofa security breach,network issues. If thetwo AD's are expected to have a mitigation plan for shutting downcontent in question was one where thepeering point and directing multicast traffic over alternative peering points. It is alsoEU expected more privacy, the EU should prefer a content bundle thatappropriateincluded this content in a large variety of other content, have all content end-to- end encrypted and the programming informationwillnot be sharedforwith AD-2 to maximize privacy. Nevertheless, thepurposeprivacy ofsecuringtheidentified breach. DRM and Application Accounting, Authorization and Authentication shouldEU against AD-2 observing traffic would still be lower than in theresponsibility ofequivalent setup using unicast, because in unicast, AD-2 could not correlate which EUs are watching themulticast application source provider and/or AD-1. AD-1 needssame content and use that towork outdeduce theappropriate agreements withcontent. Note that even thesource provider. Network has no DRM responsibilities, but might have authentication and authorization obligations. These though are consistent with normal operationssetup in Section 3.4 where AD-2 is not involved in IP multicast at all does not provide privacy against this level ofa CDN to insure end user reliability, security and network security. AD-1 andanalysis by AD-2should have mechanismsbecause there is no transport layer encryption inplace to ensure proper accounting forAMT and therefore AD-2 can correlate by onpath traffic analysis who is consuming thevolume of bytes delivered throughsame content from an AMT relay from both thepeering point(S,G) join messages in AMT andseparatelythenumber of bytes delivered to EUs. For example, [BCP38] style filtering could be deployed by both AD'sidentical content segments (that where replicated at the AMT relay). In summary: Because only content toensurebe consumed by multiple EUs is carried via IP multicast here, and all that content can be end-to-end encrypted, the onlylegitimately sourcedIP multicastcontentspecific privacy consideration isexchanged between them. Authentication and authorization of EUfor AD-2 toreceive multicastknow or reconstruct what content an EU isdone at the application layer between the client application and the source. This may involve some kind of token authentication andconsuming. For content for which this isdone at the application layer independently of the two AD's. If there are problems related to failure of token authentication when end-users are supported by AD-2, thenundesirable, somemeans of validating proper workingform ofthe token authentication process (e.g., back-end servers querying the multicast application source provider's token authentication server are communicating properly) should be considered. Implementation detailsprotections as explained above arebeyondpossible, but ideally, thescopemodel ofthis document. 7. IANA Considerations No considerations identifiedSection 3.4 could be used inthis document 8. Conclusions This Best Current Practice document provides detailed Use Case scenarios for the transmission of applications via multicast across peering pointsconjunction with future work adding e.g.: dTLS [RFC6347] encryption betweentwo Administrative Domains. A detailed set of guidelines supporting the delivery is provided for all Use Cases. For Use Cases involvingAMTtunnels (cases 3.4relay and3.5), it is recommended that proper procedures are implemented suchEU. Note that IP multicast by nature would permit thevarious AMT Gateways (at the End User devices andEU privacy against theAMT nodes in AD-2) are able to findcountent source operator because unlike unicast, thecorrect AMT Relay in other AMT nodes as appropriate. Section 4.2content source does not natively know which EU is consuming which content: In all cases where AD-2 providesan overview of one methodreplication, only AD-2 does know this directly. This document does not attempt to describe a model thatfinds the optimal Relay-Gateway combination via the usedoes maintain such level ofan Anycast IP address for AMT Relays.privacy against the content source but only against exposure to intermediate parties, in this case AD-2. 8. IANA Considerations No considerations identified in this document. 9. Acknowledgments The authors would like to thank the following individuals for their suggestions, comments, and corrections: Mikael Abrahamsson Hitoshi Asaeda Dale Carder Tim Chown Leonard Giuliano Jake Holland Joel Jaeggli Albert Manfredi Stig Venaas Henrik Levkowetz 10. Change log [RFC Editor: Please remove] Please see discussion on mailing list for changes before -111. -11: version in IESG review. -12: XML'ified version of -11, committed solely to make rfcdiff easier. XML versions hosted on https://www.github.com/toerless/ peering-bcp -13: o IESG feedback. Complete details in: https://raw.githubusercontent.com/toerless/peering-bcp/master/11- iesg-review-reply.txt o Ben Campbell: Location information about EU (End User) is Network Locatio information o Ben Campbell: Added explanation of assumption to introduction that traffic is sourced from AD-1 to (one or many) AD-2, mentioned that sourcing from EU is out of scope. o Introduction: moved up bullet points about exchanges and transit to clean up flow of assumptions. o Ben Campbell: Added picture for the GRE case, visualized tunnels in all pictures. o Ben Campbell: See 13-discus.txt on github for more details of changes for this review. o Alissia Cooper: Added more explanation for Log Management, explained privacy context. o Alissia Cooper: removed pre pre-RFC5378 disclaimer. o Alissia Cooper: removed mentioning of potential mutual compensation between domains if the other violates SLA. o Mirja Kuehlewind: created section 4.1.1 to discuss congestion control more detailled, adding reference to BCP145, removed stub CC paragraphs from section 3.1 (principle applies to every section 3.x, and did not want to duplicate text between 3.x and 4.x). o Mirja Kuehlewind: removed section 8 (conclusion). Text was not very good, not important to hae conclusion, maybe bring back with better text if strong interest. o Introduced section about broadcast peering points because there where too many places already where references to that case existed (4.2.4). o Introduced section about privact considerations because of comment by Ben Campbell and Alissa Cooper. o Rewrote security considerations and structured it into key aspects: DoS attacks, content protection, peering point encryption and operational aspects. o Kathleen Moriarty: Added operational aspects to security section (also for Alissia), e.g.: covering securing the exchange of operational data between ADs. o Spencer Dawkins: Various editorial fixes. Removed BCP38 text from section 3, superceeded be explanation of PIM-SM RPF check to provide equvialent security to BCP38 in security section 7.1). o Eric Roscorla: (fixed from other reviews already). o Adam Roach: Fixed up text about MDH-04, added reference to RFC4786. 11. References 11.1. Normative References [RFC2784] Farinacci, D., Li, T., Hanks, S., Meyer, D., and P. Traina, "Generic Routing Encapsulation (GRE)", RFC 2784, DOI 10.17487/RFC2784, March 2000, <https://www.rfc-editor.org/info/rfc2784>. [RFC3376] Cain, B., Deering, S., Kouvelas, I., Fenner, B., and A. Thyagarajan, "Internet Group Management Protocol, Version 3", RFC 3376, DOI 10.17487/RFC3376, October 2002, <https://www.rfc-editor.org/info/rfc3376>. [RFC3810] Vida, R., Ed. and L. Costa, Ed., "Multicast Listener Discovery Version 2 (MLDv2) for IPv6", RFC 3810, DOI 10.17487/RFC3810, June 2004, <https://www.rfc-editor.org/info/rfc3810>. [RFC4760] Bates, T., Chandra, R., Katz, D., and Y. Rekhter, "Multiprotocol Extensions for BGP-4", RFC 4760, DOI 10.17487/RFC4760, January 2007, <https://www.rfc-editor.org/info/rfc4760>. [RFC4604] Holbrook, H., Cain, B., and B. Haberman, "Using Internet Group Management Protocol Version 3 (IGMPv3) and Multicast Listener Discovery Protocol Version 2 (MLDv2) for Source- Specific Multicast", RFC 4604, DOI 10.17487/RFC4604, August 2006, <https://www.rfc-editor.org/info/rfc4604>. [RFC4609] Savola, P., Lehtonen, R., and D. Meyer, "Protocol Independent Multicast - Sparse Mode (PIM-SM) Multicast Routing Security Issues and Enhancements", RFC 4609, DOI 10.17487/RFC4609, October 2006, <https://www.rfc-editor.org/info/rfc4609>. [RFC7450] Bumgardner, G., "Automatic Multicast Tunneling", RFC 7450, DOI 10.17487/RFC7450, February 2015, <https://www.rfc-editor.org/info/rfc7450>. [RFC7761] Fenner, B., Handley, M., Holbrook, H., Kouvelas, I., Parekh, R., Zhang, Z., and L. Zheng, "Protocol Independent Multicast - Sparse Mode (PIM-SM): Protocol Specification (Revised)", STD 83, RFC 7761, DOI 10.17487/RFC7761, March 2016, <https://www.rfc-editor.org/info/rfc7761>. [BCP38] Ferguson,P., et al,P. and D. Senie, "Network Ingress Filtering: Defeating Denial of Service Attacks which employ IP Source Address Spoofing",BCP:BCP 38, RFC 2827, DOI 10.17487/RFC2827, May2000.2000, <https://www.rfc-editor.org/info/rfc2827>. [BCP41] Floyd, S., "Congestion Control Principles", BCP 41, RFC 2914, DOI 10.17487/RFC2914, September2000.2000, <https://www.rfc-editor.org/info/rfc2914>. [BCP145] Eggert, L., Fairhurst, G., and G. Shepherd, "UDP Usage Guidelines", BCP 145, RFC 8085, DOI 10.17487/RFC8085, March 2017, <https://www.rfc-editor.org/info/rfc8085>. 11.2. Informative References [RFC4786] Abley, J. and K. Lindqvist, "Operation of Anycast Services", BCP 126, RFC 4786, DOI 10.17487/RFC4786, December 2006, <https://www.rfc-editor.org/info/rfc4786>. [RFC6347] Rescorla, E. and N. Modadugu, "Datagram Transport Layer Security Version 1.2", RFC 6347, DOI 10.17487/RFC6347, January 2012, <https://www.rfc-editor.org/info/rfc6347>. [INF_ATIS_10] "CDN Interconnection Use Cases and Requirements in a Multi-Party Federation Environment", ATIS Standard A-0200010, December 2012. [MDH-04] Thaler, D. and others, "Multicast Debugging Handbook", IETF I-D draft-ietf-mboned-mdh-04.txt, May 2000. [Traceroute] , <http://traceroute.org/#source%20code>. [I-D.ietf-mboned-mtrace-v2] Asaeda, H., Meyer, K., and W. Lee, "Mtrace Version 2: Traceroute Facility for IP Multicast", draft-ietf-mboned- mtrace-v2-20 (work in progress), October 2017. Authors' Addresses Percy S. Tarapore (editor) AT&T Phone: 1-732-420-4172 Email: tarapore@att.com Robert Sayko AT&T Phone: 1-732-420-3292 Email: rs1983@att.com Greg Shepherd Cisco Email: shep@cisco.com Toerless Eckert (editor) Futurewei Technologies Inc. Email:tte@cs.fau.dette+ietf@cs.fau.de Ram Krishnan SupportVectors Email: ramkri123@gmail.com