[Docs] [txt|pdf|xml|html] [Tracker] [Email] [Nits]



IDR Working Group                                               S. Hares
Internet-Draft                                   Hickory Hill Consulting
Intended status: Informational                             July 14, 2020
Expires: January 15, 2021


                  BGP IPSEC VPNs - A Solution Analysis
                 draft-hares-idr-bgp-ipsec-analysis-00

Abstract

   This draft describes problems with IGP convergence time in some IPRAN
   networks that use a physical topology of grid backbones that connect
   rings of routers.  Part of these IPRAN network topologies exist in
   data centers with sufficient power and interconnections, but some
   network equipment sits in remote sites impacted by power loss.  In
   some geographic areas these remote sites may be subject to rolling
   blackouts.  These rolling power blackouts could cause multiple
   simultaneous node and link failures.  In these remote networks with
   blackouts, it is often critical that the IPRAN phone network re-
   converge quickly.

   The IGP running in these networks may run in a single level of the
   IGP.  This document seeks to briefly describe these problems to
   determine if the emerging IGP technologies (flexible algorithms,
   dynamic flooding, layers of hierarchy in IGPs) can be applied to help
   reduce convergence times.  It also seeks to determine if the
   improvements of these algorithms or the IP-Fast re-route algorithms
   are thwarted by the failure of multiple link and nodes.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on January 15, 2021.





Hares                   Expires January 15, 2021                [Page 1]


Internet-Draft             BGP-IPSEC-ANALYSIS                  July 2020


Copyright Notice

   Copyright (c) 2020 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
     1.1.  Requirements Language . . . . . . . . . . . . . . . . . .   3
     1.2.  IPSEC deployments . . . . . . . . . . . . . . . . . . . .   3
     1.3.  History of BGP passing Tunnel Endpoints . . . . . . . . .   4
     1.4.  Overview of proposals . . . . . . . . . . . . . . . . . .   5
     1.5.  Method of analysis  . . . . . . . . . . . . . . . . . . .   7
   2.  Security and Privacy  . . . . . . . . . . . . . . . . . . . .   8
     2.1.  Option 1 - Configuration Plus BGP Routes with Tunnel SA
           IDs . . . . . . . . . . . . . . . . . . . . . . . . . . .  12
     2.2.  Option 2- BGP passes client routes with SA-ID plus NLRI
           passes underlay SAs . . . . . . . . . . . . . . . . . . .  13
     2.3.  Option 3:  Secure EVPN (client routes + SA information) .  14
     2.4.  comparison of security issues . . . . . . . . . . . . . .  16
   3.  Manageability and Scaling . . . . . . . . . . . . . . . . . .  16
     3.1.  Configuration sizes - used for theoretical comparison . .  17
     3.2.  BGP Route sizes for theoretical comparison  . . . . . . .  18
       3.2.1.  Size of Tunnel encapsulation attribute with 1 SA per
               tunnel endpoint . . . . . . . . . . . . . . . . . . .  18
       3.2.2.  Size of Tunnel encapsulation attribute with 10 SAs
               per tunnel  . . . . . . . . . . . . . . . . . . . . .  19
     3.3.  Network Scenario 1  . . . . . . . . . . . . . . . . . . .  19
     3.4.  Network scenario 2  . . . . . . . . . . . . . . . . . . .  19
     3.5.  Scaling Memory sizes  . . . . . . . . . . . . . . . . . .  19
   4.  Key differences between the options . . . . . . . . . . . . .  19
   5.  Processing of BGP routes  . . . . . . . . . . . . . . . . . .  19
   6.  Future Issues - SBGP and Secure IPSEC VPNs  . . . . . . . . .  19
   7.  Security Considerations . . . . . . . . . . . . . . . . . . .  19
   8.  IANA considerations . . . . . . . . . . . . . . . . . . . . .  20
   9.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  20
     9.1.  Normative References  . . . . . . . . . . . . . . . . . .  20
     9.2.  Informative References  . . . . . . . . . . . . . . . . .  20



Hares                   Expires January 15, 2021                [Page 2]


Internet-Draft             BGP-IPSEC-ANALYSIS                  July 2020


   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  21

1.  Introduction

   This document analyzes three solutions for using a BGP based approach
   on SDWAN edge nodes to establish secure IPSec tunnels for the overlay
   routes distribution.  The solutions are:

   o  [I-D.hujun-idr-bgp-ipsec]

   o  [I-D.dunbar-idr-sdwan-edge-discovery]

   o  [I-D.sajassi-bess-secure-evpn]

   These three drafts propose an IPsec related tunnel type for an
   augmentation of [I-D.ietf-idr-tunnel-encaps] to support IPsec
   tunnels.  At IETF 105, IDR and BESS WG held a session to discuss the
   security issues in these emerging drafts with security directorate
   people.  The security people provided excellent feedback on on how to
   approach security, privacy, and scaling.  The IDR/BESS working
   members provided provided feedback on the scaling and concepts.  As a
   result of this session, it became evident that each proposal has
   started with a unique user scenario.

   Therefore, this draft simply reviews the technical qualities in terms
   of: 1) the security and privacy of each technology, and 2) how the
   technology is managed (manageability) and how the technology scales.

   The purpose of this draft to grow our joint understanding of each
   proposed IPSec tunnel type so that IDR and BESS can make informed
   decisions.  It is non-goal to determine which pf these 3 solutions
   fits a particular use case using VPN using BGP to pass IPsec tunnel
   end points.

1.1.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119].

1.2.  IPSEC deployments

   Secure VPNs based on IPsec tunnels began appearing around 2000.
   IPsec tunnels were used for secure link transport.  The IPSEC VPNs
   utilized IPsec tunnels over physical links or underlay networks to
   virtual links for these VPNs.  These IPsec tunnels can be created by
   configuring routers with tunnel endpoints and setting up security
   associations for these tunnels.  Automated processes can use IETF



Hares                   Expires January 15, 2021                [Page 3]


Internet-Draft             BGP-IPSEC-ANALYSIS                  July 2020


   network management protocols (NETCONF and RESTCONF) to configure Yang
   modules in the routers to set up these tunnels.

   Enterprise VPNs were created from these secure tunnels.  EVPNs and
   SDWANs have also deployed VPNs using IPSEC.

   The BGP client routes which use the tunnel as a pathway also
   distribute pathway information (endpoint, inner encapsulation, outer
   encapsulation) via BGP with tunnel attribute
   [I-D.ietf-idr-tunnel-encaps] For IPsec tunnels, there are three
   approaches to what security association information is included with
   the tunnel attribute.  (See [I-D.hujun-idr-bgp-ipsec],
   [I-D.dunbar-idr-sdwan-edge-discovery] and
   [I-D.sajassi-bess-secure-evpn].

   One of the newly defined user scenarios for the secure VPN is the
   SDWAN. [ [I-D.ietf-rtgwg-net2cloud-problem-statement] describes the
   problems faced by SDWAN.  [I-D.ietf-rtgwg-net2cloud-gap-analysis]
   describes the gaps in IETF technology for this use case.
   [I-D.dunbar-bess-bgp-sdwan-usage] describes how BGP is used as
   control plane to setup the SDWAN networks for various SDWAN use
   cases.  SDWAN overlay networks can run over physical forwarding by a
   wide variety of underlay networks.  SDWAN is one of the more recent
   developments in IPsec based VPNS created by an SDN controller.

   The author welcomes additional information on other use cases.

1.3.  History of BGP passing Tunnel Endpoints

   [RFC5512] defined SAFI to pass tunnel endpoint encapsulation
   information.  However, many operators and vendors preferred to pass
   this information in a BGP attribute.  [I-D.ietf-idr-tunnel-encaps]
   defines a BGP attribute for tunnels to replace [RFC5512]
   functionality, but does not address how to use RFC5566 without the
   encapsulation SAFI.  EVPN [RFC8365] also defined tunnel types for
   encapsulation.  The tunnel types registered with IANA (www.IANA.org)
   list the following tunnel types from [RFC5512], [RFC5566], and
   [RFC8365]:

   o  L2TPv3 over IP [RFC5512] [value 1],

   o  GRE [RFC5512] [value 2]

   o  Transmit tunnel endpoint [RFC5566][value 3]

   o  IPsec in tunnel mode [RFC5566] [value 4]

   o  IP in IP tunnel with IPsec Transport mode [RFC5566][value 5]



Hares                   Expires January 15, 2021                [Page 4]


Internet-Draft             BGP-IPSEC-ANALYSIS                  July 2020


   o  MPLS-in-IP tunnel with IPsec Transport mode [RFC5566][value 6]

   o  IP in IP [RFC5566] [value 7]

   o  VXLAN encapsulation [RFC8365][value 8]

   o  NVGRE encapsulation [RFC8365][value 9]

   o  MPLS Encapsulation [RFC8365][value 10]

   o  MPLS in GPE encapsulation [RFC8365] [value 11]

   o  VXLAN GPE encapsulation [RFC8365] [value 12]

   [I-D.ietf-idr-tunnel-encaps] has been created to address deficiencies
   in RFC5512 [RFC5512].  These deficiencies include: operational costs
   of using SAFI for tunnel identifiers, inability to specify egress
   endpoint of tunnel, unclear prefix-tunnel association, and inability
   to specify inner/outer encapsulation.  [I-D.ietf-idr-tunnel-encaps]
   defines new Sub-TLVs to support inner and outer encapsulation for
   these encapsulation types, and will become the main reference for
   these tunnel types.

   RFC5566 [RFC5566] defined the IP Tunnel Authenticator Sub-TLV for use
   in the SAFI, but these recent proposals have suggested different
   alternatives for replacing the Tunnel Authenticator function.

1.4.  Overview of proposals

   This section provides a technical overview of the 3 proposals
   [I-D.hujun-idr-bgp-ipsec], [I-D.dunbar-idr-sdwan-edge-discovery], and
   [I-D.sajassi-bess-secure-evpn].

   [I-D.hujun-idr-bgp-ipsec] proposes 3 new Sub-TLVs: local/remote
   Prefix Sub-TLV, Public Routing Instance Sub-TLV, and IPSec
   Configuration Sub-TLV (IPsec-Config).  The local/remote prefix Sub-
   TLV will not be discussed here as it does not clearly align to
   [I-D.ietf-idr-tunnel-encaps].  The optional Public Routing Instance
   (PRI) is used instead of a route target community so that local
   policy can filter routes for a specific community.  This feature
   provides the same feature as a Route target for a pre-configured set
   of PRIs.

   The IPsec Configuration Sub-TLV contains 4 octet opaque value to link
   the tunnel to the Tunnel Authentication entry found in a security
   association table on the local node.  This table will need to include
   which tunnel endpoints this security association is valid for.  This




Hares                   Expires January 15, 2021                [Page 5]


Internet-Draft             BGP-IPSEC-ANALYSIS                  July 2020


   analysis assumes the IETF protocols NETCONF RESTCONF configure a YANG
   module that has these security associations.

   [I-D.dunbar-idr-sdwan-edge-discovery] proposes UPDATEs from client
   routers to include the IPsec SA identifiers (ID) to reference the
   IPsec SA attributes being advertised by separate Underlay Property
   BGP UPDATE messages.  The security association table is built
   dynamically from the information passed in these Underlay Property
   BGP Updates plus some local configuration.  If a client route can be
   encrypted by multiple IPsec SAs, then multiple IPsec SA IDs are
   included in the Tunnel-Encaps Path attribute for the client route.
   This draft proposes two new Sub-TLVs: IPsec-SA-ID and IPsec-SA-Group.
   The IPsec-SA-ID is similar to [I-D.hujun-idr-bgp-ipsec] IPSec Config
   Sub-TLV passing a 2 octet pointer to into a security association
   table.  IPsec-SA-Group Sub-TLV optimizes passing the same information
   when multiple IPsec SAs with the same inner encapsulation header.

   [I-D.dunbar-idr-sdwan-edge-discovery] proposes underlay tunnel
   topology information for SDWAN in BGP UPDATEs.  The information is
   passed in a combination of NLRI with an SAFI=74 (SDWAN SAFI) and a
   Tunnel Encapsulation attribute with tunnel type being SDWAN-Underlay.

   Security association information for the tunnels in this underlay
   will be passed in the Tunnel Attribute using in the SDWAN Underlay
   tunnel type.  This new tunnel type which will support the current
   tunnel Sub-TLV plus the newly proposed IPSec SA Sub-TLV(s).  There
   are two types of IPsec SA Sub-TLVs proposed by
   [I-D.dunbar-idr-sdwan-edge-discovery], one is for general purpose
   deployment which requires a full-set of Security Association,
   including Nonce, Public Key, Proposal and Transform Sub-TLVs in the
   SDWAN SAFI NLRI (SA-TYPE =2).  Another type is for simple deployment
   which only needs one simple IPsec SA Sub-TLV included (SA-TYPE=1).
   In addition, it can also include other optional Sub-TLVs like NAT,
   WAN Port, Geo-location with the SDWAN SAFI route.

   [I-D.sajassi-bess-secure-evpn] proposes defines 2 new tunnel types
   (ESP-Transport and ESP-in-UDP-Transport) and 3 new Sub-TLVS (DIM Sub-
   TLV, Key-Exchange Sub-TLV, and proposal Sub-TLV) for these new tunnel
   types.  The new Sub-TLVs pass information regarding security
   associations.  The DIM Sub-TLV is required to be supported for the
   two new tunnel types.  As noted above, the SDWAN-WAN-Underlay tunnel
   type from [I-D.dunbar-idr-sdwan-edge-discovery] supports equivalent
   features to IPsec-SA, Public-key, and SA-Transforms.

   [I-D.dunbar-idr-sdwan-edge-discovery] and
   [I-D.sajassi-bess-secure-evpn] differ in the information included in
   the client routes.  [I-D.sajassi-bess-secure-evpn] attaches all the
   IPsec SA information to the actual client routes, whereas the



Hares                   Expires January 15, 2021                [Page 6]


Internet-Draft             BGP-IPSEC-ANALYSIS                  July 2020


   [I-D.dunbar-idr-sdwan-edge-discovery] only includes the IPsec SA IDs
   for the client routes.  The IPsec SA IDs used by
   [I-D.dunbar-idr-sdwan-edge-discovery] reference (point) to the SA-
   Information which is advertised separately.  All the SA-Information
   are attached to routes which describe the SDWAN underlay, such as WAN
   Ports or Node address.

   [I-D.sajassi-bess-secure-evpn] supports tunnel types of ESP-Transport
   and ESP-in-UDP transport, but not traditional IPsec tunnel types
   (IPsec in tunnel mode, IP in IP tunnel with IPsec transport, MPLS-in-
   IP tunnel with transport mode).  The use of the new tunnel type could
   be used in a similar fashion to [I-D.dunbar-idr-sdwan-edge-discovery]
   to pass SA-information regarding the underlay.
   [I-D.sajassi-bess-secure-evpn] seems to point to passing client
   routes upon a rekeying request.  This method will increase the amount
   of BGP traffic passed in the crash or initial start-up in the tunnel
   encapsulation attribute.

   Since [I-D.sajassi-bess-secure-evpn] draft has not recently been
   updated, it is not clear if the recent changes to
   [I-D.ietf-idr-tunnel-encaps] are reflected in this draft.
   [I-D.sajassi-bess-secure-evpn] depends on
   [I-D.carrel-ipsecme-controller-ike] which received many security
   comments at IETF 105.  Therefore, the author has analyzed
   [I-D.sajassi-bess-secure-evpn] solutions based on the following
   assumptions:

   o  ESP-Transport and ESP-in-UDP would have been aligned with the
      latest version of the [I-D.ietf-idr-tunnel-encaps],

   o  Only the DIM Sub-TLV is required to be sent during initialization,
      PE rekey requests, routing periodic updates, and node restarts
      (crash/load) for shared security controller policies.

   o  The multiple policy environments may increase the size of Tunnel
      Encapsulation attribute as transforms and transform attributes are
      sent.

1.5.  Method of analysis

   The things matter to the network operator of IP-SEC VPN in SDWAN:
   security, manageability, scaling, and privacy.  Each deployment of an
   IPSec VPN may combine different underlay networks with different
   challenges to security, manageability, scaling and privacy.  This
   analysis compares the basic technologies of these proposals in terms
   of two groups of features: 1) security and privacy, and 2)
   manageability and scaling.  This analysis drafts looks at each
   solution based on the strengths are weaknesses of each type.



Hares                   Expires January 15, 2021                [Page 7]


Internet-Draft             BGP-IPSEC-ANALYSIS                  July 2020


   Analyzing scaling can either be done at the 50,000 foot level or in
   excruciating detail.  This analysis will be at the 50,000 foot level
   using two example scenarios (small and very large)

   Scenario 1: The 3 Route Reflectors (RR) each have 5 client routers
   per router reflector.  The client routers have a potential of 5
   tunnels with 1 security association per tunnel.  Each client router
   has 200 routes.  The total number of configured tunnels is 20 tunnels
   per RR cluster and the total number of client routes is 3000.
   Diagram 1 in section 1.3 showed this simple topology for these route
   reflectors.

   Scenario 2: The 3 Route Reflectors (RR) each have 10,000 client
   routers.  Each client router supports 100 tunnels, 10K routes, and 10
   security associations per tunnel.  Each Route Reflectors will receive
   from its client routers a total of 100 million client routes with 1
   million tunnels client tunnels (100*10K client routers), and 10
   million security associations.  The totals for all 3 RR may be up to
   3 times this level (300 Million client routes, 3 million tunnels, and
   10 million security associations), but it is likely the RR will
   contain some redundancy.  Our scenario focuses on the challenges
   within a single RR clustr.

   The BGP scaling in these two scenarios contrast small IPsec VPNs and
   very large IPsec VPNs.  BGP routing products handle route
   distribution of over 100 Million routes so this scaling is well
   within the range of the BGP products.

2.  Security and Privacy

   During an initial security review of this information, Ben Kaduk made
   the following comments:

      "First off, when we start to get IPsec configuration via BGP, it's
      helpful to think of what other information we get in the same way,
      and to analyze the effects of misconfiguration or malicious
      configuration both on IPsec and the broader system.  For example,
      if we are getting NLRI from BGP, then a misconfiguration that
      gives us parameters that are incompatible with a peer's is not
      causing particularly new harm, since we could just as easily be
      told that peer is unreachable and we wouldn't try to talk to them
      anyway.  On the other hand, we could be given configuration to use
      computationally expensive algorithms which would increase the DOS
      risk in a way that may not (or may!) be already possible. " (email
      to IDR and BESS WGs after IETF 105)."

   The security analysis in this draft assumes that the route
   distribution for BGP routes is done via a mesh of Route Reflectors



Hares                   Expires January 15, 2021                [Page 8]


Internet-Draft             BGP-IPSEC-ANALYSIS                  July 2020


   which have route reflector clients associated.  The IBGP mesh of
   route reflectors within a domain is assumed to run over secure
   transport links (such as TLS).  If privacy is an issue for BGP route
   distribution, the TLS encrypts/decrypts the data in the IBGP mesh.
   If a single AS IBGP mesh of Route Reflectors connects to another AS,
   the EBGP connection is also over a secure transport (such as TLS).

         [Full mesh topology within a IBGP cloud
          LAN used to simply drawing)
                        TLS
           ===============================
          |            |                 |
          RR1---------- RR2 ------------ RR3
    TLS /  |  \      /   |   \         / |  \
       C1  C2  Cn   C1  C2    Cn      C1 C2 Cn


                         Route Reflector Topology

   This security topology has the same transport link secure topology as
   the NETCONF/RESTCONF security of set of NETCONF/RESTCONF servers.
   The example NETCONF topology is below.

           Back-end configuration database
                      TLS
           ===============================
          |            |                 |
        Netconf      Netconf            Netconf
        Client1 -------Client2 -------  Client 3
   TLS  /  |  \      /   |   \         /   |    \
      NS1 NS2 NSn  NS1  NS2  NSn     NS1  NS2   NSn

                             NETCONF Topology

   The security aspects of using network management protocols NETCONF or
   RESTCONF servers to control IPsec SA distribution has been considered
   as part of SDN-based IPSEC flow consideration (see [RFC8192]).  The
   user data traffic runs over secure IP tunnels whether the
   configuration is via NETCONF or BGP RR mesh.  Figure 3 below shows a
   Route Reflector topology with BGP sessions in a RR mesh and client
   traffic over IPSEC tunnels.

   Figure 3 - pending [Editor's note: Large scale topology needs svg
   drawings]

   Figure 4 shows an equivalent topology can be used with NETCONF
   client-servers.  Notice that the NETCONF topology requires a common
   database behind the network clients to provide the correct



Hares                   Expires January 15, 2021                [Page 9]


Internet-Draft             BGP-IPSEC-ANALYSIS                  July 2020


   configurations.  If the NETCONF servers work across administrative
   domains, a shared database must be developed to provide the
   appropriate information given the correct policy filters and access
   (NETCONF NACM).

   Diagram 4 - pending [Editor's note: Large scale BGP topologies need
   .svg ]

   There are two parts of the security for control plane traffic: link
   security and data security.  Link security entails making sure the
   data is secure as the data is transmitted across the link.

   Link security in the NETCONF configuration cloud shown in diagram 2
   entails making sure the configuration data passes across each of the
   links.  The links from the configuration database (DB in diagram) to
   each client server must be secured via TLS.  The links from the
   netconf client to the netconf server on the node must be secured via
   TLS.  Data security in the NETCONF configuration cloud entails making
   sure the data from the configuration data base (DB) travels through
   the netconf clients (e.g. netconf client1) to the node's netconf
   server (e.g.  NS1) without change.  Data privacy for configuration
   pathways traffic entails making sure no other party snoops on the
   data while it travels from configuration database (DB) to the netconf
   client to the server.

   NETCONF client/servers are designed to operate in a single
   administrative domain.  NETCONF client/servers require additional
   policy filters and checks to run between multiple administrative
   domains.  The Database to NETCONF client link is not standardized by
   IETF.

   Correspondingly, the link security in a BGP RR mesh requires that the
   data is secure across any link in the BGP RR mesh (RR to/from any RR
   client or within the RR mesh).  Data security for control plane
   traffic entails ensuring that the data placed into the BGP mesh (from
   RR clients or RR) arrives at the appropriate destination without
   change.  BGP does not provide data security on control plane traffic
   as the data may be modified via policy at each node.  SBGP does
   provide data security.

   For most networks, physical security of each node and link security
   is considered sufficient.

   The data written into a node using configuration data writes (NETCONF
   edit-config or RESTCONF PUT/POST) uses the NETCONF client to write to
   the network server on the client router.  The data which is sent from
   the route-reflector to the RR client routers is sent via BGP, saved
   in the BGP RIBs, and installed in the router.



Hares                   Expires January 15, 2021               [Page 10]


Internet-Draft             BGP-IPSEC-ANALYSIS                  July 2020


   The network management protocols (NETCONF/RESTCONF) and BGP both have
   access policy that controls the data is written into the router.  The
   error handling for incorrect data is different between these two
   network management protocols (NETCONF/RESTCONF) and BGP.  If netconf
   tries save data with the wrong format, it will provide an error
   information in the response (rpc-error).  The BGP error handling of a
   malformed Tunnel Attribute in the TLV simply ignores the tunnel
   attribute while accepting the route.

   The common resolution is that NETCONF, RESTCONF, and BGP write error
   information to a local log.  Error reports can be tracked in a Yang
   module which can be automatically streamed to central controller via
   an alternate channel NETCONF/RESTCONF logging channel.

   [Editorial note: Should I give the details of the NETCONF/RESTCONF
   logging channel?]

   The SDWAN environment or any VPN that uses BGP to transfer tunnel
   configuration and security association information SHOULD consider
   augmenting the base BGP Yang model with BGP tunnel encapsulation Yang
   model for all tunnel types including IPSEC.  The logging features or
   the reporting of the BGP errors can be combined with any error
   reporting on NETCONF/RESTCONF configuration or any operational state
   from the tunnel interface.  The NETCONF/RESTCONF logging feature
   providing throttling so any type of error reporting can be configured
   to be manageable within a large network.

   This implies the SDWAN environment should design a BGP Yang model
   augmenting the BGP base model for the BGP tunnel encapsulation
   functionality for all tunnel types including IPSEC AND provides
   logging features the reporting can be the same as NETCONF/RESTCONF.

   Given Ben's rule of thumb, the transmission of the routes, the tunnel
   end points, and the link to the security association information via
   the BGP protocol does not cause extra security risks.

   The next 3 sections summarize the security and privacy of each
   technology in terms of:

   o  what is distributed via netconf,

   o  what is distributed via BGP,

   o  link security provided,

   o  data security provided,

   o  suggested Yang models that will augment error handling,



Hares                   Expires January 15, 2021               [Page 11]


Internet-Draft             BGP-IPSEC-ANALYSIS                  July 2020


   o  privacy issues.

2.1.  Option 1 - Configuration Plus BGP Routes with Tunnel SA IDs

   Document: draft-hujun-idr-bgp-ipsec-02.txt [I-D.hujun-idr-bgp-ipsec]

   What is distributed via NETCONF: Tunnel Configuration, Security
   associations, and the mapping of the security association to a tunnel
   end point (identified y IPsec tunnel identifier), and SA (security
   association) for the each IPSec tunnel.

   What is distributed via BGP: Client Routes with IPsec Sub-TLV per
   tunnel attribute with IP-SEC tunnel.  Optionally, the Public Instance
   Sub-TLV may augment the BGP tunnel attributes Sub-TLV for tunnel
   endpoint.

   Sub-TLVs added:

      IPsec SUB-TLV in IP-Sec Tunnel Attribute (proposed): 4 octet
      opaque tag.

      Public Instance Sub-TLV: identifies the remote instance the Sub-
      TLV for Tunnel End-Point Identifier takes its address from.

   NETCONF Link Security: Distribution is secured by Client-server TLS

   Configuration Data security: Configuration clients SHOULD have host
   and data security.  This is beyond NETCONF/RESTCONF security.  Client
   synchronization of data with other clients must have security links
   and security mechanisms.

   Suggested Yang Models for Configuration and Reporting

   o  Tunnels configuration and operational state

   o  SA configuration and operational state,

   o  BGP Tunnel Attribute Yang model augmenting base BGP model with
      tunnel attributes data and error log.  (Tunnel attribute
      information includes the tunnel attributes plus the mapping of
      routes to tuples of [tunnel endpoint, security association, and
      encryption].)

   Privacy: Link privacy assumes the ability to encrypt the link data to
   provide privacy.  Node Privacy requires software secure containers
   within the NETCONF/RESTCONF clients/servers and BGP modules for each
   of these models.




Hares                   Expires January 15, 2021               [Page 12]


Internet-Draft             BGP-IPSEC-ANALYSIS                  July 2020


2.2.  Option 2- BGP passes client routes with SA-ID plus NLRI passes
      underlay SAs

   Document: [I-D.dunbar-idr-sdwan-edge-discovery]

   What is distributed via NETCONF/RESTCONF or locally configured:
   Policy and/or templates so that automation may use NLRI with SDWAN
   SAFI to configure tunnels.  This policy may be expressed in as little
   as 1 line of local configuration.

   What is distributed via BGP: Client Routes with tunnel attribute with
   IPsec Tunnel type, IPSec SA ID(s) which reference Security
   Association attributes being advertised by SDWAN-Underlay UPDATE.
   The Sub-TLVs added include:

      IPSec-SA-ID SUB-TLV (proposed): 2 octet pointer into SA table

      IPsec-SA-Group SUB-TLV: list of pointers into SA table grouped by
      inner encapsulation type

   The Underlay property is NLRI attached to port addresses or node
   address with SDWAN SAFI: Includes Site Type, IPSEC-SA-Type, Port-
   Local-ID, SDWAN-Site-ID, SDWAN Node ID.  Depending on the SITE-Type
   and IPSec-SA-type, this SAFI carries either template references for
   pre-configured security association (SAs) or full SA information.

      Note: Since the Security association information is carried in a
      different AFI/SAFI pair, this information may be transmitted in a
      different BGP update than the client routes with the Tunnel
      attribute.

   Link Security: Distribution is secured between RR to RR clients and
   between RR in the RR mesh is secured with Transport layer security.
   If the RR mesh with underlay information is compromised, it does not
   mean the route with tunnel attributes will be compromised.

   Data Security: Data distribution security of tunnel endpoints, SA
   (security association), routes and mapping (tunnel endpoint, SA,
   routes) SHOULD have RR and RR Client security on modules processing
   the data.  Full data security (with certificates that the data
   originated is what arrives at the final destination) for the BGP
   routes and attributes is beyond the normal mechanism of BGP.  These
   features may be available in SBGP.  SBGP signature processing is
   computationally expensive and requires additional memory space.
   Synchronization of the routing information on RR (routes, tunnel
   endpoints, SA-links) and underlay security association information
   (from AFI/SAFI SDWAN) may be impacted policy that distributes the
   data.



Hares                   Expires January 15, 2021               [Page 13]


Internet-Draft             BGP-IPSEC-ANALYSIS                  July 2020


      Technical Note: Many ISPs have chosen to only validate the route
      origin attribute of the BGP route to insure reduction of "human
      errors" and some classes of attacks.

   Suggested Yang Models for Reporting Errors

   o  Tunnels configuration and operational state of tunnels,

   o  SA configuration and operational state of SA information,

   o  BGP Tunnel Attribute Yang model augmenting base BGP model with
      tunnel attributes data and error log.  (Tunnel attribute
      information includes the tunnel attributes plus the mapping of
      routes to tuples of [tunnel endpoint, security association, and
      encryption].

   o  Augmentation to base BGP Model to display information passed in
      NLRI with SDWAN SAFI

   Privacy: (Same as option 1)

   o  Link privacy assumes the ability to encrypt the link data to
      provide privacy.

   o  Node Privacy requires secure containers within the netconf/
      restconf clients/servers and BGP modules for the BGP control plane
      data.

2.3.  Option 3: Secure EVPN (client routes + SA information)

   Document: draft-ietf-sajassi-bess-secure-evpn
   [I-D.sajassi-bess-secure-evpn]

   What is distributed: Client routes with tunnel attribute with ESP-
   Transport and ESP-in-UDP-Transport tunnel types.

      SUB-TLVs in ESP-Transport and ESP-in-UDP-Transport Attribute
      (proposed): BASE DIM, Key-Exchange, ESP SA, and Transform Sub-
      Structure.

      This solution would require the Tunnel TLV for the IPsec to
      contain: Tunnel Endpoint TLV and the DIM TLV.

      The DIM SUB-TLV has the following fields:

      *  ID-length

      *  Nonce length,



Hares                   Expires January 15, 2021               [Page 14]


Internet-Draft             BGP-IPSEC-ANALYSIS                  July 2020


      *  I-flag

      *  Flags

      *  Re-key counter

      *  Originator ID + (Tenant ID) + (Subnet ID) + (Tenant Address

      *  Nonce data

      Technical Note: The data rate for retransmitting the client routes
      with the DIM Sub-TLV must be done at the rekeying rate.  This
      automatic re-key counter is distributed with the data.

   Link Security: Distribution is secured between RR to RR clients and
   the RR mesh is secured with transport link security.  The regular
   data distribution of the SA nonce and the rekeying counter provides a
   potential attack vector for man-in-the middle attacks if the link
   security is compromised.

   Data Security: Data distribution security of tunnel endpoints, SA
   (security association), routes and mapping (tunnel endpoint, SA,
   routes) SHOULD have RR and RR Client security on modules processing
   the data.  In addition the processes handling SA information
   [I-D.carrel-ipsecme-controller-ike] should exist in a protected
   process.

   Full data security for the BGP routes and attributes is beyond the
   normal mechanism of BGP, but may be available in SBGP.  SBGP
   signature processing is computationally expensive and requires
   additional memory space.  Synchronization of the routing information
   on RR (routes, tunnel endpoints, SA-links) and underlay security
   association information (from AFI/SAFI SDWAN) may be impacted policy
   that distributes the data.

   Yang Models for Reporting Errors

   o  Tunnels configuration and operational state,

   o  configuration and operational state,

   o  BGP Tunnel Attribute Yang model augmenting base BGP model with
      tunnel attributes data and error log.  (Tunnel attribute
      information includes the tunnel attributes plus the mapping of
      routes to tuples of [tunnel endpoint, security association, and
      encryption].)





Hares                   Expires January 15, 2021               [Page 15]


Internet-Draft             BGP-IPSEC-ANALYSIS                  July 2020


   o  Yang models for the operational state in
      [I-D.carrel-ipsecme-controller-ike].

   Privacy: Same as option 1 and 2.

2.4.  comparison of security issues

   The security of each of these solutions utilizes similar distribution
   and error reporting.  Man in the Middle attacks based on snooping,
   would need to break the TLS security and encryption for privacy.  The
   [I-D.sajassi-bess-secure-evpn] provides more data directly linked to
   the routes which could allow an attack vector.

   [I-D.hujun-idr-bgp-ipsec] and [I-D.dunbar-idr-sdwan-edge-discovery]
   provide the route, tunnel information, and link to the SA
   information.  This indirect access to SA information could lessen the
   attack vector for the tunnel.

   [I-D.dunbar-idr-sdwan-edge-discovery] and
   [I-D.sajassi-bess-secure-evpn] have options send the SA information
   on unique tunnel types.  [I-D.dunbar-idr-sdwan-edge-discovery]
   placement of the SA data in a NLRI can allow a separate encryption
   between the SA data and the route/tunnel information.

   While all 3 solutions can be used with automated tools (SDN based on
   simply configuration based), the each solution has benefits and
   deficits.

3.  Manageability and Scaling

   Manageability involves how much manual effort is involved to set up
   IPSec tunnels using each of the three options.  The manageability
   must handle the following: initial set-up of nodes, reporting of
   status or errors, and rekeying efforts.  BGP data distribution and
   processing of routes to set-up forwarding is stressed during: initial
   start-up, crash of a RR, and start-up of RR.

   The scaling of the system should handle the data distribution and the
   manageability should handle both the network scenario 1 and network
   scenario 2.  Scenario 1 and scenario 2 both consider one security
   association per tunnel and 10 security associations per 10.  This
   comparison is given to help understand the impact of rekeying the
   security associations.  [I-D.hujun-idr-bgp-ipsec] would need to send
   rekeying via NETCONF/RESTCONF, but the rekeying that causes a tunnel
   to switch security associations can be sent via BGP.
   [I-D.hujun-idr-bgp-ipsec] use of the NETCONF/RETCONF to send a
   configuration becomes a bottleneck if the network sizes reach
   scenario 2.  [I-D.dunbar-idr-sdwan-edge-discovery] uses two parallel



Hares                   Expires January 15, 2021               [Page 16]


Internet-Draft             BGP-IPSEC-ANALYSIS                  July 2020


   BGP NLRI processes where one passes routes and security association
   identifiers, and the second process sends rekeying based on topology
   information.  Rekeying information is transmitted prior to BGP passes
   the rekeying of the tunnel.  [I-D.sajassi-bess-secure-evpn] passes
   the rekeying information with the client routes.  During initial
   start-up or RR crash, this rekeying data substantially increases the
   memory footprint.  A continual rekeying process in
   [I-D.sajassi-bess-secure-evpn] could also cause periodic BGP updates
   to continue to use bandwidth in the network.

   One alternative to the periodic rekeying is to allow the association
   of more than 1 security association (SA) per tunnel, and allow a
   local mechanism to switch security associations are a particular
   time.  This analysis looks at the scaling issues of 10 SAs per tunnel
   allows this analysis to look at the scaling in terms of memory
   required for this mechanism of rekeying.

   The estimate of 10 security associations is admittedly imperfect, but
   it may help to start the discussion on the memory usage during
   rekeying.

   NETCONF/RESTCONF data distribution scales when the client to netconf/
   restconf server ratio is low.  1 client per server is best, but 5
   servers per client is a low level.  1 client configuring 10K servers
   on network nodes is beyond most NETCONF/RESTCONF servers.  Pushing
   multiple types of data may also cause stress on the client ability to
   pull data from the back-end configuration database.

   The difference between NETCONF/RESTCONF and BGP mechanisms matter in
   for SDWAN deployments.  NETCONF/RESTCONF is optimized for a single
   administrative domain and BGP is optimized for inter-domain policy.
   In SDWAN the nodes are distributed across multiple administrative
   domains.  BGP implementations have many levels of policy.  Using BGP
   each node can be under different RR.  Each node can have default SA
   attributes such as supported encryption algorithms, the nonce, and
   the public key.  The SA ID is only locally significant to the node
   (or to the port), which is less prone to misconfiguration.  BGP also
   has policy at the route level.  Using BGP built-in RT constraint
   distribution, BGP implementations distribute the SA information to
   the nodes specified as authorized peers.

3.1.  Configuration sizes - used for theoretical comparison

   To provide a simple estimate, it is assumed that 100 items needed to
   be configured in the Yang modules prior to starting the IPsec VPN.

   o  BGP peer items per node: 20




Hares                   Expires January 15, 2021               [Page 17]


Internet-Draft             BGP-IPSEC-ANALYSIS                  July 2020


   o  Tunnel configuration items node: 20

   o  SA Configuration items per node: 40

   o  Monitoring configuration per node: 20

3.2.  BGP Route sizes for theoretical comparison

   The following estimates are for route and the tunnel Attribute are
   used for this comparison:

   o  average of 4 bytes for IPv4 prefix

   o  average of 8 bytes for IPv6 prefix.

3.2.1.  Size of Tunnel encapsulation attribute with 1 SA per tunnel
        endpoint

   The space required in the BGP packet for the tunnel attribute per 1
   tunnel with 1 Security association (SA) for each of the options is as
   follows:

   o  Tunnel TLV header [4 bytes]

   o  Sub-TLV for tunnel endpoint for IPv4 [12 bytes]

   o  IP-Sec Sub-TLVs (required) with 1 SA per tunnel endpoint:

      *  Option 1 [I-D.hujun-idr-bgp-ipsec]: 6 octets

      *  Option 2 [I-D.dunbar-idr-sdwan-edge-discovery] 4 octets

      *  Option 3 [I-D.sajassi-bess-secure-evpn] : 35 octets

         +  Sub-TLV header: 3 octets

         +  Dim: 32 octets (header (4), rekey (8), address (8), Nonce
            (12))

   The total space in the tunnel attribute for each type per tunnel
   endpoint with one security association is the following:

      Option 1 [I-D.hujun-idr-bgp-ipsec]: 22 octets

      Option 2 [I-D.dunbar-idr-sdwan-edge-discovery]: 20 octets

      Option 3 [I-D.sajassi-bess-secure-evpn] : 52 octets




Hares                   Expires January 15, 2021               [Page 18]


Internet-Draft             BGP-IPSEC-ANALYSIS                  July 2020


   Encapsulation mechanisms such as GRE and VXLAN may add 6-16 octets
   per tunnel to the Tunnel attribute per tunnel.  This addition is due
   to adding encodings for inner mechanisms (4-12), and outer encodings
   (2-4)

   The total space with encapsulations would then be:

      Option 1 [I-D.hujun-idr-bgp-ipsec]: 28-38 octets

      Option 2: [I-D.dunbar-idr-sdwan-edge-discovery]:: 26-36 octets

      Option 3 [I-D.sajassi-bess-secure-evpn] : 56-68 octets

3.2.2.  Size of Tunnel encapsulation attribute with 10 SAs per tunnel

   This will be completed in version -01

3.3.  Network Scenario 1

   This will be completed in version -01

3.4.  Network scenario 2

   This will be completed in version -01.txt

3.5.  Scaling Memory sizes

   This section includes scaling for network scenario 1 and 2.

4.  Key differences between the options

   (to be completed in version -01)

5.  Processing of BGP routes

   (to be completed in version -01)

6.  Future Issues - SBGP and Secure IPSEC VPNs

   (to be completed in version -01)

7.  Security Considerations

   This draft is analysis that includes security and privacy.  The draft
   does not cause any further security issues, but hopes to enhance the
   security considerations in other drafts.





Hares                   Expires January 15, 2021               [Page 19]


Internet-Draft             BGP-IPSEC-ANALYSIS                  July 2020


8.  IANA considerations

   This draft does not make any requests to IANA for allocations.  It is
   an analysis for review of future allocations in the BGP registry.

9.  References

9.1.  Normative References

   [I-D.carrel-ipsecme-controller-ike]
              Carrel, D. and B. Weis, "IPsec Key Exchange using a
              Controller", draft-carrel-ipsecme-controller-ike-01 (work
              in progress), March 2019.

   [I-D.dunbar-idr-sdwan-edge-discovery]
              Dunbar, L., Hares, S., Raszuk, R., and K. Majumdar, "BGP
              UPDATE for SDWAN Edge Discovery", draft-dunbar-idr-sdwan-
              edge-discovery-00 (work in progress), July 2020.

   [I-D.hujun-idr-bgp-ipsec]
              Hu, J., "BGP Provisioned IPsec Tunnel Configuration",
              draft-hujun-idr-bgp-ipsec-02 (work in progress), March
              2020.

   [I-D.ietf-idr-tunnel-encaps]
              Patel, K., Velde, G., Ramachandra, S., and J. Scudder,
              "The BGP Tunnel Encapsulation Attribute", draft-ietf-idr-
              tunnel-encaps-16 (work in progress), July 2020.

   [I-D.sajassi-bess-secure-evpn]
              Sajassi, A., Banerjee, A., Thoria, S., Carrel, D., Weis,
              B., and J. Drake, "Secure EVPN", draft-sajassi-bess-
              secure-evpn-03 (work in progress), July 2020.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

9.2.  Informative References

   [I-D.dunbar-bess-bgp-sdwan-usage]
              Dunbar, L., Guichard, J., Sajassi, A., Drake, J., Najem,
              B., and D. Carrel, "BGP Usage for SDWAN Overlay Networks",
              draft-dunbar-bess-bgp-sdwan-usage-08 (work in progress),
              July 2020.





Hares                   Expires January 15, 2021               [Page 20]


Internet-Draft             BGP-IPSEC-ANALYSIS                  July 2020


   [I-D.ietf-rtgwg-net2cloud-gap-analysis]
              Dunbar, L., Malis, A., and C. Jacquenet, "Networks
              Connecting to Hybrid Cloud DCs: Gap Analysis", draft-ietf-
              rtgwg-net2cloud-gap-analysis-06 (work in progress), May
              2020.

   [I-D.ietf-rtgwg-net2cloud-problem-statement]
              Dunbar, L., Jacquenet, C., and M. Toy, "Dynamic Networks
              to Hybrid Cloud DCs Problem Statement", draft-ietf-rtgwg-
              net2cloud-problem-statement-10 (work in progress), May
              2020.

   [RFC5512]  Mohapatra, P. and E. Rosen, "The BGP Encapsulation
              Subsequent Address Family Identifier (SAFI) and the BGP
              Tunnel Encapsulation Attribute", RFC 5512,
              DOI 10.17487/RFC5512, April 2009,
              <https://www.rfc-editor.org/info/rfc5512>.

   [RFC5566]  Berger, L., White, R., and E. Rosen, "BGP IPsec Tunnel
              Encapsulation Attribute", RFC 5566, DOI 10.17487/RFC5566,
              June 2009, <https://www.rfc-editor.org/info/rfc5566>.

   [RFC8192]  Hares, S., Lopez, D., Zarny, M., Jacquenet, C., Kumar, R.,
              and J. Jeong, "Interface to Network Security Functions
              (I2NSF): Problem Statement and Use Cases", RFC 8192,
              DOI 10.17487/RFC8192, July 2017,
              <https://www.rfc-editor.org/info/rfc8192>.

   [RFC8365]  Sajassi, A., Ed., Drake, J., Ed., Bitar, N., Shekhar, R.,
              Uttaro, J., and W. Henderickx, "A Network Virtualization
              Overlay Solution Using Ethernet VPN (EVPN)", RFC 8365,
              DOI 10.17487/RFC8365, March 2018,
              <https://www.rfc-editor.org/info/rfc8365>.

Author's Address

   Susan Hares
   Hickory Hill Consulting
   US

   Email: shares@ndzh.com










Hares                   Expires January 15, 2021               [Page 21]


Html markup produced by rfcmarkup 1.129d, available from https://tools.ietf.org/tools/rfcmarkup/