--- 1/draft-ietf-tsvwg-quickstart-02.txt 2006-04-25 01:12:29.000000000 +0200 +++ 2/draft-ietf-tsvwg-quickstart-03.txt 2006-04-25 01:12:30.000000000 +0200 @@ -1,18 +1,18 @@ Internet Engineering Task Force S. Floyd INTERNET-DRAFT M. Allman -draft-ietf-tsvwg-quickstart-02.txt ICIR -Expires: September 2006 A. Jain +draft-ietf-tsvwg-quickstart-03.txt ICIR +Expires: October 2006 A. Jain F5 Networks P. Sarolahti Nokia Research Center - 5 March 2006 + 24 April 2006 Quick-Start for TCP and IP Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. @@ -25,48 +25,122 @@ months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. - This Internet-Draft will expire on September 2006. + This Internet-Draft will expire on October 2006. Abstract This document specifies an optional Quick-Start mechanism for transport protocols, in cooperation with routers, to determine an allowed sending rate at the start and at times in the middle of a data transfer (e.g., after an idle period). While Quick-Start is designed to be used by a range of transport protocols, in this - document we describe its use with TCP. By using Quick-Start, a TCP - host, say, host A, would indicate its desired sending rate in bytes - per second, using a Quick Start option in the IP header of a TCP - packet. Each router along the path could, in turn, either approve - the requested rate, reduce the requested rate, or indicate that the - Quick-Start request is not approved. If the Quick-Start request is - not approved, then the sender would use the default congestion - control mechanisms. The Quick-Start mechanism can determine if - there are routers along the path that do not understand the Quick- - Start option, or have not agreed to the Quick-Start rate request. - TCP host B communicates the final rate request to TCP host A in a - transport-level Quick-Start Response in an answering TCP packet. - Quick-Start is designed to allow connections to use higher sending - rates when there is significant unused bandwidth along the path, and - all of the routers along the path support the Quick-Start Request. + document we only specify its use with TCP. Quick-Start is designed + to allow connections to use higher sending rates when there is + significant unused bandwidth along the path and the sender and all + of the routers along the path approve the Quick-Start Request. + + As this document describes, environments where Quick-Start requests + would not be approved include paths with routers, IP tunnels, MPLS + paths, and the like that do not support Quick-Start, or paths with + routers or middleboxes that drop packets containing IP options. + Quick-Start requests could be difficult to approve over paths that + include multi-access layer-two networks. This document also + describes environments where the Quick-Start process could fail with + false positives, with the sender incorrectly assuming that the + Quick-Start request had been approved by all of the routers along + the path. As a result of these concerns, and as a result of the + difficulties and seeming absence of motivation for routers such as + core routers to deploy Quick-Start, Quick-Start is being proposed as + a mechanism that could be of use in controlled environments, and not + as a mechanism that would be intended or appropriate for ubiquitous + deployment in the global Internet. TO BE DELETED BY THE RFC EDITOR UPON PUBLICATION: + Changes from draft-ietf-tsvwg-quickstart-02: + + * Some general editing. + + * Said that if the receiver receives a Quick-Start Request + with a rate of zero, then the receiver SHOULD NOT send + a Quick-Start response. And that if the sender + receives an acknowledgement of its packet with no + Quick-Start response, then the sender MUST assume that the + request was denied, and send a Report of Approved Rate + with a rate of zero. + + * Said that if a Quick-Start packet is dropped or marked, + the sender should not make more Quick-Start requests in this + connection. + + * Said that the Quick-Start Request SHOULD be sent on a packet + that requires an acknowledgement, e.g., a SYN, SYN/ACK, or data + packet. + + * Made changes to the section on "TCP: A Quick-Start Request in the + Middle of a Connection". + + * Added that if the TCP host is going to use the successful + Quick-Start Request, it MUST start using it within one + round-trip time of receiving the Quick-Start Response, + or within three seconds, whichever is smaller. + + * Added a stronger applicability statement, in the abstract + and in Section 10 on "Implementation and Deployment Issues". + From feedback from the working group. + + * Added a section about MPLS. From feedback from Mitchell + Erblichs. + + * Strengthened the language of the difficulties presented by + multi-access links. + + * Added a discussion in Section 10.3 about the deployment of + Quick-Start on single-hop paths. From feedback from + Mitchell Erblichs. + + * Clarified that the "router" function of approving + Quick-Start requests includes the IP-layer processing + at the sender. + + * Clarified in Section 3.3 on "Processing the Quick-Start + Request at Routers" that this document standardizes only + the semantics of Quick-Start, and not the specific + algorithms for processing Quick-Start requests at routers. + + * Clarified in Section 3.3 on "Processing the Quick-Start + Request at Routers" that a router will have to consider + the previous Quick-Start requests in approving a new one. + + * In Section 9.3 on "Quick-Start with QoS-enabled Traffic", + which says that routers are free to take into account + the diff-serv codepoint in considering QS requests, clarified + that routers are also free to take into account their own + understanding of priorities. + + * Added the Temporary bit to Appendix D on "Possible Additional + Uses for the Quick-Start Option". Clarified that the Quick-Start + mechanism is not designed to help routers achieve full link + utilization. + + * Editing from feedback from Gorry Fairhurst. + Changes from draft-ietf-tsvwg-quickstart-01: + * Added a citation to SPAND: Speeding Up Short Data Transfers. * Added a sentence in Section 8.1 on "Implementation Issues for Processing Quick-Start Requests" about multi-access links. * Mentioned the IP Router Alert option, RFC 2113, in Appendix A.1.1. * Added a discussion of lower-than-best-effort service. * Added a few sentences about the requirements for randomness in the nonce. * Changed the name of the option from the Quick-Start Request Option to the Quick-Start Option. @@ -71,26 +145,27 @@ * Changed the name of the option from the Quick-Start Request Option to the Quick-Start Option. * Changed the semantics of the Reserved field to the Function field, adding that a Quick-Start option is only interpreted as a request if this field is zero. * Changed the "Reporting Approved Rate" option from a "Possible Use" in Appendix D to a required use in Section 3.1, to allow routers and receivers some protection against misbehaving senders. * Changes from feedback from Bob Briscoe: - - Added Appendix E with a response to Sections 1-3 of + - Added Appendix E about Sections 1-3 of Bob Briscoe's document. - Added a clarification that the approval of a Quick-Start request at a router does not affect the treatment of the subsequent arriving Quick-Start data packets. + - Added the one-way hash function as an alternate implementation for the QS Nonce. - Clarified the phrase "incrementally deployable", adding the following: "We note that while Quick-Start is incrementally deployable in this sense, a Quick-Start request can not be approved for a particular connection unless both end-nodes and all of the routers along the path have been configured to support Quick-Start." - Clarified semantics about additional rate. @@ -194,144 +269,143 @@ * Added a discussion in the related work section about the possibility of optimistically sending a large initial window, without explicit permission of routers. * Added a discussion in the related work section about the tradeoffs of XCP vs. Quick-Start. * Added a section on "The Quick-Start Request: Packets or Bytes?" Changes from draft-amit-quick-start-00.txt: * The addition of a citation to [KHR02]. * The addition of a Related Work section. - * Deleted the QS Nonce, in favor of a random initial value for the QS TTL. Table of Contents - 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . 10 - 2. Assumptions and General Principles. . . . . . . . . . . . . . 11 + 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . 11 + 1.1. Conventions and Terminology. . . . . . . . . . . . . . . 12 + 2. Assumptions and General Principles. . . . . . . . . . . . . . 12 2.1. Overview of Quick-Start. . . . . . . . . . . . . . . . . 13 - 3. The Quick-Start Option in IP. . . . . . . . . . . . . . . . . 15 - 3.1. The Quick-Start Option for IPv4. . . . . . . . . . . . . 15 - 3.2. The Quick-Start Option for IPv6. . . . . . . . . . . . . 18 + 3. The Quick-Start Option in IP. . . . . . . . . . . . . . . . . 16 + 3.1. The Quick-Start Option for IPv4. . . . . . . . . . . . . 16 + 3.2. The Quick-Start Option for IPv6. . . . . . . . . . . . . 20 3.3. Processing the Quick-Start Request at - Routers . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 + Routers . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.3.1. Processing the Report of Approved - Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 - 3.4. The QS Nonce . . . . . . . . . . . . . . . . . . . . . . 21 - 4. The Quick-Start Mechanisms in TCP . . . . . . . . . . . . . . 23 - 4.1. When to Use Quick-Start. . . . . . . . . . . . . . . . . 24 + Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 + 3.4. The QS Nonce . . . . . . . . . . . . . . . . . . . . . . 23 + 4. The Quick-Start Mechanisms in TCP . . . . . . . . . . . . . . 25 + 4.1. Sending the Quick-Start Request. . . . . . . . . . . . . 26 4.2. The Quick-Start Response Option in the TCP - header. . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 - 4.3. TCP: Sending the Quick-Start Response. . . . . . . . . . 27 + header. . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 + 4.3. TCP: Sending the Quick-Start Response. . . . . . . . . . 28 4.4. TCP: Receiving and Using the Quick-Start - Response Packet . . . . . . . . . . . . . . . . . . . . . . . 27 + Response Packet . . . . . . . . . . . . . . . . . . . . . . . 29 4.5. TCP: Responding to a Loss of a Quick-Start - Packet. . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 + Packet. . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 4.6. TCP: A Quick-Start Request for a Larger Ini- - tial Window . . . . . . . . . . . . . . . . . . . . . . . . . 30 - 4.6.1. Interactions with Path MTU Discovery. . . . . . . . 30 + tial Window . . . . . . . . . . . . . . . . . . . . . . . . . 32 + 4.6.1. Interactions with Path MTU Discovery. . . . . . . . 32 4.6.2. Quick-Start Request Packets that are - Discarded by Middleboxes . . . . . . . . . . . . . . . . . 31 + Discarded by Routers or Middleboxes. . . . . . . . . . . . 33 4.7. TCP: A Quick-Start Request in the Middle of - a Connection. . . . . . . . . . . . . . . . . . . . . . . . . 32 - 4.8. An Example Quick-Start Scenario with TCP . . . . . . . . 33 - 5. Quick-Start and IPsec AH. . . . . . . . . . . . . . . . . . . 34 - 6. Quick-Start in IP Tunnels . . . . . . . . . . . . . . . . . . 34 + a Connection. . . . . . . . . . . . . . . . . . . . . . . . . 34 + 4.8. An Example Quick-Start Scenario with TCP . . . . . . . . 35 + 5. Quick-Start and IPsec AH. . . . . . . . . . . . . . . . . . . 35 + 6. Quick-Start in IP Tunnels and MPLS. . . . . . . . . . . . . . 36 6.1. Simple Tunnels That Are Compatible with - Quick-Start . . . . . . . . . . . . . . . . . . . . . . . . . 36 + Quick-Start . . . . . . . . . . . . . . . . . . . . . . . . . 38 6.1.1. Simple Tunnels that are Aware of Quick- - Start. . . . . . . . . . . . . . . . . . . . . . . . . . . 37 + Start. . . . . . . . . . . . . . . . . . . . . . . . . . . 39 6.2. Simple Tunnels That Are Not Compatible with - Quick-Start . . . . . . . . . . . . . . . . . . . . . . . . . 37 - 6.3. Tunnels That Support Quick-Start . . . . . . . . . . . . 38 + Quick-Start . . . . . . . . . . . . . . . . . . . . . . . . . 39 + 6.3. Tunnels That Support Quick-Start . . . . . . . . . . . . 40 + 6.4. Quick-Start and MPLS . . . . . . . . . . . . . . . . . . 41 7. The Quick-Start Mechanism in other Transport Pro- - tocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 - 8. Using Quick-Start . . . . . . . . . . . . . . . . . . . . . . 40 - 8.1. Determining the Rate to Request. . . . . . . . . . . . . 40 + tocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 + 8. Using Quick-Start . . . . . . . . . . . . . . . . . . . . . . 42 + 8.1. Determining the Rate to Request. . . . . . . . . . . . . 42 8.2. Deciding the Permitted Rate Request at a - Router. . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 - 9. Evaluation of Quick-Start . . . . . . . . . . . . . . . . . . 42 - 9.1. Benefits of Quick-Start. . . . . . . . . . . . . . . . . 42 - 9.2. Costs of Quick-Start . . . . . . . . . . . . . . . . . . 42 - 9.3. Quick-Start with QoS-enabled Traffic . . . . . . . . . . 44 - 9.4. Protection against Misbehaving Nodes . . . . . . . . . . 44 - 9.4.1. Misbehaving Senders . . . . . . . . . . . . . . . . 44 + Router. . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 + 9. Evaluation of Quick-Start . . . . . . . . . . . . . . . . . . 44 + 9.1. Benefits of Quick-Start. . . . . . . . . . . . . . . . . 44 + 9.2. Costs of Quick-Start . . . . . . . . . . . . . . . . . . 44 + 9.3. Quick-Start with QoS-enabled Traffic . . . . . . . . . . 46 + 9.4. Protection against Misbehaving Nodes . . . . . . . . . . 47 + 9.4.1. Misbehaving Senders . . . . . . . . . . . . . . . . 47 9.4.2. Receivers Lying about Whether the - Request was Approved . . . . . . . . . . . . . . . . . . . 45 + Request was Approved . . . . . . . . . . . . . . . . . . . 48 9.4.3. Receivers Lying about the Approved - Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 - 9.4.4. Collusion between Misbehaving Routers . . . . . . . 47 - 9.5. Misbehaving Middleboxes and the IP TTL . . . . . . . . . 49 - 9.6. Attacks on Quick-Start . . . . . . . . . . . . . . . . . 49 - 9.7. Simulations with Quick-Start . . . . . . . . . . . . . . 50 - 10. Implementation and Deployment Issues . . . . . . . . . . . . 50 + Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 + 9.4.4. Collusion between Misbehaving Routers . . . . . . . 50 + 9.5. Misbehaving Middleboxes and the IP TTL . . . . . . . . . 51 + 9.6. Attacks on Quick-Start . . . . . . . . . . . . . . . . . 52 + 9.7. Simulations with Quick-Start . . . . . . . . . . . . . . 52 + 10. Implementation and Deployment Issues . . . . . . . . . . . . 53 10.1. Implementation Issues for Sending Quick- - Start Requests. . . . . . . . . . . . . . . . . . . . . . . . 50 + Start Requests. . . . . . . . . . . . . . . . . . . . . . . . 53 10.2. Implementation Issues for Processing Quick- - Start Requests. . . . . . . . . . . . . . . . . . . . . . . . 51 - 10.3. Possible Deployment Scenarios . . . . . . . . . . . . . 51 - 10.4. Would QuickStart packets take the slow path - in routers? . . . . . . . . . . . . . . . . . . . . . . . . . 52 - 10.5. A Comparison with the Deployment Problems - of ECN. . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 - 11. Related Work . . . . . . . . . . . . . . . . . . . . . . . . 53 - 11.1. Fast Start-ups without Explicit Information - from Routers. . . . . . . . . . . . . . . . . . . . . . . . . 53 - 11.2. Optimistic Sending without Explicit Infor- - mation from Routers . . . . . . . . . . . . . . . . . . . . . 55 - 11.3. Fast Start-ups with other Information from - Routers . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 - 11.4. Fast Start-ups with more Fine-Grained Feed- - back from Routers . . . . . . . . . . . . . . . . . . . . . . 56 - 11.5. Fast Start-ups with Lower-Than-Best-Effort - Service . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 - 12. Security Considerations. . . . . . . . . . . . . . . . . . . 58 - 13. Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 58 - 14. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 59 - A. Design Decisions. . . . . . . . . . . . . . . . . . . . . . . 59 + Start Requests. . . . . . . . . . . . . . . . . . . . . . . . 54 + 10.3. Possible Deployment Scenarios . . . . . . . . . . . . . 54 + 10.4. A Comparison with the Deployment Problems + of ECN. . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 + 11. Security Considerations. . . . . . . . . . . . . . . . . . . 56 + 12. IANA Considerations. . . . . . . . . . . . . . . . . . . . . 57 + 12.1. IP Option . . . . . . . . . . . . . . . . . . . . . . . 57 + 12.2. TCP Option. . . . . . . . . . . . . . . . . . . . . . . 58 + 13. Related Work . . . . . . . . . . . . . . . . . . . . . . . . 58 + 13.1. Fast Start-ups without Explicit Information + from Routers. . . . . . . . . . . . . . . . . . . . . . . . . 58 + 13.2. Optimistic Sending without Explicit Infor- + mation from Routers . . . . . . . . . . . . . . . . . . . . . 60 + 13.3. Fast Start-ups with other Information from + Routers . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 + 13.4. Fast Start-ups with more Fine-Grained Feed- + back from Routers . . . . . . . . . . . . . . . . . . . . . . 61 + 13.5. Fast Start-ups with Lower-Than-Best-Effort + Service . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 + 14. Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 63 + 15. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 63 + A. Design Decisions. . . . . . . . . . . . . . . . . . . . . . . 63 A.1. Alternate Mechanisms for the Quick-Start - Request: ICMP and RSVP. . . . . . . . . . . . . . . . . . . . 59 - A.1.1. ICMP. . . . . . . . . . . . . . . . . . . . . . . . 59 - A.1.2. RSVP. . . . . . . . . . . . . . . . . . . . . . . . 61 - A.2. Alternate Encoding Functions . . . . . . . . . . . . . . 62 - A.3. The Quick-Start Request: Packets or Bytes? . . . . . . . 63 + Request: ICMP and RSVP. . . . . . . . . . . . . . . . . . . . 63 + A.1.1. ICMP. . . . . . . . . . . . . . . . . . . . . . . . 64 + A.1.2. RSVP. . . . . . . . . . . . . . . . . . . . . . . . 65 + A.2. Alternate Encoding Functions . . . . . . . . . . . . . . 66 + A.3. The Quick-Start Request: Packets or Bytes? . . . . . . . 67 A.4. Quick-Start Semantics: Total Rate or Addi- - tional Rate?. . . . . . . . . . . . . . . . . . . . . . . . . 64 + tional Rate?. . . . . . . . . . . . . . . . . . . . . . . . . 68 A.5. Alternate Responses to the Loss of a Quick- - Start Packet. . . . . . . . . . . . . . . . . . . . . . . . . 65 - A.6. Why Not Include More Functionality?. . . . . . . . . . . 66 + Start Packet. . . . . . . . . . . . . . . . . . . . . . . . . 69 + A.6. Why Not Include More Functionality?. . . . . . . . . . . 70 A.7. Alternate Implementations for a QuickStart - Nonce . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 + Nonce . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 A.7.1. An Alternate Proposal for the Quick- - Start Nonce. . . . . . . . . . . . . . . . . . . . . . . . 69 + Start Nonce. . . . . . . . . . . . . . . . . . . . . . . . 73 A.7.2. The Earlier Request-Approved QuickStart - Nonce. . . . . . . . . . . . . . . . . . . . . . . . . . . 70 - B. Quick-Start with DCCP . . . . . . . . . . . . . . . . . . . . 71 - C. Possible Router Algorithm . . . . . . . . . . . . . . . . . . 73 + Nonce. . . . . . . . . . . . . . . . . . . . . . . . . . . 74 + B. Quick-Start with DCCP . . . . . . . . . . . . . . . . . . . . 75 + C. Possible Router Algorithm . . . . . . . . . . . . . . . . . . 76 D. Possible Additional Uses for the Quick-Start - Option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 - E. Feedback from Bob Briscoe . . . . . . . . . . . . . . . . . . 75 - E.1. Potential Deployment Scenarios . . . . . . . . . . . . . 76 - E.2. Misbehaving Senders and Receivers. . . . . . . . . . . . 76 - E.3. Fairness . . . . . . . . . . . . . . . . . . . . . . . . 77 - E.4. Models of Under-Utilization. . . . . . . . . . . . . . . 78 - E.5. Router Algorithms as Local Policy. . . . . . . . . . . . 78 - E.6. An Alternate Proposal. . . . . . . . . . . . . . . . . . 79 - Normative References . . . . . . . . . . . . . . . . . . . . . . 79 - Informative References . . . . . . . . . . . . . . . . . . . . . 80 - F. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 85 - F.1. IP Option. . . . . . . . . . . . . . . . . . . . . . . . 85 - F.2. TCP Option . . . . . . . . . . . . . . . . . . . . . . . 85 - AUTHORS' ADDRESSES . . . . . . . . . . . . . . . . . . . . . . . 85 - Full Copyright Statement . . . . . . . . . . . . . . . . . . . . 87 - Intellectual Property. . . . . . . . . . . . . . . . . . . . . . 87 + Option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 + E. Feedback from Bob Briscoe . . . . . . . . . . . . . . . . . . 79 + E.1. Potential Deployment Scenarios . . . . . . . . . . . . . 80 + E.2. Misbehaving Senders and Receivers. . . . . . . . . . . . 80 + E.3. Fairness . . . . . . . . . . . . . . . . . . . . . . . . 81 + E.4. Models of Under-Utilization. . . . . . . . . . . . . . . 82 + E.5. Router Algorithms as Local Policy. . . . . . . . . . . . 82 + E.6. An Alternate Proposal. . . . . . . . . . . . . . . . . . 83 + Normative References . . . . . . . . . . . . . . . . . . . . . . 83 + Informative References . . . . . . . . . . . . . . . . . . . . . 84 + AUTHORS' ADDRESSES . . . . . . . . . . . . . . . . . . . . . . . 89 + Full Copyright Statement . . . . . . . . . . . . . . . . . . . . 90 + Intellectual Property. . . . . . . . . . . . . . . . . . . . . . 90 1. Introduction Each connection begins with a question: "What is the appropriate sending rate for the current network path?" The question is not answered explicitly, but each TCP connection determines the sending rate by probing the network path and altering the congestion window (cwnd) based on perceived congestion. Each TCP connection starts with a pre-configured initial congestion window (ICW). Currently, TCP allows an initial window of between one and four MSS-sized @@ -355,130 +429,99 @@ A fair amount of work has already been done to address the issue of choosing the initial congestion window for TCP, with RFC 3390 allowing an initial window of up to four segments based on the MSS used by the connection [RFC3390]. Our underlying premise is that explicit feedback from all of the routers along the path would be required, in the current architecture, for best-effort connections to use initial windows significantly larger than those allowed by [RFC3390], in the absence of other information about the path. - The Congestion Manager [RFC3124] and TCP control block sharing - [RFC2140] both propose sharing congestion information among multiple - TCP connections with the same endpoints. With the Congestion - Manager, a new TCP connection could start with a high initial cwnd - if it was sharing the path and the cwnd with a pre-existing TCP - connection to the same destination that had already obtained a high - congestion window. RFC 2140 discusses ensemble sharing, where an - established connection's congestion window could be `divided up' to - be shared with a new connection to the same host. However, neither - of these approaches addresses the case of a connection to a new - destination, with no existing or recent connection (and therefore - congestion control state) to that destination. + In using Quick-Start, a TCP host, say, host A, would indicate its + desired sending rate in bytes per second, using a Quick Start option + in the IP header of a TCP packet. Each router along the path could, + in turn, either approve the requested rate, reduce the requested + rate, or indicate that the Quick-Start request is not approved. (We + note that the `routers' referred to in this document also include + the IP-layer processing of the Quick-Start request at the sender.) + If the Quick-Start request is not approved, then the sender would + use the default congestion control mechanisms. The Quick-Start + mechanism can determine if there are routers along the path that do + not understand the Quick-Start option, or have not agreed to the + Quick-Start rate request. TCP host B communicates the final rate + request to TCP host A in a transport-level Quick-Start Response in + an answering TCP packet. Quick-Start would not be the first mechanism for explicit communication from routers to transport protocols about sending rates. Explicit Congestion Notification (ECN) gives explicit congestion control feedback from routers to transport protocols, based on the router detecting congestion before buffer overflow [RFC3168]. In contrast, routers would not use Quick-Start to give congestion information, but instead would use Quick-Start as an optional mechanism to give permission to transport protocols to use higher sending rates, based on the ability of all the routers along the path to determine if their respective output links are significantly underutilized. +1.1. Conventions and Terminology + + The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", + "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this + document are to be interpreted as described in [RFC2119]. + 2. Assumptions and General Principles This section describes the assumptions and general principles behind the design of the Quick-Start mechanism. Assumptions: * The data transfer in the two directions of a connection traverses different queues, and possibly even different routers. Thus, any mechanism for determining the allowed sending rate would have to be used independently for each direction. * The path between the two endpoints is relatively stable, such that the path used by the Quick-Start request is generally the same path - used by the Quick-Start packets one round-trip time later. [ZPS00] - shows this assumption should be generally valid, although [RFC3819] - discusses a range of Bandwidth on Demand subnets. + used by the Quick-Start packets one round-trip time later. [ZDPS01] + shows this assumption should be generally valid. However, [RFC3819] + discusses a range of Bandwidth on Demand subnets that could cause + the characteristics of the path to change over time. * Any new mechanism must be incrementally deployable, and might not be supported by all of the routers and/or end-hosts. Thus, any new mechanism must be able to accommodate non-supporting routers or end- hosts without disturbing the current Internet semantics. We note that while Quick-Start is incrementally deployable in this sense, a Quick-Start request can not be approved for a particular connection unless both end-nodes and all of the routers along the path have been configured to support Quick-Start. General Principles: * Our underlying premise is that explicit feedback from all of the routers along the path would be required, in the current architecture, for best-effort connections to use initial windows significantly larger than those allowed by [RFC3390], in the absence of other information about the path. - * A router should only approve a request for a higher sending rate - if the output link is underutilized. Any other approach will result - in either per-flow state at the router, or the possibility of a - (possibly transient) queue at the router. + * A router should only approve a Quick-Start request if the output + link is underutilized. Any other approach will result in either + per-flow state at the router, or the possibility of a (possibly + transient) queue at the router. * No per-flow state should be required at the router. Note that while per-flow state is not required, we also do not preclude a router from storing per-flow state for making Quick-Start decisions or for checking for misbehaving nodes. - There are also a number of questions regarding the Quick-Start - mechanism that are discussed later in this document. - - Questions: - - * Would the benefits of the Quick-Start mechanism be worth the added - complexity? - - The benefits and drawbacks of Quick-Start are discussed in more - detail in Section 9 on "Evaluation of Quick-Start". - - * One practical consideration is that packets with known and unknown - IP options are often dropped in the current Internet [MAF04]. - - We note that this does not preclude using Quick-Start in Intranets. - Further, [MAF04] also shows that over time the blocking of packets - negotiating ECN has become less common, and therefore an incremental - deployment story for Quick-Start based on IP Options is not out of - the question for the Internet. Appendix A.1 on "Alternate - Mechanisms for the Quick-Start Request" discusses the possibility of - using RSVP or ICMP instead of IP Options for carrying Quick-Start - Requests to routers. - - * A second practical consideration is that Quick-Start data packets - could be dropped at non-IP queues along the path, if the non-IP - queue is a point of congestion. This is discussed in more detail in - Section 9.2. - - * Apart from the merits and shortcomings of the Quick-Start - mechanism, is there likely to be a compelling need to add explicit - congestion-related feedback from routers over and above the one-bit - feedback from ECN? - - If the answer to the question above is yes, should we be considering - ways to incorporate Quick-Start in mechanisms that, while more - complex, are also sufficiently more powerful than Quick-Start, or - should Quick-Start be considered as orthogonal to such mechanisms? - This is discussed further in Appendix A.6 on "Why Not Include More - Functionality". - 2.1. Overview of Quick-Start In this section we give an overview of the use of Quick-Start with TCP to request a higher congestion window. The description in this section is non-normative; the normative description of Quick-Start with IP and TCP follows in Sections 3 and 4. Quick-Start could be used in the middle of a connection, e.g., after an idle or underutilized period, as well as for the initial sending rate; these uses of Quick-Start are discussed later in the document. @@ -486,63 +529,66 @@ end-points requesting a higher sending rate in the Quick-Start (QS) option in IP, and routers along the path approving, modifying, discarding or ignoring (and therefore disallowing) the Quick-Start Request. The receiver uses reliable, transport-level mechanisms to inform the sender of the status of the Quick-Start Request. In addition, Quick-Start assumes a unicast, congestion-controlled transport protocol; we do not consider the use of Quick-Start for multicast traffic. When sent as a request, the Quick-Start Option includes a request - for a sending rate in bytes per second, and a Quick-Start TTL (QS + for a sending rate in bits per second, and a Quick-Start TTL (QS TTL) to be decremented by every router along the path that understands the option and approves the request. The Quick-Start TTL is initialized by the sender to a random value. The transport - receiver returns the rate and information about the TTL to the - sender using transport-level mechanisms. In particular, the - receiver computes the difference between the Quick-Start TTL and the - IP TTL (the TTL in the IP header) of the Quick-Start request packet, - and returns this in the Quick-Start response. The sender uses this - information to determine if all of the routers along the path - decremented the Quick-Start TTL, approving the Quick-Start Request. + receiver returns the rate, information about the TTL and the Quick- + Start Nonce to the sender using transport-level mechanisms. In + particular, the receiver computes the difference between the Quick- + Start TTL and the IP TTL (the TTL in the IP header) of the Quick- + Start request packet, and returns this in the Quick-Start response. + The sender uses this information to determine if all of the routers + along the path decremented the Quick-Start TTL, approving the Quick- + Start Request. If the request is approved by all of the routers along the path, then the TCP sender combines this allowed rate with the measurement of the round-trip time, and ends up with an allowed TCP congestion window. This window is sent rate-paced over the next round-trip time, or until an ACK packet is received. - Figure 1 shows a successful use of Quick-Start, with both routers - along the path approving the Quick-Start Request. In this example, - Quick-Start is used by TCP to establish the initial congestion - window. + Figure 1 shows a successful use of Quick-Start, with the sender's IP + layer and both routers along the path approving the Quick-Start + Request. In this example, Quick-Start is used by TCP to establish + the initial congestion window. Sender Router 1 Router 2 Receiver ------ -------- -------- -------- | | | | Quick-Start Request - | in SYN or SYN/ACK --> + | in SYN or SYN/ACK. + | IP: Decrement QS TTL + | to approve request --> | | Decrement | QS TTL | to approve | request --> | | Decrement | QS TTL | to approve | request --> | - | - | + | + | | | Return Quick-Start | info to sender in | <-- TCP ACK packet. | | | Quick-Start approved, | translate to cwnd. | Report Approved Rate. V Send cwnd paced over one RTT. --> @@ -555,33 +601,35 @@ default congestion control mechanisms for that transport protocol, including the default initial congestion window, response to idle periods, etc. Sender Router 1 Router 2 Receiver ------ -------- -------- -------- | | | | Quick-Start Request - | in SYN or SYN/ACK --> + | in SYN or SYN/ACK. + | IP: Decrement QS TTL + | to approve request --> | | Decrement | QS TTL | to approve | request --> | | Forward packet | without modifying | Quick-Start Option. --> | - | - | + | + | | | Return Quick-Start | info to sender in | <-- TCP ACK packet. | | | Quick-Start not approved. | Report Approved Rate. V Use default initial cwnd. --> @@ -613,34 +661,37 @@ length of eight bytes. The third byte includes a four-bit Function field. If the Function field is set to "0000", then the Quick-Start Option is a Rate Request. In this case, the second half of the third byte is a four- bit Rate Request field. For a Rate Request, the fourth byte contains the Quick-Start TTL (QS TTL) field. The sender MUST set the QS TTL field to a random value. Routers that approve the Quick-Start Request decrement the QS TTL - (mod 256) by the same amount that they decrement the IP TTL. The QS - TTL is used by the sender to detect if all of the routers along the - path understood and approved the Quick-Start option. + (mod 256) by the same amount that they decrement the IP TTL. (As + elsewhere in this document, we use the term `router' imprecisely to + also include the Quick-Start functionality at the IP layer at the + sender.) The QS TTL is used by the sender to detect if all of the + routers along the path understood and approved the Quick-Start + option. For a Rate Request, the transport sender MUST calculate and store the TTL Diff, the difference between the IP TTL value and the QS TTL value in the Quick-Start request packet, as follows: TTL Diff = ( IP TTL - QS TTL ) mod 256 (1) - For a Rate Request, the second four bytes contain a 30-bit QS Nonce - and a two-bit Reserved field. The sender SHOULD set the reserved - field to zero, and routers SHOULD ignore the reserved field. The - sender SHOULD set the 30-bit QS Nonce to a random value. + For a Rate Request, bytes 5-8 contain a 30-bit QS Nonce and a two- + bit Reserved field. The sender SHOULD set the reserved field to + zero, and routers and receivers SHOULD ignore the reserved field. + The sender SHOULD set the 30-bit QS Nonce to a random value. The sender initializes the Rate Request to the desired sending rate, including an estimate of the transport and IP header overhead. The encoding function for the Rate Request sets the request rate to K*2^N bps (bits per second), for N the value in the Rate Request field, and for K set to 40,000. For N=0, the rate request would be set to zero, regardless of the encoding function. This is illustrated in Table 1 below. For the four-bit Rate Request field, the request range is from 80 Kbps to 1.3 Gbps. Alternate encodings that were considered for the Rate Request are given in Appendix A.2. @@ -693,55 +744,61 @@ After an approved Rate Request, the sender MUST report the Approved Rate, using a Quick-Start Option configured as a Report of Approved Rate with the Rate Report field set to the approved rate. The packet containing the Report of Approved Rate MUST be either a control packet sent before the first Quick-Start data packet, or a Quick-Start Option in the first data packet itself. The Report of Approved Rate does not have to be sent reliably; for example, if the approved rate is reported in a separate control packet, the sender does not necessarily know if the control packet has been dropped in - the network. + the network. If the packet contained the Quick-Start Request is + acknowledged, but the acknowledgement packet does not contain a + Quick-Start Response, then the sender MUST assume that the Quick- + Start Request was denied, and set a Report of Approved Rate with a + rate of zero. If the Rate Request is denied, the sender MUST sent a Report of Approved Rate with the Rate Report field set to zero. We note that unlike a Quick-Start Request sent at the beginning of a connection, when a Quick-Start Request is sent in the middle of a connection, the connection could already have an established congestion window or sending rate. The Rate Request is the requested total rate for the connection, including the current rate of the connection; the Rate Request is *not* a request for an additional sending rate over and above the current sending rate. If the Rate Request is denied, or lowered to a value below the connection's current sending rate, then the sender ignores the request, and reverts to the default congestion control mechanisms of the transport protocol. + The use of the Quick-Start Option does not require the additional + use of the Router Alert Option [RFC2113]. + We note that in IPv4, a change in IP options at routers requires recalculating the IP header checksum. 3.2. The Quick-Start Option for IPv6 The Quick-Start Option for IPv6 is placed in the Hop-by-Hop Options extension header that is processed at every network node along the communication path [RFC 2460]. The option format following the generic Hop-by-Hop Options header is identical to the IPv4 format, with the exception that the Length field should exclude the common type and length fields in the option format and be set to 6 bytes instead of 8 bytes. For a Quick-Start Request, the transport receiver compares the - Quick-Start TTL with the IPv6 Hop Limit field in order to calculate - the TTL Diff. (The Hop Limit in IPv6 is the equivalent of the TTL - in IPv4.) That is, TTL Diff MUST be calculated and stored as - follows: + Quick-Start TTL with the IPv6 Hop Limit field to calculate the TTL + Diff. (The Hop Limit in IPv6 is the equivalent of the TTL in IPv4.) + That is, TTL Diff MUST be calculated and stored as follows: TTL Diff = ( IPv6 Hop Limit - QS TTL ) mod 256 (2) Unlike IPv4, modifying or deleting the Quick-Start IPv6 Option does not require checksum re-calculation, because the IPv6 header does not have a checksum field, and modifying the Quick-Start Request in the IPv6 Hop-by-Hop options header does not affect the IPv6 pseudo- header checksum used in upper-layer checksum calculations. Note that [RFC2460] specifies that when a specific flow label has @@ -754,99 +811,116 @@ specification of flow labels in Appendix A of RFC 2460 is changed. We note that RFC 2460 states that the use of the flow label field in IPv6 "is, at the time of writing, still experimental and subject to change as the requirements for flow support in the Internet become clearer" [RFC2460]. 3.3. Processing the Quick-Start Request at Routers The Quick-Start Request does not report the current sending rate of the connection sending the request; in the default case of a router - that does not maintain per-flow state, a router makes the - conservative assumption that the flow's current sending rate is - zero. Each participating router can either terminate or approve the - Quick-Start Request. A router should only approve a Quick-Start - request if the output link is underutilized, and if the router - judges that the output link will continue to be underutilized if the - request is approved. Otherwise, the router terminates the Quick- - Start Request. + (or IP layer implementation at an end-node) that does not maintain + per-flow state, a router makes the conservative assumption that the + flow's current sending rate is zero. Each participating router can + either terminate or approve the Quick-Start Request. A router MUST + only approve a Quick-Start request if the output link is + underutilized, and if the router judges that the output link will + continue to be underutilized if this and earlier approved requests + are used by the senders. Otherwise, the router reduces or + terminates the Quick-Start Request. + + While the paragraph above defines the *semantics* of approving a + Quick-Start request, this document does not specify the specific + algorithms to be used by routers in processing Quick-Start Requests + or Reports. This is similar to RFC 3168, which specifics the + semantics of the ECN codepoints in the IP header, but does not + specify specific algorithms for routers to use in deciding when to + drop or mark packets before buffer overflow. A router that wishes to terminate the Quick-Start Request SHOULD delete the Quick-Start Request from the IP header. This saves - resources as downstream routers will have no option to process. If - a Quick-Start-capable router wishes to deny the request but doesn't - delete the Quick-Start Request from the IP header, then the router - SHOULD zero the QS TTL, QS Nonce, and Rate Request fields. Zeroing - the Rate Request field may be more efficient for routers to + resources because downstream routers will have no option to process. + If a Quick-Start-capable router wishes to deny the request but + doesn't delete the Quick-Start Request from the IP header, then the + router SHOULD zero the QS TTL, QS Nonce, and Rate Request fields. + Zeroing the Rate Request field may be more efficient for routers to implement than deleting the Quick-Start option. As suggested in [B05], future additions to Quick-Start could define error codes for routers to insert into the QS Nonce field to report back to the sender the reason that the Quick-Start request was denied, e.g., that the router is denying all Quick-Start requests at this time, or that this router as a matter of policy does not grant Quick-Start requests. A router that doesn't understand the Quick-Start option will simply forward the packet with the Quick-Start Request - unchanged. + unchanged (ensuring that the TTL Diff will not match and Quick-Start + will not be used). If the participating router has decided to approve the Quick-Start Request, it does the following: - * The router MUST decrement the QS TTL by one. + * The router MUST decrement the QS TTL by the amount the IP TTL is + decremented (usually one). * If the router is only willing to approve a Rate Request less than that in the Quick-Start Request, then the router replaces the Rate Request with a smaller value. The router MUST NOT increase the Rate Request in the Quick-Start Request. If the router decreases the Rate Request, the router MUST also modify the QS Nonce, as described in Section 3.4. * In IPv4, the router MUST update the IP header checksum. If the router approves the Quick-Start request, this approval SHOULD be taken into account in the router's decision to accept or reject - subsequent Quick-Start requests (e.g., in a variable that tracks the - recent aggregate of accepted Quick-Start bandwidth), but this - approval SHOULD NOT be used by the router to affect the treatment of - the data packets that arrive from this connection in the next few - round-trip times. That is, the approval by the router of a Quick- - Start request does not give differential treatment for Quick-Start - data packets at that router; it is only a statement from the router - that the router believes that the subsequent Quick-Start data - packets from this connection will not change the current under- - utilized state of the router. + subsequent Quick-Start requests (e.g., using a variable that tracks + the recent aggregate of accepted Quick-Start requests). This + consideration of earlier approved Quick-Start request is necessary + to ensure that the router only approves a Quick-Start request when + the router judges that the output link will remain underutilized if + this and earlier Quick-Start requests are used by the senders. + + In addition, the approval of a Quick-Start request SHOULD NOT be + used by the router to affect the treatment of the data packets that + arrive from this connection in the next few round-trip times. That + is, the approval by the router of a Quick-Start request does not + give differential treatment for Quick-Start data packets at that + router; it is only a statement from the router that the router + believes that the subsequent Quick-Start data packets from this + connection will not change the current under-utilized state of the + router. A non-participating router forwards the Quick-Start Request unchanged, without decrementing the QS TTL. The non-participating router still decrements the TTL field in the IP header, as is required for all routers [RFC1812]. As a result, the sender will be able to detect that the Quick-Start Request had not been understood or approved by all of the routers along the path. 3.3.1. Processing the Report of Approved Rate If the Quick-Start Option has the Function field set to "1000", then this is a Report of Approved Rate, rather than a Rate Request. The router MAY ignore such an option, and in any case it MUST NOT modify the contents of the option for a Report of Approved Rate. However, the router MAY use the Approved Rate report to check that the sender is not lying about the approved rate. If the reported Approved Rate is higher than the rate that the router actually approved for this connection in the previous round-trip time, then the router may + implement some policy for cheaters. For instance, the router MAY decide to deny future Quick-Start requests from this sender, including, if desired, deleting Quick-Start requests from future packets from this sender. Section 9.4.1 discusses misbehaving senders in more detail. From the Report of Approved Rate, the router can also learn if some of the downstream routers have - approved the Quick-Start request for a smaller rate, and adjust its - bandwidth allocations accordingly. From a Report of Approved Rate - with a Rate Report of zero, the router can learn if downstream - routers denied the earlier Quick-Start request. + approved the Quick-Start request for a smaller rate or denied the + use of Quick-Start, and adjust its bandwidth allocations + accordingly. 3.4. The QS Nonce The QS Nonce gives the Quick-Start sender some protection against receivers lying about the value of the received Rate Request. This is particularly important if the receiver knows the original value of the Rate Request (e.g., when the sender always requests the same value, and the receiver has a long history of communication with that sender). Without the QS Nonce, there is nothing to prevent the receiver from reporting back to the sender a Rate Request of K, when @@ -874,22 +948,21 @@ Bits 22-23: Rate 4 -> Rate 3 Bits 24-25: Rate 3 -> Rate 2 Bits 26-27: Rate 2 -> Rate 1 Bits 28-29: Rate 1 -> Rate 0 Table 2: The QS Nonce. The transport sender MUST initialize the QS Nonce to a random value. If the router reduces the Rate Request from rate K to rate K-1, then the router MUST set the field in the QS Nonce for "Rate K -> Rate - K-1" to a new random value, using the requirements for "randomness" - in the previous paragraph. Similarly, if the router reduces the + K-1" to a new random value. Similarly, if the router reduces the Rate Request by N steps, the router MUST set the 2N bits in the relevant fields in the QS Nonce to a new random value. The receiver MUST report the QS Nonce back to the sender. If the Rate Request was not decremented in the network, then the QS Nonce should have its original value. Similarly, if the Rate Request was decremented by N steps in the network, and the receiver reports back a Rate Request of K, then the last 2K bits of the QS Nonce should have their original value. @@ -930,80 +1003,88 @@ Section 9.4 also considers issues of receiver cheating in more detail. 4. The Quick-Start Mechanisms in TCP This section describes how the Quick-Start mechanism would be used in TCP. We first sketch the procedure and then tightly define it in the subsequent subsections. If a TCP sender, say host A, would like to use Quick-Start, the TCP - sender puts the requested sending rate in bytes per second, + sender puts the requested sending rate in bits per second, appropriately formatted, in the Quick-Start option in the IP header of the TCP packet, called the Quick-Start request packet. (We will be somewhat loose in our use of "packet" vs. "segment" in this - section.) The Quick-Start Request also includes random values for - the QS TTL and the QS Nonce. When used for initial start-up, the - Quick-Start request packet can be either the SYN or SYN/ACK packet, - as described above. The requested rate includes an estimate for the - transport and IP header overhead. The TCP receiver, say host B, - returns the Quick-Start Response option in the TCP header in the - responding SYN/ACK packet or ACK packet, called the Quick-Start - response packet, informing host A of the results of their request. + section.) When used for initial start-up, the Quick-Start request + packet can be either the SYN or SYN/ACK packet, as described above. + The requested rate includes an estimate for the transport and IP + header overhead. The TCP receiver, say host B, returns the Quick- + Start Response option in the TCP header in the responding SYN/ACK + packet or ACK packet, called the Quick-Start response packet, + informing host A of the results of their request. If the acknowledging packet does not contain a Quick-Start Response, or contains a Quick-Start Response with the wrong value for the TTL Diff or the QS Nonce, then host A MUST assume that its Quick-Start request failed. In this case, host A sends a Report of Approved Rate with a Rate Report of zero, and uses TCP's default congestion control procedure. For initial start-up, host A uses the default - initial congestion window. + initial congestion window [RFC2581]. If the returning packet contains a valid Quick-Start Response, then host A uses the information in the response, along with its measurement of the round-trip time, to determine the Quick-Start congestion window (QS-cwnd). Quick-Start data packets are defined as data packets sent as the result of a successful Quick-Start request, up to the time when the first Quick-Start packet is - acknowledged. The sender sends a Report of Approved Rate. In order - to use Quick-Start, the TCP host MUST use rate-based pacing to + acknowledged. The sender also sends a Report of Approved Rate. In + order to use Quick-Start, the TCP host MUST use rate-based pacing to transmit Quick-Start packets at the rate indicated in the Quick- Start Response, at the level of granularity possible by the sending host. We note that the limitations of interrupt timing on computers can limit the ability of the TCP host in rate-pacing the outgoing packets. The two TCP end-hosts can independently decide whether to request Quick-Start. For example, host A could sent a Quick-Start Request in the SYN packet, and host B could also send a Quick-Start Request in the SYN/ACK packet. -4.1. When to Use Quick-Start +4.1. Sending the Quick-Start Request + + When sending a Quick-Start Request, the TCP sender SHOULD send the + request on a packet that requires an acknowledgement, such as a SYN, + SYN/ACK, or data packet. In this case, if the packet is + acknowledged but no Quick-Start Response is included, then the + sender knows that the Quick-Start request has been denied, and can + send a Report of Approved Rate. In addition to the use of Quick-Start when a connection is established, there are several additional points in a connection when a transport protocol may want to issue a Rate Request. We first re-iterate the notion that Quick-Start is a coarse-grained mechanism. That is, Quick-Start's Rate Requests are not meant to be used for fine-grained control of the transport's sending rate. Rather, the transport MAY issue a Rate Request when no information about the appropriate sending rate is available and the default congestion control mechanisms might be significantly underestimating the appropriate sending rate. The following are potential points where Quick-Start may be useful: (1) At or soon after connection initiation, when the transport - has no idea of the capacity of the network, as discussed above. - (A transport that uses TCP Control Block sharing, the Congestion - Manager, or the like may not need Quick-Start to determine an - appropriate rate.) + has no idea of the capacity of the network path, as discussed + above. (A transport that uses TCP Control Block sharing + [RFC2140], the Congestion Manager [RFC3124], or other mechanisms + for sharing congestion information may not need Quick-Start to + determine an appropriate rate.) + (2) After an idle period when the transport no longer has a validated estimate of the available bandwidth for this flow. (An example could be a persistent-HTTP connection when a new HTTP request is received after an idle period.) (3) After a host has received explicit indications that one of the endpoints has moved its point of network attachment. This can happen due to some underlying mobility mechanism like Mobile IP [RFC3344,RFC3775]. Some transports, such as SCTP [RFC2960], may associate with multiple IP addresses and can switch @@ -1061,178 +1142,200 @@ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | QS Nonce | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 5. The Quick-Start Response option in the TCP header. The first byte of the Quick-Start Response option contains the option kind, identifying the TCP option (to be assigned by IANA). The second byte of the Quick-Start Response option contains the - option length in bytes. The length field MUST be set to four bytes. + option length in bytes. The length field MUST be set to 8 bytes. The third byte of the Quick-Start Response option contains a four- bit Reserved field, and the four-bit allowed Rate Request, formatted - as in the Quick-Start option. + as in the Quick-Start Rate Request option. The fourth byte of the TCP option contains the TTL Diff. The TTL Diff contains the difference between the IP TTL and QS TTL fields in the received Quick-Start request packet, as calculated in equations (1) or (2) (depending on whether IPv4 or IPv6 is used). - The last four bytes of the TCP option contain the 30-bit QS Nonce - and a two-bit Reserved field. + Bytes 5-8 of the TCP option contain the 30-bit QS Nonce and a two- + bit Reserved field. - We note that the Quick-Start Response Option for TCP contains eight - bytes, and the length of the TCP option field is generally at most - 40 bytes. Other TCP options that might be used include Time Stamp - (ten bytes), Window Scale (three bytes), Maximum Segment Size (four - bytes), Selective Acknowledgments Data (at least ten bytes), and - Selective Acknowledgments Permitted (two bytes). + We note that while there are limitations on the potential size of + the Quick-Start Response Option, a Quick-Start Response Option of + eight bytes should not be a problem. The TCP Options field can + contain up to 40 bytes. Other TCP options that might be used in a + SYN or SYN/ACK packet include Maximum Segment Size (four bytes), + Time Stamp (ten bytes), Window Scale (three bytes), and Selective + Acknowledgments Permitted (two bytes). 4.3. TCP: Sending the Quick-Start Response An end host, say host B, that receives an IP packet containing a Quick-Start Request passes the Quick-Start Request, along with the value in the IP TTL field, to the receiving TCP layer. If the TCP host is willing to permit the Quick-Start Request, then a Quick-Start Response option is included in the TCP header of the corresponding acknowledgement packet. The Rate Request in the Quick-Start Response option is set to the received value of the Rate Request in the Quick-Start option, or to a lower value if the TCP receiver is only willing to allow a lower Rate Request. The TTL Diff in the Quick-Start Response is set to the difference between the IP TTL value and the QS TTL value as given in equation (1) or (2) (depending on whether IPv4 or IPv6 is used). The QS Nonce in the Response is set to the received value of the QS Nonce in the Quick-Start option. - The Quick-Start Response will NOT be resent if it is lost in the - network. Packet loss is an indication of congestion on the return - path, in which case it is better not to approve the Quick-Start - Request. + If an end host receives an IP packet with a Quick-Start Request with + a rate request of zero, then that host SHOULD NOT send a Quick-Start + Response. + + The Quick-Start Response MUST NOT be resent if it is lost in the + network. Packet loss could be an indication of congestion on the + return path, in which case it is better not to approve the Quick- + Start Request. 4.4. TCP: Receiving and Using the Quick-Start Response Packet A TCP host, say TCP host A, that sent a Quick-Start Request and receives a Quick-Start Response in an acknowledgement first checks that the Quick-Start Response is valid. The Quick-Start Response is valid if it contains the correct value for the TTL Diff, and an equal or lesser value for the Rate Request than that transmitted in the Quick-Start Request. In addition, if the received Rate Request is K, then the the rightmost 2K bits of the QS Nonce must match those bits in the QS Nonce sent in the Quick-Start Request. If these checks are not successful, then the Quick-Start request failed, and the TCP host MUST use the default TCP congestion window that it would have used without Quick-Start. If the rightmost 2K bits of the QS Nonce do not match those bits in the QS Nonce sent in the Quick-Start Request, for a received Rate Request of K, then the TCP host MUST NOT send additional Quick-Start requests during the life of the connection. Whether the Quick-Start request was - successful or not, the TCP host MUST send a Report of Approved Rate. + successful or not, the host receiving the Quick-Start Response MUST + send a Report of Approved Rate. Similarly, if the packet containing + the Quick-Start Request is acknowledged, but the acknowledgement + does not include a Quick-Start Response, then the sender MUST send a + Report of Approved Rate. If the checks of the TTL Diff and the Rate Request are successful, - then the TCP host sets its Quick-Start congestion window (in terms - of MSS-sized segments), QS-cwnd, as follows: + and the TCP host is going to use the Quick-Start Request, it MUST + start using it within one round-trip time of receiving the Quick- + Start Response, or within three seconds, whichever is smaller. To + use the Quick-Start Request, the host sets its Quick-Start + congestion window (in terms of MSS-sized segments), QS-cwnd, as + follows: QS-cwnd = (R * T) / (MSS + H) (3) where R the Rate Request in bytes per second, T the measured round- trip time in seconds, and H the estimated TCP/IP header size in bytes (e.g., 40 bytes). Derivation: the sender is allowed to transmit at R bytes per second including packet headers, but only R*MSS/(MSS+H) bytes per second, or equivalently R*T*MSS/(MSS+H) bytes per round-trip time, of application data. The TCP host SHOULD set its congestion window cwnd to QS-cwnd only - if QS-cwnd is greater than cwnd; otherwise QS-cwnd is ignored. When - Quick-Start is used at the beginning of a connection, before any - packet marks or losses have been reported, the TCP host MAY use the - reported Rate Request to set the slow-start threshold to a desired - value, e.g., to some small multiple of the congestion window. (The - initial value of ssthresh is allowed to be arbitrarily high, and - some TCP implementations use the size of the advertised window for - ssthresh [RFC2581].) - - If QS-cwnd is used, the TCP host sets a flag that it is in Quick- - Start mode, and while in Quick-Start mode the TCP sender MUST use - rate-based pacing to pace out Quick-Start packets at the specified - Rate Request. If, during Quick-Start mode, the TCP sender receives - ACKs for packets sent before this Quick-Start mode was entered, - these ACKs are processed as usual, following the default congestion - control mechanisms. Quick-Start mode ends when the TCP host - receives an ACK for one of the Quick-Start packets. + if QS-cwnd is greater than cwnd; otherwise QS-cwnd is ignored. If + QS-cwnd is used, the TCP host sets a flag that it is in Quick-Start + mode, and while in Quick-Start mode the TCP sender MUST use rate- + based pacing to pace out Quick-Start packets at the approved rate. + If, during Quick-Start mode, the TCP sender receives ACKs for + packets sent before this Quick-Start mode was entered, these ACKs + are processed as usual, following the default congestion control + mechanisms. Quick-Start mode ends when the TCP host receives an ACK + for one of the Quick-Start packets. If the congestion window has not been fully used when the first ack arrives ending the Quick-Start mode, then the congestion window is decreased to the amount that has actually been used so far. This is necessary because when the Quick-Start Response is received, the TCP sender's round-trip-time estimate might be longer than for succeeding round-trip times, e.g., because of delays at routers - processing the IP QuickStart option, or because of delays at the + processing the IP Quick-Start option, or because of delays at the receiver in responding to the Quick-Start Request packet. In this case, an overly-large round-trip-time estimate could have caused the TCP sender to translate the approved Quick-Start sending rate in bytes per second into a congestion window that is larger than needed, with the TCP sender receiving an ACK for the first Quick- Start packet before the entire congestion window has been used. Thus, when the TCP sender receives the first ACK for a Quick-Start - packet, the sender reduces its congestion window to the amount that - has actually been used. + packet, the sender MUST reduce the congestion window to the amount + that has actually been used. As an example, a TCP sender with an approved Quick-Start request of R KBps, B-byte packets including headers, and an RTT estimate of T seconds, would translate the Rate Request of R KBps to a congestion window of R*T/B packets. The TCP sender would send the Quick-Start packets rate-paced at R KBps. However, if the actual current round- trip time was T/2 seconds instead of T seconds, then the sender would begin to receive acknowledgements for Quick-Start packets after T/2 seconds. Following the paragraph above, the TCP sender - would then reduce its congestion window from R*T/B to R*T/(B*2) - packets, the actual number of packets that were needed to fill the - pipe at a sending rate of R KBps. + would then reduce its congestion window from R*T/B to approximately + R*T/(B*2) packets, the actual number of packets that were needed to + fill the pipe at a sending rate of R KBps. (Note: this is just an + illustration and that the congestion window is actually set to the + amount of data sent before the ACK arrives and not based on + equations above.) After Quick-Start mode is exited and the congestion window adjusted if necessary, the TCP sender returns to using the default congestion control mechanisms, processing further incoming ACK packets as specified by those congestion control mechanisms. For example, if the TCP sender was in slow-start prior to the Quick-Start request, and no packets were lost or marked since that time, then the sender continues in slow-start after exiting Quick-Start mode, as allowed by ssthresh. To add robustness, the TCP sender MUST use Limited Slow-Start [RFC3742] along with Quick-Start. With Limited Slow-Start, the TCP sender limits the number of packets by which the congestion window is increased for one window of data during slow-start. + When Quick-Start is used at the beginning of a connection, before + any packet marks or losses have been reported, the TCP host MAY use + the reported Rate Request to set the slow-start threshold to a + desired value, e.g., to some small multiple of the congestion + window. A possible future research topic is how the sender might + modify the show-start threshold at the beginning of a connection to + avoid overshooting the path capacity. (The initial value of + ssthresh is allowed to be arbitrarily high, and some TCP + implementations use the size of the advertised window for ssthresh + [RFC2581].) + 4.5. TCP: Responding to a Loss of a Quick-Start Packet For TCP, we have defined a ``Quick-Start packet'' as one of the packets sent in the window immediately following a successful Quick- - Start request. After detecting the loss of a Quick-Start packet, - TCP MUST revert to the default congestion control procedures that - would have been used if the Quick-Start request had not been - approved. For example, if Quick-Start is used for setting the - initial window, and a packet from the initial window is lost, then - the TCP sender MUST then slow-start with the default initial window - that would have been used if Quick-Start had not been used. In - addition to reverting to the default congestion control mechanisms, - the sender MUST take into account that the Quick-Start congestion - window was too large. Thus, the sender SHOULD decrease ssthresh to - at most half the number of Quick-Start packets that were + Start request. After detecting the loss or ECN-marking of a Quick- + Start packet, TCP MUST revert to the default congestion control + procedures that would have been used if the Quick-Start request had + not been approved. For example, if Quick-Start is used for setting + the initial window, and a packet from the initial window is lost or + marked, then the TCP sender MUST then slow-start with the default + initial window that would have been used if Quick-Start had not been + used. In addition to reverting to the default congestion control + mechanisms, the sender MUST take into account that the Quick-Start + congestion window was too large. Thus, the sender SHOULD decrease + ssthresh to at most half the number of Quick-Start packets that were successfully transmitted. Section A.5 discusses possible alternatives in responding to the loss of a Quick-Start packet. + If a Quick-Start packet is lost or ECN-marked, then the sender + SHOULD NOT make future Quick-Start requests for this connection. + We note that ECN [RFC3168] MAY be used with Quick-Start. As is always the case with ECN, the sender's congestion control response to an ECN-marked Quick-Start packet is the same as the response to a dropped Quick-Start packet, thus reverting to slow start in the case of Quick-Start packets marked as experiencing congestion. 4.6. TCP: A Quick-Start Request for a Larger Initial Window Some of the issues of using Quick-Start are related to the specific scenario in which Quick-Start is used. This section discusses the @@ -1265,116 +1368,99 @@ The sender SHOULD send one large packet in the initial window with the DF bit set, and send the remaining packets in the initial window with a smaller MTU of 576 bytes (or 1280 bytes with IPv6). A second possibility would be for the sender to delay sending the Quick-Start Request for one round-trip time, sending the Quick-Start Request with the first window of data while also doing Path MTU Discovery. -4.6.2. Quick-Start Request Packets that are Discarded by Middleboxes +4.6.2. Quick-Start Request Packets that are Discarded by Routers or +Middleboxes It is always possible for a TCP SYN packet carrying a Quick-Start request to be dropped in the network due to congestion, or to be - blocked due to interactions with middleboxes, where a middlebox is - defined as any intermediary box performing functions apart from - normal, standard functions of an IP router on the data path between - a source host and destination host [RFC3234]. Measurement studies - of interactions between transport protocols and middleboxes [MAF04] - show that for 70% of the web servers investigated, no connection is - established if the TCP SYN packet contains an unknown IP option (and - for 43% of the web servers, no connection is established if the TCP - SYN packet contains an IP TimeStamp Option). In both cases, this is - presumably due to middleboxes along that path. + blocked due to interactions with routers or middleboxes, where a + middlebox is defined as any intermediary box performing functions + apart from normal, standard functions of an IP router on the data + path between a source host and destination host [RFC3234]. + Measurement studies of interactions between transport protocols and + middleboxes [MAF04] show that for 70% of the web servers + investigated, no connection is established if the TCP SYN packet + contains an unknown IP option (and for 43% of the web servers, no + connection is established if the TCP SYN packet contains an IP + TimeStamp Option). In both cases, this is presumably due to routers + or middleboxes along that path. If the TCP sender doesn't receive a response to the SYN or SYN/ACK packet containing the Quick-Start Request, then the TCP sender SHOULD resend the SYN or SYN/ACK packet without the Quick-Start Request. Similarly, if the TCP sender receives a TCP reset in response to the SYN or SYN/ACK packet containing the Quick-Start Request, then the TCP sender SHOULD resend the SYN or SYN/ACK packet without the Quick-Start Request [RFC3360]. - RFC 1122 and 2988 recommend that the sender should set the initial - RTO to three seconds, though many TCP implementations set the - initial RTO to one second. For a TCP SYN packet sent with a Quick- - Start request, the TCP sender SHOULD use an initial RTO of three - seconds. - - In the case of a retransmission, in addition to resending the SYN or - SYN/ACK packet without the Quick-Start Request, the TCP sender - SHOULD use an RTO of three seconds and a different Initial Sequence - Number. Using this scheme the TCP sender MUST keep track of when - each of the SYN (or SYN/ACK) packets was transmitted. In this way, - an acknowledgement for the retransmitted SYN or SYN/ACK packet can - be matched with the SYN or SYN/ACK being acknowledged, and the - transmission time of the SYN (or SYN/ACK) being acknowledged can be - used for an RTT measurement to seed the RTO. If only the - retransmitted SYN or SYN/ACK is acknowledged, the TCP sender can - reasonably assume that the earlier SYN or SYN/ACK with the Quick- - Start option was dropped by the network because of the option and - not because of congestion. In this case, the TCP sender can refrain - from performing TCP's standard congestion control state changes. + RFC 1122 and 2988 specify that the sender should set the initial RTO + to three seconds, though many TCP implementations set the initial + RTO to one second. For a TCP SYN packet sent with a Quick-Start + request, the TCP sender SHOULD use an initial RTO of three seconds. We note that if the TCP SYN packet is using the IP Quick-Start Option for a Quick-Start request, and it is also using bits in the TCP header to negotiate ECN-capability with the TCP host at the other end, then the drop of a TCP SYN packet could be due to - congestion, to a middlebox dropping the packet because of the IP - Option, or because of a middlebox dropping the packet because of the - information in the TCP header negotiating ECN. In this case, the - sender could resend the dropped packet without either the Quick- - Start or the ECN requests. Alternately, the sender could resend the - dropped packet with only the ECN request in the TCP header, - resending the TCP SYN packet without either the Quick-Start or the - ECN requests if the second TCP SYN packet is dropped. The second - choice seems reasonable, given that a TCP SYN packet today is more - likely to be blocked due to IP Options than due to an ECN request in - the TCP header [MAF04]. - - It is always possible that some middlebox that doesn't drop TCP SYN - packets containing Quick-Start options will still drop or delay TCP - data packets containing Quick-Start options as Approved Rate - reports. This would be one reason for a TCP sender to report the - Approved Rate in a separate control packet, rather than in a data - packet. + congestion, to a router or middlebox dropping the packet because of + the IP Option, or because of a router or middlebox dropping the + packet because of the information in the TCP header negotiating ECN. + In this case, the sender could resend the dropped packet without + either the Quick-Start or the ECN requests. Alternately, the sender + could resend the dropped packet with only the ECN request in the TCP + header, resending the TCP SYN packet without either the Quick-Start + or the ECN requests if the second TCP SYN packet is dropped. The + second choice seems reasonable, given that a TCP SYN packet today is + more likely to be blocked due to policies that discard packets with + IP Options than due to policies that discard packets with ECN + requests in the TCP header [MAF04]. 4.7. TCP: A Quick-Start Request in the Middle of a Connection This section discusses the following issues that arise when Quick- - Start is used by TCP to request a larger window in the middle of - connection, for example after an idle period: (1) determining the - rate to request; and (2) the response if Quick-Start packets are - dropped; + Start is used by TCP to request a larger window in the middle of a + connection, such as after an idle period: (1) determining the rate + to request; (2) when to make a request; and (3) the response if + Quick-Start packets are dropped; (1) Determining the rate to request: - In the middle of connection, an easy rule of thumb would be for the - TCP sender to determine the largest congestion window that the TCP - connection achieved since the last packet drop, to translate this - congestion window to a sending rate, and use this rate in the Quick- - Start request. If the request is granted, then the sender + For a connection that has not yet had a congestion event (that is, a + marked or dropped packet), the TCP sender is not restricted in the + rate that it requests. As an example, a server might wait and send + a Quick-Start request after a short interaction with the client. + + To use a Quick-Start Request in a connection that has already + experienced a congestion event, and that has not had a recent + mobility event, the TCP sender can determine the largest congestion + window that the TCP connection achieved since the last packet drop + and translate this to a sending rate to get the maximum allowed + request rate. If the request is granted, then the sender essentially restarts with its old congestion window from before it was reduced, for example during an idle period. - In the case of an idle period, the sender SHOULD NOT use Quick-Start - if the idle period has been less than an RTO, and the congestion - window has not decayed down to less than half of its value at the - start of the idle period. Such a use of Quick-Start requires - further investigation. + A Quick-Start Request sent in the middle of a TCP connection SHOULD + be sent on a data packet. - A Quick-Start Request sent in the middle of a TCP connection could - be carried either in a data packet or in a pure acknowledgement - packet. + (2) When to make a request: + A TCP connection MAY make a Quick-Start request before the + connection has experienced a congestion event, or after an idle + period of at least one RTO. (2) Response if Quick-Start packets are dropped: - If Quick-Start packets are dropped in the middle of connection, then the sender MUST revert to half of the Quick-Start window, or to the congestion window that the sender would have used if the Quick-Start request had not been approved, whichever is smaller. 4.8. An Example Quick-Start Scenario with TCP The following is an example scenario in the case when both hosts request Quick-Start for setting their initial windows. This is similar to Figures 1 and 2 in Section 2.1, except that it @@ -1436,45 +1522,47 @@ for IPv4. If the Quick-Start option is recognized, it MUST be treated as a mutable IPv4 option, and hence be completely zeroed for AH ICV calculation purposes. IPv6 option numbers explicitly indicate whether the option is mutable; the 3rd highest order bit in the IANA-allocated option type has the value 1 to indicate that the Quick-Start option data can change en route. RFC 2402 requires that the option data of any such option be zeroed for AH ICV computation purposes. Therefore changes to the Quick-Start option in the IP header do not affect the calculation of the AH ICV. -6. Quick-Start in IP Tunnels +6. Quick-Start in IP Tunnels and MPLS This section considers interactions between Quick-Start and IP - tunnels, including IPsec [RFC2401,2401bis] and IP in IP [RFC2003]. + tunnels, including IPsec [RFC2401,2401bis], IP in IP [RFC2003], GRE + [RFC2784], and others. This section also considers interactions + between Quick-Start and MPLS [RFC3031]. In the discussion, we use TTL Diff, defined earlier as the difference between the IP TTL and the Quick-Start TTL, mod 256. Recall that the sender considers the Quick-Start request approved only if the value of TTL Diff for the packet entering the network is the same as the value of TTL Diff for the packet exiting the network. Simple tunnels: IP tunnel modes are generally based on adding a new "outer" IP header that encapsulates the original or "inner" IP header and its associated packet. In many cases, the new "outer" IP - header may be added and removed at intermediate points along a - connection, enabling the network to establish a tunnel without - requiring endpoint participation. We denote tunnels that specify - that the outer header be discarded at tunnel egress as "simple - tunnels", and we denote tunnels where the egress saves and uses - information from the outer header before discarding it as "non- - simple tunnels". An example of a "non-simple tunnel" would be a - tunnel configured to support ECN, where the egress router might copy - the ECN codepoint in the outer header to the inner header before - discarding the outer header [RFC3168]. + header may be added and removed at intermediate points along a path, + enabling the network to establish a tunnel without requiring + endpoint participation. We denote tunnels that specify that the + outer header be discarded at tunnel egress as "simple tunnels", and + we denote tunnels where the egress saves and uses information from + the outer header before discarding it as "non-simple tunnels". An + example of a "non-simple tunnel" would be a tunnel configured to + support ECN, where the egress router might copy the ECN codepoint in + the outer header to the inner header before discarding the outer + header [RFC3168]. __ Tunnels Compatible with Quick-Start / Simple Tunnels __/ \ \__ Tunnels Not Compatible with Quick-Start (False Positives!) __ Tunnels Supporting Quick-Start / @@ -1663,20 +1750,38 @@ tunnel endpoints to allow or forbid support for Quick-Start within the tunnel. This was done for ECN for IPsec tunnels, with IKEv1 [RFC3168, Section 9.2]. This negotiation of Quick-Start capability in an IPsec tunnel will be specified in a separate IPsec document. This document will also include a discussion of the potential effects of an adversary's modifications of the Quick-Start field (as in Sections 18 and 19 of RFC 3168), and of the security considerations of exposing the Quick-Start rate request to network scanners. +6.4. Quick-Start and MPLS + + The behavior of Quick-Start with MPLS is similar to the behavior of + Quick-Start with IP Tunnels. For those MPLS paths where the IP TTL + is decremented as part of traversing the MPLS path, these paths are + compatible with Quick-Start, but do not support Quick-Start; Quick- + Start requests traversing these paths will be correctly understood + by the transport sender as having been denied. Any MPLS paths where + the IP TTL is not decremented as part of traversing the MPLS path + would be not compatible with Quick-Start; such paths would result in + false positives, where the TCP sender incorrectly believes that the + Quick-Start Request was approved by all routers along the path. + + For cases where the ingress node to the MPLS path is aware of Quick- + Start, this node should either zero the Quick-Start rate request, QS + TTL, and QS Nonce fields or remove the Quick-Start option from the + IP header. + 7. The Quick-Start Mechanism in other Transport Protocols The section earlier specified the use of Quick-Start in TCP. In this section, we generalize this to give guidelines for the use of Quick-Start with other transport protocols. We also discuss briefly how Quick-Start could be specified for other transport protocols. The general guidelines for Quick-Start in transport protocols are as follows: @@ -1686,21 +1791,21 @@ to augment their operation. * A transport-level mechanism is needed for the Quick-Start response from the receiver to the sender. This response contains the Rate Request, TTL Diff, and QS Nonce. * The sender checks the validity of the Quick-Start response. * The sender has an estimate of the round-trip time, and translates the Quick-Start response into an allowed window or allowed sending - rate. The sender sends a Report of Approved Rate. The sender + rate. The sender sends a Report of the Approved Rate. The sender starts sending Quick-Start packets, rate-paced out at the approved sending rate. * After the sender receives the first acknowledgement packet for a Quick-Start packet, no more Quick-Start packets are sent. The sender adjusts its current congestion window or sending rate to be consistent with the actual amount of data that was transmitted in that round-trip time. * When the last Quick-Start packet is acknowledged, the sender @@ -1711,25 +1816,25 @@ to the standard congestion control method of that protocol that would have been used if the Quick-Start request had not been approved. In addition, the sender takes into account the information that the Quick-Start congestion window was too large (e.g., by decreasing ssthresh in TCP). 8. Using Quick-Start 8.1. Determining the Rate to Request - As discussed in [SAF05], the data sender does not necessarily have + As discussed in [SAF06], the data sender does not necessarily have information about the size of the data transfer at connection initiation; for example, in request-response protocols such as HTTP, the server doesn't know the size or name of the requested object - during connection initiation. [SAF05] explores some of the + during connection initiation. [SAF06] explores some of the performance implications of overly-large Quick-Start requests, and discusses heuristics that end-nodes could use to size their requests appropriately. For example, the sender might have information about the bandwidth of the last-mile hop, the size of the local socket buffer, or of the TCP receive window, and could use this information in determining the rate to request. Web servers that mostly have small objects to transfer might decide not to use Quick-Start at all, since Quick-Start would be of little benefit to them. Quick-Start will be more effective if Quick-Start requests are not @@ -1765,32 +1870,32 @@ A simple way for the router to keep track of the potential bandwidth from recently-approved requests is to maintain two counters, one for the total aggregate Rate Requests that have been approved in the current time interval [T1, T2], and one for the total aggregate Rate Requests approved over a previous time interval [T0, T1]. However, this document doesn't specify router algorithms for approving Quick- Start requests, or make requirements for the appropriate time intervals for remembering the aggregate approved Quick-Start bandwidth. A possible router algorithm is given in Appendix D, and - more discussion of these issues is available in [SAF05].) + more discussion of these issues is available in [SAF06].) * If the router's output link has been underutilized and the aggregate of the Quick Start Request Rate options granted is low enough to prevent a near-term bandwidth shortage, then the router could approve the Quick-Start Request. Section 10.2 discusses some of the implementation issues in - processing Quick-Start requests at routers. [SAF05] discusses the + processing Quick-Start requests at routers. [SAF06] discusses the range of possible Quick-Start algorithms at the router for deciding whether to approve a Quick-Start request. In order to explore the - limits of the possible functionality at routers, [SAF05] also + limits of the possible functionality at routers, [SAF06] also discusses Extreme Quick-Start mechanisms at routers, where the router would keep per-flow state concerning approved Quick-Start requests. 9. Evaluation of Quick-Start 9.1. Benefits of Quick-Start The main benefit of Quick-Start is the faster start-up for the transport connection itself. For a small TCP transfer of one to @@ -1905,78 +2010,94 @@ 9.3. Quick-Start with QoS-enabled Traffic The discussion in this document has largely been of Quick-Start with default, best-effort traffic. However, Quick-Start could also be used by traffic using some form of differentiated services, and routers could take the traffic class into account when deciding whether or not to grant the Quick-Start request. We don't address this context further in this paper, since it is orthogonal to the specification of Quick-Start. + Routers are also free to take into account their own priority + classifications in processing Quick-Start requests. + 9.4. Protection against Misbehaving Nodes In this section we discuss the protection against senders, - receivers, or colluding middleboxes lying about the Quick-Start - Request. First, we note that it is not necessarily in the sender's - or receiver's interest to lie about the Quick-Start Request. If the - sender sends at too-high of an initial rate, and has a packet - dropped, this does not necessarily improve the performance of the - connection, relative to the case when the Quick-Start Request was - not approved. + receivers, or colluding routers or middleboxes lying about the + Quick-Start Request. 9.4.1. Misbehaving Senders A transport sender could try to transmit data at a higher rate than - that approved in the Quick-Start Request, or transmit at a high rate - even without using Quick-Start at all. The network could use a - traffic policer to protect against such misbehaving senders, for - example by dropping packets that exceed the allowed transmission - rate. The required Report of Approved Rate allows traffic policers - to check that the Report of Approved Rate does not exceed the Rate - Request actually approved at that point in the network in the - previous Quick-Start Request from that connection. The required - Approved Rate report also allows traffic policers to check that the - sender's sending rate does not exceed the rate in the Report of - Approved Rate. + that approved in the Quick-Start Request. The network could use a + traffic policer to protect against misbehaving senders that exceed + the approved rate, for example by dropping packets that exceed the + allowed transmission rate. The required Report of Approved Rate + allows traffic policers to check that the Report of Approved Rate + does not exceed the Rate Request actually approved at that point in + the network in the previous Quick-Start Request from that + connection. The required Approved Rate report also allows traffic + policers to check that the sender's sending rate does not exceed the + rate in the Report of Approved Rate. If a router or receiver receives an Approved Rate report that is larger than the Rate Request in the Quick-Start request approved for that sender for that connection in the previous round-trip time, then the router or receiver could deny future Quick-Start requests from that sender, e.g., by deleting the Quick-Start Request from future packets from that sender. We note that routers are not required to use Approved Rate reports to check if senders are - cheating; this is at the discretion of the router. If a router sees - a Report of Approved Rate, and did not see an earlier Quick-Start - request, then either the sender could be cheating, or the - connection's path could have changed since the Quick-Start request - was sent. In either case, the router could decide to deny future - Quick-Start requests from this sender. In particular, it is - reasonable for the router to deny a Quick-Start request if either - the sender is cheating, or if the connection path suffers from path - changes or multi-pathing. + cheating; this is at the discretion of the router. + + If a router sees a Report of Approved Rate, and did not see an + earlier Quick-Start request, then either the sender could be + cheating, or the connection's path could have changed since the + Quick-Start request was sent. In either case, the router could + decide to deny future Quick-Start requests for this connection. In + particular, it is reasonable for the router to deny a Quick-Start + request if either the sender is cheating, or if the connection path + suffers from path changes or multi-pathing. If a router approved a Quick-Start Request, but does not see a subsequent Approved Rate report, then there are several - possibilities: (1) the sender did not send a Report of Approved - Rate; (2) the Approved Rate report was dropped in the network; or - (3) the Approved Rate report took a different path from the Quick- - Start Request. In any of these three cases, the router would be - justified in denying future Quick-Start Requests from this sender. + possibilities: (1) the request was denied and/or dropped downstream + and the sender did not send a Report of Approved Rate; (2) the + request was approved but the sender did not send a Report of + Approved Rate; (3) the Approved Rate report was dropped in the + network; or (4) the Approved Rate report took a different path from + the Quick-Start Request. In any of these cases, the router would be + justified in denying future Quick-Start Requests for this + connection. - In any of the above mentioned cases (i.e., an Approved Rate report - that is larger than the Rate Request in the earlier Quick-Start - request; no Approved Rate report because of packet drops, path - changes, or the sender's failure to send one), a traffic policer may - assume that Quick-Start is not being used appropriately, or is being - used in an inappropriate environment, and take some corresponding - action. + In any of the cases mentioned in the three paragraphs above (i.e., + an Approved Rate report that is larger than the Rate Request in the + earlier Quick-Start request; a Report of Approved Rate with no + preceding Rate Request, or a Rate Request with no Report of Approved + Rate), a traffic policer may assume that Quick-Start is not being + used appropriately, or is being used in an unsuitable environment + (e.g., with multiple paths), and take some corresponding action. + + What are the incentives for a sender to cheat by over-sending after + a Quick-Start request? Assuming that the sender's interests are + measured by a performance metric such as the completion time for its + connections, sometimes it might be in the sender's interests to + cheat, and sometimes it might not; in some cases it could be + difficult for the sender to judge whether it would be in its + interests to cheat. The incentives for a sender to cheat by over- + sending after a Quick-Start request are not that different from the + incentives for a sender to cheat by over-sending even in the absence + of Quick-Start, with one difference: the use of Quick-Start could + help a sender to evade policing actions from policers in the + network. The Report of Approved Rate is designed to address this, + to make it harder to senders to use Quick-Start to `cover' their + cheating. 9.4.2. Receivers Lying about Whether the Request was Approved One form of misbehavior would be for the receiver to lie to the sender about whether the Quick-Start Request was approved, by falsely reporting the TTL Diff and QS Nonce. If a router that understands the Quick-Start Request denies the request by deleting the request or by zeroing the QS TTL and QS Nonce, then the receiver can ``lie" about whether the request was approved only by successfully guessing the value of the TTL Diff and QS Nonce to @@ -1995,24 +2116,28 @@ that host before calculating the TTL Diff, and decrement the QS TTL by two in the following received request, until the sender acts on one of the Quick-Start Requests. Unfortunately, if a router doesn't understand Quick-Start, then it is not possible for that router to take an active step such as zeroing the QS TTL and QS Nonce to deny a request. As a result, the QS TTL is not a fail-safe mechanism for preventing lying by receivers in the case of non-Quick-Start-capable routers. - As we noted above, it is not necessarily in the receiver's interests - to lie about whether the rate request was approved, particularly - since such lying could result in Quick-Start data packets dropped in - the network due to congestion. + What would be the incentives for a receiver to cheat in reporting on + a Quick-Start request, in the absence of a mechanism such as the QS + Nonce? In some cases, cheating would have been of clear benefit to + the receiver, resulting in a faster completion time for the + transfer. In other cases, where cheating would have resulted in + Quick-Start packets being dropped in the network, cheating might or + might not have improved the receiver's performance metric, depending + on the details of that particular scenario. 9.4.3. Receivers Lying about the Approved Rate A second form of receiver misbehavior would be for the receiver to lie to the sender about the Rate Request for an approved Quick-Start Request, by increasing the value of the Rate Request field. However, the receiver doesn't necessarily know the Rate Request in the original Quick-Start Request sent by the sender, and a higher Rate Request reported by the receiver will only be considered valid by the sender if it is no higher than the Rate Request originally @@ -2048,25 +2173,20 @@ spoofing attacks. A second limited form of protection would be for senders to use some degree of randomization in the requested Rate Request, so that it is difficult for receivers to guess the original value for the Rate Request. However, this is difficult because there is a fairly coarse granularity in the set of rate requests available to the sender, and randomizing the initial request only offers limited protection in any case. - Again, as we noted above, it is not necessarily in the receiver's - interests to lie about the value of the approved rate request, - particularly since such lying could result in Quick-Start data - packets dropped in the network due to congestion. - 9.4.4. Collusion between Misbehaving Routers In addition to protecting against misbehaving receivers, it is necessary also to protect against misbehaving routers. Consider collusion between an ingress router and an egress router belonging to the same Intranet. The ingress router could decrement the Rate Request at the ingress, with the egress router increasing it again at the egress. The routers between the ingress and egress that approved the decremented rate request might not have been willing to approve the larger, original request. @@ -2089,105 +2209,106 @@ all of the routers along the path. With ECN, a colluding ingress router could falsely mark a packet as ECN-capable, with the colluding egress router returning the ECN field in the IP header to its original non-ECN-capable codepoint, and congested routers along the path could have been fooled into not dropping that packet. This collusion would give an unfair competitive advantage to the traffic protected by the colluding ingress and egress routers. In contrast, with Quick-Start, the collusion of the ingress and egress routers to make it falsely appear that a Quick-Start request - was approved does not necessarily give an advantage to the traffic - covered by that collusion. If some router along the path really - does not have enough available bandwidth to approve the Quick-Start - request, then the Quick-Start packets sent as a result of the - falsely-approved request could be dropped in the network, to the - resulting disadvantage of the connection. Thus, while the ingress + was approved sometimes could give an advantage to the traffic + covered by that collusion, and sometimes would give a disadvantage, + depending on the details of the scenario. If some router along the + path really does not have enough available bandwidth to approve the + Quick-Start request, then Quick-Start packets sent as a result of + the falsely-approved request could be dropped in the network, to the + possible disadvantage of the connection. Thus, while the ingress and egress routers could collude to prevent intermediate routers - from denying a Quick-Start request, it would not necessarily be to - the connection's advantage for this to happen. In addition, the - router between the ingress and egress nodes that denied the request - could be monitoring connection performance, actively penalizing - nodes that seem to be using Quick-Start after a Quick-Start request - was denied, or that are reporting an Approved Rate higher than that - actually approved by that router. + from denying a Quick-Start request, it would not always be to the + connection's advantage for this to happen. One defense against such + a collusion would be for some router between the ingress and egress + nodes that denied the request to monitor connection performance, + penalizing connections that seeem to be using Quick-Start after a + Quick-Start request was denied, or that are reporting an Approved + Rate higher than that actually approved by that router. - If the congested router was ECN-capable, and the colluding ingress - and egress routers were lying about ECN-capability as well as about + If the congested router is ECN-capable, and the colluding ingress + and egress routers are lying about ECN-capability as well as about Quick-Start, then the result could be that the Quick-Start request falsely appears to the sender to have been approved, and the Quick- Start packets falsely appear to the congested router to be ECN- capable. In this case, the colluding routers might succeed in giving a competitive advantage to the traffic protected by their collusion (if no intermediate router is monitoring to catch such misbehavior). 9.5. Misbehaving Middleboxes and the IP TTL One possible difficulty is that of traffic normalizers [HKP01] or - other middleboxes along that path that re-write IP TTLs, in order to + other middleboxes along that path that re-write IP TTLs in order to foil other kinds of attacks in the network. If such a traffic normalizer re-wrote the IP TTL, but did not adjust the Quick-Start TTL by the same amount, then the sender's mechanism for determining if the request was approved by all routers along the path would no longer be reliable. Re-writing the IP TTL could result in false positives (with the sender incorrectly believing that the Quick- Start request was approved) as well as false negatives (with the sender incorrectly believing that the Quick-Start request was denied). 9.6. Attacks on Quick-Start - As discussed in [SAF05], Quick-Start is vulnerable to two kinds of - Quick-Start attacks: (1) attacks to increase the routers' - processing and state load; and (2) attacks with bogus Quick-Start - requests to temporarily tie up available Quick-Start bandwidth, - preventing routers from approving Quick-Start requests from other - connections. Routers can protect against the first kind of attack - by applying a simple limit on the rate at which Quick-Start requests - will be considered by the router. + As discussed in [SAF06], Quick-Start is vulnerable to two kinds of + attacks: (1) attacks to increase the routers' processing and state + load; and (2) attacks with bogus Quick-Start requests to temporarily + tie up available Quick-Start bandwidth, preventing routers from + approving Quick-Start requests from other connections. Routers can + protect against the first kind of attack by applying a simple limit + on the rate at which Quick-Start requests will be considered by the + router. - The second kind of attack, attacks to tie up the available Quick- - Start bandwidth, is more difficult to defend against. As discussed - in [SAF05]. Quick-Start Requests that are not going to be used, - either because they are from malicious attackers or because they are - denied by routers downstream, can result in `wasting' potential + The second kind of attack, to tie up the available Quick-Start + bandwidth, is more difficult to defend against. As discussed in + [SAF06]. Quick-Start Requests that are not going to be used, either + because they are from malicious attackers or because they are denied + by routers downstream, can result in short-term `wasting' potential Quick-Start bandwidth, resulting in routers denying subsequent Quick-Start Requests that if approved would in fact have been used. We note that the likelihood of malicious attacks would be minimized significantly when Quick-Start was deployed in a controlled environment such as an Intranet, where there was some form of centralized control over the users in the system. We also note that this form of attack could potentially make Quick-Start unusable, but it would not do any further damage; in the worst case, the network would function as a network without Quick-Start. - [SAF05] considers the potential of Extreme Quick-Start algorithms at + [SAF06] considers the potential of Extreme Quick-Start algorithms at routers, which keep per-flow state for Quick-Start connections, in protecting the availability of Quick-Start bandwidth in the face of frequent overly-larqe Quick-Start requests. 9.7. Simulations with Quick-Start Quick-Start was added to the NS simulator [SH02] by Srikanth Sundarrajan, and additional functionality was added by Pasi Sarolahti. The validation test is at `test-all-quickstart' in the `tcl/test' directory in NS. The initial simulation studies from [SH02] show a significant performance improvement using Quick-Start for moderate-sized flows (between 4KB and 128KB) in under-utilized environments. These studies are of file transfers, with the improvement measured as the relative increase in the overall throughput for the file transfer. The study shows that potential improvement from Quick-Start is proportional to the delay-bandwidth product of the path. - The Quick-Start simulations in [SAF05] explore the following: the + The Quick-Start simulations in [SAF06] explore the following: the potential benefit of Quick-Start for the connection; the relative benefits of different router-based algorithms for approving Quick- Start requests; and the effectiveness of Quick-Start as a function of the senders' algorithms for choosing the size of the rate request. 10. Implementation and Deployment Issues This section discusses some of the implementation issues with Quick- Start. This section also discusses some of the key deployment @@ -2201,225 +2322,319 @@ Section 4.6 discusses some of the issues with deciding the initial sending rate to request. Quick-Start raises additional issues about the communication between the transport protocol and the application, and about the use of the past history with Quick-Start in the end node. One possibility is that a protocol implementation could provide an API for applications to indicate when they want to request Quick- Start, and what rate they would like to request. In the conventional socket API this could be a socket option that is set - before a connection is established. Some applications, such those - that use TCP for bulk transfers, do not have interest in the + before a connection is established. Some applications, such as + those that use TCP for bulk transfers, do not have interest in the transmission rate, but they might know the amount of data that can be sent immediately. Based on this, the sender implementation could decide whether Quick-Start would be useful, and what rate should be - requested. Datagram-based real-time streaming applications, on the - other hand, may have a specific preference on the transmission rate - and they could indicate the required rate explicitly to the - transport protocol to be used in the Quick-Start Request. + requested. We note that when Quick-Start is used, the TCP sender is required to save the QS Nonce and the TTL Diff when the Quick-Start request is sent, and to implement an additional timer for the paced transmission of Quick-Start packets. 10.2. Implementation Issues for Processing Quick-Start Requests A router or other network host must be able to determine the approximate bandwidth of its outbound network interfaces in order to process incoming Quick-Start rate requests, including those that originate from the host itself. One possibility would be for hosts to rely on configuration information to determine link bandwidths; this has the drawback of not being robust to errors in configuration. Another possibility would be for network device drivers to infer the bandwidth for the interface and to communicate this to the IP layer. Particular issues will arise for wireless links with variable bandwidth, where decisions will have to be made about how frequently - the network host gets updates of the changing bandwidth. It seems + the host gets updates of the changing bandwidth. It seems appropriate that Quick-Start Requests would be handled particularly conservatively for links with variable bandwidth, to avoid cases where Quick-Start Requests are approved, the link bandwidth is reduced, and the data packets that are sent end up being dropped. Difficult issues also arise for paths with multi-access links (e.g., - Ethernet). Routers with multi-access links should be particularly - conservative in granting Quick-Start requests. + Ethernet). Routers or end-nodes with multi-access links should be + particularly conservative in granting Quick-Start requests. In + particular, for some multi-access links there may be no procedure + for an attached node to use to determine whether all parts of the + multi-access link have been underutilized in the recent past. 10.3. Possible Deployment Scenarios Because of possible problems discussed above concerning using Quick- - Start over some network paths, the most realistic initial deployment - of Quick-Start would most likely take place in Intranets and other - controlled environments. Quick-Start is most useful on high - bandwidth-delay paths that are significantly underutilized. The - primary initial users of Quick-Start would likely be in - organizations that provide network services to their users and also - have control over a large portion of the network path. + Start over some network paths and the security issues discussed in + section 11 , the most realistic initial deployment of Quick-Start + would most likely take place in Intranets and other controlled + environments. Quick-Start is most useful on high bandwidth-delay + paths that are significantly underutilized. The primary initial + users of Quick-Start would likely be in organizations that provide + network services to their users and also have control over a large + portion of the network path. + + Quick-Start is not currently intended for ubiquitous deployment in + the global Internet. In particular, Quick-Start should not be + enabled by default in end-nodes or in routers; instead, when Quick- + Start is used, it should be explicitly enabled by users or system + administrators. Below are a few examples of networking environments where Quick- Start would potentially be useful. These are the environments that might consider an initial deployment of Quick-Start in the routers and end-nodes, where the incentives for routers to deploy Quick- Start might be the most clear. * Centrally-administrated organizational Intranets: These intranets often have large network capacity, with networks that are - underutilized for much of the time. Such Intranets might also - include high-bandwidth and high-delay paths to remote sites. In - such an environment, Quick-Start would be of benefit to users, and - there would be a clear incentive for the deployment of Quick-Start - in routers. For example, Quick-Start could be quite useful in high- - bandwidth networks used for scientific computing. + underutilized for much of the time [PABL+05]. Such Intranets might + also include high-bandwidth and high-delay paths to remote sites. + In such an environment, Quick-Start would be of benefit to users, + and there would be a clear incentive for the deployment of Quick- + Start in routers. For example, Quick-Start could be quite useful in + high-bandwidth networks used for scientific computing. - * GPRS: Quick-Start could also be useful in high-delay environments - of Cellular Wide-Area Wireless Networks such as the GPRS [BW97] and - their enhancements and next generations. For example, GPRS EDGE - (Enhanced Data for GSM Evolution) is expected to provide wireless - bandwidth of up to 384 Kbps (roughly 32 1500-byte packets per - second) while the GPRS round-trip times range typically from few - hundred milliseconds to over a second excluding any possible - queueing delays in the network [GPAR02]. In addition, these networks - sometimes have variable additional delays due to resource allocation - that could be avoided by keeping the connection path constantly - utilized, starting from initial slow-start. Thus, Quick-Start could - be of significant benefit to users in these environments. + * Wireless networks: Quick-Start could also be useful in high-delay + environments of Cellular Wide-Area Wireless Networks such as the + GPRS [BW97] and their enhancements and next generations. For + example, GPRS EDGE (Enhanced Data for GSM Evolution) is expected to + provide wireless bandwidth of up to 384 Kbps (roughly 32 1500-byte + packets per second) while the GPRS round-trip times range typically + from few hundred milliseconds to over a second excluding any + possible queueing delays in the network [GPAR02]. In addition, these + networks sometimes have variable additional delays due to resource + allocation that could be avoided by keeping the connection path + constantly utilized, starting from initial slow-start. Thus, Quick- + Start could be of significant benefit to users in these + environments. * Paths over satellite links: Geostationary Orbit (GEO) satellite links have one-way propagation delays on the order of 250 ms while the bandwidth can be measured in megabits per second [RFC2488]. Because of the considerable bandwidth-delay product on the link, TCP's slow-start is a major performance limitation in the beginning of the connection. A large initial congestion window would be useful to users of such satellite links. -10.4. Would QuickStart packets take the slow path in routers? - - How much delay would the slow path add to the processing time for - Quick-Start request packets? In addition, if QuickStart request - packets took the slow path, this could add stress to routers, though - routers could always rate-limit the number of QuickStart request - packets they are willing to consider. + * Single-hop paths: Quick-Start should work well over point-to-point + single-hop paths, e.g., from a host to an adjacent server. Quick- + Start would work over a single-hop IP path consisting of a multi- + access link only if the host was able to determine if the path to + the next IP hop has been significantly underutilized over the recent + past. If the multi-access link includes a layer-2 switch, then the + attached host cannot necessarily determine the status of the other + links in the layer-2 network. -10.5. A Comparison with the Deployment Problems of ECN +10.4. A Comparison with the Deployment Problems of ECN Given the glacially slow rate of deployment of ECN in the Internet to date [MAF05], it is disconcerting to note that some of the deployment problems of Quick-Start are even greater than those of ECN. First, unlike ECN, which can be of benefit even if it is only deployed on one of the routers along the end-to-end path, a - connection's use of Quick-Start requires its deployment on all of - the routers along the end-to-end path. Second, unlike ECN, which - uses an allocated field in the IP header, Quick-Start requires the - extra complications of an IP Option. + connection's use of Quick-Start requires Quick-Start deployment on + all of the routers along the end-to-end path. Second, unlike ECN, + which uses an allocated field in the IP header, Quick-Start requires + the extra complications of an IP Option, which can be difficult to + pass through the current Internet [MAF05]. However, in spite of these issues, there is some hope for the deployment of Quick-Start, at least in protected corners of the Internet, because the potential benefits of Quick-Start to the user are considerably more dramatic than those of ECN. Rather than simply replacing the occasional dropped packet by an ECN-marked packet, Quick-Start is capable of dramatically increasing the - throughput of connections in underutilized environments. + throughput of connections in underutilized environments [SAF06]. -11. Related Work +11. Security Considerations + + Sections 9.4 and 9.6 discuss the security considerations related to + Quick-Start. Section 9.4 discusses the potential abuse of Quick- + Start by senders or receivers lying about whether the request was + approved or about the approved rate, and of routers in collusion to + misuse Quick-Start. Section 9.5 discusses potential problems with + traffic normalizers that rewrite IP TTLs in packet headers. All of + these problems could result in the sender using a Rate Request that + was inappropriately large, or thinking that a request was approved + when it was in fact denied by at least one router along the path. + This inappropriate use of Quick-Start could result in congestion and + an unacceptable level of packet drops along the path, Such + congestion could also be part of a Denial of Service attack. + + Section 9.6 discusses a potential attack on the routers' processing + and state load from an attack of Quick-Start Requests. Section 9.6 + also discusses a potential attack on the available Quick-Start + bandwidth by sending bogus Quick-Start requests for bandwidth that + will not in fact be used. While this impacts the global usability + of Quick-Start it does not endanger the network as a whole since TCP + uses standard congestion control if Quick-Start is not available. + + Section 4.6.2 discusses the potential problem of packets with Quick- + Start Requests dropped by middleboxes along the path. + + As discussed in Section 5, for IPv4 IPsec Authentication Header + Integrity Check Value (AH ICV) calculation, the Quick-Start option + MUST be treated as a mutable IPv4 option, and hence completely + zeroed for AH ICV calculation purposes; this is also the treatment + required by RFC 2402 for unrecognized IPv4 options. The IPv6 Quick- + Start option's IANA-allocated option type indicates that it is a + mutable option, hence, according to RFC 2402, its option data MUST + be zeroed for AH ICV computation purposes. See RFC 2402 for further + explanation. + + Section 6.2 discusses possible problems of Quick-Start used by + connections carried over simple tunnels that are not compatible with + Quick-Start. In this case it is possible that a Quick-Start + Request is erroneously considered approved by the sender without the + routers in the tunnel having individually approved the request, + causing a false positive. + + We note two high-order points here. First, the Quick-Start Nonce + goes a long way towards preventing large scale cheating. And, + second, even if a host occasionally uses Quick-Start when it is not + approved by the entire network path the network will not collapse. + Quick-Start does not remove TCP's basic congestion control + mechanisms and these will kick in when the network is heavily + loaded, relegating any Quick-Start mistake to a transient. + +12. IANA Considerations + + Quick-Start requires an IP Option and a TCP Option. + +12.1. IP Option + + Quick-Start requires that both an IPv4 and an IPv6 Option Number be + allocated. The IPv4 Option would have a copied flag of 0, a class + field of 00, and the assigned five-bit option number. The IPv6 + Option would have the first three bits of "001" [RFC 2460, Section + 4.2], with the first two bits indicating that the IPv6 node skip + over this option and continue processing the header if it doesn't + recognize the option type, and the third bit indicating that the + Option Data may change en-route. + + In both cases the name of the option would be "QS - Quick-Start", + with this document as the reference document. + +12.2. TCP Option + + Quick-Start also requires that a TCP Option Number be allocated. + The Length would be 4, and the Meaning would be "Quick-Start + Request", with this document as the reference document. + +13. Related Work The Quick-Start proposal, taken together with HighSpeed TCP [RFC3649] or other transport protocols for high-bandwidth transfers, could go a significant way towards extending the range of performance for best-effort traffic in the Internet. However, there are many things that the Quick-Start proposal would not accomplish. Quick-Start is not a congestion control mechanism, and would not help in making more precise use of the available bandwidth, that is, of achieving the goal of high throughput with low delay and low packet loss rates. Quick-Start would not give routers more control over the decrease rates of active connections. In addition, any evaluation of Quick-Start must include a discussion of the relative benefits of approaches that use no explicit information from routers, and of approaches that use more fine- grained feedback from routers as part of a larger congestion control - mechanism. We discuss several classes of proposals (no explicit - feedback from routers; explicit feedback about the initial rate; - more fine-grained feedback from routers; and proposals based on - lower-than-best-effort service) in the sections below. + mechanism. We discuss several classes of proposals in the sections + below. -11.1. Fast Start-ups without Explicit Information from Routers +13.1. Fast Start-ups without Explicit Information from Routers One possibility would be for senders to use information from the packet streams to learn about the available bandwidth, without explicit information from routers. These techniques would not allow a start-up as fast as that available from Quick-Start in an underutilized environment; one has to have sent some packets already to use the packet stream to learn about available bandwidth. However, these techniques could allow a start-up considerably faster than the current slow-start. While it seems clear that approaches *without* explicit feedback from the routers will be strictly less powerful that is possible *with* explicit feedback, it is also possible that approaches that are more aggressive than slow-start - are possible without explicit feedback from routers. + are possible without the complexity involved in obtaining explicit + feedback from routers. Periodic packet streams: [JD02] explores the use of periodic packet streams to estimate the available bandwidth along a path. The idea is that the one-way delays of a periodic packet stream show an increasing trend when the - stream's rate is higher than the available bandwidth. While [JD02] - states that the proposed mechanism does not cause significant - increases in network utilization, losses, or delays when done by one - flow at a time, the approach could be problematic if conducted - concurrently by a number of flows. [JD02] also gives an overview of - some of the earlier work on inferring the available bandwidth from - packet trains. + stream's rate is higher than the available bandwidth (due to an + increasing queue). While [JD02] states that the proposed mechanism + does not cause significant increases in network utilization, losses, + or delays when done by one flow at a time, the approach could be + problematic if conducted concurrently by a number of flows. [JD02] + also gives an overview of some of the earlier work on inferring the + available bandwidth from packet trains. Swift-Start: The Swift Start proposal from [PRAKS02] combines packet-pair and packet-pacing techniques. An initial congestion window of four segments is used to estimate the available bandwidth along the path. This estimate is then used to dramatically increase the congestion window during the second RTT of data transmission. SPAND: In the TCP/SPAND proposal from [ZQK00] for speeding up short data transfers, network performance information would be shared among many co-located hosts to estimate each connection's fair share of the network resources. Based on such estimation and the transfer size, the TCP sender would determine the optimal initial congestion window size. The design for TCP/SPAND uses a performance gateway that monitors all traffic entering and leaving an organization's network. + Sharing information among TCP connections: + The Congestion Manager [RFC3124] and TCP control block sharing + [RFC2140] both propose sharing congestion information among multiple + TCP connections with the same endpoints. With the Congestion + Manager, a new TCP connection could start with a high initial cwnd + if it was sharing the path and the cwnd with a pre-existing TCP + connection to the same destination that had already obtained a high + congestion window. RFC 2140 discusses ensemble sharing, where an + established connection's congestion window could be `divided up' to + be shared with a new connection to the same host. However, neither + of these approaches addresses the case of a connection to a new + destination, with no existing or recent connection (and therefore + congestion control state) to that destination. + While continued research on the limits of the ability of TCP and other transport protocols to learn of available bandwidth without explicit feedback from the router seems useful, we note that there are several fundamental advantages of explicit feedback from routers. (1) Explicit feedback is faster than implicit feedback: One advantage of explicit feedback from the routers is that it allows the transport sender to reliably learn of available bandwidth in one round-trip time. (2) Explicit feedback is more reliable than implicit feedback: - A second advantage of explicit feedback from the routers is that the - available bandwidth along the path does not necessarily map to the - allowed sending rate for an individual flow. As an example, if the - TCP sender sends four packets back-to-back in the initial window, - and the TCP receiver reports that the data packets were received - with roughly the same spacing as they were transmitted, does this - mean that the flow can infer an underutilized path? And how fast - can the flow send in the next round-trip time? Do the results - depend on the level of statistical multiplexing at the congested - link, and on the number of flows attempting a faster start-up at the - same time? + Techniques that attempt to assess the available bandwidth at + connection startup using implicit techniques are more error-prone + than techniques that involve every element in the network path. + While explicit information from the network can be wrong, it has a + much better chance of being appropriate than an end-host trying to + *estimate* an appropriate sending rate using "block box" probing + techniques of the entire path. -11.2. Optimistic Sending without Explicit Information from Routers +13.2. Optimistic Sending without Explicit Information from Routers Another possibility that has been suggested [S02] is for the sender to start with a large initial window without explicit permission from the routers and without bandwidth estimation techniques, and for the first packet of the initial window to contain information such as the size or sending rate of the initial window. The proposal would be that congested routers would use this information in the first data packet to drop or delay many or all of the packets from that initial window. In this way a flow's optimistically-large initial window would not force the router to drop packets from @@ -2436,30 +2651,30 @@ deployment, where some of the routers along the path might not understand the packet information describing the initial window. (2) Congestion collapse: There could also be concerns about congestion collapse if many flows used large initial windows, many packets were dropped from optimistic initial windows, and many congested links ended up carrying packets that are only going to be dropped downstream. (3) Distributed Denial of Service attacks: - - A third key question would be the potential role of optimistic - senders in amplifying the damage done by a Distributed Denial of - Service (DDoS) attack. + A third question would be the potential role of optimistic senders + in amplifying the damage done by a Distributed Denial of Service + (DDoS) attack (assuming attackers use conformant congestion control + in the hopes of "flying under the radar"). (4) Performance hits if a packet is dropped: A fourth issue would be to quantify the performance hit to the connection when a packet is dropped from one of the initial windows. -11.3. Fast Start-ups with other Information from Routers +13.3. Fast Start-ups with other Information from Routers There have been several proposals somewhat similar to Quick-Start, where the transport protocol collects explicit information from the routers along the path. An IP Option about the free buffer size: In related work, [P00] investigates the use of a slightly different IP option for TCP connections to discover the available bandwidth along the path. In that proposal, the IP option would query the routers along the path about the smallest available free buffer @@ -2469,25 +2684,27 @@ The Performance Transparency Protocol: The Performance Transparency Protocol (PTP) includes a proposal for a single PTP packet that would collect information from routers along the path from the sender to the receiver [W00]. For example, a single PTP packet could be used to determine the bottleneck bandwidth along a path. ETEN: Additional proposals for end nodes to collect explicit information - from routers include Explicit Transport Error Notification (ETEN), - which includes a cumulative mechanism to notify endpoints of - aggregate congestion statistics along the path [KAPS02]. + from routers include one variant of Explicit Transport Error + Notification (ETEN), which includes a cumulative mechanism to notify + endpoints of aggregate congestion statistics along the path + [KAPS02]. (A second variant in [KSEPA04] does not depend on + cumulative congestion statistics from the network.) -11.4. Fast Start-ups with more Fine-Grained Feedback from Routers +13.4. Fast Start-ups with more Fine-Grained Feedback from Routers Proposals for more fine-grained congestion-related feedback from routers include XCP [KHR02], MaxNet [MaxNet], and AntiECN marking [K03]. Section A.6 discusses in more detail the relationship between Quick-Start and proposals for more fine-grained per-packet feedback from routers. XCP: Proposals such as XCP for new congestion control mechanisms based on more feedback from routers are more powerful than Quick-Start, but @@ -2485,42 +2702,47 @@ Proposals for more fine-grained congestion-related feedback from routers include XCP [KHR02], MaxNet [MaxNet], and AntiECN marking [K03]. Section A.6 discusses in more detail the relationship between Quick-Start and proposals for more fine-grained per-packet feedback from routers. XCP: Proposals such as XCP for new congestion control mechanisms based on more feedback from routers are more powerful than Quick-Start, but also are more complex to understand and more difficult to deploy. + XCP routers maintain no per-flow state, but provide more fine- grained feedback to end-nodes than the one-bit congestion feedback of ECN. The per-packet feedback from XCP can be positive or negative, and specifies the increase or decrease in the sender's - congestion window when this packet is acknowledged. + congestion window when this packet is acknowledged. XCP is a full- + fledge congestion control scheme, whereas Quick-Start represents a + quick check to determine if the network path is significantly + underutilized such that a connection can start faster and then fall + back to TCP's standard congestion control algorithms. AntiECN: The AntiECN proposal is for a single bit in the packet header that routers could set to indicate that they are underutilized. For each TCP ACK arriving at the sender indicating that a packet has been received with the Anti-ECN bit set, the sender would be able to increase its congestion window by one packet, as it would during slow-start. -11.5. Fast Start-ups with Lower-Than-Best-Effort Service +13.5. Fast Start-ups with Lower-Than-Best-Effort Service There have been proposals for routers to provide a Lower Effort differentiated service that would be lower than best effort [RFC3662]. Such a service could carry traffic for which delivery is strictly optional, or could carry traffic that is important but that has low priority in terms of time. Because it does not interfere - with best-effort traffic, Lower Effort services would be used by + with best-effort traffic, Lower Effort services could be used by transport protocols that start-up faster than slow-start. For example, [SGF05] is a proposal for the transport sender to use low- priority traffic for much of the initial traffic, with routers configured to use strict priority queueing. A separate but related issue is that of below-best-effort TCP, variants of TCP that would not rely on Lower Effort services in the network, but would approximate below-best-effort traffic by detecting and responding to congestion sooner that standard TCP. TCP Nice [V02] and TCP Low Priority (TCP-LP) [KK03] are two such @@ -2528,76 +2750,35 @@ TCP connections to use the bandwidth unused by TCP and other traffic in a non-intrusive fashion. Both TCP Nice and TCP Low Priority use the default slow-start mechanisms of TCP. We note that Quick-Start is quite different from either a Lower Effort service or a below-best-effort variant of TCP. Unlike these proposals, Quick-Start is intended to be useful for best-effort traffic that wishes to receive at least as much bandwidth as competing best-effort connections. -12. Security Considerations - - Sections 9.4 and 9.6 discuss the security considerations related to - Quick-Start. Section 9.4 discusses the potential abuse of Quick- - Start by senders or receivers lying about whether the request was - approved or about the approved rate, and of routers in collusion to - misuse Quick-Start. Section 9.5 discusses potential problems with - traffic normalizers that rewrite IP TTLs in packet headers. All of - these problems could result in the sender using a Rate Request that - was inappropriately large, or thinking that a request was approved - when it was in fact denied by at least one router along the path. - This inappropriate use of Quick-Start would result in congestion and - an unacceptable level of packet drops along the path, Such - congestion could also be part of a Denial of Service attack. - - Section 9.6 discusses a potential attack on the routers' processing - and state load from an attack of Quick-Start Requests. Section 9.6 - also discusses a potential attack on the available Quick-Start - bandwidth by sending bogus Quick-Start requests for bandwidth that - will not in fact be used. - - Section 4.6.2 discusses the potential problem of packets with Quick- - Start Requests dropped by middleboxes along the path. - - As discussed in Section 5, for IPv4 IPsec Authentication Header - Integrity Check Value (AH ICV) calculation, the Quick-Start option - MUST be treated as a mutable IPv4 option, and hence completely - zeroed for AH ICV calculation purposes; this is also the treatment - required by RFC 2402 for unrecognized IPv4 options. The IPv6 Quick- - Start option's IANA-allocated option type indicates that it is a - mutable option, hence, according to RFC 2402, its option data MUST - be zeroed for AH ICV computation purposes. See RFC 2402 for further - explanation. - - Section 6.2 discusses possible problems of Quick-Start used by - connections carried over simple tunnels that are not compatible with - Quick-Start. In this case it is possible that a Quick-Start - Request is erroneously considered approved by the sender without the - routers in the tunnel having individually approved the request, - causing a false positive. - -13. Conclusions +14. Conclusions We are presenting the Quick-Start mechanism as a simple, understandable, and incrementally-deployable mechanism that would be sufficient to allow some connections to start up with large initial rates, or large initial congestion windows, in overprovisioned, high-bandwidth environments. We expect there will be an increasing number of overprovisioned, high-bandwidth environments where the Quick-Start mechanism, or another mechanism of similar power, could be of significant benefit to a wide range of traffic. We are presenting the Quick-Start mechanism as a request for the community to provide feedback and experimentation on issues relating to Quick- Start. -14. Acknowledgements +15. Acknowledgements The authors wish to thank Mark Handley for discussions of these issues. The authors also thank the End-to-End Research Group, the Transport Services Working Group, and members of IPAM's program on Large Scale Communication Networks for both positive and negative feedback on this proposal. We thank Srikanth Sundarrajan for the initial implementation of Quick-Start in the NS simulator, and for the initial simulation study. Many thanks to David Black and Joe Touch for extensive feedback on QuickStart and IP tunnels. We also thank Mohammed Ashraf, John Border, Bob Briscoe, Martin Duke, Tom @@ -2914,65 +3095,61 @@ Start failed in finding an appropriate congestion window, meaning that the congestion window after halving may easily also be wrong. A more moderate alternate would be to continue in congestion avoidance from a window of (W-D)/2, where W is the Quick-Start congestion window, and D is the number of packets dropped or marked from that window. However, such an approach would implicitly assume that the number of Quick-Start packets delivered is a good indication of the appropriate available bandwidth for that flow, even though other packets from that window were dropped in the - network. We believe that such an assumption would require more - analysis at this point, particularly in a network with a range of - packet dropping mechanisms at the router, and we cannot recommend it - at this time. + network. And, further, that using half the number of segments that + were successfully transmitted is conservative enough to account for + the possibly inaccurate congestion window indication. We believe + that such an assumption would require more analysis at this point, + particularly in a network with a range of packet dropping mechanisms + at the router, and we cannot recommend it at this time. Another drawback of approaches that don't revert back to slow-start when a Quick-Start packet in the initial window is dropped is that - any such approaches could give the TCP receiver an incentive to lie - about the Quick-Start request. That is, if the sender reverts to - slow-start when a Quick-Start packet is dropped, then it is - generally not to the receiver's advantage to report a larger rate - request than was actually approved if the result is going to be a - Quick-Start packet dropped in the network. However, if the receiver - benefits from a larger Quick-Start window even when the larger - window results in Quick-Start packets dropped in the network, then - the receiver has a greater incentive to lie about the received rate - request, in an effort to get the sender to use a larger initial - sending rate. + such approaches could give the TCP receiver a greater incentive to + lie about the Quick-Start request. If the sender reverts to slow- + start when a Quick-Start packet in the initial window is dropped, + this diminishes the benefit a receiver would get from a Quick-Start + request that resulted in a dropped or ECN-marked packet. A.6. Why Not Include More Functionality? This proposal for Quick-Start is a rather coarse-grained mechanism - that would allow connections to use higher sending rates along + that would allow a connection to use a higher sending rate along underutilized paths, but that does not attempt to provide a next- - generation transport protocol, and does not attempt the goal of - providing very high throughput with very low delay. As Section 11.4 - discusses, there are a number of proposals such as XCP, MaxNet, and - AntiECN for more fine-grained per-packet feedback from routers than - the current congestion control mechanisms, that do attempt these - more ambitious goals. + generation transport protocol or congestion control mechanism, and + does not attempt the goal of providing very high throughput with + very low delay. As Section 13.4 discusses, there are a number of + proposals such as XCP, MaxNet, and AntiECN for more fine-grained + per-packet feedback from routers than the current congestion control + mechanisms, that do attempt these more ambitious goals. Compared to proposals such as XCP and AntiECN, Quick-Start offers much less control. Quick-Start does not attempt to provide a new congestion control mechanism, but simply to get permission from routers for a higher sending rate at start-up, or after an idle period. Quick-Start can be thought of as an "anti-congestion- control" mechanism, that is only of any use when all of the routers along the path are significantly under-utilized. Thus, Quick-Start is of no use towards a target of high link utilization, or fairness in a high-utilization scenario, or controlling queueing delay during high-utilization, or anything of the like. At the same time, Quick-Start would allow larger initial windows than would proposals such as AntiECN, requires less input to routers - than XCP (e.g., XCP's cwnd and rtt fields), and would require less + than XCP (e.g., XCP's cwnd and RTT fields), and would require less frequent feedback from routers than any new congestion control mechanism. Thus, Quick-Start is significantly less powerful than proposals for new congestion control mechanisms such as XCP and AntiECN, but as powerful or more powerful in terms of the specific issue of allowing larger initial windows, and (we think) more amenable to incremental deployment in the current Internet. We do not discuss proposals such as XCP in detail, but simply note that there are a number of open questions. One question concerns whether there is a pressing need for more sophisticated congestion @@ -3004,21 +3181,22 @@ window to take effect when that particular packet is acknowledged, not about the allowed sending rate for the connection as a whole. In contrast, Quick-Start sends a single Quick-Start request, and the answer to that request gives the allowed sending rate for an entire window of data. As a result, Quick-Start could be problematic in an environment where some fraction of the packets in a window of data take path A, and the rest of the packets take path B; for example, the Quick-Start Request could have travelled on path A, while half of the Quick-Start packets sent in the succeeding round-trip time - are routed on path B. + are routed on path B. We note that [ZDPS01] shows Internet paths to + be stable on the order of RTTs. There are also differences between Quick-Start and some of the proposals for per-packet feedback in terms of the number of bits of feedback required from the routers to the end-nodes. Quick-Start uses four bits of feedback in the rate request field to indicate the allowed sending rate. XCP allocates a byte for per-packet feedback, though there has been discussion of variants of XCP with less per- packet feedback. This would be more like other proposals such as anti-ECN that use a single bit of feedback from routers to indicate that the sender can increase as fast as slow-starting, in response @@ -3161,131 +3339,124 @@ [KHF05] for a more detailed description of DCCP, and of the congestion control mechanisms. Because CCID 3 uses a rate-based congestion control mechanism, it raises some new issues about the use of Quick-Start with transport protocols. In this document we don't attempt to specify the use of Quick-Start with DCCP. However, we do discuss some of the issues that might arise. In considering the use of Quick-Start with CCID 3 for requesting a - higher initial sending rate, the following questions arise: (1) how - does the sender respond if a Quick-Start packet is dropped; and (2) - when does the sender determine that there has been no feedback from - the receiver, and reduce the sending rate? + higher initial sending rate, the following questions arise: - (1) How does the sender respond if a Quick-Start packet is dropped: + (1) How does the sender respond if a Quick-Start packet is dropped? As in TCP, if an initial Quick-Start packet is dropped, the CCID 3 sender should revert to the congestion control mechanisms it would have used if the Quick-Start request had not been approved. (2) When does the sender decide there has been no feedback from the - receiver: + receiver? Unlike TCP, CCID 3 does not use acknowledgements for every packet, or for every other packet. In contrast, the CCID 3 receiver sends feedback to the sender roughly once per round-trip time. In CCID 3, the allowed sending rate is halved if no feedback is received from the receiver in at least four round-trip times (when the sender is sending at least one packet every two round-trip times). When a Quick-Start request is used, it would seem necessary to use a smaller time interval, e.g., to reduce the sending rate if no - feedback is received from the receiver in at least two round-trip - times. + feedback arrives from the receiver in at least two round-trip times. The question also arises of how the sending rate should be reduced after a period of no feedback from the receiver. As with TCP, the default CCID 3 response of halving the sending rate is not necessarily a sufficient response to the absence of feedback; an alternative is to reduce the sending rate to the sending rate that would have been used if no Quick-Start request had been approved. That is, if a CCID 3 sender uses a Quick-Start request, special rules might be required to handle the sender's response to a period of no feedback from the receiver regarding the Quick-Start packets. Similarly, in considering the use of Quick-Start with CCID 3 for requesting a higher sending rate after an idle period, the following - questions arise: (1) what rate does the sender request; (2) what is - the response to a packet loss; and (3) when does the sender - determine that there has been no feedback from the receiver, and the - sending rate must be reduced? + questions arise: - (1) What rate does the sender request: + (1) What rate does the sender request? As in TCP, there is a straightforward answer to the rate request that the CCID 3 sender should use in requesting a higher sending rate after an idle period. The sender knows the current loss event rate, either from its own calculations or from feedback from the receiver, and can determine the sending rate allowed by that loss event rate. This is the upper bound on the sending rate that should be requested by the CCID 3 sender. A Quick-Start request is useful with CCID 3 when the sender is coming out of an idle or underutilized period, because in standard operation CCID 3 does not allow the sender to send more than twice as fast as the receiver has reported received in the most recent feedback message. - (2) What is the response to loss: + (2) What is the response to loss? The response to the loss of Quick-Start packets should be to return to the sending rate that would have been used if Quick-Start had not been requested. (3) When does the sender decide there has been no feedback from the - receiver: + receiver? As in the case of the initial sending rate, it would seem prudent to reduce the sending rate if no feedback is received from the receiver in at least two round-trip times. It seems likely that in this case, the sending rate should be reduced to the sending rate that would have been used if no Quick-Start request had been approved. C. Possible Router Algorithm This specification does not tightly define the algorithm a router uses when deciding whether to approve a Quick-Start Rate Request or whether and how to reduce a Rate Request. A range of algorithms is likely useful in this space and we consider the algorithm a particular router uses to be a local policy decision. In addition, we believe that additional experimentation with router algorithms is necessary to have a solid understanding of the dynamics various algorithms impose. However, we provide one particular algorithm in this appendix as an example and as a framework for thinking about additional mechanisms. - [SAF05] provides several algorithms routers can use to consider + [SAF06] provides several algorithms routers can use to consider incoming Rate Requests. The decision process involves two algorithms. First, the router needs to track the link utilization over the recent past. Second, this utilization needs to be updated by the potential new bandwidth from recent Quick-Start approvals, and then compared with the router's notion of when it is underutilized enough to approve Quick-Start requests (of some size). First, we define the "peak utilization" estimation technique (from - [SAF05]). This mechanism records the utilization of the link every + [SAF06]). This mechanism records the utilization of the link every S seconds and stores the most recent N of these measurements. The utilization is then taken as the highest utilization of the N samples. This method, therefore, keeps N*S seconds of history. This algorithm reacts rapidly to increases in the link utilization. - In [SAF05] S is set to 0.15 seconds, and experiments use values for + In [SAF06] S is set to 0.15 seconds, and experiments use values for N ranging from 3 to 20. Second, we define the "target" algorithm for processing incoming - Quick-Start Rate Requests (also from [SAF05]). The algorithm relies + Quick-Start Rate Requests (also from [SAF06]). The algorithm relies on knowing the bandwidth of the outgoing link (which in many cases can be easily configured), the utilization of the outgoing link (from an estimation technique such as given above) and an estimate of the potential bandwidth from recent Quick-Start approvals. Tracking the potential bandwidth from recent Quick-Start approvals is another case where local policy dictates how it should be done. The simpliest method, outlined in Section 8.2, is for the router to keep track of the aggregate Quick-Start rate requests approved in the most recent two or more time intervals, including the current time interval, and to use the sum of the aggregate rate requests over these time intervals as the estimate of the approved Rate - Requests. The experiments in [SAF05] keep track of the aggregate + Requests. The experiments in [SAF06] keep track of the aggregate approved Rate Requests over the most recent two time intervals, for intervals of 150~msec. The target algorithm also depends on a threshold (qs_thresh) that is the fraction of the outgoing link bandwidth that represents the router's notion of "significantly underutilized". If the utilization, augmented by the potential bandwidth from recent Quick- Start approvals, is above this threshold then no Quick-Start Rate Requests will be approved. If the utilization, again augmented by the potential bandwidth from recent Quick-Start approvals, is less @@ -3348,37 +3519,44 @@ Informational Request: An Informational Request codepoint in the Function field would indicate that a request is purely informational, for the sender to find out if a rate request would be approved, and what size rate request would be approved, when the Informational Request is sent. For example, an Informational Request could be followed one round-trip time later by a standard Quick-Start Request. A router approving an Informational Request would not consider this as an approval for Quick-Start bandwidth to be used, and would not be under any obligation to approve a similar - standard Quick-Start Request one round-trip time later. + standard Quick-Start Request one round-trip time later. An + Informational Request with a rate request of zero could be used by + the sender to find out if all of the routers along the path + supported Quick-Start. Use Format X for the Rate Request Field: A Quick-Start codepoint for `Use Format X for the Rate Request Field' would indicate that an alternate format was being used for the Rate Request field. Do Not Decrement: A Do Not Decrement codepoint could be used for a Quick-Start request where the sender would rather have the request denied than to have the rate request decremented in the network. This could be used if the sender was only interested in using Quick- Start if the original rate request was approved. + Temporary Bandwidth Use: A Temporary codepoint has been proposed to + indicate that a connection would only use the requested bandwidth + for a single time interval. + E. Feedback from Bob Briscoe - [B05] in a review of an earlier version of the Quick-Start proposal - discussed a number of potential problems with Quick-Start, and - argued for an alternate approach for policing congestion in networks + [B05] is a review of an earlier version of the Quick-Start proposal + that discusses a number of potential problems with Quick-Start, and + argues for an alternate approach for policing congestion in networks using re-feedback [BJCG+05,BJS05]. However, [B05] didn't oppose the move to Quick-Start to experimental status as long as its applicability is made clear. E.1. Potential Deployment Scenarios [B05] argues that because the sender's trustworthiness is not necessarily verified, Quick-Start, if it is remain stateless, should be confined to environments where every source is trusted. Section 3.2 of [B05] argues that either (1) Quick-Start should be @@ -3428,21 +3606,21 @@ used, due either to malicious requests or due to requests denied downstream). In Section 3.3, [B05] says that "the lack of a secure binding between a request and subsequent traffic means that any other node could send a burst of traffic and claim it requests it, with no-one being able to prove it didn't." In the current Internet, any node can send a burst of traffic any time it would like, even without Quick-Start. However, in the absense of Quick-Start, sending at a high rate is not always in the sender's interest, as the packets - that are send might have a high probability of being dropped in the + that are sent might have a high probability of being dropped in the network, particularly in the absense of Quick-Start. The addition of the Report of Approved Rate to Quick-Start gives traffic policers the ability to check on some of the potential abuses of Quick-Start, if they so desire. In Section 3.8, [B05] states that "not only do Quick-Start senders have to be trusted, but also other senders who could claim their data had been authorised by a Quick-Start response when it hadn't (Section 3.3)." In response, we would again clarify that with Quick- Start, the Quick-Start request makes no difference in how the @@ -3457,38 +3635,36 @@ the current under-utilized state of the router. We have clarified this in Section 3.3 of this document. E.3. Fairness In its abstract, [B05] says the following: "Because traffic variance will always blur the boundary, we argue that under-utilisation should be treated as the extreme of a spectrum where fairness is always an issue to some extent." - The specification for Quick-Start now says the following: "A router - should only approve a Quick-Start request if the output link is - underutilized, and if the router judges that the output link will - continue to be underutilized if the request is approved." - + The specification for Quick-Start now says in section 2 the + following: "A router should only approve a Quick-Start request if + the output link is underutilized, and if the router judges that the + output link will continue to be underutilized if the request is + approved." We changed the quote "for a mechanism for requesting an initial sending rate in an underutilized environment, the fairness issues of a general congestion control mechanism go away" to the following: "because Quick-Start is a mechanism for requesting an initial sending rate in an underutilized environment, its fairness issues are less severe than those of a general congestion control - mechanism." - - However, we did not add the sentence recommended in Ssection 3.4 of - [B05], that "Quick-Start is targeted at an experimental environment - where the more intractable issues can be set aside". In particular, - we don't agree that Quick-Start needs to be targeted only for - environments where fairness is not an issue. + mechanism" in section A.6 However, we did not add the sentence + recommended in section 3.4 of [B05], that "Quick-Start is targeted + at an experimental environment where the more intractable issues can + be set aside". In particular, we don't agree that Quick-Start needs + to be targeted only for environments where fairness is not an issue. E.4. Models of Under-Utilization [B05] states that "One of the under-utilisation assumptions I had in my head while reading the paper was that any one host is generally able to over-fill available capacity, but that, given a high rate, the flow would end quickly." We are sorry that this is the model that the author inferred from the draft, but we can give assurances that this `one big flow at a time" model was *never* the implicit or explicit model underlying the Quick-Start design. (We would also @@ -3515,58 +3691,61 @@ accepting or rejecting Quick-Start requests. Our approach is that, while the Quick-Start IETF documentation standardizes the semantics of Quick-Start and the format of the Quick-Start IP Option and the Quick-Start Response TCP Option, the IETF documentation should not and does not standardize the algorithms used at routers for approving or denying Quick-Start request. Appendix D in this document has presented one possible router algorithm for approving or denying Quick-Start requests, but further discussion of the range of possibilities in router - algorithms is available elsewhere [SAF05]. As an example, the + algorithms is available elsewhere [SAF06]. As an example, the fairness criteria that routers might apply in granting or denying - Quick-Start requests are discussed in [SAF05], but are not discussed + Quick-Start requests are discussed in [SAF06], but are not discussed in the same detail in this document. This document leaves routers free to apply any criteria they choose in accepting or denying Quick-Start requests, modulo the requirements given in Section 3.3 above. This approach of the Quick-Start router algorithm as a matter of local policy is consistent with the approach in RFC 3168 on standardizing Explicit Congestion Notification (ECN). RFC 3168 standardized the semantics of the ECN field, but did not standardize a router's Active Queue Management mechanisms for deciding when to set the Congestion Experienced codepoint in packets. E.6. An Alternate Proposal [B05] proposes an alternate to Quick-Start where endpoints allocate rates to themselves. [B05] argues that adding rate allocation to - interior routers is not the fundamenally correct direction. + interior routers is not the fundamentally correct direction. [B05] argues for an approach that associates senders with their ingress attachment point, with routers reporting their impairment status back to the sender [BJCG+05, BJS05]. The source declares the impairment that it believes it will accumulate along the path, and routers effectively subtract the local impairment status. If the sender is reporting correctly, and the impairment has not changed significantly from one round-trip time to the next, the reported impairment at the egress router should be close to zero. Normative References [RFC793] J. Postel, Transmission Control Protocol, RFC 793, September 1981. [RFC1191] Mogul, J. and S. Deering, Path MTU Discovery, RFC 1191, November 1990. + [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate + Requirement Levels", BCP 14, RFC 2119, March 1997. + [RFC2460] S. Deering and R. Hinden. Internet Protocol, Version 6 (IPv6) Specification. RFC 2460, December 1998. [RFC2581] M. Allman, V. Paxson, and W. Stevens. TCP Congestion Control. RFC 2581. April 1999. [RFC3168] Ramakrishnan, K.K., Floyd, S., and Black, D. The Addition of Explicit Congestion Notification (ECN) to IP. RFC 3168, Proposed Standard, September 2001. @@ -3596,60 +3775,57 @@ April 1997. [RFC2205] R. Braden, et al. Resource ReSerVation Protocol (RSVP) -- Version 1 Functional Specification. RFC 2205, September 1997. [RFC2409] D. Harkins and D. Carrel, The Internet Key Exchange (IKE), RFC 2409, November 1998. [RFC2246] T. Dierks and C. Allen, The TLS Protocol, RFC 2246. - [RFC2309] B. Braden, D. Clark, J. Crowcroft, B. Davie, S. Deering, - D. Estrin, S. Floyd, V. Jacobson, G. Minshall, C. Partridge, L. - Peterson, K. Ramakrishnan, S. Shenker, J. Wroclawski, L. Zhang, - Recommendations on Queue Management and Congestion Avoidance in the - Internet, RFC 2309, April 1998. - [RFC2401] S. Kent and R. Atkinson. Security Architecture for the Internet Protocol. RFC 2401, November 1998. [2401bis] S. Kent and K. Seo, Security Architecture for the Internet Protocol, internet-draft draft-ietf-ipsec-rfc2401bis-06.txt, work- in-progress, March 2005. - [RFC2402] S. Kent and R. Atkinson. IP Authentication Header. RFC - 2402, November 1998. - [2402bis] S. Kent, IP Authentication Header, internet-draft draft- ietf-ipsec-rfc2402bis-11.txt work-in-progress, March 2005. [RFC2415] K. Poduri and K. Nichols. Simulation Studies of Increased Initial TCP Window Size. RFC 2415. September 1998. [RFC2416] T. Shepard and C. Partridge. When TCP Starts Up With Four Packets Into Only Three Buffers. RFC 2416. September 1998. [RFC2463] A. Conta and S. Deering. Internet Control Message Protocol (ICMPv6) for the Internet Protocol Version 6 (IPv6) Specification. RFC 2463, December 1998. [RFC2488] M. Allman, D. Glover, and L. Sanchez. Enhancing TCP Over Satellite Channels using Standard Mechanisms. RFC 2488. January 1999. + [RFC2784] D. Farinacci, T. Li, S. Hanks, D. Meyer, and P. Traina, + Generic Routing Encapsulation (GRE), RFC 2784, March 2000. + [RFC2661] W. Townsley, A. Valencia, A. Rubens, G. Pall, G. Zorn, and B. Palter, Layer Two Tunneling Protocol "L2TP", RFC 2661, August 1999. [RFC2960] R. Stewart, et. al. Stream Control Transmission Protocol. RFC 2960, October 2000. + [RFC3031] E. Rosen, A. Viswanathan, and R. Callon. Multiprotocol + Label Switching Architecture. RFC 3031. January 2001. + [RFC3124] H. Balakrishnan and S. Seshan. The Congestion Manager. RFC 3124. June 2001. [RFC3234] B. Carpenter and S. Brim, Middleboxes: Taxonomy and Issues, RFC 3234, February 2002. [RFC3344] C. Perkins (ed.). IP Mobility Support for IPv4. RFC 3344, August 2002. [RFC3360] S. Floyd. Inappropriate TCP Resets Considered Harmful. @@ -3690,24 +3866,20 @@ [BJS05] Briscoe, B., Jacquet, A., and A. Salvatori, "Re-ECN: Adding Accountability for Causing Congestion to TCP/IP", internet-draft draft-briscoe-tsvwg-re-ecn-tcp-00.txt, work-in-progress, October 2005. [BW97] G. Brasche and B. Walke. Concepts, Services and Protocols of the new GSM Phase 2+ General Packet Radio Service. IEEE Communications Magazine, pages 94--104, August 1997. - [FF99] Floyd, S., and Fall, K., Promoting the Use of End-to-End - Congestion Control in the Internet, IEEE/ACM Transactions on - Networking, August 1999. - [GPAR02] A. Gurtov, M. Passoja, O. Aalto, and M. Raitola. Multi- Layer Protocol Tracing in a GPRS Network. In Proceedings of the IEEE Vehicular Technology Conference (Fall VTC2002), Vancouver, Canada, September 2002. [H05] P. Hoffman, email, October 2005. Citation for acknowledgement purposes only. [HKP01] M. Handley, C. Kreibich and V. Paxson, Network Intrusion Detection: Evasion, Traffic Normalization, and End-to-End Protocol @@ -3734,133 +3906,112 @@ Environments. ACM Sigcomm 2002, August 2002. URL "http://ana.lcs.mit.edu/dina/XCP/". [KHF05] E. Kohler, M. Handley, and S. Floyd, Datagram Congestion Control Protocol (DCCP), internet draft draft-ietf-dccp-spec-11.txt, work in progress, March 2005. [KK03] A. Kuzmanovic and E. W. Knightly. TCP-LP: A Distributed Algorithm for Low Priority Data Transfer. INFOCOM 2003, April 2003. + [KSEPA04] Rajesh Krishnan, James Sterbenz, Wesley Eddy, Craig + Partridge, Mark Allman. Explicit Transport Error Notification (ETEN) + for Error-Prone Wireless and Satellite Networks. Computer Networks, + 46(3), October 2004. + [L05] Guohan Lu, Nonce in TCP Quick Start, draft, September 2005. URL "http://www.net-glyph.org/~lgh/nonce-usage.pdf". [MAF04] Alberto Medina, Mark Allman, and Sally Floyd, Measuring Interactions Between Transport Protocols and Middleboxes, Internet Measurement Conference 2004, August 2004. URL "http://www.icir.org/tbit/". [MAF05] Alberto Medina, Mark Allman, and Sally Floyd. Measuring the Evolution of Transport Protocols in the Internet. Computer - Communications Review, April 2004. + Communications Review, April 2005. [MaxNet] MaxNet Home Page, URL "http://netlab.caltech.edu/~bartek/maxnet.htm". - [PK98] Venkata N. Padmanabhan and Randy H. Katz, TCP Fast Start: A - Technique For Speeding Up Web Transfers, IEEE GLOBECOM '98, November - 1998. - [P00] Joon-Sang Park, Bandwidth Discovery of a TCP Connection, report to John Jeidemann, 2000, private communication. Citation for acknowledgement purposes only. + [PABL+05] Ruoming Pang, Mark Allman, Mike Bennett, Jason Lee, Vern + Paxson, Brian Tierney. A First Look at Modern Enterprise Traffic. + ACM SIGCOMM/USENIX Internet Measurement Conference, October 2005. + [PRAKS02] Craig Partridge, Dennis Rockwell, Mark Allman, Rajesh Krishnan, James P.G. Sterbenz. A Swifter Start for TCP. Technical Report No. 8339, BBN Technologies, March 2002. URL "http://www.icir.org/mallman/papers/". [RW03] Mattia Rossi and Michael Welzl, On the Impact of IP Option Processing, Preprint-Reihe des Fachbereichs Mathematik - Informatik, - No. 15, October 2003. + No. 15, Institute of Computer Science, University of Innsbruck, + Austria, October 2003. [RW04] Mattia Rossi and Michael Welzl, On the Impact of IP Option Processing - Part 2, Preprint-Reihe des Fachbereichs Mathematik - - Informatik, No. 26, July 2004. + Informatik, No. 26, Institute of Computer Science, University of + Innsbruck, Austria, July 2004. [S02] Ion Stoica, private communication, 2002. Citation for acknowledgement purposes only. - [SAF05] Pasi Sarolahti, Mark Allman, and Sally Floyd. Evaluating - Quick-Start for TCP. May 2005. URL - "http://www.icir.org/floyd/quickstart.html". + [SAF06] Pasi Sarolahti, Mark Allman, and Sally Floyd. Determining + an Appropriate Sending Rate Over an Underutilized Network Path. + February 2006. URL "http://www.icir.org/floyd/quickstart.html". [SGF05] Singh, M., Guha, S., and P. Francis, "Utilizing spare network bandwidth to improve TCP performance", ACM SIGCOMM 2005 Work in Progress session , August 2005, https://www.guha.cc/saikat/pub/sigcomm05-lowtcp.pdf. [SHA1] "Secure Hash Standard", FIPS, U.S. Department of Commerce, Washington, D.C. publication 180-1, April 1995. [SH02] Srikanth Sundarrajan and John Heidemann. Study of TCP Quick Start with NS-2. Class Project, December 2002. Not publically available; citation for acknowledgement purposes only. [V02] A. Venkataramani, R. Kokku, and M. Dahlin. TCP Nice: A Mechanism for Background Transfers. OSDI 2002. [W00] Michael Welzl: PTP: Better Feedback for Adaptive Distributed Multimedia Applications on the Internet, IPCCC 2000 (19th IEEE International Performance, Computing, And Communications Conference), Phoenix, Arizona, USA, 20-22 February 2000. URL - "http://informatik.uibk.ac.at/users/c70370/research/publications/". - - [W03] Michael Welzl, PMTU-Options: Path MTU Discovery Using Options, - expired internet-draft draft-welzl-pmtud-options-01.txt, work-in- - progress. February 2003. + "http://www.welzl.at/research/publications/". - [ZPS00] Y. Zhang, V. Paxson, and S. Shenker, The Stationarity of - Internet Path Properties: Routing, Loss, and Throughput, ACIRI - Technical Report, May 2000. + [ZDPS01] Y. Zhang, N. Duffield, V. Paxson, and S. Shenker, On the + Constancy of Internet Path Properties, Proc. ACM SIGCOMM Internet + Measurement Workshop, November 2001. [ZQK00] Y. Zhang, L. Qiu, and S. Keshav, Speeding Up Short Data Transfers: Theory, Architectural Support, and Simulation Results, NOSSDAV 2000, June 2000. -F. IANA Considerations - - Quick-Start requires an IP Option and a TCP Option. - -F.1. IP Option - - Quick-Start requires that both an IPv4 and an IPv6 Option Number be - allocated. The IPv4 Option would have a copied flag of 0, a class - field of 00, and the assigned five-bit option number. The IPv6 - Option would have the first three bits of "001" [RFC 2460, Section - 4.2], with the first two bits indicating that the IPv6 node skip - over this option and continue processing the header if it doesn't - recognize the option type, and the third bit indicating that the - Option Data may change en-route. - - In both cases the name of the option would be "QS - Quick-Start", - with this document as the reference document. - -F.2. TCP Option - - Quick-Start also requires that a TCP Option Number be allocated. - The Length would be 4, and the Meaning would be "Quick-Start - Request", with this document as the reference document. - AUTHORS' ADDRESSES Sally Floyd Phone: +1 (510) 666-2989 ICIR (ICSI Center for Internet Research) Email: floyd@icir.org URL: http://www.icir.org/floyd/ Mark Allman ICSI Center for Internet Research 1947 Center Street, Suite 600 Berkeley, CA 94704-1198 - Phone: (440) 243-7361 + Phone: (440) 235-1792 Email: mallman@icir.org URL: http://www.icir.org/mallman/ Amit Jain F5 Networks Email : a.jain@f5.com Pasi Sarolahti Nokia Research Center P.O. Box 407