ISP Dual Queue Networking Deployment Suggestions

1.1. A Different Understanding of Application Needs

In the course of working to improve the responsiveness of network protocols, the IETF concluded with their L4S and NQB work that there were two main types of traffic and that these two traffic types could benefit from having separate network processing queues in order to improve the way the performance of delay-sensitive and/or interactive applications. Introducing a new queue enables incremental development of a new standard rather than changing existing congestion control algorithms for existing traffic on all networks, hosts, and clients - which would be complex.¶

One of the two major traffic types is mostly file download or upload, such as downloading an operating system update or uploading files to a cloud backup. This type of traffic is not delay-sensitive, at least on a millisecond level basis. The other type of traffic is real-time, interactive traffic that is latency-sensitive, such as video conferencing, gaming, and artificial intelligence (AI) interactions.¶

The value of dual queue networking (simply "low latency networking" hereafter) seems potentially good, and at least one major ISP has deployed it [Comcast]. It seems possible that this new capability might enable entirely new classes of applications to become possible, driving a wave of new Internet innovation, while also improving the latency-sensitive applications that people use today.¶

2.1. Throughput is Bounded by Congestion Control Algoritms

The Mathis Equation [Mathis] explains how throughput is bounded by congestion control algorithms. This equation shows that throughput scales at 1/sqrt(p), where p is the packet loss probability. The square root is the problem, because achieving a ten-fold increase in throughput requires a one-hundred-fold reduction in packet loss! One solution, now understood as problematic, was to build large packet buffers so as to minimize packet loss (p). But this led to very high delay, otherwise known as bufferbloat.¶

This nonlinear relationship described in the equation means that, to take a link from 1 Gbps to 10 Gbps for example, packet loss would need to be reduced one-hundred-fold. This makes it increasingly hard to hit ever-higher data rates in real world environments, especially with WiFi and 5G links, where those levels of loss are unachievable at the current time.¶

L4S replaces packet loss (p) with a signal (CE mark), as a rapid feedback signal to a sender to reduce their rate. This means that with L4S, throughput scales proportionally. In a manner of speaking, L4S apps can quickly react to potential congestion by gradually tapping the brakes before there is a big congestion crash. With L4S, if you increase throughput by 10x, you only need to reduce loss by 10x rather than 100x. This is far more achievable and makes high throughput, low delay networking a realistically achievable goal.¶

This table shows examples of the loss target necessary to take a link from 1 Gbps to 10 Gbps, given certain starting packet loss levels. Wireline networks will typically have loss levels between 0.1% and 0.5%, WiFi links may experience between 0.5% and 2% loss, and 5G links between 1% and 3% loss [FCC-MBA-2023].¶

2.2. Best Effort Priority

Low latency traffic to is not prioritized over other (best effort priority) "classic" Internet traffic. This best effort approach stands in contrast to prior differential quality of service (QoS) approaches or to what has been discussed for 5G network slicing [CDT-NN] [van-Schewick-1A] [van-Schewick-1B] [van-Schewick-2] [van-Schewick-3].¶

Those approaches are grounded in an assumption that there is scare bandwidth and so the network must grant higher priority to some flows, creating bandwidth winners and losers. That has raised the ire of network neutrality proponents, who have focused on negative effects this can have on flows that are not prioritized and the terms under which certain applications or destinations may be granted priority.¶

Another contrast with 5G network slicing is that IETF standards such as L4S do not require network operator-specific APIs, direct coordination, or the granting of permission by technical or legal means. Rather, marking packets with ECT(1) or CE as part of L4S will work on any network that has implemented L4S, with no need to coordinate with each network or seek permission. This exemplifies the loose coupling across layer (network and application) and permissionless innovation (at the edge by application developers) that are core tenets of the Internet's architecture.¶

The only exception for priority is within a user's in-home Wi-Fi network (in the case of a fixed network) due to the particulars of how the IEEE 802.11 wireless protocol [IEEE] functions at the current time - see [RFC8325]. Some user access points may prioritize certain traffic (such as gaming) and some traffic such as NQB may use the AC_VI Wi-Fi link layer queue [I-D.ietf-tsvwg-nqb].¶

2.3. Shared Throughput

Low latency networking flows do not get access to greater throughput than "classic" flows. Thus, a user's total provisioned or permitted throughput on an ISP access network link is shared between both classic and low latency queues. This is important, as it demonstrates that there is no difference in priority between flows - so by marking for low latency treatment a flow cannot achieve any greater bandwidth than if it did not mark, removing an incentive to game the system to the detriment of other flows.¶

Another way to thing of L4S is that it is not really an instruction to a dual queue network link to ask for low latency treatment per se. Rather, applications marking for L4S are essentially making a promise to the network that the application will rapidly reduce throughput if needed (when detecting CE marks); it promises to help keep latency low by being a good network citizen.¶

2.4. Access-Agnostic

Low latency networking can be implemented in a variety of network technologies. For example in access network technologies this could be implemented in DOCSIS [LLD], 5G [Ericsson], PON [CTI], WiFi, Low Earth Orbit (LEO) satellite and many other types of networks.¶

2.5. End-to-End Diagram

This diagram shows an example of the end-to-end path for two different user devices, one phone connected to the home WiFi network and one PC connected via Ethernet to the home LAN. Along this full path, L4S or NQB marks must be able to pass between an application on a client device and an application server. While in theory dual queue low latency networking can be implemented on any of the routers on this path or the access point, the most common bottlenecks are in the home network (WiFi), the CPE router, and the aggretation router. This alone will provide a tremendous benefit to interactive end user applications and dual queue need not in that case be deployed at any of the other hops.¶

+-------+                   +---------+
| Phone | <--802.11 WiFi--> | WiFi AP |
+-------+                   +---------+
                                ^
                                |
                                | Ethernet
                                v
+-------+                   +---------+      +--------+      +-----------+      +----------+      +--------+
|  PC   | <---Ethernet--->  |   CPE   | <--> | Agg    | <--> | Core/Edge | <--> | App Edge | <--> | App    |
|       |                   | Router  |      | Router |      | Router    |      | Router   |      | Server |
+-------+                   +---------+      +--------+      +-----------+      +----------+      +--------+

3.1. Delivery Infrastructure for L4S

Since L4S uses the Explicit Congestion Notification (ECN) field of the packet header, to ensure ECN works end-to-end, application developers need to be certain that their servers, datacenter routers, and any transit, cloud provider, or content delivery network (CDN) server involved in their application is not altering or bleaching the ECN field. For an application to use the L4S queue, they must mark their packets with the ECT(1) code point to signal L4S-capability or with the Congestion Experienced (CE) code point when appropriate. Coupled with client marking, if an application client or server detects CE marks, it should respond accordingly (e.g., by reducing the send rate), which typically means that the server must be running a "responsive" congestion controller (i.e., is able to adjust rate based the presence or absence of CE marks for L4S traffic - such as DCTCP, TCP Prague, SCReAM, and BBRv2). See Section 4.3 of [RFC9330] and Section 4.3 of [RFC9331] for more information about this.¶

3.2. Delivery Infrastructure for NQB

NQB uses the DSCP-45 code point in the DiffServ part of the packet header to ensure NQB works end-to-end. To achive end-to-end marking, application developers need to be certain that their servers, datacenter routers, and any transit, cloud provider, or content delivery network (CDN) server involved in their application is not altering or bleaching a DSCP-45 mark. The server does not need to run a special responsive congestion controller. Since it is common for networks to bleach or modify DSCP marks on ingress today, so networks will need to change that policy for NQB to work end-to-end (in contrast, ECN is rarely bleached).¶

3.3. Only Mark Delay-Sensitive Traffic for L4S or NQB

It may seem tempting to mark all traffic for L4S or NQB handling, but it may not help in all cases. For example, a video gaming service may benefit from using L4S or NQB for real-time controller inputs and gameplay, while major game software updates would best be left in the classic queue, as doing otherwise might adversely impact game performance.¶

3.4. Consider Application Needs in Choosing L4S vs. NQB

Determine whether your application needs "sparse" flows or "congestion-controlled" (higher capacity) flows. Sparse flows that are latency senstive should be marked as NQB (thus DSCP-45). This may be things like DNS queries or VoIP media flows, where maximizing the bandwidth of the flow is not necesary.¶

Latency-sensitive flows that need more bandwidth are congestion controlled, and identified via ECN marking. These types of applications are less limited by the application protocol itself (i.e., a small DNS query), which means the application quality can improve as more bandwidth is available - such as shifting a video stream or a video conference session from Standard Definition (SD) to 4K quality.¶

3.5. Example: iOS Application for L4S

Impementations will vary depending upon variables such as server operating system, transport and application protocols, and client software and operating system. To use the example of an Apple device such as an iPhone, running iOS, L4S support is built into iOS by default [L4S-Testing]. If the developer uses the QUIC protocol, they can use "URLSession" [URLSession] which enables L4S.¶

4.1. Allow ECN Across Network Boundaries

Traffic sent to a peer network marked with ECT(1) or CE in the ECN header must pass to that peer without altering or removing the ECT(1) or CE marking (see exception below). Traffic FROM peers marked with ECT(1) or CE in the ECN header must be allowed to enter the network without altering or removing the ECT(1) or CE marking (see exception below). The only exception would be when a network element is CE-aware and able to add a CE mark to signal that it is experiencing congestion at that hop. The end-to-end integrity of ECN marks is explored in Appendix C.1 of [RFC9331].¶

This part - allowing unmodified ECN across the network - is likely to be easier than DSCP-45 for NQB (see next section), since it appears rare that networks modify the ECN header of packet flows.¶

4.2. Allow DSCP-45 Across Network Boundaries

Traffic sent to a peer network marked with DSCP value 45 must pass to that peer without altering or removing the DSCP 45 marking (see exception below). Traffic FROM peers marked with DSCP value 45 must be allowed to enter the network without altering or removing the DSCP 45 marking (see exception below). The end-to-end integrity of the DSCP-45 mark is explored in Section 6.4 of [I-D.ietf-tsvwg-nqb].¶

Some networks may use DSCP 45 for internal purposes other than NQB within their network. In these cases, the peer using DSCP 45 for other purposes is responsible for remarking as appropriate. If possible, for operational simplicity, a network should try to maintain the use of DSCP 45 on an end-to-end basis without remarking in their interior network hops.¶

4.3. Last Mile Network (Access Network)

There are two hops of interest in the last mile access network. One will be a point of user aggregation, such as a Cable Modem Termination System (CMTS) or Optical Line Terminal (OLT). The second is at the user location, such as a Cable Modem (CM) or Optical Network Unit (ONU), both of which are example of CPE.¶

In theses two queues, ISPs should consider using the optional Queue Protection function [I-D.ietf-tsvwg-nqb] [I-D.briscoe-docsis-q-protection]. This can potentially detect mismarking and take corrective action as needed.¶

4.4. Customer Premise Equipment (Customer Edge)

In most residential Internet services, there are typically two equipment modes. One is a very simple CPE that hands off from the ISP's access network (i.e., DSL, 5G, DOCSIS, PON) and provides the customer with an Ethernet interface and IP address(es). The customer then connects their own router and wireless access point (often integrated into the router, typically referred to as a "wireless gateway" or "wireless router"). The other model is more typical, which is that the CPE integrates a link layer termination function (i.e., Cable Modem, 5G radio, or Optical Network Unit) as well as a wireless gateway.¶

Not all ISP networks support both of these models; sometimes only a wireless gateway is available. Even in this case, some users "double NAT" and install their own router and wireless access point(s) to get whatever functionality and control over their home network that they desire. The cases explored below are commonplace but may not apply to all networks.¶

In some cases, dual queue networking and associated packet marking is supported up to the ISP's demarcation point - such as in a cable modem. Packet markings should pass from such a demarcation point to any attached customer-administered CPE, such as a router or wireless access point. That enables a customer-administered router to implement dual queue networking, rather that it only being possible with ISP-administered CPE.¶

4.5. Inside the Home - Customer Local Area Network (LAN)

As noted above with the mention of an integrated wireless gateway, the CPE and router/wireless network gear is integrated into a single CPE device. Even though these are functionally in one piece of hardware, we can think of the wide area network interface and local area network as functionally separate for purposes of this analysis.¶

4.6. Packet Loss Effects

As networks implement low latency networking, including L4S, NQB, and new forms of AQM, they may notice somewhat less packet loss. This is because applications now have a new signals, such as a CE mark, to signal congestion. This signal is faster than the typical TCP packetloss mechanism.¶

Thus, this is expected, as noted in Section 5.1 of Section 5.1 of [RFC9330]:¶

Latency is not the only concern of L4S. The 'Low Loss' part of the name denotes that L4S generally achieves zero congestion loss due to its use of ECN. Otherwise, loss would itself cause delay, particularly for short flows, due to retransmission delay [RFC2884].¶

In addtion see Section 3 of Section 3 of [RFC2884]:¶

Since ECN marks packets before congestion actually occurs, this is useful for protocols like TCP that are sensitive to even a single packet loss. Upon receipt of a congestion marked packet, the TCP receiver informs the sender (in the subsequent ACK) about incipient congestion which will in turn trigger the congestion avoidance algorithm at the sender.¶

4.7. Avoid Middleboxes

As noted in [Tussle] there has always been a tension in the end-to-end model between how much intelligence and processing takes place along the end-to-end path inside of a network and how much takes place at the end of the network in servers and/or end user client devices and software. In this new approach to low latency networking, entry into a low latency queue depends upon marks in the packet header of a particular application flow. In practice, this marking is best left to the application edge of the network, rather than it being a function of a middlebox in the ISP network. As explored below, this is the most efficient, least prone to mis-classification, and is most acceptable from the standpoint of network neutrality. So, while a middlebox may exist on an end-to-end path, it is best not to have a middlebox actively classify traffic for a low latency queue; let applications made that decision. This limits the potential for harm by a middlebox.¶

The best approach is for applications to mark traffic to indicate their preference for the low latency queue, not the network making such a decision on its own. This is for several reasons:¶

• According to the end-to-end principle, this function is best delegated to the edge of the network as an architectural best practice (the edge being the application in this case).¶
• Application marking maintains the loose coupling between the application and network layers, eliminating the need for close coordination between networks and application developers.¶
• Application developers know best whether their application is compatible with low latency networking and which aspects of their traffic flows will or will not benefit.¶
• Only the application (not the network) knows whether a scalable congestion control algorithm congestion control is being used on the application server. Thus, only the developer and server administrator know if they are correctly responding to Congestion Experienced (CE) markings for L4S (see Section 4.1 of [RFC9331]).¶
• Application traffic is almost entirely encrypted, which makes it very difficult for networks to accurately determine application protocols and to further infer which flows will benefit from low latency and which flows may be harmed because they need to build a queue. It is likely that false positives [Lotus] and false negatives will occur if network-based inference is used; all of which can be avoided simply by relying solely on application marking.¶
• The pace of innovation and iteration is necessarily faster-moving in the application edge at layer 7, rather than in the network at layer 3 (and below) - where there is greater standards stability and a lower rate of major changes. As a result, the application layer is best suited to rapid experimentation and iteration. Network operators and equipment vendors trying to infer application needs will in comparison always be in a reactive mode, one step behind changes made in applications.¶
• This avoids issues arising from mis-classification of application flows [Lotus].¶
• Any application provider should be able to mark their traffic for the low latency queue, with no restrictions other than standards compliance or other reasonable and openly documented technical guidelines. This maintains the loose cross-layer coupling that is a key tenet of the Internet's architecture by eliminating the need for application providers and networks to coordinate and creates an environment of so-called "permissionless innovation".¶