Subject: [ledbat] "Oh Noes, the Internetz will Melt Down..."

From: Nicholas Weaver <[email-masked]>
Date: Mon, 1 Dec 2008 07:56:52 -0800

The typical Fear Mongering "UDP Death Uhz Da Internetz":

http://www.theregister.co.uk/2008/12/01/richard_bennett_utorrent_udp/

However, the question is, does anyone have insight into the "uTP"  
congestion control protocol and what it is actually doing?  Is it  
"AIMD + delay based?" which was discussed at the IETF meeting, which  
still has the multi-flow unfairness issue (the ISP congestion problem  
stays the same, but isn't made worse or better), but addresses the  
buffer-congestion problem?

Or is it something else?

_______________________________________________
ledbat mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ledbat

---

From: Lars Eggert <[email-masked]>
Date: Mon, 1 Dec 2008 18:23:05 +0200

> http://www.theregister.co.uk/2008/12/01/richard_bennett_utorrent_udp/

Wow. Talk about jumping to conclusions. That is quite an "article".

> congestion control protocol and what it is actually doing?

See http://forum.utorrent.com/viewtopic.php?id=49813

"What is in 1.9:
uTP, the micro transport protocol. This UDP-based reliable transport  
is designed to minimize latency, but still maximize bandwidth when the  
latency is not excessive. We use this for communication between peers  
instead of TCP, if both sides support it. In addition, we use  
information from this transport, if active, to control the transfer  
rate of TCP connections. This means uTorrent, when using uTP, should  
not kill your net connection - even if you do not set any rate limits.

What was in 1.8.1:
uTP, but connection attempts were not initiated by default, and there  
was no control over TCP as described above. You can enable it, but  
likely you will see the uTP connections not transfering much data,  
because they are pushed out of the way by TCP."

This last bit sounds a lot like if uTP is is using some sort of less- 
than-best-effort CC. (I believe someone mentioned in Minneapolis that  
utorrent would implement BitTorrent's DNA - maybe this is it?

Lars

---

From: Murari Sridharan <[email-masked]>
Date: Mon, 1 Dec 2008 12:28:11 -0800

Lars I think you are right, stas can confirm. I was trying to get more details on the actual protocol and here is a post I found from BitTorrent VP

------
A comment from BitTorrent on UTP:

Firon posted already about UTP:
******
uTP, the micro transport protocol. This UDP-based reliable transport is designed to minimize latency, but still maximize bandwidth when the latency is not excessive. We use this for communication between peers instead of TCP, if both sides support it. In addition, we use information from this transport, if active, to control the transfer rate of TCP connections. This means uTorrent, when using uTP, should not kill your net connection - even if you do not set any rate limits.
******

Just to re-iterate and offer a few more details (there's some pretty wild press reports popping up):
Firon described uTP completely accurately.
uTP is the result of a couple of years of work to try to make a Bittorrent protocol that works better on the internet.

The switch to uTP is at this point purely experimental, but the design objective (counter to some reports in the press) is actually to offer better congestion control than TCP offers, but maintain the same level of performance (speed).
Better congestion control is good for everyone - for users (VOIP, Gamers etc.) as well as for ISPs.
Same performance is what users have come to expect from their BitTorrent application - unless we can offer the same performance, then people will switch to a different BitTorrent client. (In reality we may be able to offer faster speeds too in many circumstances, but this is a byproduct and not the main objective.)

uTP is our UDP-based implementation of the BitTorrent protocol.
Normally BitTorrent is implemented on top of TCP which is the standard congestion control mechanism for the internet.
It so happens that the congestion control mechanism inside TCP is quite crude and problematic. It only detects congestion on the internet once "packet loss" has occurred - i.e. once the user has lost data and (probably) noticed there is a problem. The problems of TCP are fairly well known in technical circles, but it doesn't get fixed as TCP is one of those protocols that is implemented in every webserver and application (e.g. browsers, bittorrent clients) on the internet. Co-ordinating a giant upgrade is a very long process.

Because BitTorrent publishes the world's most popular BitTorrent clients AND because these clients are talking mostly to each other (not to web servers), then we have an opportunity to detect end-to-end congestion and implement a protocol that can detect problems very quickly and throttle back accordingly so that BitTorrent doesn't slow down the internet connection and Gamers and VOIP users don't notice any problems. This is our objective.

This is great news for users of the internet and even for ISPs as it should mean that people make far more efficient use of internet bandwidth, but don't over-use it to destruction. If uTP is successful, then internet congestion due to BitTorrent protocol could become a thing of the past. Of course there are many other applications that use the internet and they may also cause congestion, but we can only control what we do. Having said that, given that some press reports suggest that BitTorrent traffic constitutes half of all traffic on the internet, our technology might have a profound impact. We're trying to do our bit to be responsible citizens on the internet.

While uTP is for now a proprietary BitTorrent protocol, we are also co-chairing an IETF group to address these issues. Hopefully that will lead to solutions that can be standardized and broadly adopted in due course.

Simon Morris
VP Product Management
BitTorrent, Inc.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Lars Eggert
Sent: Monday, December 01, 2008 8:23 AM
To: Nicholas Weaver
Cc: [email-masked]
Subject: Re: [ledbat] "Oh Noes, the Internetz will Melt Down..."

"What is in 1.9:
uTP, the micro transport protocol. This UDP-based reliable transport is designed to minimize latency, but still maximize bandwidth when the latency is not excessive. We use this for communication between peers instead of TCP, if both sides support it. In addition, we use information from this transport, if active, to control the transfer rate of TCP connections. This means uTorrent, when using uTP, should not kill your net connection - even if you do not set any rate limits.

What was in 1.8.1:
uTP, but connection attempts were not initiated by default, and there was no control over TCP as described above. You can enable it, but likely you will see the uTP connections not transfering much data, because they are pushed out of the way by TCP."

This last bit sounds a lot like if uTP is is using some sort of less- than-best-effort CC. (I believe someone mentioned in Minneapolis that utorrent would implement BitTorrent's DNA - maybe this is it?

Lars

---

From: Richard Bennett <[email-masked]>
Date: Mon, 01 Dec 2008 13:03:17 -0800

I'm glad so many people have read my article, and I'd be happy to
discuss it at length if it's appropriate. Broadband Reports, the info
hub for file-sharers, ran an article on uTorrent 1.9 over the weekend
under the title: "New UDP uTorrent Takes Aim At Throttling,"
highlighting the value of UDP-based file sharing for avoiding TCP-based
fair traffic systems. It's clear that the users of BitTorrent see this
as simply another means of grabbing more network resources.

That being said, I'll gladly stipulate that BitTorrent, Inc., means
well and has a heart of gold. But what happens when users can tune
congestion avoidance and backoff parameters through a GUI? I'll go out
on a limb here and say some of them may not be altruistic.

RB

Murari Sridharan wrote:

------

Simon Morris
VP Product Management
BitTorrent, Inc.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Lars Eggert
Sent: Monday, December 01, 2008 8:23 AM
To: Nicholas Weaver
Cc: [email-masked]
Subject: Re: [ledbat] "Oh Noes, the Internetz will Melt Down..."

"What is in 1.9:

What was in 1.8.1:

Lars

---

From: "Robb Topolski" <[email-masked]>
Date: Mon, 1 Dec 2008 13:36:57 -0800

I didn't read his article.  This is in reply to his message only.

> But what happens when users can tune congestion avoidance and backoff
> parameters through a GUI?
>

1.  They can tune parameters in TCP now with REGEDIT in Windows or by using
Linux.  And even that doesn't limit the possibility that they could if they
wanted to -- users are already swapping out Windows TCPIP.SYS (have been for
years) for ones hacked to work as they want it to.  So, having this ability,
what are they changing?  Are they defeating congestion control boundaries?
Nope.  They're only defeating a half-open TCP limit designed to avoid the
fast spread of worms.

2.  (Until I'm blue in the face, I swear), P2P developers generally are fans
of the Internet and are unlikely to intentionally develop in ways harmful to
the network.  I don't know how else to day it.  They don't ##### where they
live.  They surf websites and take VOIP calls, too.

It's clear that the users of BitTorrent see this as simply another means of
> grabbing more network resources.
>

3.  I'm not agreeing with that conclusion, as it's just as plausible that
they want to reduce their own support load; a lot of effort is dedicated to
teaching users how to avoid overrunning the buffers in their modem.
However, if "more network resources" are they're there to "grab" and doing
so doesn't interfere with anybody, then so what?  The design of TCP is to
keep trying to grab more network resources, and most applications do nothing
to change that behavior.

Broadband Reports, the info hub for file-sharers,

4.  Hardly. But as clear as it is to you that users of BitTorrent see this
as simply another means of grabbing more network resources, it's clear to me
that some people are never pursuaded from their consistent droning despite
any amount of facts and evidence presented to the contrary.

Robb Topolski

On Mon, Dec 1, 2008 at 1:03 PM, Richard Bennett <[email-masked]> wrote:

>  I'm glad so many people have read my article, and I'd be happy to discuss
> it at length if it's appropriate. Broadband Reports, the info hub for
> file-sharers, ran an article on uTorrent 1.9 over the weekend under the
> title: "New UDP uTorrent Takes Aim At Throttling," highlighting the value of
> UDP-based file sharing for avoiding TCP-based fair traffic systems. It's
> clear that the users of BitTorrent see this as simply another means of
> grabbing more network resources.
>
> That being said, I'll gladly stipulate that BitTorrent, Inc., means well
> and has a heart of gold. But what happens when users can tune congestion
> avoidance and backoff parameters through a GUI? I'll go out on a limb here
> and say some of them may not be altruistic.
>
> RB
>
>
>
> Murari Sridharan wrote:
>
> From: [email-masked] [mailto:[email-masked] <[email-masked]>] On Behalf Of Lars Eggert
>  The typical Fear Mongering "UDP Death Uhz Da Internetz":http://www.theregister.co.uk/2008/12/01/richard_bennett_utorrent_udp/
>
> ledbat mailing [email-masked]://www.ietf.org/mailman/listinfo/ledbat
>
>

-- 
Robb Topolski ([email-masked])
Hillsboro, Oregon USA
http://www.funchords.com/
_______________________________________________
ledbat mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ledbat

---

From: Nicholas Weaver <[email-masked]>
Date: Mon, 1 Dec 2008 13:53:34 -0800

On Dec 1, 2008, at 1:03 PM, Richard Bennett wrote:

> discuss it at length if it's appropriate. Broadband Reports, the  
> info hub for file-sharers, ran an article on uTorrent 1.9 over the  
> weekend under the title: "New UDP uTorrent Takes Aim At Throttling,"  
> highlighting the value of UDP-based file sharing for avoiding TCP- 
> based fair traffic systems. It's clear that the users of BitTorrent  
> see this as simply another means of grabbing more network resources.
>
> congestion avoidance and backoff parameters through a GUI? I'll go  
> out on a limb here and say some of them may not be altruistic.

Where is the suggestion that users will be able to control the  
congestion control parameters?

IF the algorithm is equivalent to what was presented at the IETF,  
namely, a delay-based ADDITION to a TCP-like AIMD, this will only have  
a net benefit, as this will still stabilize out like any other TCP  
flow (so no effect on the ISP congestion problem one way or the  
other), but will have a huge benefit on the buffer-oversizing  
congestion problem.

Likewise, the notion that this can avoid traffic management is pretty  
delusional.  The only thing a UDP shift avoids is RST-injection based  
management, which is a crude tool.  Any in-path management (which,  
btw, Sandvine supports, but was simply not deployed in this way by  
Cox, Comcast, and others) can block a UDP flow just as easily as a TCP  
flow.  Likewise, router ACL injection can block flows as well.

There are plenty of things to dislike about bulk-data P2P.  But this  
is probably not one of them.

---

From: Richard Bennett <[email-masked]>
Date: Mon, 01 Dec 2008 14:55:53 -0800

This is what we've seen in P2P/BT implementations in the past. Check the 
configuration options in the Azureus agent that's part of most standard 
Linux distributions: it has a GUI option to set the DSCP value for its 
BitTorrent packets, which permits any old value you want.

So I'd expect Vuze (sponsor of Azureus) to add user-controllable knobs 
to set backoff parameters, consistent with their evident design 
philosophy.  The AIMD algorithms are an essential part of the Internet 
infrastructure, even though they're implemented in code that runs on the 
end system, so any large-scale alteration is going to have some very 
interesting consequences to other infrastructure components that 
implement RED and smart discard strategies. Consequently, I'm very 
skeptical of putting a lot of power over these things in the hands of 
the pirates. We know what their intentions are.

That being said, I'm all in favor of limited experiments. Just don't 
break the Internet in the process.

RB

Nicholas Weaver wrote:
>> highlighting the value of UDP-based file sharing for avoiding 
>> TCP-based fair traffic systems. It's clear that the users of 
>> BitTorrent see this as simply another means of grabbing more network 
>> resources.
>>

-- 
Richard Bennett

---

From: "Woundy, Richard" <[email-masked]>
Date: Mon, 1 Dec 2008 19:11:47 -0500

>IF the algorithm is equivalent to what was presented at the IETF,
namely, a delay-based ADDITION to a TCP-like AIMD, this will only have a
net benefit, as this will still stabilize out like any other TCP flow
(so no effect on the ISP congestion problem one way or the other), but
will have a huge benefit on the buffer-oversizing congestion problem.

Have we established that there is "no effect on the ISP congestion
problem one way or the other"? I missed that.

I could imagine that a LEDBAT flow with delay-based controls might
stabilize at a lower transmission rate than a TCP flow.

-- Rich

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Nicholas Weaver
Sent: Monday, December 01, 2008 4:54 PM
To: Richard Bennett
Cc: [email-masked]; Nicholas Weaver
Subject: Re: [ledbat] "Oh Noes, the Internetz will Melt Down..."

---

From: Richard Bennett <[email-masked]>
Date: Mon, 01 Dec 2008 16:17:02 -0800

How serious is this problem with cable modems that get buffer overflow 
because they don't flow-control their Ethernet links? It sounds like a 
poor implementation that can be dealt with at level 2 without disturbing 
RED and similar tweaks.

I'd be glad to see a real scavenger transport class, but remain 
skeptical that it can be implemented effectively solely in a network 
endpoint. This is one of the things that DSCP was supposed to do, after all.

RB

-- 
Richard Bennett

---

From: "Woundy, Richard" <[email-masked]>
Date: Mon, 1 Dec 2008 20:07:41 -0500

poor implementation...

I stumbled across this paper today,
http://www.imconf.net/imc-2007/papers/imc137.pdf. This text comes from
section 4.3.2:

"We found that 26.2% of the DSL hosts show a RED-style drop policy on
their upstream queues. The three providers

owned by AT&T (i.e., Ameritech, BellSouth, and PacBell) exhibit
deployment rates between 50.3% and 60.5%, whereas

all other DSL providers' deployment rates are below 23.0%. The partial
deployment of RED-style policies within ISPs

could be due to heterogeneity in the ISPs' equipment. We did not detect
RED in any of the cable ISPs measured."

I suspect there are a lot of "poor implementations" out there, across
multiple ISP technologies.

-- Rich

________________________________

From: Richard Bennett [mailto:[email-masked]] 
Sent: Monday, December 01, 2008 7:17 PM
To: Woundy, Richard
Cc: Nicholas Weaver; [email-masked]
Subject: Re: [ledbat] "Oh Noes, the Internetz will Melt Down..."

I'd be glad to see a real scavenger transport class, but remain
skeptical that it can be implemented effectively solely in a network
endpoint. This is one of the things that DSCP was supposed to do, after
all.

RB

Woundy, Richard wrote: 

	IF the algorithm is equivalent to what was presented at the
IETF,

 
 
-- Rich
 
-----Original Message-----

	I'm glad so many people have read my article, and I'd be happy
to  
	discuss it at length if it's appropriate. Broadband Reports, the

	info hub for file-sharers, ran an article on uTorrent 1.9 over
the  
	weekend under the title: "New UDP uTorrent Takes Aim At
Throttling,"  
	highlighting the value of UDP-based file sharing for avoiding
TCP- 
	based fair traffic systems. It's clear that the users of
BitTorrent  
	see this as simply another means of grabbing more network
resources.
	 
	That being said, I'll gladly stipulate that BitTorrent, Inc.,
means  
	well and has a heart of gold. But what happens when users can
tune  
	congestion avoidance and backoff parameters through a GUI? I'll
go  
	out on a limb here and say some of them may not be altruistic.

---

From: Nicholas Weaver <[email-masked]>
Date: Mon, 1 Dec 2008 17:16:02 -0800

> problem one way or the other"? I missed that.

Good point, and I messed up.  A better way of putting it in retrospect  
is:

The delay based addition can't make the ISP congestion problem worse,  
because its Min(delay based, AIMD based) in prefromance.

It is an open question whether it can make the ISP congestion problem  
better, as at the common-points of congestion, the buffer sizing may  
be too small to provide a reliable indication of impending congestion  
in an ISP network.

---

From: Richard Bennett <[email-masked]>
Date: Mon, 01 Dec 2008 17:59:28 -0800

Interesting paper, it's by the Glasnost people.

RED is a whole different animal than the buffer problem that's been 
presented here. RED (and AQM generally) is an attempt to prevent cycling 
by reducing stream rates before buffers overflow. It's quite different 
from tail-dropping an oversized buffer, and would not lead to 5 second 
delays on the DSL upstream.

RB

Woundy, Richard wrote:
>
> > How serious is this problem with cable modems that get buffer 
> overflow because they don't flow-control their Ethernet links? It 
> sounds like a poor implementation...
>
>  
>
> could be due to heterogeneity in the ISPs' equipment. We did not 
> detect RED in any of the cable ISPs measured."
>
>  
>
> ------------------------------------------------------------------------
>
> *From:* Richard Bennett [mailto:[email-masked]]
> *Sent:* Monday, December 01, 2008 7:17 PM
> *To:* Woundy, Richard
> *Cc:* Nicholas Weaver; [email-masked]
> *Subject:* Re: [ledbat] "Oh Noes, the Internetz will Melt Down..."
>
>  
>
> poor implementation that can be dealt with at level 2 without 
> disturbing RED and similar tweaks.
>
> From: [email-masked] <mailto:[email-masked]> [mailto:[email-masked]] On Behalf
> Of Nicholas Weaver
> Cc: [email-masked] <mailto:[email-masked]>; Nicholas Weaver
> [email-masked] <mailto:[email-masked]>

-- 
Richard Bennett

---

From: Stanislav Shalunov <[email-masked]>
Date: Tue, 2 Dec 2008 00:30:38 -0800

Some clarification of terminology --

uTP is, technically, a transport framing over UDP.  Congestion control  
used is, indeed, what I described in Minneapolis.

DNA is the BitTorrent Inc commercial product for content delivery.   
The protocol is BitTorrent, but DNA is a closed system with only one  
tracker and only one client implementation.  It uses only uTP for  
transport.

uTorrent is a free implementation of the BitTorrent protocol  
distributed by BitTorrent Inc.  The new version, 1.9, uses uTP along  
with TCP.

So the new version of uTorrent more widely deploys the transport, and  
the congestion control (both referred to as uTP in the quote below),  
already used in DNA.  It is designed to be less aggressive than TCP  
and has been useful to reduce latency in DNA.

-- Stas

On Dec 1, 2008, at 8:23 AM, Lars Eggert wrote:

> is designed to minimize latency, but still maximize bandwidth when  
> the latency is not excessive. We use this for communication between  
> peers instead of TCP, if both sides support it. In addition, we use  
> uTP, but connection attempts were not initiated by default, and  
> there was no control over TCP as described above. You can enable it,  
> but likely you will see the uTP connections not transfering much  
> data, because they are pushed out of the way by TCP."
>
> than-best-effort CC. (I believe someone mentioned in Minneapolis  
> that utorrent would implement BitTorrent's DNA - maybe this is it?
>
> Lars
>
>

--
Stanislav Shalunov

---

From: "Saverio Mascolo" <[email-masked]>
Date: Tue, 2 Dec 2008 10:27:47 +0100

i guess uTP could be similar to TCP santa cruz...forward Tfw is monitored so
that Tfw  increasing trigger the stop of the tcp probing phase. at that time
you may set the cwnd in different way...
is also utp implemneting rate-based features?

best
saverio

On Tue, Dec 2, 2008 at 9:30 AM, Stanislav Shalunov <[email-masked]>wrote:

> uTP is, technically, a transport framing over UDP.  Congestion control used
> is, indeed, what I described in Minneapolis.
>
> DNA is the BitTorrent Inc commercial product for content delivery.  The
> protocol is BitTorrent, but DNA is a closed system with only one tracker and
> only one client implementation.  It uses only uTP for transport.
>
> uTorrent is a free implementation of the BitTorrent protocol distributed by
> BitTorrent Inc.  The new version, 1.9, uses uTP along with TCP.
>
> So the new version of uTorrent more widely deploys the transport, and the
> congestion control (both referred to as uTP in the quote below), already
> used in DNA.  It is designed to be less aggressive than TCP and has been
> useful to reduce latency in DNA.
>
> -- Stas
>
>
>
>
>> uTP, the micro transport protocol. This UDP-based reliable transport is
>> designed to minimize latency, but still maximize bandwidth when the latency
>> is not excessive. We use this for communication between peers instead of
>> TCP, if both sides support it. In addition, we use information from this
>> transport, if active, to control the transfer rate of TCP connections. This
>> means uTorrent, when using uTP, should not kill your net connection - even
>> if you do not set any rate limits.
>>
>> What was in 1.8.1:
>> uTP, but connection attempts were not initiated by default, and there was
>> no control over TCP as described above. You can enable it, but likely you
>> will see the uTP connections not transfering much data, because they are
>> pushed out of the way by TCP."
>>
>> This last bit sounds a lot like if uTP is is using some sort of
>> less-than-best-effort CC. (I believe someone mentioned in Minneapolis that

-- 
Prof. Saverio Mascolo
Dipartimento di Elettrotecnica ed Elettronica
Politecnico di Bari
Via Orabona 4
70125 Bari
Italy
Tel. +39 080 5963621
Fax. +39 080 5963410
email:[email-masked] <[email-masked]>

http://www-dee.poliba.it/dee-web/Personale/mascolo.html

=================================
This message may contain confidential and/or legally privileged information.
 If you are not the intended recipient of the message, please destroy it.
Any unauthorized dissemination, distribution, or copying of the material in
this message, and any attachments to the message, is strictly forbidden.
_______________________________________________
ledbat mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ledbat

---

From: Richard Bennett <[email-masked]>
Date: Fri, 05 Dec 2008 11:51:52 -0800

Readers enjoyed by last article on BitTorrent uTP so much I've done a
follow-up for The Register, available here:

http://www.theregister.co.uk/2008/12/05/richard_bennett_bittorrent_udp/

It raises some issues that are relevant to the deliberation of this
group, mentions you all by name, and is a quick read. Here's a teaser:

The internet's TCP/IP protocol doesn't work very well. As the
internet's traffic cop, it's supposed to prevent applications from
overloading the network, but it's at a loss when it comes to managing
P2P applications. This deficiency, generally known to network engineers
but denied by net neutrality advocates, has been a central issue in the
net neutrality debate. BitTorrent Inc has now weighed in on the side of
the TCP/IP critics.

The next official release of the uTorrent client – currently in
alpha test – replaces TCP with a custom-built transport protocol called
uTP, layered over the same UDP protocol used by VoIP and gaming.
According to BitTorrent marketing manager Simon Morris, the motivation
for this switch (which I incorrectly characterized in The Register
earlier this week as merely another attempt to escape traffic shaping)
is to better detect and avoid network congestion.

Morris also told the media this week that TCP only reduces its
sending rate in response to packet loss, a common but erroneous belief.
Like uTP, Microsoft’s Compound TCP
begins to slow down when it detects latency increases. Even though TCP
is capable of being just as polite as BitTorrent wants uTP to be, the
fact that it hides its delay measurements from applications makes it
troublesome for P2P clients with many paths to choose from. But it’s
sensible to explore alternatives to TCP, as we’ve said on these pages
many times, and we’re glad BitTorrent finally agrees.

It remains to be seen whether uTP is the best approach. The company
has touted its close relationships with ISPs and the IETF’s LEDBAT task
group, but has so far shared none of the specifics of uTP operation in
public fora. The PowerPoints they’ve shared with the IETF are
encouraging, but they’re a long way from the source code, simulations,
and hard data from impartial sources that are prerequisite to any new
protocol standard.

Enjoy,

RB

---

From: Nicholas Weaver <[email-masked]>
Date: Fri, 5 Dec 2008 12:37:19 -0800

Unfortunately there are continued errors in your article.

On Dec 5, 2008, at 11:51 AM, Richard Bennett wrote:
> sending rate in response to packet loss, a common but erroneous  
> belief. Like uTP, Microsoft’s Compound TCP begins to slow down when  
> it detects latency increases. Even though TCP is capable of being  
> just as polite as BitTorrent wants uTP to be, the fact that it hides  
> its delay measurements from applications makes it troublesome for  
> P2P clients with many paths to choose from. But it’s sensible to  
> explore alternatives to TCP, as we’ve said on these pages many  
> times, and we’re glad BitTorrent finally agrees.
>

And Compound TCP is the exact opposite requirements as uTP, the goal  
is something which is NOT outcompeted by AIMD, if I'm groking the  
description correctly.  Compound-TCP uses a sum-of-windows method, an  
AIMD window and a delay-based window.

uTP is a "Minimum of AIMD/delay based" window sizing algorithm.

The implications are huge.

Compound TCP is designed to get the delay-based behavior on very fast,  
high latency networks, where even a very small loss rate introduces  
very bad behavior to AIMD based congestion control and, in the absence  
of congestion, an AIMD based sender would back off and not utilize the  
link.

Yet by using a sum-of-window with a second AIMD based window, it is  
fully competitive with TCP flows.  This is actually more agressive  
than conventional AIMD, except that the delay-based window gets  
outcompeted by other AIMD flows, so it becomes effectively AIMD based  
in terms of agressiveness.

Thus under competition, Compound TCP can be considered AT LEAST AIMD- 
only for fairness/agressiveness, because it uses the sum of the two  
windows so the AIMD window will compete like any other flow and  
(hopefully) the delay-based window will go to 0.

Thus Compound TCP does NOT back off earlier.  In fact, the feature is  
that it does NOT back off on a packet drop if it is on a high  
bandwidth, high latency network!

uTP, by being a "Minimum of two window" algorithm, is the opposite, as  
it is AT MOST AIMD-only for fairness/agressiveness, precisely because  
it takes advantage of the side effect of delay based congestion  
control: it backs off earlier than AIMD-based flows so it gets starved  
out by AIMD-based flows (hopefully).

In almost all other TCP applications, the starvation problem faced by  
delay-based window sizing is a problem.  For a scavenger-class  
protocol like uTP, it is potentially a feature.

Thus all TCP stacks are AT LEAST AIMD-aggressive, including Compound  
TCP, as it uses delay-based metrics to have the exact opposite  
behavior: attempting to grab more bandwidth in conditions where AIMD- 
based systems fail.  Because any TCP stack that is not AT LEAST AIMD- 
aggressive is going to face starvation under congestion.

The open question is "how much friendlier" is (min(AIMD, delay based))  
in practice.

Delay-based measurements are notoriously noisy and jitter prone, so it  
may very well become "just AIMD" in terms of ISP congestion behavior,  
which means it isn't more aggressive than TCP but isn't less  
aggressive, thus it isn't a real scavenger-class service.

And this isn't about replacing TCP, its about alternate congestion  
control algorithms which are friendlier than AIMD (TCP Reno).

The window sizing algorithms from uTP could just as easily be an  
option in a TCP stack (and I've argued it should be a single-sided  
option: one side of a TCP stack should be able to control both sending  
and receiving windows in this manner).

And its really just observing that "Not all flows are really equal  
under congestion", so if you can designate a flow as "less important",  
you get better aggregate behavior, and because you have some points of  
congestion (eg, the user's uplink) where only the individual user  
experiences it, there are incentives for applications to actually use  
such a property.

---

From: Richard Bennett <[email-masked]>
Date: Fri, 05 Dec 2008 13:10:05 -0800

That's a nice analysis of TCP congestion-avoidance mechanisms, Nick, but 
I'm not sure it's any more accurate than my simplification for the mass 
audience. I don't argue that a scavenger class isn't needed, I simply 
express my skepticism for some of the claims that have been made for the 
uTP approach. As you point out, TCP implementations have a number of 
tweaks and variations for congestion avoidance, many of them RTT-based, 
so it's not correct to say that TCP just waits for packet drop to slow 
down. TCP over ECN certainly doesn't, but I've written about that in the 
press previously.

The main concern I have about the uTP approach is that the system is 
simply an endpoint implementation, and endpoints don't have all the 
information about the status is ES-IS and IS-IS links they need for 
assessing a path. A single-product hack has to go on the information it 
has, but an IETF standard can take a larger focus and combine feedback 
from the infrastructure with path selection to get a better answer. As 
you point out, delay measurements are jitter-prone, and after smoothing 
it's not clear that they're any better than AIMD. Latency is also not a 
constant for a flow.

A secondary concern is the likelihood of false positives in path 
selection due to the inability of legacy traffic shapers to deal with 
UDP correctly, especially when the payload is obfuscated.

And the third concern is to effect on flow optimizers, routers, and NATs 
of P2P streams moving around much faster than they do today, and of P2P 
streams growing in number relative to existing practice because they're 
probing more paths. These things all impact the infrastructure.

But thanks again for your assessment of Compound TCP.

RB

Nicholas Weaver wrote:
>> its delay measurements from applications makes it troublesome for P2P 
>> clients with many paths to choose from. But it’s sensible to explore 
>> alternatives to TCP, as we’ve said on these pages many times, and 
>> we’re glad BitTorrent finally agrees.
>>
>
> Thus under competition, Compound TCP can be considered AT LEAST 
> AIMD-only for fairness/agressiveness, because it uses the sum of the 
> two windows so the AIMD window will compete like any other flow and 
> behavior: attempting to grab more bandwidth in conditions where 
> AIMD-based systems fail.  Because any TCP stack that is not AT LEAST 
> AIMD-aggressive is going to face starvation under congestion.
>
>
>

-- 
Richard Bennett

---

From: Murari Sridharan <[email-masked]>
Date: Fri, 5 Dec 2008 13:48:31 -0800

Nick is mostly right in how he summarized CTCP. It is designed to fairly compete with Standard TCP flows but unlike expectation from LEDBAT it does not backoff when other flows are present. It however has one operating region where it tries to maintain a certain backlog in the bottleneck buffers using sum of both cwnd and dwnd and during this time the loss based flow (cwnd) tries to catch up and the delay based window dwnd reduces, eventually to zero. In this operating region the total amount of packets we keep backlogged is not dependent on RTT and it tries to maintain low buffer occupancy. Once the loss based value fully assumes the total value it behaves like regular TCP and does AIMD and eventually overflows the buffers to produce a loss. Hope this makes sense.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Nicholas Weaver
Sent: Friday, December 05, 2008 12:37 PM
To: Richard Bennett
Cc: [email-masked]; Nicholas Weaver
Subject: Re: [ledbat] Another uTP story by alarmist author

On Dec 5, 2008, at 11:51 AM, Richard Bennett wrote:
> belief. Like uTP, Microsoft's Compound TCP begins to slow down when
> P2P clients with many paths to choose from. But it's sensible to
> explore alternatives to TCP, as we've said on these pages many
> times, and we're glad BitTorrent finally agrees.
>

---

From: Murari Sridharan <[email-masked]>
Date: Fri, 5 Dec 2008 13:57:32 -0800

To be more clear in this region it maintains its sending rate and does not increase. To get a better sense of this window evolution look at the CTCP window evolution slide in attached ppt. The ppt is a bit old on the results section. Its not drawn to scale but simply to illustrate the concept. The dwnd control law the second equation where we actually reduce the rate to maintain gamma packets is the key to this operating region.

Tying this back to the uTP discussion, CTCP does not have the same negative effects of a vanilla Vegas like delay implementation it is design to *compete*. LEDBAT approaches like uTP will back off and could potentially starve for long periods of time depending on the competing traffic and we need to design how we want to pace the flow. The Low Pri TCP approach I presented at the LEDBAT WG does this as well. It backs off completely and paces itself and figures out if there is residual capacity in the network.

Thanks.