Re: QUIC – Will it Replace TCP/IP?

[Michael Tuexen]

> On 29. Apr 2020, at 22:45, Matt Joras <matt.joras at gmail.com> wrote:
> 
> Full disclosure: I work on a QUIC implementation for my $DAYJOB (not msquic).
> 
> On Wed, Apr 29, 2020 at 12:58 PM Michael Tuexen <tuexen at freebsd.org> wrote:
>> 
>>> On 29. Apr 2020, at 14:13, Randall Stewart <rrs at netflix.com> wrote:
>>> 
>>> That is interesting and something we may in the future
>>> interest us.. but I am not sure what exactly is
>>> the gain we get from QUIC?
>> From https://github.com/microsoft/msquic
>> 
>> (a) Handshake authenticated with TLS 1.3
>> (b) All packets are encrypted
>> (c) Parallel streams of application data.
>> (d) Improved (compared to TCP) congestion control and loss recovery.
>> (e) Exchange application data in the first round trip (0-RTT).
>> (f) Survives a change in the clients IP address or port.
>> (g) Easily extendable for new features (such as unreliable delivery).
>> 
>> (a) and (b) are also true for the payload if you use TLS 1.3/TCP, but not
>> for the transport information.
> Indeed, this is something that is a benefit for QUIC implementers but
> also "problematic" for network operators. By keeping most of the
> transport encrypted, "middle boxes" on the path can make very few
> assumptions about how QUIC works, which prevents the gradual
> ossification of protocol features we have seen with TCP, as many
> features are not usable on the general Internet.
I guess you can consider a QUIC connection like a VPN tunnel...
> 
>> 
>> (c) is application dependent. If only a small
>> number of streams are needed, you can use multiple TCP connections. (d) is
>> interesting. The CC in MSQUIC is Cubic, which is also available for TCP. I'm not
>> sure about the relation of RACK loss recovery and QUIC loss recovery. Would
>> be interesting to know.
> 
> If you are interested, I highly recommend you read the current
> recovery draft, it is accessible:
> https://quicwg.org/base-drafts/draft-ietf-quic-recovery.html
I'm aware of it. QUIC has some benefits, like packet numbers, which
TCP right now hasn't (one could use time stamps for a similar thing),
but I don't have a feeling how much do you really gain by this
in real networks. I mean this seriously, I'm not aware of any scientific
papers on that. 
> In general the IETF charter for QUIC is to _not_ innovate in the
> recovery and congestion control space, but rather integrate the
> current best practices for TCP (from published RFCs). The same
That is what I said. The TCP variant looks like RACK, which is implemented.
> algorithms that apply to TCP generally end up being easier to
> implement in QUIC, as you do not need to deal with things like
> sequence number ambiguity for retransmissions (QUIC packet numbers are
> monotonically increasing). For congestion control, the draft describes
Sure. I'm aware of this. Just don't know how much you really gain.
> a variant of Reno, not Cubic, though most major deployments will
> probably end up using their own implementation of Cubic or BBR or
> something else.
They can't use anything else than New Reno, since they need something
described in a proposed standard.

I was referring to Cubic, since I was citing the text from the MS page
and by looking at the code I figured out they have implemented Cubic.
But Cubic and BBR are also available on TCP. So I'm wondering why this
comes up as an argument for QUIC.
> 
>> I guess (e) can be done also with TCP FO and TLS 1.3.
>> The difference might be the length of the initial flight.
> Technically yes, though TCP FO has the general problem of middle
> boxes, often home routers, blocking SYNs with payloads.
So do, some don't. But you can encapsulate TCP in UDP and would get
around it.
> 
>> (f) is not possible
>> with base TCP, but MPTCP has been deployed for that use case, as far as I know.
>> (g) is harder to achieve...
> MPTCP is much more ambitious and complex than the connection migration
> described in the QUIC drafts. MPTCP nominally provides a way to have
> Active/Active paths, rather than simply migrating to a new path. MPTCP
> has also not seen wide adoption outside of Apple's usecase, AFAIK. An
> implementation has only recently become available on Linux, and never
> passed a prototype state for FreeBSD.
I know this it is more complex, just wanted to say that it is available
if needed. I think it is used on MacOS/iOS devices. Whether it is available
on FreeBSD is also a question of someone needing it. The prototype was based
on some University activity, not on a company.
> 
>> Please note that the transport features are available via SCTP, but not used
>> that much. Partial unreliability is available in WebRTC data channels, but I
>> don't know if it is used that much.
> 
> SCTP, being a new L4, has the problems of actually being usable on the
> Internet. The whole reason QUIC is built on top of UDP is that
> operators allow a whitelist of protocol types, including UDP.
And you could use SCTP/UDP, which is specified in 2013.
> 
>> For me it looks like the benefits are:
>> * Increased level of encryption
>> * Extensibility (especially if you control both sides)
> I would agree with this but also add to it.
> 
> From personal experience, the loss recovery described in the QUIC
> drafts outperforms stock TCP in Linux even when the congestion control
> algorithms are the same. This is measurable in both strictly transport
> terms (retransmissions, TLPs, etc) but also in real application
> metrics (e.g. video MTBR, request latency, etc.). This could be
That is interesting. Are you using RACK on Linux?
> achieved with TCP with tighter control of OS's TCP stack, or by doing
> something like TCP over UDP, but I would actually argue the
> development cost of changing an OS TCP stack is higher for most places
> than utilizing or creating their own QUIC implementation. You an
> already see this, in practice. There are over a dozen QUIC
> implementations at various stages of maturity (see a list here:
> https://github.com/quicwg/base-drafts/wiki/Implementations). There are
When we did SCTP back in 2000, there were I think about 20 implementations
available, which have reduced over time to kernel implementation for
the OSes where it is relevant and some commercial userland stacks.

Recently the number grew again because they are used in WebRTC implementations.
> probably more people hacking on QUIC right now than major OS TCP
> implementations, and QUIC isn't even an RFC yet. QUIC is highly
That might be true.
> accessible as a protocol. There are only 4 drafts to read, compared to
> decades of TCP RFCs and drafts and "de facto" standards.
I'm pretty sure will change. In SCTP we started with a single spec (not
four, but we didn't capture crypto). Then there are the partial reliability
extension (which is something the QUIC datagrams), Dynamic address reconfiguration,
which allows for connection migration, Dynamic stream reconfiguration (which
allows resetting the streams (SCTP reuses stream IDs)), SCTP Authentication
which adds the capability that the endpoints ensure no-one changed anything
on the fly, UDP encapsulation, improved failover modes, and so on. It is
just a question of time, I guess.
> 
> Beyond this QUIC being typically implemented in userspace has
> advantages and disadvantages. At the cost of CPU, which is a cost that
> can gradually be brought down, applications can benefit from a tighter
> integration with the transport than you can achieve with syscalls and
> BSD sockets. This, when combined with the inherent extensibility
This assumes that the data you want to serve is available in userland.
What if your data is in the kernel, like on a disk. Something like
sendfile() allows you to send the data without looping through userland
for the transport stack. I know this might not be the common case, but
it is the case some people are focussing on.
> relative to TCP, makes it a very appealing protocol if you have
> concerns beyond throughput per CPU cycle.
> 
> I don't think TCP is going anywhere any time soon, but it's almost
> certain that within the next decade total QUIC traffic on the internet
> will be similar to the volume of TCP.
I agree the TCP will not go away anytime soon. QUIC will also evolve as
a protocol. Hard to say how that will go..

Best regards
Michael
> 
>> This might map differently to business cases.
>> 
>> Best regards
>> Michael
>>> 
>>> This is something I have debated with some of my colleagues for a while now.
>>> 
>>> Thinking in terms of NF we have:
>>> 
>>> 1) 1 audio connection
>>> <and>
>>> 2) 1 video connection (sometimes 3, but if we have 3 we take each
>>>  request and divide it amongst the 3 connections.. we have to
>>>  wait for all of them to complete).
>>> 
>>> So I don’t need QUIC for HOL blocking
>>> 
>>> We have TLS 1.3, so that means I have the quick start thing. And we
>>> also have a TCP over UDP stack that means I can run, if I want to,
>>> my own TCP stack on each client… which means I can enable the
>>> TCP-fastopen… so I can get the 0-RTT start of quick (which means
>>> I can shave 100ms off of play delay… which in terms of a 1 hour
>>> movie is not that big of deal)..
>>> 
>>> I had this same angst when I got to NF since SCTP offered all of
>>> the features QUIC does now (and some more by the way) years ago.. but
>>> I could not justify:
>>> 
>>> 1) The development cost
>>> <and>
>>> 2) The CPU cost (QUIC is more costly then TCP just like SCTP is more costly)
>>> 
>>> So I am unsure of the benefit here. There is nothing new or novel in QUIC
>>> that other transports have not done or are already doing. And unless there
>>> is some huge benefit so I get a gain in QoE or CPU performance I can not
>>> get from TCP I wonder why I need it?
>>> 
>>> I would love to know of some feature in QUIC that makes it a
>>> undisputed gain that cannot be done in TCP and that would sell it
>>> to me .. but I just don’t know what that is..
>>> 
>>> 
>>> Suggestions on this marvelous feature, that I may
>>> not know about... would be most welcome…
>>> 
>>> R
>>> 
>>> 
>>> 
>>>> On Apr 29, 2020, at 8:03 AM, Scheffenegger, Richard <Richard.Scheffenegger at netapp.com> wrote:
>>>> 
>>>> FYI only.
>>>> 
>>>> 
>>>> Richard Scheffenegger
>>>> 
>>>> 
>>>> -----Original Message-----
>>>> From: Eggert, Lars <lars at netapp.com>
>>>> Sent: Mittwoch, 29. April 2020 13:01
>>>> Subject: Re: QUIC – Will it Replace TCP/IP?
>>>> 
>>>> Hi,
>>>> 
>>>> Microsoft just announced that they are open-sourcing their QUIC stack (under an MIT license).
>>>> 
>>>> MSQUIC is suitable for application and kernel-use (it will ship as part of the Windows kernel), so it might be usable in other kernel contexts (FreeBSD/ONTAP, Linux) as well. The amount of work needed is likely (much) less than porting another QUIC stack that is only meant for user-space use. Also, based on my testing, MSQUIC is amongst the faster and most interoperable stacks.
>>>> 
>>>> In short, if we wanted to do something with QUIC, MSQUIC would probably a good starting point.
>>>> 
>>>> Lars
>>>> 
>>>> 
>>>> On 2020-4-29, at 2:38, Nick Banks <nibanks=40microsoft.com at dmarc.ietf.org> wrote:
>>>>> 
>>>>> Hey Folks,
>>>>> 
>>>>> We just open sourced the Microsoft QUIC implementation, MsQuic:
>>>>> https://github.com/microsoft/msquic. Feel free to read a little bit
>>>>> about it in this post:
>>>>> https://techcommunity.microsoft.com/t5/networking-blog/msquic-is-open-
>>>>> source/ba-p/1345441
>>>>> 
>>>>> Thanks,
>>>>> - Nick
>>>> 
>>> 
>>> ------
>>> Randall Stewart
>>> rrs at netflix.com
>>> 
>>> 
>>> 
>> 
>> _______________________________________________
>> freebsd-transport at freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-transport
>> To unsubscribe, send any mail to "freebsd-transport-unsubscribe at freebsd.org"