Last Wednesday, we were looking at approaches to congestion avoidance. I.e., approaches that allow a sender to send as much TCP (or transport layer) traffic as possible while avoiding in between routers becoming congested.
We said that we could split approaches into those where the router is given some additional capabilities, and those where the sender tries to figure out how congested the routers along a path/flow are.
We looked in detail at RED (random early detection) where routers estimate their average queue length via a weighted running average of last several queue lengths as well as maintain a MinThreshold and MaxThreshold. If the average queue length is less than MinThreshold all new packets are queued and forwarded, if it is greater than MaxThreshold, all new packets are discarded, and if it is between these bounds packets are dropped with some probability P.
We then looked at a more commonly deployed router scheme of explicit congestion notifications (ECN).
Finally, we gave some short example Source Based Congestion Avoidance schemes which led into a discussion on TCP Vegas.
We start today by reviewing what we'd said about TCP Vegas.
TCP Vegas (Brakmo Peterson 1994)
Rather than look at the flattening of the throughput to detect congestion, you could imagine comparing the measured throughput versus expected throughput.
This is the idea of TCP Vegas. TCP Vegas is not widely deployed yet, but the strategy it uses has been adopted by other implementations of congestion avoidance that are now being deployed.
As an example, the top graph above graphs the congestion window as earlier, the middle graph shows the average sending rate as measured at the source, and the bottom graph shows the average queue length at the bottleneck router.
Notice between 4.5 and 6 seconds the congestion window increases, but the throughput stays flat. This is because the throughput cannot increase beyond the available bandwidth. So increasing the window size beyond this point, only results in packets taking up buffer space at the bottleneck router (as seen on the bottom graph).
So if we measure the throughput we are achieving versus what we'd expect by increasing the window, we can tell when congestion is starting.
TCP Vegas Details
To estimate the throughput rate, we could compute (amount of data in-flight)/(RTT if no other traffic).
We define BaseRTT as the RTT of a packet when the flow is not congested. To estimate BaseRTT, TCP Vegas sets its value to the minimum
of all measured round-trip times. This usually works out to be the RTT of the packet sent by the connection, before router queues increase due to traffic.
If we are not overflowing the connection, then it is reasonable to assume that the CongestionWindow is roughly the number of bytes in transit.
This gives the estimated throughput rate that TCP Vegas uses as:
`\mbox{ExpectedRate} = \mbox{CongestionWindow}/\mbox{BaseRTT}`.
TCP Vegas calculates the current sending rate, ActualRate, by recording the sending time for a distinguished packet, recording how many bytes are sent between when that packet was first sent and when it is acknowledged, computing the sample RTT for this packet, and dividing the number of bytes transmitted by this sample RTT. This calculation is done once/RTT.
TCP Vegas computes Diff = ExpectedRate - ActualRate and uses two thresholds, `\alpha` and `\beta`, to determine what to do with the window size.
If `\mbox{Diff} < \alpha`, TCP Vegas linearly increases the congestion window during the next RTT.
If `\mbox{Diff} > \beta`, TCP Vegas linearly decreases the congestion window during the next RTT.
If `\alpha <= \mbox{Diff} <= \beta`, TCP Vegas leaves the congestion window unchanged.
TCP Vegas Behavior
The further the ActualRate gets from the ExpectedRate, the more congestion there is in the network, so it makes sense to reduce the sending rate. This is what the `\mbox{Diff} > \beta` condition on the previous slide does.
On the other hand, if the ActualRate is too close to the ExpectedRate, the connection might not be utilizing all of the available bandwidth, so we should increase the CongestionWindow. This is what the `\mbox{Diff} < \alpha` condition on the last slide does.
The top figure above shows the congestion window when TCP Vegas is in use. The hashes at the top indicate times when packets were sent.
The bottom graph traces expected (colored line) and actual (black line) throughput rates.
The wide shaded strip gives the regions between `\alpha` and `\beta`.
The goal is to keep the Actual rate within the shaded region. Observed when it goes above the strip, the CongestionWindow in the top graph is increased. Similarly, if it goes below the shaded region, the CongestionWindow is decreased.
Notice in the above, we don't get packet timeouts. For this reason, TCP Vegas' linear decrease is still okay as an early decrease before congestion is occurring. If we had a timeout, TCP Vegas would do a multiplicative decrease to ensure network stability as before.
Setting `\alpha` and `\beta`
As TCP Vegas compares the difference between the actual and expected throughput rates to the `\alpha` and `\beta` thresholds, these two thresholds are defined in terms of `kBps`.
Often people of think in terms of how many extra buffers the connection is occupying in the network.
For example, if BaseRTT is 100ms, a packet is 1kB, then if `\alpha = 30kBps` and `\beta = 60kBps` and if a buffer can hold the amount of in flight data, then `alpha` entails that the connections needs to be occupying at least 3 extra buffers, and `beta` is saying it should be occupying no more than 6 extra buffers.
In practice, people usually use a value of 1 buffer for `alpha` to 3 buffers for `beta`.
Quiz
Which of the following is true?
Slow start makes use of both a CongestionWindow and a CongestionThreshold.
If fast recovery is used, we retransmit if we see three duplicate ACKs.
Explicit Congestion Notification (ECN) is a source-based congestion avoidance scheme.
TCP BBR (Cardwell, Cheng, Gunn, Yeganeh, Van Jacobson 2016)
TCP BBR (Bottleneck Bandwidth and RTT) is a newer TCP Congestion control algorithm developed at Google.
Like Vegas, it is delay based - it tries to detect buffer growth so as to avoid congestion and packet loss.
Also like Vegas, BBR uses a minimum RTT and maximum RTT as calculated over some time interval as a control signal.
BBR introduces packet-pacing, bandwidth probing, and RTT-probing to further improve performance.
Packet-pacing spaces the packets based on an estimate of the available bandwidth so as to eliminate bursts and unnecessary queuing.
BBR periodically increases its rate, probing available bandwidth.
BBR periodically decreases its rate, probing for a new minimum RTT.
The RTT-probing mechanism attempts to be self-synchronizing - when there are multiple BBR flows, their respective RTT probes happen at the same time. This is done to give a more accurate view of uncongested path RTT.
BBR is still under-development and is rapidly evolving. One problem that is currently an area of research is fairness. For example, some experiments show CUBIC get 100 times less bandwidth when competing against BBR flow. Other experiments show that when BBR flows compete against each other, one can still get an unfair amount of service.
Another research area is avoiding high retransmission rates if using BBR.
DCTCP
As a final TCP source based congestion algorithm, we look at Datacenter TCP (DCTCP).
This algorithm was designed to work with ECN in datacenters.
As datacenters are self-contained, it is possible to deploy a tailor-made version of TCP that does not need to worry about treating other TCP flows fairly.
In the datacenter setting, one typically has low-cost white box switches that are provisioned without an excess of buffers as one doesn't have to worry about long-fat continent spanning links.
Recall in ECN, two bits in the IP TOS field are used - ECT to indicate ECN-Capable Transport, the other, CE, indicates congestion encountered. Two TCP option flags are also used: ECN-echo's sent to sender from receiver to indicate it saw a CE bit, and CWR flags to indicate from the sender to the receiver that it has reduced its window size.
In ECN, the actual active queue management (AQM) policy used is left to the implementation.
To do this in DCTCP, DCTCP first estimates the fraction of bytes that encounter congestion. At the end hosts, DCTCP then scales the congestion window based on the estimate.
Standard TCP (like CUBIC, Tahoe, or Reno) kicks in if a packet is actually lost.
DCTCP Details
Since the links are well known for the datacenter case, to decide whether the CE bit should be set, DCTCP uses:
`K > (mbox{RTT} times C)/7`
where `K` is the queue length and C is the link rate in packets/second.
The receiver then maintains a boolean variable for every flow denoted SeenCE. Using this, it processes each packet received as follows:
If the CE bit is set and SeenCE == false, set SeenCE = true, and send an immediate ACK.
If the CE bit is not set and SeenCE == true, set SeenCE = false, and send an immediate ACK.
Otherwise, ignore the CE bit.
In the "otherwise" case, the receiver continues to send delayed ACKs once every `n` packets, whether or not the CE bit is set.
The sender computes the fraction of bytes that encountered congestion during the previous observation window (usually chosen to be approximately the RTT), as the ratio of the total bytes transmitted and the bytes acknowledged with the ECE flag set.
DCTCP raises the congestion window in exactly the same way as the standard algorithm, but it reduces the window in proportion to how many bytes encountered congestion during the last observation window.
Quality of Service
Packet-switched networks support all kinds of applications and data, including multimedia data such as video and audio streams.
In the early days of the internet, there was an issue of whether there was sufficient bandwidth to handle video and audio.
Although this is less of an issue nowadays, there are other difficulties which make supporting audio and video streams difficult.
For example, if we want to carry an audio conversation between two people, we expect when one person talks, the other can hear that person
almost immediately.
Applications which are sensitive to such timeliness of data are called real-time applications.
Besides audio and video applications, one can imagine the remote control of objects such as robot arms, or player positions in an online video game, as other examples of real-time applications.
Even certain file transfer applications needed for updates of databases which need to be done in a non-busy window of time (such as overnight) can be thought of as real-time applications.
The distinguishing feature of real-time applications is that they need some assurance from the network that data are likely to arrive on time.
Unlike a nonreal-time application, where we might use non-timely retransmission to ensure data arrive correctly, we often relax correctness for timeliness.
To handle real-time applications, we would like a new service model, where services that need stronger assurances about how the network will perform can ask the network for them. The network can then respond with the level of assurance it can promise to provide.
Old-style applications that are happy with best-effort should be able to use this service model, their requirements just might be less stringent than those of real time applications.
Because of this, in this model, the network will treat some packets differently than others.
A network that supports different levels of services is said to support quality of service (QoS).
Requirements for QoS
As a first-cut on levels of quality of service that might be needed, we can divide applications into: real-time versus nonreal-time.
Nonreal-time applications include traditional applications such as ssh, file transfer, email, web browsing, and so on. These can all work without timeliness guarantees on the network.
Nonreal-time applications are sometimes called elastic because they are able to stretch gracefully in the face of increased delay.
Still, these applications can benefit from shorter delays. For example, ssh might benefit from shorter delays.
As an example of a real-time applications, consider an audio application such as the one illustrated in the above figure.
Data is generated by collecting samples from a microphone which are digitized using an A-D converter.
These samples are placed in packets, transmitted across the network, and received at the other end.
The receiver takes these packets and plays them back at an appropriate rate.
More on Audio Application Example
If the samples were collected at a rate of one sample/125 µs, they should be played back at the same rate.
So a sample can be thought of as having a playback time - the point in time at which it is needed in the receiving host.
If data arrive after their appropriate playback time, they are essentially useless.
One way to make our voice application work, would be make sure all samples take exactly the same amount of time to traverse the network. Then, since samples are injected at a rate of 125µs, they will appear at the receiver at the same rate, ready to be played back.
However, this is hard to guarantee for a packet-switched network.
To deal with variations in delivery time, the receiver can buffer data in reserve, thereby always providing a store of packets to be played at the right time.
As long as the variation in packet delivery times (early/late) is not too great, we can play the audio continuously, although it will be offset in time (this offset is called the playback point) from when the original audio was digitized.
The playback point requirements determine the kind of guarantees we want on the network. For audio communication we don't want this playback point to be greater than 300ms or people will notice.
The right figure above shows the typical packet delay variability across one path in the internet on a particular day. Although most packets easily meet the needed requirements, the distribution does have a long tail. To ensure all packets arrive on time, based on the above, we might want to set the playback buffer to 200ms.
Taxonomy of Real-Time Applications
The above diagram gives one way to classify applications, the top split being on whether they are nonreal or real-time.
Within real-time applications, the next split is based on whether they are tolerant or not to data loss. For example, an audio-app might be tolerant to the occasional lost of a packet. On the other hand, a robot control program might not be tolerant of any data loss.
The next split refers to how well the application can adapt if network circumstances change. For example, for the audio app, if the network permits we might be able to reduce the playback point of the buffering to improve the experience for the user, similarly if the network circumstance worsens we could add delay/do more buffering.
Within adaptive applications, some allow one to adjust delay, as we did for audio, other might allow one to adjust the bit-rate of data (for example send local quality images versus higher quality images). This gives rise to the last split in the above diagram - rate adaptive versus delay adaptive applications.
Approaches to QoS Support
There two broad categories of approaches to QoS:
Fined-grained approaches, which provide QoS to individual applications.
Coarse-grained approaches, which provide QoS to large classes of data or aggregated data.
Integrated Services is a QoS architecture developed in the IETF and often associated with the Resource Reservation Protocol that falls into the first category.
Differentiated Services, which is probably the most widely deployed QoS mechanism, falls into the second category.
We now start looking at each of these in turn.
Integrated Services (RSVP)
Integrated Services (aka IntServ) refers to a body of work produced by the IETF around 1995-1997.
The IntServ working group developed specifications for a number of service classes designed to meet the needs of some of the application types considered in our taxonomy.
It also defined how RSVP could be used to make reservations using these service classes.
