Net neutrality, explained

With the recent FCC action on net neutrality, a lot of people are worked up, but almost no one seems to know what it actually means. Here is what it is all about:

The Internet is a protocol for delivering small packets of data over a wide-area network. An important property of the internet protocol (for present purposes it is the most important property) is that packet delivery is best-effort. That means that internet routers will attempt to deliver every packet, but there are no promises. If a router gets overloaded, some packets will be dropped (i.e., not delivered).

The key question is this: when packets have to be dropped, how do you pick which ones? Historically, internet routers have generally chosen the packets to drop arbitrarily.

But for many applications, this is not actually what you want. For example, in streaming video (nearly 80% of all internet traffic) some frames are more important than others. Some frames are keyframes (which give the entire picture), while most frames only describe the differences from the previous frame. If you miss a regular frame, there is some degradation in the picture, but if you miss a keyframe, you have no picture until the next keyframe arrives. Consequently, it would be best if internet routers preferred keyframes to other frames. More generally, there is an area of research called quality of service, which seeks to enable applications that require a certain amount of reliable bandwidth, and typically does so by identifying and reserving network resources.

So what is network neutrality? Network neutrality is the principle that says that routers must not distinguish between packets when deciding which ones to drop.

As a technical matter, network neutrality would tie the hands of networking innovators by prohibiting any routing strategy that intelligently chooses which packets to favor. One can improve the network by adding more hardware, but not by using the hardware one already has more efficiently. (Note: some dispute this. I will return to this point later.)

So why do some people want network neutrality? There is concern about a certain sort of network management practice that ISPs might adopt. To explain this, it’s helpful to know a few statistics.

Over three-quarters of the internet traffic in North America is from video streaming. Netflix alone accounts for over a third of internet traffic. YouTube (owned by Google) accounts for another sixth. (Figures from December 2015.) This creates a tension between ISPs and content providers. The latter’s business strategy requires that they consume a brobdingnagian amount of network bandwidth, while the former are concerned that the content providers will overwhelm their networks.

Thus, the content providers are concerned that the ISPs will throttle back their content, or that the ISPs will demand that they help pay for it. And, in fact, on a few occasions, ISPs actually did something like this.

Back in 2007, when the concern was peer-to-peer file-sharing (mostly music and video), rather than video streaming, Comcast throttled back certain file-sharing protocols. On the face of it, one could argue it was reasonable, but, being Comcast, they did it in a super-sleazy way. Rather than announce they were doing it, they just did it secretly. People noticed, made a fuss, and Comcast stopped doing it. (Ironically, Comcast is now trying to position themselves as big supporters of network neutrality, while they literally did exactly the thing that net neutrality advocates are trying to prevent.)

Nevertheless, the Comcast incident is instructive, because it was corrected by the free market without any government intervention at all. The customers noticed, the customers complained, and Comcast stopped. Years later, AT&T blocked its customers from using Facetime on iPhones, and they too stopped after public outcry.

That’s the technical background. At this point it gets political. Google (which owns YouTube) made a big push for net neutrality, and it became a cause celebre on the left.

In 2010 the FCC announced its rules governing internet routing and they were quite bad for innovation. (You can find the relevant rules on pages 17944-17956 here, but the summary in the draft rules is much more accessible.) They were full of lip service about freedom to innovate, but they never defined the key terms of “nondiscrimination” and “reasonable network management.” (The final rules improved the draft rules by giving a non-circular definition of “reasonable network management,” but they defined it (¶82) in terms of “legitimate network management purposes” which they did not define.) They specifically say (¶83) that “reasonable network management” will be evaluated on a case-by-case basis. Consequently, if you want to stay out of trouble, you have to go to the FCC for permission to do anything.

At this point, there is one more important element. The FCC actually had no legal authority to regulate the internet, and when they tried to do it anyway, they lost in court. But, never ones to allow something like that to stop them, the Obama administration reclassified the internet from an information service (which they have no power to regulate) to a telecommunications service (which they do).

A telecommunications service is a service that transmits information without change in its form or content. An information service is a service for generating, acquiring, storing, transforming, processing, retrieving, utilizing, or making available information via telecommunications. (Via Ars Technica.) The sticky thing is the internet is pretty clearly both of these. Some applications are more like an information service: the web, search engines, social media, video streaming, file sharing, cloud drives; while other applications are more like telecommunications: email, instant messaging, voice over IP, online gaming, remote login. Certainly the vast majority of traffic is in the first category, but a case can be made for either.

It is harder to make a case to reclassify the internet as telecommunications service when it is already classified as an information service, particularly since most internet usage is moving in the opposite direction. But, when the matter went to the Court of Appeals, only one of the three judges was bothered by that. The other two applied the Chevron principle which requires that judges defer to administrative agencies unless their decisions are clearly wrong. (It’s a terrible principle, but that’s another topic entirely.) As it happens, the 2-1 decision was Democratic appointees versus a Republican appointee, which emphasizes how political the matter has become.

Anyway, the reclassification allowed the FCC to regulate the internet (or, more precisely, home access to the internet). Another effect was to erase all of the FTC’s internet privacy rules. Instead, FCC crafted its own rules, but the FCC is not an agency with experience with consumer privacy protection, so the result was, as the FTC put it, “not optimal.” Moreover, there were other pernicious effects that had not fully played out yet.

At this point, net neutrality has become a proxy battle between progressives and conservatives. Conservatives oppose net neutrality because they generally oppose regulation, but especially because they don’t want to set the precedent that the government can regulate the internet. Net neutrality may be well-intentioned (if ill-conceived), but they worry that net neutrality will be just the beginning. Hillary Clinton’s statement that net neutrality is “a foot in the door” plays into that concern.

Now the FCC has re-reclassified the internet as an information service again, as it was until 2015. This makes the FCC’s net neutrality regulation null and void. So what happens now? In the near term, nothing at all. The internet survived without net neutrality until 2015, with very few abuses and nearly all of those quickly corrected by the free market. If an ISP ever tried to block an internet content provider that didn’t pay the ISP (a standard scenario in net neutrality’s parade of potential horribles), they would immediately face the wrath of their consumers, and have to reverse themselves. They know it, so they won’t even try.

In the longer term, it’s hard to say. The business model of Netflix and others, who soak up most of the internet’s bandwidth without paying anything for the infrastructure that makes it possible, may turn out to be unsustainable. If so, something will have to change. The same would have been true with net neutrality. But this is a technical problem, and the repeal of net neutrality means that networking researchers can develop a technical solution. Were net neutrality in play, technical solutions would be banned, so the solution would have to be political. That would mean that rent-seeking, not innovation, would be the future of the internet, and we would all be the worse for it.

UPDATE: Added the AT&T-Facetime affair to the preceding discussion.

ANOTHER UPDATE: The discussion above focuses on policies for choosing packets to drop, but there’s another aspect of network neutrality that I left out: concern that ISPs might block certain services entirely. This too happened once: a small phone company blocked voice over IP in an effort to force customers to pay for phone service. This case runs against my general thesis, because it was the (Bush-era) FCC, not the market, that put a stop to it. (Oddly, the FCC was able to do it at a time when it had no regulatory authority over ISPs at all, which says something about the intimidation powers of federal regulators.) Nevertheless, this is a baldly anti-competitive practice and I think the FTC has the power to address it under existing antitrust rules.

(Previous post.)

3 Responses to Net neutrality, explained

  1. Philip Levis says:

    Thanks for the thoughtful post. Your key point is right: net neutrality prevents some forms of network traffic engineering which could be beneficial. But I think you have some basic facts wrong, and are extrapolating from incorrect premises to incorrect conclusions. Let’s start with the technical ones:

    “Historically, internet routers have generally chosen the packets to drop arbitrarily. But for many applications, this is not actually what you want.”

    Actually, it is: please read RFC1958. One of the foundational design principles of the Internet is that fine-grained decisions on prioritization of traffic should occur at the edges, by, the applications, not within the network. The example you give of Internet streaming doesn’t hold up today; these streams use TCP, not UDP, whosec reliable delivery means that the key frame will be delivered. Instead, the governing approach is that the source of the stream should self-regulate how quickly it sends traffic in order to avoid packet drops and the resulting slowdown as well as head-of-line blocking. The algorithms to perform this self-regulating are called “congestion control.”

    In fact, the whole idea that an ISP makes intelligent decisions on which packets to drop, or rate-limit, etc. etc., goes against RFC 1958, which greatly predates the current debate.

    ” Netflix alone accounts for over a third of internet traffic. YouTube (owned Google) accounts for another sixth. (Figures from December 2015.) This creates a tension between ISPs and content providers. The latter’s business strategy requires that they consume a brobdingnagian amount of network bandwidth, while the former are concerned that the content providers will overwhelm their networks.”

    These numbers are correct, but it’s really important where you measure them. Netflix accounts for a third of Internet traffic to end hosts; i.e., a third of the bits flowing into home and office computers over the public Internet is Netflix. However, almost none of this traffic actually traverses the public Internet (core cross-country lines, etc.). Instead, almost all of this traffic is served from a custom content distribution network (CDN) that Netflix has built, called OpenConnect. Basically, Netflix is willing to pay for CDN nodes that a last-mile ISP puts in its switching center; these CDN nodes directly serve up Netflix to the ISP’s customers from within the ISP’s network. Netflix has also placed large groups of such nodes in all of the major Internet exchange points, such that even if the ISP’s local CDN nodes happen to not have the traffic, the traffic doesn’t go very far. So while Netflix is about 1/3 of the bits to end hosts, it’s effectively zero in the wider Internet.

    “The business model of Netflix and others, who soak up most of the internet’s bandwidth without paying anything for the infrastructure that makes it possible, may turn out to be unsustainable.”

    Netflix pays for a huge amount for the infrastructure: they’ve built their own CDN. One interesting anecdote: when Netflix started rolling out its OpenConnect boxes, they observed a very simple phenomenon. In any market where there was last-mile competition, both competing parties wanted OpenConnect boxes because it let them provide much better Netflix service to their customers basically for free (they don’t need to upgrade their transit capacity). In markets where there was a single ISP, the ISP refused, instead arguing that Netflix should have to pay for the bigger transit capacity that the ISP needed. This is textbook rent-seeking behavior.

    And just to check: are you arguing that Google doesn’t pay anything for Internet infrastructure?

    “But this is a technical problem, and the repeal of net neutrality means that networking researchers can develop a technical solution. Were net neutrality in play, technical solutions would be banned, so the solution would have to be political.”

    At first glance, this argument seems sound. But unfortunately, I think it breaks down when you consider that the parties involved have antagonistic goals. One of the strongest practical arguments for why Comcast needed to stop terminating BitTorrent connections in 2008 was that it would create an arms race. Comcast would roll out a blocking technology; the clients would change their algorithms and policies to evade it. For example, BitTorrent could use relays, encrypt or obfuscate more of its traffic, try to masquerade as other applications, etc. The basic problem with this conflict is that the resulting protocols would be less efficient than their straightforward counterparts. I.e., the end result would be that BitTorrent continues to run but is slightly slower and uses more bandwidth than before. So, from Comcast’s perspective, it would spend a lot of resources, lose the fight, and have a network that works worse than before.

    In the presence of a free market, your arguments about choice, consumer uproar, and pushback make sense and could play out as you describe. But a large fraction of the U.S. does not have a choice: they have only one ISP they can get reasonable service from. The behavior of ISPs in these settings has been pretty poor historically, with repeated examples of rent-seeking behavior.

    But, let’s try to flip this around. I hope you’ll agree that there are some potential and real drawbacks to the absence of net neutrality. My reading of your argument is that they exist, but you think the benefits of innovation outweigh them. Can you present a single example of a specific policy or optimization which would be tangibly beneficial to end users today but which net neutrality prevents? E.g., would reduce YouTube buffering times, enable wide-area cooperative VR, etc. Full disclosure: I’ve seen this question asked among a group of libertarian network engineers and nobody could come up with one.

  2. foxcharlie says:

    I might be willing to accept net neutrality if it included google, twitter, and facebook.

  3. K. Crary says:

    Thanks for your thoughtful response Philip, and I’m sorry it took me so long to notice that there were comments awaiting moderation.

    You make an interesting case, but you haven’t convinced me that routers should necessarily choose packets to drop arbitrarily, much less that the government should mandate such a policy. I think that even if “end-to-end functionality should be realized by end-to-end protocols” were an immutable dictum (which it explicitly disclaims being), it leaves a lot of room for interpretation. Quality of service and active networks are two research areas that involve implementing semantics within the network, and I’ve never heard anyone complain that they were against the principles of the internet. (I haven’t heard anything about active networks in years, so it may have been a passing fad, but QoS certainly isn’t.)

    Anyway, I’m not arguing that ISPs make good decisions about which packets to drop. (In fact, I’m not even concerned primarily with ISPs as much as the backbone.) What I am arguing is that they could, and someday they may need to.

    Your concern about rent-seeking is well-taken. I am concerned with that as well. But the solution to rent-seeking is not *more* rent-seeking, and that’s what I think net neutrality would bring about. If there is a government board that decides what traffic management strategies are legal, eventually that board will be controlled by the industry. As always happens with regulatory capture, they will institute policies that create a barrier to entry for new competitors.

    There’s is another, more direct way to deal with rent-seeking: through anti-trust. I think there is a clear anti-trust case against several ISPs, if we ever had an administration inclined to pursue it. When it does pursued some day, the FTC, not the FCC, is the agency tasked with that sort of thing.

    To answer your final question: No, I don’t have a policy that could improve network performance today. (If I did, I would surely publish it.) But I’m only half a libertarian network engineer — the libertarian half. My research is in programming languages. Moreover, I think that’s the wrong question. I’m not concerned about improving the internet *today*, but retaining our ability to improve it in the future.

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: