Google's Service Outage IPv6 Related

Google's services outage today was apparently Google mucking with something on the backend, in a sort of "don't touch if it ain't broke" mistake. Admittedly, we're being snarky, as Google was working on something that definitely needs addressing, but it still made a major impact on systems around the world.

It's the old "we are running out of IP addresses problem." IP addresses identify clients on the Internet, and IPv4, the current version has a limit of 4,294,967,296 possible unique addresses. The solution to the problem is IPv6, which has a limit of about about 3.4×10³⁸ possible unique addresses.


It's a move that has to be made eventually, but until now the necessity has been ameliorated by the use of network address translation (NAT) via routers along with internal IP addresses. For example, if you have a cable modem and home and use a router to network other PCs and share broadband, only the cable modem will have a public IP address while the router and PCs will have those dedicated to LANs.

Now, Google itself didn't go into great detail, but said:

An error in one of our systems caused us to direct some of our web traffic through Asia, which created a traffic jam. As a result, about 14% of our users experienced slow services or even interruptions.

That's the short version. The long version, according to C|Net, is the following:

Dmitri Alperovitch, vice president of threat research for McAfee, said that Google this morning attempted to make changes to key Internet routing numbers--known as autonomous system numbers--as part of its ongoing transition from an older networking standard to a newer one called IPv6. An unknown "bug" inside Google's network involving some sort of hardware failure or glitch prevented Internet service providers from finding Google's new ASNs on the Internet--effectively sealing it off from many customers, he said.

Not all Internet users were affected, but some that use larger providers--such as AT&T or Verizon--appeared to be disproportionately hurt because large ISPs "peer" with Google, or interconnect their networks with Google's networks in order to improve speed and reduce bandwith costs, Alperovitch said.

That explains why AT&T was hit, as indicated earlier this morning. Of course, Google is a major promoter of IPv6, but IPv6 has not reached "critical mass."

Google's claims that only 14% of users were affected means, based on all the Twitter traffic around it, that it was a very vocal 14%.