It was a tough week for the cloud, we’ll run down the list and summarize what happened to the services we all depend on so much!
Plus a big batch of your questions, our answers, and a rocking round-up!
All that and a lot more, on this week’s TechSNAP.
SPECIAL OFFER! Save 20% off your order!
Support the Show:
- GoDaddy’s services started to drop off of the internet
- The outage lasted approximately 6 hours, from 10:00 PDT (17:00 UTC) and being fully restored about 16:00 PDT (23:00 UTC)
- A twitter account, claiming to represent part of Anonymous, took responsibility, claiming to have launched a massive DDoS attack against GoDaddy
- Some news outlets and blogs misunderstand what a DDoS attack is, and report that Anonymous has hacked GoDaddy
- “We have determined the service outage was due to a series of internal network events that corrupted router data tables.” – Interim Godaddy CEO Scott Wagner
- The issue was compounded because the downtime affected not only GoDaddy hosting customers, but also customers that only used GoDaddy for DNS
- GoDaddy hosts 5 million web sites and manages a total of 52 million domain names
- For example, the DNS for jupiterbroadcasting.com is hosted at GoDaddy, while the actual site resides at ScaleEngine, but because the DNS was down, viewers were unable to lookup the IP address of jupiterbroadcasting.com in order to connect to ScaleEngine
- DNS caching will have helped reduce the effect of this downtime somewhat, especially for more popular sites, and for users coming from larger ISPs, the DNS records for JB have a TTL of 1 day, so users would only have issues reaching the site if the records had not yet been cached, or once the cache expired. At the time of this writing, the records for JB still had 28461 seconds left in my local Google Public DNS cache, but we not cached at my local OpenDNS
- This event ruined GoDaddy’s previous 99.999% uptime record for DNS (99.999%, or 5 nines as it is called in the industry, allows for only 6 minutes of cumulative downtime in an entire year, compared to 4 nines, which allows about 53 minutes of downtime per year, or 99.9% which is nearly 9 hours)
- GoDaddy uses Anycast for the DNS servers, this means that while it looks like each domain is only assigned to 2 DNS servers, each of those two IP addresses actually exists in multiple data centers around the world. Traffic is routed to the closest server, and if that servers route fails, after a few minutes the BGP routers at your ISP or an intervening transit provider route the traffic to the next closest server
- However, due to what I assume was some human error after the failure of one or more network components, the routes that GoDaddy broadcasted to their upstream providers were in some way incorrect, and caused traffic to no longer reach the GoDaddy servers
- Anycast is commonly used for DNS but is not very often used for TCP based services due to the fact that the routes can change at any time, and suddenly the same IP address points to a different server, and your connection is dropped. There are some cases where people have successfully used Anycast for short lived TCP connections
- Additional Coverage
- Go Daddy Site Outage Investigation Completed – GoDaddy.com
- Security researcher David Schuetz was analyzing the the data posted online, and found an unusually large number of devices that mentioned Blue Toad, 19 out of the 1 million records analyzed
- Schuetz then contacted Blue Toad to report what he had found
- Schuetz also said he couldn’t say conclusively if Anonymous’ claims about the FBI were false or true
- Blue Toad makes apps for publishing companies, long known for collecting extensive data about their readers for market research and marketing purposes
- Paul DeHart, CEO of Blue Toad said his firm would not be contacting individual consumers to notify them that their information had been compromised, instead leaving it up to individual publishers to contact readers as they see fit
- The company’s forensic analysis claims to show the data had been stolen “in the past two weeks”
- This is contrary to the original claim that the data was stolen from an FBI computer months ago
- Techsnap drinking game
- Windows Server 2012 review
- Shoeboxed.com security sufficient for personal documents, eg. tax returns?
- 10 Char Password Enough?
- Bad to use scripts to set stuff up?
- Siemens Admits Issues
- Are you a PHP programmer with time to spare for an open source security project? I am considering building an open source web service to handle sensitive information disclosure, and I am looking for some help building the site and API. email email@example.com
- BEAST creators develop new SSL attack
- Thoughts on the way Amazon Glacier Pricing works
- Malicious Apache Module Injects Iframes
- Github Goes Down
- Over 11,000 Guild Wars 2 passwords hacked
- Romney 1040 Tax Returns – Part 2 – Pastebin.com
- Shane’s Software on the side: Quick Update to the Jupiter Broadcasting Android App
- BMW cars build since 2006 vulnerable to attack with blank programmable keys
- Microsoft Digital Crimes Unit gets permissions to disrupt botnet of computers sold from China with pirated copies of windows