Amazon Outage Breaks Large Parts of the Internet

Megalith

24-bit/48kHz
Staff member
Joined
Aug 20, 2006
Messages
13,000
Something hit the fan with Amazon’s S3 web-based storage service earlier, and as a result, a ton of websites, apps, and devices were/are broken and inaccessible. I read that this was “sweeping the East Coast,” but I am in the Tri-state area and haven’t run into anything so far. How about you?

If your favorite website or internet service isn't accessible today, it's probably not just you. Amazon Web Services' S3 cloud storage is experiencing "high error rates" that have caused chaos among many of the sites that depend on AWS to work. [Engadget] is one of them, but the failure has also affected some or all of Giphy, Medium, Slack, Quora and a slew of other websites and apps you likely use. Some connected home devices aren't working correctly, either. Even Amazon's ability to report problems was broken for a while -- the AWS dashboard wasn't changing color because its issue was "related" to S3's problems.
 
I noticed a few minutes ago when I turned on my computer that the Razer Synapse app unusually asked me to sign on and then wouldn't take my password. A couple financial sites aren't updating in Quicken either...more than the normal one or two. I'm in Florida.
 
A lot more then S3 was down, practically all of US-East. Only like 1 out of 15 EC2 Instances(virtual servers) were up. Right now somethings are up, but S3 puts are down. It is causing my SQS (queue) to fill up. When it comes back it will eat all my burst credits.
 
Part of Amazon's website services are down, can't access my orders or print return ship labels, etc.
 
A bunch of our stuff is down. Of course this happens just as I'm done handing in paperwork to move some of our services into our AWS based virtual machine room.

Gotta love the cloud. Management tells us it is more reliable. So far, not so much. But hey they sign the checks.
 
It was taking me 4 minutes to load the H main page, and I had to reload the forums multiple times in order to post anything earlier today.

Every time my browser would be "waiting for s3.amazonaws.com"

It seems to be working better now though.
 
A bunch of our stuff is down. Of course this happens just as I'm done handing in paperwork to move some of our services into our AWS based virtual machine room.

Gotta love the cloud. Management tells us it is more reliable. So far, not so much. But hey they sign the checks.

The president of our company is doing a demo at a conference today.
 
  • Like
Reactions: klank
like this
I shop at amazon a lot. If I want to find quality products with good prices, anyone recommend other online stores other than Amazon? not ali express but other popular online stores?
 
My GarminConnect App will not sync.
a few other cloud based storage systems are not responding.

too soon for an "Alexa Break the internet" joke?
 
And the downsides of "moving your business to the cloud" start becoming evident...
I'm no fan of AWS, but I don't think I could have the same yearly uptime percentage if everything was on site. Our ISP uptime alone would prevent that from happening.
 
And this is exactly the illustration of the weakness that server/client/cloudbase apps exhibit. Client side apps are still the way to go. I can work offline and not worry about a thing.
 
I'm in NYC...a lot of websites I visit were slow to load and had errors earlier this morning
 
All the errors seemed to be in the Northern Virginia region for AWS and yet it affected so much more than just that particular region, it's kinda messed up how that can happen but as usual it's still a "weakest link" kind of situation.
 
Yep.

H was super slow
Amazon wasn't displaying pictures
Bunch of websites came back as 404

Couple other weird issues.
 
Management: "We need to move everything to the cloud and get rid of our servers and IT staff, that way it's never going to go down and we can save lot of money!"

Checkmate, people who say stuff like that.

:p
 
thankfully pornhub was safe. But since other sites were down, i can hardly hold a glass.

And the downsides of "moving your business to the cloud" start becoming evident...

but the small downtime falls within the normal window of downtime typically associated with patches, reboots, & hardware related maintenance. Cost savings of not having staff stay after hours, power, and cooling allows SOME orgs to act like it never happened.
 
Fuck "The Cloud". People need to be grownups and get some redundant and high availability infrastructure at home and at a third party location. Shit! I have seen job offers for doing nothing but being prepared for the possibility of this kind of disaster management IN HOUSE.
 
Got the notice it took down one of our client's sites today just after 1PM EST, one he counts on for about 300k in monthly sales. Wasn't fully back up until about an hour ago. Of course, I wasn't made aware of the outage until about halfway through that, which just made things worse. Add in another hour or so after that to figure out his port 443 settings were blown away by whatever happened, and it wasn't that much fun (though the hours were billable).

Disclaimer: wasn't my idea to go with AWS. Been there in the past, done that.
 
Not sure if related but full episodes of the walking dead are unviewable on AMC. Maybe just a kwinky dink.
 
I envision businesses are going to take a step backwards in the years to come and go back to realizing self-hosting is way better if you have the funds to do it.
 
I envision businesses are going to take a step backwards in the years to come and go back to realizing self-hosting is way better if you have the funds to do it.

Probably not IMO. Some info-week reading idiots aside, a lot of existing businesses have to make a financial case to move services to the cloud. Initial roll out of a service may be different, but still likely has to meet a price point. The reality is that the cloud is more expensive unless you do one or more of the following.

1) Avoid building machine rooms.
2) Avoid buying durable hardware for a transient project.
3) Eliminate staff.
4) Avoid adding staff.

Lots of start ups love the cloud because they get 1,2, and 4 out of it. This is a massive reduction in need for capital to get going.
 
EPIC's game servers and logins were all down for a few hours yesterday essentially breaking the ability to play their new games online.
 
Back
Top