What started as a small issue affecting some instances of Amazon’s Elastic Cloud Compute (EC2) in North Virginia became a full-blown outage of AWS in North Virginia. Major services, such as Reddit, Foursquare, Minecraft and Heroku, are down. GitHub, imgur, Pocket, HipChat, Coursera and others are affected.
Reddit experienced hiccups at first when only the Elastic Bloc System was affected, but then seemed to go back to normal for most people. It is now down again.
Many other services are now down as the issue is now affecting other parts of Amazon Web Services. Amazon’s RDS database instances and Elastic Beanstalk are down in North Virginia. Minecraft, Pinterest, Foursquare and Airbnb are down. Finally, AWS’s ElastiCache and CloudWatch are experiencing delays and connectivity issues.
Heroku currently reports elevated error rates for both the production service and the development service. Many smaller services are affected by the outage.
As always, users and developers are wondering why half of the Internet is going down every time an AWS data center experiences issues.
On Amazon’s AWS status page, the company first reported issues only affecting EBS volumes:
10:38 AM PDT We are currently investigating degraded performance for a small number of EBS volumes in a single Availability Zone in the US-EAST-1 Region.
11:11 AM PDT We can confirm degraded performance for a small number of EBS volumes in a single Availability Zone in the US-EAST-1 Region. Instances using affected EBS volumes will also experience degraded performance.
11:26 AM PDT We are currently experiencing degraded performance for EBS volumes in a single Availability Zone in the US-EAST-1 Region. New launches for EBS backed instances are failing and instances using affected EBS volumes will experience degraded performance.
Then the Relational Database Service went down as well:
11:03 AM PDT We are currently experiencing connectivity issues and degraded performance for a small number of RDS DB Instances in a single Availability Zone in the US-EAST-1 Region.
11:45 AM PDT A number of Amazon RDS DB Instances in a single Availability Zone in the US-EAST-1 Region are experiencing connectivity issues or degraded performance. New instance create requests in the affected Availability Zone are experiencing elevated latencies. We are investigating the root cause.
The Elastic Beanstalk experience similar issues:
11:06 AM PDT We are currently experiencing elevated API failures and delays launching, updating and deleting Elastic Beanstalk environments in the US-East-1 Region.
11:45 AM PDT We are continuing to see delays launching, updating and deleting Elastic Beanstalk environments in the US-East-1 Region.
Update: Operations are now slowly recovering and everything should be back to normal.