UPDATE 11/18/19: Added some additional notes to include mention of AWS Config, Zelkova, and some info about tools by Rhino Security Labs like Pacu (thanks to input by @andrewbrown!)
S3NakedInPublic: The Common Misconfiguration that Lands Companies in The Wrong Spotlight
Again and Again
Companies continue to make the news. Two, in particular, led to this post. One was a dating app, where private photos were publicly available for a year. 1
They’re being fined $240,000.
The second company? An IT firm:
“The leaky AWS S3 buckets contained information on Attunity's own operations, but also data from some of its customers – Fortune 100 companies like Ford, Netflix, and TD Bank.” 2
They Are Not Alone
These others may sound familiar, and they struggled with the same mistake (bold emphasis by the author of this article):
“The UpGuard Cyber Risk team can now report that 73 gigabytes of downloadable data belonging to Washington-based internet service provider Pocket iNet was publicly exposed in a misconfigured Amazon S3 storage bucket.” 3
“Once again, an Amazon S3 bucket with no specific authorization required, merely knowledge of the file path which is obviously stored in the app itself (returned via the API)...The services sitting on top of the exposed database are able to point to the precise location of the profile pictures and voice recordings of children.” 4
“The virtual disk image contains over 100 gigabytes of data from an Army intelligence project, codenamed 'Red Disk.'...The disk image was left on an unlisted but public Amazon Web Services storage server, without a password, open for anyone to download. Unprotected storage buckets have become a recurring theme in recent data leaks and exposures.” 5
What Is Going On?
Reports have been steady for nearly the past decade. Some articles refer to it as a “leaky bucket.” Others say a company was "leaving a server unsecured,” or configuring online storage "without a password."
The service consistently cited? Amazon Simple Storage Service (S3). These articles continue referring to the very same misconfiguration of the S3 storage. It’s an issue that will continue in pushing companies into an unwelcome spotlight.
Think of these buckets as file directories with a blatant label of PUBLIC allowing anyone to read, download, and at times even modify or delete data. This can be confusing when initially learning how S3 bucket policies and AWS IAM functionality works. For people with little experience, the ability to make a dreadful mistake becomes highly probable.
By default, S3 buckets are private. A person has to explicitly make it public for it to become that way. It can be the easy button for when someone is struggling with getting permissions to work. ACLs? IAM users, groups, and roles? Setting something to public will make everything flow smooth, and people may not realize that “public” means to the internet and not restricted to all the services within the AWS account.
Manually managing and tweaking access to buckets can result in punching holes to the outside world. Being aware of what’s publicly accessible wasn’t always very clear at a glance, with the console changing more each year.
Maybe it's tempting to invoke Nike and smack the keyboard while shouting, "JUST DO IT!"
There are legitimate cases where the user wants S3 public for read access, such as with static websites hosted in S3. Another example is an access point for downloads, such as PDF reports linked to for people to download. But outside of those examples? It is hard to find justification for public access.
Regardless of these standards, granting public access to an S3 bucket is done all the time. In 2013, a Rapid7 report mentioned finding almost 2,000 public S3 buckets. Imagine what that number is now, six years later! Some studies have shown that 7% to 10% of all S3 buckets (!!) are public, with ~20% of those public buckets even having write access. 6
AWS has been working to change this.
What Can Be Done?
AWS has added bigger message banners in the console to try and grab an administrators attention. Last year, their news blog wrote about the new feature of Amazon S3 Block Public Access – Another Layer of Protection for Your Accounts and Buckets, mentioning the need:
“We want to make sure that you use public buckets and objects as needed, while giving you tools to make sure that you don’t make them publicly accessible due to a simple mistake or misunderstanding.” - Jeff Barr, AWS
S3NakedInPublic is a little landing page I created, one that has examples on visual steps that can be done manually within the AWS Management Console to check for (and secure) Public buckets. S3NakedInPublic also has simple code examples in PowerShell, Python, and bash (wrapped around the AWS CLI). They can be starting points, only taking 5 – 10 lines of code to get started on auditing.
PowerShell Example
# Requires AWSPowerShell.NetCore Module
# OR AWSPowerShell Module (if on Windows PowerShell)
# Install-Module AWSPowerShell.NetCore
# https://www.powershellgallery.com/packages/AWSPowerShell.NetCore
# Funky formatting to fit dev.to code display
foreach ($OhNoes in Get-S3Bucket) {
if (($OhNoes | Get-S3ACL).Grants.Grantee |
where {
$_.URI -eq 'http://acs.amazonaws.com/groups/global/AllUsers'
}
) {$OhNoes}
}
Checkout the repository for other examples. Also of note, the open-source Python script S3 Inspector may be able to help even further.
Outside of Simple Auditing, What Else?
Avoiding manual changes to infrastructure, by using tools like CloudFormation or Terraform, can ensure proper control and change history of an environment. There is still the problem of change being manually done after the fact and outside of configuration management frameworks, though CloudFormation had drift detection added last year: AWS Blog - New CloudFormation Drift Detection
It would be a good idea to take a look at tools such as:
- Security Monkey
- Cloud Custodian
- AWS Config - s3* rules, such as s3-bucket-public-read-prohibited* and s3-bucket-public-write-prohibited* in particular
- AWS Security Hub - for viewing alerts from services such as Amazon GuardDuty, Amazon Inspector, and Amazon Macie.
- AWS Blogpost: Zelkova - An under-the-hood look at how AWS uses automated reasoning tooling to assist in policy analyses, certain AWS Config rules, and that results in the labels of PUBLIC appearing in the AWS Console when viewing the S3 service (among many other things). exist to assist with monitoring and alerting on issues like this (not to mention paid third-party products like Redlock and UpGuard. Reports consistently come out from UpGuard, themselves, and are often referenced on the public S3 buckets that they keep discovering!).
Adding these to a security toolbelt, including implementation of AWS guidance like IAM Best Practices and the Well-Architected Framework can set an account up to better avoid these mishaps.
A Little Extra
There is also the open-source exploitation framework, Pacu, created by Rhino Security Labs. It's continuously growing with a roadmap that looks to include additional cloud providers. Rhino has also come out with CloudGoat, their "Vulnerable by Design" deployment tool for testing security tools against and really understanding the type of flaws you want to avoid in your AWS infrastructure.
Thoughts?
Have any suggestions for quick audits and wins when it comes best security practices on AWS? Or know of other, high-profile, common issues that keep coming to light? Post in the comments below!
Originally published at https://icanteven.io
-
Gay dating app fined $240,000 for leaking nude and private photos. Reported by Catalin Cimpanu @ ZDNet, 2019 ↩
-
Contractor's AWS S3 server leaks data from Fortune 100 companies. Reported by Catalin Cimpanu @ ZDNet, 2019 ↩
-
Out of Pocket: How an ISP Exposed Administrative System Credentials. UpGuard, 2018 ↩
-
Children’s Voice Messages Leaked in CloudPets Database Breach. Reported by Mike Mimoso @ ThreatPost, 2017 ↩
-
NSA leak exposes Red Disk, the Army's failed intelligence system. Reported by Zack Whittaker @ ZDNet, 2017 ↩
-
New Study Shows 20% of Public AWS S3 Buckets are Writable. Reported by Ben Layer @ Tripwire, 2018 ↩
Top comments (4)
Security Hub is just an aggerate of compliance information from multiple AWS Security or Logging Services so it's not going to help in this case.
What's going to help here is turning on AWS Config and using the off-the-shelf AWS Config rule provided by AWS which will tell you if you have public read and write
AWS Internal Service Zelkova is worth giving a read since it can reason whether your have a public bucket.
Turning on Amazon Macie is a great idea that uses ML to monitor S3.
Pacu is something you'll also want to run against your account to see what you can discover in terms of vulnerabilities around S3.
Wow, awesome recommendations. I have looked at Pacu, heard of Macie, but never knew about Zelkova! Definitely going to take a look.
I'll make the correction about Security Hub. I seem to have misunderstood it to be something more like a suite that included AWS Config, when it's really a viewer of aggregated findings from other services.
EDIT / UPDATE: I didn't know that Zelkova worked as part of the underlying tech for the PUBLIC / not public display labels that eventually appeared within the AWS console in viewing S3, and relevant AWS Config rules. Thanks for the links!
Just to clarify when you turn on Security Hub is creates a handful of AWS Config rules for you based on the CIS baseline recommendation. So it does automate the creation of some AWS Config rules for you though just distinguishing that those compliance checks are from AWS Config and not Security Hub.
Ah! Okay, that makes more sense