AWS Mountpoint: A High-Performance File Client for Amazon S3
Today, AWS announces the general availability of Mountpoint for Amazon S3.
β₯ AWS Mountpoint is an open source file client that makes it easy for Linux-based applications to connect directly to Amazon Simple Storage Service (Amazon S3) buckets and access objects using file APIs. It is ideal for large-scale analytics applications that read and generate large amounts of S3 data in parallel, but don't require the ability to write to the middle of existing objects.
β₯ It automatically translates local file system API calls to REST API calls on S3 objects. Mountpoint for Amazon S3 is optimized for high-throughput performance. It builds on the AWS Common Runtime (CRT) library, which is purpose-built for high performance and low-resource usage to make efficient use of your compute resources.
It is a powerful tool that can be used to accelerate a variety of workloads, including:
βοΈ Data lakes.
βοΈ Machine learning training.
βοΈ Image rendering.
βοΈ Autonomous vehicle simulation.
βοΈ Extract, transform, and load (ETL) processes.
β½ Here are some of the key benefits of AWS Mountpoint:
β High throughput:
AWS Mountpoint can achieve up to 100 GB/s of throughput on a single EC2 instance, and can scale to support even higher throughput requirements.
β Low latency:
AWS Mountpoint has very low latency, making it ideal for latency-sensitive applications.
β Elasticity:
AWS Mountpoint is fully elastic, so you can scale up or down your throughput requirements as needed.
β Cost-effectiveness:
AWS Mountpoint is a cost-effective way to access your data in Amazon S3. You only pay for the storage and bandwidth that you use.
Watch Devabrat Kumar - Senior Technical product leader at AWS talks about it :
Getting started:
β₯ Run these two commands to install Mountpoint for Amazon S3 on your Amazon Linux EC2 instance (for Graviton instances, replace x86_64 with arm64 in the URL):
wget https://s3.amazonaws.com/mountpoint-s3-release/latest/x86_64/mount-s3.rpm
sudo yum install -y ./mount-s3.rpm
On Ubuntu, use these commands instead (for Graviton instances, replace x86_64 with arm64 in the URL):
wget https://s3.amazonaws.com/mountpoint-s3-release/latest/x86_64/mount-s3.deb
sudo apt-get install -y ./mount-s3.deb
See Installing Mountpoint for Amazon S3 for detailed instructions and other installation options (including Docker or building from source).
β₯ Once you've got Mountpoint for Amazon S3 installed, you can mount your Amazon S3 bucket. You'll need valid AWS credentials to access your bucket; for example, Mountpoint will automatically use credentials from an IAM role associated with your EC2 instance, or you can use the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables.
β₯ To mount your bucket, run this command, replacing DOC-EXAMPLE-BUCKET with the name of your bucket and /path/to/mount with the directory you want to mount the bucket to:
mount-s3 DOC-EXAMPLE-BUCKET /path/to/mount
Now you can work with your bucket contents as if they were a local file system:
ls /path/to/mount
echo "Hello World!" > /path/to/mount/Data.txt
cat /path/to/mount/Data.txt
When you're finished accessing your bucket, you can unmount it (you might need sudo):
umount /path/to/mount
See Configuring Mountpoint for Amazon S3 for more details on how to configure and use Mountpoint, including options for providing AWS credentials.
To learn more about AWS Mountpoint, please visit the following resources:
π AWS Mountpoint documentation
π AWS Mountpoint GitHub repository
π Mountpoint for Amazon S3
by Jeff Barr - Vice President & Chief Evangelist of Amazon Web Services
π Mountpoint service page
I hope this article was helpful! If you have any questions, please feel free to leave a comment below.
Follow me on:
Linkedin.
Top comments (1)
Hey π Ahmed,
It was really a good article ..
I have a different query link, please visit once , your help is really appreciated
Thanks π