DEV Community

Chastina Li 👩🏻‍💻
Chastina Li 👩🏻‍💻

Posted on

Auto-refresh AWS Tokens Using IAM Role and boto3

The Curse of The Hour

Session management in AWS is complicated, especially when authenticating with IAM roles. A common way to obtain AWS credentials is to assume an IAM role and be given a set of temporary session keys that are only good for a certain period of time. The maximum session duration is a setting on the IAM role itself, and it is one hour by default. So if users don't specify a value for the DurationSeconds parameter, their security credentials are valid for only one hour.

A typical boto3 request to assume IAM role looks like:

response = client.assume_role(
    RoleArn='string',
    RoleSessionName='string',
    Policy='string',
    DurationSeconds=123,
    ExternalId='string',
    SerialNumber='string',
    TokenCode='string'
)
Enter fullscreen mode Exit fullscreen mode

where DurationSeconds is the duration of the role session. It can go up to the maximum session duration setting for the role. So if your IAM role is only setup to go up to an hour, you wouldn't be able to extend the duration of your sessions unless you update the settings on the IAM role itself.

Long-Lasting Credentials

So an alternative must be introduced to extend IAM role sessions. This is when I found RefreshableCredentials , a botocore class acting like a container for credentials needed to authenticate requests. Moreover, it can automatically refresh the credentials! This is exactly what I need. But it's usage is poorly documented. Looking at its source code:

def __init__(self, access_key, secret_key, token, expiry_time, refresh_using, method, time_fetcher=_local_now):
Enter fullscreen mode Exit fullscreen mode

The __init__ function takes several arguments, half of which I don't recognize. But there's a class method that can be used to initialize an object:

@classmethod
def create_from_metadata(cls, metadata, refresh_using, method):
    instance = cls(
         access_key=metadata['access_key'],
         secret_key=metadata['secret_key'],
         token=metadata['token'],
         expiry_time=cls._expiry_datetime(metadata['expiry_time']),
         method=method,
         refresh_using=refresh_using)
    return instance
Enter fullscreen mode Exit fullscreen mode

where metadata is a dictionary containing information abound the current session, ie. access_key, secret_key, token, and expiry_time, all are things we can get from boto3's STS client's assume_role() request.

To construct the metadata response, we make a simple boto3 API call:

import boto3
sts_client = boto3.client("sts", region_name=aws_region)
params = {
    "RoleArn": self.role_name,
    "RoleSessionName": self.session_name,
    "DurationSeconds": 3600,
}
response = sts_client.assume_role(**params).get("Credentials")
metadata = {
    "access_key": response.get("AccessKeyId"),
    "secret_key": response.get("SecretAccessKey"),
    "token": response.get("SessionToken"),
    "expiry_time": response.get("Expiration").isoformat(),
}
Enter fullscreen mode Exit fullscreen mode

refresh_using is a callable that returns a set of new credentials, taking the format of metadata. Remember that in Python, functions are first-class citizens. You can assign them to variables, store them in data structures, pass them as arguments to other functions, and even return them as values from other functions. So I just need a function that generates and returns metadata.

def _refresh(self):
    " Refresh tokens by calling assume_role again "
    params = {
        "RoleArn": self.role_name,
        "RoleSessionName": self.session_name,
        "DurationSeconds": 3600,
    }

    response = self.sts_client.assume_role(**params).get("Credentials")
    credentials = {
        "access_key": response.get("AccessKeyId"),
        "secret_key": response.get("SecretAccessKey"),
        "token": response.get("SessionToken"),
        "expiry_time": response.get("Expiration").isoformat(),
    }
    return credentials
Enter fullscreen mode Exit fullscreen mode

Now we're ready to create a RefreshableCredentials object:

from botocore.credentials import RefreshableCredentials
session_credentials = RefreshableCredentials.create_from_metadata(
    metadata=self._refresh(),
    refresh_using=self._refresh,
    method="sts-assume-role",
)
Enter fullscreen mode Exit fullscreen mode

and we can use the credentials to generate a IAM role session that lasts for as long as we need:

from boto3 import Session
from botocore.session import get_session
session = get_session()
session._credentials = session_credentials
session.set_config_variable("region", aws_region)
autorefresh_session = Session(botocore_session=session)
Enter fullscreen mode Exit fullscreen mode

And of course we can generate a boto client within that session, ie.:

db_client = autorefresh_session.client("rds", region_name='us-east-1')
Enter fullscreen mode Exit fullscreen mode

Top comments (13)

Collapse
 
noelleleigh profile image
Noelle Leigh

For future visitors: RefreshableCredentials isn't documented, but can be found in the source code.

Collapse
 
knoha profile image
Kostiantyn Noha • Edited

@li_chastina Thanks fir the snippets!
But one question: in order to refresh credentials do we need to call autorefresh_session.client each time we use a client functions or its enough to always use db_client reference and credentials refreshed automatically?

Collapse
 
jinyeng profile image
jinyeng

When I use this sample and print session info, looks like its generating a new credentials on every resource request !

Collapse
 
mandheer profile image
Mandheer

This post made my day.

Collapse
 
jinyeng profile image
jinyeng

Thanks For this great article. I am trying to use this code for S3 bucket listing and upload. I have two issues. I see that when I try to list bucket, it seems to again refresh credentials every time.

Also when trying to list bucket contents, it just gets stuck foerver without a response.

Collapse
 
icharle7 profile image
cmartín

This really worked for me. You helped me a lot. Thank you Chastina!
I will post again if i find something interesting about this botocore class.

Collapse
 
reka193 profile image
reka193

Perfect, thank you!!

Collapse
 
aaalllex profile image
aaalllex

Very useful
Thanks for sharing

Collapse
 
mbadawi23 profile image
melkammar

Would you be able to share the full class code here?

Collapse
 
avinashdalvi_ profile image
Avinash Dalvi

Good one

Collapse
 
sarkisvarozian profile image
Sarkis Varozian

Nice find and great post!