Multi-factor authentication is a best practice for root and administrator users. Having multiple MFA options can help you out in unexpected situations.
1. The scenario
I don't change my phones or computers frequently as I want to get the maximum value for my money (probably because I experienced what it's like not having things as a child). If these tools serve their purpose, I keep them as long as possible. I take good care of my things, but the change is sometimes inevitable.
I had my old phone for years. I loved it, but it was running on fumes in its last few months. So, I decided to retire it and buy a new phone.
2. The problem
Migrating apps and contacts to the new phone was easy, and the process went seamlessly. But somewhere in the back of my mind, a little devil kept telling me that I would be screwed if Authy, the 2FA application I had used, had an issue. I would be locked out of all accounts on all services where I have multi-factor authentication (MFA) enabled. I read the migration instructions carefully beforehand, and I immediately thought that the process was, well, fragile, to say the least.
The thought became a self-fulfilling prophecy, and I got locked out of Authy during the migration. The app asked for a backup code on my new phone, which I had set, but it wasn't working for some reason. It must have still considered the existing backup password on my old phone valid. Alas, I didn't remember it. If a password isn't in my password manager, then it doesn't exist at all. 🙂
The customer service representative was very understanding and apologetic, but understanding and apologies, while appreciated, don't solve problems. He couldn't unlock my app even though he verified my email address and phone number.
So, I had nothing left but to go to each service one by one and turn off MFA.
By the way, migration with Google Authenticator was super simple. I had to create a QR code on the old phone, and then scan the code on the new one. The accounts were transferred to my new device in under a minute. But the majority of my MFA-enabled accounts were still stored in Authy, including my AWS single sign-on page and GitHub account.
3. The solution
Since this is a blog on AWS, I'll write about resolving the issue with my AWS accounts.
I have workloads, applications, and instructor demos in them, so needless to say, I was eager to regain access to my AWS accounts.
I have multiple AWS accounts. I use an IAM Identity Center user to centrally sign in and select the account I want to work in. The solution would have been to sign into the management account, remove the MFA device, and add the new one.
To do that, I would have needed to sign in first, but the account was protected by MFA, which I had lost access to.
3.1. Recovering the root user
So I thought I would reset MFA using the root user. We should never use the root user for everyday activities. But this situation was an emergency, and as such, it was time to dust off the root user credentials.
However, since I do my best to follow the best practices, my root user was also protected by MFA! The root MFA account was also stored in Authy, which made this route a dead end, too.
I had to start the account recovery process, which includes email (Step 1) and phone number (Step 2) verifications.
First, you'll receive an email with a link. You have to click the link to prove that you own the email address. Then, in Step 2, you will receive a phone call, and you will need to enter some code provided on the screen using the phone's keypad while on the line.
I got stuck at Step 2 first because the call just didn't come. So I submitted a support ticket to AWS. They responded a few hours later with some recommended solutions. Unfortunately, these solutions all involved using my phone number in some way. In that blended state of mind which included panic and despair, I believed that I had the wrong phone number on record. And, you guessed it, I can only change it if I sign into my account, but I couldn't log in because of the lost MFA account.
Before I responded to the AWS customer service representative explaining my situation, I thought I would give the recovery process another go, and this time I had success! I received the call, which confirmed that my account details were up-to-date, and I could finally access my management account as a root user.
3.2. Changing MFA devices with the root user
Once in the account, I went to Security credentials by selecting my user in the top right corner of the console page to enable new multi-factor authentication methods for the root user.
We have three options: Passkey (security key), Authenticator app, and Hardware TOTP token. This time, learning from my mistakes, I set up multiple MFA options. So, when I sign in with my root user credentials, the multi-factor authentication page will offer me every enabled option, and I can choose which one I want to use.
The last thing I did as a root user was go to the Identity Center and remove the existing MFA device from my SSO user.
3.3. Changing MFA devices with the Identity Center user
Finally, I could sign into my management account with my SSO user's username and password to add the second factor back to protect my accounts.
Identity Center allows us to register two MFA device types, authenticator app and security key.
We can create multiple options for each type. For example, I can use two different authenticator applications or security keys. This way, if I lose my phone (or the 2FA app locks me out again), I still have other options to log into my accounts.
3.4. IAM users
AWS also recommends that IAM users with elevated permissions should have MFA enabled.
If IAM users lose access to the second factor, other administrators (or the root user if nothing else) can remove their MFA device. Users can then sign in with their username and password and set up their second factor again.
It's a good idea to have multiple but limited number of administrators to reduce the risk of account unavailability.
4. Learnings
I never thought that I could get locked out of my own accounts. Well, newsflash, it can happen. In my case, it was one innocent password I forgot to take note of 6 years ago.
So here are my learnings:
- First, stay humble and consistent. Save every code or password in your password manager, no matter if it belongs to a web or mobile app.
- Enable multiple different MFA devices. AWS offers software authenticator apps, passkeys, and hardware tokens. Store the root user authentication code in a separate device from the main admin user.
- If you want to use software authenticator apps, install at least two on your phone. Use one for the root user and one for the administrator user.
- If your contact details change, particularly your email address or phone number, update your AWS account immediately with the new records. You will need them to restore root user access as a last resort.
As the old investment advice says, don't put all your eggs in one basket. Have multiple 2FA options available if the given service (AWS or others) supports it. Avoid creating a single point of failure, like a single authentication app storing all MFA codes and be highly available not only in your workloads but also with your second factor.
One word of caution here. Having multiple MFA devices comes with greater responsibility, meaning you must protect more than one tool. Just keep that in mind.
5. End of the story
Eventually, I successfully regained access to all but one service by turning off and re-enabling multi-factor authentication. In a few cases, I needed to contact customer service who turned off MFA for me after they verified my identity.
And what is the one service that I still don't have access to? A service that keeps sending me an email every day to remind me to restore my account by clicking the link. A tiny problem is that the emails contain invalid links. 🙂 The account restoration deadline expires soon. It is what it is. I probably won't fall on my sword if I can't see who eats what for breakfast or if I don't receive the daily wisdom quotes again. 🙂
6. Summary
It's best practice to set up multi-factor authentication for the AWS root and admin users. AWS supports multiple different MFA options, and we can enable more than one in our accounts. To avoid being exposed to one MFA device and making it a single point of failure, we might want to register multiple MFA devices.
7. References, further reading
Enable a virtual MFA device for the root user (console) - How to add an MFA device to the root user
YubiKey - Hardware security key for MFA. Great option if you don't want to rely on your phone.
Top comments (0)