Generate magic tokens in Rails with generates_token_for

For a long time, and probably still today, the reference for authentication in Rails is using a gem like Devise.

Thing is, you'll probably end up customizing it a lot: views, emails, onboarding flow, etc.
Since Rails 7.1, we have access to several new features that make it easier to implement authentication with minimal extra code, making it a viable option for many projects.

One of these features is generates_token_for, which allows you to generate non-persisted tokens for your models, allowing you to implement features such as passwordless auth, password reset, email confirmation, and more.

When I stumbled upon this feature, my first thought was: That's magic.

In this post, We'll see how to use generates_token_for to generate magic tokens in Rails, then we'll dive into the code to understand how it works.

How to use generates_token_for

Here's a basic example of how to use generates_token_for in your Rails models:

class User < ApplicationRecord
  generates_token_for :account_activation
end

user = User.find(42)
token = user.generate_token_for(:account_activation) # => "sometoken===--somesignature"

User.find_by_token_for(:account_activation, token) # => #<User id: 42, ...>

Once we declare that we want to generate a token for :account_activation, we can call generate_token_for to generate a token and find_by_token_for to find a user from a given token.

This token is not persisted anywhere, it just contains the user id and a signature to verify its authenticity, making it a very convenient way to implement features that require a token.

Expiration

Token expiration is also supported, you can pass a expires_in option to generates_token_for to set the expiration time :

class User < ApplicationRecord
  generates_token_for :account_activation, expires_in: 1.day
end

If you try to find a user by an expired token, it will return nil.

Invalidating the token when something changes

generates_token_for supports making the token dependant on an arbitrary block of code, allowing to implement features like password reset tokens that are invalidated when the password changes:

class User < ApplicationRecord
  generates_token_for :password_reset, expires_in: 1.day do
    password_salt&.last(10)
  end
end

In this example, the token is dependent on the last 10 characters of the password salt.
The generated token will contain the content of the block, this is why it should be deterministic and not contain any sensitive information, and why we use the last 10 characters of the password salt in this example instead of the password hash directly.

When trying to find a user by a token, the block will be called again and compared to the value in the token, if they don't match, the token is considered invalid.

This is very powerful and allows to implement complex token invalidation logic with minimal code, you can make tokens dependent on values, state, timestamps, etc.

How it works

Let's have a look at the code to understand how generates_token_for works. Here's a portion of the ActiveRecord::TokenFor module that is included in our active record classes :

# activerecord/lib/active_record/token_for.rb

included do
  class_attribute :token_definitions, instance_accessor: false, instance_predicate: false, default: {}
end

# ...

def generates_token_for(purpose, expires_in: nil, &block)
  self.token_definitions = token_definitions.merge(purpose => TokenDefinition.new(self, purpose, expires_in, block))
end

We see that a token_definitions hash is defined, and that generates_token_for is just a method that adds a TokenDefinition to it.

The TokenDefinition is a class defined in the same file, through a Struct :

# activerecord/lib/active_record/token_for.rb

TokenDefinition = Struct.new(:defining_class, :purpose, :expires_in, :block) do # :nodoc:
  # Some methods we'll see right after
end

It's basically a class that accepts params such as defining_class purpose, expires_in and block, with some methods used to do the token generation and verification.

Before diving in, let's have a quick look at the generate_token_for method used on an instance of a model :

# activerecord/lib/active_record/token_for.rb

def generate_token_for(purpose)
  self.class.token_definitions.fetch(purpose).generate_token(self)
end

Pretty straight forward, we're looking for the token definition for the good purpose, and calling generate_token on it.

# activerecord/lib/active_record/token_for.rb

def full_purpose
  @full_purpose ||= [defining_class.name, purpose, expires_in].join("\n")
end

def payload_for(model)
  block ? [model.id, model.instance_eval(&block).as_json] : [model.id]
end

def generate_token(model)
  message_verifier.generate(payload_for(model), expires_in: expires_in, purpose: full_purpose)
end

We use the rails MessageVerifier, that can generate and verify signed messages, based on a secret key. It's also used in other features such as CSRF token validation.

We'll use it to generate our token, based on a payload consisting either only of the model id if no block was passed, or the id along the content of the block if one was passed.

The generated string will look like this :

eyJfcmFpbHMiOnsiZGF0YSI6WzQyXSwicHVyIjoiVXNlclxuc2Vzc2lvblxuIn19--7db5fb8690104cec00ec6443353c2362760e7078

The first part is the actual data, in Base64, and the second is the signature.

If we decode the first part, we obtain this :

{"_rails":{"data":[42],"pur":"User\nsession\n"}}

And here is another example for a token using expiration and a block :

{"_rails":{"data":[42,"93LHs7.oVu"],"exp":"2024-04-28T16:44:03.463Z","pur":"User\npassword_reset\n3600"}}

Basically, a token is just a big JSON object containing our purpose, our model id, and optionally the expiration time and arbitrary block value.

Now, the last part to inspect is the find_by_token_for method used on a model class to retrieve a record from a token :

def find_by_token_for(purpose, token)
  raise UnknownPrimaryKey.new(self) unless primary_key
  token_definitions.fetch(purpose).resolve_token(token) { |id| find_by(primary_key => id) }
end

and the resolve_token method on the TokenDefinition:

 def resolve_token(token)
  payload = message_verifier.verified(token, purpose: full_purpose)
  model = yield(payload[0]) if payload
  model if model && payload_for(model) == payload
end

We use the message verifier to decode the token and ensure it was generated by our rails app, this is all done inside the MessageVerifier, this will return nil if the data is invalid, not verified, or expired.

We find the model by its primary key (through yielding its id to the calling method)

Then, we recompute the payload, and compare it with the one we got from the token, if it matches, the model is returned, otherwise nil. Not so magic after all.

Conclusion

As always, reading through the code can help us broader our understanding of a feature, and understand any possible gotchas.

Features such as generates_token_for are great tools to implement strong authentication features in a Rails app, making it possible to drop big dependencies and stay pretty vanilla, with minimal extra code.