How to break Rails built-in Active Record's Validations

This post originally appeared on Arjun Rajkumar's blog.

This past week I got an error which as it turn out originated because Rails failed with its built-in Active Record validation checkers.

I have this has_many association set up in my database - a Website has many pages

class Website < ApplicationRecord
  has_many :pages, -> { order("created_at DESC")} , dependent: :destroy
end

class Page < ActiveRecord::Base
  belongs_to :website, touch: true, counter_cache: true
end

Each page has a unique URL scoped to a website. This means that pages under a website shouldn't have the same URLs.

I’m using Sidekiq workers to crawl a website in the background, get all it's on-page links and add those link as pages to the website. Sidekiq is multi-threaded - and runs thousands of jobs at the same time. Each of these jobs is creating a page with the crawled link as the URL. So, it may happen that two different jobs creates two pages with the same URL value at the same time, if it discovers two identical links on the website.

And this is exactly what happened.

Here are a few things I tried that didn’t stop Sidekiq from creating duplicate pages under the same website.

These didn't work.

#Active Record's built-in Validation checkers
class Page < ActiveRecord::Base
  validates_uniqueness_of :url, :scope => :website_id
  belongs_to :website, touch: true, counter_cache: true
end

#Transactions
ActiveRecord::Base.transaction do
  website.pages.create(url: new_url) 
end

This worked!

What worked instead was pretty simple.
Adding a unique index directly in the database solved this error.

class AddIndexToPageUrls < ActiveRecord::Migration[5.2]
  def change
    add_index :pages, [:website_id, :url], unique: true
  end
end

And this works 100% of the time. Rails has documented this properly in their docs, and I fixed the error quickly once I knew what was causing it. But diagnosing this error initially to understand why this is happening took time.

I hope this helps someone in the future.