Redirect YAML From Ghost Data Using JQ

A bit ago I realized I needed to generate redirect text for each of my current blog posts on my Casper themed Ghost website. This seemed to be an overwhelming task but after talking it through both my husband and I came up with two potential ideas. Here is his idea that uses the data file, jq, and visual filtering that I implemented, while tired, and shared in case it helps you.

Quick aside: To go over how to find the redirect section through the Ghost interface you can check out my earlier post in this series How to Find the Redirect Section on Your Ghost Account. After this one I'll share an alternative way to do this. If you are looking for my original full post you can check out Oh No, I Need to Create Redirect Text for All My Posts!.

Word of warning I was tired when doing this, it was all new to me, wasn't working, and I easily got frustrated. As such it's a bit of a journey but I figured I'd share my process in case something in it I may have otherwise omitted helps you out.

Get the Data

To use this method you will need to start with your website data that I got by exporting it from my website through the migration portion of my settings page. I found it by entering my domain and then typing in /ghost/#/settings/migration.

Parse the Data

Once I had my data I needed to extract each of my posts' slugs. We decided to use jq, a lightweight and flexible command-line JSON processor, to get that information but before we did that we first needed to figure out what the command itself, used to parse out that information, should be. To figure it out we opened the data and used ctrl+f to look for a slug and then worked my way back up resulting in the following structure. This is the json code from the Migration Export part of the Settings with only the information needed extracted:

{
  "db": [
    {
      "data": {
    "offers": [],
        "posts": [
          {
          "slug": “foo”,
    }
]}}]}

With the json structure found we next turned to ChatGPT and asked it to create the jq query I needed which gave me the command: jq '.db[].data.posts[].slug' data.json. I used that to test out the results and further iterated through versions while hoping I was getting closer to what I wanted.

Downhill

The resulting yaml from the command worked when uploaded to localhost but didn't work once uploaded to my main website as I had way too many redirects for the website page to load there. I next went back and forth with ChatGPT constantly trying to fix the yaml resulting in, at times, decreasing the number of redirects and a 404 error. In the process I got frustrated and switched over to json so I could use the pattern matching mentioned in Ghost's Implementing redirects and thus switched over to wanting the json output to look something more like:

{"from":"^\/(slug)\/$","to":"/blog/$1","permanent":true},

A Command

During this process I was getting frustrated but looking back, at some point, I was using this command where each section's results were piped | as input into the next section.

cat simply-kyra.ghost.date.json | jq '.db[0].data.posts[] | {"from":"^/(.+)/$","to":"/blog/(.slug)","permanent":true}' | pbcopy

Explained below is each section:

• cat simply-kyra.ghost.date.json: Displays the contents of the downloaded file from Ghost.
• jq '.db[0].data.posts[]: Takes the file's contents and parses out the slugs for each post.
• {"from":"^/(.+)/$","to":"/blog/(.slug)","permanent":true}': Converts the list of slugs to the content string I want consisting of this text.
• pbcopy: and then sends the output to my clipboard so I can paste it into my actual redirects file and upload back up to my website. Just a heads up this is only a Mac command you'll need to adjust it to something else if you're using a different computer.

Got It

I kept having trouble with mismatched brackets and the like so I decided to simplify the automated part of this process by switching out the complicated special string I wanted outputted and instead padded each slug with several dashes on the left and lots of zeros on the right. I had the idea that I could simply do a find all and replace on the results without worrying about the brackets themselves. With this change the new ChatGPT reply went something like this:

Which resulted in the following command:

cat simply-kyra.ghost.date.json | jq -r '.db[].data.posts[] | " ---(.slug)000000"' | pbcopy

This command extracts the slug field from each post and appends 000000 after it while also prefixing it with a triple dash. For example, if the input JSON contains:

{
  "db": [
    {
      "data": {
        "offers": [],
        "posts": [
          {
            "slug": "foo"
          },
          {
            "slug": "bar"
          }
        ]
      }
    }
  ]
}

This command would output:

---foo000000
---bar000000

And then I'd do two find and replace all commands so the 000000 is replaced with )\/$","to":"/blog/$1","permanent":true}, and the triple dashes are replaced with {"from":"^\/( leading to these results:

{"from":"^\/(foo)\/$","to":"/blog/$1","permanent":true},
{"from":"^\/(bar)\/$","to":"/blog/$1","permanent":true},

Warning

During the find and replace portion I happened to notice a page title and realized the posts' section in my data file didn't just include posts but also included all of my pages thus meaning I needed to further mine my file to remove all of the page-related lines that didn't need to be redirected. Luckily I didn't have too many pages but after uploading the new file I realized I missed my about page and it was now redirecting to a location that didn't exist. That said, I'm happy it was just the one page and I was able to fix it by simply renaming the page url (and then fixing the linked menu button) and now it's all good to go... that said, while writing this I realized any previous links to my about post no longer work but at least the menu offers a simple correction for this one.