How to Sync Your Portfolio with GitHub Repositories 😻👌

The Issue 🫤

We all (at least most of us) have a portfolio site. As developers, we want to showcase the projects we’ve worked on, which we mostly store in our GitHub profiles.

🤔 But how do we go about showing them on our portfolio site?

Previously, I used to have a ts file to store the links to GitHub repositories and card information, like this:

export const data = [
  {
    id: Math.random(),
    subtitle:
      "A Resume Builder with OpenAI that can generate a resume from given data. Supports file upload with Supabase.",
    title: "Resume Builder with OpenAI",
    github: "https://github.com/shricodev/resume-builder-openai",
    noLive: true,
  },
  {
    id: Math.random(),
    title: "PDFWhisper - OpenAI",
    subtitle: "A PDF Chatbot with OpenAI. It can answer questions from a PDF.",
    github: "https://github.com/shricodev/pdfwhisper-openai",
    liveUrl: "https://pdfwhisper-openai.vercel.app/",
  },
  // ...Rest of the project info
];

Alternatively, you might use a json file to store similar info.

The Problem with This Approach 🤨

The real problem is that this process is entirely manual. You also can’t display project content stored in your README files. If you build a new project, you’d need to manually edit the file, commit the changes, and push them to your repository. Ugh, such a work 😮‍💨.

Why not automate this and, optionally, show README content in markdown as well?

---

The Fix 🛠️

I was frustrated with this process and always felt my portfolio site was outdated because of it. The first thing that crossed my mind was: Can we use GitHub Actions to automate it?

And the answer was yes... But how?

There are a few ways to solve this problem. You could use JavaScript libraries built to interact with the GitHub API (GitHub recommends Octokit.js) or find other alternatives.

When I read the docs, it mentioned generating a personal access token, which felt unnecessary since I only needed data from public repositories. After all, GitHub has a public API (https://api.github.com) with generous rate limits, perfect for fetching repository info.

So, I decided to ditch libraries and use some Bash magic instead. 🪄🧙

Fetching Data Directly with curl

Using a simple curl command, I fetched all the names of the repository:

curl -s "https://api.github.com/users/shricodev/repos?per_page=300" | jq -r '.[] | select(.private == false and .fork == false) | .name' | uniq

This command fetches all repositories names for a given user. It filters out forked repositories (In most cases, definitely don’t want to show any forked repositories in your portfolio) using jq, and then filters the project based on uniqueness (just to be on safer side).

🤔 But how do we filter specific repositories to show in our site?

For this, what we will do is add a unique topic like showcase to the repositories you want to display (you can name it as you like but add this unique topic to all the repositories you wish to show in your portfolio).

Now, all we need to do is modify that above command jq filtering to check if the topic showcase is in the repository, if it is then only fetch all the metadata and stuffs for that repository

curl -s "https://api.github.com/users/shricodev/repos?per_page=300" | jq -r '.[] | select(.private == false and .fork == false and (.topics | index("showcase") != null)) | .name' | uniq

Next, I generated a personal access token (PAT) from GitHub’s Developer Settings page and added it to the repository’s Action secrets (Secrets and variables -> Actions).

Here’s how that looks:

Writing the GitHub Action ✍️

Here’s the complete GitHub Action workflow that uses the above curl command:

name: Fetch Public Repos README's

on:
  schedule:
    # Run once every single day.
    - cron: '0 0 * * *'
  workflow_dispatch:

permissions:
  contents: write

jobs:
  fetch-readmes:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout the repository
        uses: actions/checkout@v4

      - name: Fetch repository list and README's with metadata
        env:
          TARGET_USER: <username>
          TARGET_FOLDER: '<path_to_the_folder_to_store_readme(s)>'
        run: |
          mkdir -p "$TARGET_FOLDER"

          # Fetch list of public repos
          repos=$(curl -s "https://api.github.com/users/$TARGET_USER/repos?per_page=300" | jq -r '.[] | select(.private == false and .fork == false and (.topics | index("showcase") != null)) | .name' | uniq)

          for repo in $repos; do
            readme_file="$TARGET_FOLDER/${repo}_README.mdx"

            if [[ -f "$readme_file" ]]; then
              echo "README for $repo already exists, skipping..."
              continue
            fi

            repo_metadata=$(curl -s "https://api.github.com/repos/$TARGET_USER/$repo")
            title=$(echo "$repo_metadata" | jq -r '.name')
            description=$(echo "$repo_metadata" | jq -r '.description // ""')
            clone_url=$(echo "$repo_metadata" | jq -r '.clone_url')
            language=$(echo "$repo_metadata" | jq -r '.language // ""')
            homepage=$(echo "$repo_metadata" | jq -r '.homepage // ""')
            topics=$(echo "$repo_metadata" | jq -r '.topics | join(", ")')
            created_at=$(echo "$repo_metadata" | jq -r '.created_at')
            updated_at=$(echo "$repo_metadata" | jq -r '.updated_at')

            status_code=$(curl -o /dev/null -s -w "%{http_code}" https://raw.githubusercontent.com/$TARGET_USER/$repo/main/README.md)

            if [ "$status_code" -eq 200 ]; then
              readme_content=$(curl -s https://raw.githubusercontent.com/$TARGET_USER/$repo/main/README.md)

              {
                echo "---"
                echo "title: "\"$title\"\""
                echo "description: "\"$description\"\""
                echo "clone_url: \"$clone_url\""
                echo "language: \"$language\""
                echo "homepage: \"$homepage\""
                echo "topics:"
                for topic in $(echo "$topics" | tr ',' '\n'); do
                  echo "  - \"$topic\""
                done
                echo "created_at: \"$created_at\""
                echo "updated_at: \"$updated_at\""
                echo "---"
                echo ""
                echo "$readme_content"
              } > "$readme_file"
            else
              echo "No README found for $repo, skipping..."
            fi
          done

      - name: Commit Changes
        run: |
          git config --global user.name "github-actions[bot]"
          git config --global user.email "github-actions[bot]@users.noreply.github.com"

          git add -A

          # Commit if there are any changes
          if ! git diff --cached --quiet; then
            git commit -m "Update READMEs with meta and content from public repos"
            git push https://x-access-token:${{ secrets.<secret_name> }}@github.com/<username>/<repository_name>.git HEAD:<branch_name>
          else
            echo "No changes to commit."
          fi            

The GitHub Actions workflow runs daily, fetching all the user's repositories, writing their metadata and README content (if any), and saving them as <repository_name>_README.mdx files.

By adding workflow_dispatch: below the cron timer, we enable support for manually triggering the workflow whenever needed.

If you are following along, make sure that you change the placeholder texts with the actual name.

Now, that we have all the metadata as well as repository README contents, we can use any Markdown parsers to parse the mdx content and show it into our site.

If you want to see my projects page in action, visit 👇

https://techwithshrijal.com/projects

---

Conclusion ⚡

With this simple GitHub Action, your portfolio site will always be in sync with your GitHub projects. By using basic command-line scripting, we automated the process without relying on third-party libraries.

You can find the entire source code here: https://github.com/shricodev/portfolio.git

Check out my portfolio here: https://techwithshrijal.com

Thank you for reading! 🎉

Comments

[Mukesh Singhania]

Great, Shrijal. I really enjoy this. Using GitHub actions this way is super mind blowing.

[Shrijal Acharya]

Endless possibilities. :)

[Aayush Pokharel]

console.log("This is awesome.");

I will soon implement in my own portfolio. Good job sathi! 🫨

[Shrijal Acharya]

Thank you, Aayush! 🤝

[Peshal Bhardwaj]

What is MDX and what is the format?

[Shrijal Acharya]

You can visit mdxjs.com/ to learn more about MDX.

Basically, it's Markdown with metadata and JSX support.

[Qasimo Kaniki]

Good Job.

          if ! git diff --cached --quiet;

what is this command?

[Shrijal Acharya]

We are checking if there is any file with any changes. You can find the documentation on this here: git-scm.com/docs/git-diff

[Kudzai Murimi]

Lesson learnt, thank you so much

[Shrijal Acharya]

Glad you liked it! :)