During the holidays, I found some spare time and decided to tackle a project to scratch my own itch. In this blog post, I'll walk you through the problem I set out to solve, the approach I took to build the solution, and how the hosting infrastructure and tech stack were shaped by the product itself. The result was a streamlined workflow that allowed me to move quickly whilst adhering to some of my key engineering principles: readability, performance, and effective use of TypeScript.
As engineers, it's easy to fall into the trap of overcomplicating things—whether by over-engineering a solution or relying on a standardised tech stack that isn't always the best fit for a project. This post highlights how taking a product-minded and infrastructure-aware approach can help solve problems before they arise.
While this post isn't a development tutorial, you can check out the code on GitHub: https://github.com/ScottHarrisonDev/cc-aggregator-game/tree/main.
The Problem to Solve
As a group of car enthusiasts, my friends and I love discussing our dream cars. An ever-changing list of vehicles we would hypothetically own. A while back, we started a game where one person shares a screenshot of a Collecting Cars auction listing, and the rest of us choose a car we'd like to hypothetically own for a year with all expenses covered. The conversation that follows is always welcome, especially since we don't see each other in person as often as we'd like.
What began as a simple game soon became more complex. Initially, I would take a screenshot of whatever listings fit in my browser window. However, I quickly ran into issues:
Occasionally, there would be listings for number plates or motorcycles, which weren't part of our game. I then started applying the "car only" filter.
I had to zoom out to fit more cars into the screenshot.
Advertisements would often sneak in, requiring me to remove them manually in the Chrome Devtools before sharing the screenshot.
What used to take less than 10 seconds to capture and share now took a couple of minutes each day. That's when I realised I could automate this process, or at the very least, simplify it.
The Requirements
At a high level, the goal was to display a page with 12 cars from Collecting Cars for my friends and me to discuss in our group chat. The data would come from the live auctions page of Collecting Cars, filtered for "cars only" and possibly restricted to the UK.
Example Collecting Cars URL which the game will be based on: https://collectingcars.com/buy?refinementList[listingStage][0]=live&refinementList[regionCode][0]=UK&refinementList[lotType][0]=car
Proof of Concept
💡 Note: Fetching data from internal APIs or scraping HTML is generally messy and unreliable, and should be avoided in production applications. It can also incur additional costs for the source website.
The biggest unknown for this project was how to retrieve the data. One approach was to copy the internal API requests to get the raw data. Another was to scrape the page's HTML directly, after making a request to the page as a user would.
After some investigation, I discovered that Collecting Cars uses Algolia for their product listings, fetched client-side. I was able to replicate their request, simplify it, and use Postman to check if I could retrieve a useful response. Sure enough, it worked! I now had access to the auction data in JSON format.
The Build
I initially built a replica of the Collecting Cars page using Astro and Tailwind CSS, displaying 12 cars in a simplified layout. This mirrored the way we'd previously played the game, where I would send a screenshot of a portion of the website.
"Easy Mode"
One issue we often encountered in the original game was that it was too hard to choose just one car from the 12 available, with many people picking multiple options. To address this, I created an "Easy Mode", which limited the options to just 3 cars, making it easier to pick a single one (in theory!).
However, after a few days of playing, I realised that the "Easy Mode" was actually more fun and faster with only 3 options. So, I simplified the game further by making "Easy Mode" the only mode available.
Mobile Styling
Initially, I assumed that the game would mostly be played on desktop, as we used to share screenshots from the desktop website. However, it quickly became clear that users would now access the site directly and share their choices in the group chat, making mobile usage the primary case.
With this in mind, I made small adjustments for a more mobile-friendly design, using a mobile-first approach with Tailwind CSS. These changes were quick to implement and helped maintain a clean, readable codebase.
Infrastructure
As an Astro-based site, the project can be statically generated or server-side rendered (SSR).
One option was to use Cloudflare Worker Pages with the Astro Cloudflare adapter for SSR. However, since the game is only played once per day and there is no on-the-fly logic, SSR wouldn't provide any benefits. Additionally, different users viewing the site at different times could see different car choices which is not ideal. Using Cloudflare, I could introduce a caching layer to store the page for 24 hours, but that would add unnecessary complexity.
The alternative was to lean into static generation. While I have more experience with other hosting providers, I know GitHub Pages supports static generation for Astro projects. The main limitation here is that the car choices would only update when I rebuild and deploy the site.
I briefly considered setting up a Raspberry Pi to run on my local network and trigger this rebuild via a Cron job. However, upon researching further, I discovered that GitHub Actions supports scheduled workflows, allowing me to trigger a rebuild at a set time each day. This solution was simple, free, and preserved the benefits of static generation, while being sympathetic to the source website API (which would only be called once per day, instead of every time someone visits the site as it would be with SSR).
I added the workflow to the codebase and included a footer note to show the last update time (partly to debug, but also so users who come back after missing a day can be sure they are seeing the up to date choices). This has been running smoothly for a few days now and is performing as expected.
Conclusion
In modern web development, it's easy to default to using tools like Next.js or React, paired with hosting on platforms like Vercel, especially for personal projects where we want to rely on tech we already know. However, this can lead to suboptimal solutions, requiring extra effort to fit a square peg into a round hole.
That's not to say you should always use unfamiliar technologies, but rather that the correct tool for the job should be chosen based on the needs of the product, even if it's not something you're already familiar with. Personal projects are the perfect opportunity to experiment with this mindset, which will serve you well in a commercial context.
The source code for the project can be found on my Github https://github.com/ScottHarrisonDev/cc-aggregator-game and you can view the game at https://scottharrisondev.github.io/cc-aggregator-game/
Top comments (0)