Automating GitHub tasks with Node-RED

Some background

While majority of the Node-RED community seems to be interested in home automation (me too, but just a little bit and I honestly think it's light-years ahead of other home automation solutions), I focus mostly on automating my ops tasks as a developer/engineer such as updating cluster resources, for that I wrote Keel, updating CLI clients through continuous delivery, manifests.

Homebrew is a package manager for MacOS where anyone can create their application manifests for an easy distribution and installation. However, there are multiple ways brew formulas can work, such as compiling during installation or downloading a binary. My method of installation is downloading a pre-compiled binary but I had to always remember to update the formula with the latest sha256 https://github.com/webhookrelay/homebrew-tap/blob/master/relay.rb#L5. If a newer version would be released without an update to this manifest, installation via homebrew would fail :) Well, and there were times when I forgot that :D

Disclaimer

This flow is not directly reusable for you as it's working around my infrastructure. The purpose is to showcase some template generation, webhookrelay node and I also managed to find a problem in crypto node (more about it later).

Stack

Node-RED on RPI with node-red-contrib-webhookrelay so I don't need to expose it to the internet, GitHub account + node and Slack (that's really optional, but helpful as I push lots of updates to my Slack channel).

The flow

The actual flow looks like this:

I have uploaded flow here: https://gist.github.com/rusenask/93ddb94479fbd6f1e3dcf3308da4ec1b.

1. Initially, workflow is triggered by a build job that's being executed with Google Cloud Builder and a webhook is dispatched through this little application
2. Flow is triggered by a webhook to webhookrelay.com endpoint which is routed through a tunnel to a node here which are then translated into Node-RED events.
3. Flow then downloads the binary and saves to disk.
4. sha256 custom function node is actually required due to a problem that I found with generating it via node-red-contrib-crypto-js. The problem I suspect was due to the way data is being fed into the crypto node (not in one go but with streams). Even setting it to 'one bytes buffer' it would still present a wrong sha. So, I had to:

In settings.js enable require as we will need additional crypto libs:

    functionGlobalContext: {
        require: require
    },

And then read the file and calculate digest:

var require = global.get('require');
var crypto = require('crypto');
var fs = require('fs');

var algo = 'sha256';
var shasum = crypto.createHash(algo);

var file = '/tmp/relay-darwin-amd64-nr';
var s = fs.ReadStream(file);

s.on('data', function(d) { shasum.update(d); });
s.on('end', function() {
    var sum = shasum.digest('hex');
    node.send({payload: sum});

});

return;

1. Once we have the sha256 sum, we create two templates: one for GitHub brew formula template and one for Slack.
2. For Github we just do a simple overwrite since the file is quite small and only the sha sum ever changes.
3. As for Slack, I chose just to do a simple HTTP request with their incoming webhook integration as the available Slack node seemed a bit too heavyweight for such a tiny requirement. Have a look at the payload that needs to be sent:

{
    "response_type": "in_channel",
    "text": "Brew formula SHA updated to: {{payload}}"
}

You could expand it to add pictures, buttons, attachments and so on, but in this case I just needed to see a notification.

To sum up

So far I am really happy with Node-RED. Even though it would pretty pretty straightforward to just write everything in Go/JS/Python, for some reason it's more fun in Node-RED :) I will be automating more and more side project tasks with it. Also, function node is a life saver!

Comments

[ImTheDeveloper]

Great to see NR posts. I've been using it since the first public release on GitHub many stars ago. It has just the right blend of versatility and usability to hit sweet spots out side of just iot integrations.

I'm currently using it for:

• API calls to notification providers for my monitoring system.

• Telegram bot notifications

• Moving and uploading server back ups to s3

• Endpoint health checks using a simple heartbeat ping.

• Iot doorbell

• Incoming rain alerts based on location

• Car GPS tracking

Quite a mix and definetly a flexible solution. I highly recommend people to give it a shot there's a lot of times I've used NR to prototype before building a full application too.

[pavelsevcik]

Exactly, there's no better visual no-code / low-code / rapid development tool. Node-RED is hidden gem, probably because of being primarily presented for wiring HW, IoT?...

we use it since mid 2016 for anything from prototyping to production grade integration with 3rd party APIs (cause I don't trust to 3rd party like IFTTT or Zappier with API keys and tokens), have on top of it crawler, scraper, ETL pipelines (check jsonata which is integrated), orchestrator for self-healing / scalable infrastructure for platform on top of Docker Swarm, there are no limits, just trades...

highly recommend to check

• jsonata.org/
• npmjs.com/package/node-red-contrib...
• red-bot.io/

[Karolis]

I am actually also starting to use it for API monitoring as I currently have a mix of uptimerobot.com and custom Go application that does more things.

At first I am just going to add websocket based tests and alarms, then expand from there. Have you got any example flows for monitoring?

In my case it's quite asynchronous, as I have to send a webhook and then wait on the socket for it to arrive. Current approach will be to keep sending webhooks on one flow and on another I will have a trigger that's being constantly reset and if a message flow stops - it will send an event :)