Slackbot to the Rescue: Keeping GitLab Branches in Sync with a Slackbot

If you've ever had to deal with horrific merge conflicts from infrequently synced branches, I've felt your pain. Our team develops off of the Dev branch, promotes code to a QA branch for testing, and merges bug fixes into the QA branch before releasing to production. As we get closer to a release and do more work in QA, Dev can quickly get out of date. And it's easy to forget to down-merge QA to Dev when you're focused on getting a release out. But there's a price to pay: when it comes time to down-merge after the release, resolving the merge conflicts can become a tedious, time-consuming, and error-prone nightmare.

There's got to be a better way! To get us into the habit of down-merging QA to Dev every day, I wanted to make it as easy and frictionless as possible to keep the branches in sync. So I created a slackbot to open a new GitLab merge request ("MR") for us and post it to our team Slack channel each morning.

Each weekday morning, the slackbot checks whether QA and Dev are in sync in our three main repos. If so, it notifies us via Slack that the branches are in sync. If not, it opens a new MR from QA to Dev and posts the link to Slack. It lets us know if there are merge conflicts, and if so, it includes resolution instructions.

Avoiding Gnarly Merge Conflicts Through Continuous Integration

Ultimately, the slackbot is a tool to help us maintain good Continuous Integration ("CI") habits. CI, in its broadest definition, is the practice of frequently integrating code changes into a shared branch, kept in a working state, so that everyone on the team has the most recent code. With respect to branch management, as Martin Fowler puts it, "[t]he over-arching theme is that branches should be integrated frequently and efforts focused on a healthy mainline that can be deployed into production with minimal effort."

One of the biggest benefits of good CI habits is minimizing Git merge conflicts. Git is a fantastic tool for collaboration, and it can often integrate code changes automatically. But the less frequently branches are integrated and the further the code has diverged, the more likely it is that Git won't be able to merge the code without human intervention.

Small merge conflicts from recent changes aren't such a big deal. It's often clear which side of the conflict is correct, even to another developer on the team that didn't author the change. But infrequent integration can lead to "nasty" merge conflicts that quickly spiral out of control.

These "nasty" merges can "generate a considerable amount of work", since the developer resolving the conflicts must first get context on other developers' changes, then figure out how they should be synchronized, then ensure the code is still in working order. They're also dangerous, since human error in resolving complex conflicts can introduce hard-to-trace bugs.

CI is a Habit

To avoid these nasty merge conflicts, we need to get into the habit of frequently integrating our branches. Behavioral psychology teaches that habits are easiest to form when they are made easier. If we set out our running shoes ahead of time, there will be less friction in forming the habit of running. It's also easiest to form habits when they are preceded by a consistent cue to perform the behavior, at the same time and place. We can set an alarm on our phone for the same time every day, which will provide a consistent cue to put on our running shoes and hit the road.

The slackbot helps us form good CI habits by making integration as frictionless as possible. It creates a branch and MR for us and sends it right to where we're already looking. All we need to do is click and review. And integrating daily makes conflicts infrequent and easy to resolve. The slackbot also provides a strong, consistent cue at the same time and place: it sends the message at the same time every work day, right as we're sitting down to our desks.

Creating the Slackbot with Bash Scripting and the GitLab API

To create the slackbot, I used a Bash script, GitLab, and the GitLab API.

For my initial attempt, I decided to write a Bash shell script so I could use Git's CLI to diff QA and Dev and skip the GitLab API calls if the branches are already synced. (I used this command: DIFFCOUNT=$(git diff origin/dev..origin/qa | wc -m | tr -d " ")). That ended up being more trouble than it was worth, both because it's easier to manage multiple repos using only GitLab's API, and because GitLab's diffing algorithm is different than the diff command above, leading to unmergeable empty MRs. (See these issues.)

While I kept the script in Bash after refactoring away the use of the Git CLI, the script could easily be converted to Node or another language.

Bash Tooling

If you're relatively new to Bash scripting like me, I think you'll find that it becomes more straightforward once you get past the terse, outdated syntax and get some experience. But it does require a somewhat different way of thinking about how to solve problems than you might be used to.

Using some modern tooling in VS Code helps. I'd recommend installing the Bash IDE for syntax highlighting and code completion, Shellcheck for code formatting and linting, and Shellman for code snippets.

Explainshell is also helpful for breaking down dense and confusing lines of Bash code. Try pasting in some of the lines of code in this blog for more detailed explanations.

The Frontend Masters course Complete Intro to Linux and the Command-Line is an excellent place to start learning more. And if you'd like more depth, Learning the Bash Shell is a very thorough primer.

Once you've got your tooling set up, follow these steps.

1. Gather Credentials

Gitlab Token

To use the Gitlab API, you'll need to generate a personal access token. Follow the steps here, and make sure to choose the api scope. Save the token somewhere safe.

Gitlab Project ID(s)

For each repo in which you'd like to use the slackbot, navigate to the project homepage and copy down the "Project ID", located directly under the project name.

Slack App

You'll need to create a slack app by clicking "create app" at https://api.slack.com/apps. On your app's homepage, click "add features and functionality", then "incoming webhooks", then click activate.

Create two webhooks: one that sends you a DM for testing (search for your name), and the "production" webhook that posts to your team's Slack channel. Keep the webhook URLs secret and safe.

On your app's home page, you can add a name, image, and description that will be shown when the slackbot posts messages.

2. Setup a Slackbot Branch in one of your Repos

In one of the repos you want to manage with the slackbot, checkout a new branch (e.g., js/slackbot). You can have the slackbot create MRs for multiple repos from this single branch.

Create a .env file

Create a .env file (make sure it's git-ignored!) and paste in your GitLab variables and Slack webhook url for testing locally. (The variables will be set in the GitLab console for the deployed job). Make sure they are exported so they are available to subshells.

Because Bash lacks complex data structures, we'll use string matching to map each repo's display name to its GitLab ID in our environment variables. First, create a comma-separated string listing the single-word display names of the repos you will manage with the slackbot. For example, export REPO_DISPLAY_NAMES="Frontend, Services, Personalization". For each repo in that list, create a matching ID variable with the repo name in all caps followed by _ID, such as FRONTEND_ID, SERVICES_ID, and PERSONALIZATION_ID. The display name variable in the list and the ID variable name before the _ID must match exactly (aside from case). Here's an example:

export REPO_DISPLAY_NAMES="Frontend, Services, Personalization"

export FRONTEND_ID=
export SERVICES_ID=
export PERSONALIZATION_ID=

export GITLAB_TOKEN=
export SLACK_URL=

The benefit of this approach is that adding another repo for the slackbot to manage is as easy as adding another repo name to the list and another repo ID variable.

Create the script file

Create a file called slackbot.sh in your repo, perhaps in a ci folder. Open it and add the Bash shebang on the first line: #!/usr/bin/env bash

Make the script executable. On the command line, run chmod u+x slackbot.sh

Edit your gitlab-ci.yml

Since the slackbot doesn't really fit into the traditional build, test, and deploy stages, add a slackbot stage to the stages key.

stages:
  - slackbot

And add a slackbot job that will execute the script.

slackbot:
  stage: slackbot
  only:
    - schedules
  script:
    - ci/slackbot.sh

You'll want to ensure that only the slackbot gets triggered when the scheduled job runs. For safety, I keep the slackbot in its own branch, the scheduled job targets that branch, and all the other jobs explicitly do not run on schedules. For each other job, you could add:

  except:
    - schedules

3. Create the Slackbot Bash Script

Our slackbot will use another dedicated branch for creating the MRs from QA to Dev -- slackbot/qa-to-dev. This will help later with merge conflicts.

Generally, our script will:

1. Read in the array of repos to be managed and call the handle_repo function for each repo.
2. For each repo, start with a clean slate by deleting the slackbot/qa-to-dev branch (if it exists) and creating a fresh slackbot/qa-to-dev branch off of QA.
3. Create an MR from that branch to Dev.
4. See if merge conflicts exists.
5. Send the link to the MR and any merge conflict info to Slack.

A. Load environment variables.

First, add the following code to load your .env when testing the script locally.

if [ -f "./.env" ]; then
    source ./.env
    echo "Importing env vars from .env file"
fi

We'll exit with an error code if the necessary variables aren't present.

if [ -z "$SLACK_URL" ] || [ -z "$GITLAB_TOKEN" ] || [ -z "$REPO_DISPLAY_NAMES" ];  then
  echo "Missing environment variables: SLACK_URL: $SLACK_URL, GITLAB_TOKEN: $GITLAB_TOKEN, REPO_DISPLAY_NAMES: $REPO_DISPLAY_NAMES"
  exit 1
fi

B. Call the handle_repo function for each repo.

First, transform the comma-separated list of repo names in REPO_DISPLAY_NAMES environment variable into an array:

IFS=', ' read -r -a REPO_ARRAY <<< "$REPO_DISPLAY_NAMES"

Then, for each repo, we'll call our handle_repo function by looping over the array of repo display names, capitalizing the name and adding _ID to get the name of that repo's ID variable, then passing in the repo display name and the repo ID as function arguments.

for REPO in "${REPO_ARRAY[@]}"; do
  REPO_ID="$(echo "$REPO" | tr '[:lower:]' '[:upper:]')_ID" # Capitalize repo display name and add "_ID" to get project id's variable name

  handle_repo "$REPO" "${!REPO_ID}" # Indirect variable. REPO_ID's value is another variable's name, such as FRONTEND_ID, and we access that second variable's value.
  # https://stackoverflow.com/questions/16553089/dynamic-variable-names-in-bash
done

C. Setup the function and variables.

Create a handle_repo function and setup the variables you'll need. Note that the slackbot branch name must be url-escaped for the GitLab API calls:

handle_repo () {

  #1. Set up variables.
  local REPO=$1 # first function argument
  local PROJECT_ID=$2 # second function argument
  local SOURCE_BRANCH_DISPLAY=${3:-"QA"} # default value of QA
  local TARGET_BRANCH_DISPLAY=${4:-"Dev"} # default value of Dev

  # Use exact (lowercase) branch name for Gitlab API calls
  local SOURCE_BRANCH=$(echo "$SOURCE_BRANCH_DISPLAY" | tr '[:upper:]' '[:lower:]' )
  local TARGET_BRANCH=$(echo "$TARGET_BRANCH_DISPLAY" | tr '[:upper:]' '[:lower:]' )

  local MR_TITLE="${SOURCE_BRANCH_DISPLAY} => ${TARGET_BRANCH_DISPLAY}"

  local SLACKBOT_BRANCH="slackbot%2fqa-to-dev" # URL escaped for API calls
  local SLACKBOT_BRANCH_DISPLAY="slackbot/qa-to-dev"

  if [ -z "$REPO" ] || [ -z "$SOURCE_BRANCH" ] || [ -z "$TARGET_BRANCH" ] || [ -z "$PROJECT_ID" ];  then
    echo "Missing environment variables: REPO: $REPO, SOURCE_BRANCH: $SOURCE_BRANCH, TARGET_BRANCH: $TARGET_BRANCH, PROJECT_ID: $PROJECT_ID"
    return 1
  fi

   echo "Running handle_repo for $REPO: From $SOURCE_BRANCH to $TARGET_BRANCH for project $PROJECT_ID"

D. Start with a clean slate.

So that we can follow the same steps every time, even if we forgot to merge in yesterday's MR, we'll delete the current slackbot/qa-to-dev branch if it exists. This (usually) will delete the old MR too. Then we'll create a new slackbot/qa-to-dev branch off of QA.

#2. Delete existing slackbot branch
DELETE_BRANCH_RESP=$(curl -X DELETE --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "https://gitlab.com/api/v4/projects/$PROJECT_ID/repository/branches/$SLACKBOT_BRANCH")
echo "delete branch resp is $DELETE_BRANCH_RESP"

#3. Create remote slackbot branch off of source branch
CREATE_BRANCH_RESP=$(curl -X POST --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "https://gitlab.com/api/v4/projects/$PROJECT_ID/repository/branches?branch=$SLACKBOT_BRANCH&ref=$SOURCE_BRANCH")
echo "create branch resp is $CREATE_BRANCH_RESP"

E. Create the MR and get MR info.

We'll create a new MR from slackbot/qa-to-dev to Dev using the GitLab API, and get the MR ID from the response.

#4. Create merge request to target branch
CREATE_MR_RESP=$(curl -X POST --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "https://gitlab.com/api/v4/projects/$PROJECT_ID/merge_requests?source_branch=$SLACKBOT_BRANCH&target_branch=$TARGET_BRANCH&simple=true" --data-urlencode "title=$MR_TITLE")
echo "create MR resp is $CREATE_MR_RESP"

# Handle Gitlab occasionally retaining yesterday's unmerged MR after delete/create
ALREADY_EXISTS=$(echo "$CREATE_MR_RESP" | grep 'already exists')

# 5. Get MR ID
if [ -n "$ALREADY_EXISTS" ]; then
  MR_ID="${CREATE_MR_RESP//[^0-9]/}" # extract numbers from response string
  echo "MR already exists $MR_ID"
else
  MR_ID=$(echo "$CREATE_MR_RESP" | python3 -c "import sys, json; print(json.load(sys.stdin)['iid'])")
fi

Then, we'll get the MR's info from the GitLab API. I've found we need to sleep for 10 seconds before fetching the MR info to let GitLab finish calculating merge conflict info.

We'll use Python to parse the JSON response and extract the MR's URL, then we'll add the URL to our Slack message.

sleep 10  # Wait for GitLab to calculate merge conflicts info

MR_INFO_RESP=$(curl --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "https://gitlab.com/api/v4/projects/$PROJECT_ID/merge_requests/$MR_ID")
echo "MR info resp is $MR_INFO_RESP"

# 7. Get MR URL
URL=$(echo "$MR_INFO_RESP" | python3 -c "import sys, json; print(json.load(sys.stdin)['web_url'])")

# 8. Set initial slack message
MSG="$REPO: $MR_TITLE: $URL"
echo "New MR: $MSG"

F. Report on merge conflicts.

The MR info API response tells us whether there are merge conflicts. If so, we'll append instructions for resolving them to our Slack message. Since we're using a separate branch, anyone on the team can just pull down slackbot/qa-to-dev, merge Dev into it locally, resolve the conflicts, then push back up. That way, no one has to create a new branch.

Note that we can use Slack text formatting and emojis in our message. 🎉

Also note that when using Python to parse JSON, true values become uppercase True, and null values become None.

# 9. Report on merge conflicts
CONFLICTS=$(echo "$MR_INFO_RESP" | python3 -c "import sys, json; print(json.load(sys.stdin)['has_conflicts'])")

if [ "$CONFLICTS" == 'True' ]; then
  MSG="$MSG. *Merge Conflicts!* 🙀 _Pull down $SLACKBOT_BRANCH_DISPLAY, merge ${TARGET_BRANCH} into it locally, resolve conflicts, then push back up._ 😸"
fi

G. Close empty MRs.

GitLab will still create an "empty" MR even if the branches are in sync, so we'll need to close the MR if it comes back without any changes. If so, we'll set our MSG variable to let the team know the branches are already synced for that repo.

# 10. If MR is empty, close it
CHANGES=$(echo "$MR_INFO_RESP" | python3 -c "import sys, json; print(json.load(sys.stdin)['changes_count'])")
echo "changes count is $CHANGES"

if [ "$CHANGES" == "None" ]; then
  echo "No changes, closing MR"
  MSG="$REPO: $SOURCE_BRANCH_DISPLAY is synced with $TARGET_BRANCH_DISPLAY! 🎉"
  curl -X PUT --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "https://gitlab.com/api/v4/projects/$PROJECT_ID/merge_requests/$MR_ID?state_event=close"
  fi

H. Send to Slack!

Finally, we're ready to send our message to Slack. We'll simply send JSON to our webhook, with a key of text pointing to our MSG variable that we set above, escaping the interior quotation marks.

# 11. Send to Slack
curl -X POST -H 'Content-type: application/json' --data "{\"text\":\"$MSG\"}" "$SLACK_URL"

I. Test Locally

That's the script! We can try running it locally to send a test DM to ourselves and make sure everything works.

4. Setup the Scheduled Job

Almost there! All that's left is to configure the scheduled job on GitLab.

On the GitLab menu in your project, navigate to CI/CD -> Schedules, and click "Create New Schedule".

Set the cron signature and timezone. I set the job to run every weekday at 9am Eastern with this cron signature: 0 9 * * 1-5. (Checkout crontab.guru to test out cron signatures.)

Set the target branch to the branch in which you wrote the slackbot script. (E.g., js/slackbot, not the $SLACKBOT_BRANCH that the slackbot will use to create MRs!)

Enter in your environment variables, set the job to activated, and click save.

I'd recommend testing it out first by setting the SLACK_URL variable to send you a DM, and clicking the play button on the job. If all goes well, switch the SLACK_URL to send the message to your team's channel.

And TADA! You've got yourself a slackbot.

Further Enhancements

The current version of the script assumes that each repo has the same source/target branch names (qa and dev), but that could be made configurable for different branch names or for syncing multiple branches within a repo. And if the script's complexity grows much further in future editions, it's probably time to move over to Node.

Conclusion

And that's how you set up your very own slackbot! It'll help you and your team establish good branch integration habits and avoid frustrating, time-consuming, and dangerous merge conflicts. It's also a fun way to get experience with Bash scripting, GitLab, and Slack Apps. These building blocks open up lots of possibilities for automating away pain-points and improving your team's development experience.

The coolest thing about learning skills like these is that you're no longer limited to the productivity tools handed down to you. You're empowered to create tools that fit your team's unique needs and improve your team's experience.

Happy coding!

Full Slackbot Script

Also available at this gist.

#!/usr/bin/env bash

# Function that will be called for each repo.  Script exectuion begins below.
handle_repo () {

  #1. Set up variables.
  local REPO=$1 # first function argument
  local PROJECT_ID=$2 # second function argument
  local SOURCE_BRANCH_DISPLAY=${3:-"QA"} # default value
  local TARGET_BRANCH_DISPLAY=${4:-"Dev"} # default value

  # Use exact (lowercase) branch name for Gitlab API calls
  local SOURCE_BRANCH=$(echo "$SOURCE_BRANCH_DISPLAY" | tr '[:upper:]' '[:lower:]' )
  local TARGET_BRANCH=$(echo "$TARGET_BRANCH_DISPLAY" | tr '[:upper:]' '[:lower:]' )

  local MR_TITLE="${SOURCE_BRANCH_DISPLAY} => ${TARGET_BRANCH_DISPLAY}"

  local SLACKBOT_BRANCH="slackbot%2fqa-to-dev" # URL escaped for API calls
  local SLACKBOT_BRANCH_DISPLAY="slackbot/qa-to-dev"

  if [ -z "$REPO" ] || [ -z "$SOURCE_BRANCH" ] || [ -z "$TARGET_BRANCH" ] || [ -z "$PROJECT_ID" ];  then
    echo "Missing environment variables: REPO: $REPO, SOURCE_BRANCH: $SOURCE_BRANCH, TARGET_BRANCH: $TARGET_BRANCH, PROJECT_ID: $PROJECT_ID"
    return 1
  fi

  echo "Running handle_repo for $REPO: From $SOURCE_BRANCH to $TARGET_BRANCH for project $PROJECT_ID"

  #2. Delete existing slackbot branch
  DELETE_BRANCH_RESP=$(curl -X DELETE --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "https://gitlab.com/api/v4/projects/$PROJECT_ID/repository/branches/$SLACKBOT_BRANCH")
  echo "delete branch resp is $DELETE_BRANCH_RESP"

  #3. Create remote slackbot branch off of source branch
  CREATE_BRANCH_RESP=$(curl -X POST --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "https://gitlab.com/api/v4/projects/$PROJECT_ID/repository/branches?branch=$SLACKBOT_BRANCH&ref=$SOURCE_BRANCH")
  echo "create branch resp is $CREATE_BRANCH_RESP"

  #4. Create merge request to target branch
  CREATE_MR_RESP=$(curl -X POST --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "https://gitlab.com/api/v4/projects/$PROJECT_ID/merge_requests?source_branch=$SLACKBOT_BRANCH&target_branch=$TARGET_BRANCH&simple=true" --data-urlencode "title=$MR_TITLE")
  echo "create MR resp is $CREATE_MR_RESP"

  # Handle Gitlab occasionally retaining yesterday's unmerged MR after delete/create
  ALREADY_EXISTS=$(echo "$CREATE_MR_RESP" | grep 'already exists')

  # 5. Get MR ID
  if [ -n "$ALREADY_EXISTS" ]; then
    MR_ID="${CREATE_MR_RESP//[^0-9]/}" # extract numbers from response string
    echo "MR already exists $MR_ID"
  else
    MR_ID=$(echo "$CREATE_MR_RESP" | python3 -c "import sys, json; print(json.load(sys.stdin)['iid'])")
  fi

  # 6. Get MR info from GitLab API
  sleep 10  # Wait for GitLab to calculate merge conflicts info
  MR_INFO_RESP=$(curl --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "https://gitlab.com/api/v4/projects/$PROJECT_ID/merge_requests/$MR_ID")
  echo "MR info resp is $MR_INFO_RESP"

  # 7. Get MR URL
  URL=$(echo "$MR_INFO_RESP" | python3 -c "import sys, json; print(json.load(sys.stdin)['web_url'])")

  # 8. Set initial slack message
  MSG="$REPO: $MR_TITLE: $URL"
  echo "New MR: $MSG"

  # 9. Report on merge conflicts
  CONFLICTS=$(echo "$MR_INFO_RESP" | python3 -c "import sys, json; print(json.load(sys.stdin)['has_conflicts'])")

  if [ "$CONFLICTS" == 'True' ]; then
    MSG="$MSG. *Merge Conflicts!* 🙀 _Pull down $SLACKBOT_BRANCH_DISPLAY, merge ${TARGET_BRANCH} into it locally, resolve conflicts, then push back up._ 😸"
  fi

  # 10. If MR is empty, close it
  CHANGES=$(echo "$MR_INFO_RESP" | python3 -c "import sys, json; print(json.load(sys.stdin)['changes_count'])")
  echo "changes count is $CHANGES"

  if [ "$CHANGES" == "None" ]; then
    echo "No changes, closing MR"
    MSG="$REPO: $SOURCE_BRANCH_DISPLAY is synced with $TARGET_BRANCH_DISPLAY! 🎉"
    curl -X PUT --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "https://gitlab.com/api/v4/projects/$PROJECT_ID/merge_requests/$MR_ID?state_event=close"
  fi

  # 11. Send to Slack
  curl -X POST -H 'Content-type: application/json' --data "{\"text\":\"$MSG\"}" "$SLACK_URL"
}

# Begin script execution here:

# Setup env vars.  Located in the Gitlab project settings console.  For running locally, use a .env file at project root and export the vars.
if [ -f "./.env" ]; then
    source ./.env
    echo "Importing env vars from .env file"
fi

if [ -z "$SLACK_URL" ] || [ -z "$GITLAB_TOKEN" ] || [ -z "$REPO_DISPLAY_NAMES" ];  then
  echo "Missing environment Variables: SLACK_URL: $SLACK_URL, GITLAB_TOKEN: $GITLAB_TOKEN, REPO_DISPLAY_NAMES: $REPO_DISPLAY_NAMES"
  exit 1
fi

# Create array from comma-separated string of repo names
IFS=', ' read -r -a REPO_ARRAY <<< "$REPO_DISPLAY_NAMES"

# Call function for each repo.  For each repo name in the REPO_ARRAY, a matching variable with the GitLab project ID named with the repo name in all caps followed by _ID is required
for REPO in "${REPO_ARRAY[@]}"; do
  REPO_ID="$(echo "$REPO" | tr '[:lower:]' '[:upper:]')_ID" # Capitalize repo display name and add "_ID" to get project id's variable name

  handle_repo "$REPO" "${!REPO_ID}" # Indirect variable. REPO_ID's value is another variable's name, such as FRONTEND_ID, and we access that second variable's value.
  # https://stackoverflow.com/questions/16553089/dynamic-variable-names-in-bash
done

(First published on Medium)