My Workflow
When creating a new project with AWS Amplify life is good: you have a lot of different AWS services handy and ready to be used in a integrated way. If you ever used Amplify you know what we are talking about.
Everything is fine until you begin to use Amplify environments. This is because Amplify creates a copy of every service you are using for every environment you are creating. When it comes to Appsync there is no exception.
If you need different environment like development, test and production (maybe you even have demo!) you will find yourself with three times your cognito pools (if you have authentication), three times your lambdas and three times your graphql tables. Often times when you are developing, you will have to make changes to your schema.graphql but you don't want to reflect those changes in other environments right away because maybe the UI is still not updated or simply because your work isn't completly done yet and you don't want to deploy a broken version.
The problem you can incur into is a known Amplify problem, in fact AWS even has a section on their docs about it: Deploying multiple secondary indices (GSI), and their github repository has a lot of issues about this.
Attempting to mutate more than 1 global secondary index at the same time on the <table_name> table in the stack.
An error occurred during the push operation: Attempting to mutate more than 1 global secondary index at the same time on the <table_name> table in the stack.
If you try to modify 2 or more connections on the same table, when you try to push your changes to Cloud Formation with the command amplify push, you will get this error. When you're working on the development branch is not a big deal because you only have to make multiple "amplify push" in order to gradually change the connections state.
The REAL problem comes after you have made a number of amplify push on your development environment and you want to replicate the final schema.graphql to the test or production environment. Suppose you made 10 amplify push on your development environment but your production environment has never been updated after push number 3: now if you try to move your current schema.graphql (let's call it version 10) over your schema.graphql version 3 on production, you will have a big problem. Amplify won't be able to make multiple updates on the secondary indexes of your production tables just like it wasn't before, when you was trying to make only two updates together on the same table.
You would have to replicate the exact sequence of successful updates you made on you development environment to you production one.
Here comes into play PUSHARD!
First of all you have to make a commit on git after every amplify push on your environment.
This Git Action will then take care to grab your commit hisory from development branch, detect the exact slice of commits you need to fully replicate the development verison of schema.graphql to your target environment. In order to deploy every update it will try to merge the current development commit into target branch, launch amplify push command and finally git commit the result after every successful operation on the target branch.
If something shoud go south during the process, it will push to git everything is committed at the time, the next time it will try to pick up the process where it left off.
Now, this flow is correct from a AWS standpoint because DynamoDB will be able to update step by step its tables. But at the same time, when asking many consecutive Cloud Formation updates, especially if you created or updated a lambda in your stack too, will take longer than the time of the simple amplify push command, so, at the next iteragion of updates loop, you could have another error:
Resource is not in the state stackUpdateComplete
This means that Cloud Formation has not yet completely update your lambda resources and, because of this, it won't be able to make further updates.
Because of this you have to make a commit after every time you create or modify a lambda function. This way our Git Action will be able to successfully push the change to amplify and during the next loop iteration it will wait for Cloud Formation to be in the UPDATE_COMPLETE state before trying to update the stack again.
When the git action is starting it can send you a Telegram message writing how many commits it will take from development to the target branch; it will message you after every successful amplify push operation and if something goes wrong.
nautes-tech
/
pushard
Progressively deploy commits between AWS Amplify environments
PUSHARD Progressively deploy commits between AWS Amplify environments
Description
When creating a new project with AWS Amplify life is good: you have a lot of different AWS services handy and ready to be used in a integrated way. If you ever used Amplify you know what we are talking about Everything is fine until you begin to use Amplify environments. This is because Amplify creates a copy of every service you are using for every environment you are creating. When it comes to Appsync there is no exception.
If you need different environment like development, test and production (maybe you even have demo!) you will find yourself with three times your cognito pools (if you have authentication), three times your lambdas and three times your graphql tables. Often times when you are developing, you will have to make changes to your schema.graphql but you don't want to…
Submission Category:
DIY Deployments, Maintainer Must-Haves.
Yaml File or Link to Code
name: pushard
on:
workflow_dispatch:
inputs:
deployHash:
description: 'Target deploy hash'
required: true
skipHashes:
description: 'Deploys to skip'
required: false
dstBranch:
description: 'Destination branch'
required: true
jobs:
run_pushard:
name: Run PUSHARD
runs-on: ubuntu-18.04
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Setup Node.js environment
uses: actions/setup-node@v2.1.5
with:
node-version: '14'
- name: Create working dir
shell: bash
run: mkdir tempdir
- name: Install AWS CLI version 2
run: |
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
sudo unzip awscliv2.zip
sudo ./aws/install
sudo rm -rf awscliv2.zip
sudo rm -rf aws
aws --version
- name: Install @aws-amplify/cli node package
run: npm install -g @aws-amplify/cli@7.3.2
- name: PUSHARD
run: |
echo "Target deploy hash: ${{ github.event.inputs.deployHash }}"
echo "Deploys to skip: ${{ github.event.inputs.skipHashes }}"
echo "Destination branch: ${{ github.event.inputs.dstBranch }}"
cd builder
npm install
npm start
env:
DEPLOY_HASH: ${{ github.event.inputs.deployHash }}
SKIP_HASHES: ${{ github.event.inputs.skipHashes }}
DST_BRANCH: ${{ github.event.inputs.dstBranch }}
AMPLIFY_PROJECT_REPO: ${{ secrets.AMPLIFY_PROJECT_REPO }}
AWS_ACCESS_KEY_ID: ${{ secrets.ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.SECRET_ACCESS_KEY }}
AWS_PROFILE: ${{ secrets.AWS_PROFILE }}
GH_USERNAME: ${{ secrets.GH_USERNAME }}
GH_USER_PAT: ${{ secrets.GH_USER_PAT }}
TELEGRAM_BOT_TOKEN: ${{ secrets.TELEGRAM_BOT_TOKEN }}
TELEGRAM_CHAT_ID: ${{ secrets.TELEGRAM_CHAT_ID }}
Additional Resources / Info
Made by
Top comments (0)