Setup your own Kubernetes Cluster with Kops and AWS Infrastructure

As you started reading this article, so I believe you must known What is kubernetes and AWS.

Pre-requisite configuration:

1. AWS Account → https://portal.aws.amazon.com/billing/signup

2. Kubectl (Kubernetes Command-lin Tool):
   Install Kubectl on your system → https://kubernetes.io/docs/tasks/tools/install-kubectl/

3. KOPS (Kubernetes Operations):
   Install Kops on your system, follow this official documentation → https://github.com/kubernetes/kops#installing

4. Install AWS CLI and configure - https://aws.amazon.com/cli/

Verify installation by execcuting aws --version

You need to create new user on AWS. However you can use root user, but its not recommended at all.

• Open IAM console: https://console.aws.amazon.com/iam/
• In the navigation pane, choose Users and then choose Add user
• Type user name for new user. I am using kops as a username for simplicity
• Select type of access as Programmatic access
• Choose Next for Permission and give admin access to this user with AdministratorAccess Policy
• Choose next for Tags and Review
• This will creates an Access Key ID and Secret Access Key. Store them securely as You will not have access to the secret access key again after this step.

Use command aws configure and enter Access Key ID, Secret Access Key and Default Region Name on prompt. I am using ap-south-1 which is Asia Pacific server at Mumbai. See list of aws regions here.

Deploying Kubernetes to AWS

https://kops.sigs.k8s.io/getting_started/aws/

Setup IAM user

Create an IAM user with following permissions:

AmazonEC2FullAccess
AmazonRoute53FullAccess
AmazonS3FullAccess
IAMFullAccess
AmazonVPCFullAccess

Do it with command line>

aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/AmazonEC2FullAccess --group-name kops
aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/AmazonRoute53FullAccess --group-name kops
aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess --group-name kops
aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/IAMFullAccess --group-name kops
aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/AmazonVPCFullAccess --group-name kops

aws iam create-user --user-name kops

aws iam add-user-to-group --user-name kops --group-name kops

aws iam create-access-key --user-name kops

Buy a domain (If don't have already)

I have buy (:p) https://kubernetes.cf for this tutorial

Create Hosted Zone in AWS

Create hosted zone in AWS and update NS in DNS of domain provider

Test the DNS setup

dig ns dev.kubernetes.cf

should get something like
;; ANSWER SECTION:
dev.kubernetes.cf. 172800 IN NS ns-1.awsdns-1.net.
dev.kubernetes.cf. 172800 IN NS ns-2.awsdns-2.org.
dev.kubernetes.cf. 172800 IN NS ns-3.awsdns-3.com.
dev.kubernetes.cf. 172800 IN NS ns-4.awsdns-4.co.uk.

Create cluster state storage on S3

aws s3api create-bucket \
--bucket dev-kubernetes-cf-state-store \
--region ap-south-1
--create-bucket-configuration LocationConstraint=<region>

aws s3api put-bucket-versioning --bucket dev-kubernetes-cf-state-store --versioning-configuration Status=Enabled

aws s3api put-bucket-encryption --bucket dev-kubernetes-cf-state-store --server-side-encryption-configuration '{"Rules":[{"ApplyServerSideEncryptionByDefault":{"SSEAlgorithm":"AES256"}}]}'

Creating the Cluster

export NAME=dev.kubernetes.cf
export KOPS_STATE_STORE=s3://dev-kubernetes-cf-state-store

aws ec2 describe-availability-zones --region ap-south-1

• create secret generate local keys: ssh-keygen

kops create secret --name dev.kubernetes.cf sshpublickey admin -i ~/.ssh/id_rsa.pub

create SSL using AWS Certificate Manager

kops create cluster \
--zones ap-south-1a \
--state s3://dev-kubernetes-cf-state-store \
--api-ssl-certificate arn:aws:acm:[aws-cert-key-id} \
${NAME}
<!-- --topology private \ -->

Master and Worker nodes are configurable in termas of specification and count

If you want to edit something
kops edit cluster ${NAME}

Finally apply cluster configuration
kops update cluster ${NAME} --yes

kubectl get nodes

use following command to validate cluster is up and running, it may take up to 10-15 minutes to complete setup
kops validate cluster

Install Kubernetes Dashboard

Connect to master

ssh to the master: ssh -i ~/.ssh/id_rsa [e-mail]

Install Kube Dashboard

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta6/aio/deploy/recommended.yaml

Create the service account in the current namespace

kubectl create serviceaccount my-dashboard-sa

Give that service account root access on the cluster

kubectl create clusterrolebinding my-dashboard-sa \
--clusterrole=cluster-admin \
--serviceaccount=default:my-dashboard-sa`

Find the secret that was created to hold the token for the SA

kubectl get secrets

Show the contents of the secret to extract the token

kubectl describe secret my-dashboard-sa-token-xxxxx

run kubectl proxy

Install ngnix-controller with helm

Install Helm - https://helm.sh/docs/intro/install/

helm install nginx-cntroller nginx/nginx-ingress

apply SSL on Load Balancer

update load balancer endpoint url in Route53 as alias target

change loadbalancer instance port of 443 same as 80

create service i.e. goapp

User followign docker image for quick reference: https://hub.docker.com/r/abygawade/goapp

create an ingress with following configuration

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: dev-ingress
spec:
  rules:
  - host: dev.kubernetes.cf
    http:
      paths:
      - path: /go
        backend:
          serviceName: goapp
          servicePort: 80

Visit: http://dev.kubernetes.cf/go
https://dev.kubernetes.cf/go

Replace kubernetes.cf with your domain name