Ingress Controllers & ALB in AWS EKS

In the world of Kubernetes, managing external access to services can be a complex task. This is where Ingress Controllers come into play. In this blog, let's delve deep into Ingress Controllers & ALB, a critical component that acts as an entry point for external traffic into your Kubernetes cluster. I'll explore everything from the fundamentals to advanced configurations, enabling you to master this crucial aspect of Kubernetes networking. An Ingress Controller is a critical component that serves as an entry point for external traffic into your Kubernetes cluster.

Ingress Controller

Ingress means act of entering to access workload. Ingress is a native Kubernetes resource like pods, deployments, etc. Using ingress, you can maintain the DNS routing configurations. The ingress controller does the actual routing by reading the routing rules from ingress objects stored in etcd.

Without an Ingress Controller, accessing services within a Kubernetes cluster from outside the cluster becomes challenging. Here's what happens:

• No Defined Entry Point: By default, services within a Kubernetes cluster are only accessible internally. There is no defined entry point for external traffic.
• Direct Service Access: You can access services directly by using NodePort or LoadBalancer type services. However, this approach has limitations:
  1. NodePort: Exposes the service on a static port on each node. It requires clients to connect directly to a specific node, making it less scalable and practical for production use.
  2. LoadBalancer: Requires a cloud provider's load balancer service, which can be costly. It also introduces additional complexity for on-premises or self-hosted clusters. Complex Routing Challenges: If you have multiple services, each with its own domain or path requirements, managing the routing becomes complex and non-standardized.
• Lack of SSL Termination
• No Fine-Grained Routing Control: You can't perform advanced routing based on request headers, cookies, or other Layer 7 characteristics.
• No Built-In URL Rewriting: Without an Ingress Controller, you can't easily perform URL rewriting or redirection.

Key Concepts

Ingress Resource:

An Ingress Resource is a Kubernetes object that defines rules for how external HTTP/S traffic should be routed to services within the cluster. It acts as a Layer 7 (application layer) load balancer.

Ingress Controller:

An Ingress Controller is a specialized Kubernetes pod that interprets and enforces the rules defined in the Ingress Resource. It acts as a traffic manager, handling incoming requests and directing them to the appropriate services.

Controller Types:

There are various Ingress Controllers available, each designed to work with different cloud providers or on-premises solutions. Popular choices include NGINX Ingress Controller, Traefik, HAProxy, and more.

How Ingress Controllers Work

When a new Ingress Resource is created or an existing one is updated, the Ingress Controller interprets the rules and configurations defined. It then configures the corresponding load balancer (e.g., AWS ALB, NLB) to route traffic based on these rules.

Why Use an Ingress Controller?

Simplifies Traffic Management: Ingress Controllers simplify the process of managing external access to services by providing a single entry point for multiple services.

• SSL Termination: They can handle SSL/TLS termination, decrypting incoming traffic and forwarding it to services over HTTP.
• Virtual Host Routing: Ingress Controllers enable virtual host-based routing, allowing multiple domains to be associated with different services.
• Path-Based Routing: They support path-based routing, allowing different paths within a domain to be directed to different services.
• Load Balancing: Ingress Controllers distribute incoming traffic across multiple pods in a service, providing load balancing capabilities.
• Name-Based Virtual Hosting: They support mapping multiple domain names to the same service, allowing for efficient resource utilization.

How Ingress Controllers Work

• Ingress Resource Creation: Users create an Ingress Resource in their Kubernetes cluster, specifying the routing rules and other configurations.
• Ingress Controller Watches for Changes: The Ingress Controller continuously monitors the API server for changes to Ingress Resources.
• Interpreting Rules: When a new Ingress Resource is created or an existing one is updated, the Ingress Controller interprets the rules and configurations defined.
• Configuring Load Balancers: Depending on the cloud provider, the Ingress Controller configures the corresponding load balancer (such as AWS ALB or NLB) with the necessary settings.
• Routing Traffic: Once the load balancer is configured, it starts routing external traffic based on the rules defined in the Ingress Resource.

AWS ALB Ingress Controller

Amazon Elastic Load Balancing Application Load Balancer (ALB) is a popular AWS service that load balances incoming traffic at the application layer (layer 7) across multiple targets, such as Amazon EC2 instances, in a region. ALB supports multiple features including host or path based routing, TLS (Transport Layer Security) termination, WebSockets, HTTP/2, AWS WAF (Web Application Firewall) integration, integrated access logs, and health checks. Refer AWS ALB Ingress Controller Git Repo

The open source AWS ALB Ingress controller triggers the creation of an ALB and the necessary supporting AWS resources whenever a Kubernetes user declares an Ingress resource in the cluster. The Ingress resource uses the ALB to route HTTP(S) traffic to different endpoints within the cluster.

Steps to configure-

• Install eksctl

curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
sudo mv -v /tmp/eksctl /usr/local/bin
eksctl version

# Enable eksctl bash-completion
eksctl completion bash >> ~/.bash_completion
. /etc/profile.d/bash_completion.sh
. ~/.bash_completion

# Enable kubectl bash_completion
kubectl completion bash >>  ~/.bash_completion
. /etc/profile.d/bash_completion.sh
. ~/.bash_completion

• Launch EKS cluster

eksctl create cluster --name=sample-cluster

• Now create an IAM OIDC provider and associate it with your cluster:

eksctl utils associate-iam-oidc-provider --cluster=sample-cluster --approve

• Deploy the relevant RBAC roles and role bindings as required by the AWS ALB Ingress controller:

kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/v1.1.4/docs/examples/rbac-role.yaml

• Create an IAM policy named ALBIngressControllerIAMPolicy to allow the ALB Ingress controller to make AWS API calls on your behalf.

aws iam create-policy \
    --policy-name ALBIngressControllerIAMPolicy \
    --policy-document https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/v1.1.4/docs/examples/iam-policy.json

• Create a Kubernetes service account and an IAM role (for the pod running the AWS ALB Ingress controller):

eksctl create iamserviceaccount \
       --cluster=attractive-gopher \
       --namespace=kube-system \
       --name=alb-ingress-controller \
       --attach-policy-arn=$ALBIngressControllerIAMPolicy.Arn \
       --override-existing-serviceaccounts \
       --approve

• Deploy the AWS ALB Ingress controller:

curl -sS "https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/v1.1.4/docs/examples/alb-ingress-controller.yaml" \
     | sed "s/# - --cluster-name=devCluster/- --cluster-name=attractive-gopher/g" \
     | kubectl apply -f -

• Verify that the deployment was successful and the controller started:

$ kubectl logs -n kube-system $(kubectl get po -n kube-system | egrep -o alb-ingress[a-zA-Z0-9-]+)
-------------------------------------------------------------------------------
        AWS ALB Ingress controller
          Release:    v1.1.4
          Build:      git-0db46039
          Repository: https://github.com/kubernetes-sigs/aws-alb-ingress-controller
-------------------------------------------------------------------------------

• Create Deployment & Service (flask-nginx-deployment.yaml)

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: flask-nginx-deployment
  namespace: apps1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: flask-nginx
  template:
    metadata:
      labels:
        app: flask-nginx
    spec:
      containers:
      - name: flask
        image: yogi4289/sample-flask-app
        ports:
        - containerPort: 5000
        livenessProbe:
          httpGet:
            path: /health
            port: 5000
          initialDelaySeconds: 10
          periodSeconds: 5        
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80
        - containerPort: 443
        volumeMounts:
        - name: nginx-conf
          mountPath: /etc/nginx/nginx.conf
          # /etc/nginx/conf.d/default.conf
          subPath: nginx.conf
      volumes:
      - name: nginx-conf
        configMap:
          name: nginx-config
---
apiVersion: v1
kind: Service
metadata:
  name: flask-nginx-service
  namespace: apps1
spec:
  selector:
    app: flask-nginx
  ports:
    - name: http
      port: 80
      targetPort: 80
    - name: https
      port: 443
      targetPort: 443
  type: ClusterIP
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-config
  namespace: apps1
data:
  nginx.conf: |
    worker_processes  1;
    events {worker_connections  1024;}
    http {
      upstream flask-app {server 127.0.0.1:5000;}
      server {
        listen       80;
        server_name  localhost;
        location / {
          proxy_pass         http://localhost:5000;
          proxy_redirect     off;
          proxy_set_header   Host $host;
          proxy_set_header X-Real-IP $remote_addr;
        }
      }
    }

• Deploy the kubernetes deployment, service, configmap using

kubectl apply -f flask-nginx-deployment.yaml

• Create Ingress (flask-nginx-ingress.yaml)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: flask-nginx-ingress
  namespace: apps1
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":80}]'
    alb.ingress.kubernetes.io/backend-protocol: HTTP
    alb.ingress.kubernetes.io/load-balancer-name: apps1-ingress
    # alb.ingress.kubernetes.io/healthcheck-path: /health
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: flask-nginx-service
            port:
                name: http

Note- In prod cluster, include more annotations like-

    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/security-groups: sg-00xxxxxx
    alb.ingress.kubernetes.io/subnets: subnet-xxx, subnet-xxx, subnet-xxxx

• Deploy the Ingress resource using

kubectl apply -f flask-nginx-ingress.yaml

• Verify that the Ingress resource is enabled:

kubectl get ingress/flask-nginx-ingress
NAME           HOSTS   ADDRESS                                                                  PORTS   AGE
flask-nginx-ingress   *       DNS-Name-Of-Your-ALB   80      3m

• Open a browser and copy-paste your DNS-Name-Of-Your-ALB/first and DNS-Name-Of-Your-ALB/second to verify the results.

Ingress Controllers play a pivotal role in managing external traffic in a Kubernetes environment. They simplify the process of routing traffic to services, handle SSL termination, and provide a centralized point for traffic management. Choosing the right Ingress Controller for your environment, whether it's AWS ALB, NGINX, Traefik, or another, is crucial for optimizing your application's external access.