Ömer Berat Sezer

Posted on Jan 19

K8s Tutorial - Part 1: Learn and Master Kubernetes, Kubectl, Pods, Deployments, Network, Service

#kubernetes #devops #tutorial #webdev

In the previous post, we focused on the usage area and importance of Kubernetes in IT sector.

https://dev.to/omerberatsezer/what-is-kubernetes-a-beginners-guide-to-container-orchestration-4gbf

This post, we are diving to details. Let’s start.

Kubernetes Architecture

Control Plane: User enters commands and configuration files from control plane. It controls all cluster.
- API Server: It provides access to the Kubernetes API, serving as the front end for the Kubernetes control plane.
- ETCD: A reliable and highly available key-value store that serves as the backbone for all cluster data in Kubernetes.
- Scheduler: It monitors newly created Pods without an assigned node and selects a suitable node for them to run on.
  - Scheduling decisions consider various factors, including:
    - individual and collective resource requirements,
    - hardware/software/policy constraints,
    - affinity and anti-affinity specifications,
    - data locality,
    - inter-workload interference,
    - deadlines.
- Controller Manager: It runs controller processes.
  - Logically, each controller is a separate process, but to reduce complexity, they are all compiled into a single binary and run in a single process.
  - Some types of these controllers are:
    - Node controller: Responsible for noticing and responding when nodes go down.
    - Job controller: Watches for Job objects that represent one-off tasks, then creates Pods to run those tasks to completion.
    - Endpoints controller: Populates the Endpoints object (that is, joins Services & Pods).
    - Service Account & Token controllers: Create default accounts and API access tokens for new namespaces"
Node: "Node components run on every node, maintaining running pods and providing the Kubernetes runtime environment."
- Kubelet: The kubelet is an agent that runs on each cluster node, ensuring containers in a Pod are running and healthy based on provided PodSpecs.
- Kube-proxy: It is a network proxy that runs on each node in your cluster, implementing part of the Kubernetes Service concept.
  - It maintains network rules on nodes. These network rules allow network communication to your Pods from network sessions inside or outside of your cluster.
- Container Runtime: The container runtime is the software that is responsible for running containers. Containerd is container runtime of K8s

How to create K8s cluster?

Different ways:
- Kubeadm, Containerd (on-prem): https://github.com/omerbsezer/Fast-Kubernetes/blob/main/K8s-Kubeadm-Cluster-Setup.md
- Kubespray: https://www.kubecost.com/kubernetes-devops-tools/kubespray/
- Rancher: https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/kubernetes-cluster-setup/rke1-for-rancher
For testing, learning k8s:
- Minikube: https://minikube.sigs.k8s.io/docs/start/

Kubectl Config – Usage

Config File

You can communicate with K8s cluster in different ways:
- REST API,
- Command Line Tool (CLI-Kubectl),
- GUI (kube-dashboard)
After installation, you can find the kubernetes config file (/home/user/.kube/config) that is YAML file.
Config file contains 3 main parts:
- Clusters (cluster certificate data, server, name),
- Context (cluster and user, namespace),
- Users (name, config features, certificates, etc.)
Kubeconfig file helps us to switch between different clusters (e.g. switching to EKS, AKS, on-prem cluster )

kubectl config get-contexts
# view available contexts
kubectl config use-context <context-name>
# set the current context
kubectl config current-context
# verify the current context

Sample kubeconfig file:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZ..
    server: https://192.168.10.50:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kubernetes-admin
  name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
  user:
    client-certificate-data: LS0tLS1CRUdJTiBDRVJUSU...
    client-key-data: LS0tLS1CRUdJTiBSU0...

Usage

Kubectl is our main command line tool that connects minikube. There are many combination of commands. So it is not possible to list all commands.
When run "kubectl" on the terminal, it can be seen some simple commands. Also "kubectl <command> --help" gives more information.
Pattern: kubectl [get|delete|edit|apply] [pods|deployment|services] [podName|serviceName|deploymentName]
Example: "kubectl get pods podName", "kubectl delete pods test_pod", "kubectl describe pods firstpod", etc.
All necessary/most usable commands are listed in the "Kubernetes Commands Cheatsheet". Please have a look to get more information and usage.

Pod: Creating, Yaml, LifeCycle

Pod is the smallest unit that is created and managed in K8s.
Pods may contain more than 1 container, but mostly pods contain only 1 container.
Each pod has unique id (uid).
Each pod has unique IP address.
Containers in the same Pod run on the same Node (computer), and these containers can communicate with each other on the localhost.
Creation of the first pod, IMPERATIVE WAY (with command):

Pod: YAML File

Imperative way could be difficult to store and manage process. Every time we have to enter commands. To prevent this, we can use YAML file to define pods and pods' feature. This way is called "Declarative Way".
Declarative way (with file), Imperative way (with command)
Sample Yaml File:

Pod: Life Cycle

Pending: API->etcd, pod created, pod id created, but not running on the node.
Creating: Scheduler take pod from etcd, assing on node. Kubelet on the Node pull images from docker registry or repository.
ImagePullBackOff: Kubelet can not pull image from registry. E.g. Image name is fault (typo error), Authorization Failure, Username/Pass error.
Running:
- Container closes in 3 ways:
  1. App completes the mission and closes automatically without giving error,
  2. Use or System sends close signal and closes automatically without giving error,
  3. Giving error, collapsed and closes with giving error code.
- Restart Policies (it can defined in the pod definition):
  1. Always: Default value, kubelet starts always when closing with or without error,
  2. On-failure: It starts again when it gets only error,
  3. Never: It never restarts in any case.
- Successed (completed): If the container closes successfully without error and restart policy is configured as on-failure/never, it converts to succeed.
- Failed
- CrashLoopBackOff:
  - If restart policy is configured as always and container closes again and again, container restarts again and again (Restart waiting duration before restarting again: 10 sec -> 20 sec -> 40 sec -> .. -> 5mins), It runs every 5 mins if the pod is crashed.
  - If container runs more than 10 mins, status converted from 'CrashLoopBackOff' to 'Running'.

MultiContainer Pod, Init Container

Best Practice: 1 Container runs in 1 Pod normally, because the smallest element in K8s is Pod (Pod can be scaled up/down).
Multicontainers run in the same Pod when containers are dependent of each other.
Multicontainers in one Pod have following features:
- Multi containers that run on the same pod run on the same node.
- Containers in the same pod can be run/pause/deleted at the same time.
- Containers in the same pod can communicate with each other on localhost, there is not any network isolation.
- Containers in the same pod can use one volume commonly and they can reach same files in the volume.

Init Containers

Init containers are used for configuration of apps before running app container.
Init containers handle what it should run, then it closes successfully, after init containers close, app containers start.
Example below shows how to define init containers in one Pod. There are 2 containers: appcontainer and initcontainer. Initcontainer is polling the service (myservice). When it finds, it closes and app container starts.

apiVersion: v1
kind: Pod
metadata:
  name: initcontainerpod
spec:
  containers:
  - name: appcontainer            # after initcontainer closed successfully, appcontainer starts.
    image: busybox
    command: ['sh', '-c', 'echo The app is running! && sleep 3600']
  initContainers:
  - name: initcontainer
    image: busybox                # init container starts firstly and look up myservice is up or not in every 2 seconds, if there is myservice available, initcontainer closes. 
    command: ['sh', '-c', "until nslookup myservice; do echo waiting for myservice; sleep 2; done"]

# save as service.yaml and run after pod creation
apiVersion: v1
kind: Service
metadata:
  name: myservice
spec:
  ports:
  - protocol: TCP
    port: 80
    targetPort: 9376

Label and Selector, Annotation, Namespace

Label

Label is important to reach the K8s objects with key:value pairs.
key:value is used for labels. E.g. tier:frontend, stage:test, name:app1, team:development
prefix may also be used for optional with key:value. E.g. example.com/tier:front-end, kubernetes.io/ , k8s.io/
In the file (declerative way), labels are added under metadata. It is possible to add multiple labels.

In the command (imperative way), we can also add label to the pods.

kubectl label pods pod1 team=development  
# adding label team=development on pod1
kubectl get pods --show-labels
kubectl label pods pod1 team-  
# remove team (key:value) from pod1
kubectl label --overwrite pods pod1 team=test 
# overwrite/change label on pod1
kubectl label pods --all foo=bar  
# add label foo=bar for all pods

Selector

We can select/filter pods with kubectl.

kubectl get pods -l "app=firstapp" --show-labels
kubectl get pods -l "app=firstapp,tier=frontend" --show-labels
kubectl get pods -l "app=firstapp,tier!=frontend" --show-labels
kubectl get pods -l "app,tier=frontend" --show-labels 
#equality-based selector
kubectl get pods -l "app in (firstapp)" --show-labels  
#set-based selector
kubectl get pods -l "app not in (firstapp)" --show-labels  
#set-based selector
kubectl get pods -l "app=firstapp,app=secondapp" --show-labels 
# comma means and => firstapp and secondapp
kubectl get pods -l "app in (firstapp,secondapp)" --show-labels 
# it means or => firstapp or secondapp

Node Selector

With Node Selector, we can specify which pod run on which Node.

It is also possible to label nodes with imperative way.

kubectl apply -f podnode.yaml
kubectl get pods -w #always watch
kubectl label nodes minikube hddtype=ssd 
# after labelling node, pod11 configuration can run, because node is labelled with hddtype:ssd

Annotation

It is similar to label, but it is used for the detailed information (e.g. owner, notification-email, releasedate, etc.) that are not used for linking objects.

kubectl apply -f podannotation.yaml
kubectl describe pod annotationpod
kubectl annotate pods annotationpod foo=bar 
# imperative way
kubectl delete -f podannotation.yaml

Namespaces

Namespaces provides a mechanism for isolating groups of resources within a single cluster. They provide a scope for names.
Namespaces cannot be nested inside one another and each Kubernetes resource can only be in one namespace.
Kubectl commands run in default namespaces if it is not determined in the command.

kubectl get pods --namespaces kube-system  
# get all pods in the kube-system namespaces
kubectl get pods --all-namespaces  
# get pods from all namespaces
kubectl create namespace development  
# create new development namespace in imperative way
kubectl get pods -n development  
# get pods from the development namespace

In declerative way, it is possible to create namespaces and run pod on the related namespace.

kubectl apply -f namespace.yaml
kubectl get pods -n development  
# get pods in the development namespace
kubectl exec -it namespacedpod -n development -- /bin/sh  
# run namespacepod in development namespace

We can avoid to use -n for all command with changing of default namespace (because, if we don't use -n namespace, kubectl commands run on the default namespace).

kubectl config set-context --current  --namespace=development  
# now default namespace is development
kubectl get pods     
# returns pods in the development namespace  
kubectl config set-context --current  --namespace=default  
# now namespace is default 
kubectl delete namespaces development  
# delete development namespace

Deployment

A Deployment provides declarative updates for Pods and ReplicaSets.
We define states in the deployment, deployment controller compares desired state and take necessary actions to keep desire state.
Deployment object is the higher level K8s object that controls and keeps state of single or multiple pods automatically.
Imperative way:

kubectl create deployment firstdeployment --image=nginx:latest --replicas=2 
kubectl get deployments
kubectl get pods -w    
# on another terminal
kubectl delete pods <oneofthepodname> 
# we can see another terminal, new pod will be created (to keep 2 replicas)  
kubectl scale deployments firstdeployment --replicas=5
kubectl delete deployments firstdeployment

Replicaset

Deployment object create Replicaset object. Deployment provides the transition of the different replicaset automatically.
Replicaset is responsible for the management of replica creation and remove. But, when the pods are updated (e.g. image changed), it can not update replicaset pods. However, deployment can update for all change. So, best practice is to use deployment, not to use replicaset directly.
Important: It can be possible to create replicaset directly, but we could not use rollout/rollback, undo features with replicaset. Deployment provide to use rollout/rollback, undo features.

Rollout and Rollback

Rollout and Rollback enable to update and return back containers that run under the deployment.
2 strategy for rollout:
- Recreate Strategy: Delete all pods first and create Pods from scratch. If two different versions of SW affect each other negatively, this strategy could be used.
- RollingUpdate Strategy (default): It updates pods step by step. Pods are updated step by step, all pods are not deleted at the same time.
  - maxUnavailable: At the update duration, it shows the max number of deleted containers (total:10 containers; if maxUn:2, min:8 containers run in that time period)
  - maxSurge: At the update duration, it shows that the max number of containers run on the cluster (total:10 containers; if maxSurge:2, max:12 containers run in a time)

kubectl set image deployment rolldeployment nginx=httpd:alpine --record  
# change image of deployment
kubectl rollout history deployment rolldeployment                    
# shows record/history revisions 
kubectl rollout history deployment rolldeployment --revision=2              
# select the details of the one of the revisions
kubectl rollout undo deployment rolldeployment                              
# returns back to previous deployment revision
kubectl rollout undo deployment rolldeployment --to-revision=1              
# returns back to the selected revision=1
kubectl rollout status deployment rolldeployment -w                         
# show live status of the rollout deployment
kubectl rollout pause deployment rolldeployment                             
# pause the rollout while updating pods 
kubectl rollout resume deployment rolldeployment                            
# resume the rollout if rollout paused

Network, Service

K8s Networking Requirements

Each pod has unique and own IP address (Containers within a pod share network namespaces).
All PODs can communicate with all other pods without NAT (Network Address Translation)
All NODEs can communicate with all pods without NAT.
The IP of the POD is same throughout the cluster.

CNI (Container Network Interface)

Networking of containers and nodes with different vendors and devices is difficult to handle. So K8s give this responsibility to CNI plugins to handle networking requirements.
"CNI (Container Network Interface), a Cloud Native Computing Foundation project, consists of a specification and libraries for writing plugins to configure network interfaces in Linux containers, along with a number of supported plugins." => https://github.com/containernetworking/cni
K8s has CNI plugins that are selected by the users. Some of the CNI methods are: Flannel, calico, weave, and canal.
Calico (https://github.com/projectcalico/calico) is the one of the popular and open source CNI method/plugin in K8s.
- Network Management in the cluster:
  - IP assignments to Pods
  - IP Table Management
  - Overlay definition between Nodes without using NAT (e.g. --pod-network-cidr management)
  - Vxlan Interface implementation and etc.

Service

"An abstract way to expose an application running on a set of Pods as a network service.
Kubernetes ServiceTypes allow you to specify what kind of Service you want. The default is ClusterIP.
Type values and their behaviors are:
- ClusterIP: Exposes the Service on a cluster-internal IP. Choosing this value makes the Service only reachable from within the cluster. This is the default ServiceType.
- NodePort: Exposes the Service on each Node's IP at a static port (the NodePort). A ClusterIP Service, to which the NodePort Service routes, is automatically created. You'll be able to contact the NodePort Service, from outside the cluster, by requesting :.
- LoadBalancer: Exposes the Service externally using a cloud provider's load balancer. NodePort and ClusterIP Services, to which the external load balancer routes, are automatically created.
- ExternalName: Maps the Service to the contents of the externalName field (e.g. foo.bar.example.com), by returning a CNAME record with its value. No proxying of any kind is set up." (Ref: Kubernetes.io)
Example of Service Object Definition: (Selector binds service to the related pods, get traffic from port 80 to port 9376)

apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  selector:
    app: MyApp
  ports:
    - protocol: TCP
      port: 80
      targetPort: 9376

Liveness and Readiness Probe

Liveness Probe

"The kubelet uses liveness probes to know when to restart a container. For example, liveness probes could catch a deadlock, where an application is running, but unable to make progress." (Ref: Kubernetes.io)
There are different ways of controlling Pods:
- httpGet,
- exec command,
- tcpSocket,
- grpc, etc.
initialDelaySeconds: waiting some period of time after starting. e.g. 5sec, after 5 sec start to run command
periodSeconds: in a period of time, run command.

Readiness Probe

"Sometimes, applications are temporarily unable to serve traffic. For example, an application might need to load large data or configuration files during startup, or depend on external services after startup. In such cases, you don't want to kill the application, but you don't want to send it requests either. Kubernetes provides readiness probes to detect and mitigate these situations. A pod with containers reporting that they are not ready does not receive traffic through Kubernetes Services." (Ref: Kubernetes.io)
Readiness probe is similar to liveness pod. Only difference is to define "readinessProbe" instead of "livenessProbe".

Resource Limit, Environment Variable

Resource Limit

Pods can consume resources (cpu, memory) up to physical resource limits, if there was not any limitation.
Pods' used resources can be limited.
- use 1 cpu core => cpu = "1" = "1000" = "1000m"
- use 10% of 1 cpu core => cpu = "0.1" = "100" = "100m"
- use 64 MB => memory: "64M"
CPU resources are exactly limited when it defines.
When pod requests memory resource more than limitation, pod changes its status to "OOMKilled" and restarts itself to limit memory usage.
Example (below), pod requests 64MB memory and 0.25 CPU core, uses maximum 256MB memory and 0.5 CPU core.

Environment Variable

Environment Variables can be defined for each pods in the YAML file.

Conclusion

This post focuses K8s details (kubeconfig file, kubectl, pod, yaml file, pod life cycle, multi container pod, init container, label, selector, annotation, namespaces, deployment, replicaset, rollout and rollback, network, services, liveness, readiness probe, resource limit and environment variable).

In the next posts, we’ll make some hands-on-labs with minikube to understand better. In addition, we'll continue Part-2 to go over the other part of the details (volume, secret, affinity, taint-toleration, PV, PVC, job, RBAC, ingress).

If you found the tutorial interesting, I’d love to hear your thoughts in the blog post comments. Feel free to share your reactions or leave a comment. I truly value your input and engagement 😉

Follow for Tips, Tutorials, Hands-On Labs for AWS, Kubernetes, Docker, Linux, DevOps, Ansible, Machine Learning, Generative AI, SAAS.

https://github.com/omerbsezer/
https://www.linkedin.com/in/omerberatsezer/

DEV Community