In the previous part, we focused on the first part of the K8s Tutorial - Part 1: Learn and Master Kubectl, Pods, Deployments, Network, Service
Now, we are diving to the second part of the K8s Tutorial. Let’s start.
Table of Contents
- Kubernetes Architecture
- Volume
- Secret
- ConfigMap
- Node – Pod Affinity
- Taint and Toleration
- Deamon Set
- Persistent Volume and Persistent Volume Claim
- Storage Class
- Stateful Set
- Job, CronJob
- Authentication, Role Based Access Control, Service Account
- Ingress
Kubernetes Architecture
- It is mentioned in the Part-1 in detail.
Volume
- Ephemeral volume (Temporary volume): Multiple containers reach ephemeral volume in the pod. When the pod is terminated, the volume is also deleted. But when container is restarted, the volume is still available because the pod still runs.
- There are 2 types of ephemeral volumes:
- Emptydir
-
Hostpath
- Directory
- DirectoryOrCreate
- FileOrCreate
Emptydir Volume
- Emptydir (empty directory on the node) is created on which node the pod is created on and it is mounted on the container using
volumeMounts. Multiple containers in the pod can reach this volume (read/write). - Emptydir volume is dependent of Pod Lifecycle. If the pod is deleted/terminated, emptydir is also deleted.
spec:
containers:
- name: sidecar
image: busybox
command: ["/bin/sh"]
args: ["-c", "sleep 3600"]
volumeMounts: # volume is mounted under "volumeMounts"
- name: cache-vol # "name" of the volume type
mountPath: /tmp/log # "mountPath" is the path in the container.
volumes:
- name: cache-vol
emptyDir: {} # "volume" type "emptydir"
Hostpath Volume
- It is similar to emtpydir,
hostpathis also created on which node the pod is created on. In addition, the hostpath is specifically defined path on the node.
apiVersion: v1
kind: Pod
metadata:
name: hostpath
spec:
containers:
- name: hostpathcontainer
image: ImageName # e.g. nginx
volumeMounts:
- name: directory-vol # container connects "volume" name
mountPath: /dir1 # on the container which path this volume is mounted
- name: dircreate-vol
mountPath: /cache # on the container which path this volume is mounted
- name: file-vol
mountPath: /cache/config.json # on the container which file this volume is mounted
volumes:
- name: directory-vol # "volume" name
hostPath: # "volume" type "hostpath"
path: /tmp # "path" on the node, "/tmp" is defined volume
type: Directory # "hostpath" type "Directory", existed directory
- name: dircreate-vol
hostPath: # "volume" type "hostpath"
path: /cache # "path" on the node
type: DirectoryOrCreate # "hostpath" type "DirectoryOrCreate", if it is not existed, create directory
- name: file-vol
hostPath: # "volume" type "hostpath"
path: /cache/config.json # "path" on the node
type: FileOrCreate # "hostpath" type "FileOrCreate", if it is not existed, create file
Secret
- Secret objects store the sensitive and secure information like username, password, ssh-tokens, certificates.
- Secrets (that you defined) and pods (that you defined) should be in the same namespace (e.g. if defined secret is in the "default" namespace, pod should be also in the "default" namepace).
- There are 8 different secret types (basic-auth, tls, ssh-auth, token, service-account-token, dockercfg, dockerconfigjson, opaque). Opaque type is the default one and the mostly used one.
- Secrets are called by the pod in 2 different ways:
volumeandenvironment variable - Imperative way, run on the terminal (generic in the command = opaque):
kubectl create secret generic mysecret2 --from-literal=db_server=db.example.com --from-literal=db_username=admin --from-literal=db_password=P@ssw0rd!
- Imperative way with file to hide pass in the command history:
kubectl create secret generic mysecret3 --from-file=db_server=server.txt --from-file=db_username=username.txt --from-file=db_password=password.txt
- Imperative way with json file to hide pass in the command history:
kubectl create secret generic mysecret4 --from-file=config.json
ConfigMap
- It is same as
secrets. The difference is that configmap does not save sensitive information. It stores config variables. - Configmap stores data with key-value pairs.
- Configmaps are called by the pod in 2 different ways: volume and environment variable.
Node – Pod Affinity
- Affinity means closeness, proximity, familarity.
Node Affinity
- With the node affinity, specific pods can enable to run on the desired node (Node selector also supports that feature, but node affinity is more flexible).
- If the node is labelled with key-value, we can run some of the pods on that specific node.
-
Terms for Node Affinity:
- requiredDuringSchedulingIgnoredDuringExecution: Find a node during scheduling according to "matchExpression" and run pod on that node. If it is not found, do not run this pod until finding specific node "matchExpression".
- IgnoredDuringExecution: After scheduling, if the node label is removed/deleted from node, ignore it while executing.
-
preferredDuringSchedulingIgnoredDuringExecution: Find a node during scheduling according to "matchExpression" and run pod on that node. If it is not found, run this pod wherever it finds.
- weight: Preference weight. If weight is more than other weights, this weight is higher priority than others.
Pod Affinity
- Some of the pods should run with other pods on same node or same availability zone (e.g. frontend pods run with cache pod on the same availability zone)
- If the pod affinity is defined for one pod, that pod runs with the related pod on same node or same availability zone.
- Each node in the cluster is labelled with default labels.
-
kubernetes.io/hostname: e.gkubernetes.io/hostname=minikube -
kubernetes.io/arch: e.gkubernetes.io/arch=amd64 -
kubernetes.io/os: e.gkubernetes.io/os=linux
-
- Each node in the cluster that runs on the cloud is labelled with following labels.
-
topology.kubernetes.io/region: e.g.topology.kubernetes.io/region=northeurope -
topology.kubernetes.io/zone: e.g.topology.kubernetes.io/zone=northeurope-1
-
apiVersion: v1
kind: Pod
metadata:
name: frontendpod
labels:
app: frontend # defined labels
deployment: test
spec:
containers:
- name: nginx
image: nginx:latest
ports:
- containerPort: 80
---
apiVersion: v1
kind: Pod
metadata:
name: cachepod
spec:
affinity:
podAffinity:
requiredDuringSchedulingIgnoredDuringExecution: # required: if not found, not run this pod on any node
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- frontend
topologyKey: kubernetes.io/hostname # run this pod with the POD which includes "app=frontend" on the same worker NODE
preferredDuringSchedulingIgnoredDuringExecution: # preferred: if not found, run this pod on any node
- weight: 1
podAffinityTerm:
labelSelector:
matchExpressions:
- key: branch
operator: In
values:
- develop
topologyKey: topology.kubernetes.io/zone # run this pod with the POD which includes "branch=develop" on the any NODE in the same ZONE
podAntiAffinity: # anti-affinity: NOT run this pod with the following match ""
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: deployment
operator: In
values:
- prod
topologyKey: topology.kubernetes.io/zone # NOT run this pod with the POD which includes "deployment=prod" on the any NODE in the same ZONE
containers:
- name: cachecontainer # cache image and container name
image: redis:6-alpine
Taint and Toleration
- Node affinity is a property of the pods that attracts/accepts them to a set of nodes. Taints are the opposite, they allow a node to repel/reject a set of pods.
-
TAINTsare assigned to theNODEs.TOLERATIONsassigned to thePODs-
kubectl describe nodes minikube, at taint section, it can be seen taints. - To add taint to the node with commmand:
kubectl taint node minikube app=production:NoSchedule - To delete taint to the node with commmand:
kubectl taint node minikube app-
-
- If pod has not any toleration for related taint, it can not be started on the tainted node (status of pod remains pending)
- Taint Types:
- key1=value1:effect: (e.g."kubectl taint node minikube app=production:NoSchedule")
- Taint "effect" types:
- NoSchedule: If pod is not tolerated with this effect, it can not run on the related node (status will be pending, until toleration/untaint)
- PreferNoSchedule: If pod is not tolerated with this effect and if there is not any untainted node, it can run on the related node.
- NoExecute: If pod is not tolerated with this effect, it can not run on the related node. If there are pods running on the node before assigning "NoExecute" taint, after tainting "NoExecute", untolerated pods stopped on this node.
Deamon Set
- It provides to run pods on EACH nodes. It can be configured to run only specific nodes.
- For example, you can run log application that runs on each node in the cluster and app sends these logs to the main log server. Manual configuration of each nodes could be headache in this sceneario, so using deamon sets would be beneficial to save time and effort.
- If the new nodes are added on the cluster and running deamon sets on the cluster at that time period, default pods which are defined on deamon sets also run on the new nodes without any action.
Persistent Volume and Persistent Volume Claim
- Volumes are ephemeral/temporary area that stores data. Emptydir and hostpath create volume on node which runs related pod.
- In the scenario of creating Mysql pod on cluster, we can not use emptydir and hostpath for long term. Because they don't provide the long term/persistent volume.
- Persistent volume provides long term storage area that runs out of the cluster.
- There are many storage solutions that can be enabled on the cluster: nfs, iscsi, azure disk, aws ebs, google pd, cephfs.
- Container Storage Interface (CSI) provides the connection of K8s cluster and different storage solution.
Persistent Volume
-
accessModestypes:-
ReadWriteOnce: read/write for only 1 node. -
ReadOnlyMany: only read for many nodes. -
ReadWriteMany: read/write for many nodes.
-
-
persistentVolumeReclaimPolicytypes: It defines the behaviour of volume after the end of using volume.-
Retain: Volume remains with all data after using it. -
Recycle: Volume is not deleted but all data in the volume is deleted. We get empty volume if it is chosen. -
Delete: volume is deleted after using it.
-
# Creating Persistent Volume on NFS Server on the network
apiVersion: v1
kind: PersistentVolume
metadata:
name: mysqlpv
labels:
app: mysql # labelled PV with "mysql"
spec:
capacity:
storage: 5Gi # 5Gibibyte = power of 2; 5GB= power of 10
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Recycle # volume is not deleted, all data in the volume will be deleted.
nfs:
path: /tmp # binds the path on the NFS Server
server: 10.255.255.10 # IP of NFS Server
Persistent Volume Claim (PVC)
- We should create PVCs to use volume. With PVCs, existed PVs can be chosen.
- The reason why K8s manage volume with 2 files (PVC and PV) is to seperate the management of K8s Cluster (PV) and using of volume (PVC).
- If there is seperate role of system management of K8s cluster, system manager creates PV (to connect different storage vendors), developers only use existed PVs with PVCs.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mysqlclaim
spec:
accessModes:
- ReadWriteOnce
volumeMode: Filesystem # VolumeMode
resources:
requests:
storage: 5Gi
storageClassName: ""
selector:
matchLabels:
app: mysql # choose/select "mysql" PV that is defined above.
Storage Class
- Creating volume with PV is manual way of creating volume. With storage classes, it can be automated.
- Cloud providers provide storage classes on their infrastructure.
- When pod/deployment is created, storage class is triggered to create PV automatically (Trigger order: Pod -> PVC -> Storage Class -> PV).
# Storage Class Creation on Azure
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: standarddisk
parameters:
cachingmode: ReadOnly
kind: Managed
storageaccounttype: StandardSSD_LRS
provisioner: kubernetes.io/azure-disk
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
-
storageClassNameis added into PVC file.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mysqlclaim
spec:
accessModes:
- ReadWriteOnce
volumeMode: Filesystem
resources:
requests:
storage: 5Gi
storageClassName: "standarddisk" # selects/binds to storage class (defined above)
- When deployment/pod request PVC (claim), storage class provides volume on the infrastructure automatically.
Stateful Set
- Pods/Deployments are stateless objects. Stateful set provides to run stateful apps.
- Differences between Deployment and Statefulset:
- Name of the pods in the statefulset are not assigned randomly. It gives name statefulsetName_0,1,2,3.
- Pods in the statefulset are not created at the same time. Pods are created in order (new pod creation waits until previous pod's running status).
- When scaling down of statefulset, pods are not deleted in random. Pods are deleted in reverse order.
- If PVC is defined in the statefulset, each pod in the statefulset has own PV.
Job, CronJob
Job Object
- "A Job creates one or more Pods and will continue to retry execution of the Pods until a specified number of them successfully terminate". If the container is not successfully completed, it will recreated again.
- "When a specified number of successful completions is reached, the task (ie, Job) is complete."
- After finishing a job, pods are not deleted. Logs in the pods can be viewed.
- Job is used for the task that runs once (e.g. maintanence scripts, scripts that are used for creating DB)
- Job is also used for processing tasks that are stored in queue or bucket.
spec:
parallelism: 2 # each step how many pods start in parallel at a time
completions: 10 # number of pods that run and complete job at the end of the time
backoffLimit: 5 # to tolerate fail number of job, after 5 times of failure, not try to continue job, fail the job
activeDeadlineSeconds: 100 # if this job is not completed in 100 seconds, fail the job
Cron Job Object
- Cron job is a scheduled job that can be started in scheduled time.
# ┌───────────── minute (0 - 59)
# │ ┌───────────── hour (0 - 23)
# │ │ ┌───────────── day of the month (1 - 31)
# │ │ │ ┌───────────── month (1 - 12)
# │ │ │ │ ┌───────────── day of the week (0 - 6) (Sunday to Saturday;
# │ │ │ │ │ 7 is also Sunday on some systems)
# │ │ │ │ │
# │ │ │ │ │
# * * * * *
#
# https://crontab.guru/
# Examples:
# 5 * * * * : (means) For every day start at minute 5: 00:05 - Second day 00:05 ....
# */5 * * * * : (means) At every 5th minute: 00:05 - 00:10 - 00:15 ...
# 0 */2 * * * : (means) At minute 0 pass every 2d hour: 00:00 - 02:00 - 04:00 ...
# "*" means "every"
# "/" means "repetitive"
spec:
schedule: "*/1 * * * *" # At every 1st minute: 00:01 - 00:02 ...
jobTemplate:
spec:
template:
spec:
containers:
- name: hello
image: busybox
imagePullPolicy: IfNotPresent
command: # start shell and echo
- /bin/sh
- -c
- date; echo Hello from the Kubernetes cluster
restartPolicy: OnFailure
Authentication, Role Based Access Control, Service Account
Authentication
- It is related to authenticate user to use specific cluster.
- Theory of the creating authentication is explained in short:
- user creates .key (key file) and .csr (certificate signing request file includes username and roles) with openssl application
- user sends .csr file to the K8s admin
- K8s admin creates a K8s object with this .csr file and creates .crt file (certification file) to give user
- user gets this .crt file (certification file) and creates credential (set-credentials) in user's pc with certification.
- user creates context (set-context) with cluster and credential, and uses this context.
- now it requires to get/create authorization for the user.
Role Based Access Control (RBAC, Authorization)
- It provides to give authorization (role) to the specific user.
-
Role,RoleBindingK8s objects are used to bind users for specific "namespace". -
ClusterRole,ClusterRoleBindingK8s objects are used to bind users for specific namespace.
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: default
name: pod-reader
rules:
- apiGroups: [""] # "" indicates the core API group
resources: ["pods"] # "services", "endpoints", "pods", "pods/log" etc.
verbs: ["get", "watch", "list"] # "get", "list", "watch", "create", "update", "patch", "delete"
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: read-pods
namespace: default
subjects:
- kind: User
name: username@hostname.net # "name" is case sensitive, this name was defined while creating .csr file
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role #this must be Role or ClusterRole
name: pod-reader # this must match the name of the Role or ClusterRole you wish to bind to
apiGroup: rbac.authorization.k8s.io
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: secret-reader
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "watch", "list"]
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: read-secrets-global
subjects:
- kind: Group
name: DevTeam # Name is case sensitive
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: secret-reader
apiGroup: rbac.authorization.k8s.io
Service Account
- RBACs are used for real people.
- Service accounts are used for pods/apps that can connect K8s API to create K8s objects.
Ingress
- "An API object that manages external access to the services in a cluster, typically HTTP." (ref: Kubernetes.io)
- "Ingress may provide load balancing, SSL termination and name-based virtual hosting" (ref: Kubernetes.io)
- Ingress is not a Service type, but it acts as the entry point for your cluster.
- Ingress resource only supports rules for directing HTTP(S) (L7) traffic.
- "Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the cluster. Traffic routing is controlled by rules defined on the Ingress resource." (ref: Kubernetes.io)
- Ingress controller is a L7 Application Loadbalancer that works in K8s according to K8s specification.
- Ingress Controllers: Nginx, HAproxy, Traefik
# Simple Ingress Object Definition
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: minimal-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
ingressClassName: nginx-example
rules:
- http:
paths:
- path: /testpath
pathType: Prefix
backend:
service:
name: test
port:
number: 80
Conclusion
With this post, we concluded the K8s tutorial. Later, we'll focus on how to apply these terms/objects on K8s cluster/minikube.
If you found the tutorial interesting, I’d love to hear your thoughts in the blog post comments. Feel free to share your reactions or leave a comment. I truly value your input and engagement 😉
For other posts 👉 https://dev.to/omerberatsezer 🧐
Follow for Tips, Tutorials, Hands-On Labs for AWS, Kubernetes, Docker, Linux, DevOps, Ansible, Machine Learning, Generative AI, SAAS.
Top comments (0)