Kubernetes End To End Project:

9 min readAug 9, 2023

Introduction:

In this detailed project we will explore kubernetes cluster installation using AWS Elastic Kubernetes Service and application deployment on it, we will also monitor the application’s health through famous tools like kiali and jaeger.

Microservices Architecture:

Lets start!

First we will create AWS EC2 AMI linux t2.medium instance:

Note: Use region us.west-1

Now take the ssh of instance:

Install kubectl on that:

Note:

kubectl is needed to interact and manage Kubernetes clusters, allowing users to deploy, monitor, and scale applications in a containerized environment.

⇒ Curl -o kubectl https://amazon-eks.s3-us-west-2.amazonaws.com/1.14.6/2019-08-22/bin/linux/amd64/kubectl

⇒ chmod +x ./kubectl

⇒ mkdir -p $HOME/bin

⇒ cp ./kubectl $HOME/bin/kubectl

⇒ export PATH=$HOME/bin:$PATH

⇒ echo ‘export PATH=$HOME/bin:$PATH’ >> ~/.bashrc

⇒ source $HOME/.bashrc

⇒ kubectl version — short –client

Verify kubectl :

Now we will install eksctl with this we will create our kubernetes cluster:

⇒ curl — silent — location “https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz” | tar xz -C /tmp

⇒ sudo mv /tmp/eksctl /usr/bin

Verify version after installation:

Now add IAM role to EC2 instance so that EC2 will access to EKS.

First stop the instance then modify the IAM role:

Create new IAM role:

Select EC2 as this role is for EC2:

Select all EKS policies for this role:

Now attach this role to EC2:

Now we will create cluster through eksctl, but before that create user in IAM then configure its access and secret keys at ec2 instance so that ec2 instance can create eks cluster through that user:

User:

Download its csv file from user’s security credentials section that contains access and secret key:

Then install aws cli at ec2 instance:

Then type aws configure to put keys details and region (us-west-1) there:

Now create eks cluster through eksctl command:

When you hit this command it will create the cloud formation stack at AWS, it is resource management service in AWS same like terraform.

We can also check the events details and diagnose issue through events:

Now check the EKS in AWS there will be cluster:

Now we will configure OIDC(open ID connect) it is a special approval which we give to the kubernetes so kubernetes access the other resources.

Now we will create node group which will be a collection of worker nodes that shares the common properties and configurations.

Note: replace the cluster and ssh public keys name to yours.

This will create two worker nodes that are EC2 instances in our cluster.

Now we will install istio, Istio is a tool that helps manage and secure communication between different parts of applications running in a Kubernetes environment.

Now set the ISTIO path:

Next step is to install istio with demo profile, It’s like using a template to quickly apply common settings to your services.

Once we install istio we will have istio namespace:

Note:

A namespace is like a virtual partition that helps organize and isolate different applications or resources within a cluster. It’s like having separate folders on a computer to keep your files organized and prevent them from interfering with each other.

It will also have pods in it:

Now using the kubectl apply command to apply Kubernetes configuration files for the Istio Bookinfo sample application. This command is deploying the Bookinfo application components onto your Kubernetes cluster. The URL links provided in the command are pointing to the configuration files for the Bookinfo application.

Now you will see some pods in default namespace:

As of now we are seeing there is one container in each pod (1/1) in Ready option.

Once we will enable istio we will see one pod with 2 containers as istio will add new container called proxy container with every pod as mention in below istio structure:

In Istio, when you enable it for a specific service or application, it adds a new container called a “sidecar proxy” alongside the main container of your application. This sidecar proxy acts as a dedicated intermediary for handling network communication, traffic routing, security, and observability tasks for that specific application.

If we talk the architecture we will have a istio → service running in front of pods→ then pod → and each pod will has 2 containers one is sidecar and one is for real application:

Now we will check our product page is running or not with below command:

Kubectl exec is use to go inside the container.

Till now istio in not enabled in namespace, Now we will hit below command to inject istio as an init container, now 2 pods will run. Actually we are enabling the istio in default namespace:

After this command we inject the istio but our application pods are not taking it fo first we will delete the pods,

Once we delete the pods K8s will re create them to keep our application up but this time each pod will have 2 container as we discussed earlier:

Now analyze the istio to check if there is any issue:

Now go to networking directory in istio and install other things:

We will create new pod here also:

The command kubectl apply -f bookinfo-gateway.yaml is used to apply a Kubernetes configuration stored in the file named “bookinfo-gateway.yaml” to your cluster. In the context of Istio, this likely refers to a configuration for an Istio Gateway, which is used to manage external traffic entering the Istio service mesh.

See the ingress gateway:

Now set the ingress ip and port:

Check the port:

Now check your load balancer in AWS EC2 section:

Copy the dns name of load balancer:

And paste in below command that will set your load balancer as ingress gateway:

Then:

The provided commands set an environment variable named GATEWAY_URL using the previously defined INGRESS_HOST variable and an additional variable named INGRESS_PORT. Then, it prints the value of the GATEWAY_URL variable.

Now hit your load balancer url on browser along with /productpage:

Now we will see how to deploy kiali dashboard:

Kiali is like a map for Istio. It helps you see how your different services talk to each other, spot problems, and understand the flow of data in your applications. It’s like having a picture of your traffic patterns to keep things running smoothly.

Basically in this directory ‘addons’ we are installing all tools.

Note:

The command kubectl apply -f . is used to apply Kubernetes configurations that are present in the current directory (.). It tells Kubernetes to process and apply all the YAML or JSON files in the current directory to your cluster. This is useful when you have multiple configuration files that you want to apply in one go.

Now you will see multiple pods of different services like grafana, prometheus, kiali, jaegers etc:

Now we will do port forward for kiali:

⇒ kubectl port-forward — address 0.0.0.0 svc/kiali 9008:20001 -n istio-system

Allow the port 9008 in security group of EC2 instance then hit its public ip on browser with port 9008:

Here is the kiali dashboard.

Now we will port forward for jaeger, remember to allow its port in security group of EC2 instance also:

Note:

Jaeger in Istio is like a detective for your applications. It helps you track and understand how requests move through your services. Think of it as a tool that lets you follow the path of requests to find out where things might be going wrong or slowing down in your application.