9 min readJun 2, 2024

Migrating the Github project on Azure Devops (Azure Repo & Pipelines)

In this project we will be deploying the microservices application which is written in different programming languages, For CI part we will use pipelines in azure pipelines. This voting app is written in python and we have a memory data store called “Redis” and we have a worker which is written in .Net framework which takes the information from the redis and save in the database and finally there is another microservice called result which read the information from database and display it for you. To perform this task you should have a good understanding of CI and CD.

Below is the source code of the application:
https://github.com/dockersamples/example-voting-app

First we will clone the project locally and run the project on our local system through docker compose:

Once app is deployed in containers just open it on browser on port 5000 for voting service and port 5001 for result service as mentioned in docker compose file.

We will deploy each CI pipeline for each microservice like one for vote, result and one for worker. If you go through the each microservice folder you will see the separate docker file for each service.
Lets create new project on azure devops:

Although azure pipeline can also be use with github but we will fully do on azure devops so first go to repo section and import git hub repo there:

Select the main branch for azure pipeline and set it as default branch:

Now we have to create azure pipelines for our microservices, we will start with result services with template name “build and push docker image to azure container registry”:
For thai you have to create a container registry so create that first on azure portal:

Now select this registry when creating the pipeline:

When it create the yaml template for us we will modify some changes in it , like for trigger we will use path base triggering like result’s pipeline only trigger when there is commit in result folder only and same for other pipelines also:

we will also update the agent details in yaml file because on free account azure does not provide you agent to run pipelines, so we will create our vm on azure portal and use that VM to run our pipelines:

We will create build and push stages separate so remove pool from here and below step click on setting and there it will give you options select there only build and give docker file path it will create the build step for you:

copy this stage add new one for push:

Full pipeline:
trigger:

paths:

include:

- result/*

resources:

- repo: self

variables:

# Container registry service connection established during pipeline creation

dockerRegistryServiceConnection: ‘7ca84e89–4b16–4eb1-bb52–62c5c450da13’

imageRepository: ‘resultapp’

containerRegistry: ‘microserviceregistryci.azurecr.io’

dockerfilePath: ‘$(Build.SourcesDirectory)/result/Dockerfile’

tag: ‘$(Build.BuildId)’

pool:

name: ‘azureagent’ # Ensure this pool name is correct and matches the one where your self-hosted agent is registered

stages:

- stage: Build

displayName: Build

jobs:

- job: Build

displayName: Build

steps:

- task: Docker@2

displayName: Build an image

inputs:

containerRegistry: ‘$(dockerRegistryServiceConnection)’

repository: ‘$(imageRepository)’

command: ‘build’

Dockerfile: ‘result/Dockerfile’

tags: ‘$(tag)’

- stage: Push

displayName: Push

jobs:

- job: Push

displayName: Push

steps:

- task: Docker@2

displayName: Push an image to container registry

inputs:

containerRegistry: ‘$(dockerRegistryServiceConnection)’

repository: ‘$(imageRepository)’

command: ‘push’

tags: ‘$(tag)’

Now create the azurevm then hit save and run:
VM:

ssh the VM and configure it as agent for our azure devops pipeline, for this below is the document:
https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/linux-agent?view=azure-devops

Go in azure devops→ project setting → agent pool → add pool → self host pool and create new pool:

Inside agent pool create new agent and follow the steps:

Note: During this process it will ask for presonal access token you can generate it from azure devops→ user setting.

once these all command run, install the docker.io and assign permission to docker user:
⇒ sudo usermod -aG docker azureuser
⇒ sudo systemctl restart docker

Now check your agent should be in online state:

Now run the pipeline:

We have done one pipeline for microservice called result now we will do the same for other two pipelines for worker and vote:

Our all three pipelines are created and ran:

Check the docker registry there will be three docker images for each service:

So, we have completely migrated the github to azure devops pipelines ad our CI part of this project is completed.

Now we will start the CD part of this project, For continuous delivery we will use GitOps approach.
We will use thre stage in pipeline with name update basically its a shell script which clones the repo and update the manifest versions of our K8 yaml files then commit it:

and then we will run argoCD it will detect changes in yaml files and fetch the changes and deploy on K8s cluster.

Let start the configuration for this CD part:

First we will create kubernetes service on azure kubernetes services, we will also need agent pool in AKS because to run pod or services in kubernetes AKS required compute power so that agent pool provides that in agent pool we can also select scale method and minimum maximum nodes we needed etc,
Note:
Also expose the public ip for node, and choose all default configurations.

Cluster creation is in progress:

Now we will install argocd in K8s cluster. First login to the cluster:

Setting up argocd:
https://argo-cd.readthedocs.io/en/stable/getting_started/

argocd will create its pods:

Now we will configure argocd, first we will connect argocd with our git repo so it will its single source of truth and argocd will do continuous monitoring to this git repo whenever any change in the any K8s manifest it will deploy those changes on K8s cluster, and if someone will try to make any change directly in the cluster argocd will revert it on the basis on this git single source of truth.

First get the argocd credentials to login in it:

secrets are base64 encoded so need to decode the secret first:
After decoding the password we can login on argocd UI, for this we have to expose argocd server in nodeport mode:

Now access the argocd:

But first open the node port in security group:

We have access argocd now:

Now we will connect argocd with our azure repository, we will create personal access token on repo and use that in argocd:

Now copy the https clone url of repo and put in the argocd but replace the first part of link with the token:

Now create new app on argocd:

It have started deploying app on kubernetes cluster:

Now we will write the update script which will link our CI and CD, mean when CI runs in last stage we will run the update script which will modify the image tag of my manifests, so when CI pipelines runs this update script also update the image tag or number, so argos will fetch that changes and re deploy it on kubernetes cluster.

update the upper shared stage in all three microservices pipelines.

Now put the update script in new script folder at azure repo:

Below is the script:
#!/bin/bash

set -x

# Set the repository URL

REPO_URL=”https://<ACCESS-TOKEN>@dev.azure.com/<AZURE-DEVOPS-ORG-NAME>/voting-app/_git/voting-app”

# Clone the git repository into the /tmp directory

git clone “$REPO_URL” /tmp/temp_repo

# Navigate into the cloned repository directory

cd /tmp/temp_repo

# Make changes to the Kubernetes manifest file(s)

# For example, let’s say you want to change the image tag in a deployment.yaml file

sed -i “s|image:.*|image: <ACR-REGISTRY-NAME>/$2:$3|g” k8s-specifications/$1-deployment.yaml

# Add the modified files

git add .

# Commit the changes

git commit -m “Update Kubernetes manifest”

# Push the changes back to the repository

git push

# Cleanup: remove the temporary directory

rm -rf /tmp/temp_repo

Now run th each pipeline they will update the manifest file with image tag so that argocd will fetch the changes and deploy on cluster.

You will notice that argocd will deploy the changes but in the pod there will be an error of pulling image because we are now pulling the image from private container registry to resolve this we will use concept “ImagePullSecrets” to pull image from private registry.

enable the the admin user in your registry and copy the access keys from there:

Then at cluster:

Once secret created then come to the manifests file and update it also:

Thats all in this project!
— — — — — — — — — — — — — — — — — — -

Migrating the Github project on Azure Devops (Azure Repo & Pipelines)

Below is the source code of the application:https://github.com/dockersamples/example-voting-app

First we will clone the project locally and run the project on our local system through docker compose:

Once app is deployed in containers just open it on browser on port 5000 for voting service and port 5001 for result service as mentioned in docker compose file.

We will deploy each CI pipeline for each microservice like one for vote, result and one for worker. If you go through the each microservice folder you will see the separate docker file for each service.Lets create new project on azure devops:

Although azure pipeline can also be use with github but we will fully do on azure devops so first go to repo section and import git hub repo there:

Select the main branch for azure pipeline and set it as default branch:

Now we have to create azure pipelines for our microservices, we will start with result services with template name “build and push docker image to azure container registry”:For thai you have to create a container registry so create that first on azure portal:

Now select this registry when creating the pipeline:

When it create the yaml template for us we will modify some changes in it , like for trigger we will use path base triggering like result’s pipeline only trigger when there is commit in result folder only and same for other pipelines also:

we will also update the agent details in yaml file because on free account azure does not provide you agent to run pipelines, so we will create our vm on azure portal and use that VM to run our pipelines:

We will create build and push stages separate so remove pool from here and below step click on setting and there it will give you options select there only build and give docker file path it will create the build step for you:

copy this stage add new one for push:

Full pipeline:trigger:

paths:

include:

- result/*

resources:

- repo: self

variables:

# Container registry service connection established during pipeline creation

dockerRegistryServiceConnection: ‘7ca84e89–4b16–4eb1-bb52–62c5c450da13’

imageRepository: ‘resultapp’

containerRegistry: ‘microserviceregistryci.azurecr.io’

dockerfilePath: ‘$(Build.SourcesDirectory)/result/Dockerfile’

tag: ‘$(Build.BuildId)’

pool:

name: ‘azureagent’ # Ensure this pool name is correct and matches the one where your self-hosted agent is registered

stages:

- stage: Build

displayName: Build

jobs:

- job: Build

displayName: Build

steps:

- task: Docker@2

displayName: Build an image

inputs:

containerRegistry: ‘$(dockerRegistryServiceConnection)’

repository: ‘$(imageRepository)’

command: ‘build’

Dockerfile: ‘result/Dockerfile’

tags: ‘$(tag)’

- stage: Push

displayName: Push

jobs:

- job: Push

displayName: Push

steps:

- task: Docker@2

displayName: Push an image to container registry

inputs:

containerRegistry: ‘$(dockerRegistryServiceConnection)’

repository: ‘$(imageRepository)’

command: ‘push’

tags: ‘$(tag)’

Now create the azurevm then hit save and run:VM:

ssh the VM and configure it as agent for our azure devops pipeline, for this below is the document:https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/linux-agent?view=azure-devops

Go in azure devops→ project setting → agent pool → add pool → self host pool and create new pool:

Inside agent pool create new agent and follow the steps:

Note: During this process it will ask for presonal access token you can generate it from azure devops→ user setting.

once these all command run, install the docker.io and assign permission to docker user:⇒ sudo usermod -aG docker azureuser⇒ sudo systemctl restart docker

Now check your agent should be in online state:

Now run the pipeline:

We have done one pipeline for microservice called result now we will do the same for other two pipelines for worker and vote:

Our all three pipelines are created and ran:

Check the docker registry there will be three docker images for each service:

So, we have completely migrated the github to azure devops pipelines ad our CI part of this project is completed.

Now we will start the CD part of this project, For continuous delivery we will use GitOps approach. We will use thre stage in pipeline with name update basically its a shell script which clones the repo and update the manifest versions of our K8 yaml files then commit it:

and then we will run argoCD it will detect changes in yaml files and fetch the changes and deploy on K8s cluster.

Let start the configuration for this CD part:

Cluster creation is in progress:

Now we will install argocd in K8s cluster. First login to the cluster:

Setting up argocd:https://argo-cd.readthedocs.io/en/stable/getting_started/

argocd will create its pods:

Written by Ghazanfar Ali

Below is the source code of the application:
https://github.com/dockersamples/example-voting-app

We will deploy each CI pipeline for each microservice like one for vote, result and one for worker. If you go through the each microservice folder you will see the separate docker file for each service.
Lets create new project on azure devops:

Now we have to create azure pipelines for our microservices, we will start with result services with template name “build and push docker image to azure container registry”:
For thai you have to create a container registry so create that first on azure portal:

Full pipeline:
trigger:

Now create the azurevm then hit save and run:
VM:

ssh the VM and configure it as agent for our azure devops pipeline, for this below is the document:
https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/linux-agent?view=azure-devops

once these all command run, install the docker.io and assign permission to docker user:
⇒ sudo usermod -aG docker azureuser
⇒ sudo systemctl restart docker

Now we will start the CD part of this project, For continuous delivery we will use GitOps approach.
We will use thre stage in pipeline with name update basically its a shell script which clones the repo and update the manifest versions of our K8 yaml files then commit it:

Setting up argocd:
https://argo-cd.readthedocs.io/en/stable/getting_started/