Python Real Time Project For Devops Engineers:
Python Real Time Project For Devops Engineers:
As a devops/cloud engineering team we take care of the AWS environment and make sure it is in compliance with the organizational policies. We use AWS cloud watch in combination with AWS Lambda to govern the resources according to the policies.
For example we triggered a lambda function when an Amazon Elastic Block Storage volume is created we use Amazon Cloud Watch events. Cloud-watch event that allow us to monitor and respond to EBS volume that are of type of GP2 and convert them to type GP3, we will use lambda functions integrate with cloudwatch to restrict users on company policies. we will use python as a scripting language for lambda functions.
The scope of this project is not just limited to EBS but it can use for many services like we can restrict developers on various AWS services like EC2 instances, RDS ,S3, EKS as per company policies etc.
Lambda Basic Introduction:
In AWS (Amazon Web Services), Lambda is a serverless computing service that lets you run code without provisioning or managing servers. It enables you to execute code in response to specific events, such as changes to data in an Amazon S3 bucket, updates to a database, or HTTP requests through Amazon API Gateway.
Lambda allows you to write your code in a variety of programming languages, and it automatically scales to handle the incoming requests without any manual intervention.
Lets start the hands on!
First we will create the Lambda function on AWS:
Aws has created very basic and simple lambda function in json for us, this function simply return hello!.
Now go to cloud watch :
We will configure the rule at cloudwatch which will trigger the lambda function.
We are creating the rule for ebs volume creation so select the same:
This rule will trigger lambda function so select lambda function in target:
We have successfully created the cloudwatch rule:
Now we will verify that our cloudwatch is able to access the lambda function for that we create a dummy ebs volume in EC2 window:
By default it will create the EBS volume:
Now, as per our expectation cloud watch should triggered the lambda function as we have created new EBS volume.
lets check in log group of cloud watch:
Cloud watch has triggered the lambda function but it does nothing so we will write python code in lambda function that will detect it ebs volume is created in gp2 it will convert it to gp3:
We will edit the default lambda function, so first just add the print line that print event , basically event value is provided by cloudwatch as it invoke this function:
Now deploy and test it after that delete old ebs volume and create the new one to test this:
This type it will print the full event detail we will use these event detail and make the python function according to it:
Copy this json data log and put in any online json formatting tool to format it:
We only concern with name and id of volumein lambda function:
we will use python module boto3 to convert volume type from gp2 to gp3:
Note: Boto3 is a popular Python library used to interact with Amazon Web Services (AWS) APIs. It provides a convenient and easy-to-use interface for developers to work with various AWS services programmatically.
With Boto3, you can use Python to create, configure, manage, and interact with AWS resources like EC2 instances, S3 buckets, DynamoDB tables, SNS topics, and more.
Now we will write new function that will extract volume name and id from events details provided by cloudwatch to lambda when lambda_handler function trigger, we will get id and name of volume through arn provided in events:
Now replace below mentioned code with lambda function code:
Now we will create IAM role to allow our lambda function to modify ebs volume:
In roles section of IAM there will be a role we just need to attach new policy to this role:
We will create new inline policy:
Now delete old volume and create new volume again:
Once we create gp2 volume it sucessfully converted into gp3:
Thats all in this project!
