Overview
AWS CloudWatch has a built-in event and trigger system which allows for regular cleanup and cost control tasks.
The problem
In a busy development house, a significant amount of time is often spent on cleaning up resources. AWS can be cost effective… unless you leave behind assets that were created for an ad-hoc purpose or experimentation. AWS has several cost control and reporting tools, but we want to go further than that, and eliminate the costs automatically.
Elastic IP Addresses
We have several cost control processes running, but for this blog I will focus only on Elastic IPs.
EIPs are super handy, but pesky. Once you have allocated an EIP to your account, that EIP must be associated with a resource otherwise there is an hourly cost while you hold it.
In our specific case, we build out training stacks of our Infrastructure automation tool MintPress. Each use of this tool might result in 60-100 servers and other assets being built – some of which need EIP addresses. As the tools are automated for build, we do also automate the tear-down. Even so – EIPs are sometimes added during training, or for other purposes, and sometimes are forgotten.
The Solution
We had to set up two things in AWS to automate this.
- A functional script that can release unassociated EIPs.
- A means to trigger that script on a regular basis like a cron tab.
AWS provides serverless code execution with its Lambda service. Lambda functions allow you to script in a language you are comfortable with and provides enormous flexibility in how you can run your function. Lambda is integrated with all the aws services via the importable sdk which is available to all the language types.
In the Lambda service I created a function “CleanupEIPAddresses”, I chose Ruby 2.7.
The code is quite simple so I will point out only the important parts.
In line 2 we require the aws-sdk-ec2 library, which has all the tools we need to interact with the EC2 service.
In line 6 we create a client, and then in line 7 we ask for the Elastic IPs for that region.
For those EIP objects we check to see if ‘instance_id’ is set, if it is not set, that EIP can be released in line 13.
Simple. Anytime I want to clean up EIP addresses I can login to the console and run this Lambda, or call invoke from the aws-cli.
Now that we have the functional script we need to automate how and when it is called.
AWS CloudWatch is where you go for metrics and finding logs and other ops data.. CloudWatch also has tools for raising alerts such as breaching thresholds or reporting on up-time. CloudWatch also has an EVENTS system. This tool can be used to trigger activities in other parts of the platform in response inbound events. The EVENT tool also has simple time based scheduler.
To set this up I added a rule called CleanupEIPaddresses.
We can choose between a cron expression or a simple time setting. In the targets panel I have chosen the Lambda I made earlier.
That’s it. Every 6 hours, CloudWatch triggers the Cleanup Lambda, and any unassociated EIPs are released.
Conclusion
Elastic IP addresses aren’t the most expensive things to forget about in AWS, but using these simple tools built into the platform you could expand its use to terminating long running instances, auditing snapshots and other unused storage. Another common setup would be to terminate and remove things based on tags.
Using AWS Lambda and CloudWatch events together is an elegant and simple way to manage costs and life cycles in your AWS account.
