Overview
VPC management is an important part of running an AWS account. In many organisations the default VPC for all systems and even for production systems and dev and integration systems side by side. Initially there is less to worry about and less things to configure, but with this arrangement security problems will creep in and new projects might accidentally expose systems to unnecessary or even dangerous access.
The best practice is to create a separate VPC for each system and for each production or development system. When creating each VPC we have the opportunity to lock it down and only expose necessary ports, and limit internet access for running machines. Each developer in your team should set up a personal VPC for their development exercises and to reinforce their understanding of basic application security.
For this example we will create a VPC with three subnets locked to my IP address. The VPC will allow webserver traffic on port 80 and 443 as well as SSH traffic on port 22. I will also set up a netcat service to listen on port 8082 to verify we can’t see it from the internet and that our assumptions are current when setting up the Network Access Control List (NACL) and our Security Groups (SGs).
Steps
In the VPC panel in the AWS I choose to create a new VPC.
The names you choose should describe who owns it and what it is for. This is a demo so I will name it “CIMAC-DEMO”. Names are important, a colleague working in the VPC screen is less likely to delete or alter a VPC named “PRODUCTION-OURWEBSHOP” than they are “shop”, or “test-shop”. The CIDR range is also important to get right. If the range is too small it cannot be altered later, so make sure it is large enough for any scaling you might need to do in the future. The CIDR range has to fit in 3 subnets which will need to be large enough for your instances. For the demo I will use /16 which is the largest available size.
Once we click the create VPC button the console does a few things for us:
- The VPC is created
- A corresponding NACL is created
- A corresponding Route Table is created
I will now take a minute to name them “CIMAC-DEMO” so that is less easy to pick the wrong route table or NACL when editing access.
The next step is to add some subnets. For highly available web systems there should be at least one subnet per availability zone. The region I am working in has four but I find that the “d” region is quite under resourced so I will make just three.
Each subnet in the VPC is a /24 sized network each having 256 IP addresses available. The VPC itself is /16 which can have as many as 65536 addresses. So there is plenty of room if I need to add additional subnets and at 256 I have plenty of addresses in each subnet for running a few machines.
To connect our VPC to the internet we need to create an “Internet Gateway” and associate it with the DEMO VPC.
Now the VPC can access the internet, but we need to specify a route for external traffic in the DEMO route table,
where 0.0.0.0/0 means “the internet”.
Now we need to configure some access for the VPC. We would like to:
- ssh to machines over port 22
- Send and receive HTTP traffic over port 80 and 443
- Dis-allow traffic on other ports such as 8082 – but any other port.
I will set up the NACL which governs the VPC, to allow traffic in both directions on those ports, and configure separate security groups for each of ssh and HTTP traffic.
Setting up multiple security groups is important as you can separate task specific access and apply each one to a machine or group of machines. We can also remove SGs from machines when that access is not needed. Imagine the example where you want to disallow ssh for security reasons, but add it back from the office IP address, or on a different day from your home IP address.
In the NACL we have to specify inbound and outbound traffic, so here I have opened those ports to the internet, and also configured them for outbound as well.
I do a similar thing in the SGs.
And for this I add a single rule for my IP address to keep things tight.
SG entries are stateful which means they automatically permit bi-directional traffic for each port you specify. Now let’s launch a machine and SSH to it.
In the launch configuration I need to:
- Choose the DEMO VPC
- Choose a subnet I just made
- Check the box that will assign a public IP to the machine
- Choose the DEMO-SSH SG that we opened port 22 for
- Create a new PEM key!
- Never use the same key for multiple systems
- Never use a production key for a test or a demo system
- Download the new key (and chmod 0400 the key)
You should also add a “Name” in the tag section, but you can name it in the interface anytime…
If everything has gone well we should be able to ssh from this IP address to the new machine.
… and there it is.
Now let’s set up a quick webserver and allow access for it:
sudo yum install -y epel-release && sudo yum update -y && sudo yum install nginx -y
Using curl I can see nginx is running:
<h1>Welcome to <strong>nginx</strong> on the Amazon Linux AMI!</h1>
At this point nginx is serving on this machine to port 80 as verified using curl but it should not be accessible yet. Remember we gave the VPC internet access and opened the ports in the NACL to allow it, but we did not add a SG for HTTP ports. Lets verify we cannot see nginx in a browser, then create a separate SG to give us access.
Open a new tab and put your machine IP into the address bar, it should timeout the request.
Now create a new security group in the VPC Security Group panel, and allow access for HTTP ports.
Here again, I am using my home IP address to allow access.
Having done so I now need to attach it to the machine which is done in the EC2 panel.
It works! Now I have two SGs I can use for giving myself access to systems using ssh or web ports provided I can edit the NACL.
We have already demonstrated that the SG can limit access to certain ports, now let’s verify that something like port 8082 remains closed. For this I will set up a listener on port 82 using netcat.
[ec2-user@ip-10-0-20-198 ~]$ nc -l 8082
So netcat is listening on that machine for TCP connections on port 8082. If I use nc on my laptop to try to connect – it times out. When I use netcat to connect to port 80, it gives me the status of “open”. So my netcat is working, but the port on the machine is closed to traffic. Success.
Were you listening?
Yes. nc was listening on that machine but it was only available via localhost.
Conclusion
VPC configuration can be confusing to navigate and consists of many interdependent parts. This post has shown you a clear path to breaking away from using the default VPC for all tasks, and to take better control of security for your production and test systems. As an exercise, each person in your development team should set up a personal VPC with minimal access. Perhaps also – admins should take a look at the state of their VPC configs and clear some technical debt.