The Backdrop
Containers have grown in popularity over the last few years. ‘Build once and Run anywhere’ is the maxim, and it is an attractive one for all stakeholders involved.
Over and above the ‘build once, run anywhere’ moniker lies the fact that containers offer greater resource efficiency in comparison to virtual machines. Add to these benefits the open source nature of the technology and both developers and sysadmins become fans.
What are containers, and how do they affect software testing?
So, what are ‘containers’?
A container is a standard unit of software. It packages up the code along with all the dependencies of the code. This packaging enables the application to run quickly and reliably from one computing environment to another. Containers are lightweight, standalone and executable packages of software. They include everything needed to run an application: code, runtime, system tools, system libraries, settings etc.
Containers are highly portable software-defined environments that can host an application or a service, and as such, have an impact on software testing. Since containers are portable, they can be easily moved from one server to another.
Although containers are similar to virtual machines but they provide advantages over virtual machines. For instance:
- Containers can easily access bare-metal resources on the host server (in virtual machines this can be technically difficult to do)
- Containers don’t need a hypervisor to run. Fewer resources need to be devoted to hosting containers. This equates to more resources becoming available for your actual applications.
Containers have become the more popular, leaner, meaner and more efficient solution for deploying applications. Above all, Docker and its complementary tools are open sources, making them freely and readily available.
How are containers being used for testing?
Currently, the testing community is on a quest to make testing faster, smarter and more embedded in continuous delivery pipelines. While there was a period when testing projects could be elaborate and drawn out over days, often now, testers have only hours and minutes to test new software batches. In an extremely fast-paced development environment, a lack of time and limited resources eventuates into bottlenecks. Automation and parallelisation are being leveraged to solve this problem.
Technologies such as Docker allows teams to spin up different containers with a local laptop or cloud service, making concurrency at a level of “a hundred of parallel machines” possible. Furthermore, it transforms the way testing fits into the CI/CD pipeline. In this way, a multi-container strategy reduces and solves bottlenecks.
The advantages of leveraging containers for software testing
Containerisation makes software delivery and testing more predictable. This means, you can use the same environment – a container – to host your software whether you are building, testing or deploying software in production. Also, containerisation uses fewer resources than virtual machines making it the go-to solution.
1. Fever environment variables
Leveraging containers, you can package most of the application’s dependencies and variables in one container. External variables won’t affect the way the application runs. When you test, you can run tests based on the variables defined in the host container. Due to this, testing the application under such a wide range of environment configurations, is not needed.
2. What you test is what you get
Containerisation provides a consistent application environment for software testing and for deployment. You can be sure that the testing will accurately reflect how the application will run in production (with a couple of exceptions – see disadvantages below) because the test and production environments are the same. Although there can be a slight difference between a containerized testing- and deployment- environment, the differences are not as significant as they would be without containers.
3. Simpler test branches
Software testers often test multiple versions of an application. They might have to test Windows and Linux versions for example. These versions need to descend from the same codebase but are tested and deployed separately. If you use containers to verify the above, they can make it easier to maintain multiple test branches. Containers will allow you to set up an isolated environment for testing each version of your application. It is easier and cleaner than trying to maintain consistent test environments for multiple branches on a virtual machine or bare metal server. Changes to the server could create inconsistencies in your test environment if you do not use containers.
The disadvantages of containers for software testing
1. Containers are not hardware agnostic
There are also a few disadvantages of using containers to do software testing. Although you can create portable environments that abstract an application away from underlying host environments with containers, they are not immune to hardware variables.
For example, a containerized application can behave in different ways depending on the GPU that its host server contains and whether or not the application interacts with the GPU. Using a container for the above does not make software testing any harder than it would if one were testing a non-containerized application on bare metal. It can be more challenging to test software that is designed to deploy in a virtual machine where the underlying server hardware won’t affect the behaviour of the application. To ensure this doesn’t happen, QA teams must test containerized apps on all the different hardware profiles that one uses in production.
2. Version differences
The “build it once, run it anywhere” benefit also has a downside. Most container technologies come in different versions. This can have an impact on application behaviour and performance. In general, there is compatibility between the different versions, but there are subtle differences. A container that is designed to run on one version of a container technology might experience problems on a modified version because of feature deprecation or other issues. This means testers need to know which version of the container to use in production. They must then test those versions. In a perfect world, testers should stay abreast of planned application and deployment changes so that they can test new versions of containers in time.
3. Testing for micro-services
Containers are particularly suited for micro-service deployment. They make the implementation and support of micro-services much easier. Although micro-services have several benefits, they also have drawbacks. One of the drawbacks is that microservices make testing more complicated. Each micro-service needs to be tested separately, and then also as a whole. Also, to ensure they interact as expected, all micro-services must run to test the application. This makes testing more complex. To achieve this, you need to rely on automated tests to cover multiple micro-services. And test API’s in addition to basic application code.
In conclusion, containers offer advantage and disadvantage. They make software delivery faster, more predictable and much easier to manage. At the same time, they can make software testing more complex. You can certainly solve the problems they create, but they require QA teams to adjust processes and tools that one currently uses.
How can software testers embrace containers?
Software testers are often asked to test new builds to ensure consistent core functionality of the application is retained with each build. It’s a critical process for manual deployments to validate that all the significant artefacts are installed and properly configured.
When you use containers in continuous deployment, the installation and configuration of dependencies are automated. This means you no longer need to run the tests. If you use continuous integration tools like Jenkins, most DevOps pipelines will leverage automated build tests. If you are a manual tester and does not want to transition, you will need to add value in other ways.
So how do you succeed as a tester in a world of containers and containerisation? You need to become an expert in the business use cases related to the software. You can also focus on new testing techniques around user experience and agility, like exploratory testing, to increase the value of manual testing for DevOps organisations.
Automated testers often build complex, UI-driven tests that are brittle and take too long to complete. They also have the potential to breakdown frequently. Infrequent, manual deployments do not slow the pace down as things are progressing slowly. Automated tests allow you enough time to debug and fix flaky test results.
When you transition to continuous development, builds are typically more frequent and require quick and robust (not brittle) tests that run in parallel. Containers allow relatively easy creation of multiple machines to run tests on. You will need to develop tests that can run in parallel across these numerous environments. Various tests can transition from interacting with the UI to interfacing directly with APIs to allow for less brittle and much faster tests. It is crucial that automation testers develop processes that enable them to identify script defects quickly, quarantine the scripts and fix them to ensure that the build process is not interrupted by problems in the test code.
How do enterprises enable continuous deployment and delivery?
It takes significant time and complicated configuration to deliver new products and features for your business applications. When the deployments become complex and diverse, the challenges can escalate fast.
LimePoint Automation Engineers can assist you in automating your infrastructure provisioning, application deployment, software releases as well as your policy and Configuration Management techniques.
While one of the key points to success is finding the right technology, choosing a partner who can help you with the successful implementation of your selected solutions, is equally important. LimePoint provides sound expertise in advanced technology services and offers integrated products and solutions that drive your pace of deployment. We have proven experience in dealing with all-around DevOps services and provide a wide range of DevOps automation services that simplify automation in a process driven by DevOps.
If you are an enterprise that is looking for a capable technology partner that delivers continuously across multiple technology platforms and services, get in touch with LimePoint.
