DevOps Workflow that Incorporates and Enhances Security

Although it may seem like they’re at odds, the DevOps principles that drive good outcomes for software development can also lead to more secure infrastructure. Read this article to see how you can achieve speed without sacrificing security.

DevOps Workflow that Incorporates and Enhances Security
DevOps Workflow that Incorporates and Enhances Security

The primary goal of most DevOps initiatives is to increase the speed and agility of software development and delivery. Unfortunately, this objective often seems to be at odds with good security practices, reducing the window of time for security assessment, and introducing new attack vectors.

But DevOps and security don’t have to be mutually exclusive. The principles that drive good outcomes for software development can also lead to a more secure infrastructure. Read the article for guidance on designing a DevOps workflow that incorporates and enhances security.

Content Summary

Introduction
The DevOps workflow: four key components
Version control
Automated testing
Continuous integration
Infrastructure as code
Make DevOps a security advantage

Introduction

While the objectives of a DevOps initiative may vary from company to company, there is at least one nearly universal goal: DevOps is a means of increasing speed and agility. By expanding automation and facilitating collaboration, companies can offload manual work and deliver software to their customers more quickly and frequently.

Sounds like a win-win, right? Unfortunately, as is the case with most organizational change, the benefits aren’t always universal. When it comes to DevOps, security teams often feel that they are on the losing end. The tools and processes that speed software delivery can introduce new attack vectors and reduce the window of time for security assessment. Not a win for the teams tasked with protecting infrastructure and sensitive data, while also ensuring compliance with internal and external security policies.

Through that lens, DevOps and security would seem to be at odds. But they do not need to be, and should not be, mutually exclusive. We found that 22 percent of firms at the highest level of security integration has reached an advanced state of DevOps evolution. The DevOps principles that drive good outcomes for software development — culture, automation, measurement, and sharing — are the same principles that drive good security outcomes.

It is possible to achieve speed without sacrificing security. It starts by rethinking how security fits into your delivery workflow.

The DevOps workflow: four key components

There is no single, standard definition of DevOps, but there are a few core components that are common to the majority of successful initiatives:

Four key DevOps components

  • Version control
  • Automated testing
  • Continuous integration
  • Infrastructure as code

Each of these practices carries implications for your organization’s security posture and adherence to compliance requirements. Understanding these risks can help your team take the necessary steps to mitigate them.

Version control

Adopting version control is often one of the first steps in establishing a DevOps practice, and for good reason. A version control system provides an authoritative source for application configurations, with the ability to track file history and changes. This offloads the burden of manual reconciliation, allowing development teams to produce deployable code faster and more frequently, and building the foundation for continuous integration.

Security implications

Version control systems are an appealing attack vector because they host code that will be propagated throughout your entire infrastructure. If that code is compromised, it can have disastrous, far-reaching implications.

As development gets more distributed, teams often put both application code and app configuration files in version control, which violates the best practice of storage separation and presents a security risk.

How to mitigate risk

Development and security teams should collaborate on threat models to identify assets that are most vulnerable to attack. This helps prioritize risk, so the threats that pose the greatest danger to your organization’s infrastructure can be addressed first.

Application code should be stored outside of your version control system, separate from configuration files. Institute strict role-based access controls (RBAC) to limit the number of people who can alter and commit code.

Automated testing

One of the goals of a DevOps initiative is the ability to make predictable and reliable changes. Deploying quickly is great, but if code isn’t reviewed and validated beforehand, you’re really just introducing bugs into your production environment at a faster rate, delivering poor user experience, and possibly exposing your organization to risk.

A standard, repeatable testing process lets you verify that everything works as it should before it’s deployed. Automated testing is faster and more reliable than manual validation, and frees up employees to focus on more innovative work. It also enables testing at earlier stages in development, when the effort and cost of fixing bugs are much lower.

Security implications

Removing manual intervention from the testing process results in faster, more stable deployments, but it also means having fewer human eyes on code throughout the development process, and fewer opportunities to assess risk in a way that a machine isn’t capable of.

Additionally, automated tests are only as good as the people behind them. If you don’t “tell” the computer to check something as part of the validation process, it won’t get checked. And machines cannot make a judgment call or inference.

How to mitigate risk

In theory, automated testing should help improve your security posture, because security checks can be built into the process. You can ensure, for instance, that the changes you’re rolling out do not violate compliance requirements; or automate tests for known weaknesses and configurations. To be sure these considerations are taken into account, your security team should be involved in designing the test pipeline.

Collaborating with security at this stage not only minimizes threats to your infrastructure; it highlights the fact that DevOps and strong security are not opposed. Working together, they can produce outcomes that benefit both teams and provide value to the business.

Continuous integration

Continuous integration (CI) combines and expands on version control and automated testing, automating the build and testing of code with each commit. When a developer commits code, it is merged into a central code base to ensure that the individual components of an application work together. A CI tool takes the pain out of collaboration, which can be especially difficult to manage among large, distributed development teams.

Because testing occurs each time a chunk of code is committed, rather than after an application is fully built, CI tools make it easier to isolate (and subsequently fix) errors. A CI pipeline streamlines each stage of development, ultimately enabling faster, more frequent deployments. It is the precursor to continuous delivery — often the goal of a DevOps initiative — in which all code changes are automatically deployed to a testing or staging environment.

Security implications

CI accelerates the pace of development, and at a certain point, it becomes all but impossible to perform manual security checks on each batch of code. If these checks are not incorporated into automated testing, applications are likely to be deployed with vulnerabilities.

A CI server provides a single point of entry to your infrastructure. If a bad actor can gain access to the appropriate permissions, they can alter code and commit changes to the master, fundamentally altering the application.

How to mitigate risk

The security measures that pertain to version control systems and automated testing apply here as well. Make your security team a close partner in test design, so that security checks are carried out consistently and automatically. It’s also essential to strictly control and audit access to your CI pipeline by adopting a policy of least-privilege access.

Infrastructure as code

Defining infrastructure as code allows you to automate the deployment and configuration of IT infrastructure using configuration files, essentially machine-readable instructions. Before the introduction of infrastructure as code (IaC), the configuration was done manually. It was slow, error-prone, and didn’t scale, making the operations team a serious roadblock to speed and innovation.

With IaC, the same configuration can quickly be applied to any number of machines. This repeatability enables an operational scale and makes the software delivery lifecycle drastically more efficient. IaC allows operations teams to function more like their development counterparts; without it, DevOps wouldn’t exist.

Security implications

The primary benefit of IaC is also its greatest risk: with the push of a button, changes can be deployed to vast swaths of your infrastructure. A malicious intruder can change the type of configuration applied to a server, turning a test server into a production environment, for instance.

The scale enabled by IaC also increases the attack surface, creating more potential access points and opportunities for exploitation.

How to mitigate risk

IaC allows your infrastructure to expand more rapidly, which in turn increases the potential attack surface. Collaborative threat-modeling between security, operations, and development teams helps maintain a defensive position. The goal is to think like a hacker — which assets would be the greatest targets, where are the weak points, and how might someone gain access to them? — and develop countermeasures.

Vulnerability scanners aid the security team in identifying threats, which are then prioritized and remediated by operations. Ops teams can benefit from tools that help prioritize vulnerabilities based on the level of risk and infrastructure context.

Despite these concerns, IaC can also be a boon to security. Security policy can be defined as code and consistently enforced across deployment environments using a configuration management tool. You can then generate reports to demonstrate compliance to auditors.

Make DevOps a security advantage

While partnering with security may slow you down initially, the upfront investment will almost certainly pay off down the road, preventing bugs and vulnerabilities that are slow and expensive to address. A study by IBM System Science Institute found that it costs 6 times more to fix a bug found during implementation than to fix one identified during design; 15 times more if it’s identified in testing; and 100 times more during regular maintenance once the code is in production.

In addition to collaborating through threat-modeling exercises, look for opportunities to employ automation as a security benefit, including:

  • Trigger vulnerability scans on deploy to test
  • Incorporate security checks, such as those for known weaknesses and configurations, into automated tests
  • Define security policy as code and ensure it is continuously enforced with a configuration management tool
  • Use your delivery pipeline to push OS patches to production

Accelerating software delivery has obvious (and potentially negative) implications for your security practice. But when dev and ops teams understand the concerns of the security organization and prioritize security at each stage of planning and design, DevOps and security are not mutually exclusive, but mutually beneficial.

Source: Puppet

Thomas Apel Published by Thomas Apel

, a dynamic and self-motivated information technology architect, with a thorough knowledge of all facets pertaining to system and network infrastructure design, implementation and administration. I enjoy the technical writing process and answering readers' comments included.