QA and DevOps Together: Is It Happily Ever After?

May 3, 2016 Alice Robbins

DevOps and QA teams

As originally seen on Illinois Technology Association, this is a guest blog post by Narrative Science CTO Adam Kanouse.

At Narrative Science, we’re always seeking new ways to be more productive and increase the quality of our work. That being said, we’re not afraid to experiment with our team organization.

Our latest experiment is combining our QA and DevOps team under a single ownerThe combination looks good on paper, but it’s still early so we’ll report back in a few quarters and share how it’s going.

What do DevOps and QA have in common?

Quality as a key goal

For a QA team, it’s obvious that quality is the goal. Most commonly in the software world, QA is focused on testing application functionality from development through production. Similarly, DevOps best practices are important throughout the software development life-cycle, but the focus is non-functional testing and effective production support.

Another link between QA and DevOps in terms of quality is how you measure success for each team. For QA, the goal is to proactively catch all functional problems as early as possible in the SDLC. Success is measured by the number of defects that make it to production.

For DevOps, the same measurement holds true. DevOps processes are successful if proactive non-functional testing and solid monitoring and alerting enables engineers to fix incidents (defects) before a customer is impacted.


Automated functional testing has been around for 15+ years and most organizations implement automated testing to increase test coverage, reduce the time needed for regression cycles, and improve the consistency and quality of testing.

Organizations who use agile development must have test automation to keep up with the speed at which the code is changing and the need for shorter testing cycles. Development speed also applies pressure to DevOps. Releases need to move through environments faster. Product features need to be deployed to production faster. The ultimate goal is “continuous integration” which ultimately requires automation tools.

Release and environment management

In many organizations, product releases are managed by QA for early testing environments, and are passed on to operations or a dedicated release team for production deployments. In addition, the teams who “own” those stages also manage the environments that are supporting these levels of the release. This nuance can cause inconsistency in the environment configuration as well as communication issues during release hand-offs.

A better model is to have releases roll up to a single owner who can assign a QA lead to own the testing and release readiness process and a DevOps lead who will own code deployment and troubleshooting across all environments. The other benefit is that DevOps often owns the tools used to actually promote the code. Having those engineers directly responsible for the release encourages continuous improvement.

These major points of commonality have lead me to the conclusion that DevOps and QA belong together. Looking ahead, we’ll measure success by two metrics:

  • Release time - did our releases get faster?

  • Quality of releases - did our releases improve in quality? What is our reduction in defects and/or incidents detected by our customers in production? How many defects are being flagged in production?

Perhaps you too can consider such questions and set up a benchmark to measure your team's processes. We’ll be sure to follow up in a few months and share any key learnings from our organizational alignment.

Jobs at Narrative Science

Previous Article
It's Not Me, It's You: An Open Letter to My Financial Advisor
It's Not Me, It's You: An Open Letter to My Financial Advisor

Find out why Financial Advisory firms should focus on personalized communication and on-the-go access to re...

Next Article
5 Ways Creative Employees Can Drive Innovation
5 Ways Creative Employees Can Drive Innovation

Five simple ways, regardless of your job or employer, that you can harness creativity and innovation in you...


Get Narrative Science blog posts in your Inbox

Keep an eye out for your confirm email!
Error - something went wrong!