DevOps for your Organization – Part 6: Automated Unit Tests

In our previous post, we covered what branch policies are, how to create them, and how they can improve your team’s development experience.

The next step in your DevOps process is to add automated unit tests to your continuous integration builds. By including automated unit tests, you guarantee that all new code checked in to your master branch via pull requests will pass all your tests. It is one more quality gate to maintain the integrity of your code.

Azure DevOps has native support for a variety of testing frameworks in both their build and release processes. Additionally, they support third-party plugins for the frameworks that they do not support. As of this writing (November 2019), they support:

  • VSTest
  • Selenium
  • Appium
  • Coded UI
  • MSTest
  • xUnit
  • NUnit
  • Chutzpah

As well as many more. See Microsoft’s official documentation for more information about the capabilities of the tool.

Adding Unit Tests to your Build

In order to add unit tests to your build or release, you need only add the Visual Studio Test task to your build. If you’ve got only one solution in your and your test project has the word test in it, the default configuration should do:

# Run tests
- task: VSTest@2
  displayName: Run Tests
  inputs:
    testSelector: 'testAssemblies'
    testAssemblyVer2: |
      **\*test*.dll
      !**\*TestAdapter.dll
      !**\obj\**
    searchFolder: '$(System.DefaultWorkingDirectory)'

Otherwise, you can use the task’s configuration pane to help you fill out the YAML. The information bubbles are a significant help.

Automated unit tests are a great way to ensure that code changes coming in to your repository do not break your current functionality. They can be a quick and easy sanity check that code changes in one segment of the solution haven’t had unintended downstream effects. In a great article by Munil Shah, former director of engineering, developer services, cloud and AI division at Microsoft, he covers Azure DevOps transition from traditional development practices to DevOps. In it, he specifically address how they transformed their unit test practice.

Pull Request to Merge takes about 30 minutes. We run about 60,000 unit tests in the Pull Request. From code merge to CI build is about 22 minutes. The first quality signal from CI (SelfTest) is about an hour. By that time most of the product is tested with that change. Within 2 hours (Merge to SelfHost), the entire product is tested and the change is ready to go into production.

Munil Shah, Former Director of Engineering, Developer Services, Cloud and AI Division at Microsoft

Let Us Help You Out!

Code Vanguard has worked with clients of all sizes, helping them to use DevOps practices to accelerate their time to delivery. If you’d like Code Vanguard to help your organization with your DevOps process, feel free to reach out to us!


This post is part of the DevOps for your Organization series, where we walk through some basic steps you can take to move your organization towards the DevOps process. The full series can be found here. If you’d like Code Vanguard to help your organization with its DevOps process, feel free to reach out to us!

James Stephens About the author

James is the founder of Code Vanguard and one of its developers. He is an applied mathematician turned computer programming. His focuses are on security, DevOps, automation, and Microsoft Azure.

No Comments

Post a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.