VSTest (Visual Studio)
  • 14 May 2021
  • 1 Minute to read
  • Dark
    Light
  • PDF

VSTest (Visual Studio)

  • Dark
    Light
  • PDF

Running NUnit, xUnit, MSTest, and more with VSTest Runner

Visual Studio Testing Tools is the development framework centered around unit testing, code coverage, and load testing. Developers are able to run tests directly in Visual Studio using the Test Explorer window, and their unit testing framework of choice: NUnit, xUnit, MSTest, and more.

Within a CI build pipeline, tests can be executed with VSTest.Console.exe, allowing organizations to run unit and functional tests (Selenium, Appium, Coded UI, etc.) as early as possible in the CI/CD process.

Running VSTest with BuildMaster

The VSTest Runner in BuildMaster essentially runs the VSTest.Console.exe command line tool against a unit test container like MSTest-based tests, and any test frameworks that have a Visual Studio test adapter, such as xUnit, NUnit, Chutzpah. These adapters are referenced using NuGet packages, which must be restored prior to running MSBuild.

An example build plan that gets source and runs tests is as follows:

GitHub::Get-Source
(
    Organization: Inedo,
    Repository: ProfitCalc
);
 
NuGet::Restore-Packages();

MSBuild::Build-Project ProfitCalc.sln
(  
    To: ~\Output
);

WindowsSdk::Execute-VSTest
(
    TestContainer: ~\Output\ProfitCalc.Tests.dll
);

Test Result Behavior

Test results are logged in the build execution log, and also to the Unit Test section of a build, with a more specific breakdown and/or test history. If a unit test fails, the default behavior is to halt the build. If the desired behavior is the continue regardless of failure, wrap the operation in an OtterScript try/catch statement, for example:

try
{
    WindowsSdk::Execute-VSTest
    (
        TestContainer: ~\Output\ProfitCalc.Tests.dll
    );
}
catch
{
    Log-Warning Ignoring test failures for now...;
}

Preventing Deployment of Builds with Failed Unit Tests

To ensure that a build with failed unit tests is never deployed, add a "Unit Tests Passed" automated check to the pipeline stage immediately following the build stage (this is typically integration). This will prevent a build with failed unit tests (and optionally inconclusive tests) from being promoted regardless if errors were logged or ignored in the build plan. Of course, this build can still be forced to the next stage, but special administrative permissions are required in order to do so.


Was this article helpful?

What's Next