Unit testing JavaScript as part of TFS Build

by Matt Perdeck 11. July 2012 00:45

Contents

Introduction

Visual Studio makes it very easy to unit test your C# or Visual Basic code. Even better, TFS can run your unit tests as part of a build, providing a bit more confidence in the quality of your code.

However, these days, a large proportion of a site's functionality is often not coded in C# or Visual Basic, but in JavaScript. Unfortunately, there is no obvious support for unit testing JavaScript in Visual Studio as of July 2012.

This article shows how to integrate JavaScript unit testing in your ASP.NET MVC solution. The objective here is to have the JavaScript unit tests execute whenever you run your unit tests - whether from the Test View window in Visual Studio or as part of a TFS build.

Working sample code

To provide a working example of all this, the download contains:

  • A very simple ASP.NET MVC site without JavaScript testing
  • The same site, with JavaScript testing added

You may have to unblock the zip file before unzipping it, so Visual Studio will properly run the solution that's inside - right click zip file | Properties | Unblock.

Comparing QUnit versus Jasmine

There are many JavaScript unit testing frameworks out there (overview). The two most popular ones at the time of writing however are QUnit and Jasmine.

  QUnit 1.8.0 Jasmine 1.2.0
Runs in web page web page
Written in JavaScript JavaScript
Style Similar to xUnit
test("increment a variable", function() {
  var a = 0;
  a++; 

  equal(a, 1, "should be 1");
});
Behavior Driven Development (BDD)
it('should increment a variable', 
   function () {
     var a = 0;
     a++; 

     expect(a).toEqual(1); 
});
Grouping of tests
module("core");
test("increment a variable", function() {
   ...
});

test("decrement a variable", function() {
   ...
});
describe('Calculator', 
  function () {
     it('can increment', 
        function () {
           ...
        });

     it('can decrement', 
        function () {
           ...
        });
});
Groups can contain groups No Yes
ReSharper support Yes (version 6) Yes (version 7)
Supports spies, stubbing, mocking No1 Yes
Supports async tests2 Yes Yes
Supports testing for exceptions Yes Yes
Built in assertions 5 unique + 4 not versions 12 + separate not modifier
Supports custom assertions No Yes
Setup/Teardown functions3 Yes Yes
Can mock the JavaScript clock4 No Yes
Can check for missing vars5 Yes No
Automatically resets test DOM area after each test Yes
<div id="qunit-fixture">
   content wiped before each test
</div>
No
Optionally disables try catch6 Yes No

1A separate package, SinonJs, can provide support for spies and stubbing and mocking for QUnit tests
2Pause the runner to wait for an async response from the server, setTimeout, etc.
3Executed before/after each test.
4So you don't have to wait 10 seconds for a 10 second timeout.
5Missing var in variable declaration makes variable global.
6Depending on your browser, can make it easier to get a stack trace.

Both QUnit and Jasmine are competent, well supported packages. However, because Jasmine currently has the edge in features, the rest of this article focusses on adding Jasmine tests to an ASP.NET MVC solution.

Visual Studio 2012 and Chutzpah

Microsoft has made unit testing extensible in Visual Studio 2012. If you already use that version, you can use the Chutzpah package to make JavaScript unit testing a first class citizen inside Visual Studio (details).

The rest of this article assumes you use Visual Studio 2010, where things a bit more complicated.

Integrating Jasmine unit tests in Visual Studio 2010 and TFS 2010

To make this work, we can take this approach:

  1. Install Jasmine and add JavaScript unit tests.

     

    This allows us to run the tests inside a browser, which is good. However, we also want to run those tests as part of a unit test inside Visual Studio, and have it fail that test when the JavaScript unit tests fail.
  2. Install Chutzpah. This allows us to run the Jasmine unit tests from the command line. It will write a failure message to the console if the tests fail.
  3. Create a unit test within Visual Studio that executes that command line. If the output contains the failure message, fail the test.

Lets make this work. You will find a worked out example in the download.

Install Jasmine and write some tests

  1. I'm assuming you have a project that looks a bit like this:

     

    MyProject.Site
       Controllers
       Models
       Views
       Scripts
       ...
    MyProject.Tests
       Controllers
       ...
    
  2. Download Jasmine zip file
  3. Unzip the file. This will result in a few directores and an html file. Create a new directory JsUnitTests in your Test project and move the Jasmine directories and files in there:
    MyProject.Site
       Controllers
       Models
       Views
       Scripts
       ...
    MyProject.Tests
       Controllers
       JsUnitTests spec src lib SpecRunner.html
       ...
    
  4. spec contains demo tests, and src contains demo JavaScript code that the demo tests run against. Open SpecRunner.html in a browser to see Jasmine in action.
  5. Remove the src directory. Update SpecRunner.html with script tags that load the actual JavaScript code you want to test.

     

    Use relative paths in your script tags, not absolute paths. That makes it so much easier if you ever want to move or copy your solution. And especially if you use TFS Build.
  6. Get rid of the demo tests in spec and write a few tests of your own against your own code. An excellent tutorial on writing Jasmine tests is on the Jasmine home page. Update SpecRunner.html with script tags that load your tests.
  7. Open SpecRunner.html again in a browser to see whether your code passes.
  8. Make sure that all your new files have been included in the project and that they are checked in.

Install Chutzpah

Chutzpah allows you to run JavaScript unit tests from the command line.

  1. Install NuGet if you haven't done so before.
  2. In Visual Studio, open the Package Manager Console - click Tools | Library Package Manager | Package Manager Console.
  3. At the PM> prompt, enter
    Install-Package Chutzpah
    

    This creates a new packages directory in your solution and installs the Chutzpah files in there. At the time of writing, Chutzpah was at version 1.4.2, which results in:

    packages Chutzpah.1.4.2 tools chutzpah.console.exe ...
    MyProject.Site
       Controllers
       Models
       Views
       Scripts
       ...
    MyProject.Tests
       Controllers
       JsUnitTests
          spec
          lib
          SpecRunner.html
       ...
    
  4. Now you can run the Jasmine tests from the command line (update this if the Chutzpah version number has changed):
    cd <directory containing SpecRunner.html>
    ..\..\packages\Chutzpah.1.4.2\tools\chutzpah.console.exe SpecRunner.html
    

Create unit test inside Visual Studio

Finally, we'll create a unit test that executes the command line you just saw.

  1. Install the NuGet package ExecConsoleProgram. This makes it easy to run a console program from managed code, such as a unit test. Open the PM> prompt again and enter:
    Install-Package ExecConsoleProgram
    

     

    NuGet stores package information in a packages directory in the root of your solution (where you .sln is located). Be sure to check in that packages directory, so the build controller can access it when you do a TFS Build.
  2. Add a reference to the ExecConsoleProgram dll to your Tests project. You'll find the dll in the packages directory.
  3. To your JsUnitTests directory, add a new unit test - right click JsUnitTests | Add | New Test | Basic Unit Test
  4. Make it look like this:
    [TestMethod]
    public void JsTest()
    {
        string stdOut;
        string stdErr;
    
        try
        {
            // Correct paths when running unit tests in Visual Studio
            ExecConsoleProgram.ConsoleProgram.Execute(
                @"..\..\JsUnitTests", 
                @"..\..\..\packages\Chutzpah.1.4.2\tools\chutzpah.console.exe", 
                @"SpecRunner.html",
                out stdOut, out stdErr);
        }
        catch (System.ComponentModel.Win32Exception)
        {
            // Correct paths when running unit tests as part of TFS Build
            // >>>>>> In the code below, replace JasmineMVC.Tests with the name 
            // of your own test project.
            ExecConsoleProgram.ConsoleProgram.Execute(
                @"..\Sources\JasmineMVC.Tests\JsUnitTests", 
                @"..\Sources\packages\Chutzpah.1.4.2\tools\chutzpah.console.exe", 
                @"SpecRunner.html",
                out stdOut, out stdErr);
        }
    
        Assert.IsFalse(stdOut.Contains("[FAIL]"));
        Assert.IsFalse(stdOut.Contains("ERROR OCCURRED"));
        Assert.IsTrue(string.IsNullOrEmpty(stdErr));
    }
    

    The Execute method runs the command line. It takes these parameters:

    • Working directory. This is relative to the directory where the currently running assembly is located.
    • Path to the executable (relative to the directory where the currently running assembly is located).
    • Parameters to the executable (relative to working directory).
    • stdOut and stdErr receive the output of the program. As you see, the unit test fails if Jasmine reported that the tests failed or if something went wrong.
  5. Done! When you now run your unit tests from the Test View window, they'll run the JavaScript tests as well. Change one of your tests to force it to fail and see what happens.

Comments (1) -

Damian
Damian Luxembourg
7/17/2012 11:14:41 AM #

Hi,

It made my day! Thank you very much.

I am going to improve the result parsing a bit to get some information, but the idea is great.

Best,
Damian

Reply

Add comment

  Country flag

biuquote
  • Comment
  • Preview
Loading

Books

Book: ASP.NET Site Performance Secrets

ASP.NET Site Performance Secrets

By Matt Perdeck

Details and Purchase

About Matt Perdeck

Matt Perdeck Presenting

Matt has written extensively on ways to improve web site performance.

more >>