Automated JavaScript Tests Using Grunt, Phantomjs, and QUnit

Many developers and development shops have embraced testing in their server side code – either using a TDD model, or sometimes just with unit or functional tests created on the fly – but very few developers I talk to test their front end code beyond eyeballing it in a few browsers (or if they’re lucky, with a service like BrowserStack). And even fewer of those use any kind of automation. In this post I’m going to try and help out that majority of you who aren’t testing – or maybe just aren’t automating.

Lately I’ve been converting a number of my jQuery plugins over to Github – something the new jQuery plugin site is requiring – and while doing so I’m refactoring and adding tests with QUnit (created by the same folks that write the jQuery core). I realized pretty quickly that having to go to the browser, hard refresh, and then run the tests again after every change was a time killer, but I didn’t want to write a whole bunch of tests and then go check the results, otherwise I could be tracking down bugs (in my code and the tests) for days. Instead I set up some automated testing using Grunt, PhantomJS, and a couple Grunt plugins. Here’s how.

The Setup

First things first, you’ll need to install the Node (we’ll be using the Node Package Manager, a.k.a. “npm”, specifically) and Grunt. These examples are from my Ubuntu Linux machine, but the code is pretty similar for Windows or Mac (but consult the links above for details).

~$ sudo apt-get install nodejs

~$ sudo npm install -g grunt-cli

Two Notes: (1) The “-g” makes Grunt install globally versus just within this project, and (2) I had some issues with having an older version of Nodejs, so I installed a Node version manager through `npm` after this and switched to a more current version of Nodejs.

We’ll also need QUnit, which is a simple JavaScript library you can drop into your testing directory in your project. Below is what my project structure looks like for a jQuery plugin. Of course, yours will be different, but there are a few things that will be required to be in specific places (I’ll let you know where).

└ project
  ├ src   // plugin source, project files, etc (could be diff for you)
  ├ tests // we'll be working in here mostly
  │ ├ lib
  │ │ ├ jquery-1.x.x.min.js // if you need it (QUnit doesn't)
  │ │ ├ qunit-1.10.0.js
  │ │ └ qunit-1.10.0.css
  │ ├ index.html  // our QUnit test specification
  │ └ tests.js    // your tests (could be split into multiple files)
  ├ Gruntfile.js  // you'll create this, we'll get to it shortly
  ├ package.json  // to specify our project dependencies
  └ ...

Okay, so this first thing you need to create is the “package.json” file in your root directory. This is a standard for Node.js projects, but npm uses it to see what dependencies exist and what to install. If you’re not creating a Node.js project, then this file can be very minimal. Here’s mine for a jQuery plugin project:

{
  "name": "projectName",
  "version": "1.0.0",
  "devDependencies": {
    "grunt": "~0.4.1",
    "grunt-contrib-qunit": ">=0.2.1",
    "grunt-contrib-watch": ">=0.3.1"
  }
}

What’s important here are the “devDependencies”, which specify what npm needs to install for this project. I’ve included two Grunt plugins which I found useful. The “qunit” one is obvious, but the “watch” plugin allows our tests to run any time a specified file is changed, without us having to manually re-run the tests! Now head back to your command line, navigate to the root directory of the project and run this:

~$ npm install

Node will go find the correct packages and put them into a “node_modules” directory. So why not just download the packages manually and put them in your project? For one, if you have multiple developers, then this method will allow any new developer to get up and running quickly, without you having to have the test libraries in your source control. On that note, you might want to add “node_modules/” to your .gitignore file!

Grunt Runner

In order to get Grunt ready to run we need to create (and fill in) our Gruntfile, which tells Grunt many things about the tasks we want to execute. The basic structure is simple:

module.exports = function(grunt) {
  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json'), // the package file to use

    taskName: { // internal task or name of a plugin (like "qunit")
      // options, etc (see the task/plugin for details)
    },
    ...
  });
  // load up your plugins
  grunt.loadNpmTasks('grunt-contrib-pluginName');
  // register one or more task lists (you should ALWAYS have a "default" task list)
  grunt.registerTask('default', ['taskToRun']);
  grunt.registerTask('taskName', ['taskToRun', 'anotherTask']);
};

Of course, we need to fill this in, we’ll start with the “qunit” task, which is pretty easy: simple replace the “taskName” in our initConfig call with this:

...
    qunit: {
      all: ['tests/*.html']
    }
...

This defines the “qunit” task in Grunt and tells the qunit plugin to execute all tests found in any .html file in the “tests” folder. But we also need to load the plugin and we should go ahead and make it the lone default task:

...
  grunt.loadNpmTasks('grunt-contrib-qunit');
  ...
  grunt.registerTask('default', ['qunit']);

Creating Your Tests

At this point Grunt is ready (although we haven’t done the “watch” task yet, we’ll get to it later); now it’s time to create some QUnit tests. I won’t get into too much detail on actually writing your tests, instead I’ll point you to the QUnit Cookbook which has some good examples. So let’s talk about setting up QUnit in the first place. Hopefully you have already downloaded the library and put it in the “tests/lib/” folder (like the folder structure above). Now create an “index.html” file in your “tests/” folder and make it look like this:

<!doctype html>
<html>
  <head>
      <meta charset='UTF-8' />
      <meta http-equiv='content-type' content='text/html; charset=utf-8' />

      <title>jQuery.simpleFAQ QUnit Test Runner</title>

      <link rel='stylesheet' href='lib/qunit-1.10.0.css'>

      <!-- add any external libraries your code needs -->
      <script src='lib/jquery-1.7.2.min.js'></script>

      <script src='../src/your.project.code.js'></script>
      <!-- add any JS files under test (or put them in different .html files) -->

      <script src='lib/qunit-1.10.0.js'></script>

      <!-- your tests, any and all to run with the given fixtures below -->
      <script src='tests.js'></script>

  </head>
  <body>
    <div id="qunit"></div> <!-- QUnit fills this with results, etc -->
    <div id='qunit-fixture'>

      <!-- any HTML you want to be present in each test (will be reset for each test) -->

    </div>
  </body>
</html>

Great, now we need to write some tests. Create a “test.js” file, also in your “tests/” folder. QUnit tests are pretty easy to write, first we’ll create a test “module” – which is just an organizational structure, and has no bearing on your tests running – and one simple test:

module("Basic Tests");

test("truthy", function() {
  ok(true, "true is truthy");
  equal(1, true, "1 is truthy");
  notEqual(0, true, "0 is NOT truthy");
});

Of course, you’ll want to put your actual tests in here. You can reference any elements you put in your “qunit-fixture” div (see the html above) and it will be reset for each test. To run the tests you *could* go to http://localhost/path/to/project/tests/index.html and see the results… but we want to run these automatically – and that means without having to open a browser. This can be accomplished with PhantomJS.

Headless Horse-browser

To set up PhantomJS I used npm, it was quite easy:

~$ npm install -g phantomjs

PhantomJS has a lot of options, but I can’ get into them now… mostly because I don’t know a tenth of them. :) I recommend checking out their Wiki on Github which is quite flushed out. For our purposes, we’ll just use all of the defaults.

We should be ready to run our tests now, so navigate to your project directory and run Grunt!

~$ cd /path/to/project/root/

~$ grunt
Running "qunit:all" (qunit) task
Testing tests/index.html..............OK
>> 3 assertions passed (10ms)

Done, without errors.

Hopefully you see the output above. So what happened? You told grunt to run, but didn’t specify a task, so Grunt ran the “default” task, which if you recall, runs the “qunit” task. The QUnit plugin for Grunt runs a headless browser through PhantomJS, loading your test html file and executing the JavaScript therein, then the plugin reports back to Grunt with the output of the QUnit test runner.

As an aside, you could have also run: “grunt qunit” which specifies to grunt that you only want to run the “qunit” task by itself (and nothing else).

Who Watches the Watchers

I told you earlier that you wouldn’t have to reload a browser every time you wanted to run your tests, and that those tests would run any time you change something… let’s make that happen.

The first step is to alter your Gruntfile. We’re going to add a new task. Remember how our “package.json” specified a second “grunt-contrib” file? Yep, that one.

grunt.initConfig({
  ...
  watch: {
    files: ['tests/*.js', 'tests/*.html', 'src/*.js'],
    tasks: ['qunit']
  }
});
...
grunt.loadNpmTasks('grunt-contrib-watch');

So we added options for another task: “watch” – it states that we want to watch certain files (our tests, our test fixtures in the html files, and our source javascript) and that when any of those change (or when new files are added or existing files deleted) run certain tasks – in our case “qunit”.

That’s pretty much it! Head back to your command line and instead of running the “qunit” task directly, run the “watch” task and then go change your test file (just tweak it, or add a test) and watch all of the tests run in your command line console!

~$ grunt watch
Running "watch" task
Waiting...OK
>> File "tests/index.html" changed.
Running "qunit:all" (qunit) task
Testing tests/index.html..............OK
>> 3 assertions passed (10ms)

Done, without errors.

Completed in 0.771s at Sun Apr 21 2013 11:44:29 GMT-0500 (CDT) - Waiting...

Notice that Grunt waits for something to change, sees the change and runs the “qunit” task, which runs our tests and reports the results, then Grunt returns to a “Watiing…” state! You can leave this window up and running, work on your code or tests, and almost instantly see the results without having to pause and go refresh (or start up) a browser.

Hope you enjoyed the tutorial, let me know if you have questions, corrections, or improvements!

Tags: , , ,

Sunday, April 21st, 2013 javascript, testing

15 Comments to Automated JavaScript Tests Using Grunt, Phantomjs, and QUnit

  • Riesgo says:

    Yep. I’ll be giving these a try.

  • Awesome piece!

    Thanks!!

  • Michael says:

    Excellent overview – thanks Jordan!

    A few thoughts for improvements, in order of appearance:

    1) Personal preference, but no need to install grunt-cli globally – just include it in the package.json and symlink:

    $ ll bin/grunt
    lrwxrwxrwx 1 michael michael 35 Jun 7 19:56 bin/grunt -> ../node_modules/grunt-cli/bin/grunt

    2) I’m not sure if there’s a reason why you mention setting up/installing phantomjs globally at all? I *think* the qunit plugin is using node_modules/grunt-contrib-qunit/node_modules/grunt-lib-phantomjs (I haven’t double-checked)

    3) grunt-notify is a nice addition. It’s not perfectly integrated with Ubuntu’s notify-send (ie. no icon on notifications), but it works.

    Thanks!

    • jordan says:

      Michael:

      Thanks! Installing the grunt-cli globally just means I don’t have to do that sym linking with every project I have (about a dozen that use Grunt currently). As for the PhantomJS global install, similar answer, I use it for other things, and you may want it installed globally for those things.

      The grunt-notify plugin looks pretty sweet, thanks for the reference!

  • Lewis Nixon says:

    Thanks for the post, really helped

  • Kyle says:

    Hi!. I am running into an issue following your steps.
    c:\qunit>grunt qunit
    Running “qunit:all” (qunit) task
    Testing index.html
    >> PhantomJS timed out, possibly due to a missing QUnit start() call.

    Any idea why this would be occurring? Only difference between what you did and what I did was that my tests.js and index.html are in the same place that the grunt file is located.

    Thanks!

    • Kyle says:

      Nvm, I got it.

      • jordan says:

        Glad you figured it out, sorry for the lack of reply! October was a crazy month for me. Care to share what it was?

        I find that the errors that the grunt plugin gets from Phantom are not very helpful, but this one is usually either a JS syntax error in the tests or source, or it’s that an external file couldn’t be found in the test page.

  • Grsmto says:

    Great article!

    I have a question, I see everywhere projects with tests including the qUnit library in their source control. Why they don’t use it via the node-module ? If we have to include it in our source control, isn’t it useless to use a dev dependencies tool (like npm with Grunt)?

  • jordan says:

    That’s a good question (and point). My best guess is that (like me) most people want to know exactly what version of the QUnit library they are using. The grunt-contrib-qunit plugin uses whatever version they see fit, you may have different requirements. Additionally, I can see someone wanting to be able to run the tests without having Node or Grunt installed – this is possible if you have the QUnit library checked into source control as you can just load the test page in a browser.

    Thanks for reading!

  • Thanks for the write up.

    I’m new to grunt so was wondering why we cannot define the tests/lib directory somehow to add these to .gitignore?

    This is a big +42 :p

    • jordan says:

      Thanks Clemens!

      As for the tests/lib directory, we need that in our project when we run our tests… yes, we could add those to npm as a devDependency, then we would need to change where our test file points to for the QUnit library. Certainly something to look at doing.

  • Eduardo J. says:

    I’ve added an initial javascript dummy project ready to test using your article:

    https://github.com/eduardoj/js-qunit-phantomjs-grunt-initial-project

    Thanks!

  • Leave a Reply

    April 2013
    S M T W T F S
    « Mar   Aug »
     123456
    78910111213
    14151617181920
    21222324252627
    282930