In a previous blog post, I described how I took Emscripten-created JS and turned it into an UMD module. One of the reasons I did this was because I wanted more control over the generated JavaScript and for it to be usable in more contexts, such as with the RequireJS module loader.

As I am a responsible developer, I desired to create a number of automated unit tests to ensure that the client-visible API for my Emscripten module works as I intended. I began by searching for a browser automated test framework and settled upon Nightwatch.js. Now I just had to figure out how to get Nightwatch.js tests running in my existing, CMake-based build system. Here’s how I did it.

Configurig Nightwatch.JS

In order to use Nightwatch.JS, you must first configure it by creating a file called nightwatch.json. The first major decision you need to make is which WebDriver-implementing system you wish to use. Most users use Selenium, but you can also run an individual browser driver directly.

As I was not concerned with cross-browser compatibility – I assume that if the test works on one browser it will work on all major browsers – and I was looking for a system with a minimum number of build-time dependencies, I decided to use ChromeDriver automatically as my WebDriver implementation.

To make everything work, I did the following:

To automatically download chromedriver, add the following to CMakeLists.txt:

# Install chromedriver
add_custom_command(
    OUTPUT node_modules/chromedriver/package.json
    COMMAND npm install chromedriver
)
  
add_custom_target(
    chromedriver ALL
    DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/node_modules/chromedriver/package.json
)

To configure Nightwatch.JS to use chromedriver, create a nightwatch.json which looks like this (the purpose of nightwatch-globals.js will become clear shortly):

{
    "globals_path": "nightwatch-globals.js",
    "selenium" : {
        "start_process" : false
    },
    "test_settings" : {
        "default" : {
            "selenium_host": "localhost",
            "selenium_port": 9515,
            "default_path_prefix": "",
            "desiredCapabilities": {
                "browserName": "chrome",
                "chromeOptions" : {
                    "args" : ["--no-sandbox"]
                },
                "acceptSslCerts": true
            }
        }
    }
}

To start and stop chromedriver when running tests, create a nightwatch-globals.js which looks like this:

var chromedriver = require("chromedriver');

module.exports = {
    before: function(done) {
        chromedriver.start();
        done();
    },
    after: function(done) {
        chromedriver.stop();
        done();
    }
};

CMake will run the unit tests from ${CMAKE_CURRENT_BINARY_DIR}, so we’ll need to copy the above config files to ${CMAKE_CURRENT_BINARY_DIR}. Here’s how to do that:

# Copy nightwatch config files to target directory
add_custom_command(
    OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/nightwatch.json
    COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_SOURCE_DIR}/nightwatch.json ${CMAKE_CURRENT_BINARY_DIR}/nightwatch.json
    DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/nightwatch.json
)
add_custom_target(
    nightwatch.json ALL
    DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/nightwatch.json
)

add_custom_command(
    OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/nightwatch-globals.js
    COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_SOURCE_DIR}/nightwatch-globals.js ${CMAKE_CURRENT_BINARY_DIR}/nightwatch-globals.js
    DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/nightwatch-globals.js
)
add_custom_target(
    nightwatch-globals.js ALL
    DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/nightwatch-globals.js
)

Automatically Download Nightwatch.JS and Run Unit Tests

First, we need to create a Nightwatch.JS unit test. Here’s a sample test case from the Nightwatch.JS home page:

// google.js
module.exports = {
    'Demo test Google' : function (browser) {
        browser
            .url('http://www.google.com')
            .waitForElementVisible('body', 1000)
            .setValue('input[type=text]', "nightwatch')
            .waitForElementVisible('button[name=btnG]', 1000)
            .click('button[name=btnG]')
            .pause(1000)
            .assert.containsText('#main', 'Night Watch')
            .end();
    }
};

To automatically download the Nightwatch.JS library, add the following lines to CMakeLists.txt:

# Install nightwatch
add_custom_command(
    OUTPUT node_modules/nightwatch/package.json
    COMMAND npm install nightwatch
)
add_custom_target(
    nightwatch ALL
    DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/node_modules/nightwatch/package.json
)

To run the above unit test as a CMake unit test, add the following lines to CMakeLists.txt:

add_test(
    NAME nightwatch_test
    COMMAND ./node_modules/nightwatch/bin/nightwatch -t ${CMAKE_CURRENT_SOURCE_DIR}/google.js
)

You may want to separate your tests into multiple JavaScript files and execute them independently. Here’s one way to do that from CMake:

file(GLOB TESTCASE_SRC tests/*.js)
foreach (testPath ${TESTCASE_SRC})
    get_filename_component(testName ${testPath} NAME_WE)

    # Test all unit tests
    add_test(
        NAME browser_${testName}
        COMMAND ./node_modules/nightwatch/bin/nightwatch -t ${testPath}
    )
endforeach()

Using Local Web Server when Running Test Cases

In certain cases, your unit tests be able to refer to local file: URLs, but things tend to be a lot easier if your unit tests reference URLs from a local web server. It’s really easy to get one up and running:

Download Node’s http-server module by adding the following to your CMakeLists.txt

# Install http-server
add_custom_command(
    OUTPUT node_modules/http-server/package.json
    COMMAND npm install http-server
)
add_custom_target(
    http-server ALL
    DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/node_modules/http-server/package.json
)

Modify nightwatch-globals.js to start and stop the web server as part of the tests:

var chromedriver = require("chromedriver');
var http = require("http-server');

module.exports = {
    before: function(done) {
        this.server = http.createServer();
        this.server.listen(8080);
        chromedriver.start();
        done();
    },
    after: function(done) {
        this.server.close();
        chromedriver.stop();
        done();
    }
};

Once this is done, your tests can refer to http://localhost:8080.

Note that http-server reads files from the current working directory, and CMake runs unit tests from ${CMAKE_CURRENT_BINARY_DIR}, so you may need to copy your test HTML and JavaScript to ${CMAKE_CURRENT_BINARY_DIR}. Here’s some CMake code which you might find helpful:

# Copy all .HTML files to binary directory
file(GLOB HTML_SRC *.html)
foreach (htmlPath ${HTML_SRC})
    get_filename_component(htmlFileName ${htmlPath} NAME)

    # Copy HTML to binary folder so they can be referred to by the test
    add_custom_command(
        OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/${htmlFileName}
        COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_SOURCE_DIR}/${htmlFileName} ${CMAKE_CURRENT_BINARY_DIR}/${htmlFileName}
        DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/${htmlFileName}
    )
    add_custom_target(
        browser_copy_${htmlFileName} ALL
        DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/${htmlFileName}
    )
endforeach()

For a real-life, working example of all this in action, see the source code for my streaming percentiles library.