A bit of background first. I'm working on a large ColdBox project and one of the first things I did when I joined the project was to start creating some unit tests and integration tests. ColdBox has integration with MXUnit so you can fairly easily write unit tests for parts of your model and integration tests for your event handlers. Over time we've built a suite of over 100 MXUnit-based tests. If I had my way, we'd have a lot more by now but test-infecting your colleagues isn't always easy!
Once we had enough of the system working, testing some of the complex flows through the application became pretty tedious and some sort of automation became necessary. We picked Selenium since it offers a plugin for Firefox that lets you record a click-thru session for your application and play it back. Selenium tests are saved as simple HTML documents with a table listing actions, targets and values in three columns. You turn this into a test by adding rows into the table that assert text is present (in the rendered page). You might end up with rows like this:
<td>open</td>
<td>/</td>
<td></td>
</tr>
<tr>
<td>assertTextPresent</td>
<td>Home Page</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>username</td>
<td>testuser</td>
</tr>
<tr>
<td>type</td>
<td>password</td>
<td>secret</td>
</tr>
<tr>
<td>clickAndWait</td>
<td>FONT COLOR=GRAY<>I<>//input[@value='login']</td> <td></td>
</tr>
<tr>
<td>assertTextPresent</td>
<td>Welcome, testuser!</td>
<td></td>
</tr>
The final step is to fully automate these suites by running them automatically as part of your ant build script. You have an ant build script, right?
When you download Selenium, you get an IDE to install into Firefox (an XPI file) and a JAR file, which is what you need for use with ant. A good idea is to create an "init-selenium" target in your ant script that looks something like this:
<property name="selenium-library.jar" value="${selenium.home}/selenium-server.jar"/>
<taskdef resource="selenium-ant.properties">
<classpath>
<pathelement location="${selenium-library.jar}" />
</classpath>
</taskdef>
</target>
suite="${tests}/selenium/suites/${suite.name}.html"
results="${tests}/results/selenium/${suite.name}-results.html"
browser="*${browser}"
multiWindow="false"
timeoutInSeconds="300"
startURL="${selenium.host}" />
I hope that gives you a taste of how to get automated testing going with Selenium. In addition to our 100+ MXUnit tests, we now have three suites of automated Selenium tests covering several areas of our application. Our ant build script resets the entire state of the server and database and runs our Scala test suite (I blogged about that a bit before), our MXUnit test suites and our Selenium test suites in anything between three and eight minutes depending on the speed of each developer's machine.
Can you elaborate on the "resets the entire state of the server and database" part?
The database part makes sense -- any data-dependent tests need to start with a clean (maybe empty or nearly empty) database. Are there ANT components that specifically assist with say MySQL resets, or do you just invoke a command line call to execute a SQL script?
The server part I'm more curious about. Are you referring to the CFML engine and/or servlet container? What gets reset?
Thanks!
The overall ant script does a lot of stuff :) It builds the Scala code, runs the ScalaTest suite, auto-generates ScalaDocs, restarts Railo, reloads the database with seed data, repopulates the search engine, runs the unit test suite (component-based tests via MXUnit) and the integration test suite (controller-level tests via ColdBox/MXUnit) and then the Selenium test suites.
Takes about 7-8 minutes on most people's machines (but only 2-3 minutes on my 2.8GHz quad core iMac :)
http://corfield.org/entry/Restarting_Railo_with_Ant
Automation via ant isn't a huge deal either and, for any multi-developer project, I'd prefer to have automated testing in place.
