Last month I resolved to take a crack at test driven development with Qt. The notion of automated testing was first introduced to me by a friend, who was working in a QA role. These conversations stuck mostly in the conceptual realm, never quite landing for me. At least not in a way that would empower me to sit down and begin writing unit tests for real code. In my more recent forays into Rails, and this Qt project, the source of this disconnect became more clear.
I really like the Rails methodology of testing. Here are several directories corresponding to different types of tests, here’s some assertions, and hey we’ve configured a test database for you to use, so lets go! Seeing these concepts in an immediately usable context does wonders. To an application developer, it’s useful to divide the world into unit tests, and the rest. A unit test takes the smallest independent unit of a code base (e.g. a class in C++), and tests the consequences of any notable combination of inputs and their output (think “public methods”, those that take in or return data). In Rails, unit tests are most frequently written for Active Record models. These are easily isolated from the rest of your code base, allowing you to test the full range of their functionality without needing to instantiate any additional objects.
Starting with a Qt code base, it’s no wonder I got hung up on where and how to write unit tests. With Rails it’s very clear how to structure the directories containing your tests, how to execute them, how to setup a non-production/non-development environment. While Qt provides QTestLib (all the assertions and tools needed to write tests), the steps for implementing them in a usable manner is unclear. More importantly, Rails documentation and directory structure makes it very clear what kinds of tests to write for which pieces of code. When writing Qt code, it’s common to write UI code tightly coupled with business logic, making it unclear what is really unit testable.
It would be best to start by to determining which parts of the application are ripe for unit testing. I decided to make a simple Contacts application, which would save data to disk in JSON format (I wanted an excuse to play around with the new QJsonDocument class in Qt5). The code for Qontacts (sorry…) can be found on GitHub. Subclassing QAbstractItemModel to understand Json arrays and objects would let me take advantage of the Item View classes for both viewing and editing.
A word about test driven development. It’s a bit mind bending to wrap your head around, frankly. Your immediate goal is no longer to write a piece of an application, bringing you professional / hobbyist pride. It’s to write a series of test cases, then writing the application as a mere implementation detail of the test. Albeit one you can be assured works precisely as intended, and can easily be verified as working in the future (after the inevitable code change for additional functionality). Then there’s the whole notion of “red-light, green-light”, where you first ensure the tests fail (a test that returns success on non-working code isn’t terribly useful, no?) before getting them to work. This is easier to accomplish in Ruby, where sending a message to a non existent object or method merely results in a message to nil, rather than a failure to compile. In C++, you’ll need to write stub implementations of your methods that return an invalid value.
Lets begin by figuring out the the structure of your testable application. Much of this is inspired, if not outright lifted, from a Qt Developer Days talk: Introduction to automated testing.
For each class we wish to implement, we will also write a QObject subclass class comprised of tests (As seen in the QTestLib tutorial) with one addition detail. At the bottom of this class, we add the following line:
QTEST_MAIN(tst_JsonObjectModel)
This lovely macro will generate a main method, meaning this class will compile as an executable program. When run, it will go through the individual tests and report their outcome. This class will need it’s own project file (any given project cannot contain multiple executable binaries). Instead of writing tedious project files for each test, including specific header and implementation files, we’ll structure the application in such a manner that all code (except main.cpp) is compiled into a static library. The library can then easily be linked into the application proper, as well as any tests we write.
Each test case, comprised of a .cpp and a .pro file, gets it’s own directory. Each of these directories exists in a /tests directory, also containing a project file of the SUBDIRS variety. That is to say, a qmake project that will compile a number of sub-projects (i.e. the test cases).
This structure works well enough for my simple app, but may not be appropriate for larger projects. If you’re pursuing some home-grown way of executing all tests for an application, you could try removing the QTEST_MAIN macro, instead adding Qt+=testcase to your project file. Rather than running make, run “make test”. This command will compile and immediately run the test case. You could then write a script that would generate the MAKEFILEs with qmake, and run all test cases.
As for writing the tests, the QTestLib tutorial is an excellent place to start. QTestLib provides a number of convenient methods and macros to test your class. Best practice dictates each test program contain one test case. A test case corresponds to a class in your application code. A test case contains many tests, and every private slot is considered a test.
For my Json models, I knew I would be re-implementing a number of abstract methods which item views would be relying on to function properly. I began by writing tests where in I provided the model a QJsonValue (depending on the model, either an object or an array), and tested it’s data functions to ensure the correct QVariants were produced for the relevant indexes. I also took advantage of the QSignalSpy class to ensure the various “rows about to be removed / added” signals were emitted. Each method I wrote a test for also needed a stub implementation returning an invalid value. This constituted the “red light” portion of test driven development. Once I had written enough tests to cover the basic functionality needed, I began implementing the model, verifying successful tests as I went along (“green light”). As new functionality was needed, I continued this pattern. The tests can be found in the GitHub Qontacts repo.
The rest of the Qontacts code base (such as it is) is widget code, gluing the bits and pieces together into a coherent ui. Testing these would fall outside the scope of unit testing, instead falling under what Rails would probably call integration testing. Should you be interested in testing a Qt application in all it’s button-clicking, document-scrolling, menu-selecting glory, you’ll want to take a look at froglogic’s Squish, an automated GUI testing tool. It’s pretty amazing, letting you script complicated sequences of actions and testing the state of QObjects.
In conclusion, my hangup with unit testing Qt code stemmed from not understanding when testing was appropriate. This can be resolved by better separation, de-coupling if you will, UI code from business logic. In Qt widget code, these tend to be pretty tightly glued together. I’d be curious to see whether testing (or rather, the inclinations of coders towards writing testable code) becomes easier with Qt Quick, where a clear division exists between the two. For further reading, I’ve heard good things about The Art of Unit Testing. I’m also looking forward to reading Eric Steele’s book, What do I test?, which looks to address the question of when and where it’s appropriate to focus your testing skills.