Have You Tested the Contract?

I’ve been grappling with the concept of contract testing over the past couple of months. I think they can offer me the confidence in my tests that I never really felt with integration tests (any of which was demolished when I watched Integrated Tests Are a Scam).

My approach so far has been simple enough. Whenever I stub or mock a method on a collaborator (Mocks Aren’t Stubs) I ensure that there is a unit test for that method, which accepts the same parameters and returns the same result. If such a test doesn’t exist then I write one.

For example, here is a unit test for a method on a repository for books to find a single book by its title. It’s a PHPUnit test using the built-in support for Prophecy for test doubles.

/**
 * @test
 */
public function itCanFindABookByItsTitle()
{
    $db = $this->prophesize('DatabaseConnection');
    $repo = new BookRepository($db->reveal());

    $expectedBook = array(
        'id' => 101,
        'title' => "The Count of Monte Cristo",
        'author_id' => 20200,
    );

    $db->fetchRow("SELECT * FROM books WHERE title = ?", array("The Count of Monte Cristo"))
        ->willReturn($expectedBook);

    $book = $repo->findByTitle("The Count of Monte Cristo");

    $this->assertSame($expectedBook, $book);
}

In this test the DatabaseConnection‘s fetchRow method is stubbed. I must now ensure that the contract that is assumed by this test double exists. I check my tests for the database connection and find the following test.

/**
 * @test
 */
public function itCanFetchASingleRow()
{
    $dbal = $this->prophesize('Doctrine\DBAL\Connection');
    $db = new DatabaseConnection($dbal->reveal());

    $query = "SELECT * FROM some_table WHERE some_column = ?";
    $params = array(123);
    $expectedRow = array(
        'some_column' => 123,
        'other_column' => 456,
    );

    $dbal->fetchAssoc($query, $params)
        ->willReturn($expectedRow);

    $row = $db->fetchRow($query, $params);

    $this->assertSame($expectedRow, $row);
}

This test fulfils the contract assumed by the test double in the first example. The $query string and the $params array are not the exact same values, but are similar enough for practical purposes. If these values were edge cases (eg an empty string) or didn’t quite match the expectations of the test double (eg the $params array has additional elements) then we’d need to find another contract test or write a new one.

A big problem I’ve had is that this a very manual process and it can be hard to keep track of exactly which test doubles have been checked. This has led me to start using annotations on my unit tests to systematically keep track.

Going back to the example, once I’ve found the contract test I can go back to my initial unit test and add the following annotation.

/**
 * @test
 * @contract DatabaseConnectionTest::itCanFetchASingleRow
 */
public function itCanFindABookByItsTitle()

If a unit test has more than one test double then I add a @contract annotation for each test double. If a unit test has no test doubles then I add a @noContracts annotation, to differentiate it from a unit test that hasn’t been checked for contract tests yet. The annotation syntax is based on that used by PHPUnit for its @covers annotation. My unit tests are namespaced according to PSR-4 and the annotations contain the appropriately qualified class name.

The idea behind this approach is that I could eventually create an automated tool to help with this process. It could detect which tests are missing annotations. It could follow the path of dependencies created by the annotations and run those tests. It could even give another coverage statistic to futilely strive to improve.