API's that Suck

October 3, 2011

Rules for Writing Automated Tests: Label your Assertions

Filed under: Testing — Grauenwolf @ 6:37 pm

Before one can determine how a test has failed they need to determine what part of it failed. Simply having the message “expected blah, but got foo” doesn’t really help someone reading the test results, they need to know why blah was expected in the first place.

Single Assertion per Test

There are several ways to label assertions. One of the more popular ideas for the last few years has been the “single assertion per test” theory.  Under this pattern the test name itself is the label for the assertion, no further clarification is needed.

Unfortunately this leads to a lot of repetitive code.  Consider this simple test to ensure that an object can be serialized to XAML and back without losing any information.

[TestMethod()]
public void XamlParseCreateTest()
{
    var original = new Foo() { FooBar = "Sam" };
    original.Bars.Add(new Bar() { Weight = 5, Cost = 1.1M });
    original.Bars.Add(new Bar() { Weight = 10, Cost = 2.2M });
    original.Bars.Add(new Bar() { Weight = 15, Cost = 3.3M });

    var xaml = XamlUtilities.CreateXaml(original);

    var copy = XamlUtilities.ParseXaml<Foo>(xaml);

    Assert.AreEqual(original.FooBar, copy.FooBar);
    Assert.AreEqual(original.Bars.Count, copy.Bars.Count);
    for (int i = 0; i < original.Bars.Count; i++)
    {
        Assert.AreEqual(original.Bars[i].Weight, copy.Bars[i].Weight);
        Assert.AreEqual(original.Bars[i].Cost, copy.Bars[i].Cost);
    }
}

Even though this is an incredibly simple test using the “single assertion per test” theory would require writing 8 separate tests. Even worse, each additional property added to Bar would require three more tests.  These tests are not only time consuming to write, they are also more expensive to run and create an unnecessarily high maintenance burden.

Labeled Assertions

A far more effective means of ensuring that the right information is conveyed is to use the message parameter on the assertion.

[TestMethod()]
public void XamlParseCreateTest()
{
    var original = new Foo() { FooBar = "Sam" };
    original.Bars.Add(new Bar() { Weight = 5, Cost = 1.1M });
    original.Bars.Add(new Bar() { Weight = 10, Cost = 2.2M }); 
    original.Bars.Add(new Bar() { Weight = 15, Cost = 3.3M });

    var xaml = XamlUtilities.CreateXaml(original);

    var copy = XamlUtilities.ParseXaml<Foo>(xaml);

    Assert.AreEqual(original.FooBar, copy.FooBar, "String field was not copied");
    Assert.AreEqual(original.Bars.Count, copy.Bars.Count, "Collection count is wrong");
    for (int i = 0; i < original.Bars.Count; i++)
    {
        Assert.AreEqual(original.Bars[i].Weight, copy.Bars[i].Weight, string.Format("Integer property Weight at Bars[{0}] was not copied", i));
        Assert.AreEqual(original.Bars[i].Cost, copy.Bars[i].Cost, string.Format("Decimal property Cost at Bars[{0}] was not copied", i));
    }
}
Advertisements

1 Comment »

  1. I’ve found that labels are not entirely necessary especially when the label used as an error message doesn’t offer all that much more information as compared to the default error message. Granted it is annoying when you do hit an error and encounter a useless error message like “Expected a string of length 6 but was of length 4.”

    Although some do define a single assert as a single block of assert i.e. if you could refactor out the asserts into one method, then it still counts as a single assert. As opposed to having asserts, followed by some code, then some more asserts which counts as two assert blocks.

    Comment by xTRUMANx — October 8, 2011 @ 8:21 am


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: