We use the ReSharper test runner here at GojiSoft to run NUnit tests from within Visual Studio. It’s a great test runner, but doesn’t play nicely with multi-threaded components where exceptions may occur on non-test threads. Unhandled exceptions on non-test threads are hidden and tests which should fail, instead pass. Here’s a very very contrived example:

[Test]
public void NonTestThreadException()
{
    Thread thread = new Thread(obj => { throw new Exception("Oops..."); });
    thread.Start();
    thread.Join();
}

If an exception isn’t raised on the test thread, the test passes – Oops!

The problem lies in the fact that the ReSharper test runner is configured to behave in the same way as .Net 1.0 and 1.1 apps where unhandled exceptions on non-main threads were swallowed. The situation improves from .Net 2.0 where all unhandled exceptions flatten the process. However, Microsoft had to allow existing .Net 1.0 and 1.1 apps the option of behaving as before on new .Net frameworks. They introduced the app.config setting: legacyUnhandledExceptionPolicy.

The ReSharper test runner is configured by default to use the .Net 1.0 and 1.1 policy, so if there is an unhandled non-test thread exception it does not bubble up and cause the test to fail – the test passes, and we get a false positive instead.

If unhandled exceptions on non-test threads should fail tests, the app.config for the ReSharper test runner has to be updated.

Configuring the test runner to fail on unhandled thread exceptions

1. Firstly find out which test runner executable is actually being used. Run a test in debug mode and look in the Visual Studio output window:

2. Locate the executable’s app.config file, e.g.:

C:\Program Files\JetBrains\ReSharper\v5.0\Bin\JetBrains.ReSharper.TaskRunner.CLR4.MSIL.exe.config

3. Turn the legacy unhandled exception policy off by editing legacyUnhandledExceptionPolicy:

<legacyUnhandledExceptionPolicy enabled="0" />

4. Now multi-threaded tests fail as expected when they raise exceptions on non-test threads:

Buyer beware…

There is a caveat to this. Exceptions on non-test threads will now flattened the test runner and test suite execution will be halted when they happen. This is in contrast to normal test runs where failed tests are marked as failed, and the test runner continues. A pain – annoying? Maybe, but failed multi-threaded tests should be a pain and fixed ASAP. Of course, if they’re not your tests, you can always delete them from the test runner session - move along, nothing to see here, dum-de-dum-de-dum…

One Response to “ReSharper test runner – hidden thread exceptions”

1. Gleb says:

   November 1, 2011 at 10:15 am

   Thanks for the post, it’s very useful

   Reply