Specific output header for each test case generated by a Theory

Topics: ReSharper test runner
Jan 20, 2010 at 1:06 PM
Edited Jan 20, 2010 at 1:11 PM

Hi,

When running a Theory test method (which generates one or more test cases), it is usually necessary to have specific output which indicates which InlineData attribute was used for each test case (i.e. which parameters were supplied). Otherwise it is very difficult to tell which case(s) failed, in the situation where some cases fail and some pass.

Unfortunately when I run such tests with Resharper, the output window (in the lower half of the R# unit test window) does not show which particular test case is running.

I'm not sure if this is something that the contrib project can provide or whether it is a limitation on the Resharper unit test GUI.

When running such tests with TestDriven.net, the information is available in both the VS Output and Error List windows.

Regards
Adam

Jan 28, 2010 at 8:48 AM

Hi,

I am missing this feature, too!
It would be great, if this will be implemented in future versions.

Regards,
Ramon

Mar 2, 2010 at 9:42 PM

I'm able to run the test but I don't receive any test results in the results window.  Any ideas on what i'm doing wrong?

Photobucket

Mar 3, 2010 at 6:57 AM

Your test results are in the top half of the window. As you can see, they all passed.

The bottom half of the window represents the console output. If you explicitly write message to the console, they get written here. Also, if a test fails, the output of the assertion/verification exception is also displayed here.

The issue I have raised is that neither xUnit.net contrib/Resharper print out details of the parameters passed when running Theory+InlineData test methods, as TestDriven.net does. This means is it is impossible to see which exactly which test case failed since the console output (bottom half of the window) only show the assertion exception details and nothing else.

Mar 3, 2010 at 3:58 PM
Given the architecture of the Resharper test runner (i.e., its API assumes that everything works exactly like NUnit), this may be a limitation that you'll be forced to live with.
Mar 4, 2010 at 7:28 AM
Edited Mar 4, 2010 at 7:29 AM

@Brad - I was fearing this. Perhaps someone can come up with something clever to get round it though ;-)

For the time being, whenever I'm running a Theory test, I am writing details of all parameters to the console.

However, this only seems to work when the test passes, i.e. no assertion exception is thrown. If an assertion exception is thrown then the console output seems to be suppressed (the call to Console.Write is made before the assertion) so whilst it does help in that I can see which cases passed, it's far from ideal. I wonder if this is something which can be fixed by xunit.net contrib or whether it is also a R# limitation?

Mar 4, 2010 at 2:50 PM
This (the suppressed console output) is a bug in xUnit.net, not in the Contrib project. It's fixed in the tree already, and will be in the next drop of xUnit.net. You can do a private build now if the issue is pressing, or wait for the next release.

Mar 4, 2010 at 5:05 PM

@Brad - thanks for the info. The suppression of console output is not pressing so I'm happy to wait for the next release.

For anyone reading this discussion, this is a side issue and does not resolve the original issue which I raised with this discussion. If anyone has any comments regarding the main issue (see above) please post them.