TestDriven.NET by Jamie Cansdale

I've been doing some experiments to find out if it's possible to execute NUnit on a Silverlight project. I knew that Visual Studio 2008 and Expression Blend both host Silverlight for use in the designer window. It turns out that rather than host a separate instance of the CoreCLR, the designer simply loads the Silverlight assemblies into the host runtime. This simplifies the business of creating a compatible unit testing framework considerably!

With a little bit twiddling, I've managed to tweak the 'nunit.framework' assembly so that it's compatible with Silverlight projects. In order for your tests to execute successfully, you also need to ensure that all Silverlight assemblies (except 'mscorlib') are set to 'Copy Local: True'. After this is done you can run, debug and even do code coverage on your Silverlight unit tests!

Because creating one of these projects is a fiddle to set up, I've created Silverlight NUnit Project template that does the work for you. To install the template simply open the 'SilverlightNUnitProject.vsi' file and ignore the unsigned content warning. Once installed, you will find it under the 'Visual C# / Silverlight' section of the 'New Project' dialog. You can download the template from here.

 

To execute one or more unit tests you should use 'Run Test(s)' rather than 'Test With > Silverlight'.

In summary you now have 3 options when testing Silverlight projects.

1. For unit testing you can create a Silverlight NUnit Project and use the usual TestDriven.Net commands.
2. For integration testing (within the context of a browser) you can use the Silverlight Testing Framework.
3. For spelunking individual methods inside the CoreCLR you can use 'Test With > Silverlight'.

I'd be interested to hear how you get on.

The latest release of TestDriven.Net is now compiled against NUnit 2.4.7. This version of NUnit includes Andreas Schlapsi's popular RowTest extension. The RowTest attributes have been moved to the 'nunit.framework.extensions' assembly (which you can find here: \Program Files\TestDriven.NET 2.0\NUnit\2.4\nunit.core.extensions.dll).

Here is an example RowTest method that tests a prime number function:

The prime number function might look like this:

As you can see the intention of the test is very clear and there is no repetition. I've written more about parameterized unit tests here.

I've just uploaded a new version of TestDriven.Net with support for Silverlight 2.0 Beta 1. Microsoft have certainly kept me on my toes as there have been lots of changes since Silverlight 1.1. I'm sorry it has taken a while!

At the moment you're limited to running individual public methods (ad-hoc tests). If you need to run a suite of tests I recommend you use this in conjunction with the Silverlight Testing Framework that was released at MIX. Jeff Wilcox has posted a detailed tutorial that shows how to use the framework here. When running your tests using 'Test With > Silverlight', bear in mind that you're simply executing the test method and any test attributes (TestInitialize etc.) will be ignored.

I've also included an application called 'agx.exe' that lets you run console applications using the Silverlight/CoreCLR from the command line. After you've installed TestDriven.Net you will find this standalone application here: \Program Files\TestDriven.NET 2.0\agx.exe. This is simply an application that I use for my own testing purposes that I thought other people might find useful.

You can download the new version of TestDriven.Net from here.

TestDriven.Net has always supported parameterized test methods when used with the MbUnit testing framework. When using MbUnit, it is common for a single test method to execute multiple tests with different parameter inputs. The most famous of these test types is the MbUnit RowTest.

Until now there has been little reason to add support for executing parameterized tests using NUnit (historically NUnit has only supported parameterless test methods). However Andreas Schlapsi has recently written an implementation of MbUnit's RowTest using NUnit 2.4's Addin extensibility mechanism.

I've updated TestDriven.Net 2.11 to better support NUnit add-ins and enable the targeting of RowTests and other parameterized test types. This version also includes a workaround for a log4net related issue that was causing a noticeable delay when launching the NUnit 2.4 GUI. You can find the release notes for TestDriven.Net 2.11 here.

To install the RowTest Extension for NUnit you will need to do the following:

Download and install TestDriven.Net 2.11. Download the RowTest Extension for NUnit 2.4.5 (Binary). Create a directory called 'addins' in '%ProgramFiles%\TestDriven.NET 2.0\NUnit\2.4'. Copy the 'NUnitExtension.RowTest.AddIn.dll' file into the 'addins' directory (don't put any non-assembly files there). Add a reference to 'NUnitExtension.RowTest.dll' from your NUnit test project.

 

You can then start writing and executing MbUnit style RowTests inside your NUnit projects! You can find Peli's original RowTest example here.

To view your RowTests inside the NUnit GUI you will need to use 'Test With > NUnit 2.4'. You will find this option on the 'Solution Explorer' project context menu.

Thanks to Wayne Brantley for letting me know about the RowTest Extension for NUnit.

Over the past year the 'Go To Reflector' command has become a first class citizen inside TestDriven.Net. You will find the 'Go To Reflector' button on many different context menus inside Visual Studio. The ones I use most often during development are the 'Code Context' and 'Project Reference' menus. When I'm debugging I tend to use the 'Call Stack' and 'Modules' context menus.

For a long time I've put off attempting to add support for generics to the 'Go To Reflector' command. The Visual Studio CodeModel and StackFrames APIs don't really support generics, so I wasn't even sure if this would be possible. This was becoming a problem with more and more code being written that uses generics. I decided it was time to bite the bullet and see what could be done.

I'm happy to say that TestDriven.Net 2.10 now has pretty decent support for generics.

You can 'Go To Reflector' from your generic class definitions. Generic methods, classes, fields, properties and nested classes are all supported.

You can round trip and 'Go To Source Code' from inside Reflector. I often find using Reflector is the fastest way to navigate my own code.

 

When you're debugging you can 'Go To Reflector' from any stack frame in the 'Call Stack' window. This is particularly useful when the debugging option 'Just My Code' is turned off. When an exception is thrown you can quickly see what caused it by selecting the top of the call stack.

Note: For updated 'Go To Reflector' on 'Call Stack' support you will need to be using TestDriven.NET 2.10.2173 or later (I released this a few days after the original 2.10 build). You can read the release notes and download the latest version from here.

There are a number of new features in TestDriven.Net 2.10 that I want to highlight (apart from the VS 2008 crash workaround). The one I'm going to focus on here is subtle, but significant I believe - especially for people working with large solutions.

Smart build is a new optimization that allows you to skip the build step before test execution when there are no source code changes. Anyone working on a solution with a large number of projects will know how time consuming the build can be before any tests can be executed. Somewhat surprisingly this remains true even when no actual source code edits have been made and nothing needs to be compiled. I've had reports of the build check taking as long as 45 seconds before any tests could be executed! (Thanks to Brian Genisio in particular for bringing this to my attention)

The new smart build feature overcomes this particular problem by monitoring your solution for source code edits and automatically skipping the build step if there is nothing new to compile. This can significantly improve performance when running multiple tests in the same solution. For example there is now no penalty for choosing to execute all tests in a fixture after an individual test starts to pass: a common usage pattern.

I will elaborate further on other new features in future posts. In the meantime you can read the latest release notes and download TestDriven.Net 2.10 from here. If you find any issues, please don't hesitate to contact me.

Now that Visual Studio 2008 RTM is out I have started getting reports of Visual Studio crashing on startup when TestDriven.Net is installed. I was pretty dismayed as TestDriven.Net has included Orcas support since January of this year and this support has been pretty well tested. I certainly wasn't expecting to see any fatal crashes!

 

It turns out that this crash only happens when the user has opted not to install the 'Code Analysis Tools' feature. This is quite common when someone is doing a minimal install of Visual Studio with just the language feature they need.

The issue appears to be related to the Visual Studio command bars. If you simply query for a command bar that doesn't exist, Visual Studio will quit with an error (rather than throwing the expected ArgumentException)!

I'm happy to say there are now two possible fixes for the issue:

If you go into the Visual Studio 2008 Team Suite setup and enable the 'Code Analysis Tools' feature, you should find the crash goes away. I've just released TestDriven.Net 2.10, which includes a workaround for this issue. Please read the release notes and download it from here. If you're using Visual Studio 2008 I strongly encourage you to install this version.

Thanks to Aaron Jensen, Chad Myers and Dru Sellers for reporting the problem.

Peter Waldschmidt of NCover fame has recently pulled together a team of developers under a new company called Gnoso. They have been working in conjunction with Grant Drake on enhanced versions of NCover and NCoverExplorer (now combined into a single tool suite). These have been in development for many months and there are lots of new features and stability improvements. Rather than try to summarize here, I recommend you read what Grant and Peter have to say about the new NCover 2.0 and NCoverExplorer 2.0 versions.

If you would like to use NCover 2.0 in conjunction with TestDriven.Net, you will need to install TestDriven.Net 2.9 or higher. If NCover 2.0 isn't installed then TestDriven.Net will use the original NCover 1.5.8 and NCoverExplorer 1.4.8 versions. When NCover 2.0 is installed, TestDriven.Net will default to using the enhanced 2.0 versions. This version should also work if you are using TypeMock.

To perform code coverage on 64-bit Windows using TestDriven.Net, you will need to install NCover 2.0.2 Enterprise or higher. There was a compatibility issue with the original NCover 2.0 release, which prevented it from working when launched from TestDriven.Net. This is fixed in NCover 2.0.2.

Syndicated from Kiwidude's Geek Spot

I've had this "quietly available" on the download page for a few weeks to gain feedback on any issues and with no known issues outstanding it seems appropriate to more publicly announce it.

A big change in NCoverExplorer 1.4.0 is that it is no longer published under the GPL license. I made this change for a number of reasons; amongst which was the concern that a number of companies have of deploying GPL based tools into their development process. This roadblock has been removed and should no longer prevent your companies from using this toolset.

My original choice of GPL was based on using the ICSharpCode 1.x texteditor control. I have now dropped that component in favour of using the superior (but commercial) Actipro SyntaxEditor control instead. The improvements are not only in stability and features but also visual as you can see below in the code highlighting screenshot. A splitter window is available as also shown.

So why should you upgrade to NCoverExplorer 1.4?

This release is less about adding new features (although there are some to be found) and more about stabilisation for the future. The core model code has been rewritten and better tested. There have been countless code coverage corner cases (say that five times in a row!) arise that should be better catered for. Better performance for .NET 2.0 users, lower resource usage, better error handling and some minor gui quirks addressed.

Some of the "internal" changes have spilled over externally. For instance the configuration file format used by NCoverExplorer.Console with /c has changed to support new features, although I have retained legacy compatibility for those who choose not to upgrade their build scripts immediately.

What about the new features? Well, the ones I did squeeze in were...

Support for the //pm (profiling a process by name) argument in NCover 1.5.8; Copy command in the source code text area via right-click; Print preview command in the File menu; /fc argument for NCoverExplorer.Console to fail a build if the combined total coverage of all assemblies does not meet the minimum specified. This was the original behaviour of the /f command before it was changed to fail if any individual assembly does not meet the minimum; Coverage exclusions now support the '?' wildcard and multiple '*' wildcards within the expression. Report xml contains a "totalSeqPoints" attribute as requested for developers writing alternate xsl reports that need to calculate the excluded code at each level; Update the NAnt/MSBuild task dlls to support the latest changes; Redesigned the options dialog as shown below;

It appears that Jon Johansen (DVD Jon) is now a .NET developer. He has created a little .NET 2.0 application to activate your iPhone without giving any personal information to AT&T. The application runs as an activation server on your local machine. You then redirect albert.apple.com to 127.0.0.1 in your hosts file. You can find the activation server and source code here.

It has been over a month since my last update. I just want to put to rest any rumors of TestDriven.Net's early demise. With the help of my lawyers, I responded to Microsoft's lawyers' letter and am now attempting to come to a resolution of the issues. Continuing to play this out on the blogosphere would unnecessarily antagonize the situation and undo the good progress which has been made.

To be clear I am not anti-Microsoft. I am simply a passionate advocate of unit testing for developers of all experience levels. To my mind test driven development is like climbing with ropes. Beginner climbers can gain the confidence to climb without the fear of landing in a mess at the bottom. Expert climbers can take on overhangs and other hard problems that they wouldn't otherwise have the confidence to try. Climb without ropes if you like, but I am certainly not going to encourage it for anything outside your comfort zone.

To end on a happy note the Wells for Zoë charity that many of you supported last year through the .NET Developer's Charity Auction is about to enter a new phase. You can read about the pump workshop which is being set up in Mzuzu, Malawi later this year. There is also a photo blog which is being updated by our guy in the field (when the Internet works). Perhaps the people who suggested I put up a PayPal button to help with legal fees would prefer to save their money for something worthwhile and make a donation here instead. :-)

Jamie A comments to an article on Channel Register:

Here's a slightly less technical way of looking at the issue:

Microsoft essentially have 2 editions of a program. The first edition speaks English (User Interface) to the user, but in behind can speak French (COM) to other components.

The second edition also speaks English and French, but can also speak Russian (VSIP - Visual Studio Integration Programme).

Microsoft intends that your components speak Russian in order to extend Visual Studio. Their VSIP license agreement also states that you can only try to speak Russian to the second edition of the software. Attempting to speak Russian to the first edition is not allowed (even if it does speak the language).

Rather than try and speak Russian, Jamie has been using French. He has asked Visual Studio if it talks a certain dialect (COM interface), and it has said "oui". So he uses that dialect to talk to Visual Studio in order to do what he needs.

The main problem from Microsoft's point of view is that they didn't intend for anyone to use that dialect of French in order to talk to Visual Studio. They only expected Russian to be used. However, there's nothing that explicitly states this. Microsoft even has public notes on the French dialect.

Therefore, Jamie thinks he's in the right because Microsoft have not said that speaking French is forbidden. Microsoft sees it the other way. And the way things are going, it looks like the decision is going to come from a judge or jury.

There will be some French lessons in a follow up post.

Today Microsoft kindly extended my deadline by 24 hours so that I could further consult my lawyers. So far they have done a very good job of keeping the pressure on. This isn't simply a case of knocking out a quick email Pirate Bay style!

Before I released support for Visual Studio Express, I made a point of carefully reading the Express EULA. After reading it I was confident that the technique I was using wasn't in breach of the license. I am now acutely aware that if I turn out to be wrong I could end up losing everything. This isn't a situation that any developer wants to find themselves in.

Unfortunately the alternative to fighting this is bowing down to a vague catchall clause. I am an anxious person and don't want live in fear of this kind of thing.

I have just received another courier delivered letter from Microsoft's lawyers. That makes a total of 3 letters in 4 working days! I can see how these things can get expensive very quickly.

I'd like to highlight a couple of things in the letter.

They say that I have been in correspondence with Microsoft about these issues for many months. What they don't take into account is that in over a year of correspondence - Microsoft consistently refused to tell me which license I was allegedly in violation of.

For example on Feb 26, 2007 Jason Weber said:

Jamie, for the reasons we discussed at great length, we believe your
various extensions to the Visual Studio Express products necessarily
violated the relevant license terms. We don't think it's productive to
rehash those discussions.

We may have discussed this at great length, but I was never told what that the "relevant license terms" actually were! I only re-enabled Express support when Microsoft yet again failed to tell me where I was in violation. A straight answer with something I could tell my users would have resolved this.

I'm not sure where on my website this was suggested. It's possible that they're referring  to one of the comments on the last post. They do however bring up an interesting point. The license attached to their first letter was the one for "Visual Studio 2005 Standard and Academic Editions". It didn't matter that the license wasn't the Express SKU license because the wording is the same. What if it turns out that the reason I can't add buttons to Express SKU also applies to Visual Studio 2005? I would then be forced to take down TestDriven.NET entirely. What if it also means I can't use PopFly Explorer for Visual Studio Express? :-(

There are a few things I need to get off my chest.

When I started working on TestDriven.Net I was a hobbyist .NET developer. My day job was as a Java consultant. TestDriven.Net was something I did for fun between consultancy gigs. Working on open source projects can be a great way to learn a new technology. I enjoyed being part of the fledgling .NET community and positive reception TestDriven.Net (then NUnitAddIn) was getting.

When TestDriven.Net 1.0 was released I was still hobbyist .NET developer. It was only natural that I use the Visual Studio Express editions which were being targeted at other hobbyist developers. In fact I developed the whole of TestDriven.NET 1.0 using C# Express, MSBuild and WiX (as described in this post).

A few months after TestDriven.Net 1.0 was released Microsoft made me an MVP. According to Ben Miller (my then MVP lead) I was "very well known" for having created TestDriven.Net. As far as I'm aware this was the primary reason I received the award.

On Dec 1, 2005 I received an email from Jason Weber the lead for the Visual Studio IDE and Visual Studio SDK. Apparently Jason wanted to better understand my product and strategy. It was clear from the email subject that Jason incorrectly assumed TestDriven.Net was a VSIP Package. The interesting thing about VSIP packages being that they require a special key from Microsoft in order to function.

This is where the story begins. I'm not going to summarize what happened. I'm going to give you all the information so you can make up your own mind.

The first set of emails are between Dec 1, 2005 and Mar 30, 2006. They culminate in a teleconference between Craig Symonds (the General Manager for Visual Studio), Grant Drake, Jason Weber (who doesn't say a word) and myself. The second set of emails are between March 31, 2006 and Apr 17, 2007. They culminate with Jason finally letting me know which license I'm supposedly in breach of (the Express EULA). I'm still none the wiser about which clause. Finally I receive two letters (delivered by motorcycle courier) from Microsoft's UK lawyers. For the first time ever I am told which exact clause I'm supposedly in breach of (allegedly I'm working around some unspecified limitation). The second letter lets me know that they are reading my weblog and the TestDrivenUsers group.

Less than a year ago I was still a hobbyist .NET developer. I created TestDriven.Net as a tool for myself and something that I hoped other .NET developers would find useful. I have no intention of selling out and giving in to this kind of petty bullying.