In a previous post, I explored an idea of using Microsoft's built-in T4 code-gen framework to generate BDD-style tests from a simple text specification. Since then, I've improved the T4 some and also reevaluated the idea some. First, let's look at the updated code-gen. My goals from the last post were:
- A very simple, 'traditional' class where the test writers 'fill in the blanks'.
- intellisense to help discover the methods to implement or override
- throw a 'Not Implemented Exception' for any feature not yet implemented.
- the nice HTML report MSpec generates.
To that, I also decided it would be nice to update the tests whenever the spec was updated. I ended up dropping the HTML report, but adding this feature. There are two ways I can think of to do this. One would be to maintain a "generated" class, which the tests would inherit and override test methods from. This, though, has the downside of requiring more typing from the developers. Since NUnit doesn't run [Test]s from a base class, developers would have to essentially re-write the entire base class, overriding each test method and marking them with the test attribute, defeating the purpose of code generating altogether. The second approach is to create a generated class, and update it with test methods as needed. This is tricky, but doable, and the approach I ended up taking. Let's look at some code.
The spec, as last time:
Transferring between from account and to account
*Should debit the from account by the amount transferred
*Should credit the to account by the amount transferred
Since T4 now needs to read the spec and either a) create a new file or b) update an existing one, it becomes more complex. I broke it up into a couple of methods for generating the code:
<#@template language="C#" hostspecific="True" #>
<#@import namespace="System.Collections.Generic" #>
<#@import namespace="System.IO" #>
<#+
void SaveOutput(string outputFileName)
{
string templateDirectory = Path.GetDirectoryName(Host.TemplateFile);
string outputFilePath = Path.Combine(templateDirectory, outputFileName);
string output = this.GenerationEnvironment.ToString().Trim();
File.WriteAllText(outputFilePath, output );
this.GenerationEnvironment.Remove(0, this.GenerationEnvironment.Length);
}
void UpdateSpecFile(string path, StreamReader specReader)
{
string fileText = File.ReadAllText(path);
int specInsertionPoint = fileText.IndexOf("//Additional specification methods go here.");
if (specInsertionPoint == -1)
{
#><#= fileText #><#+
SaveOutput(path);
return;
}
else
{#><#= fileText.Substring(0, specInsertionPoint)#>
<#+
}
bool endOfSpec;
string line = "";
do{
line = specReader.ReadLine();
endOfSpec = (line == String.Empty || line == Environment.NewLine || specReader.EndOfStream);
string methodName = line.Replace(" ","_").Replace("*","").Replace("-","");
if (!endOfSpec && !fileText.Contains("public void " + methodName+ "()")){#>
[Test]
public void <#=methodName#>()
{
//TODO: Write code to perform the work indicated by "<#=line#>"
//For example:
//Assert.AreEqual(this.toAccount.Balance, 120);
throw new NotImplementedException();
}
<#+
}
}while(!endOfSpec);
#><#= fileText.Substring(specInsertionPoint)#><#+
SaveOutput(path);
}
void GenerateAllSpecs()
{
DirectoryInfo currentDir = new FileInfo(Host.TemplateFile).Directory;
FileInfo[] specFiles = currentDir.GetFiles("*.spec");
foreach(FileInfo spec in specFiles){
StreamReader reader = spec.OpenText();
bool eof = false;
string currentClassName = "";
bool endOfSpec = false;
string currentFileFullPath = "";
do{
string line = reader.ReadLine();
eof = reader.EndOfStream;
if(!String.IsNullOrEmpty(line) && !line.StartsWith("*") && line != Environment.NewLine){
currentClassName = line.Replace(" ","_");
currentFileFullPath = currentDir.ToString() + "/" + currentClassName + ".cs";
//If source control has this locked, skip it.
if (File.Exists(currentFileFullPath) && new FileInfo(currentFileFullPath).IsReadOnly) continue;
if (File.Exists(currentFileFullPath)) {
//If the file exists and isn't locked, update it.
UpdateSpecFile(currentFileFullPath, reader);
}else if (!File.Exists(currentFileFullPath)){
//If the file doesn't exist, create it
#>using System;
using NUnit.Framework;
//From <#=spec.Name#> Specification
[TestFixture]
public partial class <#=currentClassName#>{
[SetUp]
public void Setup()
{
InitializePreTestContext();
Execute_<#=currentClassName#>();
}
public void InitializePreTestContext()
{
//TODO: Set up any fields used by this specification.
//For example:
//this.toAccount = new Account(100);
//this.fromAccount = new Account(100);
}
public void Execute_<#=currentClassName#>()
{
//TODO: Write code to perform the work indicated by "<#=line#>"
//For example:
//this.toAccount.TransferFrom(fromAccount, 20);
throw new NotImplementedException();
}
<#+ do{
line = reader.ReadLine();
endOfSpec = (line == String.Empty || line == Environment.NewLine || reader.EndOfStream);
if (!endOfSpec){#>
[Test]
public void <#=line.Replace(" ","_").Replace("*","").Replace("-","")#>()
{
//TODO: Write code to perform the work indicated by "<#=line#>"
//For example:
//Assert.AreEqual(this.toAccount.Balance, 120);
throw new NotImplementedException();
}
<#+ }
}while(!endOfSpec); #>
//Additional specification methods go here. If you delete this line, then code gen will no longer be able to update this file.
}
<#+
SaveOutput(currentFileFullPath);
}
} //if ! startswith *
}while(!eof);
}//foreach
} //GenAllSpecs #>
This code reads the .spec and generates unit tests, but works differently in that it renders the output to a file instead of relying on the Visual Studio tooling to output the file. If the file exists, it checks that it isn't read-only and looks for a comment "//Additional specification methods go here". If it finds it, it inserts any new test methods in that space. If the method already exists, then it is not overwritten. Because of the way T4 works, these methods have to be called from a second tt file:
<#@ include file="SpecsToTests.tt" #>
<# GenerateAllSpecs(); #>
The effect is that the code gen works to keep the test class up-to-date with the spec, but doesn't overwrite any of the developer's code. The end result looks like this:
using System;
using NUnit.Framework;
//From Account.spec Specification
[TestFixture]
public partial class Transferring_between_from_account_and_to_account_with_balance_of_100
{
[SetUp]
public void Setup()
{
InitializePreTestContext();
Execute_Transferring_between_from_account_and_to_account_with_balance_of_100();
}
public void InitializePreTestContext()
{
//TODO: Set up any fields used by this specification.
//For example:
//this.toAccount = new Account(100);
//this.fromAccount = new Account(100);
}
public void Execute_Transferring_between_from_account_and_to_account_with_balance_of_100()
{
//TODO: Write code to perform the work indicated by "Transferring between from account and to account with balance of 100"
//For example:
//this.toAccount.TransferFrom(fromAccount, 20);
throw new NotImplementedException();
}
[Test]
public void Should_debit_the_from_account_by_the_amount_transferred()
{
//TODO: Write code to perform the work indicated by "*Should debit the from account by the amount transferred"
//For example:
//Assert.AreEqual(this.toAccount.Balance, 120);
throw new NotImplementedException();
}
[Test]
public void Should_credit_the_to_account_by_the_amount_transferred()
{
//TODO: Write code to perform the work indicated by "*Should credit the to account by the amount transferred"
//For example:
//Assert.AreEqual(this.toAccount.Balance, 120);
throw new NotImplementedException();
}
[Test]
public void Should_add_log_to_transaction_history()
{
//TODO: Write code to perform the work indicated by "*Should add log to transaction history"
//For example:
//Assert.AreEqual(this.toAccount.Balance, 120);
throw new NotImplementedException();
}
[Test]
public void Should_update_last_transaction_date_on_both_accounts()
{
//TODO: Write code to perform the work indicated by "*Should update last transaction date on both accounts"
//For example:
//Assert.AreEqual(this.toAccount.Balance, 120);
throw new NotImplementedException();
}
//Additional specification methods go here. If you delete this line, then code gen will no longer be able to update this file.
}
All that said, I've started to rethink the benefit of this some. It's cool, but I'm not sure how much it actually saves over just writing the class from scratch. Especially with some code templates and ReSharper, it just wouldn't take much more effort to write the tests than it does to write the .spec and then generate the code. Then again, the .spec is something you could type up in a meeting with the product owners, so you may as well generate some code from it. So, I'm still on the fence about the actual usefulness of this, but it's been worth it for the T4 learning experience alone. At this point, I'm going to wait for a project it makes sense to try this on before going much further with it. But do let me know if you have any questions or suggestions for this, or if you do something similar for your organization!
