Tuesday, October 20, 2009

DataWings: Convention over Configuration

I've just released some new functionality for the increasingly popular (?) framework for data driven integration testing: DataWings. This new functionality aims at letting you register conventions as to how your domain entities map to the database tables, and in this way you can declare your assertions in a much more concise and elegant manner. The purpose is to reduce the amount of ceremony needed to execute the assertions.

Get the bits here.

An example:

The Old Way:

   1:  [Test]


   2:  public void CreatePersonInTransaction_ScopeCompleted_PersonExistsInDatabase()


   3:  {


   4:      Person person;


   5:      using (var scope = new TransactionScope())


   6:      {


   7:          person = new Person{ Id = Guid.NewGuid() };


   8:          IoC.GetInstance<IDomainObjectProvider>().Save(person);


   9:          scope.Complete();


  10:      }


  11:      // Assertion using DataWings


  12:      // The old fashioned way


  13:      DbAssert.ForTable("PERSON")


  14:          .WithColumnValuePair("ID", person.Id)


  15:          .Exists();


  16:  }






The New Way (using conventions):





   1:  [Test]


   2:  [DbTableNameConvention(DbTableNameConventionType.ClassNameEqualsTableName)]


   3:  [DbIdConvention("ID")]


   4:  public void CreatePersonInTransaction_ScopeCompleted_PersonExistsInDatabase()


   5:  {


   6:      Person person;


   7:      using (var scope = new TransactionScope())


   8:      {


   9:          person = new Person{ Id = Guid.NewGuid() };


  10:          IoC.GetInstance<IDomainObjectProvider>().Save(person);


  11:          scope.Complete();


  12:      }


  13:      // Assertion using DataWings


  14:      // The new way with conventions


  15:      DbAssert.Exists(person);


  16:  }









All conventional assertions (such as the one on line 15) are available as extension methods, so that the code on line 15 can be replaced with this code:



person.AssertExistsInDatabase();


Attribute based configuration



Notice how the assertion on lines 13 to 15 in the “old way” is replaced by just a single line (number 15) in the sample using conventions. In order to use this concise notation, DataWings will have to know how to map the object to its corresponding table, i.e. what the conventions are. These conventions are specified with the help of attributes, and in the sample above you can see how the two attributes DbTableNameConvention and DbIdConvention set up such conventions.



Such attributes may be put on methods or classes, the functionality walks the stack looking for an attribute to use. This is done by examining the method and class of each stack frame until a suitable attribute decoration is located. If no suitable attribute can be found, the conventional convention (see below) will be used.



 



The conventional conventions



You can start using the conventions functionality immediately, if you accept the conventional conventions, that is. And these conventions are:




  • Table name equals name of class


  • Name of unique key column in table is on the format Id[ClassName]



Registering conventions for mapping class to table



The convention for how the class maps to a database table is specified through the usage of DbTableNameConvention. The following example show how to use this attribute:



Class name and table name match exactly



[DbTableNameConvention(DbTableNameConventionType.ClassNameEqualsTableName)]


Class name as part of the table name



[DbTableNameConvention(DbTableNameConventionType.Custom, Convention = "TBL_{0}")]


Overriding for specific classes



The attribute also has a property EntityType, and this is used in cases where the conventions for a specific class do not match the conventions of the other entities that are in play in the test.



[DbTableNameConvention(DbTableNameConventionType.Custom, Convention = "TBL_{0}", EntityType = typeof(Address))]

[DbTableNameConvention(DbTableNameConventionType.ClassNameEqualsTableName, EntityType = typeof(Relation))]



Registering conventions for mapping primary key



The convention for how the primary key of the table is  mapped to a property of the class is specified by using DbIdConvention. Some examples:



Name of primary key column is identical for all tables



[DbIdConvention("ROWID")]



Class name is part of primary key column name



[DbIdConvention("Id{0}")]

Thursday, May 28, 2009

DataWings – Data driven integration testing





Yes, I’m now officially an open source contributor, and the project’s even got it’s logo, so you know it’s gotta be good.



So what we have attempted to do is to make a lightweight, easy to use, no set up tool to be used when testing code that sits on top of a database. With this tool, DataWings, you can set up the database so that your tests are accessing known data and assert that the database is in the expected state after the tests have executed. And all of this is done directly in the test code.



Get the bits her.



Here’s a more detailed description of what the tool does, and how it works:



First, a word of caution



DataWings is designed to be a tool to be used at design time and during testing. No attention has been paid to security issues, and we definitely do not recommend using this code in production.



Configuring the connection string



The first thing you need to do when using DataWings is to configure the connection string(s) to be used. The functionality for doing this is purposefully designed with two goals in mind: a) making it easy to set up the connection string in code in order to "get going" as fast as possible, and b) making is easy to maintain the connection string outside of code thus helping to ensure that the test will remain operative in the future.



The connection string is set by decorating either the class or method with an attribute. There are several different kinds of attributes thata can be used, but here we'll focus on ConnectionStringFromConfigFile. As the name implies, this attribute is used when the connection string is registered in the standard <connectionStrings> section of the configuration file. A typical usage of this attribute might look like this:



[ConnectionFromConfigFile(SqlVendor.Oracle, Key = "MyConnection", AlternativeConnection = "TheConnectionString")]


Here, the sql vendor (input to the constructor) dictates which ADO.NET provider that will be used behind the scenes, the Key property specifies the name of the connection string in the configuration file to use, and the AlternativeConnection property is set with the specified connection string is not found in the configuration file or if any other problem is detected while trying to look up this string.



How the attribute is resolved



As mentioned, the connection string attribute can be used to decorate both methods and classes. The process of resolving which attribute to be used is carried out by walking the stack looking for an appropriate attribute. The algorithm first looks at the executing method of the stack frame, and if this method does not have an appropriate decoration, the class of the executing method is examined. This process continues for each stack frame until a decoration is found. If no such decoration can be located, an exception is raised.



Named connection string



The ConnectionFromConfigFile attribute also has a property called Name. This property is useful in situations where the tests touch more than one database. All the static gateway classes into the DataWings functionality (DataBoy, DbAssert and Adversary - see below) have a ForConnection() method, through which the named connection attribute to be used can be specified.



Here's a sample of such a named decoration:



[ConnectionFromConfigFile(SqlVendor.Oracle, Name="Default", Key = "MyConnection", AlternativeConnection = "TheConnectionString")]


DataBoy



Standard usage



DataBoy provides functionality for keeping the data in the database in a consistent state so that the tests are running against known data . This sample shows the standard usage of DataBoy.



DataBoy
.ForTable("Person")
.Row("IdPerson", 1).Data("Surname", "Obama").DeleteFirst()
.Row("IdPerson", 2).Data("Surname", "Bush").DeleteFirst()
.ForTable("Address")
.Row("IdAddress", 100).Data("Street", "Main street").DeleteFirst()
.Commit();


Internally DataBoy keeps track of changes in a session and this session resides only in memory until the Commit() method is invoked. The table to insert data into is specified with the ForTable() method, and this table will be the one that is used for all subsequent row specifications until another call to ForTable() is encountered.



Each row to be inserted is marked by the Row() method, and this method takes the column name and the value the (presumably) uniquely identifies the row as input. The row can receive the DeleteFirst() method, and if this method has been invoked, DataBoy will delete the row from the database before it is inserted.



Order of execution



The rows of the session are traversed twice, first from back to front and then from front to back. In the first parse (backwards), any deletions are performed (i.e. rows marked with DeleteFirst()) while the insert are performed in the second traversal. In this way it should largely be possible to order the statements so that any errors due to foreign key constraint violations are avoided.



Updating instead of deleting



By invoking the ForUpdate() method an update statement (instead of an insert) will be generated and invoked the row in question. Example:



DataBoy.ForTable("Person").Row("IdPerson", 1).Data("Surname", "Obama").ForUpdate().Commit();




Just deleting





Rows can be deleted through the usage of the ForDelete() method. Example:



DataBoy.ForTable("Person").Row("IdPerson", 1).ForDelete().Commit();


Executing custom queries





The ExecuteNonQuery() method supplies a way to invoke custom queries directly against the database:



string sqlQuery = "INSERT INTO Address (IdAddress, Street) VALUES (99, 'Some Street')";
DataBoy.ExecuteNonQuery(sqlQuery).Commit();


 


DbAssert



The static class DbAssert is for asserting that data in the database is in the expected state. This class is quit similar to the familiar Assert class of many unit test frameworks.



In order to set up an assertion you first need to specify which table you are testing against. As with DataBoy, this done through the ForTable() method. When the table has been specified we need to tell the framework which row we are interested in, and this is accomplished with the method WithColumnValuePair(). This method takes a column name and corresponding value as input, and generally this will be the column name of primary key and the primary key for the row of interest. If more than one row exist for this column value pair, the first row encountered (randomly) will be used.



When the assertion has been set up, the actual assertion can be specified:



Exists(), NotExists()





Determines whether the row in question exists at all. Example:





string columnName = "Surname";
string columnValue = "Obama";
DbAssert
.ForTable("Person")
.WithColumnValuePair(columnName, columnValue)
.Exists();




AreEqual()





Determines whether the value in the specified row equals the specified value. Example:



DbAssert.ForTable("Person")
.WithColumnValuePair("IdPerson", 1)
.AreEqual("FirstName", "Barack");


Evaluate()



This method returns the entire row, and you can perform arbitrarily complex tests on the values of this row by using a lambda expression. Example:





DbAssert.ForTable("Person")
.WithColumnValuePair("IdPerson", 1)
.Evaluate(row =>
row.GetResult("FirstName") == "Barack" &&
.row.GetResult("IsPresident") == true);




Adversary



Adversary is a static gateway class providing functionality for provoking conflicts in optimistic concurrency scenarios. This code is still in a very early phase, and hopefully will mature in the future



Sql provider provisioning and built in providers



DataWings natively support SQL Server and Oracle database engines by using the System.Data.SqlClient and System.Data.OracleClient of the .NET framework. Additionally, DataWings supports SQLite through the separate assembly DataWings.SQLite. This support for SQLite has been realized by use of the built-in provider provisioning infrastructure. This is a model where you can develop support for your favorite database engine by implementing two simple interfaces. Hopefully, I'll be able to get into more details about this at a later stage.

Wednesday, May 13, 2009

Fluent Castle Windsor and Configured Parameters

Castle Windsor version 2.0 has just been released (despite the fact that version 1.0 has never existed). The biggest new feature in this release is the fluent configuration interface which lets you set up your components in code in an elegant way (as opposed to configuring in xml).

A component which in xml is set up like this:

<component
id="service"
lifestyle="transient"
service="Some.Namespace.IService, MyAssembly"
type="Some.Namespace.Service, MyAssembly">
<parameters>
<lang>Norwegian</lang>
</parameters>
</component>


can now be configured fluently like this:



var container = new WindsorContainer();
container.Register(
Component
.For<IService>()
.ImplementedBy<Service>()
.LifeStyle.Transient
.Parameters(Parameter.ForKey("lang").Eq("Norwegian")));


For more examples of how the fluent interface works, read this.



I am really enjoying the experience of using this new fluent interface; it is much easier to configure a component first time, and you get the full support of your compiler and from ReSharper. At last I am able to rename classes (through the refactoring functionality in ReSharper) without having to hunt down and fix the configuration for the component in the xml file.



Configuring parameters



Notice the parameter lang in the example above; the value Norwegian is hardwired directly in as a parameter (is presumably injected into the component's constructor). In the real world you would probably want the keep track of all such properties separately in the properties node, thus promoting reuse and easing maintenance. Your xml configuration might look like this:



<properties>
<language>Norwegian</language>
</properties>
<components>
<component
id="service"
lifestyle="transient"
service="Some.Namespace.IService, MyAssembly"
type="Some.Namespace.Service, MyAssembly">
<parameters>
<lang>#{language}</lang>
</parameters>
</component>
</components>


There is a tension between the benefits and drawbacks of having the configuration in code as opposed to in separate xml files: in code it is easier to manage the configuration while under development, while xml-based configuration supports easy changes to a system that has already been deployed. I feel that the smartest path would be to configure the components in code, but keep the properties defined in xml.



I am currently retrofitting fluent castle configuration on a relatively large application that is totally castle.windsor based. My initial gut feeling was that this new version of castle as a matter of course supported this "smartest path". Unfortunately, it doesn't, and so I was left to my own devices: enter ConfiguredParameter.



ConfiguredParameter



With the ConfiguredParameter functionality in place, I am able to configure my component as follows:



Properties in xml:



<castle>
<properties>
<language>Norwegian</language>
</properties>
</castle>


Component in code:



var container = new WindsorContainer();
container.Register(
Component
.For<IService>()
.ImplementedBy<Service>()
.LifeStyle.Transient
.Parameters(ConfiguredParameter.ForKey("lang").GetValue("language")));


To make it clear, the line ConfiguredParameter.ForKey("lang").GetValue("language") will look up the value for the configured property language, and this value will be injected into the component at lang (which presumably is a parameter in the constructor of the type Service).



The functionality must be bootstrapped as your application starts (before your container is initialized), and this will typically be accomplished like this:



InitializeConfiguredParameters.Initialize();


Here the application configuration file will be parsed, and any additional castle configuration files (included through the use of the <include> element of castle) will be included. There is a single overload to the Initialize() method where you can explicitly indicate which config file (.config or .xml) to parse, but this is mostly useful in testing scenarios.



This functionality is rather simple, and the implementation consists of just two types in addition to a couple of parser classes responsible for the actual parsing of the configuration files. These two types are ConfiguredParameter and InitializeConfiguredParameters. Below you will find the definition of the two types.



On the Road Map



The ability to use configured key value pairs from the appSettings element of the application configuration file might be nice, and I'll implement it whenever I need it.





The Code



using System;
using System.Collections.Generic;
using System.Configuration;
using Castle.MicroKernel.Registration;

/// <summary>
/// Used to access parameters that are configured within a standard
/// castle.windsor properties element
/// </summary>
public class ConfiguredParameter
{
#region Static API

private static readonly IDictionary<string, string> configuredParameters = new Dictionary<string, string>();
private static readonly object syncLock = new object();

/// <summary>
/// Adds each parameter in the incoming dictionary to the internal
/// cache of configured parameters
/// </summary>
/// <param name="parameters">The parameters.</param>
internal static void AddParameters(IDictionary<string, string> parameters)
{
// Thread safe!
lock (syncLock)
{
foreach (var pair in parameters)
{
// Skip if already contained, assuming that
// it's some kind of race condition. So, if
// the configuration contains two or more
// identical keys, one of them will "win"
// unpredictably
if (!configuredParameters.ContainsKey(pair.Key))
{
configuredParameters.Add(pair);
}
}
}
}

/// <summary>
/// Resets the ConfiguredParameter infrastructure by clearing all loaded
/// configured parameters. NB! This method should normally not be invoked,
/// and it is defined mostly for testing purposes.
/// </summary>
public static void Reset()
{
configuredParameters.Clear();
}

/// <summary>
/// Sets the name of the parameter on a new instance of
/// ConfiguredParameter and returns it
/// </summary>
/// <param name="parameterKey">The key.</param>
/// <returns></returns>
public static ConfiguredParameter ForKey(string parameterKey)
{
if (configuredParameters.Count == 0)
throw new InvalidOperationException("ConfiguredParameter infrastructure not initialized.");
return new ConfiguredParameter(parameterKey);
}

private static string GetVal(string key)
{
try
{
return configuredParameters[key];
}
catch (KeyNotFoundException e)
{
string message = String.Format("No configured parameter named {0} can be found", key);
throw new ConfigurationErrorsException(message, e);
}
}

#endregion

private readonly string parameterKey;

/// <summary>
/// Initializes a new instance of the <see cref="ConfiguredParameter"/> class.
/// </summary>
/// <param name="parameterKey">The parameter key.</param>
private ConfiguredParameter(string parameterKey)
{
this.parameterKey = parameterKey;
}

/// <summary>
/// Returns a Parameter with the value at the propertyKey in the
/// castle configuration
/// </summary>
/// <param name="propertyKey">The property key.</param>
/// <returns></returns>
public Parameter GetValue(string propertyKey)
{
return Parameter.ForKey(parameterKey).Eq(GetVal(propertyKey));
}
}


using System.IO;
using System.Reflection;

/// <summary>
/// Resposible for initializing the ConfiguredParameter functionality
/// by getting hold of and parsing any relevant configuration files
/// containing castle.windsor parameters
/// </summary>
public static class InitializeConfiguredParameters
{
/// <summary>
/// Initializes this instance by getting hold of the application's
/// configuration file (app.config or web.config) and parsing it
/// looking for configured parameters. If the castle configuration
/// of this file contains include elements, the castle files referenced
/// in these elements are also parsed.
/// </summary>
public static void Initialize()
{
string configFile = Path.GetFileName(Assembly.GetEntryAssembly().Location) + ".config";
if (File.Exists(configFile))
{
InitializeWithFile(configFile);
}
}

/// <summary>
/// Initializes the with file. Valid file types are application files
/// (app.config or web.config) as well as stand alone castle config
/// files
/// </summary>
/// <param name="filename">The filename.</param>
public static void InitializeWithFile(string filename)
{
ReaderBase reader;
if (Path.GetExtension(filename).ToLower() == ".config")
{
reader = new ConfigFileReader(filename);
}
else
{
reader = new PropertiesReader(filename);
}
ConfiguredParameter.AddParameters(reader.GetConfiguredProperties());
}
}

Tuesday, March 31, 2009

My First PowerShell Script

For a while now, I've been planning on getting my hands dirty using  PowerShell. There are at least four features that make this a pretty compelling scripting environment:

  • The ability to write scripts combining regular scripting commands  with the full power of the .net base class library (also including Cmdlets that you implement youself).
  • The ability to easily define functions as a part of the script.
  • The Cmdlet Verb-Noun naming convention, giving the scripting syntax a consistent and easy to discover feel that is completely missing in the jungle of cryptic abbreviated commands of the scripting environments of yore.
  • Everything is object based, so that when you, for instance, loop through all files in a directory by using the built in Get-ChildItem function, you are in fact accessing objects that represent the files and not just a textual path.

I hereby announce that I have completed my first PowerShell script (see below).

The good thing about being a latecomer is that you get to use the newest version, so I went directly for PowerShell 2.0 CTP3. A very nice thing about this version is that it comes complete with its own Integrated Scripting Environment, and this has been tremendously helpful in the process of understanding the basics and weeding out bugs.

So, what does my very first PowerShell script do? It recursively copies the contents of one directory to another one. As input to this process it takes a list of directory matching filters and a similar list of file extensions to be used as a filter. All in all this serves the purpose of copying the entire contents of a directory will keeping away from certain paths and files as dictated by the filters.

So by giving in this: "_ReSharper","\obj","\bin", "\.svn" as directory filter and this: ".user", ".suo", ".resharper" as file extension filter, I get functionality for copying .NET source code directories without also copying all the crud that is lying around as a by-product of the VS build process, SubVersion, ReSharper and so on.

I guess that everyone with some PowerShell experience will view this script as childishly amateurish, but at least it works .

Your welcome!


function Passes-Filter($dir, $filters) {
foreach($filter in $filters){
if($dir.Contains($filter)) {
return ''
}
}
return 'True'
}

## Checks to see whether the extension of the
## file matches any of the filters. If so
## returns false, else true
function Passes-FileFilter($file, $filters){
$ext = [System.IO.Path]::GetExtension($file).ToLower()
foreach($filter in $filters){
if ($filter.Equals($ext)){
return ''
}
}
return 'True'
}

function Get-DestinationPath($basedir, $candidate, $destdir){
$baseLength = [int]$basedir.Length
$candidateLength = [int]$candidate.Length
if ($candidateLength.Equals($baseLength)){
return ''
}
else {
#Write-Host -ForegroundColor GREEN $candidate
$rightSide = $candidate.Substring($baseLength, ($candidateLength - $baseLength))
$dir = $destdir + $rightSide
return $dir
}
}

function Copy-CodeFile($basedir, $candidate, $destdir) {
$newFile = Get-DestinationPath $basedir $candidate.FullName $destdir
copy $candidate.FullName $newFile
}

function Make-CodeDirectory($basedir, $candidate, $destdir){
$newDir = Get-DestinationPath $basedir $candidate $destdir
if ([System.String]::IsNullOrEmpty($newDir)){
}
else {
mkdir $newDir
}
}

function Traverse-Directory($basedir, $destdir, $dir, $dirfilters, $filefilters) {
# #Write-Host 'About to traverse dir: ' $dir
foreach($candidate in Get-ChildItem $dir) {
if ([System.IO.File]::Exists($candidate.FullName)){
# It's a file
if(Passes-FileFilter $candidate $filefilters) {
Copy-CodeFile $basedir $candidate $destdir
}
}
else {
# It's a directory
if (Passes-Filter $candidate.FullName $dirfilters){
Write-Host -ForegroundColor GREEN $candidate
Make-CodeDirectory $basedir $candidate.FullName $destdir
Traverse-Directory $basedir $destdir $candidate.FullName $dirfilters $filefilters
}
else {
Write-Host -ForegroundColor RED "Stopped in dir filter: " $candidate.FullName
}
}
}
}

## Script entry point
Clear-Host
$dirfilters ="_ReSharper","\obj","\bin", "\.svn"
$filefilters = ".user", ".suo", ".resharper"
$sourceDir = 'C:\depot\MyProject\trunk'
$destDir = 'C:\temp\CopyOfMyProject'
Traverse-Directory $sourceDir $destDir $sourceDir $dirfilters $filefilters

Monday, March 9, 2009

Smart Clients and System.Transactions Part 5 – Fixing the Timeout Problem

This is the fifth installment of an ongoing series about using System.Transactions as the fundament for client-side infrastructure for managing changes among loosely coupled components.

Previous posts: Introduction, Timeout, Enlistments, The transaction sink


Earlier I discussed the time-out problem which was a serious setback to the plan of building a client side change gathering infrastructure based on System.Transactions. How did we fix this problem? Well, we didn't. Instead we had to resort to cheating:

var scopeFactory = IoC.GetInstance<ITransactionScopeFactory>();
using (var scope = scopeFactory.Start())
{
// Transactional code

scope.Complete();
}


 



The above code shows how we now start client side transactions. As you can see, we no longer start a System.Transactions.TransactionScope, but rather look up a factory class (ITransactionScopeFactory) through a service locator, and ask this instance to start a transaction scope. This scope implements the interface ITransactionScope which is defined as  follows:



/// <summary>
/// A scope mimicking the API of System.Transactions.TransactionScope.
/// Defines a single method Complete() used for marking the scope
/// as "successful". This interface extends IDisposable, and when
/// Dispose() is invoked, this scope will instruct the ambient
/// transaction to commit or rollback depending on whether the
/// scope has been completed or not.
/// </summary>
public interface ITransactionScope : IDisposable
{
/// <summary>
/// Marks the scope as complete, resulting in this scope
/// instructing the ambient transaction to commit when
/// Dispose() is invoked later on. If Complete() is never
/// invoked, the scope will force the ambient transaction
/// to rollback upon Dispose().
/// </summary>
void Complete();
}


 



The responsibility of the factory is to determine the type of scope to generate, and this it does by querying the ambient transaction as to whether or not a transaction already has been started. If such a transaction does not exist an instance of ClientTransactionScope is created, and if a transaction already exists, an instance of NestedClientTransactionScope is created. The difference between these two classes lie mainly in their respective constructors and in the Dispose() method:



Constructor and Dispose() of ClientTransactionScope



/// <summary>
/// Initializes a new instance of the <see cref="ClientTransactionScope"/> class.
/// The ambient transaction is automatically started as this instance constructs.
/// </summary>
public ClientTransactionScope()
{
GetClientTransaction().Begin();
}

/// <summary>
/// If Complete() has been invoked prior to this, the
/// ambient transaction will be instructed to commit
/// here, else the transaction will be rolled back
/// </summary>
public virtual void Dispose()
{
if (Completed)
{
GetClientTransaction().Commit();
}
else
{
GetClientTransaction().Rollback();
}
}


 



Constructor and Dispose() of NestedClientTransactionScope



/// <summary>
/// Initializes a new instance of the <see cref="NestedClientTransactionScope"/> class.
/// This constructor does nothing since an ambient transaction already has been
/// started if an instance
/// </summary>
public NestedClientTransactionScope()
{}

/// <summary>
/// If Complete() has been invoked prior to this, nothing happens
/// here. If Complete() has not been invoked, the ambient transaction
/// will be marked as "non commitable". This has ne immediate
/// consequence, but the transaction is doomed and it will be
/// rollback when the outermost scope is disposed regardless of
/// if this scope attempts to rollback or commit the tx.
/// </summary>
public override void Dispose()
{
if (!Completed)
{
GetClientTransaction().MarkInnerTransactionNotCompleted();
}
}


 



The comments in the code explain the distinction between these two classes.



Commit and Rollback



The actual task of finishing the transaction, either  by commit or rollback, is the responsibility of the ambient transaction.  Throughout the lifetime of the scope, enlistments that have detected changes have enlisted with the ambient transaction. The exact details of how this enlistment procedure is done is kept hidden from the enlistments, but what actually happens is that the ambient transaction maintains a dictionary in which the enlistments are added.



When the time to commit or roll back has finally arrived, a real System.Transactions.TransactionScope is started, the registered enlistments are enlisted with the transaction, and Complete() is either invoked or not on the scope depending on whether or not the transaction is meant to be committed or rolled back:



/// <summary>
/// Instructs the transaction to begin the two-phased commit procedure.
/// This will be done except if any nested inner transaction scope
/// have instructed the transaction to rollback prior to this. If this
/// is the case the transaction will roll back the transaction at this
/// point in time.
/// </summary>
public void Commit()
{
if (!canCommit)
{
Rollback();
throw new TransactionAbortedException("Inner scope not completed");
}
using (var scope = new TransactionScope())
{
EnlistAll();
scope.Complete();
}
}

/// <summary>
/// Instructs the transaction to rollback. This will happen at
/// once if the sending scope is the outer scope (fromInnerScope == true)
/// else the rollback will be postponed until when the outer scope
/// requests a commit
/// </summary>
public void Rollback()
{
using (new TransactionScope())
{
EnlistAll();
// Don't Complete the scope,
// resulting in a rollback
}
}

private void EnlistAll()
{
var tx = Transaction.Current;
tx.EnlistVolatile(this, EnlistmentOptions.None);
tx.EnlistVolatile(sink, EnlistmentOptions.None);
foreach (var notification in enlistments.Values)
{
tx.EnlistVolatile(notification, EnlistmentOptions.None);
}
}


 



Conclusion



This concludes this series which has been an attempt at showing the benefits and problems that we have seen when realizing a novel idea: using the functionality of System.Transactions as a "change gathering infrastructure". The idea has proved viable, however the "timeout problem" proved to be a serious bump in the road, and forced us to implement code so that the actual functionality of System.Transactions only comes into play in the final moments of the logical scope.

Thursday, February 12, 2009

On the Common Service Locator

A couple of years ago I found myself evaluating which IoC container to use in a reusable framework that we where developing at my shop.  A lot of back and forth, I recall, but I ended up choosing Castle.Windsor.

Then I thought: what if I change my mind, or what if a new and hot container hits the streets while my entire code base  is married to Windsor? I decided to implement a static class with the sole purpose of hiding the details of how the actual dependency injection was taking place from the application code (I called it ObjectFactory as a homage to StructureMap, a container which was a strong contender). Later I discovered that I had employed the Gateway Pattern.

The problem with rolling my own gateway, of course, is that I would be forever condemned to writing adapters for each new IoC framework that I would want to support. Additionally, the code base that I was working on at the time could conceivably fall into the hands of devs from other shops (it's a framework). It would be nice to be able to tell them that the code used dependency injection, but that they could choose which ever container they fancied.

Enter the Common Service Locator (released around October 2008, I believe) - CSL for short. This is a light weight framework for solving the exact problem described above. More than anything else it's a standard that the IoC vendors have agreed upon, so that they make the adapters and you don't have to. Presumably, the makers of the containers are going to do a better job than you would here, anyway. Already adapters exist for Windsor, Spring.NET, Unity, StructureMap, Autofac and even for MEF.

I urge you to download the source code. It's a pleasant experience in that you'll understand the entire code base in just a matter of minutes. It immediately becomes clear that the real value on offer here isn't the actual code, it's the fact that this is, and hopefully will remain, an "industry standard" for accessing IoC container functionality on the .net platform.

So, what's the name of the gateway in the CSL framework? It's ServiceLocator.Current, so you resolve instances doing something like this:

var service = ServiceLocator.Current.GetInstance<IMyService>();


ServiceLocator.Current? Quite a few letters, don't you think? According to ctavares (one of the coordinators of this project):



“We were following the precendent set by the .NET framework - Thread.Current, HttpContext.Current, etc.”



Fine by me, but I still don't like it, and I can't see the purpose for me in following this precedent. What you have to realize here, however, is that the thing with CSL isn't the gateway, it's the interface IServiceLocator. This is the standard that the IoC vendors implement. So, it's trivially simple to roll your own gateway that simply gives access to the instance implementing this interface.



Another thing: The standard defined by IServiceLocator exposes only the bare minimum of what a container is. Chances are that your code base is using functionality that is not defined in the interface. In my case, for instance, I need to resolve objects based on its key, i.e. I have the key, but I don't know anything about the type of either of the interface or the concrete type. This functionality is used by an engine that dynamically parses and executes code at runtime.



One more small thing: the only way to register the instance implementing IServiceLocator with ServiceLocator.Current is by feeding it a delegate of type ServiceLocatorProvider which is invoked  to produce the actual locator. Sometimes (in testing scenarios), I already have the locator. So, I expanded my static gateway with an initialization method which takes a locator directly.



I give you: IoC, my static gateway into all that IoC goodness:



/// <summary>
/// This static class implements a gateway to IoC functionality
/// through the Common Service Locator framework
/// </summary>
public static class IoC
{
private static IServiceLocator serviceLocator;
private static ServiceLocatorProvider locatorDelegate;

#region Public API - Initialization

/// <summary>
/// Sets the service locator to use internally. Any previously
/// registered ServiceLocatorProvider registered through the
/// Initialize(ServiceLocatorProvider lctrDelegate) overload
/// will be removed
/// </summary>
/// <param name="locator">The service locator to use internally</param>
public static void Initialize(IServiceLocator locator)
{
serviceLocator = locator;
locatorDelegate = null;
}

/// <summary>
/// Sets the delegate used to get the IServiceLocator used internally. This
/// delegate will be invoked for every call to the GetInstance and
/// GetAllInstances overloads. Any instance set through the
/// Initialize(IServiceLocator locator) overload is nulled, and will be
/// "lost" forever.
/// </summary>
/// <param name="lctrDelegate">The delegate to be used to provide the inner service locator</param>
public static void Initialize(ServiceLocatorProvider lctrDelegate)
{
serviceLocator = null;
locatorDelegate = lctrDelegate;
}

/// <summary>
/// Gets the instance that is registered with the given
/// key.
/// </summary>
/// <param name="key">The key.</param>
/// <returns>The object registered by the given key</returns>
public static object GetByKey(string key)
{
var locator = GetServiceLocator();

if (typeof(IExtendedServiceLocator).IsAssignableFrom(locator.GetType()))
return ((IExtendedServiceLocator) locator).GetByKey(key);

throw new NotSupportedException("Registered locator " + locator.GetType() + " doesnot implement IExtendedServiceLocator");
}

private static IServiceLocator GetServiceLocator()
{
if (serviceLocator != null) return serviceLocator;
if (locatorDelegate == null)
throw new InvalidOperationException(
"IoC must be initialized through Initialize(IServiceLocator locator) or " +
"Initialize(ServiceLocatorProvider lctrDelegate)");
return locatorDelegate.Invoke();
}

#endregion

#region Public API - ServiceLocator

/// <summary>
/// Get an instance of the given <typeparamref name="TService"/>.
/// </summary>
/// <typeparam name="TService">Type of object requested.</typeparam>
/// <exception cref="ActivationException">if there is are errors resolving
/// the service instance.</exception>
/// <returns>The requested service instance.</returns>
public static TService GetInstance<TService>()
{
return GetServiceLocator().GetInstance<TService>();
}

/// <summary>
/// Get an instance of the given <paramref name="serviceType"/>.
/// </summary>
/// <param name="serviceType">Type of object requested.</param>
/// <exception cref="ActivationException">if there is an error resolving
/// the service instance.</exception>
/// <returns>The requested service instance.</returns>
public static object GetInstance(Type serviceType)
{
return GetServiceLocator().GetInstance(serviceType);
}

/// <summary>
/// Get an instance of the given named <typeparamref name="TService"/>.
/// </summary>
/// <typeparam name="TService">Type of object requested.</typeparam>
/// <param name="key">Name the object was registered with.</param>
/// <exception cref="ActivationException">if there is are errors resolving
/// the service instance.</exception>
/// <returns>The requested service instance.</returns>
public static TService GetInstance<TService>(string key)
{
return GetServiceLocator().GetInstance<TService>(key);
}

/// <summary>
/// Get an instance of the given named <paramref name="serviceType"/>.
/// </summary>
/// <param name="serviceType">Type of object requested.</param>
/// <param name="key">Name the object was registered with.</param>
/// <exception cref="ActivationException">if there is an error resolving
/// the service instance.</exception>
/// <returns>The requested service instance.</returns>
public static object GetInstance(Type serviceType, string key)
{
return GetServiceLocator().GetInstance(serviceType, key);
}

/// <summary>
/// Get all instances of the given <typeparamref name="TService"/> currently
/// registered in the container.
/// </summary>
/// <typeparam name="TService">Type of object requested.</typeparam>
/// <exception cref="ActivationException">if there is are errors resolving
/// the service instance.</exception>
/// <returns>A sequence of instances of the requested <typeparamref name="TService"/>.</returns>
public static IEnumerable<TService> GetAllInstances<TService>()
{
return GetServiceLocator().GetAllInstances<TService>();
}

/// <summary>
/// Get all instances of the given <paramref name="serviceType"/> currently
/// registered in the container.
/// </summary>
/// <param name="serviceType">Type of object requested.</param>
/// <exception cref="ActivationException">if there is are errors resolving
/// the service instance.</exception>
/// <returns>A sequence of instances of the requested <paramref name="serviceType"/>.</returns>
public static IEnumerable<object> GetAllInstances(Type serviceType)
{
return GetServiceLocator().GetAllInstances(serviceType);
}

#endregion

}

Thursday, January 29, 2009

Smart Clients and System.Transactions Part 4 – The Transaction Sink

This is the fourth installment of an ongoing series about using System.Transactions as the fundament for client-side infrastructure for managing changes among loosely coupled components.

Previous posts: Introduction, Timeout, Enlistments


This series is supposed to be about something I called "change gathering infrastructure", but up until now there has been precious little change gathering going on. But now at last we're ready to take a closer look at this.

Here's the entire definition of the IAmbientTransaction interface. I've showed it before without the ReceiveChanges signature.

public interface IAmbientTransaction
{
void Enlist(IEnlistmentNotification enlistmentNotification, EnlistmentOptions options);
void ReceiveChanges(string changeKey, object changes);
}


This is the method that the providers use in order to give notice of the changes that they have collected during the current transaction. The natural method where the providers do this is in the Prepare() method, which constitutes the first part of the two phase commit implemented by System.Transactions. So, the Prepare() method of the transactional provider typically looks like this:



public void Prepare(PreparingEnlistment preparingEnlistment)
{
ambientTransaction.ReceiveChanges(typeof(T).FullName, GetChanges());
preparingEnlistment.Prepared();
}


The ambient transaction delegates the task of receiving changes to an object implementing IAmbientTransactionSink. The work is shared between them as follows: the transaction acts as the facade (the providers don't know anything about the sink) and in addition is responsible for knowing when every provider has sent their changes, while the sink knows how to transform all received changes and transform them into a message to be sent to the server for replay.



Here's the definition of the sink:



public interface IAmbientTransactionSink : IEnlistmentNotification
{
void ReceiveChanges(string changeKey, object changes);
void WriteAllChanges();
}


How does the ambient transaction know when all changes have been received?




Originally I thought I had found a simple and elegant solution to this problem, one that didn't even burden the transaction with the additional task of knowing when every change has been received. Notice the commit coordinated by System.Transactions is two-phase; first every enlistment receives the Prepare() method, and then the Commit() method. It is recommended that all enlistments do the brunt of their work in the Prepare() method, and this is where the transactional providers send in their changes. So, I thought, if I design the sink so that it does nothing during Prepare(), it can be sure that all providers have registered their changes when it receives the Commit() message: everybody does their work during Prepare() except the sink which does its work (assembling a change request and sending it to the server for replay ) during Commit().



However, this idea is flawed, and to understand why you have to know how and when the Rollback() message is invoked on the enlistments:



When and how may a System.Transactions Rollback carried through?





There are two possible ways that the Rollback() message is distributed to the enlisted providers:



1. Complete() never sent to the TransactionScope


    a. Because the logic in the program explicitly decides not to do so


    b. Because an exception causes program execution to jump directly to the implicit


2. Prepare() not sent to preparing enlistment in the Prepare() method of a provider


    a. Because the enlistment deliberately decides not to set the flag because it for some reason wants the transaction to be aborted.


    b. Because an exception in the Prepare code of the enlistment is raised.



Notice that all of these methods for instigating a rollback happen before the Commit() phase has been reached. This means that it is impossible to start a rollback after this phase has been reached. My original idea of having the sink do its work in the Commit method therefore is a no go: if something goes wrong here (and it inevitably will) I have no way of rolling back graciously.



So what have I learned here: don't do any work in the Commit() method of your enlistments. This method should only be used for doing risk free management of internal data structures (clearing lists, resetting counters and similar).



The solution




So the solution is pretty simple. Let the ambient transaction keep track of how many enlistments there are in total, and how many that have delivered their changes. When all enlistments are done, the transaction sink can be ordered to send the change message. Below are some of the code snippets involved in this process:



public void ReceiveChanges(string changeKey, object changes)
{
enlistmentCount--;
transactionSink.ReceiveChanges(changeKey, changes);

if (enlistmentCount == 0)
transactionSink.WriteAllChanges();
}
public void Commit(Enlistment enlistment)
{
if (enlistmentCount > 0)
{
throw new TransactionException(String.Format("enlistmentCount = {0} in Commit()", enlistmentCount));
}
ExitTransaction();
enlistment.Done();
}

public void Rollback(Enlistment enlistment)
{
ExitTransaction();
enlistment.Done();
}
private void ExitTransaction()
{

enlistments.Clear();
enlistmentCount = 0;
}


Ok, this almost wraps it up for this series, however I still haven't explained how we solved the timout problem. This will be the topic of the next (an final) post in this series.

Monday, January 12, 2009

Smart Clients and System.Transactions Part 3 - Enlistments

This is the third installment of an ongoing series about using System.Transactions as the fundament for client-side infrastructure for managing changes among loosely coupled  components.

Previous posts: Introduction, Timeout


The way you hook into the change infrastructure is by building a provider that implements System.Transactions.IEnlistmentNotification. The provider in this context is an object that knows how to get hold of domain objects, and its main purpose is being a facade that abstracts away all details having to do with getting hold of objects from the server or cache. The provider knows how to detect when one of its objects changes. I usually do this by having the provider listen to events that the domain objects raise whenever changed. One elegant way of doing this is by having the domain objects implement System.ComponentModel.INotifyPropertyChange.

As changes are detected the provider will want to register with the ambient transaction. This is an operation known as enlisting. The System.Transactions functionality implements a two-phase commit scheme, and once enlisted the provider will be a part of this scheme: it will automatically receive method calls (Prepare, Commit, Rollback) at the appropriate times.

Here's some code showing a prototypical provider (a lot of functionality that you probably would need is not shown here):

public class TransactionalProvider<T> : IEnlistmentNotification where T : INotifyPropertyChanged
{
private IRepository repository;
private readonly Dictionary<INotifyPropertyChanged, INotifyPropertyChanged> changeMap =
new Dictionary<INotifyPropertyChanged, INotifyPropertyChanged>();
private bool enlisted = false;

public TransactionalProvider(IRepository repository)
{
this.repository = repository;
}

public T GetObject(Guid id)
{
T result = repository.GetObject<T>(id);
if (result == null) return default(T);
result.PropertyChanged += OnObjectChanged;
return result;
}

void OnObjectChanged(object sender, PropertyChangedEventArgs e)
{
if (Transaction.Current == null)
throw new TransactionException(
String.Format("Attempted to change object {0} while not in ntransaction", sender));

if(!enlisted)
{
Transaction.Current.EnlistVolatile(this, EnlistmentOptions.None);
enlisted = true;
}

T changedObj = (T) sender;
if(!changeMap.ContainsKey(changedObj))
{
changeMap.Add(changedObj, changedObj);
}
}

public void Prepare(PreparingEnlistment preparingEnlistment)
{
// Code for sending changes (stored in changeMap)
// into the transaction change infrastructure
// goes here.

preparingEnlistment.Prepared();
}

public void Commit(Enlistment enlistment)
{
enlisted = false; // Consider the consequences if I forget this
changeMap.Clear();
enlistment.Done();
}

// Code omitted for clarity
}


The provider enlists with the ambient transaction as it detects changing objects in the method OnObjectChanged. What happens if the provider by accident enlists two times? You might be tempted to think that the transaction notices this and either raises an exception or just lets it pass, i.e. that the list of enlistments is represented internally as a hashtable/dictionary. However, this is not the case. What happens if you enlist twice is that you will receive all the transactional messages (Prepare, Commit, Rollback) twice. This almost certainly is not what you want.



So, the task of knowing if the provider already is enlisted or not falls on you. It would have been very helpful if there existed an API for checking if any given object was enlisted or not (e.g. Transaction.Current.IsEnlisted(this) ), but unfortunately no such API exists, hence the boolean instance variable enlisted that you can see in the example above. Not too complicated, and it works, but this approach is a little too brittle for my taste. First, you have to implement this kind of enlistment checking functionality in each and every transactional provider you implement (thus violating the DRY principle), and second consider the consequences if you forget to reset the boolean when exiting the transaction. This would amount to an absolutely catastrophic bug: the application does not throw exceptions, and seems to be working as intended, however only the first transaction the provider participated in went as planned. After that the provider thought it already was enlisted, and it never enlisted again. None of the changes under its jurisdiction where ever sent to the server! Better solve this problem once and for all.



Enter IAmbientTransaction and AmbientTransaction which implements it.



public interface IAmbientTransaction
{
void Enlist(IEnlistmentNotification enlistmentNotification, EnlistmentOptions options);
}


public class AmbientTransaction : IAmbientTransaction, IEnlistmentNotification
{
private readonly Dictionary<IEnlistmentNotification, IEnlistmentNotification> enlistmentMap =
new Dictionary<IEnlistmentNotification, IEnlistmentNotification>();

public AmbientTransaction()
{}

public void Enlist(IEnlistmentNotification enlistmentNotification, EnlistmentOptions options)
{
if (Transaction.Current == null)
throw new TransactionException(
String.Format(
"Attempted to enlist enlistment notification {0} while not inntransaction",
enlistmentNotification));

if (!enlistmentMap.ContainsKey(enlistmentNotification))
{
enlistmentMap.Add(enlistmentNotification, enlistmentNotification);
}
if(!enlistmentMap.ContainsKey(this))
{
enlistmentMap.Add(this, this);
}
}

public void Prepare(PreparingEnlistment preparingEnlistment)
{
preparingEnlistment.Prepared();
}

public void Commit(Enlistment enlistment)
{
enlistmentMap.Clear();
enlistment.Done();
}

public void Rollback(Enlistment enlistment)
{
enlistmentMap.Clear();
enlistment.Done();
}
// Code omitted
}


This is functionality implements a gateway to the System.Transactions functionality. I.e. the providers no longer reference System.Transactions directly, but the IAmbientTransaction (the ambient transaction is auto-wired into the provider by the IoC container of choice) contract. A few interesting points about this code:




  • The ambient transaction itself implements IEnlistmentNotification. The sole reason for this is so that it too can receive the message sends from the transaction in the coordinated two-phase commit. It wants this messages in order to clean up its its internal state (enlistentMap) at exactly the correct point in time (in the Commit method).


  • It raises an exception when and if an enlistment ever tries to enlist when not in transaction. This eliminates the need for having such a check in every provider.



Check out the new and improved version of the method OnObjectChanged of the provider:



void OnObjectChanged(object sender, PropertyChangedEventArgs e)
{
ambientTransaction.Enlist(this, EnlistmentOptions.None);

T changedObj = (T) sender;
if(!changeMap.ContainsKey(changedObj))
{
changeMap.Add(changedObj, changedObj);
}
}


Much better, I think. All details concerning transactions (enlisting and raising exceptions when changes occur outside of transaction) are delegated to the ambient transaction, while the sole remaining responsibility of the provider is to maintain its own state.



Next time: the transaction sink.




Friday, January 9, 2009

Smart Clients and System.Transactions Part 2 - Timeout

This is the second installment of an ongoing series about using System.Transactions as the fundament for client-side infrastructure for managing changes among loosely coupled components.

Previous posts:
Introduction


Pop quiz: How long before the transaction times out here:

var scopeTimeout = new TimeSpan(
0, // Days
2, // Hours
0, // Minutes
0); // Seconds
using (var scope = new TransactionScope(
TransactionScopeOption.Required,
scopeTimeout))
{
// Do some work
scope.Complete();
}


You might be tempted to answer 2 hours, however it's impossible to say with the information I've given you here. The correct answer is probably 10 minutes. As described here and here, there are settings in the configuration that override whatever the programmer has set, and the setting with the final say is in machine.config. So the 100% correct answer is that the timeout will vary from machine to machine and application to application, and in order to determine the timeout you would have to examine the code, app.config and machine.config on the given machine.



As an aside, the API for the constructor of TransactionScope here leaves a lot to be desired, don't you think? As pointed out by Alkampfer in the comments here, the API communicated by the constructor here violates the principle of least surprise.



What does the official documentation for the constructor of TransactionScope have to say? Well, this:



Initializes a new instance of the TransactionScope class with the specified timeout value, and sets the specified transaction as the ambient transaction, so that transactional work done inside the scope uses this transaction.



No mention of all this business with the configuration. I don't know about you, but I read this basically as a confirmation of what I already "knew": the timeout is what I decide and supply in the constructor. Even the Microsoft technical writers thought this one was a no-brainer, I guess.



How should Microsoft fix this problem? Update the documentation for a start. I think it might be a good idea if the logic in the constructor checked whether the supplied timeout was larger than what the configuration allows, and that an exception was thrown if this was the case.



Yet another aside: this kind of problem with timing almost always lead to problems that go undetected by both developers and testers, but that blow up in the face of the end users almost immediately. It's a conflict of interest between the developers and testers on the one hand, and the users on the other. More precisely, everybody wants to get their work done as quickly as possible, but they don't do the same work. For a start, the developer tests the code by means of automated tests which for the most part run much quicker than any timeout. And when the developer/tester tests the application by hand, he or she is not interested in the work done by the application, but in getting the testing done. The end user on the other hand, actually uses the application.



Anyhow, this puts me between a rock and a hard place. My application is running client-side, and I can't get my hands on the machine.config on all these machines. And the users of my application do a lot of fact checking while it is "in transaction"; they read documents, make telephone calls, talk to colleagues, and so on. They spend time, and this timeout issue amounts to a serious bug rendering the application close to unusable for some of the users.



So, how did your brave correspondent get around this serious setback? I'll come to it at the end of this series. I just have to explain some other stuff first. Hang in there!

Thursday, January 8, 2009

Smart Clients and System.Transaction - Introduction

System.Transactions! I've been excited about it ever since I heard about it the first time, and generally I have not been disappointed. The ability to write data to several sources at once and easily handle the "transactional atomicity stuff" with just a couple lines of code: easy, nice, elegant.

If you have read any of the rhetoric coming out from Microsoft concerning this functionality (here in the documentation of TransactionScope, for instance), you will have noticed the term Ambient Transaction popping up all over the place. It's an interesting term, and to me it communicates something akin to infrastructure (ambient = surrounding space = scaffolding that is easily reachable from anywhere ). I started envisioning using System.Transactions as an infrastructure component for use in smart clients. The whole point being to be able to coordinate the "transactional needs" of loosely coupled components on the client.

To make it a bit clearer: I'm operating in the Smart Client, Composite Application area. I have several visual components that know nothing of each other or the environment that they are running in with the exception of a few interfaces describing the shell. Some of these components change data, and these changes must be persisted on the server. Often several unrelated components change data at the same time in what the user perceives as an atomic transaction (all or nothing). I want these changes to be gathered and assembled into a single change request which is sent to the server for replay. Needless to say, I also want the ability to rollback at any time.


Why not let System.Transactions become the backbone of this "change gathering framework"?
  • It's already installed on every machine running .net
  • It's easy to communicate how to hook into the infrastructure (just implement IEnlistementNorification !) to others.
  • Well tested, well documented, well known
  • And, indeed, ambient

So what we're talking about is using System.Transactions for something that I think it is incidentally well suited for, but which it was not primarily designed for.

And now I have built it. Overall it has been a success and a pleasant experience. However there have been some surprises and hurdles to overcome. My plan is to document my voyage in a series of posts (about 4, I guess) on this blog.

Stay tuned!