DataWings – Data driven integration testing

Yes, I’m now officially an open source contributor, and the project’s even got it’s logo, so you know it’s gotta be good.

So what we have attempted to do is to make a lightweight, easy to use, no set up tool to be used when testing code that sits on top of a database. With this tool, DataWings, you can set up the database so that your tests are accessing known data and assert that the database is in the expected state after the tests have executed. And all of this is done directly in the test code.

Get the bits her.

Here’s a more detailed description of what the tool does, and how it works:

First, a word of caution

DataWings is designed to be a tool to be used at design time and during testing. No attention has been paid to security issues, and we definitely do not recommend using this code in production.

Configuring the connection string

The first thing you need to do when using DataWings is to configure the connection string(s) to be used. The functionality for doing this is purposefully designed with two goals in mind: a) making it easy to set up the connection string in code in order to "get going" as fast as possible, and b) making is easy to maintain the connection string outside of code thus helping to ensure that the test will remain operative in the future.

The connection string is set by decorating either the class or method with an attribute. There are several different kinds of attributes thata can be used, but here we'll focus on ConnectionStringFromConfigFile. As the name implies, this attribute is used when the connection string is registered in the standard <connectionStrings> section of the configuration file. A typical usage of this attribute might look like this:

[ConnectionFromConfigFile(SqlVendor.Oracle, Key = "MyConnection", AlternativeConnection = "TheConnectionString")]

Here, the sql vendor (input to the constructor) dictates which ADO.NET provider that will be used behind the scenes, the Key property specifies the name of the connection string in the configuration file to use, and the AlternativeConnection property is set with the specified connection string is not found in the configuration file or if any other problem is detected while trying to look up this string.

How the attribute is resolved

As mentioned, the connection string attribute can be used to decorate both methods and classes. The process of resolving which attribute to be used is carried out by walking the stack looking for an appropriate attribute. The algorithm first looks at the executing method of the stack frame, and if this method does not have an appropriate decoration, the class of the executing method is examined. This process continues for each stack frame until a decoration is found. If no such decoration can be located, an exception is raised.

Named connection string

The ConnectionFromConfigFile attribute also has a property called Name. This property is useful in situations where the tests touch more than one database. All the static gateway classes into the DataWings functionality (DataBoy, DbAssert and Adversary - see below) have a ForConnection() method, through which the named connection attribute to be used can be specified.

Here's a sample of such a named decoration:

[ConnectionFromConfigFile(SqlVendor.Oracle, Name="Default", Key = "MyConnection", AlternativeConnection = "TheConnectionString")]

DataBoy

Standard usage

DataBoy provides functionality for keeping the data in the database in a consistent state so that the tests are running against known data . This sample shows the standard usage of DataBoy.

DataBoy
.ForTable("Person")
  .Row("IdPerson", 1).Data("Surname", "Obama").DeleteFirst()
  .Row("IdPerson", 2).Data("Surname", "Bush").DeleteFirst()
.ForTable("Address")
  .Row("IdAddress", 100).Data("Street", "Main street").DeleteFirst()
.Commit();

Internally DataBoy keeps track of changes in a session and this session resides only in memory until the Commit() method is invoked. The table to insert data into is specified with the ForTable() method, and this table will be the one that is used for all subsequent row specifications until another call to ForTable() is encountered.

Each row to be inserted is marked by the Row() method, and this method takes the column name and the value the (presumably) uniquely identifies the row as input. The row can receive the DeleteFirst() method, and if this method has been invoked, DataBoy will delete the row from the database before it is inserted.

Order of execution

The rows of the session are traversed twice, first from back to front and then from front to back. In the first parse (backwards), any deletions are performed (i.e. rows marked with DeleteFirst()) while the insert are performed in the second traversal. In this way it should largely be possible to order the statements so that any errors due to foreign key constraint violations are avoided.

Updating instead of deleting

By invoking the ForUpdate() method an update statement (instead of an insert) will be generated and invoked the row in question. Example:

DataBoy.ForTable("Person").Row("IdPerson", 1).Data("Surname", "Obama").ForUpdate().Commit();

Just deleting

Rows can be deleted through the usage of the ForDelete() method. Example:

DataBoy.ForTable("Person").Row("IdPerson", 1).ForDelete().Commit();

Executing custom queries

The ExecuteNonQuery() method supplies a way to invoke custom queries directly against the database:

string sqlQuery = "INSERT INTO Address (IdAddress, Street) VALUES (99, 'Some Street')";
DataBoy.ExecuteNonQuery(sqlQuery).Commit();

DbAssert

The static class DbAssert is for asserting that data in the database is in the expected state. This class is quit similar to the familiar Assert class of many unit test frameworks.

In order to set up an assertion you first need to specify which table you are testing against. As with DataBoy, this done through the ForTable() method. When the table has been specified we need to tell the framework which row we are interested in, and this is accomplished with the method WithColumnValuePair(). This method takes a column name and corresponding value as input, and generally this will be the column name of primary key and the primary key for the row of interest. If more than one row exist for this column value pair, the first row encountered (randomly) will be used.

When the assertion has been set up, the actual assertion can be specified:

Exists(), NotExists()

Determines whether the row in question exists at all. Example:

string columnName = "Surname";
string columnValue = "Obama";
DbAssert
  .ForTable("Person")
  .WithColumnValuePair(columnName, columnValue)
  .Exists();

AreEqual()

Determines whether the value in the specified row equals the specified value. Example:

DbAssert.ForTable("Person")
      .WithColumnValuePair("IdPerson", 1)
      .AreEqual("FirstName", "Barack");

Evaluate()

This method returns the entire row, and you can perform arbitrarily complex tests on the values of this row by using a lambda expression. Example:

DbAssert.ForTable("Person")
  .WithColumnValuePair("IdPerson",  1)
  .Evaluate(row =>
          row.GetResult("FirstName") == "Barack" &&
          .row.GetResult("IsPresident") == true);

Adversary

Adversary is a static gateway class providing functionality for provoking conflicts in optimistic concurrency scenarios. This code is still in a very early phase, and hopefully will mature in the future

Sql provider provisioning and built in providers

DataWings natively support SQL Server and Oracle database engines by using the System.Data.SqlClient and System.Data.OracleClient of the .NET framework. Additionally, DataWings supports SQLite through the separate assembly DataWings.SQLite. This support for SQLite has been realized by use of the built-in provider provisioning infrastructure. This is a model where you can develop support for your favorite database engine by implementing two simple interfaces. Hopefully, I'll be able to get into more details about this at a later stage.

Fluent Castle Windsor and Configured Parameters

Castle Windsor version 2.0 has just been released (despite the fact that version 1.0 has never existed). The biggest new feature in this release is the fluent configuration interface which lets you set up your components in code in an elegant way (as opposed to configuring in xml).

A component which in xml is set up like this:

<component
  id="service"
  lifestyle="transient"
  service="Some.Namespace.IService, MyAssembly"
  type="Some.Namespace.Service, MyAssembly">
    <parameters>
       <lang>Norwegian</lang>
    </parameters>
</component>

can now be configured fluently like this:

var container = new WindsorContainer();
container.Register(
   Component
     .For<IService>()
     .ImplementedBy<Service>()
     .LifeStyle.Transient
     .Parameters(Parameter.ForKey("lang").Eq("Norwegian")));

For more examples of how the fluent interface works, read this.

I am really enjoying the experience of using this new fluent interface; it is much easier to configure a component first time, and you get the full support of your compiler and from ReSharper. At last I am able to rename classes (through the refactoring functionality in ReSharper) without having to hunt down and fix the configuration for the component in the xml file.

Configuring parameters

Notice the parameter lang in the example above; the value Norwegian is hardwired directly in as a parameter (is presumably injected into the component's constructor). In the real world you would probably want the keep track of all such properties separately in the properties node, thus promoting reuse and easing maintenance. Your xml configuration might look like this:

<properties>
  <language>Norwegian</language>
</properties>
<components>
<component
       id="service"
       lifestyle="transient"
       service="Some.Namespace.IService, MyAssembly"
       type="Some.Namespace.Service, MyAssembly">
       <parameters>
         <lang>#{language}</lang>
       </parameters>
     </component>
</components>

There is a tension between the benefits and drawbacks of having the configuration in code as opposed to in separate xml files: in code it is easier to manage the configuration while under development, while xml-based configuration supports easy changes to a system that has already been deployed. I feel that the smartest path would be to configure the components in code, but keep the properties defined in xml.

I am currently retrofitting fluent castle configuration on a relatively large application that is totally castle.windsor based. My initial gut feeling was that this new version of castle as a matter of course supported this "smartest path". Unfortunately, it doesn't, and so I was left to my own devices: enter ConfiguredParameter.

ConfiguredParameter

With the ConfiguredParameter functionality in place, I am able to configure my component as follows:

Properties in xml:

<castle>
 <properties>
   <language>Norwegian</language>
 </properties>
</castle>

Component in code:

var container = new WindsorContainer();
container.Register(
   Component
       .For<IService>()
       .ImplementedBy<Service>()
       .LifeStyle.Transient
       .Parameters(ConfiguredParameter.ForKey("lang").GetValue("language")));

To make it clear, the line ConfiguredParameter.ForKey("lang").GetValue("language") will look up the value for the configured property language, and this value will be injected into the component at lang (which presumably is a parameter in the constructor of the type Service).

The functionality must be bootstrapped as your application starts (before your container is initialized), and this will typically be accomplished like this:

InitializeConfiguredParameters.Initialize();

Here the application configuration file will be parsed, and any additional castle configuration files (included through the use of the <include> element of castle) will be included. There is a single overload to the Initialize() method where you can explicitly indicate which config file (.config or .xml) to parse, but this is mostly useful in testing scenarios.

This functionality is rather simple, and the implementation consists of just two types in addition to a couple of parser classes responsible for the actual parsing of the configuration files. These two types are ConfiguredParameter and InitializeConfiguredParameters. Below you will find the definition of the two types.

On the Road Map

The ability to use configured key value pairs from the appSettings element of the application configuration file might be nice, and I'll implement it whenever I need it.

The Code

using System;
using System.Collections.Generic;
using System.Configuration;
using Castle.MicroKernel.Registration;

/// <summary>
/// Used to access parameters that are configured within a standard
/// castle.windsor properties element
/// </summary>
public class ConfiguredParameter
{
   #region Static API

   private static readonly IDictionary<string, string> configuredParameters = new Dictionary<string, string>();
   private static readonly object syncLock = new object();

   /// <summary>
   /// Adds each parameter in the incoming dictionary to the internal
   /// cache of configured parameters
   /// </summary>
   /// <param name="parameters">The parameters.</param>
   internal static void AddParameters(IDictionary<string, string> parameters)
   {
       // Thread safe!
       lock (syncLock)
       {
           foreach (var pair in parameters)
           {
               // Skip if already contained, assuming that
               // it's some kind of race condition. So, if
               // the configuration contains two or more
               // identical keys, one of them will "win"
               // unpredictably
               if (!configuredParameters.ContainsKey(pair.Key))
               {
                   configuredParameters.Add(pair);
               }
           }
       }
   }

   /// <summary>
   /// Resets the ConfiguredParameter infrastructure by clearing all loaded
   /// configured parameters. NB! This method should normally not be invoked,
   /// and it is defined mostly for testing purposes.
   /// </summary>
   public static void Reset()
   {
       configuredParameters.Clear();
   }

   /// <summary>
   /// Sets the name of the parameter on a new instance of
   /// ConfiguredParameter and returns it
   /// </summary>
   /// <param name="parameterKey">The key.</param>
   /// <returns></returns>
   public static ConfiguredParameter ForKey(string parameterKey)
   {
       if (configuredParameters.Count == 0)
           throw new InvalidOperationException("ConfiguredParameter infrastructure not initialized.");
       return new ConfiguredParameter(parameterKey);
   }

   private static string GetVal(string key)
   {
       try
       {
           return configuredParameters[key];
       }
       catch (KeyNotFoundException e)
       {
           string message = String.Format("No configured parameter named {0} can be found", key);
           throw new ConfigurationErrorsException(message, e);
       }
   }

   #endregion

   private readonly string parameterKey;

   /// <summary>
   /// Initializes a new instance of the <see cref="ConfiguredParameter"/> class.
   /// </summary>
   /// <param name="parameterKey">The parameter key.</param>
   private ConfiguredParameter(string parameterKey)
   {
       this.parameterKey = parameterKey;
   }  

   /// <summary>
   /// Returns a Parameter with the value at the propertyKey in the
   /// castle configuration
   /// </summary>
   /// <param name="propertyKey">The property key.</param>
   /// <returns></returns>
   public Parameter GetValue(string propertyKey)
   {
       return Parameter.ForKey(parameterKey).Eq(GetVal(propertyKey));
   }
}

using System.IO;
using System.Reflection;

/// <summary>
/// Resposible for initializing the ConfiguredParameter functionality
/// by getting hold of and parsing any relevant configuration files
/// containing castle.windsor parameters
/// </summary>
public static class InitializeConfiguredParameters
{
   /// <summary>
   /// Initializes this instance by getting hold of the application's
   /// configuration file (app.config or web.config) and parsing it
   /// looking for configured parameters. If the castle configuration
   /// of this file contains include elements, the castle files referenced
   /// in these elements are also parsed.
   /// </summary>
   public static void Initialize()
   {
       string configFile = Path.GetFileName(Assembly.GetEntryAssembly().Location) + ".config";
       if (File.Exists(configFile))
       {
           InitializeWithFile(configFile);
       }
   }

   /// <summary>
   /// Initializes the with file. Valid file types are application files
   /// (app.config or web.config) as well as stand alone castle config
   /// files
   /// </summary>
   /// <param name="filename">The filename.</param>
   public static void InitializeWithFile(string filename)
   {
       ReaderBase reader;
       if (Path.GetExtension(filename).ToLower() == ".config")
       {
           reader = new ConfigFileReader(filename);
       }
       else
       {
           reader = new PropertiesReader(filename);
       }
       ConfiguredParameter.AddParameters(reader.GetConfiguredProperties());
   }
}