Thursday, January 29, 2009

Smart Clients and System.Transactions Part 4 – The Transaction Sink

This is the fourth installment of an ongoing series about using System.Transactions as the fundament for client-side infrastructure for managing changes among loosely coupled components.

Previous posts: Introduction, Timeout, Enlistments


This series is supposed to be about something I called "change gathering infrastructure", but up until now there has been precious little change gathering going on. But now at last we're ready to take a closer look at this.

Here's the entire definition of the IAmbientTransaction interface. I've showed it before without the ReceiveChanges signature.

public interface IAmbientTransaction
{
void Enlist(IEnlistmentNotification enlistmentNotification, EnlistmentOptions options);
void ReceiveChanges(string changeKey, object changes);
}


This is the method that the providers use in order to give notice of the changes that they have collected during the current transaction. The natural method where the providers do this is in the Prepare() method, which constitutes the first part of the two phase commit implemented by System.Transactions. So, the Prepare() method of the transactional provider typically looks like this:



public void Prepare(PreparingEnlistment preparingEnlistment)
{
ambientTransaction.ReceiveChanges(typeof(T).FullName, GetChanges());
preparingEnlistment.Prepared();
}


The ambient transaction delegates the task of receiving changes to an object implementing IAmbientTransactionSink. The work is shared between them as follows: the transaction acts as the facade (the providers don't know anything about the sink) and in addition is responsible for knowing when every provider has sent their changes, while the sink knows how to transform all received changes and transform them into a message to be sent to the server for replay.



Here's the definition of the sink:



public interface IAmbientTransactionSink : IEnlistmentNotification
{
void ReceiveChanges(string changeKey, object changes);
void WriteAllChanges();
}


How does the ambient transaction know when all changes have been received?




Originally I thought I had found a simple and elegant solution to this problem, one that didn't even burden the transaction with the additional task of knowing when every change has been received. Notice the commit coordinated by System.Transactions is two-phase; first every enlistment receives the Prepare() method, and then the Commit() method. It is recommended that all enlistments do the brunt of their work in the Prepare() method, and this is where the transactional providers send in their changes. So, I thought, if I design the sink so that it does nothing during Prepare(), it can be sure that all providers have registered their changes when it receives the Commit() message: everybody does their work during Prepare() except the sink which does its work (assembling a change request and sending it to the server for replay ) during Commit().



However, this idea is flawed, and to understand why you have to know how and when the Rollback() message is invoked on the enlistments:



When and how may a System.Transactions Rollback carried through?





There are two possible ways that the Rollback() message is distributed to the enlisted providers:



1. Complete() never sent to the TransactionScope


    a. Because the logic in the program explicitly decides not to do so


    b. Because an exception causes program execution to jump directly to the implicit


2. Prepare() not sent to preparing enlistment in the Prepare() method of a provider


    a. Because the enlistment deliberately decides not to set the flag because it for some reason wants the transaction to be aborted.


    b. Because an exception in the Prepare code of the enlistment is raised.



Notice that all of these methods for instigating a rollback happen before the Commit() phase has been reached. This means that it is impossible to start a rollback after this phase has been reached. My original idea of having the sink do its work in the Commit method therefore is a no go: if something goes wrong here (and it inevitably will) I have no way of rolling back graciously.



So what have I learned here: don't do any work in the Commit() method of your enlistments. This method should only be used for doing risk free management of internal data structures (clearing lists, resetting counters and similar).



The solution




So the solution is pretty simple. Let the ambient transaction keep track of how many enlistments there are in total, and how many that have delivered their changes. When all enlistments are done, the transaction sink can be ordered to send the change message. Below are some of the code snippets involved in this process:



public void ReceiveChanges(string changeKey, object changes)
{
enlistmentCount--;
transactionSink.ReceiveChanges(changeKey, changes);

if (enlistmentCount == 0)
transactionSink.WriteAllChanges();
}
public void Commit(Enlistment enlistment)
{
if (enlistmentCount > 0)
{
throw new TransactionException(String.Format("enlistmentCount = {0} in Commit()", enlistmentCount));
}
ExitTransaction();
enlistment.Done();
}

public void Rollback(Enlistment enlistment)
{
ExitTransaction();
enlistment.Done();
}
private void ExitTransaction()
{

enlistments.Clear();
enlistmentCount = 0;
}


Ok, this almost wraps it up for this series, however I still haven't explained how we solved the timout problem. This will be the topic of the next (an final) post in this series.

Monday, January 12, 2009

Smart Clients and System.Transactions Part 3 - Enlistments

This is the third installment of an ongoing series about using System.Transactions as the fundament for client-side infrastructure for managing changes among loosely coupled  components.

Previous posts: Introduction, Timeout


The way you hook into the change infrastructure is by building a provider that implements System.Transactions.IEnlistmentNotification. The provider in this context is an object that knows how to get hold of domain objects, and its main purpose is being a facade that abstracts away all details having to do with getting hold of objects from the server or cache. The provider knows how to detect when one of its objects changes. I usually do this by having the provider listen to events that the domain objects raise whenever changed. One elegant way of doing this is by having the domain objects implement System.ComponentModel.INotifyPropertyChange.

As changes are detected the provider will want to register with the ambient transaction. This is an operation known as enlisting. The System.Transactions functionality implements a two-phase commit scheme, and once enlisted the provider will be a part of this scheme: it will automatically receive method calls (Prepare, Commit, Rollback) at the appropriate times.

Here's some code showing a prototypical provider (a lot of functionality that you probably would need is not shown here):

public class TransactionalProvider<T> : IEnlistmentNotification where T : INotifyPropertyChanged
{
private IRepository repository;
private readonly Dictionary<INotifyPropertyChanged, INotifyPropertyChanged> changeMap =
new Dictionary<INotifyPropertyChanged, INotifyPropertyChanged>();
private bool enlisted = false;

public TransactionalProvider(IRepository repository)
{
this.repository = repository;
}

public T GetObject(Guid id)
{
T result = repository.GetObject<T>(id);
if (result == null) return default(T);
result.PropertyChanged += OnObjectChanged;
return result;
}

void OnObjectChanged(object sender, PropertyChangedEventArgs e)
{
if (Transaction.Current == null)
throw new TransactionException(
String.Format("Attempted to change object {0} while not in ntransaction", sender));

if(!enlisted)
{
Transaction.Current.EnlistVolatile(this, EnlistmentOptions.None);
enlisted = true;
}

T changedObj = (T) sender;
if(!changeMap.ContainsKey(changedObj))
{
changeMap.Add(changedObj, changedObj);
}
}

public void Prepare(PreparingEnlistment preparingEnlistment)
{
// Code for sending changes (stored in changeMap)
// into the transaction change infrastructure
// goes here.

preparingEnlistment.Prepared();
}

public void Commit(Enlistment enlistment)
{
enlisted = false; // Consider the consequences if I forget this
changeMap.Clear();
enlistment.Done();
}

// Code omitted for clarity
}


The provider enlists with the ambient transaction as it detects changing objects in the method OnObjectChanged. What happens if the provider by accident enlists two times? You might be tempted to think that the transaction notices this and either raises an exception or just lets it pass, i.e. that the list of enlistments is represented internally as a hashtable/dictionary. However, this is not the case. What happens if you enlist twice is that you will receive all the transactional messages (Prepare, Commit, Rollback) twice. This almost certainly is not what you want.



So, the task of knowing if the provider already is enlisted or not falls on you. It would have been very helpful if there existed an API for checking if any given object was enlisted or not (e.g. Transaction.Current.IsEnlisted(this) ), but unfortunately no such API exists, hence the boolean instance variable enlisted that you can see in the example above. Not too complicated, and it works, but this approach is a little too brittle for my taste. First, you have to implement this kind of enlistment checking functionality in each and every transactional provider you implement (thus violating the DRY principle), and second consider the consequences if you forget to reset the boolean when exiting the transaction. This would amount to an absolutely catastrophic bug: the application does not throw exceptions, and seems to be working as intended, however only the first transaction the provider participated in went as planned. After that the provider thought it already was enlisted, and it never enlisted again. None of the changes under its jurisdiction where ever sent to the server! Better solve this problem once and for all.



Enter IAmbientTransaction and AmbientTransaction which implements it.



public interface IAmbientTransaction
{
void Enlist(IEnlistmentNotification enlistmentNotification, EnlistmentOptions options);
}


public class AmbientTransaction : IAmbientTransaction, IEnlistmentNotification
{
private readonly Dictionary<IEnlistmentNotification, IEnlistmentNotification> enlistmentMap =
new Dictionary<IEnlistmentNotification, IEnlistmentNotification>();

public AmbientTransaction()
{}

public void Enlist(IEnlistmentNotification enlistmentNotification, EnlistmentOptions options)
{
if (Transaction.Current == null)
throw new TransactionException(
String.Format(
"Attempted to enlist enlistment notification {0} while not inntransaction",
enlistmentNotification));

if (!enlistmentMap.ContainsKey(enlistmentNotification))
{
enlistmentMap.Add(enlistmentNotification, enlistmentNotification);
}
if(!enlistmentMap.ContainsKey(this))
{
enlistmentMap.Add(this, this);
}
}

public void Prepare(PreparingEnlistment preparingEnlistment)
{
preparingEnlistment.Prepared();
}

public void Commit(Enlistment enlistment)
{
enlistmentMap.Clear();
enlistment.Done();
}

public void Rollback(Enlistment enlistment)
{
enlistmentMap.Clear();
enlistment.Done();
}
// Code omitted
}


This is functionality implements a gateway to the System.Transactions functionality. I.e. the providers no longer reference System.Transactions directly, but the IAmbientTransaction (the ambient transaction is auto-wired into the provider by the IoC container of choice) contract. A few interesting points about this code:




  • The ambient transaction itself implements IEnlistmentNotification. The sole reason for this is so that it too can receive the message sends from the transaction in the coordinated two-phase commit. It wants this messages in order to clean up its its internal state (enlistentMap) at exactly the correct point in time (in the Commit method).


  • It raises an exception when and if an enlistment ever tries to enlist when not in transaction. This eliminates the need for having such a check in every provider.



Check out the new and improved version of the method OnObjectChanged of the provider:



void OnObjectChanged(object sender, PropertyChangedEventArgs e)
{
ambientTransaction.Enlist(this, EnlistmentOptions.None);

T changedObj = (T) sender;
if(!changeMap.ContainsKey(changedObj))
{
changeMap.Add(changedObj, changedObj);
}
}


Much better, I think. All details concerning transactions (enlisting and raising exceptions when changes occur outside of transaction) are delegated to the ambient transaction, while the sole remaining responsibility of the provider is to maintain its own state.



Next time: the transaction sink.




Friday, January 9, 2009

Smart Clients and System.Transactions Part 2 - Timeout

This is the second installment of an ongoing series about using System.Transactions as the fundament for client-side infrastructure for managing changes among loosely coupled components.

Previous posts:
Introduction


Pop quiz: How long before the transaction times out here:

var scopeTimeout = new TimeSpan(
0, // Days
2, // Hours
0, // Minutes
0); // Seconds
using (var scope = new TransactionScope(
TransactionScopeOption.Required,
scopeTimeout))
{
// Do some work
scope.Complete();
}


You might be tempted to answer 2 hours, however it's impossible to say with the information I've given you here. The correct answer is probably 10 minutes. As described here and here, there are settings in the configuration that override whatever the programmer has set, and the setting with the final say is in machine.config. So the 100% correct answer is that the timeout will vary from machine to machine and application to application, and in order to determine the timeout you would have to examine the code, app.config and machine.config on the given machine.



As an aside, the API for the constructor of TransactionScope here leaves a lot to be desired, don't you think? As pointed out by Alkampfer in the comments here, the API communicated by the constructor here violates the principle of least surprise.



What does the official documentation for the constructor of TransactionScope have to say? Well, this:



Initializes a new instance of the TransactionScope class with the specified timeout value, and sets the specified transaction as the ambient transaction, so that transactional work done inside the scope uses this transaction.



No mention of all this business with the configuration. I don't know about you, but I read this basically as a confirmation of what I already "knew": the timeout is what I decide and supply in the constructor. Even the Microsoft technical writers thought this one was a no-brainer, I guess.



How should Microsoft fix this problem? Update the documentation for a start. I think it might be a good idea if the logic in the constructor checked whether the supplied timeout was larger than what the configuration allows, and that an exception was thrown if this was the case.



Yet another aside: this kind of problem with timing almost always lead to problems that go undetected by both developers and testers, but that blow up in the face of the end users almost immediately. It's a conflict of interest between the developers and testers on the one hand, and the users on the other. More precisely, everybody wants to get their work done as quickly as possible, but they don't do the same work. For a start, the developer tests the code by means of automated tests which for the most part run much quicker than any timeout. And when the developer/tester tests the application by hand, he or she is not interested in the work done by the application, but in getting the testing done. The end user on the other hand, actually uses the application.



Anyhow, this puts me between a rock and a hard place. My application is running client-side, and I can't get my hands on the machine.config on all these machines. And the users of my application do a lot of fact checking while it is "in transaction"; they read documents, make telephone calls, talk to colleagues, and so on. They spend time, and this timeout issue amounts to a serious bug rendering the application close to unusable for some of the users.



So, how did your brave correspondent get around this serious setback? I'll come to it at the end of this series. I just have to explain some other stuff first. Hang in there!

Thursday, January 8, 2009

Smart Clients and System.Transaction - Introduction

System.Transactions! I've been excited about it ever since I heard about it the first time, and generally I have not been disappointed. The ability to write data to several sources at once and easily handle the "transactional atomicity stuff" with just a couple lines of code: easy, nice, elegant.

If you have read any of the rhetoric coming out from Microsoft concerning this functionality (here in the documentation of TransactionScope, for instance), you will have noticed the term Ambient Transaction popping up all over the place. It's an interesting term, and to me it communicates something akin to infrastructure (ambient = surrounding space = scaffolding that is easily reachable from anywhere ). I started envisioning using System.Transactions as an infrastructure component for use in smart clients. The whole point being to be able to coordinate the "transactional needs" of loosely coupled components on the client.

To make it a bit clearer: I'm operating in the Smart Client, Composite Application area. I have several visual components that know nothing of each other or the environment that they are running in with the exception of a few interfaces describing the shell. Some of these components change data, and these changes must be persisted on the server. Often several unrelated components change data at the same time in what the user perceives as an atomic transaction (all or nothing). I want these changes to be gathered and assembled into a single change request which is sent to the server for replay. Needless to say, I also want the ability to rollback at any time.


Why not let System.Transactions become the backbone of this "change gathering framework"?
  • It's already installed on every machine running .net
  • It's easy to communicate how to hook into the infrastructure (just implement IEnlistementNorification !) to others.
  • Well tested, well documented, well known
  • And, indeed, ambient

So what we're talking about is using System.Transactions for something that I think it is incidentally well suited for, but which it was not primarily designed for.

And now I have built it. Overall it has been a success and a pleasant experience. However there have been some surprises and hurdles to overcome. My plan is to document my voyage in a series of posts (about 4, I guess) on this blog.

Stay tuned!