BizTalk moving to the cloud… and to Windows Server AppFabric

For those of you who may have missed Sriram’s
blog post, there will be a CTP of the Integration as a Service version of BizTalk appearing
in Azure sometime before July 2011 (interestingly, the blog entry was posted at 10am
on October 28^th, i.e. an hour after PDC10 started…!).

Additionally, BizTalk vNext will transition to being AppFabric based.

To anyone who’s been watching AppFabric for a while now, this will come as no surprise:
when Dublin was announced, it was clear that there were too many striking similarities
between Dublin (now called Windows Server AppFabric) and BizTalk to be a coincidence.

Sriram’s post makes one thing clear: Microsoft won’t be abandoning BizTalk customers
(10,000+ at last count) who have a large investment in the product and are understandably
concerned about the future of their investment. And this is a very important message
to get across: At one client I’m working with, Microsoft have been saying that AppFabric
is now the choice-du-jour for low-latency integration solutions – not BizTalk.

This can be very worrying to an organisation that may have spent significant sums
on BizTalk/SQL Licenses, and the associated infrastructure: BizTalk is not a cheap
product, but AppFabric is free, and so this is a message that Microsoft has to be
careful in spreading. And I think that Sriram has hit the right tone here.

Will AppFabric replace BizTalk?

I think that’s the wrong way to look at it: I think the way that Sriram has phrased
it is better: BizTalk will migrate to AppFabric. That is, AppFabric will not replace BizTalk,
and this is an important distinction to make. BizTalk 2010 has the ability to
integrate (partly) with AppFabric, whilst BizTalk vNext will actually be part
of AppFabric.

Why AppFabric won’t replace BizTalk

A reasonable question to ask would be: why can’t I write an AppFabric application
today which effectively replaces what BizTalk does? The answer is: you can. Sort
of. For the right kind of application.

Let’s look at some of the features of BizTalk, and their equivalents in AppFabric
today (note: this is not an exhaustive list, I’m just trying to prove a point):

* Notes:
The WF Rules activity has the ability to perform simple business rules; the BizTalk
BRE has the ability to perform complex rules. Which ones is better depends on the
situation: the WF engine requires you to write C# classes (i.e. little visibility
of rules), whilst the BRE provides the Rules Composer which, while arcane as it can
be, still gives powerful visibility of rules (for a comparison of BRE with the earlier
version of WF Rules, see Charles
Young’s post)

Looking at the features above, we can only see three differences:

a) Concurrent/scheduled service instances

This is an interesting one: If you want to support concurrency in WF4, you
have to code it yourself. For example, if from inside a workflow you wanted to make
two concurrent service calls, you’d need to do all the thread plumbing yourself. The
WF4 engine is inherently single-threaded, so you don’t get much out of the box support. UPDATE: Matt
Winkler (in the comments below) has corrected me on this: WF4 offers the Parallel and ParallelForEach activities
which will schedule tasks to be performed in parallel: although a workflow instance
is single-threaded, the WF4 engine is multithreaded – much good work has gone
on for the WF4 release.  In BizTalk, however, the pub-sub mechanism
looks after this for you: you can drop multiple messages to the message box, multiple
service instances spin up to process them, and then you correlate the results back
when finished (much like the await-all functionality in the Visual
Studio Asynchronous talk the Anders did at the PDC yesterday).>

b) Distributed in-memory persistence

This has always been BizTalk’s Achilles heel (whilst also being one of its strengths):
BizTalk persists everything to SQL server. This gives reliability at the cost of performance.
AppFabric, on the other hand, can use AppFabric Caching as its persistence mechanism.
Which is why AppFabric is a better choice for true low-latency situations, where latency
and throughput is more important than the security of persisting everything to disk.

c) Pub/Sub mechanism

This is the one major piece missing from AppFabric (to my mind): a flexible pub-sub
style mechanism (or something that allows store-and-forward) and which allows
us to spin up multiple concurrent subscribing workflows to process messages… and
which implements comprehensive throttling.

AFAIK there is nothing like this in Windows Server AppFabric yet (although you could
implement it in Azure via Message Buffering). But I wouldn’t be surprised if we see
something soon. This functionality would provide the last piece in the puzzle: It
would now be possible to write a services-based application that can process multiple
concurrent requests in a reliable, distributed, and throttle-able manner; which could
activate business workflows, call rules; and make use of BizTalk Adapters.

Conclusion

As you can see, Microsoft are very close. However there’s still a lot of work to do.
Providing Integration as a Service doesn’t come without its difficulties: AppFabric
is not yet feature rich or mature enough to provide the same functionality that BizTalk
does. In the meantime, BizTalk is still a great platform to develop integration solutions
on, and as Biztalk Architects and Developers we now have more tools to use to develop
those solutions.

>