Application Server and Integration Server: Growing Closer Together

September 24, 2010

 

This blog post is about the evolution of Microsoft’s application server – a collection of products including IIS and .NET framework libraries such as WCF and WF. Over the last few releases Microsoft has added 1) runtime container and activation service, via WAS; 2) multi-protocol support that goes beyond Http, via IIS 7.0; and 3) manageability extensions and caching, via AppFabric. Some of these enhancements overlap with the capabilities offered by Microsoft’s flagship integration server, BizTalk. BizTalk is well known for its robust hosting and integration capabilities. This overlap has stirred conversation in blogs and news stories over the last several months.

My goal in this post is to step back and look into some of the reasons causing application and integration servers to grow closer; we will do a side-by-side comparison of a scenario that is right in the middle of the overlap which I alluded to earlier. Additionally, as pure conjecture on my part, we will take a look at an approach to making them live together closely.

Application and Integration Server

Application servers are well known for the capabilities that enable hosting of application logic. As you would expect, they offer capabilities such as robust hosting – auto start, load balancing, fault tolerance etc. Integration servers are well known for enabling communication with external systems and, as you would expect, offer support for various message exchange patterns, guaranteed delivery, and family of adapters to connect to desperate systems and so on.

Note -as stated earlier, BizTalk is more than a typical integration server. The diagram below is intended to be more generic and depicts typical workloads of application and integration servers.

image

 

With emphasis on composite applications, application designers are increasingly looking for ways to combine the best of the capabilities offered by application and integration server. For example, they want to deploy a business service (exposed as a SOAP endpoint) and at the same time, expose it as a service endpoint that is consumable as an external system via MSMQ. Conversely, they want to expose integration endpoints that have low latency requirements and don’t necessarily need features such as guaranteed delivery and business activity monitoring.

Another factor that is driving application and integration servers closer is the move towards the cloud. Traditional on-premise based application logic is being refactored to take advantage of cloud based services such as storage, compute and access services. This is also placing a demand for integration capabilities to be added to application servers.

Not surprisingly, Microsoft’s recent announcements suggest that the line between AppFabric and BizTalk is being blurred. For instance, the BizTalk team has stated publicly that they are looking to build the next generation of BizTalk on top of AppFabric – http://www.microsoftpdc.com/2009/SVR15.

Current Dilemma

If you are architecting an enterprise application today, should you just move all your integration logic to AppFabric? There are a number of reasons why you would be tempted to do this, including 1) AppFabric is part of the OS; 2) most integration is now SOA/ WS based anyways, thus reducing the need for custom adapters; 3) operational setup and maintenance cost of running an AppFabric WebFarm is likely lower than a BizTalk farm; and 4) development of components is more approachable to a .NET developer who does not have prior knowledge of BizTalk. However, if you go down in this direction, be aware that there will be some features that will not be available. To better understand what is available and what is not available, let us use a scenario that is well supported by both – hosting a WCF endpoint. The following table compares the hosting a WCF endpoint in BizTalk and AppFabric.

 

WCF hosted Inside BizTalk

WCF hosted inside App Fabric

Binding flexibility – support for custom WCF

configuration

Binding flexibility – support for custom WCF configuration

Load balanced, fault tolerant setup

· Central administration covers all aspects. For example, a change to the WCF binding needs to be applied to a single location (port setting dialog).

Load balanced, fault tolerant setup

· Central administration for tracking purposes only. A change to a WCF binding needs to be replicated to all nodes using tools such as msdeploy.

http://download.microsoft.com/download/3/1/C/31CED722-2E5F-48D6-96B1-E73AAFD9873F/AppFabricWebFarm.docx

Common tracking database

Common tracking database

Tooling to review tracked instances

· Tooling is somewhat dated –BAM portal.

Tooling to review tracked instances

· More modern tooling.

Cross – cutting concerns – exception handling, BAM, pre and post processing (formatting etc.)

· Advanced support in the form of suspended message queue, BAM portal.

Cross – cutting concerns – exception handling, BAM, pre and post processing

(Formatting etc.)

· Not as advanced as BTS yet.

Guaranteed delivery

No guaranteed delivery (except when provided by the transport such as msmq)

Better suited for async message exchange. Request /Response for low latency scenarios are not easily handled.

Better support for different message exchange patterns – Request/Response, Asynchronous.

Extensibility points for transforming incoming and outgoing messages

· Pipeline components (, EDI, AS2, MIME, etc.)

· Mapper

· Flat File Parser

Extensibility points for transforming incoming and outgoing messages as defined by WCF

· Custom Channels

Business logic is baked into a set of code and configuration components spread across the system

· Custom pipeline component

· Receive and send port definition

· Custom mapper

· Orchestration

This setup allows flexibility in handling changes. For example, a new message decoder can be added for incoming messages, by simply adding a custom decoder step in the receive pipeline.

Business logic is encapsulated into the body of method (of course, factored appropriately into a class libraries etc.).

This setup is makes it easy to single step through the code. But this also means that every aspect of the logic manifests itself as code. In essence, you need to roll out a new version of binary to make a change.

Large message handling

· Every message in BizTalk by default is routed via the message box. This makes it a challenge to handle large messages. However, there are well known workarounds for this. One of them is described in detail by Paolo

http://blogs.msdn.com/b/paolos/archive/2010/05/25/large-message-transfer-with-wcf-adapters-part-1.aspx.

Large message handling

Type safety

· WCF adapters use untyped contracts for receiving messages. This allows it to receive any type of message.

Type safety

· Contracts can be typed and un-typed

No built-in caching

Caching

Secure Storage Service(SSS) to secure store credentials

No equivalent capability

What can we expect next?

The latest release of BizTalk 2010 is already bridging the gap between the two products. For instance, AppFabric connect feature of BizTalk 2010 offers ways to leverage BizTalk capabilities inside AppFabric hosted programs. Specifically:

1) Ability to leverage the BizTalk mapping capability inside a WF program using the Mapper activity.

2) Ability to generate WF activities that invoke operations on WCF LOB adapters provided by BizTalk.

For more information see – http://blogs.msdn.com/b/biztalk_server_team_blog/archive/2010/06/09/biztalk-appfabric-an-introduction.aspx

I expect this trend to continue, in fact, I envision that a future version of App Fabric would look something like the diagram below (again, this is pure conjecture on my part). As you can see, the BizTalk provided capabilities will be moved into AppFabric (shown in red). This will allow WCF endpoints to take advantage of guaranteed delivery, as well as, the configuration flexibility offered by BizTalk ports (pipeline components, mapper etc.).

 

image

 

 

Call to Action

It is clear that AppFabric and BizTalk are moving closer together. In future, we can expect to have access to capabilities offered by both, rather than choosing between them.

Messaging, with its defining characteristics including adapters, reliable communication and mediation, is generally accepted as the preferred way to connect disparate systems (look no further than Gregor Hohpe’s seminal book on Enterprise Integration Patterns for more on messaging). As we have seen in this post, BizTalk has built-in support for messaging. This is why it is recommended that BizTalk be considered for most integration scenarios

AppFabric should be considered for hosting most of the application logic. Microsoft is making strategic investment in AppFabric to not only to bring advanced management and robust hosting capabilities but also as a way to seamlessly enable application migration between on-premise and the cloud based deployments (commonly referred to as Server / Service symmetry).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: