M13

M13 Application Development Paradigms for large-scale enterprise applications:

1. All business functionality is thought of in terms of one or more services. An application includes a set of services and their implementers/providers.
2. A Service does not automatically imply Web Service.
3. A Service is defined by a Java interface and its implementers are Java classes. A Service can have multiple implementers.
   
4. Service definitions and implementers are independent of how the services are provisioned (Web Service, Java API, EJBs etc.), and are independent of the UI Channel (Browser DHTML, Browser Flash, Adobe AIR, MS WPF, Java Swing etc.).
5. Service provisioning is declarative meta-data driven.
   
6. Service implementers can be switched with configuration entry changes and without any code change. Services can optionally be versioned and it is possible to have multiple versions of a service active in the system at the same time.
   
7. For remote clients, a client-side service API is available for clients to code against.
   
8. Services produce and consume strongly typed objects and not XML. Optimized binary formats are preferred over text-based formats for data transfer across the wire
   
9. UI Components bind to strongly-typed, well-defined data models produced directly by the services without any intermediate transformations.
   
10. A common canonical form for clients across UI Channels is impractical and infeasible for any real-life application. Clients are written separately for each UI Channel in its own technology but they share the same services layer.
11. All application data can be broken up into 2 clear and distinct categories - meta-data and data.
12. Application structure and flow is meta-data driven.
13. All meta-data is efficiently query-able and stored in a relational data store. Meta-data is versionable and upgradable, and it is possible to have multiple versions of a meta-data element active in the system at the same time.
    

Web Services Abuse

Web Services is arguably one of the most significant advance in enterprise software systems in the last 5 years. Web Services have provided a mechanism to unlock and expose data and business logic buried within legacy systems in a uniform and standardized fashion. This is significant because most enterprises use a multitude of different software systems for various operational and strategic functions. These systems tend to be very different in their architectures, platforms, programming languages, interfaces and data stores. Web Services has made it is possible to have a single standard mechanism to access all systems.

Business processes in enterprises often involve accessing more than one system and require information flow among different systems. With Web Services-based standardized programmable access to all systems, we now have an opportunity to create so-called "composite applications" which allow seamless transfer of data and invocation of business logic across multiple systems.

SOAP is an important standard for Web Services. While the simpler REST-based Web Services are popular, SOAP is important in Web Services where security and reliability are important. For both SOAP- and REST-based Web Services, XML is the de facto standard for the format in which data is produced and consumed by services. This text-based format with a standardized and well understood structure is one of the reasons for the easy adoption of Web Services.

While Web Services is a great choice for enterprise system integration, it is also widely used where it probably shouldn't be. The primary scenario where Web Services is not the best choice is when it is used to integrate sub-systems within a system built by the same vendor, and in particular when there is a large amount of data transfer among these sub-systems. Often sub-systems owned by different development sub-groups are built independently without attempting to standardize on common infrastructure and inter sub-system service contracts. This is typically a result of development groups growing unmanageably large (see A Few Good Men) and sub-groups inherited from mergers and acquisitions. Each sub-system tends to adopt its own architecture, frameworks, metadata and data formats, and access mechanisms, ending up looking very different even though they may all be part of the same product suite sold by the same vendor. When it comes time for these sub-systems to talk to each other, the challenge is easily addressed with the magic words - Web Services. Each sub-system builds a Web Services layer and now they can all talk to each other fine. Problem solved. In reality though, there is a great deal of inefficiency in this approach. Often these sub-systems live in the same runtime and communication among them does not involve crossing process boundaries. In these cases, what should be simple co-located service API calls exchanging objects between sub-systems, becomes an inefficient process with unnecessary penalty of protocols such as SOAP and the overhead of XML processing. XML is by nature verbose and parsing of XML is expensive even with modern parsers. When the volume of data is large, the performance overhead of object->XML->Object transformations can be very significant. Data exchanges in binary format are typically orders of magnitude more efficient that text-based formats. This study illustrates graphically a benchmark that support this argument.

The need for loose coupling among sub-systems is often cited as a reason for integrating them through Web Services. The specious argument goes like this - if two sub-systems expose APIs and share object models, any change to one sub-system will require re-compilation of the other which is undesirable. However, what is often overlooked is that even with Web Services, if the format of the XML data produced by a service changes, the consumer of the XML has to re-work the business logic code requiring re-compilation and re-deployment.

The right paradigm for designing systems is to adopt the M13 services paradigm. All business functionality is thought of in terms of services. Services do not necessarily mean Web Services. One defines Java service interfaces that produce and consume serializable Java objects, and builds their implementations. This should be independent of how the services are exposed and invoked. An external declarative provisioning mechanism is available that dictates the best mode of service consumption - plain Java APIs, Web Services, EJBs etc. The underlying infrastructure insulates the author of the service from the provisioning and lets him/her focus on the service contract and business logic. It also allows for employing the optimal service invocation mechanism depending on the scenario. Of course, this requires all sub-systems to conform to a common service authoring paradigm such as M13.

Web Services are great for exchanging small targeted data between disparate systems. It may be the wrong choice when you have full control over systems on either end of the wire. In those cases, consider more efficient alternatives.

Checked versus Unchecked Exceptions in Java

There has always been a debate in the Java community about checked versus unchecked exceptions. Java purists would argue that unchecked exceptions should be used only for conditions you cannot recover from at runtime. On the other hand, in recent years the thinking has shifted a bit (or is it muddled a bit?) with successful frameworks such as Spring promoting heavy use of unchecked exceptions. In the case of lightweight frameworks such as Spring, one of the principal value proposition is the fact that you write very little code to get your job done. Use of unchecked exceptions helps promote that value because you don't need to have try/catch blocks in your code or be forced to declare the exceptions thrown by a method. I believe a significant percentage of software applications are lightweight in nature (see Who produces the most applications software?) with their creators looking for simple, lightweight frameworks such as Spring to get the job done. This is the reason for the widespread adoption and success of such frameworks. For lightweight applications, the use of unchecked exceptions works just fine.

As you move up the chain to more complex applications, the reasoning on exceptions changes. When you have an application with many sub-components written by different groups containing hundreds of thousands of line of code, there is more of a need for rigor and discipline. Building complex software applications is still very difficult and costly. One of the biggest cost in such systems is diagnosing and fixing bugs after the software has been shipped to, and deployed at customer sites. For such systems anything that helps reduce the chances of code problems at development time, or helps diagnose problems after deployment is of immense value. Checked exceptions help towards that goal. Checked exceptions are needed to ensure compile-time detection and enforcement of handling abnormal conditions. Checked exceptions force the developer to think through exception scenarios and code in appropriate exception handlers. These handlers can take appropriate action such as logging state at the time of occurrence of the exception for easy error diagnosis, freeing of any resources for avoiding leaks, applying some corrective business logic such as re-initializing state, or re-casting the exception to fit a service contract. With unchecked exceptions where there is a handler for all exceptions at the top level of the application, all this is not possible.

The natural question that comes to mind is of course - what is considered a complex application and what is considered a lightweight application? Some applications clearly fall in either ends of the complexity spectrum, but many applications fall in a big gray area in between. Complexity also tends to a relative term - complex for one development group may be simple for another. For the majority of such applications in the middle, there is no formula that can be applied to pick the right choice of exception management. The correct approach involves a judicious mix of both checked and unchecked exceptions. I tend to follow the following rules:

• Use checked exceptions for coarse service contracts to external clients you have no control over. Exceptions are part of the formal service contract you are publishing to your clients.
  
• Use checked exceptions for situations that are likely to occur due to invalid input to your services e.g. client called a login service without providing a user id.
  
• Use unchecked exceptions within your own service implementation code that performs very basic operations, is exercised very frequently, and where exceptions are unlikely to occur unless there is a very basic flaw in the programming logic. e.g. indexing into a list with an index that is out of bounds. Imagine how painful our code would be if the java.lang.IndexOutOfBoundsException exception of the widely-used java.util.List::get(int) was a checked exception! In fact I use the List analogy and ask myself if the exception scenario is at the same level as the List indexing example. If it is, I use an unchecked exception; if not I use a checked exception.
  
• Use unchecked exceptions for conditions from which the program cannot or should not attempt to recover e.g. running out of system memory.
  

SOA

TSS Article
http://www.theserverside.com/tt/articles/article.tss?l=ChurchandState