Sunday, February 10, 2008

M13

M13 Application Development Paradigms for large-scale enterprise applications:
  1. All business functionality is thought of in terms of one or more services. An application includes a set of services and their implementers/providers.
  2. A Service does not automatically imply Web Service.
  3. A Service is defined by a Java interface and its implementers are Java classes. A Service can have multiple implementers.
  4. Service definitions and implementers are independent of how the services are provisioned (Web Service, Java API, EJBs etc.), and are independent of the UI Channel (Browser DHTML, Browser Flash, Adobe AIR, MS WPF, Java Swing etc.).
  5. Service provisioning is declarative meta-data driven.
  6. Service implementers can be switched with configuration entry changes and without any code change. Services can optionally be versioned and it is possible to have multiple versions of a service active in the system at the same time.
  7. For remote clients, a client-side service API is available for clients to code against.
  8. Services produce and consume strongly typed objects and not XML. Optimized binary formats are preferred over text-based formats for data transfer across the wire
  9. UI Components bind to strongly-typed, well-defined data models produced directly by the services without any intermediate transformations.
  10. A common canonical form for clients across UI Channels is impractical and infeasible for any real-life application. Clients are written separately for each UI Channel in its own technology but they share the same services layer.
  11. All application data can be broken up into 2 clear and distinct categories - meta-data and data.
  12. Application structure and flow is meta-data driven.
  13. All meta-data is efficiently query-able and stored in a relational data store. Meta-data is versionable and upgradable, and it is possible to have multiple versions of a meta-data element active in the system at the same time.

Tuesday, January 15, 2008

Enums in ActionScript 3

The introduction of Enums in Java 1.5 was a welcome addition. Prior to Java 1.5, one had to simulate Enums with some special patterns. In ActionScript 3 there is no native enum construct in the language. So we need to apply patterns similar to what we did in Java prior to 1.5. While similar, the pattern for ActionScript is a bit different.

Let's start with the basic Enum pattern in ActionScript:

package enumexample
{
public class Color
{
public var _value:String;

public function Color(val:String):void {
this._value = val;
}

public static const RED:Color = new Color("RED");
public static const BLUE:Color = new Color("BLUE");
public static const GREEN:Color = new Color("GREEN");
}
}


The issues with the above Enum implementation are:
  • The constructor is public. So one could do: var c:Color = new Color("FOO"); which violates the notion of the Color Enum constants being restricted to RED, GREEN or BLUE. ActionScript does not allow private constructors.
  • Color does not have a default no-arg constructor which is needed if an instance of Color is de-serialized by the Flash Player from data from the back-end e.g. when using LCDS's RemoteObject to invoke a Java method that returns an instance of Color (see this for LCDS Enum support). For default serialization based on property-value pairs, not only do we need a no-arg constructor but we also need to make _value public or provide a setter. Both these are undesirable because one could change an enum as follows: var c:Color = Color.RED; c._value = "MAROON";
  • If an Enum cannot guarantee uniqueness of enum constant instances, object comparisons based on == will fail. var c:Color = new Color("RED"); if (c == Color.RED); // false. ActionScript does not have the notion of operator overloading.
  • When a Color instance is de-serialized, an instance is first created using the default no-arg constructor and the _value property is then populated. There is no object substitution mechanism available in ActionScript like the readResolve() Java serialization mechanism. So created instances will fail the == comparison as Enum constant instance uniqueness cannot be guaranteed.

The solution I use in conjunction with LCDS RemoteObject is to have Color implement Externalizable and have readExternal() and writeExternal() methods read/write value for _value. In addition, the class is "locked" after the initial construction of the valid Enum constant instances - see the lock member in the Color class below. An exception is thrown when an attempt is made to construct an Enum instance after it is locked. However, since the de-serialization will construct an Enum instance with no args, the exception is not thrown if no args are specified to the constructor. Unfortunately there is no way to detect in the constructor that the instance is being created in the context of de-serialization. The _value member is made private. An equals() method is added which is the way to compare enum instances rather than using ==. However, this is just a guideline/convention and the use of == cannot be enforced. Here's what Color looks like finally - the best one can do given the limitations of the language.

package enumexample
{
[RemoteClass(alias="javaenumexample.Color")]
public class Color implements IExternalizable
{
private var _value:String;
private static var lock:Boolean=false;

public function Color(val:String=null):void {
if (lock && val != null)
throw new Error("Cannot instantiate Enum");

this._value = val;
}

public function get value():String {
return this._value;
}

public static const RED:Color = new Color("RED");
public static const BLUE:Color = new Color("BLUE");
public static const GREEN:Color = new Color("GREEN");

{
lock = true;
}

public function equals(enum:Color):Boolean {
if (enum == null)
return false;

return enum.value == this._value;
}

public function readExternal(input:IDataInput):void {
this._value = input.readUTF();
}

public function writeExternal(output:IDataOutput):void {
output.writeUTF(this._value);
}
}
}

Saturday, January 5, 2008

Why Software Quality Sucks

It was an important customer demo at a big industry trade show. We had worked franticly all through the previous night to get the systems setup for this demo. The demo began very well but halfway through, the program crashed. It was a disaster! Since the demo was for a large and important potential customer, there were a whole bunch of my company suits (non-techie executives) present and they were clearly upset. Apparently they had seen this all too often and couldn't understand why we developers were so incapable of producing bug-free software, and why we should be paid so much money for the junk we produce! At the hospitality suite that night, after everyone was happily buzzed and relaxed, I accosted some of these suits to try and explain to them what had happened and provide excuses for the failed demo. After the usual excuses about demo'ing with the latest unstable dev build, lack of sufficient resources to code, this other team not delivering their fixes on time, bad hardware etc. (all of which they had probably heard many times before), the conversation drifted on to a more generic discussion on software and bugs. I found myself trying to impress upon them the difficulty of building bug-free software. Now this was easy and I felt confident I could convince them because I had Bill Gates on my side. That's right, Bill Gates of Microsoft. It so happened that around the same time as this trade show, Bill Gates was demo'ing Windows 98 at COMDEX, introducing it to the whole world for the first time. And it crashed and burned in the middle of the demo! An embarrassed Gates mumbled something about needing to iron out some more bugs. I mentioned this incident to my suits. Some of them had read about it in the papers. I told them Microsoft had 2000 very bright engineers working on Windows 98 and asked rhetorically why they still couldn't guarantee bug-free code. The software we were building and demo'ing was incredibly complex large-scale, real-time, network monitoring and control systems with hardware interfaces to receive 100s of thousands of data points, and heavy-duty number crunching in mission-critical, high-availability systems. I reminded them that we had just 3 engineers working on the portion of the demo that crashed. The point was that if Microsoft, with its huge army of the world's best developers, cannot produce bug-free software, how could we, with far fewer resources to build equally complex systems, produce anything better? The suits seemed to see the point. By the end of the night I had totally convinced them that software is, by nature, very complex and it was virtually impossible to build systems of better quality than Windows. One had to just live with the fact that software systems will malfunction and crash periodically. Thanks Bill!

That was 10 years ago. Windows has come a long way since those days and we now see less of the infamous "blue screen of death". Software in general has matured more in the last decade. However, we are nowhere close to the level of maturity in commercial software where high quality is taken for granted. Any enterprise-scale software is fraught with bugs requiring continuous application of hot-fixes, patches, upgrades etc. Ask any IT professional in any company about the quality of software he supports and invariably you will hear complaints. Complaints are likely to be more vociferous when it comes to business applications software.

So why does software quality in general and business applications software in particular suck so badly? Why is it that software is not as reliable as (say) bridges? Some would argue that the software industry is still very new. After all, they argue, we've been building software for less than 50 years while we've been building bridges for 1000s of years. Others would argue that software is much more complex than building bridges or cars, and as complexity increases, so does the probability of defects. There are some who would point to lack of standardization in the way certain common software elements are built resulting in defects when trying to integrate elements into one application. Then there is the camp that feels that a significant number of software engineers lack the proper training and skills and that the software industry is probably the only industry where we don't make a distinction between technicians and engineers (think of electrician versus electrical engineer, plumber versus civil engineer etc.) - in software, anyone who works with code is a software engineer and problems arise when technicians try to do the job of engineers.

While all of the above reasons may be valid, I believe the fundamental reason for poor software quality is the way we build it. This is more true for applications software than systems software. In spite of all the advances in languages, IDEs, design patterns, open-source libraries, runtimes and standards, building large applications software remains horrendously tedious, labor intensive and error prone. I think we need a radical shift in the fundamental mechanics of writing code in a programming language. We need a new vocabulary for application developers to state their business logic rather than code it. Modeling tools have attempted to address some basic part of this but have been very code and programming oriented e.g. thinking in terms of classes in UML. Model Driven Architectures (MDA) never lived up to the hype and simply provide a language to describe the high-level problem without providing a real-life solution.

I have no idea what this new vocabulary and mechanics for building software would look like in its final form (although I have a name for it - Yeti), but it seems to me intuitively that it will have the following characteristics:
  • Rules would figure in it prominently, and application building will primarily involve describing business rules
  • Application logic would be expressed in a natural language syntax
  • Application builders would not be programming (as we know it) in languages such as Java, C++ or JavaScript
  • Applications would be able to process incomplete information employing fuzzy logic, and learning

Maybe I'll never see Yeti in my lifetime. Maybe Yeti simple cannot exist given the basic von Neumann model for computers.
Or maybe not.

Sunday, December 30, 2007

Web Services Abuse

Web Services is arguably one of the most significant advance in enterprise software systems in the last 5 years. Web Services have provided a mechanism to unlock and expose data and business logic buried within legacy systems in a uniform and standardized fashion. This is significant because most enterprises use a multitude of different software systems for various operational and strategic functions. These systems tend to be very different in their architectures, platforms, programming languages, interfaces and data stores. Web Services has made it is possible to have a single standard mechanism to access all systems.

Business processes in enterprises often involve accessing more than one system and require information flow among different systems. With Web Services-based standardized programmable access to all systems, we now have an opportunity to create so-called "composite applications" which allow seamless transfer of data and invocation of business logic across multiple systems.

SOAP is an important standard for Web Services. While the simpler REST-based Web Services are popular, SOAP is important in Web Services where security and reliability are important. For both SOAP- and REST-based Web Services, XML is the de facto standard for the format in which data is produced and consumed by services. This text-based format with a standardized and well understood structure is one of the reasons for the easy adoption of Web Services.

While Web Services is a great choice for enterprise system integration, it is also widely used where it probably shouldn't be. The primary scenario where Web Services is not the best choice is when it is used to integrate sub-systems within a system built by the same vendor, and in particular when there is a large amount of data transfer among these sub-systems. Often sub-systems owned by different development sub-groups are built independently without attempting to standardize on common infrastructure and inter sub-system service contracts. This is typically a result of development groups growing unmanageably large (see A Few Good Men) and sub-groups inherited from mergers and acquisitions. Each sub-system tends to adopt its own architecture, frameworks, metadata and data formats, and access mechanisms, ending up looking very different even though they may all be part of the same product suite sold by the same vendor. When it comes time for these sub-systems to talk to each other, the challenge is easily addressed with the magic words - Web Services. Each sub-system builds a Web Services layer and now they can all talk to each other fine. Problem solved. In reality though, there is a great deal of inefficiency in this approach. Often these sub-systems live in the same runtime and communication among them does not involve crossing process boundaries. In these cases, what should be simple co-located service API calls exchanging objects between sub-systems, becomes an inefficient process with unnecessary penalty of protocols such as SOAP and the overhead of XML processing. XML is by nature verbose and parsing of XML is expensive even with modern parsers. When the volume of data is large, the performance overhead of object->XML->Object transformations can be very significant. Data exchanges in binary format are typically orders of magnitude more efficient that text-based formats. This study illustrates graphically a benchmark that support this argument.

The need for loose coupling among sub-systems is often cited as a reason for integrating them through Web Services. The specious argument goes like this - if two sub-systems expose APIs and share object models, any change to one sub-system will require re-compilation of the other which is undesirable. However, what is often overlooked is that even with Web Services, if the format of the XML data produced by a service changes, the consumer of the XML has to re-work the business logic code requiring re-compilation and re-deployment.

The right paradigm for designing systems is to adopt the M13 services paradigm. All business functionality is thought of in terms of services. Services do not necessarily mean Web Services. One defines Java service interfaces that produce and consume serializable Java objects, and builds their implementations. This should be independent of how the services are exposed and invoked. An external declarative provisioning mechanism is available that dictates the best mode of service consumption - plain Java APIs, Web Services, EJBs etc. The underlying infrastructure insulates the author of the service from the provisioning and lets him/her focus on the service contract and business logic. It also allows for employing the optimal service invocation mechanism depending on the scenario. Of course, this requires all sub-systems to conform to a common service authoring paradigm such as M13.

Web Services are great for exchanging small targeted data between disparate systems. It may be the wrong choice when you have full control over systems on either end of the wire. In those cases, consider more efficient alternatives.

Checked versus Unchecked Exceptions in Java


There has always been a debate in the Java community about checked versus unchecked exceptions. Java purists would argue that unchecked exceptions should be used only for conditions you cannot recover from at runtime. On the other hand, in recent years the thinking has shifted a bit (or is it muddled a bit?) with successful frameworks such as Spring promoting heavy use of unchecked exceptions. In the case of lightweight frameworks such as Spring, one of the principal value proposition is the fact that you write very little code to get your job done. Use of unchecked exceptions helps promote that value because you don't need to have try/catch blocks in your code or be forced to declare the exceptions thrown by a method. I believe a significant percentage of software applications are lightweight in nature (see Who produces the most applications software?) with their creators looking for simple, lightweight frameworks such as Spring to get the job done. This is the reason for the widespread adoption and success of such frameworks. For lightweight applications, the use of unchecked exceptions works just fine.

As you move up the chain to more complex applications, the reasoning on exceptions changes. When you have an application with many sub-components written by different groups containing hundreds of thousands of line of code, there is more of a need for rigor and discipline. Building complex software applications is still very difficult and costly. One of the biggest cost in such systems is diagnosing and fixing bugs after the software has been shipped to, and deployed at customer sites. For such systems anything that helps reduce the chances of code problems at development time, or helps diagnose problems after deployment is of immense value. Checked exceptions help towards that goal. Checked exceptions are needed to ensure compile-time detection and enforcement of handling abnormal conditions. Checked exceptions force the developer to think through exception scenarios and code in appropriate exception handlers. These handlers can take appropriate action such as logging state at the time of occurrence of the exception for easy error diagnosis, freeing of any resources for avoiding leaks, applying some corrective business logic such as re-initializing state, or re-casting the exception to fit a service contract. With unchecked exceptions where there is a handler for all exceptions at the top level of the application, all this is not possible.

The natural question that comes to mind is of course - what is considered a complex application and what is considered a lightweight application? Some applications clearly fall in either ends of the complexity spectrum, but many applications fall in a big gray area in between. Complexity also tends to a relative term - complex for one development group may be simple for another. For the majority of such applications in the middle, there is no formula that can be applied to pick the right choice of exception management. The correct approach involves a judicious mix of both checked and unchecked exceptions. I tend to follow the following rules:

  • Use checked exceptions for coarse service contracts to external clients you have no control over. Exceptions are part of the formal service contract you are publishing to your clients.
  • Use checked exceptions for situations that are likely to occur due to invalid input to your services e.g. client called a login service without providing a user id.
  • Use unchecked exceptions within your own service implementation code that performs very basic operations, is exercised very frequently, and where exceptions are unlikely to occur unless there is a very basic flaw in the programming logic. e.g. indexing into a list with an index that is out of bounds. Imagine how painful our code would be if the java.lang.IndexOutOfBoundsException exception of the widely-used java.util.List::get(int) was a checked exception! In fact I use the List analogy and ask myself if the exception scenario is at the same level as the List indexing example. If it is, I use an unchecked exception; if not I use a checked exception.
  • Use unchecked exceptions for conditions from which the program cannot or should not attempt to recover e.g. running out of system memory.



Who produces the most Applications Software?


If I asked you who produces the largest amount of business applications software, you'd probably say Oracle, SAP, Microsoft, IBM or one of many software vendors that produce and sell such software. Until recently I would have agreed with you completely. But I'm beginning to realize that the maximum amount of applications software is produced by IT groups within companies and not by large software vendors. Although companies spend millions of dollars on buying software applications to run their business, they also produce a lot of code internally for their own consumption. This could be enhancements or customizations to applications they purchased because the purchased software did not meet their business needs out-of-the-box. It could also be brand new applications they build specific to their business for which no commercial software is readily available. Most of these applications tend be simple (and interestingly short-lived) since IT groups tend to be very constrained in terms of budgets and skills to engage in full-fledged, large-scale software development. However, given the massive number of companies globally that use software for their daily operations, the total number of such applications is very large. Interestingly, it is because of this large market for simple, light-weight applications that light-weight frameworks such as Spring have gained a lot of popularity.

So who produces the most applications software? Combined IT groups of companies do.

Saturday, December 29, 2007

A Few Good Men


One of the classic problems in software development is to accurately estimate how many developers are needed to complete a task. In my 15+ years of commercial software development in small and large companies, I've yet to come across a project where the resource estimate at the beginning of the project ended up being anywhere close to being correct. Invariably the estimate is much lower than what it actually takes to complete the project. Why is this? I think it is because some of the intangible aspects of developer efficiency are overlooked in the resource estimation process.

What is the optimal size of a developer group engaged in a project with shared, common, inter-dependent tasks? I believe the optimal size is 1! One brilliant developer with all the required skills produces software in the most efficient fashion. Of course, it is going to take forever for a team of one developer to complete any reasonable size project, and obviously most teams need to have more than one developer. My thesis is that with every additional developer, efficiency of the group decreases by 5% over the previous level. So if a team of 1 developer works at 100% efficiency, a team of 2 developers works at 95% efficiency (0.95 x 100), a team of 3 developers works at 90.25% efficiency (0.95 x 95), a team of 4 developers works at 85.74% efficiency (0.95 x 90.25), and so on. This is shown graphically in Figure 1 below where the yellow line indicates the decrease in efficiency level as the number of developers increases.

Figure 1. Developer Value


Why is it that efficiency decreases as the number of developers increases? The decrease is due to many factors. The principal factor is related to source code dependency and version management. When a group of developers share a common code base, there is invariably inefficiencies related to:
  • module inter-dependency ("my change broke his code"), and
  • code sharing ("I have to rebase to his changes first before delivering my changes to the same file").
Other factors leading to lower efficiency of larger group sizes include:
  • difficulty in ensuring a common understanding of the project design goals and principles ("I thought you meant this rather than that 3 months ago"),
  • differing developer backgrounds, styles and skill levels ("this guy's code is so cryptic, I'd rather re-write it than try to figure it out"), and
  • the increasingly distributed nature of development with language, culture and time-zone related communication problems ("wish I could quickly and easily get this guy in Bangalore or Minsk to understand exactly what I mean")

As the number of developers increases, the output obviously increases. This increase is linear and is shown graphically in Figure 1 as the blue line. However, this output does not factor in the cost of inefficiencies mentioned above. The inefficiencies introduce a cost penalty that needs to be factored in to the calculation of true "developer value". This value is shown in Figure 1 as the green line. Note that the rate of increase of developer value is lower than the rate of increase of developer output as the number of developers increases. Resource estimates are typically based on the output curve, when in fact they should be based on the value curve. The difference between the output curve and the value curve is called the "estimate gap" and is indicated by red double-arrow lines between the blue (output) and green (value) curves. Missing this estimate gap is the reason for poor and lower estimation of required resources for projects.

So what does this value curve mean? Let's say you estimate that you need 5 developers to complete a task based on amount of code that needs to be written. In Figure 1, assume that the y-axis is re-scaled so that 20% output is 100% for your task - so 5 developers are estimated to be required to complete 100% of the task. From the output and value curves, you actually need 6.7 developers when you factor in the estimate gap. The table below shows the mapping between estimated number of developers based on output and required number of developers based on value. This mapping can be used to arrive at more accurate resource estimates without just going with the incorrect estimate based on output.

# Developers Estimated
(Output)
# Developers Actual
(Value)
1
1
2
2.1
3
3.3
4
5
5
6.7
6
9
7
13

An interesting aspect of the value curve is that there is an inflection point around 21 developers (indicated in Figure 1) after which the value actually starts decreasing! This means that a group of 21 or more developers working on shared, common, inter-dependent project tasks is likely to be a loss making proposition. In terms of estimated number of developers based on output corresponding to 21 developers based on value, the number turns out to be between 7 and 8. This means that if your estimate arrives at more than 7 developers based on output (which is more than 13 developers based on value), you run a huge risk of failure. In such a scenario, re-visit your project tasks and try to break it up into more independent sub-projects. If that is not possible, you are dealing with an immensely complex software system that given today's state of the art, is impossible to build effectively. Hopefully such systems are rare.

So for your next project, bridge the estimate gap to arrive at better resource estimates, and remember the magic upper limit numbers - 7 and 13.