3-Stage Software Testing

One of the 7 Habits of Highly Effective Software Developers relates to Unit Testing. Software Unit Testing is one of the most important but least popular and poorly employed aspects of software development. Often we developers put too much onus on QA (Quality Assurance) teams to test software for bugs. Unfortunately QA's role is only to test things that are not possible for every developer to test e.g. making sure the software runs on all versions of all Browsers or OSs.

The 7 Habits talks of considering unit tests after completion of every quarter of your task and having a test completed by the end of the task. When you write a test, how should it be written? Should it be a full fledged test that can be run and verified with automated testing frameworks or should it be informal tests? While the answer may vary with the size of development groups and the complexity of applications, I like to think of 3 stages in writing unit tests - stages which progressively lead to complete tests.

Stage 1: Crash Test - The first stage is to write a test that can simply verify that things don't crash i.e. no unexpected exceptions are thrown and no abnormal conditions occur. Here's a simple test that invokes a service on another server that returns a value object. Successful running of this test merely ensures that nothing is fundamentally broken.

@Test
public void testReport() throws M13Exception
{
Report report = (Report)svs.getReport("foo");
}

Stage 2: Eyeball Test - The next stage is to examine the results of invoking the functionality being tested. In the example test below, the contents of the Report object returned from the server is printed to a log output. What this test lets you do is to "eyeball" the results printed to get some sense of whether the functionality is working right or not. This stage is an extension to the Crash Test stage.

@Test
public void testReport() throws M13Exception
{
Report report = (Report)svs.getReport("foo");
logger.info("Report: " + report.toXML());
}

Stage 3: Automated Result-Comparison Test - This is the final stage in writing a test and extends the Eyeball Test stage. In this stage, you write code that compares the results of invoking the functionality being tested with pre-defined, expected results. In the example below, the XML representation of the Report object got from the server is compared against a XML data file; the test passes if the XML matches the XML in the data file and fails otherwise. Once this stage is done, the test is complete.

@Test
public void testReport() throws M13Exception
{
Report report = (Report)svs.getReport("foo");
String fooXML = report.toXML();
logger.info("Report: " + fooXML);
// read from data file to compare results with
String reportXML = FileUtils.readFileToString(file, "UTF-8");
assertTrue(fooXML.equals(reportXML));
}

One of the common mistakes people make is to attempt to get to the third stage directly right from the beginning. While this may work for some simpler tasks, for more complex tasks and software, this results in constantly having to change the data set being compared against since it is likely that the definition of the data is changing continuously until the later stages of development.

So the best practice may be to ensure you have Crash tests and Eyeball tests in place after each task (making sure you at least think of testing at each quarter of the task). If the data sets are stable enough, you can also write Automated Result-Comparison tests at this stage. If not, come back to this stage later in the development cycle.

Note that for all stages you employ a formal testing framework such as JUnit. So you start right from scratch with a formalized test that can be run in an automated fashion even if you are only at the Crash Test Stage. This way you are simply expanding on and extending the tests at each stage.

Happy testing! ... and give those QA guys a break!

Adobe MAX 2008

Highlights of the Adobe MAX conference in San Francisco, Nov 2008

Flash Player 10

Flash 10's new text engine ("Vellum") provides advanced text rendering capabilities. Layouts such as text wrapping around images, which is very easy to do in HTML, is now supported in Flash 10. Bi-directional (R-to-L) languages are natively supported. Text rotation does not require embedded fonts in 10. New York times demo-ed a new AIR-based news reader application that takes advantage of Flash 10's new text capabilities. CS4 and Flex Gumbo components use the new text engine which is written entirely in ActionScript.

The ability to allow search engines to index/search content inside a SWF file was unveiled. While it requires some extra work, the fact that this now possible is significant.

The new text and search capabilities address what used to be one of the major drawbacks of Flash for text-oriented content. One wonders if Flash 10 will usher in a new era where we will see more traditional consumer-facing Web sites adopting Flash.

One intriguing announcement was the "Alchemy" project which allows your C++ code to run in the Flash player. This project provides libraries that can be used to generate ActionScript code from C++ which can then run in the Flash Player. A demo of OpenSSL code ported to run in Flash was very interesting. Encryption may be a good example of established, existing C++ code that one may want to run in the Flash Player.

Another cool demo was the new feature that enables direct Player to Player communication using the new RTMFP (Real Time Media Flow Protocol) protocol. This is peer-to-peer communication but does not support file or document sharing. It requires a server for the initial connection hookup between peers.

Flash Catalyst (formerly known as Thermo)

This is a new design tool that allows designers to import creative graphics content from Photoshop or Fireworks and turn them into real application components. The designers can control the look and feel of the application including state transition animations. The output of Catalyst is a "FXP" package that can then be imported into Flex by a developer and used directly.

Catalyst leverages the new XML-based FXG graphics language. This language lets you declaratively define look and feel of components - effectively this is doing programmatic skinning without writing code. Hopefully this will reduce the need for graphical skinning and reduce the proliferation of image assets for skinning and leverage all the advantages of programmatic skinning.

Couple of concerns I have about Catalyst are 1) Will this end up in "XML hell" with projects containing hundreds of FXG XML files, and 2) Will round-tripping between Catalyst and Flex (designer and developer) really work well for large commercial-grade projects?

Flex Gumbo (Flex 4)

This is the next generation of Flex which was previewed and is due to be released next year. The main feature seems to be the new MVC separation at the component level. This ties in with Catalyst - the separation allows for designers to have more control over the look and feel of components using the Catalyst tool. Other than this separation for Catalyst, there does not seem to be anything of major significance. There is some new ColdFusion integration but that is of no use to those of us who don't live in the proprietary world of ColdFusion.

A disappointment for me was that there is no change to facilitate runtime changing of CSS. CSS files are still compiled into the SWF. The same applies to FXG files. If you need to be able to change the CSS/FXG files after deployment of your application at the customer site, it requires custom code to be written to allow this.

AIR

There seems to be more buzz around AIR with several well attended sessions around this technology. The application that won the top award in the Enterprise category was a trading desktop AIR application from NASDAQ.

Adobe announced the new "Wave" AIR application for desktop notification. Using this application, users can get desktop notifications from (say) their favorite social networking sites without having to browse to each site to look for anything new. There is a notification service in the back-end that Adobe hosts and that content producers integrate with.

Sneak preview of "Nitro" widgets provided glimpses of a formalized framework for building widgets that can run on multiple screens. Drag and drop of widgets from the Browser to the desktop reminded me of the JavaFX demos of similar functionality. The "Durango" preview showcased AIR mashups - dragging and dropping application components to weave together a composite AIR app.

AIR, with its platform independence, local database, WebKit embedded Browser, JavaScript and ActionScript scripting capabilities, and of course the Flash Player runtime, seems poised to be a powerful channel for widgets/gadgets. It is possible that AIR on the desktop could be as ubiquitous as Flash in the Browser. Interestingly, downloading the free Adobe Reader 9 also installs AIR by default showing Adobe's desire to try and get AIR on as many desktops as Reader. Smart strategy!

PDF

Acrobat sessions introduced new features of version 9 released recently. The most interesting feature to me was the new embedded Flash player in Reader/Acrobat and better support for a SWF file running inside PDF. It is possible to script JavaScript to interact with the SWF within the PDF. Adobe is moving Acrobat from being a viewer for static documents to a full-blown application platform. Given the popularity of Reader on desktops, this is very promising. However, the ability to debug/profile code in Acrobat needs to be improved if it needs to really be an application platform.

One wonders if Adobe will start blurring the lines between its desktop runtimes - AIR, Flash Desktop Player and Reader.

Business Intelligence

SAP Business Objects had a demo booth where they were showcasing their XCelsius product line. A new demo that I haven't seen before was the ability to export a dashboard to PDF that contains a SWF providing rich interactive capabilities. See demo here on their Web site. Another new demo was their use of AIR for desktop widgets/gadgets delivering BI content. It ships a component SDK that can be used to put together custom dashboards - this is oriented towards IT and consultants rather than the end user. XCelsius uses a home-grown server-side SWF generation/manipulation library.

XCelsius is clearly ahead of the competition in terms of taking advantage of Adobe technologies such as Flash, Flex, AIR and PDF in its BI suite. BI aside, SAP in general seems to have a solid partnership with Adobe. Adobe's PDF generation libraries are baked into all of SAP's OLTP products. They use Adobe Connect Web Conferencing product for their training solutions.

ILOG (soon to be part of IBM) demo-ed their Flash/Flex-based visualization library Elixir. ILOG does have a history of experience in the advanced visualization space although their traditional forte is optimization and business rules. The library is quite rich including Charts, Maps, Org Charts, Gantt Charts, Treemaps and Gauges (they don't have Pivot table). Their library is re-sold by Adobe for $799 per license, but the catch is the hefty deployment fees/royalties which reportedly could cost tens of thousands of dollars a year.

Cocomo

This is a new Platform as a Service offering for building real-time collaboration capabilities leveraging Acrobat.com cloud services and the Connect infrastructure. Cocomo promises to make it easy to add chat, live file sharing, screen sharing, white-boarding and VOIP audio to applications. Given the high quality of Adobe's Connect product, Cocomo is likely to be a winner.

Mobile

While CTO Kevin Lynch demo-ed a number of phones running Flash, the big one was missing - iPhone. Kevin held up an iPhone and said Flash on it was still baking in the oven and that Apple's chief taster hadn't approved it yet! Looks like the obstacle for getting Flash on the iPhone is not a technical one but a business one. Apple is concerned that allowing Flash on the iPhone will result in a Adobe monopoly on the default channel for content and applications with sizzle. Apple however is doing a great disservice to developers because there is a phenomenal advantage to coding with one platform for all screens although you are likely to build the same application differently (scaled down) for a mobile device. I'm still in the process of picking up Objective C, XCode etc. which I need to build an iPhone app. I wish I could write the same ActionScript for the iPhone that I write for the Browser, AIR and PDF.

Miscellaneous

Some random thoughts from the conference:

• One way to feel the pulse of the market and understand trends is to look at which sessions at a conference are the most popular ones. At MAX the most popular and sold-out sessions included (in no particular order) CS4, Search-able SWF, Flex Introduction and ColdFusion. The majority of attendees at the conference seemed to be folks producing creative content using CS4 products and/or folks building Web sites using ColdFusion. CS4 remains Adobe's flagship product suite and hugely popular. I was quite surprised by the fact that ColdFusion, which I had assumed to be legacy stuff, is alive and well. Interestingly ColdFusion seems to enjoy a cult following among its users who don't seem to care that it is a proprietary platform. The popularity of the Search-able SWF session shows that consumer-facing Web sites using Flash are very keen on a solution that lets search engines get to content within their SWFs. This may have been the single biggest concern of Web sites using Flash for whom search hits are important. The fact that Flex introductory sessions were so popular shows that while Flex is gaining a lot of popularity, it is still an early-stage technology that more people are beginning to discover.
• Google's session on Flash support in Google Maps was interesting not just from the perspective of users now being able to use maps natively in their Flash/Flex applications, but for what Google mentioned were the advantages of Flash over JavaScript-based Maps solution - 1) performance advantage of Flash in manipulating 5-10K vertex polygons in real-time and 3D; 2) vector graphics drawing APIs of Flash to be able to do specialized markers such as a rectangle with glow filters or video markers overlay-ed on Maps.
• Scrapblog is a company that won an award in the RIA category. Scrapblog is a tool for creating online multimedia scrapbooks and is an example of what still remains the primary use of Flash - creative design with graphical and audio/video content.

JavaFX - Applets Part II

SUN entered the RIA foray with its announcement at JavaOne of the JavaFX Rich Client Technology - "a family of products for creating RIAs with immersive media and content across all screens of your life". JavaFX technology includes the following:

• A runtime plugin similar to Adobe's Flash Player
• Development SDK including IDE plugins, compiler and debugger
• A new declarative scripting language for building UIs called JavaFX Script
  

JavaFX adds a third player in the RIA competitive landscape - the other two being Adobe's Flash/Flex/AIR and Microsoft's Silverlight/WPF. It validates and reinforces the Web 2.5 future we are heading towards - a future where users enjoy applications that are rich, interactive, engaging, cinematic, responsive and stylish, and where developers don't have to struggle with trying to use Web-page-centric DHTML/AJAX technologies to get even very basic, minimally rich applications to work consistently across Browsers and other "screens of your life". Kudos to SUN for recognizing the future and creating a promising, open-source-based RIA solution.

The JavaFX technology for RIAs is conceptually similar to Adobe's Flex and Microsoft's Silverlight in that it has the following key ingredients:

• A formalized client-side component and event model which is absolutely essential for any RIA with heavy client-side business logic, and which is sorely lacking in the DHTML/AJAX/JavaScript world
• A JIT-compiled Virtual Machine runtime that is far more efficient than interpreted JavaScript in the Browser
• A development SDK with editors, designers, debuggers and profilers
• Built-in support for hardware acceleration and superior vector graphics with a library that supports animations and effects out-of-the-box
  

The runtime plugin is the Java SE 6 Update 10 runtime. For old-timers that probably brings back bad memories of Applets - the poorest contribution from SUN to the Java world (some would argue that abysmal position belongs to EJB 2.x!). When most people hear Applets they think of massive downloads, install and configuration issues, browser freezes/crashes, Browser back button problems, and ugly UIs. Of course some of the issues are really issues with the Swing technology whose failure caused the emergence of superior technologies such as IBM's (now Eclipse's) SWT. JavaFX is fundamentally a similar Applets technology - in fact the JavaFX site refers to JavaFX programs as Applets (if I were the marketing guy at SUN, I'd call it anything but Applets). However, it looks like there have been some improvements such as:

• Size of the plugin core needed for typical applications is only a few MB
• Grahics capabilities look much more refined, competing with Flash and Silverlight UIs
• Installation, configuration and launch may be easier with the deployment toolkit and JNLP support
• Runtime performance has been improved
  

In spite of all of the above, SUN's team would have to work very hard to convince developers that they have a real, commercial-grade solution. The demos at JavaOne kept crashing, but of course that was because of Moscone center's bad network right?! Seriously, early stage demos apart, given the Applets and Swing history, I wouldn't risk real money on this technology until I am fully convinced that it is really more that a fun project for the Computer Science PhDs at SUN. Once bitten, twice shy.

One interesting aspect of this technology is the new language - JavaFX Script. JavaFX Script is a declarative, statically typed language for defining GUIs and application behavior. It provides data binding capabilities for synchronizing UI element state with application data. It is possible to call Java code from JavaFX Script making it easy to leverage existing Java APIs such as the Swing toolkit. It is unclear if we will have something analogous to the Flex-AJAX bridge for JavaFX Script-to-JavaScript communication. With JavaFX Script you can do true multi-threaded applications but I am reminded of Swing developers often shooting themselves in the foot with UI update and the event-dispatch thread. JavaFX Script is compiled to JVM bytecode providing superior execution performance. Some other academically interesting features of the language are described later in this post.

JavaFX Script can be compared to Adobe's MXML/ActionScript and Microsoft's XAML/C#. However, unlike MXML and XAML, JavaFX does not have an XML description of the application UI tree. The UI tree description and the business logic behavior is combined into JavaFX scripts. As a developer looking at Flex and Silverlight to produce commercial software, the first reaction to JavaFX Script is - oh no, another language! Why not just do what Adobe did with ActionScript? - adopt the ECMA Script standard and enhance it where needed. While the ability to work with Java from within JavaFX Script is appealing to Java developers, ActionScript's object oriented-ness and semantics is also quite natural for a Java developer. I wonder if SUN could have served developers better by sticking with the ECMA Script standard which is the basis for ActionScript and JavaScript, and extended it for supporting Java invocation and any other features as needed.

Some other features of the JavaFX technology include:

• Ability to run an application both in the Browser and on the desktop. In fact you could drag an application out of the Browser and drop it on to your desktop and continue working as a full desktop application. JavaFX applications run in a separate process. Conceivable a JavaFX application crash will not crash the browser - along the same lines of tabs in Google's Chrome Browser running in separate process spaces.
  
• JavaFX applications communicate with the server back-end either through Web Services or through RMI. Web Services are extremely useful; but for heavy-duty applications, more efficient mechanisms such as AMF3 over HTTP that Flash/Flex supports is needed (see this). Perhaps someone could implement AMF3 over HTTP support to JavaFX as an open-source project. RMI has issues with firewalls requiring complex HTTP-tunneling solutions.
  
• Mobile and TV platform support coming in the future
  

Coming back to the JavaFX Script language itself, some interesting features that may sound unfamiliar to Java developers include:

• Classes have attributes (analogous to fields in Java), functions and operations (analogous to methods in Java)
• The distinction between functions and operations is interesting. One wonders if it adds additional complexity and could have been combined into one construct. Functions are for implementing simple business logic while operations are for complex logic with exception handling. More interestingly, return values of functions are always re-evaluated when the value of any variable referenced by the function changes. This is useful with data binding described below. On a data-binding related note, functions are independent entities that don't need to be associated with a class - this is more of a procedural concept rather than object-oriented one. While the separation of state and behavior is valid on the server for SOA (see this), applying the concept on the client feels weird although it may make sense from a data binding perspective.
• Function and Operation definitions are outside of the class declaration. The class declaration just contains the declaration of the function/operation. Similarly, attribute initialization is defined outside of the class declaration. This feels more C++-ish rather than Java-ish.
• Data binding is a useful feature for UI development. An attribute can be defined to be bound to another attribute or to a function so that its value is dynamically (and optionally lazily) updated whenever the bound target value changes.
• Blocks of code within a "do" or "do later" block execute in a separate thread. While Java developers are familiar with convenient multi-threading mechanisms available in Java, this is the first time a scripting language has provided multi-threading capabilities. While this is a great advantage, it also opens the door for pitfalls for average developers as multi-threading in inherently hard to design and implement correctly for complex applications.
• Triggers is an exciting feature where you can write code that kicks in on state change events such as an attribute value change. This is not unlike database triggers that developers are familiar with.
• Attributes can have cardinality operators such as *, ?, + (think regex) to indicate optional, one or more, and zero or more.
• An attribute can have an inverse declaration indicating a bi-directional relationship with another attribute with automatic update of the inverse on attribute change. This feels like something one does in ORM (Object Relational Mapping) with persistence to a datastore. It seems out of place in a UI development-oriented scripting language.
• Arrays with "list comprehensions" seem extremely useful. This is like writing SQL statements on lists e.g. var a:Integer* = select n*n from n in [1..10] where (n%2 == 0); New keywords such as insert as first, into, before and after facilitate useful list manipulations.
  
• Object literals allow for creating object instances in a JSON-like fashion. While this seems useful, it is likely to lead to very verbose and difficult to read code - see this TableNodeExampleApplet.fx example from a JavaFXpert.
• Multiple inheritance, which Java avoided due to its inherent complexity, is supported by JavaFX Script.
  

JavaFX Script is an exciting new language with some cool features that developers have not used in other scripting languages. Obviously some brilliant minds have been applied to its creation. However, my gut feel is that this is a language that, given its many new constructs, is going to have a steep learning curve and a slow adoption.

Overall, the JavaFX technology is very promising and will help usher in the new Web 2.5 era faster. If SUN plays its cards right, JavaFX may catch up with Silverlight and maybe even with Flash/Flex someday.

Flex TabNavigator Image Snapshot Icons

Example files below illustrate generating thumbnail snapshots of tabs in a Flex TabNavigator. The thumbnails are shows as tab icons.
See this flexcoders thread.

ImgSnapshot.mxml - Main Application File:

<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" creationComplete="init()">

<mx:Script>
<![CDATA[
import mx.events.IndexChangedEvent;
import mx.containers.TabNavigator;

private var tabNav:TabNavigator = new TabNavigator();

private function init():void
{
  // tab1
  var tabChild1:MyComponent = new MyComponent(0);
  this.tabNav.addChild(tabChild1);

  // tab2
  var tabChild2:MyComponent = new MyComponent(1);
  this.tabNav.addChild(tabChild2);

  this.tabNav.percentHeight = 100;
  this.tabNav.percentWidth = 100;
  this.addChild(this.tabNav);
}     
]]>
</mx:Script>

</mx:Application>

MyComponent Class - MyComponent.as:

package
{
  import flash.events.Event;
 
  import mx.charts.PieChart;
  import mx.charts.series.PieSeries;
  import mx.collections.ArrayCollection;
  import mx.containers.HBox;
  import mx.containers.TabNavigator;
  import mx.core.Container;
  import mx.events.FlexEvent;
 
  public class MyComponent extends Container
  {
 private var box:HBox = new HBox();
 private var chart:PieChart = new PieChart();
 private var index:int;
 
 public function MyComponent(indx:int)
 {
     super();
   
     this.index = indx;
     this.label = "Tab " + this.index;
     this.percentHeight = 100;
     this.percentWidth = 100;
  
     this.addEventListener(FlexEvent.UPDATE_COMPLETE, updateComplete);
 }
  
 override protected function createChildren():void
 {
     super.createChildren();        
     this.box.addChild(this.getPie());   
     this.addChild(this.box);
 }
  
 override protected function updateDisplayList(
   unscaledWidth:Number, unscaledHeight:Number):void
 {
     super.updateDisplayList(unscaledWidth, unscaledHeight);
     this.box.setActualSize(unscaledWidth, unscaledHeight);
     this.chart.setActualSize(unscaledWidth, unscaledHeight);
 }
  
 private function updateComplete(event:Event):void
 {
     var tabNav:TabNavigator = TabNavigator(this.parent);
     if (this.width == 0 && this.height == 0) {
  var selTab:MyComponent = MyComponent(tabNav.getChildAt(tabNav.selectedIndex));    
    
         this.width = selTab.width;
         this.height = selTab.height;
         this.updateDisplayList(selTab.width, selTab.height);
        
         return;
     }         
     if (this.icon == null)
         this.icon = IconUtil.getIconClass(this, tabNav.getTabAt(this.index), 150, 100); 
      }
  
 private function getPie():PieChart
 {   
     chart.percentWidth = 100;
     chart.percentHeight = 100;
     chart.showDataTips = true;
     chart.dataProvider = this.getChartDataProvider();

     var series:PieSeries = new PieSeries();             
     series.nameField = "label";
     series.field = "data";
     series.filters = [];
     chart.series = [series];
   
     return chart;
 }
  
 private function getChartDataProvider():ArrayCollection
 {
     var arr:Array = [];
           arr.push({label:"East", data:24});
           arr.push({label:"West", data:32});
           arr.push({label:"North", data:22});
           arr.push({label:"South", data:11});
           arr.push({label:"Texas", data:5});  
           return new ArrayCollection(arr);
 }
  }
}

IconUtil.as:

package
{
import flash.display.BitmapData;
import flash.events.Event;
import flash.geom.Matrix;
import flash.utils.Dictionary;

import mx.core.BitmapAsset;
import mx.core.UIComponent;
import mx.graphics.ImageSnapshot;

public class IconUtil extends BitmapAsset
{
    private static var dictionary:Dictionary = new Dictionary(true);

    public function IconUtil()
    {
        addEventListener(Event.ADDED, addedHandler, false, 0, true);            
    }

    private function addedHandler(event:Event):void
    {
        if (this.parent == null)
            return;
    
        var value:Object = dictionary[this.parent];
        if (value == null)
            return;
        
        this.bitmapData = value.src;
        UIComponent(this.parent).invalidateSize();
    }

    public static function getIconClass(source:UIComponent, target:UIComponent,
            width:int, height:int):Class
    {         
        if (source.width <= 0 || source.height <= 0)
            return null;
       
        var scaleWidth:Number = width/source.width;          
        var scaleHeight:Number = height/source.height;          
        var mtrx:Matrix = new Matrix(scaleWidth, 0, 0, scaleHeight);          
        var bitmap:BitmapData = ImageSnapshot.captureBitmapData(source, mtrx);
        dictionary[target] = {src:bitmap, w:width, h:height};             

        return IconUtil;                      
    }           
 } 
} 

7 Habits of Highly Effective Software Developers

1. Design before Coding - We developers often tend to get straight to code spending little time on design. While too much formal and complete design is useless in building software systems, some basic design before coding is extremely helpful. Design does not imply writing a text document with pictures or UML models. Design can be skeletal code that is the basis for the full implementation. I try to first build data structures that hold system state and service interface contracts (not implementations) for the entire application as part of my design exercise. I then try to think through the application flow for various usage scenarios. Good design typically pays off in the end in terms of good quality code developed in a shorter period of time.
2. Re-factor continuously - This seems contradictory to the first habit of designing before coding. After all, if the design is good why should there be a need for re-factoring? The answer is - it is impossible to design software systems accurately and fully at the very beginning. Designs tend to evolve as the system is being built. Often times requirements keep changing. No one can get it right the first time unless you are building a very simple, trivial piece of software. Re-visiting previously written code and continuously re-factoring it is one of the keys to building good software.
3. Unit Test every quarter - No that's not company's fiscal or calendar quarter. It is the 1/4 milestone of your task completion. Break up your task into four, and after each quarter, think of how you can unit test the code you have written. Note that I'm not suggesting you should write a unit test, rather you should have a good mental idea of how one would go about effectively testing what's built so far. If what you have written so far is not unit-testable, re-factor the code to facilitate unit testing before moving on to implementing the next quarter of your task. At the end of a complete task, make sure a formal unit test gets written.
4. Write squeaky-clean code - All brilliant developers that I've seen write very clean code. Clean code means paying attention to removing dead code, avoiding compiler warnings, indentation, spacing, naming, and consistency in code style. This is more than just cosmetics - if your code is not clean, it is unreadable, unmaintainable, less likely to be reused and more likely to get re-written by someone else.
5. Comment code - The only documentation we developers will ever write is comments in the code. Keeping this up to date is of critical importance for the long life of the code. Any code where the logic may not be obvious to a new peer programmer should be documented. Documenting your code is not just for others but for yourself too - how many times have you gone back to your own code written a year ago and wondered what the heck the implementation logic was?
6. Be lazy - Yes, be lazy! Don't write your own code when there is good code already available for the same task. Don't re-invent the wheel. There are a ton of open-source utility libraries that are extensively used; make use of these wherever possible. Why waste time writing code when you can borrow someone else's code which may even be better that what you could write?
7. Get your code peer-reviewed - Peer reviews often help generate new ideas and perspectives on implementation that you may not have thought about. Set your ego aside and solicit peer reviews of your code.

Flex Uncertainty Doubt

If you are evaluating Flex and attempting to pitch it to your management, these are some issues that you are likely to encounter. As with any new technology, you encounter a mix of reality and FUD. This is my attempt at objective analysis of real and perceived issues with Flex that one typically hears.

1. Flex applications are too bulky - An empty Flex 3 application produces a swf file of roughly 150K size. Full-fledged applications can produce swf files of hundreds of kilo-bytes or even mega-bytes. This has historically been a problem. However, with current versions of Flex and the Flash Player, this can be mitigated in 2 ways - modules and framework RSL caching.
   
• A Flex application can be broken down into modules with each module being loaded only when needed e.g. on application startup, only the login module is loaded. This reduces the amount of bytes going across the wire each time, as the application is loaded in pieces depending on user interactions.
  
• Bulk of the size of a swf file is due to the Flex framework code. The Flex framework can be loaded once as a runtime shared library (RSL) across all applications and cached by newer versions of Flash Player 9 even across domains. So your application does not have to statically link in the Flex framework thereby reducing its size.
2. Why not just do everything with Flash Authoring (CS3?) - Flash Authoring is a designer tool while Flex is a developer tool. Flash Authoring is for producing creative, stylized content such as animations and graphics. Flex is the framework for application development. The typical usage is to build graphical assets in Flash Authoring, and bring them into Flex to incorporate into Flex components. Of course, if all you are doing is producing intro-screen Flash movies, stick to Flash Authoring. If you are building applications with re-usable components and business logic, use Flex.
   
3. Flex involves license fees to Adobe - The Flex SDK with its UI component library and command-line compilers is free. Flex Builder Eclipse IDE Plugin has to be licensed (no runtime license involved though). If your application uses server-side Flex Data Services (now called Live Cycle Data Services), that involves a significant runtime license to Adobe although a portion of it has been open sourced as Blaze Data Services.
4. Flex is proprietary - Yes, Flex is proprietary. But Adobe plans to open-source the Flex SDK - see this.
   
5. Flex applications are poor at rendering text/HTML - This is truly one of the big issues with Flex applications (actually this is a Flash Player issue rather than a Flex issue). Text layouts taken for granted in HTML are difficult to do with Flex. Flex has issues with system fonts and font embedding results in very large swfs. Adobe plans to provide better text support in Flash Player 10 - see this. Adobe's acquisition of Buzzword - the Flash-based word processor - shows its commitment to filling this gap.
   
6. Flex is not internationalization friendly - The lack of support for RTL (right-to-left) languages such as Arabic and Hebrew is a huge drawback. This is really a Flash Player issue and Player 10 is supposed to fix this - see this.
7. Flex applications are not skinnable - Flex supports applying CSS-based styling. However the drawback is that style sheets are compiled into swfs and cannot be changed without compilation at runtime by editing a .css text file (See this enhancement request).
   
8. Flex is architecturally inferior to Java Server Faces (JSF) - The comparison to JSF might not be an apples to apples comparison.
• JSF is a server-side framework while Flex is client-side. Flex can take advantage of the power of client-side processing that typical JSF applications cannot.
  
• JSF provides a formalized Model-View-Controller (MVC)-based framework with a component and event model for Web applications. While Flex provides an excellent component and event model, it does not have a full, formalized MVC framework out-of-the-box. One has to use frameworks such as Cairngorm if a rigorous, formalized MVC framework is required.
• In contrast to JSF Web-page type applications, building Flex applications are like building thick desktop applications (that happen to run in a Browser). Unlike JSF, in Flex one does not think in terms of pages rather in terms of application state. Therefore, the JSF notion of a controller that orchestrates page flow does not directly apply to Flex.
  
• JSF render kits for different UI channels is a great concept that works for output channels where there is server-generated markup such as HTML for browsers and WML for phones. Stretching this concept to server-side rendering of full-fledged thick client applications such as Swing, WPF or Flex is practically infeasible beyond a "Hello World" application.
9. Flex applications don't play well with the Browser back button and URL bookmarking - This is a valid concern. As noted earlier, Flex applications are like thick desktop applications where one does not think in terms of pages as in traditional Web applications. However, users are used to Browser Back/Forward functionality and bookmarking pages, and are likely to use it. Hitting the Back button with a Flex application may take the user to the page he was on before navigating to the page that loaded the swf. Bookmarking may not bookmark the state the user was in within the Flex application. The solutions to these issues include Flex's deep-linking and toolkits such as UrlKit. The solutions however require extra coding effort.
   
10. Flex applications are slow - Flex applications may be slow to load initially because of the size of the swf files (see 1. for ways to mitigate this). However, Flex applications allow for client-side processing that can provide superior interactivity on subsequent interactions by reducing server round trips. Using the optimized, binary AMF3 message format that Flash natively understands, one can achieve significant performance improvements over text-based formats. The new ActionScript Virtual Machine (AVM) in Flash Player 9 with JIT-compilation provides an efficient runtime for executing swf byte code. JavaScript in the Browser may some day come close to this performance, but it is not going to be happen anytime soon (see Tamarin).
11. ActionScript is yet another language to learn - ActionScript is very much like JavaScript (both based on ECMAScript). ActionScript 3.0 is based on Edition 4 of ECMAScript which is also the same base for JavaScript 2.0. which is not finalized yet. Adobe is ahead of the game with ActionScript 3.0 but it is likely that in the future ActionScript 3.0 == JavaScript 2.0.
12. With Flex I'm tied to the Flash Player and locked into Adobe - Yes you are tied to Adobe's Flash Player but the Flash Player is ubiquitous and free. As noted earlier, Flex is being open sourced. With the Flash Player, there is of course the issue of being dependent on Adobe for any enhancements you may need to the Player. For example customizing Flash right mouse-click context menus is not possible to the extent an application may desire (see this) and one has no choice but to depend on Adobe to fix it if and when it chooses to.
    

Flex is still relatively new and is likely to improve along with improvements to the Flash Player. But it is part of an emerging and significant change in the Web 2.0 RIA landscape. Flash-Flex/AIR and Silverlight/WPF are ushering in the next revolution in Web and Desktop applications which are easier to build and provide superior user experience. The days of DHTML AJAX applications and the struggle to provide rich user interfaces that work consistently across Browsers are numbered. The Web 2.5 (r)evolution is here.

M13

M13 Application Development Paradigms for large-scale enterprise applications:

1. All business functionality is thought of in terms of one or more services. An application includes a set of services and their implementers/providers.
2. A Service does not automatically imply Web Service.
3. A Service is defined by a Java interface and its implementers are Java classes. A Service can have multiple implementers.
   
4. Service definitions and implementers are independent of how the services are provisioned (Web Service, Java API, EJBs etc.), and are independent of the UI Channel (Browser DHTML, Browser Flash, Adobe AIR, MS WPF, Java Swing etc.).
5. Service provisioning is declarative meta-data driven.
   
6. Service implementers can be switched with configuration entry changes and without any code change. Services can optionally be versioned and it is possible to have multiple versions of a service active in the system at the same time.
   
7. For remote clients, a client-side service API is available for clients to code against.
   
8. Services produce and consume strongly typed objects and not XML. Optimized binary formats are preferred over text-based formats for data transfer across the wire
   
9. UI Components bind to strongly-typed, well-defined data models produced directly by the services without any intermediate transformations.
   
10. A common canonical form for clients across UI Channels is impractical and infeasible for any real-life application. Clients are written separately for each UI Channel in its own technology but they share the same services layer.
11. All application data can be broken up into 2 clear and distinct categories - meta-data and data.
12. Application structure and flow is meta-data driven.
13. All meta-data is efficiently query-able and stored in a relational data store. Meta-data is versionable and upgradable, and it is possible to have multiple versions of a meta-data element active in the system at the same time.
    

Enums in ActionScript 3

The introduction of Enums in Java 1.5 was a welcome addition. Prior to Java 1.5, one had to simulate Enums with some special patterns. In ActionScript 3 there is no native enum construct in the language. So we need to apply patterns similar to what we did in Java prior to 1.5. While similar, the pattern for ActionScript is a bit different.

Let's start with the basic Enum pattern in ActionScript:

package enumexample
{
  public class Color 
  {
    public var _value:String;

    public function Color(val:String):void {            
        this._value = val;
    }

    public static const RED:Color = new Color("RED");
    public static const BLUE:Color = new Color("BLUE");
    public static const GREEN:Color = new Color("GREEN");
  }
}

The issues with the above Enum implementation are:

• The constructor is public. So one could do: var c:Color = new Color("FOO"); which violates the notion of the Color Enum constants being restricted to RED, GREEN or BLUE. ActionScript does not allow private constructors.
  
• Color does not have a default no-arg constructor which is needed if an instance of Color is de-serialized by the Flash Player from data from the back-end e.g. when using LCDS's RemoteObject to invoke a Java method that returns an instance of Color (see this for LCDS Enum support). For default serialization based on property-value pairs, not only do we need a no-arg constructor but we also need to make _value public or provide a setter. Both these are undesirable because one could change an enum as follows: var c:Color = Color.RED; c._value = "MAROON";
  
• If an Enum cannot guarantee uniqueness of enum constant instances, object comparisons based on == will fail. var c:Color = new Color("RED"); if (c == Color.RED); // false. ActionScript does not have the notion of operator overloading.
  
• When a Color instance is de-serialized, an instance is first created using the default no-arg constructor and the _value property is then populated. There is no object substitution mechanism available in ActionScript like the readResolve() Java serialization mechanism. So created instances will fail the == comparison as Enum constant instance uniqueness cannot be guaranteed.
  

The solution I use in conjunction with LCDS RemoteObject is to have Color implement Externalizable and have readExternal() and writeExternal() methods read/write value for _value. In addition, the class is "locked" after the initial construction of the valid Enum constant instances - see the lock member in the Color class below. An exception is thrown when an attempt is made to construct an Enum instance after it is locked. However, since the de-serialization will construct an Enum instance with no args, the exception is not thrown if no args are specified to the constructor. Unfortunately there is no way to detect in the constructor that the instance is being created in the context of de-serialization. The _value member is made private. An equals() method is added which is the way to compare enum instances rather than using ==. However, this is just a guideline/convention and the use of == cannot be enforced. Here's what Color looks like finally - the best one can do given the limitations of the language.

package enumexample
{
  [RemoteClass(alias="javaenumexample.Color")]
  public class Color implements IExternalizable
  {
    private var _value:String;
    private static var lock:Boolean=false;

    public function Color(val:String=null):void {            
        if (lock && val != null)
               throw new Error("Cannot instantiate Enum");

        this._value = val;
    }

    public function get value():String {
        return this._value;
    }

    public static const RED:Color = new Color("RED");
    public static const BLUE:Color = new Color("BLUE");
    public static const GREEN:Color = new Color("GREEN");

    {
        lock = true;
    }

    public function equals(enum:Color):Boolean {
        if (enum == null)
            return false;
         
        return enum.value == this._value;
    }

    public function readExternal(input:IDataInput):void {
        this._value = input.readUTF();
    }

    public function writeExternal(output:IDataOutput):void {
        output.writeUTF(this._value);
    }
  }
}

Why Software Quality Sucks

It was an important customer demo at a big industry trade show. We had worked franticly all through the previous night to get the systems setup for this demo. The demo began very well but halfway through, the program crashed. It was a disaster! Since the demo was for a large and important potential customer, there were a whole bunch of my company suits (non-techie executives) present and they were clearly upset. Apparently they had seen this all too often and couldn't understand why we developers were so incapable of producing bug-free software, and why we should be paid so much money for the junk we produce! At the hospitality suite that night, after everyone was happily buzzed and relaxed, I accosted some of these suits to try and explain to them what had happened and provide excuses for the failed demo. After the usual excuses about demo'ing with the latest unstable dev build, lack of sufficient resources to code, this other team not delivering their fixes on time, bad hardware etc. (all of which they had probably heard many times before), the conversation drifted on to a more generic discussion on software and bugs. I found myself trying to impress upon them the difficulty of building bug-free software. Now this was easy and I felt confident I could convince them because I had Bill Gates on my side. That's right, Bill Gates of Microsoft. It so happened that around the same time as this trade show, Bill Gates was demo'ing Windows 98 at COMDEX, introducing it to the whole world for the first time. And it crashed and burned in the middle of the demo! An embarrassed Gates mumbled something about needing to iron out some more bugs. I mentioned this incident to my suits. Some of them had read about it in the papers. I told them Microsoft had 2000 very bright engineers working on Windows 98 and asked rhetorically why they still couldn't guarantee bug-free code. The software we were building and demo'ing was incredibly complex large-scale, real-time, network monitoring and control systems with hardware interfaces to receive 100s of thousands of data points, and heavy-duty number crunching in mission-critical, high-availability systems. I reminded them that we had just 3 engineers working on the portion of the demo that crashed. The point was that if Microsoft, with its huge army of the world's best developers, cannot produce bug-free software, how could we, with far fewer resources to build equally complex systems, produce anything better? The suits seemed to see the point. By the end of the night I had totally convinced them that software is, by nature, very complex and it was virtually impossible to build systems of better quality than Windows. One had to just live with the fact that software systems will malfunction and crash periodically. Thanks Bill!

That was 10 years ago. Windows has come a long way since those days and we now see less of the infamous "blue screen of death". Software in general has matured more in the last decade. However, we are nowhere close to the level of maturity in commercial software where high quality is taken for granted. Any enterprise-scale software is fraught with bugs requiring continuous application of hot-fixes, patches, upgrades etc. Ask any IT professional in any company about the quality of software he supports and invariably you will hear complaints. Complaints are likely to be more vociferous when it comes to business applications software.

So why does software quality in general and business applications software in particular suck so badly? Why is it that software is not as reliable as (say) bridges? Some would argue that the software industry is still very new. After all, they argue, we've been building software for less than 50 years while we've been building bridges for 1000s of years. Others would argue that software is much more complex than building bridges or cars, and as complexity increases, so does the probability of defects. There are some who would point to lack of standardization in the way certain common software elements are built resulting in defects when trying to integrate elements into one application. Then there is the camp that feels that a significant number of software engineers lack the proper training and skills and that the software industry is probably the only industry where we don't make a distinction between technicians and engineers (think of electrician versus electrical engineer, plumber versus civil engineer etc.) - in software, anyone who works with code is a software engineer and problems arise when technicians try to do the job of engineers.

While all of the above reasons may be valid, I believe the fundamental reason for poor software quality is the way we build it. This is more true for applications software than systems software. In spite of all the advances in languages, IDEs, design patterns, open-source libraries, runtimes and standards, building large applications software remains horrendously tedious, labor intensive and error prone. I think we need a radical shift in the fundamental mechanics of writing code in a programming language. We need a new vocabulary for application developers to state their business logic rather than code it. Modeling tools have attempted to address some basic part of this but have been very code and programming oriented e.g. thinking in terms of classes in UML. Model Driven Architectures (MDA) never lived up to the hype and simply provide a language to describe the high-level problem without providing a real-life solution.

I have no idea what this new vocabulary and mechanics for building software would look like in its final form (although I have a name for it - Yeti), but it seems to me intuitively that it will have the following characteristics:

• Rules would figure in it prominently, and application building will primarily involve describing business rules
• Application logic would be expressed in a natural language syntax
• Application builders would not be programming (as we know it) in languages such as Java, C++ or JavaScript
  
• Applications would be able to process incomplete information employing fuzzy logic, and learning
  

Maybe I'll never see Yeti in my lifetime. Maybe Yeti simple cannot exist given the basic von Neumann model for computers.
Or maybe not.