SOA - What is it good for ?

Well, I won't go as far as the song and say "Absolutely Nothing" but the project I worked on definitely proved that too much of a SOA thing is bad for you.

Somewhere along the line the project management drank the SOA Kool Aid all the way to dregs and mandated SOA everywhere without thought for performance and the impact of having too many services of a low granularity.

We implemented our SOA services using Oracle Fusion (10.1.3.X) middleware as a combination of BPEL processes, EJBs, web services and ESB components.  All SOA services required a WSDL to enable them to be called from other services.  Some WSDLs could be generated from the EJBs but others were pre defined so required JAXB to generate the java objects for interaction with the service.  Some services used WSIF (ie calling an EJB directly rather than going via a SOAP conversion) to improve performace.  

WSDLs and their related XSDs and namespaces are unweildy, verbose and repetitive.  Changing them by hand is time consuming and error prone.  A decent set of high quality robust tools could almost make dealing with them manageable - unfortunately we had Oracle Fusion where the tools were incomplete, buggy and more often than not got in the way.

Testing the services also meant dealing with the WSDLs and the idiosyncracies of the testing tool.  We used SoapUI Pro as our test tool which certainly improved over the years so that it started offering native functionality that we were at first forced to write ourselves using the provided Groovy scripting language.  SoapUI produces XML test files but they were not designed to be changed by humans so when the GUI tool came up short for certain tasks (eg changes affecting multiple tests) it became very painful.

Both Oracle Fusion and SoapUI can be very unforgiving to an error in the WSDL or XSD so that it can be an extensive debugging campaign to determine an error.

Apart from the technology challenges of implementing a service there is the design and documentation side of it.  In theory a Service should have a simple and complete interface, be a well designed, well documented reusable component that can be easily selected from a service catalog.  It should be implemented in a reliable secure environment and enable change management and different versions.  A defect management process is also required since different organisational groups may be managing different services

The above prelude is to demonstrate that there is a fair amount of overhead involved in every Service.  With lots of services you either cut corners (eg documentation, interface completeness) or invest a lot of time and you have to take the performance hit of the service call a lot more often.

In our case most "services" ended up being single purpose and tightly coupled to their dependent services so lost most of the advantages of being a service.  We spent the better part of 2 years modifying and re-modifying our services to achieve better performance and enable more and more production aspects (logging, security, performance metrics) which involved modifying WSDLs and test cases and retesting.  This was very time consuming and inefficient and as a programmer it felt like walking in knee high mud.

Alright so SOA is not good for low granularity service decomposition so what is it good for ?

SOA makes sense for high value reusable components that need to be invoked by different systems.  It is also a good way to make existing functionality (legacy systems) available to other systems in a standardised way by providing a Service Adapter component as an interface to the legacy system.

Ironically that is exactly the way we didn't use it.  There was a legacy system in place that needed to be upgraded.  The decision was made to replace everything in a big bang approach with an unrealistic deadline.  So lots of people implemented lots of services with a new technology stack (when we started Oracle Fusion was lucky to be Beta quality) and focused on functionality.  We ended up creating a big mess that is still being ironed out 3 years later. 

If we had broken the legacy functionality into large granularity services and provided adapters for some of the functionality whilst providing a smaller chunk as rewritten services then we could have focused on getting the smaller chunk production ready and developed our development templates and best practices.  This would have made the project more manageable and given the stakeholders earlier evidence of the project success.  Later parts of the project could then build on the best practices and be more of a known quantity. 

Of course it would have been a harder sell from a project completion date perspective but sometimes you have to be realistic and recognise that when a project gets above a certain size the only way to get success is to be conservative and manage the risk appropriately.