STKI Summit – SOA Perspective: Same Old Architecture or Same Old Mistakes? – Part 1

On March 24^th I participated in the STKI Summit by the local analysts company STKI.
The company headed by Dr. Jimmy Schwartzkopf is focused on the Israeli market. The theme of this year conference was: Back to Basics.

Presentations were based on an Israeli market survey performed by the company.
The company's analysts' presentations topics were Infrastructure Trends, Applications Trends, Office of the CIO trends and Infrastructure Services Trends, followed by wrap up IT Trends presentation by Jimmy.

The survey may not be accurate but the information is valuable and probably reflects the overall market status.

A less accurate on the spot survey results were presented afterwards.
The following bullets highlights SOA perspective as it was presented:

• Disappointments from SOA implementations results

• SOA implementations are lagging behind expectations: It takes longer time than expected and consumes more resources.
  

• Benefits form SOA projects include better and faster integration, Faster changes to production, Better day to day operations and migration from Legacy products.
  

• On the spot survey results: 48% - No SOA initiative, 22% - SOA benefits are in IT budget savings, 28% - SOA benefits are in Agility

• Development, SOA and Middleware Radar in which SOA implementation is in the interest circles. SOA technology Selection, SOA Governance initiative and SOA Road map projects are in the Implementing phase and no SOA aspects in the more mature circles titled Using ( EAI, Java and .Net are in the Using phase).
  

• Quantitative results such as number of services or interface and a list of installations of SOA/EAI/Messaging/Development tools.

My Take:

• I am not able to interpret the quantitative results because SOA and non-SOA are intermixed (I assume that both users and vendors label Enterprise Application Integration (EAI) and Message Oriented Middleware (MOM) implementations as SOA). Separation is needed for analyzing SOA Infrastructure Trends.
  

• Separating SOA from non- SOA in a survey is difficult (see my previous Post SOA and SCS). If it can be done I would add another quantitative question: What is the Service Reuse Ratio?
  

• Is it disappointment or disillusionment? I do think that SOA if implemented correctly is not the acronym Same Old Architecture. However, repeating the same IT mistakes will create the Same Old Architecture and not SOA. SOA should not be an IT only initiative, but a joint Business and IT initiative.
  

• Expectations of Enterprises hiring external experienced and knowledgeable SOA consultants would be more realistic. The significant difference between these enterprises and others will probably not be in deviation from expectations but in actual measurable results.
  

My next post will analyze the survey results in comparison to other surveys.

Automated Business Services Modeling – Reality or Vision?

A new SOA product VeN4 ServiceWare developed by Venotion Technologies

was presented in the 20^th meeting of the SOA Forum of the Israeli Association of Information Processing on March 12^th.

Venotion Technologies is an Israeli startup, presenting a unique approach to SOA implementation. The presenter was Ronen Yochpaz the company's CTO.

The product is focused on the Finance sector especially the Banking and Insurance industry.

Many Core Banking application systems and Insurance applications are legacy systems developed over more than 20-30 years ago on IBM Mainframes and z/OS operating system. Venotion's product aimed at using existing assets as part of a SOA implementation. Currently the product is in Beta. General Availability is expected on April 2008.

Basic Concepts and product highlights

• The product composes automatically the Business Model by Reverse Business Engineering of existing User Interface and IT assets.
  

• Execution is in the Testing environment and not in the Production environment in order to prevent performance degradation.
  

• The entry point is the User Interface (UI), so B2B and Service to Service connections which were not initiated via User Interface are excluded. The assumption is that the majority of SOA related resources are invoked via User Interface.
  

• Batch modeling is based upon Schedulers such as BMC's Control-M using the Scheduler control cards instead of the User Interface.
  

• A UDDI based Registry is automatically populated by the product using the artifacts of the Business Model which were created by the first step of the product execution.

• UDDI entries populated by the product are augmented by Business context using Business Taxonomy.
  

• Ability to catalog services in the UDDI repository based on business context.
  

• IT Application Decomposition is based upon recording of the path of each service starting from the user click via User Interface.The recording is based on tracing tools of each technological environment containing part of the Service implementation.
  

• A Graphical tool enables Business Model presentation.
  

• Another tool is a Semantic Search tool (search by Context and not by Keyword).
  

• Currently the product supports J2E .Net and CICS
  

Role in SOA initiatives and Deployment

The role of the tool is for assessing SOA readiness for domains targeted for transformation to SOA (the tool is positioned by the company as a "Jump start to SOA" i.e. a tool which accelerates the first steps in SOA adoption).

The automatic Business Model Composition cycle time is few hours.

The short time for building Business Model Composition enables many iterations of that process.

The product can also be used for justification of a SOA initiative because it is possible to show results and Return On Investment (ROI) in relatively short time.

Competitive Positioning

• Venotion positions its product as the only tool except IBM's tools for modeling existing and Legacy assets to Services. According to the company, IBM's tools (SOMA, CBM) are not fully automated and therefore implementing them may consume more time and resources (IBM had presented SOMA in a meeting of the SOA Forum of the Israeli Association of Information Processing two years ago as an internal tool implemented only by IBM's experts).

• Leumi Bank is the Beta site for the product.
  

• Deutsche Bank USA is in negotiations for evaluation of the product.

• No information about other candidates is disclosed currently by the company.

• Most of the leading SOA vendors are Business partners: IBM, BEA, Microsoft, Oracle, Software AG, Sun and HP

The partnership is based upon population of the UDDI Registry of the partner by
VeN4 ServiceWare. Populating the registry in short time could justify buying the registry.

Ronen said that business people told Venotion people that the model helps them to know which components are used by their application, so they can quantify the number and percentage of the elements used by them. They said that quantifying the percentage of elements can help them in estimating the accounting model. For example a Business unit paying half of the IT budget but using only 10% of the IT artifacts would ask for lower charge.

My First Take
My first take is based upon my limited knowledge of the product (The presentation and information in the company's Web site), so it could be wrong.

• Unique innovative approach

The unique innovative approach is promising especially for enterprises thinking of Legacy Modernization without Mainframe replacement.

Product strengths include:

1. Automation which enables short time for producing a model

2. Drilling down from services to assets composing services

3. Automatically adding Business Context and Taxonomy to UDDI registry.

Lack of Business Context is a well known limitation of UDDI standard.

• Too good is not good enough

This product as well as any other product is not a Panacea.Some organizations could see it as a
Panacea.
The short time for creating the model could be misleading. The product may be valuable in
the long journey to SOA, but thinking of it as a replacement of Business Modeling tools
and Governance tools could be a very costly mistake.

It is compelling to think that VeN4 ServiceWare or any other tool could replace the
process of building the Service Model. No tool can do the job instead of Business and IT
professionals collaboration. Sure if wisely used, tools like VeN4 ServiceWare could
aid the professional people collaboration process by supplying valuable information.
Do not assume that the tool can address lack of organizational maturity for SOA.

• Does the product produce a Business Services model or Web Services model?
  

For answering this crucial question I need to investigate the product more deeply. However, using a UDDI based registries and the automatic process makes me suspicious: Is the model built of Business Services or technical Web Services? Is the tool capable of resolving redundancies such as multiple Web Services with overlapping or identical functionality? Is the granularity of Services in the created model coarse enough?

• Creation of Dynamic Assets Repository

VeN4 ServiceWare builds an Assets Repository. Due to the ease of building

it could be more dynamic and flexible than the traditional Metadata Repositories

such as ASG's Rochade or Ca's Repository products.

I am not sure if it is more flexible than the new SOA related Metadata Repositories e.g.

BEA's Flashline or LogicLibrary's Logidex.

It should be remembered that the traditional Metadata Repositories manage

Production Metadata and are capable of supporting wider range of technological

environments.

• Inability to discover services which were not initiated via UI
  

Currently the product addresses only UI online applications.

Mapping non-UI initiated services or UI initiated services outside the enterprise and the enterprise's Web site (e.g. UI initiated Service in a partner enterprise which invokes services within the enterprise) could be important.

It is especially important to organizations partially transitioned to SOA paradigm, enabling consumption of services outside the enterprise domain and consuming services beyond the enterprise boundaries.

• Additional environments should be supported
  

Although VeN4 ServiceWare supports three frequently used environments (CICS, J2E and .Net) Enterprises are usually more heterogeneous.

The result is exclusion of these applications and environment from the model, so no Full Business Services Model could be derived by this tool.

Legacy Visual Basic up to version 6 including version 6 is an example of a frequently used environment which is not supported by the tool.

Mainframe examples of non supported environments are IMS/DC OLTP monitor and Adabas/Natural.

It should be remembered that the product is in Beta of its first release, therefore support for too many technological environments is not a good practice and is not expected, however I think that support of additional environments will be a required enhancement in next releases.

SOA VDS – AmberPoint Example

ZapThink's analyst David Linthicum wrote a zapflash titled: Avoid Vendor Driven Architecture (VDA).In this Zapflash David Linthicum describes two prevailing SOA implementation patterns: buying SOA products from existing vendors and letting these vendors to design and define the solution.

According to his analysis the outcome of such approach will be a failed SOA implementation because SOA is Architecture and not Technology, so the expectation of out of the box magic technology solution to improper architecture is both unrealistic and misleading.

As an experienced consultant I would add to this that whatever the technology or architecture is vendors are biased towards their own solutions and vendors employees rewards are dependent upon sales.

This post is not about VDA but about VDS or Vendor Driven Surveys. Same as Architectures build by vendors are suspect to bias towards their solutions; results of Surveys executed by vendors probably will be biased towards their solutions.

AmberPoint is a leading SOA Management and Governance products vendor. The company has impressive line of products and partnerships (e.g. Microsoft, BEA, Software AG) for competing with companies like IBM, HP and CA.

On January 2008 the company published a survey titled "State of SOA Adoption Survey".The survey results show more mature and successful SOA adoption than other surveys. Unsurprisingly, the survey results show that AmberPoint's customers SOA initiatives are more successful: approximately 41.3% of AmberPoint's customers reported that their SOA initiatives met all their goals in comparison to 28% of non-customers.

A Skeptic general results analysis

Let us try to interpret survey results in non VDS attitude.

Issue 1: The stage of SOA adoptionResults:
Internal multiple Department – 22%
External – 11%
Internal Single department – 9%
In Development – 22%
Experimental – 36%

My interpretation: SOA is implemented by only a third of the survey participants (Internal multiple Department and External). The others are in early stages of implementation. Notice that 100% of the survey participants declared a SOA initiative in any stage. This could be a vendor driven result and undoubtfully biased. I am aware of many enterprises which has not begun yet a SOA initiative. The reason for this finding: the survey participants are all IT professionals that "have an understanding of SOA". Those engaged in a SOA or SCS initiative are those who think they have an understanding of SOA.

Issue 2: meeting SOA Goals

The survey results show none of the respondent labeled his enterprise SOA initiative as a Fiasco and only 1.5% claiming that they did not complete the project and therefore did not achieve desired goals. 38% met all their goals and 60% met most of their goals.

Are these initiatives Real World initiatives or these initiatives executed in heaven?Were have all the damages of billions of dollars of failed SOA initiatives due to lack of adequate Governance gone (I read Research Notes by Gartner Group and by Burton Group including these billions of dollars estimation)?

Notice:

• Failures admitting are rare in IT and SOA is not an exception.

• Few years ago I looked for SAP ERP Case Studies and found only one failure. Unsurprisingly, the roots of the failed initiative were explained by the succeeding new project manager replacing the previous fired project manager.

• Success of SOA initiatives gaols should be measured by quantified Business related benifits. Is the high success rate a result of such benefits in early stages adoption or the result ofIT only related goals which are not related to Business? Was the "success" a result of IT only SOA initiatives including IT only goals?

• What was completed successfully: A SOA Project? Probably, yes, A complete Architecture? Probably not. Will the temporary success of a project or few projects turn into an Architectural disaster due to lack of Reuse, No Agility and overwhelming Complexity?

In summary, it seems that the high success rate reported is reflecting more biases of IT professionals involved in SOA initiatives than actual success.

Issue 3: Non-SOA Components involved in SOA systems

ERP and Mainframe Components are the most frequent answers. This finding is reasonable and manifesting the challenges of building a complete SOA architecture including existing Legacy and ERP and CRM systems.

Issue 4: Issues best addressed by SOA

The most prevailing isssues cited by the participants in the survey (in descending order) are: Inflexibility, Restricted Information flow due to Stovepipes, Lengthy Development Cycles and Costs.

Well Inflexibility or Agility is a common SOA justification aligned with Business expectations of Responsive organization.

What the hell are Stovepipes as an issue? StovePipes are a reason for lack of flexibility, high costs and other issues and not a reason only for restricted information flow.

Notice that improved maintenance is not a common answer (Was it posted as a valid answer option?). The benefits of improved maintenance are more significant than the benefits of shortening Development Cycle because maintenance efforts and costs consume 70% to 80% of IT budgets.

Issue 5: Topics to be addressed for successful SOA

The most frequent obstacles facing SOA implementations are in descending order:
Lack of SOA Expertise (67. 6%), Organizational reluctance to change from the status quo (58.9%), Security (51.8%), Challenge of managing SOA systems (48.5%) and Immaturity of SOA standards (43.8%).

The findings are reasonable. Notice that Organizational issues are facing SOA initiatives. Not all SOA challenges are technological.
The only missing item is SOA Governance. According to analysts research lack of Governance is one of the main reasons for SOA initiatives failures. I would expect a 50% or more votes for that topic.

Is it because the SOA experts participating in the survey think of SOA Governance as SOA Management?

The common link

The common element to all results is the participants labeled by AmberPoint as "IT professionals that have an understanding for Service Oriented Architecture (SOA)".

My discussion of Issue 2: meeting SOA Goals raises some of the possible biases due to IT only surveys, but it is not the only limitation: as discussed in Issue 1: The stage of SOA adoption most of the survey participants are probably participating in a SOA Initiative execution and therefore enterprises without ongoing SOA initative were excluded.

Vendor specific findings as VDS

AmberPoint specific results section is for sure Vendor Driven.The results are reasonable: A larger percent of AmberPoint's customers reported more SOA applications in Production (Nearly half of the non customers reported having 10 or less than 10 services in production) as well as more successful implementation (met their goals).

The devil is not hidden in the details; it is hidden in the interpretation.The vendor driven comparison is between AmberPoint's customers implementing the company's Management and Governance tools and between all other participants.

The non-customers are heterogeneous group composed of the following enterprise profiles:

1. Enterprises in Early Stages of SOA initiative
In many cases they did not implemented yet any Management and Governance tool (perhaps they do not need such a tool in this stage). 
Surely they report less successful and less sophisticated SOA implementation than enterprises in more
advanced stages.

2. Enterprises implementing a large number of services without Management and Governance
tools.
Would anybody expect these enterprises to be successful? Surely, they are leading candidates for SOA initiative failure. No doubt that AmbrePoint's customers will be more successful than these enterprises.

3. Enterprises implementing other vendors' tools.

Comparison of AmberPoint's customers to these enterprises is the interesting comparison.

In short the comparison should be between enterprises in the same sophistication level divided into three groups: AmberPoint Customers, Competitors Customers and Enterprises which did not deploy any SOA Government or SOA Management tools.

Another point raised in this section of the survey is: AmberPoint Customers deployed its tools in early stages even in Development. This point is Vendor driven, suggesting that adopters of SOA Governance and SOA Management tools at early stages are more successful.

In my opinion Governance and Management implementation at early stages is improper and costly.

The only reason for implementing these tools in an early stage is related to most frequent obstacle Lack of SOA Expertise (Issue 5). Due to lack of expertise tools implementation too early is better than tool implementation too late. However, the best timing is the implementing these tools when needed consulting by experienced SOA experts is capable of planning and implementing these tools just in the right time.

There is no need for such tools in early stages during SOA Pilot. If the SOA Pilot is planned and executed correctly, at Pilot completion a SOA application will be in production. In that case, Surveyed Enterprises completing the Pilot will be classified as In Development or Experimenting (Issue 1).

Conclusions

• Surveys can be vendors driven (VDS).

• Vendors Surveys should be evaluated carefully same as Vendors Tools and Vendor Architectures

• AmberPoint's SOA Survey is vendor driven. The interpretation of the results is misleading.

• Some of the findings are valid such as non-SOA components involvement (Issue 3) and partially Issues 4 (Issues best addressed by SOA) and 5 (Topics to be addressed for successful SOA)

• Overcoming the challenge of Lack of SOA expertise (the most cited topic) is a key for successful SOA implementation. It is recommended to deploy external consulting by experienced SOA experts. This conclusion is based upon other non-vendors surveys (e.g. Aberdeen Group found that Best In Class organizations tend to use outside expertise more than other organizations).