SOA Lookalike

The most central issue to the topic of SOA Projects is determining what kind of SOA implementation the business needs. Does the business need agility, more integration, more visibility, lower costs or the best ability to adapt to change at all costs?

Amazingly when I did talk around about this to active SOA practitioners, I found many organizations that were big on SOA and said they need this agility for business competitiveness actually wanted rapid development shops that would band aid existing legacy applications with web services. Basically they were trying to roll out pilot projects that use latest web 2.0 technologies to solve some niche business problem that was hanging in the organization without being solved for a long time.

So before you are asked to lead or participate in a supposedly enterprise SOA project you need to ask yourself several questions to help determine what kind of SOA implementation you are working on:

<!--[if !supportLists]-->· <!--[endif]-->How strong is the organization need for rapid change?

<!--[if !supportLists]-->· <!--[endif]-->Does the project's emphasis on agility arise because it is really one of the common SOA lookalikes?

<!--[if !supportLists]-->· <!--[endif]-->Is your project limited by any weaknesses that would prevent a SOA success?

Not doing due diligence on finding SOA look-alikes will actually lead to the SOA leader being viewed as a stumbling block for organizational direction.

Here are some of my suggestions to identify the real SOA projects are,

Organization with Strong need for Agility

Organizations that truly need agility are in a market place where they are under constant competitive threat. If you observe these organizations they will have a fairly large portfolio of product and services with broad client base that demands services fairly quickly. The company must have attempted to reinvent itself once in its life time. The management will constantly be in a sense of urgency and looking for the next market leader by researching new technologies.

Organizations with Strong Standards Orientation

These organizations will have a strong culture oriented towards standards and systems. Many of them would have or will be in the process of getting certified by standards such as CMM or ISO or MSF etc. Keep away from organizations that say they have their own needs and hence need a standards and procedures that is unique to them, nine out of ten times these kind of organizations would have attempted the industry standard certifications and failed in adhering to them. Many times the reasons for not following industry standards and deriding them are basically sour grapes story.

Organizations with Strong IT Leaders

These organizations will have a strong IT professionals leading the production and delivery systems. With professional line managers in place the conceptual foundation for implementing a SOA architecture can be sold without difficulty. By having a high caliber production team will help the architectural team to work on design issues rather than fighting fires everyday.

Organizations with resilience

SOA is a long story and needs patience. If the organization cannot stomach failures and delays, then the SOA effort will change direction midway and will definitely turn into a SOA look alike.

Organizations with Strategic Sense

Finally you need to find whether your organization is attempting SOA as a strategic tool or using it as a tactical weapon. SOA for strategy is real SOA... SOA for tactical purpose SOA look alike. How do you find it... look at the past project history. If majority of the projects were done to pluck the low hanging fruits then the organization lack the will to mount the effort to grab the difficult but juicy fruits on the top of the tree.

Wizard of Oas

Once upon a time in the IT Land there was a village called Olis. Everybody in Olis was looking for a place called IT nirvana... ok first let me define IT nirvana. IT Nirvana is a place where there is IT but No IT, where there is perfect interoperability and agility, where providing a technology upgrade path also preserves the investment in legacy systems and simplifies deployment of new systems..... it is a place where there are no borders, everybody is free of vendors, where developers, analysts, managers are biological clones so anybody can be replaced by anybody, where time, cost and schedule are independent and so on and so forth.

The people in Olis had looked far and wide and not found this dream place. They felt very lonely in IT land. Many times they tried to break from this isolation and get to IT Nirvana but without any success.

Their Village was so built so complex that people could not even find their own homes some time. It was like a maze and they could not build any trails to outside their land and do business.

Lots of wizards have come and gone in the village and everyone of them promised the people of Olis the elusive IT Nirvana where Olis is the center of IT Land and all trails are marked and interconnected. 
Every wizard had his own magic method and they promised if the villagers followed their ways and did what the wizards wanted them to do they can have IT Nirvana. The people in Olis tried all of the methods of the wizards and still could not break loose.They were desperate, if Olis did not find the dream place, it may perish.

One day a new wizard came to the village, his name was AOS, and he said he has this new magic that can get Olis to IT Niravana.

All of them very initally doubtful of AOS but still they wanted to believe in him as they desperately needed IT Nirvana.

Reluctantly they agreed to let AOS do his magic. Since they were desperate, they wanted to start learning the wizards magic fast... but the wizard warned them that the path ahead is long and arduous. He told them that his methods take a while to learn and he also needed the old and forgotten elders of the village has his apprentices.

The young villagers did not like this, they thought the old ones are slow and lazy but AOS said that his magic is from the ancient masters with new power and as such he needed apprentices from the village who knew the methods of the ancient masters. AOS also told them that his magic is not free and will cost the village 100 bushels of corn.

Now that was a problem, the villagers got together and discussed about the ways to get the 100 bushels, they knew for sure that the village headman would not easily give them the 100 bushels, so they devised a plan...

Everybody in Olis went to the head man and told him that the wizard and his methods are so powerful that when Olis attains IT Nirvana he, the village head will have everything he wanted in the village, every villager will work his land, his neighbors land and any other land he needs them. They also told that when Olis reaches IT Nirvana he can sell Corn bushels to all of the IT land as he will have trails leading everywhere and make a lot of wealth, plus they told the village head that if he pays the wizard then the wizard will get him new Headgear that is flashy and he will also get a new magic oracle that will tell him about all things happening in the village right inside his own house.

The villagers made AOS do his smoke screen pyro techniques showing IT Nirvana, which convinced the headman to give AOS his 100 bushels.

AOS started his magic, Slowly and steadily new trails started appearing out of nowhere connecting Olis to other places in IT Land, but those trails were not smooth enough to walk. It took lot of effort to walk on those paths and many in Olis were frustrated by it. The village headman was getting impatient, he wanted his riches, fancy head gear and his own private oracle NOW

Many of the people in Olis decided that AOS's magic is no good, his ancient ways were boring and more work and they need a new wizard.  As time went by people were getting tired of AOS's's methods, some started commenting that he may be a fake wizard, some others said he is just teaching them what they already knew and there was nothing new in his magic, but some believed in AOS and continued to do what he wanted from them.

The village headman was getting impatient and villagers were panicking , they wanted to stop AOS and get some new wizard.

Now the question before them is do they have ONE.???

Colloborative BPM - A new way for SOA, and Enterprise IT projects.

Last week I had the opportunity to look at the process flow diagram of one of our BPM projects. Looking at the exiting process flow it was no surprise to see a wall to wall snapshot of the long work flow diligently documented by our BPM team. We had the normal conversation of how complex our processes have become and how inefficient the existing operations are. One point that was brought up was how simple tasks have grown into complex entities due to N number of intermediate steps in them.

This made me think about the work flow presented to me. Basically I was wondering whether the complex setup I see before my eyes was developed by a dedicated team for solving a particular problem or grew by itself to solve a business requirement.

I took a piece of the work flow and started inquiring around the organization about why it is being done like that. Amazingly I had answers that were logical and reasonable that supported every twists and turns that made the flow complex. For me it looked like nothing was really amiss in the microscopic level, at the macroscopic scale I found the asynchronous nature of organizational tasks lead to all this complex work flow.

For example task A which when completed leads to task B after some time gets a new business requirement task A.1, since the implementor of task A.1 changed the original task A to be safe he added a check point C to Task A before it goes to task B.So a Simple task of going from A to B now has morphed into A to A.1 to C to B and this happens to a task cycle were there is no interplay with other enterprise components. If we take the enterprise effect into account the A.1 and C may effect processes X and Y and they may add new steps X.1 and Y.1 to solve the issues of A.1 and C

As the organization grows there are events that happen asynchronously all the time and we have a situation with an ugly looking BPM work flow chart and we have all the six sigma's and SOA architects attacking it to create simplicity and efficiency.

Now the question is after this iteration will we have a stable system that will be predictable and provide an efficient work flow for now and the future? The answer in the SOA/BPM space for that is governance. Tighter control, standardizations and process management when implemented and governed properly will provide that stability. However the after effect of tighter governance is decline of innovation, creativity and spontaneity. How does organizations will deal with this side effect of the BPM/SOA initiative. This becomes critical in a global marketplace were innovation is the competitive advantage for organizations.

Looking into my spyglass I see answer in the application of Chaos Theory in organizational development. The challenge of change management is pervasive in the BPM/SOA space for creating an agile organization receptive to change. Chaos management can provide organizations an environment that would thrive on change. Embracing chaos and constantly being on the edge is a nervous state for organizations but the payback is a creative, innovative and agile organization. So my recipe would be embrace chaos and use BPM/SOA to act as the moderator in controlling the chaos.Have loose governance, create a plan to do a BPM work flow analysis in iteration after the first run as often possible to find the instability and use SOA to manage.

Innovation with SOA and R&D, Business Perceptions and Engineering Challenges

There I was sitting in a meeting with business stake holders discussing on the next set of deliverables for a new product we have been developing for the past five months. This is a project I am involved with new business development in my company. To give a brief background, we are developing a retail reward system that is both hardware and software based. We have two dedicated teams, one for business and one for technology. The teams are small and focused on getting this done with the shortest possible time.

My this post is not on developing new product but about the relationship between the business teams and technical teams on driving the product development. As with any new product development we have fewer resources, more complex problems and tighter deadlines.

As with any new innovation project, the challenge I face everyday in the project is the uncertainty involved in solving complex engineering and technology issues. Historically in our company business for the most of the time has been dealing only with software projects, even those projects were most of the time limited to the scope of enhancing our existing applications or trans migrating our existing applications.

This product I am working on now is unique in the sense that it has hardware development and software development. As we are going through the development phases I see a expectations gap between the business and the engineering team. The complexity involved in the engineering solutions is seldom reflected in the perception of the business for a particular business requirement.

The core concern of the business is its ROI on the project and the delivery schedule. While the concerns are legitimate the problem is the expectations arising out of them are not rooted in reality. The biggest gap I see in this issue is the concept of risk management. While businesses are within their rights to demand the ROI which includes the business requirements and delivery time, as with any business investments they should also realize that they are taking up risk in their investment expecting a higher return.

When this uncertainty in engineering and technical challenges or in other words the risk in the project is brought up to the table in the form of prototype testing, tighter scoping of the business requirements, avoiding complexity by eliminating scope creep etc., business gets rattled.

We see in many blogs in EAI space the importance of ROI, especially in the BPM and SOA space we see arguments forcefully made to sell the SOA project to the business with relevant ROI. Many SOA proponents advocate that the SOA practitioner do that job of selling the project to Business with ROI. Doing this I see an agency conflict.Since it is in the benefit of the SOA practitioner to sell his goods he is going to underplay the risk involved in the project, which many times translates to direct costs in the form of change management, process correction and the need for outside resources in the form of outsourcing.

So how do we attack this gap in the Business perception of engineering challenges and bring to the table the Real Risk in ROI presented for the project. Looking through my spyglass I see some answers in financial management theory. In the CAPM model the expected rate of return for any investment takes into account the risk free return, the risk premium for the investment and the market sensitivity factor.

In our case applying the CAPM model to SOA or R&D projects we should factor in the historic success rate of IT projects in the enterprise, the risk premium associated with attempting new technology and the industry success rate for projects of similar nature (if available) for the ROI calculation . When a dialog on ROI is based on this framework, the risk factor is automatically included in the dialog and should address the gap in engineering challenges and business perceptions about it. I would welcome feedback on any other the ways to bring to the table the risk factor.

In conclusion it is not about IT department showcasing ROI of SOA, IT or R&D projects that leads to business alignment, rather it is about what kind of ROI we present and what type of dialog it initiates in the business about the project. A ROI rooted in the reality of what is the risk associated with the project and what is the expected rate of return for the business it provides will decide the true business alignment.

SOA, BPM, Six Sigma and IT Projects...what is the scope here?

The past three weeks I had a hectic training schedule. First I was in a SOA Architecture Boot camp, week later I was in a course ware on Lean Management from MIT looking at ways to accelerate process efficiency. The thing that struck me in both the courses were the involvement of Six Sigma Black belts in show casing process improvement methodologies. I had the opportunity to look at real life case studies and ask questions about them. In an economy looking at recession, it was no wonder to see all this BPM initiatives in companies looking at ways to "Cut Costs".

As expected BPM was the entry point for IT projects where some of them ended up as SOA initiatives and black belts were the enablers for those BPM projects. Looking through my spyglass (plugin for my blog) I find many Q's popping out in the landscape. There was a general distrust of the six sigma methodologies and its efficiency among the participants.

One of the points that was raised in the seminar was, the black belts definitely bring in the horsepower to do the DMA part of the DMAIC six sigma process, however when it comes to the Improve and Control piece of the process they seem to falter. They face obstacles and road blocks all the way after the DMA cycle to implement their recommendations. When I asked the black belts about the buy in from IT groups on their recommendations, all of them unanimously mentioned IT as their biggest bottle neck. For that matter one Master Black Belt told me that in his organization some of the IT managers have expressly prohibited his team members attending the BPM meetings without his permission.

One other participant put forth an interesting observation, he said "American companies in the 90's have invested a lot on the BPM initiatives and while they have gained a lot of incremental improvements only some of them has got the biggest bang for the buck and it is so late now for them to change course on this.

So after this three weeks of SOA, BP, Six Sigma and Lean... I come out with more questions than answers. Common sense tells us there is no one size fits all, at the same time companies are implementing solutions based on these tool sets which they feel will solve all of their problems. The challenge ahead is how to improve efficiency of the existing processes and at the same time create an environment that fosters innovation (this is another whole topic for a later blog). Looking through my spyglass I found one place where there was something interesting. U.S Army has embarked upon a ambitious plan to transform its business process and they have taken best of two disciplines Lean Management and Six Sigma and have attempted to chart a new course. Check it out at http://www.army.mil/ArmyBTKC/focus/cpi/tools3.htm.

Looks like Lean can partner at where six sigma falters and make it a success. The process map and variance analysis that was rigorously done by the six sigma black belts can now be made to real processes that will bring the change by using Lean Management principles. I feel IT can be a great partner in doing this phase of the BPM initiative. By involving IT in a lean initiative we can create an environment in BPM Projects where, IT groups feels that they are an important enabler of the initiative rather than just a trench warriors executing everybody's wishes. The Improve and Control part of the DMAIC can now be done through a partnership with Business, IT and the Black Belts using Lean as the tool.