SOA Real-World challenges

 This post discusses some of the challenges architects/designers face while trying to implement SOA in their Enterprise. Our team has been working with customers for the past two and half years to implement SOA. This post reflects collective experience of our team.

 

Most of the discussions around SOA revolves around basic issues as RPC vs Doc/Lit, Remoting vs Web Services, usage of WS-* protocols etc. These are relevant issues and has to be discussed in detail. But, there are other important issues that needs to tackled in order to implement SOA in an enterprise. In my opinion, WS-* is necessary, but not sufficient. So far we have been talking about the necessary part. This post will cover the sufficient part.

 

Today, SOA is mainly used in connecting applications. In other words, the sweet spot for SOA is primarily in the integration space. If you look at the solutions that are built in a typical enterprise today, they fall into two classes of solutions - Service Delivery and Information Analysis. Service delivery - as its name suggests - is about delivering services offered by business systems (like ERP) to end-users directly. Self-Service solutions such as Employee Self-Service portal falls into this category. Primary motivation for self-service type solutions is to reduce the cost by dis-intermediating the interaction between user and system.

 

Let's take Employee Self-service portal as an example. Building this portal involves integrating multiple systems - HR System, Payroll system and Benefits system. Usually, these are separate systems and are autonomous in nature. Some of these systems may even be outsourced. For example, Payroll capability is usually outsourced.

 

Now, let's look at the challenges involved in building a self-delivery application.

 

1. Identity management - Usually, every business system maintains their own identity profile. They also define their own authorization realms - they define roles specific to the business functionality they expose. So, managing identity across multiple systems is a major challenge. Some of the issues you have to resolve are
   1. Single Sign-On (both web-based SSO and system level SSO)
   2. Federation - For example, Payroll systems and benefits systems may be managed by a different enterprise. How do I federate identities across organization boundaries.
   3. Provisioning - When a new identity is created (during hiring for example), what is the process involved in creating appropriate identities in other systems.
   4. Synchronization - Usually Single-Sign on solution requires some sort of credential mapping. How do we make sure when password changes occur in a business system and how do we synchronize that SSO credential mapping store.
   5. Authorization - How do I authorize in this world?

 

1. EII (Enterprise Information Integration)

   1.  Every system maintain its own view of employee. For example, payroll system defines the payroll specific view for the employee entity, benefits system defines benefits specific view for the employee entity etc. But, building a service-delivery application requires a unified view of employee. A better example would be 'Single view of customer' issue that every enterprise faces.

   2. Horizontal partition - There are cases where entity instances are partitioned across multiple services. How do we present an aggregated view.

   3. There are cases where your solution requires cross join of entities encapsulated by different systems. In that case, you have no choice but to bring those entities to a central store. Cross-join scenarios are so devastating that your mind set will shift from service-orientation to data-orientation. I have gone through that experience :)
2. Schema Harmonization - Every service maintains its own view of an entity. How do we harmonize the schematic differences between services? Canonical schema is one practical way to harmonize the differences. But, the real challenge is to get all the stake-holders to agree on the schema. This is where a strong enterprise architecture team can make a difference. Another challenge is - would you go for a big-bang approach with enterprise modeling or would you go for incremental approach. Current thinking is to go with incremental approach. Start with few commonly used entities such as Customer, Product etc and then add more stuff later on.

 

1. Service Management - How do I manage, bill, meter, provision, de-provision services?

 

There are more issues. But, these are the primary challenges one would face while trying to design a solution based on services. In my future posts, I will spend some time on each of these issues and share with you some of our experiences and key learnings.

 

I would love to hear your opinions on this subject. I am sure some of you may have implemented SOA in your IT and faced similar challenges. I am also sure that you may have come up with better solutions and have a better insight into this problem. Please, please share your learnings. We can all learn from each other.

Comments

• Anonymous
  April 23, 2004
  Ram,
  
  some of the challenges we encounter as we march towards a SOA vision in our $2B financial/benefits services provider
  
  While every one understands web services and its plus, its unfortunate that most developers relate SOA directly to web services and assume it as the end and that they have enabled SOA from their perspective ( i think we are not even close to beginning). perhaps lack of education and exposure on SOA and WS-*? (let me be honest and put myself also, in this lack of knowledge group, there is just too much to keep track of). Is the industry growing too fast too soon to be called mature?
  
  I also fear in our urgency to enable SOA, we would end up creating scores of web services all over the enterprise, still talking only point-to-point, without a proper intermediary platform. Another mesh of mess?
  
  we are partially approaching the schema harmonization problem by recommending standards payloads. I guess we wanted to reuse what the consortiums have done to define the business entities...but the industry standards themselves are split, promoting more than a few standards...we see no convergence? no one common set of vocabulary? If we folks who should clear are confused, how can we expect a stable SOA anytime soon?
  
  In a huge organizations like ours, there are potentially 1000s of applications, ranging from individual MS access/excels to monsterous mainframe systems. Resistance to change is a major factor across this spectrum. A multi-year plan to dig out the business from these applications(in some cases, should I say files?) is a sure pre-cursor to SOA initiatives. On top of that, owners of legacy apps, perhaps as old as their systems are, feel things like enterprise architecture and SOA are making too much noise. Trust me, tough sell with them.
  
  What are you thoughts on these?

• Anonymous
  April 28, 2004
  Sudhar,
  
  I agree with your second point that concerns about creating scores of web services all over enterprise. In my opinion, there are two key things that act as driver for creating services in the enterprise. They are a) Business process b) Service delivery solutions.
  
  Business process is a very important driver. Activities in a business process almost always spans system boundaries. In order to support a business process, you may have to expose certain functionalities across the service boundary, preferably through web services.
  
  Second scenario is about building solutions (like self-service) that requires integration of systems. Again, this scenario also drives exposing functionalities across service boundaries.
  
  Thanks
  Ram

• Anonymous
  July 14, 2004
  Ram,
  
  I feel in the industry many people are not applying the SOA model properly.
  Can you please confirm me that SOA model can be used for all type of app's. I feel choosing an architecture is totally depended on the requirements & available infrastructure. Can you throw some lights on my views...
  
  Ram-- just to give a background about myself--- i have just started blogging.. I work at wipro, currently working at the dallas office... I think i have met u when u came to our campus. I am also an MVP in C#...

• Anonymous
  May 29, 2009
  PingBack from http://paidsurveyshub.info/story.php?title=runofthemillblog-soa-real-world-challenges

• Anonymous
  June 09, 2009
  PingBack from http://insomniacuresite.info/story.php?id=6906

• Anonymous
  June 19, 2009
  PingBack from http://debtsolutionsnow.info/story.php?id=12676