Showing posts with label Web services. Show all posts
Showing posts with label Web services. Show all posts

Wednesday, August 13, 2008

Web Service Message Level Implementations

If you're like me and prefer working with the raw XML in your web service implementations instead of dealing with the mess of generated data binding code, this article will show you how to do so using JAX-WS and XPath. In this example, we use this approach to implement a WS-Notification consumer service that receives WSDM-based messages for monitoring the health and performance of services.

JAX-WS has the Provider interface that you can implement to get access to the raw XML message. You have the choice of implementing Provider<Source>, Provider<DataSource>, or Provider<SOAPMessage>. We'll be using the Provider<Source> in this example. The invoke(Source) method is what gets called to process a service request. We'll break down the implementation of that method step-by-step.

First we send the source through a Transformer to get a DOM tree that we can use for XPath processing:

DOMResult dom = new DOMResult();
Transformer trans = TransformerFactory.newInstance().newTransformer();
trans.transform(source, dom);

Next, we instantiate an XPathFactory and create an XPath that we'll use to process the DOM tree against. The third line calls the setNamespaceContext() method and uses a custom class I created that provides a mapping of namespace URI's to prefixes and vice-versa. You can find more information on how to implement such a class here. We need this because we'll be using namespace prefixes in our expressions.

XPathFactory xpf = XPathFactory.newInstance();
XPath xp = xpf.newXPath();
xp.setNamespaceContext(MyNamespaceContext.getInstance());

Now we'll process our first expression to extract the NotificationMessage elements from the message, passing in the XPath expression for the NotificationMessage and our DOM tree to evaluate the expression against. A WS-Notification message may contain multiple NotificationMessages so we get a NodeList in return.

NodeList resultList = (NodeList)xp.evaluate("/wsnt:Notify/wsnt:NotificationMessage", dom.getNode(), XPathConstants.NODESET);

Then we'll iterate through each item in the NodeList, i.e. each NotificationMessage, and process another expression against it to extract the ResourceId element and print it out.
String exprPrefix = "";
int len = resultList.getLength();
for (int i=1;i<=len;i++) {
String expr1 = "/wsnt:Notify/wsnt:NotificationMessage[" + i
+ "]" + "/wsnt:Message/muws1:ManagementEvent"
+ "/muws1:SourceComponent/muws1:ResourceId";
String resourceId = xp.evaluate(expr1, dom.getNode());
System.out.println("Resource ID: " + resourceId);
}
The entire source code listing can be found here. You'll notice that the class has the following two annotations:

@WebServiceProvider()
@ServiceMode(value=Service.Mode.PAYLOAD)

The first one tells JAX-WS that this class is a Provider endpoint. The second one says that we only want to work with the message at the payload level, i.e. we don't need to access the message headers. In addition to these annotations, we have to customize the WSDL to mark the port as a Provider interface. This can be done by embedding the customizations directly within the WSDL or through a separate customization file. We'll do it using a separate provider-customize.xml file--you can find this file here.


Bookmark and Share

Tuesday, April 01, 2008

SOA Message Exchage Patterns

There are a variety message exchange patterns that can be used to implement an SOA. Know them and when to apply them. It can mean the difference between an SOA that scales and performs vs. one that doesn't. The traditional synchronous request/response may not always be the most appropriate in all scenarios. Here are some examples of other types of MEPs:
  • Asynchronous request/response using a callback--the consumer sends a request and provides a reference to a callback service as part of the request; instead of responding synchronously to the request, the service responds asynchronously by sending the response to the callback service
  • Publish/subscribe--consumers register their interest in receiving some data that the service provides by subscribing to the service; whenever the service gets or creates that data, it publishes it to all subscribed consumers. This is somewhat of an extension of the asynch req/resp with callback MEP. The difference is that the pub/sub MEP is used in more of a one-to-many type of scenario, i.e. the service has some data that may be applicable/consumed by multiple consumers; whereas the asynch req/resp with callback is used in more of a one-to-one scenario, i.e. the consumer is requesting some data or functionality that is specific to it (not applicable to other consumers) and doesn't want/need to wait for the response to return immediately
  • Request and poll--the consumer sends a request to the service but doesn't want/need to wait for the response so the service returns some type of identifier that the consumer can use later to poll the service for the response. This MEP is useful for when you're trying to achieve the behavior of the asynch req/resp with callback but can't provide a callback service perhaps because firewall settings don't allow inbound messages.
  • One-way push, aka "fire and forget"--the consumer sends the service a request and doesn't need a response so it just fires it off and forgets about it; perhaps it would be more accurate to call it a message instead of a request since the consumer's not expecting a response. Also it doesn't have to be the consumer firing off the message, it can be the service doing that as well. This is in fact the publish half of the pub/sub MEP for scenarios that don't have any type of guaranteed delivery requirements.


Bookmark and Share

Wednesday, January 30, 2008

SOAP/WS-* vs. REST and The Art of Politecture

Those of you who've worked on very large scale enterprise-wide architectures know that it's always the people and politics that are the biggest hurdles. I came across the term "politecture" in Nick Malik's blog here. You can follow the link for the definition, but it's essentially an architecture that's been designed with the influence of political factors. Some may say this isn't right, but I think it's perfectly fine, even necessary when you're designing a large scale architecture that has to be adopted by many people and organizations.

So what's this have to do with the SOAP/WS-* vs. REST debate? I tend to agree that there are very many scenarios in large scale distributed systems in which a simpler RESTful approach is a technically superior solution. However, IBM, Microsoft, OASIS et al. have done such a good job touting the power of standards in the WS-* world that most folks within large enterprises are sold on the WS-*-based architecture. Whether or not those standards actually help you is a topic for another discussion, but the point is that within most large enterprises you're going to have a much better chance of getting people to adopt your WS-* "standards-based" architecture than you will with a "maverick" REST-based architecture. Is this a problem? Maybe, but from what I've seen usually not. So what if you end up with a technically more complex solution? I can solve the technical problems that result from the added complexity of WS-*. Like I said at the beginning of this post, it's the people and political issues that are the biggest hurdles. From my experience, it's usually the people and politics that cause projects to fail or be canceled. If I can't get people to adopt my architecture because I'm going with a "non-standards-based RESTful architecture", I'm not even going to have a project to worry about it failing because WS-* is more complex, less interoperable, etc.

As an architect, you're not implementing, but you need to get everybody to agree to what you've defined so that they will implement it. If this means making some compromises that perhaps result in a less technically superior solution, but will allow you to achieve consensus then go ahead and do it. As I've painfully realized in my transition from developer to architect over the past few years, things are not so black and white with architecture, i.e. you can't always just go with the solution that makes the most technical sense. If you fail to see this or cannot adapt to this, then you will not be a successful architect.

Bookmark and Share

Wednesday, September 26, 2007

Scenarios for RESTful Web Services

The RESTful approach is very attractive for large scale integration scenarios that cross many organizational boundaries. This is because the constraints imposed by the REST principles emphasize interoperability and scalability. The constraint of uniform interfaces supports those scenarios in which the consumer base for the services is so broad that it makes it difficult to create and maintain a large set of custom interfaces. In such cases, it makes more sense to apply a design in which a single interface can support all the required interactions. Unfortunately, modeling everything as a set of resources that are all exposed through a uniform interface is not always easy. Developers are accustomed to designing a specific interface for each piece of functionality or data that they wish to expose; forcing them to always use a uniform interface is antithetical to this. However, scenarios in which services are just providing access to data can easily support the uniform interface constraint. This is because any kind of data can be manipulated through the same set of create, read, update, and delete operations. Thus, scenarios in which it is primarily data that needs to be shared through web services make REST an easy choice.

, , ,

Scenarios for SOAP WS-* Web Services

The SOAP WS-* approach provides a broad set of standards and specifications for quality of service features and also gives developers a lot of flexibility to define custom interfaces for the services that they wish to expose. This flexibility is useful for application-to-application integration scenarios internal to an organization. This is also useful in scenarios in which legacy applications need to be exposed to the rest of the enterprise. In either of these scenarios, the existing applications often constrain how the services may be exposed, so the flexibility to design service interfaces that can adapt to these constraints is important. Additionally, in these scenarios the services are often providing complex functionality and processes that may be difficult to model in a resource-oriented manner with uniform interfaces. It is also these types of scenarios that typically require many of the complex quality of service features that the SOAP WS-* approach has broad support for. Finally, these types of scenarios are more commonly found inside a single organization and less so across organizational boundaries. The SOAP WS-* approach typically results in large number of custom interfaces, but when this is occurring within a single organization, they are a lot easier to control and maintain than in scenarios where there are many organizations that are dependent on those interfaces.

, , ,

Thursday, August 30, 2007

What a F!#@ing Waste of Time!

Been doing some work with web services management lately and looking at the various standards and specs for that stuff, most notably WSDM and WS-Management. It's always frustrating when you have competing standards. So not only are WSDM and WS-Management competing standards but they are built on top of other competing standards, e.g. WS-Notification vs. WS-Eventing, WS-ResourceFramework vs. WS-Transfer, etc. So the vendors from both camps have decided that they should converge the two sets. After spending a couple years creating these competing standards, now they're going to spend some more time to converge them. So you screw the customers in the first place by creating the competing specs and now you screw them again because you're going to create a whole other set of specs and the previous set will eventually get deprecated. Not to mention the time you waste yourselves creating the competing sets only to get back together to converge them. Those of us in the customer community need to put an end to this madness.

, ,

Friday, April 20, 2007

Guard Your Data Services Carefully

A common thing to do these days is to create data access web services directly on top of your databases to open up access to that data to other services and applications, essentially creating a data services layer for your SOA. I've even recommended such an approach before. While conceptually this may be a good idea, this also presents an increased security risk.

To be widely usable (or reusable), these data services tend to need to support a general querying interface. A lot of EII products will in fact let you create data services that will take XQuery to query the data source. This is in effect analogous to widely opening up a SQL interface into your database. There's a reason why direct connections to databases are not made widely available--either someone can issue malicious queries to the database or some inept developer can create some massive query that will bring the database to its knees.

Besides not widely opening up direct database connections, RDBMSes also have pretty fine-grained access control so that you can specify what users have what kind of access to specific tables and columns. The data services in your SOA will also need such fine-grained access control policies. In this case, they will be specifying access control on the specific operations and XML elements and attributes. The key is to be able to specify and enforce the policies down to the specific elements and attributes.

Ideally, this would be done through a tight integration between the service's policy enforcement point and the XQuery processor (assuming you're supporting XQuery on your data service) so that if your XQuery references attributes and elements you don't have access to, the query wouldn't even be processed. I don't know of any products that do this, but there are some other ways to do it.

Assuming the source of your data service is an RDBMS, you could set up an access control policy internal to the RDBMS that maps to the access control policy of the data service so that once the XQuery is translated to SQL and executed in the RDBMS you let the RDBMS's security mechanisms enforce the access control.

If maintaining two sets of mapped access control policies like this sounds like a pain in the ass, the other approach is to have a post processing step where you can strip out data from the XML using XSLT for the parts that the client does not have access to. With this approach, you need to make sure that you've designed the XML schema such that once you've stripped out those parts you still have a valid XML document. Also this approach may not be the most efficient--bringing back all the data as XML and then applying XSLT to filter stuff out. It would be much more efficient to do that in the RDBMS.

And then lastly, similar to how direct access to databases are not made widely available, you're going to have to be more restrictive about who you grant access to your data services. This will also prevent any Joe Schmoe developer in your enterprise from issuing some crazy XQuery that will bring down your data service and the database behind it. Such restrictiveness may be in direct conflict with one of your reasons for creating data services in the first place, but it's probably a necessary compromise.

Wednesday, September 27, 2006

XML Compression is Not the Answer to Your SOA Performance Woes

Articles like this are misleading. XML compression will reduce the size of the message and thus reduce the time to transfer the message. But if you've ever done any performance testing and profiling on Web services, you will know that most of the times the performance bottlenecks are not in the message transfer but the XML processing, i.e. parsing the message. Thus, compressing the message will not do you much good. Some may say that in a bandwidth constrained environment, the compression does help. But if you're working in a bandwidth constrained environment, you probably shouldn't even be using such technologies in the first place. What does help tremendously is offloading the XML processing to dedicated XML appliances and the article does point that out.

, ,

Wednesday, September 06, 2006

Separation of Concerns in Web Services Article

My article on separation of concerns in Web service implementations is now available on the O'Reilly site onjava.com. Here's the link.

Sunday, July 30, 2006

AOP and SOA Implementations

We are all familiar with the good old fashioned principle of separation of concerns and thanks to Thomas Erl we know how it's a fundamental principle behind Service Oriented Architectures. But we mostly think about separation of concerns at the architectural level. It's also a good idea to apply the separation of concerns at the implementation level, when coding the services that make up the SOA. This is where the techniques of Aspect Oriented Programming come in. While separation of concerns is a design principle, AOP allows you to carry it through to implementation. The Spring Framework is a nice lightweight Java framework with very good facilities for AOP. There's a companion security system for Spring called the Acegi Security System that allows you to secure your applications developed using Spring. Both of these frameworks provide very nice alternatives to JEE for developing Web services. I'm working on an article that shows you how to develop secure Web services using Apache Axis, Spring, and Acegi. Sure, there are plenty of ways to implement Web services but using Spring allows you to use its AOP facilities to drive the separation of concerns down into the implementation. Spring itself also has support for Web services, but since I'm more familiar with Axis, I decided to go with Axis for the example in the article. Acegi has some pretty nice security functionality but to be able to use it to secure Web services developed with Axis requires a bit of work to bridge the Web service security context from Axis into the Acegi security context. This is one of the things that I will show in the article. Stay tuned, I will post a link to the article once it becomes available.

, , , ,

Monday, June 05, 2006

Java SDK for Web Services Security

Came across this announcement from Forum Systems on the JDJ website about their release of a Java SDK for Web Services security:

Forum Systems Releases Java Web Services Security
— Forum released the Forum Java Web Services Security (JWSS) Software Development Kit (SDK) version 1.0 offering developers a comprehensive library of application programming interfaces (API's) to leverage in coding Web Services applications. The JWSS SDK v1.0 addresses the need for security to be enforced within the application itself in order to ensure privacy and integrity of Web Services and SOA applications.
Haven't had a chance to actually check out the SDK but there's definitely a need for those types of tools. Sun has their own set with the XWS Security that's part of the Java Web Services Developer Pack.

While I'm all for separation of concerns and not mixing security code in with the business logic code, there are definitely certain scenarios where you can't avoid it. Although I think they could have come up with a better example to justify when you may need to mix security logic in with your business logic. The example they used is to verify the origin of messages at the point of consumption to ensure that they were not physically intercepted. If that's what I needed to do, I would do that in a request interceptor that is deployed within the endpoint. Most web services toolkits allow you to set up request and response interceptors to do pre and post processing such as this. No need to mix this type of security code in with your business logic.

One scenario I have come across is when you have to make an authorization decision based on certain characteristics of the data that is being processed. In such cases, only the business logic understands the semantics of the data enough to make such a decision. So to do this you either have to put the security code into the business logic or you have to add business logic into your security code so that it understands the data semantics.

So the important questions for the guys at Forum Systems about this SDK are:
  • What standards/specifications is this SDK built on top of/compliant with?
  • What kind of interoperability tests have been conducted for Web Services that are secured using this SDK?
, ,

Sunday, May 07, 2006

Popularity of UDDI in Federal Agencies

Not sure how popular UDDI is in the private sector with the Fortune companies (as I've been working in the government sector for almost a couple years now), but UDDI is alive and well in the federal government, especially in the Department of Defense. The DoD is so large and the amount of money they spend on this stuff is so much that even if they were the only ones using UDDI, that would be enough to sustain it for a very long time. The various Agencies and Services in the DoD already or are planning to have UDDI registries.

In fact so many different organizations are rolling out UDDI registries that they need to start looking at how to federate all these registries. To that regard, there's been a lot of talk that the UDDI specs don't adequately support federation. Admittedly, I'm no expert in UDDI federation but I did take a quick look at the UDDI 3.0 specs the other day and to me it looked like there was decent support for federation in there. You can create a federation by setting up a root registry and a bunch of afiliate registries that get blocks of keys allocated to them from the root registry so that there are no ID collisions in the registry. Then there are also the Replication and Subscription APIs. So if you don't like the affiliates model they have, then you can set up your own federation model and use these APIs to synchronize the data among all the instances. So what else is missing?

,

Friday, April 28, 2006

Security Data Conflicts in Large Scale SOAs

Most large scale SOAs will use some type of role-based or attribute-based security mechanisms to control access to the data and services. In most organizations, this will involve integrating multiple siloes of security domains containing user profiles, policies, access control lists, etc. Each system can in fact be viewed as a siloed security domain since most systems have their security models and infrastructures designed and developed independently of other systems. When you look at it, this is really just another integration scenario. Just as with other integration scenarios, you will run into conflicts with the data. Here, you're dealing with security data--such as attributes about the users (or principals) and attributes about the resources. For an organization that's using attribute-based access control, here are some examples of some of the types of conflicts that might be encountered:
  1. The names of the attributes that describe the principals may not match across the different security domains. You'll probably have security policies that say something like "grant access to this resource if attribute ABC of the user equals XYZ". With multiple security domains, there's no guarantee that that attribute about a user will always be called ABC. If that's the case, how does the service deciding whether or not to grant access evalute the policy and make a decision?
  2. The attributes that describe a principal may not exist in all domains. In other words, some domains may use more attributes to describe their principals and hence reference these attributes in their policies while in other domains principals aren't even described with such attributes. Using the example from #1, how can you evaluate the policy if the user doesnt' even have an attribute ABC?
  3. The semantics of the attributes vary across different domains. This one's actually pretty common. There are a lot of access control rules out there that say "grant access to this resource if the user is an admin". But the concept of admin can vary greatly from one domain to another. For example you can have system admins and HR admins. Does that mean you should grant some system admin access to some HR service that deals with personnel files?
Some of these issues get into the areas of federated identity management and some of the standards out there such as SAML, WS-Federation, WS-Trust can help. Also there are products out there by companies such as Vordel and Layer 7 based on some of these standards that helps you to mediate some of the conflicts. If you have to deal with some of the issues I've described here, it's worthwhile to take a look at what they have to offer. But at the end of the day it will still require some coordinated effort to sync up all the participating parties so that there is some commonality in the security language used to describe the principals, resources, and access control rules.

, , , , ,

Tuesday, April 18, 2006

Would You Build an SOA Using CORBA Today?

Had an interesting discussion with some colleagues today. If you were building an SOA today and didn't have to worry about any legacy issues, would you always go with web services or would you still consider CORBA in some cases? My position was that in some cases it may still make sense to go with CORBA instead of web services. For example if you didn't have wide interoperability requirements (a lot of external partners to integrate with) and performance is a major concern, then you may be better off using CORBA. I think the main advantage of web services over CORBA is the improved interoperability due to the use of XML and http, but if you're working strictly within the organization then perhaps you don't need all that. Maybe you go with a hybrid approach where within the organization the services use CORBA and any services that are exposed externally use web services. Bottom line is don't rule out CORBA just yet, there are many situations in which it still makes sense.

, ,

Thursday, April 13, 2006

Excel--The End User Friendly Rich Client for Web Services?

StrikeIron recently announced the ability to integrate web services from their Business Network into Excel. Web services have been lacking an end user-friendly client for a while. Will this announcement from StrikeIron push Excel to fill that gap? We all know that Excel has always been the tool of choice for business analysts. Now that they can directly tap into reusable web services from Excel, maybe we'll really start to see the use of web services as true business services that are reusable by people on the business side instead of just developers.

, ,