Saturday, February 13, 2010

The simplest possible GWT application using UiBinder

I've been thinking about building a application to help my daughters practice math. The idea is to generate a simple daily quiz that they take online. The application should send them an SMS or email reminder to take the quiz, and then track their results. Each completed quiz should earn them points (number of correct answered problems) that they can redeem with me (in exchange for something to-be-decided). Each day they fail to get over some threshold or fail to take the quiz at all, they lose some number of number of points.

Google App Engine seems like a good fit for implementing this app, and anyway I've been wanting to try it out and also learn how to use GWT. I just installed the latest Eclipse plugin (App Engine 1.3.1 and GWT 2.0.2).

Where to begin? Whenever I learn a new technology, I find it helpful to start with the simplest possible scenario and then iterate, making sure I understand each additional layer of complexity. A logical starting point was just to build an app that outputs static HTML.

GWT 2.0 introduced a new declarative technique for defining the UI in XML markup rather than code. The UiBinder documentation has examples but doesn't have a complete tutorial that assumes no previous knowledge of GWT. I started with the Mail sample app and pared it down to the simplest possible example.

The first step was to create a new Web Application Project in Eclipse using org.sample.uitest as the default project package. Then I deleted all of the boilerplate code that gets generated for the new project, leaving a src directory with empty client, server and shared packages. I also removed the servlet mapping from war/WEB-INF/web.xml.

The first file that was clearly necessary was the 'module definition' in src/org/sample/uites/Uitest.gwt.xml. The simplest possible module definition inherits the basic toolkit functionality and stylesheet and defines an entry point into the application:

<module rename-to='uitest'>

  <inherits name='com.google.gwt.user.User'/>

  <inherits name='com.google.gwt.user.theme.standard.Standard'/>

  <entry-point class='org.sample.uitest.client.Uitest'/>

</module>

Next I needed to implement the entry point class. I started by copying the entry point from the Mail sample app and paring it down:

package org.sample.uitest.client;

import com.google.gwt.core.client.EntryPoint;
import com.google.gwt.core.client.GWT;
import com.google.gwt.uibinder.client.UiBinder;
import com.google.gwt.user.client.Window;
import com.google.gwt.user.client.ui.DockLayoutPanel;
import com.google.gwt.user.client.ui.RootLayoutPanel;

public class Uitest implements EntryPoint {

  interface Binder extends UiBinder { }

  private static final Binder binder = GWT.create(Binder.class);

  public void onModuleLoad() {

    DockLayoutPanel outer = binder.createAndBindUi(this);

    Window.enableScrolling(false);
    Window.setMargin("0px");

    RootLayoutPanel root = RootLayoutPanel.get();
    root.add(outer);
  }
}

The Binder requires a matching layout file named Uitest.ui.xml:

<ui:UiBinder
  xmlns:ui='urn:ui:com.google.gwt.uibinder'
  xmlns:g='urn:import:com.google.gwt.user.client.ui'>

  <g:DockLayoutPanel unit='EM'>

    <g:center>
      <g:HTML>
        <p>UI Test</p>
      </g:HTML>
 
    </g:center>

  </g:DockLayoutPanel>

</ui:UiBinder>

Finally, an HTML page named Uitest.html is required in the war directory to 'host' the application:

<!doctype html>

<html>
  <head>
    <meta http-equiv="content-type" content="text/html; charset=UTF-8">
    <title>UI Test</title>
    <script type="text/javascript" language='javascript' src='uitest/uitest.nocache.js'></script>
  </head>
  <body>
    <noscript>
      <div style="width: 22em; position: absolute; left: 50%; margin-left: -11em; color: red; background-color:white; border: 1px solid red; padding: 4px; font-family: sans-serif">
        Your web browser must have JavaScript enabled
        in order for this application to display correctly.
      </div>
    </noscript>
  </body>
</html>

At this point my project consisted of five files, including WEB-INF/web.xml. I ran the project in Eclipse and was able to see the message in the browser.

Out of curiosity, I looked at the requests in Firebug when the page loads. The initial response is simply an unmodified version of the 'host' page, which loads uitest.nocache.js to actually render the page. It appears that uitest.nocache.js executes an onload handler that in turn triggers a request for hosted.html?uitest. I didn't try very hard to decipher the obfuscated JavaScript, but it appears that it is writing a SCRIPT tag to load the dynamic content.

Sunday, August 30, 2009

Log4J and Spring

I just spent a half hour trying to increase the logging for a unit test that uses SpringJUnit4ClassRunner. First I thought I could just add a log4j.xml configuration file at the root of the classpath, but that didn't work. After looking around the Spring code a bit it dawned on me that I didn't even have log4j on the classpath for the project, so Commons Logging was using the standard java.util.logging package instead. I added log4j as a dependency to the project, and now log4j configuration works as expected.

Saturday, August 15, 2009

Getting started with Spring Web Services

I needed to quickly implement a simple (document/literal wrapped) web service, so I thought I would try Spring Web Services. Unfortunately, just following the tutorial did not result a working webapp. I had to do some searching and reading of the source code to figure out a few problems. Here are my notes.

Like every other part of the Spring portfolio, Spring Web Services provides a wide variety of ways to accomplish the same task. I chose the following options:

  • Use the @Endpoint and @PayloadRoot annotations to map methods to SOAP operations.
  • Use JAXB 2 for XML binding.

This combination seemed to require the least amount of coding and configuration directly related to web services, allowing me to focus on the actual service implementation.

Set up your environment

There are only two things you absolutely need to get started with Spring Web Services:

  1. JDK 6, including the xjc command-line tool for generating XML binding classes from your schema.
  2. Maven (http://maven.apache.org) for managing dependencies and building the project

Maven eliminates the need to download Spring Web Services or any of its dependencies directly.

Generate the project skeleton

Maven makes it really easy to create the basic skeleton for the project:

mvn archetype:create \
  -DarchetypeGroupId=org.springframework.ws \
  -DarchetypeArtifactId=spring-ws-archetype \
  -DgroupId=org.xocoatl.ws \
  -DartifactId=web-service

This command downloads the required dependencies to your local repository and builds the following directory structure:

$ ls -R web-service/
web-service/:
pom.xml  src

web-service/src:
main

web-service/src/main:
resources  webapp

web-service/src/main/resources:

web-service/src/main/webapp:
WEB-INF

web-service/src/main/webapp/WEB-INF:
spring-ws-servlet.xml  web.xml

This structure just follows the Maven conventions for a webapp.

Write the web service contract

Spring Web Services only supports a contract-first service development model, so the next task is to write the XML schema that defines your service. Create web-service/src/main/webapp/WEB-INF/web-service.xsd and write your contract:

<?xml version="1.0" encoding="UTF-8"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
  xmlns:tns="urn:ws.xocoatl.org" elementFormDefault="qualified"
  targetNamespace="urn:ws.xocoatl.org">

  <xsd:element name="EchoRequest">
    <xsd:complexType>
      <xsd:sequence>
        <xsd:element name="Message" type="xsd:string" />
      </xsd:sequence>
    </xsd:complexType>
  </xsd:element>

  <xsd:element name="EchoResponse">
    <xsd:complexType>
      <xsd:sequence>
        <xsd:element name="Message" type="xsd:string" />
      </xsd:sequence>
    </xsd:complexType>
  </xsd:element>

</xsd:schema>

At minimum, your schema will include types representing the payload of the request and response messages for each operation in your service. Spring-WS includes the ability to automatically generate a WSDL that includes your schema. As I discovered, however, the default WSDL generator requires that your payload types end in Request or Response. This seems to go against the typical convention of having your request payload elements match the operation name (i.e. Echo rather than EchoRequest.

Generate the XML binding classes

Once you have defined your contract, you can use the xjc command-line tool to generate the source code for the XML binding classes that your service will use. The simplest thing is to just add this to the maven build configuration. Here is how my pom.xml wound up:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>org.xocoatl.ws</groupId>
  <artifactId>web-service</artifactId>
  <packaging>war</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>Web Service</name>

  <build>
    <plugins>
      <!-- use JDK 6 -->
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <configuration>
          <source>1.6</source>
          <target>1.6</target>
        </configuration>
      </plugin>
      <!-- use the command-line xjc tool that comes with JDK 6 
           to generate XML binding classes from schema -->
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>exec-maven-plugin</artifactId>
        <version>LATEST</version>
        <executions>
          <execution>
            <id>mkdir</id>
            <phase>generate-sources</phase>
            <goals>
              <goal>exec</goal>
            </goals>
            <configuration>
              <executable>mkdir</executable>
              <arguments>
                <argument>-p</argument>
                <argument>target/generated-sources/xjc</argument>
              </arguments>
            </configuration>
          </execution>
          <execution>
            <id>xjc</id>
            <phase>generate-sources</phase>
            <goals>
              <goal>exec</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <executable>xjc</executable>
          <arguments>
            <argument>-p</argument>
            <argument>org.xocoatl.schema</argument>
            <argument>-d</argument>
            <argument>target/generated-sources/xjc</argument>
            <argument>src/main/webapp/WEB-INF/web-service.xsd</argument>
          </arguments>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.mortbay.jetty</groupId>
        <artifactId>maven-jetty-plugin</artifactId>
        <version>LATEST</version>
      </plugin>
    </plugins>
  </build>
  <dependencies>
    <dependency>
      <groupId>org.springframework.ws</groupId>
      <artifactId>spring-oxm</artifactId>
      <version>1.5.7</version>
    </dependency>
    <dependency>
      <groupId>org.springframework.ws</groupId>
      <artifactId>spring-oxm-tiger</artifactId>
      <version>1.5.7</version>
    </dependency>
    <dependency>
      <groupId>org.springframework.ws</groupId>
      <artifactId>spring-ws-core</artifactId>
      <version>1.5.7</version>
    </dependency>
    <dependency>
      <groupId>org.springframework.ws</groupId>
      <artifactId>spring-ws-core-tiger</artifactId>
      <version>1.5.7</version>
    </dependency>
  </dependencies>
</project>

There is also a maven plugin to invoke xjc, but maven reported errors while trying to download it, and with JDK 6 it is easy to just execute the command-line tool directly. Note that it is necessary to create the output directory first, for some reason xjc will not do this for you. Windows users may need to be tweak this step.

This task is bound to the generate-sources phase of the build lifecycle, so if necessary it will always run before compiling your own code.

While editing the POM file, I made a few other additions:

  1. Plugin configuration for building the project for Java 6
  2. Dependencies required for using the annotation features of Spring-WS
  3. Plugin configuration for running the webapp in Jetty during development

Now you are ready to actually generate the XML binding classes:

$ mvn generate-sources
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Building Web Service
[INFO]    task-segment: [generate-sources]
[INFO] ------------------------------------------------------------------------
[INFO] [exec:exec {execution: mkdir}]
[INFO] [exec:exec {execution: default}]
[INFO] parsing a schema...
[INFO] compiling a schema...
[INFO] org/xocoatl/schema/EchoRequest.java
[INFO] org/xocoatl/schema/EchoResponse.java
[INFO] org/xocoatl/schema/ObjectFactory.java
[INFO] org/xocoatl/schema/package-info.java
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2 seconds
[INFO] Finished at: Sat Aug 15 21:07:19 PDT 2009
[INFO] Final Memory: 7M/79M
[INFO] ------------------------------------------------------------------------

The generated source code is written to target/generated-sources/xjc, where it is available for building with your own code.

Implement the endpoint

The next step is to implement the endpoint itself. Maven requires that your source code reside in src/main/java. I created a file at src/main/java/org/xocoatl/ws/EchoEndoint.java for my implementation:

package org.xocoatl.ws;

import org.springframework.ws.server.endpoint.annotation.Endpoint;
import org.springframework.ws.server.endpoint.annotation.PayloadRoot;
import org.xocoatl.schema.EchoRequest;
import org.xocoatl.schema.EchoResponse;

@Endpoint
public class EchoEndpoint {

  @PayloadRoot(localPart = "EchoRequest", namespace = "urn:ws.xocoatl.org")
  public EchoResponse echo(EchoRequest echoRequest) {

    EchoResponse echoResponse = new EchoResponse();
    echoResponse.setMessage(echoRequest.getMessage());
    
    return echoResponse;
  }
}

Usually your endpoint will delegate to an underlying service to perform some logic. The code in the endpoint simply extracts data from the request, passes it to the service, and then builds an appropriate response.

Write the Spring configuration

By convention, the Spring configuration resides in src/main/webapp/WEB-INF/spring-ws-servlet.xml. The samples in the Spring-WS distribution are a tossed salad of various approaches, but this is the minimal configuration that seems to be necessary for the annoation + JAXB approach:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:sws="http://www.springframework.org/schema/web-services"
  xsi:schemaLocation="http://www.springframework.org/schema/beans 
                      http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
                      http://www.springframework.org/schema/web-services 
                      http://www.springframework.org/schema/web-services/web-services-1.5.xsd">

  <bean id="echoEndpoint" class="org.xocoatl.ws.EchoEndpoint" />

  <bean
    class="org.springframework.ws.server.endpoint.adapter.GenericMarshallingMethodEndpointAdapter">
    <constructor-arg ref="marshaller" />
    <constructor-arg ref="marshaller" />
  </bean>

  <bean id="marshaller" class="org.springframework.oxm.jaxb.Jaxb2Marshaller">
    <property name="contextPath" value="org.xocoatl.schema" />
  </bean>

  <bean
    class="org.springframework.ws.server.endpoint.mapping.PayloadRootAnnotationMethodEndpointMapping" />

  <bean id="schema" class="org.springframework.xml.xsd.SimpleXsdSchema">
    <property name="xsd" value="/WEB-INF/web-service.xsd" />
  </bean>

  <bean id="echo"
    class="org.springframework.ws.wsdl.wsdl11.DefaultWsdl11Definition">
    <property name="schema" ref="schema" />
    <property name="portTypeName" value="Echo" />
    <property name="locationUri" value="http://localhost:8080/" />
    <property name="targetNamespace" value="urn:ws.xocoatl.org" />
  </bean>

</beans>

Every endpoint requires two form of configuration:

  1. some way to map methods to the payload of the request message. In our case the job is handled by PayloadRootAnnotationMethodEndpointMapping in conjunction with the PayloadRoot annotation.
  2. an adapter for reading the request and writing the response messages. In our case we are using JAXB marshalling and unmarshalling for these tasks.

Note that the id of the DefaultWsdl11Definition bean winds up in the URL of the WSDL.

Run the application

Once the configuration is complete, you can run the application from the command line:

$ mvn jetty:run

At this point you should be able to browse to http://localhost:8080/echo.wsdl. You should also be able to load the WSDL into a tool such as soapUI and try out the operation.

Saturday, February 23, 2008

Recipe for Building a Maven Plugin

Recently I've become a maven convert. One of the hardest parts of making the switch from Ant was the fact that I couldn't implement custom logic directly in pom.xml the way I could in an Ant build file. I had to make the mental shift to writing custom plugins instead, which at first glance seemed a lot more complicated than writing Ant targets.

This weekend I finally sat down and worked through the process of writing a custom plugin. It's actually pretty easy, and the end result is a nice reusable component.

I found that I had to refer to both the "Better Builds with Maven" book as well as the plugin developers' guide on the maven web site to get the gist of writing a plugin. Here is the general recipe I came up with.

A mojo implements a maven goal. If you want to be able to type: mvn my:mygoal

and have something happen, then you need to implement a single mojo.

To do this, first use maven to generate the boilerplate for a plugin project:


mvn archetype:create -DgroupId=com.mycompany.maven.plugins \
  -DartifactId=maven-my-plugin \
  -DarchetypeArtifactId=maven-archetype-mojo

Edit pom.xml:

  • Change name and artifact ID
  • Set build source and target to Java 5:
    
      <build>
        <plugins>
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>very
            <configuration>
              <source>1.5</source>
              <target>1.5</target>
            </configuration>
          </plugin>
        </plugins>
      </build>
    
  • Add dependencies

Rename the generated MyMojo.java file to the desired name of your mojo, and edit the file.

First change the class-level mojo annotations. For an explicitly invoked task (not automatically bound to a lifecycle phase), remove the @phase annotation and only specify the @goal annotation.

Update the field-level annotations. Use @parameter to set a field value based on the plugin configuration.

Finish implementing and testing mojo.

Install the finished plugin to your local repository using mvn install. If you look in target/classes/META-INF/maven/plugin.xml, you'll see that maven has extracted all the annotations from your mojo into a plugin descriptor file. Maven uses this file at runtime to map configuration properties onto mojo fields.

Next switch over to a module where you want to use your newly implemented goal. Add your new plugin to the module's pom.xml:


  <build>
    <plugins>
      <plugin>
        <groupId>com.mycompany.maven.plugins</groupId>
        <artifactId>maven-my-plugin</artifactId>
        <executions>
          <execution>
            <id>mygoal</id>
            <goals>
              <goal>mygoal</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <myfield>myvalue</myfield>
        </configuration>
      </plugin>
    </plugins>
  </build>

Now you can execute the goal: mvn my:mygoal

Certainly not as easy as adding an Ant target, but the nice thing about maven is that you rarely have implement custom build logic if you just follow the maven conventions.

Sunday, February 10, 2008

Multiple Persistence Units with Spring and JPA: Part II

In my last post I explained how I'd like to implement a persistence unit as a self-contained module that I could drop into another application that already has its own persistence unit. Both persistence units should talk to their respective database schemas without problems.

The two basic parts to the solution I worked out are: (1) ensure that the location of all configuration files and the names of all Spring-managed beans are unique; and (2) always specify which persistence unit and transaction manager a class requires.

To implement the solution, I established a set of conventions based on the persistence unit name:

First convention. A reusable code module that contains a persistence unit should have three configuration files located in META-INF/persistence-unit-name. If the persistence unit is named myunit, here is the structure of the jar file that contains the module:

  • com/**/**/*.class (Java classes related to the persistence unit
  • META-INF
    • myunit
      • META-INF/myunit/applicationContext.xml
      • META-INF/myunit/database.properties
      • META-INF/myunit/persistence.xml

The persistence.xml only has to specify the persistence unit name:

<persistence xmlns="http://java.sun.com/xml/ns/persistence"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://java.sun.com/xml/ns/persistence 
                     http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd"
 version="1.0">

 <persistence-unit name="myunit" transaction-type="RESOURCE_LOCAL">
 </persistence-unit>

</persistence>

The database.properties file contains a set of overridable connection settings:

database.url=jdbc:oracle:thin:@localhost:1521:ORCL
database.user=mydata
database.password=mydata

Second convention. The Spring configuration file for the reusable module should define a database connection pool, entity manager factory and transaction manager for the module's persistence unit. To avoid naming conflicts, all bean names and properties should be prefixed with the persistence unit name:

<?xml version="1.0" encoding="UTF-8"?>

<beans xmlns="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:context="http://www.springframework.org/schema/context"
  xmlns:tx="http://www.springframework.org/schema/tx"
  xmlns:aop="http://www.springframework.org/schema/aop"
  xsi:schemaLocation="http://www.springframework.org/schema/beans 
                     http://www.springframework.org/schema/beans/spring-beans.xsd 
                     http://www.springframework.org/schema/tx 
                        http://www.springframework.org/schema/aop 
                        http://www.springframework.org/schema/aop/spring-aop-2.5.xsd
                        http://www.springframework.org/schema/tx/spring-tx-2.0.xsd
                        http://www.springframework.org/schema/context 
                        http://www.springframework.org/schema/context/spring-context-2.5.xsd">

  <context:annotation-config />
  
  <context:component-scan base-package="com.mycompany.myunit" />

  <bean id="myunitDatabaseConfigurer"
    class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
    <property name="locations">
      <value>classpath:/META-INF/myunit/database.properties</value>
    </property>
    <property name="systemPropertiesModeName">
      <value>SYSTEM_PROPERTIES_MODE_OVERRIDE</value>
    </property>
    <property name="placeholderPrefix" value="$MYUNIT{"/>
  </bean>

  <bean id="myunitDataSource"
    class="com.mchange.v2.c3p0.ComboPooledDataSource">
    <property name="driverClass"
      value="oracle.jdbc.driver.OracleDriver" />
    <property name="jdbcUrl" value="$MYUNIT{database.url}" />
    <property name="user" value="$MYUNIT{database.user}" />
    <property name="password" value="$MYUNIT{database.password}" />
  </bean>

  <bean id="myunitEntityManagerFactory"
    class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
    <property name="dataSource" ref="myunitDataSource" />
    <property name="jpaVendorAdapter" ref="myunitJpaAdapter" />
    <property name="persistenceXmlLocation" value="classpath:/META-INF/myunit/persistence.xml"/>
  </bean>

  <bean id="myunitTransactionManager"
    class="org.springframework.orm.jpa.JpaTransactionManager">
    <property name="entityManagerFactory" ref="myunitEntityManagerFactory" />
  </bean>

  <tx:advice id="myunitTxAdvice" transaction-manager="myunitTransactionManager">
    <tx:attributes>
      <tx:method name="*" read-only="true"/>
    </tx:attributes>
  </tx:advice>
  
  <aop:config>
    <aop:pointcut id="myunitTxPointcut" expression="execution(* com.mycompany.mypackage.*.*(..))"/>
    <aop:advisor advice-ref="myunitTxAdvice" pointcut-ref="myunitTxPointcut"/>
  </aop:config>

  <bean id="myunitJpaAdapter"
    class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
    <property name="database" value="ORACLE" />
    <property name="showSql" value="true" />
  </bean>

</beans>

Third convention. Always specify the persistence unit name when using the PersistenceContext annotation to inject an entity manager into a class that is part of the module:

  @PersistenceContext(unitName = "myunit")
  private EntityManager entityManager;

Fourth convention. Use explict AOP configuration in the Spring configuration file for declaring transaction boundaries. The Transactional annotation does not support multiple local transaction managers tied to different persistence units, so it cannot be used in the context of a reusable module.

Note that the module should not publicly expose any non-transactional methods involving database access. This ensures that the correct transaction manager is used for the persistence unit.

With these conventions, I was able to build the module as a jar file that can be tested in isolation and also deployed to other applications in three steps:

  1. add the jar file to the classpath
  2. add classpath:/META-INF/myunit/applicationContext.xml to the list of configuration locations for the application's Spring container.
  3. (optional) override database settings via system properties as necessary

The conventions may be too restrictive in some cases but they work well for my situation.

Saturday, February 09, 2008

Multiple Persistence Units with Spring and JPA

Recently I've been building an application with Spring, Hibernate and JPA. The application uses a single persistence unit with a number of entities stored in a single database schema.

Next I need to add some functionality that queries data from another database schema. I'd like to be able to implement this as a self-contained module packaged as a jar file that could be dropped into any application, not just the one I'm currently implementing. The module therefore needs to configure its own persistence unit and all supporting objects, namely a database connection pool and transaction manager. I don't need global transaction support since the module is only reading information from the second schema.

To meet these criteria, the jar file for the module needs to include:

  1. Annotated classes reponsible for executing queries using some combination of JPA and Hibernate APIs.
  2. JPA configuration file for defining the persistence unit
  3. Spring configuration file that declares beans for the database connection pool, entity manager factory and transaction manager and initializes the persistence unit.

Applications should be able to integrate the module in three steps:

  1. Add the jar file to the classpath
  2. Add the module's Spring configuration file to the list of files used to configure the application's Spring container.
  3. Define system properties to override the database connection information (url, user name, password) for the module

The Spring documentation has a brief section on dealing with multiple persistence units. The instructions there seemed relevant but the solution I ultimately came up with did not require defining my own PersistenceUnitManager. I'll explain the solution in my next post.

Saturday, February 17, 2007

Removing nodes from an XML document in Flex 2

E4X is the ECMAScript language extension for accessing and manipulating XML. One odd thing about E4X is that it has no methods for removing nodes. This has stumped me for a while, but I just discovered that you simply use the delete operator:

var doc:XML = <folder label="Parent" name="parent">
   <folder label="Child A" name="childa"/>
   <folder label="Child B" name="childb"/>
 </folder>;

// remove the child folders
delete doc.folder;

trace(doc.toXMLString());

Easy once you know how to do it.

Saturday, February 10, 2007

Debugging in Flex 2

Until now I have been adding Alert.show() statements in ActionScript to help me debug my Flex application. This is very cumbersome, so I tried to figure out how to enable trace() statements that would write messages to a local log file.

I couldn't find any up-to-date instructions, but here is what I was able to piece together for Firefox on Windows at least:

  1. Download and install the latest debug version of the Flash plugin.
  2. Create or edit the mm.cfg file (located in C:\Documents and Settings\<user>\ in Windows). The file should have these three entries:
    TraceOutputFileEnable=1
    ErrorReportingEnable=1
    TraceOutPutFileName=c:\Documents and Settings\<user>\Application Data\Macromedia\Flash Player\Logs\flashlog.txt
  3. Install the FlashTracer extension for Firefox.
  4. Restart Firefox

Prior to the latest player revision it was necessary to set the HOMEDRIVE and HOMEPATH environment variables, but apparently not anymore. Also note that it is not possible to simply change TraceOutPutFileName to another location.

You do not need to compile your Flex application with the -debug option to use the trace method.

Saturday, February 03, 2007

Flex 2: Adding a toolbar to the list pane

I wanted to add a toolbar to the top of the list pane of my remote file explorer. The toolbar would contain icon buttons for some basic file operations like add, move, copy and delete.

After some poking around, it seemed that Flex does not have a built-in component for this purpose. The closest container for this purpose seemed to be ApplicationControlBar. I was able to create something that looks and acts like a typical desktop application toolbar with this declaration:

<mx:ApplicationControlBar dock="true" width="100%" 
  paddingTop="2" paddingRight="2"
  paddingLeft="2" paddingBottom="2" horizontalGap="2">
  <mx:PopUpButton id="newItem" label="" height="36" 
    styleName="toolbarButton" toolTip="Add Item..."
    icon="@Embed(source='assets/24x24/document_add.png')"/>
  <mx:Spacer width="8"/>
  <mx:Button id="newFolder" label="" height="32" 
    styleName="toolbarButton" toolTip="Add Folder..."
    icon="@Embed(source='assets/24x24/folder_add.png')"/>
  <mx:Spacer width="8"/>
  <mx:Button id="copyItems" label="" height="32" 
    styleName="toolbarButton" toolTip="Copy Selected Items"
    enabled="false" alpha="0.4"
    icon="@Embed(source='assets/24x24/copy.png')"/>
  <mx:Button id="moveItems" label="" height="32" 
    styleName="toolbarButton" toolTip="Move Selected Items"
    enabled="false" alpha="0.4"
    icon="@Embed(source='assets/24x24/document_into.png')"/>
  <mx:Button id="deleteItems" label="" height="32" 
    styleName="toolbarButton" toolTip="Delete Selected Items"
    enabled="false" alpha="0.4"
    icon="@Embed(source='assets/24x24/delete.png')"/>
</mx:ApplicationControlBar>

One additional puzzle was how to disable the copy, move and delete buttons until the user selects one or more items. The enabled flag on the Button component controls user interaction, but setting this flag to false does not change a button's appearance unless you specify the disabledColor, disabledIcon, or disabledSkin properties. I didn't want to create a second set of disabled icons. Eventually I figured out I can use the alpha property to dim out the button when disabled. I added this code to the change handler for the item list:

copyItems.enabled = moveItems.enabled = deleteItems.enabled = true;
copyItems.alpha = moveItems.alpha = deleteItems.alpha = 1.0;

I haven't written handlers for clicking on any of the buttons yet.

Saturday, January 27, 2007

Sorting a DataGrid in Flex 2

The default sorting behavior of DataGrid needs some customization to suit the list pane of my remote file explorer. Rather than a simple alphabetical sort in every case, the behavior depends on the sort column:

  1. The last modified column should sort in chronological order.
  2. The file size column should sort by actual size (i.e. 1 KB, 500 KB and 1.5 MB should sort correctly).
  3. Text columns should sort in case-insenstive order.

The documentation for DataGridColumn mentions the optional sortCompareFunction property, which specifies a comparison function with the contract:

private function compare(a:Object, b:Object):int {
   return (a < b) ? -1 : (a == b) ? 0 : 1;
}

However, the documentation doesn't say what the arguments a and b actually are. Turns out that they are the data for the rows being compared, not the column values.

This is convenient for writing generic functions that can be applied to multiple columns. First I wrote a function to do a case-insensitive comparison for text columns:

/**
  * Compares items by the case-insensitive value of the current sort
  * column.
  **/
private function compareCaseInsensitive(a:Object, b:Object):int {

  var sortColumn:String = itemData.sort.fields[0].name;

  var valueA:String = (a as XML)[sortColumn].toString().toLowerCase();
  var valueB:String = (b as XML)[sortColumn].toString().toLowerCase();

  return (valueA < valueB) ? -1 : (valueA == valueB) ? 0 : 1;
}

The sort.fields collection indicates the current sort column. Note that the < and > operators work for strings as well as numbers in ActionScript.

For the Last Modified and Size columns of the list pane, I added a sortable value attribute to the XML data supplied by the server, i.e.:

<item>
  <title>clone</title>
  <size value="1024">1 KB</size>
  <type>Web Page</type>
  <lastModified value="200606241611">24 Jun 06 04:11 PM PDT</lastModified>
</item>

This convention allowed me to use a single comparison function for all columns that do not sort alphabetically:

/**
  * Compares items by the value attribute of the current sort column.
  **/
private function compareByValue(a:Object, b:Object):int {

  var sortColumn:String = itemData.sort.fields[0].name;

  var sizeA:int = (a as XML)[sortColumn].@value;
  var sizeB:int = (b as XML)[sortColumn].@value;

  return (sizeA < sizeB) ? -1 : (sizeA == sizeB) ? 0 : 1;
}

I applied these functions to the corresponding columns by modifying their declarations in the MXML file:

<mx:DataGridColumn dataField="title" headerText="Title" 
                   sortCompareFunction="compareCaseInsensitive"/>
<mx:DataGridColumn dataField="size" headerText="Size" 
                   sortCompareFunction="compareByValue"/>
<mx:DataGridColumn dataField="type" headerText="Type" 
                   sortCompareFunction="compareCaseInsensitive"/>
<mx:DataGridColumn dataField="lastModified" headerText="Modified" 
                   sortCompareFunction="compareByValue"/>

I tested this arrangement and it worked fine.

Sunday, January 21, 2007

Incremental Loading of a Tree in Flex 2

In my last post I started building a remote file explorer in Flex 2. Implementing the list pane was simple. The next task was to implement the tree pane on the left side of the explorer window.

The requirements for the tree pane were:

  1. The tree must load incrementally. When the explorer first loads it should only retrieve the top-level directories from the server. The explorer should retrieve additional subdirectories one parent directory at a time, when the user expands a tree node for the first time.
  2. The explorer must accept the path to a nested subdirectory and automatically expand to the tree down to that subdirectory when the explorer loads.

I couldn't find any good examples to meet these requirements. After some trial and error with XMLListCollection, XMLList and XML data providers, I came up with a solution that seems to work.

First I added an HTTPService, data provider and Tree component to my initial MXML file:

<?xml version="1.0"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml"
  creationComplete="initApp();">

  <mx:Script source="explorer.as"/>

  <mx:HTTPService id="folderService" url="/flex/folders.jsp" 
    resultFormat="e4x" showBusyCursor="true"
    result="addFolders(event)"
    fault="handleFault(event)">
  </mx:HTTPService>

  <mx:XML id="folderData"/>

  <mx:HTTPService id="itemService" url="/flex/items.jsp" 
    resultFormat="e4x" showBusyCursor="true"
    fault="handleFault(event)">
  </mx:HTTPService>

  <mx:XMLListCollection id="itemData" 
      source="{itemService.lastResult.item}"/>

  <mx:HDividedBox width="100%" height="100%">

    <mx:Tree id="folderTree" 
      width="40%" height="100%" 
      labelField="@label"
      showRoot="false" 
      dataProvider="{folderData}" 
      folderOpenIcon="@Embed(source='assets/folder.png')"
      folderClosedIcon="@Embed(source='assets/folder_closed.png')"
      defaultLeafIcon="@Embed(source='assets/folder_closed.png')"
      itemOpening="loadChildFolders(event)"
      change="loadList(event)"/>

    <mx:DataGrid id="itemGrid" width="60%" height="100%" 
      dataProvider="{itemData}">
      <mx:columns> 
 <mx:DataGridColumn dataField="title" headerText="Title"/>
 <mx:DataGridColumn dataField="size" headerText="Size"/>
 <mx:DataGridColumn dataField="type" headerText="Type"/>
 <mx:DataGridColumn dataField="lastModified" headerText="Modified"/>
      </mx:columns>
    </mx:DataGrid>

  </mx:HDividedBox>

</mx:Application>

Note a couple significant differences between the data provider for the Tree and the DataGrid components:

  1. The data provider for the list pane is automatically bound to the corresponding HTTPService. Each time the itemService is called, the DataGrid clears and displays the last data received from the server. Since the Tree needs to load its data incrementally, it seemed that Flex requires manual handling of the folderService response using a result event handler.
  2. Because the directory tree represents a single hierarchy, the data provider can be a simple XML object rather than a XMLList or XMLListCollection. I was more or less able to get the tree to work using the latter two types of data providers as well, but the code was messier. I also could not convince the root node to hide itself that way, in spite of the showRoot attribute being set to false.

The event handlers for the tree pane required much more ActionScript than the list pane (comments inline):

import mx.controls.Alert;
import mx.events.TreeEvent;
import mx.rpc.events.FaultEvent;
import mx.rpc.events.ResultEvent;

/**
 * Handles the creationComplete event for the application.
 *
 * Expects a comma-separated list of folder IDs representing the
 * initial expansion path for the explorer.
 **/
private function initApp():void {
  
  var expandIDs:Array = Application.application.parameters.folderID.split(",");

  var lastIndex:Number = expandIDs.length - 1;

  var parameters:Object = {folderID:expandIDs[lastIndex]};
  itemService.send(parameters); 

  parameters.folderID = expandIDs.shift();
  var token:Object = folderService.send(parameters);
  token.expandIDs = expandIDs;
}

private function handleFault(e:FaultEvent):void {
  Alert.show("Failed to contact the server.");
}

/**
 * Handles the change event (selection of a directory) for the
 * explorer tree.
 *
 * Expands the selected directory to show subdirectories.  Initiates
 * retrieval a list of subdirectories from the server if user is
 * selecting this directory for the first time.
 *
 * Also retrieves a list of directory files for display in the list
 * pane.
 **/
private function loadList(e:Event):void {

  var node:XML = Tree(e.target).selectedItem as XML;

  if (loadChildFolderNode(node, e.type) && ! folderTree.isItemOpen(node)) {
    var open:Boolean = true;
    var animate:Boolean = true;
    folderTree.expandItem(folderTree.selectedItem, open, animate);
  }

  var parameters:Object = {folderID:node.@id};
  
  itemService.send(parameters);
}

/**
 * Handles the itemOpening event (clicking the expand arrow next to a
 * directory) for the explorer tree.
 *
 * Initiates retrieval of a list of subdirectories from the server and
 * cancels immediate opening if user is selecting this directory for
 * the first time.  The explorer then opens the directory after
 * receiving the server response and adding the subdirectories to the
 * tree.
 **/
private function loadChildFolders(e:TreeEvent):void {

  var node:XML = e.item as XML;

  if (! loadChildFolderNode(node, e.type)) {
    e.stopPropagation();
    e.preventDefault()
  }
}

/**
 * Checks if a tree node is loaded and initiates retrieval if not.
 **/
private function loadChildFolderNode(node:XML, eventType:String):Boolean {

  if (node.@isBranch && node.children().length() == 0) {

    var parameters:Object = {folderID:node.@id};
    var token:Object = folderService.send(parameters);
    token.parent = node;
    token.eventType = eventType;
    return false;

  } else {

    return true;
  }
}

/**
 * Handles the result event for folderService by inserting
 * subdirectories into the tree under the parent node and then
 * expanding the tree to display the newly inserted nodes.
 *
 * Initiates retrieval of the next level in the starting expansion
 * path if appropriate.
 **/
private function addFolders(e:ResultEvent):void {

  var node:XML = e.result as XML;
  var parent:XML = e.token.parent;

  if (! parent) {
    // insert root node here to work around apparent problem with
    // showRoot property of the Tree component
    parent = folderData = ;
  }

  // add nodes from server response
  for each (var child:XML in node.children()) {
    parent.appendChild(child);    
  }

  folderTree.expandItem(parent, true, true);
  // only update tree selection for change events
  if (folderTree.selectedItem != parent && 
      e.token.eventType == "change") {
    folderTree.selectedItem = parent;
  }

  // expandIDs tracks the expansion path
  var expandIDs:Array = e.token.expandIDs;
  if (expandIDs != null && expandIDs.length > 0) {

    var folderID:String = expandIDs.shift();

    var list:XMLList = parent.folder.(@id == folderID);

    if (list.length() > 0) {
      var parameters:Object = {folderID:folderID};
      var token:Object = folderService.send(parameters);
      token.expandIDs = expandIDs;
      token.parent = list[0];
      token.eventType = (expandIDs.length > 1) ? "init" : "change";
    }
  }
}

I took advantage of the Asynchronous Completion Token (ACT) pattern to keep track of the parent directory when retrieving its subdirectories from the server, as well as for completing the initial expansion of the tree over a series of retrievals. It seems a little odd to set the token properties after making the request, but that's the way it seems to be done.

On the server side, I wrote a simple JSP that accepts a directory ID and returns a list of subdirectories in the form:

<folder id="parent ID">
  <folder id="first child ID" label="first child label" isBranch="true or false"/>
  <folder id="second child ID" label="second child label" isBranch="true or false"/>
  ...
</folder>

Maybe there's an easier way to do it, but this is still a very small amount of code and markup compared to what would be required to implement anything close to the equivalent functionality using DHTML.

Sunday, January 14, 2007

Implementing a Remote File Explorer in Flex 2: List View

The first application I want to implement in Flex is a standard split-pane file explorer, with a folder tree in the left pane and a list of folder contents in the right pane.

A basic implementation of the list pane turned out to be very simple:

<?xml version="1.0"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml"
  creationComplete="initApp(); itemsService.send();">

  <mx:Script source="explorer.as"/>

  <mx:HTTPService id="itemsService" url="/flex/items.jsp" 
    resultFormat="e4x" showBusyCursor="true"
    fault="handleFault(event)">
    <mx:request>
      <folderID>{folderID}</folderID>
    </mx:request>
  </mx:HTTPService>

  <mx:XMLListCollection id="itemsList" 
      source="{itemsService.lastResult.item}"/>

  <mx:DataGrid id="itemGrid" width="100%" height="100%" 
 dataProvider="{itemsList}">
    <mx:columns> 
      <mx:DataGridColumn dataField="title" headerText="Title"/>
      <mx:DataGridColumn dataField="size" headerText="Size"/>
      <mx:DataGridColumn dataField="type" headerText="Type"/>
      <mx:DataGridColumn dataField="lastModified" headerText="Last Modified"/>
    </mx:columns>
  </mx:DataGrid>
</mx:Application>

Working from the bottom of the file up:

4. The DataGrid is a scrollable, sortable table component. Viewers can resize and reposition columns by dragging the column headers.

3. The XMLListCollection is the data source for the table. Flex automatically updates the table when the data source changes.

2. The HTTPService defines how to retrieve data from the server as a simple XML document, in this case returned by a JSP. Flex automatically unmarshals the XML into the data source whenever it receives a successful response. The request element defines the dynamic query parameters that should be added to each request.

3. The Script references a small bit of ActionScript to handle initialization and request faults:

import mx.controls.Alert;
import mx.rpc.events.FaultEvent;

[Bindable]
public var folderID:String;

private function initApp():void {
  folderID = Application.application.parameters.folderID;
}

private function handleFault(e:FaultEvent):void {
  Alert.show("Failed to contact the server.");
}

The XML document returned by the server corresponds to the rows and columns in the table:

<items>
  <item>
    <title>myscript.as</title>
    <size>10.1K</size>
    <type>ActionScript</type>
    <lastModified>8:34 AM 01/15/07</lastModified>
  </item>
  ...
</items>

The only other thing I had to do was modify the wrapper HTML page to pass a folder ID to Flex:

<html lang="en">
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
      <title>File Explorer</title>
      <script src="AC_OETags.js" language="javascript"></script>
      <style>
 body { margin: 0px; overflow:hidden }
      </style>
  </head>

  <body scroll='no'>
    <script language="JavaScript" type="text/javascript">
      <!--
      AC_FL_RunContent(
      "src", "explorer",
      "width", "100%",
      "height", "100%",
      "align", "middle",
      "id", "flexstore",
      "quality", "high",
      "bgcolor", "#869ca7",
      "name", "explorer",
      "flashVars", "folderID=<%=folderID%>",
      "allowScriptAccess","sameDomain",
      "type", "application/x-shockwave-flash",
      "pluginspage", "http://www.adobe.com/go/getflashplayer"
 );
      // -->
    </script>
    <noscript>
      <object classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000"
 id="flexstore" width="100%" height="100%"
 codebase="http://fpdownload.macromedia.com/get/flashplayer/current/swflash.cab">
 <param name='flashVars' value='folderID=<=folderID%>'></param>
 <param name="movie" value="explorer.swf"></param>
 <param name="quality" value="high"></param>
 <param name="bgcolor" value="#869ca7"></param>
 <param name="allowScriptAccess" value="sameDomain"></param>
 <embed src="explorer.swf" quality="high" bgcolor="#869ca7"
   width="100%" height="100%" name="flexstore" align="middle"
   flashVars="folderID=<%=folderID%>"
   play="true"
   loop="false"
   quality="high"
   allowScriptAccess="sameDomain"
   type="application/x-shockwave-flash"
   pluginspage="http://www.adobe.com/go/getflashplayer">
 </embed>
      </object>
    </noscript>
  </body>
</html>

I compiled the MXML file and was able to view the file list in the browser. Very nice!

Friday, January 12, 2007

Getting Started with Flex 2

After reading about Flex 2, I wanted to give it a try for myself. Adobe provides a free command-line compiler in the Flex SDK, downloadable with registration from their surprisingly clunky web site.

The download is just a zip file that you can extract and immediately begin using. Within a few minutes I was able to compile and test the sample applications, and from there it wasn't too hard to figure out the minimum setup I needed to create my own first application:

  1. Create an empty directory under the document root of your web server.
  2. Copy samples/explorer/AC_OETags.js to the new directory.
  3. Copy samples/explorer/explorer.html to the new directory. Rename it first.html
  4. Edit first.html and replace all instances of explorer with first
  5. Copy samples/explorer/build.sh to the new directory.
  6. Edit build.sh and change to the relative path to mxmlc so it points to wherever you extracted the SDK.

Once that was done, I created a new main layout file named first.mxml to declare my application:

<?xml version="1.0"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml">
</mx:Application>

I used mxmlc to compile this into first.swf. I was then able to view first.html in a browser and successfully access my first (empty) Flex application.

Building user interfaces with Flex 2

After several years in the trenches, I am very sick of building rich browser-based user interfaces using DHTML. Even relying on fancy libraries like scriptaculous or the Yahoo User Interface Library, the interface never acts quite as nice as a desktop application. Loading can be slow. Uneditable labels and components are selectable. Objects can stutter and flicker or get stuck on unexpected boundaries when dragged. Different combinations of browsers and operating systems yield different results and bugs.

The whole programming model is also tedious. Not so much because of JavaScript, but because HTML is awkward for declaring layouts and wiring together components and data sources.

Adobe Flex 2 seems to offer a very compelling alternative to DHTML for rich browser-based application interfaces. With Flex you build interfaces in three parts:

  1. XML files that declare layouts, components and other static objects. Flex includes a full set of standard interface components such as trees, lists, menus and form controls. You can also create custom components.
  2. Script files (ActionScript) for implementing event handlers and other dynamic behavior.
  3. Server-side data sources and callbacks to handle data flow to and from the client.

Besides being compelling from a programming perspective, the standard components look and feel beautiful and responsive on all browsers and platforms. This demo is my favorite since it shows off each component along with the code required to use it.

Sunday, December 17, 2006

Solving 403 Forbidden Error in Apache on Fedora Core 6

I am trying out Fedora Core 6 in a VMWare image downloaded from thoughtpolice. I started apache using apachectl, created a test file in /var/www/html/test.html, tried to access it from a browser, and received a 403 Forbidden error.

This incantation solved the problem:

# chcon -R -t httpd_sys_content_t /var/www/html/

Sunday, November 26, 2006

Naming Exceptions

When defining a WebMethod for JAX-WS 2.0, it appears that the best naming convention is to leave off the word Exception when defining method-specific exceptions. For example:
  @WebMethod(operationName = "GetMember", action = "GetMember")
  public void getMember(@WebParam(name = "MemberID")
                        Long memberID,
                        @WebParam(name = "Member",
                                  mode = Mode.OUT)
                        Holder member) 
    throws InvalidMemberID {

    if (! isValid(memberID)) {
      throw new InvalidMemberID(memberID);
    }

    ...
  }

I originally named the exception InvalidMemberIDException, but this results in a SOAP fault with the same name in the WSDL, rather than just InvalidMemberID. The client tool wsimport will then use this WSDL to generate an exception named InvalidMemberIDException_Exception.

Monday, October 23, 2006

Implementing Web Service Security: Accessing a Secure Service

In my last post I finished a handler to verify the signature in a SOAP message following a simple web service security scheme.

Next I was ready to use JAX-WS 2.0 to create a client to call my secure service. The wsimport tool does the work of generating the client code, but I needed to figure out how to insert my custom security element into the message header before it gets sent to the server.

The process of preparing to send a message from the client should look something like this:

  1. Create the message body, specifying the operation and relevant data or parameters.
  2. Add a SOAP Header element to the message if it does not already have one.
  3. Add a Security element to the header.
  4. Add the Timestamp and AccessID elements to the Security element.
  5. Create the Signature element based on the access ID, timestamp and canonicalized text of the message body and add it to the Security element.

The client code that invokes a specific web service operation should only have to worry about the first step of this process. The remaining steps are the same for all operations and should be handled transparently from the viewpoint of the caller.

The message processing pipeline for JAX-WS is similar whether you are sending or receiving messages. A message always passes through a handler chain before and after the message is actually processed. In this case, I needed to create a handler to insert the security element into the message header as described above.

My signature creation handler extended the same base handler I used on the server. In this handler the action happens in the handleOutbound method:

package member.client;

import java.util.Iterator;
import java.util.Map;
import java.util.Set;

import java.util.logging.Logger;

import org.w3c.dom.*;

import javax.xml.namespace.QName;
import javax.xml.ws.handler.MessageContext;
import javax.xml.ws.handler.soap.SOAPHandler;

import javax.xml.soap.SOAPBody;
import javax.xml.soap.SOAPElement;
import javax.xml.soap.SOAPException;
import javax.xml.soap.SOAPMessage;
import javax.xml.soap.SOAPPart;
import javax.xml.soap.SOAPEnvelope;
import javax.xml.soap.SOAPHeader;
import javax.xml.soap.SOAPHeaderElement;

import javax.xml.ws.handler.soap.SOAPMessageContext;

import member.common.Canonicalizer;
import member.common.Signature;
import member.common.BaseHandler;
import member.common.Timestamp;

/*
 * Adds a signature to outbound messages.
 */
public class SignatureCreationHandler extends BaseHandler {

  /**
   * TO DO: replace these with configuration properties
   **/
  private static final String ACCESS_ID = "karl";
  private static final String SECRET_KEY = "secret";

  private static final Logger log = Logger.getLogger("member");

  public boolean handleOutbound(SOAPMessageContext smc) throws SOAPException {

    SOAPMessage message = smc.getMessage();
    SOAPPart part = message.getSOAPPart();
    SOAPEnvelope envelope = part.getEnvelope();
    
    SOAPBody body = envelope.getBody();
    if (body == null) { 
      body = envelope.addBody();
    }
    
    SOAPHeader header = envelope.getHeader();
    if (header == null) {
      header = envelope.addHeader();
    }

    addSecurityElement(header, body);

    return true;
  }

  private void addSecurityElement(SOAPHeader header, SOAPBody body) 
    throws SOAPException {

    QName qName = getQName(body, SECURITY_ELEMENT);
    if (qName == null) { return; }
    SOAPHeaderElement securityElement = header.addHeaderElement(qName);
    
    qName = new QName("", TIMESTAMP_ELEMENT, "");
    SOAPElement timestampElement = securityElement.addChildElement(qName);
    
    String timestamp = Timestamp.create();
    timestampElement.addTextNode(timestamp);
    
    qName = new QName("", ACCESS_ID_ELEMENT, "");
    SOAPElement clientIDElement = securityElement.addChildElement(qName);
    clientIDElement.addTextNode(ACCESS_ID);

    qName = new QName("", SIGNATURE_ELEMENT, "");
    SOAPElement signatureElement = securityElement.addChildElement(qName);

    String bodyText = Canonicalizer.toText(body);
    String signedText = timestamp + ":" + ACCESS_ID + ":" + bodyText;

    String signature = Signature.create(SECRET_KEY, signedText);

    signatureElement.addTextNode(signature);
  }
}

Once the handler was written, the next challenge was to incorporate it into the service endpoint interface (SEI) that the wsimport tool generates based on a WSDL file. The endpoint interface is what a client uses to access the service.

To customize the endpoint interface, I created a binding file declaring a handler-chains element and passed it to the wsimport tool. The tool then added the specified nodes to the WSDL before generating the SEI:

<bindings 
    xmlns:xsd="http://www.w3.org/2001/XMLSchema"
    xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"
    wsdlLocation="http://localhost:8082/member?wsdl"
    xmlns="http://java.sun.com/xml/ns/jaxws">    
  <bindings node="wsdl:definitions">
    <package name="member.client"/>
  </bindings>
  <bindings node="wsdl:definitions" 
    xmlns:javaee="http://java.sun.com/xml/ns/javaee">
    <javaee:handler-chains>
      <javaee:handler-chain>
 <javaee:handler>
   <javaee:handler-class>member.client.SignatureCreationHandler</javaee:handler-class>
 </javaee:handler>
      </javaee:handler-chain>
    </javaee:handler-chains>
  </bindings>
</bindings>

After running the wsimport tool, I was able to use the generated classes to run a simple test program that successfully executed an operation on the secure service:

package member.client;

import java.math.BigInteger;

import java.util.List;
import java.rmi.RemoteException;

public class Client {

  public static void main (String[] args) throws Exception {

    MemberService service = new MemberService();
    MemberPort port = service.getMemberServicePort();

    BigInteger memberID = new BigInteger("2");

    List sn = port.memberGroups(memberID);
  }
}

As illustrated by this simple test program, the handler manages security transparently for the client.

Implementing Web Service Security: Verifying the Message Signature

In my last post I sketched out some of the implementation details for a simple web service security scheme. Next I wanted to flesh out the implementation and incorporate it into my sample web service.

The process of verifying a message from the client should look something like this:

  1. Receive the message and parse it into a DOM.
  2. Retrieve the security element from the header.
  3. Retrieve the signature submitted by the client from the security element.
  4. Recreate the signature based on the submitted access ID, timestamp and canonicalized text of the message body (along with the shared secret key).
  5. If the signatures match, allow processing to continue. Otherwise generate a fault and abort processing.

The verification process should happen transparently from the standpoint of any particular web service operation, such as the getMembers method I wrote about earlier.

JAX-WS provides handlers to support this type of transparent pre- or post-processing. A handler that has access to the entire SOAP message (including the header) must implement the javax.xml.ws.handler.soap.SOAPHandler interface.

The SOAPHandler interface honestly seems a bit odd to me. The same method handleMessage is invoked on each message both before and after processing (See the Tech Tip for some good diagrams). Except in the seemingly unlikely event that you want the same behavior in both these situations, you need to test a context property and split the implementation of handleMessage into two branches. There is probably an explanation for this, but for the moment it is lost on me.

I started by implementing a generic starting point for this and other handlers I might need to write. The base handler simplifies handling of either the inbound or outbound processing phase:

package member.common;

import java.util.Iterator;
import java.util.Set;

import javax.xml.namespace.QName;
import javax.xml.ws.handler.MessageContext;
import javax.xml.ws.handler.soap.SOAPHandler;

import javax.xml.soap.*;

import javax.xml.ws.handler.soap.SOAPMessageContext;

public abstract class BaseHandler 
  implements SOAPHandler {

  protected static final String SECURITY_ELEMENT = "Security";
  protected static final String ACCESS_ID_ELEMENT = "AccessID";
  protected static final String TIMESTAMP_ELEMENT = "Timestamp";
  protected static final String SIGNATURE_ELEMENT = "Signature";

  private static SOAPFactory factory;

  public Set getHeaders() {
    return null;
  }

  public boolean handleMessage(SOAPMessageContext smc) {

    try {
      return isOutbound(smc) ? handleOutbound(smc) : handleInbound(smc);
    } catch (SOAPException e) {
      throw new RuntimeException(e);
    }
  }

  /**
   * Handles pre-processing of messages before being sent by a client
   * or processed by a server.
   **/
  protected boolean handleInbound(SOAPMessageContext smc) 
    throws SOAPException {
    return true;
  }

  /**
   * Handles post-processing of messages after being received by a client
   * or processed by a server.
   **/
  protected boolean handleOutbound(SOAPMessageContext smc) 
    throws SOAPException {
    return true;
  }

  public boolean handleFault(SOAPMessageContext smc) {
    return true;
  }
    
  public void close(MessageContext messageContext) {
  }
    
  /**
   * Tests whether the message is being sent or received.
   **/
  private boolean isOutbound(SOAPMessageContext smc) {

    Boolean outboundProperty = (Boolean)
      smc.get(MessageContext.MESSAGE_OUTBOUND_PROPERTY);
        
    return outboundProperty.booleanValue();
  }

  /**
   * Returns a factory required for generating SOAP faults.
   **/
  protected SOAPFactory getFactory() throws SOAPException {

    if (factory == null) {
      factory = SOAPFactory.newInstance();
    }

    return factory;
  }

  /**
   * Returns a qualified name based on the namespace declared by the
   * root element in the SOAP body.
   **/
  protected QName getQName(SOAPBody body, String localName) {

    Iterator i = body.getChildElements();
    if  (! i.hasNext()) { return null; }

    SOAPElement methodElement = (SOAPElement) i.next();
    String namespaceURI = methodElement.getNamespaceURI();
    String prefix = methodElement.getPrefix();

    return new QName(namespaceURI, localName, prefix);    
  }
}

Once I had the base handler in hand, I could write a signature verification handler focused exclusively on pre-processing of each incoming message. The handleInbound method does the actual work:

package member.server;

import java.util.Date;
import java.util.Iterator;
import java.util.Map;
import java.util.Set;

import java.util.logging.Logger;

import org.w3c.dom.*;

import javax.xml.namespace.QName;
import javax.xml.ws.handler.MessageContext;
import javax.xml.ws.handler.soap.SOAPHandler;

import javax.xml.soap.SOAPBody;
import javax.xml.soap.SOAPElement;
import javax.xml.soap.SOAPException;
import javax.xml.soap.SOAPFault;
import javax.xml.soap.SOAPMessage;
import javax.xml.soap.SOAPPart;
import javax.xml.soap.SOAPEnvelope;
import javax.xml.soap.SOAPHeader;
import javax.xml.soap.SOAPHeaderElement;

import javax.xml.ws.soap.SOAPFaultException;

import javax.xml.ws.handler.soap.SOAPMessageContext;

import member.common.Canonicalizer;
import member.common.Signature;
import member.common.BaseHandler;
import member.common.Timestamp;

/*
 * Verifies the signature on inbound messages.
 */
public class SignatureVerificationHandler extends BaseHandler {

  private static final Logger log = Logger.getLogger("member");

  private static final long MAX_TIME_RANGE_MILLIS = 1000 * 60 * 5;

  public boolean handleInbound(SOAPMessageContext smc) throws SOAPException {

    SOAPMessage message = smc.getMessage();
    SOAPPart part = message.getSOAPPart();

    SOAPEnvelope envelope = part.getEnvelope();
    
    SOAPBody body = envelope.getBody();
    if (body == null) { 
      throw new RuntimeException("Missing required message body.");
    }
    
    SOAPElement securityElement = getSecurityElement(envelope);
    String accessID = getChildText(securityElement, ACCESS_ID_ELEMENT);
    String timestamp = getChildText(securityElement, TIMESTAMP_ELEMENT);
    Date submittedDate = Timestamp.parse(timestamp);
    
    long timeRangeMillis = 
      Math.abs(System.currentTimeMillis() - submittedDate.getTime());
    if (timeRangeMillis > MAX_TIME_RANGE_MILLIS) {
      SOAPFault fault = 
        getFactory().createFault("Timestamp outside allowed range",
     getQName(body, "InvalidTimestamp"));
      throw new SOAPFaultException(fault);
    }
    
    String submittedSignature = 
      getChildText(securityElement, SIGNATURE_ELEMENT);
    
    String bodyText = Canonicalizer.toText(body);
    String signedText = timestamp + ":" + accessID + ":" + bodyText;
    String secretKey = getSecretKey(accessID);
    String verifiedSignature = Signature.create(secretKey, signedText);
    
    if (! submittedSignature.equals(verifiedSignature)) {
      SOAPFault fault = 
        getFactory().createFault("Failed to verify the message signature",
     getQName(body, "InvalidSignature"));
      throw new SOAPFaultException(fault);
    }

    return true;
  }

  private SOAPElement getSecurityElement(SOAPEnvelope envelope) 
    throws SOAPException {

    SOAPHeader header = envelope.getHeader();

    if (header == null) {
      throw new RuntimeException("Missing required message header");
    }

    QName qName = getQName(envelope.getBody(), SECURITY_ELEMENT);
    if (qName == null) {
      throw new RuntimeException("Missing required method element.");
    }

    Iterator i = header.getChildElements(qName);
    if (! i.hasNext()) {
      throw new RuntimeException("Missing required security element.");
    }

    SOAPElement securityElement = (SOAPElement) i.next();

    return securityElement;
  }

  private String getChildText(SOAPElement element, String name) {
    
    Iterator i = element.getChildElements(new QName("", name, ""));
    if (! i.hasNext()) {
      return "";
    }
    
    SOAPElement child = (SOAPElement) i.next();

    Node textNode = child.getFirstChild();
    if (textNode == null) { 
      return ""; 
    }

    if (textNode.getNodeType() != Node.TEXT_NODE) {
      return "";
    }

    return ((CharacterData) textNode).getData();
  }

  /**
    * Placeholder method.
    **/
  private String getSecretKey(String accessID) {
    return "secret";
  }
}

The handler throws a fault if the timestamp or the signature is invalid. A real implementation of the getSecretKey method would also throw a fault if the access ID was invalid.

To add the handler to the message processing chain, I created a handlers.xml file in the member.server package directory:

<?xml version="1.0" encoding="UTF-8"?>

<handler-chains xmlns="http://java.sun.com/xml/ns/javaee">
  <handler-chain>
    <handler>
      <handler-class>member.server.SignatureVerificationHandler</handler-class>
    </handler>
  </handler-chain>
</handler-chains>

This apparently gets picked up by the apt tool and applied to all the web services in that package.

Before I could actually test the security scheme, I had to create a a client that would supply the correct security element in the message header. Keep reading for the shocking finale.

Sunday, October 22, 2006

Implementing Web Service Security

In my last post I looked into adding a security scheme to the example web service I implemented with JAX-WS 2.0.

I shied away from WS-Security because I wanted my service to be easy to access even with a minimal toolkit, without compromising authentication, confidentiality or data integrity. I opted to rely on SSL as the transport mechanism, and to define my own concise notation for signing each message with a shared secret key:

<S:Envelope xmlns:S="http://www.w3.org/2001/12/soap-envelope"
                 xmlns:ns1="http://xocoatl.blogspot.com/schema/1/">
  <S:Header>
    <ns1:Security>
      <AccessID>karl</AccessID>
      <Timestamp>2005-01-31T23:59:59.183Z</Timestamp>
      <Signature>DJbchm5gK...</Signature>
    </ns1:Security>
  </S:Header>
  <S:Body Id="SoapBody">
    <ns1:MemberGroups>
      <MemberID>2</MemberID>
    </ns1:MemberGroups>
  </S:Body>
</S:Envelope>

The elements of the scheme are:

AccessID A unique identifier assigned to each user who is authorized to access the web service.
Timestamp The current time to millisecond precision. The timestamp submitted by the user must be expressed in the UTC time zone, and must be within 5 minutes of the server clock for the message to be authenticated successfully.
Signature A message authentication code (MAC) generated using the HMAC_SHA1 algorithm, based on a shared secret key and a text string.

The text string used to generate the MAC has the following form:

Timestamp:AccessID:CanonicalizedMessage

The canonicalized message is the content of the Body element from the SOAP message. All spaces between elements must be stripped, and the top-level element must only have a namespace prefix. For example:

2006-09-28T07:05:07.484Z:karl:<ns1:MemberGroups><MemberID>2</MemberID></ns1:MemberGroups>

To simplify the creation of the security element for outbound messages from the client, as well as the authentication of inbound messages to the server, I wrote three small utility classes described below.

The simplest utility to handle the timestamp element of the security header. The timestamp string must conform to the standard Internet time/date format, and must furthermore be in the UTC time zone:

package member.common;

import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.text.ParseException;

import java.util.Date;
import java.util.TimeZone;

/**
 * Creates and parses timestamps in IETF RFC 3339 format.
 **/
public class Timestamp {

  private static DateFormat getFormatter() {

    DateFormat f = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'");
    f.setTimeZone(TimeZone.getTimeZone("UTC"));

    return f;
  }

  public static String create() {
    return getFormatter().format(new Date());
  }

  public static Date parse(String timestamp) {
    try {
      return getFormatter().parse(timestamp);
    } catch (ParseException e) {
      throw new RuntimeException("Invalid timestamp " + timestamp);
    }
  }
}

Generating a canonicalized version of the message is simple as long our schema consists of simple elements without attributes:

package member.common;

import java.util.Iterator;

import org.w3c.dom.CharacterData;
import org.w3c.dom.Node;
import javax.xml.soap.SOAPElement;

/**
 * Transforms the contents of a SOAP element into a standardized text
 * string.
 **/
public class Canonicalizer {

  public static String toText(SOAPElement element) {
    
    StringBuilder buffer = new StringBuilder();
    appendElement(buffer, element.getChildElements());

    return buffer.toString();
  }

  private static void appendElement(StringBuilder buffer, Iterator i) {
    
    while (i.hasNext()) {
      
      Node node = (Node) i.next();
      if (node.getNodeType() == Node.TEXT_NODE) {

        buffer.append(((CharacterData) node).getData());

      } else if (node instanceof SOAPElement) {

        SOAPElement element = (SOAPElement) node;
        String tag = element.getTagName();

        buffer.append("<").append(tag).append(">");
        appendElement(buffer, element.getChildElements());
        buffer.append("");
      }
    }
  }
}

Creating the signature is also a straightforward application of the HMAC_SHA1 algorithm:

package member.common;

import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;

public class Signature {

  private static final String MAC_ALG = "HmacSHA1";
  private static final String DIGEST_ALG = "SHA-1";

  /**
   * Creates a messsage authentication code for a text string.
   *
   * @param key A shared key
   * @param message A text message
   * @return A hex string representing the message authentication code
   **/
  public static String create(String key, String message) {

    try {

      byte[] keyBytes = key.getBytes("UTF-8");
      byte[] messageBytes = message.getBytes("UTF-8");

      Mac mac = Mac.getInstance(MAC_ALG);
      mac.init(new SecretKeySpec(keyBytes, MAC_ALG));

      byte[] signature = mac.doFinal(messageBytes);

      return toHex(signature);

    } catch (Exception e) {
      throw new RuntimeException(e);
    }
  }

  /**
   * Converts a binary messsage authentication code into a hex string.
   **/
  public static String toHex(byte[] bytes) {
    
    StringBuffer buffer = new StringBuffer(bytes.length * 2);
    for (int i = 0; i < bytes.length; i++) {

      int b = bytes[i];
      int hi = (b & 0xF0) >>> 4;
      buffer.append(Character.forDigit(hi, 16));
      int lo = b & 0x0F;
      buffer.append(Character.forDigit(lo, 16));
    }

    return buffer.toString();
  }
}

My next post will focus on using these utility classes to add a security header to outbound messages from a client.

Sunday, September 24, 2006

Web Service Security

In my last two posts I used the JAX-WS 2.0 API to implement a simple web service that I can access with either a SOAP or REST request.

The next thing I wanted to do was to add security to the service:

  1. Authentication. The sender and recipient should both be able to verify each other's identiy.
  2. Confidentiality. The data exchanged between the sender and recipient should be kept confidential from anyone watching network traffic.
  3. Data integrity. The sender and recipient should both have a way to verify that no one has tampered with the data they exchange.

I wanted to meet these security requirements without compromising interoperability, including access via REST. Consumers should be still able to access the service with a minimal toolkit consisting of an HTTP client, an XML parser and commonly used cryptographic functions.

My initial thought was to use the WS-Security standard as implemented by the XWSS project. This project builds in turn on the XML Digital Signature and XML Encryption APIs.

The focus of WS-Security seems to be on message-based security. The goal is to provide a standard for signing and encrypting all or part of a SOAP message regardless of the underlying transport mechanism (HTTP, SMTP, JMS, etc.). It avoids any reliance on transport-based security, notably SSL.

The complexity of the WS-Security standard seemed to pose an obstacle to ensuring universal access to my service. I looked at the public web services offered by a number of major companies and found that none of them used WS-Security. Instead they all used a mix of more established protocols and algorithms, along with their own simple conventions. Most of their approaches could be expressed using the WS-Security schema, but I have not seen a major public service that uses it.

Common security schemes for public web services seem to take one of two forms:

  1. Signature Verification. Consumers generate a unique signature for each request based on the submitted data, a timestamp (to prevent a replay attack) and a secret key (typically using the HMAC_SHA1 algorithm). The service re-generates the signature for each request and verifies a match.
  2. Session Authentication. A consumer always starts by submitting a user identifier a key or password to an authentication service. The service responds with a short-lived session id or token for authenticating subsequent requests.

Session authentication not guarantee data integrity by itself, but may be combined with signature verification to achieve that goal. Such hybrid schemes generate signatures with the temporary session token rather than with a permanent key.

Both schemes rely on SSL to encrypt requests and responses and therefore ensure confidentiality.

One drawback to the simple signature verification scheme using HMAC seems to be that it requires the server to permanently store a retrievable secret key for each consumer. That could be avoided by generating an RSA public and private key pair for each consumer and using that to sign messages instead. With RSA the server would only have to store the public key for each consumer. Despite this advantage the major services seem to use HMAC, perhaps because RSA implementations are not as widely available.

Hybrid schemes avoid the problem of storing a secret key because login passwords can be stored in hashed form. However, I am not sure how inconvenient it would be to require the initial login request as part of each service invocation.

After researching this I decided to start by implementing signature verification. Using WS-Security, requests to my example web service would look something like this:

<S:Envelope xmlns:S="http://www.w3.org/2001/12/soap-envelope"
                 xmlns:ns1="http://xocoatl.blogspot.com/schema/1/">
  <S:Header>
    <wsse:Security xmlns:wsse="http://schemas.xmlsoap.org/ws/2002/04/secext">
      <wsse:UsernameToken Id="AccessKey">
        <wsse:Username>karl</wsse:Username>
      </wsse:UsernameToken>
      <ds:Signature>
        <ds:SignedInfo>
          <ds:CanonicalizationMethod 
            Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/>
          <ds:SignatureMethod 
            Algorithm="http://www.w3.org/2000/09/xmldsig#hmac-sha1"/>
          <ds:Reference URI="#SoapBody">
            <ds:DigestMethod 
              Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/>
            <ds:DigestValue>LyLsF0Pi4wPU...</ds:DigestValue>
          </ds:Reference>
        </ds:SignedInfo>
        <ds:SignatureValue>DJbchm5gK...</ds:SignatureValue>
      </ds:Signature>
    </wsse:Security>
  </S:Header>
  <S:Body Id="SoapBody">
    <ns1:MemberGroups>
      <MemberID>2</MemberID>
    </ns1:MemberGroups>
  </S:Body>
</S:Envelope>

I would have to fit the timestamp in there somewhere as well. If I were just specifying my own notation, the request would look like this:

<S:Envelope xmlns:S="http://www.w3.org/2001/12/soap-envelope"
                 xmlns:ns1="http://xocoatl.blogspot.com/schema/1/">
  <S:Header>
    <ns1:Security>
      <AccessID>karl</AccessID>
      <Timestamp>2005-01-31T23:59:59.183Z</Timestamp>
      <Signature>DJbchm5gK...</Signature>
    </ns1:Security>
  </S:Header>
  <S:Body Id="SoapBody">
    <ns1:MemberGroups>
      <MemberID>2</MemberID>
    </ns1:MemberGroups>
  </S:Body>
</S:Envelope>

The only significant piece of information missing from my simpler notation is a "canonicalization method." Standardization of XML attributes and elements does not seem necessary as long as the service mandates a few simple conventions:

  1. All requests use UTF-8 character encoding.
  2. Request elements such as MemberGroups never have attributes.
  3. Request elements do not use namespace prefixes.
  4. No white space between elements.

The extra verbosity of WS-Standard seems warranted if it makes it easier for most consumers to use the service. I am not sure this is the case.