Friday, October 23, 2009

Extender Pattern vs. Lifecycle Interceptor

With this post I'd like to give a little more background on the well known OSGi Extender Pattern, when this pattern is applicable and what might be an alternative when it is not applicable. There will be coverage of
  • Basics of extending an OSGi Bundle
  • Advantages and drawbacks of Extenders
  • Notion of "self ordering" LifecycleInterceptor
Extending an OSGi Bundle

The basic functional sequence of an OSGi Extender goes like this
  1. Extender registers itself as BundleListener
  2. Bundle gets installed/started
  3. Framework fires a BundleEvent
  4. Extender picks up the BundleEvent (e.g. STARTING)
  5. Extender reads metadata from the Bundle and does its work
There is no extender specific API. It is a pattern rather than a piece of functionality provided by the Framework. Typical examples of extenders are the Blueprint or Web Application Extender.

The following illustration shows how a WAR Extender might process an OSGi Web Application to register servlets that are declared in WEB-INF/web.xml with the OSGi HttpService.




Client code that installs, starts and uses the registered endpoint could look like this

// Install and start the Web Application bundle
Bundle bundle = context.installBundle("mywebapp.war");
bundle.start();

// Access the Web Application
String response = getHttpResponse("http://localhost:8090/mywebapp/foo");
assertEquals("ok", response);


This seemingly trivial code snippet has a number of issues that are probably worth looking into in more detail
  • The WAR might have missing or invalid web metadata (i.e. an invalid WEB-INF/web.xml descriptor)
  • The WAR Extender might not be present in the system
  • There might be multiple WAR Extenders present in the system
  • Code assumes that the endpoint is available on return of bundle.start()
It should be noted that the Extender, which is essentially a synchronous or asynchronous BundleListener has no way to veto the install or start transition of the bundle. Even if the Extender cannot process the metadata or fails for some other reason, the bundle will start just the same. Client code would see an ACTIVE bundle, which in fact might not be active at all in the colloquial sense of the word.

Most Blueprint or WebApp bundles are not useful if their Blueprint/Web metadata is not processed. Even if they are processed but in the "wrong" order a user might see unexpected results (i.e. the webapp processes the first request before the underlying Blueprint app is wired together).

As a consequence I would say that the extender pattern is useful in some cases but not all. IMHO it is mainly useful if a bundle can optionally be extended in the true sense of the word.

There is an RFP in the works that addresses many of the above issues. Adrian Colyer from SpringSource has initiated the work around RFP-118 Multiple Extenders in Nov-2008. The document is currently in the non-public review phase of the OSGi Alliance Enterprise Expert Group. (EEG)

Intercepting the Bundle Lifecycle

If the use case requires the notion of "interceptor" the extender pattern is less useful. The use case might be such that you would want to intercept the bundle lifecycle at various phases to do mandatory metadata processing.

An interceptor could be used for annotation processing, byte code weaving, and other non-optional/optional metadata processing steps. Traditionally interceptors have a relative order, can communicate with each other, veto progress, etc.

Lets look at how multiple interceptors can be used to create Web metadata and publish endpoints on the HttpService based on that metadata.



Here is how it works
  1. The Wep Application processor registers two LifecycleInterceptors with the LifecycleInterceptorService
  2. The Parser interceptor declares no required input and WebApp metadata as produced output
  3. The Publisher interceptor declares WebApp metadata as required input
  4. The LifecycleInterceptorService reorders all registered interceptors according to their input/output requirements and relative order
  5. The WAR Bundle gets installed and started
  6. The Framework calls the LifecycleInterceptorService prior to the actual state change
  7. The LifecycleInterceptorService calls each interceptor in the chain
  8. The Parser interceptor processes WEB-INF/web.xml in the invoke(int state, InvocationContext context) method and attaches WebApp metadata to the InvocationContext
  9. The Publisher interceptor is only called when the InvocationContext has WebApp metadata attached. If so, it publishes the endpoint from the WebApp metadata
  10. If no interceptor throws an Exception the Framework changes the Bundle state and fires the BundleEvent.
Client code is identical to above.

// Install and start the Web Application bundle
Bundle bundle = context.installBundle("mywebapp.war");
bundle.start();

// Access the Web Application
String response = getHttpResponse("http://localhost:8090/mywebapp/foo");
assertEquals("ok", response);


The behaviour of that code however, is not only different but also provides a more natural user experience.
  • Bundle.start() fails if WEB-INF/web.xml is invalid
  • An interceptor could fail if web.xml is not present
  • The Publisher interceptor could fail if the HttpService is not present
  • Multiple Parser interceptors would work mutually exclusiv on the presents of attached WebApp metadata
  • The endpoint is guaranteed to be available when Bundle.start() returns
The general idea is that each interceptor takes care of a particular aspect of processing during state changes. In the example above WebApp metadata might get provided by an interceptor that scans annotations or by another one that generates the metadata in memory. The Publisher interceptor would not know nor care who attached the WebApp metadata object, its task is to consume the WebApp metadata and publish endpoints from it.

The current BundleContext API does not allow for arbitrary attachments as contextual information to be passed between interceptors. The InvocationContext adds that plus the notion of a VirtualFile that represents the Bundle's content. The VirtualFile is necessary to access arbitrarily deeply nested resources regardless of their actual location.

Here a sneak preview of the ParserInterceptor that will come as part of the Interceptor example in JBoss OSGi 1.0.0.Beta5

public void invoke(int state, InvocationContext context)
{
  // Do nothing if the metadata is already available  
  HttpMetadata metadata = context.getAttachment(HttpMetadata.class);
  if (metadata != null)
    return;

  // Parse and create metadta on STARTING
  if (state == Bundle.STARTING)
  {
    try
    {
      VirtualFile root = context.getRoot();
      VirtualFile propsFile = root.getChild("/metadata.properties");
      if (propsFile != null)
      {
        log.info("Create and attach HttpMetadata");
        metadata = createHttpMetadata(propsFile);
        context.addAttachment(HttpMetadata.class, metadata);
      }
    }
    catch (IOException ex)
    {
      throw new LifecycleInterceptorException(ex);
    }
  }
}


and here the associate PublisherInterceptor

public void invoke(int state, InvocationContext context)
{
  // HttpMetadata is guaratied to be available because we registered
  // this type as required input
  HttpMetadata metadata = context.getAttachment(HttpMetadata.class);

  // Register HttpMetadata on STARTING 
  if (state == Bundle.STARTING)
  {
    String servletName = metadata.getServletName();
    try
    {
      log.info("Publish HttpMetadata: " + metadata);

      // Load the endpoint servlet from the bundle
      Bundle bundle = context.getBundle();
      Class servletClass = bundle.loadClass(servletName);
      HttpServlet servlet = (HttpServlet)servletClass.newInstance();

      // Register the servlet with the HttpService
      HttpService httpService = getHttpService(context, true);
      httpService.registerServlet("/servlet", servlet, null, null);
    }
    catch (RuntimeException rte)
    {
      throw rte;
    }
    catch (Exception ex)
    {
      throw new LifecycleInterceptorException(ex);
    }
  }

  // Unregister the endpoint on STOPPING 
  else if (state == Bundle.STOPPING)
  {
    log.info("Unpublish HttpMetadata: " + metadata);
    HttpService httpService = getHttpService(context, false);
    if (httpService != null)
      httpService.unregister("/servlet");
  }
}


This post and the initial implementation that is based on these concepts is a follow up of a discussion that I had with Hal Hildebrand from Oracle at one of our recent OSGi EEG meetings. Please stay tuned to find out in what shape or form these ideas develop.

In parallel to the standards efforts you will of course get this functionality as part of the next JBoss OSGi release, which should come out 01-Dec-2009.

That's it for now. In my next post I will explore the Framework launch API in more detail and show how you can bootstrap an OSGi Framework independent of the actual Framework implementation.

May this be useful.

Wednesday, October 14, 2009

JBossOSGi 1.0.0.Beta4 Released

I am happy to announce the release of JBossOSGi-1.0.0.Beta4.

You can download the binary here: jboss-osgi-installer-1.0.0.Beta4.jar

The release comes with improvements in the following areas
For details please have a look at the latest version of our User Guide.

Here are the change log details

Feature Request

  • [JBOSGI-110] - Support WAR deployments in the OSGi runtime
  • [JBOSGI-127] - Provide build and hudson support for MC facade
  • [JBOSGI-128] - Add Framework launch API

Task

Bug

  • [JBOSGI-78] - ResourceNotFoundException too verbose
  • [JBOSGI-130] - Bundle is still in state INSTALLED after bundle.start()
  • [JBOSGI-135] - Cannot handle package being imported and exported by the same bundle
  • [JBOSGI-136] - Cannot resolve dependency against unstarted bundle
  • [JBOSGI-140] - Invalid delivery of framework events
  • [JBOSGI-144] - Framework does not handle ServiceFactory provided services properly
  • [JBOSGI-151] - Cannot resolve circular dependencies
  • [JBOSGI-155] - OutOfMemoryError on repeated install/unstall bundle
  • [JBOSGI-158] - Framework itself must be a bundle
  • [JBOSGI-161] - Cannot use commons-logging
  • [JBOSGI-162] - Cannot load classes from Bundle-ClassPath
  • [JBOSGI-163] - Husky invocation before async extender processing

Enjoy

OSGi Unit Testing

With this post I'd like to give a little more background on OSGi unit testing. There will be coverage of
  • Basic problem of OSGi bundle testing
  • Running a unit test inside an OSGi Framework
  • Interaction between a test runner and the OSGi Framework
Introduction

One of the main characteristics of an OSGi environment is its high dynamicity. Services come and go at any time or get updated by newer versions. To verify that functionality provided by an OSGi Bundle does indeed behave as expected, you need good test coverage of that functionality. This of course is a general truth that is not limited to OSGi alone.

The benefit of unit testing is to isolate functionality and show that it behaves as expected. A unit test provides a strict, written contract that the piece of code must satisfy.

The basic problem in OSGi is that you cannot access the artefacts that you deploy in a bundle directly from the test case that runs external to the OSGi Framework. The test case is loaded from different classloader than the artefacts contained in the bundle.



Embedded OSGi Framework

This is probably the most common usage. A test or a wrapper around a collection of tests bootstraps the OSGi Framework installs a number of bundles and verifies their correct behaviour. Husky test cases leverage the OSGi Framework Launch API. Framework launching should not require prior knowledge of the actual framework implementation. OSGi Framework and test runner share the same VM. From the perspective of the test runner, the test can be executed like an ordinary unit test, which gives the developer the freedom of choice with respect to the runner (i.e. Eclipse, IntelliJ, Maven, Ant). The test reports results back to the runner like an ordinary unit test, which preserves the runner's test report functionality.

Remote OSGi Framework

In an enterprise scenario the OSGi Framework is more often than not a standalone container. The same test as above should be deployable in a remote framework that is already running. It is a configuration option of the test environment of whether the test gets installed into an embedded OSGi Framework or deployed to a remote framework instance. OSGi Framework and test runner do not share the same VM. From the perspective of the test runner, the test can still be executed like an ordinary unit test and it is completely transparent that the test gets actually executed in a remote environment. Conceptually, this is similar to what Apache Cactus is about.


Requirements
  1. The solution MUST support plain JUnit4 POJO test cases
  2. There SHOULD be no requirement to extend a specific test base class
  3. There MUST be no requirement on a specific test runner
  4. There SHOULD be a minimum test framework leakage into the test case
  5. The test framework MUST support embedded and remote OSGi Runtimes with no change required to the test
  6. The same test case MUST be executable from outside as well as from within the OSGi Framework
  7. There SHOULD be a pluggable communication layer from the test runner to the OSGi Framework
  8. The test framework MUST NOT depend on OSGi Framework specific features
  9. There MUST be no automated creation of test bundles required by the test framework
Architecture

JBoss OSGi Husky has client side interceptor that fields the test request to an embedded/remote OSGi Framework where the test case is then actually executed.



How does it work
  1. A Bridge intercepts a test and determines the FQN of the test case and the test method from the call stack. It then delegates the execution to the same (or another) test in and isolated test environment. An isolated test environment is one that does not have the same class loading space as the test itself.
  2. A Bridge is associated with an Invoker. Invokers may be arbitarily complex. Local 'in proccess' invokers are possible just as well as remote invokers.
  3. The Invoker sends the Request to a Connector in the isolated test environment.
  4. A Connector has associated PackageListeners that are responsible for processing test cases for their respective test packages.
  5. A PackageListeners delegates the Request to a test Runner, typicaly this would be a JUnitRunner.
  6. The Runner injects the Context into the test case and returns a Response, which the Connector returns to the Invoker.
  7. The Bridge finally translates potential Failures that may be contained in the Result, to test failures on the client side.
The JBoss OSGi jboss-osgi-husky.jar bundle registers the Connectors. The JMXConnector is always registered. The SocketConnector is registered when the appropriate configuration options are set. It then registers the HuskyExtender, which is a BundleListener that inspects every incomming bundle for the Test-Package manifest header. The Extender creates a PackageListener for every package in the 'Test-Package' manifest header and registers them with the available Connectors.

Configuration

In the target OSGi Framework, which is the one that has the jboss-osgi-husky.jar bundle installed, you set these properties.
# Husky socket connector properties
org.jboss.osgi.husky.runtime.connector.host=localhost
org.jboss.osgi.husky.runtime.connector.port=5401
Both properties must be set for the SocketConnector to be available.

On the client side, you configure the Invoker that you want to use.
# Husky invoker properties
org.jboss.osgi.husky.invoker=org.jboss.osgi.husky.internal.OSGiInvoker

This particular invoker will also look for the host and port properties and if availbale will use socket invocation.

Writing Husky Tests

In a typical Husky test you have
  • A discriminator to distinguish between client and 'in container' execution
  • An interceptor that delegates the call to an Invoker
For OSGi, the "discriminator" would be the BundleContext that gets injected by the 'in container' test Runner. The "interceptor" would be a call to one of the Bridge.run() variants.

public class SimpleHuskyTestCase
{
@ProvideContext
public BundleContext context;

...

@Test
public void testSimpleBundle() throws Exception
{
// Tell Husky to run this test method within the OSGi Runtime
if (context == null)
BridgeFactory.getBridge().run();


// Stop here if the context is not injected
assumeNotNull(context);

// Get the SimpleService reference
ServiceReference sref = context.getServiceReference("SimpleService");
assertNotNull("SimpleService Not Null", sref);

// Access the SimpleService
SimpleService service = (SimpleService)context.getService(sref);
assertEquals("hello", service.echo("hello"));
}
}

The bundle that contains the test case must have the Test-Package manifest header configured.
Bundle-SymbolicName: example-simple-husky

# Export the package that contains the test case
Export-Package: org.jboss.test.osgi.example.simple

# Tell Husky that there are test cases in this package
Test-Package: org.jboss.test.osgi.example.simple
That's it for now. JBoss OSGi 1.0.0.Beta4 is ready and waiting for the files to show up on Sourceforge. In a follow up post you can read about our new native OSGi Framework implementation that is based on the JBoss Microcontainer and how JBoss OSGi uses Pax Web to provide support for OSGi WebApps.

May this be useful