Thursday, November 11, 2010

JBoss AS7 OSGi Integration

Now that JBossAS-7.0.0.Alpha1 has come out I'd like to talk about how JBoss OSGi integrates with it and what benefits our community will gain from the OSGi integration. In this post I'll show how non-OSGi components like EJB3, Servlet, CDI Beans, ManagedBeans, MBeans, etc. can access OSGi services and vice versa.

OSGi in the EE Landscape

Generally I believe that OSGi is very much an all-or-nothing technology. If your application is an OSGi bundle (or many of them) you must run within an OSGi Framework and can only have access to services that are provided by other OSGi bundles. In todays EE appplication landscape where folks take (standard) functionality like JPA, Servlet3, CDI, EJB3, JCA, JMS, etc for granted they would need a container that offers these services not only to EE applications but also to OSGi bundles.

In Mar-2010 the OSGi Enterprise Expert Group, has come up with its first EE OSGi specification, which IMHO offers a fraction of what we currently see in modern EE applications. There is no JSP, EJB3, JCA, JMS, CDI not even Annotation processing. The EEG is working on completing the picture, but this is unlikely to become generally available in the near future. David and I are members of this group, so its partially also our call.

Of course, if your application is such that it only requires what is available in OSGi Core, Compendium and Enterprise already (and many of todays larger OSGi applications do) OSGi is a good candidate to solve general modularity and service integration issues.

The JBoss approach to OSGi

At JBoss we take a different approach. With the upcoming AS7 the basic building block is a "module" which can have dependencies on other modules. There are services with basic lifecycle that can also have dependencies on each other. Our OSGi Framework implementation builds on top of this basic infrastructure like any other JBoss subsystem. We integrate at the lowest possible level. Such it becomes possible to access an OSGi service from a non-OSGi component and vice versa.

The questions about the suitability of OSGi is at the very heart of these integration issues IMHO. If EJB3, CDI, etc. offer value-add then it must be possible to leverage that value from an OSGi application. Otherwise you have an either/or situation where you gain benefits from one technology but at the same time loose benefits from another technology that cannot be accessed any more.

You can think of this in terms of two intersecting sets of technology (i.e. EE6, OSGi). I'm interested in the superset of the two and not so much in the smaller individual sets. The aim is that JBoss users who already write great and non trivial EE applications can now "also" use OSGi to address modularity issues in these very applications.

The Theory behind JBoss OSGi

Any deployment, and in-fact any jar you see in AS7, is a Module. A Module may have dependencies and a number of attached ResourceLoaders. A ResourceLoader is typically backed by a VirtualFile. A Module can choose to export a subset of the resource paths from its attached ResourceLoaders. This is equivalent to the Export-Package notion in OSGi. The dependencies on other Modules are equivalent to Require-Bundle. A module can choose which paths to import from a given dependency. A Module can choose to re-export the paths that it imports from a dependency.

An OSGi Bundle is a jar with a set of metadata in it's Manifest. To name a few, we have
Generally speaking however, these are Requirements and Capabilities that a Bundle may have. In JBoss OSGi we translate the OSGi metadata to these general Requirements and Capabilities and feed the Standalone Resolver with it. At bundle resolution time, the Resolver tries to find a consistent set of Wires, such that every non-optional Requirement is wired to a Capability. This is a non-trivial process that leads to consistent class spaces. If successful, the Bundle is said to be RESOLVED.

At the end of the day, every OSGi Framework conceptually creates dependencies between the installed bundles and limits the set of paths at the exporting and importing side of the wires.

So we get the set of Requirements and Capabilities from somewhere, feed them to our resolver abstraction and finally use the resulting Wires to setup the ModuleSpec. The source for our Requirements and Capabilities is not limited to the OSGi manifest. Any AS7 deployment can generate resolver metadata and make itself known to the OSGi layer.

At deploy time every Bundle becomes an AS7 Module and every AS7 Module may become an OSGi Bundle when its provides it's set of resolver metadata and registers with the OSGi layer. OSGi bundles are therefore a subset of the modules that exist in the AS7 module layer.

Lets look at some code ...

Bundle accessing a Non-OSGi Module

In a test case we construct and deploy a module that provides a simple Echo service to the running AS7.

JavaArchive archive = ShrinkWrap.create(JavaArchive.class, "example-xservice-target-module");
archive.addClasses(Echo.class, EchoService.class, TargetModuleActivator.class);
String activatorPath = "META-INF/services/" + ServiceActivator.class.getName();
archive.addResource(getResourceFile("xservice/target-module/" + activatorPath), activatorPath);

The module contains a few classes and a ServiceActivator. The ServiceActivator is the equivalent of a BundleActivator. When the module gets activated we register a simple EchoService.

public static void addService(BatchBuilder batchBuilder)
{
  serviceBuilder = batchBuilder.addService(SERVICE_NAME, new EchoService());
  serviceBuilder.addAliases(ServiceName.of(Constants.JBOSGI_PREFIX, Echo.class.getName()));
  serviceBuilder.setInitialMode(Mode.ACTIVE);
  log.infof("Service added: %s", SERVICE_NAME);
}

Please note, that the service is also registered with an alias. The JBoss OSGi service registry uses the ServiceContainer that contains all services that exist in AS7. For service lookup the OSGi API uses the FQN of an interface that the requested service implements. The alias is needed for this OSGi API call to succeed

ServiceReference sref = context.getServiceReference(Echo.class.getName());
Echo service = (Echo)context.getService(sref);
service.echo("hello world");

When the module is deployed, it is not automatically registered with the OSGi layer. A bundle that imports the package of the Echo interface would not resolve.

Any module can however be registered with the OSGi layer.

ModuleIdentifier moduleId = ModuleIdentifier.create("deployment.example-xservice-target-module");
registerModuleWithBundleManager(moduleId);

Currently this must be done explicitly - there is no automatic registration. In our case the module does not provide resolver metadata explicitly, so it will be generated. We add a PackageCapability for every exported path.

ResolverPlugin resolverPlugin = getPlugin(ResolverPlugin.class);
XModuleBuilder builder = resolverPlugin.getModuleBuilder();
builder.createModule(symbolicName, version, 0);
builder.addBundleCapability(symbolicName, version);
for (String path : module.getExportedPaths())
{
  if (path.startsWith("META-INF"))
    continue;

  String packageName = path.replace('/', '.');
  builder.addPackageCapability(packageName, null, null);
}
XModule resModule = builder.getModule();

Now that the module is known to the OSGi layer. The resolver can use the module's capabilities for bundle resolution. We can now install and start a real OSGi bundle that imports the echo package and calls the Echo service in its BundleActivator.

mvn -Dtarget.container=jboss70x -Dtest=BundleAccessesModuleServiceTestCase test

Running org.jboss.test.osgi.example.xservice.BundleAccessesModuleServiceTestCase
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.009 sec

17:21:16,469 INFO  Activating deployment: example-xservice-target-module
17:21:16,732 INFO  Service added: service jboss.osgi.xservice.target
17:21:16,733 INFO  ModuleIdentifier: module:deployment.example-xservice-target-module:main
17:21:16,854 INFO  Install bundle: deployment.example-xservice-target-module:0.0.0
17:21:18,135 INFO  Install bundle: example-xservice-client-bundle:1.0.0
17:21:18,196 INFO  Echo: hello world
17:21:18,199 INFO  Bundle started: example-xservice-client-bundle:1.0.0
17:21:18,222 INFO  Bundle uninstalled: example-xservice-client-bundle:1.0.0
17:21:18,235 INFO  Bundle uninstalled: deployment.example-xservice-target-module:0.0.0
17:21:18,292 INFO  Undeployed example-xservice-target-module 

Going forward, we can now define a set of rules for every possible AS7 deployment on whether or not it gets registered with the OSGi layer and how it generates it's caps/reqs.

A possible question could be:
"Why isn't the target module a real OSGi bundle in the first place? In other words, why does it not define its capabilities in a standard way in the OSGi manifest"
The target module may have dependencies on other services that are not available in the OSGi layer. Or simply because the folks that provide the target module do not feel sufficiently familiar with OSGi. The important thing here is that with AS7 we have a migration path to and from OSGi.

Module accessing an OSGi Service

This scenario requires a little more thought. Let's first explore the seemingly obvious approach.
  • The target bundle contains the Echo interface
  • The client module has a dependency on the target bundle
Every Module has a ModuleIdentifier, which in case of an OSGi bundle is constructed according to the following pattern
jbosgi.[bundle-symbolic-name]:[bundle-version]
In case of the target bundle that is used in this test case, we would have
jbosgi.example-xservice-target-bundle:1.0.0
This is the Require-Bundle semantic, which is generally considered bad practise in OSGi. In the OSGi core spec we find the following chapter

3.12.3 Issues With Requiring Bundles

The preferred way of wiring bundles is to use the Import-Package and
Export-Package headers because they couple the importer and exporter to a
much lesser extent. Bundles can be refactored to have a different package
composition without causing other bundles to fail.

The Require-Bundle header provides a way for a bundle to bind to all the
exports of another bundle, regardless of what those exports are. Though this
can seem convenient at first, it has a number of drawbacks:

Split Packages – Classes from the same package can come from different bundles with Require bundle, such a package is called a split package. Split packages have the following drawbacks:
  • Completeness – Split packages are open ended, it is difficult to guarantee that all the intended pieces of a split package have actually been included.
  • Ordering – If the same classes are present in more than one required bundle, then the ordering of Require-Bundle is significant. A wrong ordering can cause hard to trace errors, similar to the traditional class path model of Java.
  • Performance – A class must be searched in all providers when packages are split. This potentially increases the number of times that a ClassNotFoundException must be thrown which can potentially introduce a significant overhead.
  • Confusing – It is easy to find a setup where there is lots of potential for confusion. For example, the following setup is non-intuitive.

    A: Export-Package: p;uses:=q
       Import-Package: q
    B: Export-Package: q
    C: Export-Package: q
    D: Require-Bundle: B, C
       Import-Package: p
    


    In this example, bundle D merges the split package q from bundles B and
    bundle C, however, importing package p from bundle A puts a uses constraint on package p for package q. This implies that bundle D can see the valid package q from bundle B but also the invalid package q from bundle C. This wiring is allowed because in almost all cases there will be no problem. However, the consistency can be violated in the rare case when package C.q contains classes that are also in package B.q.

Mutable Exports – The feature of visibility:=reexport that the export signature of the requiring bundle can unexpectedly change depending on the export signature of the required bundle.

Shadowing – The classes in the requiring bundle that are shadowed by those in a required bundle depend on the export signature of the required bundle and the classes the required bundle contains. (By contrast, Import-Package, except with resolution:=optional, shadows whole packages regardless of the exporter.)

Good practise is to use Export-Package to declare a PackageCapability and Import-Package to declare a PackageRequirement. In this way importer does not need to care where a package comes from and the provider of the exported package is free to move the package around from one bundle to another without breaking the importer.

OSGi supports the notion of bundle revisions. This does not show up in the public API. However, every time a bundle is updated (i.e. the bytes change) the framework creates a new revision with potentially the same Bundle-SymbolicName and Bundle-Version. Internally we append the revision suffix to the ModuleIdentifier. As a consequence, binding to a particular ModuleIdentifier identifier ultimately means binding to a particular revision, which negates the purpose update.


Having said all this, we decide against  putting the Echo interface (i.e. the API)  in the target bundle. Instead we put it in a Module which can be registered with the OSGi layer as described above.

JavaArchive getAPIModuleArchive() throws Exception
{
  JavaArchive archive = ShrinkWrap.create(JavaArchive.class, "example-xservice-api-module");
  archive.addClasses(Echo.class);
  archive.addDirectory("META-INF");
  return archive;
}

// Deploy the module that contains the API
String apiDeploymentName = getRemoteRuntime().deploy(getAPIModuleArchive());

// Register the API module with the OSGi layer
ModuleIdentifier apiModuleId = ModuleIdentifier.create("deployment." + apiDeploymentName);
registerModuleWithBundleManager(apiModuleId);

The target bundle is a plain OSGi bundle that imports the package of the Echo service and registers an instance of that service with the OSGi service registry.

@Override
   public void start(final BundleContext context) throws Exception
   {
      context.registerService(Echo.class.getName(), new EchoImpl(), null);
   }

   static class EchoImpl implements Echo
   {
      @Override
      public String echo(String message)
      {
         log.infof("Echo: %s", message);
         return message;
      }
   }

The client module needs to define a dependency on the API module. In JBoss AS module dependencies can be defined in the manifest like this

Manifest-Version: 1.0
Dependencies: org.osgi.core,deployment.example-xservice-api-module

Ouch, this uses an AS7 proprietary manifest header as well as Require-Bundle semantics. For AS7 internal modules (i.e. the ones that we control) this direct and hard coded approach is key to bootstrap performance. It should however be carefully considered before putting in user deployments.

Never mind, for sake of this exercise we continue. When the client module activates it registers a Service.

public class EchoInvokerService implements Service<Void>
{
   InjectedValue<BundleContext> injectedBundleContext = new InjectedValue<BundleContext>();

   static void addService(BatchBuilder batchBuilder)
   {
      EchoInvokerService service = new EchoInvokerService();
      BatchServiceBuilder serviceBuilder = batchBuilder.addService(SERVICE_NAME, service);
      serviceBuilder.addDependency(ServiceName.parse("jboss.osgi.context"), BundleContext.class, service.injectedBundleContext);
      serviceBuilder.setInitialMode(Mode.ACTIVE);
      log.infof("Service added: %s", SERVICE_NAME);
   }

   @Override
   public void start(StartContext context) throws StartException
   {
      BundleContext systemContext = injectedBundleContext.getValue();
      ServiceReference sref = systemContext.getServiceReference(Echo.class.getName());
      Echo service = (Echo)systemContext.getService(sref);
      service.echo("hello world");
   }

   @Override
   public void stop(StopContext context)
   {
   }

   @Override
   public Void getValue() throws IllegalStateException
   {
      return null;
   }
}

This is a native AS7 service that gets the OSGi system BundleContext injected before it is started. Currently, when you download and try jboss-7.0.0.Alpha1 you will see that it bootstraps in very little time (i.e. less than 3sec). This is also because the OSGi subsystem is activated lazily. Installing a service like the one above with initial mode ACTIVE would cause all services that it depends on also to become ACTIVE. The OSGi subsystem would activate and the framework would start.

In the start method of the EchoInvokerService you see the usage of OSGi API that calls the target service.

mvn -Dtarget.container=jboss70x -Dtest=ModuleAccessesBundleServiceTestCase test

Running org.jboss.test.osgi.example.xservice.ModuleAccessesBundleServiceTestCase
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.009 sec

11:10:22,040 INFO  Activating deployment: example-xservice-api-module
11:10:22,228 INFO  Install bundle: deployment.example-xservice-api-module:0.0.0
11:10:23,140 INFO  Install bundle: example-xservice-target-bundle:1.0.0
11:10:23,251 INFO  ModuleIdentifier: module:jbosgi.example-xservice-target-bundle:1.0.0
11:10:23,252 INFO  Bundle started: example-xservice-target-bundle:1.0.0
11:10:23,413 INFO  Activating deployment: example-xservice-client-module
11:10:23,516 INFO  Service added: service jboss.osgi.xservice.invoker
11:10:23,518 INFO  ModuleIdentifier: module:deployment.example-xservice-client-module:main
11:10:23,520 INFO  Echo: hello world
11:10:23,602 INFO  Undeployed example-xservice-client-module
11:10:23,661 INFO  Bundle uninstalled: example-xservice-target-bundle:1.0.0
11:10:23,687 INFO  Bundle uninstalled: deployment.example-xservice-api-module:0.0.0
11:10:23,739 INFO  Undeployed example-xservice-api-module

Conclusion

I have shown how in principle an OSGi bundle can access an arbitrary AS7 service and vice versa. With this distribution we have reached about 80% OSGi Core TCK compliance, which we hope to complete by Q2/2011. The final version of AS7 is currently scheduled for May/2011.

I am hopeful that by the time AS7 goes final we have examples of various EE components interacting with OSGi and vice versa. The aim is that you can use the benefits of OSGi modularity and its service layer in non-trivial applications that make use of AS7 middle ware.

You may also want to check out David's post on "Using OSGi in JBoss AS7".

May this be useful
-thomas

1 comment:

  1. Hi Thomas,
    That is a gr8 insight, however if you can present a Spring Hello world sample with a JSP, I think will be better example as the the BundleContext should be used for lifecycle mgmt.

    ReplyDelete