Tuesday, October 26, 2010

JBossOSGi Hudson on Amazon Elastic Cloud

Following Amazons announcement for an AWS Free Usage Tier, I thought I would give it a try. Here my account.

Creating the Instance

If you are new to Amazon EC2 you need to create an account and sign in. Once you are in the AWS Management Console, you can create a key pair like this




I like to have control over the connection ports that are accessible on my instance, so I create a new Security Group, which I can later use with my instance.


Next, you navigate to Instances and launch an new Amazon Linux Micro instance. Basic 32-bit Amazon Linux is a good choice because it can be upgraded to small. A 64-bit instance can only be upgraded to large and above.


Select the key and security group that you created earlier, hit launch and your Linux Micro instance should be up and running.

Accessing you Linux Instance

When you created your keypair you were offered to download and save the private key to a secure location. You can now ssh into you newly created instance like this

chmod 600 .ssh/tdiesler-ec2.pem
ssh -i .ssh/tdiesler-ec2.pem ec2-user@ec2-50-16-7-151.compute-1.amazonaws.com
Last login: Mon Oct 25 13:32:07 2010 from p5b04e10a.dip.t-dialin.net

       __|  __|_  )  Amazon Linux AMI
       _|  (     /     Beta
      ___|\___|___|

See /etc/image-release-notes for latest release notes. :-)
[ec2-user@ip-10-112-99-96 ~]$ 

Installing Hudson on Tomcat

Amazon Liunx supports the Yum Package Manager, which I happen to know quite well because I run Fedora. Lets first check if Tomcat is there

$ yum list installed | grep tomcat
$ 

Ok, that's not the case, so see what's available.

$ yum list available | grep tomcat
...
tomcat6.noarch                               6.0.24-8.12.amzn1 
tomcat6-admin-webapps.noarch                 6.0.24-8.12.amzn1 
tomcat6-docs-webapp.noarch                   6.0.24-8.12.amzn1 
tomcat6-javadoc.noarch                       6.0.24-8.12.amzn1 
tomcat6-log4j.noarch                         6.0.24-8.12.amzn1 
tomcat6-webapps.noarch                       6.0.24-8.12.amzn1 

Let's install tomcat6

[ec2-user@ip-10-112-99-96 ~]$ 
$ sudo yum install tomcat6
...
Installed:
  tomcat6.noarch 0:6.0.24-8.12.amzn1
...
Complete!

Furthermore, we need Git, Ant and Maven. Git and Ant we can install with yum like above. Maven however does not seem to be available in yum. So lets get it from the Apache Maven download page.

$ wget http://apache.linux-mirror.org/maven/binaries/apache-maven-3.0-bin.zip
...
2010-10-25 14:40:23 (44.4 KB/s) - “apache-maven-3.0-bin.zip” saved [2919967/2919967]
$ unzip apache-maven-3.0-bin.zip
$ sudo mkdir /usr/java
$ sudo mv apache-maven-3.0 /usr/java
$ sudo ln -s /usr/java/apache-maven-3.0 /usr/java/apache-maven
$ mkdir bin
$ ln -s /usr/java/apache-maven/bin/mvn bin/mvn
$ mvn
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.108s
[INFO] Finished at: Mon Oct 25 14:44:26 UTC 2010
[INFO] Final Memory: 2M/7M
[INFO] ------------------------------------------------------------------------

Amazon Linux comes with openjdk, which is fine for most purposes.  At the time of this writing however, I saw issues with the maven build and Hudson Captcha. So lets install the Oracle JDK 

$ cd /usr/java
$ sudo wget http://cds.sun.com/is-bi/...jdk-6u22-linux-i586.bin
$ sudo chmod +x jdk-6u22-linux-i586.bin 
$ sudo ./jdk-6u22-linux-i586.bin 
$ sudo ln -s jdk1.6.0_22 jdk1.6
$ sudo rm jdk-6u22-linux-i586.bin

$ vi ~/.bash_profile
...
export JAVA_HOME="/usr/java/jdk1.6"
export PATH=$HOME/bin:$JAVA_HOME/bin:$PATH

$ java -version
java version "1.6.0_22"
Java(TM) SE Runtime Environment (build 1.6.0_22-b04)
Java HotSpot(TM) Client VM (build 17.1-b03, mixed mode, sharing)

 
Building JBoss OSGi

The JBoss OSGi umbrella project that holds the Hudson setup as an integral part of the code base is hosted on GitHub.

$ mkdir git; cd git
$ git clone git://github.com/jbosgi/jbosgi.git; cd jbosgi
$ git submodule init
$ git submodule update

JBoss community projects often do not contain the JBoss Nexus Configuration. Follow the instructions on Maven Getting Started - Users to setup your ~/.m2/settings.xml

Now, lets build the umbrella for the first time

$ mvn install
...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] JBossOSGi JMX ..................................... SUCCESS [1:00.822s]
[INFO] JBossOSGi JMX API ................................. SUCCESS [1:44.154s]
[INFO] JBossOSGi VFS ..................................... SUCCESS [0.008s]
[INFO] JBossOSGi VFS API ................................. SUCCESS [5.442s]
[INFO] JBossOSGi SPI ..................................... SUCCESS [24.504s]
[INFO] JBossOSGi JMX Bundle .............................. SUCCESS [15.142s]
[INFO] JBossOSGi JMX iTests .............................. SUCCESS [2:37.160s]
[INFO] JBossOSGi VFS VFS30 ............................... SUCCESS [9.390s]
[INFO] JBossOSGi Deployment .............................. SUCCESS [12.598s]
[INFO] JBossOSGi Framework ............................... SUCCESS [0.047s]
[INFO] JBossOSGi Framework Core .......................... SUCCESS [32.110s]
[INFO] JBossOSGi Framework iTest ......................... SUCCESS [2:33.963s]
[INFO] JBossOSGi ......................................... SUCCESS [0.026s]
[INFO] JBossOSGi Reactor ................................. SUCCESS [0.006s]
[INFO] JBossOSGi Testsuite ............................... SUCCESS [0.017s]
[INFO] JBossOSGi Testsuite Examples ...................... SUCCESS [3:39.319s]
[INFO] JBossOSGi Testsuite Functional .................... SUCCESS [1:40.045s]
[INFO] JBossOSGi Testsuite Performance ................... SUCCESS [19.179s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 16:48.812s
[INFO] Finished at: Tue Oct 26 12:30:18 UTC 2010
[INFO] Final Memory: 22M/68M
[INFO] ------------------------------------------------------------------------

This will take a while, so lets continue with Tomcat and Hudson.

Installing Hudson

In our security group we opened port 8280. Tomcat however runs on 8080 by default. Lets change that.

$ sudo vi /etc/tomcat6/server.xml 
...
    <connector port="8280" connectiontimeout="20000" 
        protocol="HTTP/1.1" redirectport="8443">

Now we configure tomcat to run as the ec2-user, which allows Hudson to reuse the ec2-user's maven repository. HUDSON-HOME can also be set in the same config file.

$ sudo vi /etc/tomcat6/tomcat6.conf
...
# Where your java installation lives
JAVA_HOME="/usr/java/jdk1.6"

# Where HUDSON has its home
HUDSON_HOME=/usr/share/tomcat6/workspace/hudson-home

# What user should run tomcat
TOMCAT_USER=ec2-user

Make sure the ec2-user is in the 'tomcat' group so it can write files to HUDSON_HOME during setup.

$ sudo usermod -g tomcat ec2-user
$ mkdir -p /usr/share/tomcat6/workspace/hudson-home

You might need to re-login for the initial login group change to take effect.

JBossOSGi simplifies Hudson setup a great deal, by including configuration and jobs in it codebase. This also allows us to recreate the QA environment for any given version by running a simple Ant task.

$ cd git/jbosgi/hudson
$ cp ant.properties.example ant.properties
$ ant hudson-setup
...
init-hudson:
     [echo] hudson.home = /usr/share/tomcat6/workspace/hudson-home
...
get-hudson:
      [get] Getting: http://hudson-ci.org/download/war/1.382/hudson.war
      [get] Getting: http://hudson-ci.org/download/plugins/github/0.2/github.hpi
      [get] Getting: http://hudson-ci.org/download/plugins/git/1.1/git.hpi
...
hudson-tomcat-setup:
     [copy] Copying 1 file to /usr/share/tomcat6/workspace/hudson-home/plugins
     [copy] Copying 1 file to /usr/share/tomcat6/workspace/hudson-home/plugins

hudson-setup:
     [copy] Copying 2 files to /usr/share/tomcat6/workspace/hudson-home
     [copy] Copying 9 files to /usr/share/tomcat6/workspace/hudson-home/jobs
     [echo] 
     [echo] *************************************
     [echo] * Hudson setup successfully          
     [echo] * sudo service tomcat6 restart       
     [echo] *************************************

$ sudo service tomcat6 restart
Stopping tomcat6:                                          [  OK  ]
Starting tomcat6:                                          [  OK  ]

Now we should be able to connect to the running Hudson instance.


Hudson Security


By default, Hudson is setup with strict project based security. Only the 'ec2-user' or whoever you configured in ${hudson.admin} can modify the configuration and run the jobs. Lets sign-up as ec2-user


You should now be able to run the jobs.

Upgrading the Instance Type

The Amazon free offer is limited to Micro instances. With it's 613MB of RAM not surprisingly it turns out that this is not sufficient to run the JBossOSGi QA in an acceptable time.

This is however not a problem, because the instance can be stopped and it's type can be changed to m1.small or m1.large depending on what you are willing to pay.

$ ec2-stop-instances i-10429a7d
$ ec2-modify-instance-attribute --instance-type=m1.large i-10429a7d
$ ec2-start-instances i-10429a7d

Note, that this can only be done with instances that are backed by the Elastic Block Store.

Creating an Elastic IP

Every time you stop/start an EC2 instance it's public DNS entry changes. Amazon's solution to this is the Elastic IP. You pay for the IP when your instance is not connected.



This IP address can be associated with your favorite domain name. In our case its a dyndns.org forward. So what was formerly known as


now points to the EC2 instance that we've just setup.


May this be useful
-thomas

1 comment: