Sunday, August 24, 2014

TomEE - Detecting Webapp Deployment

Setup your environment for a TomEE debugging session

To understand the TomEE startup process

Once TomEE jars are loaded into Tomcat and initialized, There should be a way of detecting deployments of the other webapps. This is again achieved through a Tomcat's favourite Lifecycle Listener named GlobalListenerSupport.

The TomcatWebappBuilder class can be considered as the main contirbutor of this process. When TomEE initialized,

  • It creates a new instance of TomcatWebappBuilder.
  • The TomcatWebappBulder creates an instance of GlobalListenerSupport.
  • GlobalListenerSupport holds a reference to TomcatWebappBulderInstance.
  • The GlobalListenerSupport is registered to the Server as a lifecycle listener

Now when a lifecycle event occurred for any webapp,
  • The Tomcat will trigger the GlobalListenerSupport since it is a lifecycle listener.
  • The GlobalListener will simply filter the relevant events and call the relevant method of the TomcatWebappBuilder  reference.

This is how the TomcatWebappBuilder get to know about webapp deployment & undeployment. 



Monday, August 11, 2014

TomEE - OpenEJB SystemInstance class

The SystemInstance  class is one of the most important class in TomEE. It acts as a centralized repository which holds all the important properties and components of the entire TomEE system. Further it fires events to its observers whenever a component is registered/unregistered.

This class follows the singleton pattern. Which means that there can be only one instance per any given classloader and its child classloaders. Further, it is the only static in the entire TomEE system.

The SystemInstance.init(Properties) method initializes the SystemInstance. That method guarantees one time initialization by  "if (initialized) return;" statement at the beginning. Next it loads properties from various sources.

Who Initializes the SystemInstance?


There are many places that calls this method. In TomEE (jars in the tomcat/lib), the TomcatLoader calls this method.

In the dropin-war deployment, this init method calls twise. One on the mini-webapp classloader by TomcatHook.hook() method and the other one on the Tomcat common class loader by TomcatLoader.

Reading Properties


The class has the following methods for read/write properties.

// Read all the properties
public Properties getProperties(){ ... }

// Read a property value by passing the property key.
public String getProperty(String key) { ... }

As a Component Repository


The SystemInstance class maintains a HashMap to hold the components of various classes. The key is the class and the actual Object is kept as the value.

Here are the two methods that put values into the above HashMap.

// Simply sets the input as key value pair.
// Publish the "ComponentAdded" or "ComponentRemved" event to Observers
public T setComponent(final Class type, final T value) 

// Look for an existing value in the HashMap
// If found
//     return it
// else
//     Create an instance of the given class using reflxion
//     Store that in the HashMap
//     Fire the "ComponentAdded" event
//     Return that instance.
public T getComponent(final Class type)

Thursday, August 7, 2014

TomEE Startup - Source Code Explained

The TomEE website explains the TomEE's architecture in one sentence.
"We start with Apache Tomcat, add our jars and zip up the rest. The result is Tomcat with added EE features - TomEE"
In other words, TomEE is just adding jars to vanilla Tomcat and it does not change the one single line of the Tomcat code. So how these jars getting mixed up with Tomcat and cook the TomEE soup?

There are two (2) recipes:
  1. Add TomEE jars to Tomcat's lib directory
  2. Deploy TomEE jars as a droppin  war

Recipe 1 - Add TomEE jars to Tomcat's lib directory

 

  • The TomEE source code is built into set of jar files.
  • These jar files (including dependencies) are copied into the Tomcat /lib directory.
  • Register org.apache.tomee.catalina.ServerListener class in server.xml as a lifecycle listener.

This ServerListener is the trigger point of TomEE code.
Tomcat calls the ServerListener.lifecycleEvent method when a server lifecycle event occurres.

When Tomcat's "after_init" event occurred, the listener sets the required properties and calls the TomcatLoader.init.

TomcatLoader is the responsible class for initializing the rest.


Recipe 2 - Deploy TomEE jars as a droppin war

 

The goal of this deployment method is, load all the tomee libs to tomcat's common class loader and call the TomcatLoader.init method. 

To achieve this goal, a war file is created with the following directory structure

/ -
  | - lib                            - root-lib
  | - | - [All the TomEE jar files]
  | - WEB-INF
      | - classes
      |   | - juli.properties
      | - lib                         - webinf-lib
      |   | - tomee-loader*.jar
      | - web.xml

This war file has something unusual. that is duplicate lib directories. one is root-lib which exist on the root directory of the webapp. The other one is the typical lib directory which exist in the WEB-INF.

This war file is yet-another-war which exists on the webapp directory of the Tomcat. So tomcat will load this as a normal webapp in Tomcat startup.

The entire process is summarized to the following diagram.
A diagram explains the TomEE startup in code level


Explanation:

The Tomcat first reads the web.xml file under the WEB-INF. The web.xml file has defined the LoaderServlet with <load-on-startup> set to 1.

  <servlet>

    <servlet-name>LoaderServlet</servlet-name>

    <servlet-class>org.apache.tomee.loader.LoaderServlet</servlet-class>

    <load-on-startup>1</load-on-startup>

  </servlet>


This load-on-startup 1 signal the tomcat to load the LoaderServlet at the start up. This is the triggering point of the TomEE code with this recipe.


This org.apache.tomee.loader.LoaderServlet class is included in the tomee-loader*.jar file. So it should be placed in the WEB-INF/lib directory, otherwise Tomcat cannot resolve the LoaderServlet class.

What does the LoaderServlet do?


All the TomEE jars are placed in the lib directory which exists in the root of the tomee war. These libraries should ultimately be loaded and initialized within the Tomcat's common class loader.

The LoaderServlet sets the physical path of the webapp and call the TomcatEmbedder.embed() method. The Tomcat's classloader is passed as a parameter to this embed method.

 TomcatEmbedder.embed(properties, config.getClass().getClassLoader());

The TomcatEmbedder.embed() method creates a new WebappClassLoader (a mini webapp in other words). and load both tomee-loader*.jar and openejb-loader*.jar files to that WebappClassLoader using reflexion.

// Use reflection to add the tomee-loader.jar file to the repository of WebappClassLoader. 

File thisJar = getThisJar();

String thisJarUrl = thisJar.toURI().toString();

webappClClass.getMethod("addRepository", String.class).invoke(childCl, thisJarUrl);



// childCl.addRepository(openejb-loader.jar)

// Use reflection to add the openejb-loader.jar file to the repository of WebappClassLoader. 

File jarFile = findOpenEJBJar(openejbWar, OPENEJB_LOADER_PREFIX);

String openejbLoaderUrl = jarFile.toURI().toString();



webappClClass.getMethod("addRepository", String.class).invoke(childCl, openejbLoaderUrl);




**This mini webapp is optional for the LoaderServlet. The exact requirement of this Mini webapp will be discussed later.**

Finally, the TomcatHook.hook() mehtod is called using reflection by the TomcatEmbedder.

Important to notice that The TomcatHook class is loaded by the  newly created WebappClassLoader.

The TomcatHook sets some of the important properties and constructs an Embedder object. This Embedder is the class which does the magic of adding tomee libs to the Tomcat's common classloader.

// set the embedder

final Embedder embedder = new Embedder("org.apache.tomee.catalina.TomcatLoader");

SystemInstance.get().setComponent(Embedder.class, embedder);

embedder.init(properties); 


The embedder adds all the tomee, openejb & dependency jars to the classpath of the Tomcat common classloader.

FileUtils home = SystemInstance.get().getHome();

libsDir = home.getDirectory("lib");



classPath.addJarsToPath(libsDir); 


Now this Tomcat instance is similar to TomEE jars copied into the Tomcat's lib folder.

The Embedder calls the TomcatLoader.init method using reflection at last

// get the init method

Method init = loaderClass.getMethod("init", Properties.class);



// create the instance

Object instance = loaderClass.newInstance();



// invoke init method

Object value = init.invoke(instance, properties);



TomcatLoader does the initialization of the rest of the system.


Recipe 3 - Pre-cooked Recipe 2

 

One of the noticeable problem under the  dropin war deployment is, the possibility of deploying other webapps before the TomEE dropin war.

This delay is avoided by changing the triggering point of the war from Servlet to LifecycleListener. This lifecycle listener is named as "OpenEJBListener". This class resides in the tomee-loader*.jar

Steps:

  • put the TomEE dropin war into the webapp directory of Tomcat.
  • Copy tomee-loader*.jar file to Tomcat lib folder.
  • configure OpenEJBListener as a lifecycle listener in server.xml

The OpenEJBListener


When Tomcat starts, this lifecycle listener receives its first lifecycle event before deployment of any of the webapps.

The Listener does the following thing:
  1. Check whether the dropin war is available in the webapps directory
  2. extract if its still a war file
  3. sets the path of the webapp as a property.
  4. calls the TomcatEmbedder.embed
From this point onwards It is similar to the functionality of "Recipe 2".

The below diagram represents this architecture.


Why a mini-webapp

Time to explain the usage of mini-webapp creation of the TomcatEmbedder class.

The OpenEJBListener class is picked by Tomcat from the tomee-loader*.jar which is placed in the Tomcat/lib directory. When OpenEJBListener is calling the TomcatEmbedder, the TomcatEmbedder is loaded by the same Tomcat classloader. What we exactly need is to call the TomcatHook class within a WebappClassLoader. That is why the TomcatEmbedder creates a new WebappClassLoader and call the TomcatHook within that class loader.


In the LoaderServlet way, The TomcatEmbedder is loaded by a WebappClassLoader. So it was optional to create another WebappClassLoader and re-triggering the process.

Conclusion

The TomEE jars should be loaded to the Tomcat's Common classloader and calling the TomeeLoader within that class loader is the way that TomEE integrates with the Tomcat.

This is done in two ways.
  1. Put Tomee jars to lib folder and call TomeeLoader.
  2. pack tomee jars in a webapp and adding those jars to the ParentClassloader of that webapp's classloader. Call the TomeeLoader
  3. The 2nd way is triggered by either LoaderServlet or OpenEJBLIstener

Friday, July 18, 2014

Start TomEE Contribution - Setting Up your Development Environment


Debug through the TomEE source code is a must-to-follow step if you want to understand how TomEE works and do some contribution.  This is a guide to quickly start your debugging session with TomEE as a TomEE developer.

This guide assumes
  • Linux is the OS
  • IntelliJ IDEA 13.1.3 is the IDE
  • Maven 3.x.x is installed

Here we go!

Download the source code 


For beginners I will recommend not to start with the trunk, because It is normal to have some blockers, non-stable functionalists which may cause your learning crashes at some point.

So first start with the latest stable released source code. Go to trunk once you are ready to do some code modification on TomEE.

Click here to download TomEE 1.6.0.2 Source code

Build the Source Code


First extract the zip file named openejb-4.6.0.2-source-release.zip to any location. Lets assume it is your home folder.
unzip openejb-4.6.0.2-source-release -d ~
 The above command will create the openejb-4.6.0.2 directory in your home directory.

Even though you can do a full build, We will run the following command to do a quick build so that you can have your meal before your hungry kills you.
mvn -Pquick -Dsurefire.useFile=false -DdisableXmlReport=true -DuniqueVersion=false -ff -Dassemble -DskipTests -DfailIfNoTests=false clean install
 More details about building the product from the source can be found here.

Deploy TomEE


The TomEE build builds several distributions (zip & war files) to cater the different needs of different users. Here we discuss about the tomee plus distribution & TomEE war distribution only. TomEE+ is the full feature packed distribution from TomEE.

TomEE+ zip location:
~/openejb-4.6.0.2/tomee/apache-tomee/target/apache-tomee-plus-1.6.0.2.zip
Unzip the zip into your home directory (or any other location)
unzip  ~/openejb-4.6.0.2/tomee/apache-tomee/target/apache-tomee-plus-1.6.0.2.zip -d ~
You will find the directory apache-tomee-plus-1.6.0.2 in your home folder. Lets run the TomEE.

cd ~/apache-tomee-plus-1.6.0.2/bin
./catalina.sh run
"INFO: Server startup in xxxx ms" is the Green light!

Prepare your IDE


Lets prepare our IntelliJ IDEA for the debugging session.

Start IntelliJ IDEA and Click the Import Project link
  

Select ~/openejb-4.6.0.2to and press OK
 

 Select import project from external model & Maven as the external model.

Press Next on this screen.

Select the main profile.

Select the org.apache.openejb:openejb:4.6.0.2

Select the JDK you want to use with.

Give the project name and press Finish.


Now your IDE will load the project.

First Breakpoint


Next step is to put a breakpoint at the place where the code is triggered. Lets understand how the code is triggered.

TomEE+ is created on top of Tomcat. TomEE registers a Tomcat Lifecycle Listener "org.apache.tomee.catalina.ServerListener" on server.xml file.

All the Tomcat lifecycle events i.e. before_init, after_init, start, before_stop etc... are received by the lifecycleEvent method of the ServerListener.

The execution of TomEE code starts in this lifecycleEvent method. So the first breakpoint should be on the lifecycleEvent method.

Run TomEE+ in debug mode


If you simply run catalina.sh jpda run in the bin folder of tomee deployment, the server starts in the debug mode but it will quckly pass your breakpoint before you attach your IDE to the server process.

So we set JPDA_SUSPEND="y"  before we start our debugging. This will notice the server, "Do not proceed until the Debugger tool is attached to the process"

The convenient way of doing this is adding this line to catalina.sh file right after the #!/bin/sh line.

#!/bin/sh
JPDA_SUSPEND="y"
 Now to time to run TomEE+ on debug mode.
~/apache-tomee-plus-1.6.0.2/bin/catalina.sh jpda run
 
 The terminal should hang with the message "Listening for transport dt_socket at address: 8000"

Attach IntelliJ IDEA debugger


  • Menu Bar > Run > Edit Configurations
  • Press the "+" button on the top left corner to get the Add new configuration menu
  •  Select "Remote" from the Add new configuration menu
  • Give a name (I gave "TomEE DEBUG") to this new configuration and set the Port to 8000
  • Click OK.

To start debugging your TomEE+
Main Menu > Run > Debug TomEE DEBUG

Congratulations! You hit the break point you put at the startup of the TomEE code. Carry on with your debugging session to learn more.