Pentaho Cluster: Installing JGroups

As I mentioned in the previous post I am going to attempt to make a Pentaho Cluster, based on this presentation(PDF) and outlined in this preparation post.

So the fundamental building block for this cluster is the JGroups Java libraries, so first cab off the rank is to download and install the jar files and get demo to work.

After some mishaps with my CLASSPATH I got that sorted and tested the demo using two JGroups instances running on the same node. See the picture. Drawing in one window was immediately reflected in the second window on the right.

The install.html which is part of the zipped download file was good and explained the procedure well.

So the next thing is try the JGroups clustering on two separate EC2 nodes. That is next...

Have Fun

Paul

Installing JGroups onto a EC2 node with Java already installed.

Check Java is installed

java -version

java version "1.5.0_12"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_12-b04)
Java HotSpot(TM) Client VM (build 1.5.0_12-b04, mixed mode, sharing)

Download JGroups

http://labs.jboss.com/jgroups/downloads
wget http://easynews.dl.sourceforge.net/sourceforge/javagroups/JGroups-2.3.bin.zip

Unzip and copy files to /usr/local

unzip JGroups-2.3.bin.zip -d /usr/local

Run the checker to make sure you have installed correctly

java -jar JGroups-2.3.bin/jgroups-all.jar

Version:        2.3
CVS:            $Id: Version.java,v 1.35 2006/06/11 19:15:23 belaban Exp $
History:        (see doc/history.txt for details)

Start X-window to EC2

ssh -i id_rsa-gsg-keypair -X root@yourEC2host

Check the display is set correctly

echo $DISPLAY

localhost:10.0

Run the demo twice, use & to background each command from command line.

cd /usr/local
java -cp JGroups-2.3.bin/concurrent.jar:JGroups-2.3.bin/jgroups-all.jar:JGroups-2.3.bin/commons-logging.jar org.jgroups.demos.Draw

Sep 20, 2007 7:34:29 AM org.jgroups.protocols.UDP createSockets
INFO: sockets will use interface 10.253.22.176
Sep 20, 2007 7:34:29 AM org.jgroups.protocols.UDP createSockets
INFO: socket information:
local_addr=10.253.22.176:32772, mcast_addr=228.8.8.8:45566, bind_addr=/10.253.22.176, ttl=32
sock: bound to 10.253.22.176:32772, receive buffer size=64000, send buffer size=32000
mcast_recv_sock: bound to 10.253.22.176:45566, send buffer size=64000, receive buffer size=64000
mcast_send_sock: bound to 10.253.22.176:32773, send buffer size=64000, receive buffer size=64000

-------------------------------------------------------
GMS: address is 10.253.22.176:32772
-------------------------------------------------------
** View=[10.253.22.176:32769|1] [10.253.22.176:32769, 10.253.22.176:32772]
** View=[10.253.22.176:32769|1] [10.253.22.176:32769, 10.253.22.176:32772]

Pentaho Business Suite Cluster : Research and preparation

As I mentioned in this article on running the Pentaho Demo on EC2, according to a recent presentation, Pentaho has the ability to be clustered using JBoss Clustering (JBoss JGroups), JBoss AS and Apache.

This post is to outline the background documentation I am using to give this cluster a go on EC2.
There are some posts indicating that the lack of multicasting with EC2 is a issue, however comments and the doco suggest that it will revert to TCP, although that is slower.

Given I already have Pentaho installed and saved as an AMI, I am going to build on that to make the single node. The documentation and wikis confident that it should auto discover and be a piece of cake, however we shall see.

Have Fun

Paul

Here is the specification of the cluster node (from that presentation)

Single Node

Single CPU
2 GB RAM
JBoss AS 4.0.3
JBoss JGroups?
Pentaho 1.1.5

Cluster Master:

Apache HTTP server 2.0.58 with mod_jk module version 1.2.15
JGroups cluster master
JMS / Web Services for Operational BI

Doco (Documentation)

http://docs.jboss.org/jbossas/jboss4guide/r4/html/jbosscache.chapt.html
http://docs.jboss.org/jbossas/jboss4guide/r4/html/cluster.chapt.html
http://www.onjava.com/pub/a/onjava/2002/07/10/jboss.html
http://www.jboss.org/wiki/Wiki.jsp?page=JBossHA
http://www.jboss.org/developers/projects/jboss/clustering
http://clusterstore.demo.jboss.com/

Forums and Blogs:

http://www.jgroups.org/javagroupsnew/docs/Perftest.html
http://blog.decaresystems.ie/index.php/2007/01/29/amazon-web-services-the-future-of-datacenter-computing-part-1
http://blog.decaresystems.ie/index.php/2007/02/12/amazon-web-services-the-future-of-datacenter-computing-part-2

Apache
http://jakarta.apache.org/tomcat/tomcat-3.3-doc/mod_jk-howto.html

Downloads

http://labs.jboss.com/jbossas/downloads
http://labs.jboss.com/jgroups/downloads
http://labs.jboss.com/jbosscache/download/index.html

Mondrian OLAP on MySQL EC2 Part 1

If you are wondering why there has been no recent postings, it is due to mainly to struggling to get the Mondrian installed. It had nothing to do with the environment and mostly due to documentation missing vital examples or explanation.

It is with relief I can say I have passed the test and can pass over to the fair lands of using Mondrian OLAP and running the demo.

You can download Mondrian from the Pentaho website or directly from sourceforge.
Use this installation document as a guide however it glosses over some of the nice little details which will make the demo work or not.

Comments:

1. Installing the demo was way too hard, just bundle tomcat or build a include everything binary for MySQL, PostgreSQL, Oracle or whatever. You want people to try the program, not many people will persist like I did to get it to work.
   
2. Having to hand edit every file which connects to the database or is used as a tomcat configuration file sucks, ever heard of storing that stuff in a single file or in the database.
3. Provide a simple SQL script to create schema objects and load data. Running the Foodmart loader did not prove you could run the demo! use this SQL script otherwise from Gizzar
   
4. Some kind of example of either a tomcat or other verification tool would be good, rather than having to lather, rinse and repeat over java, JDBC, Tomcat and red herring errors.

Many thanks to some other blogs and sites which helped solved various issues which cropped up
Gizzar article on Mondrian Open Source OLAP with MySQL
University of Vienna (Wien) for Tomcat FAQ solving tomcat shutdown on missing X
Mondrian Forum

Have Fun

Paul

Install:
Note: This really requires a HOWTO document. I will work on that based on this. Hopefully you can use this in conjunction with installation guide and the various other blogs who had fun with this.

1. Install some linux packages: yum install gcc autoconf
   
2. Download the last release of Mondrian non-embedded files from SourceForge.
3. Download and install Java 1.5 or better from Sun Java Downloads.
   
4. Download and install MySQL 5.0 from MySQL Downloads.
5. Download and unzip the MySQL JDBC driver
   
6. Download and install Apache Tomcat 5.0.28.
7. Verify that Java, MySQL and Tomcat are working by doing the following:
   
8. java -version
9. mysql -V
10. /usr/local/tomcat/bin/startup.sh
11. Point your browser at http://yourhostname:8080, if it works, tomcat is working.
    
12. unzip Mondrian.zip -b /usr/local/mondrian
13. Create the MySQL database: mysqladmin create foodmart -u root -p
14. Create the Foodmart User: create user 'foodmart'@'yourhostname' identified by 'foodmart';
15. Grant permissions: grant all privileges on *.* to 'foodmart'@'yourhostname' identified by 'foodmart';
16. Explode the mondrian.war into /usr/local/tomcat/webapps/mondrian: jar -xvf mondrian.war
17. Locate the 4 jar files eigenbase-properties.jar,eigenbase-resgen.jar,eigenbase-xom.jar and log4j-1.2.9.jar
    
18. Run Mondrian.FoodmartLoader to create tables and load data, pass in the full path to all the required JAR files, otherwise it will fail with a Class notFound error. This is an example:
    



java -cp "lib/mondrian.jar: /usr/local/mysql-5.0.45-linux-i686/mysql-connector-java-5.0.7/src/lib/log4j-1.2.9.jar: 
lib/eigenbase-xom.jar:lib/eigenbase-resgen.jar:lib/eigenbase-properties.jar:
/usr/local/mysql/mysql-connector-java-5.0.7/mysql-connector-java-5.0.7-bin.jar" \
mondrian.test.loader.MondrianFoodMartLoader \
-verbose -tables -data -indexes \
-jdbcDrivers=com.mysql.jdbc.Driver \
-inputFile=demo/FoodMartCreateData.sql \
-outputJdbcURL="jdbc:mysql://localhost/foodmart?user=foodmart&password=foodmart"



19. cp /usr/local/jakarta-tomcat-5.0.28/webapps/mondrian/WEB-INF/lib/xalan.jar /usr/local/jakarta-tomcat-5.0.28/common/endorsed/
20. Modify the query files, web.xml, datasource.xml and mondrian.properties file to replace localhost with your hostname



sed -i`date +%y%m%d` -e "s/localhost/yourhostname/" fourhier.jsp
sed -i`date +%y%m%d` -e "s/localhost/yourhostname/" mondrian.jsp
sed -i`date +%y%m%d` -e "s/localhost/yourhostname/" colors.jsp
sed -i`date +%y%m%d` -e "s/localhost/yourhostname/" arrows.jsp
sed -i`date +%y%m%d` -e "s/localhost/yourhostname/" web.xml
sed -i`date +%y%m%d` -e "s/localhost/yourhostname/" datasource.xml
sed -i`date +%y%m%d` -e "s/localhost/yourhostname/" mondrian.properties



21. Modify the connection string in each file as per install guide, including the #38 stuff it is not a browser character issue.
22. Test connectivity for mysql user foodmart@yourhostname. Connection errors can cause this error



Mondrian Error:Internal error: Error while creating SQL dialect



23. export CATALINA_OPTS='-Djava.awt.headless=true' or add to your shell profile, this stops tomcat dying when X-windows is not found!! Thanks to this link for solving that.