Mashups and the Enterprise Mashup Markup Language (EMML)

The Structure and Content of a Mashup Script

The Mashup script has <mashup> as the root element. Within the root element, you can add other elements to define variables, invoke other services, and process the results to return the mashup result. EMML provides a large set of elements that let you perform different operations easily. Some of the operations supported by EMML include:

• Declare mashup input, output parameters and variables to hold input/output for mashup scripts
• Declare RDBMS data sources that define connection details.
• Issue statements to execute a SQL query to a data source.
• Invoke component services such as REST or SOAP web services and HTML- or RSS-based web sites.
• Web clipping or screen scrapping functionality for HTML from any URI using XHTML.
• Transform intermediate results such as assign one variable to another, filter, group, sort, annotate data across heterogeneous services.
• Defining a script to execute at runtime.
• Combining results from different component services
• Defining the Mashup Output, Input or Intermediate Variables.
• Controlling the Mashup Processing Flow through if, for, while, and parallel elements.
• Defining and using custom mashup statements with macros.
• Send messages to the console or logs to support debugging.
• Adding Metadata to Mashups to provide user or any other information.
• The result of a mashup script can also be used within another mashup script.

Developing and Deploying Mashups

To illustrate what's involved in building EMML-based mashups, I create Stock Ticker mashup and deploy it in the EMML Runtime reference implementation deployed on Tomcat Web Server. The sample mashup uses web clipping from Google Finance for the stock price details based on ticker code from a HSQL database which is combined with a company specific news feed from Yahoo Finance News to display a real-time stock ticker for few of the popular stocks. The sample also showcases the use of JavaScript by adding some information to the final result. The complete XML and EMML source code for the StockTicker example is available here.

I've tested this example with:

• Apache Tomcat 6.0 or any other J2EE compliant Application Server
• EMML Runtime Engine Reference Implementation 1.0
• HSQL library (included in the EMML Runtime)

The first step, of course, is to install and setup the deployment environment:

1. Download the latest Apache Tomcat server from the Apache site (http://tomcat.apache.org/download-60.cgi).
2. Download the EMML related zip file from the OMA site (http://www.openmashup.org/download).
3. Deploy the EMML reference runtime engine web application emml.war in the server by copying the war file to the %TOMCAT_HOME%/webapps folder where %TOMCAT_HOME% is the folder where Tomcat has been installed.
4. Copy the testing class emmlclient.class to %TOMCAT_HOME%/webapps/emml/WEB-INF/classes. This is used to test the mashup through the command line. In this example, however, I will be testing through the URL.

For this example, I'm using a HSQL database to retrieve the stock names and company names. The HSQL server needs to be started before the mashup can be tested.

1. Go to the %TOMCAT_HOME%/temp folder and start the HSQL database using this command:

java -classpath ..\webapps\emml\WEB-INF\lib\hsqldb.jar org.hsqldb.Server

2. From the same folder, start the HSQL Database Manager GUI using this command:

java -cp ..\webapps\emml\WEB-INF\lib\hsqldb.jar org.hsqldb.util.DatabaseManagerSwing

3. Create a table of the name STOCK_TICKER using this query:

CREATE TABLE STOCK_TICKER (STOCKNAME VARCHAR, COMPANYNAME VARCHAR);

4. Insert some values into the STOCK_TICKER table using this query:

INSERT INTO STOCK_TICKER VALUES ('INFY', 'Infosys');
INSERT INTO STOCK_TICKER VALUES ('GOOG', 'Google');
INSERT INTO STOCK_TICKER VALUES ('MSFT', 'Microsoft');
INSERT INTO STOCK_TICKER VALUES ('YHOO', 'Yahoo');

Developing and Deploying a Mashup Script

Here are the steps necessary to create and execute a mashup as per the EMML specification:

1. The first step is to create an emml script as per the specifications. Open a new XML file in any editor and add the following content to it. Save the file as StockTicker.emml in your local hard drive.

<mashup xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  	xmlns:xhtml="http://www.w3.org/1999/xhtml"
        xsi:schemaLocation="http://www.openemml.org/2009-04-15/EMMLSchema ../schemas/EMMLSpec.xsd"
        xmlns="http://www.openemml.org/2009-04-15/EMMLSchema"
 	name="StockTicker">
</mashup>

2. Next add the input and output variables to the mashup script. The output of the mashup could be of type boolean, date, document, number, string or any token that identifies a data type defined by a service. The input of the mashup can only be one of the primitive types. Input cannot be of document type.

<!-- Input and Output tags to declare input/output parameters to the mashup.  There can be only one output for a mashup -->
<input name="ticker" type="string" default="INFY"/>
<output name="result" type="document"/>

3. For this mashup, I retrieve the list of stock codes and company names from a HSQL database based on which the details are retrieved from Google Finance and Yahoo Finance News sites. EMML provides tags to configure and connect to any database and retrieve the required values. Here is the code that does this:

<!-- Connect to a SQL data source -->
<datasource url="jdbc:hsqldb:hsql://localhost:9001" username="sa" password="" />
	
<!-- Execute select statement to retrieve the stock and company name --> 
<sql query="select stockname, companyname from STOCK_TICKER" outputvariable="itemNames"/>

The output is stored in the variable itemNames as an XML document in this format:

<records>
	<record>
		<stockname>INFY</stockname>
		<companyname>Infosys</companyname>
	</record>
	…
</records>

4. We would now need to retrieve the stock price information and news for the list of stocks retrieved above. EMML provides a many conditional operations that allow developers to control the processing flow. We use <foreach> tag to loop through a set of nodes:

<foreach variable="value" items="$itemNames/records/record">
	<!-- Retrieve the ticker code and the company name from the SQL query result -->
	<assign fromexpr="$value/stockname/string()" outputvariable="ticker"/>
	<assign fromexpr="$value/companyname/string()" outputvariable="companyname"/>
</foreach>

5. EMML provides a common command to invoke any publically accessible web service or website using the <directinvoke> operation. I access the Google Finance web page and retrieve the financial information for the ticker through the web clipping feature provided by EMML. Web Clipping converts the HTML result of any URI into XHTML and gives you a clip of the required content from the web page:

   
   <foreach variable="value" items="$itemNames/records/record">
   	…
   	<template expr="http://finance.google.com/finance?q={$ticker}" outputvariable="wholeURL"/>
   
   	<!-- invoke the Google Finance web page that has stock information -->
   	<directinvoke outputvariable="clipresult" endpoint="$wholeURL"/>
   
   	<assign fromexpr="$clipresult//xhtml:div[@class='g-section sfe-break-bottom-8']" outputvariable="clipresult2"/>
   	<assign fromexpr="$clipresult2//xhtml:h3/string()" outputvariable="$company"/>
   	<assign fromexpr="$clipresult//xhtml:span[@class='pr']" outputvariable="clipresult3"/>
   	<assign fromexpr="$clipresult3//xhtml:span/string()" outputvariable="$price"/>
   	<assign fromexpr="$clipresult//xhtml:div[@id='price-change']" outputvariable="clipresult4"/>
   	<assign fromexpr="$clipresult4//xhtml:span[@class='chg']/string()" outputvariable="change"/>
                  …
   </foreach>
   
   

   The company name, price, and percentage change details are retrieved from the web page and stored in temporary variables.

6. Next I retrieve the news feed for the provided company using Yahoo Finance News RSS feed. As mentioned earlier this is also accessed using the <directinvoke> command. EMML also provides various commands to transform intermediate results using commands such as filter, sort, group, script, etc. In this sample, I only retrieve the first three news results and title, link, description details from the RSS feed:

<foreach variable="value" items="$itemNames/records/record">
	…
	<!-- Get the news feed from Yahoo Finance News -->
	<template expr="http://news.search.yahoo.com/news/rss?p={$companyname}" outputvariable="newsURL"/>

	<!-- Add the additional values to the URL using JavaScript -->
	<script type="text/javascript" inputvariables="newsURL" outputvariable="newsURL">
	<![CDATA[
	    newsURL += "&ei=UTF-8&fl=0&x=wrt";
	]]>
	</script>

	<directinvoke endpoint="$newsURL" outputvariable="companynews"/>

               <!-- Filter RSS feed using xpath regular expression function to retrieve only 3 news feeds -->
	<filter inputvariable="companynews"
		filterexpr = "/rss/channel/item[position() = (1 to 3)]"
		outputvariable="companynews"/>
		
	<select inputvariable="$companynews" outputvariable="$companynews" 
		  selectexpr="/rss/channel/item"> 
		<columns>
			<column>title</column>
			<column>link</column>
			<column>description</column>
		</columns>
	</select>
               …
</foreach>

The company news retrieved from the RSS is stored in the temporary variable companynews.

7. Using the required content stored in the temporary variables, a custom result xml is prepared using the <appendresult> command:

 

<foreach variable="value" items="$itemNames/records/record">
	…
	<!-- build the  xml result -->
	<appendresult outputvariable="result">
		<stock>
			<ticker>{$ticker}</ticker>
			<company>{$company}</company>
			<price>{$price}</price>
			<change>{$change}</change>
			<news>{$companynews}</news>
		</stock>
	</appendresult>
</foreach>

8. Finally an additional information tag is added to the final result using JavaScript:

<!-- Add a final source and generated time using JavaScript -->
<script type="text/javascript" inputvariables="result" outputvariable="result">
<![CDATA[
	var m_names = new Array("January", "February", "March", 
			"April", "May", "June", "July", "August", "September", 
			"October", "November", "December");
	var d = new Date();
	var curr_date = d.getDate();
	var curr_month = d.getMonth();
	var curr_year = d.getFullYear();
	var curr_hour = d.getHours();
	var curr_min = d.getMinutes();
		
	// Add order details to EMML Variable - result
	var info = <info/>;
	info.source = "Google Finance and Yahoo Finance News";
	info.datetime = "Auto Generated on " + curr_date + "-" + m_names[curr_month] + "-" + curr_year + " at " + curr_hour + ":" + curr_min;

	result.info = info;
]]>
</script>