Dr. Dobb's | Writing SMTP-Based SOAP Messages in PHP

Writing SMTP-Based SOAP Messages in PHP

With the buzz around web services, most SOAP usage has focused around the HTTP protocol, but other protocols can be used as well. SMTP has advantages not found with HTTP, including SMTP's ability to store and forward messages, implement one-to-many broadcast messaging, and use attachments to embed extra data into a SOAP message.

October 01, 2002
URL:http://www.drdobbs.com/writing-smtp-based-soap-messages-in-php/184416579

October 2002/Writing SMTP-Based SOAP Messages in PHP

SOAP, the Simple Object Access Protocol, allows decentralized and distributed communications using XML as a common data format. Having worked on a SOAP implementation for PHP, I would argue that it's not so simple! However, from a user's point of view, most SOAP implementations strive for ease of use.

SOAP is commonly hyped as "web services," but it's much more than a web service. SOAP can be used over many transports, in many systems and situations, to provide a common communication protocol. This is where "simple" comes in. Code written to use SOAP over other protocols will have little difference from code written to use SOAP over HTTP. Most of the differences are handled transparently.

For example, PEAR::SOAP (PHP's SOAP implementation) supports HTTP and SMTP; Perl's SOAP::Lite includes support for a number of protocols such as Jabber and TCP, in addition to HTTP and SMTP. Several other SOAP implementations provide SMTP support. Support for alternate protocols exists, and is fairly easy to implement. However, with the hype around web services, most SOAP usage has focused around the HTTP protocol.

This article will provide a view of how SOAP over SMTP works, and how it functions within a couple of popular scripting languages, Perl and PHP. As an example, I'll illustrate a simple system that catalogs images. Images are added via SOAP calls made over SMTP; a SOAP message containing the URL to the image is returned to the sender.

Why SMTP?

So why use SOAP over SMTP? SMTP works well for certain kinds of data transmission and has built-in functionality unavailable in the HTTP protocol. For example, SMTP has the inherent ability to store and forward messages. SMTP servers will resend a message numerous times before giving up and returning a failure message. HTTP, on the other hand, provides an immediate response to a request. With SMTP, we can send a one-way message and expect a response some time in the future (if we want one at all).

Another interesting possibility with SMTP is the use of a list server to send one-to-many SOAP messages. Imagine pushing catalog updates out to your distributors via e-mail, and having their systems update without intervention. While you can certainly reproduce store-and-forward, one-way, or asynchronous messaging over HTTP, why reinvent the wheel (yet again)? There are a number of instances where you could consider using SMTP, such as:

To implement one-to-many messaging.
When the endpoint is intermittently connected.
When the return route does not have a web server.
When there is a systems constraint on HTTP usage.
When the HTTP endpoint is behind a firewall.
When asynchronous messaging is required.

When deploying SOAP services over SMTP, you must ensure that you are using a SOAP implementation that supports SMTP on the client and, on the server, either a mail server that allows script execution upon message delivery, or POP3/IMAP access and a scheduler to check for e-mail messages on a timely basis, and process them appropriately.

How Does It Work?

Deploying SOAP over SMTP does not require changes to your e-mail system or additional e-mail-related software (beyond the capability of sending and receiving e-mail). Your SOAP client sends the request to an address using SMTP. When your e-mail server receives the message, it stores it in the normal manner. Then a process picks up that e-mail from the server using POP3 or IMAP (much like any e-mail client would) and processes the SOAP envelope contained in the message. After processing the envelope, a response could be sent via SMTP (or even some other protocol). If a response is expected, the client side of this transaction will also have to watch an e-mail box for responses, and handle them. It's a "simple" process.

Note that if your e-mail server can spawn processes when an e-mail is received, you could simply pipe the e-mail message to your request handler, which is a common method of processing e-mail.

Writing a Simple SOAP SMTP Client

First, I want to illustrate by writing a simple script that uses SOAP over SMTP, as compared to HTTP. When using SOAP over HTTP, you generally only provide a URL for the endpoint and a namespace for the function you are calling. For SMTP, you need to provide a little more information, such as the "From" address. Here's a quick example of sending a SOAP message over SMTP using Perl's SOAP::Lite module:

#!perl -w
use SOAP::Lite;

SOAP::Lite
 -> uri('http://soapinterop.org/')
 -> xmlschema('2001')    
 -> proxy('mailto:[email protected]', 
      smtp => 
      'smtp.activestate.com', 
      From => 
      '[email protected]', 
      Subject =>
       'SOAP message')
 -> echoString('Hello World');

As the next example shows, it's not all that different from using SOAP over HTTP:

#!perl -w
use SOAP::Lite;

my $response = SOAP::Lite
 -> uri('http://soapinterop.org/')        
 -> xmlschema('2001')
 -> proxy('http://pear.php.net/
      soap_interop/
      server_round2.php')
 -> echoString('Hello World');

print $response->result;

Operationally, the primary difference between the two is that the second example produces an immediate response, whereas in the first example, I would receive a return e-mail containing a SOAP message that could be processed offline. The SOAP envelopes are identical; there is only a small difference in headers that are specific to the transport protocol.

One of the most useful SMTP headers is the Message-ID. When you receive a reply to a SOAP request over SMTP, the Message-ID is reflected back to you in the In-Reply-To header. This behavior is defined in the SOAP over SMTP binding (see References). This allows you to link what you have sent to what you receive. Another noteworthy difference is that SOAP over SMTP requires an encoding using Mime or DIME. These encodings are optional for other transport layers, and are possibly not supported in many SOAP implementations. The SMTP and HTTP envelopes the scripts produce are shown in Listing 1.

Sending SOAP Attachments

What's an e-mail without attachments? SOAP attachments really don't have anything to do with using e-mail to send SOAP messages. However, they show how you can embed data, such as images, into a SOAP message. There are two protocols, MIME and DIME, with bindings defined for SOAP. The benefit of using attachments with SOAP is that they allow you to send data (such as an image) as part of the SOAP message in a more efficient manner. Also, you are not limited by XML syntax.

The first protocol that handles SOAP attachments is SwA, SOAP Messages with Attachments. This method uses MIME encoding to add attachments to a SOAP message. The SOAP envelope is encoded in the first MIME part and will have elements within the envelope that refer to external IDs. These external references are additional MIME parts that are encoded after the SOAP envelope.

The second protocol is DIME, or Direct Internet Message Encapsulation. DIME is a lightweight binary format that can be used to encode data. DIME is not really specific to SOAP; there is a separate IETF draft on binding DIME with SOAP (see References). DIME has benefits over MIME, such as a lower overhead in sending large amounts of data by using a chunking feature that is part of the specification (similar in some ways to HTTP/1.1 chunking). Aside from its advantages over MIME, DIME is important simply because Microsoft has chosen to support it instead of SwA (at the time of writing). While DIME is relatively new, many non-Microsoft SOAP implementations have been quickly adopting DIME, and the SOAP Builders community has been developing tests for DIME interoperability with SOAP messages.

Listing 2 is a SOAP envelope with an attachment, using MIME encoding. Essentially, the SOAP envelope is created first, with one element in the envelope containing a reference to external data. The external data is serialized into a separate message part using MIME or DIME, and the Content-ID is the ID referenced in the SOAP envelope by an 'href' attribute on one of the elements. It looks much scarier than it really is and, fortunately, you should never need to see these details. The code to produce the SOAP message in Listing 2 is shown in Listing 3. This example uses MIME. However, we can easily switch to DIME by removing the 'Attachments' option, or setting its value to 'DIME'. The PHP library defaults to DIME, so you don't really have to specify DIME. However, if new encodings are added in the future, DIME support may require the setting. Unfortunately, at the time of this writing, Perl's SOAP::Lite does not support SwA or DIME.

A Simple Image Archive

Now that I've gone through some of the basic components for using SOAP over SMTP, I'll illustrate the image archive system I mentioned in the introduction. This system is extremely simple, providing only the functionality I need to demonstrate a system using SOAP over SMTP.

First, a SOAP client must send messages to our archive. This client is basically the same as the client shown above, except that in the options array, I've added a namespace. In the earlier SOAP::Lite examples, this is the equivalent to the 'uri' parameter. This is important for the server side of this application, as it will only execute functions attached to a namespace; see Listing 4.

The E-mail SOAP Server

Implementing a SOAP server is almost as easy as a SOAP client, but takes a little more effort...this is server side, after all! The script in Listing 5 will retrieve messages via the POP3 protocol, and then process them one at a time. When discussing the client side, I pointed out the namespace option. In the server code below, you will see a method_namespace in the imageArchiver class. The client must match the namespace of the class. Usually this would be defined for the client automatically through the use of WSDL, but I'm leaving that out in this example.

Handling The Response

Finally, you need to handle the last link in the chain. Once you've made your SOAP request over SMTP, you will receive a return response. With some minor alterations of the previous server script, we can handle the response using the client method in the SOAP_Server_Email class. Listing 6 can be used to parse an incoming e-mail, but return native data types that your response handler script can deal with.

As you can see from all the source code, there is very little you need to know about SOAP at all. SOAP should be as transparent as possible to the developer.

If you want to run these examples, you will need to remove the two functions that were left unimplemented and provide usernames, passwords, and a host address to an available SMTP server. Be sure to do this behind the privacy of your own firewall, as I have not included any kind of security.

Installing

To use the examples in this article, a number of software components must be installed on your system.

Installing Perl and SOAP::Lite. If you do not already have Perl, you can download it from ActiveState at http://www.ActiveState.com/. Once Perl is installed, you can install SOAP::Lite and the Mime module using the Perl Package Manager (PPM). The Mime module will be installed when you update SOAP::Lite. To install SOAP::Lite:

C:\perl\bin\> ppm verify  — upgrade SOAP-Lite

Installing PHP and PEAR. For the PHP examples, you will need to install PHP from http://www.php.net/. If you want to use the installer for PHP, also download the larger package to get the necessary PHP modules. After running the installer, extract the larger zip file and overwrite the installation. The benefit of the installer is that it automatically configures IIS to use PHP. Once PHP 4.3 is released, you can use the PEAR Package Installer to install PEAR::SOAP (as shown later). To install PEAR::SOAP on PHP 4.2, see the installation instructions at the PEAR web site (http://pear.php.net/).

C:\php\pear\scripts\>pear install MODULE_NAME
Replace MODULE_NAME with the following: 
Mime
Net_DIME
Net_Socket
Net_POP3
SOAP.

You may need to configure the PEAR installer by editing pear.bat in the scripts directory and providing the proper paths for PHP.

You can also get the packages from the Web at http://pear.php.net/, or via CVS at http://cvs.php.net/.

Other Requirements

Any e-mail system will do just fine for the examples in this article, as long as you can set up an e-mail account and can access it via POP3. The e-mail system does not need to be on the same system that you use to run these examples.

Want to use a different language? You can find a list of SOAP implementations for just about any language and platform at SOAPWare.org. For some languages you will not have a choice of SOAP implementations. For those that do provide a choice, you should examine the interoperability reports for the implementation to see if it is complete enough to work for your application.

Conclusion

Hopefully, one thing you've picked up from this article is that SOAP over SMTP, or any other protocol, is indeed simple, and relatively the same as using SOAP over HTTP. However, I've left the hardest part for you to figure out on your own — what to do with it! The important thing to realize about SOAP is that it is no Holy Grail nor Killer App. It simply provides a standardized protocol for communication between systems.

I've focused mostly on PHP here, and for good reason; I've spent the last couple of months working on the PEAR SOAP implementation for PHP. However, there is nothing that prevents any of these techniques from being used with Perl, Python, Java, C#, or any other language, and there are SOAP libraries for most languages that support the SMTP protocol.

References

Box, D., D. Ehnebuske, G. Kakivaya, A. Layman, N. Mendelsohn, H.F. Nielsen, S. Thatte, and D. Winer, "Simple Object Access Protocol (SOAP) 1.1," May 2000
SMTP Transport Binding for SOAP 1.1: http://www.pocketsoap.com/specs/smtpbinding/
SOAP Messages with Attachments protocol: http://www.w3.org/TR/SOAP-attachments
Direct Internet Message Encapsulation protocol: http://gotdotnet.com/team/xml_wsspecs/dime/default.aspx
IETF draft on encapsulating SOAP in DIME: http://search.ietf.org/internet-drafts/draft-nielsen-dime-soap-00.txt
PEAR (PHP Extension and Application Repository): http://pear.php.net/
PHP: http://www.php.net/
SOAP::Lite: http://www.soaplite.com/
SOAP 1.2: http://www.w3.org/TR/soap12-part1/, http://www.w3.org/TR/soap12-part2/.
SOAP Builders: http://groups.yahoo.com/group/soapbuilders/
XML Web Services Interoperability Resources: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnsvcinter/html/globalxmlwebsrvinterop.asp
See www.xmethods.com for a listing of public web services.

Shane Caraveo is a senior developer at ActiveState, where he is the Technical Lead for Web Services and PHP. He also works on Komodo (the Mozilla-based IDE) and on PHP integration issues. He ported PHP to Windows, and currently maintains the PEAR::SOAP library for PHP.