Continuing the Java network communication theme, let's examine how to make HTTP requests from a Java application, including how to add header parameters and proxy information. I've made requests to HTTP servers from Java in the past, but believe it or not I only recently had to consider adding proxy server information. As usual for me, I started at a low level, making the requests using a Java Socket.

To begin, you need to parse the given URL request to extract the host, path, port, and protocol (i.e., HTTP, HTTPS, and so on). For example, let's parse the host and path first:

            String urlStr = "http://www.someserver.com/getdata?param1=abc¶m2=xyz"; // some URL
            URI uri = new URI( urlStr); 
            String host = uri.getHost( ); 
            String path = uri.getRawPath( ); 
            if (path == null || path.length( ) == 0) {
                path = "/";
            } 

            String query = uri.getRawQuery( ); 
            if (query != null && query.length( ) > 0) {
                path += "?" + query;
            } 

Next, let's extract the protocol and port, and make sure they match:

            String protocol = uri.getScheme( ); 
            int port = uri.getPort( ); 
            if (port == -1) {
                if (protocol.equals("http")) { 
                    port = 80; // http port 
                }
                else if (protocol.equals("https")) {
                    port = 443; // https port 
                }
                else {
                    return null;
                }
            }

Now that the required information has been extracted, we need to do three main things. First, make a socket connection to the server; second, send a correctly formatted HTTP request; third, listen for the response. Connecting is simple; just create a new Java Socket with the host and port:

Socket socket = new Socket( host, port ); 

Sending the request is a little trickier. If the HTTP server requires authentication for the request, you need to encode this so it isn't sent in plain text (and hence viewable by anyone looking at the network traffic). The HTTP specification requires you to format it as ":", as in "ericbruno:mypassword" (see http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html). To encode it using Base64 encryption, I used the Apache Commons Codec library. Here's the code:

            import org.apache.commons.codec.binary.Base64;
            …

            String username = "ericbruno";
            String password = "mypassword";
            String auth = username + ":" + password;
            String encodedAuth = Base64.encodeBase64String(auth.getBytes());

The HTTP request format for this request needs to look like this, according to the spec:

            GET /getdata?param1=abc¶m2=xyz HTTP/1.1
            Host: www.someserver.com
            Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=
            Connection: close

The code to send this request uses the Socket's OutputStream and simply "prints" the request as a String to it:

            PrintWriter request = new PrintWriter( socket.getOutputStream() );
            request.print(  "GET " + path + " HTTP/1.1\r\n" + 
                                   "Host: " + host + "\r\n" + 
                                   "Authorization: Basic " + encodedAuth + "\r\n" +
                                   "Connection: close\r\n\r\n"); 
            request.flush( ); 

The "\r\n" (line feed, new line) combinations at the end of each line in the String ensure that the request meets the HTTP specification formatting requirements. To get the response, use the Socket's InputStream and read the resulting text line-by-line:

            InputStream inStream = socket.getInputStream( ); 
            BufferedReader rd = new BufferedReader(
                    new InputStreamReader(inStream));
            String line;
            while ((line = rd.readLine()) != null) {
                // ...
            }

A Better Way

Using this code works, and it's obvious what's happening since it involves low-level socket communication. There are cases when understanding the low-level workings is important, and for some reason I tend to feel comfortable working at this level. I wrote the basis for this code years ago, and I simply copy/paste from my personal code library each time I need it. However, I ran into a snag when I had to add the proxy information. I'm sure it can be done using this low-level code, but I decided to find a better way. For me, that search ended with the Apache HTTP Client classes, part of the Apache HTTP Components library. Using this library, the code to make the request via a proxy server looks like this:

            DefaultHttpClient httpclient = new DefaultHttpClient();
            if ( useProxy == true ) {
                HttpHost proxy = new HttpHost(proxyStr, 80, "http");
                httpclient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
            }

            HttpGet httpget = new HttpGet(urlStr);
            httpget.addHeader("Authorization", "Basic " + encodedAuth);

            HttpResponse response = httpclient.execute(httpget);

Here, the proxy I needed to use was a basic HTTP proxy, but there's support for others (i.e., SOCKS). If you don't require a proxy, simply set the useProxy flag to false. The response is returned when you make the request via the execute() method. Here's the code to extract the call status (i.e., "HTTP/1.1 200 (OK)"), the headers, and the body of the response:

            String status = response.getStatusLine().toString();

            Header[] headers = response.getAllHeaders();

            HttpEntity entity = response.getEntity();
            BufferedReader rd = new BufferedReader(
                    new InputStreamReader(entity.getContent()));
            String line;
            while ((line = rd.readLine()) != null) {
                // ...
            }

That's much better! Happy coding!
-EJB