Dr. Dobb's is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Channels ▼

RESTful Web Service in Go Powered by the Google App Engine

It's Alive! Alive!

You'll need to download and install the Google App Engine SDK for Go. Next, you need to add its installation directory to your paths. GAE SDK for Go is available for Windows, Linux, and Mac. I used Mac, but also tested on a VM running Ubuntu on Windows.

Once you have done this, create a directory somewhere, for example, /Users/alexander/Projects/usvisa. Inside this directory, create another subdirectory also called usvisa/ and place the usvisa.go file there (the full name should be /Users/alexander/Projects/usvisa/usvisa/usvisa.go).

Now we need a configuration file. Each GAE project has a file describing its properties. Create a /Users/alexander/Projects/usavisa/app.yaml file and put the following into it:

    application: usvisa-service
	  version: 1
    runtime: go
    api_version: go1


    - url: /batch/.*
      script: _go_app

    - url: /.*
      script: _go_app
      login: admin

The name of the application is usvisa-service. Of course, if you are going to give it a real try deploying to GAE, I would recommend changing the name because it could be already occupied. You need to go to the App Engine, create an application with your unique name, and update the app.yam file accordingly.

Also important in the handlers descriptions are the - url: sections, which define a mask of URLs and their properties. We only have three URLs: /batch/, /action/print/, and /action/refresh/. We indicate that the first one is visible to everybody, but the last two require administrator permissions (literally, they will be visible to you only) by adding the login: admin option.

Recall that we talked about scheduled tasks. In our application, it would be convenient if the /action/refresh/ URL could be invoked automatically on a regular basis, in a way similar to the UNIX cron service. Let's create another file, called cron.yaml, place it in the same directory as app.yaml, and put the following into it:

    - description: data refresh
      url: /action/refresh/
      schedule: every 60 minutes

This specifies that the /action/refresh/ URL will be invoked by GAE every hour (don't skip the trailing "/" at the third line).

Let's review the directory structure:

  • /Users/alexander/Projects/usvisa/app.yaml
  • /Users/alexander/Projects/usvisa/cron.yaml
  • /Users/alexander/Projects/usvisa/usvisa/usvisa.go

Now everything is ready. Prior to uploading to the GAE cloud, you can test your application locally. Execute the following:

    cd /Users/alexander/Projects/usvisa
    dev_appserver.py .

If everything is correct, the last few lines in your terminal console will contain something like this:

    INFO     2012-06-10 15:44:14,864 dev_appserver_multiprocess.py:647] Running application dev~usvisa-service on port 8080: http://localhost:8080
    INFO     2012-06-10 15:44:14,864 dev_appserver_multiprocess.py:649] Admin console is available at: http://localhost:8080/_ah/admin

Go to the http://localhost:8080/_ah/admin URL to make sure that the local development server is up the running. Then try the http://localhost:8080/action/print/ URL. You should see something like the following in the browser window:

 # records: 0, updated: 0001-01-01T00:00:00Z, now: 2012-06-10T15:44:58Z, age: 2259641h1m17.462093152s, Original PDF

This simply means that we have no information in Memcache so far. In the terminal console, you also should see the lines like this:

    2012/06/10 15:44:58 INFO: > Print
    2012/06/10 15:44:58 INFO: No [table] record found in cache, [memcache: cache miss]
    2012/06/10 15:44:58 INFO: Read 0 records, updated at []
    2012/06/10 15:44:58 INFO: Print finished

Now let's load up the data by invoking the http://localhost:8080/action/refresh/ URL in the browser. In the console, it should print the following (in your case, the particular numbers could be different):

    2012/06/10 15:45:36 INFO: > Refresh
    2012/06/10 15:45:36 INFO: - Refreshing a table
    2012/06/10 15:45:36 INFO: Started downloading
    2012/06/10 15:45:42 INFO: Loaded 2212885 bytes
    2012/06/10 15:45:43 INFO: Storing a table to memcache
    2012/06/10 15:45:43 INFO: Storing 6037 records, updated at [2012-06-10T15:45:43Z]
    2012/06/10 15:45:43 INFO: Stored
    2012/06/10 15:45:43 INFO: Refresh finished

Now invoke http://localhost:8080/action/print/ again. The console output should be something like:

    2012/06/10 15:46:18 INFO: > Print
    2012/06/10 15:46:18 INFO: Read 6037 records, updated at [2012-06-10T15:45:43Z]
    2012/06/10 15:46:18 INFO: Print finished

And in the browser window, you may get something like this (indicating that the age of our data is ~35 seconds):

    # records: 6037, updated: 2012-06-10T15:45:43Z, now: 2012-06-10T15:46:18Z, age: 35.481546s, Original PDF

    20010070231	 Pending, 7-Jan-10 
    20010072631	 Pending, 7-Jan-10
    20010072641	 Pending, 7-Jan-10
    20080791731	 processed, 30-Sep-11

Finally, let's check the batch information retrieval by invoking http://localhost:8080/batch/20080791731. In my particular example, it printed:


In the console, it printed:

    2012/06/10 15:50:20 INFO: > Batch: [20080791731]
    2012/06/10 15:50:20 INFO: Read 6037 records, updated at [2012-06-10T15:45:43Z]
    2012/06/10 15:50:20 INFO: Last updated: 2012-06-10 15:45:43 +0000 UTC, now 2012-06-10 15:50:20.386815 +0000 UTC, gap 1h0m0s
    2012/06/10 15:50:20 INFO: Check time 2012-06-10 16:45:43 +0000 UTC
    2012/06/10 15:50:20 INFO: [{processed 30-Sep-11}]
    2012/06/10 15:50:20 INFO: Batch done

Now we need to check the situation when the contents of the Memcache record table disappear. Go to http://localhost:8080/_ah/admin, Memcache Viewer, and delete the record called table, then invoke http://localhost:8080/batch/20080791731 again. It should come up with the same output in the browser window. In the console output (below), we see that the record had not been found, the table was reloaded, and the batch information was retrieved.

    2012/06/10 15:55:52 INFO: > Batch: [20080791731]
    2012/06/10 15:55:52 INFO: No [table] record found in cache, [memcache: cache miss]
    2012/06/10 15:55:52 INFO: - Refreshing a table
    2012/06/10 15:55:52 INFO: Started downloading
    2012/06/10 15:55:58 INFO: Loaded 2212885 bytes
    2012/06/10 15:55:59 INFO: Storing a table to memcache
    2012/06/10 15:55:59 INFO: Storing 6037 records, updated at [2012-06-10T15:55:59Z]
    2012/06/10 15:55:59 INFO: Stored
    2012/06/10 15:55:59 INFO: Read 6037 records, updated at [2012-06-10T15:55:59Z]
    2012/06/10 15:55:59 INFO: Last updated: 2012-06-10 15:55:59 +0000 UTC, now 2012-06-10 15:55:59.702255 +0000 UTC, gap 1h0m0s
    2012/06/10 15:55:59 INFO: Check time 2012-06-10 16:55:59 +0000 UTC
    2012/06/10 15:55:59 INFO: [{processed 30-Sep-11}]
    2012/06/10 15:55:59 INFO: Batch done

That's it. Everything has been tested. Interestingly, you don't need to stop the development server when you change the source of your project. The GAE development server automatically detects changed files and rebuilds them on-the-fly upon the next request.

It is time now to deploy our nifty Web service to the GAE cloud. Stop your development server by CTRL-C and type in the following command:

 appcfg.py update .

If your application is properly created via the GAE administrative console and app.yaml contains the correct information, you will be prompted for your Google credentials, then the application will uploaded and install automatically. Afterwards, you can try exactly the same scenarios that we used on the local server. Simply replace localhost:8080 in your application URL (for example, usvisa-service.appspot.com).


Summing up, I have developed a Web service in Go that does useful data retrieval and exposes the results via a RESTful API, and, now, I've rolled it out on the Google App Engine cloud. If your volumes (traffic and data) are less than a certain limit, you can use GAE for free. Beyond that, Google will start charging you under their flexible pricing model. Because different GAE APIs are charged differently, it is better to always choose APIs that suit your requirements best.

My full implementation is running here. The full project is available on GitHub.

Alexander Demin is a software engineer who works in London.

Related Reading

Go Tutorial: Object Orientation and Go's Special Data Types

Getting Going with Go

A Brief Tour of the Go Standard Library

Why Not Go?

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.