It's Alive! Alive!
You'll need to download and install the Google App Engine SDK for Go. Next, you need to add its installation directory to your paths. GAE SDK for Go is available for Windows, Linux, and Mac. I used Mac, but also tested on a VM running Ubuntu on Windows.
Once you have done this, create a directory somewhere, for example, /Users/alexander/Projects/usvisa. Inside this directory, create another subdirectory also called usvisa/ and place the usvisa.go file there (the full name should be /Users/alexander/Projects/usvisa/usvisa/usvisa.go).
Now we need a configuration file. Each GAE project has a file describing its properties. Create a /Users/alexander/Projects/usavisa/app.yaml file and put the following into it:
application: usvisa-service
version: 1
runtime: go
api_version: go1
handlers:
- url: /batch/.*
script: _go_app
- url: /.*
script: _go_app
login: admin
The name of the application is usvisa-service. Of course, if you are going to give it a real try deploying to GAE, I would recommend changing the name because it could be already occupied. You need to go to the App Engine, create an application with your unique name, and update the app.yam file accordingly.
Also important in the handlers descriptions are the -
url: sections, which define a mask of URLs and their properties. We
only have three URLs: /batch/, /action/print/, and /action/refresh/. We
indicate that the first one is visible to everybody, but the last two require
administrator permissions (literally, they will be visible to you only) by
adding the login: admin option.
Recall that we talked about scheduled tasks. In our application, it would
be convenient if the /action/refresh/ URL could be invoked automatically on a
regular basis, in a way similar to the UNIX cron service. Let's
create another file, called cron.yaml, place it in the same directory as
app.yaml, and put the following into it:
cron:
- description: data refresh
url: /action/refresh/
schedule: every 60 minutes
This specifies that the /action/refresh/ URL will be invoked by GAE every hour (don't skip the trailing "/" at the third line).
Let's review the directory structure:
- /Users/alexander/Projects/usvisa/app.yaml
- /Users/alexander/Projects/usvisa/cron.yaml
- /Users/alexander/Projects/usvisa/usvisa/usvisa.go
Now everything is ready. Prior to uploading to the GAE cloud, you can test your application locally. Execute the following:
cd /Users/alexander/Projects/usvisa
dev_appserver.py .
If everything is correct, the last few lines in your terminal console will contain something like this:
INFO 2012-06-10 15:44:14,864 dev_appserver_multiprocess.py:647] Running application dev~usvisa-service on port 8080: http://localhost:8080
INFO 2012-06-10 15:44:14,864 dev_appserver_multiprocess.py:649] Admin console is available at: http://localhost:8080/_ah/admin
Go to the http://localhost:8080/_ah/admin URL to make sure that the local development server is up the running. Then try the http://localhost:8080/action/print/ URL. You should see something like the following in the browser window:
# records: 0, updated: 0001-01-01T00:00:00Z, now: 2012-06-10T15:44:58Z, age: 2259641h1m17.462093152s, Original PDF
This simply means that we have no information in Memcache so far. In the terminal console, you also should see the lines like this:
2012/06/10 15:44:58 INFO: > Print
2012/06/10 15:44:58 INFO: No [table] record found in cache, [memcache: cache miss]
2012/06/10 15:44:58 INFO: Read 0 records, updated at []
2012/06/10 15:44:58 INFO: Print finished
Now let's load up the data by invoking the http://localhost:8080/action/refresh/ URL in the browser. In the console, it should print the following (in your case, the particular numbers could be different):
2012/06/10 15:45:36 INFO: > Refresh
2012/06/10 15:45:36 INFO: - Refreshing a table
2012/06/10 15:45:36 INFO: Started downloading
2012/06/10 15:45:42 INFO: Loaded 2212885 bytes
2012/06/10 15:45:43 INFO: Storing a table to memcache
2012/06/10 15:45:43 INFO: Storing 6037 records, updated at [2012-06-10T15:45:43Z]
2012/06/10 15:45:43 INFO: Stored
2012/06/10 15:45:43 INFO: Refresh finished
Now invoke http://localhost:8080/action/print/ again. The console output should be something like:
2012/06/10 15:46:18 INFO: > Print
2012/06/10 15:46:18 INFO: Read 6037 records, updated at [2012-06-10T15:45:43Z]
2012/06/10 15:46:18 INFO: Print finished
And in the browser window, you may get something like this (indicating that the age of our data is ~35 seconds):
# records: 6037, updated: 2012-06-10T15:45:43Z, now: 2012-06-10T15:46:18Z, age: 35.481546s, Original PDF
20010070231 Pending, 7-Jan-10
20010072631 Pending, 7-Jan-10
20010072641 Pending, 7-Jan-10
20080791731 processed, 30-Sep-11
...
Finally, let's check the batch information retrieval by invoking http://localhost:8080/batch/20080791731. In my particular example, it printed:
processed
30-Sep-11
In the console, it printed:
2012/06/10 15:50:20 INFO: > Batch: [20080791731]
2012/06/10 15:50:20 INFO: Read 6037 records, updated at [2012-06-10T15:45:43Z]
2012/06/10 15:50:20 INFO: Last updated: 2012-06-10 15:45:43 +0000 UTC, now 2012-06-10 15:50:20.386815 +0000 UTC, gap 1h0m0s
2012/06/10 15:50:20 INFO: Check time 2012-06-10 16:45:43 +0000 UTC
2012/06/10 15:50:20 INFO: [{processed 30-Sep-11}]
2012/06/10 15:50:20 INFO: Batch done
Now we need to check the situation when the contents of the Memcache record
table disappear. Go to http://localhost:8080/_ah/admin, Memcache
Viewer, and delete the record called table, then invoke
http://localhost:8080/batch/20080791731 again. It should come up with the same
output in the browser window. In the console output (below), we see that the
record had not been found, the table was reloaded, and the batch information
was retrieved.
2012/06/10 15:55:52 INFO: > Batch: [20080791731]
2012/06/10 15:55:52 INFO: No [table] record found in cache, [memcache: cache miss]
2012/06/10 15:55:52 INFO: - Refreshing a table
2012/06/10 15:55:52 INFO: Started downloading
2012/06/10 15:55:58 INFO: Loaded 2212885 bytes
2012/06/10 15:55:59 INFO: Storing a table to memcache
2012/06/10 15:55:59 INFO: Storing 6037 records, updated at [2012-06-10T15:55:59Z]
2012/06/10 15:55:59 INFO: Stored
2012/06/10 15:55:59 INFO: Read 6037 records, updated at [2012-06-10T15:55:59Z]
2012/06/10 15:55:59 INFO: Last updated: 2012-06-10 15:55:59 +0000 UTC, now 2012-06-10 15:55:59.702255 +0000 UTC, gap 1h0m0s
2012/06/10 15:55:59 INFO: Check time 2012-06-10 16:55:59 +0000 UTC
2012/06/10 15:55:59 INFO: [{processed 30-Sep-11}]
2012/06/10 15:55:59 INFO: Batch done
That's it. Everything has been tested. Interestingly, you don't need to stop the development server when you change the source of your project. The GAE development server automatically detects changed files and rebuilds them on-the-fly upon the next request.
It is time now to deploy our nifty Web service to the GAE cloud. Stop your development server by CTRL-C and type in the following command:
appcfg.py update .
If your application is properly created via the GAE administrative console and app.yaml contains the correct information, you will be prompted for your Google credentials, then the application will uploaded and install automatically. Afterwards, you can try exactly the same scenarios that we used on the local server. Simply replace localhost:8080 in your application URL (for example, usvisa-service.appspot.com).
Conclusion
Summing up, I have developed a Web service in Go that does useful data retrieval and exposes the results via a RESTful API, and, now, I've rolled it out on the Google App Engine cloud. If your volumes (traffic and data) are less than a certain limit, you can use GAE for free. Beyond that, Google will start charging you under their flexible pricing model. Because different GAE APIs are charged differently, it is better to always choose APIs that suit your requirements best.
My full implementation is running here. The full project is available on GitHub.
Alexander Demin is a software engineer who works in London.
Related Reading
Go Tutorial: Object Orientation and Go's Special Data Types
A Brief Tour of the Go Standard Library
