Learning a Lesson from the Past

Fortunately, we've seen this exact suite of problems in the past, and can learn a lesson from one of the great success stories of the computer industrySQL. During the 1970s, the focus on databases brought to light weaknesses in the programming languages of the day. While adequate for many things, they simply were far from ideal for database work. Database programs took too long to write. The code required was unnecessarily lengthy. Each database vendor's product had a proprietary means of being accessed. The skill level required for database work was too high.

Structured Query Language (SQL) was a solution to the database programming problem. Lengthy code was replaced with short SQL queries. Database programming became much faster and simpler. Differences between database products were hidden behind a layer of abstraction, making it possible for companies to switch database vendors or even intermix them without concern. With SQL, a junior- level programmer could do more database programming than in the past. Despite its name, SQL became more than just a query language; it eventually became possible to perform database actions as well. The success of SQL can't be denied. More than 25 years later, we take it for granted.

We can draw an exact parallel between the database programming problems of yesteryear and network/Internet programming problems of today. The symptoms are the same. Today's situation cries out for a similar solution to SQL. NQL was created to be that solution. While NQL is not grammatically similar to SQL, it shares SQL's characteristics: Programs are short, code is easy to write and read, development is rapid, and even less experienced people can use it.

Introducing NQL

NQL is a scripting language for developing connected applications. It has building blocks for bots and agents, and a strong emphasis on communications, data conversion, automation, and intelligent behavior. NQL is currently available for Windows 98/NT/2000. A Java edition of NQL is nearing release, which is initially being qualified for Linux, Macintosh, and Solaris. NQL Inc. is also contemplating porting NQL to PDA platforms such as Palm and Windows CE. Trial versions of NQL are available at http://www.nqli.com/.

Listing One is a sample script that retrieves news articles from the CNN main page and outputs headlines and links as XML. I'll trace through the script line-by-line later on, but even at first glance you should find it easy to follow.

NQL stresses simplicity. Statement keywords are made up of words found in the dictionary. Odd punctuation is avoided. The syntax of the language is simple: one statement per line. If a statement requires multiple parameters, they are separated by commas. String constants are enclosed in single quotes (') or double quotes ("). Lines beginning with // are single-line comments. Multiline /* and */ comments can also be used as in C, C++, and Java.

Like most languages, NQL supports standard control-flow statements including if, for, and while. Curly braces are used to enclose statement blocks. One of the control-flow statements is thread, which executes a block of code in a separate thread of execution.

NQL variables are not strongly typed and may contain any kind of data. You can require variables to be declared or not, as you see fit, and you can choose from three kinds of error handling: program halt, executing error-trapping code, or ignoring errors altogether. In addition to variables, NQL can also hold data on a stack. The stack is what allows NQL's statements to interact with each other smoothly.

NQL supports a range of communications' built-in protocols: HTTP and HTTPS for web access; FTP for moving files; MAPI, POP3, and SMTP for sending and reading e-mail; ODBC, JDBC, OLEDB, and ADO for database access; NNTP for reading newsgroups; LDAP for accessing directory servers; SNMP for network monitoring; and TELNET for interacting with legacy systems. Communications in NQL is accomplished with a small set of statement keywords designed to logically fit together. Differences in protocols are hidden. For example, the method of reading MAPI e-mail and the method of reading POP3 e-mail is the same. Listing Two(a) reads unread e-mail, while Listing Two(b) reads through the articles in a newsgroup. Finally, Listing Two(c) logs on to a Linux system and captures a directory listing.

NQL scripts can work with many kinds of information, including text, web pages, XML, and multimedia types. NQL's variables and stack can hold string or binary information of arbitrary size. NQL has particularly strong support for XML, with direct capabilities for reading, parsing, and writing XML.

Because connected applications frequently need to be automated, NQL includes features for scheduling. Scripts can be set to execute at a specific date and time, and to run again at regular intervals. Scripts code can wait for a specific time to arrive, wait for a specific interval to pass, or repeat execution of a code block at regular intervals. Scripts can declare time limits.

For intelligent behavior, NQL supports practical outpouring of AI research, including fuzzy logic and neural networks. The use of fuzzy logic can greatly improve the decision making of agents. NQL's fuzzy logic support includes language constructs for declaring condition evaluation functions and performing logical operations on fuzzy values. Neural networks have great application in predictive systems and pattern recognition. NQL makes it simple to create, train, save, and run neural networks, as Listing Three illustrates.

Paradoxically, NQL's compactness of code is due in part to two notions borrowed from assembly language: stacks and condition codes. A stack is a list of values, kind of like a stack of dinner plates. A new value may be pushed onto the stack. The top value on the stack may be popped off of the stack. Unlike variables, which hold information by name, a stack can hold lots of information as a temporary holding area. For connected applications, stacks can be useful for recursive operations such as crawling web sites. NQL uses the stack as a way to let multiple statements work on the same piece of information.

NQL allows every statement to set a success or failure condition. For example, a pattern match statement results in a successful condition if a match is found, or a failed condition if a match is not found. Condition codes allow NQL's statements to be combined with control-flow statements in a compact way.

Listing One: Line by Line

Going back to Listing One, in line 1, a get statement retrieves a web page. Where does the web page go? On the stack. Line 2 is a match statement that searches for its pattern in the top item on the stack, which just happens to be the HTML web page that was just retrieved. If the pattern match was successful, three things happen: The variable names in the pattern (link and headline) are set to values from the web page; the HTML on the stack is shortened in anticipation of more pattern matching; and a success condition is set. Line 3 is a while statement, which may look a bit odd since no condition is specified. This means that the success/failure condition (set by the previous statement) is what determines whether the block of code is executed. In the code block, line 5 contains an output statement, which outputs the variables headline and link as XML. Line 6 is a nextmatch statement, which repeats the previous pattern match — this time, on the remainder HTML on the stack, which will find the next match. Like the match statement in line 2, nextmatch sets variables and a success condition if it finds another match, and will shorten the HTML on the stack again. Thus, the while loop continues as long as there are matches. That's a lot of explanation, but the code itself is quite small and fairly clear, even to someone who hasn't previously been exposed to NQL.

The match Statement

The action of the match statement deserves some explanation. The pattern includes HTML tags and text, but may also contain wildcards. Although an asterisk may be used to indicate a wildcard area, it is more common to specify a variable name enclosed in curly braces. When a match occurs, variables are created automatically with these names that contain data from the pattern match. This is called "automatic variable mapping." Database queries also utilize automatic variable mapping, as Listing Four shows. For each record retrieved, the fields are moved into variables and are ready for immediate use.

NQL is designed to play a role wherever it is needed. An NQL script can run on the desktop, on a middleware system, on a server, or on a web server as a CGI application. You can also combine NQL with another programming language. Virtually all languages, including VB, ASP, C++, and Java, can call NQL as a component. This lets you continue using your preferred development tools and still tap into the power of NQL when you need it.

NQL can be used to drive web applications. NQL program code and HTML can be interspersed, just as is the case with ASP and JSP. In this context, NQL provides mobile device support for Palm VIIs, Internet-enabled smart phones, and Pocket PCs. NQL has built-in features for recognizing these devices and responding appropriately in their native content language. This means that a single NQL script can be accessed through a myriad of devices. NQL can also detect these devices' serial numbers, which is handy for mapping them to specific user IDs.

Conclusion

NQL was designed for same-day agent development, and it delivers on that promise. The built-in functionality, simple grammar, and compact design create a rapid development environment for realizing connected applications.

DDJ

Listing One

get "http://www.cnn.com"
match '<li><a href="{link}">{headline}</a>'
while
{
    output headline, link
    nextmatch
}

Back to Article

Listing Two

openmail
firstunread
while
{
    ...do something with mail message...
    nextmessage
}
closemail

(b)
opennews "news.server.com", "alt.news.topic"
firstarticle
while
{
    ...do something with newsgroup article...
    nextarticle
}
closenews

(c)
openterm "linuxserver2"
waitterm "login:"
sendterm "jsmith\r"
waitterm "password:"
sendterm "johnny\r"
waitterm "$ "
clearterm
sendterm "ls -l\r"
waittermquiet 2
getterm
closeterm
 ...do something with captured session data...




Back to Article

Listing Three



///////////////////////////////////////////////////////////////////////////
// Script name: geology
// Description: Demonstrates use of a neural network in the area of geology.
//           Given indirect measurements, determines lithology.
//           Inputs:  Gamma Ray, Neutron, and Density.
// Inputs:   Gamma Ray log value, Neutron log value, and Density log value
// Outputs:  Identification (Shale, Dolomite, Limestone, Sandstone)
// Notes:    The first time the script is run, the network will be created,
//             trained, and saved (in geology.nnf). Thereafter, running the
//             script will process input and generate output.
//           The file required for training is geology.trn (text data file).
//           The file required for processing is geology.inp (text data file).
//           Results are output to the file geology.out (text data file).
////////////////////////////////////////////////////////////////////////////

//if this is the first time this script is being run,
//create, train, and save the neural network

lookup "geology.nnf"
else
{
  setneural 0.45, 0.90
  createneural "backprop", 3, 4, 4
  trainneural "geology.trn"
  saveneural "geology.nnf"
  closeneural
  show "Network has been created and saved. Run script again to process data."
  end
}
//open previously saved neural network and run it against real data
openneural "backprop", "geology.nnf"
processneural "geology.inp", "geology.out"
closeneural
show "Network has been run against input data. Processing is complete."





Back to Article

Listing Four



opendb "customers"
select "SELECT Customers WHERE BALANCE > 0.00;"
while
{
    show CustID, Company, Addr1, Addr2, City, State, Zip, Phone
    nextrecord
}




Back to Article




        

Terms of Service | Privacy Statement | Copyright ©  2024 UBM Tech, All rights reserved.