Dr. Dobb's is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


Channels ▼
RSS

VNOS: The Visual Network Operating System


Dec01: Programmer's Toolchest

Mark is founder and CTO of Lone Wolf Technologies and can be reached at [email protected].


In 1988, my music and business partner David Warman and I were working on songs for my first CD. Like any other activity where you bring subtle, fleeting thoughts into reality, making music requires that you stay rooted in the creative process and not be distracted by technology. Whenever I had an inspiration, Dave would set up the studio and get the idea recorded. Sometimes this process would take an hour because of the myriad settings on each piece of gear, multiplied by the numerous pieces of gear in recording studios. By the time everything was ready to go, my creative inspiration had drifted off. After these sessions, Dave and I would talk about the problems we had tweaking the gear and orchestrating the connections and parameters, as well as the need for an all-encompassing system to make it easier to setup, use, and save gear settings.

Being hardware, software, and network protocol engineers, we decided to create technology for our personal studios that would help us stay focused on the task at hand — making music. Over the following months, we designed a fiberoptic system (to eliminate the dreaded ground loop hum) to tie together all the equipment, and devised a LAN protocol that was light, efficient, and isochronous. We called the system "MediaLink," and launched our company Lone Wolf Technologies (http://www.lonewolfusa.com/). MediaLink solved most of our problems and made working on music much easier, not to mention more fun. On top of the physical system, we created the Virtual Studio software interface, which provided a visual metaphor for studios and individual pieces of gear. To connect two pieces of gear, you simply use a mouse to draw a line between icons on the computer screen. To edit parameters on some gear, double click on its icon and a panel opens with all the settings. When everything is properly set up, you press a button to save the state of the entire system as a snapshot, creating what we call a "LanScape." The frustration was finally gone.

Friends heard about the system and, within months, the music industry was abuzz over MediaLink. Magazine articles about the system appeared worldwide, ranging from LAN Magazine to a cover feature in German Keyboards. Suddenly our system was being used in major studios and on tour with bands such as Herbie Hancock and INXS. Clearly, there was a need to be able to tie together many disparate products from multiple manufacturers, enabling them to work together as a coherent system.

The Birth of VNOS

Building on MediaLink, we moved into the pro audio market, augmenting the system with the more sophisticated Visual Network Operating System (VNOS) — a visual, event-driven, data-flow platform that lets you monitor and control physical devices on any network and quickly change or adapt those systems based on changes in the event data flowing through them. VNOS (http://www.vnos.net/) lets OEMs create small plug-in applications, called "panels," for each of their hardware products. These panels drop into the VNOS framework and integrate/interoperate with other VNOS-compatible products. In addition, VNOS lets sound engineers and venue designers create multiviewed graphical operating environments to model, monitor, and manage the networked equipment in their venue. Within months, a significant percentage of pro audio manufacturers adopted MediaLink and VNOS, and reengineered their gear to work in a tightly integrated system that could span entire stadiums. VNOS was installed in large venues such as the Houston Astrodome, the Grand Ole Opry, Portland's Rose Garden Stadium, and Vancouver's GM place, using gear from Bose, JBL, Altec Lansing, Yamaha, Carver, QSC, Rane, and more. It was also used on national tours by The Grateful Dead, Jimmy Buffet, and Brooks and Dunn, among others.

The principal breakthrough VNOS provided was the ability to alias panel control widgets — volume faders, volume unit (vu) meters, clipping lights, mute buttons, and the like — from any manufacturer's panel, and aggregate them onto a new VNOS view to build intent-oriented, human interfaces. Each view would contain only the mix of information and controls important for a specific task. In addition, you could paste an image into the background of your new view to give it the appearance of your own custom panel. Some venues used actual scanned photographs of their rooms and racks, and hid buttons and controls under the image to give a more direct and realistic look to the panel. Icons for the various views could be aliased into other views to provide an intuitive, hyperlinked pathway from any view to any other view as needed.

Subsequently, manufacturers in the home theater and home automation markets started using VNOS to integrate their disparate products together into seamless systems. It is also being applied in systems for e-commerce, sprinklers, heating and air-conditioning, X-10 lighting, and stock trading. Then, in 1994, we began adapting VNOS to IP-based networks and the Internet, because it was clear that the VNOS paradigm could be used to monitor and manage the flow of information and the devices on any IP-based network.

Throughout the development of VNOS, we have maintained a number of key characteristics for ease of use, performance, and multiplatform compatibility. For one thing, VNOS is written in ANSI C for portability. The VNOS core has a very small footprint in memory, taking up less than 400 KB. VNOS is an object-oriented, event-driven system with data flow characteristics. In spite of being a message-based system, it is very responsive due to a small, tight main-event processor.

Rather than the typical seven-layer OSI model, VNOS is designed with three layers:

  • The top layer is the perceptual plane where human interaction happens. Graphical widgets, physical hardware, buttons, and so on live in the perceptual plane. Each object in the perceptual plane has a small amount of local data containing its perceptual plane properties. All perceptual plane objects are optimized completely for humans.

  • The middle layer is the data plane, a distributed, cached database containing the mirrored data computers and VNOS needed for each object in the perceptual plane. The data plane is optimized for computers.

  • The lowest layer is the communications plane and forms the connectivity layer for the data between mirrored objects in computers and devices. When mirrored objects need to stay in sync because of a change event, an event packet is transferred.

VNOS has a series of Inventory windows that contain things it discovers. Discovered widgets are visible in the widgets palette, and devices and processes discovered on the network appear in the network inventory.

In short, VNOS is an operating framework for a visual, component-based system that lets you build higher level components from base components. Furthermore, VNOS is a recursive system, meaning most of VNOS is made from VNOS's basic control widgets. If you open a properties panel for a VNOS object, you see various property settings. If you highlight one of those properties and open it, you get a properties panel for that property, and so on. This is because, at startup, VNOS instantiates every object in every VNOS window and dialog from the widgets toolbox.

A collection of views and their widgets is called an "environment." In previous markets, VNOS was used to construct environments that represented all the elements of a stadium and any associations and interactions that were needed to monitor and control the various processes and devices as a whole.

VNOS and the Internet

In recasting VNOS for IP-based networks, we had to add widgets to the collection of base controls. We already had popup menus, text boxes, numeric displays, sliders, LED indicators, checkboxes, push buttons, timers, comparators, logic gates, and such. Next came the higher level widgets: TCP port, RS-232 port, value gate, URL cracker, text matcher, web server, mail sender, and others.

The higher level widgets are basically composed of primitive VNOS widgets with some added logic. If the properties of a particular widget don't provide enough — or correct — functionality, you can embed scripts into the widget to intercept its messages, letting you programmatically change the widget's basic behavior. (Think of it as visual Legos for networks and devices with scriptable message intercepts.) We also added a tag mechanism so that you can extract the value of a widget when code is executed. This lets a visual VNOS widget be referenced in lines of script. When the script is executed, the value of the referenced object is substituted inline. A typical Vtag looks like this: [vtag Path/to/Some/object/in/the/VNOS/hierarchy].

The TCP port widget is a visual human interface panel that listens for, or opens a connection on, an IP network — somewhat like a two-windowed terminal with TCP protocol state indicator LEDs and buttons to open or listen to a URL on a specific port. The various controls on the panel are used to set the URL and port that will be connected when the TCP port's Open button is pressed. Any incoming data shows up in the Receive Data text box. Data to be sent gets entered in the Send Data text box. After all operations are complete, pressing the Close button closes the TCP connection. With the aliasing capability of VNOS, you can drag various TCP port controls and indicators from the panel into views and draw lines to connect them to other things for automatic operation, or to build other widgets. For example, one of our interns recently built an SMTP mail sender without writing a single line of code. It was completely built using visual techniques and base VNOS widgets.

An interesting aspect of VNOS is that you construct your applications while the system is live and running — there is no write/compile/debug cycle. You can immediately see the result of every action at each step, which makes debugging straightforward. If you need more debugging detail as you are constructing something, simply open the event monitor widget, which is essentially a message-viewing data scope, and watch the message information and values flow through the system with pinpoint precision.

Building a World-Wide Monitoring System

I was tasked with building an application for a potential customer who wanted to see if VNOS was capable of monitoring the thousands of web servers the customer has in data centers around the world. Within 45 minutes, I built a working graphical application to monitor an HTTP-based web server using no lines of code. My application measures the opening latency and serving latency on a periodic basis and generates information for use in other areas. The networking aspects were hidden by the TCP port widget, and the human interface aspects were provided by VNOS's built-in widgets. If I had coded the same application in C, it would have taken days to get the networking, application logic, and human interface to the same level. When I finished, the HTTP monitor application became a widget for use in a larger application that could monitor a multitude of web servers in data centers around the world.

Using a top-down approach, I created a new view, called it "world view," and pasted a map of the world that I found on the Internet into this view; see Figure 1. This gave me a top-level view of the cities that contained our customer's data centers. I then placed hidden view widgets under each monitored city that would open a view window into each city's data center floor plans. A LED status indicator would be placed over each city in the world view to show the aggregate status of all the servers in each city. The customer would have a view to monitor the status of thousands of servers in one focused window.

Once this overall view of the network was created, I needed a widget to monitor the HTTP status. A new view was instantiated and named "HTTP monitor." The key widget for this phase was a TCP port (Figure 2), which was cloned into the HTTP monitor view from the widgets palette and let me open a TCP connection to a web server. I entered the appropriate header information in the Transmit Data text box of the TCP port widget to make VNOS generate an HTTP request, masquerading as Netscape Navigator. Upon entering the address of one of my web servers, I could manually press the Open button on the TCP port and get a web page back in the TCP widget's Receive Data text box. I next needed to automate the requests and check for valid returns.

A timer widget was used to create periodic events to trigger the TCP port widget, triggering it to collect the web page from the server. The timer's status light was cloned into the HTTP monitor's graphical source view (Figure 3) and linked to the TCP widget's Open button to automate this process. When a web page is returned from the server, a text-matcher widget is used to check for the valid "http/1.0 200 OK" string. The output from this widget clears an error counter that is incremented every time the timer widget initiates a check. A number of other widgets were applied to finish out the HTTP monitor that can monitor many aspects of the server, including TCP opening latency and web server response latency.

After finishing the graphical source code and validating that everything was working properly, I aliased the necessary indicators and controls from the check view into a new view. This view would provide the UI for the HTTP monitor. I then created a graphic in Photoshop to finish up the panel; see Figure 4.

To quickly and visually identify server status, an image that looks like a rack of servers was found and the HTTP monitor view widget was cloned in each server location in the rack and each view icon was hid under the graphic. This lets users click on any server in the rack to open its associated HTTP monitor. In addition, I aliased each monitor's status light and placed it on the server rack for easy monitoring at the rack level. Each rack was cloned into the floorplan of each city (Figure 5) and an aggregated status light of each rack into a master status light for each city, was created using the min/max widget. Finally, I aliased the master status light for each city onto the world map view to provide worldwide server monitoring.

If any server in any city fails, the status bubbles up to the master status light for that city, and with three clicks the Network Operations Center (NOC) operator will be able to drill down and determine what is causing the server to fail.

A New Start

Versions of VNOS are being made available for UNIX, MacOS X, and Windows. VNOS environment files (applications, widgets, connections, and scripts) are saved in XML format so that they can be moved transparently from platform to platform. All platforms running VNOS can interact seamlessly as a distributed application. Views can be opened into a remote VNOS's object space, and a widget can be instantiated there by dragging and dropping from your local widgets palette. The XML description of the widget is transferred and instantiated in the remote VNOS and is immediately usable. In essence, the remote VNOS is a daemon with a "box of objects" that are persistent component-based applications, executing and doing their job in a greater system. It is possible to make powerful data collector/aggregators in this manner.

Many organizations are using four or more disparate software products to collect data on the operation and health of their network. These disparate products have various areas of usefulness, but can't be interconnected easily. Most of these products are good at collecting data, but they fall short when it comes to visually representing it or creating policies. VNOS can be used to integrate the many different networked devices such as routers, servers, HVAC, lighting, and the like, and the various software systems you might have in place, melding them into a complete and cohesive systems.

The Future

People are starting to talk about a concept called the "X-Internet," where a mesh or lattice of distributed applications all share and process data as a fabric — something we have been envisioning and building for a decade. While the Web is useful for reading published documents (such as the news), its major limitation has always been that information in the browser devolves into only graphical pixels long before you get it. The actual value of the data is lost at the server. With VNOS widgets, any data value stays a value, even after being displayed. This provides the ability to drive additional actions anywhere in your network.

If you draw a line from a remote stock market feed widget to a local display widget, you see its value in the format you choose. You can then draw a line from your local display widget to a comparator widget, for example, and the value will continue on its way. The comparator could then tell you, by sending you an e-mail using the e-mail sender widget, that your stock has reached a certain value and you should make a trade. Even better, you could create a widget that makes the trade automatically when certain conditions are met. The pixels in the browser don't become the final dead end for any useful information — they stay live, useful, and manipulable.

DDJ


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.