BioMOBIUS Software Environment
Drag-and-drop environments ideally support the concept of rapid application prototyping through the development of applications, by selecting blocks from a palette of capabilities categorized by function, such as input/output, data management, digital signal processing (DSP), etc. Sophisticated applications can be quickly assembled, including hardware interfacing and real-time data processing, by using the functional blocks that are connected in sequence to represent the real-time data flow.
EyesWeb, developed by the University of Genoa, was selected as the drag-and-drop GDE [43, 44]. Originally focused on visual arts applications, EyesWeb provides fusion of motion and music for emotion capture during artistic performance . It is also utilized to develop applications based on multimedia techniques for therapy and rehabilitation; e.g., Parkinson's, Autism [46, 47]. The EyesWeb software platform was originally developed in EU 5th Multisensory Expressive Gesture Applications (MEGA) and 6th Framework Programme, the Tangible Acoustic Interfaces for Computer Human Interaction (TAI-CHI). The EyesWeb environment has been adapted and enhanced by TRIL for biomedical oriented applications as in Figure 1.
The EyesWeb environment comprises a GDE that supports drag and drop application development in the form of patches. BioMOBIUS provides a Graphical User Interface (GUI) development tool that enables the user to design an application to control a patch and view its output. The GUI Designer is built by using the JUCE C++ class library . This is a cross-platform, high-level, graphical API and provides the potential to develop BioMOBIUS applications on non-Windows platforms.
Use of the GUI development tool allows the user to be abstracted away from the underlying complexities of the connected patch. The resultant application is called a GUI application. The runtime environment within BioMOBIUS comprises two key components:
- Kernel Runtime Server. The EyesWeb Kernel Runtime is a Windows component that supports the execution of EyesWeb patches outside of the EyesWeb GDE. It is used in conjunction with the BioMOBIUS Runtime to execute GUI applications and patches as if they are native Windows applications.
- BioMOBIUS GUI Runtime. This is a Windows console application that supports the execution of a BioMOBIUS application outside of the GUI development tool. Typically, a BioMOBIUS application is first developed and tested within the GUI development tool. Thereafter, the BioMOBIUS Runtime is used to run the application as a Windows executable in conjunction with the EyesWeb Kernel Runtime component, via a TCP/IP connection that supports execution of the patch and GUI on two physically separated networked systems, if required.
Both patches and GUI applications are defined in XML and interpreted by their respective runtimes. A number of benefits are achieved by separating these two frameworks:
- Runtime efficiency. Applications consume less system resources; for example, output and display rendering is automatically disabled in the runtime environment.
- Separation of duty. Patches and UI applications may be developed in parallel.
- Distribution. An application and its patch may execute on separate platforms across a network.
- Extendibility. Other graphical development environments may be used to develop feature-rich applications such as FLASH-based interfaces.
Blocks, as shown in Figure 2, are the fundamental building units of the environment which are dragged and dropped onto a patch from a palette of blocks grouped by functionality. A block is coded as a C++ class with inherited attributes and methods. The number of inputs and outputs for each block is determined by the developer during the development process. Inputs and outputs are referred to as pins. There are two categories of inputs:
- Main inputs (block receives its primary data for processing).
- Parameter inputs (configuration details determines block execution).
Blocks are executed in two ways at runtime. Active blocks initiate their own execution by requesting the kernel to schedule its execution. Initiation of execution is triggered periodically by using kernel polling, i.e., a call-back mechanism. The second type of execution is used by reactive blocks. These blocks are executed by the kernel when data arrives at its main input. The kernel is alerted to the availability of these data by a system flag in the block setting.
New blocks can easily be developed for the GDE by using the EyesWeb software development kit (SDK) and integrated development environment (IDE) wizards. Typically, a user creates a new block to support a new hardware device or execute a specific DSP function. In addition, EyesWeb applies data-type checking on block data pins, and the SDK supports the creation of user-specific data types. The developer of a new block has the choice to either distribute the source and/or binary to the development community.
Blocks are organized in two ways with the GDE. Firstly, they can be aggregated into catalogs that are physical collections of blocks within a standard Windows environment. Secondly, they can be structured into libraries which are logical collections organized according to functionality/use; e.g., mathematical operations (Figure 3).
To build a functioning application, blocks are dragged from libraries or catalogs in the GDE environment and connected together to form a patch. A patch comprises a series of blocks linked together in order of execution to form a fully-functional application. Figure 4 shows an example of a patch designed to allow the user to manually trigger the acquisition of data from a SHIMMER sensor, process the received data, and to display the acceleration values sensed by the sensor to the user. A patch is executed via the Kernel Runtime Server. The EyesWeb Kernel Runtime is a Windows component that supports the execution of EyesWeb patches both inside and outside the EyesWeb GDE.
As the complexity of an application increases so too does the need to structure patch development into logical or functional units. Complex patch development can be managed through the use of the subpatch feature within the EyesWeb environment. The subpatch supports the integration of a functional group of interconnected blocks into a single block-like subpatch structure.
Digital Signal Processing Capabilities for Biosignal Processing
BioMOBIUS features a variety of digital signal-processing capabilities. For real-time video processing, an extensive range of processing options exists including, baricenter/centroid calculations, quantity of motion from a silhouette, background subtraction, image concatenation, and image splitting. A variety of more advanced techniques is also available including real-time, full-body motion tracking and blob extraction. Time domain digital filtering is served by Finite Impulse Response (FIR) filters. A variety of frequency domain digital filters are available including Fourier Transform (FT), Hilbert Transform, and Kalman. Support is included for matrix-handling operations. A variety of specialist filters, such as Cochleogram generation in both image and matrix format from an audio buffer, are available.