|
When your application is running a mile or so under water, it makes sense to control it remotely.
June 01, 2004
URL:http://www.drdobbs.com/windows/remotely-controlling-windows-application/184405678
Ruben is currently working on his Ph.D. at the Institute Of Marine Research in Norway, focusing on autonomous and stationary remote sensing of marine resources. He can be contacted at ruben.patelimr.no.
A common way of collecting scientific data involves the use of electronic sampling equipment. Typical scenarios for collecting biological data include long-time surveillance, surveillance near specific biomasses, and surveillance using autonomous platforms. In this article, I describe how I communicate with an EK60 SIMRAD echo sounder embedded in the High-precision Underwater Geosurvey and Inspection System (HUGIN) Autonomous Underwater Vehicle (AUV). Although this autonomous vehicle was originally designed for seabed mapping, its software can accommodate any sensor that can connect to the AUV.
As it turns out, the Institute of Marine Research in Norway uses scientific echo sounders for biomass measurements. While the echo sounders work well for remote sensing, they are unfortunately attached to a very large mother ship, which fish tend to react to because of engine and propeller noise. Consequently, the behavior of the fish is altered, introducing bias into the measurements. However, due to its low noise level, the AUV can get closer to schools of fish without introducing fish reaction. Because of this, we decided to embed our sensor in the AUV, thereby letting us move sensors closer to the biomass and collect detailed information and less biased data.
The AUV has the ability to run in autonomous or controlled mode.
The High Precision Acoustic Positioning system (HiPAP) tracks the position of the AUV. When running the AUV in the beam of the mother ship's echo sounder, we get detailed information of the biomass location relative to the AUV. This helps us steer the AUV close to the biomass of interest. Figure 1 shows the trajectory of the AUV while approaching and penetrating an enormous school of herring. This biological aggregation is common in some of the fjords in Norway during the herring's wintering phase. Since no cables are attached between the AUV and mother ship, the AUV can maintain a speed of 4 knots at its maximum depth of 2000 meters. In Figure 1, data was collected by running the mother ship directly above the AUV. The sea bottom is the thick red line and the herring school the biggest red aggregation. Two smaller schools of fish can be seen in the beginning of the image, and one small school directly above the biggest. The AUV trajectory can bee seen as a red line enhanced by blue. Dispersed fish (blue dots) can be seen distributed in the image. The image is contaminated due to acoustic interference form the AUVs sensor and communication system. The depth range is 550 meters and the distance from the start of the image to the end is around 3 kilometers. Initially, the stepwise AUV trajectory was to approach the school gradually. This was not successful as the school turned downward and disappeared. Still, this is an excellent performance compared to towed bodies.
The communication protocol between the sensor in the AUV and the control program on the mother vessel is complex. Figure 2 is an overview of the top side and bottom side of the HUGIN system. The horizontal dashed line indicates physical separation of the top side and bottom side. The vertical dashed line is the network connection when the AUV is connected directly to the topside system, this is done only when the AUV is onboard the mother ship. All commands and data sent to or received from the AUV go through the HUGIN Operator Station (HUGIN OS). This computer is also used for navigation, mission planning, and monitoring AUV performance. Sensors are controlled from Payload Operator Stations (POS); if a POS wants to send a command to its corresponding sensor, it has to first send it to the HUGIN OS. In turn, the command is sent over the acoustic data link to be received by the control processor. The control processor sends the command to the payload processor, which addresses it to the correct sensor plugin. All sensor plugins rely on the payload processor and are implemented as Dynamic Link Libraries (DLLs). From this point on, it is up to the programmer of the DLL to forward the message to its sensor. In our case, we send the command using TCP/IP to a bridge program. This design abstracts much of the complexity of the infrastructure in the AUV, and sensors can be added in a systematic manner.
The Windows-based applications we run for these studies remotely control the EK60 sensor in the AUV from the mother ship. You have several options when remotely controlling Windows applications. Commercial software packages that let you do this include PcAnywhere and Citrix. While these products give total control over the PC, they don't meet the AUV's communication speed and protocol needs. Consequently, I developed an alternative approach that, as a side benefit, can be used over any protocol.
The basic idea behind the approach I present here is to send events from one application to another using interprocess communication (IPC) based on Windows messages. This lets me send commands to applications from programs on the same machine for opening dialog boxes, pressing buttons, and reading and setting the states of different controls. In other words, I can send commands to applications just as if I were using a keyboard and mouse. This concept opens the possibility to remotely controlling many Windows applications. You can implement complex timers for sampling data at different hours. Sensors can be connected to other sensors and programs. In our case, we wrote a bridge program that translates messages between two communication protocols, letting us communicate with the sensor from different machines.
Figure 3 shows the three abstraction layers of the remote-control system. The first layer defines the fundamental functions for finding Windows handlers and performing simple commands on controls (Listing One). The next layer is a generic dialog box class, which encapsulates some of the fundamental functions (Listing Two). The last classes are dialog box classes that reflect the dialog boxes in our application. In this example, we need to access the BI500 Dialog and Surface Range Dialog dialog boxes (see Listings Three and Four, respectively).
Before writing the bridge program, I had to decide what commands to send to the application and what data to retrieve. Typical steps for executing a command are to open the correct dialog box, alter one or more of the controls in it, then press the OK button on the opened dialog boxes. This means that I have to map all the necessary events that are generated during the command execution. Since the control of the application lies in different windows, I have to map the events to open the corresponding dialog boxes, then I have to identify events for executing different commands, and finally the event for pressing the OK button. I did this using Microsoft's Spy++ program to spy on the message loop in the application we want to remotely control while executing the commands manually.
The Set Surface Range command controls the vertical depth range across the echogram. All the other commands are implemented in a similar manner. To set the surface range to 200 meters, for instance, I have to:
Using Spy++, I map the events that have to be sent to the application to perform each of the steps mentioned. The events are collected in a header file in Listing Five.
Using the defined classes, I can set the range to 200 meters using Listing Six. Line 3 starts the application if it is not started. We then control the range value to see if it is within the valid range using the macro in line 1. A new instance of the Surface Range Dialog box is created in lines 8 and 9. The parent and child window names are set. These names are used to find the windows handler in the window tree. In line 10, the command for setting the range is executed; see Listing Four. Line 41 shows the start of the method for setting the range. This method calls the SetText method in line 33. In line 35 the BI500 dialog is first opened, then the Surface Range Dialog box is opened by pressing the Surface Range button in the BI500 dialog box. Line 36 inserts the range value and closes the dialog boxes by pressing the OK button in each of them. As you see from the code, the PostMessage and SendMessage functions are the core of the communication. These functions send specified messages to a window and call the window procedure of the specified window. SendMessage does not return until the window procedure has processed its message. In contrast, PostMessage returns immediately without waiting for the window procedure to process the message.
The example works only if no errors are cast by the application. If an error is thrown, a dialog box appears, notifying users of the error. In most cases, the error box blocks the application for further input. We use two methods of dealing with this. Before any command is executed, we search for error or warning boxes and close them. The other method to avoid errors is by checking the commands that are sent to the application. One problem I had with our sensor was that our application sometimes lost connection to the general-purpose transceiver (GPT). This is the piece of hardware that does the actual sampling and signal processing of the raw data collected from the transducer. This caused an error box to appear. After pressing Retry three or four times, it worked fine and data was collected. I solved this problem by sending events to press the Retry button as long as the error dialog box was open; see Example 1.
Controls in dialog boxes often have a range limit and, if you try to set some value out of range, an error box appears. I avoided these types of errors by testing the range of each control and checking the range of the value before it was sent to the control, as in Listing Six, line 4.
The bridge program was made for running in two modes. For testing purposes, a command-line interface gives me the ability to send commands to the application. In remote mode, the bridge programs listen on a TCP/IP port. Received messages are translated to events and sent to the application. Example 2 is pseudocode for the bridge program.
Between every command translation, I test for potential GPT error boxes. These rarely occur, but would halt the application for further input if not closed. I then test if we are in test or remote mode. These modes receive commands differently and, therefore, we have two different functions for each mode. If a valid command is received, the nBytes variable contains the number of bytes the command occupies. If the command contains a valid command number of bytes, it is translated to events and sent to the application. If the client has lost connection with the bridge, we wait for the client to reconnect.
You need to be careful when remotely controlling applications. It is important to map all the potential errors that can appear during application use. One unknown error can halt the whole communication. Dialog boxes can use some time before they appear or close. It is therefore important to halt any commands before we are sure that the dialog box is open or closed. In my case, I poll the window tree to see if we can find the dialog box. Some programs are unstable and occasionally shut down or halt. A good rule is to check whether the sensor program is running before sending any commands. If it is not running, then start it before the command is sent. It was necessary to implement commands for stopping the sensor, restarting the computer, and shutting down the computer. I needed to stop the sensor and shutdown the computer before the AUV was launched and recovered. This was to reduce the risk for damaging the transducer and hard disk. There is always the danger of getting a total machine halt, like the blue screen in Windows. To recover from this, the control processor can recycle power on all the sensors at command from the HUGIN OS.
I tested the system during a cruise period over two weeks. My experience during this cruise is that this way of remote controlling and reading data from an application can indeed be used for remote sensors. The method is easy to implement and can be used in many ways. If the application behavior is well investigated a robust communication protocol can be developed.
DDJ
1 // Header
2 #ifndef __GLOBAL_HH__
3 #define __GLOBAL_HH__
4 #include \windows.h
5
6 typedef struct
7 {
8 HWND hwnd;
9 const char *title;
10 } FindWnd;
11
12 CALLBACK CheckWindowTitle( HWND hwnd, LPARAM lParam );
13 HWND FindWinTitle(const char *title);
14 HWND FindWndByTitle(const char *parent,const char *child);
15 HWND WaitForDialogToOpen(char *parent,int timeout);
16 HWND WaitForDialogToClose(char *parent);
17 HWND OnShotOpenDialog(char *parent);
18 HWND LeftClickInAt(const char *parent,const char *child,int x,int y);
19
20 #endif
1 // Cpp file
2 #include "Global.h"
3
4 // Se if the specified window has a specified title
5 CALLBACK CheckWindowTitle( HWND hwnd, LPARAM lParam )
6 {
7 char buffer[MAX_PATH];
8 // Get the window title form window
9 GetWindowText( hwnd, buffer, sizeof( buffer ) );
10 FindWnd * fw = (FindWnd *)lParam;
11 // Compare window tile with title to be checked.
12 if(strcmp( buffer, fw-title ) == 0 )
13 {
14 fw-hwnd = hwnd;
15 return FALSE;
16 }
17 return TRUE;
18 }
19 // Find a parent window by it window title
20 HWND FindWinTitle(const char *title)
21 {
22 FindWnd fw;
23 fw.hwnd = 0;
24 fw.title = title;
25 EnumWindows( (WNDENUMPROC) CheckWindowTitle, (LPARAM) &fw );
26 return fw.hwnd;
27 }
28 // Find a child window by it window title
29 HWND FindWndByTitle(const char *parent,const char *child)
30 {
31 FindWnd fw;
32 fw.hwnd = 0;
33 fw.title = child;
34 HWND hWnd = FindWinTitle(parent);
35 if(child==NULL) return hWnd;
36 else
27 {
28 ::EnumChildWindows(hWnd, (WNDENUMPROC) CheckWindowTitle, (LPARAM) &fw);
29 return fw.hwnd;
40 }
41
42 }
43 // Halt until a dialog is opened
44 HWND WaitForDialogToOpen(char *dlgName,int timeout)
45 {
46 HWND hWndDlg;
47 hWndDlg = NULL;
48 // Loop until the window is opened
49 do
50 {
51 hWndDlg= FindWndByTitle(dlgName,NULL);
52 } while(!hWndDlg);
53 return hWndDlg;
54 }
55 // Halt until a dialog is closed
56 HWND WaitForDialogToClose(char *dlgName)
57 {
58 HWND hWndDlg;
59 hWndDlg = NULL;
60 // Loop until the window is opened
61 do
62 {
63 hWndDlg= FindWndByTitle(dlgName,NULL);
64 } while(hWndDlg);
65 return hWndDlg;
66 }
67 // Open dialog
68 HWND OnShotOpenDialog(char *dlgName)
69 {
70 HWND hWndDlg;
71 hWndDlg = NULL;
72 // Get the handler
73 hWndDlg= FindWndByTitle(dlgName,NULL);
74 return hWndDlg;
75 }
76 // Left click in a child window at position x,y
77 HWND LeftClickInAt(const char *parent,const char *child,int x,int y)
78 {
79 // Get window handler
80 HWND hWnd = FindWndByTitle(parent,child);
81 WPARAM wParam = MK_RBUTTON;
82 LPARAM lParam = MAKELPARAM(x,y);
83 // simulating left mouse click in window
84 if(!::PostMessage(hWnd, WM_RBUTTONDOWN ,wParam,lParam))
85 return NULL;
86 return hWnd;
87 }
Back to article1 // RDialog.h: interface for the CRDialog class.
2 #ifndef __CRDialog_H
3 #define __CRDialog_H
4
5 #include \windows.h
6
7 class CRDialog
8 {
9 public:
10 CRDialog(char *parent,char *child);
11 ~CRDialog();
12
13 HWND IsDialogOpen(char *dlgName); // Check if the dialog
// with name in dlgName is open
14 long SetText(int nIDDlgItem,char *text); // Set text in a control
15 DWORD SetCheck(int nIDDlgItem,BOOL checked); // Check or
// uncheck a check box
16 HWND CloseDialog(void); // Close this dialog box
17 BOOL PressButton(int nIDDlgItem); // Press a button
18
19 char *m_sParent; // String to parent window
20 char *m_sChild; // String to this dialog box
21
22 };
23 #endif
1 // CRDialog class implementation
2 #include \stdio.h
3 #include "CRDialog.h"
4 #include "Global.h"
5 #include "EK60MK1ID.h"
6
7 // Set string of parent and child window
8 CRDialog::CRDialog(char *parent,char *child)
9 {
10 int len1=strlen(parent)+1;
11 int len2=strlen(child)+1;
12 m_sParent = new char[len1];
13 m_sChild = new char[len2];
14 sprintf(m_sParent,"%s",parent);
15 sprintf(m_sChild,"%s",child);
16 }
17 CRDialog::~CRDialog()
18 {
19 delete[] m_sParent;
20 delete[] m_sChild;
21 }
22 // Close dialog box
23 HWND CRDialog::CloseDialog(void)
24 {
25 // Find handler of dialog box form string
26 HWND hWndDlg = FindWinTitle(m_sChild);
27 // Close dialog by pressing the OK button
28 ::SendDlgItemMessage(hWndDlg,RIDC_BUTTON_OK,BM_CLICK,0,0);
29 // wait for dialog to close
30 return WaitForDialogToClose(m_sChild);
31 }
32 // Return handler of dialog specified by window name
33 HWND CRDialog::IsDialogOpen(char *dlgName)
34 {
35 return FindWndByTitle(dlgName,NULL);
36 }
27 // Set text in control in dialog box
28 long CRDialog::SetText(int nIDDlgItem,char *text)
29 {
40 return::SendDlgItemMessage(FindWndByTitle(m_sChild,NULL),
nIDDlgItem,WM_SETTEXT,0,(LPARAM)text);
41 }
42 // Check or uncheck a check control
43 DWORD CRDialog::SetCheck(int nIDDlgItem,BOOL checked)
44 {
45 // manipulate control in dialog
46 DWORD wParam ;
47 // Check it
48 wParam = (WPARAM) (checked)?(BST_CHECKED):(BST_UNCHECKED);
49 ::SendDlgItemMessage(FindWndByTitle(m_sChild,NULL),nIDDlgItem,
BM_SETCHECK,wParam,0);
50 // Check if success
51 return::SendDlgItemMessage(FindWndByTitle(m_sChild,NULL),
nIDDlgItem,BM_GETSTATE,0,0);
52 }
53 // Press a button
54 BOOL CRDialog::PressButton(int nIDDlgItem)
55 {
56 HWND hWndDlg;
57 // Check if the dialog is open
58 hWndDlg = IsDialogOpen(m_sChild);
59 // get handler of button to press
60 HWND hWndCont = ::GetDlgItem(hWndDlg,nIDDlgItem);
61 // Press it
62 ::PostMessage(hWndCont,BM_CLICK,0,0);
63 return TRUE;
64 }
Back to article1 // BI500RemoteDlg.h: interface for the BI500RemoteDlg class.
2 #ifndef __BI500RemoteDialog_H
3 #define __BI500RemoteDialog_H
4
5 #include "EK60MK1ID.h"
6 #include "CRDialog.h"
7
8 class CBI500RemoteDlg:public CRDialog
9 {
10 public:
11 CBI500RemoteDlg(char *parent,char *child);
12 virtual ~CBI500RemoteDlg();
13
14 BOOL PressButtonSurfaceRange();
15 BOOL PressButtonOK();
16 private:
17 HWND OpenDialog(void);
18 BOOL SetText(int nIDDlgItem,char *text);
19 BOOL PressButton(int nIDDlgItem);
20
21 };
22
23 #endif
1 // BI500RemoteDlg.cpp: implementation of the BI500RemoteDlg class.
2
3 #include "CBI500RemoteDlg.h"
4 #include "global.h"
5
6
7 CBI500RemoteDlg::CBI500RemoteDlg(char *parent,char *child)
8 :CRDialog(parent,child)
9 {}
10 CBI500RemoteDlg::~CBI500RemoteDlg()
11 {}
12 HWND CBI500RemoteDlg::OpenDialog(void)
13 {
14 HWND hWndDlg;
15 // Check if dialog is already open
16 hWndDlg = IsDialogOpen(m_sChild);
17 if(hWndDlg) return hWndDlg;
18 // Open the dialog
19 if(!::PostMessage(FindWinTitle(m_sParent), WM_COMMAND,
RID_INSTALL_BI500,0 )) return NULL;
20 // get dialog handler
21 hWndDlg = WaitForDialogToOpen(m_sChild,1000);
22 return hWndDlg;
23 }
24 BOOL CBI500RemoteDlg::SetText(int nIDDlgItem,char *text)
25 {
26 OpenDialog();
27 CRDialog::SetText(nIDDlgItem,text);
28 CloseDialog();
29 return TRUE;
30 }
31 BOOL CBI500RemoteDlg::SetSurfVals(char *Surf)
32 {
33 SetText(RIDC_LIST_NOSURFVALS,Surf);
34 return TRUE;
35 }
36
37
38 BOOL CBI500RemoteDlg::PressButton(int nIDDlgItem)
39 {
40 HWND hWndDlg;
41 hWndDlg = IsDialogOpen(m_sChild);
42 if(!hWndDlg) hWndDlg=OpenDialog();
43 HWND hWndCont = ::GetDlgItem(hWndDlg,nIDDlgItem);
44 ::PostMessage(hWndCont,BM_CLICK,0,0);
45 return TRUE;
46 }
47
48 BOOL CBI500RemoteDlg::PressButtonSurfaceRange()
49 {
50 return PressButton(RIDC_BUTTON_SURFRANGE);
51 }
52 BOOL CBI500RemoteDlg::PressButtonOK()
53 {
54 return PressButton(RIDC_BUTTON_OK);
55 }
Back to article1 // CSurfRangeRemoteDlg: interface.
2 #ifndef _SURFRANGEREMOTEDLG_H
3 #define _SURFRANGEREMOTEDLG_H
4
5 #include "EK60MK1ID.h"
6 #include "CRDialog.h"
7 #include "CBI500RemoteDlg.h"
8 #include "Global.h"
9
10 class CSurfRangeRemoteDlg :public CRDialog
11 {
12 public:
13 CSurfRangeRemoteDlg(char *parent,char *child);
14 virtual ~CSurfRangeRemoteDlg();
15
16 BOOL SetRange(char *range);
17 BOOL SetStart(char *start);
18
19 private:
20 HWND OpenDialog(void);
21 HWND CloseDialog(void);
22 BOOL SetText(int nIDDlgItem,char *text);
23 };
24
25 #endif
1 // CSurfRangeRemoteDlg: implementation.
2 #include "SurfRangeRemoteDlg.h"
3
4 CSurfRangeRemoteDlg::CSurfRangeRemoteDlg(char *parent,char *child)
5 :CRDialog(parent,child)
6 {}
7 CSurfRangeRemoteDlg::~CSurfRangeRemoteDlg()
8 {}
9 HWND CSurfRangeRemoteDlg::OpenDialog(void)
10 {
11 HWND hWndDlg;
12 // Open BI500 dialog
13 CBI500RemoteDlg BI500RDlg(m_sParent,"BI500 Dialog");
14 // Press the Surface Range button in the BI500 dialog
15 BI500RDlg.PressButtonSurfaceRange();
16 hWndDlg = WaitForDialogToOpen(m_sChild,1000);
17 return hWndDlg;
18 }
19 HWND CSurfRangeRemoteDlg::CloseDialog(void)
20 {
21 HWND hWndDlg = FindWinTitle(m_sChild);
22 // Close dialog
23 ::SendDlgItemMessage(hWndDlg,RIDC_BUTTON_OK,BM_CLICK,0,0);
24 // wait for dialog to close
25 WaitForDialogToClose(m_sChild);
26 //CRDialog::CloseDialog();
27 CBI500RemoteDlg BI500RDlg(m_sParent,"BI500 Dialog");
28 BI500RDlg.PressButtonOK();
29 return NULL;
30 }
31
32
33 BOOL CSurfRangeRemoteDlg::SetText(int nIDDlgItem,char *text)
34 {
35 OpenDialog();
36 CRDialog::SetText(nIDDlgItem,text);
37 CloseDialog();
38 return TRUE;
39 }
40
41 BOOL CSurfRangeRemoteDlg::SetRange(char *range)
42 {
43 SetText(RIDC_LIST_SRANGE,range) ;
44 return TRUE;
45 }
46
47 BOOL CSurfRangeRemoteDlg::SetStart(char *start)
48 {
49 SetText(RIDC_LIST_STARTSURF,start);
50 return TRUE;
51 };
Back to article1 #define RID_INSTALL_BI500 32878 // ID to activate BI500 dialog
2 #define RIDC_BUTTON_SURFRANGE 0x510 // Button to push for activating
// Surface Range Dialog box.
3 #define RIDC_LIST_SRANGE 0x3ec // ID for surface range text box
4 #define RIDC_BUTTON_OK 0x01 // Ok button id
Back to article1 #define IsInRange(val,min,max) if(val=min && val\=max)
2 nRang=200;
3 startEK60MK1App();// Start sensor program if not started
4 IsInRange(nRange,0,15000) // Check range
5 {
6 char val[5];
7 itoa(nRange,val,10);
8 CsurfRangeRemoteDlg *surfRangeDlg
9 = new CSurfRangeRemoteDlg("SIMRADEK60","SurfaceRange Dialog");
10 surfRangeDlg-SetRange(val); // Set range
11 delete surfRangeDlg;
12 }
Back to article
//handle GPT transceiver not found dialog box.
CGPTTransceiverNotFoundDlg *gptNotFound =
new CGPTTransceiverNotFoundDlg("SIMRAD EK60",
"GPT Transceiver Not Found");
// Check if GPTTransceiverNotFound dialog is open
// if so press the retry button until ok
while(gptNotFound-isOpen())
{
gptNotFound-PressButtonRetry();
Sleep(2000);
}
For(;;)
{
nBytes=-1;
If(test) nBytes=getCmdLineCommand(cmd);
// Manually sending commands from cmd line
Else if(remote) nBytes=getRemoteCommand(cmd);
// Receiving commands over TCP/IP
If(nBytes0) translateCmdToEventAndSen...(cmd)
if(lostConnectionWithClien()) waitForClient()
sleep(400);
}
Terms of Service | Privacy Statement | Copyright © 2024 UBM Tech, All rights reserved.