Channels ▼
RSS

Tools

Multitouch Gains Momentum


Developer APIs

There are several options for writing multitouch-enabled code: Win32 SDK for unmanaged code; Windows 7 Integration Library (available on CodePlex) for managed code; and when .Net 4.0 releases, controls such as ScrollViewer will have native support for multitouch in Windows Presentation Foundation.

All applications receive messages from WM_Gesture. In normal scenarios, the operating system takes up to 30 WM_Gesture messages per second.

WM_Gesture can call one of two parameters. wParam provides information identifying the gesture command and gesture-specific argument values. This is the same information passed in the ullArguments member of the GESTUREINFO structure. And lParam provides a handle to information identifying the gesture command and gesture-specific argument values. This information is retrieved by calling the GetGestureInfo method.

Return values include zero if an app processes the message, and a call to DefWindowProc if the app doesn't process it. Anything else causes the application to leak memory because the touch input handle won't be closed and associated process memory won't be freed.

The GESTUREINFO structure, which stores the actual gesture interaction, looks like this:


typedef struct _GESTUREINFO {
   UINT cbSize;
   DWORD dwFlags;
   DWORD dwID;
   HWND hwndTarget;
   POINTS ptsLocation;
   DWORD dwInstanceID;
   DWORD dwSequenceID;
   ULONGLONG ullArguments;
   UINT cbExtraArgs;
} GESTUREINFO, *PGESTUREINFO;

Table 3 lists the gesture commands indicated by the dwID value in the GESTUREINFO structure.

Table 3: Gesture commands indicated by the dwID value.

For an example of unmanaged code that demonstrates intercepting the WM_GESTURE message and handling the gesture see Listing One. Keep in mind that an app will always process WM_GESTURE unless told to suppress the gesture.


LRESULT DecodeGesture(HWND hWnd, UINT message, WPARAM wParam, LPARAM lParam){
   // Create a structure to populate and retrieve the extra message info.
   GESTUREINFO gi;
   ZeroMemory(&gi, sizeof(GESTUREINFO));
   gi.cbSize = sizeof(GESTUREINFO);
   BOOL bResult = GetGestureInfo((HGESTUREINFO)lParam, &gi);
   BOOL bHandled = FALSE;

if (bResult){
   // now interpret the gesture
   switch (gi.dwID){
      case GID_ZOOM:
          // Code for zooming goes here
          bHandled = TRUE;
          break;
   case GID_PAN:
         // Code for panning goes here
        bHandled = TRUE;
        break;
   case GID_ROTATE:
       // Code for rotation goes here
       bHandled = TRUE;
       break;
   case GID_TWOFINGERTAP:
      // Code for two-finger tap goes here
      bHandled = TRUE;
      break;
   case GID_PRESSANDTAP:
      // Code for roll over goes here
      bHandled = TRUE;
      break;
   default:
      // A gesture was not recognized
      break;
  }
}else{
   DWORD dwErr = GetLastError();
   if (dwErr > 0){
      //MessageBoxW(hWnd, L"Error!", L"Could not
     // retrieve a GESTUREINFO structure.", MB_OK);
   }
}
if (bHandled){
   return 0;
}else{
   return DefWindowProc(hWnd, message,wParam, lParam);
  }
}

Listing One

The Windows 7 Integration Library in managed code in CLR 3.5 SP1 provides an entire API that you have access to from the Windows7.Multitouch.dll and Windows7.Multitouch.WPF.dll. To get started, you first check if there is a digitizer that supports multitouch:


if
(!Windows7.Multitouch.TouchHandler.DigitizerCapabilities.IsMultiTouchReady)
{
    MessageBox.Show("Multitouch is not availible");
    Environment.Exit(1);
}

This checks if the hardware supports touch. If it does, it intercepts the stylus events on the digitizer which is enabled by using the EnableStylusEvents factory class from the Windows 7 Integration Library. You then write code using the Windows 7 Integration Library to track and manage the touch-id's from the stylus events which translate into the objects moving on the screen. In the case of the previous code which detects whether a stylus is supported, you would handle the WM_GESTURE messages with StylusDown, StylusUp, and StylusMove:


public MainWindow()
{
    InitializeComponent();
    if
(!Windows7.Multitouch.TouchHandler.DigitizerCapabilities.IsMultiTouchReady)
    {
        MessageBox.Show("Multitouch is not availible");
        Environment.Exit(1);
    }
    this.Loaded += (s, e) =>
      { Factory.EnableStylusEvents(this); LoadObjects(); };
    //Register for stylus (touch) events
    StylusDown += ProcessDown;
    StylusUp += ProcessUp;
    StylusMove += ProcessMove;
}

In this example, the ProcessDown, ProcessUp, and ProcessMove functions are tracking the location of the object that is being processed by the touch events. As a finger touches the digitizer, a unique touch-id is available for you to track to make decisions on the events that are occurring. In the Windows 7 Integration Library example, this touch-id management is handled by two classes:

  • The ObjectTracker class, which contains the ProcessDown, ProcessUp, and ProcessMove functions that track the location of the object on the screen.
  • The ObjectTrackerManager class, which actually handles the touch events, and contains a Dictionary object which maps the current touch event to the correct ObjectTracker instance which represents the actual object being interacted with on the screen.

In the ObjectTrackerManager class, the following scenarios need to be considered when determining which ObjectTracker (the concrete object that is being touched on the screen) to forward the touch event to.

  • ProcessDown
    • The finger touches an empty spot, so nothing should happen.
    • The finger touches new object (for instance, a Picture object that is being acted upon), and a new ObjectTracker instance is created and a new Dictionary item is registered for this touch-id.
    • A second (or more) finger touches an already tracked object, and a new touch type is correlated with the existing touch type.

    
    public void ProcessDown(object sender, StylusEventArgs args)
    {
        Point location = args.GetPosition(_canvas);
        ObjectTracker objectTracker =
            GetObjectTracker(args.StylusDevice.Id, location);
        if (objectTracker == null)
            return;
        objectTracker.ProcessDown(location);
    }
    
    

  • ProcessMove
    • The finger touch-id is not correlated with the ObjectTracker, so nothing should happen.
    • The finger touch-id is correlated with an ObjectTracker instance, so we need to forward the event to it.

    
    public void ProcessMove(object sender, StylusEventArgs args)
    {
        ObjectTracker objectTracker =
            GetObjectTracker(args.StylusDevice.Id);
        if (objectTracker == null)
            return;
        Point location = args.GetPosition(_canvas);
        objectTracker.ProcessMove(location);
    }
    
    

  • ProcessUp
    • One finger touch-id is removed, but there is at least one more correlated touch-id. This entry must be removed from the Dictionary.
    • The last correlated touch-id is removed, so this entry must be removed from the Dictionary.

    
    public void ProcessUp(object sender, StylusEventArgs args)
    {
        Point location = args.GetPosition(_canvas);
        ObjectTracker objectTracker =
            GetObjectTracker(args.StylusDevice.Id);
        if (objectTracker == null)
            return;
        objectTracker.ProcessUp(location);
        _objectTrackerMap.Remove(args.StylusDevice.Id);
    }
    
    

In both examples, it takes work to do something more interesting with multitouch than simple default legacy support. If you want a richer experience and don't want to write too much code, we recommend that you consider the controls for WPF that will ship with .Net 4.0.


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 

Video