Channels ▼
RSS

.NET

Windows Phone 8 App Development: Using Voice Commands


To add a new VCD file, right-click on the Recipes project in Solution Explorer, select Add | New Item… and select Voice Command Definition (Figure 11). Enter VoiceCommandDefinition.xml in the Name textbox and click Add.

Windows Phone 8 App Development Part 2
Figure 11: Adding a new Voice Command Definition file to the project.

The default template for the XML VCD file defines sample voice commands for a game. The following lines replace the default template with a VCD file that defines a voice command for US English that allows the user to create a new recipe:

<?xml version="1.0" encoding="utf-8"?>

<VoiceCommands xmlns="http://schemas.microsoft.com/voicecommands/1.0">
  <CommandSet xml:lang="en-US">
    <CommandPrefix>Recipes</CommandPrefix>
    <Example>create new recipe</Example>

    <Command Name="CreateNewRecipe">
      <Example>create new recipe</Example>
      <ListenFor>create [a] [new] [recipe]</ListenFor>
      <ListenFor>add [a] [new] [recipe]</ListenFor>
      <Feedback>Creating a new recipe... </Feedback>
      <Navigate Target="/AddNewRecipePage.xaml"/>
    </Command>
  </CommandSet>
</VoiceCommands>

The schema for the XML VCD file is easy to understand and is composed of the following elements:

  • VoiceCommands. The root element for all voice commands. This element contains one or more CommandSet children. You should define at least one CommandSet child.
  • CommandSet. Defines all the voice commands that the app will accept for the language specified in the required xml:lang attribute. The xml:lang attribute requires an IETF-style culture code such as "en-US" or "it-IT." You cannot specify more than one language for the CommandSet. Thus, if you want to support voice commands for both English (United States), xml:lang="en-US", and English (United Kingdom), xml:lang="en-GB", you must define two CommandSet, one with xml:lang="en-US" and the other for xml:lang="en-GB." That's bad news when you want to provide the same commands for different cultures because you have to duplicate the same content for each CommandSet. The optional Name attribute allows you to make the CommandSet accessible by the Windows.Phone.Speech.VoiceCommands API and make changes to it in the code. I'll dive deeper on this later with a complete example. This element contains one or more Command children. You should define at least one Command child.
  • CommandPrefix. Specifies a user-friendly name for the app. This element is optional and allows you to specify a name for the app that the user can say to identify the app as the prefix for the available voice commands. For example, you have an app named Recipes4u, you can specify "Recipes for you" as the CommandPrefix to match the way it sounds.
  • Example. Specifies a sample voice command for the CommandSet. Windows Phone will display this example on the "Apps" pivot of the "What can I say?" helper page, as an example on the "Listening…" page and on some error pages and dialogs.
  • Command. Defines a voice command with a unique name (specified by the required Name attribute) within the CommandSet. The Command element specifies what the users can say to initiate the action, the feedback provided to the user when the GSE recognizes the command and the page to which the app must navigate when the command is recognized. Each Command element contains one or more ListenFor children. You should define at least one ListenFor child. A command Element has the following required children elements:
    • Example. Specifies a sample phrase for this command. The sample phrase doesn't have to include the prefix specified in CommandPrefix. When you tap on the app name in the "Apps" pivot of the "What can I say?" helper page, Windows Phone displays the "did you know?" helper page that shows all the sample phrases defined for the registered CommandSet for the app.
    • ListenFor. Defines a phrase that the app will recognize for the Command. You can include optional words or phrases enclosed in square braces ([]). For example, "create [new] recipe" will match both "create recipe" and "create new recipe," because new is enclosed in square braces and is an optional word. Each Command may contain up to ten ListenFor elements. It is also possible to use PhraseList elements but I'll dive deeper on them later.
    • Feedback. When GSE recognizes the command, it displays the text specified in the Feedback element and reads it back to the user. You can also use PhraseList elements in the text that specifies the feedback.
    • Navigate. Specifies that the command must navigate to the app. This element may define the Target attribute with the specific page within the app to navigate when the command is activated.

Now, it is time to initialize the VCD file from within the app, using these steps:

  • Open App.xaml.cs.
  • Add the following using statement: using Windows.Phone.Speech.VoiceCommands;
  • Then, add the async modifier to the Application_Launching event handler because it is necessary to call an API method with an asynchronous execution. In addition, add a line to install the command sets for the global speech engine language:

private async void Application_Launching(object sender, LaunchingEventArgs e)
{
    try
    {
        await VoiceCommandService.InstallCommandSetsFromFileAsync(
            new Uri("ms-appx:///VoiceCommandDefinition.xml"));
    }
    catch (Exception ex)
    {
        //// You must add the necessary code to handle exceptions for your app
        //// caused by any possible compilation error in the VCD file
    }
}

  • Build and start debugging the app in the emulator. To make sure the app runs the previously added line that installs the command sets that match the global speech engine language, it is necessary to launch the app once.
  • After the debugging session executes the app, activate GSE, tap on the help button, and the "What can I say?" helper page will appear. Swipe over to the "apps" pivot and you will see the list of examples from the apps that support voice commands, including the new example for the Recipes app (see Figure 12).

Windows Phone 8 App Development Part 2
Figure 12: The "apps" pivot of the "What can I say?" helper page displaying the sample voice command for the Recipes app.

  • Tap on Recipes and the phone will display all the available voice commands for the app in the "did you know?" helper page (Figure 13). As you can see, if you say "Recipes create new recipe," the phone should recognize the command, launch the Recipes app and navigate to the AddNewRecipePage. In fact, the AddNewRecipePage doesn't exist in the project, so an exception will be thrown. I'll explain how to handle the navigation later because the idea here is to develop an app that uses speech-related features already available in Windows Phone 8 (which I still need to explain). I'll keep the focus on the voice commands for now.

Windows Phone 8 App Development Part 2
Figure 13: The list of supported voice commands for the Recipes app.

  • Go to Settings, Speech. The emulator will display the Speech settings, including the active language. Make sure that the "Enable Speech Recognition Service" checkbox is activated and that the Speech Language is English (United States), see Figure 14.

Windows Phone 8 App Development Part 2
Figure 14: The Speech settings page.

Now, you can activate GSE and say any of the following phrases. Remember that "Recipes" is the CommandPrefix:

  • Recipes add
  • Recipes add a new recipe
  • Recipes add new
  • Recipes add new recipe
  • Recipes create
  • Recipes create a new recipe
  • Recipes create new

Any of the these phrases will match one of the two ListenFor elements for the voice command defined for English (United States):

  • create [a] [new] [recipe]
  • add [a] [new] [recipe]

When the GSE matches the phrase said by the user with the registered CreateNewRecipe command, Windows Phone will display the app icon, its name, and the feedback text defined for the voice command. In this case, the feedback is "Creating a new recipe..." and you will hear the phone's voice reading back the text (Figure 15). Then, the debugger will show an exception because there is no AddNewRecipePage yet.

Windows Phone 8 App Development Part 2
Figure 15: Windows Phone starting the Recipes app and navigating to the AddNewRecipePage as a result of the "Recipes create new recipe" voice command.

Because the navigation target for the voice command was "/AddNewRecipePage.xaml" in the VCD file, the URI generated for the phrase "Recipes create new recipe" is:

//AddNewRecipePage.xaml?voiceCommandName=CreateNewRecipe&reco=open%20Recipes%20create%20new%20recipe

The URI includes two query string parameters:

  • voiceCommandName: The Command name that the GSE matched with the phrase said by the user. In this case, the Command name is CreateNewRecipe.
  • reco: The full text that the GSE recognized, including the CommandPrefix. The full text includes words separated by "%20," (the encoded representation of the space character). In this case, the recognized text is "open%20Recipes%20create%20new%20recipe."

Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 

Video