Channels ▼
RSS

Web Development

Silverlight Media Elements


Hack: Video Scrubbing

A common feature of most video players is the ability to "scrub" video or "scrub" the timeline of a video. When we say "scrub," we basically mean being able to change the position of the video by clicking on a line or icon and dragging it along a line that represents the video and the video changes in real time. That leads us into how we make the timeline visually, then how to wire it up such that we can actually "scrub" it. The first thing we do is then is to create our XAML; see Listing Five.

<Rectangle x:Name="TBStatusBarBg" Opacity="0.5" Width="327.167"
Height="2" Fill="#FF8C8989" RadiusX="0" RadiusY="0" Canvas.Left="15"
Canvas.Top="7.667" />
   <Rectangle x:Name="TBStatusBarDownloadProg" Opacity="0.5"
Width="0" Height="2" Fill="#FF6C6B6B" RadiusX="0" RadiusY="0"
Canvas.Left="15.333" Canvas.Top="8" />
   <Rectangle x:Name="TBStatusBarPlayProgress" Opacity="0.5"
Width="0" Height="2" Fill="#FF353434" RadiusX="0" RadiusY="0"
Canvas.Left="15" Canvas.Top="8.333" />
   <Rectangle x:Name="ScrubIcon" Opacity="1" Width="3" Height="8"
Fill="#999999" RadiusX="5" RadiusY="5" Canvas.Left="15" Canvas.Top="6"
Cursor="Hand" />
   <Rectangle x:Name="ScrubMask" Width="327.167" Opacity="0"
Height="10" Fill="#FFF0E8E8" Canvas.Left="15" Canvas.Top="5"
Cursor="Hand" />
Listing Five: Video Scrubbing Rectangles

You can make your timeline any way you want. In our case, we use five simple rectangles. Using our timeline, we will show how much the current video is downloaded and the current position in the playback. So with that our first rectangle is the background. This is the core visual representation of the video length. It doesn't actually do anything and is static, regardless of the video source, but contrast of the progress and position indicator is relative to it on a percentage basis.

The next rectangle is used to show how much the video is downloaded. This rectangle is animated by a media element event.

The third rectangle is our position indicator that shows where the video is in playback and will be the key thing that changes when we actually "scrub" the video.

Following that we have a rectangle that we use as our indicator icon. The indicator icon is rendered perpendicular to the others and is generally what users click on to scrub the timeline.

The last rectangle is actually an invisible mask we place over the whole set. It actually is what we bind mouse events to so that things are not so hard to click on.

The next two XAML components are what we need to make for the timeline to work. They are a media element and storyboards for animation. There are two storyboards -- one is a timer and the other is the position play back animation to smooth out the position change. Together this smoothes out the changes in position when scrubbing and during playback as seen in Listing Six.

<Canvas.Resources>
<Storyboard x:Name="Timer"
Storyboard.TargetName="MediaPlayer" >
   <DoubleAnimation x:Name="TimerWidth"
Storyboard.TargetProperty="Width" Duration="0:0:.1" />
</Storyboard>
<Storyboard x:Name="PositionPlayBack"
Storyboard.TargetName="TBStatusBarPlayProgress" >
   <DoubleAnimation x:Name="PositionPlayBackWidth"
Storyboard.TargetProperty="Width" To="0" Duration="0:0:.1" />
   <DoubleAnimation x:Name="PositionIconLeft"
Storyboard.TargetName="ScrubIcon"
Storyboard.TargetProperty="(Canvas.Left)" To="0" Duration="0:0:.1" />
</Storyboard>
</Canvas.Resources>
<MediaElement x:Name="MediaElement" Canvas.Left="0"
Canvas.Top="0" Width="361" Height="280" ></MediaElement>
Listing Six: Storyboard as a resource

Now the hard part is actually to wire things up. All the eventing we are doing is in XAML, but you could do it programmatically as well. In the root of our page class that is our media player, we will create the values or objects we need.

private bool _ScrubIconMouseCapture = false;
private bool _MediaOpened = false;
private SilverlightHost SLHost = new SilverlightHost();
private System.Windows.Threading.DispatcherTimer MyTimer = new
System.Windows.Threading.DispatcherTimer();
Listing Seven: Setting values and binding events and other initialization in code.

Here we can see a number of values. The _ScrubIconMouseCapture value is used to determine if the mouse is captured so we can perform the drag operation on the MouseMove event. The _MediaOpened is used to see if the video is open or not and if so then we know we can scrub the video timeline. We also have a reference to the Silverlight host and an instance of the DispatcherTimer object that we can use to fire a thread. Now we take a look at our constructor code in Listing Eight.

MyTimer.Interval = new TimeSpan(0, 0, 0 , 0, 1);
MyTimer.Tick += new EventHandler(Timer_Completed);
MyTimer.Start();
Listing Eight: Code in the constructor needed for scrubing.

In Listing Eight, we see us setup a timer or thread that fires at the set interval, in this case 1 millisecond. We also define the event that is fired at every interval and start our thread.

Next let's look at the timer completed event (Listing Nine).

private void Timer_Completed(object sender, EventArgs e)
   {
   if (_MediaOpened && !_ScrubIconMouseCapture)
   {
       double Width = (this.TBStatusBarBg.Width /
       ThisMediaElement.NaturalDuration.TimeSpan.Seconds) * ThisMediaElement.Position.Seconds;
       if (Width > TBStatusBarBg.Width)
   {
       Width = TBStatusBarBg.Width;
   }
       else if (Width < 10)
   {
       Width = 10;
   }
       PositionPlayBackWidth.To = Width;
       PositionIconLeft.To = Width + 10;
       PositionPlayBack.Begin();
   }
}
Listing Nine: Timer completed event.

In this event, we check that the MediaOpened flag has been set and that that mouse is captured or not and starts the timer again. The media opened event that is bound to the media element sets the media opened flag when the video can start playing. Now if the media opened flag is set and the mouse is not captured, then it calculates the width based on the width of the background of the timeline vs. the length of the video taken from the natural direction based on seconds and then the current position. Next we set the two values, one for the position bar and the other for the "icon" or fourth rectangle we used as our scrub icon. After we set the critical values, we then start the animation again so the change is smooth as implemented in this sample.

Now we are ready to setup our scrub timeline. First we go over the download progress changed event. We bound this to show on our timeline how much of the video is downloaded, so this event then sets the width of the download progress bar on our timeline, once downloaded it starts the media element playing if our property auto play we created in our class is set to True.

private void ThisMediaElement_DownloadProgressChanged(object sender, RoutedEventArgs e)
  {
     TBStatusBarDownloadProg.Width =
     ThisMediaElement.DownloadProgress * TBStatusBarBg.Width;
       if (ThisMediaElement.DownloadProgress == 1)
  {
      if (AutoPlay)
     {
     Play();
     }
  }
}
Listing Ten: Download progress changed event code

A scrub operation starts once a user clicks on our mask rectangle. This fires off the event Mouse Left Button Down. Here we first set the mouse capture flag, and then we call the pause method we created on the application class. We could just "pause" the video ourselves, but since we might want to call this method in a number of places and in our application we also want to change the play icon; we just wrap it in a method called pause which we call here. Once the flag is set and pause is executed, we actually "capture" the mouse. Remember on this method we lose the capture without being able to track it if the user moves off the Silverlight control. If we do nothing else, if the user clicks somewhere on the timeline to scrub then position will stay somewhere else until they move the mouse. So to make sure this effect doesn't happen we also fire our mouse move event so that the position immediately goes to the correct position on the timeline and the video play back.

private void ScrubMask_MouseLeftButtonDown(object sender, MouseButtonEventArgs e)
   {
   _ScrubIconMouseCapture = true;
   Pause();
   ScrubMask.CaptureMouse();
   ScrubMask_MouseMove(sender, e);
   }
     private void ScrubMask_MouseMove(object sender, MouseEventArgs e)
     {
        if( _ScrubIconMouseCapture )
        {
           double x = e.GetPosition(null).X;
           if( x > TBStatusBarBg.Width + 10 )
        {
           x = TBStatusBarBg.Width;
        }
        else if( x < 10)
     {
          x = 10;
     }
        PositionPlayBackWidth.To = x - 10;
        PositionIconLeft.To = x;
        PositionPlayBack.Begin();
        double NewTime = x / ( TBStatusBarBg.Width /
        ThisMediaElement.NaturalDuration.TimeSpan.Seconds);
        if (NewTime >
        ThisMediaElement.NaturalDuration.TimeSpan.Seconds)
     {
        NewTime = ThisMediaElement.NaturalDuration.TimeSpan.Seconds;
     }
        try
     {
        double TotalSeconds = Math.Round(NewTime);
          //get time
        int Minutes = int.Parse( Math.Floor(TotalSeconds / 60).ToString() );
        int Seconds = int.Parse( (TotalSeconds - (Minutes * 60)).ToString() );
        ThisMediaElement.Position = new TimeSpan(0, Minutes, Seconds);
     } catch(Exception) { }
  }
}
Listing Eleven: Mouse events for the drag operation

The mouse move event this code implements as the user holds down the mouse button affects our timeline and video and is actually "scrubbing" the video.

First, this method fires any time the mouse moves over the mask> So as to not mess with the user too much, we want to make sure we have the mouse capture before doing anything. If we do, we then grab the x value of the mouse and change it by our offset of the control. Next, we need to make sure the value is not too big or too small. For example, if the user moves the mouse way past either end of the timeline we would effectively be past the end or before the beginning, so we reset our x value that accordingly by using an "if else if" statement based on the bounds of the timeline.

Since the position change animation is currently paused during the capture event and because we called pause on mouse down we just set the two double animation values. Then we restart the animation. Also on regular mouse moves we just set the values and call the begin method. After this we create a new time value conversion for what the new play back position will be. This is based on our x value divided by the line width divided by natural duration of the video. If the new time is greater than the natural duration, we set it to the end of the media; otherwise we go ahead and set the new position and use some error handling to make sure this operation doesn't blow up.

When the user is done with scrubbing the video with our mouse move event and lifts the mouse button up, we call our mouse up button to let the video continue playing. In this event, we set our mouse capture flag back to False, call the play method which we wrapped so we could change icons, and so on -- like we did with the pause method we implemented. We then can release our mouse capture.

private void ScrubMask_MouseLeftButtonUp(object sender, MouseButtonEventArgs e)
   {
      _ScrubIconMouseCapture = false;
      Play();
      ScrubMask.ReleaseMouseCapture();
   }
Listing Twelve: Left button up event that ends the drag operation.

You can implement this a number of ways but this is how I hacked it together and it works well. You can scrub the video timeline and become dangerous to yourself and others. This actually provides us with the ability to build the complete media player experience.


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 
Dr. Dobb's TV