David James Kelley is a senior software architect at IdentityMine and author of the upcoming book Hacking Silverlight 2 on which this article is based. Courtesy of Manning Publications. All rights reserved.
Silverlight is all about next-generation user interfaces on the web, and one of the coolest aspects of presenting next-generation UIs on the web is video and sound. When developing a next-generation user or Web 2.0 experience, it is the smooth, subtle animation in the background, the video playing in the text, and the sound reacting to a user's action that brings UIs alive. The media element in Silverlight is the way we add video and sound, from streaming video or scrubbing the timeline, to using audio and special effects with video brushes. Here we talk about all the ins-and-outs of the media element and all the things we can do with it. The Silverlight media experience even supports high-definition video streaming online. With all the cool things about using media elements in Silverlight, it is easy to over look the possible downside of it -- performance. So first let's talk a little about performance of media elements in Silverlight.
I recall sitting in the room and writing a class called MediaClip that wrapped all the functionality I wanted for videos -- manipulating video, starting video, stopping and pausing video. It even had embedded controls and did source swapping with different versions of a media source, so that different resolutions could be used, depending on what the current video size setting was for the object. The best part was that in its smallest mode, you could drag the video. For example, if in the application you had a line or group of videos, then you could click-and-drag the clips to the editor tool and make a play list of sorts. Each one of these little clips was an instance of my MediaClip class. Thinking this source swapping and drag-and-drop was cool, we went built out all functionality for this rich immersive media experience. And we were feeling pretty good about ourselves.
When we finally had something to play with and things were coming together, a few of the guys decided to spend time playing with the video, maybe with our little Vista CPU meter running and the memory usage thing going on down in the task bar. We played a few videos, dragged a few things around. Life was good.
Then I selected a different category and the application swapped out the current media group on the page. No worries right? No one really noticed much on the meters. Dragged some media, play a clip or two... no worries. Well, so the meter was slowing crawling up... hm... no worries right? (Say it with me... "no worries...") Pretty soon the memory meter was pegged and the drag operation was getting slow. Hmm.. no worries? Click on another category... a new set of videos animate out... and it started going sloooower and sloooower.
At this point a few red flags started coming up -- or maybe about 100 red flags, one for each media clip that was on or had been on the screen. This was not a good sign. Through a lot of jumping around, we found a number of key performance issues for Silverlight and the media element.
Rule #1: Don't have 500 media elements in the same application at the same time all bound to some media somewhere.
Rule #2: Do not leave all the media elements around if you're not using them. It is one thing to have 10 on the screen, but another to have 10 on the screen and 90 hiding somewhere at --9000 for Canvas.Left. (And yes I have actually seen that done.)
Rule #3: See Rule #2.
Rule #4: Encode your media for the size that it will be displayed. Do NOT stretch the video....
All this video is cool, but if you can't serve it up and make it fast, then there's no point. Let's get down to business and learn out to work with the media element and how to use it in such a way as to be reasonably fast.
