Route Control & Streaming Video

Route Control

Route control is the ability to select the path of data packets as they leave your video server. Route control is possible when your server resides in a "multihomed" network—one in which your server's network possesses more than one distinct external connection to the Internet. In multihomed networks, you control which network path video packets travel to their specific IP destinations. Differences in the quality of these paths can vary enormously. For instance, Figure 1 shows a typical network with network egress points for the streaming video server. Of the three paths, video streams on path 1 travel on a more stable path with lower jitter than those on path 2 or 3. Because of jitter in this example, the ratio of worst-path to best-path exhibits 70 times more instability than the preferred path.

In general, maintaining multihomed sites is complex and paths are assigned using the metrics of the routing protocol through which the paths were acquired. The assignment of these paths doesn't take into account application-specific metrics (jitter for streaming video, for instance) that have more meaning to the quality of the delivery of the data.

As data packets travel across the Internet, routers forward them to neighboring routers by determining which network prefix is associated with a destination address, then sending the packet to the next hop-gateway router associated with that prefix. Typically, there are 10 to 20 routers that forward data packets (up to a limit of 255) before a packet is dropped. Each path possesses different data transmission characteristics.

From a client's point of view, video stream playback is sensitive to variability in arrival times of the data packets at the client. Video streams are usually divided into temporal (time windowed) chunks of data, delivered with the expectation that they will arrive at the client within a certain time window. The inability for a client to predict this arrival time window for streaming packets could cause faults in the video playback. These changes in the elapsed travel time of packets are referred to as "jitter." Jitter that exceeds a threshold can play havoc with the buffer that the client has set up to receive the data. If the buffer has been exhausted before the next video packet arrives, the playback displays artifacts (halted playback or broken images), depending on the codec/player used. To deliver the highest quality video experience, jitter needs to be as low as possible.

Route control lets the source network minimize jitter by choosing the best available streaming path. By selecting the best route, you can experience an average improvement of 50 percent over the default path. Even better, this is a low-level optimization that affects all stream types, transports, video-on-demand, or live-streaming for the delivery of video.

Controlling Video Routes

Some multihomed networks have two network connections, others may have dozens of external connections. Each external connection represents a potential path to a specific destination. Finding out which path has better characteristics, then selecting that path, can be difficult—but this is the essence of route control.

Among the capabilities that a video server's network route-controlling system should possess are the ability to:

• Communicate with client routers by using, for example, the BGP-4 protocol (see the accompanying text box entitled "Routing/BGP/CIDR").
• Probe or collect data on network path characteristics.

• Store and correlate results for decision making.

The best way to support these capabilities is by using individual functional components. Figure 2 illustrates the functional relationships between these components and responsibilities (arrows denote communication paths between components). This route-control design is based on individual components: Manager, Measurement, Decision, Router Communication Interface, and Listener. Each component runs on a separate schedule and contains a collection of objects. The Router Communication Interface communicates with client routers, the Measurement module probes/collects path characteristics, and the Decision module stores/correlates decision-making data. The Manager is a support module for controlling the communication between BGP modules, and the Listener is a module for communicating with external (nonBGP) modules.

Jitter-based route control follows this sequence:

1. On a video stream request, dispatch the requestor's IP address to the route-control process.

2. Route control identifies the prefix and potential egress points.

3. Route control collects specific path jitter measurements.

4. The path with the lowest jitter is inserted as the new default path.

5. Repeat upon regular intervals until the connection is dropped.

These steps can be executed in a straightforward process. I've avoided details about the interface with BGP and communication with router peers; you can find information on this from BGP implementations such as GateD (http://www.nexthop.com/), MRT (http://www.mrtd.net/), GNU Zebra (http://www.zebra.org/), and ZebOS (http://www.ipinfusion.com/).

Collecting Data

There are several techniques you can use to collect data on potential routes to specific destinations. The route-control design I present here supports a generic interface that allows implementation of these techniques.

• Passive metrics. This requires a complex integration with the web service and router systems. Web-site designers need to include two 1-pixel images per egress path that (through the network routing configuration) directs HTTP traffic over each multihomed path. Data retrieved from the images quantifies the jitter. Therefore, once a request is made, jitter data can be collected though the original request. This only measures data quality over the HTTP protocol and may not be representative of all traffic types.
• Active collection. This is closer to a representative metric of jitter for streaming video. The route-controller system "pings" near the client destination at regular intervals and records the response through each protocol path. This approach requires coordination of test routes with ping measurements to collect data.

• Externally processed. There can be ways to characterize the quality of the path remotely and dispatch the computed measurement directly to the route controller for processing and control of the best path.

The route-controller design lets you derive from the MeasurementBase parent class to implement specific measurement methodologies for collecting and storing jitter data. As you can see in Listing One, the base class requirements are simple. You need only the capacity to handle an incoming request through processRequest(). When the measurement process has completed, submit the data to storage via submit(). MeasurementBase provides additional support for communication between external requests for measurements through the listen() method that binds to a specific port during the construction of the object. Any number of derived measurement objects can coexist within this design, but coordination of measurements between these modules is required and is the responsibility of the Measurement manager object.

In the active data-collection approach, the video request is received in the form of an IP address from the video server (or appropriate surrogate). The IP address is matched up to a routing prefix. Because this request was received at the server, you know that this IP address (and corresponding prefix) is an actively used prefix and can be tested and optimized to reduce jitter. The MeasurementBase class receives the request, and the derived active measurement module is responsible for inserting test routes and pinging the target through each test path. Once a measurement has been completed, the jitter measurement is computed and stored.

Listing Two presents an active measurement implementation of the MeasurementBase pure virtual method processRequest(). ProcessRequest() receives a MsmtRequest object that contains a target prefix for active measuring. The processRequest() method inserts a route for testing and pings the destination. Once the response has either been received or timed-out over iNumProbes measurements, the active route is removed from the client router. The jitter is computed in computeScore, then stored via the submit() method. A different approach is required for a passive measurement, where the MsmtRequest object wraps the raw measurements that have been captured through a web page request. The measurements are then used for computing jitter.

The approach used to collect the jitter data can result in different characteristics of the paths being measured. In addition, this design lets you support a mixture of data-collection methods. Again, since the active measurements and the external measurement process can potentially affect test paths, these measurement techniques are coordinated within the Measurement component by the manager.

With either measurement approach, the data is collected in the form of latency values (round-trip travel times) and is converted into jitter measurements. Jitter can be computed from received latency values using the formula:

Jitter=(p2received-p2sent)-(p1received-p1sent)

Jitter is computed in milliseconds and represents a single measurement. This measurement represents a round-trip computation of jitter. Because data usually returns on the same path for a given prefix, the return trip will be constant based on the computed round-trip value. This measurement can be assigned a sample time midway between the times probe1 and probe2 are sent. A weighted temporal window can be applied to multiple samples so as to provide a more accurate representation of the current jitter value. A single measurement is useful, but might indicate a singularity in that particular measurement, therefore, multiple measurements are performed over time.

Selecting the Best Route

Listing Three shows the route-decision process within the RouteSelection object. With the collected and processed path scores, the RouteSelection object iterates through the vector of prefixes and inserts paths that possess the lowest jitter score. The RouteSelection::engineer() method accepts a prefix object that contains the rating or score of the prefix along with routing information. This path is then inserted as the new route to the router. If an error occurs during insertion of this route, the path is removed from the prefix and a recursive call is made to RouteSelection::engineer() until a path is inserted without error or no paths are available. Upon a successful insertion, this information is recorded and this method is exited. RouteSelection::engineer() is called on a regular basis to update routes based on the current statistics collected for these paths.

Because path characteristics change over time, it is important to use current path metrics. The design I present here allows that a destination be measured upon video-stream request. Therefore, paths that are not frequently engineered are reduced in priority compared to data from frequently requested prefix sources. Routes and paths can also become available/unavailable at various times; therefore, it is important to reevaluate newly collected metric data and reinsert the best path on a regular basis (for example, engineering every 15 minutes would be reasonable). However, recently received requests from prefixes should be engineered after they have been tested.

Optimized Delivery

You can achieve significant improvement in the delivery of streaming video when your server is able to make intelligent decisions regarding which paths to select. Figure 3 is an example of jitter data for multiple paths and the resulting route selection decision of the lowest jitter path for a destination using a commercial route-control device over a 24-hour period. The gain in performance of the stream is not through improving the delivery path itself (for example, modifying the nature of the path on which the packet would travel), but intelligently selecting the best delivery path at that time from all the paths that are available in a multihomed environment.

Further optimizations in the design can be applied to the areas of concurrent measurements, additional path characteristic measurements, advanced target identification and testing to provide further accuracy in the data collected, and sampling theory to create a picture of the path characteristics over time and to be somewhat predictive in the behavior of the path characteristics. Finally, commercially available appliances from Proficient Networks (the company I work for), NetVMG, and RouteScience support route control in different forms.

If your server is operating in a multihomed environment and it is important to squeeze every bit of performance from the quality of your video streams, then this is one optimization that you can't ignore.

DDJ

Listing One

class MeasurementBase
{
public:
     bool processRequest(MsmtRequest &req) = 0; //implemented by derived class

     bool shutdown(); //shutdown this component

protected:
     MeasurementBase(RouteComm *pRouteComm, int iPort) : m_pRouteComm(pRouteComm), m_iPort(iPort) {;}
     virtual MeasurementBase() {;}

     //commit collected data to data store
     bool submit(Target &target, double dSample);

private:
     bool listen(); //initiates a listen to port, base class will take care of initial contact 

private: //variables
     RouteComm *m_pRouteComm;
     int m_iPort;
};

Back to Article

Listing Two

bool JitterActive::processRequest(MsmtRequest &msmtReq)
{
	Prefix prefix;
	PrefixPaths paths;
	PrefixPaths::iterator iter;
	bool fErr;
	int i;
	double dValue, dResult;
	vector<double> respColl;
	Target target;

	prefix = msmtReq.getPrefix();
	paths = prefix.getPaths();

	iter = paths.begin();
	if (iter == paths.end())
		return Konstants::bFail;

	while (iter != paths.end())
	{
		target = iter->getTarget();
		try
		{
			fErr = m_pRouteComm->insertTargetRoute(target);
		}
		catch (Exception e)
		{
			cerr << "Error in insertion of route" << endl;
			return Konstants::bFailure;
		}
		
		if (fErr == Konstants::bSuccess)
		{
			for (i = 0; i < Konstants::iNumProbes; ++i)
			{
				fErr = sendProbe(target);
				if (fErr == Konstants::bFail)
				{
					cerr << "Error in sending probe" << endl;
				}
				else
				{
					dValue = waitForResponse(target);
					if (dValue < 0)
					{
						dValue = Konstants::dJitterMax;
						cerr << "Error in receiving data" << endl;
					}
					respColl.push_back(dValue);
				}
			}
			
			//compute score for this path
			fErr = computeScore(target, respColl, dResult);
			if (fErr == Konstants::bFail)
			{
				cerr << "Error computing rank for this path" << endl;
			}
			else
			{
				fErr = submit(target, dResult);
				if (fErr == Konstants::bFail)
				{
					cerr << "Error in writing response" << endl;
				}	
			}
		}
		//empty result vect
		respColl.erase(respColl.begin(), respColl.end());
		//move to next path
		++iter;
	}

	try
	{
		//remove last target route
		fErr = m_pRouteComm->removeTargetRoute(target);
	}	
	catch (Exception e)
	{	
		cerr << "Error in removal of route" << endl;
		return Konstants::bFailure;
	}
	if (fErr == Konstants::bFail)
	{
		cerr << "Error in removal of route" << endl;
		return Konstants::bFailure;
	}	

	//finished performing measurements for this target
	return Konstants::bSuccess;
}

Back to Article

Listing Three

bool RouteSelection::engineer(Prefix& prefix)
{
	PrefixPaths paths;
	PrefixPaths::iterator iter;	
	int iVal;
	bool fErr;

	paths = prefix.getPaths();
	if (paths.empty() == true)
		return Konstants::bFail;

	//collection is already sorted by path score
	iter = paths.begin(); //best path

	try
	{
		fErr = m_pRouteComm->insertRoute(*iter);
		if (fErr == Konstants::bFail)
			return fErr;
	}
	catch(Exception e)
	{
		cerr << "Exception thrown by routerInterface " << e << endl;
		return Konstants::bFail;
	}
	

	if (fErr == Konstants::bFail)
	{
		//recursive call until we get this right
		prefix.removePath(*iter);	

		fErr = engineer(prefix);
		if (fErr == Konstants::bFail);
			return fErr;
	}
	else
	{
		try
		{
			fErr = m_pState->recordEngineered(*iter);
			if (fErr == Konstants::bFail)
				return fErr;
		}
		catch(Exception e)
		{
			cout << "Exception thrown by State " << e << endl;
			return Konstants::bFail;
		}
	}
	return Konstants::bSuccess;
}

Back to Article