What About Caching?
No doubt some of you are wondering, "What about caching?" Couldn't the web server avoid repeatedly sending the same file over and over, if only it were stored in web caches much nearer the clients? The answer is yes, it could. However, it is not clear whether it will be any easier to get caches deployed than IP multicast. For very popular software, you will often see a list of mirror sites in different regions that you can download from. This amounts to manual caching and requires the content provider to set up equipment around the world, or to cut deals with third parties. When automated caches are set up, many servers operate in cache-busting mode, deliberately trying to force clients to bypass the cache so that they can get direct web hits.
Supposing that caches can be deployed, they do reduce waste of network bandwidth by keeping the file nearer the client. Notice, however, that the total number of transfers of the file is the same -- one for each client. So the total server power needed remains the same -- it is just spread out onto the caches.
However, the number of caches may be large, so it may still be worthwhile to reduce the load further by multicasting to the caches. Also, the caches themselves may need to serve a large number of clients, making multicast from the cache desirable. So caching may be used in combination with multicast, rather than necessarily eliminating it.
For campus-wide software distribution, caching may not make sense at all. Consider the task we face at Microsoft in distributing the nightly builds of Windows NT. Every night, all the source code checked in during the day is compiled (built). This latest build, typically of several hundred megabytes, needs to be distributed back out to several hundred developers and installed on their machines for testing and further development. Doing this using web servers requires about 15 high-end server machines. Adding caches would simply move some of these machines to different buildings. Some network traffic may be saved, but the server equipment requirement remains the same. Using Fcast would allow the file distribution using the network bandwidth of a single transfer, and would require only one server.
-- J.G.
