Simba
A Distributed Video File Server

Satyam Vaghani, Pranav Kantawala
{svaghani, pranavk} @ cs.stanford.edu

Abstract

We have designed and implemented Simba, a scalable, fault-tolerant and low-cost video delivery system constructed from a collection of lightweight servers running on a network of workstations (NOW). Video files are striped into very small fragments on network attached storage or the local file system of each workstation. The fragments are streamed across to the Simba client, which seamlessly integrates all these fragments in real-time to reproduce the entire movie without incurring high buffering costs. The system does not store persistent state or scheduling information at any delivery server. Consequently, Simba scales with the addition of delivery servers without com-promising consistency or fault-tolerance. The ideas presented in the report can also be applied for designing peer-to-peer streaming media delivery systems.

Motivation

The Simba architecture is motivated by the fact that networked workstations have enough residual bandwidth, computing power and storage capacity to store and deliver small fragments of video files reliably. A highly available video delivery system can be constructed by distributing the workload over these localized NOWs. To illustrate the point, lets do a back-of-the-envelope calculation for the Stanford Campus Network, which consists of approximately 45000 hosts spanning over 350 active subnets: A conservative estimate of 50MB of residual storage space on each machine would yield an aggregate storage capacity for approximately 2250 full-length movies. Similar arguments apply to residual processing power and network bandwidth. We have presented a detailed discussion of fault tolerance and scalability later in our report.

More Info...

Real-time video delivery systems have evolved over the past few years to stream video to large audiences with disparate connection speeds. The delivery servers used for this purpose usually require large processing power, main memory and secondary storage. Real-time video delivery over IP networks also requires a high aggregate bandwidth at the server side to support a large number of concurrent users. We have designed and implemented Simba, a distributed video server that amortizes the cost of real-time video delivery by distributing the workload and storage requirements over a network of workstations.

Video files that are served over Simba are assigned universally unique identifiers (UUID) and striped into a number of small fragments of the order of 1-4 MB. These fragments are stored on network-attached storage, or locally on the workstations. Computers that intend to participate as Simba video delivery servers listen for commands from a central server at a unique multicast address. The delivery servers are known as workers and constitute a dynamic set that changes with time. We shall refer to the central server as the master. The master occasionally sends a multicast message directing the workers to send load information. A scheduler on the master chooses a good worker to handle an incoming request for a video fragment from the client and hands off the request to the worker. Simba supports a single master and an unlimited number of workers.

Click here to retrieve the Simba project report in PDF format

A few screenshots...

	We decided to run 8 Simba clients simultaneously on a single machine to see how the client and the server performed. The 8 client windows you see on the left are running different fragments of the same movie and the fragments are being served from different servers.
	Same as above, except that we decided to run fragments from different movies.