backbone of a large-file streaming system

[Aggelis Aggelis]

On 11/8/05, Nick I <clusterbuilder at gmail.com> wrote:
> Hi,
> I help maintain a Web site at
> www.clusterbuilder.org<http://www.clusterbuilder.org/>.
> You might have seen before that I have a section called Ask the Cluster
> Expert, where I am building a knowledgebase of cluster and grid information.
> When someone asks a question I am researching the answer to build this
> knowledgebase out. I received the following question:
>
> *"I am looking for a variety of solutions to be a backbone of a large-file
> streaming system providing thousands of concurrent download streams.
> Preferably commodity hardware and Linux, though I'm open to commercial
> solutions."*
>
> I am wondering if anyone here has suggestions of what applications will work
> best for this type of setup. You can respond to the question at
> www.clusterbuilder.org/FAQ
> <http://www.clusterbuilder.org/FAQ>or respond in email.
>
> Thanks,
> Nick
> _______________________________________________
> freebsd-cluster at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-cluster
> To unsubscribe, send any mail to "freebsd-cluster-unsubscribe at freebsd.org"
>
it all depends on job. This is the moto of clustering.
>From what i have read the solution is load balancing/redundancy  cluster.
Probably something like what is described in the article  at

http://www.countersiege.com/doc/pfsync-carp/

at the section "Something Bigger"

"This larger configuration is in place at a large educational
institution, providing load balancing and redundancy for a cluster of
web servers:"

As far  as the hardware , that depends on the kind and magnitude of
load that you expect.

there are also devices like eg this
http://www.coyotepoint.com/extreme.htm

Also read the pdf for what these devices are offering
http://www.coyotepoint.com/pdfs/extreme_05.pdf.

Anyway the key point here is on load balancing  and redundancy


      client
        |
        |
    internet
        |
 Load Balancer/Firewall
    |          |             . ..........  |
    |          |                           |
    |          |                           |
  server1 server2                  serverN