client-less distributed file system?

T

Thuleman

Supreme [H]ardness
Joined
Apr 13, 2004
Messages
5,833
What I am looking for is a file system that will allow a Windows user to download a file from multiple sources without having to install a client application on the Windows users machine.



Basically like a BitTorrent, except without the client application. In the above example each LAN could have an "aggregator" that the users accesses to download the file, and then the aggregator assembles the blocks from different sources.

Ideally this would work through a download link on a web page, although that's not essential. What is important is that no software should have to be installed on the end-user computer to make this work.

Suggestions?
 
I don't know of any such thing. What's your goal with this?
 
There are online Bittorent services that don't require the use of a client. Just copy and past the torrent link and you're good to go. Torrent Relay and BitLet are the first two online torrent client services I can think of.
 
unhappy_mage said:
I don't know of any such thing. What's your goal with this?
Click to expand...

We have site licenses for a number of products, and thus far their distribution hasn't been an issue, just put up the ISOs on a sharepoint download site and that's that. However, the ISO for some of these products are becoming too large (4GB+) and there are issues with traditional http based download. It's also becoming an I/O and network bottleneck issue.

The idea is to distribute the ISO datastores across the different main offices to allow local users quicker access to the data, as well as getting around the 4GB issue that seems to creep up during web-based download.

Installing a BT application on the client machines is out of the question.

Danny Bui said:
There are online Bittorent services that don't require the use of a client. Just copy and past the torrent link and you're good to go. Torrent Relay and BitLet are the first two online torrent client services I can think of.
Click to expand...
That's exactly what I am looking for, except I need the Torrent Relay server, can't be using the 3rd party service, we'd need to run our own.
 
Why is additional clients a no go? One solution I can think of is using utorrent which is only a single exe. Than setup a private tracker. Since only your ppl will be using the tracker and have access to the torrents it should be just as private as any.
 
Why don't you just install a NAS at each location and have every local user use the NAS on their LAN? Just replicate the ISO between each NAS.

You don't need expensive hardware too, just a normal PC with a couple of hard drives + FreeNAS http://www.freenas.org/ would do the trick.

Will be MUCH MUCH faster to replicate the data once then access it locally on the NAS than what you are trying to do...
 
Thuleman said:
What I am looking for is a file system that will allow a Windows user to download a file from multiple sources without having to install a client application on the Windows users machine.



Basically like a BitTorrent, except without the client application. In the above example each LAN could have an "aggregator" that the users accesses to download the file, and then the aggregator assembles the blocks from different sources.

Ideally this would work through a download link on a web page, although that's not essential. What is important is that no software should have to be installed on the end-user computer to make this work.

Suggestions?
Click to expand...

OpenAFS might work. I have read about it but never tried it. Edit: I think openafs might need a client, but if you had a server at each location you could make IT the openafs client and then re-serve data via samba.
 
Thuleman said:
We have site licenses for a number of products, and thus far their distribution hasn't been an issue, just put up the ISOs on a sharepoint download site and that's that. However, the ISO for some of these products are becoming too large (4GB+) and there are issues with traditional http based download. It's also becoming an I/O and network bottleneck issue.

The idea is to distribute the ISO datastores across the different main offices to allow local users quicker access to the data, as well as getting around the 4GB issue that seems to creep up during web-based download.

Installing a BT application on the client machines is out of the question.


That's exactly what I am looking for, except I need the Torrent Relay server, can't be using the 3rd party service, we'd need to run our own.
Click to expand...

Now reading this, I'd suggest a linux box, drbd (async most likely) and samba. Another option would be to put a riverbed at each location.
 
re: why no BT client
There's a HUGE stigma attached to BitTorrent due to its predominant use for illegal file sharing. Installing a BT client on the desktop machines is a no-go due to political reasons. I agree that it would be the best solution, but the mere suggestion raised a bunch of eye-brows.

re: more hardware
Not going to happen.

The current situation is inadequate because there is only a single 100Mbit source for all of the ISOs that are being distributed (Microsoft, Adobe, SPSS, LabView, bunch of other stuff). That box gets hammered at times. The solution as I see it is to essentially replicate this ISO datastore locally, and then have some mechanism to pool all datastores whenever a client initiates a download to spread the bandwidth throughput among four or more source systems.

BT is the ideal technology, unfortunately the traditional implementation just won't work for us. I'll email the Torrent Relay folks and see what their thoughts on this are.
 
You could install a caching HTTP proxy (like Squid, which is free and works on Windows or any *nix platform) at each location and force clients to go through it for the ISO files. Then each ISO will be downloaded only once from the central location; further downloads will be site-local and thus not touch the central server. Only a few gigabytes of disk space are needed, but making the cache directory have as much room as the central repository will make sure that you never have to re-download the ISO images again.

This method has the advantage that only bandwidth that is needed is actually used: if nobody at site X wants app Y, that ISO isn't transferred there. The downside is that until someone requests Y, it isn't cached locally, so the download will go slowly the first time it's requested.

If the sites are all fairly reliable, you could configure clients to have a few parent proxies. Then each uncached download that happens may be routed through a few branches, distributing that ISO to more than one place at a time. The disadvantage is that initial download time will be limited by the slowest of the branch office connections.
 
Can't server 2003/2008 do distributed file system natively?I've never used it, but I thought it could do that.

Do that with replication turned on so all the data is on a few different boxes.
 
You must log in or register to reply here.
Back
Top