New fold-server release

U

unhappy_mage

[H]ard|DCer of the Month - October 2005
Joined
Jun 29, 2004
Messages
11,455
Hey guys,
For those of you interested in farming, a new release of my fold-server distro has just hit the virtual shelves at sourceforge. If you've looked at the idea in the past but gave up on it or tried and failed, this is the one to try it on. Instructions on installing and using fold-server are available here, or you can email me for help.

I hope this will make things a bit easier for those of you with farms, and enjoy!

Fold on!
Unhappy_mage
 
Sweet. If I had the time and money, I would definately do it. However, college doesn't pay the bills (yet). Someone else try it!
 
So, does this use samba yet so I can monitor the clients using EM3 ?
 
I was really looking for something like this back in the day when G@H was still going.

I still have most of that old crap. Too bad it's too slow for Folding.
 
I'd suggest it. If you've got a spare disk sitting around, install it on that instead of overwriting so you've got a fallback position if I screwed something up, but it should go smoothly.

Reasons: the client dns issue is resolved.
the netconfig script is run over SSL (and everything http, too) now so it isn't quite as hackable as before.
it's based on DSL 0.9.3 instead of 0.9.2.
nerk: it does indeed use EM3! go to \\192.168.1.15 in a my computer window to check it out. everything's in share/, based on client name.

If you need help setting stuff up for this, I should be on around 9 or 9:30 at the latest.

 
SWEET, I'm just waiting for the CPU of node #1 to arrive, once things are running smoothly the orders for #2-4 go out.
 
Q: The saved work that is stored on the server, is that in a "Linux" only format?

Could I move the WU's from this to a XP platform?

I know this sounds wierd, reason is because I can fold 24/7 longer (2 weeks) with my corp boxes if they are under XP (I need my app/drivers ect. loaded to burn in other gear).
 
marty9876 said:
Q: The saved work that is stored on the server, is that in a "Linux" only format?

Could I move the WU's from this to a XP platform?

I know this sounds wierd, reason is because I can fold 24/7 longer (2 weeks) with my corp boxes if they are under XP (I need my app/drivers ect. loaded to burn in other gear).
Click to expand...
I don't know, I'll try it. If it works at all, it'd be really easy to do: the clients all save their data to a samba share, so you could just copy over the entire directory. The client itself isn't copied over every time, so there's no linux client executables or anything. It does create an empty FAH500-Console.exe for EM3 to notice, but it's easy enough to just skip copying that file over.
 
unhappy_mage said:
I don't know, I'll try it. If it works at all, it'd be really easy to do: the clients all save their data to a samba share, so you could just copy over the entire directory. The client itself isn't copied over every time, so there's no linux client executables or anything. It does create an empty FAH500-Console.exe for EM3 to notice, but it's easy enough to just skip copying that file over.
Click to expand...

One thing I don't know about is how Stanford keeps track of whom it sent what. The reg keys in XP for example. I'm not sure if the box that uploads the data needs to be the same one which down loads the data? Make any sense? I'm thinking it don't matter.
 
marty9876 said:
One thing I don't know about is how Stanford keeps track of whom it sent what. The reg keys in XP for example. I'm not sure if the box that uploads the data needs to be the same one which down loads the data? Make any sense? I'm thinking it don't matter.
Click to expand...

It doesn't matter. Stanford will take the unit back no problem. I've done this many times when I was stuck on dial up and WUs wouldn't upload properly on one box but work fine on another. They were both WinXP boxen, though they did have the different IDs. I've also done XP->2k and 2k->XP although that is the extent of what I've done. I have no Win98 boxen or anything else right now to test anything like that.

 
Nice sig, I just killed off a few brain cells... It looked like I was talking to myself, took me a few to figure things out.
 
I tried copying over the WU this morning, but I hadn't set the date on my fold-server box, so it thought it was 2002 and the box I copied it to said "Oh, deadline passed, try for a new WU". So I set the date right on fold-server, and it said, Oh wait, deadline passed, get another WU. I suppose the smart thing to do would be set the XP box's time to 2002 temporarily and see if it works, but I've never been known to do that. :rolleyes:

And then I had to leave for school :( . Oh well. More tests to come this afternoon.
No [H]'s here because of school.
 
Is it possible to get the scripts to do all this running on a standard linux install? That would be top class, as i could get it running on my file server, and save myself some grief when i eventually get round to starting the farm im planning.

Also, sorry to thread hijack, but as farm as proc choices for farming goes, is it worth getting the hyperthreaded chips over say, an equivalent speed celeron D? In essence, does hyper-threading add ~50% to the points per day, for maybe a 10% cost increase?
 
I think the HT adds about 10-20% to your points... There are others here who can better answer that (and probably back it up with data)...

back ontopic... I need to find some time to give this a try... I need another PC though... I don't want to switch mine or my mom's over to linux just yet... she has a hard enough time as it is....


Keep on Folding!!

 
Herulach said:
Is it possible to get the scripts to do all this running on a standard linux install? That would be top class, as i could get it running on my file server, and save myself some grief when i eventually get round to starting the farm im planning.

Also, sorry to thread hijack, but as farm as proc choices for farming goes, is it worth getting the hyperthreaded chips over say, an equivalent speed celeron D? In essence, does hyper-threading add ~50% to the points per day, for maybe a 10% cost increase?
Click to expand...
It is *possible*, and if you want to get this going, email me and I'll send you more information, but the distro is definitely 10x easier.

Like OSUguy said, 10-20% increase in points. This is also true of non-celeron chips. Stanford prefers that you only run one instance per real processor, though, so if you're in it for the science celeron D's are a win.

Osuguy: keep in mind this takes almost no machine to get going. You can run the server on a pentium machine, it doesn't take anything fancy. My server is an duron 1200, but I've run it in a virtual machine in Bochs before, which is really slow. I want to try it on a real box at this kind of speed, though. I'll have to dig through all my old junk.

Come to think of it, I should look into integrating smoothwall/m0n0wall features in this. Router machines would make a really easy place to set this up.
 
unhappy_mage said:
IOsuguy: keep in mind this takes almost no machine to get going. You can run the server on a pentium machine, it doesn't take anything fancy. My server is an duron 1200, but I've run it in a virtual machine in Bochs before, which is really slow. I want to try it on a real box at this kind of speed, though. I'll have to dig through all my old junk.
Click to expand...

Well, there are 5 PCs in the house.... 3 are running.... one is dead right now (KT7A that won't post)... and one is my mom's old P-133.... I'm just looking for an excuse to get another folding box in the house ;-)



Keep on Folding!!

 
Ok cool, prob gonna go for the digging an old machine out option though, Just so i understand everything right, it goes
network->NIC1
then the farm all connects into NIC2 right?
Also how well does it play with dhcp? Is it a sort of fine in principle but maybe a bit of a ball ache kind of thing? Or a definate no go area? I ask because experimenting in the past ive had trouble with setting up static ips on the lan, mainly because for some reason my isp decides to change DNSs every couple of days, same 4 or 5 ips, just cycles them, annoying as hell.
 
Herulach said:
Ok cool, prob gonna go for the digging an old machine out option though, Just so i understand everything right, it goes
network->NIC1
then the farm all connects into NIC2 right?
Also how well does it play with dhcp? Is it a sort of fine in principle but maybe a bit of a ball ache kind of thing? Or a definate no go area? I ask because experimenting in the past ive had trouble with setting up static ips on the lan, mainly because for some reason my isp decides to change DNSs every couple of days, same 4 or 5 ips, just cycles them, annoying as hell.
Click to expand...
The network is set up like you said.

As long as the router (gateway) you point fold-server to (a linksys box, m0n0wall, whatever) knows how to do DNS you should be OK. I think. Maybe. If it doesn't work automatically, then you'd have to set the DNS for fold-server itself manually, but it does dns for all its clients, so as long as it knows where to look, they'll know where to look too.

BTW, it has to have a static IP in order to route properly. Imagine there's someone relaying all conversation between you and a friend. In order to talk to your friend, you have to know where to find the middleman. If he keeps moving around, then you'll have no idea where to locate your friend.

OSUguy: I know the feeling. Right now we're down to 7 computers in the house. I feel lonely ;)
 
Cool, d/led the iso, reading the notes, i assume external netmask is the network side, and internal the farm side? Think thats about it i was confused by. FOund an old p2 in the loft its going on after tea
 
Yep, you got it. I'll clarify that documentation when I get a chance. Right now it's sort of a Frankenstein of stuff I wrote for the first release and stuff I've tacked on since, so it may be a little inconsistent. I'll get to it :cool:
 
Hey!!! Don't forget about me!!!!

I'm not sure is I missed the answer, possible to move WU's from Linux server to XP enviroment?
 
I remember you! You were that guy...

I'm still waiting on Stanford for this, basically. Squid (my mandatory proxy server :( ) is still not completely working, so it takes a while to get a WU. When I get one I'll try, but until then I really can't do anything to check. Updates will be coming when I get a WU...

Edit: woohoo, got a WU. Copied over the files, everything worked. Finished a frame with no problems. Didn't wait for an entire WU to finish (this machine lacks th3rmal 3ffici3ncy) but it looks good to go.
 
So just move everything over(/work logs config ect.) and replace the fah502-???.linux thing with the fah502-console.exe (winblows deal) and all should be cool?

Ahh second question: What server should I use with 50 nodes/2.53 gzh each........

Ahh third question: You think this would work for 126.5 gzh of boxen?

If I got the 50 boxes, they will be set to LAN boot. I can have them shipped w/ blank hard drives as they will net boot. Open box, place on shelve and fold. Then ~ 2 weeks before deployment I'd just load my XP image, move the WU's over and keep going till the boxes go out the front door.
 
So just move everything over(/work logs config ect.) and replace the fah502-???.linux thing with the fah502-console.exe (winblows deal) and all should be cool?
Click to expand...
Actually, the problem there is that the Linux end makes a fake (empty) FAH500-Console.exe so EM3 will find it as a valid folding@home directory. Since it isn't the real executable, you wouldn't want to overwrite the real one with the empty one.
Ahh second question: What server should I use with 50 nodes/2.53 gzh each........
Click to expand...
Anything really. The only limiting factor I can tell in terms of expansion is the network bandwidth. Eventually you'll have so many systems sending/receiving WUs that you'll swamp the t1... Oh, and don't turn them all on at once. Each machine has about 12 MB of data to get before it's completely booted. You can opt to make this smaller by removing some of the cores from one of the files (every core in use by Stanford is sent over the wire on boot!) since you'll probably never use most of them. At 100mbit speeds, that means about 1 second (at least) has to elapse between client boots. 3 or 4 would be better, since the traffic isn't all at once.
Basically, the server just handles a few simple network things that it can do with half its attention, and the real work it does is to fold. A pentium II could probably handle 500 boxen, if it had gigabit ethernet.
Ahh third question: You think this would work for 126.5 gzh of boxen?
Click to expand...
It damn well better. ;)
If I got the 50 boxes, they will be set to LAN boot. I can have them shipped w/ blank hard drives as they will net boot. Open box, place on shelve and fold. Then ~ 2 weeks before deployment I'd just load my XP image, move the WU's over and keep going till the boxes go out the front door.
Click to expand...
How can I get into your line of work? I need the points... :D
edit: and the $ :rolleyes:
 
true dat... where DO you work?

oh and congrats on the update (don't think i told you yet:p)
 
One last question (yea not really but...)

My "reason" for running these systems 24/7 is to break them in. Yea, yea it's a 1/2 way valid reason...sort of.

Say you take 10 boxes, boot them off the server (they grab DHCP IP's) and one craps out, how am I going to physically know which one of the boxes are dead? All headless boxes.

My last run of these, ~ 30 or so, I sat them on a bench, imaged them with an XP build-installed the folding doo hicky, set to a static IP and labled them on the outside of the box with IP. It's not bad, just not great either.

So, in theroy if I paid attention to EM3/Remote desktoped into them I can tell what's working. Somethings EM3 would complain about a box, I'd just reboot it.

My plan this time was remove from packaging, net boot and let run, install in end use envirmont with XP image move the work WU's over and jam some more till the last minute possible when I need to image with the real XP build.

And no, the MAC's are not listed on the outside of the box. Maybe I could ask/pay extra to have the MAC's listed onthe outside since the builder needs to power the systems with a monitor anyways.

Thanks for all you help!
 
the only way I can think of is put mac addresses on the outside of the boxen. then compare ip address on the non-responding box to the dhcp logs.

that, or put them in a row, and turn them on by going down the row, starting one every few seconds... it should get sequential IP addresses.
 
Power loss would really screw up the IP scheme from the one by one power up.

IE ummm now how many can I plug in here....
 
marty9876 said:
Power loss would really screw up the IP scheme from the one by one power up.

IE ummm now how many can I plug in here....
Click to expand...
The dhcp server will remember them for 2 days, I think *checks* yep. So as long as you turn them back on within 2 days they'll get the same IP. I gotcha covered, man.

You can usually get those power strips at Radioshack @ 2/$10... that's my suggestion. As for lan connectivity, you're on your own. Cheap 24-port hubs from ebay?
 
Does your server act as the dhcp server? (I'm too lazy to look it up).

I've got plenty-o surge strips (spendy ISOBAR ones...hehehe, crap I've got enough voltage regulators if needed too). Before I was running one surge out of another, way not cool.

I've got an order in to get a few more 20 amp circuits installed, now that's sad. I should have around 80 amps across 4x20 amp lines. Think I can run a few boxen from that?

I dunno know about all this. I'll use this for sure with the "personal" farm, on these corp boxen, XP is included so it's almost why not use it. It's gonna be 5-6 long hours to image/setup all these under XP, vs. your insto farm. Huh, insto farm I like the sound of that.

Not if I can figure out how to make a ghost bootable CD with nic drivers my life would be better. Right now, it's two floppies to boot ghost for a ghost cast session. The actual image lay is 1.5 minutes over the wire.
 
Yeah, it does dhcp on its own subnet so it won't screw up your existing network.

I have in the past figured out ghost bootable CDs, but I only have the home edition so you'd need to type in the serial # every time you imaged a machine. Ghost 9.0 (i believe) comes with a bootable WinPE cd that has NIC drivers, but because it's booting Windows from a CD it takes like 5 minutes to start up. And I don't think you can script it so it would immediately start in on the imaging... Oh well.
 
I'm close to the bootable CD, I'm just gonna finish it before these arrive.

I got a guys CD form the [H], just won't connect to the session.

I've got a bootable USB drive, crapper is it's works great w/o a hard disk in the system. Config the bios priorty and it works, till you remove the USB drive. So to boot from USB, config bios, boot every time, config bios....boot.

yuk
 
If i've having some problems with it, what's the best way to get some help?

I'm not a *nix noob, but I'm not much better. As far as I can tell the modprobe to set the nics up are failing. I'll post the exact error msg tonight.


 
I have a few to try, one is an intel pro 100, the other two are 3com 905s ( one is the netboot version, the other the non-netboot).

I've tried all three combinations of them.
 
Hmmm, funny, The two NICs in mine are both 3c905s. Mage will probably be the one to ask.

Oldbenwa
 
Those should definitely work, all 3 of them... What do you get if you type "ifconfig -a" at a console?

This thread is old, wow.
 
ok, nm, NICS are fine, other than my home network is 192.168.0.0

but i still get these errors:

Configuring network interfaces.... route: SIOC[ADD|DEL]RT: Network is unreacable

route: SIOC[ADD|DEL]RT: File exists

Starting HTTP Server : Socket: Address family not supported by protocol

fold start called
tar (child): /foldsave/fold.tar.gz: Cannot open: no such file

any clue?
 
nerk01 said:
ok, nm, I don't think it's the nics, I think it's because I'm using 192.168.0.0 for my home network. Looks like it fails to setup dns, gateway etc. What would it take to fix this?
Click to expand...


Correct me if I'm wrong.

Change your network to 192.168.1.x

Fold server will be at 192.168.1.15

Need to do a static route throw the router "Network" 10.x.x.x to 192.168.1.x or vice versa on that one.
 
You must log in or register to reply here.
Back
Top