GoofyxGrid Multi-client setup

[Gilthanis]

GoofyxGrid is a non-CPU intensive project. At least for now it is. However, some people would like to run more work units on their machines to better support the project. In this event, you would either need to run virtual machines, multiple clients, or both.

I recommend if running Windows to run your multiple clients from within a virtual machine if the box is used for anything else. Otherwise bear metal would be best.

 

[Gilthanis]

Credit for the procedure goes to DigiK-oz

Go to your c:\ drive or wherever you want to store the multiple clients and create a folder. Name it Multiboinc

Inside the folder you just created, create another folder and name it BOINC.

Inside the BOINC folder you just created, you need to create a cfg file using Notepad.exe. Name it gui_rpc_auth.cfg . The only thing inside the xml file should be whatever password you want to use in the event you want to connect to the clients using tools like BOINCtasks. Something simple like multi would simplify things.

Next, navigate to the BOINC data directory. The below photo is of XP and can be found under c:\Documents and Settings\All Users\Application Data\BOINC\

In Vista or newer, you will find the file in c:\Program Data\BOINC\
Copy the account_goofyxgridathome.net.xml file

and paste it into the c:\multiboinc\BOINC\ folder

Open a command prompt. Navigate to the c:\multiboinc\ directory Type
FOR /L %X IN (100,1,199) DO mkdir BOINC_%X

Hit Enter

You should see it create 100 BOINC directories

Next, type

FOR /L %X IN (100,1,199) DO copy BOINC\*.* BOINC_%X\*.*

hit Enter

It should copy a couple files into each of those directories

 

[Gilthanis]

Now that you have the multi-clients set up, you will want to run the command to start them all.
From the command prompt type:

FOR /L %X IN (100,1,199) DO start "BOINC_%X" /MIN "c:\Program Files\BOINC\boinc.exe" -allow_multiple_clients -dir c:\Multiboinc\BOINC_%X -gui_rpc_port 50%X

Hit Enter and you should see 100 clients start up. When the clients first start up, it will peg the CPU due to BOINC running benchmarks on each client. This will go down once it has finished.

Here are a few other commands.

To close all boinc.exe instances gracefully -
TASKKILL /IM boinc.exe

To force close all boinc.exe instance -
TASKKILL /IM boinc.exe /F

To set all the clients to No New Work -
FOR /L %X IN (100,1,199) DO start "BOINC_%X" "c:\Program Files\BOINC\boinccmd.exe" --host 127.0.0.1:50%X --passwd multi --project goofyxgridathome.net nomorework

To set all the clients to Allow New Work -
FOR /L %X IN (100,1,199) DO start "BOINC_%X" "c:\Program Files\BOINC\boinccmd.exe" --host 127.0.0.1:50%X --passwd multi --project goofyxgridathome.net allowmorework

To force all instances to update -
FOR /L %X IN (100,1,199) DO start "BOINC_%X" "c:\Program Files\BOINC\boinccmd.exe" --host 127.0.0.1:50%X --passwd multi --project goofyxgridathome.net update

 

[Gilthanis]

You can attempt to run more than 100 on a host by adjusting (100,1,199) to (100,1,XXX) on each of the lines above. However, I have seen that XP doesn't reliably run more than 100.

If you have spinner HDD's you may have IO latency, so keep an eye on your work units.

Each BOINC.exe instance will show up at GoofyxGrid as a separate host with the same name as the core system.

I recommend if you are spinning up a lot of these instances that you stagger them. Again, just change the (100,1,199) line as needed.

 

[pututu]

Now that you have the multi-clients set up, you will want to run the command to start them all.
From the command prompt type:

FOR /L %X IN (100,1,199) DO start "BOINC_%X" /MIN "c:\Program Files\BOINC\boinc.exe" -allow_multiple_clients -dir D:\Multiboinc\BOINC_%X -gui_rpc_port 50%X

View attachment 16370

Hit Enter and you should see 100 clients start up. When the clients first start up, it will peg the CPU due to BOINC running benchmarks on each client. This will go down once it has finished.

View attachment 16371

Here are a few other commands.

To close all boinc.exe instances gracefully -
TASKKILL /IM boinc.exe

To force close all boinc.exe instance -
TASKKILL /IM boinc.exe /F

To set all the clients to No New Work -
FOR /L %X IN (100,1,199) DO start "BOINC_%X" "c:\Program Files\BOINC\boinccmd.exe" --host 127.0.0.1:50%X --passwd multi --project goofyxgridathome.net nomorework

To set all the clients to Allow New Work -
FOR /L %X IN (100,1,199) DO start "BOINC_%X" "c:\Program Files\BOINC\boinccmd.exe" --host 127.0.0.1:50%X --passwd multi --project goofyxgridathome.net allowmorework

To force all instances to update -
FOR /L %X IN (100,1,199) DO start "BOINC_%X" "c:\Program Files\BOINC\boinccmd.exe" --host 127.0.0.1:50%X --passwd multi --project goofyxgridathome.net update

That should be C: not D: for this command line ""FOR /L %X IN (100,1,199) DO start "BOINC_%X" /MIN "c:\Program Files\BOINC\boinc.exe" -allow_multiple_clients -dir D:\Multiboinc\BOINC_%X -gui_rpc_port 50%X"", correct?

I'm still having problem. Let me check again.

 

[Gilthanis]

fixed the typo

 

[pututu]

I think I solved the problem after rebooting the rig. Let see how it performs tomorrow.

 

[motqalden]

Can confirm this is still working!

 

[ChristianVirtual]

The principle is still working and I do the similar way under Linux for all kind of projects, even GPU based ... copy the boinc folder and start multiple instances on different ports, not 100%sure for Windows but if you get the folder right I don’t see a problem why not

 

[motqalden]

Gilthanis what is the max number i can run like this on one host? i have been doing this for a few days now and results are good and seems stable. Added to 2 VM's today for more but i also wanted to try pushing past 100. How many are you running on a host?

Not sure why everyone isn't doing this as it's a really good way to boost our teams RAC without really affecting your other work.

 

[pututu]

motqalden, a max of 250 clients or 1000 (250 clients x 4 tasks per client) application tasks per machine/VM.

Sometimes ran into insufficient RAM, so had to scale some back when running projects that consumes huge amount of RAM like CSG.

 

[motqalden]

Thanks!

 

[MN Scout]

This appears to not work with Sztaki. All the instances will say work was requested too recently. I sat and watched the cmd output for one of the instances and the get work was 4 minutes apart, but the notice back from Sztaki was that work was requested 2 seconds ago. Maybe someone will say differently?

 

[Gilthanis]

Not all of the projects play well with multi-clients. I believe GPUGrid also does not play well with it IIRC.

 

[auntjemima]

This was pretty helpful! Running 350 instances.

 

[Deleted member 88227]

The max is 250. There is a 1000 task limit. So 250 clients running 4 tasks.

 

[Gilthanis]

That limit is per "host" (not client) and also keep this task limitation in mind if you are also crunching other work units on the same "host". It can cause issues going over that limit. I recommend limiting your clients per host to just 200 to give head room.

 

[auntjemima]

The max is 250. There is a 1000 task limit. So 250 clients running 4 tasks.

I have two systems. One running 100 and another running 250.

 

[auntjemima]

To force all instances to update -
FOR /L %X IN (100,1,199) DO start "BOINC_%X" "c:\Program Files\BOINC\boinccmd.exe" --host 127.0.0.1:50%X --passwd multi --project goofyxgridathome.net update

Setting up a VM to run some more instances and I noticed when running this command that it gives errors in the little command windows that quickly flash on the screen. I imagine the errors were present on other machines as well, but the VM was lagging and I just happened to see it.

To fix it I have to change goofyxgridathome.net to nci.goofyxgridathome.net.

 

[Gilthanis]

Yeah..I haven't updated it since Goofy changed the project (2 different times by the way). It used to be goofyxgridathome.net and then they changed it to gridcomputingcenter and then they made 2 projects. one for NCI and one for cpu intensive. When I have time, I may go back and do some editing.

 

[auntjemima]

Anyone else notice that their system used to run 4 tasks per instance and now they all run two? Not sure what changed, but I noticed my "in progress" was half what I would expect. Checked my 5 systems and they are all running two tasks at a time now versus 4. Hasn't really affected my output, as far as I can tell.

 

[auntjemima]

Nevermind, figured it out. You can download 1 tasks per type from GxG at a time. Turns out they don't have any ready for 2 of their tasks on the server status page.

 

[Deleted member 88227]

Project is down for maintenance.

http://nci.goofyxgridathome.net/forum_thread.php?id=302#2694

 

[auntjemima]

Project is down for maintenance.

http://nci.goofyxgridathome.net/forum_thread.php?id=302#2694

LOL! 3 to 5 days down time eh. Well, gotta do what they gotta do!

 

[pututu]

Time to take a rest and do something else....

 

[Deleted member 88227]

I think he is waiting for all the tasks to deplete before starting the maintenance which is probably why so many days. So keep crunching until the tasks run out so he can get started.

 

[auntjemima]

I think he is waiting for all the tasks to deplete before starting the maintenance which is probably why so many days. So keep crunching until the tasks run out so he can get started.

Can't stop, won't stop.

 

[Deleted member 88227]

But feel free to stop for a little while pututu so I can pass you up.

 

[pututu]

Skillz, if you want to overtake me, you need to get this.

 

[Deleted member 88227]

Skillz, if you want to overtake me, you need to get this.

Damn I wish that was yours.

 

[NickOfTime]

Hmm here a startup.cmd file that has a pause for each instance starting...

: Echo off

FOR /L %%A IN (%1,1,%2) DO call :doit %%A
goto :eof

:doit
start "BOINC_%1" /MIN "c:\Program Files\BOINC\boinc.exe" -allow_multiple_clients -dir c:\multiboinc\BOINC_%1 -gui_rpc_port 50%1
timeout /t 5 /nobreak
goto :eof

 

[Deleted member 88227]

Awesome thanks man

 

[pututu]

Thanks NickOfTime.

Not sure what I did wrong but somehow it only works when I changed the dos variables %1 and %2 to fix numbers as shown in the red hilited text below when running 250 instances.

: Echo off

FOR /L %%A IN (100,1,349) DO call :doit %%A
goto :eof

:doit
start "BOINC_%1" /MIN "c:\Program Files\BOINC\boinc.exe" -allow_multiple_clients -dir c:\multiboinc\BOINC_%1 -gui_rpc_port 50%1
timeout /t 5 /nobreak
goto :eof

 

[Deleted member 88227]

NickOfTime

We were discussing in Slack about your script. Could you possibly make a script that does all the steps above with one clickable file?

Once you get to the step of creating the multiboinc directory and on it could do all that automatically.

create dir
create sub dir
copy file(s) over
create multiple directories
copy file to multiple directories
start all instances

And after typing that you could eliminate the sub directory and the first file copy by just creating all the mutliple directories and copying the file directly from the primary install