FlexRAID (Flexible RAID)

hm, I suppose its possible I might not have closed one out correctly, but I definitely only intended to have one running. I'll try it again tonight.
 
Thanks.

FYI, RC3 is coming out tonight with the much needed performance improvements.
 
Great news. I have been putting off doing the parity backups of my drives after I tried one and estimated a backup time of abut 27 hours :).
 
LurkerLito said:
Great news. I have been putting off doing the parity backups of my drives after I tried one and estimated a backup time of abut 27 hours :).
Click to expand...

What are your system specs?

I am currently getting a 5x increase on my "production" system and 8x on a test system.
 
spectrumbx said:
What are your system specs?

I am currently getting a 5x increase on my "production" system and 8x on a test system.
Click to expand...

Just a stock Q6600. I hope to see that kind of speed up, I don't mind leaving the machine to work while I am not around, but when processing times hit when I want/need to use the machine I have a tendency to put off those time consuming things ;)
 
LurkerLito said:
Just a stock Q6600. I hope to see that kind of speed up, I don't mind leaving the machine to work while I am not around, but when processing times hit when I want/need to use the machine I have a tendency to put off those time consuming things ;)
Click to expand...

You can easily reduce the number of CPU cores FlexRAID uses at any time while it is running through the "max" command.
In my setup, I can stream multiple DVDs and write files to the storage server just fine while the RAID process is running without even reducing the CPU load.

Anyway, RC3 should be up shortly.
 
Update: 05-27-2008:
RC3 is out!
This release brings some seriouly needed performance improvements.
5x more performance on average over RC2 on my test systems.
Better systems should see even more performance scalling.

Make sure to read the change logs and the updated documentation on the website.
 
Excellent work. I just started a processes and it is significantly faster. 27 hours to 3 hours is my first estimate but if that rate keeps up that is huge improvement. It's 10% done and it took a few hours to get this far with RC2.
 
LurkerLito said:
Excellent work. I just started a processes and it is significantly faster. 27 hours to 3 hours is my first estimate but if that rate keeps up that is huge improvement. It's 10% done and it took a few hours to get this far with RC2.
Click to expand...

The first few runs will be your slowest runs.
It is long to explain, but it gets faster with every run.

So, it is better to do a few warm up test runs before RAID'ing large data sets on a fresh install.

Also:
- What is your processes property value?
- How is the CPU utilization?
 
Doh it crapped out on me at 48%. I can't seem to find the log. all i got was a Error Read error. I have my create.txt set to 4 processes. Hmm my metadata is 0 bytes. Maybe one of my drives has a read error on one of the files.
 
I erased the data and removed one directory (from a drive that is louder than the others and I think might have bad blocks) from my create.txt. and I'll give it another go.

I am still using 4 as my processes property value in create.txt and my CPU Utilization maxes at around 60% max, but usually it is lowerin the 30-50% range.
 
Trying increasing the processes value till you get to 95% CPU utilization.

Using the max command you can increase/decrease the processes value.

So, play with it and see if performance improves.

Of course, make sure you have your file permissions set right and that no other process is locking any of the files.
 
Well it stopped again at about 66-67% with a read error. Is there anyway to find out what files it was processing at the time of the error? The log file really has no information in it other than the actual error message:
Code: 
[2008-05-28 00:22:34,179] ERROR: Read error
java.io.IOException: Read error
	at java.lang.Void.<unknown>(Unknown Source)
	at java.lang.Void.<unknown>(Unknown Source)
	at java.lang.Void.<unknown>(Unknown Source)
	at java.lang.Void.<unknown>(Unknown Source)
It also seems that when it gets the error it tries to abort but never finishes aborting. It just gets stuck. The problem is I have to close the server box and it the metadata becomes just a zero byte file with no info in it. I'll run check disk on all my hard drives just to be sure.
 
LurkerLito said:
Well it stopped again at about 66-67% with a read error. Is there anyway to find out what files it was processing at the time of the error? The log file really has no information in it other than the actual error message:
Code: 
[2008-05-28 00:22:34,179] ERROR: Read error
java.io.IOException: Read error
	at java.lang.Void.<unknown>(Unknown Source)
	at java.lang.Void.<unknown>(Unknown Source)
	at java.lang.Void.<unknown>(Unknown Source)
	at java.lang.Void.<unknown>(Unknown Source)
It also seems that when it gets the error it tries to abort but never finishes aborting. It just gets stuck. The problem is I have to close the server box and it the metadata becomes just a zero byte file with no info in it. I'll run check disk on all my hard drives just to be sure.
Click to expand...
Alright, I will pm you shortly with a special build.
 
Update: RC3 build 002 is out!
Build 001 had issues (code intrusion from the redesign code base) I had to fix.

Please uninstall 001 and install 002.
 
Update: 05-29-2008:
Alert! RC3 build 003 is out.
Build 001 and 002 were bad. I did not properly test those earlier builds given my busy schedule.
My apologies to all those inconvenience. :)

For those affected, please run the "rsynch" task and then one of the "validate" or "verify" tasks.
If the "validate" or "verify" task fails, then you will need to re-create the RAID.
 
"forgot to update this thread" bump... :)

Build 008 and Web UI Client are up.

Preparing for build 009.
 
You must log in or register to reply here.
Back
Top