Looking for some advice

[Booker The Geek]

Mod's, Please feel free to move this thread if I have inadvertently posted it in the wrong forum.

First, please take a gander at my crude picture attempt that I made to help me describe my question better.

Link to this Picture Link to "Home Network"

So my questions center around the Server and SAS Array's

1. Can I (Do I need to) use multiple servers for load balancing?
2. Will each server be able to access the same information on the SAS Array's?
3. Assuming each SAS Array represents a server rack, each w/x5 4u case's for storage, and x1 2u server, what would be a good way to set this up?

My idea is to be able to offer other local people storage space for their use to backup stuff (Movies, music, photos, business stuff, whatever they wanted)

I would like to be able to put about 250 TB per SAS Array. (Henceforth to be referred to as "S.A.") Which would make it about 50TB per 4U case. 50TB after Raid setup.

btw...Is it possible to use This Card to calculate parity in a raid?


I am planning to piecemeal this together. FYI... I have about $3k for the server and about $8-9k for the first S.A. I plan on expanding later down the road.

• What would be a good operating system to run on the S.A.?
• With write speed not being to much of a concern, (More worried about read speed and data security), would it be better to use a hardware raid 60 w/two hot spare's, or a triple parity software raid w/three spares?
• Best Hardware based Firewall?

 

[Booker The Geek]

Reserved for continuation of OP

 

[ST3F]

Can I (Do I need to) use multiple servers for load balancing?

No.

Will each server be able to access the same information on the SAS Array's?

Yes, by Ethernet protocol

Assuming each SAS Array represents a server rack, each w/x5 4u case's for storage, and x1 2u server, what would be a good way to set this up?

Why 2U ... why Not 3U with 16x HDD Hotswapp ?

I would like to be able to put about 250 TB per SAS Array. (Henceforth to be referred to as "S.A.") Which would make it about 50TB per 4U case. 50TB after Raid setup.

Why SAS ?
.... SAS drive are up to 1 TB capacity ; on 4U case : 24 drives = 24 To Raw

btw...Is it possible to use This Card to calculate parity in a raid?

Never ever seen in the universe.

I am planning to piecemeal this together. FYI... I have about $3k for the server and about $8-9k for the first S.A. I plan on expanding later down the road.

Euhhhhhh .... oO
For Less on ZFS, I'm about $ 10 800 for the server node !!

[*]What would be a good operating system to run on the S.A.?

Solaris 11 / OpenIndiana / Nexenta

With write speed not being to much of a concern, (More worried about read speed and data security), would it be better to use a hardware raid 60 w/two hot spare's, or a triple parity software raid w/three spares?

ZFS RaidZ2 with 4 hotspare / 4 U[/QUOTE]

 

[Booker The Geek]

Originally Posted by Booker The Geek
Can I (Do I need to) use multiple servers for load balancing?

No.

I was thinking that If I were to have a lot of people accessing the server at the same time, (upward of 50), that it would speed things up to have to servers to make things go faster. I assume that one server, if powerful enough, would be able to handle it, but are you sure that two of them, maybe not quite so powerful (or as expensive) might be able to handle it just as well or not better? I was also thinking of redundancy.

Answered by ST3F
Why 2U ... why Not 3U with Cox HDD Hotswapp

I think your are talking about this case? Or one of these?
The reason why I chose a 2U, either this or this one, is because I am thinking of putting the main computer in it and the storage computers in the 4U case's.

I guess I am imagining a tower. The main lobby would be the 2U server, controlling access to the tower. The other floors of the tower would be the 4U case's containing all the storage. (I don't know if that illustrates what I have pictured in my head or not, but it is the best analogy I can come up with. )

Answered by ST3F
Why SAS ?
.... SAS drive are up to 1 TB capacity ; on 4U case : 24 drives = 24 To Raw

Really? What about Western Digital, Seagate or either of theses links, 2TB or 3TB

This case, your right, is 24 drives. So 24 3TB drives, if put into raid 6+0 w/spares, per case, would leave me with what...?

• 20 / 2 = 10 Remember the "+0".
• 10 - 2 = 08 Losing Parity Drives
• 08 - 1 = 07 Lost the spare
• 07 * 2 = 14 Remember the "+0"
• 14 * 3 = 42 ... still more then 24TB raw..

Why else SAS?

• Better Made, so better
  • MTBF
  • BER
• SCSI command set
• Hot Plugging

Answered by ST3F

Originally Posted by Booker The Geek
btw...Is it possible to use This Card to calculate parity in a raid?

Never ever seen in the universe.

Interesting, as it is for sale here, and all of here. btw... that did not answer my question. I was wondering if using that card would be faster then the CPU to scrub a raid or to calculate parity. I believe that both of those tasks are highly parallel.

Answered by ST3F
Euhhhhhh .... oO
For Less on ZFS, I'm about $ 10 800 for the server node !!

... what?

Proposed build for Storage array...

• Case @ $300
• MB @ $470
• CPU @ $625
• Ram @ $100 (x4, so $400)
• HDD @ $400 (x20, so $8k)
• PSU @ $470
• Extras @ about $250 ish

So looking at it, I guess I was off a little, SA will cost about 10.5k.
Not an unreasonable ammount, I bet I can get that price down by at least 10-15% by shopping around and looking for deals.


Would this picture help understanding in what I would like to do?

 

[vr.]

I think what ST3F is saying is that the Tesla card has to have some way to interface to the "calculate parity in a raid" you mention. I too have never seen RAID able to take processing inbound from another source either so I don't know that it's doable without some development effort. Maybe it is, but I've never seen it.

Appliance based load balancers I've looked at recently ($5K-$15K per node) were all active/passive and limited to 2 nodes. I'd like to think there are beefier ones out there(?) but it's certainly a scale vs cost factor you will want to research more.

A 4U, 45-bay Supermicro chassis would get my vote over a Norco + PSU. You add those prices together and you're already there and SMI has much wider distribution so getting replacement parts 3-7 years from now should be easier.

Since you're using Ethernet anyway, separating your access/server network from your storage network, I'd probably use NFS between the storage/access servers. I've never liked how iSCSI makes me hard-carve out buckets of storage that I can't dynamically-on-the-fly tap into when one customer isn't (and won't ever be) utilizing the space at their speculated growth rate.

 

[hotcrandel]

Trying to BYO a system with that much disk space might be a mistake, especially if its business critical.

Maybe you should start with a more basic explanation of what you need, like this;

Several servers serving up large multi TB network shares over CIFS to Windows 7 Desktops.

Total space - [250TB?]
Number of concurrent users - 50
Gigabit infrastructure throughout the building? FC, or Infiniband available?
Average estimated data delta per backup / Frequency of backups
Maximum amount of downtime / maintenance per year.

Before you even start, you should consider how you're going to back it all up. For what you're trying to do, unless it doesn't matter if the system fails, I would solve this problem by purchasing nearline SAS SAN units, many companies make good products to do this. If you're some sort of ZFS / Solaris ninja, I could see it being done that way, but trying to self admin the system seems like it would be the job of more than just one man.

 

[Booker The Geek]

I think what ST3F is saying is that the Tesla card has to have some way to interface to the "calculate parity in a raid" you mention. I too have never seen RAID able to take processing inbound from another source either so I don't know that it's doable without some development effort. Maybe it is, but I've never seen it.

Makes sense when you explain it that way. I was hoping/thinking that a software raid setup might be able to take advantage of that card.

Appliance based load balancers I've looked at recently ($5K-$15K per node) were all active/passive and limited to 2 nodes. I'd like to think there are beefier ones out there(?) but it's certainly a scale vs cost factor you will want to research more.

lol... Out of my budget.

A 4U, 45-bay Supermicro chassis would get my vote over a Norco + PSU. You add those prices together and you're already there and SMI has much wider distribution so getting replacement parts 3-7 years from now should be easier.

Were you thinking of this case? It is a good point, and I will most likely save some cash on being able to get 1TB drives other then 2TB drives.

Since you're using Ethernet anyway, separating your access/server network from your storage network, I'd probably use NFS between the storage/access servers.

Researching.... Found This paper on it. While a slightly heave read, makes sense and I have to agree with you. It was my understanding that with iSCSI multiple "access" servers (for lack of a better term) could access the same storage servers, basically at the same time. Would this be possable with NFS? ( or am I completly out of the balpark and making no sense at all?)

Trying to BYO a system with that much disk space might be a mistake, especially if its business critical.

wahahaha...
It probable is a mistake. But, it is not business critical, and I would like to try/learn. Best way to learn how to do something is by trying, then failing, and then trying again. (Best way, not the fastest or least expensive, but I think that I will learn the most this way. )

Maybe you should start with a more basic explanation of what you need, like this;

Several servers serving up large multi TB network shares over CIFS to Windows 7 Desktops.

Total space - [250TB?]
Number of concurrent users - 50
Gigabit infrastructure throughout the building? FC, or Infiniband available?
Average estimated data delta per backup / Frequency of backups
Maximum amount of downtime / maintenance per year.

hm......

I need a massive amount of storage space that is to be accessed from over the internet by mulitiple users using hardware and/or operating systems that I have no controll over. Most of the users will have READ ONLY access to a shared portion of the storage that I will be administrating, while other will need READ/WRITE access to variable amounts of storage space that they only have access to.

• Number of cuncurrent users?
  About 10 to start, but hopefull around 150 within a year.
• Infrastucture in the building? None.
  All that I will have is a room (A very well A/C'd room) apx 15' x 25'. The internet interface is Fiber provided by the local municapality. I have been told that I will be recieving 150Mbps, both upstream and down.
• hm....
  People backing up, I assume about once a day.
  People accessing other parts of storage? Imagine a large collect of user created content being shared.
• Maintenance
  I am planning on apx 8hr downtime one day a month for maintenance. I assume that figure is off, but I do not know by how much or in what direction.

Before you even start, you should consider how you're going to back it all up. For what you're trying to do, unless it doesn't matter if the system fails, I would solve this problem by purchasing nearline SAS SAN units, many companies make good products to do this. If you're some sort of ZFS / Solaris ninja, I could see it being done that way, but trying to self admin the system seems like it would be the job of more than just one man.

I have considered this. And I do not know at this momement in time as to how I will be backing it all up. It does matter if the system fails, that would not be good. You think it would be better/easier/cheaper to get "SAS SAN" units from an OEM and do it that way? I most likely would be, lol... But where would be the fun in that? And no, I'm not a ZFS / Solaris ninja, as I have never used them, or even been in the same room as a computer that used them. But I will enjoy learning as I go along.