SAN: DS4000 guru's please help

S

sixpac

Limp Gawd
Joined
Jun 8, 2004
Messages
272
I have a DS4300 (at work).. now I can't get a straight answer from IBM san support, so I am asking you all.

Is it recommended to have READ and WRITE cache enabled on Raid 5 logical drives?

How about Raid 0 drives or the flash copy logical drives?

I've been trying to find a decent Red Paper on this but I can't find anything yet.

Thanks
 
On the DS4000 series, you want Read and Write Caching Turned on for Raid 5, and Raid 0, Raid 10 or Raid 1 you want it turned off. Write Caching on a Mirrored system is not the best.
 
Thanks for that. now in Windows 2003 (where I have these logical drives attached). should I turn on (in the drives policies) "Enable write caching on the disk" and/or "enable advanced performance"?

Thanks for you help.
 
Nope, I dont recommend that because its software based, therefore susceptible to the OS.

The Idea of Drive Caching in the operating system is a backfetch to ram, where it queues write operations to the disk, allowing bulk write operations to occur without I/O Delay.

The problem with this is that its not battery backed, E.G. If you lose power, or the server has a hard stop (problem on the system board) and windows goes kaput, (blue screen, etc.al) you "may" have data inconsistencies by data waiting in cache that hasnt written yet (which causes data corruption or loss).

You have to evaluate this on a needs basis. Considerations you need to take a look into is, how much are your storage processors being used on the DS4000. Typically people who use a DS4000 make one gigantic Raid 5/10 Set across the entire set of disks, and create luns myself personally,(and usually the larger enterprise community) who use larger SAN infrastructures make multiple raid sets based upon the "customer's" needs.


Things that you should consider when developing your SAN

1. Am I Redundant? (E.G. Storage Processors, SAN Switches, Dual Path HBA's)
2. In Calculating I/O Rate - How many Disks am I using, How many Raid Sets Do I have
3. How many hosts am I serving with my SAN?
4. What type of disks am I using (Yes They Sell PATA/SATA/SCSI/SAS Drives) for my San

When it comes down to it, your write caching wont be truly effective unless you are on the higher end >70% of your Max I/O Rate for the Disks you are writing to.

Go by your actual need, not just what sounds good on paper. If you want further help in discussing your SAN go ahead and pm me, ill give you supplemental contact info, and we can work through it. A successful SAN implementation is key to ensuring a high uptime, which with a SAN you need to have, cause you can take out all of your hosts abilities to serve business functions with one pull of a power plug.
 
NauticaONE said:
The problem with this is that its not battery backed, E.G. If you lose power, or the server has a hard stop (problem on the system board) and windows goes kaput, (blue screen, etc.al) you "may" have data inconsistencies by data waiting in cache that hasnt written yet (which causes data corruption or loss).
Click to expand...
Even at the application level, this can cause problems. If your app makes a transaction, and some of those IOs make it across the wire to the storage device and some don't, you can still end up with an inconsistent state from the application level.

OTOH, if your application is properly designed (logging, etc), though, it won't have an inconsistent state no matter where the fault occurs, and you don't need to worry in the long run about whether the host machine goes down.

The whole concept of "susceptible to the OS" has always seemed strange to me. If the OS has a showstopper bug (data corruption, whatever) you're screwed anyways. That doesn't mean you have to blindly trust it, but looking for problems that you can't solve seems counterproductive.
 
Thanks, for the questions you posed.. I put my answers in red. Do you think this is an ok setup?

NauticaONE said:
Nope, I dont recommend that because its software based, therefore susceptible to the OS.

Perfect.. I am leaving this turned off.


Things that you should consider when developing your SAN

1. Am I Redundant? (E.G. Storage Processors, SAN Switches, Dual Path HBA's)

Yes, dual controllers (DS4300 turbo), two Fiber switches, dual HBA's on each server using RDAC drivers (dual pathing failover drivers).

2. In Calculating I/O Rate - How many Disks am I using, How many Raid Sets Do I have

I am using 14 disks,

Raid 1 (two drives) - SQL 2005
Raid 1 (two drives) - Exchange 2003 User files
Raid 5 (seven drives) - Windows File share (typical files of Office, media.. etc).
2 drives used for Flash copy of the Raid 5 array
1 Hot spare

3. How many hosts am I serving with my SAN?

4 - SQL server, Exchange server, File server, Flash copy server for Netbackup use.

4. What type of disks am I using (Yes They Sell PATA/SATA/SCSI/SAS Drives) for my San

All disks are 10,000 rpm Fiber drives - 136GB
Click to expand...


Oh and I have to add that all my servers have dual power supplies (at least) and all equipment is connected to a heavy duty APC power backup unit (which is connected to a diesel generator that kicks on after about 30 seconds of no power).

All servers are running 2003 SP2 withe primary OS on local SCSI 10,000 rpm drives in RAID 1.
 
Oh and unfortunately I have my Exchange databases on a Raid 1 array (didn't know any better when I set it up) but I am seeing from googling that they really should be on a Raid 5 array.

We have only about 450 mailboxes and about 80GB of data right now for the Exchange.

Should I move this to a Raid 5 array? would I see increased performance?

Also I have write cache enabled on my Raid 1 arrays. Should I turn it off and only have read cache enabled?

Thanks for your help thus far.
 
sixpac said:
Oh and unfortunately I have my Exchange databases on a Raid 1 array (didn't know any better when I set it up) but I am seeing from googling that they really should be on a Raid 5 array.
Click to expand...
What gives you that impression? Everything I've seen strongly recommends against raid 5 for mail service.
 
http://msexchangeteam.com/archive/2007/01/15/432199.aspx
RAID10 is the ideal configuration for Exchange, and it works well with large capacity disks.

With Exchange 2003, RAID5 provides the best capacity efficiency, though its poorer performance seldom allows the extra space to be utilized.
Click to expand...
http://technet.microsoft.com/en-us/library/bb738141.aspx (Microsoft's documentation on how to deal with Exchange storage) doesn't even mention raid-5.

http://technet.microsoft.com/en-us/library/c5a9c0ed-e43e-4bc7-99fe-7d1a9cb967f8.aspx
For example, most administrators who select a RAID-5 solution do so to maximize storage usage. However, in many cases, properly sizing the performance of your spindles requires you to use more physical disks for RAID-5 than you would use in a RAID-1+0 configuration.
Click to expand...

However, with only 450 users it's not a big deal. If you had to worry about expanding to 10k users tomorrow you'd definitely want to stick with raid 10 (over more disks, one would hope) but if you want to try raid 5, have fun.
 
You must log in or register to reply here.
Back
Top