• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

Lets play the disk shuffle game!

R

RiDDLeRThC

2[H]4U
2FA
Joined
Jun 13, 2002
Messages
3,963
So as a few know, one of our VNX 5300's arrived and while it seems to have a decent layout from the factory I know we have people on here that do this daily. Plus it will be interesting to compare and challenge whatever we come up with in this thread with what the tech wants to do come Tuesday.

With that said I have a screenshot and the excel file if anyone would like to play. The disks and DAE's can all move around as people see fit.

current%20disk%20layout%20v1.PNG


current disk layout v1.xlsx

Feel free to upload screenshots to this uploader. If you make an account you can also upload xlxs files.
http://www.cyclonesworld.net/upload/


EDIT: I sometimes wonder if things like this should be posted in the Data Storage Systems.... But we have a lot of VNX stuff here along with this is used for just VMs.
 
Last edited:
It all depends on what you need the disks to do.

When I create a storage design for a client I want to know what servers are going to be using it, how much space each needs, what each server will be doing (SQL, Virtual Machines, Exchange, File shares, etc.), and how many IOPs each server is currently using, plus what your future needs may be and if we have the ability to account for that. Depending on performance needs we need to make sure we keep SQL DB files and SQL Transaction logs separated, Exchange should be by itself, and so on.

From there, I then decide how I want to carve up the disks, what RAID levels to use, and so on.

But.... even with all that information I don't think my company would be very happy if I created a storage design for you so a competitor could implement it. :)
 
From what I remember of the best practise guide make sure you put some of your DAE's on the second SAS port, don't forget that things in the DEP use SAS port 0
 
Child of Wonder said:
It all depends on what you need the disks to do.

When I create a storage design for a client I want to know what servers are going to be using it, how much space each needs, what each server will be doing (SQL, Virtual Machines, Exchange, File shares, etc.), and how many IOPs each server is currently using, plus what your future needs may be and if we have the ability to account for that. Depending on performance needs we need to make sure we keep SQL DB files and SQL Transaction logs separated, Exchange should be by itself, and so on.

From there, I then decide how I want to carve up the disks, what RAID levels to use, and so on.

But.... even with all that information I don't think my company would be very happy if I created a storage design for you so a competitor could implement it. :)
Click to expand...

This is more for fun and kinda to learn a little for both myself and others on [H]. I will be receiving the design documents tomorrow and while I won't share the doc itself I will post the recommendations. Everyone has their own way to do things but I personally love to see how others might do things even if it differs from my own way or the way of our vendor.

For us our workload is all over the place but its all vmfs.

We have a new exchange cluster coming online soon but I haven't done much of the work for it yet. Haven't decided if the exchange vm's will use RDM. Think I'm just going to stick to vmfs. I know at one point one or the other wasn't supported. Not sure where all that stands now.
 
Can someone answer if a DAE or the DEP have different buses within them? From what I understand (thanks ManateeMatt) the DEP uses SAS port 0. But is there anything else I should take into consideration with the disk layout.

Reason I ask is it seems a little odd that from the factory EMC put 4 15k drives, then 3 SSD, then another 4 15k drives.

I've got a lot of room in one DAE so I would rather free up 3 slots in each of the other DAE's for future SSD drives IF there is two buses within each enclosure.
 
Don't judge too much but after reading a little bit last night I started rearranging the disks within Excel. Yes part of this is just organizing but also trying to follow best practices.

The empty slots on DAE1-3 are for future flash drives for Tier 1 storage OR to build some additional raid groups using the empty spots on the DPE.

One question, I remember reading that you should split your raid groups 3 disks on one DAE and 2 on another. Should they also be split across buses or should they remain on the same bus?

Each raid group is a different color. Red are the hot spares. Flash Cache is labeled along with the vault drives.

The workbook from our VAR hasn't been sent yet so I'm still not sure what they have planned.

my%20disk%20layout%20v2.PNG
 
So short of my VAR splitting the disks across backend buses they don't seem to have much of a plan for the where the disks or how the raid groups will be setup. Very interesting.
 
Why RAID groups and not just FAST VP pool(s)?

Very few partners are going to go this deep on a storage design. I'll be honest, most customers just don't need it. This makes placement and config much more complex if it's not well documented.

Whether this config works is up to you and your workload. You spread RGs across two shelves, but if the workload is equal you haven't solved anything, really.

I'd put those non-cache EFDs, 600GB 15Ks and those 1TB NL-SAS in a FAST pool and call it good. Pull out some drives if you need something setup a specific way, or create a second pool..but you'll drive yourself nuts doing all this complex "best practice" stuff for often very little gain.
 
NetJunkie said:
Why RAID groups and not just FAST VP pool(s)?

Very few partners are going to go this deep on a storage design. I'll be honest, most customers just don't need it. This makes placement and config much more complex if it's not well documented.

Whether this config works is up to you and your workload. You spread RGs across two shelves, but if the workload is equal you haven't solved anything, really.

I'd put those non-cache EFDs, 600GB 15Ks and those 1TB NL-SAS in a FAST pool and call it good. Pull out some drives if you need something setup a specific way, or create a second pool..but you'll drive yourself nuts doing all this complex "best practice" stuff for often very little gain.
Click to expand...

They will all be fast pools. I was under the impression I had to still break them out into RAID groups but thats just my misunderstanding. I just tell it to build the pool with 25 disks and it will break it into the 5 disk raid5 groups.

Here is the simple design, i went back and re-read about the RAID groups across DAE's and it didn't really said you had to but if you did to make sure 2 drives were on one.

my%20disk%20layout%20v3.PNG


BTW, thanks for the response. As you said, you can drive yourself nuts trying to follow these 200 page best practice for performance and availability.
 
Yep. Using pools it'll distributed the load better than you. It's the "easy button" for arrays these days.
 
NetJunkie said:
Why RAID groups and not just FAST VP pool(s)?
Click to expand...

For small clients with minimal storage performance needs, FAST VP works well.

However, there are often exceptions where FAST VP just isn't practical. I'm currently setting up a VNX5300 for a client that has 4 SQL DB servers that each average 500 IOPs. They also have a few VM datastores, file servers, and app servers. It just doesn't make sense to put everything into a FAST VP pool since the SQL DBs, SQL Transaction Logs, VMs, and app servers would all have to share the 15k disks. Having all those different kinds of IO profiles sharing the same disks would create a lot of head movement and increase IO response times.
 
Child of Wonder said:
For small clients with minimal storage performance needs, FAST VP works well.

However, there are often exceptions where FAST VP just isn't practical. I'm currently setting up a VNX5300 for a client that has 4 SQL DB servers that each average 500 IOPs. They also have a few VM datastores, file servers, and app servers. It just doesn't make sense to put everything into a FAST VP pool since the SQL DBs, SQL Transaction Logs, VMs, and app servers would all have to share the 15k disks. Having all those different kinds of IO profiles sharing the same disks would create a lot of head movement and increase IO response times.
Click to expand...

As you know...that depends on many things and I'm not saying do pools for EVERY array. But what's the I/O requirements of all those? 500 IOPS per DB? That's nothing. File servers? Nothing. Most VM datastores are pretty low. Take all that, average the spread across all disks in the pool and you'll find that I/O latency is just fine.

We have much larger arrays with much larger mix of larger requirements in pools together. It's all about averaging it out over the spindles.

EDIT: And I very much disagree with the "For small clients with minimal storage performance needs..." statement. The beauty of FAST VP isn't that it's a simple pool, it's that you get all the spindles in that pool for your performance. The only way to match that sort of I/O with RGs is with Metas and that gets real ugly, real fast.
 
NetJunkie said:
As you know...that depends on many things and I'm not saying do pools for EVERY array. But what's the I/O requirements of all those? 500 IOPS per DB? That's nothing. File servers? Nothing. Most VM datastores are pretty low. Take all that, average the spread across all disks in the pool and you'll find that I/O latency is just fine.

We have much larger arrays with much larger mix of larger requirements in pools together. It's all about averaging it out over the spindles.

EDIT: And I very much disagree with the "For small clients with minimal storage performance needs..." statement. The beauty of FAST VP isn't that it's a simple pool, it's that you get all the spindles in that pool for your performance. The only way to match that sort of I/O with RGs is with Metas and that gets real ugly, real fast.
Click to expand...

Yes, I love using Storage Pools over RAID Groups when I can since it reduces complexity and negates the need for Metas.

I should have also clarified my "small business" statement. I didn't mean only small businesses make sense for FAST VP, just that when you have a small business with minimal IO requirements, then I'd agree that FAST VP is a no brainer or an "Easy Button."

FAST VP is a great technology, I just feel one should be careful when deciding to use it. It's not a panacea.
 
Ah..yeah, then we agree. It's not the answer to everything and we still do manually build out arrays, depending on the workload.

One of my favorite statements (came from a session at VMware Partner Exchange) is "Knowing your workload is better than any best practice.". Been using that a lot.
 
Our workload is mostly dev/qa/uat Postgres servers that never get used.

Then some reporting Postgres servers for reporting databases.

We did the workload eval and i talked to the vendor today. They do have a workbook that lays everything out for our workload. The installer will have the documents.
 
NetJunkie said:
Ah..yeah, then we agree. It's not the answer to everything and we still do manually build out arrays, depending on the workload.

One of my favorite statements (came from a session at VMware Partner Exchange) is "Knowing your workload is better than any best practice.". Been using that a lot.
Click to expand...

Words to live by.

I also rarely get a situation to even consider FAST VP because of the way my company handles sales.

A pre-sales engineer (who is not very technically inclined) will talk with the client and come up with a rough guess on what they need for disks (usually based on capacity only) then places the order.

Then me, the post sales guy, gets assigned to the project and has to find a way to shoehorn the client's servers into what the pre-sales guy decided to order. When they do order EFDs, it's almost always a 5 pack which would only allow me to put the drives in a 3+1 RAID 5 plus 1 HS if I wanted to do something else with them.

It's really infuriating and I wish I could just work with the client on the pre-sales side myself to help decide what to purchase, build a design, then implement it.
 
RiDDLeRThC said:
Our workload is mostly dev/qa/uat Postgres servers that never get used.

Then some reporting Postgres servers for reporting databases.

We did the workload eval and i talked to the vendor today. They do have a workbook that lays everything out for our workload. The installer will have the documents.
Click to expand...

Then it sounds like FAST VP might work well for you, especially with all those EFDs.
 
Child of Wonder said:
It's really infuriating and I wish I could just work with the client on the pre-sales side myself to help decide what to purchase, build a design, then implement it.
Click to expand...

The way you guys are is the way MOST partners are..and, honestly, why I am where I am. Our pre-sales guys are really good at storage sizing and array design. We do all that on the front end with the customer. We actually used to be responsible for the detailed design workbook but that now goes to a delivery engineer. Even then the pre-sales guy has an internal call to talk about the design and the "why we did this" things with the engineers. It's one very big way we differentiate ourselves...we do a ton of pre-sales work and design..stuff most others don't.

Going to be at EMC World? Looks like I might be there.
 
NetJunkie said:
The way you guys are is the way MOST partners are..and, honestly, why I am where I am. Our pre-sales guys are really good at storage sizing and array design. We do all that on the front end with the customer. We actually used to be responsible for the detailed design workbook but that now goes to a delivery engineer. Even then the pre-sales guy has an internal call to talk about the design and the "why we did this" things with the engineers. It's one very big way we differentiate ourselves...we do a ton of pre-sales work and design..stuff most others don't.

Going to be at EMC World? Looks like I might be there.
Click to expand...

Not sure. Trying to figure out what 2 I want to hit this year between EMC World, Cisco Live, and VMworld.

Right now I'm thinking of skipping VMworld for EMC World and Cisco Live and then going to PEX 2013 instead.
 
Child of Wonder said:
Not sure. Trying to figure out what 2 I want to hit this year between EMC World, Cisco Live, and VMworld.

Right now I'm thinking of skipping VMworld for EMC World and Cisco Live and then going to PEX 2013 instead.
Click to expand...

That's what I'd do. Had this conversation with a customer today. We aren't sending many people at all to VMworld this year. I probably won't even be there unless I'm presenting. I feel like with the large virtualization community I get everything I need from blogs and Twitter. Nothing really new at VMworld. The EMC World stuff is more informative as that community isn't as big/vocal. This year was my first PEX and I really enjoyed it...good content plus I wasn't doing constant meetings. VMworld for me is now just customer meetings, partner meetings, and interviews. Last year I went to one session that wasn't one of mine.
 
Is it worth a small biz customer to go to EMC world? My rep said yes and is trying to secure a free pass but I have my doubts.
 
NetJunkie said:
That's what I'd do. Had this conversation with a customer today. We aren't sending many people at all to VMworld this year. I probably won't even be there unless I'm presenting. I feel like with the large virtualization community I get everything I need from blogs and Twitter. Nothing really new at VMworld. The EMC World stuff is more informative as that community isn't as big/vocal. This year was my first PEX and I really enjoyed it...good content plus I wasn't doing constant meetings. VMworld for me is now just customer meetings, partner meetings, and interviews. Last year I went to one session that wasn't one of mine.
Click to expand...

If I do go I'll let you know. We could get together and BS for a while if you have time.
 
You must log in or register to reply here.
Back
Top