Need some help with Storage, Hyper-V and Exchange decision

Z

zanderkid

n00b
Joined
Sep 2, 2011
Messages
13
Hey Guys,

I'm new to the forum. I found a lot of positive feedback here on VNXe3100. I was actually considering Storageworks x1600 G2, but at the cost of VNXe, I might as well get a SAN....
Environment is 100% Windows, with 110 users.

Physical Exchange 2003 Standard - HP DL380 G5, DB size is approximately 60GB, considering Exchange 2010; also my backup server to Tape and USB (Backup Exec 12)

2 x File Servers with combined file data of 650GB give or take; one of the file servers also runs SQL database (only 4 or 5 users ever connect to it)

1 physical DC - HP DL380; this box also runs Symantec End Point VM (MS Virtual Server 2005 R2)

1 physical DC - Dell PowerEdge 2600

DL 360 4Gp - runs ESXi 4 with 2 VMs: BES (Blackberry enterprise server) and BCM (Business Contact manager for Outlook - SQL DB, low user access)

We also have another Database server running SPSS (IBM), it`s facing the public network with DB size of approxiately 40GB or so.

I spoke to vendors, CDW recommended VNXe, another vendor recommended Storageworks X1600G 2 NAS, Dell recommended Equallogic.
My original plan was to go with 3-2-1 design from Dell (3 servers, 2 switches, 1 NAS/SAN), realistically, whatever it is we get, it will litarelly store everything (File data, exchange, SQL, VMs), so any of the vendor ideas would have a signle point of failure, be it SAN or NAS.

I can also post the IOPS results that we ran later. However it looks like 80% read and 20 % write...

My solution from HP (3x DL 380 G7, 2 x 24port switches & 1 x1600 G2 6TB SAS) came up to $40K, Dell (3 x R710, 2 x switches, 1 equallogic, Vsphere 4 came up to 45K
Problem is I need some form of live replication or failover.
Any suggestions or recommendations for or against VNXe3100 or Storageworks. Ultimately, so that I could sleep at night, I would like to get 2units, preferably SAS in one and SATA (to keep cost low) in the other.
We will probably go with Hyper-V to keep cost low, I also plan to combine the 2 file servers into one. Also, anybody has an idea on the cost of support and warranty on VNXe for years 4 and 5 ?

Thanks in advance, I really appreciate the help.
 
Have you looked the HP P4000?

It's a clustered SAN made of storage nodes. You can put them in the same rack, or ideally drop them on a dedicated iSCSI network in different locations and you have storage with failover.

That's the really brief summary, suggest you take a look on their website and see what you think.

The key word you've mentioned is live replication/failover. I'm not aware of any other SAN that gives you that option without purchasing a second, expensive, unit, which typically sits doing nothing until you have a problem.
 
Have you looked the HP P4000?
Click to expand...

God no..stay away. I've implemented the P4000 and absolutely hate it when compared to our EMC CX4-240.

There is nothing "technically advanced" about the P4000. It's a cluster of HP servers essentially running a small linux app which handles all node failover/redundancy via their shitty software.

You do not get the ability to break out physical spindles and assign them to an application/db..etc.
There is ALWAYS sharing of disk. The nodes will come RAID 5 or RAID 10, on top of that, you can utilize their implementation of Network RAID to accomodate for node failover..etc, which penalizes you on disk space so not only will it require redundancy at the single internal node level, but also the cluster node level as well. You can't assign physical spindles to a particular application/db to dedicate required I/O's to the app/db, it's one large pool. Of course, the idea is the more spindles you add, the more performance, and less contention.

Go with the VNXe, it's tighly integrated with both Hyper-V and VMware and has really nice Exchange functionality built into Unisphere. There are others using it, i'm sure they will chime in here.
 
Last edited:
You're right that you can't dedicate spindles with the P4000, that may or may not be an issue. There are benefits like being able to do a stretch cluster for failover, or buy a couple of cheap VSA's and drop them on your existing hardware and BYO SAN.

The VNXe looks nice, but there are licensing considerations and hardware considerations if you want to do replication or failover so Zander needs to be aware of what he wants to do now, and what he may want to do in six months or a year as there will be costs involved.

Equallogic are nice too, I like the all-in pricing model but they aren't cheap and like you said they share disks as does the P4000 or most scale-out/virtualised SAN.

Unless you have deep pockets there's always some compromise and Zander needs to consider what he's prepared to compromise on as he said "Problem is I need some form of live replication or failover.".
 
Buy 20 units of Crucial M4 SSD 512GByte 770 x 20
Allocate some 10 as spare, RAID the remaining. Base on your stated GByte used, the remaining is more than enough
Buy Enterprise Grade SAS RAID Controllers
Installed into ECC-based Server of your choice with proper case to house many SSD. Socket 1155 Server is more than sufficient to run this config.
Run Solaris ZFS-SAN.
Pay Oracle for official Solaris maintenance support. (Last line of defense, when in doubt, call Oracle for storage support)
Buy some mechanical disks for backup and nearline secondary line of defense Norco 20-hot-plug

The only uncertainty is MLC SSD endurance, which I spare 10 units, I think reasonable.
The other problem is not sure how much Oracle wants to charge you.

Triple check with your support vendor, Oracle and this Forum's experts before committing budget. Doing properly, the performance maybe fantastic for your current usage for several years. Multiple verification with SAS Controller compatibility with MLC SSD to avoid weird problems.

Only consider this if you are comfortable with ZFS command-line. Else revert back to standard Enterprise Storage options.

This post is half light-tone because some friends want to push some ZFS message. But there are basis in the post. For example, ZFS snapshot and recovery.

Others
A serious FLASH-based SAN such as TEXAS MEMORY RAM-SAN models higher cost but guaranteed IOPS performance for workload. Some say overspec. Useful for many years.

Cheers
 
Last edited:
Take a look at the Dell PowerVault MD3200i--it is cheaper than the Equalogic's and are solid.
 
The base VNXe3100 only comes with a single Storage Processor which means you'll have only 2x 1Gb network ports for your hosts to connect to. If you plan to keep Exchange physical you'll want one port dedicated to iSCSI. Sounds like you also want to move CIFS to the VNXe which means you'll want to dedicate the other port to CIFS. Then you've used up all your network ports unless you want to piggy back different services on each (you can run more than one storage protocol on one NIC).

If you get 2 Storage Processors, then you'll have much better failover ability and another 2 network ports to work with. However, bear in mind that you'll need to configure your network switch identically for the ports on SPA and SPB (same VLANs, Jumbo frames, LACP) because when one SP fails over to the other, it essentially moves its ports and services to the ports on the other SP. So if you enable Jumbo Frames on port 1 on the SPA, Jumbo Frames will automatically be enabled on port 1 on SPB.

Also bear in mind you can only provision disks in 5 or 6 disk increments depending on the RAID level you want to use. RAID 5 Storage Pools can only be created in groups of 5, 10, 15, etc. disks, RAID 10 in 6, 12, 18, etc. groups of disks, and RAID 6 SATA pools in 6, 12, 18, etc. groups. If you purchase a VNXe3300, the RAID 5 increment increases from 5 to 7.
 
Guys, this is the IOPS report using Dell Pack that we ran for 24 hours

Server--------------------IOPS----------- MB/Sec-----Capacity----Free Capacity----Read/Write ratio
DC -----------------------188-------------- 12.9---------136.7GB------61.3GBs----------54% R/46% W
DC2 ----------------------4 ------------------6.8----------204.7GB------49.6GBs----------52% R/48% W
File Server 1-----------613----------------21-----------465.8GBS---107.3GBs---------97% R/3% W
BES-VM ---------------- 19----------------0.7-------------- 30GBs------15.9GBs-----------7% R/93% W
Symantec-VM -------- 204---------------11----------------40GBs------19.4GBs-----------58% R/42% W
Exchange -------------- 584--------------19.2------------1944GBs-----322.9GBs---------93% R/7% W
File Server 2---------- 129---------------33.8-------------135.6GBs----28.7GBs----------54% R/46% W
Total ----------------------- -------------------------------------- 2.9TBs--------0.6TBs-----------80% R/20% W

I don't think that this report is 100% accurate, as for instance Exchange shows as 322.9GB of free space...
in reality, I have 6x146GB drives (2 x Raid1 OS & 4 drives for data), well the 4 data drive partition, only has 40 - 50GB free....
 
Last edited:
It looks like if I was able to get approval for VNXe3100, there is no way I would be able to approve a second unit. Realistically hardware quote w 1 x storageworks x1600, 3 x dl380 & 2 switches, came down to approximately 40K
I guess I can try to push to 45, maybe even 50, but not much more then that.
I'm gonna need to find some sort of solution where I could recover quickly if things go South....
Am I able to replicate from VNXe to a low budget NAS?
So, let`s say VNXe is down (asa unlikely as it may seem, but not impossible) I just point my VMs and exchange to another NAS, point file services as well. Yeah, peformance would suck for a day maybe two while VNXe would be down, but it would still be possible to operate business....
I also want to dump the data to either Autonomy, Unitrends or another vendor.
 
Recovering quickly if things go south depends how "south" things go :)

Most SANs will do snapshots so you can roll-back a volume to a previous point in time.

Of course, that doesn't help you if the whole SAN/room/closet disappears.

Pretty much all SAN vendors make it so that their SANs will only replicate volumes to another of their SANs, and most vendors will charge you quite a lot to do this, Equallogic and P4000 being the two main exceptions that I'm aware of.

I'm not clear from your post which machines you're planning on keeping physical and which you're moving to VM's?

If you can do your backups and restores at VM level you make it quicker to get back up and running as, in theory, you just need your backup server available and a vsphere box with somewhere to dump the data - and these days that could be a server stuffed with local storage.

Incidentally, how did you come up with those IOPS numbers? Some of those numbers look pretty high in terms of IOPS, I'd be interested to know if those are peak or 95th percentile number or what exactly.
 
Based on iOPS in that chart, what would you recommend I get in terms of quantity of drives and capacity of drives whether it is VNXe or not ?
 
Dell provided this tool called DPACK or dell Pack, I have a full report where it shows everything per server, but it would be to long to post here, so I posted only the summary. It ran for 24 hours though...
It ran during the week where I do not run Full backups too.

in terms of what`s virtual and what`s physical goes. When I was discussing STorageworks, since it runs Windows Storage Server 2008, I don't need the file servers for that, i could just post shares on NAS.
I would then host BES, SEP(Symantec end point), also build a new VM for Accounting (only 4 or 5 people would access it) and also consider virtualizing Exchange (haven't determined whether to keep it physical or virtual). I would also keep 1 DC physical and virtualize the sedond one.
I was told a few times that IOPS look way to high, I guess I would have to run another tool,perhaps capacity planner or something, let me see if I can get my hands on it...
Vsphere would be nice, but in terms of future planning, I wouldn't go with essentials since ver 5 only limits me to 32GB of vRAM and the nexxt version up will throw my budget out of the water, so Hyper-V would be my choice due to cost
 
Last edited:
Let's roll it back a step, what are you looking to achieve?

Let's say you go out and buy three servers and a SAN, let's ignore the vendor but it's one SAN in one room.

If you run an application i.e. Exchange on a physical server your SPOF becomes the physical server as it doesn't matter if all your data is on your SAN if the motherboard on the server fails (unlikely but you see where I'm going).

Similarly if you virtualise everything you could have half a dozen servers with all your applications virtualised but if all your data is on your SAN that becomes your SPOF.

I would definitely take a close look at the raw logs used to produce those IOPS numbers too, they don't look right unless they're peak figures (I can't believe a DC for 110 people average almost 200 IOPS unless it's doing a lot more than just being a DC, ditto your Exchange server).
 
Yeah, totally agree with you. DCs don't have that much load. Unfortunately my basement flooded and we are throwing everything out, so I'm posting back here as I take a break to relax. As soon as have the opportunity, I will get either entire report posted here or perhaps stick it on box.net....
 
I'd like to place my vote for 100% virtualized environment on ESXi, you loads are not significant so you probably be ok with 2x 360 G7 with CF boot 2x5660 and as much ram as possible - we running 144gb each node, but you'd be fine with probably 64gb
All OS should be stored on NFS volumes, but data disks iSCSI and a CIFS volume for file access
What you need is SAN solution which could do all three and manage your backup, replication and high availability. Netapp 2040HA would probably be too much money for you, but I think you should checkout managed appliance from PogoLinux - ZFS based storage.
http://www.pogostorage.com/products/managed-appliances.php
 
Explanation for one of the DCs having IOPS this high is because that same DC hosts Microsoft Virtual Server 2005 with SEP VM on it.
 
Here is probably the important part that I should`ve listed in te first place, i apologize with this flood here, can't really think straight anymore....
Servers Recorder: 7
Throughput: 37.1 MB/Sec peak
IOPS: 1005 avg and 1705 at peak (peak is at night when backups run)
Read/Write ration: 80%read / 20% write
Total Capacity: 2957.7GB (but that includes OS and everything)
Free Capacity: 605.1 GB Free


Also, I will be upgrading domain to 2008 functional level from 2003, so I build the VMs form scratch, there is no point of virtualizing most of the existing servers. With exchange being upgraded from 2003 to 2008, that will also be a fresh new 2008 Serevr install, so the total capacity will decrease in the beginning, but I still need to allocate some spare storage for future...
What I'm trying to accomplish is a solution where I could virtualize everything but keeping 1 DC physical and Exchange, well Exchange physical or virtual, haven't determined that yet. I'd like to go with Vmware, however unless I go with Vsphere 4, boss will not approve it due ot cost.
Perhaps I keep the Exchange local be it physical or virtual and separated from NAS/SAN, so if SAN is down, Exchange is separated. Possibly also add a low budget NAS like QNAP perhaps for File Storage and VMs.Replicate data from SAN to this low budget NAS.
Another problem is the way company spends money, we pay for everythign up front, I will try to push lease instead of buyout up front. Solution that I think needs to be implemented is the one that will sustain at least 3 - 4 year period.
 
Last edited:
Personally I would go 100% virtual unless you have a compelling reason not to do so, and nothing you've mentioned so far is a compelling reason IMO.

The days of needing a physical DC are gone IMO as are the days of wanting to keep Exchange on bare metal. Physical was perhaps a good idea with Exchange 2003 if you had a large number of users, but personally for 110 users, even allowing for some expansion, you'll be just fine running it on a VM, better yet look into upgrading to Exchange 2010 as the IO drops by 90% or so.

The bottom line is you have a ton of options because as an SMB you're in a situation where basically, every single SAN you look at is going to do what you need, which doesn't help you to choose one.

The two things I would consider are TCO and support.

By TCO I don't mean trying to analyse every single potential cost, but look at the basics like what will a second unit cost when you want to do some kind of replication? What licenses does a solution from vendor x include? If it doesn't come with a license to do replication what will that cost now and in 12 months? Does it come with snapshots? if so how many and is there a license cost?

By support I mean pass-the-buck. Personally, if you're looking at vmware I think there is a lot to be said for looking at one vendor for everything, be it Dell or HP or some other vendor, basically you don't want to end up in a situation where you have an issue and the SAN vendor blames the NIC in your server and the server vendor blames the SAN vendor and nobody takes ownership - buy it all from one vendor you have one vendor to deal with.

If I was in your shoes, my next moves would be to download a few storage virtual appliances and set them up and try them out.

I'll repeat my suggestion to check out the HP P4000, that seems to be where HP is headed at the mid-range along with their 3PAR line-up once you get beyond the entry level MSA range.

Also check out things like FreeNAS/OpenFiler just to get a feel for what you may want to do.

As one very specific suggestion of a solution, consider 2, possibly three DL380 G7's loaded with RAM and disks each running a P4000 VSA with the vSphere Essentials Plus Kit (if you want vMotion and HA, if not consider the Essentials Kit). You'd come out of it with something that offers quite a bit of resilience and won't cost a lot of money.
 
You must log in or register to reply here.
Back
Top