Help with virtual SAN appliance

[matt_payne666]

Hi, I need some help configuring a virtual SAN in my ESXi host... I know its been covered before (to death probably!) and I have used the search and read an unhealthy amount of posts, but I just can get my setup to perform...

My original attempts produced dismal speeds so after much reading I revised my disk controller hardware, but still only seeing about 20MB/s copying from a VM on the same host via an iSCSI connection... What sort of speed would you expect I should see from this? I was hoping for about 100MB/s

HardForum,
Your mission, should you choose to accept it is to convert this pile of bits into a nice fast Virtual SAN appliance which will live on a host with a couple of low use VM's.

Hardware intel S5400 motherboard
2x 2.33ghx quad core XEONs
16GB ECC FB RAM
LSi SAS 92112 4i4e HBA flashed with IT firmware
4x 3TB Seagate HDD
1x 128GB SSD configured as ESXi datastore with 3 VM's hosted on it (Win2012 DC, Win2012 Direct access server, Nexentastor community SAN appliance)

Storage strategy - idealy maximum space (only 4 drive bays in a 1u chassis)

Im not fussed with freenas, openfiler, nexentastor, etc, but I do have a preference to the feel of the nexenta gui (im a windows man so zero Linux/solaris/unix experience and am mostly gui orientated)

Thanks

 

[_Gea]

It can be better, your options:

Nic-problem: check e1000 vs vmxnet3 driver in ESXi on all guests
iSCSI problem: try NFS (i would use NFS in general on a virtualized SAN)
sync-problem: disable sync (NFS) or enable writeback cache (iscsi)

avoid jumboframes and link-aggregation (at least on first tests)

read also my mini-howto for other settings (based on OI/OmniOS but Nexentastor is similar,
at least the new NexentaStor4): http://www.napp-it.org/doc/downloads/all-in-one.pdf

 

[matt_payne666]

Thanks for the tips... Just swapped to the vmxnet3 cards and the thoughput is upto 50MB/s.
Writebackcache on or off makes no difference
How do i mount an nfs share on a server2012 machine?

I had previously read your howto, but i need almost all the availiabke space exposed to just one vm in one large chunk so creating a vm data store is a nogo with its 2TB limit.

What sort or read/write speed do you think i should be able to achieve? IM transfering 10gb files to and from the device

 

[_Gea]

-I would use NFS only as ESXi datastore (no 2TB Limit).
-If you use your second Win2012 as filer, you should think about using Solaris as a SMB filer.
(can be a AD member)

-Maybee your storage VM has not enough RAM, try
- assign 10GB to your storage VM and share a target via Comstar
- assign 4 GB to a Win2012 VM and connect the target, check speed (no other VM active)

 

[matt_payne666]

I can create a big ESXI datastore, but how would I get a 7TB disk into my VM?

Ill rejig the memory commitments and give that a go... (I have another 8GB memory arriving in a few weeks time which will help) The web interface for nexenta shows that there is 3GB spare at pretty much all time....

 

[_Gea]

I can create a big ESXI datastore, but how would I get a 7TB disk into my VM?
.

Not using ESXi.
You can either create a 7TB Comstar iSCSI target and format if to NTFS or
you put your data directly to a ZFS SMB share with all ZFS features like snaps
(as Windows previous version)

 

[matt_payne666]

I just put the 4 3TB disks into my old microserver (36NL, 1.3ghz, 8GB ram) and built a physical nexentastor box - 80MB/s consistently over iSCSI

Why cant I get the same from the appliance virtualized?

 

[_Gea]

I just put the 4 3TB disks into my old microserver (36NL, 1.3ghz, 8GB ram) and built a physical nexentastor box - 80MB/s consistently over iSCSI

Why cant I get the same from the appliance virtualized?

ESXi internal traffic is capable of much higher values but it is hard to compare two configs.
There are too many hardware, software or setting options and possible problems.

http://blog.cyberexplorer.me/2013/03/improving-vm-to-vm-network-throughput.html
http://napp-it.org/manuals/tuning.html