Openfiler newbie storage strategy help

[@theincredibleaf]

Hi guys,

I have a VMware ESXi 5.5 host, 32 GB RAM, 6 core AMD, FC HBA card, with 4 NICs.

It's running vCenter, a few Windows Server 2012 VMs and some Ubuntu server VMs.

It uses a NAS device for datastore, with NFS access protocol, and both devices connect using a gigabit L2 switch.

The NAS has 2 x 2TB drives configured as a striped array, and operates quite slowly, delivering peak read performance of about 60 MB/s. It has a single Gbit port so no chance to use link aggregation to speed this up.

I am building a low cost additional host, as I want to remove my Cisco router and NAS from my setup, and virtualise their services.

It will have 4 NICs, 8 GB RAM, old quad core AMD, a 4 Gbps Fibre Channel HBA card and probably 4 x SSD drives. It will boot ESXi off USB stick.

Now here is where I need some help with the software side.

Ideally, I would want the primary host to connect to the secondary using direct connect fibre, booting off it and using it as DAS for all data storage (it won't have it's own storage at all).

On the secondary, what should I use to present this storage?

An Openfiler VM with direct access to the HBA card and hard disk controller? (this VM would boot off the USB stick also)

ESXi PCI pass-through works fine for me so this isn't a problem.

Would it be possible to use Openfiler to present iSCSI also, using the same drives?

iSCSI would be used internally by VMs running on the secondary host?

Is there a better way of doing all this that doesn't involve buying extra hardware?

I do have access to a fibre switch, and could possibly build a third host just purely for storage duties, but I really would rather avoid the extra noise and electricity.

Advice appreciated.

Cheers.

 

[_Gea]

I would
- use NFS only for ESXi and iSCSI only when needed
- use nothing without ZFS

my preferred OS for a virtualized SAN is OmniOS (free Oracle Solaris fork)
I use such a setup for years. You may read my miniHowto: http://www.napp-it.org/doc/downloads/napp-in-one.pdf

You can download my ready to use ZFS storage appliance that you can use for free from
http://napp-it.org/downloads/index_en.html

It comes with ESXi 5.5 vmware tools, NFS, SMB, iSCSI and FC support together with a Web-UI

 

[@theincredibleaf]

Thanks I will look at that.

 

[@theincredibleaf]

my preferred OS for a virtualized SAN is OmniOS (free Oracle Solaris fork)
I use such a setup for yearsI

So on a storage host running this software that has with 4 x 480 GB SSD drives and FC and Ethernet ports...

Can I make a number of partitions over say 4 drives, and then allocate a connection type per partition?

Example....

Partition 1 - 300 GB - presented as DAS using direct connect FC to another host to boot from
Partiton 2 - 500 GB - NFS datastore for VMs on another host using Ethernet connection

etc

Is this possible?

Thanks.

 

[_Gea]

Yes, no problem
With ZFS you create a datapool say a raid-10 (960GB) or a raid-z1 (1440GB).

1.
You can then create filesystems on that pool. Size of ZFS filesystems is not fixed as you may know with traditional partitions (this is called storage virtualisation) managed by reservations and quotas.

You can share such a filesystem via NFS

2.
You can create a zvol (a ZFS blockdevice) on a ZFS filesystem that is treated like a disk. Such a zvol can be resized as well like a filesystem. From this zvol you create a LU (Logical unit) and then a target.

Next you create a target group with the target as member. Then a view from your LU to your target group to make it visible. This can all done via Web-UI in menu Comstar.

Napp-it supports beside that a simplified iSCSI sharing where a iSCSI LUN is a property of a ZFS filesystem (on, set size and ok)

for details, read Oracle docs about FC
http://docs.oracle.com/cd/E23824_01/html/E23097/agkco.html

a perfect OmniOS guide is
http://docs.oracle.com/cd/E19963-01/pdf/821-1448.pdf

more manuals, see
http://napp-it.org/manuals/index_en.html

 

[@theincredibleaf]

This sounds incredible, and this software is free?

Does it make any serious impact on performance with all these features...Where can I find performance benchmarks?

The storage guy at work said he had bad results when he trialled FreeNAS some time ago but asked me to send him what you posted because he was interested.

 

[@theincredibleaf]

Sorry one last question...for future project...Do you know about Hadoop and HDFS format?

Am I right in thinking that I wouldn't use such a solution as you posted with Hadoop and Map Reduce tasks?

Thanks for your help.

 

[_Gea]

With ZFS you can build fast Petabyte Storage but it is not a distributed filesystem (You may build such things upon).

FreeBSD (ex FreeNAS) is another ZFS platform (beside Linux and Solaris).
I prefer Solaris based ones as this is where ZFS and the development comes from and current Solaris/ZFS performance was never a problem.

I have no absolute performance values only a pdf with benchmarks with differenz ZIL devices for securs sync writes.
http://napp-it.org/doc/manuals/benchmarks.pdf

But ZFS scale very well over hardware and offers advanced features like ZIL, ARC and L2Arc Cache.
IFor me it was the decision that I want something mostly comparable to a highend solution like NetApp but without the price. There is a reason why NetApp tried to stop ZFS with a lawsuit years ago.

And yes, ZFS is free at least with those that are based on the free Solaris fork Illumos. BSD and Linux use this as upstream as well. Only the origin Oracle Solaris is not free but we have a free fork.