ESXi/ZFS Problems

[Synthetickiller]

Warning: I am totally new to ESXI & Solaris!

After reading the nearly 80 page long thread on ZFS, I decided to build my own ZFS array. Here's my hardware:

Mobo: SUPERMICRO MBD-H8SGL-F-O Info
CPU: AMD Opteron 6134 2.3ghz octo-core w/ Noctua HSF (testing 120mm and 92mm versions. 120mm blocks the 16x slot)
Ram: DDR3 RDIMM 1066 64gb (16gb x4) (all same samsung chips)
Raid Card: LSI 9211-8i flashed to IT firmware
PSU: 1kw Enermax Galaxy
Case: Silverstone TJ06
Hotswap bay: Supermicro CSE-M35T-1 (not installed yet)
EXSi drive: Samsung 470 64gb SSD (will probably put 2 or 3 VMs on this)
Storage Drives: 3x3tb Hitachi 7200RPM
Other drives: 500gb and 1000gb Samsung F3

So far here's what I've done successfully.
Install ESXi 4.1, install vSphere, connect to ESXi server, install Windows 2008 R2 and OpenIndiana (settings were for Solaris 10 64bit, not sure if this is right).

Open Indiana works fine if I set the hdd controller to parallel. If I change the setting to SAS, I get an infinite loop going from post mode to the beginning of loading openIndiana. I cannot load the OS, period. Same goes for Open Solaris 11.

Open Indiana will manage my ZFS array.

Does anyone know of a work around or anything else or am I completely doing something wrong? I can't seem to find a work around.

 

[spankit]

Leaving it to parallel should work fine. Once its installed you'll have to pass your HBA through to the OS. Your motherboard has to support PCI passthrough and will most probably need to be enabled in the BIOS before it will allow you do do it. If you find the setting in your BIOS and it is enabled you should be able to configure a passthrough device in vSphere on the Host > Configuration > Advanced Settings (Hardware). If its still not letting you, chances are your motherboard does not support the feature.

 

[Synthetickiller]

I was reading for about an hour after posting and realized I didn't have IMMOU enabled. That'll fix any passthrough errors. I beleive all C32/G34 socket motherboards support IMMOU, just like all C202/204/206 boards have Vt-d, along with Q67.

I'll have to take a look at it tomorrow.

One other question, when you go to set up Napp-it, and you use the su command, where about do you type all of that in? Is there a phrase I should search for on google, llike Open Indiana commands or something?

I'm a total n00b to this, so when someone says, "use the SU command" and then gives a command string, I don't know where to start.

I'll post tomorrow if I was successful or not. Thanks buddy!

 

[styx0r]

assuming your motherboard supports passthrough, and you've enabled AMD's version of VT-D, I think the mistake you're making is one I was making yesterday when setting up my all-in-one napp-it box.

The SCSI controller you see in the vm (the one that's there by default) should be set to parallel - as you've discovered, if you set it to SAS, you get the infinite loop reboot. (this is not actually your controller, even though it has LSI in the name)

You need to pass the card through. This involves selecting the card for passthrough, and then in your Vsphere, click on your solaris VM, selecting "edit settings".

Click "add"
and chose "pci device"

then you'll be able to select your adapter to pass.

That's what worked for me.

 

[danswartz]

One issue: after marking the HBA as passthrough, you need to reboot the ESXi host. Then you can add it to the VM. Question: is there a reason you put the windows VM on the local datastore?

 

[TheLastBoyscout]

Here's a quite detailed blog post about configuring pass-through in ESXi (with pictures):
http://www.servethehome.com/configure-passthrough-vmdirectpath-vmware-esxi-raid-hba-usb-drive/

-TLB

 

[Synthetickiller]

I'm making progress, thanks guys.

 

[Synthetickiller]

Hey guys. I am stuck on 8.2 of the all-in-one napp-it walkthrough:

set ESXi-IP and Storage IP to a manual IP-Adress (OI-menu System)
usually you have to manuall enter a dns-server
open a terminal and su
start nautilus
edit /etc/resolv.conf
add a nameserver, enter (8.8.8.8 is a free Google DNS):
nameserver 8.8.8.8

I don't understand this step.

I have a DynDns account (although my router can't seem to connect to it). Not sure if this goes here or something else.

 

[Synthetickiller]

I've made some progress and found some really odd issues.

I was able to install napp-it off the bat and it would work. I hit a wall and started over to see if I could move forward.

I have followed the all-in-one pdf and each time, multiple things fail.
1. Installing Napp-it fails. I get the following error (this occurs regaurdless of using su or not)

I'm thinking installation of VMware Tools is the cause. I originally did VMware Tools out of order, much later and Napp-it worked fine until VMware tools was installed.

2. Changing the DNS resolv.conf file will not work, even w/ su command. I believe VMware tools is also to blame.

I'm really lost as to what's going on.

 

[TheLastBoyscout]

Hey Synthetickiller,

The hyphen (minus) is NOT directly in front of the www; there is an extra space:

# wget -O - www.napp-it.org/nappit | perl

For issue #2 I didn't try to use the Google DNS, but used by corporate server one without a problem (I may have set it in OpenInidiana's GUI though, not in the resolve.conf file...)

-TLB

 

[Synthetickiller]

Hey Synthetickiller,

The hyphen (minus) is NOT directly in front of the www; there is an extra space:


For issue #2 I didn't try to use the Google DNS, but used by corporate server one without a problem (I may have set it in OpenInidiana's GUI though, not in the resolve.conf file...)

-TLB

Thanks dude.

I went back and caught the error. I hate linux font! I can't read it!

Anyways, I have everything installed. I'm now looking at how to build the ZFS pool. I'll be here for a while, lol.

 

[Synthetickiller]

So I have gone all the way to napp-it. I am stuck here.

I have one drive shown that I can add to creat a pool. That drive is 0gb.

How do I add drives?

I am not sure if OI should see the three 3TB drives on my raidcard that's set up as passthrough. I cannot use Gparted partitioner since I cannot get rights, even with su command.

This is not covered in the walkthrough, so please do not refer to it. That won't help. I don't want to re-creat the wheel. I just need to know how to add drives to the zpool and I can figure it out from there.

 

[danswartz]

That drive seems to be the vmware virtual cdrom. Your other drives are not apparently visible. Are you sure you passed the HBA thru correctly?

 

[Synthetickiller]

Danswartz, you're 100% correct. I thought about that myself when i saw it and came to that conclusion.

The only want to get everything to work was startnig over from scratch. Something wasn't ticked as I went through the walkthrough or maybe ESXi just wasn't playing nicely.

I have it working. Thanks for the help.

 

[danswartz]

Cool!