OpenSolaris derived ZFS NAS/ SAN (OmniOS, OpenIndiana, Solaris and napp-it)

PigLover

[H]ard|Gawd
Joined
Jul 11, 2009
Messages
1,181
...To all of you - thanks a lot for your help! This forum rocks!

People here helped me when I was stuck, so I try to pass on the limited things I know when it seems appropriate. Stick around and help others - that's all the thanks any of us could ask for!

BTW, your welcome. Glad you got it sorted out.
 

Jim G

Limp Gawd
Joined
Jun 2, 2011
Messages
221
@dilidolo, you're great. Thanks a lot, this has immediately solved my problem.
<snip>
I think that's about it, I hope I have not forgotten anything. Last thing now is that the plumb is gone after a reboot. Does anyone know where I have to add this so that it remains persistent?

To all of you - thanks a lot for your help! This forum rocks!

Well, your post helped me a lot so hopefully this will help you - look at the Configuration files section:

http://solaris-x86.org/documents/guides/interfaces.mhtml

I did what you did and then followed the instructions on that page and my network settings are now persistent - at least after I figured out that I couldn't use the same name as the host name.
 

Freak1

Limp Gawd
Joined
Sep 9, 2009
Messages
191
Hi guys.

I want to update my All-In-One server software, and swap my OS/virtuel harddisk controller to a Adaptec 6405.

So i hope you guys can help me with the best order to do it and how.

I already tryed swapping the controller but ESXi does boot but it dosent find the storage, even though it on the same controller. It seems like i need to get newer ESXi first then get the right driver.

All the virtuel machines and ESXi is on my RAID 10

Software now
ESXi 4.1
Openindiana 148
Napp-it 0.500s Nightly

New software?
VMware vSphere Hypervisor 5
Openindiana 151a
Napp-it 0.6k

The server
ESXi server
1x SuperMicro SC846E1-1200B
1x MBD-X8DAH+-F -B
2x Intel Xeon E5606, 2.13 GHz - Quad Core/1066/8 MB
2x Heatsink SNK-P0038P (Rev. A & B) 2U+ DP Server
6x Kingston DDR3 ECC Reg, 1333 MHz, DR x4, 4 GB (24 GB)
1x Adaptec 51245 RAID controller for Hybrid RAID
2x Intel 510 Series SSD, 120 GB, 450/210 MB/sec (Hybrid RAID 10)
2x Seagate 1TB es2 (Hybrid RAID 10)
1x LSI SAS9201 PCIE 16 PORT INT
2x SAS EL2/EL1 Cascading Cable (External), 68cm CBL-0166L

Software:
ESXi 4.1
Openindiana 148 with Napp-it
Windows 2008R2

External jbod case 45 bay
1x SuperMicro 847E16-RJBOD1
24x Hitachi Ultrastar 7K3000 3TB (RAID-Z3) 72TB advertised 49,7TB usable
 
Last edited:

_Gea

2[H]4U
Joined
Dec 5, 2010
Messages
4,042
Hi guys.

I want to update my All-In-One server software, and swap my OS/virtuel harddisk controller to a Adaptec 6804.

So i hope you guys can help me with the best order to do it and how.

I would not update but reinstall:
- install ESXi 5 and activate pass-through to SAS controller, reboot
- install OpenIndiana 151 live with pass-through adapter, set to autostart first
(use ESXi filebrowser to copy boot-iso to local datastore, use it as virtual DVD file)
- install vmware-tools
- install napp-it via wget, set root pw, reboot
- import datapool

- import NFS shared datafolder in ESXi
- start ESXi filebrowser and import VM's (right-click on .vmx file)
confirm moved when asked if the VM is copied or moved

up and running in 30min up to an hour!
 

Captainquark

Weaksauce
Joined
Dec 14, 2011
Messages
103
@Jim G
Thanks for the link, but even after reading it, I'm still a bit confused... I cannot find any config files for my NICs. I do have 3 NICs: 1 for Mgmt (net3), 2 for Aggregate (aggr0). So if the hostname of my box is NAS1, I should have at least one file /etc/nas1.net3 , right? I don't have this.... And then, the Aggregate is temporary right now because it is configured through ifconfig, so does this mean I have to create a blank file /etc/nas1.aggr0 and only have one line in it, like 192.168.1.224/24? Under what hostname would I enter this IP in the hosts file?
Then why could you not use the same name as the hostname - in all documentation, it says exactly that you need to do that...
And why is there no config file for my Mgmt interface?
Appreciate some clarifying hints... thanks!
 

_Gea

2[H]4U
Joined
Dec 5, 2010
Messages
4,042
@Jim G
Thanks for the link, but even after reading it, I'm still a bit confused... I cannot find any config files for my NICs. I do have 3 NICs: 1 for Mgmt (net3), 2 for Aggregate (aggr0). So if the hostname of my box is NAS1, I should have at least one file /etc/nas1.net3 , right? I don't have this.... And then, the Aggregate is temporary right now because it is configured through ifconfig, so does this mean I have to create a blank file /etc/nas1.aggr0 and only have one line in it, like 192.168.1.224/24? Under what hostname would I enter this IP in the hosts file?
Then why could you not use the same name as the hostname - in all documentation, it says exactly that you need to do that...
And why is there no config file for my Mgmt interface?
Appreciate some clarifying hints... thanks!

its a bit confusing at the moment because configuration on Solaris 11 and mostly OpenIndiana is different from Nexenta* or Solaris 10/ OpenSolaris where you do all via ifconfig and files like /etc/host.nic

Current config is done via tools like ipadm, best is to read Oracle docs about:
http://docs.oracle.com/cd/E23824_01/html/821-1458/gdyrf.html#scrolltoc
 

sjalloq

n00b
Joined
Jun 20, 2011
Messages
57
Yes I have gone through what I think is the flow. I can successfully connect from OS X so it looks like it works. But as previously stated, it doesn't work from Ubuntu.

To close my own thread...I finally found the answer here: http://www.justlinux.com/nhf/Filesystems/Mounting_smbfs_Shares_Permanently.html

When you connect to an SMB share from Linux only the local root user is given write permission. It doesn't matter what the ACL's of the share are or the fact that you have connected using an OI user's credentials. If you want to give write access to Linux users/groups then you need to specify them when you mount the share. In my case I have to do the following:

Code:
sudo mount -t cifs //ipaddr/smb_share /mnt/mnt_point -o username=smbuser,password=smbpasswd,uid=linuxuser

Hope that helps someone else too.

linux cifs smb write permission zfs ubuntu server
 

Captainquark

Weaksauce
Joined
Dec 14, 2011
Messages
103
@_Gea,
I tried what was described in the manual. In fact, after the creation of the aggregate, I do:
Code:
ipadm create-ip aggr0
ipadm create-addr -T static -a 192.168.1.224/24 aggr0/v4

These commands run successfully. Nevertheless, after a reboot, this is the result:
Code:
root@nas1:~# ipadm show-addr
ADDROBJ           TYPE     STATE        ADDR
lo0/v4            static   ok           127.0.0.1/8
net3/_a           static   ok           192.168.1.222/24
lo0/v6            static   ok           ::1/128
aggr0/v4          static   disabled     192.168.1.224/24

Why is it always disabled? According to the doc, I have done everything necessary... I think, at least.

Oh yeah, this is my hosts file:
Code:
::1 nas1 localhost
127.0.0.1 nas1 localhost loghost
192.168.1.222 nas1
192.168.1.224 nas1aggr

Also, is there a way to force gdm to always and only bind to 192.168.1.222?

Thanks,
Cap'
 

tladuke

n00b
Joined
Sep 26, 2011
Messages
52
So, I have Open Indiana and Napp-it running and I just realized, I think I made a huge mistake: The boot drive is only one disk. It should definitely be on a mirror right?

And how do you backup (image) the root pool? Hopefully I don't need to boot off another media to do it... Still have a lot of Solaris to get used to.
 

tladuke

n00b
Joined
Sep 26, 2011
Messages
52
I'm out of SATA ports on the motherboard. Anyone know a good Mini PCI-e to SATA card? Just one or two ports
 

tladuke

n00b
Joined
Sep 26, 2011
Messages
52
Man...
I can't find anything in the cheap, non-raid category that looks like it will work with OI. I don't need it to be fast. 2 SATA ports.
 

s0rce

Limp Gawd
Joined
Jan 17, 2011
Messages
495
Man...
I can't find anything in the cheap, non-raid category that looks like it will work with OI. I don't need it to be fast. 2 SATA ports.

Try to find a Br10i or M1015 on ebay (or in the forum buy/sell) and get a SAS 8087 -> 4x SATA forward breakout cable from monoprice.

Some Silicon Image SiI3124 or 3132 cards might work, google around to find more information.
 

Captainquark

Weaksauce
Joined
Dec 14, 2011
Messages
103
Still no luck with my aggregate... anyone has an idea as to why my aggregate is always disabled after a reboot? Googled and read Oracle docs for hours...
Thanks!
Cap'
 

Origin_Unknown

Limp Gawd
Joined
Dec 12, 2009
Messages
257
is it just me or has solaris express 11 disapeared from the oracle site? I've clicked the link in the OP (and used google) but the link + google only take me to solaris 11. just curious about the licencing issues of using solaris, i cant see anywhere to say its not free or anything about a time limit

(napp-it related as its the host os :p )
 

s0rce

Limp Gawd
Joined
Jan 17, 2011
Messages
495
is it just me or has solaris express 11 disapeared from the oracle site? I've clicked the link in the OP (and used google) but the link + google only take me to solaris 11. just curious about the licencing issues of using solaris, i cant see anywhere to say its not free or anything about a time limit

(napp-it related as its the host os :p )

As far as I've read the licensing for solaris 11 is the same as solaris 11 express, free for non commercial development/testing use.
 

Origin_Unknown

Limp Gawd
Joined
Dec 12, 2009
Messages
257
Went from the machine in my sig (asus e350 and a 1 port sata card) with read 262MB/s / Write 400ishMB/s to a Intel e5300 with an M1015 in IT mode and the benchys make me smile :D

benchmark_m1015.jpg
 

s0rce

Limp Gawd
Joined
Jan 17, 2011
Messages
495
I got nice improvements going from an e350 to a Athlon II X2 250, the e350 is a nice low power system but its pretty limited.
 

levak

Limp Gawd
Joined
Mar 27, 2011
Messages
386
I think you could get nice performance if you would just put a M1015 in e350 motherboard?

Matej
 

Origin_Unknown

Limp Gawd
Joined
Dec 12, 2009
Messages
257
^ yeah, i tried that but the asus board i have either didnt see the card or tried to think it was a graphics card. its by far the worst board ive ever bought
 

levak

Limp Gawd
Joined
Mar 27, 2011
Messages
386
I see the problem. You need a x8 PCIe port, but ASUS only has 4x.

Unfortunatelly, non of the AMD fusion chipsets support more than 4x PCIe:(
 

zfs-player

n00b
Joined
Dec 26, 2011
Messages
4
Hi,
since this is the napp-it thread I'll post my question here. Hope it's right!

napp-it currently seems not fully adjusted to Solaris 11. It seems that power.conf is no longer supported by Solaris 11. (see Oracle documentation - not sure if I'm allowed to post links).

How can I control disk spin down and other power settings?

thanks!
 

_Gea

2[H]4U
Joined
Dec 5, 2010
Messages
4,042
Hi,
since this is the napp-it thread I'll post my question here. Hope it's right!

napp-it currently seems not fully adjusted to Solaris 11. It seems that power.conf is no longer supported by Solaris 11. (see Oracle documentation - not sure if I'm allowed to post links).

How can I control disk spin down and other power settings?

thanks!

Yes, Solaris 11 is dfferent and this is not yet supported in napp-it
see http://docs.oracle.com/cd/E23824_01/html/821-1451/gjwsz.html
 

zfs-player

n00b
Joined
Dec 26, 2011
Messages
4
Thanks, Gea!

The link you posted is what I was refering to. However I can't find anything wrt spinning down disks. power.conf is no longer availible but it does not really state what is the alternative... Any ideas?

thanks!
 

_Gea

2[H]4U
Joined
Dec 5, 2010
Messages
4,042
Thanks, Gea!

The link you posted is what I was refering to. However I can't find anything wrt spinning down disks. power.conf is no longer availible but it does not really state what is the alternative... Any ideas?

thanks!

I have no more info nor seen a howto elsewhere

ps
Its too new and Oracle has only focus on datacenter use
where power management is not evident
 

PigLover

[H]ard|Gawd
Joined
Jul 11, 2009
Messages
1,181
I have no more info nor seen a howto elsewhere

ps
Its too new and Oracle has only focus on datacenter use
where power management is not evident

Half right. Oracle is definitely only focused on data center use. But power management is very evident in that environment. Data Center managers care a lot more about it than your average SOHO user.
 

_Gea

2[H]4U
Joined
Dec 5, 2010
Messages
4,042
Half right. Oracle is definitely only focused on data center use. But power management is very evident in that environment. Data Center managers care a lot more about it than your average SOHO user.

ok
seems not evident for Oracle, especially the disk spin down.

I have not yet seen any more complete information about new Solaris 11 settings
beside the very basic ones from the link above.
 
Last edited:

gasu

n00b
Joined
Dec 3, 2011
Messages
47
Can someone tell me how to mount a ntfs drive under openindiana ? need to take some stuff off a ntfs drive and if i could do it directly on the server it would be better.

Thanks
 

_Gea

2[H]4U
Joined
Dec 5, 2010
Messages
4,042
Can someone tell me how to mount a ntfs drive under openindiana ? need to take some stuff off a ntfs drive and if i could do it directly on the server it would be better.

Thanks

it may be possible, but if you need it 'just working',
connect your disk to a pc or a mac and copy your files over the net.
- use a cheap sata/ide to usb (best usb3) adapter if you have bare disks
 

gasu

n00b
Joined
Dec 3, 2011
Messages
47
Yeah it's like 60gb so that's why i would like to transfer from the disk but worst comes i will have to do it like that i guess except just transfer it over samba or so from my mac
 
Last edited:

gasu

n00b
Joined
Dec 3, 2011
Messages
47
Ok figured it out using openindiana SEF i managed to install ntfs-3g and then with some linking since it wasn't working by default it's working now.

Is there any way to patch openindiana 151a spool for 4k drives the one where you don't lose much performance?

Cheers
 

bexamous

[H]ard|Gawd
Joined
Dec 12, 2005
Messages
1,670
Hmm... so been running OI on ESX5 for a month now and twice now the OI guest has reset and I've found it at login screen. I don't know if its KPing or something and automatically rebooting or what exactly is happening. Nothing in logs looked interesting, nothing in ESX events. Kinda confused wtf happens.
 

_Gea

2[H]4U
Joined
Dec 5, 2010
Messages
4,042
Ok figured it out using openindiana SEF i managed to install ntfs-3g and then with some linking since it wasn't working by default it's working now.

Is there any way to patch openindiana 151a spool for 4k drives the one where you don't lose much performance?

Cheers

i would not because you may find more troubles than performance

but if you want, you may use a modified zpool binary from OI 148 http://digitaldj.net/2010/11/03/zfs-zpool-v28-openindiana-b147-4k-drives-and-you/
or you may create a pool via ZFSguru and import it.
 

zfs-player

n00b
Joined
Dec 26, 2011
Messages
4
I created the zpool using FreeNAS 8.0.2.
Export zpool.
Import under Solaris, etc. and optionally upgrade to latest zpool version. (you can't go back to an older version anymore if you do this).

Worked fine so far. ashift 12 is shown and there is definitly an increase in performance.
 

gasu

n00b
Joined
Dec 3, 2011
Messages
47
Thanks for the help on the 4k drives , i have zfs guru so i will do it like that when the time comes.

I have to format the drives as ugh forgot but not gpt on zfsguru for solaris to see it tho , right ? gdm or so

Cheers
 
Top