OpenSolaris derived ZFS NAS/ SAN (OmniOS, OpenIndiana, Solaris and napp-it)

e2g said:
I have been trying to install openindiana all day, and everytime I reboot (power off and power on the machine), I am greeted with

grub>

I didn't have this problem the first time around, but recently just picked up two hard drives, and basically threw the old one out that was running openindiana. Did I need to do something special with the OS hard drive before taking it out and installing a fresh copy of OI?

Is there a guide that walks you through install OI? I am also trying to export a zfs pool from zfsguru. I have been formatting them using GEOM but I didn't not select the zero erase/ random/ secure erase because of time. I guess I may try that and erase this disk using zero erase
Click to expand...

if you have a supported and welll known to work hardware-
at least with disabled USB (seems to be a problem on some boards)

- boot OI live DVD
- Install OI on a single disk, use total disk and erase/initialize it during install

should work always. if you need a mirrrored boot drive, do it afterwards.
i would not use any special unsupported settings like ashift modifications or you may get problems (problems not worth a minimal speed advantage -
even when its 20%, its not a aahh and oohh effect, more for benchmarks than important in real live)
 
Last edited:
Today my file server was going slow and I logged into napp-it to see a drive listed as "FAULTED".

Why didn't napp-it send me an e-mail? I just re-ran the SMTP test and it worked fine.

Also does a "FAULTED" drive pretty much mean it should be replaced ASAP?

Here is what napp-it shows me:
Code: 
c0t5000CCA369C437F8d0	2.00 TB	tank	raidz	FAULTED	Error: S:0 H:0 T:12	Hitachi HDS5C302	-	-	-	-	-
 
jmk396 said:
Today my file server was going slow and I logged into napp-it to see a drive listed as "FAULTED".

Why didn't napp-it send me an e-mail? I just re-ran the SMTP test and it worked fine.

Also does a "FAULTED" drive pretty much mean it should be replaced ASAP?

Here is what napp-it shows me:
Code: 
c0t5000CCA369C437F8d0	2.00 TB	tank	raidz	FAULTED	Error: S:0 H:0 T:12	Hitachi HDS5C302	-	-	-	-	-
Click to expand...

replace it
test it with a manufacturers low level tool
or send it back

and:
have you enabled auto-service in napp-it?
did you get status mails about your pool?
 
jmk396 said:
Today my file server was going slow and I logged into napp-it to see a drive listed as "FAULTED".

Why didn't napp-it send me an e-mail? I just re-ran the SMTP test and it worked fine.

Also does a "FAULTED" drive pretty much mean it should be replaced ASAP?

Here is what napp-it shows me:
Code: 
c0t5000CCA369C437F8d0    2.00 TB    tank    raidz    FAULTED    Error: S:0 H:0 T:12    Hitachi HDS5C302    -    -    -    -    -
Click to expand...

I had the same thing today. A drive showed up with a few thousand hard and transfer errors, but no security errors. Only been running for half a week. I pulled the drive, put it back in and cleared the errors. Been running fine with no errors after that. Will have to keep an eye on it though. If it acts up again I'll have it replaced.
 
_Gea said:
have you enabled auto-service in napp-it?
did you get status mails about your pool?
Click to expand...
Here is what the auto service page shows me:

Code: 
auto-service: enabled every 15 minutes

last execution:

## last run was sat 16.jul 2011 22:15 Jobid:

I also never get status e-mails... I didn't realize that I was supposed to but the alert page shows this:

Code: 
Send status info and logs to [email protected]

Month (1..12) - every
Day (1..31 sun..sat) - sun	
Hour (0..23) - 23
Minute (15 Min interval) - 0

...but I never get any of those e-mails either.


Maximus825 said:
I had the same thing today. A drive showed up with a few thousand hard and transfer errors, but no security errors. Only been running for half a week. I pulled the drive, put it back in and cleared the errors. Been running fine with no errors after that. Will have to keep an eye on it though. If it acts up again I'll have it replaced.
Click to expand...
What's the process for doing this? I tried removing/reinserting the drive but nothing changed.
 
Last edited:
_Gea said:
if you have a supported and welll known to work hardware-
at least with disabled USB (seems to be a problem on some boards)
Click to expand...

Yes I do
_Gea said:
- Install OI on a single disk, use total disk and erase/initialize it during install
Click to expand...

Not sure if I know what you mean here. During install, I choose one disk, and tell it to use the entire disk. That is it.

I am not sure what the problem is. I was able to install on my original hard drive just fine. Does OpenIndiana have something against 2.5" hard drives? I just installed it again, and it installed just fine. When I shutdown the server, and then power it back up...I am greeted with "grub>" . No errors on the screen. What am I doing wrong? i have tried installing from a usb external dvd drive, and an internal dvd drive. This is driving me crazy.

EDIT: Maybe its because the drives are formatted with GEOM...?
 
Last edited:
e2g said:
Yes I do


Not sure if I know what you mean here. During install, I choose one disk, and tell it to use the entire disk. That is it.

I am not sure what the problem is. I was able to install on my original hard drive just fine. Does OpenIndiana have something against 2.5" hard drives? I just installed it again, and it installed just fine. When I shutdown the server, and then power it back up...I am greeted with "grub>" . No errors on the screen. What am I doing wrong? i have tried installing from a usb external dvd drive, and an internal dvd drive. This is driving me crazy.

EDIT: Maybe its because the drives are formatted with GEOM...?
Click to expand...

Geom is a BSD thing. I expected a reformat if you choose "use the whole disk"
Otherwise try re-partition the disk first to something like old mbr
 
Decided to put my old hard drive in, and remove the 2 drives I just received. I installed open indiana without a problem. Once I remove that hard drive, and shutdown the system, I should be able to install another hard drive and do a fresh install without a problem, correct?

I formatted earlier today, the hard drives (the new ones), and still I am not getting anything. When I reboot for the first time (when it actually does a power cycle), I am greeted with the grub> prompt. On my original WD Blue hard drive, i do not have this problem but with my new black hard drives, I do (I even tried to install it on both and same results. Why is OpenIndiana being racist against my Black hard drives? I am going to try one more thing, but I don't know what else I can try. I mean, I assume OpenIndiana has the drives for this hard drive as I can see it off the live dvd.

If it matters, I am trying to install OI to this: Western Digital Scorpio Black wd5000bpk

Any suggestions?

EDIT: _Gea, I will try this.
 
Last edited:
Maximus825 said:
zpool clear <poolname>
Click to expand...
This didn't work too... the pool is still degraded.

When/how do I remove the bad disk and then re-initialize it? I must be missing something :(

EDIT: Also, has anybody been able to install Net::SSLeay on Open Solaris? I'm getting an error about invalid option KPIC and no such file math.h (using cpan)
 
Last edited:
jmk396 said:
This didn't work too... the pool is still degraded.

When/how do I remove the bad disk and then re-initialize it? I must be missing something :(

EDIT: Also, has anybody been able to install Net::SSLeay on Open Solaris? I'm getting an error about invalid option KPIC and no such file math.h (using cpan)
Click to expand...

I'm going to tell you command line wise..

(Physically) Replace the disk. You said you're running OS? You might have to enable hotplug service, IIRC it is svcadm enable hotplug. Rescan with cfgadm... Off the top of my head, and early in the morning,cfgadm -a should suffice. You should see your new disk available/connected. If you put it in the same physical location, it would be there, but this time it should show connected :)

Replace the disk for the zpool, if it is in the same physical location then its zpool replace yourpool newdisk ... this is assuming you don't have auto-replace on.

If its in another location then its zpool replace yourpool baddisk newdisk then when its replaced zpool detach yourpool baddisk.

I didn't see any mention of spares, but just detach the spare once the replace has been completed. This should return the hotspare into normal operation (as a spare).

PS If you're trying to get the zpool to re-use the disk that is faulted, don't bother. the Zpool will re-grab the disk if its good and resilver/correct the bad data, but if its bad then replace the physical disk. cfgadm -a should also show you if the disk is not readable/accessible.
 
Last edited:
Using Zfsguru:

The working 'disc information'

ada1
........512....................# sectorsize
........500107862016....# mediasize in bytes (466G)
........976773168.........# mediasize in sectors
........0.......................# stripesize
........0.......................# stripeoffset
........969021..............# Cylinders according to firmware.
........16.....................# Heads according to firmware.
........63.....................# Sectors according to firmware.

The non-working 'disc information'

ada0
........512...................# sectorsize
........500107862016...# mediasize in bytes (466G)
........976773168........# mediasize in sectors
........4096................# stripesize
........0.....................# stripeoffset
........969021............# Cylinders according to firmware.
........16...................# Heads according to firmware.
........63..................# Sectors according to firmware.


Again, non-working means, after installing openindiana, the 'grub>' prompt comes up rather than the grub OS selection screen. One thing that is interesting above is 'stripe size'. As far as I can remember, the working (Western Digital 3.5" Blue drive --WD5000AAKX) is a 4K drive but I am beginning to think it is not, and that may explain something. I hope I don't have to return these Scorpio drives. So I guess I need to find out how do I install OI on a 4K drive.
 
Has anyone here installed their OS onto a 4K aligned drive? Searching around on the internet reveals ppl working with zpools, but not anything to do with OS installs. I guess I need to manually align it (somehow).
 
Last edited:
quick question for everyone

how easy is it to set something like this up for someone who has no experience with operating systems other than windows and os x?

I would be buying a new system, something along the lines of a norco 4224, supermicro x9 board, xeon 1220, 16gb ram, recommended controller cards, and hitachi 3tb drives

I don't want something that is going to require a lot of maintenance, I'd like to set it up, and it just work as a huge external drive I can access from my windows 7 computers

I'd likely start with 1 controller card and 8x3tb drives and add more drives in sets of 8 when I need them
I'd probably do raid z2 or z3 on each set of 8 drives
 
e2g said:
Has anyone here installed their OS onto a 4K aligned drive? Searching around on the internet reveals ppl working with zpools, but not anything to do with OS installs. I guess I need to manually align it (somehow).
Click to expand...

There is no problem with 4k disks beside a slightly reduced pereformance.
Just use it.
 
EnderW said:
quick question for everyone

how easy is it to set something like this up for someone who has no experience with operating systems other than windows and os x?

I would be buying a new system, something along the lines of a norco 4224, supermicro x9 board, xeon 1220, 16gb ram, recommended controller cards, and hitachi 3tb drives

I don't want something that is going to require a lot of maintenance, I'd like to set it up, and it just work as a huge external drive I can access from my windows 7 computers

I'd likely start with 1 controller card and 8x3tb drives and add more drives in sets of 8 when I need them
I'd probably do raid z2 or z3 on each set of 8 drives
Click to expand...


Installation is easy, if you have a dhcp server for networking:
- boot OpenIndiana Live Iso and install it to your disk (not more than klick some ok's, enter hostname, username and pw)
- login as user, start a shell and install napp-it with: sudo wget -O - www.napp-it.org/nappit | perl
(installs and configures all NAS related things + Web-GUI)
- manage your NAS via Web-GUI http://serverip:81

You need to become familiar with some ZFS and Raid basics (how to create a pool, how to share and set permissions, what to do on disk errors)
Its good to know about some unix specials like permissions/ ACL, basic directory structure, user/ groups but thats similar to OSX


Problems:
If you want to install tools, not in the basic install, you need to learn a bit more about Unix
 
Last edited:
Hi gea


isit possible to install OI on a usb drive? will the drive degrade over time? as too much read/write on the drive.

Hoping to free up some sata port

Thanks
 
kegler said:
Hi gea


isit possible to install OI on a usb drive? will the drive degrade over time? as too much read/write on the drive.

Hoping to free up some sata port

Thanks
Click to expand...

mostly it does not work and when, you will not be happy,
it is not reliably enough and unusable slow
 
_Gea said:
There is no problem with 4k disks beside a slightly reduced pereformance.
Just use it.
Click to expand...

Danke.

As far as I know, there isn't but I wanted to know if people have had success installing the OS to a 4K disk and not simply created pools after the OS is installed. I am not sure why, when I have a non-4K drive, the OS installs without a problem, but when I have a 4K drive (formated in all kinds of ways), whatever i try, it always boots up to the grub> prompt. Perhaps i should post a thread in the Operating Systems forum to get more feedback? I don't know.

Thanks.
 
e2g said:
Danke.

As far as I know, there isn't but I wanted to know if people have had success installing the OS to a 4K disk and not simply created pools after the OS is installed. I am not sure why, when I have a non-4K drive, the OS installs without a problem, but when I have a 4K drive (formated in all kinds of ways), whatever i try, it always boots up to the grub> prompt. Perhaps i should post a thread in the Operating Systems forum to get more feedback? I don't know.

Thanks.
Click to expand...

Two possibilities:
1. You discovered a new problem
(You are the first to use this special 4k disk, Thats possible, because 4k disks are
mostly high capacity disks not used for OS)

2. This disk is bad
I have had a similar problem with a disk (non 4k) that i could use only for datapools
The supposed problem was a bad sector in boot area
If you have one, try a second disk
 
_Gea said:
Two possibilities:
1. You discovered a new problem
(You are the first to use this special 4k disk, Thats possible, because 4k disks are
mostly high capacity disks not used for OS)
Click to expand...

Uh oh. Then again, this would surprise me.

_Gea said:
2. This disk is bad
I have had a similar problem with a disk (non 4k) that i could use only for datapools
The supposed problem was a bad sector in boot area
If you have one, try a second disk
Click to expand...

Actually, I know one of the disk may be bad (random clicking) of the two I bought. For my final test, i have unplugged all the other drives in the system and going to try and install OI on both of them one at a time. Another staying up all night again :(
 
I installed only one hard drive (one of the two 2.5" 4K drives i recently bought. None of the 2TB drives plugged in). I install OI, and after the few screens where it gets to load the OS, it doesn't. It keeps restarting my computer. So, i swapped the first hard drive with the second one (again, I bought 2 4K 2.5" laptop drives to be used as a rpool mirror) and same thing happened.
 
danswartz said:
If the disk is, say, c1t1d1 and the new one is c2t2d2, you would do like:

zpool attach POOL c1t1d1 c2t2d2

it should start resilvering automatically.
Click to expand...

how about adding 2 more disk to form raidz1 or another 3 more disk to form raidz4 later on? as in add 2 or 3 at the same time would the first single disk able to maintain the data and rebuild into raidz1 or 2 ?
Thanks
 
with current ZFS you can only add disks to a basic or mirrored vdev
basic -> mirror or 2 x mirror to 3 x mirror
you can not add to a raid-z or transform a vdev type.

maybe with one of the next ZFS versions....
 
_Gea said:
that could really become a problem but currently I expect a 8 GB limit per VM not per ESXi
see http://communities.vmware.com/message/1790276
Click to expand...
I sure hope so, but the thread is still going on and it still seems unclear whether the 8 GB are "total host system memory", "total host system memory per CPU" or "maximum system per VM" (in my own words). VMware didn't help by introducing new terms (vRAM) that seem to cause more confusion than anything... :mad:

_Gea said:
as i know, you can currently register as much 4.1 versions as you need [...]
Click to expand...
That may help for the immediate future (to get at least the planned systems up and running). Yeah, I found the following statement at the very bottom of the download page:
"This license key can be deployed on an unlimited number of physical hosts, but is restricted to deployment on less than or equal to 5 Physical Servers."
:confused:
Could someone educate me on the difference between "physical hosts" and "physical server"?

Thanks!
-TLB
 
Hi gea

Can I install lsi 9211-8i and update driver in oi? Does oi contain the driver? Thinking of getting a new 9211-8i today.
Instead of the traditional way to install driver on booting as I have already install Oi, don't want to reinstall again. Boot disk remains frOm mb sata Port
Thanks
 
TheLastBoyscout said:
I sure hope so, but the thread is still going on and it still seems unclear whether the 8 GB are "total host system memory", "total host system memory per CPU" or "maximum system per VM" (in my own words). VMware didn't help by introducing new terms (vRAM) that seem to cause more confusion than anything... :mad:

Thanks!
-TLB
Click to expand...

It's total amount of vRAM assigned to running virtual machines. So, if you had 4 VMs with 2GB each, you would be at the limit. You could have 8 VMs with 2GB each, but only have 4 running and that would be OK. From what I understand, it's per-socket too. So if you had 2 CPUs, then you would be able to have 16GB vRAM. It doesn't matter how much physical RAM you have.
 
kegler said:
Hi gea

Can I install lsi 9211-8i and update driver in oi? Does oi contain the driver? Thinking of getting a new 9211-8i today.
Instead of the traditional way to install driver on booting as I have already install Oi, don't want to reinstall again. Boot disk remains frOm mb sata Port
Thanks
Click to expand...

There isn't a flash binary that will run under OI. You shouldn't have to reinstall. Get a bootable USB stick with FreeDOS and put the firmware and flasher on it. Boot from the stick, flash, then reboot from your OS drive.
 
_Gea said:
with current ZFS you can only add disks to a basic or mirrored vdev
basic -> mirror or 2 x mirror to 3 x mirror
you can not add to a raid-z or transform a vdev type.

maybe with one of the next ZFS versions....
Click to expand...

So I can just create 2x2tb mirror and add another 2x2tb to the same pool to create raid 10? Any limit on how many more mirror I can add to the same pool?
 
Maximus825 said:
There isn't a flash binary that will run under OI. You shouldn't have to reinstall. Get a bootable USB stick with FreeDOS and put the firmware and flasher on it. Boot from the stick, flash, then reboot from your OS drive.
Click to expand...

How about flashing fr a win7 pc?
 
kegler said:
So I can just create 2x2tb mirror and add another 2x2tb to the same pool to create raid 10? Any limit on how many more mirror I can add to the same pool?
Click to expand...

AFAIK, you can add mirrors to the pool as much as you want. So, say, you had 4 disks in a 2x2 config (two mirrors striped - raid10), you could add two more as another mirror and have 3x2 config.
 
I'm a little confused on this step:
9.1 mount NFS SAN datastore
Klick on the ESXi icon (red 1.), then select Storage (red 2) and Add Storage (red 3.)
Select Network Files System (NFS) and enter
host: your host ip
folder: ex /data/nfs
name : ex nfs
Thats it, you can now create new virtual machines on this datastore with name nf
Click to expand...
Would I be able to install a vm of Debian on the nfs share? That's what it looks like but what do I do when linux asks to format the drive?
 
smellyeagle said:
I'm a little confused on this step:

Would I be able to install a vm of Debian on the nfs share? That's what it looks like but what do I do when linux asks to format the drive?
Click to expand...

If I understand you correctly, yes.

When you create a debian vm on the nfs share, it creates a file that represent the vms harddisk, just format it like you would do with any physical debian installation.
 
Varg said:
If I understand you correctly, yes.

When you create a debian vm on the nfs share, it creates a file that represent the vms harddisk, just format it like you would do with any physical debian installation.
Click to expand...

From the guest's point of view, it cannot tell the hard drive is virtual or that it is on an NFS share. That is all hidden by esxi.
 
Varg said:
If I understand you correctly, yes.

When you create a debian vm on the nfs share, it creates a file that represent the vms harddisk, just format it like you would do with any physical debian installation.
Click to expand...

So I'd format the OI with ZFS and then when install Debian on top of that it's ok to format the vm as ext3/4 or whatever? To me it seems unelegant.
 
You must log in or register to reply here.
Back
Top