Svar`s DAS/NAS project

S

svar

Weaksauce
Joined
Aug 13, 2010
Messages
83
Hi everyone :)

The past week I have looked for an new disk solution for my mac mini server.
I have looked at stuff like QNAP, ReadyNAS and likly every other NAS/DAS we can get here in Norway.

While the Qnap units seems nice, according to the qnap forum, they has big problems with iSCSI, that is one of my requirements for this project.
(mac mini only has USB,FW800, LAN), and the FreeNAS, people say it make lot of noice.

So, I found out that maybe making my own NAS/DAS can provide me with what I want, and more flexible.

My requirements are as follows:
-Iscsi
-Raid that is easy to upgrade storage.
- Fast enough to stream music and some video.

For the software, I have looked at FreenNAS, FreeBSD and the offcause this project

Im not that happy in CLI, so I prefer GUI to work in....

So, I have looked at Intel Atom motherboards, but they are small (few PCI-e slots), and Im not shure if atom has the right power for an main Das system, running ZFS.
And I want as many SATA connectors on mobo as possible, hehe.

For the chassie, I have looked at Lian-Li PC-A70F
Yes, I know its huge, but it has nice HDD slot design and it should not be hard to keep cool.

At the CPU side, I have looked at Atom, i3 and i5.
I have an idea of that an i3 wil be much faster than an Atom and that an i5 is way over the top.

I have looked at the folloing mobos (in order)
Asus P7P55D-E
Asus P7P55 WS Supercomputer

I guess the mobos itself work with FreeBSD, but Im wondering about the SATA controllers.

Well, thats my plan so far, I keep up as I go on, and I realy like to get some feedback :)
 
Im back :)

Now I have looked around and I think I have found another chassis that wil be nice:
Define R3

This one is not possible to get in Norway yet, so Im using the time to look for an nice mobo.

I wil keep this post updated as I go on.
 
you may want to have a look at ASRock A330ION Atom 330 + NVIDIA MCP7A-ION:
4*Sata + 2 e-Sata +1 PCI-E + up-to 4 Gb RAM DDR3

Although idle power consumption between ATOM and i3 seems similar, under-load consumption is vastly increased for i3
 
Hey there!

Regarding the motherboard; if you have the money i would go for Core i3; much faster and virtually the same idle power consumption! For a NAS, idle power consumption is all that cares really, but Core i3 would give you the CPU power when you need it. It should still idle 99%+ on average, for most home users. Of course your power supply has to be able to deliver the maximum power, etc. But for heat, noise and your electricity bill you would only have to look at the idle consumption.

There are some very nice Mini-ITX and Micro-ATX Core i3 boards; try those, and look for the new 32nm chips with onboard video. The motherboard should have DVI/HDMI output as well.
 
HI again, and thanks for all the replys :D

The cpu Im looking at is the Intel Core i3 530 / 2.93Ghz to be spesific.
I want an mobo with as many SATA ports at possible, my setup wil be at least 6drives, and thats kinda impossible to find on an mini-itx board (im trying to avoid HDD controller card if I can)

Or, maybe not, I yust come across the Gigabyte GA-H55M-USB3.
(the rev1.0 version, havent find the rev2 in any store here in Norway)

Why I want an normal tower over an mini tower (yes, I know its mini towers out there that can hold 6HDD), is because of more air flow, if anyone wonder.

I mostly gona use the unit as an DAS over iSCSI, thats why I think intel i3 is better than an atom.
 
Hi :)

Im in full testing now, havent got the HDD`s yet, so Im testing with 2 I dont use and trying ZFS Stripe.

So far, connecting to my existing LAN, using my iMac as iSCSI initiator (they are connected to the same switch) I get around 50-60MB/secn write and around 70MB/sec read.
(around 2mins to transfer an DVD image)

This is with the internal nic, and then the server is doing nothing else.
Tried the same with an dvd streaming from it, and offcause the speed dropped.

Then you people recommend an Intel NIC, is it the desctop or server edition of the Intel PCI-E 1x card?

ow, one thing I can confirm now if anyone wonders, the current relese of FreenNAS works with the integrate GPU chip in the i3 CPU.
 
I'm interested in that Fractal Design case.

Is there enough room on the HDD trays for the SATA and power cables (between the HDD and the side of the case)? If you use straight SATA connectors, does the cable get a severe bend?
 
Hopefully I get it tomorrow, the R3 has been very difficult to get here in norway, mostly because its so new.

Then I get it I put out some pictures and stuff here :)
 
Hi :)

Sorry Im late, but here are some pictures:




The system is latest FreeNAS.
Things working good, but I have some questrions/problems:

-In the FreeNAS front page, it show nice the free space and the used space, but then I delete data, the used space dont show the extra free space.
An bug or something with the ZFS structure?

-The NAS/DAS is connected to my mac mini running OSX 10.5.8 server over iSCSI, using the globalSAN iSCSI Initiator for OS X, the server takes care of filesharing, users and stuff. Sometimes then I transfer lots of data at once, the iSCSI connection to the server is lost, and I have to reconnect and restart the AFP service to get access from the clients.
It seems the NAS is ok, and no data is lost, I guess its OSX or the GlobalSAN indicator that is making the problem, I heard that indicator has some problems, but I dont know about anyone else for OSX.
 
If you keep snapshots, deleting stuff won't free up space since it won't actually be deleted just de-referenced. If it isn't snapshotted, then you should get more free space available; but it can take a few seconds until next transaction group before the free space becomes available, i think.
 
I see, I guess I have to learn more about ZFS and how to manage it in the FreeNAS, because I dont know howto take snapshoots or if it maybe is on an auto, hehe :D

One more thing; the mobo Im using have 8SATA ports, 6 on the intel chipset (5 internal, 1 external) and 2 on an GIGABYTE chip.
My disks is 5 on the intel chip and 1 one the gigabyte chip, thats no problem or?
(FreeNAS finds all the ports, so it seems ok, but maybe I should use the ESATA port for the last internal drive?)
 
Last edited:
I now try to use an Ubuntu server with Ubuntu gui as the filserver.

I manage to connect to the FreeNAS over iSCSI, and format it as EXT3 at 8,59TB using GParted.

I guess the formating is no quick format, because it take ages for it to complete.
Alsa, if I refresh the front page of the FreeNAS webGUI, the "Used Space" yust keep growing, even over the max space of the ZFS array :confused:
The used space in the % bar, is 0%, even more :confused:

Any good advice for me?
First time Im trying this and I cant that mutsh about disk managment in Linux then it comes to large disks/arrays.
 
I've never used ext3 on a large filesystem with linux, I usually use XFS, without partitions (filesystem installed directly on block device). It only takes a few seconds to initialize XFS on a large RAID volume (greater than 20TB).

I am not really clear on what you tried. You mounted the FreeNAS device via iSCSI onto an Ubuntu server? Then ran gparted to do what, exactly? What formatting are you talking about, something gparted did?
 
I try to explain as good as I can what I did :)

1: I got an FreeNAS box, with 6*2TB HDDs, in one ZFS raid5 array, and have setup this as an 8,5TB iSCSI target.

2: I have installed Ubuntu server (+Gnome) on another box, and configured the iSCSI as described here: http://www.howtoforge.com/using-iscsi-on-ubuntu-9.04-initiator-and-target
(I skiped the fdisk steps, as fdisk seems to have max 2TB pr volume)

3: Ran GParted, setting the partition table to "gpd".
4: Maked an new volum (right click on free/unallocated space), used all free space as one primary volum, and sat the format to EXT3.
5: Waithed LONG TIME (as in many hours)

Thanks for youre reply :)
 
I assume you skipped the iscsi target setup in the howto, since your FreeNAS is already set up as an iscsi target. You just need to set up open-iscsi on the linux box so it can be an iscsi initiator.

After you have connected with iscsi, it would be helpful to see what fdisk displays:

$ sudo fdisk -l

so we can verify that the iscsi volume seems to be properly recognized by the linux box.

And fdisk has no 2TB limitation that I know of. You should be able to follow the howto.

Also, I think the howto uses the jargon misleadingly. They are not "formatting" the volume, they are partitioning it. To me, formatting a disk involves zeroing out the media, either by physically writing zeros, or at least clearing the partition table and any other special sectors.

If you want, you could try initializing an XFS filesystem, as a test.

$ sudo aptitude install xfsprogs
$ sudo mkfs.xfs -f -L freenas /dev/sd_

be careful to use the correct block device on the end, /dev/sd_ , which is where the fdisk -l that I mentioned before comes in handy. Also, do not specify a partition (like /dev/sdb1), just give the block device (like /dev/sdb). This will overwrite whatever you had on the specified block device, including the partition table (although it does not write to every sector of the device, so it should run fairly quickly).

If that seems to succeed, you can check it with:

$ sudo file -s /dev/sd_

and you should see something like:

/dev/sd_: SGI XFS filesystem data (blksz 4096, inosz 256, v2 dirs)
 
Last edited:
Hi :)

I tried it out some time ago, found out by google, as you say, that sudo aptitude install xfsprogs was needed.
Then I was able to use the "Disk Utility" in Ubuntu to make an XFS volume :D
( I like Disk Utility over GParted so far because D.U sett my user as owner) and used the Samba GUI in Webmin to setup samba shares for my OSX client.
(Yes, I like GUI :p)

Webmins hardware-->Partitions on Local Disks show the volum as an 2TB disk, but I realy dont care becuase the "df -l" command and prefs on the share in OSX show the right free/used space, so no Im happy :)

Seems like XFS is the right way to go for raid/ZFS volums, insted of the ext1/2/3/4

Ow, and it seems like the open-iscsi initiator is more stable than the GlobalSAN free one for OSX, so far no iSCSI drop-outs, but its still some hours before all files are copied over.
(my intel-atom linux box has a wooping 100Mbit/s nix :p)

Thanks for the help :D
 
svar said:
I tried it out some time ago, found out by google, as you say, that sudo aptitude install xfsprogs was needed.
Then I was able to use the "Disk Utility" in Ubuntu to make an XFS volume :D
Click to expand...

Well, xfsprogs is only needed if you are using a disk with XFS. It should have been possible to install an ext3 or ext4 filesystem on your volume -- I'm not sure why that did not work. But I prefer XFS for large RAID volumes, and since XFS worked for you, it sounds like everything is good now.
 
Well, I dident say ext3/4 dont work, I said it take ages to format/partition it (hours, I dident count)
And the Used space number in FreeNAS web-gui yust keept jumping up and up.
Then I canceled the ext format, ReadyNAS showed that the array had used like 50GB of 8.8TB, and that was before the partition/formatting was done and I had started moving files over.

Making the xfs volume took only 10-20 secounds and FreeNAS used space dident do all koko. :p
 
Hi again :)
Been testing FreeNAS for some time now, but I cant get the speed over 30MB/sec...
I use around 37sec to transfer 1GB file (testet with 2 diffrerent pc`s)

Im still using 6*2TB WD green HDDs in an raid5/Zraid and tried bouth SMB and AFP filesharing (using the FreeNAS as an nas atm)

More testing comming.
 
HI again :)

I have now changed the cabels and stuff.
I got 2 switches, one that the servers are connected to and one that the clients are connected to (because the servers and clients are in differents parts of my appartment and I dont wana use 10meters cabels for every node)

The switches are now connected together with an 10m cat6 FTP cabel, the servers with 1,5m cat6 UTP cabels.
The clients use standar cat5(e) cabels.
(before it was cat5(e) all over)

The FreeNAS server is same as my post before.

I first tried to copy over an folder of Apple Lossless files (only music in this folder):
Server uptime 5mins
59GB in 2766 objects took 34min = 1735MB/Min = 28,9MB/sec

Some time later I copyed my "collection of everything".
This folder has big ISO files, small ZIP files, and almost everything between:
Server uptime 6hours and 40min.

73,31GB in 10072 objects took 125mins =586MB/min = 9,7MB/sec

This was over SMB sharing direct to the NAS.

Any sugestions anyone?
 
You must log in or register to reply here.
Back
Top