FreeNAS and ESXi 4 slow through NFS

J

jadams

2[H]4U
Joined
Mar 14, 2010
Messages
4,086
I'm getting absolutely terrible transfer speeds from my ESXi box to my FreeNAS machine.

I can get around 50-60MBps from the Windows shares.

Running the VMware converter on a machine I created in VMWare Workstation, and sending it directly to the NFS datastore I see about 2-3MBps

http://community.spiceworks.com/topic/102686-vm-on-nfs-has-slow-network-performance?page=4

Pretty much identical to that guy's problem.

System graphs show the FreeNAS system isnt under any funky load. Why would SMB/CIFS speeds be fine and NFS so slow?
 
NFS is used in sync mode for ESX datastores. Unless you have a BBWC, or turn off sync mode, performance will be pretty slow. That's why people use nexenta with a ZIL.
 
Not recommending this for production, but to prove/disprove this is your issue, set sync=disabled on the nfs shared folder and retest.
 
From a shell prompt Just do 'zfs set sync=disabled XXX', where 'XXX" is the nfs share folder.
 
Hmmm, maybe I was reading too fast, but the linked article mentioned using ZFS. OP?
 
its worth noting that I have an earlier version of FreeNAS connected to this ESXi box and it doesnt exhibit these problems. I verified all settings in the GUI are identical.
 
Also using ZFS? Either it has sync disabled or the theory is wrong. Can't really comment further until I know if this is a red herring.
 
I'll see what i can do. I might not have access to the machine till Tuesday. Apparently setting up SSH isnt very easy either.
 
Hmmm, may not need shell access. Does the webui give you any way to update that setting? If not, does it allow you to execute shell commands from the webui?
 
No it doesnt, i tried setting up a cron job to run the command, but i didnt see any output. It has a console viewer, but you cant enter commands.
 
actually just got shell access,

Your command tells me "invalid dataset name"

I'm using

Code: 
zfs set sync=disabled /mnt/volume0/testesxi
 
No, not the pathname, the dataset name. From the output of 'zfs list'. My nfs share is 'tank/esxi-datastore', so I would do 'zfs set sync=disabled tank/esxi-datastore'.
 
it doesnt like "sync"

output of zfs set is:

Code: 
[root@freenas] /# zfs set
missing property=value argument
usage:
        set <property=value> <filesystem|volume|snapshot> ...

The following properties are supported:

        PROPERTY       EDIT  INHERIT   VALUES

        available        NO       NO   <size>
        compressratio    NO       NO   <1.00x or higher if compressed>
        creation         NO       NO   <date>
        mounted          NO       NO   yes | no
        origin           NO       NO   <snapshot>
        referenced       NO       NO   <size>
        type             NO       NO   filesystem | volume | snapshot
        used             NO       NO   <size>
        usedbychildren   NO       NO   <size>
        usedbydataset    NO       NO   <size>
        usedbyrefreservation  NO       NO   <size>
        usedbysnapshots  NO       NO   <size>
        aclinherit      YES      YES   discard | noallow | restricted | passthrough | passthrough-x
        aclmode         YES      YES   discard | groupmask | passthrough
        atime           YES      YES   on | off
        canmount        YES       NO   on | off | noauto
        casesensitivity  NO      YES   sensitive | insensitive | mixed
        checksum        YES      YES   on | off | fletcher2 | fletcher4 | sha256
        compression     YES      YES   on | off | lzjb | gzip | gzip-[1-9]
        copies          YES      YES   1 | 2 | 3
        devices         YES      YES   on | off
        exec            YES      YES   on | off
        jailed          YES      YES   on | off
        mountpoint      YES      YES   <path> | legacy | none
        nbmand          YES      YES   on | off
        normalization    NO      YES   none | formC | formD | formKC | formKD
        primarycache    YES      YES   all | none | metadata
        quota           YES       NO   <size> | none
        readonly        YES      YES   on | off
        recordsize      YES      YES   512 to 128k, power of 2
        refquota        YES       NO   <size> | none
        refreservation  YES       NO   <size> | none
        reservation     YES       NO   <size> | none
        secondarycache  YES      YES   all | none | metadata
        setuid          YES      YES   on | off
        shareiscsi      YES      YES   on | off | type=<type>
        sharenfs        YES      YES   on | off | share(1M) options
        sharesmb        YES      YES   on | off | sharemgr(1M) options
        snapdir         YES      YES   hidden | visible
        utf8only         NO      YES   on | off
        version         YES       NO   1 | 2 | 3 | 4 | current
        volblocksize     NO      YES   512 to 128k, power of 2
        volsize         YES       NO   <size>
        vscan           YES      YES   on | off
        xattr           YES      YES   on | off
        userused@...     NO       NO   <size>
        groupused@...    NO       NO   <size>
        userquota@...   YES       NO   <size> | none
        groupquota@...  YES       NO   <size> | none

Sizes are specified in bytes with standard units such as K, M, G, etc.

User-defined properties can be specified by using a name containing a colon (:).

The {user|group}{used|quota}@ properties must be appended with
a user or group specifier of one of these forms:
    POSIX name      (eg: "matt")
    POSIX id        (eg: "126829")
    SMB name@domain (eg: "matt@sun")
    SMB SID         (eg: "S-1-234-567-89")
 
i loaded up Crystal Disk Mark on the VM. It actually reads @ 90MB/sec. Writes at <1MB/sec
 
Oh, you know what? I think freenas might be old enough zfs to not have that. Google for disabling the zil on freenas and see - I'm pretty sure there is a setting for that. That said, I agree with lopo.
 
thanks for your help. i believe i found the loader.conf in /boot folder.

i gotta go google how to use vi again. *sigh*.

Two NIX classes in college that I hated. :(
 
If you have GenMay access you'll get a kick out of my thread in there regarding this issue. Though everyone was extremely helpful trying to talk me through my small *NIX knowledge.

Long story short. I FINALLY edited the /boot/loader.conf file and put in the appropriate line of code.

Rebooted.... Web interface down, SSH down, but still pings.... I'm not in the office again until Tuesday. I'll get someone to give it a hard reset and we'll see what happens.

Good thing this wasnt in production yet. :D
 
Got someone in the office to hard reset it.... doesnt come back up. I wont be in the office for quite some time.
 
Probably! lol

But i put the line in character for character that it said to.
 
I'm looking at other options for a NAS. Someone suggested earlier versions of FreeNAS, I may try napp-it.
 
No offense, but if you thought freenas was hard to configure being too unixy, you will NOT like anything opensolaris related.
 
i toyed around with napp-it before. and other than entering the single command to download and install the app i never touched the command line. though I never got around to seeing if the nfs issue was there as well.
 
So I got into the office today only to find that someone hard reset the wrong machine. No wonder it didnt come back up lol!

After restarting the NAS i ran another CDM and found that now my READ speeds are about 6MB/sec and my write speeds still suck @ less than than 1MB/sec.

SMB shares still do not have this problem.
 
Hello

I have the same problem. But only with ESXi and an poor write performance. Reading from NFS-Share have the same performance how an access to an CIF/SMB-Share.

I use FreeNAS 8.0.4 installed on a USB-Stick.
The hardware:
CPU: Intel(R) Xeon(R) CPU 3040 @ 1.86GHz (dual core)
RAM: 2 GB
NIC: 2 x 1Gbit


A test with an 1 GB big file:
CIF/SMB access with Windows-box: writing: ~ 30 sec, reading 30 sec
NFS access with Linux-box: writing: ~ 30 sec, reading 30 sec
NFS access with ESXi 4 or 5: writing: ~ 3 - 4 min, reading ~30s

Writing with an ESXi to NFS-Share is 10x slower!!! And only with ESXi. A Linux-box have not this poor issue.

With an UFS-Store i have not this poor performance. Writing is a litte bit slower then reading.

I tried to disable ZIL, but the command is not supported in FreeNas 8.0.4.

zfs set sync=disable storage
cannot set property for 'storage': invalid property 'sync'

Any other ideas?
 
Last edited:
chaosranger, that is exactly my problem.

yesterday i said FUCK IT and opened up NFS shares on a Windows Server 2008 R2 with hardware raid and called it a day.

As far as ZIL you cant do it with the command. Towards the end of this thread we found that you had to edit the loader.conf file (?). I did and that still didnt help.

My VM's were getting <1MB/sec write speeds with their virtual hard drives on the NFS share with FreeNAS 8.0.4. I also had a FreeNAS 8.0.1 that didnt have this problem.
 
Last edited:
I also tried to edit the loader.conf, but he system tells me this file is only read-only. This is sh**. ESXi and NFS are a well used combination. But why not does this work with FreeNAS. I don't will try an other free NAS OS. I like FreeNAS.

Is the performance with the Windows NFS-Share better?
 
HAH!!! I since you're new to this board you probably dont have access to GenMay. But I had a *NIX rant thread about how hard it was to do some things. Edit that file being one of them. They helped me through it. Even root doesnt have access to that file. Let me search and post the solution to editing it. Who knows, maybe i did it wrong.
 
Yea I said it had no change. But I'm by no means a *nix master. I might have done it wrong. Either way I got fed up with it and decided I didnt want to have to support something I wasnt comfortable with.

Yes my Windows based NFS share get 85MB/sec write speed. Right around the point the network starts to limit it.
 
You must log in or register to reply here.
Back
Top