ESXi NFS mount I/O error

[webbrowser]

I'm scratching my head as to why ESXi refuses to mount the nfs datastore with an I/O error. Here are the details.

From the ESXi ssh shell,
- "vmkping 192.168.9.249" (the nfs server) succeeds.
- "nc -z 192.168.9.249 2049" succeeds.
- "esxcfg-nas -a -o 192.168.9.249 -s /z/vmdatastore nas" gives:

Connecting to NAS volume: nas
Unable to connect to NAS volume nas: Sysinfo error on operation returned status : I/O error. Please see the VMkernel log for detailed error information

- /var/log/vmkernel.log says:

2013-07-28T12:13:30.769Z cpu6:30335)NFS: 157: Command: (mount) Server: (192.168.9.249) IP: (192.168.9.249) Path: (/z/vmdatastore) Label: (nas) Options: (None)
2013-07-28T12:13:30.769Z cpu6:30335)StorageApdHandler: 692: APD Handle 3ab89597-5160e1d0 Created with lock[StorageApd0x410016]
2013-07-28T12:13:30.770Z cpu3:30335)StorageApdHandler: 739: Freeing APD Handle [3ab89597-5160e1d0]
2013-07-28T12:13:30.770Z cpu3:30335)StorageApdHandler: 802: APD Handle freed!
2013-07-28T12:13:30.770Z cpu3:30335)NFS: 218: NFS mount 192.168.9.249:/z/vmdatastore status: I/O error

The NFS server is Debian 7 Wheezy, running nfs-kernel-server.
- /etc/exports is
/z/vmdatastore 192.168.9.0/24(rw,no_root_squash,no_subtree_check)
- /z/vmdatastore is ZFS formatted SSD with 777 permissions.

# ls -al /z/vmdatastore/
total 13
drwxrwxrwx 2 root root 2 Jul 28 19:37 .
drwxr-xr-x 7 root root 4096 Jul 28 11:21 ..

- /var/log/syslog doesn't show any error messages at all when I attempt to do the esxcfg-nas add.
- nfs mount on the NFS server localhost, that works.
- If I do an nfs mount on the NFS server localhost but with the wrong path (eg. /vmdatastore instead of /z/vmdatastore) yields an error message in syslog:
rpc.mountd[3340]: refused mount request from 192.168.9.249 for /vmdatastore (/): not exported
- esxcfg-nas add with the same wrong path does not yield any error message in syslog. So something is screwing up even before it gets to this stage. But the ping and nc connectivity tests succeeded


More details if needed:

ESXi server:
- vswitch0: (internet connected), vmnic0
vmkernel: 192.168.2.250/24, gateway 192.168.2.254
- vswitch1: (storage network), not connected to physical nic
vmkernel: 192.168.9.250/24

# esxcfg-route -l
VMkernel Routes:
Network Netmask Gateway Interface
192.168.2.0 255.255.255.0 Local Subnet vmk0
192.168.9.0 255.255.255.0 Local Subnet vmk1
default 0.0.0.0 192.168.2.254 vmk0

 

[webbrowser]

Can't seem to delete my post....

Found my problem. Had left an entry in /etc/hosts.deny from before i had decided to just use iptables to firewall it off.

 

[m1abram]

Generally not very helpful to delete a post like this just because you found the answer. Best to post saying how you solved it so if someone else has a similiar issue they can get an answer.

Pet peeve of mine when people ask for help, but find the solution elsewhere but do not post back to there and give back that info.

And yes you did answer it which is great. But your first reaction was to delete it.

 

[Chandler]

Generally not very helpful to delete a post like this just because you found the answer. Best to post saying how you solved it so if someone else has a similiar issue they can get an answer.

Pet peeve of mine when people ask for help, but find the solution elsewhere but do not post back to there and give back that info.

And yes you did answer it which is great. But your first reaction was to delete it.

++1 - Like to have things come up in search results and be indexed by Google xD

 

[unholythree]

++1 - Like to have things come up in search results and be indexed by Google xD

Double true, of course the absolute worst is when you're searching and all the promising treads are just filled with suggestions that the OP search.

But now I've taken this way off topic. Glad you figured it out OP!

 

[webbrowser]

Guess what. Almost exactly TWO years later, I got the same error message (NFS Mount I/O Error in ESXi) while upgrading my NFS server from wheezy to jessie.

Did some searching, this was the second hit in google. Wahey, my own post, so I tried it out... and nope. of course, I didn't modify tcpwrappers and the debian upgrade certainly didn't do it for me.

So now I can add another possible cause of I/O errors. After the jessie upgrade somehow the nfs-kernel-server init script had --no-nfs-version 3. ESXi 5 only supports NFSv3, so I had to remove that.

 

[xxx31fr]

Hi,

Can you explain me how you fix your problem, because i'd exactly the same issue.

Thanks.