"Failed to log into NFC server"

[AMD_Gamer]

I noticed my Veeam replication jobs where failing and while investigating the host the replicas are being sent to, i noticed that if I try to download something from the datastore I can't do it ans get a "Failed to log into NFC server" error.

I check permissions, the firewall, services, etc and rebooted a few times and i cant get this fixed. I can create a virtual machine ok but it seems like upload/download from the client or veeam from a remote host do not work.

What could cause this?

 

[lopoetve]

What's /var/log/hostd.log (or /var/log/vmware/hostd.log if pre 5.0) say?

 

[AMD_Gamer]

What's /var/log/hostd.log (or /var/log/vmware/hostd.log if pre 5.0) say?

This is what comes up after I try to download a file from the datastore

2011-12-28T14:24:52.767Z [2C755B90 error 'SoapAdapter.HTTPService'] HTTP Transaction failed on stream TCP(local=127.0.0.1:0, peer=127.0.0.1:56159) with error N7Vmacore15SystemExceptionE(Connection reset by peer)
2011-12-28T14:24:56.440Z [2C755B90 verbose 'SoapAdapter'] Responded to service state request
2011-12-28T14:25:05.418Z [2C714B90 verbose 'HttpSvc.HTTPService'] User agent is 'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)'
2011-12-28T14:25:10.061Z [2C714B90 verbose 'HttpSvc.HTTPService'] HTTP Response: Auto-completing at 8628356/8628356 bytes
2011-12-28T14:25:26.442Z [2C381B90 verbose 'SoapAdapter'] Responded to service state request
2011-12-28T14:25:33.967Z [2C444B90 verbose 'HttpSvc.HTTPService'] User agent is 'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)'
2011-12-28T14:25:37.867Z [2C444B90 verbose 'HttpSvc.HTTPService'] HTTP Response: Auto-completing at 8628724/8628724 bytes
2011-12-28T14:25:52.550Z [2C689B90 verbose 'HttpSvc.HTTPService'] User agent is 'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)'
2011-12-28T14:25:55.421Z [2C755B90 verbose 'Cimsvc'] Ticket issued for CIMOM version 1.0, user root
2011-12-28T14:25:56.443Z [2C403B90 verbose 'SoapAdapter'] Responded to service state request
2011-12-28T14:25:57.844Z [2C689B90 verbose 'HttpSvc.HTTPService'] HTTP Response: Auto-completing at 8628999/8628999 bytes
2011-12-28T14:26:26.444Z [2C140B90 verbose 'SoapAdapter'] Responded to service state request
2011-12-28T14:26:45.770Z [2C381B90 verbose 'Default'] Timed out reading between HTTP requests. : Read timeout after approximately 50000ms. Closing stream TCP(local=127.0.0.1:8309, peer=127.0.0.1:54377)
2011-12-28T14:26:56.445Z [2C6D3B90 verbose 'SoapAdapter'] Responded to service state request
2011-12-28T14:27:17.315Z [2C714B90 verbose 'DvsManager'] PersistAllDvsInfo called
2011-12-28T14:27:25.784Z [2C6D3B90 verbose 'Cimsvc'] Ticket issued for CIMOM version 1.0, user root
2011-12-28T14:27:26.446Z [2C140B90 verbose 'SoapAdapter'] Responded to service state request
2011-12-28T14:27:42.742Z [2C403B90 verbose 'Proxysvc Req01383'] New proxy client SSL(TCP(local=192.168.7.2:443, peer=192.168.4.11:59311))
2011-12-28T14:27:42.797Z [2C6D3B90 verbose 'HttpSvc.HTTPService'] User agent is 'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)'
2011-12-28T14:27:47.956Z [2C6D3B90 verbose 'HttpSvc.HTTPService'] HTTP Response: Auto-completing at 8630268/8630268 bytes
2011-12-28T14:27:56.447Z [2C444B90 verbose 'SoapAdapter'] Responded to service state request

 

[AMD_Gamer]

I see this grouping a lot also

2011-12-26T14:54:05.939Z [2C403B90 error 'SoapAdapter.HTTPService'] HTTP Transaction failed on stream TCP(local=127.0.0.1:0, peer=127.0.0.1:55008) with error N7Vmacore15SystemExceptionE(Connection reset by peer)
2011-12-26T14:54:19.139Z [2C6D3B90 verbose 'SoapAdapter'] Responded to service state request
2011-12-26T14:54:49.140Z [2C6D3B90 verbose 'SoapAdapter'] Responded to service state request
2011-12-26T14:55:19.142Z [2C403B90 verbose 'SoapAdapter'] Responded to service state request
2011-12-26T14:55:24.350Z [FFC8EA90 verbose 'Cimsvc'] Ticket issued for CIMOM version 1.0, user root
2011-12-26T14:55:49.143Z [FFC8EA90 verbose 'SoapAdapter'] Responded to service state request
2011-12-26T14:56:19.144Z [2C689B90 verbose 'SoapAdapter'] Responded to service state request
2011-12-26T14:56:49.145Z [2C6D3B90 verbose 'SoapAdapter'] Responded to service state request
2011-12-26T14:56:54.834Z [2C381B90 verbose 'Cimsvc'] Ticket issued for CIMOM version 1.0, user root
2011-12-26T14:57:16.695Z [2C689B90 verbose 'DvsManager'] PersistAllDvsInfo called
2011-12-26T14:57:19.146Z [FFC8EA90 verbose 'SoapAdapter'] Responded to service state request
2011-12-26T14:57:49.147Z [2C689B90 verbose 'SoapAdapter'] Responded to service state request
2011-12-26T14:58:04.927Z [2C381B90 verbose 'DvsTracker'] FetchSwitches: added 0 items
2011-12-26T14:58:04.927Z [2C381B90 verbose 'DvsTracker'] FetchDVPortgroups: added 0 items
2011-12-26T14:58:04.927Z [2C381B90 info 'DvsTracker'] FetchUplinkDVPortgroups: added 0 items
2011-12-26T14:58:19.148Z [2C6D3B90 verbose 'SoapAdapter'] Responded to service state request
2011-12-26T14:58:25.207Z [2C444B90 verbose 'Cimsvc'] Ticket issued for CIMOM version 1.0, user root
2011-12-26T14:58:49.150Z [2C444B90 verbose 'SoapAdapter'] Responded to service state request
2011-12-26T14:59:06.548Z [2C444B90 verbose 'Proxysvc Req00553'] New proxy client TCP(local=127.0.0.1:80, peer=127.0.0.1:50705)
2011-12-26T14:59:06.550Z [2C140B90 info 'Vmomi'] Activation [N5Vmomi10ActivationE:0x9c13188] : Invoke done [waitForUpdates] on [vmodl.query.PropertyCollector:ha-property-collector]
2011-12-26T14:59:06.550Z [2C140B90 verbose 'Vmomi'] Arg version:
--> "2"
2011-12-26T14:59:06.550Z [2C140B90 info 'Vmomi'] Throw vmodl.fault.RequestCanceled
2011-12-26T14:59:06.550Z [2C140B90 info 'Vmomi'] Result:
--> (vmodl.fault.RequestCanceled) {
--> dynamicType = <unset>,
--> faultCause = (vmodl.MethodFault) null,
--> msg = "",
--> }

 

[AMD_Gamer]

Actually I think those are both the same but one had the /host/hostd.log requests mixed in.

That looks like the culprit, whatever it is?

 

[lopoetve]

what are you copying to?

 

[AMD_Gamer]

what are you copying to?

My veeam replications are failing. And when i browse the datastore and try to upload or download something i get this error.

The veeam errors match this problem.

 

[AMD_Gamer]

So any idea on this?

 

[lopoetve]

sadly, I don't work with NFC much.

What's the network infrastructure - what's kind of hardware, firewalls, etc?

 

[rykker]

Not sure if this will work for you, but I had the same error when trying to upload something to my datastore, for me, it turned out to be a DNS issue so I manually added the hostnames of my esx servers to the hosts file on my vcenter server