• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

iSCSI howto?

TeeJayHoward

TeeJayHoward

Limpness Supreme
2FA
Joined
Feb 8, 2005
Messages
13,332
I've got two ESXi boxes with a NIC dedicated to iSCSI. I've got a Windows Server 2012 box with two NICs dedicated to iSCSI (teamed, but I can undo that if I need to). I set up an iSCSI target on the Server 2012 rig, and I can access it from the two ESXi machines.

Now, I hear it's a good idea to separate your iSCSI traffic from your regular traffic. Makes sense. But how? Right now, everything's 192.168.0.x. If I put the NICs on the Server 2012 box that I plan on using for iSCSI on the 192.168.1.x subnet, and put the ESXi NICs on the same subnet, ESXi can't find the target. (And, to be honest, I wasn't sure what to put for the gateway. There is no 192.168.1.1, so I set it to 192.168.1.12, which didn't work. It's been years since I've done even basic networking stuffs!) I can't find a way to force Windows to use certain NICs for iSCSI traffic. And then there's the whole VLAN bit. My v1810-24G supports VLANs. I put the two ESXi ports and the two Server2012 ports on the same VLAN. Didn't work.

What settings need to be set to get this working? I'm really wanting to learn how to use Windows Server as an iSCSI target, but the more I screw around with it, the more tempted I am to run a Hyper-V *NIX VM and just set it up that way. (Of course, I don't know if it's possible to pass through a NIC on Hyper-V without VT-x/VT-d... Much less a team of them.)

TL;DR: How do I iSCSI on a different subnet/VLAN?
 
I'm making up numbers so you get the idea:

Windows Server 2012

NIC1 - iSCSI network 192.168.2.10 on VLAN2 on the switch. No default gateway necessary since they are on the same layer 2 network.

ESXi Server

NIC1 - iSCSI network 192.168.2.11 on VLAN2 on the switch. Default gateway doesn't matter on the ESXi host because again, this NIC is on the same layer 2 network.

Connect the ESXi host to 192.168.2.10. Create another VLAN, VLAN1 or whatever, and put the management interface on that so you can get to the host on a different network. It's not multipathed but it will work. Windows will use whatever NIC the initiator connects over.

It's that easy.
 
QHalo said:
I'm making up numbers so you get the idea:

Windows Server 2012

NIC1 - iSCSI network 192.168.2.10 on VLAN2 on the switch. No default gateway necessary since they are on the same layer 2 network.

ESXi Server

NIC1 - iSCSI network 192.168.2.11 on VLAN2 on the switch. Default gateway doesn't matter on the ESXi host because again, this NIC is on the same layer 2 network.

Connect the ESXi host to 192.168.2.10. Create another VLAN, VLAN1 or whatever, and put the management interface on that so you can get to the host on a different network. It's not multipathed but it will work. Windows will use whatever NIC the initiator connects over.

It's that easy.
Click to expand...
I figured it should be that easy. That's basically how I had it set up, at least as I understand it. It didn't work. Here's what I had:
2012-NIC-1: 192.168.0.10 (no VLAN, Trunk 1)
2012-NIC-2: 192.168.0.10 (no VLAN, Trunk 1)
2012-NIC-3: 192.168.1.12 (VLAN 2, Trunk 2)
2012-NIC-4: 192.168.1.12 (VLAN 2, Trunk 2)
ESXi-1-NIC-1: 192.168.0.5 (no VLAN)
ESXi-1-NIC-2: 192.168.1.7 (VLAN 2)
ESXi-2-NIC-1: 192.168.0.6 (no VLAN)
ESXi-2-NIC-2: 192.168.1.8 (VLAN 2)

Management goes through NIC-1 on both ESXi boxes. NIC-2 on both ESXi boxes is the iSCSI software adapter I set up (which works on the 192.168.0.x network). The 2012 box accesses the internet and the ESXi hosts through the NIC-1/NIC-2 LACP'd team. I intended to use the other NIC team for iSCSI traffic, but can't find a way to force the box to only use those NICs. I gave up and assumed that Windows would just figure it out.
 
Do you have the iSCSI ports on the ESXi hosts bound to the vmkernels on VLAN2?

iscsi_port_binding.png
 
Got it working. Tried it with Starwind, and everything worked fine. So it was definitely a setting somewhere. Turns out, I can't use IQN for Windows Server 2012. I have to use IP Address.

Now I've got a new issue. Why is my iSCSI share so slow?
Code: 
Solaris VM:
time dd if=/dev/zero of=/home/tim/6gb.file bs=1024 count=6291456
real   1m42.838s
user   0m1.941s
sys    0m38.924s

ESXi Host:
time dd if=/dev/zero of=/vmfs/volumes/51249d07-3c27686a-7997-002590a51b55/10gb.file bs=1024 count=10485760
real    4m 9.34s
user    0m 38.98s
sys     0m 0.00s

This is with a 200GB iSCSI Virtual Disk on a Samsung 840 Pro 256GB SSD hosted via Window's built-in iSCSI dohickey. Gonna try switching back to Starwind now. I keep hearing about battery-backup and write-caching. Hopefully Starwind's write caching is a suitable substitute!

edit - Starwind results:
Code: 
Solaris VM:
time dd if=/dev/zero of=/home/tim/6gb.file bs=1024 count=6291456
real   1m42.684s
user   0m1.944s
sys    0m39.445s

ESXi Host:
time dd if=/dev/zero of=/vmfs/volumes/51243b7b-da55da20-bb97-0025907384df/10gb.file bs=1024 count=10485760
real    4m 4.97s
user    0m 38.69s
sys     0m 0.00s

Ugh. Here's something I won't try again:
Code: 
ESXi Host:
time dd if=/vmfs/volumes/51243b7b-da55da20-bb97-0025907384df/10gb.file of=/dev/null
real    14m 11.93s
user    2m 38.70s
sys     0m 0.00s

Okay, I changed the cache type from Write-through to write-back and enabled Jumbo Frames on both NICs and the switch. Looks like it helped a little, but I'm still only at 45.5MB/s.
Code: 
ESXi Host:
time dd if=/dev/zero of=/vmfs/volumes/51243b7b-da55da20-bb97-0025907384df/10gb.file bs=1024 count=10485760
real    3m 45.88s
user    0m 31.50s
sys     0m 0.00s

Another edit:

Okay, what the snark?

esxtop, then D shows a decent transfer speed: 91.14MB/s
iSCSI_speed_3_zps5c013191.png


Task Manager shows a decent transfer speed: 800Mbps, or about 100MB/s
iSCSI_speed_1_zps9c847658.png


Resource Monitor shows a decent transfer speed: ~95MB/s
iSCSI_speed_2_zpsbfc63094.png


So is my math just off? 3 minutes, 45 seconds = 225 seconds. 10GB=10240MB. 10240MB/225s = 45.5MB/s... What the hell? I'm 20% below line speed, but 60% below write speed? Does this make any sense to anyone?

And copying my VMDKs back to the datastore results in an unusual speed as well... 68MB/s from StarWind, and 34MB/s from VMWare? ~100MB/s in utilization, but only 34MB/s (or is it 68MB/s?) in actual transfer speed? How the hell does iSCSI work?
iSCSI_speed_4_zps344ab1fb.png
 
Last edited:
What if you get rid of the trunking for a moment and just use the adapters as 1 active and the other in standby? What kind of speeds to you get then? What if you then take both up links and set them to round Robin without trunking them?
 
peanuthead said:
What if you get rid of the trunking for a moment and just use the adapters as 1 active and the other in standby? What kind of speeds to you get then? What if you then take both up links and set them to round Robin without trunking them?
Click to expand...
Damn, I didn't read this until I got to work this morning. I'll give it a shot when I get home. I also want to bench it with Windows iSCSI and Jumbo Frames, see if that helps any. How do you set the adapters for round robin or failover on Windows 2012?
 
TeeJayHoward said:
Damn, I didn't read this until I got to work this morning. I'll give it a shot when I get home. I also want to bench it with Windows iSCSI and Jumbo Frames, see if that helps any. How do you set the adapters for round robin or failover on Windows 2012?
Click to expand...

Okay. I am referring to setting the vnic/vmk port to round robin in the port group in vmWare.
 
You must log in or register to reply here.
Back
Top