OpenIndiana horrible SMB performance

S

spankit

Limp Gawd
Joined
Oct 18, 2010
Messages
262
I have been working on my server for two days now and really can't seem to figure out why I am getting such terrible performance over samba. After doing some googling I found that the e1000g driver has been problematic for many people in OpenSolaris. In my searches I found that for some people enabling jumbo frames had fixed the issue. I went ahead and followed this guide (link) and changed the MaxFrameSize to 2 all across the board and then when that did not make a difference I tried with 3 with the same result. I can't seem to be able to get anything more than 3MBps over samba from my Windows 7 Pro machine. I have also tried using the onboard SATA ports as well as the ports on the controller card both with the same result. Drives have been configured with stripping, mirror, raidz1 and raidz2. Also note, I used the same rig in FreeBSD (ZFSGuru) and was able to hit well above 70MBps.

Mobo: Intel DG33BU
Proc: Intel Core2 Duo 2.66ghz
RAM: 8GB DDR2
HBA: LSI 9201-16i
HDD: 4x Hitachi 5K3000 3TB, 4x Seagate ST32000542AS 2TB
Chipset: Intel® G33 Express
NIC: Intel 82566DC
OS: OpenIndiana 151 w/ napp-it 0.500s (also tried 148)

Any help would be great.

-Thanks
 
SMB or CIFS is the name of the protocol, The ZFS property behind is smbshare.
Samba is an alternative widely used SMB-server, also available on Solaris

(Solaris Kernel based SMB/CIFS server is faster, integrated in ZFS and has better ACL support but
overall less features)

About your problem:
check pool performance (menu pool benchmark)
check net-performance (use iperf, installed on Solaris from CLI, available on Windows)
opt: use a cross-over cable between Solaris and Windows (no switch needed)

use base network settings without modifications
try another NIC
 
Last edited:
Thanks for the replies. Here are the results of iperf so far. Using a crossover did not make a difference in the results and I had to use the default settings in OpenSolaris as it was taking far too long to complete.

## OpenIndiana 151, On Switch, MaxFrameSize=2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2;
Code: 
c:\>iperf -c 192.168.1.75
------------------------------------------------------------
Client connecting to 192.168.1.75, TCP port 5001
TCP window size: 8.00 KByte (default)
------------------------------------------------------------
[156] local 192.168.1.3 port 49365 connected with 192.168.1.75 port 5001
[ ID] Interval       Transfer     Bandwidth
[156]  0.0-10.0 sec  16.7 MBytes  14.0 Mbits/sec

## OpenIndiana 151, On Switch, (default) MaxFrameSize=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0;
Code: 
c:\>iperf -c 192.168.1.75
------------------------------------------------------------
Client connecting to 192.168.1.75, TCP port 5001
TCP window size: 8.00 KByte (default)
------------------------------------------------------------
[156] local 192.168.1.3 port 49404 connected with 192.168.1.75 port 5001
[ ID] Interval       Transfer     Bandwidth
[156]  0.0-10.0 sec  14.6 MBytes  12.2 Mbits/sec

## FreeBSD 8.2, On Switch
Code: 
c:\>iperf -c 192.168.1.73 -n 10000000000
------------------------------------------------------------
Client connecting to 192.168.1.73, TCP port 5001
TCP window size: 8.00 KByte (default)
------------------------------------------------------------
[156] local 192.168.1.3 port 49626 connected with 192.168.1.73 port 5001
[ ID] Interval       Transfer     Bandwidth
[156]  0.0-37.1 sec  1.31 GBytes   304 Mbits/sec


EDIT: Forgot to mention. I also tried with a different NIC in the server. It is also an Intel NIC and seemed to use the same driver (e1000g) so that could very well be the issue.
 
Last edited:
8 KByte
256MBytes 217 Mbits/sec
zpool wd green raidz2 8 disks
server :z68 realtek
client: windows 7 realtek H57
I use maxframesize=33333333333......
and I was running something when iperf

your zpool have problem?
 
Last edited:
Judging by the iperf results, chances are really good that its an issue with the chipset or something. So I stopped messing with cfis or any zpool configurations.
 
spankit said:
(...) 12.2 Mbits/sec(...)
Click to expand...

12.2 / 8 = 1.5 MBytes / second.

As was pointed out, you have a networking issue, not a CIFS/SMB issue.

Also realize if you are trying to use jumbo frames you need to support them on the client, switch, and server. But really they are a bit of a red herring now - this isn't a matter of incremental performance improvements by jumbo frames, but some sort of underlying networking issue.
 
I'm in the process of installing Solaris 11 Express. Well see if that makes any difference.
 
Same result in Solaris 11 Express. Can't get anything higher than 6MBps with iperf.
 
Changed motherboard and the issue went away. Must have been something with the chipset that Solaris based OS's didn't like.
 
spankit said:
Changed motherboard and the issue went away. Must have been something with the chipset that Solaris based OS's didn't like.
Click to expand...

Solaris based systems (and ESXi systems) are known to work only on
a limited set of hardware without problem and with a lot of fun.
 
spankit, which motherboard did you use this time?

BTW I am sort of having similar issues with getting good speeds using Dell XR997 10gbe NICs (point-to-point). I can get 9.7 gigabits/sec using iperf. Server command line: "iperf -s -w 512K -l 56K", client command line: "iperf -c 11.0.0.1 -w 512K -l 56K -i 10 -t 60".

But with Windows 7 CIFS I can't get more than 185 megabytes/sec read and 270 megabytes/sec writes running CrystalDisk Mark v3. Maybe I'm asking too much? Doesn't matter if I have jumbo frames on or off.

I have no idea how others can get 500+ MB/sec read/writes...maybe I can blame the hardware pickiness of OpenIndiana? Using an Asus P5Q-E motherboard, Intel Q9300 CPU at 2.5 GHz (stock), 8 GB of RAM.

I haven't tried iSCSI yet.
 
Even though I'm getting approx 300MB/s read and write locally on the box using the DD bench utility. I can only hit about 50-70MB/s using CFIS to my Win7 Pro box. I have no idea what model of motherboard I am using as it was just one I had kicking around. I'm saving my money to buy a proper server board and more memory. If your hitting 9.7gigabit/s using iperf than chances are good that your issue is not a network issue.

What kinda numbers are you getting in your benchmarks locally?
 
10x Samsung F4 2TBs in RAIDZ2
8x 1TB drives of various makes (samsung, WD blacks) in RAIDZ2

I will experiment with iSCSI soon by destroying the 8x 1 TB array.
I may also make it into a ZFS RAID10 (RAID0 4x 1TB mirrored). Hopefully I'll have time tonight.

Using 2x LSI 1068E SAS controllers in IT mode and a couple of hdds are on the ICH9 controller.
 
Code: 
  pool: stripe1
 state: ONLINE
 scan: none requested
config:

	NAME        STATE     READ WRITE CKSUM
	stripe1     ONLINE       0     0     0
	  c10t2d0   ONLINE       0     0     0
	  c10t5d0   ONLINE       0     0     0
	  c24t20d0  ONLINE       0     0     0
	  c24t21d0  ONLINE       0     0     0
	  c24t22d0  ONLINE       0     0     0
	  c24t4d0   ONLINE       0     0     0
	  c25t0d0   ONLINE       0     0     0
	  c25t1d0   ONLINE       0     0     0

errors: No known data errors
bzgz0.png


In OpenIndiana 148, Bonnie reports Seq-Write of 315 MBytes/sec, Seq-Read of 783 MBytes/sec.

(sequential writes can actually hit 485 MBytes/sec if I run mkfile).
 
You must log in or register to reply here.
Back
Top