Samba performance help

D

drizzt81

[H]F Junkie
Joined
Jan 21, 2004
Messages
12,361
Hi lads,

so I have finally managed to get my Samba server up, integrated into my AD...

I transferred a bunch of files from my Win2K3 server to the Samba server. It went around 25 MiB/s. Not too shabby for a non-jumbo framed GbE transfer, I think. The source -in this case- was a single 500GB Seagate 7200.10, the destination a degraded (ie. two devices + missing) Linux Software RAID-5 array, composed of a WD5000YS and a Seagate 7200.10.

Now I am trying to transfer data from my WinXP machine to the Linux server (RAID-5 is now complete and online, two 7200.10s and the WD5000YS), but cannot get the network utilization above 5% (~5 MiB/s). I have activated jumbo frames to no avail.

This is what bonnie++ spits out:
Code: 
Version  1.03       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
deb-serv         4G 43782  79 51149  20 25930   9 48950  83 93583  21 243.2   1
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
deb-serv,4G,43782,79,51149,20,25930,9,48950,83,93583,21,243.2,1,16,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++
If I interpret that correctly, I should be able to write at up to 50 MB/s, and rewrite at up to 25MB/s, neither of which appear to be limiting. The source HDD is a WD Raptor, which ought to be able to perform better as well.

When I copy from the samba server (400 MB file), I get ~20% utilization in the task manager, when I copy to it, I get ~8% on average. Is this to be expected? Can I improve upon this?
 
try adding this to the smb.conf file (or to the relevant section in SWAT)

socket options = TCP_NODELAY IPTOS_LOWDELAY SO_SNDBUF=65535 SO_RCVBUF=6

VAST improvement in samba packet handling
 
You should actually tune those buffer sizes based on the bandwidth delay product and the average packet payload size. For my home gigabit network, that worked out to 21900 after some extensive testing and tweaking. 65535 is almost certainly too large for a low latency network and would degrade performance slightly. My Samba transfers average 45MB/sec (which is not disk I/O limited). I occasionally try to improve on this, but haven't been able to so far.

Jumbo frames, in my experience, don't help unless your hosts are CPU constrained.

edit: Do your network performance testing on tmpfs and avoid the disks completely.
 
So I moved a 400MB file to /dev/shm (tmpfs) and copied it over to/ from my workstation's iRAM. I have tried 8192, 21900 and 65535. The smallest value caused the performance (pulling form samba to windows) to drop. The other two were indistinguishable from the original behavior. Still, I can pull from much faster than push to.

I used iperf to test the network bandwidth available and came up with ~700-800 Mbit/second, so it appears that my network is working ok.

I read the samba user guide a bit and it mentioned that the socket options shouldn't make much of a difference with recent kernels (Debian with vanilla 2.6.20 here).

I am open for other suggestions.

ps.: yes I used /etc/init.d/samba restart after changing the parameters

edit:
I used the following share (now commented out)
Code: 
;[test]
;  path = /dev/shm
;  browsable = yes
;  writable = yes
;  guest ok = no
;  inherit acls = yes
;  inherit permissions = yes
;  force create mode = 0666
;  force directory mode = 0777
;  hide files = /DESKTOP.INI/desktop.ini/Desktop.ini
 
I can't think of anything that will greatly impact your performance... most of the stuff I've tried has been a wash for me. There are a few things that might be worth a try for you, though.

That's a custom 2.6.20 kernel compile you're using, right? Do you have the CIFS stuff turned on?

Samba has an asynchronous IO option that is experimental. I don't know if your samba binary was compiled with it, but it is worth checking out. It didn't do anything for my transfers with Windows boxes, but it gave the Linux boxes a (very) slight boost. In the end, I left it turned off, but maybe you would have better results. The smb.conf options are undocumented, despite what the changelog said about it:

Code: 
Asynchronous IO Support
=======================

Experimental support for async IO has been added to smbd for 
certain platforms.  To enable this new feature, Samba must be 
compiled to include the --with-aio-support configure option.
In addition, the "aio read size" and "aio write size" to non-zero
values.  See the smb.conf(5) man page for more details on these 
settings.
 
Bones said:
That's a custom 2.6.20 kernel compile you're using, right? Do you have the CIFS stuff turned on?
Click to expand...
Yes to custom kernel, how can I check for CIFS support?
edit:
Code: 
deb-serv:/usr/src/linux# cat /boot/config-2.6.20.1-gwksmp |grep CIFS
CONFIG_CIFS=m
# CONFIG_CIFS_STATS is not set
# CONFIG_CIFS_WEAK_PW_HASH is not set
# CONFIG_CIFS_XATTR is not set
# CONFIG_CIFS_DEBUG2 is not set
# CONFIG_CIFS_EXPERIMENTAL is not set
so I guess I am using CIFS?
Bones said:
It didn't do anything for my transfers with Windows boxes, but it gave the Linux boxes a (very) slight boost. In the end, I left it turned off, but maybe you would have better results.
Click to expand...
I am using the debian packaged version of samba. I doubt that it is compiled with that switch. Experimental sounds a bit too cutting edge for me at the moment.
 
CIFS is a network file system. In menuconfig, it's under File Systems --> Network File Systems.

You should be using CIFS to mount a remote Samba or Windows share on a Linux machine, and not the older SMB protocol.
 
Bones said:
CIFS is a network file system. In menuconfig, it's under File Systems --> Network File Systems.

You should be using CIFS to mount a remote Samba or Windows share on a Linux machine, and not the older SMB protocol.
Click to expand...

Right now I am mounting Samba shares on Windows not vice versa.
 
Ah, well I'm fresh out of ideas, then :eek:

Maybe I'll think of something else after sleeping on it.
 
thanks for everybody's help up to now :) goodnight.

edit:
I do not know what happened, but I rebooted my linux server this morning and now I am getting a utilization of up to 40%, i.e. 40MB/s.
*shrugs*
color me confused.

edit2: Apparently, it's a CPU utilization issue. Running F@H via VMWare appears to be limiting my outgoing speed via Samba.
 
Have you tried nicing VMWare? I'd try out nice 10 or 15. It is strange that one process would starve Samba for CPU that badly. I run several virtual machines on my server 24/7 and they don't impact Samba like that, even when eating every available CPU cycle for compiling.
 
Bones said:
Have you tried nicing VMWare? I'd try out nice 10 or 15. It is strange that one process would starve Samba for CPU that badly. I run several virtual machines on my server 24/7 and they don't impact Samba like that, even when eating every available CPU cycle for compiling.
Click to expand...

The VMWare server is running on windows. I guess I could lower its priority and see how it does.
 
You must log in or register to reply here.
Back
Top