Poor Performance under raid 5

[golemcito]

Hi guys

I need some help. At this moment i have a 6 disks (Samsung SpinPoint F4 2Tb) RAID5 under mdadm in ubuntu.

I have made some tests and I get this with hdparm:

/dev/md0:
Timing cached reads: 1848 MB in 2.00 seconds = 924.42 MB/sec
Timing buffered disk reads: 320 MB in 3.02 seconds = 106.00 MB/sec

/dev/sdb:
Timing cached reads: 1758 MB in 2.00 seconds = 878.70 MB/sec
Timing buffered disk reads: 360 MB in 3.02 seconds = 119.37 MB/sec

/dev/sdc:
Timing cached reads: 1782 MB in 2.00 seconds = 890.83 MB/sec
Timing buffered disk reads: 362 MB in 3.01 seconds = 120.15 MB/sec

/dev/sdd:
Timing cached reads: 1800 MB in 2.00 seconds = 899.79 MB/sec
Timing buffered disk reads: 360 MB in 3.01 seconds = 119.77 MB/sec

/dev/sde:
Timing cached reads: 1800 MB in 2.00 seconds = 899.91 MB/sec
Timing buffered disk reads: 356 MB in 3.01 seconds = 118.13 MB/sec

/dev/sdf:
Timing cached reads: 1796 MB in 2.00 seconds = 898.18 MB/sec
Timing buffered disk reads: 366 MB in 3.01 seconds = 121.79 MB/sec

/dev/sdg:
Timing cached reads: 1474 MB in 2.00 seconds = 736.53 MB/sec
Timing buffered disk reads: 394 MB in 3.01 seconds = 130.81 MB/sec

But when i use dd i get those:

Writing
dd if=/dev/zero of=/mnt/raid/zerofile.000 bs=2M count=10000
10000+0 registros de entrada
10000+0 registros de salida
20971520000 bytes (21 GB) copiados, 189,424 s, 111 MB/s

Reading
dd if=/mnt/raid/zerofile.000 of=/dev/null bs=2M
10000+0 registros de entrada
10000+0 registros de salida
20971520000 bytes (21 GB) copiados, 190,884 s, 110 MB/s

Any clue... what is happening? should i get more speed reading?

Thanx in advance

 

[drescherjm]

should i get more speed reading?

Your numbers should be 3 to 4 times as high.

Is the array rebuilding?

cat /proc/mdstat

 

[golemcito]

Your numbers should be 3 to 4 times as high.

Is the array rebuilding?

cat /proc/mdstat

No. It is fine

Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sde[4] sdd[2] sdc[1] sdb[0] sdg[6] sdf[5]
9767566720 blocks super 1.2 level 5, 128k chunk, algorithm 2 [6/6] [UUUUUU]

unused devices: <none>

 

[Red Falcon]

Do you have encryption enabled? Also, the 128K chunk might hold it back a bit.

 

[golemcito]

No, encryption is disabled...

 

[drescherjm]

Can you try a dd read test from all 6 individual drives at the same time?

 

[golemcito]

I think i cant do that because they are working and i have 6 Tb stored...

 

[drescherjm]

You should be able to do a read only

dd if=/dev/sdb of=/dev/null bs=8M count=1000 &
dd if=/dev/sdc of=/dev/null bs=8M count=1000 &
dd if=/dev/sdd of=/dev/null bs=8M count=1000 &
dd if=/dev/sde of=/dev/null bs=8M count=1000 &
dd if=/dev/sdf of=/dev/null bs=8M count=1000 &

without harming the raid array at all.

 

[golemcito]

You should be able to do a read only

dd if=/dev/sdb of=/dev/null bs=8M count=1000 &
dd if=/dev/sdc of=/dev/null bs=8M count=1000 &
dd if=/dev/sdd of=/dev/null bs=8M count=1000 &
dd if=/dev/sde of=/dev/null bs=8M count=1000 &
dd if=/dev/sdf of=/dev/null bs=8M count=1000 &

without harming the raid array at all.

Here we go:

dd if=/dev/sdb of=/dev/null bs=8M count=1000
1000+0 registros de entrada
1000+0 registros de salida
8388608000 bytes (8,4 GB) copiados, 142,411 s, 58,9 MB/s

dd if=/dev/sdc of=/dev/null bs=8M count=1000
1000+0 registros de entrada
1000+0 registros de salida
8388608000 bytes (8,4 GB) copiados, 143,444 s, 58,5 MB/s

dd if=/dev/sdd of=/dev/null bs=8M count=1000
1000+0 registros de entrada
1000+0 registros de salida
8388608000 bytes (8,4 GB) copiados, 145,961 s, 57,5 MB/s

dd if=/dev/sde of=/dev/null bs=8M count=1000
1000+0 registros de entrada
1000+0 registros de salida
8388608000 bytes (8,4 GB) copiados, 145,007 s, 57,8 MB/s

dd if=/dev/sdf of=/dev/null bs=8M count=1000
1000+0 registros de entrada
1000+0 registros de salida
8388608000 bytes (8,4 GB) copiados, 100,736 s, 83,3 MB/s

dd if=/dev/sdg of=/dev/null bs=8M count=1000
1000+0 registros de entrada
1000+0 registros de salida
8388608000 bytes (8,4 GB) copiados, 95,336 s, 88,0 MB/s

sdf & sdg are plugged in another sata controller pci-e 1x