Recommendation wanted >= 20 Port Switch with LACP (for NAS use) at LAN parties.

Discussion in 'Networking & Security' started by Archaea, May 23, 2018.

  1. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004
    LACP (802.3ad) would be ideal to make the most of my QNAP NAS with two network ports.


    I'm looking at options in the $100-$300 range. More ports is good, but at least 20 ports. This will be pulled out a couple times a year for LAN parties.


    Here are two I'm evaluating at current:

    Netgear GS724T-400NAS (newest version)
    https://www.amazon.com/dp/B00I5W5EG...olid=3BIRYPVDF1DFH&psc=0&ref_=lv_ov_lig_dp_it

    and

    Trendnet TEG-240WS (comes in 20, 24, 28 port forms)
    https://www.amazon.com/gp/product/B0001LS36Q/ref=ox_sc_act_title_1?smid=ATVPDKIKX0DER&th=1



    Both have 48Gbps switching, lifetime warranties, LACP. Any preference between the two, or recommendation otherwise?
     
  2. Vengance_01

    Vengance_01 [H]ardness Supreme

    Messages:
    5,555
    Joined:
    Dec 23, 2001
    I have used a Netgear switch that has a web mgt interface. Once you get used to it performance was fine. Not sure on the trendnet? Why not a used Cisco switch?
     
  3. Cmustang87

    Cmustang87 [H]ardness Supreme

    Messages:
    4,300
    Joined:
    Oct 4, 2007
  4. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004
    Thanks for the tips guys ---

    As to why not just an older Cisco switch off business lease

    Frankly, the mgmt. interface seems easier on the consumer level products than it does on something professional like Cisco.
    I learned Cisco IOS in college with my IT degree, 15 years ago, and haven't used it since as I'm not in a networking role.

    I was just trying to read up in the Cisco manual how to use LACP port trunking, it's a bit convoluted to me.

    LACP search term
    https://www.cisco.com/c/en/us/td/do...pter_0100101.html?bookSearch=true#con_1275688
    Setup
    https://www.cisco.com/c/en/us/td/do...f_2960p_cg_chapter_01101.html?bookSearch=true


    Looks like I set it to a vlan - (IE Vlan 1), do I then have to setup each of the other 46 ports to be on the same vlan or would those all default? How much optimizing would there need to be with an old professional switch like this Cisco 2960s you recommended --- vs. a new consumer switch. Seems like the new web GUIs - drag and drop is worlds easier for someone not in the day to day configuration routine?
     
  5. Cmustang87

    Cmustang87 [H]ardness Supreme

    Messages:
    4,300
    Joined:
    Oct 4, 2007
    The 2960S is still a relatively recent switch. It would totally be a set and forget appliance. It's probably more capable than any brand new COTS switch you'd buy like Netgear.

    If you're doing LACP, you're essentially just creating a virtual L2 interface that can carry tagged traffic (trunk). If you don't need VLANs then you can just keep the portchannel as default and it will just be untagged VLAN1.
     
  6. bds1904

    bds1904 Gawd

    Messages:
    999
    Joined:
    Aug 10, 2011
    Mikrotik Routerboard CSS326-24G-2S+RM

    LACP (hardware based, not software based like RouterOS used to be) and it’s got 2x 10gb SFP+ ports. Silent and low power. They run switchos which is pretty lightweight but it handles VLAN’s and LACP just fine.

    I’ve been running 3 of them for quite a while and they just work, including the LACP to my NAS. I picked them up for $120 a piece.
     
    Archaea likes this.
  7. Cmustang87

    Cmustang87 [H]ardness Supreme

    Messages:
    4,300
    Joined:
    Oct 4, 2007
    That's a killer switch for the price if you say it works as well you describe. I've had excellent luck with their routers (they just work, like you said), so I imagine the switches are just as reliable. I think this is a hard option to pass up, Archaea
     
    Archaea likes this.
  8. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004
    Seems just about what the doctor prescribed...

    That $120 price. What vendor di you find them at for that. Looks more like $150 range shipped that I'm seeing.
     
  9. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004
    Would you recommend the Mikrotik or the Cisco switch you recommended previously for $140. The cisco switch is still available.

    Any idea if there would be any discernable difference in performance?
     
  10. Vengance_01

    Vengance_01 [H]ardness Supreme

    Messages:
    5,555
    Joined:
    Dec 23, 2001
    For your needs no.. The only gotcha with Cisco is getting newer code for older gear that has been re-sold as its against the TOS.
     
    Cmustang87 likes this.
  11. Eickst

    Eickst [H]ard|Gawd

    Messages:
    1,769
    Joined:
    Aug 24, 2005
    The cisco switches are loud in comparison to the other brands. They don't expect them to be in homes
     
  12. Blue Fox

    Blue Fox [H]ardForum Junkie

    Messages:
    11,643
    Joined:
    Jun 9, 2004
    Why bother with Cisco when you can get an HP 1820-24G for about $150? Far easier to manage through the web interface vs CLI and they're fanless. They also have a lifetime warranty.
     
    Archaea, Ocellaris and MrGuvernment like this.
  13. MrGuvernment

    MrGuvernment Stay [H]ard

    Messages:
    19,810
    Joined:
    Aug 3, 2004
    Go buy a used HP Ent. switch, i got mine for $70 off kijiji in canada.
     
  14. Vengance_01

    Vengance_01 [H]ardness Supreme

    Messages:
    5,555
    Joined:
    Dec 23, 2001
    Hp procurves are nice
     
  15. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004
    I'm going to take advantage of the 20% off ebay coupon today.

    Thanks for all the help guys. Newegg had a decent price on the HP 1820-24G mentioned above at $165.

    With 20% off it comes to $131. $131 is the cheapest I've seen for a brand new 24 port switch with LACP (802.3ad), lifetime warranty, and fanless.

    HPE OfficeConnect 1820 24G Switch (J9980A)
    Specs: https://h20195.www2.hpe.com/v2/GetPDF.aspx/c04518995.pdf
    Source: newegg via ebay - https://www.ebay.com/itm/HPE-Office...599253?hash=item4415af8015:g:IQMAAOSwZC1Z9JLF
     
  16. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004
    The switch arrived.
    It came with firmware version 2.0.1 and the newest was 2.0.5. It's nearly impossible to find firmware for the switch with a websearch. I eventually found a link to a specific firmware search page in a HP forum user question - where the HP tech who responded admitted that HP in their revamp efforts made it seriously hard to find that stuff for cutomers and lamented the current site design. I updated to the newest firmware finally - and briefly tested it last night. Turned off SNMP, configured password, changed default IP address, etc. I don't see LACP in the manual, but it does speak to trunking - I guess I'll just futz with it until I figure it out. There doesn't seem to be much user community threads or youtube videos on the switch - surprisingly.

    I guess this 1820 is old and keeps getting updated? I think mine says it is version 23. It seems solid enough build quality and the power cord it came with must be 10 or 12 gauge wire - which is silly for a 19 watt max draw switch. It has both active and passive(backup) bios functions. I'll update more when I have more time to mess with it.
     
  17. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004
    So I finally got around to trying LACP and it's not working.

    Section 6-1 of this manual has the HPE 1820 switch config trunking section:
    https://support.hpe.com/hpsc/doc/public/display?docId=c04622710

    And I followed this guide from QNAP. And just chose layer 2. (not layer 2 +3 (MAC+IP, because as far as I can tell the Office Connect 1820 is only a layer 2 switch).
    https://www.qnap.com/en/how-to/tuto...o-increase-the-bandwidth-via-802-3ad-protocol

    Then I setup the tunk on the switch like this. But I don't know what drop down to use?
    What's highlighted is default.
    upload_2018-6-16_11-21-19.png

    I've tried both dynamic (disabled) and static (configurable)


    Now I can't access the QNAP at all. Any suggestions?

    upload_2018-6-16_11-23-32.png
     
    Last edited: Jun 16, 2018
  18. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004
    This helped:
    https://community.hpe.com/t5/Web-and-Unmanaged/Does-j9980A-HP1820-24G-support-LACP/td-p/6867909


    I got it figured out.

    For me the fix was disabling Static Mode in the existing configuration option above as it relates to the HPE Office Connect 1820, and rebooting everything. Supposedly it should work with static too - but with static I couldn't get I to work for whatever reason.

    Now it appears to be operational. I did verify that the Office Connect 1820 only supports layer 2 (mac address) not Layer 2 + IP address (layer 3 - which is a bit superior option).


    upload_2018-6-16_13-37-3.png
     
  19. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004
    Ugh - when I test this with my desktop and my laptop - both devices can get about 112MB/s when copying solo - but went copying big files together - same file or different - they both drop to about 56MB/s or some slightly lopsided division of the max throughput of a single port - priority in bandwidth given to the device started first...........

    So it's not working afterall. I've rebooted everything - but it's still acting like a single 1GB port. Any ideas? All devices are connected directly to the Office Connect 1820 Switch. (NAS two cables, Desktop cable, and laptop cable) The router isn't seeing network traffic data use from any of them in this testing - so I know the file copy data traffic is staying local to the switch.

    QNAP Trunk Setting -
    upload_2018-6-16_15-4-56.png


    Office Connect 1820 Setting -
    upload_2018-6-16_15-6-22.png


    Desktop 1 copy speed by itself
    upload_2018-6-16_15-8-8.png

    Laptop 2 copy speed by itself
    upload_2018-6-16_15-14-17.png




    Together - desktop
    upload_2018-6-16_15-15-33.png

    together - laptop
    upload_2018-6-16_15-16-13.png



    It’s worth pointing out that QNAP CPU and HPE Office Connect 1820 switch CPU both never peaked above 30% use during this testing...
     
    Last edited: Jun 16, 2018
  20. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004
    alright I'm at a loss. I put another laptop into the mix to determine if I could get use more bandwidth. This one was connected to the wifi. So now two laptops (one wired, one wifi), and one desktop that is wired.

    Still I'm limited to the speed of a single Gb port speed shared. All three throttled down to the 30-40MB/s range when all copying at the same time. When any two were connected or just one at a time they'd speed up to saturate the 1Gb/s link speed individually.

    With all three copying at the same time (different large files - BluRay ISOs)

    CPU on NAS is at 30%
    QNAP NAS Upload speed was 116MB/s
    CPU on 1820 Switch was 17%


    upload_2018-6-16_22-58-55.png
     
  21. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004
    Any ideas on this Blue Fox? I've spent hours today on this - but no joy yet.
     
  22. mda

    mda [H]ard|Gawd

    Messages:
    1,290
    Joined:
    Mar 23, 2011
    I would have chimed in and said SG300 even if the die hard networking guys don't like it, it seems to do the job (200$ over at amazon) -- no need for CLI and I got LACG just set up today and it was pretty simple. I got the NAS (synology) reporting a total throughput/read of 200MB/s from the NAS, 2 clients pulling 100MB/s each. Best of luck to you! I'm going down the networking rabbithole too..
     
  23. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004
    I called HP today and spoke to a CSR who remote controlled my machine, switch, router for 1.5 hours. We couldn't get it working, he tried the same things I'd already tried and got the same result.

    One thing we did test that I hadn't before was pulling 1 cable.

    We figured out pulling one of the paired adapters didn't interrupt copies (so If you pulled adapter 1, then adapter 2 would take over without an issue and the copy would continue - when you put adapter 1 back in place, then the data copy would move from adapter 2 back to 1 --- they weren't sharing the load --- and so learned the failover/redundancy aspect of the trunking is working fine --- but so far, just one adapter is carrying the full data load, instead of both adapters splitting it.

    I just submitted a QNAP ticket.
     
  24. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004
    I took the QNAP over to my brothers house who is a Cisco Network tech. He had it working properly in a couple minutes on a Cisco 2960S (Cisco IOS Software, C2960S Software (C2960S-UNIVERSALK9-M), Version 12.2(55)SE5)
    Both computers were easily copying down 80-112MB/s range at the same time. So I know it's not the QNAP.

    My brother spent about 1.5 hours trying to get it to work on the HPE 1820 and we couldn't get it figured out. He still suspects its a configuration mismatch, but it's not an obvious one --- I spent 1 hour with a HPE tech, a session with my brother for 1.5 hours, and then another 1.5 hours with a different HPE tech and we still can't figure it out.

    Now the QNAP tech is starting to advise me from my submitted ticket --- but knowing that the LACP trunking functionality works fine on the Cisco 2960S means to me this layer 2 HPE Office Connect 1820 switch is going back if we don't get this sorted out soon, and I'll try something else.

    I'm super annoyed with this --- given how much time I've spent on something that should be easy to implement - and the fact I tried to do my research before hand is salt in the wound.
     
    Last edited: Jun 22, 2018
    Ranulfo likes this.
  25. BlueLineSwinger

    BlueLineSwinger Gawd

    Messages:
    523
    Joined:
    Dec 1, 2011
    My best guess is that the Cisco is using a different load balancing mechanism than the HP is. According to this, the Cisco's default is to use the MAC of the source, but double-check what was actually used. Have you tried setting up the HP the same (and is a reload/restart required to properly enable the change)?

    Likewise, does the QNAP have a setting for this? Maybe one of the IP-based options would work better.
     
    Archaea likes this.
  26. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004
    That's a good assessment, and one both the HP reps tried and my brother tried. I feel like we've done every option with the HP. Disabling and enabling the trunk in between. It's possible I missed a combination - because there are a lot, but I don't think so. I've tried both the MAC and IP options on the HP switch.


    1820 switch has these manual options (and it looks like 3 and 6 are the combination choices for 1,2, and 4, 5)
    vs. Static Mode disabled, which is just Dynamic 802.3ad (LACP)
    upload_2018-6-22_12-3-30.png

    The QNAP NAS has these options:
    upload_2018-6-22_12-6-39.png


    NAS options explained:

    Balance-rr (Round-Robin) -> Must Be Configed From Switch First
    Round-Robin mode is good for general purpose load balancing between two Ethernet interfaces. This mode transmits packets in sequential order from the first available slave through the last. Balance-rr provides load balancing and fault tolerance.
    Supports static trunking. Make sure static trunking is enabled on the switch.

    .
    Balance XOR -> Must Be Configed From Switch First
    Balance XOR balances traffic by splitting up outgoing packets between the Ethernet interfaces, using the same one for each specific destination when possible. It transmits based on the selected transmit hash policy. The default policy is a simple slave count operating on Layer 2 where the source MAC address is coupled with destination MAC address. Alternate transmit policies may be selected via the xmit_hash_policy option. Balance XOR mode provides load balancing and fault tolerance.
    Supports static trunking. Make sure static trunking is enabled on the switch.

    .

    Broadcast -> Must Be Configed From Switch First
    Broadcast sends traffic on both network interfaces. This mode provides fault tolerance.
    Supports static trunking. Make sure static trunking is enabled on the switch.

    .

    IEEE 802.3ad (Dynamic Link Aggregation) -> Need Supported Switch, Must Be Configed From Switch First
    Dynamic Link Aggregation uses a complex algorithm to aggregate adapters by speed and duplex settings. It utilizes all slaves in the active aggregator according to the 802.3ad specification. Dynamic Link Aggregation mode provides load balancing and fault tolerance but requires a switch that supports IEEE 802.3ad with LACP mode properly configured.
    Supports 802.3ad LACP

    .

    Balance-tlb (Adaptive Transmit Load Balancing)-> General switches
    Balance-tlb uses channel bonding that does not require any special switch. The outgoing traffic is distributed according to the current load on each Ethernet interface (computed relative to the speed). Incoming traffic is received by the current Ethernet interface. If the receiving Ethernet interface fails, the other slave takes over the MAC address of the failed receiving slave. Balance-tlb mode provides load balancing and fault tolerance.

    .

    Balance-alb (Adaptive Load Balancing) -> General switches
    Balance-alb is similar to balance-tlb but also attempts to redistribute incoming (receive load balancing) for IPV4 traffic. This setup does not require any special switch support or configuration. The receive load balancing is achieved by ARP negotiation sent by the local system on their way out and overwrites the source hardware address with the unique hardware address of one of the Ethernet interfaces in the bond such that different peers use different hardware address for the server. This mode provides load balancing and fault tolerance.
     
    Last edited: Jun 22, 2018
  27. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004
    So for giggles I tried the QNAP 231P on my ASUS RT-AC3100 router (physical ports LAN1 and LAN2) which also features layer 2 (only) trunking.

    I then rebooted both Qnap and Asus Router.

    Here are the trunking instructions for a sister model Asus Router (the 8 port model instead of the 4 port model I have).
    https://www.asus.com/support/FAQ/1016088


    Here is a screenshot showing my RT-AC3100 router and the config I see which matches the 8 port config in the instructions above.

    upload_2018-6-22_21-22-31.png

    Same problem. One adapter gets all the bandwidth. This is two computers copying BluRay ISO's locally to their C: drive.

    Max data pull is 113 ish GB/s.

    It's really starting to feel like these switching devices need layer 2 + layer 3 IP for QNAP to get the trunking speed increase over both NICs.

    upload_2018-6-22_21-21-12.png

    upload_2018-6-22_21-24-11.png
     
  28. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004
    Okay now this is interesting. I asked for help over at reddit and someone suggested with MAC (layer 2) I didn’t have enough machines running (with 2 or 3) to guarantee I wasn’t just unlucky with the clumping of adapters. That seems to have some truth.

    I fired up four machines. Two wireless and two over LAN and started copying large data folders. Now the NAS shows 135MB/s download over the NICs and its no longer going through just one adapter.

    Mostly through one, but the second adapter is picking up some slack now

    This is using the ASUS RT-AC3100 I tried in the previous post.

    F7E7CDB1-FCE6-4491-B9AA-44DE23E8C147.jpeg
     
    Last edited: Jun 22, 2018
  29. BlueLineSwinger

    BlueLineSwinger Gawd

    Messages:
    523
    Joined:
    Dec 1, 2011
    Sounds like a reasonable explanation. The Cisco may simply have a better balancing algorithm than the other two, or maybe during that test it was simply dumb luck the two clients were split across the link aggregation.
     
  30. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004
    I was told by my network tech brother that I could prove or disprove the HPE Office Connect 1820 switch is working correctly with a simple test, now that I have it pulling uniquely from both adapters.


    Using 802.3AD + Layer 2+3 (MAC + IP) setting on the NAS device get two different machines associated with two different adapters. Prove each machine can max the GB line pulling down from the NAS to the local hard-drive. Then run them simultaneously. If they don't stay at about 100MB/s each then something is wrong. And since we know that worked with his Cisco 2960S on Thursday night - then it's got to be something wrong with the HPE Office Connect 1820 switch. So here goes.

    This is the test he suggested.

    PC 1: Connected to the HPE 1820 switch via cat 5e. My gaming desktop is pulling down a large folder BluRay movie ISO. I'm maxing out adapter 1 - and it's only using one of the two adapters as expected.
    upload_2018-6-23_9-16-23.png

    PC 2: Connected to the HPE 1820 switch via cat 5e. This is one of the laptops that utilized adapter 2 and as you can tell it's basically maxing out adapter 2 -- as expected.
    PC 2 bandwidth.PNG
    NAS when PC 2 is copying down.PNG

    Now - I'll screenprint them both copying together. It should be around 200MB throughput from the NAS. If it's not - somethings not working properly because the Cisco 2960S could do that with these same movie ISO files.



    Now as you can see, I unpaused the first copy (windows 10) and I'm still stuck at cruddy speeds. Just barely over single adapter speeds. Since this didn't happen with the Cisco 2960S - what does that mean?
    upload_2018-6-23_9-34-48.png


    It's not the switch is overloaded. Look at the CPU use while this simultaneous copy is happening.
    upload_2018-6-23_9-36-28.png

    For reference here are my Switch trunk settings - which shouldn't matter because the read is happening on both workstations from the NAS as the source.
    upload_2018-6-23_9-37-22.png
    upload_2018-6-23_9-38-41.png
    upload_2018-6-23_9-38-15.png
     
    Last edited: Jun 23, 2018
    IdiotInCharge likes this.
  31. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004
    About 135MB/s is the best I've seen so far with the Office connect 1820 and QNAP 231P (pulling down from the NAS) with any test I've done. I got there with 5 machines and I go there with 3 machines. 2 machines doesn't typically go faster than about 125MB/s for whatever reason.


    This is three wired hosts copying down using XOR Trunking on the NAS.

    231P setting:

    upload_2018-6-24_19-53-52.png


    1820 Switch setting:
    upload_2018-6-24_19-55-14.png




    Note each machine is getting about 45MB/s. It's got two machines working on adapter 1, and just one machine assigned to adapter 2 - and each gets ~45MB/s. QNAP NAS CPU use is about 40%
    upload_2018-6-24_19-44-17.png


    One machine's copy finished and now the other two machines jump to 65MB/s. --- one on each adapter. CPU use drops to 25%.
    upload_2018-6-24_19-45-37.png

    when the second host finishes and just one machine is copying it averages about 100MB/s but with peaks of 113MB/s. CPU is basically unchanged at about 25%.
    upload_2018-6-24_19-52-26.png
     
    Last edited: Jun 24, 2018
  32. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004
    Okay - now this is super interesting. A reddit guy told me to kick the copies off at the exact same time on a machine on each adapter and see what happened.

    BINGO -- now approaching that 200MB/s range. That's the highest I've seen on this by far on the 1820 Switch!

    upload_2018-6-24_20-5-41.png

    But it doesn't stay there very long. Those spikes you see that I labeled with a red X are the short term spike on copy down. After just a few moments it settles down to the 60MB/s range again.
    upload_2018-6-24_20-9-19.png


    Further testing has shown that one of my workstations is faster than the other and will slightly get ahead. (spinning drive on one of the two that happened to get assigned to the second adapter and SSD on the one that got assigned to the first adapter). If I pause the machine that is slightly faster and let the second machine go past, further into the copy and then resume my faster machine's copy (Win10 lets' me pause copy functions) then at the time the two workstations are around the same point in the bluray ISO the copy speed on it's own skyrockets again towards the 200MB/s range. Same behavior as when I kick them off at the same time.

    That's starting to sound more like a NAS read or memory limitation at this point? BUUTTTTTT

    I can't say we spent this much time testing on the Cisco to validate that it didn't slow way down over the course of a long copy. But I can tell you the Cisco does seem to hold the fast copy much longer than the 1820, because when using the 1820, that speed slows down within just a few seconds. (and hasn't even been accidentally encountered until I started doing this testing with starting and stopping the same file copy at the same time) -- and with the Cisco it just was at 180-200MB/s without trying anything specifically synchronized.

    upload_2018-6-24_20-47-42.png
     

    Attached Files:

    Last edited: Jun 24, 2018
  33. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004
    So here is my QNAP disk drive benchmark and information. At this point I'm copying from the nAS a single 25GB bluray
    ISO to 2 workstations c:\temp simultaneously. I'm seeing 135MB/s combined on XOR setting with the QNAP 231P.


    I don't think the 5TB Western Digital RED drives should be a limitation right? Since they are in RAID 1, it should read faster than just a single disk I would think --- even as high as those 200MB/s peaks we saw above. But maybe when they exceed the RAM cache of the NAS the speed has to slow down???

    upload_2018-6-24_21-37-27.png


    And again with the Linux disk command "iostat -x 2" which tells the iostat command to update every 2 seconds. It looks like these drives are 100% busy in this command even though they don't necessary show such in the QNAP GUI pages??? Is this my problem?
    upload_2018-6-24_22-40-59.png

    Finally a single PC maxing out adapter 1 with a iso. iostat - 2 shows 60-65% busy on the drives.
    upload_2018-6-24_23-15-53.png


    I'm going to borrow my brothers Cisco 2960S and test more with it --- QNAP quotes higher speeds with aggregation. 176MB/s write and 224MB/s read --- and those numbers with lesser 7,200RPM 1 TB drives that they used for their benchmarks.

    https://www.qnap.com/en-us/product_x_performance/product.php?type=2&II=258

    They also used a Cisco 2960 switch for their testing -- so I'll be real curious to do some further testing with the Cisco and determine if it can actually hold the higher copy rates the whole time - or just when the workstation hosts are actually synced and the file is more or less in the same state of being copied down like we've determined in post #33 above. Right now this is looking like a disk/NAS limitation --- but we didn't consider that when we did the brief testing at my brothers house on Thursday with the Cisco switch and saw 180MB-200MB/s.
     
    Last edited: Jun 25, 2018
  34. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004
    The suggestion of a reddit user to enable flow control under the switching menu on the 1820 switch seemed to help.

    Now up to 148.

    However - I had to switch back to static XOR - because my three machines kept trying to all go to the same adapter on dynamic 802.3ad this morning in my testing.



    upload_2018-6-27_8-19-26.png



    upload_2018-6-27_8-19-5.png
     
  35. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004
    I borrowed my brother's Cisco 2960S tonight and did some additional testing. The Cisco switch does seem to be faster in identical testing. I spent two weeks troubleshooting this and probably 20+ hours at least. 2.5 hours on the phone and remote screen share with HP Enterprise support, and then a QNAP ticket later -- I'd had no success.

    Plug in the Cisco 2960S and 5 minutes later it's working better.

    Take a look at these numbers...

    230MB/s pulling from the NAS device from two different workstations. It never slowed down the whole time I was downloading. This is kicking off the same file at the same time. (I tried this with the HPE 1820 and I'd get about 5-10 seconds of identical speed and then it'd drop to about 60MB/s each. With the Cisco it stayed identical, maxed out the whole time).

    This is frustrating -- me trying to cheap out on price, and moreso skip what I deemed to be complication by getting something with a web GUI so I didn't have to learn CLI. Those expectations, were upended by a couple lines of CLI to trunk the ports (that my brother literally setup in two minutes) and it just works!!!!!!!!!


    arghhh I'm so darn irritated right now.

    This is starting a ~6GB ISO copy at the same time on each machine:

    upload_2018-6-29_20-52-25.png

    This is starting a ~25GB ISO copy with about a 10GB difference in start points. To test a switch buffer size theory, that it all relates to starting the file at the same time. It maintained about 85-95MB/s per NIC. On the HPE it maintained only about 55-65MB/s per NIC.

    upload_2018-6-29_22-26-34.png

    Further testing with two different files, and copies initiated at two different times shows that the copy slows down to about 63MB/s or so. So ultimately - at the end of the day --- that' the real wold use case --- different files, with the copies started at different times --- and in the real use case the switches perform about the same in my hour or two of playing with the switches. But if you are copying the same file - or if you start copying the file at the same time - that Cisco really pours on the coals!
     
    Last edited: Jun 30, 2018
    IdiotInCharge likes this.
  36. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004

    That 200MB/s number you quote. Do you have any of the idiosyncrasies I noted above with my HPE in post 33 above. Where it slows down dramatically if you are pulling different files, or if you don't start the copy at the same time? Or does it keep a pretty constant 200 no matter what you are doing?
     
  37. Archaea

    Archaea [H]ardForum Junkie

    Messages:
    8,655
    Joined:
    Oct 19, 2004
    I give up...

    just end me.

    I tried the HP 1820 again tonight to give it one last chance so I could prove it was different on the same tests and so be armed with intent to send it back --- ...….and I'll be buggered. It's not different tonight, in my testing. I'm seeing identical performance from the HPE now as I did from the Cisco.

    I HAVE NO IDEA WHY.

    I'm using 802.3ad layer 2 + 3 (Mac + IP) on the NAS
    Dynamic 802.3ad on the switch
    the generic setting combination I've tried more than any of them.

    and for whatever reason I'm getting the solid performance now over the entire length of the BluRay ISO when both copies are started at the same time on my desktop and laptop. I've never seen over 185MB/s with the HPE before now, and it was very brief. Here I got over 220-233MB/s sustained for the entire copy. The files were copied down at the same time, and so were in Sync, and pulled down at the same time, but I've done this test a dozen times on the HPE before and it always slowed down to 115-130MB/s ish range in previous tests.

    I don't get it.

    I'm going to bed. Threaten to get rid of a piece of electronics gear and it starts to finally behave.....

    upload_2018-7-1_0-36-7.png

    start the copies about 10GB apart on the same file and get about the same speed as the Cisco testing above too.

    upload_2018-7-1_0-52-38.png

    With two different files it's now mimicking the Cisco too. like 55-60MB/s for each adapter.
     
    Last edited: Jul 1, 2018
    IdiotInCharge likes this.
  38. IdiotInCharge

    IdiotInCharge Not the Idiot YOU are Looking for

    Messages:
    8,105
    Joined:
    Jun 13, 2003