PCIE Bifurcation

Discussion in 'Small Form Factor Systems' started by jb1, Jul 28, 2015.

  1. TheArkratos

    TheArkratos n00b

    Messages:
    50
    Joined:
    Nov 20, 2017
    C_Payne do you have a recommended motherboard for your x4x4x4x4 itx riser?
     
  2. C_Payne

    C_Payne [H]Lite

    Messages:
    121
    Joined:
    Jan 6, 2017
    Either asrock x299e-itx or x99e-itx.
    Or the server counterparts from asrock.

    I don't have any of the behind itx in stock currently.

    Let me know!
     
  3. TheArkratos

    TheArkratos n00b

    Messages:
    50
    Joined:
    Nov 20, 2017
    That's so disappointing that I'd have to step all the way up to that board.... With people demoing a 4x m.2 card for Ryzen 3000 maybe we'll get x4x4x4x4 support on x570. Because I'm really pondering a PfSense router with 10G ethernet cards.... And your riser board looks perfect if someone would just support it on a reasonably priced platform. I'll potentially order one in a few months... IF x570 supports it. I love your work though! As a case scratch builder I'm seriously looking at your risers and pondering builds.
     
  4. TheArkratos

    TheArkratos n00b

    Messages:
    50
    Joined:
    Nov 20, 2017
    C_Payne and Neapolitan6th like this.
  5. C_Payne

    C_Payne [H]Lite

    Messages:
    121
    Joined:
    Jan 6, 2017
    That is awesome news! Ryzen x4x4x4x4!
     
    TheArkratos and Neapolitan6th like this.
  6. C_Payne

    C_Payne [H]Lite

    Messages:
    121
    Joined:
    Jan 6, 2017
    Update on my PLX work:
    IMG-20190620-WA0002.jpg
    It does work with some Lanes missing (maybe bad BGA solder joints?)
    Assembling took me a whole day though.
    Lets see if I can make these available sooner or later.
     
  7. haestrod

    haestrod n00b

    Messages:
    1
    Joined:
    Jun 20, 2019
    Has anyone been successful in modding a BIOS to engage in bifurcation? Specifically an Asus Strix X470-I? Although I would be interested to hear success stories with any board.

    Edit: I'm aware of www. win-raid. com/t3323f16-Guide-How-to-Bifurcate-a-PCI-E-slot.html
     
    Last edited: Jun 21, 2019
  8. TheArkratos

    TheArkratos n00b

    Messages:
    50
    Joined:
    Nov 20, 2017
    How does this work? Does the bios have to support the PLX application? I'm familiar with the theory of PLX, but how does this turn out in practice when it's not implemented directly on the motherboard?
     
  9. chemist_slime

    chemist_slime [H]ard|Gawd

    Messages:
    1,100
    Joined:
    Jul 24, 2005
    When can I buy one? :)
     
  10. C_Payne

    C_Payne [H]Lite

    Messages:
    121
    Joined:
    Jan 6, 2017
    BIOS support is not necessary AFAIK. Otherwise dual GPU cards wouldn't work either. It works fine with my X99E and it should work with any mainboard.

    The issue I currently have is that I haven't figured out a reasonable way to manufacture these in little quantities.
    Hand assembly is NOT an option, took me a complete day.

    I will keep you posted.
     
    chemist_slime and TheArkratos like this.
  11. DaHans

    DaHans n00b

    Messages:
    37
    Joined:
    May 21, 2015
    Do you have any news for the DAN A4?
     
  12. C_Payne

    C_Payne [H]Lite

    Messages:
    121
    Joined:
    Jan 6, 2017
    Unfortunately I had no time yet to work on the dan-project. Also I will be on a vacation for the next 3 weeks, so it will have to wait a bit longer. But I won't forget about it!
     
  13. DaHans

    DaHans n00b

    Messages:
    37
    Joined:
    May 21, 2015
    Thanks for the quick reply.

    No problem. Happy Holidays! ;)
     
  14. TheArkratos

    TheArkratos n00b

    Messages:
    50
    Joined:
    Nov 20, 2017
    Any word on Ryzen 3000, X570, and pcie gen 4 with existing bifurcation risers?
     
  15. C_Payne

    C_Payne [H]Lite

    Messages:
    121
    Joined:
    Jan 6, 2017
    Have a ryzen 3000 and rx 5700 ordered, will report next week when back from my trip.
     
  16. extide

    extide 2[H]4U

    Messages:
    3,450
    Joined:
    Dec 19, 2008
    I'm fairly sure the MSI Creation supports it at least -- it comes with a M.2 adapter to put 2x M.2's in a PCIe 8x slot -- no PLX chip. I actually have the board but no cpu/ram yet but I will be getting the rest of my parts in over the next few weeks.
     
    TheArkratos likes this.
  17. chemist_slime

    chemist_slime [H]ard|Gawd

    Messages:
    1,100
    Joined:
    Jul 24, 2005
    Oh, and PLX update please. :)
     
  18. C_Payne

    C_Payne [H]Lite

    Messages:
    121
    Joined:
    Jan 6, 2017
    OK: First results with ryzen 3000 and PCIe gen4:

    Summary:
    The Gigabyte X570 I Aorus Pro WIFI Mini-ITX supports bifurcation even with its stock BIOS, all modes (x8x8, x8x4x4 and x4x4x4x4 are available... YAY!!!!!)
    I have Sucessfully tested x8x8 and x8x4x4 with 4 different versions of my existing risers.

    Using one of my risers with the card in the topmost slot(longest traces) did not impact performance (pic 1+2)
    gen4 x8 yields around 13500 MByte/sec and there is no difference between card in Slot and card in Riser. (Raw should be 16000, but one has to deduct overhead!)
    I also tested an 50cm LinkUp Riser cable which works without speed-drops as well.

    What did NOT work was plugging my Riser into the LinkUp cable. Speed dropped to gen1. (Although I could probably set it to gen3 via BIOS, had it set to Auto here)

    I have ordered A Dan-Cases A4-SFX and will report how the Flex-Riser in that case will work together with my Risers.

    Also I enquired DIODES inc. for a datasheet for their gen4 redrivers (the ones that are used on pretty much every 3 Slot ATX board for the second slot)
    Sadly no reply yet. Those Chips are available on digikey though... Maybe i will go through the hoops and reverse engineer the Pinout...

    Anyways I would conclude this to be good news.
    Bifurcated gen4 builds are in fact possible with my risers as they are right now. (For example with the GPU placed directly next or above the mainboard without using any additional cabling)
    Further testing will be neccessary and follow as soon as I get the time and hardware.

    IMAG2994.jpg
    IMAG2995.jpg
    IMAG2996.jpg
     
  19. DaHans

    DaHans n00b

    Messages:
    37
    Joined:
    May 21, 2015
    Wow, thanks for testing. Sounds great that Bifurcation is suported out of the box.

    DAN also mentioned, that PCIe4.0 doesnt work with the risers in Dan CAse v3 and 4. Link

    So maybe some peaople have to order a new riser for getting PCIe 4.0.

    Looking forward for the Bifurcation solution in the DAN A4 with you risers. :)
     
  20. TheArkratos

    TheArkratos n00b

    Messages:
    50
    Joined:
    Nov 20, 2017
    Btw those looking for a mini-tower case that supports bifurcation with very little work, I do recommend the Neo-G (http://metallicgear.com/products/Neo-G-Series-mITX). It's not the smallest case, but it's pretty, does a decent job supporting AIOs/watercooling, and is super easy to support a bifurcation riser. To use a bifurcation riser just get some motherboard standoffs, some washers, and drill 4 holes. I can post pictures of my set up Thursday probably, I'm planning to build my own case eventually but wanted a case for my new 3900x so I can use it in the meantime.
     
  21. TheArkratos

    TheArkratos n00b

    Messages:
    50
    Joined:
    Nov 20, 2017
  22. Meepcmx

    Meepcmx n00b

    Messages:
    2
    Joined:
    Aug 20, 2019
    Does anone have any insights into how these risers behave with IOMMU groups? Will any devices attached to the split ports end up in the same group, or will/can they be in seperate groups (for device passthrough to virtual machines)
     
  23. C_Payne

    C_Payne [H]Lite

    Messages:
    121
    Joined:
    Jan 6, 2017
    On my x99 with x8x8 the GPUs ended up in separate IOMMU groups.
    I suppose this would be the standard behavior.
     
    Meepcmx likes this.
  24. Meepcmx

    Meepcmx n00b

    Messages:
    2
    Joined:
    Aug 20, 2019
    Thanks for that insight. I have a slightly different use case than discussed here in that I’m interested in maximizing pcie slot availability for virtualization and device pass through (usb adapters and the like).

    I had a look at your site and will likely be placing an order. Wondering, however, if there’s a version of the 4x4x4x4 breakout card with a lower slot pitch. They are set up for dual slot cards, but I’d be needing only single slot, so could tolerate the slots being closer together, and therefore taking up less space overall.
     
  25. C_Payne

    C_Payne [H]Lite

    Messages:
    121
    Joined:
    Jan 6, 2017
    I will gladly make a custom one for you with single slot spacing. you can contact me via email risers@peine-braun.net
     
  26. TheArkratos

    TheArkratos n00b

    Messages:
    50
    Joined:
    Nov 20, 2017
    C_Payne I would love to design some cases for your amazing risers, sadly I only have experience making acrylic cases (similar to Parvum). I wonder if you've explored reaching out to some of the SFF case companies such as Sliger to see if they would be interested in working with you for bifurcation cases?
     
    chemist_slime likes this.
  27. fireworm

    fireworm n00b

    Messages:
    3
    Joined:
    Sep 2, 2019
    Hey guys, wondering if anyone could help me troubleshoot / debug my pci bifurcation setup:

    MOBO: Gigabyte z270 WIFI Itx
    CPU: i7-7700k
    GPU: EVGA 1080 ti FTW3

    Splitter: ARC1-PERY423-C20
    Other PCI card: Intel 10gbit card

    When I plug it all up and boot into Windows, I can successfully see my 10gb pci card.

    Using the onboard graphics, I cannot see my 1080ti in device maanger (it is connected to another monitor). I can't install nvidia drivers either (at least the one's I downloaded from the nvidia site, 435.xx).

    I looked in hwinfo, and saw the 'gp(102)' (which is the code name for the 1080ti), but couldn't see it in device manager, and couldn't manually install the driver either.

    I rebooted, and now I don't see the 1080ti in hwinfo anymore. Card does light up and spin fans on boot.

    Any ideas would be much appreciated.
     
  28. cyklondx

    cyklondx Limp Gawd

    Messages:
    388
    Joined:
    Mar 19, 2018
    that mobo does not support bifurcation. Only couple z270 mobo's supported it (mining mobo's with bios option for it).
     
  29. Neapolitan6th

    Neapolitan6th [H]ard|Gawd

    Messages:
    1,078
    Joined:
    Nov 18, 2016
    It was in their marketed features so I'd be inclined to say it should work.
     
  30. fireworm

    fireworm n00b

    Messages:
    3
    Joined:
    Sep 2, 2019
    BIOS for this MOBO does indeed have the 2x8 PCI Split option. Anyway, I disassembled everything and laid it out on my desk trying different combos, (found I did have one bad riser, GPU wouldn't boot when it was connected through this riser to my MOBO, with no splitter).

    After trying some some combos and risers and no risers and a different computer (to verify the GPU even still worked):

    The splitter didn't like the gpu when I had a riser between them.

    So, this setup did NOT work: mobo > 20cm riser > Splitter > 5cm riser > GPU.
    But this setup DID work: mobo > 20cm riser > Splitter > GPU.

    This is kind of annoying because I can't put my GPU into the bottom slot of the splitter, and use another riser out of the top slot (because the splitter is now trying to go into my Dan Case / mobo). So, I'll have to put the gpu in the 'top' slot, and a riser between the GPU and splitter in the bottom slot. I tried this, and it doesn't fit vertically due to the GPU's heatsink being very close to the slot, so I'm gonna file down another riser for the 10gb nic and make it sit deeper into the splitter, and hope that works.
     
  31. fireworm

    fireworm n00b

    Messages:
    3
    Joined:
    Sep 2, 2019
    Some more experimenting showed the following result:

    MOBO > Splitter > [20cm Riser > GPU] + [20cm Riser > 10gb card] = Works fine, slot doesn't matter.

    Looks like the real issue was doing MOBO > Another Riser > Splitter.

    This means you need a splitter which has a long riser [MOBO > Splitter's Riser > Splitter's PCB], if you hope to move the Splitter PCB out of the CPU chamber.

    My Splitter has a small 5cm riser on itself (it's not just a pluggable pcb), so I'm SOL for keeping my CPU chamber breathable. :(

    Hope that's not confusing, and hope it helps someone else out there.
     
  32. AP2

    AP2 [H]ard|Gawd

    Messages:
    1,436
    Joined:
    Jun 25, 2007
    Just curious would bifurcation plus NVLink affect performance of 2 2080tis on mini itx setup?
     
  33. C_Payne

    C_Payne [H]Lite

    Messages:
    121
    Joined:
    Jan 6, 2017
    It will be just the same compared to a mainstream CPU (16 Lanes total) on an ATX board. Nothing different happening.
     
  34. AP2

    AP2 [H]ard|Gawd

    Messages:
    1,436
    Joined:
    Jun 25, 2007
    @c-Payne, I sent you an email.
     
  35. ChronoDog

    ChronoDog [H]Lite

    Messages:
    110
    Joined:
    Oct 15, 2009
    Does anyone know of a bifurcated (x8x8) riser that's both rigid and flexible?
    Basically, the idea would be to use a 905P (or any other HHHL card) along with a 2-slot GPU in an NCase M1.
    The SSD would sit over the PCIe slot, with the graphics card one slot below it.

    What I'm getting at, is something like this:

    uOgreZW.jpg

    With the top part of this (but in x8) attached to the left side of it it:

    3flJXt0.jpg

    I looked around, but can't seem to find anything like that.
     
  36. TheArkratos

    TheArkratos n00b

    Messages:
    50
    Joined:
    Nov 20, 2017
    Like this? http://www.ameri-rack.com/ARC2-PELY423-C7_m.html
    If you do want it you have to call/email them and they'll get you an invoice to pay and then ship it out to you, those risers are excellent, I have several.
     
  37. ChronoDog

    ChronoDog [H]Lite

    Messages:
    110
    Joined:
    Oct 15, 2009
    I already have exactly that in an A4-SFX, see here, but that's not what I mean and doesn't fit the purpose, unfortunately.
    What I'm loking for is a rigid riser that will basically turn a half-height (low-profile) single-slot PCIe card into a full-height one, whilst also adding another PCIe x16 (x8 electrically) slot right next to the existing one using a flexible cable.

    Here's a limpy Paint representation, which hopefully explains it better:

    unUuZLn.png
     
    Last edited: Sep 28, 2019
    JokinenK and TheArkratos like this.
  38. TheArkratos

    TheArkratos n00b

    Messages:
    50
    Joined:
    Nov 20, 2017
  39. ChronoDog

    ChronoDog [H]Lite

    Messages:
    110
    Joined:
    Oct 15, 2009
    Upward risers do exist, just not bifurcated ones.
    The goal is bifurcating a PCIe slot while keeping a low-profile expansion card in the same place without using multiple risers.
     
    TheArkratos likes this.
  40. C_Payne

    C_Payne [H]Lite

    Messages:
    121
    Joined:
    Jan 6, 2017
    I have actually just recently made a low profile to full height riser and also made a long time ago a bifurcated one for an M.2 slot. (search this thread)
    Spacing is very tight though and I don't have the tools to work with small twinax cables. (yet) Maybe I can figure something out with an angled slot...
     
    TheArkratos likes this.