PCIE Bifurcation

J

jb1

Limp Gawd
Joined
Dec 27, 2014
Messages
500
Hi all,
I'm interested in doing a build with two GPUs in SLI using one 16x PCI-E slot on a mini ITX motherboard. My understanding is that the Z97 chipset supports PCI-E bifurcation so you can just split the signal into two 8x ports. However, my motherboard has no setting for PCI-E bifurcation. Is it necessary to enable this, or will the chipset just figure it out when it sees two devices on the same PCI-E slot?
 
PCI-E Bifurcation requires bios support, I'm not actually sure the Z97 supports this feature anyway. As an example ASRock's support site says only their X99 boards support it.
 
Some ASRock motherboards have support but also require a specific BIOS to enable them, like the Z87E-ITX.

Mind you that all Z87 and Z97 chipsets support PCIe Bifurcation, but only a select few BIOSes have this enabled.
 
Okay great, thanks for the note–– I couldn't find bifurcation mentioned anywhere in the X99E/ITXac manual. Anyone know about gigabyte's GA-Z97N-Gaming 5?
 
Just called gigabyte... whomever I spoke with hadn't heard of PCI-E bifurcation before and said that it won't work. I believe this.

Somehow I feel cheated of this excellent feature of the Z97 chipset–– should have anticipated this desire a year ago.
 
Last edited:
dude, I'm doing the exact same thing with the asrock x99e itx board, what a small world. I have actually tried it with a few diff splitters and have had no luck so far.

Products I have tried are, this one failed to show anything.

http://www.amazon.com/Supermicro-RS...r=8-3-fkmr0&keywords=supermicro+pci+explitter

I have also tried this one, also failed to show any graphics card display.

http://www.ameri-rack.com/ARC2-PELY423-C7_m.html

I just got into today: http://www.acmemicro.com/Product/13...E16-A-LHS-Active-PCI-E-2U-Riser-Card?c_id=356

This product is different from others in that it has an active PCI splitter but I have yet to verify if it has a PLX chip yet because it's under the heatsink. I'll do that this weekend when I have more time with the system. Unfortunately, this splitter card is too long for the case I'm using so even if I get it working I'll have to find an alternative solution but at least I'll know it works.

Anyone know if this is bios related or not? Do I need to turn something on/off?

I'm really hoping to get this all working for this special project I'm working on which will be a one of a kind but getting this dual graphics card working in SLI would be amazing!
 
As an Amazon Associate, HardForum may earn from qualifying purchases.
Nice, glad to see you're trying this!

You might want to call Asrock and see what they say–– maybe you need a special version of the BIOS to bifurcate? As posted above, the X99 ITX should work with bifurcation according to Asrock:
http://forum.asrock.com/forum_posts.asp?TID=355&title=pcie-bifurcation-in-z97eitxac

Also, for the active splitter that you just got (RSC-R2UG-A2E16-A), could you post the dimensions of the card? I want to see if this will fit in my case and I can't find concrete details on its length and width.

Please let us know how it goes!
 
jb1 said:
Nice, glad to see you're trying this!

You might want to call Asrock and see what they say–– maybe you need a special version of the BIOS to bifurcate? As posted above, the X99 ITX should work with bifurcation according to Asrock:
http://forum.asrock.com/forum_posts.asp?TID=355&title=pcie-bifurcation-in-z97eitxac

Also, for the active splitter that you just got (RSC-R2UG-A2E16-A), could you post the dimensions of the card? I want to see if this will fit in my case and I can't find concrete details on its length and width.

Please let us know how it goes!
Click to expand...


Got to try out the new Supermicro PCIE Riser and unfortunately it did not work either. No boot screen at all. I sent Asrock a support ticket and we'll see what they say. Perhaps a bios update is needed to make this all work in the end. BlueFox, from another thread has it, v1.35 so we'll see. You can see progress here:

http://hardforum.com/showthread.php?t=1855235&page=15


As for the super micro rise, it's about 6.8 inches in length, a big longer than the actual ITX form factor of 6.7".
 
Btw, just tried the bios 1.35 from blue fox and no luck. Looks like we'll have to wait for Asrock to officially respond.
 
In order to have PCIe Bifurcation to work for 2 PCIe devices you need the following:

1) Bios Support, the Bios needs to have th code to detect en change the PCIe lane assignments from 16x to 8x/8x if the code is not there it doesn't work

2) Each PCIe device needs a Reference Clock. Multi PCIe boards have multiple Reference clocks 1 for each PCIe slot. In order to get more then 1 PCIe reference clock into 1 PCIe slot you need either

A) deviate from the PCIe specification and use some of the pins to bring in another reference clock. This is what SuperMicro does.

B) Multiplex the Reference clock on the PCIe riser PCB.

I've had a look into this as well with the same idea of running X99 ITX with 2 GPUs. I've measured all of the pins on this SuperMicro card and in my excel sheet below you can see they hacked up the PCIe slot specification to get all the signals in, that is why they only work on a select number of their own server boards.

RSC-R2UG-2E4E
$T2eC16F,!ygE9s7HHghgBQGTInErNg~~60_1.JPG


The actual pin-out
full.png


The other option I've looked into is the PLX chip, but those will be very expensive, this is old information but these things can cost as much as $80, plus the validation that needs to be done because of the switching involved. But with a PLX chip you can get any dual GPU setup to run on a single PCIe slot, but that riser will more then likely costs $100+
http://www.anandtech.com/show/6170/four-multigpu-z77-boards-from-280350-plx-pex-8747-featuring-gigabyte-asrock-ecs-and-evga/31
 
chemist_slime said:
Btw, just tried the bios 1.35 from blue fox and no luck. Looks like we'll have to wait for Asrock to officially respond.
Click to expand...

Unfortunate! By any chance do you have the width of the card in addition to the length?

Maybe this isn't really a possible option given QinX's post though...

QinX said:
The other option I've looked into is the PLX chip, but those will be very expensive, this is old information but these things can cost as much as $80, plus the validation that needs to be done because of the switching involved. But with a PLX chip you can get any dual GPU setup to run on a single PCIe slot, but that riser will more then likely costs $100+
http://www.anandtech.com/show/6170/four-multigpu-z77-boards-from-280350-plx-pex-8747-featuring-gigabyte-asrock-ecs-and-evga/31
Click to expand...

I wonder, can consumers even purchase PLX chips and soldier the PCI-E connections to the chip?
 
jb1 said:
Unfortunate! By any chance do you have the width of the card in addition to the length?

Maybe this isn't really a possible option given QinX's post though...

I wonder, can consumers even purchase PLX chips and soldier together the PCIE connections, to the chip?
Click to expand...

It should be possible, the problem lies only with the motherboard manufacturer, if they have the code in the BIOS to bifurcate then you can make a PCB for it.

PLX chips are BGA packages, yes you can solder it, but not with your regular soldering iron. Also because the PLX chip is a rather advanced switching chip, implementing it isn't easy, there is a lot of validation required to be sure it works properly.

The best route to go is to build a PCIe riser card to duplicates the Reference clock.
I've been wanting to design the PCB for it and I think I know what it needs, but because I couldn't find a ITX board that has a BIOS for PCIe bifurcation I haven't bothered to give it a try.
 
QinX said:
It should be possible, the problem lies only with the motherboard manufacturer, if they have the code in the BIOS to bifurcate then you can make a PCB for it.

PLX chips are BGA packages, yes you can solder it, but not with your regular soldering iron. Also because the PLX chip is a rather advanced switching chip, implementing it isn't easy, there is a lot of validation required to be sure it works properly.

The best route to go is to build a PCIe riser card to duplicates the Reference clock.
I've been wanting to design the PCB for it and I think I know what it needs, but because I couldn't find a ITX board that has a BIOS for PCIe bifurcation I haven't bothered to give it a try.
Click to expand...

Interesting. I'll be curious to see how the Asrock X99 ITX testing goes since they have a hardware rep confirming bifurcation support.

As for building a PCIe riser to duplicate the reference clock, do you mind me asking what would be involved?
 
It seems that the only thing you really need to do is multiply the reference clock signal.
All the other signals can be daisychained as seen in the Supermicro example. The only thing the added to the PCIe fingers was more power and 2 more RefClks.

Something like the IDT 9DB233 should be able to double up the Reference Clock signals from the looks of it.

You mentioned you had tried the ARC1-PELY423-C7 and the RSC-R2UT-2E8R.
Could you identify what chip is on those 2 cards? It might be that they have the same type of chips.
 
QinX said:
In order to have PCIe Bifurcation to work for 2 PCIe devices you need the following:

1) Bios Support, the Bios needs to have th code to detect en change the PCIe lane assignments from 16x to 8x/8x if the code is not there it doesn't work

2) Each PCIe device needs a Reference Clock. Multi PCIe boards have multiple Reference clocks 1 for each PCIe slot. In order to get more then 1 PCIe reference clock into 1 PCIe slot you need either

A) deviate from the PCIe specification and use some of the pins to bring in another reference clock. This is what SuperMicro does.

B) Multiplex the Reference clock on the PCIe riser PCB.

I've had a look into this as well with the same idea of running X99 ITX with 2 GPUs. I've measured all of the pins on this SuperMicro card and in my excel sheet below you can see they hacked up the PCIe slot specification to get all the signals in, that is why they only work on a select number of their own server boards.

RSC-R2UG-2E4E
$T2eC16F,!ygE9s7HHghgBQGTInErNg~~60_1.JPG


The actual pin-out
full.png


The other option I've looked into is the PLX chip, but those will be very expensive, this is old information but these things can cost as much as $80, plus the validation that needs to be done because of the switching involved. But with a PLX chip you can get any dual GPU setup to run on a single PCIe slot, but that riser will more then likely costs $100+
http://www.anandtech.com/show/6170/four-multigpu-z77-boards-from-280350-plx-pex-8747-featuring-gigabyte-asrock-ecs-and-evga/31
Click to expand...

Qinx, did you try to tackle this same issue a while back on another forum? I remember reading through your thread hoping that you'd come ups with a solution but I do not believe you did? Anyhow, I'd like to re-ignite this topic and get things working!!! once and for all. I can front some of the cost for the PLX chip and dev cost if that helps, I really want to get a dual 970 GTX working on my asrock x99e board and also get it working on a Hackintosh :) What can I do to help??
 
  • Like
Reactions: D10S
like this
jb1 said:
Unfortunate! By any chance do you have the width of the card in addition to the length?

Maybe this isn't really a possible option given QinX's post though...



I wonder, can consumers even purchase PLX chips and soldier the PCI-E connections to the chip?
Click to expand...

It's about 2.5 inches in height and 6.8 in length. It did not work for me though.
 
chemist_slime said:
Qinx, did you try to tackle this same issue a while back on another forum? I remember reading through your thread hoping that you'd come ups with a solution but I do not believe you did? Anyhow, I'd like to re-ignite this topic and get things working!!! once and for all. I can front some of the cost for the PLX chip and dev cost if that helps, I really want to get a dual 970 GTX working on my asrock x99e board and also get it working on a Hackintosh :) What can I do to help??
Click to expand...

I've had a topic going over on EEVblog, but they told me it wasn't possible, but what do they know right ;) .

I'm considering to make the PCBs and such for it to check it out, only thing is I don't have the money right now to try it out. I'll take a look into design the PCB and then seeing if there is interest from the community. Maybe I can get ASRock onboard to sponsor me a X99 ITX board.

The cheapest way to go is going with the ASRock X99 ITX board because if it has PCIe bifurcation the components needed to get it working shouldn't be that expensive.

You have 2 types of cases for this.
PCIe perpendicular
Where the cards are at 90 degrees with the motherboard, you'd need a 5 slot case for aircooled cards or 3 slots for watercooled cards that have single slot I/O minimum. The riser also needs to be flexible, so you'd need 2 PCBs

PCIe parallel
The cards are inline with the motherboard the PCIe slot needs to make a 90 degree turn. This could allow for 1 PCB and has a minimum requirement of 4 PCIe brackets for aircooled and 2 for watercooled cards.
 
QinX said:
I've had a topic going over on EEVblog, but they told me it wasn't possible, but what do they know right ;).
Click to expand...

Yeah! If AMD and NVIDIA can implement this on their dual GPU cards then it has to be possible to do... doesn't mean it will be easy though.

QinX said:
I'm considering to make the PCBs and such for it to check it out, only thing is I don't have the money right now to try it out. I'll take a look into design the PCB and then seeing if there is interest from the community. Maybe I can get ASRock onboard to sponsor me a X99 ITX board.
Click to expand...


This sounds like a really fun project–– how much do you think it would cost to get a prototype made without the PLX chip assuming Asrock bifurcation support?

QinX said:
You have 2 types of cases for this.
PCIe perpendicular
Where the cards are at 90 degrees with the motherboard, you'd need a 5 slot case for aircooled cards or 3 slots for watercooled cards that have single slot I/O minimum. The riser also needs to be flexible, so you'd need 2 PCBs

PCIe parallel
The cards are inline with the motherboard the PCIe slot needs to make a 90 degree turn. This could allow for 1 PCB and has a minimum requirement of 4 PCIe brackets for aircooled and 2 for watercooled cards.
Click to expand...

I was actually considering having both cards parallel to the motherboard, one right behind the other, both mini ITX length. Only issue is that one card wouldn't have accessible ports, but it would allow for the thinnest design.

chemist_slime said:
I can front some of the cost for the PLX chip and dev cost if that helps, I really want to get a dual 970 GTX working on my asrock x99e board and also get it working on a Hackintosh :)
Click to expand...

Note that SLI is not supported in OS X. Both cards will register in OS X so you can use them for CUDA or OpenCL compute, but they won't operate together in a gaming situation.
 
jb1 said:
Yeah! If AMD and NVIDIA can implement this on their dual GPU cards then it has to be possible to do... doesn't mean it will be easy though.
Click to expand...
AMD and Nvidia both implement the PLX chip, but this is more because their cards have to work with ALL motherboards, so even the ones that don't have PCIe bifurcation. They also have a lot more resources for it.

jb1 said:
This sounds like a really fun project–– how much do you think it would cost to get a prototype made without the PLX chip assuming Asrock bifurcation support?
Click to expand...
Like most things this scales with quantities, making 1 is probably not worth it, making 5+ reduces price a lot. a guestimate would be that if I'd make around 10 boards you'd pay 40 bucks for each.


jb1 said:
I was actually considering having both cards parallel to the motherboard, one right behind the other, both mini ITX length. Only issue is that one card wouldn't have accessible ports, but it would allow for the thinnest design.
Click to expand...

Do you mean turning two 17cm cards into one 34cm card? The downside is with Nvidia, you can't SLI because the bridge won't reach.

Of course because this has never been done before decide on the orientation isn't easy. Most ITX cases have cards perpendicular to the motherboard so that would mean a flexible riser, doable but more expensive then the parallel version, however with the parallel version you have the issue that most ITX only have 2, maybe 3 slots available where you'd need 4 to get this working with aircooled cards.

Going for watercooled cards you only need 2 PCIe slots if you use either a reference GTX970 or a Fury X. Not the cheapest cards but if you already have the X99 ITX board you probably aren't looking at a GTX960.

Going with aircooled cards means a minimum of 4 PCIe slots are needed. I could design a chassis for it, but then we go to a whole other level.
 
QinX said:
Like most things this scales with quantities, making 1 is probably not worth it, making 5+ reduces price a lot. a guestimate would be that if I'd make around 10 boards you'd pay 40 bucks for each.

Do you mean turning two 17cm cards into one 34cm card? The downside is with Nvidia, you can't SLI because the bridge won't reach.
Click to expand...

Well, it sounds like there are at least three here who might want something like this!

And yes, kindof. I was actually planning something even crazier, mounting one normally, and the other pointing vertically to reduce the length.

This would probably be done with AMD, since CrossFireX works right over PCI-e
 
jb1 said:
Don't really see anything insightful in that thread...
Click to expand...

Well I was just showing some people got it working correctly on a supermicro board w/ non supermicro PCIe extenders, and listed the part numbers they used. Whether those same part numbers work on an Asrock board is another story.
 
Diverge said:
Well I was just showing some people got it working correctly on a supermicro board w/ non supermicro PCIe extenders, and listed the part numbers they used. Whether those same part numbers work on an Asrock board is another story.
Click to expand...

Ah got it, must have missed that when reading through. Sorry!
 
Ofcourse any configuration can be made, but for just 1 piece you can count on it costing some where between $150 and $250, because making a single PCB isn't cheap, it has to be a 4 layer PCB and you'd need gold plating on it.

I think it's best if we approach it from a somewhat "normal" use case.

Having a perpendicular setup is a no go imho. because if you have them perpendicular why even have the ITX formfactor because you'd be right at the mATX size. Unless you go watercooled single slot cards, you'd might be able to cram 2 cards into the NCASE M1 or a Bitfenix Prodigy

If you'd go parallel on the side of the motherboard you can have 2 cards in 4 slots. the height of the cards also allows for a ATX PSU in the front. You'd need a flexible PCIe riser.

Watercooling the cards and CPU brings a lot of options, although you'd need space for the radiator.

@chemist_slime

Would it be possible for you to make some high-resolution photo's of all the different riser you have? Front and Back if possible.
I'd like to to a look at the layout and tracing.

You mentioned you tried the supermicro card and it didn't boot?
You mean Supermicro RSC-R2UG-A2E16-A didn't work?
Could you remove that heatsink and check if it's a PLX chip, because if it is it should work without a problem, unless they also f****d up the signal like the did with my card, which I bought because I want to retrace all the signals.
 
Last edited:
QinX said:
Ofcourse any configuration can be made, but for just 1 piece you can count on it costing some where between $150 and $250, because making a single PCB isn't cheap, it has to be a 4 layer PCB and you'd need gold plating on it.

I think it's best if we approach it from a somewhat "normal" use case.

Having a perpendicular setup is a no go imho. because if you have them perpendicular why even have the ITX formfactor because you'd be right at the mATX size. Unless you go watercooled single slot cards, you'd might be able to cram 2 cards into the NCASE M1 or a Bitfenix Prodigy

If you'd go parallel on the side of the motherboard you can have 2 cards in 4 slots. the height of the cards also allows for a ATX PSU in the front. You'd need a flexible PCIe riser.

Watercooling the cards and CPU brings a lot of options, although you'd need space for the radiator.
Click to expand...

Yes, definitely agreed normal use case would be best. I would implement my idea with flexible risers off of this PCB.

Also agree parallel to the motherboard is the way to go, otherwise, yes, you have an mATX board! Maybe best to assume flexible risers will be in use so people can get the GPU's where they want them depending on the case they are using.
 
If you're planning on doing your own PCB you should review the PCIe card electromechanical spec. It covers things like differential skew requirements (less than 0.127mm), impedances, etc.

You should be able to google it.
 
jb1 said:
Yes, definitely agreed normal use case would be best. I would implement my idea with flexible risers off of this PCB.

Also agree parallel to the motherboard is the way to go, otherwise, yes, you have an mATX board! Maybe best to assume flexible risers will be in use so people can get the GPU's where they want them depending on the case they are using.
Click to expand...

I'll look into making a draft of de PCB including a "optimal" computer case layout.

jimmyb said:
If you're planning on doing your own PCB you should review the PCIe card electromechanical spec. It covers things like differential skew requirements (less than 0.127mm), impedances, etc.

You should be able to google it.
Click to expand...

I've made a PCIe Riser before and it worked fine. unless you are going to do some weird stuff, such as PLX chips or flexible risers you need to look out.

I've gathered most of the important info from that last riser PCB.
 
Nice, looking forward to seeing what you come up with! I guess we're assuming that Asrock will come through on proper bifurcation...
 
I've done some quick drafting in CAD to get a feeling at what kind of dimensions we are looking at.
If you stick to a SFX PSU, Silverstone has a 700W version coming, I can stay just shy of 10 liters, a little smaller. Rough dimensions would be 35cm*30cm*9cm=9.54 liters

You'd be able to run 2x GTX 980 ti without too much trouble.

KjnA476.jpg


kvcbMYL.jpg
 
jb1 said:
Nice, looking forward to seeing what you come up with! I guess we're assuming that Asrock will come through on proper bifurcation...
Click to expand...

It's funny how a company like ASrock, that had the image of a cheapo subbrand of ASUS a few years ago is now the leading innovator in mITX boards.
They are the ones we suspect to do things others don't have the guts for, and they deliver. Goes to show what you can make of a company if you put work and creativity into it.
 
QinX said:
I've done some quick drafting in CAD to get a feeling at what kind of dimensions we are looking at.
If you stick to a SFX PSU, Silverstone has a 700W version coming, I can stay just shy of 10 liters, a little smaller. Rough dimensions would be 35cm*30cm*9cm=9.54 liters

You'd be able to run 2x GTX 980 ti without too much trouble.

[...]
Click to expand...

Why would you need the two additional risers if you're going with the Impact VII? Other than that, freaking awesome!
 
iFreilicht said:
Why would you need the two additional risers if you're going with the Impact VII? Other than that, freaking awesome!
Click to expand...

It's a placeholder board, I don't have a ASRock x99e-itx/ac 3D model. The 2 additional risers are there to space the PCIe brackets further away from the board, without some extra clearance the PCIe bracket might interfere with audio ports.
The other option is to have a flexible riser like you are using in Brevis S and flip the GPUs. But then the question would be if the signal stays good, so at least for now I'm designing with this layout in mind.
 
I've attached below some images of the 3 PCI-E risers that I've used in conjunction with the Asrock x99e-iTX that did not work. Hopefully some of the images will help. I have yet to take off the heatsink on the Supermicro one, will do that tomorrow in the evening when I get back from work.

I have also PM'ed the tech support person who replied to the PCI-E bifurcation support on the x99e motherboard. He has read the PM but has yet to respond. Asrock support has not yet responded either. I will post results when they do. Let's do this!

MinH0ZV.jpg

qBe76G7.jpg

aVZDNIj.jpg

G8NqMsz.jpg

FKR88Ev.jpg

DYBObj1.jpg
 
You must log in or register to reply here.
Back
Top