PCIE Bifurcation

jb1

Limp Gawd
Joined
Dec 27, 2014
Messages
500
Hey, jb1 - I saw your post on ASRock forum about the bifurcation on Z170 Gaming-ITX. Did you manage to get a direct answer from them? As I see that the thread wasn't answered...
Nope, they never replied. I just bumped the thread over there in hopes that they get back about it.

From the manual for this board it doesn't seem like there are any options for bifurcation, which seems to suggest that the BIOS at publication didn't support it. However, I remember seeing something that suggested that they would add support for 100 series chipsets.

I ended up going the X99 route!
 

bleedinge

n00b
Joined
Feb 2, 2011
Messages
2
I'm the guy who originally asked in the Asrock forums about the Z97E-ITXac and had the thread deleted.
I needed that particular board because I need the board to have its own video out port for the Intel graphic card.

I got to talk to some guys from Asrock NL and they told me that they modify the board at the factory, and it looks like it has a different code: Z97E/ITX/ACN.

I just needed one board for some tests but finally gave up on it.

Let's hope they live up to their word and some Z170 board provides bifurcation support.
 

jb1

Limp Gawd
Joined
Dec 27, 2014
Messages
500
I'm the guy who originally asked in the Asrock forums about the Z97E-ITXac and had the thread deleted.
I needed that particular board because I need the board to have its own video out port for the Intel graphic card.

I got to talk to some guys from Asrock NL and they told me that they modify the board at the factory, and it looks like it has a different code: Z97E/ITX/ACN.

I just needed one board for some tests but finally gave up on it.

Let's hope they live up to their word and some Z170 board provides bifurcation support.
Wait, so are you saying there is some special Z97 Asrock variant (Z97E/ITX/ACN) that has PCIe bifurcation support?
 

bleedinge

n00b
Joined
Feb 2, 2011
Messages
2
Yes, and it's actively being sold on request. That's the model name Asrock referred to when I asked them about it.

It looks like you can get it from Ameri-Rack too, as the guy there told me he had just sold 5K modified boards to a client.
 

KazeoHin

Supreme [H]ardness
Joined
Sep 7, 2011
Messages
8,084
Okay, instead of reading through ten pages:

Has anyone here accomplished PCI-E Bifurcation yet? Last I checked it was many pages of maybes and nopes.
 

QinX

Limp Gawd
Joined
Jan 15, 2015
Messages
230
Okay, instead of reading through ten pages:

Has anyone here accomplished PCI-E Bifurcation yet? Last I checked it was many pages of maybes and nopes.
Instead of explaining in detail then, yes, on asrock x99e-itx, PCI-e bifurcation is possible with a simple splitter and the correct BIOS.
 

extide

2[H]4U
Joined
Dec 19, 2008
Messages
3,494
Hey guys, I think you are missing an important point. PCIe bifurcation is a lot more common than you think. It is indeed the method behind boards that have, for example, 2 16x slots that become x8/x8 when 2 cards are inserted. The clock splitter is on the motherboard, along with the physical switches that move the lanes to the other slot when a second card is inserted.
 

jb1

Limp Gawd
Joined
Dec 27, 2014
Messages
500
Hey guys, I think you are missing an important point. PCIe bifurcation is a lot more common than you think. It is indeed the method behind boards that have, for example, 2 16x slots that become x8/x8 when 2 cards are inserted. The clock splitter is on the motherboard, along with the physical switches that move the lanes to the other slot when a second card is inserted.
Right, I think what everyone means is PCIe bifurcation on mITX motherboards. This requires BIOS support to split the lanes on the one 16x slot, which is not common on consumer boards.
 

KazeoHin

Supreme [H]ardness
Joined
Sep 7, 2011
Messages
8,084
Instead of explaining in detail then, yes, on asrock x99e-itx, PCI-e bifurcation is possible with a simple splitter and the correct BIOS.
That's a maybe. Not a "yes, someone on this forum has accomplished this"

So I ask again. Has anyone ACCOMPLISHED this yet?
 

KazeoHin

Supreme [H]ardness
Joined
Sep 7, 2011
Messages
8,084
Yes, on the Asrock X99 ITX board with a splitter and the correct BIOS.
AWESOME! So this means Dual-Card SFF is possible...

Imagine a PC that is exactly 7" x7" x7" and houses two ITX cards in SLI/CFX and a 6-core Haswell-E CPU... That would be a tiny beast.
 

extide

2[H]4U
Joined
Dec 19, 2008
Messages
3,494
Right, I think what everyone means is PCIe bifurcation on mITX motherboards. This requires BIOS support to split the lanes on the one 16x slot, which is not common on consumer boards.
I know, I am just saying that VGA card manufacturers probably DO test to make sure their cards work with bifurcation, as it's not some super rare feature that's never used in the wild, but something pretty common, even if not common on mini ITX.


And also some of you guys are curious as to why cards like a R9 295x2 or a GTX 690 "just work" and I bet that it's because the BIOS on the video card itself has stuff in it to make things go a bit more smoothly. Where as the BIOS on a single gpu video card isnt going to have stuff like that. Just a guess. I am talking about how you needed that custom bios to make even the splitter with the PLX chip work. -- The original BIOS you had probably had some issues with the PCIe spec in general. Perhaps the VGA BIOS on some of those dual gpu cards can help alleviate those issues, there might be some known workarounds for certain common issues or something.
 

EdZ

Gawd
Joined
Jan 28, 2008
Messages
773
The BIOS support is needed more on the motherboard side than the GPU side.
 

jtd871

Weaksauce
Joined
Feb 11, 2015
Messages
91
I would expect that dual-GPU cards do not require bifurcation of the single slot. All the magic happens on the card. The system sees only 1 PCIe device.
 

extide

2[H]4U
Joined
Dec 19, 2008
Messages
3,494
The BIOS support is needed more on the motherboard side than the GPU side.
Yes, but it is curious why the dual gpu cards work out of the box in basically any mobo, where as a bios update was needed earlier in the thread to get a similar setup working (two gpu's + PLX chip)

I would expect that dual-GPU cards do not require bifurcation of the single slot. All the magic happens on the card. The system sees only 1 PCIe device.
They don't use bifurcation, they use a PLX chip. Although, the system does see two PCIe devices.
 

EdZ

Gawd
Joined
Jan 28, 2008
Messages
773
Yes, but it is curious why the dual gpu cards work out of the box in basically any mobo, where as a bios update was needed earlier in the thread to get a similar setup working (two gpu's + PLX chip)
What was gotten working earlier with the specific BIOS for the X99 ITX was the use of a splitter without a PLX chip. AFAIK, nobody has gotten multiple single cards to work over a PLX-chip splitter.
They don't use bifurcation, they use a PLX chip. Although, the system does see two PCIe devices.
The PLX chip is what allows them to work without BIOS support from the motherboard.

PCIe Bifurcation (no PLX on the splitter): needs to be supported in the BIOS of the motherboard. Should in theory be agnostic of the PCIe devices, but may not be in practice (e.g. issues people occasionally have with motherboards that use bifurcation to rearrange PCIe lanes).
PLX splitter: Bifurcation support not required by motherboard BIOS, but work needs to be done by card vendor to ensure cards play nicely with the PLX chip.

tl;dr: Bifurcation is Motherboard-dependant & card agnostic, PLX chip splitters are card-dependant and motherboard-agnostic.
 
  • Like
Reactions: al007
like this

extide

2[H]4U
Joined
Dec 19, 2008
Messages
3,494
What was gotten working earlier with the specific BIOS for the X99 ITX was the use of a splitter without a PLX chip. AFAIK, nobody has gotten multiple single cards to work over a PLX-chip splitter.
No, actually, the BIOS update went from no splitters working (PLX chip or not) to all splitters working (2 w/out PLX, and one with).

That's why it's kinda curious.

EDIT: Here is the post:
Alright guys, I just tested all 3 risers with the 1.20E bios and I'm happy to report that all 3 now work! I am elated right now, I have poured countless hours into researching this and pretty much found nothing online except speculation and now it all works. Let's this day 7/31/2015 be known that we have the first working PCIE bifurcation support for a mini-ITX board and my dream build machine can be achieved! However, what's puzzling is that the PLX riser from super micro also now works although previously it did not without the bios support. So it seems that some bios support is indeed needed even if a PLX chip is used!

@Qinx, looks like you no longer need to create your own custom board unless you want a specific PCB layout that isn't achievable with a flexible riser. :)
 

EdZ

Gawd
Joined
Jan 28, 2008
Messages
773
Whoops, I completely forgot chemist_slime had PLX a riser to test too. A few possibilities then:
- While messing with PCIe bifurcation support, ASRock incidentally fixed a bug that was preventing the Supermicro PLX riser from working (its HCL even with Supermicro boards is pretty narrow, so it may have some Weirdness going on in it's implementation)
- ASRock were working on supporting that particular riser (or PLX chip) for some unrelated reason, and just happened to roll in that code into the BIOS update that also enabled PCIe bifurcation support
- That Supermicro riser PLX has some sort of 'Bifurcation fallback mode' that got enabled when Bifurcation was exposed
- General Weirdness from dealing with ultra-high-speed busses
 

jb1

Limp Gawd
Joined
Dec 27, 2014
Messages
500
@chemist_slime,

How's your project going? Any progress on the thin liquid in/out adapter? How's the 12 core holding up?
 

jamall

n00b
Joined
Oct 19, 2015
Messages
1
Wow, I've learnt more in the last 30 minutes reading these eleven pages than I was able discover in three weeks of scouring some of the loneliest corners of the internet, enjoying classics like PCI Express Base Specification Revision 3.1 and Designated Vendor-Specific Extended Capability ECN. It seems to be a bit of a meeting of the minds here - you guys have unravelled the answers to questions that have had most of the rest of the world completely stumped for years.


I've run out of PCIe slots on my Apple Xserve and have had to spread my fibre channel card, SAS RAID card and video cards between two Xserves to maintain all the functionality I need. I want to consolidate everything back into one box and maybe add some fast solid state storage into the mix, and seeing as most of my applications only require a x4 connection I figured it would be cheap and easy. How wrong I was!

I almost pulled the trigger on a Magma box before I realised it didn't come with a host interface card or cable, and how rare and expensive replacements are. I could find plenty of Nvidia Tesla host interface cards going quite cheaply, sporting the same connector and performing essentially the same function, but no information whatsoever about interconnect standards and whether the two company's systems would be interchangeable.

I've got some bits and pieces to start experimenting with on the way, but in the meantime I have some unexpected news to announce: there is actually one slightly useful page on the Supermicro website after all. No clues as to which chips are on which cards but it makes sorting the wheat from the chaff a little bit easier. If chemist_slime has any trusting, Mac owning friends I'd love to know whether that active Supermicro card is recognised properly in any Mac Pro or Xserve slots, but if the project you're undertaking now is any indication I'm guessing people started hiding their IT gear and swallowing their thumb drives when they heard you were coming a long time ago :eek:
 
Last edited:

jb1

Limp Gawd
Joined
Dec 27, 2014
Messages
500
@jamall, it's great to see interest on the Mac side of things for PCIe bifurcation! I think a tricky thing with macs is that you don't have any access to the BIOS–– the most reliable solution right now seems to be Asrock's X99 ITX BIOS which let's you pick how you want to split up your PCIe lanes.

That being said, with a PLX chip on your splitter it should in theory be possible. I unfortunately don't have access to a Mac Pro, but hopefully someone on here might! Not sure what your tolerance for costs are, but these splitters aren't too expensive in the grand scheme of things when you're working on an Xserve so you could try yourself and I imagine little chance of damaging your components.
 

jb1

Limp Gawd
Joined
Dec 27, 2014
Messages
500
@chemist_slime, Have you gotten the PCIe splitters to work with the 1.40 X99E-itx/ac BIOS from Asrock? My board came with this BIOS and will downgrade to P1.20E if 1.40 doesn't enable it. Is there a setting somewhere in the BIOS that lets you pick your bifurcation settings?
 

chemist_slime

[H]ard|Gawd
Joined
Jul 24, 2005
Messages
1,100
@chemist_slime, Have you gotten the PCIe splitters to work with the 1.40 X99E-itx/ac BIOS from Asrock? My board came with this BIOS and will downgrade to P1.20E if 1.40 doesn't enable it. Is there a setting somewhere in the BIOS that lets you pick your bifurcation settings?
jb1, I haven't gotten a chance to do 1.40, have just been super swamped at work. But in 1.20E in Bios there was indeed a place where you could set the split level. It's very visible, you can't miss it.
 

jb1

Limp Gawd
Joined
Dec 27, 2014
Messages
500
jb1, I haven't gotten a chance to do 1.40, have just been super swamped at work. But in 1.20E in Bios there was indeed a place where you could set the split level. It's very visible, you can't miss it.
Got it, in that case I don't think bifurcation made it into 1.40!
 

veryrarium

Weaksauce
Joined
Apr 12, 2014
Messages
125
Digging up an old post...
PS:
I've just sent an email to Avago with the question(simplified):
Do your PLX chips needs any hardware or software/BIOS support from a motherboard in order to work.

If it is no, then we know that the X99e just had a bit of flimsy PCIe code in the BIOS. That could happen on any motherboard.
If it is yes, then lets hope we get a nice detailed answer of what or why
QinX, have you gotten a reply from Avago? Curious to know their answer.
 

chemist_slime

[H]ard|Gawd
Joined
Jul 24, 2005
Messages
1,100
Got it, in that case I don't think bifurcation made it into 1.40!
@jb1: Just had dinner with the ceo and one of the founders of ekwb, they were somehow connected with the ceo of our company by simply being slovenian! I guess for a country of 2million, everyone knows everyone. Anyways, I told him what I was building and they're creating the custom part for me, maybe they'll create a few and I'll simply mail you one! Fingers crossed!
 

iFreilicht

[H]ard|Gawd
Joined
Sep 23, 2014
Messages
1,348
@jb1: Just had dinner with the ceo and one of the founders of ekwb, they were somehow connected with the ceo of our company by simply being slovenian! I guess for a country of 2million, everyone knows everyone. Anyways, I told him what I was building and they're creating the custom part for me, maybe they'll create a few and I'll simply mail you one! Fingers crossed!
You're a lucky man, you know that?
 

Sanguineus

n00b
Joined
Aug 14, 2014
Messages
35
@jb1: Just had dinner with the ceo and one of the founders of ekwb, they were somehow connected with the ceo of our company by simply being slovenian! I guess for a country of 2million, everyone knows everyone. Anyways, I told him what I was building and they're creating the custom part for me, maybe they'll create a few and I'll simply mail you one! Fingers crossed!
If they have some more, I would be very grateful if I could purchase one ;) I am also contemplating a build with two Radeon Nanos in NCase M1. A custom bridge to join two waterblocks next to each other would come in handy... My NCase is already sitting in the box for over a year ;) And I am still wondering if it's not worth waiting for the next gen video cards. I set myself a goal of building a PC capable of 60 FPS in 4K but not only for current games, but let's say I would like it to be at least a bit futureproof. I have really high hopes for the next generation of AMD/nVidia cards - if they come in small form factor, like the Nano, maybe it will be possible to use just one card to achieve my goal, but so far - this waterblock bridge could be useful ;)
 
Last edited:

jb1

Limp Gawd
Joined
Dec 27, 2014
Messages
500
@jb1: Just had dinner with the ceo and one of the founders of ekwb, they were somehow connected with the ceo of our company by simply being slovenian! I guess for a country of 2million, everyone knows everyone. Anyways, I told him what I was building and they're creating the custom part for me, maybe they'll create a few and I'll simply mail you one! Fingers crossed!
That's incredible! If you could convince them to make a few that would be wonderful... I'd be gladly willing to pay for a part like this! I've already got the 12 core X99 build up and running, so this is the final step.

In the mean time, I've been talking to ekwb tech support to try and get the exact dimensions of their in/out flow port connector so we could continue with the design. I haven't gotten any detailed specifications yet, should have gone straight to the CEO! ;)
 

chemist_slime

[H]ard|Gawd
Joined
Jul 24, 2005
Messages
1,100
That's incredible! If you could convince them to make a few that would be wonderful... I'd be gladly willing to pay for a part like this! I've already got the 12 core X99 build up and running, so this is the final step.

In the mean time, I've been talking to ekwb tech support to try and get the exact dimensions of their in/out flow port connector so we could continue with the design. I haven't gotten any detailed specifications yet, should have gone straight to the CEO! ;)
Yea, I'll see what I can do. They seem willing to make a few for 'friends'. What 12 core processor are you using? The 2676 V3?
 

jb1

Limp Gawd
Joined
Dec 27, 2014
Messages
500
Yea, I'll see what I can do. They seem willing to make a few for 'friends'. What 12 core processor are you using? The 2676 V3?
That's great! I'm assuming that the in/out flow port on the top of these EK blocks can be removed–– I actually haven't ever purchased one, but am I right to assume that it can be done easily?

Was able to find a good deal on a 2690 V3.

I remember you were looking into running OS X–– everything runs well on 10.10.5! The biggest concerns I had were sleep/wake and speed stepping, both of which work if USB 3 is disabled. Turbo, however, does not work. Temperatures are ~15 degrees hotter at idle than in Windows, making me think that voltages are kept high even with low clocks at idle due to ineffective power management. It would be great to get 10.11 running, but it looks like legacy PCI kexts from 10.9.5 are necessary for booting and they don't seem to work in 10.11 as far as I can tell. I'm hoping Apple releases a refresh Mac Pro soon so full support is within reach!
 

chemist_slime

[H]ard|Gawd
Joined
Jul 24, 2005
Messages
1,100
That's great! I'm assuming that the in/out flow port on the top of these EK blocks can be removed–– I actually haven't ever purchased one, but am I right to assume that it can be done easily?

Was able to find a good deal on a 2690 V3.

I remember you were looking into running OS X–– everything runs well on 10.10.5! The biggest concerns I had were sleep/wake and speed stepping, both of which work if USB 3 is disabled. Turbo, however, does not work. Temperatures are ~15 degrees hotter at idle than in Windows, making me think that voltages are kept high even with low clocks at idle due to ineffective power management. It would be great to get 10.11 running, but it looks like legacy PCI kexts from 10.9.5 are necessary for booting and they don't seem to work in 10.11 as far as I can tell. I'm hoping Apple releases a refresh Mac Pro soon so full support is within reach!
Which installation instructions did you use to install 10.10.5?
 
Joined
Dec 1, 2015
Messages
3
Hello, long time lurker here. I recently got into contact with Ameri-rack and they do in fact sell a bifurcation variant of the z97e-itx under the designation z97e-itx/acn. Sales say it's $155 per unit, but you need to contact them in order to get a link to purchase it.
Have asked about a z170 variant but sales haven't gotten back to me yet. I'll update when I hear back.
 
Joined
Dec 1, 2015
Messages
3
Hi sorry for the double post, Ameri-rack got back to me. According to the person at sales the z170m-itx/ac supports bifurcation if you update the bios.

Edit: He says this holds true for the gaming variant as well. Does Bios 1.50 have any bifurcation settings available?
 
Last edited:

jb1

Limp Gawd
Joined
Dec 27, 2014
Messages
500
Hi sorry for the double post, Ameri-rack got back to me. According to the person at sales the z170m-itx/ac supports bifurcation if you update the bios.

Edit: He says this holds true for the gaming variant as well. Does Bios 1.50 have any bifurcation settings available?
Good to hear Asrock is supporting bifurcation across multiple products!
 
Joined
Dec 1, 2015
Messages
3
Shame that a modified version of the board is needed for z97, but I guess it's the price I pay for wanting broadwell :c
 

Phuncz

2[H]4U
Joined
Apr 12, 2009
Messages
2,630
Hi sorry for the double post, Ameri-rack got back to me. According to the person at sales the z170m-itx/ac supports bifurcation if you update the bios.

Edit: He says this holds true for the gaming variant as well. Does Bios 1.50 have any bifurcation settings available?
With the Z87E-ITX I have, AmeriRack provided a specific BIOS because it wasn't available on the AsRock website.
 
Top