Fall 2015 Solid State Drive Technology Update @ [H]

[Trimlock]

> For those MB's that don't connect the M.2 or certain PCIe slots to the PCH you won't be limited by the DMI link.

Can you provide examples from ASUS, Gigabyte and MSI, please?

If those "certain PCIe slots" are not connected to the PCH,
what are they connected to? PLX chips?

I'd like to examine their technical documentation very closely
(and hopefully NOT run into a slew of typos


MRFS

Sorry, I was misunderstanding your question at first. You do have a good understanding of how storage communicates with the cpu.

As for the PLX that should only be used as a multiplexer with certain ports, like USB3.1/C. I am unaware of any different set ups.

As for being limited by the DMI link, you should be fine for storage, retrieving is where your bottleneck will occur.

 

[Hakuren]

At this moment in time it is really difficult to make heads or tails with storage options. Except venerable HDDs other things looks like in the state of flux.

Next year should bring some really exciting stuff to the PC market. One thing I can't see myself using in any capacity at current price tag is TLC. Basically it's kind of NAND which hates anything remotely substantial. Best used for data which is not moved at all or very rarely (like video libraries). But... and it's a big, fat but. Not with pricing as at the moment. For 1/3rd of price I would take the risk, but there is no chance in hell going with TLC in any capacity if it cost as much as - miles more reliable - MLC. Putting NVMe aside (which is great and using it everyday with good effect on very large directories where ordinary NAND is choked to death) I can't see ordinary NAND-SSD living for much more (by ordinary I also include 3D/V NAND). It's just trying to avoid the inevitable which is EoL for this particular technology.

Right now X-Point (Optane?!? what a stupid name) is way forward. And it will also spark certain evolution/revolution in motherboard designs. With more and more lanes required for storage to run and SSDDIMMs coming to PC near you soon, motherboard with 40 DIMM slots doesn't sound that ridiculous anymore. SATA ports will be dropped (except for some rudimentary, legacy connectivity).

 

[Zarathustra[H]]

I don't think TLC is anything to be scared of.

The good TLC drives today have similar or better write endurance than most MLC drives did a couple of years ago.

There is a reason SLC is mostly gone now, even in enterprise circles.

Today's MLC is as reliable as yesterday's SLC. Today's TLC is taking the place of yesterday's MLC.

 

[wirk]

Right now X-Point (Optane?!? what a stupid name) is way forward. And it will also spark certain evolution/revolution in motherboard designs. With more and more lanes required for storage to run and SSDDIMMs coming to PC near you soon, motherboard with 40 DIMM slots doesn't sound that ridiculous anymore. SATA ports will be dropped (except for some rudimentary, legacy connectivity).

SSDDIMMs is the future but it will require new systems and apparently will not be available for PCs, at least in the beginning: It will also ship memory DIMMs for use as main system memory in servers, though it wasn’t immediately clear if they’re coming in 2016 as well. The DIMMs will work with future Intel Xeon server processors. So PC SSDDIMMs are a couple of years away.

 

[GoodBoy]

Here's an MSI chipset block diagram, illustrating the placement of an x4 "switch" in the X99 chipset:

http://www.anandtech.com/show/8557/...rock-x99-ws-msi-x99s-sli-plus-intel-haswell-e

Scroll down to: "as shown in this MSI chipset diagram"


Without owning a Z170 motherboard, I'm still a little confused
by some conflicting advice from above: can I install a RAID controller
that uses the CPU lanes instead of the PCH lanes in the Z170 chipset?

If so, can someone point me to a motherboard make and model, please?

MRFS

I think the terminology on this one might be confusing: http://media.bestofmicro.com/5/9/515997/original/Intel-Z170-chipset-block-diagram.jpg

It mentions "PCIe 3.0 Graphics" on the 16 lanes going directly to the cpu. But they can be used for anything. Just plug a PCIe raid card in. It will all boil down to what you plug into the PCIe slots, and which slots, the layout of which will vary from mobo to mobo.
Typically for what you are wanting to do, a GPU will eat x8, and a raid card in the second x8 will finish off the PCIe slots. the remainder are in the DMI3 link for usb, ethernet, etc.

So you can get an x8 PCIe 3.0 worth of bandwidth for either a raid card, or add-in ssd card.

It's more than enough, and even being limited to the x4 worth of bandwidth will be fine for pretty much any desktop setup.

If you are doing something else that really needs to chew thru 16Gb/sec of bandwidth, you shouldn't be buying Z170, and instead be looking at workstation mobo's or server setups.

 

[evilsofa]

p.s. I would prefer to tell technology vendors what I want,
rather than to work with the stuff they think I should have.

You gain that ability sometime after you earn your first few billion dollars. See: Elon Musk

 

[GoodBoy]

> a GPU will eat x8, and a raid card in the second x8 will finish off the PCIe slots

Thanks for that.

My personal focus is to "push" the technology as far as possible,
to find and experimentally confirm the real limits aka MAX HEADROOM.

But, I want a RAID card with an x16 edge connector
to be installed in the first x16 PCIe slot --
NOT an x8 edge connector, and NOT in any other slots.

Think of this exercise as "overclocking storage" in a manner very similar
to overclocking CPUs and DRAM. I wrote a paper several years ago in 2010
titled "Overclocking Data Storage Subsystems". FYI, here it is:

http://supremelaw.org/patents/SDC/Overclocking.Storage.Subsystems.Version.3.pdf


To that end, I would want to install a capable x16 RAID controller
in the first PCIe slot, and utilize the GPU that is integrated in the CPU.

Yeah, that's one other option if you want all 16 pcie lanes for the raid card. Use the integrated graphics (only certain cpu's and mobo's will support that). Install the raid card in the main x16 slot, and don't use any other pcie slots.

Furthermore, my "WANT AD" specifies an x16 edge connector on that RAID controller, because I want MAX HEADROOM
to feed four Intel NVMe 2.5" SSDs, as follows:

each U.2 cable uses 4 PCIe 3.0 lanes
four U.2 cables use 16 PCIe 3.0 lanes

http://supremelaw.org/systems/nvme/want.ad.htm

16 PCIe 3.0 lanes <=> x16 edge connector (an engineering design objective)

That, in turn, opens up all of the other modern RAID modes
as practical and feasible design options.

If graphics subsystems can support multiple slots and
very high bandwidth, then the up-and-coming memory
technologies on the visible horizon should have the
same opportunity i.e. "option" available to the designer,
DIY builder and PC Enthusiast.

Agree.

Now consider this: now that memory and video controllers have been integrated into the CPU die, why not integrate a high-performance
RAID controller onto the next CPU die?

I wouldn't be surprised if they did this, but I wouldn't use it. I don't use any onboard raid. I use a raid card.

A limitation of onboard raid is that if I decide I need to upgrade my motherboard, this raid set may not be recognized or supported on the next mobo, because most likely the raid controller would be a different chipset/brand. So the data would have to be backed up elsewhere, drives reformatted in the new set up, data restored. Pain in the ass, and for ssd's not a good idea for the write wearing to have to completely re-write the data. Plus onboard mobo raids usually have shit performance compared to a good raid card (be prepared to spend $400 or more on just the raid card). Good pcie raid cards usually support write caching with battery backup, have their own cpu, which is a big performance boost. Mobo's raids aren't going to have battery backup, RAM, or have any real write caching.

Also, onboard raids (probably mainly those in the past) were finicky and sensitive, and the raid 0's blew up all the time on my friends. I am not going to reinstall my os every 3 months like they did becuase the raid set took a shit. I want reliability and stability, along with data redundancy. Your needs may vary.

A nice big plus to a raid on pcie card, is that when I build a new system, or switch the mobo, I just pick up the raid set and move it over the the new machine. I've gone from AMD Athlon/chipset to Intel i7 chipset boards without having to reformat my os or raid. The os critical boot drivers are the storage device. The driver in my os is already for the raid card, so even though the rest of the guts of my pc have changed, the os boots just fine. It finds a bunch of new devices, installs the drivers for those, reinstalls drivers for peripherals like sound card or video card, reboot, done. When I switched from amd to intel I just had to install the intel chipset drivers. And you can have some of that stuff installed ahead of time, or at least the drivers all downloaded and ready to be installed. Was up and running lickity split and I didn't even lose any of my installed apps.

My raid card is an x16 card, but only needs x8 lanes to work. It's older sata devices (in a Raid 5) and even still, my windows 10 boot time is like less than 8 seconds, once you get past the bios post and the raid cards post. I'm happy with it (get about 650Mb/sec throughput from 5 mechanical drives), even though going to a ssd raid would get 5x performance if not better, even when getting bottlenecked by the available pcie lanes to about 4gb/sec total throughput.

My wishlist for many years has always been more PCIe lanes, and for basically the same reasons as you.

I think you would be better suited with an x99 mobo. Skip the Z170.

 

[Zarathustra[H]]

= 3.938 GB/sec MAX HEADROOM.

Our desired controller needs 16 lanes @ 8 GHz = 128 GHz / 8.125 bits per byte
= 15.75 GB/sec MAX HEADROOM.

 

[AbRASiON]

If anything, this article revealed how messy the storage situation is, the whole U.2 or M.2 thing is not good.

I'm sitting this one out until it's a bit more stable, current SSD's seem fine for me.

 

[Zarathustra[H]]

If anything, this article revealed how messy the storage situation is, the whole U.2 or M.2 thing is not good.

I'm sitting this one out until it's a bit more stable, current SSD's seem fine for me.

Meh,

Drives, (especially SSD's) are not long term investments. Hard drives wear out and die, and SSD's use up their write cycles, and both are quickly outclassed by faster/cheaper/more power efficient and quieter models.

If a drive connector standard only lasts for one build, it's not a big deal. The drive would probably have had to be replaced (or you'd want to) anyway.

That and they are all electrically PCIe, so there will always be cheap-ish passive adapters.

I'm not concerned about the volatility. In fact, it is a good thing, as it is a symptom of a rapidly developing industry, as opposed to CPU's and GPU's which are stagnating.

 

[Zarathustra[H]]

p.s. If anyone is interested, here's a Highpoint RAID card with an x16 edge connector:

http://www.newegg.com/Product/Product.aspx?Item=9SIA24G3FG6779&Tpk=9SIA24G3FG6779


/s/ Paul

Not sure I trust Highpoint. have had nothing but trouble with them over the years.

When it comes to SAS controllers, I think I have some real LSI brand loyalty, and that is rare for me, as I usually don't have much in the way of brand loyalty at all.

I currently have two LSI 9211-8i's in my server, and they have been damned near flawless. The only thing I don't like is having to chase the firmware update every time BSD updates the LSI driver, as apparently LSI controllers are really sensitive to having drivers and firmware on the same "phase" as they like to call it.

 

[GoodBoy]

Haven't used the Highpoint cards. In the past they were limited to low end products. That card might be good, but I would want a few reviews first.

I use Areca cards, most have the option to add the battery backup, which you can order separately. Adds $100 to $130 to the price. Use the battery backup so you can enable write caching, which is a big boost.

Learn all about RAID, stripe sizes, etc. Before setting up your raid. Choosing the wrong stripe size can cause performance to be slower than desired. These 2 articles aren't bad : http://www.tomshardware.com/reviews/RAID-SCALING-CHARTS,1735-4.html and http://www.pcguide.com/ref/hdd/perf/raid/concepts/perfStripe-c.html Likely you will want to use 64kb, 128kb, or 256kb. Another article just said use the largest but that's oversimplifying. If you aren't sure, 128kb is a safe first try.

 

[777]

Is it generally assumed that SATA and M.2 Xpoint drives will be simple drop-in replacements on existing systems? (Performance handicaps aside)

I don't think I've seen much info on this. Intel has only been talking up the enterprise with their Xpoint DIMMs.

 

[extide]

Is it generally assumed that SATA and M.2 Xpoint drives will be simple drop-in replacements on existing systems? (Performance handicaps aside)

I don't think I've seen much info on this. Intel has only been talking up the enterprise with their Xpoint DIMMs.

Yeah, although we will probably not see them in SATA form,rather PCIe, but yeah, they will function exactly the same.

 

[777]

Yeah, although we will probably not see them in SATA form,rather PCIe, but yeah, they will function exactly the same.

I've been reading that Union Point 200-series chipsets will have Optane/Xpoint support, whatever that means. Intel being too coy here. I wish they would give out more information.

 

[extide]

I've been reading that Union Point 200-series chipsets will have Optane/Xpoint support, whatever that means. Intel being too coy here. I wish they would give out more information.

I think they may be talking about using it in place of (some of) the DRAM. (In that case)

Although it can also be used as storage, like a SSD.