Bootable PCIe NVMe RAID 0 on ASUS Z170-A

A

amp88

Limp Gawd
Joined
Sep 3, 2007
Messages
133
I'm trying to get a bootable PCIe RAID 0 working on an ASUS Z170-A, but I'm stuck. The RAID volume appears in the Intel RST section of the BIOS as bootable, but the volume doesn't appear in the Boot Menu of the BIOS. I have managed to install Windows 8.1 on the RAID volume (by installing from a UEFI bootable USB flash drive and supplying the Intel RST RAID Driver during the Windows install), however I can't boot from it.

I've tried a number of different settings in the BIOS (e.g. toggling most of the CSM stuff), but no joy. I've set the "PCI-EX16_3 Bandwidth" in 'Onboard Devices Configuration' to "x4 Mode", from "Auto".

intel_rst_bootable_raid.png

pci-ex16_3_bandwidth_mode.png


System specs:

CPU: Intel Core i7-6700K
CPU Cooler: be quiet! Dark Rock Pro 3
Motherboard: ASUS Z170-A LGA1151 (BIOS version 0604)
Memory: Corsair Vengeance LPX 16GB (2x8GB) DDR4-3200 C16
Primary Storage: 2 x Samsung 128GB SM951 M.2 NVMe SSDs in PCIe RAID 0
Secondary Storage: Western Digital Caviar Green 3TB
Video Card: 4GB Asus GTX 970 STRIX Direct CU II
Case: Fractal Design Define R5 Titanium Windowless
Power Supply: Corsair AX760 80+ Platinum Fully Modular
Optical Drive: LG BH10LS38 Blu-ray Writer
Operating System: Windows 8.1 64-bit

(Note that the WD Green drive and Blu-ray burner are disconnected while I try to get the RAID volume sorted)

Has anyone here got a PCIe RAID volume to be bootable on a Z170 board? Or are there any suggestions for things to try? I'm getting desperate here, and welcome any input. Thanks for reading.

edit: For diagnostic purposes I have also successfully installed Windows 8.1 on a single SM951 and was able to boot from it with no problems. The problem only occurs when I try to boot from the RAID volume.
 
Last edited:
I am particularly interested in your endeavour, as it's precisely what I'd like to do (albeit with a Z170-Deluxe.)

What does your boot device selection screen show as options?

Also, where on the planet did you find the NVMe SM951s? They are sold out everywhere on the globe...
 
i haven't messed with RAID yet in a long time so please don't laugh -_- What is the difference from doing a RAID in BIOS/IRST verse doing it in windows manager?

Did you at least try to make it in windows manager to see if a RAID at least worked that way?
 
SomeGuy133 said:
i haven't messed with RAID yet in a long time so please don't laugh -_- What is the difference from doing a RAID in BIOS/IRST verse doing it in windows manager?

Did you at least try to make it in windows manager to see if a RAID at least worked that way?
Click to expand...

Software RAID is not bootable (by definition ... it is managed by the OS.) He is trying to setup a bootable hardware RAID 0 across two M.2 (PCIe) devices.
 
I was under the impression that nvme RAID wasn't bootable... Hell until a few months ago nvme wasn't even bootable, period.
 
macksomerville said:
I was under the impression that nvme RAID wasn't bootable... Hell until a few months ago nvme wasn't even bootable, period.
Click to expand...

This is what I thought as well, I know PCper did a review on the Intel 750s and one of the cons was that you could not boot raid 0, but that is before skylake and Z170 boards.


The OPs testing seems to agree that it doesn't work, but since he is using a drive that isn't officially supported by anyone it isn't the best benchmark to go by. Z170 is still new and maybe this could be fixed in a future bios update.
 
Astyrrean said:
Software RAID is not bootable (by definition ... it is managed by the OS.) He is trying to setup a bootable hardware RAID 0 across two M.2 (PCIe) devices.
Click to expand...

the more I think about it the more I could have sworn my first raid back in 2000/XP was windows RAID 0 (striped in drive management) and was bootable. eh whatever.
 
Sorry for the late reply. First off, I did manage to boot into the RAID 0 volume (unfortunately I didn't copy down the successful BIOS settings I tried though, I was basically just trying every single combination in CSM and Boot settings menus), but I had constant problems with system stability. Within a short period of starting to write data to the drives (typically between 30 seconds and 3 minutes), the system would stop working properly then hang and need to be reset. You can see an example of this behaviour here (sorry for the poor quality). After a few cycles of this behaviour I got a notification from Intel RST that the RAID volume had failed, and on the next reboot the RAID volume had disappeared from the boot menu.

The only conclusion that I can draw from the experience is that there was an issue with the BIOS (which might be solved in a later version...) or with the way Windows was interacting with the RAID/NVMe driver.

After deleting the RAID volume I've had no problems running the drives separately (installing Windows 8.1 on the SM951 in the motherboard's M.2 slot and using the second M.2 drive in the adapter as additional storage), so there doesn't appear to be a problem with the drives, the adapter, or Windows' handling of NVMe drives.

At this point in time I'd advise going for a single drive of larger capacity (I wish I'd bought the 256GB version instead of two 128GBs). I got mine from Flexx Memory, delivered on the 14th of August.

A couple of benchmarks of the RAID volume:

sm951_raid_crystal.png

sm951_raid_atto.png


...and the failed volume information in the Intel RST manager:

intel_rst_failed_1.png

intel_rst_failed_2.png

intel_rst_failed_3.png
 
Well that's a bummer... Why'd you go for 2x128GB anyway? What were ya planning to do with them? (besides the obvious, OS, etc)

I went with a single 256GB from RAMCity thru Amazon for like $210. Second guessed myself a little on whether I should go larger but the money was a bit of a factor and the truth is half a TB would kinda be no man's land.

Would be able to install some games on a 500GB, but not all and they wouldn't benefit greatly, would be able to keep some media/data I'm working on but again not all... And I've managed with a 256GB OS volume for years (and smaller still).

I figure for a similar kind of money a smaller SM951 + 850 EVOs for games/data made more sense. 256 SM951 + 500GB EVO would actually be cheaper than a 500GB SM951 at current prices... Tho I went 1TB (x2!).

For sequential speeds off an SM951 array to matter much in daily use you'd probably need a second array that can feed it fast enough anyway...
 
Impulse said:
Well that's a bummer... Why'd you go for 2x128GB anyway? What were ya planning to do with them? (besides the obvious, OS, etc)

I went with a single 256GB from RAMCity thru Amazon for like $210. Second guessed myself a little on whether I should go larger but the money was a bit of a factor and the truth is half a TB would kinda be no man's land.

Would be able to install some games on a 500GB, but not all and they wouldn't benefit greatly, would be able to keep some media/data I'm working on but again not all... And I've managed with a 256GB OS volume for years (and smaller still).

I figure for a similar kind of money a smaller SM951 + 850 EVOs for games/data made more sense. 256 SM951 + 500GB EVO would actually be cheaper than a 500GB SM951 at current prices... Tho I went 1TB (x2!).

For sequential speeds off an SM951 array to matter much in daily use you'd probably need a second array that can feed it fast enough anyway...
Click to expand...

In my original plan for the build I was going to use a single 256GB drive (at the time I wasn't sure if I'd be using the AHCI or NVMe version), but I ended up going for the two 128GB drives for 2 reasons: availability was better for the 128GB drives at the time and I was enticed by the potential for performance scaling with NVMe. The low queue depth performance of the 128GB AHCI model looked really good in this Tom's Hardware review, and I'd expected the lower overhead of NVMe to really fly with these. Most of the heavier use on the drives will be photo/video editing and some database work. The 256GB version definitely looks like the sweet spot at the moment, so I think you made the right choice.

My intention has always been to add a larger SATA SSD at some point down the line (probably next year), when I run out of storage on the M.2s. The way SSD prices have been falling lately I'll probably be able to get a decent 512GB drive (about the performance of an 850 EVO) for a reasonable sum. The total local storage in my (soon to be retired) desktop has been about 270GB for a few years now (a 128GB Samsung 830 and a 150GB WD Velociraptor) and I mostly coped OK (though Assetto Corsa taking up about 25GB on the SSD was a bit of a pain ;) ).
 
So this isnt just an issue with this ASUS board

From Gigabyte review
http://www.legitreviews.com/gigabyte-z170x-ud5-atx-motherboard-review_170496/20

One of the features of the Intel Z170 chipset we were looking forward to checking out is the option of creating a bootable Raid arrary with the Kingston HyperX Predator M.2 PCI Express SSDs. Unfortunately, we were unable to implement the M.2 Raid capability of the Z170X-UD5. We have brought this to Gigabyte’s attention and have waited for a solution for 14 days without receiving one. If you wanted to run a bootable M.2 setup we do not suggest using this board for the time being, but that is such a small group it should be a deal breaker.
 
zaniix said:
So this isnt just an issue with this ASUS board

From Gigabyte review
http://www.legitreviews.com/gigabyte-z170x-ud5-atx-motherboard-review_170496/20

One of the features of the Intel Z170 chipset we were looking forward to checking out is the option of creating a bootable Raid arrary with the Kingston HyperX Predator M.2 PCI Express SSDs. Unfortunately, we were unable to implement the M.2 Raid capability of the Z170X-UD5. We have brought this to Gigabyte’s attention and have waited for a solution for 14 days without receiving one. If you wanted to run a bootable M.2 setup we do not suggest using this board for the time being, but that is such a small group it should be a deal breaker.
Click to expand...

Interesting. Thanks for the link - at least this should give us hope for the future...
 
Lone Gunman said:
This is how ASRock created a 3 way M.2 raid 0 in their UEFI Bios, it may be similar. But because of the bottleneck DFI 3.0 the max read speed was still only 3.5 Gbs.

https://www.youtube.com/watch?v=Kp3eOrbQeiA
Click to expand...

That looks as though they're not creating a bootable RAID volume though, which is what I was trying to do. I didn't actually try creating a non-bootable RAID (i.e. installing Windows on a different SSD and using the M.2 RAID 0 volume as additional storage), so I don't know if that would have been more stable for me. Thanks for the link though.

zaniix said:
right there is zero reason to use RAID0 with PCIi NVME drives as the board will cause bottlenecks, you are just wasting money
Click to expand...

The ~3.5GB/s DMI bottleneck isn't really a concern, and I think most people looking to PCIe RAID would be doing it for the potential of increased random read/write performance, especially at lower queue depths. That was certainly the case for me anyway.
 
Isn't RAID usually more beneficial for sequential I/O than random? Latency tends to go up with RAID...
 
Impulse said:
Isn't RAID usually more beneficial for sequential I/O than random? Latency tends to go up with RAID...
Click to expand...

It depends on the members and implementation. One of the promises of NVMe (which early consumer drives, including the Samsung SM951 and Intel 750 have delivered on, to slightly differing extents) is the reduction in overhead should improve latency and IOPS. The hardware in the SM951 AHCI and NVMe versions is apparently identical (according to AnandTech), but the NVMe version offers between 10-20% better performance compared to AHCI. For mechanical hard drives (and possibly for SATA/AHCI SSDs, though that may depend on implementation) a RAID would be likely to increase latency.

Now, my particular use case wasn't a perfect demonstration of the potential of PCIe NVMe RAID 0 performance for a couple of reasons. Clearly stability and support were lacking, but the stripe size for the volume was 128KB, which acted to limit small transfer performance compared to single drives. That's why it took until a transfer size of 8KB until the performance of the RAID 0 surpassed that of a single drive (and past 8KB the scaling was really quite good). With a smaller stripe size I'd expect the small transfer performance to improve accordingly, though I haven't tested that myself.

For comparison, here are ATTO benchmarks for the RAID 0 and a single boot drive:

sm951_raid_atto.png
single_sm951_atto.png
 
Very interesting, thanks for sharing that. Will be interesting to see how support develops down the line.
 
Impulse said:
Very interesting, thanks for sharing that. Will be interesting to see how support develops down the line.
Click to expand...

Yep, with improved BIOS/driver support and refinements on hardware we could see even bigger gains from NVMe too. It's quite an exciting prospect.
 
I am coming across the same troubles with my MSI z170a M7 board. I am really interested to know anyone that has got it working and how. Thanks.
 
As I write this I am sitting at my new machine and I have got NVMe Raid0 working and booting just fine. Asus support could not help me (They said that these drives were not supported) but after hours of experimenting with the UEFI settings I finally got it working. I am using 2 x SM-951 500GB NVMe. One in the onboard m.2 slot and the other in an expansion card inserted into PCI-E_16 Slot 3.

Raid0.jpg


Diskmngmnt.jpg


Rest assured that it is possible to get NVMe Raid0 working as the bootable partition with the Asus z170-A and the above NVMe Samsung drives.

I will post screenshots of the proper Motherboard UEFI settings in the next day or two.

Out
 
Last edited:
depending on my budget maybe I should try 2 950 Pros for 1 TB hummmm

Benchmarks? Pretty please :D
 
Janpaul said:
As I write this I am sitting at my new machine and I have got NVMe Raid0 working and booting just fine. Asus support could not help me (They said that these drives were not supported) but after hours of experimenting with the UEFI settings I finally got it working. I am using 2 x SM-951 500GB NVMe. One in the onboard m.2 slot and the other in an expansion card inserted into PCI-E_16 Slot 3.

Rest assured that it is possible to get NVMe Raid0 working as the bootable partition with the Asus z170-A and the above NVMe Samsung drives.

I will post screenshots of the proper Motherboard UEFI settings in the next day or two.

Out
Click to expand...
Very interesting. I look forward to reading some more details. Thanks for posting.
 
Here are the benchmarks I just took of my RAID0 NVMe boot partition.


NVMe_RAID0.jpg


I am collecting the UEFI settings and will provide a step by step as to how I got this working very soon.

Out
 
Ok here are the UEFI settings needed for the ASUS Z170-A and NVMe RAID0 for boot. I am using the latest release from Asus Which is UEFI/BIOS Version 1101.

151023100523.BMP

151023100549.BMP

151023100629.BMP

151023100652.BMP

151023100722.BMP

151023100752.BMP


Once these settings were configured I used the WINDOWS 7 USB/DVD DOWNLOAD TOOL (https://www.microsoft.com/en-us/download/windows-usb-dvd-download-tool) to create a Windows 10 x64 bootable USB flash drive from my Windows 10 x64 ISO file. (Any UEFI boot disk should work).

At this point I rebooted the machine and entered the UEFI/BIOS. In the boot section I selected the UEFI version of my flash drive as the boot override option.

On reboot Windows begins the installation; however, the newly created RAID0 volume is not detected. Here you must feed the installation (add) the Intel Rapid Storage Drivers (Browse to the inf file in the X64 folder (iaStorAC.inf)). You can download these at the link below.....

http://station-drivers.com/index.php?option=com_content&view=article&id=1613:intel-rapid-storage-technology-rst-version-14-6-1-1030-whql&catid=16:articles&Itemid=171&lang=fr

After the RST driver is loaded the RAID0 volume was detected and installation proceeded as normal from there.

I hope this is helpful.

~Paul
 
Last edited:
Astyrrean said:
Very helpful

Will it boot also with CSM disabled?
Click to expand...

It will but I have found that if I change the CSM settings from those I posted to either AUTO or DISABLED the system becomes unstable. Also trying to install Windows with these settings would result in a crash during installation. Only the options I posted for CSM seem to work for me for both flawless installation and system stability.

~Paul
 
Paul - thanks for posting your settings. I plan to attempt the same thing, made 2 USB bootable drives last night and downloaded the RST drivers. Now just waiting for the Samsung Pro 950's to ship out. I have the same motherboard so hopefully I can get similar results.
 
I will watch the thread here. Please let me know if there is anything I can do to help along the way. I would be sure to update the UEFI/BIOS to the latest first thing.

Best of luck and I know you are going to love this build. Everything has been rock stable for me since I got this up and running so that is a very good sign.

Enjoy!

~Paul
 
Paul - I did send you a PM with a couple of quick questions. Thanks.
 
WxChaser said:
Paul - I did send you a PM with a couple of quick questions. Thanks.
Click to expand...

Tried to reply by PM not sure they made it.



Happy to help.

Once you have extracted the RST drivers to a folder copy the x64 folder from it to a flash drive.

At the screen where you choose the volume to install windows on there should be an option to browse for a storage driver. Browse to the x64 folder on the flash drive containing the RST driver.

Once this driver is loaded and windows searches again for disks/volumes, the NVMe RAID0 array should populate as an available volume for the WinX install.

Proceed as normal from here. On reboot make sure to select the NVMe Raid0 volume in the UEFI/BIOS as the first boot option.

Let me know if there is anything else I can do to help.

~Paul
 
Last edited:
Would be really interesting to see if Asus would chime in here. With the 950 Pro M.2 coming out they should.

I've held off getting the second SM951 - the first one alone is already blazing fast!
 
You must log in or register to reply here.
Back
Top