Need 2TB SSD for IO intensive work

M76

[H]F Junkie
Joined
Jun 12, 2012
Messages
13,139
Writes / Reads simultaneously with up to 3 full write/erase cycles daily. Probably half on average.

I wish I could go for NVME Samsungs but as far as I can tell there are only 512GB versions avaialable, and I fear RAID would cripple IOPS.
 

M76

[H]F Junkie
Joined
Jun 12, 2012
Messages
13,139
How many IOPS do you really need?

The more the better. So the workflow can be accelerated. I don't have an exact number just more :-D I use 840EVO-s now and it's terrible. No, worse than terrible. It takes 10-20 minutes to erase a few hundred GBs of files. I know that's not IOPS, but I want something good this time. That can take the beating. And doesn't roll over dead when I run 12 separate processes, each writing and reading hundreds of GBs.
 
Last edited:

NetJunkie

[H]F Junkie
Joined
Mar 16, 2001
Messages
9,682
The more the better. So the workflow can be accelerated. I don't have an exact number just more :-D I use 840EVO-s now and it's terrible. No, worse than terrible. It takes 10-20 minutes to erase a few hundred GBs of files. I know that's not IOPS, but I want something good this time. That can take the beating. And doesn't roll over dead when I run 12 separate processes, each writing and reading hundreds of GBs.

Doesn't sound like an I/O problem to me.
 

JBark

2[H]4U
Joined
Dec 13, 2002
Messages
2,048
It's the simultaneous read/writes that are killing you on the EVO. Well, that will kill just about any SSD, but something like the EVO will be affected much more than say the PRO.

Check a few reviews with mixed read/write workloads, that's where the high-end and enterprise drives really show their worth. Some of the cheaper SSDs, especially the TLC ones, sometimes show up to 90% performance drops in mixed read/write vs. peak.

If the P3608 is too expensive, take a look at the 750. Yeah, not quite as fast, but it still crushes any consumer level SSD in mixed read/write. $1000 USD on Amazon for the 1.2TB model.

Also, you talk of it taking 10-20 minutes to delete files on the SSD. I'm assuming this must be hundreds of thousand/millions of small files? If so, any sort of NVMe drive should show some impressive improvements then due to the greatly reduced overhead.
 

ToddW2

2[H]4U
Joined
Nov 8, 2004
Messages
4,018
To increase speed of what you're doing it sounds like multiple enterprise SSD will take care of it for you, or a couple NVME drives. However, in reality it doesn't sound like you have the budget for any of the things you say you "need"... to get the performance and TBs of capacity you're going to have to spend the $$$.
 

M76

[H]F Junkie
Joined
Jun 12, 2012
Messages
13,139
It's the simultaneous read/writes that are killing you on the EVO. Well, that will kill just about any SSD, but something like the EVO will be affected much more than say the PRO.
I know, ironically the 830 Series outperformed the 840 EVOs by a mile under these loads. But right now I'm stuck with 840EVOs as someone rushed into buying a boatload of these last year, so now every workstation is equipped with these.

Check a few reviews with mixed read/write workloads, that's where the high-end and enterprise drives really show their worth. Some of the cheaper SSDs, especially the TLC ones, sometimes show up to 90% performance drops in mixed read/write vs. peak.
I did, and the 850EVO seems unaffected by the same performance drop as the 840EVO, it only performs slightly slower than the 850PRO in mixed workload tests, but understandably I'm a bit skittish after the 840EVO debacle.

If the P3608 is too expensive, take a look at the 750. Yeah, not quite as fast, but it still crushes any consumer level SSD in mixed read/write. $1000 USD on Amazon for the 1.2TB model.
I'd need 2TB as a bare minimum, so I would need two of those, and then face the problem of raiding them somehow. Plus I don't think I can push the 2000 cost trough. So I think I'm stuck with the consumer market. The 850PRO would still be a huge leap over the 840EVO I believe, if there are no other options in this price range.
Also, you talk of it taking 10-20 minutes to delete files on the SSD. I'm assuming this must be hundreds of thousand/millions of small files? If so, any sort of NVMe drive should show some impressive improvements then due to the greatly reduced overhead

Nope, it doesn't matter. It can be 100 1GB files, or 1 100GB file, it will still take 10 minutes to erase it. It's some kind of issue between the intel chipset and this specific model of SSDs, as it happens on both X58, and H87 machines. If I use an Adaptec RAID add-on card the delete times are normal, but that just cripples read-write performance even further.

Edit: It actually seems to be getting worse. I'm just erasing 250GB in 429 files, and it currently half way trough after 30 minutes.
 
Last edited:

M76

[H]F Junkie
Joined
Jun 12, 2012
Messages
13,139
To increase speed of what you're doing it sounds like multiple enterprise SSD will take care of it for you, or a couple NVME drives. However, in reality it doesn't sound like you have the budget for any of the things you say you "need"... to get the performance and TBs of capacity you're going to have to spend the $$$.

It's company politics. The big bosses said we can get anything we need. So if it was only up to them I'd have 3608 drives delivered no questions asked. The problem is the department head, whose reputation depends on the profit/cost ratio of the department, refuses to authorize the purchase of even a pencil unless it's absolutely needed. Typical small mindedness, where he doesn't realize that to produce more, we need tools that we can work faster with.

And I'm stuck, from one side I'm not allowed to spend anything, from the other side they'll say "why didn't you tell us that you need it". So I need to find a compromise. Something I can push past the stingy guy but still performs relatively well.
 
Joined
Mar 25, 2012
Messages
33
But right now I'm stuck with 840EVOs as someone rushed into buying a boatload of these last year, so now every workstation is equipped with these.

What OS are you running and is it trim enabled? What is your workflow that is so IO intensive and requires 6TB writes per day per workstation and would a shared storage solution be more suitable?
 

SomeGuy133

2[H]4U
Joined
Apr 12, 2015
Messages
3,447
What OS are you running and is it trim enabled? What is your workflow that is so IO intensive and requires 6TB writes per day per workstation and would a shared storage solution be more suitable?

i am wondering how those TLC drives haven't died....they aren't known for great P/E
 

Zangmonkey

Supreme [H]ardness
Joined
Jul 6, 2005
Messages
4,609
The more the better. So the workflow can be accelerated. I don't have an exact number just more :-D I use 840EVO-s now and it's terrible. No, worse than terrible. It takes 10-20 minutes to erase a few hundred GBs of files. I know that's not IOPS, but I want something good this time. That can take the beating. And doesn't roll over dead when I run 12 separate processes, each writing and reading hundreds of GBs.

Is your erasure secure, and does it need to be?
Can you just quick-format the drive rather than erase it?
 

SomeGuy133

2[H]4U
Joined
Apr 12, 2015
Messages
3,447
Is your erasure secure, and does it need to be?
Can you just quick-format the drive rather than erase it?

i don't think he is wiping but editing and the edit is in the 100s of GB and when you delete that large I think garbage collection is kicking in but I could be wrong but thats my thought.
 

HammerSandwich

[H]ard|Gawd
Joined
Nov 18, 2004
Messages
1,126
This. Samsung has issued two firmware updates, the last in April 2015, to resolve issues with performance degradation.
Because this problem affects old, static data, it probably doesn't apply in OP's 3DWPD situation.

That said, it's certainly worth checking & updating.
 

M76

[H]F Junkie
Joined
Jun 12, 2012
Messages
13,139
I would make sure you have the latest firmware on those evos.

I upgraded right after each new firmware relase, hoping it will solve the issue, but no such luck. I even ran the performance restoration on some of the drives, even trough there are absolutely no old files on these SSDs, still no luck.

What OS are you running and is it trim enabled? What is your workflow that is so IO intensive and requires 6TB writes per day per workstation and would a shared storage solution be more suitable?

Windows 7 64 bit Enterprise or Professional. I'm processing LIDAR and Image data. RAW data is processed creating 10x data, then from that I Export the data to a common format, and it even needs to be transformed into a local coordinate system after that. 6TB of writes may be an extreme case, but up to that possible. Basically I generate about 20x amount of intermediate data before I get the end result.

I already use shared storage for the RAW data, only the processed data is generated on the workstation. But I only have gigabit network avaialable so I can't use it for anything else.

i am wondering how those TLC drives haven't died....they aren't known for great P/E

I don't know what do you mean by P/E. The drives are not used every day, but when they are used they get about 2-4TB writes in a day. Depending on how much data needs to be processed.

The 840EVOs I'm currently using are about 18 months old and each 1TB drive has about 60-70TB writes.

Is your erasure secure, and does it need to be?
Can you just quick-format the drive rather than erase it?

No it's not secure erase, I simply select the files and delete them. I can't format because I only need to erase some of the intermediate files at a time. While other files need to be kept during the entire workflow.

i don't think he is wiping but editing and the edit is in the 100s of GB and when you delete that large I think garbage collection is kicking in but I could be wrong but thats my thought.

Garbage collection could be a good guess. Which probably doesn't work in case the drives are in RAID, that's why I don't get the delays with the Adaptec RAID controller. But since no advanced functions work in RAID the performance is very poor.
 

SomeGuy133

2[H]4U
Joined
Apr 12, 2015
Messages
3,447
ah so its not 2-4 TB every day. Gotcha. I read somewhere these drives die as soon as 1TB written to them so I am shocked they have lasted this long. Or was the warranty only 1 TB?

I think chris is talking about warranties but i am a little confused by his posts. I generally loathe TLC for a number of reasons but this adds to the reasons why I won't touch it.

EDIT: BTW I thought I did a lot of writes but wow....You write a lot of stuff.
The price per GB of the 750 is right in line with the SM951 AHCI, and will probably also basically be a wash with the NVMe version. The SM951 (AHCI or NVMe) are still OEM-only products with finicky compatibility (not to say that NVMe support is great anywhere, but there's something to be said about retail support of a high-end SSD). Furthermore, the Intel 750 is cheaper per GB than any of the other PCIe options in the test roundup. So, dominant performance, significant new tech and value vs. peers? Seems like Gold to me.

While I'm going to very nicely ask Samsung for an SM951 to test, from what I've heard and read they've been iffy about review samples for non-retail drives, and I'm also a little disappointed at the lack of 1TB option.

In terms of endurance, 219TB Total Bytes Written (TBW)/70GB/day is quite nice, but perhaps not market-leading. Gotta provide some incentive to move up to the P3xxx drives.

Another MAJOR, MAJOR point is that the quoted reliability numbers from manufacturers aren't comparable. Have a look at the description of JEDEC testing standards here- http://www.flashmemorysummit.com/English/Collaterals/Proceedings/2011/20110810_T1B_Cox.pdf

You will often have to go into the fine print on a product's datasheet to find the testing methodology used, but the Intel 750's endurance is quoted on the JESD218 standard, while the crazy enormous numbers shown by Kingston for the HyperX Predator are based on JESD219A. What does that mean in practice as we try to compare these numbers? It means that they're apples and oranges.

Compared to some other SSDs in the test or of interest, and with the above caveats, here's how the Intel 750's 219TB/70GB/day stacks up:

  • Samsung SM951- because it's OEM-only, Samsung won't be held to endurance figures. One reseller, RamCity, apparently warranties it to 72TBW on the 512gb version (methodology unknown)
  • Kingston HyperX Predator - 882TB total (JESD219A)
  • Samsung 850 Pro- 300TB total (methodology unknown)
  • Samsung 850 EVO 1TB- 150TB total (methodology unknown)
  • OCZ RevoDrive 350 480GB - 50GB/day (methodology unknown)
  • G.Skill Phoenix Blade 480GB - 192TB total (methodology unknown)
  • SanDisk Extreme Pro 960GB- >70TB total (methodology unknown)

Samsung's Pro drives are seemingly unkillable, but beyond that, we can't really discern much from these numbers. If you're doing >70GB of writes a day, you're a prime client for a workstation-class drive anyway.
 
Joined
Mar 25, 2012
Messages
33
What file system is the SSD formatted as?

Cache write through enabled for maximum throughput?

Any recycling bin rubbish going on?
 

cyclone3d

[H]F Junkie
Joined
Aug 16, 2004
Messages
15,451
What about using a very large RAM cache for the SSD in addition to whatever drive(s) you get?

That should speed things up quite a bit.

MaxVeloSSD is pretty cheap AND the RAM cache should help a lot.
The software is meant more for caching HDDs with SSDs and RAM, but you could use just the RAM cache to cache the SSD.
 

jyi786

Supreme [H]ardness
Joined
Jun 13, 2002
Messages
5,669
How about a boatload of RAM and configure them as RAM drives??? :D
 

Blue Fox

[H]F Junkie
Joined
Jun 9, 2004
Messages
11,814
Have fun paying $25000+ for that. 8 socket systems are incredibly expensive, as are 32GB DIMMs.
 

evilsofa

[H]F Junkie
Joined
Jan 1, 2007
Messages
10,078
ah so its not 2-4 TB every day. Gotcha. I read somewhere these drives die as soon as 1TB written to them so I am shocked they have lasted this long. Or was the warranty only 1 TB?

I can only assume you are thinking of PB (petabytes) instead of TB (terabytes). 1 PB = 1000 TB. In TechReport's big SSD endurance test, they tested an 840 (a TLC drive that was replaced by the 840 EVO) and it failed just before it hit 1 PB of writes.
 

cyclone3d

[H]F Junkie
Joined
Aug 16, 2004
Messages
15,451
Have fun paying $25000+ for that. 8 socket systems are incredibly expensive, as are 32GB DIMMs.

Even at Ebay prices, 4TB of RAM alone would be ~$38,400

Edit: found some for $210 per stick, which would bring it down to ~$26,880 for the RAM.
 

jyi786

Supreme [H]ardness
Joined
Jun 13, 2002
Messages
5,669
Have fun paying $25000+ for that. 8 socket systems are incredibly expensive, as are 32GB DIMMs.

I was being semi-serious. :p

If he indeed needs that kind of speed (and endurance), maybe RAM drives are the way to go. He's going to get sorely disappointed using any of the consumer-grade stuff out there, kinda like what he's going through with those 840s.

At the very least, go with the Samsung Pro drives. For the cost, might as well look into RAM drives.
 

SomeGuy133

2[H]4U
Joined
Apr 12, 2015
Messages
3,447
I can only assume you are thinking of PB (petabytes) instead of TB (terabytes). 1 PB = 1000 TB. In TechReport's big SSD endurance test, they tested an 840 (a TLC drive that was replaced by the 840 EVO) and it failed just before it hit 1 PB of writes.

not according to chris....read my quote. They were 850s but they are supposed to be better then 840s

EDIT: HAHAHAHAHAHAHA

I have read that post probably 40 times and I just noticed its the 1TB model with 150 TB. -_- I thought it was 1-150TB was what has been reported by people with them dying. hahahahahahahhahahah -_-

Samsung 850 EVO 1TB- 150TB total (methodology unknown)
bad formatting lol
 

M76

[H]F Junkie
Joined
Jun 12, 2012
Messages
13,139
What file system is the SSD formatted as?

Cache write through enabled for maximum throughput?

Any recycling bin rubbish going on?

NTFS, Recycle bin not used.

As for RAM caching, it's not feasible, I'd need at least 512GB of RAM for it to make a difference. And for that price I can get a P3700 2TB SSD.
 

SomeGuy133

2[H]4U
Joined
Apr 12, 2015
Messages
3,447
I can only assume you are thinking of PB (petabytes) instead of TB (terabytes). 1 PB = 1000 TB. In TechReport's big SSD endurance test, they tested an 840 (a TLC drive that was replaced by the 840 EVO) and it failed just before it hit 1 PB of writes.

NTFS, Recycle bin not used.

As for RAM caching, it's not feasible, I'd need at least 512GB of RAM for it to make a difference. And for that price I can get a P3700 2TB SSD.

is all data equally used or is some used more then others?
 

dilidolo

Limp Gawd
Joined
Jun 30, 2011
Messages
180
SanDisk 2TB Optimus
3 DWPD, 6Gb SAS interface.

Price is reasonable, well below $2K
 
Joined
Mar 25, 2012
Messages
33
NTFS, Recycle bin not used.

As for RAM caching, it's not feasible, I'd need at least 512GB of RAM for it to make a difference. And for that price I can get a P3700 2TB SSD.

Sorry my bad, I meant write back, not write through. As in, is the drive configured to allow immediate confirmation of writes. Probably an option in Intel RST if memory serves me.
 

rsq

Limp Gawd
Joined
Jan 11, 2010
Messages
246
I try to offer some non-conventional options:

1.
I would try running the various steps sequentially and timing that.

If the bottleneck is the concurrent access on the SSD's it might be faster to not do multiple things at the same time. This used to be the case with spinning drives.

It maybe also brings your in-flight volume down enough to use a RAM disk to bring up your sequential speed.

It does worry me that you mention X58 and H87 mainboards. It indicates you are using regular computers to run this very heavy workload.

2.
Break chain of command and go directly to the boss that said you get whatever you need. Your stingy boss will not like you for doing that.

3.
Use clever programming and farm out work to other workstations. If the parcels are small enough, it can all sit in RAM and be processed there. You would be able to use the resources of the receptionist's computer (at least the resources not used for solitaire). It would involve setting up worker agents on the other computers. There are frameworks to help you do that.

4.
Leverage NVIDIA CUDA and run the workload in small bits on the GPU.

5.
Combine 3 and 4.
 

CombatChrisNC

[H]ard|Gawd
Joined
Apr 3, 2013
Messages
1,140
http://www.ebay.com/itm/New-2015-IN...080693?hash=item3ab68211b5:g:2fIAAOSwnGJWTjOC

Get yourself two of those. Might be tough to get corporate to order something off ebay, but IMO the price is pretty good. It'll obliterate EVO's in the kind of workload you're doing.

I wish we could whitebox servers which would actually utilize this kind of speed, space, and durability... I can only whitebox our non-mission critical things like WSUS and cold storage/archive.... neither of which can be justified in getting hardware like this. :(
 
Top