Intel Has a Problem: Vast majority of Game Crashes come from 13th/14th gen Intel CPU's during Decompression

Zarathustra[H]

Extremely [H]
Joined
Oct 29, 2000
Messages
39,798
Wendell over at Level1Techs reached out to some of his contacts in the gaming industry to allow him to peek at game crash data in an attempt to figure out what is going on with Intel CPU instability. Two developers agreed, and the data is - well - staggering.

I'm not going to provide any spoilers. Just watch for yourself:


View: https://www.youtube.com/watch?v=QzHcrbT5D_Y

Figured you guys might find this interesting.

Edit: OK, I'll do some spoilers. (but still watch the video, the context and full details are worth it, and Wendell is a good dude who deserves your clicks and ad revenues)

1.) Out of 1584 game decompression errors during the 90 day data retention period across both databases, 1431 of the errors are on Intel 13th or 14th gen CPU's. Second place (after 13900KS/K/KF and 14900KS/K/KF) is the i7-9700H with only 11 instances. AMD CPU's only had four crashes in total in this dataset.
2.) Crash rates occur on average every 2 hours of gameplay for affected users.
3.) Frequency of crash rates increase over time. (silicon or something else is degrading?)
4.) This is not due to overclocking.
5.) Datasets from game servers using either Asus or Supermicro W680 chipsets (no overclocking support, more conservative designs) show similar crash rates.
6.) Crash rates are also increasing over time on the W680 game server datasets...

Game servers are an outlier in the datacenter in that the single threaded performance often warrants using consumer CPU's instead of Xeons for servers, but that may be changing with 13th and 14th gen Intel CPU's.

"...support incidents have been unusually high for that configuration... so recently we've had to update the bios, disable e-cores or do CPU swaps to get the issues resolved. And we're not sure the issue is fully resolved, so we are charging a support premium for those systems right now..."

-- A Datacenter Service provider


"...we had really good luck with the 12900ks, and have always had good luck with xeons [...] something isn't right with the 13900k and 14900k. We already replaced a lot of customer's 13900k with 14900k and the issues don't seem fully resolved. [...] been steering customers towards 7950c systems instead. They're almost always faster anyway."

-- A Datacenter Service provider


"I might lose over $100k in like lost players from these [multiplayer server] crashes"

-- A Game Developer


...so, as the video title states. It seems Intel has a real problem with their 13th and 14th gen CPU's.

It sounds like Intel is telling large system integrators (Dell, HP, etc.) that they should expect between 10% and 25% of CPU's will have a problem or are marginal in some way, and they are not really sure what the root issue is, and this number Intel is admitting is likely low when looking at the Game Crash databases.

When AMD motherboard partners were caught juicing CPU's to the point of premature failure, AMD was pretty quick to announce that customers would be made whole. Intel does not appear to be doing the same...

I don't know about you guys, but I probably wouldn't buy an Intel 13th or 14th gen CPU right now. Luckily the 9xxx series Ryzens are looking to be pretty nice, at least if early rumors and leaks are anything to go by.
 
Last edited:
Wendell over at Level1Techs reached out to some of his contacts in the gaming industry to allow him to peek at game crash data in an attempt to figure out what is going on with Intel CPU instability. Two developers agreed, and the data is - well - staggering.

I'm not going to provide any spoilers. Just watch for yourself:


View: https://www.youtube.com/watch?v=QzHcrbT5D_Y

Figured you guys might find this interesting.

Edit: OK, I'll do some spoilers. (but still watch the video, the context and full detail is worth it, and Wendell is a good dude who deserve your clicks and ad revenues)

1.) Out of 1584 game decompression errors during the 90 day data retention period across both databases, 1431 of the errors are on Intel 13th or 14th gen CPU's. Second place (after 13900KS/K/KF and 14900KS/K/KF) is the i7-9700H with only 11 instances. AMD CPU's only had four crashes in total in this dataset.
2.) Crash rates occur on average every 2 hours of gameplay for affected users.
3.) Frequency of crash rates increase over time. (silicon or something else is degrading?)
4.) This is not due to overclocking.
5.) Datasets from game servers using either Asus or Supermicro W680 chipsets (no overclocking support, more conservative designs) show similar crash rates.
6.) Crash rates are also increasing over time on the W680 game server datasets...

Game servers are an outlier in the datacenter in that the single threaded performance often warrants using consumer CPU's instead of Xeons for servers, but that may be changing with 13th and 14th gen Intel CPU's.

"...support incidents have been unusually high for that configuration... so recently we've had to update the bios, disable e-cores or do CPU swaps to get the issues resolved. And we're not sure the issue is fully resolved, so we are charging a support premium for those systems right now..."

-- A Datacenter Service provider


"...we had really good luck with the 12900ks, and have always had good luck with xeons [...] something isn't right with the 13900k and 14900k. We already replaced a lot of customer's 13900k with 14900k and the issues don't seem fully resolved. [...] been steering customers towards 7950c systems instead. They're almost always faster anyway."

-- A Datacenter Service provider


"I might lose over $100k in like lost players from these [multiplayer server] crashes"

-- A Game Developer


...so, as the video title states. It seems Intel has a real problem with their 13th and 14th gen CPU's.

It sounds like Intel is telling large system integrators (Dell, HP, etc.) that they should expect between 10% and 25% of CPU's will have a problem or are marginal in some way, and they are not really sure what the root issue is, and this number Intel is admitting is likely low when looking at the Game Crash databases.

When AMD motherboard partners were caught juicing CPU's to the point of premature failure, AMD was pretty quick to announce that customers would be made whole. Intel does not appear to be doing the same...

I don't know about you guys, but I probably wouldn't buy an Intel 13th or 14th gen CPU right now. Luckily the 9xxx series Ryzens are looking to be pretty nice, at least if early rumors and leaks are anything to go by.

That was very impressive !!!

I am referring to the data collection mechanisms & number of sources used in a single story. I simply cannot recall reading a tech news story with so many sources & rigor of data collection.

Seriously impressive 👏
 
Actually way worse than I thought, staggering numbers.

But yeah it makes sense that server owners and not only gamers are seeing those issues. The CPUs need to decompress the same data and Intel is popular there (performs very well for this type of task, after all).

It's pretty crazy how decompression (+shader compilation) nowadays is such a heavy workload, even when if still often single threaded, it makes modern AMD and Intel CPUs spike extremely high in terms of temperatures, even when the wattage is not necessarily that high (I reported something similar with my 7800X3D when loading Age of Empires 2 maps, insane temperature spikes despite very low watts, though no stability issues obviously, my AM5 has not crashed a single time since release). Forget P95 or CB for testing stability, shader compilation and map loading is where it's at.
 
I knew that some people were having troubles with 14th gen with fewer experiencing trouble with 13th, too, but this is much worse than I expected. Explains why I was able to find a cheap 13700k perhaps. Maybe I won't replace my 12900k after all.
 
Last edited:
So did Intel identify/clarify/publish the root cause. This seems to affect a significant number of users and doesn't seem to be going away

So far this is what I understood that affected customers could do:
  1. RMA
  2. Overvolt & restrict current
  3. Underclock boost freq
  4. Disable e-cores
  5. Underclock memory speeds
  6. Replace 8gb Nvidia cards with more VRAM cards
  7. Replace weak Nvidia cards with more powerful Nvidia/AMD cards
Anything else ?
 
I don't think 6 and 7 do anything for those with genuine CPU issues. But all the first points help, without necessarily fixing what seems to be a deep problem. Like hardware flaw maybe.
 
When all if this started breaking a while back my first instinct was maybe a bad batch or something along those lines. Like some manufacturing runs were not up to par and they shipped anyway. As more information comes out it really does seem like the parts are just pushed to the raged edge form go and even good manufacturing can't save them all. Right now I have 2 LGA 1700 boards in use and of course both are on 12th gen i5s and work perfectly. At this point I may just top one off with a cheap 12900k or 14th gen i5. That or just run them as is till they don't do what I want. Intel seems like they are going to just move on the the next socket and walk away from all this. Maybe some more heat from Techtubers and industry people will help hold them accountable. Reddit threads full of rants don't seem to get anywhere.
 
Its good to see more coverage of this, its a problem that's been going on for months and keeps fading in and out of the spotlight.

I can't watch the video right now but until this point, I was under the impression that this was stemming from motherboard manufacturers and Intel not agreeing on what "stock" is and they were just doing whatever for default out of the box operation. In many cases, no power limits at all.

I know at least Asus has since released BIOS updates to be more conservative by default but now i'm not sure if this issue goes deeper.
 
Its good to see more coverage of this, its a problem that's been going on for months and keeps fading in and out of the spotlight.

I can't watch the video right now but until this point, I was under the impression that this was stemming from motherboard manufacturers and Intel not agreeing on what "stock" is and they were just doing whatever for default out of the box operation. In many cases, no power limits at all.

I know at least Asus has since released BIOS updates to be more conservative by default but now i'm not sure if this issue goes deeper.

My understanding from reading is that Intel originally blamed it on motherboards, and it initially looked like a similar issue to what was going on with AMD CPU's when motherboard manufacturers were recently running stock voltages too high resulting in degraded and killed CPU's over time (GamersNexus has an excellent series on this), but there appears to be much more to it than that, as evidenced by similar crash rates on super conservative Supermicro W680 based motherboards.

The new BIOS:es with tweaked settings apparently help, but not much. More extreme measures like disabling E cores help a little as well, but again not much. The only tweak that appears to really help a lot is drastically reducing RAM clocks, but this comes at a significant performance premium.

There simply appears to be something fundamentally wrong with these CPU's.

If intel is secretly admitting to large scale integrators to expect that up to 25% of CPU's are bad (and the real rate seems much higher than that) there really is something to it.
 
My understanding from reading is that Intel originally blamed it on motherboards, and it initially looked like a similar issue to what was going on with AMD CPU's when motherboard manufacturers were recently running stock voltages too high resulting in degraded and killed CPU's over time (GamersNexus has an excellent series on this), but there appears to be much more to it than that, as evidenced by similar crash rates on super conservative Supermicro W680 based motherboards.
It really did look like that too since the boards were juicing the processors as hard as they could, and only having thermals at the limiting factor. Also some people, like me, noticed the limits were disabled, turned them on, and have had no issues. So it did sound like it was a board thing.

Unfortunate that is is a larger problem. I'm crossing my fingers I don't develop any issues because all in all I really like my 13900 build and barring issues I see no reason to reaplce it for a number of years. However if it starts crashing all the time, I'll probably have to look at something new because that'll annoy the hell out of me.
 
It's a shame. I really like my 13600k and 14700k, but I wish I had waited another generation. Hopefully Intel can get this sorted, or at least correct it in the next gen.
 
It's a shame. I really like my 13600k and 14700k, but I wish I had waited another generation. Hopefully Intel can get this sorted, or at least correct it in the next gen.
Whole new architecture on a whole new process for all new problems I’m sure.

I’m holding off on a 7800x3d to see how it shapes up all the same.
 
Whole new architecture on a whole new process for all new problems I’m sure.

I’m holding off on a 7800x3d to see how it shapes up all the same.

I'm due for an upgrade, but for various reasons it is going to have to wait. (just spent my upgrade budget on a major HVAC project for the house, and really want to find a motherboard that has a secondary 8x PCIe slot that works at the same time as a 16x Video card without dropping it down to 8x)

If I were shopping today though, I'd probably just hold off for the Ryzen 9xxx's. The launch isn't that far off, and the leaks look very promising.
 
Last edited:
I just got two 13900hx (8p, 16e cores) notebooks 4 months ago which I love. If they end up affected, I hope they do so under warranty :eek:. They've been 100% fine but I'm still worried.
 
I just got two 13900hx (8p, 16e cores) notebooks 4 months ago which I love. If they end up affected, I hope they do so under warranty :eek:. They've been 100% fine but I'm still worried.
the mobile series seems to be fine so far strictly the desktop bits, I suspect the voltage and power limits on mobile don't allow it to get to a place where it can be problematic
 
You know it's a big problem when Intel can't figure out what's wrong with their CPUs. I think Intel has a hardware flaw and they need to do a massive recall but are avoiding it.
This, they wont because it will cost them a fortune, and sadly most companies that buy intel for DC's and such, either arent impacted by this, or what would be involved for them to trade out gear...massive scale and work.
 
This, they wont because it will cost them a fortune, and sadly most companies that buy intel for DC's and such, either arent impacted by this, or what would be involved for them to trade out gear...massive scale and work.
More likely they won't because it's not the entire lineup probably closer to 1/3'rd than 1/4 and it could very well be a series of batches from one of the plants that had a bad QC process and they released failed silicon that should never have entered the wild to begin with. It is easier and cheaper for everybody to deal with those units as they come in and extend the warranty for suspected units than do a full recall.

Besides if they recall them, what do they replace them with?
 
1720724814788.jpeg
 
I highly doubt this is any sort of manufacturing or QC problem. This is just Intel juicing the CPUs as high as they can go and maybe a little bit more in order to get performance as high as possible for benchmarking purposes. It's not the first time they've done this. Motherboard makers didn't help things because they were pushing the CPUs even farther to make their boards look better than someone else's.

Intel can claim it's a bug of some sort but it likely boils down to the boost algorithm being way too aggressive. I'm sure they knew it was going to degrade CPUs but nowhere near as fast as it has. It seems most of the worst affected CPUs are the very upper tiers which were already at the edge of stability. The lower tiered CPUs don't get pushed to the ragged edge at stock and also don't generate nearly as much heat (nor do they use as much voltage) which is why they are generally fine.

Intel overclocked the highest tiered CPUs to the what they thought was the edge and it resulted in the CPUs being pushed over the edge outside of some of the best silicon.
 
So did Intel identify/clarify/publish the root cause. This seems to affect a significant number of users and doesn't seem to be going away

So far this is what I understood that affected customers could do:
  1. RMA
  2. Overvolt & restrict current
  3. Underclock boost freq
  4. Disable e-cores
  5. Underclock memory speeds
  6. Replace 8gb Nvidia cards with more VRAM cards
  7. Replace weak Nvidia cards with more powerful Nvidia/AMD cards
Anything else ?
Intel did not want to RMA my 13900k.

They simply told me to set "Intel Fail Safe" in the BIOS which cranked the vcore to 1.6v and that TBH did fix the issue... For now.
 
Intel did not want to RMA my 13900k.

They simply told me to set "Intel Fail Safe" in the BIOS which cranked the vcore to 1.6v and that TBH did fix the issue... For now.
That sounds like a terrible solution.

I haven't had any issues with my 13700k but tbh I haven't had time to do anything on it in months either. This whole scenario has me second guessing my choice of platforms to say the least.

Edit- if the normies actually catch wind of this, Intel sales are going to die off quick like. Posing an obviously massive dual headed loss for them.
 
Last edited:
This, they wont because it will cost them a fortune, and sadly most companies that buy intel for DC's and such, either arent impacted by this, or what would be involved for them to trade out gear...massive scale and work.
I'm almost certain this will lead to a class action lawsuit. I think the flaw is something that requires Intel to make entirely new chips, but that's going to cost Intel a lot of money to send out flawless chips. On the other hand a class action lawsuit may result in Intel having to send everyone out $20 checks, and maybe coupon to buy a new chip from Intel. If I was a owner of one of these chips, I'd want either a new flawless chip or my money back. What's the point of warranty when Intel can weasel out of it?
I highly doubt this is any sort of manufacturing or QC problem. This is just Intel juicing the CPUs as high as they can go and maybe a little bit more in order to get performance as high as possible for benchmarking purposes. It's not the first time they've done this. Motherboard makers didn't help things because they were pushing the CPUs even farther to make their boards look better than someone else's.
Juicing up a CPU and it being unstable is in fact a defective product. I've overclocked in the past and damaged CPU's because they can't handle that amount of power draw without causing permanent damage. I feel this is the same problem.
Intel overclocked the highest tiered CPUs to the what they thought was the edge and it resulted in the CPUs being pushed over the edge outside of some of the best silicon.
This is probably because Intel is trying to stay competitive with AMD and like most companies when their product is slower they just pump up the clock speed, but this can quickly degrade the chip. Meanwhile AMD is putting in level 4 cache and getting crazy performance from it, without degrading the chip.
 
I'm almost certain this will lead to a class action lawsuit. I think the flaw is something that requires Intel to make entirely new chips, but that's going to cost Intel a lot of money to send out flawless chips. On the other hand a class action lawsuit may result in Intel having to send everyone out $20 checks, and maybe coupon to buy a new chip from Intel. If I was a owner of one of these chips, I'd want either a new flawless chip or my money back. What's the point of warranty when Intel can weasel out of it?

Juicing up a CPU and it being unstable is in fact a defective product. I've overclocked in the past and damaged CPU's because they can't handle that amount of power draw without causing permanent damage. I feel this is the same problem.

This is probably because Intel is trying to stay competitive with AMD and like most companies when their product is slower they just pump up the clock speed, but this can quickly degrade the chip. Meanwhile AMD is putting in level 4 cache and getting crazy performance from it, without degrading the chip.
I could see an incentive from Intel to appease 13 and 14 gen owners, and offering the next gen on a completely different socket.
Here's 10% off a new cpu and you foot the bill for a new platform /s
 
I highly doubt this is any sort of manufacturing or QC problem. This is just Intel juicing the CPUs as high as they can go and maybe a little bit more in order to get performance as high as possible for benchmarking purposes. It's not the first time they've done this.

I remember this one time...

From the Pentium III Wikipedia page:
A 1.13 GHz version (S-Spec SL4HH) was released in mid-2000 but famously recalled after a collaboration between HardOCP and Tom's Hardware discovered various instabilities with the operation of the new CPU speed grade. The Coppermine core was unable to reliably reach the 1.13 GHz speed without various tweaks to the processor's microcode, effective cooling, higher voltage (1.75 V vs. 1.65 V), and specifically validated platforms. Intel only officially supported the processor on its own VC820 i820-based motherboard, but even this motherboard displayed instability in the independent tests of the hardware review sites. In benchmarks that were stable, performance was shown to be sub-par, with the 1.13 GHz CPU equalling a 1.0 GHz model. Tom's Hardware attributed this performance deficit to relaxed tuning of the CPU and motherboard to improve stability. Intel needed at least six months to resolve the problems using a new cD0 stepping and re-released 1.1 GHz and 1.13 GHz versions in 2001.

Some time has passed, but trying to crank up the voltage and push out chips that are unstable and may degrade as a result still seems to be the Intel approach if/when they lose the performance crown :p


If this is truly what is going on, I just don't understand how they thought they were going to get away with it. These issues must have shown up in internal validation...
 
Juicing up a CPU and it being unstable is in fact a defective product. I've overclocked in the past and damaged CPU's because they can't handle that amount of power draw without causing permanent damage. I feel this is the same problem.
Dumping too much voltage and current into a CPU is not defective manufacturing. Without the extra voltage and current the product would be fine but in this case wouldn't have clockspeeds as high. Pushing a CPU via overclocking past the limits of the silicon to the point where the CPU degrades is not a manufacturing defect or fault. The silicon and manufacturing process are not at fault. The act of overclocking is the fault. In this case it's Intel factory overclocking but that doesn't mean the silicon or the manufacturing process has a problem.

The base problem lies with Intel no matter what but the distinction between an innate hardware problem and an overly aggressive algorithm is important. This is basically a repeat of the Intel 1.13Ghz Coppermine fiasco. The AMD Thunderbird was slapping Intel around and in response Intel took the Coppermine 1.0Ghz CPU and overclocked it in order to squeeze a little bit more out because there was no other way of responding to AMD at that time. It didn't work because Coppermine was already maxed out at 1Ghz and outside of the silicon lottery couldn't go any higher. It was unstable and was recalled before it ever really released. This does not mean that Coppermine was flawed nor does it mean that it was a quality control issue or a manufacturing defect.
 
4.) This is not due to overclocking.
5.) Datasets from game servers using either Asus or Supermicro W680 chipsets (no overclocking support, more conservative designs) show similar crash rates.
6.) Crash rates are also increasing over time on the W680 game server datasets...

If those part are true, the 13900k-14900k had no room left at all and you would think consumer board with aggressive board would have more issues

These issues must have shown up in internal validation...
Crash rates are also increasing over time, maybe not that bad the first month ?
 
If this is truly what is going on, I just don't understand how they thought they were going to get away with it. These issues must have shown up in internal validation...
I suspect two things. The first is Intel didn't count on board manufacturers pushing the CPUs as hard as they did which is what made the problem show up earlier than expected. The second thing is either Intel only tested this using the absolute very best silicon available which showed much slower degradation which wouldn't have shown up so soon or Intel pushed this on silicon not properly binned.

This problem seems to be most prominent on xx900 CPUs which are the CPUs pushed the hardest. The lower tier CPUs aren't pushed to the same clock speeds and don't gobble up nearly as much power and therefore won't exhibit the same degradation since the conditions which cause the degradation aren't likely to be met.
 
4.) This is not due to overclocking.
5.) Datasets from game servers using either Asus or Supermicro W680 chipsets (no overclocking support, more conservative designs) show similar crash rates.
6.) Crash rates are also increasing over time on the W680 game server datasets...

If those part are true, the 13900k-14900k had no room left at all and you would think consumer board with aggressive board would have more issues


Crash rates are also increasing over time, maybe not that bad the first month ?
The CPUs have built in overclocking called boosting. It doesn't matter if the board doesn't have enthusiast level overclocking options or not because the CPU is set to push higher speeds along with additional voltage and current via an algorithm by design and in stock form.
 
The crash rates increasing over time would explain a lot of the posts I've seen in various forums. Happy owners suddenly reporting issues after formatting, changing paste, etc.
 
The CPUs have built in overclocking called boosting. It doesn't matter if the board doesn't have enthusiast level overclocking options or not because the CPU is set to push higher speeds along with additional voltage and current via an algorithm by design and in stock form.
I would have thought something like max voltage and power limits to be influenced by those Normal, performance type setting motherboard tend to have. Would it be simply that the CPU are pushed too much for benchmark the fix of pushing them less for those who want stabiltiy over performance should be relatively easy to make, it is probably deeper than that.

Wendel could be all wrong here obviously, seem to be pure speculation going from screenshots and second hands talks.
 
Last edited:
Wendel could be all wrong here obviously, seem to be pure speculation going from screenshots and second hands talks.

Did you watch the video? He directly talked to the people that sell these server parts to people and provide support for those business customers. They're jacking prices up for Intel platforms because support for them is proving more expensive, and urging them to move to AMD 7950X platforms instead. Seems to me like he also actually ran manual testing himself. The third party data for end users is gathered directly from crash report data that they let him have. This isn't secondhand screenshots. It's live data. Far from pure speculation. It's definitely a little speculative, but his conclusion is data driven. Considering he's not an engineer at Intel, and he's having to look from the outside in, this is probably about as good as it gets. It's a large data set with a lot of diagnostic data. The data center tidbit is even more direct than that.

There's some weird stuff going on in Intel land, and trying to claim otherwise at this point is just cope.
 
Yea I would prefer a text article over a video, but there is some data in this one (from general crash reports AND datacenters), and not just pure speculation. Of course he does a lot of that too, but can't blame him given the circumstances.
 
I have used a 13900K and now a 13700K and 14700K with zero issues (yes I'm going to be that guy). Not sure if it's luck or just because I only enable XMP and leave the other cpu settings in the bios at default. I have played all types of games and few more recent ones and have the same results.
 
Dumping too much voltage and current into a CPU is not defective manufacturing. Without the extra voltage and current the product would be fine but in this case wouldn't have clockspeeds as high. Pushing a CPU via overclocking past the limits of the silicon to the point where the CPU degrades is not a manufacturing defect or fault. The silicon and manufacturing process are not at fault. The act of overclocking is the fault. In this case it's Intel factory overclocking but that doesn't mean the silicon or the manufacturing process has a problem.

The base problem lies with Intel no matter what but the distinction between an innate hardware problem and an overly aggressive algorithm is important. This is basically a repeat of the Intel 1.13Ghz Coppermine fiasco. The AMD Thunderbird was slapping Intel around and in response Intel took the Coppermine 1.0Ghz CPU and overclocked it in order to squeeze a little bit more out because there was no other way of responding to AMD at that time. It didn't work because Coppermine was already maxed out at 1Ghz and outside of the silicon lottery couldn't go any higher. It was unstable and was recalled before it ever really released. This does not mean that Coppermine was flawed nor does it mean that it was a quality control issue or a manufacturing defect.

I remember that review. The CPU may have been a fail, but the cooler it came equipped with looked awesome at the time. I remember my own 1GHz Coppermine wouldn't post at anything over 1GHz no matter how much vcore I threw at it.
 
I have used a 13900K and now a 13700K and 14700K with zero issues (yes I'm going to be that guy). Not sure if it's luck or just because I only enable XMP and leave the other cpu settings in the bios at default. I have played all types of games and few more recent ones and have the same results.
If you're serious about the issue, I would run the high stress loads that Wendell mentioned in his vid to see.... and of course, there's no guarantee you'll see the issue until after many tests are done.
 
Back
Top