Hardware Unboxed Analyzes Intel's Commissioned Core i9-9900K Benchmarks

Discussion in '[H]ard|OCP Front Page News' started by cageymaru, Oct 9, 2018.

  1. cageymaru

    cageymaru [H]ard|News

    Messages:
    19,490
    Joined:
    Apr 10, 2003
    Intel has commissioned Principled Technologies to run benchmarks of the upcoming i9-9900K CPU and pit it against older Intel CPUs and some AMD CPUs such as the AMD Ryzen 2700X, Threadripper 2990WX, and Threadripper 2950X. PCGamesN wrote a glowing article that has since been heavily edited, about how Intel's new chip was up to 50% faster than AMD's best. Reading over the original PCGamesN article you would have thought that sliced bread had just been invented and Intel was delivering us from the Dark Ages. To PCGamesN's credit they did run this article acknowledging that Hardware Unboxed found bias in the benchmarks, but they still have the flawed and biased Principled Technologies charts showcased on their website for all to see.

    Hardware Unboxed did a short analysis of a few of the benchmarks as their team felt that the i7-8700K benchmarks and the AMD Ryzen 2700X numbers were incorrect. They found that Principled Technologies had allegedly gimped the AMD CPUs by using different coolers, incorrect ram timings, and possibly even disabled some of the cores on the AMD Ryzen 2700X. To put this into perspective, on the Ashes of the Singularity benchmark that Hardware Unboxed ran, the AMD Ryzen 2700X was 18% faster and the i7-8700K was 4% slower, than the commissioned testing that Intel has published. They even showed how over a suite of games that the i7-8700K was only 9% faster than the AMD Ryzen 2700X in previous pure gaming benchmarks conducted by Hardware Unboxed. Yet in Intel's commissioned benchmark results, the AMD Ryzen 2700X was far, far, behind the Intel i7-8700K in performance metrics. This is why we never trust a manufacturer's benchmarks. Always wait for the review before buying hardware.

    "Why is PCGamesN publishing this misleading data? Why aren't they tearing this obviously paid report; because they're very transparent that this is a paid report commissioned by Intel, why aren't they tearing it to shreds? Do they simply not know any better? I'm a bit worried or wondering if we might see other websites covering this report; perhaps paid to cover it. And I'm not accusing or saying that PCGamesN were paid by Intel. Doesn't look great and I suppose the very least it is very shoddy journalism."
     
  2. jardows

    jardows [H]ard|Gawd

    Messages:
    1,297
    Joined:
    Jun 10, 2015
    Intel sounds desperate. I'll wait for the [H] review.
     
  3. Armenius

    Armenius [H]ardForum Junkie

    Messages:
    16,262
    Joined:
    Jan 28, 2014
    AMD does the same thing. I agree with cageymaru that we should never trust any manufacturer's benchmarks, or those where money has traded hands to a third party as in this case.
     
  4. jmilcher

    jmilcher 2[H]4U

    Messages:
    3,902
    Joined:
    Feb 3, 2008
    Even if the performance numbers were significantly higher, it's coming in at double the cost of the 2700x.

    If you have to start advertising "5 GHz (2 core Max boost out of 8 physical cores ) " that looks meh to your enthusiast type consumer.

    Not to mention the price of the Z390 boards is nuts. And so far no one has found any significant changes from Z370. Intel doing it's thing again.

    I'm glad this round I settled on X470 and a 2700x. Now I'll wait for Zen 2 and pop one in my board.

    Let's go AMD, and competition. Consumers win.
     
  5. TheHig

    TheHig Limp Gawd

    Messages:
    389
    Joined:
    Apr 9, 2016

    Exactly. Everyone has cherry picked marketing benchmarks. The GOOD 3rd party reviews show where the rubber meets the road.

    Pretty shoddy article if the numbers are that far off though.
     
    AceGoober and Armenius like this.
  6. Monkey34

    Monkey34 [H]ardness Supreme

    Messages:
    5,011
    Joined:
    Apr 11, 2003
    Exactly.....but this only helps those who are at least moderately informed (and I hope the vast majority of us here are in that category). This kind of craptastic reporting, misleading benchmarking, and manufacturer self-gratification preys on the weak.

    Be the first to throw out a "we're so much better" headline, and the uninformed masses fall for it; leaving the subsequent headlines to try and play damage control.
     
  7. ccityinstaller

    ccityinstaller 2[H]4U

    Messages:
    3,631
    Joined:
    Feb 23, 2007
    Well said.
     
  8. pgaster

    pgaster [H]ard|Gawd

    Messages:
    1,300
    Joined:
    May 17, 2008
    "AMD does the same thing."

    I don't think so.


    Check out the latest post on youtube:

    " It's even worse than we thought, Principled Technologies are running the 2700X as a quad-core. "
     
  9. 1_rick

    1_rick Limp Gawd

    Messages:
    361
    Joined:
    Feb 7, 2017
    PCGamesN has now updated the showcase article to mention the disputed scores.

    If you click through Hardware Unboxed's Patreon link from the video, he says that "Principled" Technologies enabled XMP and then adjusted timings for the Intel system, but not for AMD.
     
  10. MaZa

    MaZa 2[H]4U

    Messages:
    2,553
    Joined:
    Sep 21, 2008
    I'm not so sure about that. Manufacturers giving false impressions in their marketing graphs is nothing new, everybody does that. But they are often very vague what they were testing against on purpose because it is AFAIK illegal in many places to give pure false information about your competitions product. IE in hypothetical situstion if Nvidia were claiming that their new GTX9001 does 500fps in game X while at the same time claiming AMD HD9000 does only 100fps while in reality it does 450fps Nvidia would get sued on the spot. I do not remember AMD doing this (correct me if I am wrong) but Intel is doing this right now and only reason they get around lawsuits is because its a 3rd party doing the "testing". Plausible deniablity and all that.
     
  11. thebufenator

    thebufenator [H]ard|Gawd

    Messages:
    1,028
    Joined:
    Dec 8, 2004
    Is there evidence of AMD gimping Intel hardware on tests? Or just showing tests that show AMD in a good light?

    This seems to clearly indicate that AMD hardware was gimped for the "benchmarks".
     
    testicle, Darth Kyrie, N4CR and 7 others like this.
  12. Shmee

    Shmee [H]ard|Gawd

    Messages:
    1,117
    Joined:
    Sep 12, 2014
    Always wait for the real reviews. I am sure the Intel chip will be fast, just not as fast as they are saying it is.
     
    cageymaru likes this.
  13. DrBorg

    DrBorg Gawd

    Messages:
    547
    Joined:
    Jan 22, 2005
    Regardless of how fast this chip is; it still has No internal mitigations of all the various attacks, and with the microcode and software mitigations for those problems, will be 20-40% slower than without them.

    I wouldn't have even released this.

    Gotta feed the suckers, tho: "OOH! It says 5GHZ!! * "

    * (unreadable fine print that includes the slower actual speed)
     
  14. cageymaru

    cageymaru [H]ard|News

    Messages:
    19,490
    Joined:
    Apr 10, 2003
    Skip to the 5 minute mark if in a rush to see some benchmarks by MSI. If you watch the entire thing Kyle_Bennett talks about some of the performance numbers being tossed around.

     
  15. ir0nw0lf

    ir0nw0lf [H]ardness Supreme

    Messages:
    6,150
    Joined:
    Feb 7, 2003
    HUGE UPDATE from Hardware Unboxed, this is getting really good!

    https://www.patreon.com/posts/21950120

    Top tidbit:
     
  16. JosiahBradley

    JosiahBradley [H]ard|Gawd

    Messages:
    1,653
    Joined:
    Mar 19, 2006
    PC gaming getting as bad as friggin politics. I just want a fast PC, but now I have to do wallet voting to stop the madness. Ryzen 2 here I come.
     
    Dudhunter, HockeyJon, Denjoy and 9 others like this.
  17. Kyle_Bennett

    Kyle_Bennett El Chingón Staff Member

    Messages:
    54,890
    Joined:
    May 18, 1997
    Wow, if you are going to hire a third party to do benchmarks for you, make sure they are not idiots.
     
  18. Stoly

    Stoly [H]ardness Supreme

    Messages:
    6,019
    Joined:
    Jul 26, 2005
    Anyone remember the first bulldozer benchmarks by AMD? Recall the "Massive Overclocking"?
     
    AceGoober likes this.
  19. ir0nw0lf

    ir0nw0lf [H]ardness Supreme

    Messages:
    6,150
    Joined:
    Feb 7, 2003
    This is why we are happy to have you brutha. :woot:
     
  20. DukenukemX

    DukenukemX 2[H]4U

    Messages:
    3,997
    Joined:
    Jan 30, 2005
    Why doesn't PCGamesN throw these tests away? Short answer, capitalism. Long answer, nobody knows who PCGamesN is so getting paid by Intel compensates for the lack of people visiting their site. Benchmark tests are very manipulative as shown by AdoredTV, so every website seems to have different results.
     
  21. Nightfire

    Nightfire [H]ard|Gawd

    Messages:
    1,031
    Joined:
    Sep 7, 2017
    Cageymaru is all over it, I was just going to post this. We all know the 9900k is going to be faster the the 2700x, but do they really need to fudge the results??

    I have never heard of them but PCgamesN seems to be a bunch of shills.

    Looking forward to the epic ball busting that will be done by Gamers Nexus
     
    dgz, dragonstongue and Madmeerkat55 like this.
  22. ManofGod

    ManofGod [H]ardForum Junkie

    Messages:
    9,593
    Joined:
    Oct 4, 2007
    LOL! Intel getting desperate, perhaps?

    Not really. Also, I know they did get massive overclocking but, that was strictly with LN2.
     
  23. Teenk9

    Teenk9 Gawd

    Messages:
    939
    Joined:
    Jul 22, 2011
    But did they gimp the other company's product to make it look bad?
     
  24. Nightfire

    Nightfire [H]ard|Gawd

    Messages:
    1,031
    Joined:
    Sep 7, 2017
    We can stop with the moral equivelancy defense. Every time brand x does something shady, we don't need a "well brand y does the same thing!!", especially without giving relative examples.
     
    jnemesh, Space_Ranger, Creig and 12 others like this.
  25. Kyle_Bennett

    Kyle_Bennett El Chingón Staff Member

    Messages:
    54,890
    Joined:
    May 18, 1997
    I would lean way towards idiocy rather than conspiracy. Intel already has the fastest gaming CPU on the planet with the last gen parts.
     
    Space_Ranger, Maxx, Chunder and 10 others like this.
  26. kju1

    kju1 2[H]4U

    Messages:
    2,984
    Joined:
    Mar 27, 2002
    A paid review is a influenced review. Even if its not directly influenced you will subconsciously moderate yourself so as to not bite the hand(s) that feed you. Some exceptions do apply for cranky old guys ;)
     
    jnemesh and PaulP like this.
  27. sirmonkey1985

    sirmonkey1985 [H]ard|DCer of the Month - July 2010

    Messages:
    20,845
    Joined:
    Sep 13, 2008
    typically it's just fudging the graphs so that 10% looks more like 50% in a normal graph.. even though i'm not an intel fan i can't possibly see this being 100% intentional by intel.. this feels more like complete and utter incompetence by PCgamesN.. cough like the 2000 dollar computer being built by some one that's never used anything other than apple products...
     
  28. MaZa

    MaZa 2[H]4U

    Messages:
    2,553
    Joined:
    Sep 21, 2008

    True. You can make a mountain out of a mole hill if you zoom in close enough, but that is what I was referring to when I mentioned "false impression". But as long as the numbers themselves are accurate and no number was intentionally gimped then that is okay, it is up to the viewer to keep their eyes open and understand what they are seeing correctly. The zoomed in graph is technically still not a lie.

    If the PCgamesN test came from Intel or not is impossible to tell unless someone leaks a paycheck or email. Plausible deniability, Intel can play dumb and nobody can prove a thing. :D
     
  29. umeng2002

    umeng2002 Gawd

    Messages:
    819
    Joined:
    May 23, 2008
    Next we'll find out that Ryzen raped an AMD intern in high school.
     
  30. MaZa

    MaZa 2[H]4U

    Messages:
    2,553
    Joined:
    Sep 21, 2008
    Holy crap, maybe I gave Intel too much credit for the "plausible deniability" thing, because Forbes also covered this debacle and Intel PR rep actually contacted them and had balls to claim that the tests were "correct" and in spec? WTF!?

    https://www.forbes.com/sites/jasone...enchmarks-are-misleading-period/#3be794c84e4e

     
  31. brink668

    brink668 n00bie

    Messages:
    12
    Joined:
    Jun 23, 2006
    Game Mode is listed on the page 16 of the PDF report. Obviously purposeful.

    https://www.principledtechnologies.com/Intel/PC_gaming_processor_study_interim_1018.pdf

    "On AMD systems download and install the AMD Ryzen Master Utility.
    a. Launch the utility, select Game mode, and click Apply."


    Also clearly states "PC gaming processor study: Intel Core i9-9900K processor vs. competitors Commissioned by Intel Corp. October 2018"

    Edited: For context
     
  32. kju1

    kju1 2[H]4U

    Messages:
    2,984
    Joined:
    Mar 27, 2002
    jnemesh, Dudhunter, Shadowed and 4 others like this.
  33. AceGoober

    AceGoober Live! Laug[H]! Overclock!

    Messages:
    23,144
    Joined:
    Jun 25, 2003
    I'll wait for independent review from [H] before coming to any conclusion. Either way, the benchmarks seem way off.
     
  34. Nightfire

    Nightfire [H]ard|Gawd

    Messages:
    1,031
    Joined:
    Sep 7, 2017
    I think that is the point. Most of the gamers would have bought the 8700k already. Intel would like to find more people to dump their 8700k for a shiny new 9900k.

    They are not so bold to gimp the 8700k directly, but if the new chip is 30-50% (edit) faster than the 2700x at gaming, surely it is faster than the 8700k!
     
    Last edited: Oct 9, 2018
  35. fs123

    fs123 Kyle is a Bad Motherfucker

    Messages:
    125
    Joined:
    Jun 26, 2010
    What this means is that Intel has given the testers some 'recommended settings' to replicate their in-house results. The blame appears to lie squarely with Intel here. The fact that Intel is claiming 50% better performance than the 2700X suggests that this is no mistake.

    Surely blatant misrepresentation like this must be a sue-able offense.
     
    jnemesh, Dudhunter, testicle and 5 others like this.
  36. Gideon

    Gideon [H]ard|Gawd

    Messages:
    1,801
    Joined:
    Apr 13, 2006
    I agree with you and looks like many others, then you look at some of their posts and you see them defending a manufactures benchmarks as fact. You can look at the Nvidia launch and see some do just that, thus why we need places like the [H] to show us the truth in the performance to expect. Always understand that a manufacturer is always going to show their new product in the best light possible. A paid third party site is really almost never a good source for solid info on a new product. Independent news is more important then most think and usually pretty close to what you should expect if you buy it.
     
    jnemesh likes this.
  37. Joust

    Joust [H]ard|Gawd

    Messages:
    1,561
    Joined:
    Nov 30, 2017
    Yeah...but the appearance of impropriety is pretty potent. Conspiracy is strong. Unsupervised sponsored negligent testing favoring the sponsoring party...that just doesn't taste very good.
     
    jnemesh and Dudhunter like this.
  38. sirmonkey1985

    sirmonkey1985 [H]ard|DCer of the Month - July 2010

    Messages:
    20,845
    Joined:
    Sep 13, 2008
    lol i foresee an intel PR rep looking for a new job by the end of the week..
     
    jnemesh and LightsOut41 like this.
  39. Advil

    Advil [H]ard|Gawd

    Messages:
    1,813
    Joined:
    Jul 16, 2004
    Yeah, and I have a nicely modded 8700k, but when right now today using Amazon and Newegg as examples a 2700x is $299 on Amazon and the 9900k is $579 on Newegg...

    "Houston we have a marketing problem!"

    If they can't somehow claim it's around 50% faster that price tag is nearly impossible to swallow.

    And it doesn't help that the 9900k is essentially the successor to the 8700k whether or not Intel calls it an "i9". The removal of the HT from the 9700k makes it more of an oddity than a comparable product. It's an "i5+" by any enthusiast's measure. It takes time to shift the specs of an entire product space and make people ok with it.
     
    DF-1, jnemesh and Nightfire like this.
  40. bigdogchris

    bigdogchris Wii was a Novelty

    Messages:
    17,598
    Joined:
    Feb 19, 2008
    https://www.pcgamesn.com/nvidia-confirms-gpp-doesnt-prevent-amd-gaming-brands

    https://www.pcgamesn.com/nvidia-geforce-partner-program-amd-impact

     
    Last edited: Oct 9, 2018