Principled Technologies Deny Bias in Recent Intel vs. AMD Testing Controversy

Discussion in '[H]ard|OCP Front Page News' started by cageymaru, Oct 10, 2018.

  1. cageymaru

    cageymaru [H]ard|News

    Messages:
    19,110
    Joined:
    Apr 10, 2003
    Principled Technologies has issued a statement about the testing methodology that they employed for the commissioned Intel report. First they address the accusations of bias by stating that they have worked for AMD, Intel, HP, Dell, etc in the past. Then they addressed issues such as not matching CPU coolers during testing, using Game Mode on AMD Ryzen, memory speeds and why they used 4 sticks, XMP usage, resolution, quality settings, motherboards used, etc. Interesting read for those interested in how they arrived at their conclusions.

    For almost 16 years, we have tested products for our clients because they trust our integrity. We have worked not just for any one company but for dozens of the leading technology firms, including rivals such as Intel and AMD, Microsoft and Google, Dell and HP, and many others. Those clients trust PT in part because our integrity and our technical knowledge are beyond reproach. We work hard to be the best in both of those areas. We chose our company name to emphasize our commitments to both technology and our principles. Before going further, we thus must categorically deny any dishonesty in our work on this project for Intel or in any of our other projects.
     
    viscountalpha and GDI Lord like this.
  2. Pieter3dnow

    Pieter3dnow [H]ardness Supreme

    Messages:
    6,362
    Joined:
    Jul 29, 2009
    "facts don't matter" deal with it.
    Much shorter ;)
     
    Sulphademus, Rahh, jnemesh and 8 others like this.
  3. Sikkyu

    Sikkyu I Question Reality

    Messages:
    2,901
    Joined:
    Jan 21, 2010
    "we don't check our work, we don't answer to you, go fuck yourself"
     
  4. OnceSetThisCannotChange

    OnceSetThisCannotChange [H]Lite

    Messages:
    107
    Joined:
    Sep 15, 2017
    In other words, they work hard, they were not dishonest.

    They are showing that in principle it's possible to cut performance of your system up to 50%, and maybe even more, if you work hard enough.

    Intel clearly appreciated the effort.
     
  5. Chebsy

    Chebsy Gawd

    Messages:
    534
    Joined:
    Jan 24, 2013
    This guy says he has been testing hardware longer that you (the interviewer) have been alive, fine, but is that is true, why doesn't he know basic details about CPU coolers, case air flow and thermal throttling and the impact of that on performance ? What has he been testing all this time ?
     
  6. MMitch

    MMitch Limp Gawd

    Messages:
    500
    Joined:
    Nov 29, 2016
    Well, this guy has balls for sure... sad that he didn't know anything. He should have brought his best employee to pass him some answers..
    Anyway, pretty sure the fact that he couldn't answer anything is the key to say dumb things or get the cat out the bag.

    GN did an amazing job and was patient, I would have gone nuts been told the same thing over and over lol.
     
    Vokar, Rahh, Malurt and 7 others like this.
  7. SixFootDuo

    SixFootDuo [H]ardness Supreme

    Messages:
    5,378
    Joined:
    Oct 5, 2004
    I would be unapologetic ASF. People need to stop answering to others, apologizing. If anything, I would just say, Intel is faster, period. Even the long haired guy at Gamer's Nexus very grudgingly admitted over and over in an interview the day before that Intel is going to be faster. That they didn't have to test AMD in the weird ways they did.

    Give me one of these 9900k's and I promise I could sell a gazillion of them for Intel using readily available PC parts off the shelf.

    I personally think it will be possible to get 5.1 or 5.2GHz across all cores with the S-TIM in place and good cooling. I might even add 3 more fans to my Corsair H150i Pro for a push / pull config. Plenty of room in my Corsair Obsidian 500d Premium case which I absolutely love.
     
    Mav451 likes this.
  8. Master_shake_

    Master_shake_ Little Bitch

    Messages:
    7,763
    Joined:
    Apr 9, 2012
    our client asked us to show their product in the best light.

    money is way way more important then our self worth or image.

    TL;DR i got mine bitch.
     
  9. viper1152012

    viper1152012 [H]ard|Gawd

    Messages:
    1,031
    Joined:
    Jun 20, 2012
    They should stop using labs that don't do gaming reviews because this guy was clearly missing some information about the testing methodology.

    For a gaming facing review Intel should have had a known reviewer just do it
     
  10. adder1971

    adder1971 n00b

    Messages:
    32
    Joined:
    Nov 19, 2010
    Steve was taking him to School! lol
     
  11. Kyle_Bennett

    Kyle_Bennett Editor-in-Chief HardOCP Staff Member

    Messages:
    52,053
    Joined:
    May 18, 1997
    I made it till he said he would not be able to answer a bunch of the tech questions. As I posted in the other thread, the outrage over this eludes me. This is literally what the analysts have been doing for decades. Some do it well, others obviously do a bad job of it. Test this and make up look good, get paid. All companies do it. How do you think analyst groups make millions of dollars a year?
     
  12. cjcox

    cjcox [H]ard|Gawd

    Messages:
    1,073
    Joined:
    Jun 7, 2004
    Now calling for a one week FBI investigation and based on that, we can move forward...
     
  13. DTN107

    DTN107 [H]ardness Supreme

    Messages:
    4,669
    Joined:
    Jun 30, 2008
    Ah dang... they are located in Durham, NC.

    Pretty much right down the road from me.
     
    MHzTweaker and cageymaru like this.
  14. psyclist

    psyclist Gawd

    Messages:
    851
    Joined:
    Jan 25, 2005
    And thats why you arent high up and public facing in a company... So in the face of the scientific method, you would just stick your fingers in your ears, close your eyes and just go your own way, because you know best? Jesus... Pride, its a funny thing at times... You can tell that dude didnt have a leg to stand on, he was just trying to smooth things over. It still stands they fucked up the testing hardcore. Absolutely the 9900k is gonna be faster...why they gotta lie then? Just so much BS out of the blue team as of late to mislead its users.

    I hope folks enjoy the new chips, I love new hardware, but dont expect a huge gap like what youve seen here. Im interested to see the other side as well, not just games. At these prices the 2700X is where my money would be and upgrade the GPU, much better gaming performance gain staying at the same budget.
     
  15. STEvil

    STEvil 2[H]4U

    Messages:
    2,816
    Joined:
    Oct 17, 2000
    Cant wait to get home and watch this
     
  16. Twisted Kidney

    Twisted Kidney 2[H]4U

    Messages:
    3,552
    Joined:
    Mar 18, 2013
    His hair is so... Majestic...
     
  17. alamox

    alamox Gawd

    Messages:
    612
    Joined:
    Jun 6, 2014
    the choices we made they put AMD in a disadvantage and intel in a clear advantage, but they are all innocent choices made to keep a level plain field, i can't really get into technical stuff and tell you how many choices fit this description, my assistant here will look into it, but i can assure you that no matter what you think and as long as i keep repeating that we did what we did to keep a level plain field, then we are golden, even when you see a weird choice completely counter intuitive, keep in mind that we did it to keep a level plain field, that's all i can say, and also your camera makes me nervous.
    ...seriously all this political correctness is absurd, you don't need to acuse him, just be blant and lay it out to him, look dude the sheer number of weird choices you made that favors iIntel and cripples AMD, is weirdly high, so high in fact that it's hard to believe it's a product of incompetence and feels alot more targeted, care to comment ?
    the same crap in politics and other stuff we see in the media, interviewer keep it so civil that the guy spends his time tap dancing around him and avoiding question with a universal answer broad enough to be a best selling fortune cookie.
     
    Last edited: Oct 10, 2018
    cyclone3d and adder1971 like this.
  18. Motley

    Motley 2[H]4U

    Messages:
    2,431
    Joined:
    Mar 29, 2005
    Oh boy, first the Verge, now Principled Technologies. They were able to crush their credibility in one fail swoop. haha
     
  19. Formula.350

    Formula.350 [H]ard|Gawd

    Messages:
    1,141
    Joined:
    Sep 30, 2011
    I totally agree, but I don't know specifically if I'd say PT or Intel lied... They just, well just as you said, stated BS to mislead consumers.
    But yea, the Co-Founder, I understood the logic behind him doing the interview, except he's clearly not the man to be doing it either. HOWEVER, clearly they weren't a competent company to be doing these tests, either, or more fairly that their employee(s) weren't. The whole 'logic' behind using the AMD Wraith on the 2700X was so painful... lol

    That's also taking their and Intel's word for it on whether they were instructed to do these this way. If it was indeed PT's choice to do it this way, that definitively says that whomever devised those test configurations and conditions was not the person to do it. I don't know what Intel is generally like when it comes to reviews and whether they apply strong arm tactics on how the review and tests need to be conducted, so I can't speak to how likely it'd be for Intel to do that sort of thing.

    It's as Kyle said, the numbers that get used in a launch event by a company are generally (I dare say always) going to only be cherry picked results to show their product in the best light. That's nature of the beast. I think where the issue comes from is that there was (in my opinion) a bit more transparency on the test systems than what people are generally used to seeing, and it in turn outlined why exactly the numbers that are used aren't reliable. What I've come to expect from the fine print is "CPU A, B, C were used, with X-amount of RAM, and Y Graphics card", and maybe what heatsink, but I dunno. So I'm surprised that what case used was mentioned.

    I mean I applaud that they were so transparent, but it just illustrated the shady practices that go on in these situations. Coming off the heels of the whole nVidia GPP fiasco, I think part of why this is being made a big deal is that people are a bit sick of companies slinging so much bullshit, particularly when the companies are doing it all the while puffing their chests up and boast of their products superiority. In this case, if it's superior, it will damn well be evident to everyone even with a totally level playing field! This is just Intel trying to save face and still appear to have a dominating lead over AMD like during the tail end of Phenom II years and during the whole Bulldozer era.

    It's as if winning the race just isn't good enough anymore unless you've also spun your rival into the wall.
     
    psyclist and dragonstongue like this.
  20. LightsOut41

    LightsOut41 Limp Gawd

    Messages:
    423
    Joined:
    Mar 5, 2017
    "We didn't lie. We're just incompetent."
     
  21. DrBorg

    DrBorg Gawd

    Messages:
    566
    Joined:
    Jan 22, 2005
    Fleshlights, judging from his work.

    I'm 55, been building PC's since I slaved to afford an 8088 DFI clone in about 1988; I've bought an upgraded PC every two years since then, until 2011.

    There's not enough improvement since then to warrant an upgrade, but AMD is looking Great again.

    My last AMD system was a Barton core 2500M pricessor, 25W at 1.8GHz, andit Easily overclocked to over 3GHz.

    I've replaced the caps in that mobo twice; it's still running. It has PCI DataQ cards in it, and doesn't need any more power.

    I still have my Socket-7 mobos, they have ISA slots with PCA3 cards in them, lol. (no one outside the nuclear instrumentation crowd even knows what those are anymore.)


    Just because you've been doing something for a long time Does Not mean you're good at it, much as you can give someone the best education, but you can't make them smart.

    I've worked with some of the dumbest people, every one of them had an advanced degree. :zdunno: A 1" thick piece of aluminum is Not going to hold 10 tons in a dynamic shear application, not matter how much the math tells you so. :)


    This guys arguments are feeble at best; Intentions are hard to prove, but it was still a hatchet job, any way you look at it.
     
  22. cageymaru

    cageymaru [H]ard|News

    Messages:
    19,110
    Joined:
    Apr 10, 2003
    A few hour's ride from me. Now I wonder where Gamer's Nexus is located...
     
    Vader1975 likes this.
  23. DrBorg

    DrBorg Gawd

    Messages:
    566
    Joined:
    Jan 22, 2005
    Drop in with Beer/wine/JackD at the end of the day; I've never been thrown out of an office using that technique. :)

    Research their likes first; that's so easy these days, lol.

    I miss the days you could call the office Manager and Prank people, now everyone thinks you're Social Engineering their network.


    I researched their inner email structure, and sent email to the head of Analog Devices, years ago, bitching about their product availability, but was pleasant enough to get a reply. :)

    I met him years later, at a different job, and he remembered my name, lol. (AD made custom chips for us; and then it was LT who made them look bad: http://www.analog.com/media/en/technical-documentation/data-sheets/6090fe.pdf.)

    He said they completely reworked their email in response. :)
     
  24. Chebsy

    Chebsy Gawd

    Messages:
    534
    Joined:
    Jan 24, 2013
    The other thing that was wrong with this test is that it was run at 1080p. Everyone knows that Ryzen CPU's are weaker than most intel CPU's at this resolution. The guy said all through the interview that he wanted to just represent the average gamer. The "average gamer" who buys the latest flagship Intel CPU is not going to play at 1080p. The test was clearly devised to show AMD CPU's in a bad light. Some will say that doing the test at 1080p removes the GPU bottleneck which is true, but it still isn't what the avarage gamer would use with the setup they were using.
     
    F.E.A.R., Zxubair, N4CR and 3 others like this.
  25. SixFootDuo

    SixFootDuo [H]ardness Supreme

    Messages:
    5,378
    Joined:
    Oct 5, 2004
    The older I get, the less explaining I give. So that's my mindset. I've learned you don't always have to answer someone and give them reasons or validation. The PC community is already well versed enough that a majority of people will be able to accurately read into those numbers. And they have. The boat corrected itself. I think this man, and the company he spoke for would have just been better off not attempting to provide any additional information.

    Like Kyle said, companies do this. It's all around us. Companies use radio, television and print to constantly BS us every single minute of every hour, each and every day. It's a non-stop barrage.

    Also, web sites like Gamer's Nexus, which I happen to really like and enjoy ( other than his constant snarkiness and obvious contempt ) is doing his best grandstanding for subs just like anyone else. It's all just one big dance that we all do that at the end of the day when you break it down, is all about really nothing. A few FPS here, a few FPS there.

    Had there not been a response this would have blown over in a few days. If you know ahead of time people may not agree, then don't give them any momentum whatsoever.
     
    Mav451, Patton187, Aioeyu and 2 others like this.
  26. Denpepe

    Denpepe Gawd

    Messages:
    917
    Joined:
    Oct 26, 2015
    While I can agree there were a lot of errors in their tests and methods, I don't agree with everything Steve from GN said either, he was picking on some minute things it started to get silly an he was loosing some credibility to the guy.
     
  27. Z3r0k3wl

    Z3r0k3wl [H]Lite

    Messages:
    125
    Joined:
    Feb 13, 2014
    Is it rational for someone who deals in absolutes. That's what I got from this old guy who thinks this interview has been irrational?
     
  28. oldmanbal

    oldmanbal [H]ard|Gawd

    Messages:
    2,044
    Joined:
    Aug 27, 2010
    Only thing I read a new cpu review for is to see the performance of the new cpu. Couldn't care less about their one time numbers for an amd part that has thousands of reviews, benchmarks, and comparisons already posted on the net. In my opinion, they did what they felt was a valid comparison. I don't agree with everyones testing methods, and am one of the people that don't skip straight to the benchmarks, but take pride in reading their interpretation of how a benchmarking session should be setup, regardless of how long and droning it may come across as (lets face it, most hardware reviewers aren't exactly turning their content into a rick and morty episode.

    Not everyone enjoys the engineering behind parts, and just wants to see if they're gonna get the highest fps in given game, or a general xx% increase over their current hardware. Either way, you're only as good a consumer as you want to be. If you think that consumers are being taken advantage of because they don't research their buying decisions? With everything so general, unless there's a wanton deception taking place, it's all fair in marketing terms. That's a lot different than selling a 3.5gb videocard as a 4gb product.
     
    psyclist likes this.
  29. IdiotInCharge

    IdiotInCharge [H]ardForum Junkie

    Messages:
    8,359
    Joined:
    Jun 13, 2003
    You should run tests at both 1080p and higher- particularly, 1080p (or other low resolutions) will show the performance delta and show a representation of increasing GPU speed as would happen with a GPU upgrade on the platforms tested.

    Higher-resolution tests will show performance for gameplay and when compared to the 1080 and lower results, will show if the performance delta holds.

    Running just one set of tests or the other set alone would be incomplete.
     
    Sulphademus likes this.
  30. Brackle

    Brackle Old Timer

    Messages:
    7,255
    Joined:
    Jun 19, 2003
    LOL Poor PT. They will now be the laughing stock of all reviews. I do feel bad for them, but it looks like they don't know what they are doing AND wanted to get paid.

    I mean Intel does deserve some blame for this IMO. It makes Intel look bad as well, specially when the reviews hit.
     
    Incontentia Buttocks likes this.
  31. IdiotInCharge

    IdiotInCharge [H]ardForum Junkie

    Messages:
    8,359
    Joined:
    Jun 13, 2003
    I'm sure they did feel this way- and if I were to take the perspective of someone that knows jack and squat about CPU reviews with respect to gaming, I could see where they are coming from.

    But it's excruciatingly clear that either through ignorance or malice, they produced results that are alien to someone that has even a rudimentary grasp of CPU performance in gaming.
     
  32. Nauseous

    Nauseous Gawd

    Messages:
    866
    Joined:
    Jun 5, 2004
    this video really bothers me, they just nit pick everything to death. "why you put a cooler on the intel one when it doesnt come with one?" i mean of course they are gonna put a cooler on it. if amd didnt send one with theirs they would have put something on it too... i mean come on. it just feels like they are out to get this dude for every little thing cuz they didnt like the results.
     
    Denpepe likes this.
  33. Brackle

    Brackle Old Timer

    Messages:
    7,255
    Joined:
    Jun 19, 2003
    Well when you are professional review site, and you are reviewing a new system....Well you need to review with that is Retail from both companies. That is how you would be honest. I mean why wouldn't you do that?
     
    Vokar, d8lock and dragonstongue like this.
  34. SixFootDuo

    SixFootDuo [H]ardness Supreme

    Messages:
    5,378
    Joined:
    Oct 5, 2004

    Exactly this. I like Gamer's Nexus and the other review websites out there. But I cannot stand all the grandstanding BS they do. This shit really does not matter. I will sum it all up for you guys. Basically, this. Millions of man hours and billions of dollars are spent on ... a few FPS here ... a few FPS there. It really is that simple. Now, if you're smart enough and you pay enough attention, do your due diligence all of this PC business becomes like a second language. You live and breath it. it guides you and you just automagically know BS when you see and smell it.

    I hope for a vast majority of you on these forums you were able to cut thru all the BS and save your time for something else more worth while.
     
  35. ghostwich

    ghostwich [H]ard|Gawd

    Messages:
    1,692
    Joined:
    Sep 10, 2014
    You're mentioned a bit in the beginning of the video:
     
  36. Kyle_Bennett

    Kyle_Bennett Editor-in-Chief HardOCP Staff Member

    Messages:
    52,053
    Joined:
    May 18, 1997
    Yeah, well I lead the entire canned benchmarks suck and are lying to you movement for quite a while, alone, for the the entire industry. Tom did not like that either. From the thread about the video...


    This all rings some bells.

    Optimizing or Cheating Radeon 8500 Drivers - 2001

    Benchmarking Wrong - 2003

    Benchmarking Future - 2003

    [H]ard|OCP 11/13/03 Editorial - 2003

    Cheating the Cheaters - 2004

    Editorial: How Far We Have Come - 2005

    Benchmarking the Benchmarks - 2008
     
  37. IdiotInCharge

    IdiotInCharge [H]ardForum Junkie

    Messages:
    8,359
    Joined:
    Jun 13, 2003
    Fuck yeah you did. It's one of the main reasons I'm still here!

    [it's certainly not the spell and grammar checking ;) ]
     
  38. JStamsek

    JStamsek [H]ardForum Junkie

    Messages:
    8,929
    Joined:
    Mar 24, 2016
    They should’ve had The Verge do the testing. Would’ve been more accurate.
     
  39. SixFootDuo

    SixFootDuo [H]ardness Supreme

    Messages:
    5,378
    Joined:
    Oct 5, 2004
  40. psyclist

    psyclist Gawd

    Messages:
    851
    Joined:
    Jan 25, 2005
    Then pick something comparable, im not exactly sure what matches up a Hyper 212 perhaps? Not one of the best coolers available against a box cooler ... Or just test both with the same cooler, and let the CPU's do the talking, not the cooling solutions... again they handicapped at too many spots to just plead ignorance I believe, and are now getting called on it. I do love when we can hold these companies accountable and not just let them post this BS and have everyone swallow it.

    Im really interested to see the real results, Intel wins of course! but im guessing by a much smaller margin. Competition is great, cheating...not so much. Looking forward to the review Kyle!
     
    d8lock, Malurt, thejalla and 2 others like this.