Google gets in on benchmark shenanigans

[1_rick]

https://www.notebookcheck.net/Exclu...-benchmark-blocking-controversy.761277.0.html

Notebookcheck recently broke the story about the fact that Google appears to have blocked writers reviewing its new Pixel 8 and Pixel 8 Pro devices from being able to easily install popular benchmark apps like Geekbench 6 and 3D Mark during the embargo period. It was a highly unusual move and raised questions about Google’s motivations for doing so. Google has made about the capabilities of the Tensor G3 chip at the heart of its latest Pixels, in particular its AI performance. However, in being unable to install the app, many reviewers were unable to test Google’s claims by running the benchmarks, as is standard practice for most reviews as it provides an objective and standardized way of assessing chip performance.

Looks like Google's afraid the new G3 SoC isn't very high-powered, so they're blocking reviewers from being able to even install benchmark software during the embargo period (of course, embargoes in general are just bogus to begin with.)

As one blogger said: " And why did Google do that? One has to assume, because the scores weren't very good. And checking some reviews, that seems to be the case. The Mediatek chip I mentioned yesterday (the Dimensity 9300, which apparently is only about 2% slower in multicore than the A17) is significantly faster than Google's powerhouse - 40% single core and nearly 100% multi-core."

 

[uOpt]

Well, the benchmarks don't test the machine learning interference coprocessor. And they apparently saved a couple bucks on the actual CPU.

This is like advertising "our CPU sucks" double page in the NYT.

 

[Lakados]

Well the AI core is there to improve camera and media playback.
At some point the performance of the rest of the system is “good enough”.

I am not upset that the the CPU performance isn’t great, if the AI stuff makes up for it that’s not the issue.

The issue is Google is blocking it, they could just say hey the benchmark doesn’t accurately reflect the performance of the actual system and give that as a preface but blocking something is bad.

 

[Meeho]

Can't think of a more useless feature than AI hardware on a phone. Well, at least for the user.

 

[Lakados]

Can't think of a more useless feature than AI hardware on a phone. Well, at least for the user.

Auto correct, noise cancellation and isolation, camera focus, image enhancement, video upscaling, auto translation, language translation, facial recognition, voice recognition, … All cellphone staples and all running off AI enhancements.

AI/ML is what makes the existing cellphones work as well as they do, for years it was just called Machine Learning, but AI gives it a trendy smart name.

 

[1_rick]

Auto correct, noise cancellation and isolation, camera focus, image enhancement, video upscaling, auto translation, language translation, facial recognition, voice recognition, … All cellphone staples and all running off AI enhancements.

AI/ML is what makes the existing cellphones work as well as they do, for years it was just called Machine Learning, but AI gives it a trendy smart name.

Great. When will it be able to properly focus the camera on my 3d print so you can see the layers?

 

[sfsuphysics]

Auto correct, noise cancellation and isolation, camera focus, image enhancement, video upscaling, auto translation, language translation, facial recognition, voice recognition, … All cellphone staples and all running off AI enhancements.

How good is language translation these days? I know point camera and poof translation isn't half bad, but are they at a point where translating speech works any good yet?

 

[Lakados]

How good is language translation these days? I know point camera and poof translation isn't half bad, but are they at a point where translating speech works any good yet?

Speech is getting better the best ones still need a back end that is still more power hungry than any phone is capable.

Microsoft offers it as a service for Teams and Skype and while I wouldn’t call it a human parity, the times when we have needed it neither side of the call was frustrated by it.

You would get the odd “the cats nose is dirty” or “tomato glasses” sort of thing because the expression doesn’t translate but those were obvious on both sides that there was a translation issue.

But on most phones even that camera text translation is going back to a cloud based AI uploading your camera footage and GPS coordinates. Newer phones can do that locally and don’t need to upload that.

 

[Twisted Kidney]

Just do a graph with no y-axis and a completely indecipherable x-axis.

It's called an AppleGraph.

Apple's shit is really good and they still obfuscate the data. I'm not going to panic about Google's parts.

 

[DukenukemX]

https://www.notebookcheck.net/Exclu...-benchmark-blocking-controversy.761277.0.html



Looks like Google's afraid the new G3 SoC isn't very high-powered, so they're blocking reviewers from being able to even install benchmark software during the embargo period (of course, embargoes in general are just bogus to begin with.)

As one blogger said: " And why did Google do that? One has to assume, because the scores weren't very good. And checking some reviews, that seems to be the case. The Mediatek chip I mentioned yesterday (the Dimensity 9300, which apparently is only about 2% slower in multicore than the A17) is significantly faster than Google's powerhouse - 40% single core and nearly 100% multi-core."

To be fair to Google, Geekbench favors Apple heavily. I don't put much weight on synthetic benchmarks anyway. There's a reason why PC hardware reviewers haven't migrated back to synthetic benchmarks. On the other hand, if people want to run Geekbench then that's none of Google's business. Bad reviewers will use bad tools like Geekbench to determine the performance of devices. People need to realize on their own that no reviewer is worth looking at if they use Geekbench.


View: https://youtu.be/X4DXk6v73DY?si=tICAf6-QqthgjNl9

 

[Meeho]

Auto correct, noise cancellation and isolation, camera focus, auto translation, language translation, facial recognition, voice recognition, …

None of those require AI hardware on the phone and work fine as is.

image enhancement, video upscaling

Useless.

All cellphone staples and all running off AI enhancements.

AI/ML is what makes the existing cellphones work as well as they do, for years it was just called Machine Learning, but AI gives it a trendy smart name.

Which is it? Is it staples that work well or we need AI hardware for it to work?

 

[Lakados]

None of those require AI hardware on the phone and work fine as is.


Useless.


Which is it? Is it staples that work well or we need AI hardware for it to work?

The AI engine in the ARMv8 and v9 cores is what has made those features work on cellphones for the past 7 years or so.

But demand for camera image quality is quickly exceeding what the default ARM core AI is capable of. And the Apple Neural Engine greatly surpasses it.
If you ever wondered how it is that the cameras on cellphones manage to do as well as they do given the tiny lenses thank AI.
They call it AI but it’s just their ML accelerators for OpenML code.

But these accelerators are what makes most cellphones do half of what they do, they just call it AI because it has better marketing buzz than fixed function OpenML acceleration chip.