Saturday , August 19 2017

Samsung caught manipulating benchmark results – again

Back in July, we reported that Samsung was boosting benchmark scores on its Galaxy S4. Scores of over 10% higher than similarly equipped phones brought the attention of AnandTech, who ran tests and confirmed that the Samsung Galaxy S4, when it detects certain popular benchmark programs, runs a code that forces its quad-core chip to run on all cores at maximum frequency, even when in idle.

Samsung, through their Korean website, said: “[We] did not use a specific tool on purpose to achieve higher benchmark scores”, and “under normal conditions, the Galaxy S4 operates up to 533MHz at its best performance.”   

This is a slightly comical statement, as AnandTech found code in the Galaxy S4 marked “Benchmark Booster”.

Fast forward to the release of the Galaxy Note 3, and you would have thought that Samsung have learnt their lesson. Ah, no. The crew over at ars technica ran the Note 3 through its gamut of benchmarks, and guess what? The Note 3 scores over 20% better than the similarly equipped LG G2, with both running the 2.3GHz Snapdragon 800 System on Chip (SoC).

To get a “real world” score for the Note 3, ars technica devised a cunning yet simple plan: rename some benchmarks so that Samsung’s Benchmark Booster can’t detect it, and thus doesn’t inflate the score by running at full throttle throughout the entire test.

Below we have the results, with Geekbench 3 and StealthBench 3 being identical except for the name. Geekbench 3 shows that all of the 4 cores of the CPU running at the maximum 2.3GHz, even though it is in idle. Stealthbench 3 (renamed so that Benchmark Booster doesn’t identify it and sees it as a normal app), runs correctly when in idle, with only one core operating at 300MHz to conserve power:

Note 3’s benchmarking “adjustments” inflate scores by up to 20    Ars Technica

Here is Geekbench 3 Multicore, showing that the “normal” Note 3 is just about equal to the LG G2, with the Benchmark Boosted Note 3 romping home for a stunning win:

Note 3’s benchmarking “adjustments” inflate scores by up to 20    Ars Technica 2

Really Samsung? Do you think that people determine that one phone is better than another  just because of an artificially inflated score in a just as artificial benchmark? High scores do not improve the users experience with a phone. What makes it is how fluidly the operating system performs, how intuitive it is, how the phone feels in the hand and other indefinable factors.

What do think of Samsung’s shenanigans? Is there a point to them in real life use of the phone? Let us know in the comments!

 
Source: SamMobile.
Via: ars technica.

Ausdroid Staff  

Join the Ausdroid Conversation

11 Comments on "Samsung caught manipulating benchmark results – again"

avatar
Sort by:   newest | oldest | most voted
Joshua Hill
Valued Guest
Joshua Hill

My posts questioning whether this article is spinning the issue have now been deleted despite atleast two users upvoting my initial post, so presumably agreeing with it’s motif. Very disappointing Ausdroid. Not only do I suspect your ethics are in breach but now you’re censoring posts that question your professional integrity. Can an Ausdroid representative explain the situation?

David Anderton
Valued Guest
David Anderton

man there are some serious samsung fanboys on this site…

Joshua Hill
Valued Guest
Joshua Hill

Now that Anandtech have an article up detailing which other manufacturers are running optimisations on benchmarks and most other android sites have mentioned the news when is Ausdroid going to provide their readers with this news or do you like unfairly singling out Samsung?

Joshua Hill
Valued Guest
Joshua Hill

So, it looks like this article has been removed from Ausdroid’s main news feed. I hope this is temporary while you do some extra fact checking, fix it and you’re not just spreading misinformation and then removing it without correcting the situation.

Joshua Hill
Valued Guest
Joshua Hill
And it’s back up on the main news feed with no corrections or no new article to set the facts straight. This is farcical Ausdroid! Your ethics statement states: ‘there will be no positive or negative spin put on the review that can’t be backed up’. The main source for this article is Ars Technica but you also mention Anandtech in the article, they’ve also been used as a source before. Anandtech clearly states other manufacturers are doing the same thing. You fail to mention this and single out Samsung. To me that is providing negative spin against Samsung. A… Read more »
Joshua Hill
Valued Guest
Joshua Hill

Anandtech also claims the HTC One does the same thing. You could force your android cpu governor to always run at max cpu speed and thus emulate this on most phones. It’s not overclocking or cheating imo just tweaking for specific purposes.

Happy Dog
Valued Guest
Happy Dog

Agree. Its like running a higher boost when you put your car on the track. They aren’t changing scores

Tim
Valued Guest

Benchmarks are to show the raw power of the device.

Exactly what Samsung is doing. Showing what is under the hood and what the beast is capable of. ..

Are we going to start asking Ferrari to test drive on 6 cylinders, in 5th gear and under 120kph as that’s all the roads allow …

If you have the power … flaunt it.

It is not like they are making a 1.8Ghz chip falsely run and state 2.3Ghz.

JeniSkunk
Valued Guest
JeniSkunk
Tim, it’s not that Samsung are falsely showing what the device is capable of, it’s that they are showing non-standard results for specific test apps, and not opening up dev access to the remove the performance lock down code so any app can make use of the full performance the device is capable of. You use a motoring analogy, fine. In your Ferrari analogy, it would be a case of being speed limited to 120Kph by Ferrari despite there being no actual maximum speed limit on the roads. When known test-review drivers are in the car the full performance of… Read more »
Andrew
Valued Guest
Andrew

Really lame Shamesung, really, really lame.

Pretty pathetic that they have to cheat to try to prove their phones are better. Anyone who’s used a Galaxy S4 with the ridiculous amounts of lag in the UI will know something isn’t right with the high benchmark scores and how bad it performs in the hand.

I expect nothing else from a ‘fast follow’ company.

vijay alapati
Valued Guest
vijay alapati

Thumbs up for the gorilla image 🙂

wpDiscuz

Check Also

Lenovo announces Android Wear 2.0 for Moto 360 Sport has started rolling out

After an extended wait, Lenovo has announced on their support page that the Android Wear …