Samsung have done away with its controversial benchmark-influencing modifications that surfaced last year after the release of the Galaxy S4, in which Anandtech alleged the South Korean conglomerate of essentially “gaming” its devices’ CPU and GPU benchmark scores by leaving cores at “full throttle” when benchmarking applications were launched.
The benchmark influencing modifications allowed Samsung to achieve somewhat marginally higher scores in benchmark tests than its rivals – although a large number of those rivals were also found to be gaming the system by Anandtech – but, it gave the Galaxy S4 and Note 3 the appearance of being a little quicker than a typical benchmarking environment would otherwise show them to be.
The Android 4.4 update for the Galaxy S4 and Note 3 which is beginning to rollout worldwide (albeit slowly) don’t just brings the sweet goodness of KitKat, but they also show that Samsung’s benchmark shenanigans have been eradicated according to tests conducted by arstechnica and confirmed by John Poole from Primate Labs makers of benchmarking App – Geekbench.
The results of the testing were pretty conclusive. The Geekbench scores for both the Note 3 and Galaxy S4 running Android 4.4.2 are significantly lower than those shown and obtained when the devices ran Android 4.3 originally, with Poole finding that the CPU showed an initial spike when launching the Geekbench App but then appeared to run the Benchmark software ‘noramlly’.
With all this data backing up the findings from the investigation, the conclusion is pretty obvious: Samsung have indeed removed the offending code which caused the S4 and Note 3 to behave differently in benchmarking apps.
It has also be reconfirmed via a tweet from Developer Wanam confirming that the code responsible for the “boosting” behavior has been removed in the Android 4.4 ROMs for both the Note 3 and Galaxy S4.
While Samsung has apparently stopped this practice, we’re still yet to see a level playing field from other manufacturers, so the issue of trusting benchmark results is still present. With the release of new flagship smartphones and updates rolling out these will no doubt be investigated in the near future and hopefully there will be a level playing field in benchmarks in the near future.
Do you think Samsung have done the right thing by removing the benchmark-influencing modifications code from the Galaxy S4 and Note 3 in the Android 4.4 update? Do you think this will create a more level playing field when it comes to benchmark reporting?
Samsung weren’t the only one’s, any news on the other OEM’s?, Asus HTC etc.
So the conspiracy theorist in me questions whether Samsung really wants these phones to be the fastest any more. We have a new flagship coming! How interesting it would be if the new S5 still has the dodgy benchmark results, making it look much faster than the S4 and Note3. Reviewers! Want to beat this? Record battery life results with the benchmark application running!
Y’all really don’t know what you are reporting on.
How is involving more configurable layers leveling the playing field?
Run all devices full throttle, all dynamics removed and you’ll see who has the best hardware.
That’s the only way to see the real differences.
I disagree. The benchmark needs to run like a normal application to be indicative of real world performance. What devices can do at a specialised full throttle is not indicative of this if the processor will not reach that state through normal equivalent use. There is a reason the hardware is not behaving that way under normal circumstances, which can vary for a multitude of reasons (eg thermal considerations).
Benchmarks aren’t exactly the best method of “gauging real world performance” rigged or not
I agree, but at least they show what the hardware is capable of compared with what another phones hardware is capable of. What Samsung did skewed the result to make them almost completely useless, when they would normally be much more consistent across devices…