Huawei Accused Of Brazen Benchmark Cheating, Claims Everyone Else Is Doing It
As you might expect, when the software detects a popular benchmark, it ratchets up system performance beyond what the device would normally be capable of in its stock configuration. In essence, the company is overclocking the SoC to make its devices look good in benchmarks at the expense [potentially] of lessening the life of critical components.
We've seen instances of this in the past from a litany of smartphone OEMs, most recently with OnePlus. However, most OEMs simply tuck their tail between their legs, admit defeat and promise not to install cheating software in future devices.
Huawei, on the other hand, defends the practice, and says that it's actually a "feature" that leverages artificial intelligence to ensure that the processor is operating at a peak efficiency at all time. "In normal benchmarking scenarios, once Huawei's software recognizes a benchmarking application, it intelligently adapts to 'Performance Mode' and delivers optimum performance," said Huawei in a statement to The Inquirer. "Huawei is planning to provide users with access to 'Performance Mode' so they can use the maximum power of their device when they need to."
On the one hand, Huawei seems to be saying that this is simply smart engineering, with its Kirin 970 SoCs operating at the limit (and beyond) when maximum performance is required. However, in an impromptu interview with AnandTech at IFA 2018, Dr. Wang Chenglu, Huawei's President of Software, admitted, "Others do the same testing, get high scores, and Huawei cannot stay silent".
That's an interesting way to look at the situation, as Wang complains that other Chinese OEMs -- some of which don't even sell devices in the United States -- are gaming the system in order to score higher on popular benchmarks.
UL has gone so far as to delist 3DMark benchmark results for the P20, P20 Pro, Nova 3 and Honor Play. "After testing the devices in our own lab and confirming that they breach our rules, we have decided to delist the affected models and remove them from our performance rankings," wrote UL. "We found that the scores from the public 3DMark app were up to 47% higher than the scores from the private app, even though the tests are identical... This kind of detection and optimization is forbidden by our rules for manufacturers."
Given Huawei's stance on optimizing cheating, we should be playing closer attention to the benchmark numbers that come from Huawei's upcoming Kirin 980 SoC, which was first announced last week at IFA 2018. This is the first commercial 7nm chip coming from TSMC and will pack in 6.9 billion transistors while promising a 20 percent overall performance uplift compared to the Kirin 970.
