AMD Gives BAPCo the Bird, Chirps About Unfair SYSMark 2012 Benchmark
In a shocking move this morning, AMD bid the BAPCo (Business Applications Performance Corporation) consortium a not-so-found farewell and stated it will no longer endorse SYSmark 2012 (SM2012), a benchmark that historically has been accused of being pro-Intel. Among other things, one of AMD's chief complaints is that SM2012 virtually ignores GPU performance, a particularly egregious offense in AMD's eyes given the chip maker's infatuation with Accelerated Processing Units (APUs).
"SM2012 doesn’t represent the evolution of computer processing and how that evolution is influencing average users’ experience," AMD's Nigel Dessau wrote in a blog post. "SM2012 focuses only on the serial processing performance of the CPU, and virtually ignores the parallel processing performance of the GPU. In particular, SM2012 scores do not take into account GPU-accelerated applications that are widely used in today’s business environments."
AMD has long been a member of BAPCo, an organization whose goal is to create meaningful benchmarks in an applications environment. But it's a goal AMD felt BAPCo was failing, in particular with SYSmark, and the chip maker says it tried in vain to get the next-generation SM2012 benchmark right. What's more, BAPCo's failure to turn SYSmark into what AMD feels would be a meaningful benchmark could cost the industry billions of dollars, AMD claims.
"The heart of our complaint is this: the SYSmark benchmark is not only comprised of unrepresentative workloads (workloads that ignore the importance of heterogeneous computing and, frankly, favor our competitor’s designs), but it actually generates misleading results that can lead to very poor purchasing decisions, causing governments worldwide to historically overspend somewhere in the area of approximately $8B!," Dessau estimates.
As bold a move as this is for AMD, the chip maker might not be the only one turning its nose up at SM2012 and resigning from BAPCo. Semi Accurate reports that Nvidia and VIA are pulling out too, though neither company is speaking out like AMD has chosen to do. Remaining members include ARCIntuition, Dell, Hewlett-Packard, Hitachi, Intel, Lenovo, Microsoft, Samsung, Seagate, Sony, Toshiba and others.
AMD brings up an interesting discussion point in terms of benchmarking, one we'd like our readers to chime in on. The question is this: Do you think the computing paradigm is shifting towards GPU performance, and if so, should benchmarks like SYSmark put a bigger focus on GPU performance, or do you think it's still too early in the game for that?
"SM2012 doesn’t represent the evolution of computer processing and how that evolution is influencing average users’ experience," AMD's Nigel Dessau wrote in a blog post. "SM2012 focuses only on the serial processing performance of the CPU, and virtually ignores the parallel processing performance of the GPU. In particular, SM2012 scores do not take into account GPU-accelerated applications that are widely used in today’s business environments."
AMD has long been a member of BAPCo, an organization whose goal is to create meaningful benchmarks in an applications environment. But it's a goal AMD felt BAPCo was failing, in particular with SYSmark, and the chip maker says it tried in vain to get the next-generation SM2012 benchmark right. What's more, BAPCo's failure to turn SYSmark into what AMD feels would be a meaningful benchmark could cost the industry billions of dollars, AMD claims.
"The heart of our complaint is this: the SYSmark benchmark is not only comprised of unrepresentative workloads (workloads that ignore the importance of heterogeneous computing and, frankly, favor our competitor’s designs), but it actually generates misleading results that can lead to very poor purchasing decisions, causing governments worldwide to historically overspend somewhere in the area of approximately $8B!," Dessau estimates.
As bold a move as this is for AMD, the chip maker might not be the only one turning its nose up at SM2012 and resigning from BAPCo. Semi Accurate reports that Nvidia and VIA are pulling out too, though neither company is speaking out like AMD has chosen to do. Remaining members include ARCIntuition, Dell, Hewlett-Packard, Hitachi, Intel, Lenovo, Microsoft, Samsung, Seagate, Sony, Toshiba and others.
AMD brings up an interesting discussion point in terms of benchmarking, one we'd like our readers to chime in on. The question is this: Do you think the computing paradigm is shifting towards GPU performance, and if so, should benchmarks like SYSmark put a bigger focus on GPU performance, or do you think it's still too early in the game for that?
