Logo   Banner   TopRight
TopUnder
Transparent
Marking Time: 3DMark 11 Performance Explored
Transparent
Date: Dec 13, 2010
Section:Graphics/Sound
Author: Joel Hruska
Transparent
Introduction, Benchmark Design
Last week, Futuremark released the latest version of 3DMark.  We've taken the new benchmark for a spin using CPUs from Intel and AMD as well as GPUs from AMD and NVIDIA. The new version, dubbed 3DMark 11 (in reference to its level of DirectX support, not the upcoming year) includes a fresh set of tests, a game demo sequence, and measures CPU performance rather differently than its predecessor, 3DMark Vantage.


3DMark 11: Now with additional oceanic encrustations.

We've run the benchmark through multiple tests, comparing it across AMD and Intel CPUs as well as AMD vs NVIDIA GPUs. According to Futuremark, 3DMK11 was designed meet four specific goals. These are:
  • Produce consistent results that are repeatable and verifiable.
  • Represent technology and workloads fairly and accurately
  • Remain relevant over a long period of time
  • Allow for result comparisons across a wide variety of systems
These are more-or-less the same goals of every version of 3DMark, but in this case Futuremark has made a number of technical and methodological changes. 3DMK11 shares Vantage's goals, but it pursues them in a different (and in our opinion, superior) manner.
Transparent
Updates and Changes
We compiled a list of the more important changes based on information included in the Reviewer's Guide and technical whitepaper for both benchmarks. These are presented below in no specific order.

New Physics Engine
: Back when FM developed Vantage, it opted to include support for Ageia's PhysX PPU. If Ageia had stayed independent, the second CPU test would've been a nod to what consumers with a PPU (Physics Processing Unit) could potentially look forward to-but given how few people had bought Ageia cards, it would never have had much impact on the benchmark's comparative database.

After NVIDIA bought Ageia and ported PhysX to its G80-derived GPUs it became impossible to compare both the overall Vantage score and the CPU score in particular between AMD and NVIDIA systems. Although the CPU score counted for much less than the graphics score, CPU results for an NV card could be as much as 4x higher than when using its ATI counterpart. This problem could be fixed by tagging the "Disable PPU" option, but the flag was unset by default.

3DMark 11 avoids this snarl by using Bullet physics. Bullet is a FOSS (Free/Open Source Software) package that's been used in a number of commercial products and films. The project's open nature ensures the underlying software code is available to anyone and prevents last minute surprises.


3DMark 11 offers much improved fine-grain control over test workloads compared to Vantage

Different Benchmark Focus: Vantage's two gaming benchmarks, Jane Nash and New Calico, each tested on a different set of GPU capabilities. Part of New Calico's focus was GPU ray tracing, which made sense at the time. Intel had unveiled its Larrabee GPU not too many months before, promising that it would transform game visuals through the use of real-time ray-tracing. AMD and NVIDIA's collective response boiled down to "No, you won't," and the two established GPU manufacturers ended up winning that argument.

The four gaming tests in 3DMK11 focus on different areas of performance. Graphics Test 1 uses a large number of spot lights in both shadow casting and non shadow casting flavors. It also includes non-shadow casting point lights and volumetric illumination. This test contains no tessellation. Test two tones down the number of spot lights but includes a fair amount of tessellated geometry. (shown above) Test three takes place outside with the sun providing a shadow casting directional light—Futuremark states that tessellation is applied to various statues and some of the vegetation.


Is that jeep 80Plus?

Test 4 is by far the nastiest--it combines single shadow-casting directional light (moonlight), and a few shadow casting spot lights. This scene appears to use more tessellation than any of the others; Futuremark states that "the majority of the rendering workload comes from drawing tessellated geometry to shadow maps and [the] G-buffer.

Scaling and Scoring: 3DMark 11 weights the relative value of CPU and GPU performance differently than Vantage. 3DMK:V split GPU/CPU weighting 75/25 in Performance mode and 95/5 in Extreme mode. 3DMK11's Performance preset  uses a 75/15/10 split between Graphics, Physics, and Combined; its Extreme preset breaks down to 80/10/10. CPU performance counts for more at the upper end, possibly because DX11's multi-threading support is much improved over DX10s.

Enough talk. Let's do some numbers.
Transparent
Performance Comparisons
We tested 3DMark 11 on the following systems:

Graphics Comparison System
Phenom X6 1100T (3.3GHz)
OCZ-built single-stage phase changer.
6GB DDR2-800
Asus M3A78-T (AM2+, 780GX + SB750)
Windows 7 64-bit

For our AMD/NVIDIA tests we used an AMD HD 5970 and an NVIDIA GTX 480. Both cards were tested with the latest drivers (Catalyst 10.11 and Forceware 260.990. We opted to use 5970's in both systems for our AMD vs Intel comparison. Our AMD rig was configured as above, our Intel rig is below:

Additional Processor Comparison System
Intel Core i7-920 @ 3.3GHz
Koolance KIT1000BK water cooler
6GB DDR3-1067
EVGA X58 SLI Deluxe
Windows 7 64-bit

We chose the older AMD motherboard because it's currently the only AMD board we have on hand that's fitted for sub-zero cooling. Despite its age, the M3A78-T had no trouble running AMD's fastest Phenom II X6 once the BIOS was updated.

AMD vs NVIDIA


Keep in mind that the ATI 5970 is a dual-GPU solution. We would've preferred to match our tests by either price or number of GPUs, but the only ATI cards available were 5970s and NVIDIA doesn't yet support SLI for 3DMark 11. The results below shouldn't be considered an endorsement of any video card but keep in mind that the GTX 480 is ~$200 cheaper than the HD 5970.


The total 3DMark 11 score shows the GTX 480 at just 74 percent of the Radeon HD 5970s performance, but that's far from the only story here. The four game tests show very disparate results.



In the first test, which doesn't use tessellation, the HD 5970 is over 70 percent faster than the GTX 480. The second test does use tessellation; the gap between the two cards decreases to 'just' 54 percent. This shrinks to 25 percent in test three and the two cards are just eleven percent apart in test four. We know the last graphics test consists largely of tessellation processing, which has always been a traditional strength of Fermi-based GPUs. What we like about 3DMark 11 is that it tests various API features without going overboard on any single aspect and it offers end-users the opportunity to adjust the degree of tessellation in a scene (though such adjustments result in no official 3DMark score).



The gap between the two cards expands in Extreme mode. The GTX 480 is 80 percent as fast as the HD 5970 in Performance but just 67 percent as fast at the higher preset. The GF100 loses ground in Tests 3&4, for reasons that aren't entirely clear.

AMD vs. Intel



We tested our AMD and Intel systems at multiple clock speeds but ran into some performance scaling oddities that we've since sent back to Futuremark for further analysis. Ultimately we decided to discuss clock-for-clock performance between the two architectures while we waited for other information. The graph below shows both processors at 3.3GHz with HT disabled on the Intel system.


In a word, ow. The clock-for-clock performance gap between Intel and AMD here is huge, especially since we're comparing six AMD cores against just four from the Core i7.  If we assume that last year's agreement between AMD and Intel actually did mark the end of all compiler/benchmark shenanigans, then AMD's Bulldozer will have to deliver in spades in order to close the performance differential core-for-core and clock-for-clock between the two companies.
Transparent
Conclusion

3DMark 11 doesn't just add support for a new API; it offers a much finer degree of control over specific test options. While the uptake on DirectX 11 is rumored to be faster than DX10's ever was, the console-centric nature of the gaming industry means it could be quite some time before we see a number of titles on the market that actually prioritize DX11 as opposed to supporting it for marketing's sake.


We're just wondering what Antec paid for all the on-screen advertising...

The changes Futuremark made to this latest version of the benchmark should help ensure it remains a neutral standard in the years to come. It's difficult to determine how accurately the suite forecasts DX11 performance in real-world games--the dearth of titles prevents a reasonably thorough comparison--but we think there's a good chance the benchmark will prove to be a relative predictor and a useful tool for comparing various aspects of DX11 performance.  You can expect to see 3DMark 11 in our upcoming graphics card evaluations and the like.


Content Property of HotHardware.com