The State of DirectX 11 - Image Quality & Performance
Scope, Test Setup & Methodology
As previously mentioned, we intend to explore the state of DirectX 11 in this article. That's a very tall order and one we cannot hope to fulfill unless we set out some boundaries and determine a specific scope for our examination. It would be nearly impossible and rather frivolous to encompass an evaluation of DirectX 11 from the perspective of everyone affected by its introduction -- such as the game developers, hardware manufacturers and end consumers -- into a single article.
To help keep the article at a manageable scope, we are only going to explore the current state of DirectX 11 from the perspective of the end consumer. Anything that is or should be transparent to the consumer, such as API optimizations that let developers program more efficiently, will not be covered. Note that this is not necessarily the same as exploring the current end user experience. We want to focus on examining the potential of currently available DirectX 11 software. We will focus our attention on two fronts; performance and image quality. Specifically, we are interested in examining the differences in performance between DirectX 9.0c and DirectX 11 and the image quality enhancements of DirectX 11, if any.
Performance Tests: For our performance tests, we used a single system configuration (see above). Our goal is to compare the performance of DirectX 11 rendering paths in currently available games with other rendering paths like DirectX 10 and especially DirectX 9. The April 2010 Steam hardware survey reveals that Windows XP 32bit is still the most popular OS with 36.79% of all Steam users who participated in the survey, although Windows 7 64bit is quickly catching up with 26.39%, a 2.1% gain from the last survey. So it would seem that a lot of users are still stuck with using DirectX 9 rendering paths, even if most of them have DirectX 10 and 11 hardware. Therefore we focused on the comparison between DirectX 9 and 11 for most of our testing, though we also tested DirectX 10 and we'll be reporting any interesting results there too.
Since the focus of the test is on contrasting the different DirectX rendering paths, very little attention will be given to the hardware running the tests. This is not a hardware focused article. Our only goal with hardware selection was to choose a respectable gaming system that many gamers upgrading to DirectX 11 will likely have. We are interested in how these games and DirectX rendering paths perform in a real-world mid-range system.
While our goal was to use a single consistent setup for all tests, we ultimately chose to test two different video cards since there seems to be two separate price 'sweet spots'. The first sweet spot appears at around $150, and it is represented here by the Diamond Radeon 5770. Video cards in this price point are currently an excellent value and hard to beat without shelling out significantly more, making it our value sweet spot. The other sweet spot is at the high end at $400, represented here by the Diamon Radeon 5870. This is the edge of single-card performance and a more powerful setup will require multiple cards which significantly bumps up the price, making this price point the high-end sweet spot. We expect both of these price points and video cards to be very popular and chose them to represent our hypothetical "average" PC gamer.
It's worth noting that this article was conceived before NVIDIA's DirectX 11 products were available and they are not represented here for that reason alone. You may also notice that the test system is a full AMD platform, which is largely due to convenience at the time of article conception. We'd like to remind everyone that this article is about software, and the hardware chosen is purely for the purposes of representing popular price points in order to illustrate a hypothetical "average" gamer. This article makes no further comment about hardware.
Image Quality Tests: For each of the games in our test, we compared the image quality of the game's DirectX 9 rendering path with its DierctX 11 rendering path. All image quality tests were performed with the Diamond Radeon 5870 and the same system configuration used in the performance tests. The video settings for each game were set to their highest settings for all of the image quality tests.
There are cases where a certain game will not offer certain features, like anti-aliasing or screen-space ambient occlusion (SSAO), in a DirectX 9 rendering path. This means that if both the DirectX 9 and 11 rendering paths were configured to their maximum available settings, the DirectX 11 rendering path would be doing significantly more work, perhaps losing some performance and also gaining image quality. We view this as a necessary part of comparing the different rendering paths and made no attempt to perform a direct setting-for-setting comparison.
Image quality was judged both in-game and with the aid of a large series of screenshots. Screenshot of each game were taken using the FRAPS utility in both DirectX 9 and 11 rendering modes. Studious effort was put into ensuring that screenshots were taken in matching sets, so the DirectX 9 screenshots would have matching DirectX 11 counterparts which are the best recreations of the DirectX 9 shot we could manage. Several dozen sets of matching screenshots were produces for each game and only the best 4 for each game are included in this article. The screenshots were judged on how well they illustrate differences between DirectX 9 and 11.
The screenshots were taken at 1920x1200 resolution and saved by FRAPS as uncompressed bitmaps. The screenshot sets chosen for inclusion in the article were cropped and resized in Adobe Photoshop and saved as high quality JPEGs, no other alterations were made to the images.While using JPEG does introduce a certain amount of compression artifacts to the images, we felt that they did not interfere with the illustration of the image quality differences. After all, if the difference is so subtle that simple JPEG compression artifacts could prevent them from being noticed in a direct side-by-side comparison, how are you supposed to notice it in-game?
To help keep the article at a manageable scope, we are only going to explore the current state of DirectX 11 from the perspective of the end consumer. Anything that is or should be transparent to the consumer, such as API optimizations that let developers program more efficiently, will not be covered. Note that this is not necessarily the same as exploring the current end user experience. We want to focus on examining the potential of currently available DirectX 11 software. We will focus our attention on two fronts; performance and image quality. Specifically, we are interested in examining the differences in performance between DirectX 9.0c and DirectX 11 and the image quality enhancements of DirectX 11, if any.
 |
|
|
AMD Phenom II X4 955 |
Video Cards: Diamond Radeon HD 5770 1024MB Diamond Radeon HD 5870 1024MB Games: Aliens vs Predator BattleForge Battlefield: Bad Company 2 Dirt 2 Unigine Heaven Benchmark |
|||
Performance Tests: For our performance tests, we used a single system configuration (see above). Our goal is to compare the performance of DirectX 11 rendering paths in currently available games with other rendering paths like DirectX 10 and especially DirectX 9. The April 2010 Steam hardware survey reveals that Windows XP 32bit is still the most popular OS with 36.79% of all Steam users who participated in the survey, although Windows 7 64bit is quickly catching up with 26.39%, a 2.1% gain from the last survey. So it would seem that a lot of users are still stuck with using DirectX 9 rendering paths, even if most of them have DirectX 10 and 11 hardware. Therefore we focused on the comparison between DirectX 9 and 11 for most of our testing, though we also tested DirectX 10 and we'll be reporting any interesting results there too.
Since the focus of the test is on contrasting the different DirectX rendering paths, very little attention will be given to the hardware running the tests. This is not a hardware focused article. Our only goal with hardware selection was to choose a respectable gaming system that many gamers upgrading to DirectX 11 will likely have. We are interested in how these games and DirectX rendering paths perform in a real-world mid-range system.
While our goal was to use a single consistent setup for all tests, we ultimately chose to test two different video cards since there seems to be two separate price 'sweet spots'. The first sweet spot appears at around $150, and it is represented here by the Diamond Radeon 5770. Video cards in this price point are currently an excellent value and hard to beat without shelling out significantly more, making it our value sweet spot. The other sweet spot is at the high end at $400, represented here by the Diamon Radeon 5870. This is the edge of single-card performance and a more powerful setup will require multiple cards which significantly bumps up the price, making this price point the high-end sweet spot. We expect both of these price points and video cards to be very popular and chose them to represent our hypothetical "average" PC gamer.
It's worth noting that this article was conceived before NVIDIA's DirectX 11 products were available and they are not represented here for that reason alone. You may also notice that the test system is a full AMD platform, which is largely due to convenience at the time of article conception. We'd like to remind everyone that this article is about software, and the hardware chosen is purely for the purposes of representing popular price points in order to illustrate a hypothetical "average" gamer. This article makes no further comment about hardware.
Image Quality Tests: For each of the games in our test, we compared the image quality of the game's DirectX 9 rendering path with its DierctX 11 rendering path. All image quality tests were performed with the Diamond Radeon 5870 and the same system configuration used in the performance tests. The video settings for each game were set to their highest settings for all of the image quality tests.
There are cases where a certain game will not offer certain features, like anti-aliasing or screen-space ambient occlusion (SSAO), in a DirectX 9 rendering path. This means that if both the DirectX 9 and 11 rendering paths were configured to their maximum available settings, the DirectX 11 rendering path would be doing significantly more work, perhaps losing some performance and also gaining image quality. We view this as a necessary part of comparing the different rendering paths and made no attempt to perform a direct setting-for-setting comparison.
Image quality was judged both in-game and with the aid of a large series of screenshots. Screenshot of each game were taken using the FRAPS utility in both DirectX 9 and 11 rendering modes. Studious effort was put into ensuring that screenshots were taken in matching sets, so the DirectX 9 screenshots would have matching DirectX 11 counterparts which are the best recreations of the DirectX 9 shot we could manage. Several dozen sets of matching screenshots were produces for each game and only the best 4 for each game are included in this article. The screenshots were judged on how well they illustrate differences between DirectX 9 and 11.
The screenshots were taken at 1920x1200 resolution and saved by FRAPS as uncompressed bitmaps. The screenshot sets chosen for inclusion in the article were cropped and resized in Adobe Photoshop and saved as high quality JPEGs, no other alterations were made to the images.While using JPEG does introduce a certain amount of compression artifacts to the images, we felt that they did not interfere with the illustration of the image quality differences. After all, if the difference is so subtle that simple JPEG compression artifacts could prevent them from being noticed in a direct side-by-side comparison, how are you supposed to notice it in-game?
