|
** Section Updated 5/23/2003 - 3PM EST **
By now, most
likely many of you have heard about the alleged driver
cheat that NVIDIA has incorporated into their 44.03
Detonator FX driver release, the driver we're using for
this article. While NVIDIA will neither confirm or
deny that they have intentionally taken steps to
manipulate the scores in 3DMark 2003, there is definitely
something completely unethical going on in our opinion and
we're not sure exactly where to point the finger.
Here's a
snippet of FutureMark's perspective.... (Taken from
FutureMark's recent 3DMark 03 Audit Report)
"In our
testing, all identified detection mechanisms stopped
working when we altered the benchmark code just trivially
and without changing any of the actual benchmark workload.
With this altered benchmark, NVIDIA?s certain products had
a performance drop of as much as 24.1% while competition?s
products performance drop stayed within the margin of
error of 3%. To our knowledge, all drivers with these
detection mechanisms were published only after the launch
of 3DMark03. According to industry?s terminology, this
type of driver design is defined as ?driver
cheats?.
Members of Futuremark?s BETA program first noticed how
parts of the tests in 3DMark03 were rendered differently
on different hardware. When testing NVIDIA hardware on
3DMark03 with socalled developer?s version?s free camera
enabled, they noticed how some parts of tests were
rendered strangely, and informed Futuremark of their
findings. Futuremark investigated further and our findings
show that certain NVIDIA drivers seem to detect when
3DMark03 is running and then replace the 3DMark03?s
rendering requests with manually implemented alternative
rendering operations. These alternative rendering
operations reduce the amount of rendering work and thereby
increase the obtained benchmark result."
And the NVIDIA
perspective...
"Since
NVIDIA is not part in the FutureMark beta program (a
program which costs of hundreds of thousands of dollars
to participate in) we do not get a chance to work with
Futuremark on writing the shaders like we would with a
real applications developer. We don't know what they
did, but it looks like they have intentionally tried to
create a scenario that makes our products look bad. This
is obvious since our relative performance on games like
Unreal Tournament 2003 and Doom 3 shows that the GeForce
FX 5900 Ultra is by far the fastest graphics on the
market today."
There you have
it. Frankly, we're growing tired of this little
struggle and it's a damn shame that FutureMark and NVIDIA
can't resolve their differences and get beyond this.
The REAL losers in this scenario are you folks, our
readers. There is definitely a place, as we've said
before here at HH, for Synthetic Benchmarks. Do we
have any other DX9 capable benchmark to use right now,
other than 3DMark 03? The answer is no... and since
we can't trust the scores with this benchmark now, sites
and publications like HotHardware will have to run DX7 and
8 game titles for our tests. You'll just have to go
without a real understanding of how a product will perform
in a DX9 gaming environment, at least until a true DX9
game engine ships. Isn't that nice? Don't tell
us there isn't a place for Synthetic Benchmarks... There
certainly is, if they are allowed to be engineered and
validated properly.
So, we'll
include scores from both 3DMark 03 before the 330 patch
and after.
Before FutureMark's
330 Patch
After
FutureMark's 330
Patch
And we're
completely reserving comment here on these scores and this
completely ridiculous situation....
|
|
|
Serious Sam SE Testing |
|
OpenGL With Lots Of Texture |
|
Serious Sam
The Second Encounter, will provide a decent perspective of
OpenGL gaming performance, with its bright and detailed
textures and DX7 lighting effects. Let's have a
look...
At a medium
resolution of 1024X768, Serious Sam's game engine is
hardly stressing any of the cards in this round-up.
Even the GFFX 5800 Ultra keeps close pace with the GFFX
5900 UItra and the Radeon 9800 Pro 256MB. However,
at 1600X1200, the pack spreads out a bit with the GeForce
FX 5900 Ultra stealing the show with 4X AA enabled.
Without AA, again all of the cards are within close
proximity of each other. However, we're reminded of
NVIDIA's
lesser quality AA output, that we showed you in our
GeForce FX 5900 Ultra launch article. In our opinion,
4X AA image quality is not quite up to par with Radeon
9800 Pro 4X AA image quality, at this point in time.
We are hopeful that NVIDIA will release a driver update
soon that will tighten up their AA fidelity. At this
point in time, users need to ask themselves if they want a
few extra frames per second at the slight expense of image
quality. We feel that although the GFFX 5900 Ultra
is faster than the Radeon 9800 Pro 256MB card, in certain
AA modes, the card just isn't doing the work that a Radeon
9800 Pro is capable of. Again, in the end it's up to
you to decide for yourself on the IQ of each card and the
associated performance levels you're getting. In our
opinion, NVIDIA has more homework to do on their drivers.
Next Up - Quake 3
Arena Time Demos
| 
