Windows XP vs. Windows Vista Performance Part 2: NVIDIA February 07, 2007

Introduction

Page:: ( 1 / 14 )

Getting competent drivers out for Vista is critical for both companies as many consumers have been waiting for Vista’s release before upgrading, as well as the OEMs like Dell, HP, and Gateway, who are busy shipping systems with Vista pre-installed to their customers. In fact, in Dell’s case they’ve had to hold shipments of their XPS 710 and XPS 710 H2C PCs with Vista, shipping these systems with Windows XP instead.

Until the driver situation is fully sorted out, things will continue to be in limbo and that’s not a state you want to be in long if you are any of these manufacturers. That’s why we’re taking the time out to evaluate both AMD and NVIDIA’s driver offerings for both 32-bit and 64-bit flavors of Vista in comparison to their counterparts for Windows XP.

In last week’s article we took a look at AMD’s Vista driver, and while we found a lot of good things to like under D3D apps, AMD’s OpenGL driver wasn’t nearly as polished. In fact it was downright ugly -- literally. Because of this, we didn’t even bother running official Quake 4 benchmarks under Vista, although for curiosity’s sake we did jot down a few numbers and saw that performance wasn’t bad. Performance of AMD’s Catalyst Control Center was very good also. User reports from AMD card owners were pretty consistent with this as well.

In comparison, NVIDIA’s Vista transition doesn’t appear to be going quite as smoothly so far. Whereas ATI was able to release WHQL-certified Vista drivers for their entire range of cards with the launch of Catalyst 7.1, NVIDIA introduced two beta drivers for GeForce 8800 card owners under Vista last week. This led to a flurry of complaints from GeForce users that were expecting fully certified, signed drivers in time for Vista’s launch.

To be honest, in the past we’ve viewed the whole WHQL certification process and whether or not a driver is signed or unsigned with a grain of salt. After all, as any Radeon or GeForce card owner can tell you, there are plenty of bugs in fully WHQL-certified Catalyst and ForceWare driver releases. All you have to do is check out the driver release notes from either company to see the long list of issues resolved (and unresolved) with various games: you’ll quickly see that there are quite a few issues out there whether the driver is beta or WHQL-certified. With this in mind, again, we’ve historically never really harped on the certified versus non-certified driver issue, as both have their fair share of problems. There’s no such thing as the perfect driver.

With that being said, we are disappointed that NVIDIA’s latest drivers continue to lack SLI support for GeForce 6/7 card owners. There are an awful lot of GeForce 6 and 7 card owners out there with SLI setups, and it’s a shame that NVIDIA doesn’t have a Vista driver for these users, arguably some of NVIDIA’s most loyal customers if they dished out the bucks for two GeForce cards and an SLI motherboard. It’s these users that should rightfully be the most upset in our opinion.

There’s also been some angst from GeForce 8800 owners over the fact that NVIDIA’s latest Vista drivers for these cards are beta, unsigned drivers, but honestly we don’t feel this is as big an issue, both beta ForceWare 100 drivers NVIDIA released last week install just fine under both 32-bit and 64-bit flavors of Windows Vista. For this article, we tested all GeForce 7 and GeForce 8800 cards using NVIDIA's beta ForceWare 100.59 Vista driver. During installation a warning message will pop up asking you if you’re sure you want to install unsigned drivers, but that’s it. All you have to do is click yes and the driver installs just as seamlessly as NVIDIA’s WHQL-certified Vista driver released at the beginning of January for GeForce 6/7 cards.

Again, in our experience, we’ve seen beta, non WHQL-certified drivers that are just as good as the WHQLs from both AMD and NVIDIA in the past. Many users download the latest betas off nZone and other websites religiously and haven’t complained up to now, while tons of Radeon users downloaded ATI’s Chuck driver last year to enable HDR+AA in games like Oblivion.

At the same time however we’re not going to let NVIDIA completely off the hook. Their latest beta drivers definitely seem to have a problem successfully coming out of suspend mode under Windows Vista with GeForce 8800 cards. NVIDIA’s beta Vista driver is also missing features that are found in their Windows XP driver.

In this article we’ll also be taking a look at NVIDIA’s Vista performance with a variety of graphics cards, but first lets discuss some of these topics with NVIDIA’s Vice-President of Software Engineering (read: the head honcho of driver development), Dwight Diercks.

Dwight Diercks Q&A

Page:: ( 2 / 14 )

FiringSquad: When did you begin work on your drivers for Vista and how many people are involved on the project?

Dwight Diercks: We have been actively involved with Windows Vista driver development for over two and a half years. We have spent over 350 man-years of development on Windows Vista driver development.


FiringSquad: Both AMD and NVIDIA’s OpenGL drivers for Vista seem to be unpolished at this time. Is this because optimizing for OpenGL is a lower priority than D3D (after all, there are fewer OpenGL apps than D3D out there) or is coding for OpenGL more difficult in Vista than it was under Windows XP? What seems to be the holdup?

Dwight Diercks: Architecturally, Windows Vista does not impact OpenGL application performance when compared to Windows XP. We constantly measure and chart our performance on a variety of 3D applications in Windows Vista and use it to help us target future optimizations.

Keep in mind that we have been optimizing our Windows XP drivers for the past seven years, and some of that optimization came with years of driver work. We have software engineers who do nothing but figure out how to get games to run better and faster, and while some of the bottlenecks under Vista have changed, we are focusing the team on a series of optimizations that will show the biggest bang for the buck as we speak. It will take time, but NVIDIA will continue to work on optimizing our drivers to ensure maximum performance for Microsoft DirectX and OpenGL. We’re working hard to get there as soon as possible.



FiringSquad: One of your stated goals is to deliver Windows Vista performance that is comparable to Windows XP in games. Realistically how long do you think it will take to truly accomplish this goal?

Dwight Diercks: Optimizing drivers for any new operating system is a key focus for a core team of software engineers here at NVIDIA. We focused first on implementing the major driver model architectural changes in Windows Vista without focusing solely on performance, and that’s why our initial drivers are slower on some applications compared to Windows XP. Now, we are making sure performance optimizations are at the top of our list. We expect to deliver frequent driver updates over the next few months that will show strides in performance for top 3D applications and games. Over the next couple of weeks we are going to have more details on our driver release plan, and we will be happy to share these dates with Firing Squad readers and end users.


FiringSquad: When do you expect to have WHQL-certified drivers for the GeForce 8800 cards?

Dwight Diercks: We estimate the WHQL driver for GeForce 8800 will be ready to release by the end of February. We are working very closely with Microsoft to finalize the WHQL certified DirectX 10 driver. Both companies are working together to ensure that the driver is certified and delivered to customers as soon as possible. After this driver, we intend to release a GeForce 7 series SLI driver in the March timeframe.


FiringSquad: Will your 32-bit Vista driver be higher priority than your 64-bit Vista driver? Are older, more prevalent, GeForce 6/7 cards given higher priority when it comes to Vista driver development than GeForce 8?

Dwight Diercks: 32-bit and 64-bit drivers are both equally important to NVIDIA since there are so many platforms out there that support the x64 instruction set. From an OEM perspective, we are seeing slightly more interest in 32-bit drivers than 64-bit. GeForce 6 and 7 series card are being given the same priority as GeForce 8 series.

Our reason for doing these first two ForceWare 100 releases for GeForce 8 series was that we have OEMs who require a WHQL driver in order to go into production. We have already obtained WHQL certification for two previous drivers for GeForce 6 and 7 series GPUs, v96.85 and 97.46. Once we have completed the GeForce 8 series WHQL requirement, we are ensuring that we have a top to bottom driver that supports all of the hardware features in all three GPU series. I do want to point out that ForceWare Release 100 is a solid beta driver for GeForce 6 and 7 series GPUs.

Q&A (cont’d)

Page:: ( 3 / 14 )

FiringSquad: With its radically different stream architecture, you’ve split off your GeForce 8800 driver from the rest of your older GeForce graphics cards. Did this cause added strain on your driver development for Vista, and how is the driver group organized now?

Dwight Diercks: No, this did not cause strain and, in fact we have not split off the GeForce 8800 driver from the rest of our GeForce GPUs. ForceWare Release 100 was originally released as a Beta for GeForce 8800 series GPUs since these customers did not have a driver to use yet, and our focus was on getting the WHQL logo for those products. The fact is Release 100 supports GeForce 6 and 7 series GPUs, and you can download and run v100.54 or v100.59. We updated the v97.46 driver download pages on January 30th to let those customers know we had a Beta driver for them to download.


FiringSquad: Could you go over some of the differences in coding for the GeForce 8800 driver versus previous GeForce cards? In light of its unified shading architecture, this would make life a little more difficult for you driver guys would it not? As you’ve got to keep the stream processors running as efficiently as possible, surely that’s an added challenge correct?

Dwight Diercks: GPU’s are probably the largest programmable device in the world, and the level of programming complexity seems to grow each year. For example, the GeForce 8800 family adds new driver requirements for DirectX 10 in addition to DirectX 9 support, HD DVD/Blu-ray drivers for NVIDIA PureVideo HD, and NVIDIA SLI for both DirectX 9 and DirectX 10. All of this adds to the complexity factor and to the volume of code required to keep a GPU operating at peak efficiency.

We will be the first company to submit hardware for DirectX 10 WHQL certification, and the challenge in getting WHQL certification for a new API is that you’ll find some issues where the tests need to change, and other issues where the drivers need to change. At this point we’re down to a handful of bugs that we still need to fix, and we expect to finish this week.



FiringSquad: How hard is it programming for Vista’s new driver model?

Dwight Diercks: Vista requires an entirely new driver model for graphics. It moves much of the driver from kernel space to user space. It changes how basic display is handled, and it removes older driver portions of the code that have been there since NT4.0 days. In addition, high definition video (Blu-ray Disc and HD DVD) has a completely new architecture for video acceleration. So it is not hard to program for the Vista driver model, its just different than Windows XP. We have tens of millions of users of our XP drivers, so we are maintaining two large simultaneous driver bases, which results in additional software engineering complexity.

FiringSquad: Last year you stated that you planned to add additional overclocking functionality in nTune for GeForce 8800 cards, including the ability to independently overclock the stream processors. How is work progressing on this version of nTune, and when can we expect it to be released?

Dwight Diercks: This is already supported in nTune, but it requires some special code in our GPU drivers to support it. We are working on this now and we expect to support it in the March timeframe.

FiringSquad: Can you give us an ETA on when you’ll have nForce drivers available for Vista for the following chipsets: nForce4 family; nForce 5 family (both AMD and Intel); nForce 680i.

Dwight Diercks: Vista drivers support all nForce products out of the box. Users can now download WHQL nForce drivers directly from www.nvidia.com for all nForce4, nForce5 and nForce 600 series products.



On behalf of FiringSquad, we’d like to thank NVIDIA’s Dwight Diercks for taking time out of his day to answer our questions on ForceWare support for Vista. We look forward to seeing how the new ForceWare 100 series drivers progress over the course of 2007, and definitely can’t wait to get our hands on the updated drivers with additional overclocking support built-in!

Test Systems

Page:: ( 4 / 14 )

System Setup

Intel Core 2 Extreme X6800

ASUS P5B Deluxe WiFi-AP

2GB Corsair TWIN2X2048-6400C4

EVGA GeForce 7600 GS
EVGA e-GeForce 8800 GTX
NVIDIA GeForce 7900 GTX
Gigabyte GeForce 7900 GS
ForceWare 100.59 (for all GeForce cards under both versions of Vista)
ForceWare 97.92 (for GeForce 8800 GTX under WinXP

ForceWare 93.71 (for GeForce 7 cards under WinXP

300GB Western Digital Caviar SE

Windows XP Professional SP2
Windows XP Professional x64
Windows Vista Ultimate x86 (32-bit)
Windows Vista Ultimate x64

Benchmarks

Company of Heroes 1.3
F.E.A.R. 1.08
Elder Scrolls IV: Oblivion
Battlefield 2142 1.10
Call of Duty 2 1.3
Half-Life 2 Lost Coast (64-bit version when needed)

Notes

We were unable to test our Gigabyte GeForce 7900 GS board under 64-bit Windows Vista due to issues we ran into with nTune. You see, we underclocked the board to stock 7900 GS speeds using nTune and/or RivaTuner under the other operating systems, unfortunately however we couldn’t get either of these utilities to work under Vista x64.

We also ran our GeForce 7600 GS card under the same high graphics settings as the other cards. As you’ll see in the forthcoming benchmarks, those settings were definitely way too high for the card and as a result, we probably would have been better off swapping it out with a GeForce 8800 GTS instead. Oh well.

3DMark 06

Page:: ( 5 / 14 )

3DMark 06 – Direct3D

Notes

Vista performance was down slightly under 3DMark 06 for both the GeForce 8800 GTX and 7900 GS. In the case of the 8800 GTX, performance was down just 2%, but 32-bit Vista was 5% slower than 32-bit WinXP with the GeForce 7900 GS. Performance was slightly up for the GeForce 7900 GTX under 64-bit Vista. Go figure.

Battlefield 2142

Page:: ( 6 / 14 )

Battlefield 2 – Direct3D

Notes

Performance was significantly down for the GeForce 8800 GTX under Windows Vista. Approximately 22% at 1280x1024 under 32-bit Vista, and 21% when comparing the 64-bit operating systems. We noted similar margins at higher resolutions.

The other GeForce cards don’t seem to be affected as severely, with the 7900 GTX running anywhere from 4-9% slower and the 7900 GS 12% slower at 1280x1024 and 9% at 1600x1200. The 7600 GS fares better than the other cards, but looking at how low the frame rates are, we’re obviously running at graphics settings that are too high for the card (we left the graphics settings at the highest level for all four cards).

F.E.A.R. Performance

Page:: ( 7 / 14 )

F.E.A.R. – Direct3D

Notes

The performance picture looks much better for the 8800 GTX under F.E.A.R. At 1280x1024 and 1920x1200 performance is unchanged for all OS’s while performance is down about 5% at 1600x1200. The GeForce 7900 GTX runs fine under 64-bit Vista, but performance was down significantly (5-22%). We saw 18+% declines in performance for the GeForce 7900 GS as well.

Oblivion Mountains HDR

Page:: ( 8 / 14 )

Oblivion – Direct3D

Notes

Interestingly enough, both the GeForce 7900 GTX and 8800 GTX both run faster under 64-bit Vista than 32-bit Vista. The opposite is the case for the GeForce 7600 GS however. In any case, among the 64-bit operating systems, Vista is actually faster than Windows XP x64.

Here we should note that we use manual walkthroughs for our Oblivion testing, so there is a little variability, but we do repeat our runs three times and average the results out.

Oblivion Foliage HDR

Page:: ( 9 / 14 )

Oblivion – Direct3D

Notes

Under the greater demands of our foliage test, the GeForce 8800 card is only slightly slower under 64-bit Vista when compared to 64-bit Windows XP (about 3%). The 7900 GTX however did deliver better frame rates under 64-bit rather than 32-bit Vista. Under the 32-bit OS’s, the margin is as great as 5% for the 8800 GTX, but that’s at just 1280x1024.

The GeForce 7900 GTX actually ran faster under 64-bit Vista than 64-bit WinXP, while the GeForce 7900 GS performed similarly under both 32-bit operating systems in our foliage test.

Call of Duty 2

Page:: ( 10 / 14 )

Call of Duty 2 – Direct3D

Notes

We ran into video display issues with the GeForce 8800 GTX card in Call of Duty 2 under Windows Vista that prevented us from testing, hence we left those fields blank. When looking at the performance of the other cards though, we clearly see a decline in performance under Windows Vista, with the GeForce 7900 GTX running 11% slower at 1600x1200 and the 7900 GS 28% slower at that same resolution.

The GeForce 7600 GS continues to put up similar performance numbers regardless of OS, but again, that’s because the settings are so high that the hardware is maxed out, so subtle differences between the various drivers doesn’t impact performance.

HL2 Lost Coast HDR

Page:: ( 11 / 14 )

Half-Life 2 Lost Coast – Direct3D

Notes

We witnessed significant performance declines under HL2 Lost Coast for the GeForce 8800 GTX and 7900 GS running Vista; 8800 GTX performance is down over 40% when comparing both the 32-bit and 64-bit operating systems at 1600x1200, while the 7900 GS saw a drop of over half across the board.

The other cards also saw declines, but not nearly as significant, anywhere from roughly 9-31% for the 7900 GTX.

Clearly NVIDIA’s got a lot more work to do when it comes to Half-Life 2 performance.

Company of Heroes

Page:: ( 12 / 14 )

Company of Heroes – Direct3D

Notes

In comparison to the numbers we just saw under HL2, the 8800 GTX puts up pretty respectable performance numbers in Vista under Company of Heroes, performance is only down by at most 4%.

The other cards also turn in declines under Vista, but nothing in the double-digit range.

Quake 4

Page:: ( 13 / 14 )

Quake 4 – OpenGL

Notes

NVIDIA has made it clear that their OpenGL driver is still a work in progress and you can clearly see that in the results above. Despite these numbers we still feel NVIDIA’s OpenGL driver is more polished than AMD’s – at least their driver renders everything correctly. We noticed several areas under AMD’s OpenGL driver that lacked shadows as well as missing textures. When you’re not rendering everything present in a scene obviously your performance is going to be improved and thus we didn’t conduct any official tests with the AMD driver under Quake 4.

Final thoughts

Page:: ( 14 / 14 )

As you just saw in our performance results, NVIDIA’s Vista driver performance is definitely behind that of Windows XP. In some cases it’s not too bad, F.E.A.R. and Company of Heroes ran about 4-5% slower for the GeForce 8800 GTX for instance, although the margin was a little greater for the 7900 GTX under F.E.A.R. But there were also notable titles such as Half-Life 2 Lost Coast that really stood out. The 8800 GTX’s performance nearly dropped in half in Lost Coast.

Looking at the latest Gamespy stats, HL2 is the #2 shooter played online, trailing only the original Half-Life, so this is a pretty big deal. Battlefield 2 and BF2142 occupy the next two spots on Gamespy’s list, they’re both based on the same basic game engine, and the 8800 GTX saw significant performance declines under Vista in our testing with BF2142. That’s definitely not good.

Based on all this, it certainly looks to us at least that NVIDIA’s Vista drivers are definitely behind that of AMD, at least on the Direct3D front (we still give the nod to NVIDIA in OpenGL). Not that AMD’s Vista drivers are perfect, we noted performance drops there as well, and ran into significant problems when running OpenGL, but their D3D driver is more polished overall than NVIDIA’s right now. At least that’s how it looks on the outside looking in.

Thankfully, NVIDIA’s Dwight Diercks made it clear that NVIDIA isn’t resting on this issue. We eagerly await more details on their upcoming driver release plan for Vista, and we’re sure many of you do too. It’s a shame though that it appears that GeForce 7 SLI owners will have to wait until March before they get a driver, and we’re crossing our fingers that perhaps NVIDIA could provide a beta a little sooner.

Considering all this, waiting a few months is the advice we’re giving to all NVIDIA-based gamers who have been tempted to take the plunge on Vista. Clearly there are titles like Company of Heroes where the performance hit is reasonably acceptable, but those titles are unfortunately a little too rare at this point. If you’re not heavily into gaming, or perhaps you spend more time playing older titles and/or less graphically-intense games, by all means, feel free to give Vista a shot, but for everyone else, the performance issues and other quirks may not be worth the headache. Clearly OEMs like Dell aren’t shipping Vista pre-installed on their high-end gaming rigs for a reason – that part of the market just isn’t ready yet. Hopefully in a few months time, it will be though. We’ll certainly be keeping an eye on how the whole situation plays itself out…