The future
Here’s a question for you, after seeing CPU test benchmarks that show identical performance between three processors because the video card is the bottleneck, what have we learned about the CPU’s performance? How do we know how it compares to other processors in its class? Moreover, those tests indicate nothing about what we can expect from it if it is paired with a better video card.
Remember, CPUs tend to be longer-term purchases. The CPU industry typically doesn’t move as fast as the graphics industry, up to this point AMD and Intel only introduce radically new CPU microarchitecture’s roughly every 4-5 years. In comparison ATI and NVIDIA introduce their next-gen GPUs roughly every 12-24 months. In the graphics world going from one generation to the next often equates to a doubling in performance. We saw this when going from the GeForce 4 to Radeon 9700, and Radeon 9700/9800 to the X800, from Radeon X800 to GeForce 7800. The list of examples goes on and on.
This isn’t necessarily always the case for CPUs however, particularly when a new architecture is first introduced. We didn’t see this when Intel transitioned from Pentium III to Pentium 4 for example, or for AMD, Athlon XP 3200+ to Athlon 64 FX-51. Now granted in the latter example the difference between the two CPUs was significant, a 30% increase wasn’t uncommon, but performance didn’t double.
Keep in mind that with the debut of previous next-gen GPUs, it wasn’t unheard of for the new graphics card to be CPU-bound at resolutions as high as 1600x1200.
Now consider that just a few months from now, we are expecting ATI and NVIDIA's latest creations - the R600 and G80 GPUs. Unlike the last two revisions, this is truly next-generation hardware, the kind of leap we saw last when the GeForce 6800 and Radeon X800 came out to replace the 5900 and 9800 lines.
When these next-gen video cards come out, how will you be able to choose the appropriate processor to match them, if all your CPU benchmarks were run with older video cards that are maxed out? Even if you cannot afford a Core 2 Duo E6700 with G80 now, next summer after price cuts you may be able to.
With traditional benchmarks, you'll still be able to see its relative performance to other CPUs with no bottlenecks, while "real-world" benchmarks will uselessly indicate that all CPUs are created equal and there is no point trying to discern between them.
Meanwhile, the benchmark standard, refined over the years also shows that once you bump up resolution and settings, the differences between the CPUs are more or less eliminated. It becomes obvious to anyone with even nominal tweaking experience that there are two forces at play, that one is more important at lower resolutions, and the other at higher settings. Of course, if upcoming next-gen video cards make another leap forward as they did with the last generational leap, even high resolutions may become CPU-bound.