Carmack's Research

FS: Can you give us an overview on what you are researching now?

John: One of the rendering engine things that I'm working on specifically is a better lighting model. Quake 2 added radiosity lighting to the calculations where you would get realistic bouncing of light on there, but a lot of people actually didn't like that because it took away from some of the harder shadows where it kind of murked out more things, even if it was in a more realistic way - a lot of people liked the stronger shadows. In Quake 3 we went back to just a direct cast light, at least partially because I didn't want to do the finite element radiosity on the curve patches. The other thing is allowing us to manually make things look better by giving the artist the ability to texture and cut geometry everywhere. So those are the two directions and in fact, they can be combined or used together. They both have interesting technical challenges associated with them. Those are things which may very well show up into our next project.

There's some other research stuff I'm working on which isn't game engine related. I'm working on some vision research with webcams, like some head tracking stuff. There might be some kind of gimmick things I can integrate into games with that. Like when you're bobbing and shaking your head, your character can do that. Or maybe like the holographic window scenes where you move your head and it actually changes the view for you. I don't think the webcams have a fast enough latency to do that without some lag, but that's still some interesting stuff. There are other levels you can take this to, with facial expressions, etc.

I think there are probably more applications for all this than I can think of. All the little interesting discontinuities happened with changes in input/output devices. Things changed when people started using the mouse for everything. I think "computer vision" is practical enough that it may very well have a "mouse-like" effect on things. After all, there were things that you could do with the mouse that you wouldn't do with the keyboard. That's still in the random guess phase of things.

There are interesting things now that we have enough CPU power to do some pretty good stuff. And it also, unlike a lot of the graphics stuff, has a lot of unsolved research problems. There's active research going on but we don't have "that SGI example" to know how to put things together and just worry about making it fast.

The last direction I'm interested in is getting some more generalization of 3D technology, moving towards being able to do things with other rendering stuff like HTML inside 3D worlds, and being able to build little cyber space things that are not specifically game related. I believe there's an infrastructure framework that we can build. We can build a first person shooter, but the framework is general enough that we can do other things with it.

All the existing software is built clearly as a first person shooter game, but then people do other interesting things with the technology. It always costs a bit to generalize. You can always make something more efficient by specializing, but at some point you reach a decision where the cost to generalize is not that painful and the benefits by having a more general platform can be pretty significant. I think we're at that point for 3D games right now.

We clearly weren't two years ago. We were still caring about every last little frame to squeeze out any bit of performance. This isn't the case anymore with the trends in hardware acceleration, especially with geometry acceleration, much faster CPUs, more RAM, etc. I think we can do things that are 90% or more as fast as the specialized version, and still have the possibility to do a ton of other interesting things with it.