Caches Relieve Some of the Pressure

Cache to the Rescue!

The cache sits between the processor and the main memory, feeding the processor whatever information it has on hand. Obviously, the cache cannot hold all of the information that the main memory holds, so it only keeps what information the processor has most recently used and discards the rest. There are several levels of cache, denoted by the names L1, L2, L3… etc., in descending order of speed. A L1 cache is on the same silicon die as the processor, and generally runs at the same speed. This makes it extremely fast, but electrical propagation laws limit its physical size. Some modern processors devote 80% of their transistors to the L1 cache. A typical L1 cache size is 32KB, as with the PIII.

L2 cache used to be on the motherboard, but starting with the Pentium Pro, Intel has moved the L2 cache on a separate die inside the same physical package as the processor. The L2 cache generally runs at a fraction of the processor speed, with a 128bit or 64bit interface to the processor die. By placing the L2 cache physically close to the processor, the leads in between are much shorter and the cache speed can be much higher. The older Katmai PIIIs have 512KB of discrete cache running at ½ processor speed, the new PIII Coppermines carry 256KB of cache right on the core which allows it to run at full processor speed. Some high end systems, especially ones equipped with Digital Alpha chips, come with L3 caches on the motherboard that range from 1 to 16MB. They are rare in today's new PCs, however.

All Cache?

Why do we have normal system memory if the cache is so much faster? Physical and monetary limitations disallow large caches, and modern operating systems and programs (cough, Microsoft, cough) need at least 64MB of RAM. How do the caches fit in? UC Berkeley Professor, David Patterson, uses the analogy of food on the dinner table. When you eat, you take the food off of your plate and put it in your mouth. IF you need more food, you generally bring it on the dinner table, so you don't have to go back the kitchen. When the dinner table is empty (you're really hungry), you take more food from the kitchen. When the kitchen is empty, you drive to the supermarket. Although the supermarket contains all of the food, you don't drive there between every bite, you cache the food in successively closer locations: the kitchen, the table, the plate. Likewise with a processor, the L1 cache is the processor's plate, the L2 is the kitchen, the main memory is the supermarket, and the hard drive is the farm. The food? Instructions and data.