Page 89 - A Practical Guide from Design Planning to Manufacturing

Computer Components 65

If the data is not found (a cache miss), it will be retrieved from main
memory after 50 cycles. This gives an average access time of 21.5 cycles.
Increasing the size of the cache increases the hit rate and the latency
of the cache. For this example, the average access time is improved by
using a 32-kB cache but begins to increase as the cache size is increased
to 128 kB. At the larger cache sizes the improvement in hit rate is not
enough to offset the increased latency.
The last column of the table shows the most common solution to this
trade-off, a multilevel cache. Imagine a processor with a 4-kB level 1
cache and a 128-kB level 2 cache. The level 1 cache is always accessed
first. It provides fast access even though its hit rate is not especially
good. Only after a miss in the level 1 cache is the level 2 cache accessed.
It provides better hit rate and its higher latency is acceptable because
it is accessed much less often than the level 1 cache. Only after misses
in both levels of cache is main memory accessed. For this example, the
two-level cache gives the lowest overall average access time, and all
modern high performance processors incorporate at least two levels
of cache memory including the Intel Pentium II/III/4 and AMD Athlon/
Duron/Opteron.
If a needed piece of data is not found in any of the levels of cache or
in main memory, then it must be retrieved from the hard drive. The hard
drive is critical of course because it provides permanent storage that is
retained even when the computer is powered down, but when the com-
puter is running the hard drive acts as an extension of the memory
hierarchy. Main memory and the hard drive are treated as being made
up of fixed-size “pages” of data by the operating system and micro-
processor. At any given moment a page of data might be in main memory
or might be on the hard drive. This mechanism is called virtual memory
since it creates the illusion of the hard drive acting as memory.
For each memory access, the processor checks an array of values
stored on the die showing where that particular piece of data is being
stored. If it is currently on the hard drive, the processor signals a page
fault. This interrupts the program currently being run and causes a por-
tion of the operating system program to run in its place. This handler
program writes one page of data in main memory back to the hard drive
and then copies the needed page from the hard drive into main memory.
The program that caused the page fault then continues from the point
it left off.
Through this slight of hand the processor and operating system
together make it appear that the needed information was in memory all
the time. This is the same kind of swapping that goes on between main
memory and the processor cache. The only difference is that the operating
system and processor together control swapping from the hard drive to
memory, whereas the processor alone controls swapping between memory