Page 90 - A Practical Guide from Design Planning to Manufacturing

66 Chapter Two

and the cache. All of these levels of storage working together provide the
illusion of a memory with the capacity of your hard drive but an effective
latency that is dramatically faster.
We can picture a processor using the memory hierarchy the way a man
working in an office might use filing system. The registers are like a
single line on a sheet of paper in the middle of his desk. At any given
moment he is only reading or writing just one line on this one piece of
paper. The whole sheet of paper acts like the level 1 cache, containing
other lines that he has just read or is about to read. The rest of his desk
acts like the level 2 cache holding other sheets of paper that he has
worked on recently, and a large table next to his desk might represent
main memory. They each hold progressively more information but take
longer to access. His filing cabinet acts like a hard drive storing vast
amounts of information but taking more time to find anything in it.
Our imaginary worker is able to work efficiently because most of time
after he reads one line on a page, he also reads the next line. When finished
with one page, most of the time the next page he needs is already out
on his desk or table. Only occasionally does he need to pull new pages
from the filing cabinet and file away pages he has changed. Of course,
in this imaginary office, after hours when the business is “powered
down,” janitors come and throw away any papers left on his desk or
table. Only results that he has filed in his cabinet, like saving to the hard
drive, will be kept. In fact, these janitors are somewhat unreliable and
will occasionally come around unannounced in the middle of the day to
throw away any lose papers they find. Our worker would be wise to file
results a few times during the day just in case.
The effective latency of the memory hierarchy is ultimately deter-
mined not only by the capacity and latency of each level of the hier-
archy, but also by the way each program accesses data. Programs that
operate on small data sets have better hit rates and lower average
access times than programs that operate on very large data sets.
Microprocessors designed for computer servers often add more or larger
levels of cache because servers often operate on much more data than
typical users require. Computer performance is also hurt by excessive
page faults caused by having insufficient main memory. A balanced
memory hierarchy from top to bottom is a critical part of any computer.
The need for memory hierarchy has arisen because memory per-
formance has not increased as quickly as processor performance. In
DRAMs, transistor scaling has been used instead to provide more
memory capacity. This allows for larger more complex programs but
limits the improvements in memory frequency. There is no real advantage
to running the bus that transfers data from memory to the processor at a
higher frequency than the memory supports.