UNIT - V

1. Memory System Design

 Memory System Design

• Memory hierarchy

• Cache Memory

• Main Memory
 Memory Hierarchy
• The memory unit is needed for storing programs
and data.

• The memory unit that communicates directly with

the CPU is called the main memory.

• Devices that provide backup storage are called

auxiliary memory (ex:- magnetic disks and tapes)

• Only programs and data CURRENTLY NEEDED by

the processor reside in main memory
• The memory hierarchy system consists of all storage
devices employed in a computer system from the
slow but high-capacity auxiliary memory to a
relatively faster main memory, to an even smaller
and faster cache memory accessible to the high-
speed processing logic. (Figure below)

• At the bottom of the hierarchy are the relatively slow magnetic tapes used to
store removable files.
• Next are the magnetic disks used as backup storage.
• The main memory occupies a central position by being able to communicate
directly with the CPU and with auxiliary memory devices through an I/O
processor.

• When programs not residing in main memory are needed by the CPU, they are
brought in from auxiliary memory. Programs not currently needed in main memory
are transferred into auxiliary memory to provide space for currently used
programs and data.
 Memory Hierarchy in Computer
• Memory Hierarchy is to obtain the highest possible access speed while
minimizing the total cost of the memory system
Auxiliary memory
Magnetic
tapes I/O Main
processor memory
Magnetic
disks

CPU Cache
memory

As one goes DOWN the hierarchy,

the following occur:

a. Decreasing cost per bit

b. Increasing capacity

c. Increasing access time

d. Decreasing frequency of access

of the memory by the processor
• Registers:
– internal to the processor
– fastest, smallest, and most expensive type of memory
– Processor contains a few dozen registers to hundreds of registers

• Main memory:
– communicates DIRECTLY with the CPU
– communicates with auxiliary memory devices (disks, tapes) through an I/O
processor
– is the PRINCIPAL INTERNAL memory system of the computer
– Each location in main memory has a unique address
– Main memory is usually extended with a higher-speed, smaller cache

• Cache:
– is NOT visible to the programmer or to the processor
– very-high speed memory
– Used to compensate for the speed differential between main memory
access time and processor logic
– Its access time is close to processor logic clock cycle time
• Auxiliary memory or Secondary memory:
– Devices that provide backup storage are called
auxiliary memory
– Ex:- magnetic disks, magnetic tapes
– used for storing system programs, large data files,
and other backup information
– Only programs and data currently needed by the
processor reside in MAIN MEMORY
– All other information is stored in auxiliary memory
and transferred to main memory when needed
 Main Memory
• The main memory is the central storage unit in a
computer system.

• It is a relatively large and fast memory used to

store programs and data DURING THE COMPUTER
OPERATION.

• The stored information remains valid as long as

power is applied to the unit.

• Integrated circuit (Random-Access Memory) RAM

chips are available in two possible operating
modes, static and dynamic.
• Most of the main memory in a general-purpose
computer is made up of RAM integrated circuit
chips, but a PORTION OF THE MEMORY may be
constructed with (Read-Only Memory) ROM chips.

• RAM is used for storing the bulk of the programs and

data that are subject to change.

• ROM is used for storing programs that are

permanently resident in the computer and for tables
of constants that do not change in value once the
production of the computer is completed.
• ROM portion of main memory is needed for storing an
initial program called a BOOTSTRAP LOADER.

• The bootstrap loader function is to start the computer

software operating when power is turned on.

• When power is turned on, the hardware of the

computer sets the program counter to the first
address of the bootstrap loader.

• The bootstrap program loads a portion of the

operating system from disk to main memory and
control is then transferred to the operating system,
which prepares the computer for general use.
RAM and ROM Chips
Typical RAM chip
• A RAM chip can communication with the CPU if it has one or
more control inputs that select the chip only when needed.

• A bidirectional data bus that allows the transfer of data either

from memory to CPU during a read operation, or from CPU to
memory during a write operation.

• (Above diagram) The capacity of the memory is 128 words of

eight bits (one byte) per word.

• This requires a 7-bit address and an 8-bit bidirectional data bus.

• The read and write inputs specify the memory operation and
the two chips select (CS) control inputs are for enabling the
chip only when it is selected by the microprocessor.
• (Fig.(b)) The unit is in operation only when CS1 = 1 and CS2-
bar = 0.

• If the chip select inputs are not enabled, or if they are

enabled but the read or write inputs are not enabled, the
memory is inhibited and its data bus is in a high-impedance
state.

• When CS1 = 1 and CS2-bar = 0, and

• when the WR input is enabled, the memory stores a byte
from the data bus into a location specified by the address
input lines.
• When the RD input is enabled, the content of the selected
byte is placed into the data bus.
• The RD and WR signals control the memory operation as well
as the bus buffers associated with the bidirectional data bus
 Typical ROM chip

• A ROM can only read, the data bus can only be in an output mode.

• The nine address lines in the ROM chip specify any one of the 512 bytes stored
in it.

• The two chip select inputs must be CS1 = 1 and CS2_bar = 0 for the unit to
operate. Otherwise, the data bus is in a high-impedance state.

• There is no need for a read or write control because the unit can only read.

• Thus when the chip is enabled by the two select inputs, the byte selected by the
address lines appears on the data bus.
 Memory Address Map
• A memory address map, is a pictorial representation of assigned address
space for each chip in the system.
• Assume that a computer system needs 512 bytes of RAM and 512 bytes of
ROM.
• The component column specifies whether a RAM or a ROM chip is used.
• The hexadecimal address column assigns a range of hexadecimal equivalent
addresses for each chip.
• Although there are 16 lines in the address bus (third column), the table shows
only 10 lines because the other 6 are not used (assumed zero).
• The small x's designate those lines that must be connected to the address
inputs in each chip.
TABLE: Memory Address Map for Microprocomputer
• The RAM chips have 128 bytes and need seven address lines.

• The ROM chip has 512 bytes and needs 9 address lines.

• The x's are always assigned to the low-order bus lines: lines 1
through 7 for the RAM and lines 1 through 9 for the ROM.

• When line 10 is 0, the CPU selects a RAM, and when this line is
equal to 1, it selects the ROM.

• The low-order lines in the address bus select the byte within
the chips and
• other lines in the address bus select a particular chip through
its chip select inputs.
 Memory Connection to CPU
• This configuration gives a memory
capacity of 512 bytes of RAM and 512
bytes of ROM.

• Each RAM receives the seven low-order

bits of the address bus to select one of
128 possible bytes.

• The particular RAM chip selected is

determined from lines 8 and 9 in the
address bus.

• This is done through a 2 x 4 decoder

whose outputs go to the CS1 inputs in
each RAM chip.

• Thus, when address lines 8 and 9 are

equal to 00, the first RAM chip is
selected. When 01, the second RAM
chip is selected, and so on.
• The RD and WR outputs from the 512 x 8

microprocessor are applied to the

inputs of each RAM chip.
• The RAMs are selected when the bit
in this line 10 is 0, and the ROM
when the bit is 1 .

• The other chip select input (CS1) in

the ROM is connected to the RD
control line for the ROM chip to be
enabled only during a read
operation.

• Address bus lines 1 to 9 are applied

to the input address of ROM
without going through the
decoder. This assigns addresses 0
to 511 to RAM and 512 to 1023 to
ROM.

• The data bus of the ROM has only

an output capability, whereas the
data bus connected to the RAMs
can transfer information in both
directions.
 Cache Memory

• Cache Hit: The refers to memory and finds the

word in cache, it is said to produce a hit.
• Cache Miss: If the word is not found in cache it is
in main memory and it counts as a miss.

Hit ratio= Hit/(Hit + Miss)

• If the active portions of the program and data are
placed in a fast small memory, the average memory
access time can be reduced, thus reducing the total
execution time of the program. Such a fast small
memory is referred to as a CACHE MEMORY.

• It is placed between the CPU and main memory

• The cache memory access time is less than the

access time of main memory by a factor of 5 to 10.

• The fundamental idea of cache organization is that

by keeping the most frequently accessed
instructions and data in the fast cache memory.
Basic operation of the cache:

• When the CPU needs to access memory, the cache is examined. If

the word is found in the cache, it is read from the fast memory. If
the word addressed by the CPU is not found in the cache, the
main memory is accessed to read the word.

• A block of words containing the one just accessed is then

transferred from main memory to cache memory.
• The block size may vary from one word (the one just accessed) to
about 16 words adjacent to the one just accessed.

• In this manner, some data are transferred to cache so that future

references to memory find the required words in the fast cache
memory.
 Hit Ratio
• The performance of cache memory is frequently measured in
terms of a quantity called Hit Ratio.

• When the CPU refers to memory and finds the word in cache, it
is said to produce a HIT .
• If the word is not found in cache, it is in main memory and it
counts as a MISS .

• The ratio of the number of hits divided by the total CPU

references to memory (hits plus misses) is the Hit Ratio.

• This high ratio verifies the validity of the locality of reference

property.
(Analysis of a large number of typical programs has shown that the references to memory at any
given interval of time tend to be confined within a few localized areas in memory. This
phenomenon is known as the property of locality of reference.)
 Mapping
• The transformation of data from main memory
to cache memory is referred to as a mapping
process.

• Three types of mapping procedures are

considering the organization of cache memory:

• 1. Associative mapping
• 2. Direct mapping
• 3. Set-associative mapping
• The main memory can store 32k words of 12 bits each.
• The cache is capable of storing 512 of these words at any given time.
• The CPU communicates both.
• For every word stored in cache, there is a duplicate copy in main
memory.

• It first sends a 15-bit address to cache.

• If there is a hit, the CPU accepts the 12-bit data from cache.
• If there is a miss, the CPU reads the word from main memory and the
word is then transferred to cache.
 Associative Mapping

15 bits as a 12 bits as a
5 digit octal 4 digit octal
• The fastest and most flexible cache organization uses an
associative memory.

• The associative memory stores both the address and

content(data) of the memory word.

• This permits any location in cache to store any word from main
memory.

• If the data not available in cache , the address and data pair are
transferred to associative memory.
• The above diagram shows three words presently stored in the cache.

• The address value of 15 bits is shown as a five-digit octal number and its
corresponding 12-bit word is shown as a four-digit octal number.

• A CPU address of 15 bits is placed in the argument register and the associative
memory is searched for a matching address.
• If the address is found, the corresponding 12-bit data is read and sent to the CPU.
• If no match occurs, the main memory is accessed for the word.
• The address-data pair is then transferred to the associative cache memory.

• If the cache is full, an address-data pair must be displaced to make room for a pair
that is needed and not presently in the cache.

• The decision as to what pair is replaced is determined from the replacement

algorithm that the designer chooses for the cache.

• A simple procedure is to replace cells of the cache in round-robin order whenever

a new word is requested from main memory. This constitutes a first-in first-out
(FIFO) replacement policy.
 Direct Mapping
Addressing relationships between main and cache memories
• Associative memories are expensive compared to random-access
memories because of the added logic associated with each cell.

• Possibility of using a RAM for the cache is investigated in below

Fig.

• The CPU address of 15 bits is divided into two fields.

• The nine least significant bits constitute the INDEX field and the
remaining six bits form the TAG field.

• The figure shows that main memory needs an address that

includes both the tag and the index bits.

• The number of bits in the index field is equal to the number of

address bits required to access the cache memory.
• In the general case, there are 2k words in cache memory and 2n words in main
memory.

• The n-bit memory address is divided into two fields:

• k bits for the index field and (n-k) bits for the tag field.
• The DIRECT MAPPING cache organization uses the n-bit address to access the
main memory and the k-bit index to access the cache.

• (Fig.(b)) Each word in cache consists of the data word and its associated tag.

• When the CPU generates a memory request, the index field is used for the
address to access the cache.
• The tag field of the CPU address is compared with the tag in the word read from
the cache.
• If the two tags match, there is a hit and the desired data word is in cache.
• If there is no match, there is a miss and the required word is read from main
memory.
• It is then stored in the cache together with the new tag, replacing the previous
value.

• Disadvantage: the hit ratio can drop considerably if two or more words whose
addresses have the same index but different tags are accessed repeatedly.

• However, this possibility is minimized by the fact that such words are relatively
far apart in the address range (multiples of 512 locations in this example. )
• Consider example shown in Fig. below.
• The word at address zero is presently stored in the cache (index = 000, tag = 00, data = 1220).

• Suppose that the CPU now wants to access the word at address 02000.
• The index address is 000, so it is used to access the cache. The two tags are then compared. The cache
tag is 00 but the address tag is 02, which does not produce a match.
• Therefore, the main memory is accessed and the data word 5670 is transferred to the CPU.
• The cache word at index address 000 is then replaced with a tag of 02 and data of 5670.
• The direct-mapping organization using a block size of B words is shown in Fig below.

• The index field is now divided into two parts: the block field and the word field.
• In a 512-word cache there are 64 blocks of 8 words each, since 64 x 8 = 512.

• The block number is specified with a 6-bit field and the word within the block is specified with a 3-bit field.
• The tag field stored within the cache is common to all eight words of the same block.

• Every time a miss occurs, an entire block of eight words must be transferred from main memory to
cache memory.

• Although this takes extra time, the hit ratio will most likely improve with a larger block size because of
the sequential nature of computer programs.

Figure:
Direct mapping cache with
block size of 8 words
 Set-Associative Mapping
• Set-associative mapping, is an improvement over the direct mapping organization in that
each word of cache can store two or more words of memory under the same index
address.
• Each data word is stored together with its tag and the number of tag-data items in one
word of cache is said to form a set.
• An example of a set-associative cache organization for a set size of two is shown in Fig
below.

12 Bits data
word

CPU address
of 15 bits

Figure: Two-way set -associative mapping cache

• Each index address refers to two data words and their associated
tags.

• Each tag requires six bits and each data word has 12 bits, so the
word length is 2(6+12)=36 bits.

• An index address of nine bits can accommodate 512 words.

• Thus the size of cache memory is 512 x 36.

• It can accommodate 1024 words of main memory since each word
of cache contains two data words.

• In general, a set-associative cache of set size k will accommodate

k words of main memory in each word of cache.
• The words stored at addresses 01000 (tag|index) and 02000
(tag|index) of main memory are stored in cache memory at index
address 000.
• Similarly, the words at addresses 02777 and 00777 are stored in
cache at index address 777.

• When the CPU generates a memory request, the index value of the
address is used to access the cache.

• The tag field of the CPU address is then compared with both tags in
the cache to determine if a match occurs.

• The comparison logic is done by an associative search of the tags in

the set similar to an associative memory search: thus the name
"set-associative."

• The hit ratio will improve as the set size increases because more
words with the same index but different tags can reside in cache.
Replacement Algorithm
• When a miss occurs in a set-associative cache and the set is full, it is
necessary to replace one of the tag-data items with a new value.

• The most common replacement algorithms used are: random replacement,

first-in, firstout (FIFO), and least recently used (LRU).

• With the random replacement policy the control chooses one tag-data item
for replacement at random.

• The FIFO procedure selects for replacement the item that has been in the set
the longest.

• The LRU algorithm selects for replacement the item that has been least
recently used by the CPU.

• Both FIFO and LRU can be implemented by adding a few extra bits in each
word of cache.
Writing into Cache
• When the CPU finds a word in cache during a read operation, the main memory
is NOT involved in the transfer.

• If the operation is a write, there are two ways that the system can proceed.

• The simplest and most commonly used procedure is to update main memory
with every memory write operation, with cache memory being updated in
parallel if it contains the word at the specified address.
• This is called the WRITE-THROUGH method.

• The second procedure is called the WRITE-BACK method.

• In this method ONLY the cache location is updated during a write operation.
• The location is then marked by a flag so that later when the word is removed
from the cache it is copied into main memory.
• The reason for the write-back method is that during the time a word resides in
the cache, it may be updated several times; however, as long as the word
remains in the cache, it does not matter whether the copy in main memory is
out of date, since requests from the word are filled from the cache.
• It is only when the word is displaced from the cache that an accurate copy need
be rewritten into main memory.
Cache Initialization
• The cache is initialized when power is applied to the computer or when the
main memory is loaded with a complete set of programs from auxiliary
memory.

• After initialization the cache is considered to be empty, but in effect it

contains some non valid data.
• It is customary to include with each word in cache a VALID BIT to indicate
whether or not the word contains valid data.
• The cache is initialized by clearing all the valid bits to 0.

• The valid bit of a particular cache word is set to 1 the first time this word is
loaded from main memory and stays set unless the cache has to be initialized
again.

• The introduction of the valid bit means that a word in cache is not replaced by
another word unless the valid bit is set to 1 and a mismatch of tags occurs.
• If the valid bit happens to be 0, the new word automatically replaces the
invalid data.
• Thus the initialization condition has the effect of forcing misses from the cache
until it fills with valid data.