The design of computer memory is based on two fundamental principles that guide how architects balance speed, cost, and efficiency.

Principle of Locality

The principle of locality states that programs access a relatively small portion of their data and instructions at any instant of time. This principle allows computers to run faster and use resources more efficiently by storing frequently accessed data closer to the processor.

Temporal Locality refers to the tendency of a program to reuse recently accessed data or instructions soon again. For example, when solving a mathematical problem, you might refer back to a previously calculated result rather than recalculating it.

Spatial Locality refers to the tendency of a program to access data or instructions close to those it just accessed. For instance, if you read one paragraph in a textbook, you are likely to read the next paragraph next because the information is spatially close.

Principle of Cost-Performance Ratio

The principle of cost-performance ratio states that faster and smaller memories are more expensive and consume more power, while slower and larger memories are cheaper and more energy-efficient. Computer architects use this principle to create a memory hierarchy that balances speed, cost, and power consumption.

Memory hierarchy optimizes computer performance by using various memory levels with distinct speeds and capacities. The hierarchy consists of:

• Registers: The fastest and most expensive memory units, located within the CPU itself.
• Cache: Smaller, faster memory closer to the processor for frequently accessed data.
• Main Memory (RAM): The primary memory providing accessible storage for running programs.
• Secondary Storage (Hard Drive): The largest and slowest memory for permanent data storage.

Key Concepts in Memory Interfacing

When the processor requests data, the system checks cache first:

• Cache Hit: Occurs when the requested data is found in cache memory, allowing quick delivery to the processor.
• Cache Miss: Occurs when the data is not found in cache, requiring retrieval from lower memory levels.
• Hit Rate: The percentage of times data is found in cache without accessing lower levels.

$\text{Hit rate} = \frac{\text{Total hits}}{\text{Total hits} + \text{Total misses}}$

• Miss Rate: The percentage of times data is not found in cache.

$\text{Miss rate} = \frac{\text{Total misses}}{\text{Total hits} + \text{Total misses}}$

• Miss Penalty: The extra time required to fetch data from lower memory levels.

Average Memory Access Time (AMAT)

AMAT represents the average time to access memory considering both hits and misses:

$\text{AMAT} = \text{Hit time} + \text{Miss rate} \times \text{Miss penalty}$

Example: Calculate AMAT for a processor with a cache hit rate of 95%, hit time of 4ns, and miss penalty of 80ns.

Solution:

• Miss rate = 100% - 95% = 5% = 0.05
• AMAT = 4ns + (0.05 × 80ns) = 4ns + 4ns = 8ns

Four primary memory technologies balance speed, size, and cost:

• DRAM (Dynamic Random Access Memory): Forms main memory, storing data as electrical charges in capacitors that need constant refreshing. It offers balance between cost-effectiveness and capacity but is slower than SRAM.

• SRAM (Static Random Access Memory): Used in registers and cache, it has faster access times using transistor-based circuits that hold data without refreshing. However, it is more expensive and has limited capacity.

• Flash Memory: A non-volatile memory using floating-gate transistors to store data even when power is off. Ideal for storing files, applications, and operating systems in devices like USB drives and smartphones.

• Magnetic Disk: Used in hard disk drives (HDDs), offering the most cost-effective way to store large amounts of data permanently, though with slower access times due to mechanical components.

Cache memory is a smaller, faster memory that stores copies of data from frequently used main memory locations.

Types of Cache Memory

• Split Cache: Features separate caches for data and instructions, allowing simultaneous fetching but dividing available space rigidly.

• Unified Cache: Data and instructions reside together, offering more flexibility and potentially higher hit rate.

Cache Levels

• Level 1 (L1): The primary cache closest to the CPU, offering fastest access time. Small size stores only most frequently used data.

• Level 2 (L2): Secondary cache with larger capacity than L1, either on the CPU chip or connected via high-speed bus.

• Level 3 (L3): Larger capacity supplement to L1 and L2, with speed approximately double that of main memory (DRAM).

Memory mapping refers to techniques used to bring data from lower levels of memory to higher levels. Cache memory mapping determines where in the cache the data should be placed.

Cache Mapping Techniques

Direct Mapping: Each block of main memory maps to a single, specific cache line using the formula:

Cache line number = (Main memory block address) modulo (Total cache lines)

Example: For a cache with 8 lines and main memory block address 29:

• Cache line = 29 % 8 = 5
• Block 29 maps to cache line 5

Fully Associative Mapping: Any block of main memory can be loaded into any cache line. When full, replacement algorithms like LRU (Least Recently Used) determine which block to replace.

Set-Associative Mapping: Cache is divided into sets, each containing a fixed number of lines (k-way). Each main memory block maps to a specific set and can occupy any line within that set.

Number of sets = (Block address) modulo (Number of sets)

Example: Calculate tag, set, and offset for a 2-way set-associative cache with cache size 256 bytes, block size 32 bytes, and main memory address 768.

• Number of lines = 256/32 = 8 lines
• Number of sets = 8/2 = 4 sets
• Block offset bits = log₂(32) = 5 bits
• Set bits = log₂(4) = 2 bits
• Tag bits = Address bits - (set bits + offset bits)

Primary Memory (main memory) includes RAM and cache, providing fast, temporary storage directly accessible by the CPU for active processes. It is volatile, meaning data is lost when power is off.

Secondary Memory includes hard drives, SSDs, and optical discs, offering larger, permanent storage for data and programs not in immediate use. It is non-volatile and has slower access speeds compared to primary memory.

Virtual Memory

Virtual memory expands a computer's usable memory by using both primary (RAM) and secondary storage. The operating system creates a virtual memory space divided into pages, loading only active pages into RAM while keeping less-used pages in secondary storage.

The Translation Lookaside Buffer (TLB) is a cache in the CPU's Memory Management Unit (MMU) that stores recently used virtual-to-physical address translations to speed up address conversion.

In Tanzania, understanding computer memory helps in troubleshooting everyday devices. For example, when a student's smartphone becomes slow because many applications are running simultaneously, they can close unused apps to free up RAM (primary memory), similar to how cache memory works by keeping frequently used data close to the processor for quick access. This understanding also helps in choosing appropriate devices—knowing that SSDs (secondary memory) are faster but more expensive than HDDs assists small business owners in Dar es Salaam making informed decisions when purchasing computers for their shops.