This appendix describes the CPU types used in Silicon Graphics workstations and servers. These CPU types are summarized in Table A-1.

To determine which CPU a Silicon Graphics system uses, type:

% uname -m

at a shell prompt. This command reports the CPU type. For details on this command, see the uname(1) man page.

For a more detailed listing of a Silicon Graphics system's hardware configuration, type:

% hinv

at a shell prompt. See the hinv(1M) man page.

All CPUs use the MIPS R2000, R3000, R4000, or R8000 (formerly known as “TFP”) series processor chips (R4400 and R4600 are the same as the R4000 for all practical purposes). These chips use a data cache to maximize the efficiency of fetching of heavily used memory.

The IP5, IP7, IP11, IP15, IP19, and IP21 multiprocessor CPUs have bus-watching caches that automatically invalidate the data cache when the system performs DMA (direct memory access) into physical memory. For these CPUs, no data cache write back or invalidation is required by software because these functions are performed by hardware.

DMA operations are categorized as DMA reads or DMA writes. DMA operations that transfer from memory to a device, and hence read memory, are DMA reads. DMA operations that transfer from a device to memory are DMA writes. Thus, you may want to think of DMA operations as being named from the point of view of what happens to memory.

The single-processor CPUs based on the R2000 or R3000 employ a cache architecture known as a write through cache. This means that all stores generated by the processor to a cached memory location go into the cache and into memory at the same time. Therefore, the data caches on these systems never contain data that is more recent than memory. However, after the system performs DMA to physical memory, the data lines in the processor's cache corresponding to this physical memory contain data that is stale with respect to memory. Therefore, a driver running on a single-processor CPU must explicitly invalidate the data cache before reading from the corresponding cached address, after the DMA completes.

R4000s and R8000s employ a cache architecture known as a write back cache. This means that stores generated by the processor go only into cache; they are written back from cache to memory only when a cache miss causes that cache line to be replaced. For this type of cache, the cache contains data that is newer than the corresponding memory locations. Memory is then stale with respect to the cache. Drivers that perform DMA reads from memory to device must specifically cause the cache to be written back to memory before the DMA starts. On IP19 and IP21 platforms, DMA pulls data from the processor caches, if necessary, thus providing coherent I/O.

Recall the code examples that read kernel data in Chapter 3, “Writing a VME Device Driver,” and Chapter 5, “Writing a SCSI Device Driver.” Before reading the data, the driver code used the dki_dcache_inval() function to invalidate the appropriate data cache lines. When the data cache lines are invalidated, accessing the kernel data causes a cache miss and, thus, forces a read from physical memory. Therefore, to ensure driver portability, your driver must always use dki_dcache_inval() to invalidate the data cache. This is the case even though the dki_dcache_inval() functions defined for the IP5 and IP7 use stub functions that do not do any actual work, although they use dki_dcache_wb() before starting a DMA from memory to device. See Table A-2 for a summary of cache line sizes for various MIPS processors.

If your driver uses the functions userdma() or physio() (physio calls userdma() internally), the data cache is automatically written back and invalidated for you no matter what system you are using. If your driver does not use these functions for a DMA write into cached memory, your driver must use dki_dcache_inval() to invalidate the data cache explicitly after the DMA completes.^[18] Further, if your driver does a DMA read from memory to device, it must use dki_dcache_wb() to write back the data cache explicitly before the DMA is started.

Another consideration worth mentioning is that of buffer alignment for DMA. The R4000 processor implements a secondary cache line size of between 4 and 32 words (the secondary cache line size is dependent upon the CPU board implementation). Buffers used for DMA must be aligned on a byte boundary that is equal to the cache line size. To accomplish this, use the kmem_alloc() function with the KM_CACHEALIGN flag. This returns a buffer with the necessary alignment for the system.

---

Note: The R8000 has the same DMA alignment problems in general as the R4000. This is true for all systems with write back caches.

---

Why is this alignment necessary? Suppose you have a variable, X, followed by a buffer you are going to use for DMA write. If you invalidate the buffer prior to the DMA write, but then reference the variable X, the resulting cache miss brings part of the buffer back into the cache. When the DMA write completes, the cache is stale with respect to memory. If, however, you invalidate the cache after the DMA write completes, you destroy the value of the variable X.

You may, in some cases, want to call flushbus() to ensure that any writes in the write buffer have actually been flushed to the system bus. This is sometimes necessary when a device requires delays between PIOs, particularly between a write and a read, since they might otherwise arrive at the device back-to-back.

For example, you write to the device, delay, write to the device, delay. These writes may be buffered regardless of the delay and still be sent to the device in quick succession.

Variables not declared volatile may be optimized to registers in the processor, since there are multiple processors. Multiple inconsistent copies of a variable may exist in registers if volatile is not declared.

---

Note: Use the -O compiler flag to turn optimization on. The -g compiler flag for debugging disables optimization.

---

The multiprocessor CPUs implement test-and-set variables in hardware. These variables are known as spinlocks. Your code can use functions such as LOCK_ALLOC(D3) and LOCK(D3) to take advantage of these variables to protect a critical region of code or data on one processor from interference from other processors.

All kernels for all Silicon Graphics CPUs support the spinlock functions (although these functions perform no actual work on single-processor systems). Therefore, a fully semaphored multiprocessor device driver can run unmodified on a single-processor system.

---

Note: Because IRIX kernels cannot, as a rule, be preempted, any driver that sits in a loop waiting for some condition to be satisfied may tie a processor up for as long as it wants. Real-time processes, such as audio, are very sensitive to such delays.

---

Recall that these base addresses are passed to the driver through the device edtinit() function. Therefore, while you may need a different system file for each Silicon Graphics platform, the driver code itself can still be portable.

Table A-3, Table A-4, and Table A-5 show the mapping of kernel virtual address, physical address, and VME-bus address for each addressing mode. For IP6 and IP12 systems, block mode and burst mode transfers are supported only during DMA, where the VME device is the master. No predefined addresses are available on CHALLENGE/Onyx series systems.

---

Note: When consulting Table A-3 through Table A-7, note that VME space is set up by software on various hardware platforms. Always use the macro #defines in /usr/include/sys/vmereg.h to refer to the various VME resource address ranges by their logical names.

---

When a VME device uses DMA to access main memory, it acts as a “VME master.” Silicon Graphics systems support both A24 and A32 VME master addressing. Although there are a few minor differences in the addressing modes supported (non-privileged versus supervisor), the main difference is that the 4D/100, 4D/200, 4D/300, 4D/400, and Crimson series systems support dynamic address mapping. This allows IP5, IP7, IP9, and IP17 CPUs to access all of physical memory for A24 addressing and allows scatter-gather for both A24 and A32 addressing.

You can still write an IRIS-portable device driver if the dma_mapalloc() function does not return –1 and if the device driver can take advantage of the DMA mapping functions described in Chapter 5, “Writing a SCSI Device Driver.” Otherwise, you must use one of the other DMA methods.

Table A-6 describes the mapping between the address generated by a VME device and physical memory. You can perform A32 master addressing on the IP5, IP7, IP9, and IP17 in either mapped or unmapped mode.

On POWER Series workstations, ranges of VME-bus address space were mapped one-to-one with K2 segment addresses. This made accessing the VME bus easy but was also limiting. Only a small amount of K2 space is available for use by VME, so very little of the VME address space was made available. Even worse, for dual VME-bus systems, the space previously available was now cut in half as it was shared between the two buses.

The CHALLENGE series supports up to five VME buses. Since K2 space is a limited resource, and dividing up what is available by five would make the extra VME buses next to useless, a new approach was tried. The CHALLENGE series does not have a direct K2 address map into VME-bus space. Each VME bus adapter has the ability to map fifteen 8 MB windows of VME-bus space into K2 space. These windows can be slid around at will to give the illusion of a much larger address space.

To access a VME space, a user must allocate a PIO map, which provides a translation between a kernel address and VME space. These mappings can be “FIXED” or “UNFIXED.” As on POWER Series platforms, a FIXED mapping is a one-to-one mapping of a range of VME-bus space into the driver's address space. An UNFIXED window takes advantage of the sliding window ability on the CHALLENGE series, which supports both FIXED and UNFIXED mappings.

In an UNFIXED map, VME-bus space cannot be accessed directly; instead, access is provided through special bcopy() routines used to move data between VME space and kernel buffers. While it is not always possible to get a FIXED mapping, an UNFIXED mapping is always available. The special bcopy() routines work for both FIXED and UNFIXED mappings. On POWER Series and earlier workstations, UNFIXED mappings are treated as FIXED mappings.

The PIO mapping routines also have a general interface that allows them to be used for mapping in bus spaces other than VME.

The support routines for PIO mapping are:

These PIO maps are normally set up in the driver's drvedtinit() routine.

Table A-7 shows the VME-bus space reserved for customers.

---

Note: For information that may not have been available when this guide went to press, refer to the comments in the /var/sysgen/system/irix.sm file.

---

For POWER Indigo2 or POWER CHALLENGE M processors, uncached
(K1 segment) writes to main memory must be double-word (64-bit) writes, and they must be aligned on a double-word boundary. The reason for this has to do with the POWER Indigo2's hardware support for ECC-protected memory. Since the ECC calculation requires that data be written to main memory in 64-bit (8-byte) chunks, smaller (1-, 2-, and 4-byte) uncached writes to main memory tend to produce memory corruption. Cached accesses (both read and write), as well as uncached read operations do not cause problems; neither do accesses that are not to main memory, such as device register reads and writes.

Any device driver that performs uncached writes to main memory must always do writes in 8-byte quantities.^[19] If the driver needs to write a smaller piece of memory, then it must do the write as a read-modify-write operation of an 8-byte piece of memory. For example, a driver with code like the following (assuming that the structure is being accessed uncached), would corrupt memory:

struct foo_s {
    char    b0;
    char    b1;
    char    b2;
    char    b3;
    char    b4;
    char    b5;
    char    b6;
    char    b7;
} bar1;

driver_write_bar1_b3 ()
{
    bar1.b3 = 5;
}

Instead, it would have to be modified to perform a read-modify-write operation on the whole double-word containing b3, as in the following example (on a big-endian system):

struct foo_s {
    union {
        __uint64    dw;    /* A 64-bit quantity */
        struct {
            char    b0;    /* READ ONLY */
            char    b1;    /* READ ONLY */
            char    b2;    /* READ ONLY */
            char    b3;    /* READ ONLY */
            char    b4;    /* READ ONLY */
            char    b5;    /* READ ONLY */
            char    b6;    /* READ ONLY */
            char    b7;    /* READ ONLY */
        } byte;
    } dw0;
} bar1;

driver_write_bar1_b3 ()
{
    bar1.dw0.dw = (bar1.dw0.dw & ~(0xff<<32)) | (5<<32);
}

If at all possible, use cached accesses to memory, along with cache coherency operations where necessary, instead of uncached operations. Cache coherency operations are necessary only for data shared by the CPU and a device that needs to perform DMA. See “Data Cache Write Back and Invalidation” for more information on use of cache coherency operations.