PCA does not recognize or generate directives that were only recently added to the multiprocessing C compiler. If PCA finds one of these new multiprocessing C compiler directives in your code, it prints a warning message and discards it. This guide clearly notes the new directives that are not processed by PCA. In future releases, PCA will recognize (and where appropriate, generate) these new directives. Thus, you should feel free to use these new directives in your code, but add them only after you have finished with PCA.

Power C pragmas are modeled after the Parallel Computing Forum (PCF) directives for parallel FORTRAN. The PCF directives define a broad range of parallel execution modes and provide a framework for defining C pragmas.

Some changes have been made to make the pragmas more C-like:

To make it easier to use pragmas, you can put several keywords on a single pragma line, or spread the keywords over several lines. In either case, you must put the key words in the correct order, and each pragma must contain an initial keyword.

For example:

#pragma parallel shared(a,b,c, n) local(i) pfor
#pragma iterate(i=0;n;1)
for (i=0; i<n; i++) a[i]=b[i]+c[i];

does the same thing as:

#pragma parallel
#pragma shared( a )
#pragma shared( b, c, n )
#pragma local( i )
#pragma pfor
#pragma iterate(i=0;n;1)
   for (i=0; i<n; i++) a[i]=b[i]+c[i];

To start a parallel region, use the parallel pragma. This pragma has a number of modifiers, but to run a single loop in parallel, the only modifiers you usually use are shared, byvalue, and local. These options tell the multiprocessing C compiler which variables to share between all threads of execution and which variables should be treated as local.

The code that comprises the parallel region is delimited by curly braces
({ }) and immediately follows the parallel pragma and its modifiers.

The syntax for this pragma is:

#pragma parallel shared (variables) byvalue (variables)
#pragma local (variables) optional modifiers
{ code }

The parallel pragma has six modifiers: shared, byvalue, local, if, ifinline, and numthreads.

Their syntax is:

shared ( variable names )
byvalue ( variable names )
local ( variable names )
if ( integer valued expr )
[no]ifinline
numthreads ( expr )
numthreads (percent=expr)
numthreads (expr)

Where:

shared		Tells the multiprocessing C compiler the names of all the variables that the threads must share. (If PCA creates a parallel region, it does this for you.)
byvalue		Puts a variable in the variable names list after this option to tell the multiprocessing C compiler that it can pass those shared variables as values rather than by reference. This fine-tuning option helps the multiprocessing C compiler optimize code. PCA will generate this variable as appropriate. However, used incorrectly, this option can generate erroneous code. Be careful what you put in this variable list or you may generate incorrect code. You can put a variable in this list only if the variable is: a scalar not already in the shared list read only
local		Tells the multiprocessing C compiler the names of all the variables that must be private to each thread. (When PCA sets up a parallel region, it does this for you.)
if		Lets you set up a condition that is evaluated at run time to determine whether or not to run the statement(s) serially or in parallel. At compile time, it is not always possible to judge how much work a parallel region does (for example, loop indices are often calculated from data supplied at run time). Avoid running trivial amounts of code in parallel because you cannot make up the overhead associated with running code in parallel. PCA will also generate this condition as appropriate. If the if condition is false (equal to zero), then the statement(s) runs serially. Otherwise, the statement(s) run in parallel.
[no]ifinline		Helps the multiprocessing C compiler optimize code when you also use the if option. This option is a fine-tuning option. Using the ifinline option (which is the default unless you use noifinline) causes a slight increase in code size but faster execution. This feature is turned off if you use noifinline (that is, the code is smaller but a little slower).
numthreads		(min=expr; max=expr)
numthreads		(percent=expr)
numthreads		(expr) Tells the multiprocessing C compiler the number of available threads to use when running this region in parallel. (The default is all the available threads.) The min clause instructs the compiler that this section is not to run in parallel unless at least expr threads are available. The max clause indicates that at most expr threads out of the available threads should be used. The actual number used is the smaller of expr and the number of threads available. The percent clause instructs the compiler to use expr percent of the available threads. In general, you should never have more threads of execution than you have processors, and you should specify numthreads with the MPC_NUM_THREADS environmental variable at run time (see Appendix C, “Run Time Environment Variables”). If you want to run a loop in parallel while you run some other code, you can use this option to tell the multiprocessing C compiler to use only some of the available threads. The usage #pragma numthreads (expr) is equivalent to #pragma numthreads (max=expr). expr should evaluate to a positive integer.

For example, to start a parallel region in which to run the following code in parallel:

for (idx=n; idx; idx--) { a[idx] = b[idx] + c[idx]; }

you or PCA must enter:

#pragma parallel shared( a, b, c ) byvalue(n) local( idx )

or:

#pragma parallel #pragma shared( a, b, c ) #pragma byvalue(n) #pragma local(idx)

before the statement or compound statement (code in curly braces, { }) that comprises the parallel region.

Any code within a parallel region but not within any of the explicit parallel constructs (pfor, independent, one processor, and critical) is termed local code. Local code typically modifies only local data and is run by all threads.

Figure 5-2 shows local code execution.

Figure 5-2. Execution of Local Code Segments

Use #pragma pfor to run a for loop in parallel only if the loop meets all of these conditions:

If the code after a pfor is not dependent on the calculations made in the pfor loop, there is no reason to synchronize the threads of execution before they continue. So, if one thread from the pfor finishes early, it can go on to execute the serial code without waiting for the other threads to finish their part of the loop.

The #pragma pfor directive takes several modifiers; the only one that is required is iterate. Figure 5-3 shows #pragma parallel, which starts a parallel region and tells the multiprocessing C compiler that the i variable must be local (private) to each processor. #pragma pfor tells the compiler that each iteration of the loop is unique and to partition the iterations among the threads for execution.

The syntax for #pragma pfor is:

#pragma pfor iterate ( ) optional modifiers
for ...
  { code ... }

The pfor pragma has three modifiers. Their syntax is:

iterate( index variable=expr1; expr2; expr3 )
schedtype ( type )
chunksize  ( expr )

Figure 5-3 shows parallel code segments using #pragma pfor.

Figure 5-3. Parallel Code Segments Using #pragma pfor

Where:

iterate		Gives the multiprocessing C compiler the information it needs to identify the unique iterations of the loop and partition them to particular threads of execution. index variable is the index variable of the for loop you want to run in parallel. expr1 is the starting value for the loop index. expr2 is the number of iterations for the loop you want to run in parallel. expr3 is the increment of the for loop you want to run in parallel. For example, for the for loop for (idx=n; idx; idx--) { a[idx] = b[idx] + c[idx]; } the iterate modifier to pfor should be: iterate(idx=n;n;-1) This loop counts down from the value of n, so the starting value is the current value of n. The number of trips through the loop is n, and the increment is -1.
schedtype (type)		Tells the multiprocessing C compiler how to share the loop iterations among the processors. The schedtype chosen depends on the type of system you are using and the number of programs executing (see Table 5-2). Table 5-2. Choosing a schedtype Single-User System * Multiuser System simple (iterations take same amount of time) gss (data-sensitive iterations vary slightly) gss (data-sensitive iterations vary slightly) dynamic (data-sensitive iterations vary greatly) dynamic (data-sensitive iterations vary greatly)   * If you are on a single-user system but are executing multiple programs, select the scheduling from the Multiuser column.

Figure 5-4 shows how loop iterations can vary.

Figure 5-4. Variance of Loop Iterations

You can use the following valid types to modify schedtype:

simple		(the default) tells the run time scheduler to partition the iterations evenly among all the available threads.
runtime		tells the compiler that the real schedule type will be specified at run time.
dynamic		tells the run time scheduler to give each thread chunksize iterations of the loop. chunksize should be smaller than (number of total iterations)/(number of threads). The advantage of dynamic over simple is that dynamic helps distribute the work more evenly than simple. Depending on the data, some iterations of a loop can take longer to compute than others, so some threads may finish long before the others. In this situation, if the iterations are distributed by simple, then the thread waits for the others. But if the iterations are distributed by dynamic, the thread does not wait, but goes back to get another chunksize iteration until the threads of execution have run all the iterations of the loop.
interleave		tells the run time scheduler to give each thread chunksize iterations (described below) of the loop, which are then assigned to the threads in an interleaved way.
gss		(guided self-scheduling) tells the run time scheduler to give each processor a varied number of iterations of the loop. This is like dynamic, but instead of a fixed chunksize, the chunk size iterations begin with big pieces and end with small pieces. If I iterations remain and P threads are working on them, the piece size is roughly: I/(2P) + 1 Programs with triangular matrices should use gss.

Figure 5-5 shows the effects of the different types of loop scheduling.

Figure 5-5. Loop Scheduling Types

chunksize (expr)

Tells the multiprocessing C compiler how many iterations to define as a chunk when you use the dynamic or interleave modifier (described above). expr should be positive integer, and should evaluate to the following formula: number of iterations -------------------- X where X \xb2 2* - 10* the number of threads. Select 2* the number of threads when iterations vary slightly and 10* the number of threads when iterations vary greatly. Performance gain may diminish after 10*.

To run the example:

for (idx=n; idx; idx--){ a[idx] = b[idx] + c[idx]; }

in parallel, PCA or you must enter the pragmas:

#pragma parallel #pragma shared( a, b, c ) #pragma byvalue(n) #pragma local(idx) #pragma pfor iterate(idx=n;n;-1) for (idx=n; idx; idx--){ a[idx] = b[idx] + c[idx]; }

A #pragma one processor directive causes the statement that follows it to be executed by exactly one thread.

The syntax of this pragma is:

#pragma one processor
{ code }

Figure 5-6 shows code executed by only one thread. No thread may proceed past this code until it has been executed.

Figure 5-6. One Processor Segment

If a thread is executing the statement following this pragma, then other threads that encounter this statement must wait until the statement has been executed by the first thread, then skip the statement and continue on.

If a thread has completed execution of the statement preceded by this pragma, then all threads encountering this statement skip the statement and continue without pause.

Sometimes the bulk of the work done by a loop can be done in parallel, but the entire loop cannot run in parallel because of a single data-dependent statement. Often, you can move such a statement out of the parallel region. When that is not possible, you can sometimes use a lock on the statement to preserve the integrity of the data.

In Power C, use the critical pragma to put a lock on a critical statement (or compound statement using { }). When you put a lock on a statement, only one thread at a time can execute that statement. If one thread is already working on a critical protected statement, any other thread that wants to execute that statement must wait until the other thread has finished executing it. Figure 5-7 shows critical segment execution.

---

Note: The current release of Power C allocates one global lock that is shared among all #pragma critical directives by default. Most uses of the critical pragma are to protect access to a very limited set of data, data that is usually referenced in many places in the program. By sharing one lock, all references by all guarded statements are properly protected. See “The lock Clause” for more information.

---

Figure 5-7. Critical Segment Execution

The syntax of the critical pragma is:

#pragma critical
{ code }

The statement(s) after the critical pragma will be executed by all threads, but only by one at a time.

Running a loop in parallel is a class of parallelism sometimes called “fine-grained parallelism” or “homogeneous parallelism.” It is called homogeneous because all the threads execute the same code on different data. Another class of parallelism is called “coarse-grained parallelism” or “heterogeneous parallelism.” As the name suggests, the code in each thread of execution is different.

Ensuring data independence for heterogeneous code executed in parallel is not always as easy as it is for homogeneous code executed in parallel. (And assuring data independence for homogeneous code is not a trivial task.)

The independent pragma has no modifiers. Use this pragma to tell the multiprocessing C compiler to run code in parallel with the rest of the code in the parallel region. Figure 5-8 shows an independent segment with execution by only one thread. However, other threads may proceed past this code as soon as it starts execution.

Figure 5-8. Independent Segment Execution

The syntax for #pragma independent is:

#pragma independent
{ code }

---

Note: The Power C Analyzer does not yet know how to generate this new pragma. Do not include it in code that you intend to pass through PCA. Insert this pragma only after you are finished with PCA.

---

A #pragma synchronize tells the multiprocessing C compiler that within a parallel region, no thread can execute the statements that follows this pragma until all threads have reached it. This directive is a classic barrier construct. Figure 5-9 shows this synchronization.

Figure 5-9. Synchronization

The syntax for this pragma is:

#pragma synchronize

You can use two additional pragmas to coordinate the processing of code within a parallel region. These additional pragmas work as a matched set. They are #pragma enter gate and #pragma exit gate.

A gate is a special barrier. No thread may exit the gate until all threads have entered it. Figure 5-10 shows execution using gates.

Figure 5-10. Execution Using Gates

This construct gives you more flexibility when managing dependencies between the work-sharing constructs within a parallel region.

For example, suppose you have a parallel region consisting of the work-sharing constructs A, B, C, D, E, and so forth. A dependency might exist between B and E such that you could not execute E until all the work on B was completed, as shown below:

#pragma parallel ...
{
..A..
..B..
..C..
..D..
..E.. (depends on B)
}

One way to handle this would be to put a synchronize before E. But this directive is wasteful if all the threads have cleared B and are already in C or D. All the faster threads would pause before E until the slowest thread completed C and D:

#pragma parallel ...
{
..A..
..B..
..C..
..D..
#pragma synchronize
..E..
}

To reflect this dependency, put a #pragma enter gate (name) after B and a #pragma exit gate (name) before E. Putting the enter gate after B tells the system to note which threads have completed the B work-sharing construct. Putting the exit gate pragma prior to the E work sharing construct tells the system to allow no thread into E until all threads have cleared B.

#pragma parallel ...
{
..A..
..B..
#pragma enter gate (foo)
..C..
..D..
#pragma exit gate (foo)
..E..
}

#pragma enter gate

The syntax of this pragma is:

#pragma enter gate ( name )

name

is a name you create to uniquely identify the work construct controlled by this pragma.

For example, construct D might be dependent on construct A, and construct F might be dependent on construct B, but you would not want to stop at construct D because all the threads had not cleared B. By using enter/exit gate pairs, you can make subtle distinctions about which construct is dependent on which other construct.

Put this pragma after the work-sharing construct that all threads must clear before the #pragma exit gate of the same name.

---

Note: The Power C Analyzer does not yet know how to generate this new pragma. Do not include it in code that you intend to pass through PCA. Insert this pragma only after you are finished with PCA.

---

#pragma exit gate

The syntax of this pragma is:

#pragma exit gate( name )

Put this pragma before the work-sharing construct that is dependent on the #pragma enter gate of the same name. No thread enters this work-sharing construct until all threads have cleared the work-sharing construct controlled by the corresponding #pragma enter gate of the same name.

---

Note: The Power C Analyzer does not yet know how to generate this new pragma. Do not include it in code that you intend to pass through PCA. Insert this pragma only after you are finished with PCA.

---