Appendix A. Sample Programs
Prev |  | Next

## Appendix A. Sample Programs

A number of example programs are distributed with the REACT/Pro Frame Scheduler. This section describes them. Only one is reproduced here (see “The simple_pt Pthreads Program”; the others are found on disk).

The source for the example programs distributed with the Frame Scheduler are found in the directory `/usr/share/src/react/examples` and the executables are in `/usr/react/bin`. They are summarized in Table A-1 and are discussed in more detail in the topics that follow.

Table A-1. Summary of Frame Scheduler Example Programs

Directory | Features of Example
--- | ---
`simple simple_pt r4k_intr` | `simple` shows two processes and `simple_pt` shows two threads scheduled on a single CPU 0 at a frame rate slow enough to permit use of `printf`() for debugging. The examples differ in the time base used; and the `r4k_intr` code uses a barrier for synchronization.
`` | Like `simple`, but the scheduled processes are independent programs.
`0"multi multi_pt ext_intr user_intr vsync_intr` | Three synchronous Frame Schedulers running lightweight processes (or pthreads in `multi_pt`) on three processors. These examples are much alike, differing mainly in the source of the time base interrupt.
`complete stop_resume` | Like `multi` in starting three Frame Schedulers. Information about the activity processes is stored in arrays for convenient maintenance. The `stop_resume` code demonstrates `frs_stop`() and `frs_resume`() calls.
`driver dintr` | `driver` contains a pseudo-device driver that demonstrates the Frame Scheduler device driver interface. `dintr` contains a program based on `simple` that uses the example driver as a time base.
`sixtyhz` `memlock` | One process scheduled at a 60 Hz frame rate. The activity process in the `memlock` example locks its address space into memory before it joins the scheduler.
`upreuse` | Complex example that demonstrates the creation of a pool of reusable processes, and how they can be dispatched as activity processes on a Frame Scheduler.

## Basic Example

The example in `/usr/react/src/examples/simple` shows how to create a simple application using the Frame Scheduler API. The code in `/usr/react/src/examples/r4kintr` is similar.

### Real-Time Application Specification

The application consists of two processes that have to periodically execute a specific sequence of code. The period for the first process, process A, is 600 milliseconds. The period for the other process, process B, is 2400 ms.

| Note: Such long periods are unrealistic for real-time applications. However, they allow the use of `printf`() calls within the “real-time” loops in this sample program.

### Frame Scheduler Design

The two periods and their ratio determine the selection of the minor frame period—600 ms—and the number of minor frames per major frame—4, for a total of 2400 ms.

The discipline for process A is strict real-time (`FRS_DISC_RT`). Underrun and overrun errors should cause signals.

Process B should run only once in 2400 ms, so it operates as Continuable over as many as 4 minor frames. For the first 3 frames, its discipline is Overrunnable and Continuable. For the last frame it is strict real-time. The Overrunnable discipline allows process B to run without yielding past the end of each minor frame. The Continuable discipline ensures that once process B does yield, it is not resumed until the fourth minor frame has passed. The combination allows process B to extend its execution to the allowable period of 2400 ms, and the strict real-time discipline at the end makes certain that it yields by the end of the major frame.

There is a single Frame Scheduler so a single processor is used by both processes. Process A runs within a minor frame until yielding or until the expiration of the minor frame period. In the latter case the frame scheduler generates an overrun error signaling that process A is misbehaving.

When process A yields, the frame scheduler immediately activates process B. It runs until yielding, or until the end of the minor frame at which point it is preempted. This is not an error since process B is Overrunable.

Starting the next minor frame, the Frame Scheduler allows process A to execute again. After it yields, process B is allowed to resume running, if it has not yet yielded. Again in the third and fourth minor frame, A is started, followed by B if it has not yet yielded. At the interrupt that signals the end of the fourth frame (and the end of the major frame), process B must have yielded, or an overrun error is signalled.

## Example of Scheduling Separate Programs

The code in directory `/usr/react/src/examples/mprogs` does the same work as example `simple` (see “Basic Example”). However, the activity processes A and B are physically loaded as separate commands. The main program establishes the single Frame Scheduler. The activity processes are started as separate programs. They communicate with the main program using SVR4-compatible interprocess communication messages (see the `intro`(2) and `msgget`(2) man pages).

There are three separate executables in the `mprogs` example. The master program, in `master.c`, is a command that has the following syntax:

    master [-p cpu-number] [-s slave-count]

The cpu-number specifies which processor to use for the one Frame Scheduler this program creates. The default is processor 1. The slave-count tells the master how many subordinate programs will be enqueued to the Frame Scheduler. The default is two programs.

The problems that need to be solved in this example are as follows:

- The FRS master program must enqueue the activity processes. However, since they are started as separate programs, the master has no direct way of knowing their process IDs, which are needed for `frs_enqueue`().

- The activity processes need to specify upon which minor frames they should be enqueued, and with what discipline.

- The master needs to enqueue the activities in the proper order on their minor frames, so they will be dispatched in the proper sequence. Therefore the master has to distinguish the subordinates in some way; it cannot treat them as interchangeable.

- The activity processes must join the Frame Scheduler, so they need the handle of the Frame Scheduler to use as an argument to `frs_join`(). However, this information is in the master's address space.

- If an error occurs when enqueueing, the master needs to tell the activity processes so they can terminate in an orderly way.

There are many ways in which these objectives could be met (for example, the three programs could share a shared-memory arena). In this example, the master and subordinates communicate using a simple protocol of messages exchanged using `msgget`() and `msgput`() (see the `msgget`(2) and `msgput`(2) man pages). The sequence of operations is as follows:

1. The master program creates a Frame Scheduler.

2. The master sends a message inviting the most important subordinate to reply. (All the message queue handling is in module `ipc.c`, which is linked by all three programs.)

3. The subordinate compiled from the file `processA.c` replies to this message, sending its process ID and requesting the FRS handle.

4. The subordinate process A sends a series of messages, one for each minor queue on which it should enqueue. The master enqueues it as requested.

5. The subordinate process A sends a “ready” message.

6. The master sends a message inviting the next most important process to reply.

7. The program compiled from `processB.c` will reply to this request, and steps 3-6 are repeated for as many slaves as the `slave-count` parameter to the master program. (Only two slaves are provided. However, you can easily create more using `processB.c` as a pattern.)

8. The master issues `frs_start`(), and waits for the termination signal.

9. The subordinates independently issue `frs_join`() and the real-time dispatching begins.

## Examples of Multiple Synchronized Schedulers

The example in `/usr/react/src/examples/multi` demonstrates the creation of three synchronized Frame Schedulers. The three use the cycle counter to establish a minor frame interval of 50 ms. All three Frame Schedulers use 20 minor frames per major frame, for a major frame rate of 1 Hz.

The following processes are scheduled in this example:

- Processes A and D require a frequency of 20 Hz

- Process B requires a frequency of 10 Hz and can consume up to 100 ms of execution time each time

- Process C requires a frequency of 5 Hz and can consume up to 200 ms of execution time each time

- Process E requires a frequency of 4 Hz and can consume up to 250 ms of execution time each time

- Process F requires a frequency of 2 Hz and can consume up to 500 ms of execution time each time

- Processes K1, K2 and K3 are background processes that should run as often as possible, when time is available.

The processes are assigned to processors as follows:

- Scheduler 1 runs processes A (20 Hz) and K1 (background).

- Scheduler 2 runs processes B (10 Hz), C (5 Hz), and K2 (background).

- Scheduler 3 runs processes D (20Hz), E (4 Hz), F (2 Hz), and K3.

In order to simplify the coding of the example, all real-time processes use the same function body, `process_skeleton()`, which is parameterized with the process name, the address of the Frame Scheduler it is to join, and the address of the “real-time” action it is to execute. In the sample code, all real-time actions are empty function bodies (feel free to load them down with code).

The examples in `/usr/react/src/examples/ext_intr`, `user_intr`, and `vsync_intr` are all similar to multi, differing mainly in the time base used. The examples in `complete` and `stop_resume` are similar in operation, but more evolved and complex in the way they manage subprocesses.

| Tip: It is helpful to use the `xdiff` program when comparing these similar programs—see the `xdiff`(1) man page.

### Example of Device Driver

The code in `/usr/react/src/examples/driver` contains a skeletal test-bed for a kernel-level device driver that interacts with the Frame Scheduler. Most of the driver functions consist of minimal or empty stubs. However, the `ioctl`() entry point to the driver (see the `ioctl`(2) man page) simulates a hardware interrupt and calls the Frame Scheduler entry point, `frs_handle_driverintr`() (see “Generating Interrupts” in Chapter 4). This allows you to test the driver. Calling its `ioctl`() entry is equivalent to using `frs_usrintr`() (see “The Frame Scheduler API” in Chapter 4).

The code in `/usr/react/src/examples/dintr` contains a variant of the simple example that uses a device driver as the time base. The program `dintr/sendintr.c` opens the driver, calls `ioctl`() to send one time-base interrupt, and closes the driver. (It could easily be extended to send a specified number of interrupts, or to send an interrupt each time the return key is pressed.)

### Examples of a 60 Hz Frame Rate

The example in directory `/usr/react/src/examples/sixtyhz` demonstrates the ability to schedule a process at a frame rate of 60 Hz, a common rate in visual simulators. A single Frame Scheduler is created. It uses the cycle counter with an interval of 16,666 microseconds (16.66 ms, approximately 60 Hz). There is one minor frame per major frame.

One real-time process is enqueued to the Frame Scheduler. By changing the compiler constant `LOGLOOPS` you can change the amount of work it attempts to do in each frame.

This example also contains the code to query and to change the signal numbers used by the Frame Scheduler.

The example in `/usr/react/src/examples/memlock` is similar to the `sixtyhz` example, but the activity process uses `plock`() to lock its address space. Also, it executes one major frame's worth of `frs_yield`() calls immediately after return from `frs_join`(). The purpose of this is to “warm up” the processor cache with copies of the process code and data. (An actual application process could access its major data structures prior to this yield in order to speed up the caching process.)

### Example of Managing Lightweight Processes

The code in `/usr/react/src/examples/upreuse` implements a simulated real-time application based on a pool of reusable processes. A reusable process is created with `sproc`() and described by a `pdesc_t` structure. Code in `pqueue.c` builds and maintains a pool of processes. Code in `pdesc.c` provides functions to get and release a process, and to dispatch one to execute a specific function.

The code in `test_hello.c` creates a pool of processes and dispatches each one in turn to display a message. The code in `test_singlefrs.c` creates a pool of processes and causes them to join a Frame Scheduler.

### The simple_pt Pthreads Program

This section is a variation of the `simple` program, implemented using the pthreads programming model.

    #include <math.h>
    #include <stdio.h>
    #include <signal.h>
    #include <semaphore.h>
    #include <pthread.h>
    #include <sys/schedctl.h>
    #include <sys/sysmp.h>
    #include <sys/frs.h>
    /*
     * frs_abort: If a pthread calls exit, then all pthreads within the process
     *            will be terminated and the FRS will be destroyed.
     *
     *            For some failure conditions, this is the desired behavior.
     */
    #define frs_abort(x)    exit(x)
    sem_t sem_threads_enqueued;
    pthread_attr_t pthread_attributes;
    int cpu_number = 1;
    /*
     * Some fixed real-time loop parameters
     */
    #define NLOOPS_A 20
    #define NLOOPS_B 15
    #define LOGLOOPS_A 150
    #define LOGLOOPS_B 30000
    void Thread_Master(void);
    void Thread_A(frs_t* frs);
    void Thread_B(frs_t* frs);
    void setup_signals(void);
    /*
     * NOTE: The initial thread of a pthread application (i.e., the thread
     *       executing main) cannot be an FRS controller or an FRS scheduled
     *       activity thread.  This is because all FRS controller and activity
     *       threads must be system scope threads.  The initial thread, however,
     *       is process scope (see pthread_attr_setscope(3P)).
     *
     *       In this example, the initial thread simply performs some set-up
     *       tasks, launches the system scope Master Controller thread, and
     *       exits.
     */
    main(int argc, char** argv)
    {
            pthread_t pthread_id_master;
            int ret;
            /*
             * Usage: simple [cpu_number]
             */
            if (argc == 2)
                    cpu_number = atoi(argv[1]);

            /*
             * Initialize semaphore
             */
            if (sem_init(&sem_threads_enqueued, 1, 0)) {
                    perror("Main: sem_init failed");
                    frs_abort(1);
            }
            /*
             * Initialize signals to catch FRS termination
             * underrun, and overrun error signals
             */
            setup_signals();
            /*
             * Initialize system scope thread attributes
             */
            if (ret = pthread_attr_init(&pthread_attributes)) {
                    fprintf(stderr,
                            "Main: pthread_attr_init failed (%d)\n", ret);
                    frs_abort(1);
            }
            ret = pthread_attr_setscope(&pthread_attributes, PTHREAD_SCOPE_SYSTEM);
            if (ret) {
                    fprintf(stderr,
                            "Main: pthread_attr_setscope failed (%d)\n", ret);
                    frs_abort(1);
            }

            /*
             * Launch Master Controller Thread
             */
            ret = pthread_create(&pthread_id_master,
                                 &pthread_attributes,
                                 (void *(*)(void *)) Thread_Master,
                                 NULL);
            if (ret) {
                    fprintf(stderr,
                            "Main: pthread_create Thread Master failed (%d)\n", ret);
                    frs_abort(1);
            }
            /*
             * Once the Master Controller is launched, there is no need for
             * us to hang around.  So we might as well free-up our stack by
             * exiting via pthread_exit().
             *
             * NOTE: Exiting via exit() would be fatal, terminating the
             *       entire process.
             */
            pthread_exit(0);
    }
    void
    Thread_Master(void)
    {
            frs_t* frs;
            pthread_t pthread_id_a;
            pthread_t pthread_id_b;
            int minor;
            int disc;
            int ret;
            /*
             * Create the Frame Scheduler object:
             *
             *      cpu = cpu_number,
             *      interrupt source = CCTIMER
             *      number of minors = 4
             *      slave mask = 0, no slaves
             *      period = 600 [ms] == 600000 [microseconds]
             */
            frs = frs_create_master(cpu_number,
                                    FRS_INTRSOURCE_CCTIMER,
                                    600000,
                                    4,
                                    0);
            if (frs == NULL) {
                    perror("Master: frs_create_master failed");
                    pthread_exit(0);
            }
            /*
             * Thread A will be enqueued on all minor frame queues
             * with a strict RT discipline
             */
            ret = pthread_create(&pthread_id_a,
                                 &pthread_attributes,
                                 (void *(*)(void *)) Thread_A,
                                 (void*) frs);
            if (ret) {
                    fprintf(stderr,
                            "Master: pthread_create Thread A failed (%d)\n", ret);
                    pthread_exit(0);
            }
            for (minor = 0; minor < 4; minor++) {
                    ret = frs_pthread_enqueue(frs,
                                              pthread_id_a,
                                              minor,
                                              FRS_DISC_RT);
                    if (ret) {
                            perror("Master: frs_pthread_enqueue Thread A failed");
                            pthread_exit(0);
                    }
            }
            /*
             * Thread B will be enqueued on all minor frames, but the
             * disciplines will differ. We need continuability for the first
             * 3 frames, and absolute real-time for the last frame.
             */
            ret = pthread_create(&pthread_id_b,
                                 &pthread_attributes,
                                 (void *(*)(void *)) Thread_B,
                                 (void*) frs);
            if (ret) {
                    fprintf(stderr,
                            "Master: pthread_create Thread B failed (%d)\n", ret);
                    pthread_exit(0);
            }
            disc =  FRS_DISC_RT | FRS_DISC_UNDERRUNNABLE |
                    FRS_DISC_OVERRUNNABLE | FRS_DISC_CONT;
            for (minor = 0; minor < 3; minor++) {
                    ret = frs_pthread_enqueue(frs,
                                              pthread_id_b,
                                              minor,
                                              disc);
                    if (ret) {
                            perror("Master: frs_pthread_enqueue ThreadB failed");
                            pthread_exit(0);
                    }
            }
            ret = frs_pthread_enqueue(frs,
                                      pthread_id_b,
                                      3,
                                      FRS_DISC_RT | FRS_DISC_UNDERRUNNABLE);
            if (ret) {
                    perror("Master: frs_pthread_enqueue ThreadB failed");
                    pthread_exit(0);
            }
            /*
             * Give all FRS threads the go-ahead to join
             */
            if (sem_post(&sem_threads_enqueued)) {
                    perror("Master: sem_post failed");
                    pthread_exit(0);
            }
            if (sem_post(&sem_threads_enqueued)) {
                    perror("Master: sem_post failed");
                    pthread_exit(0);
            }

            /*
             * Now we are ready to start the frame scheduler
             */
            printf("Running Frame Scheduler on Processor [%d]\n", cpu_number);
            if (frs_start(frs) < 0) {
                    perror("Master: frs_start failed");
                    pthread_exit(0);
            }
            /*
             * Wait for FRS scheduled threads to complete
             */

            if (ret = pthread_join(pthread_id_a, 0)) {
                    fprintf(stderr,
                            "Master: pthread_join thread A (%d)\n", ret);
                    pthread_exit(0);
            }
            if (ret = pthread_join(pthread_id_b, 0)) {
                    fprintf(stderr,
                            "Master: pthread_join thread B (%d)\n", ret);
                    pthread_exit(0);
            }
            /*
             * Clean-up before exiting
             */
            (void) pthread_attr_destroy(&pthread_attributes);
            (void) sem_destroy(&sem_threads_enqueued);
            pthread_exit(0);
    }
    void
    Thread_A(frs_t* frs)
    {
            int counter;
            double res;
            int i;
            int previous_minor;
            pthread_t pthread_id = pthread_self();
            /*
             * Join to the frame scheduler once given the go-ahead
             */
            if (sem_wait(&sem_threads_enqueued)) {
                    perror("ThreadA: sem_wait failed");
                    frs_abort(1);
            }

            if (frs_join(frs) < 0) {
                    perror("ThreadA: frs_join failed");
                    frs_abort(1);
            }

            fprintf(stderr, "ThreadA (%x): Joined Frame Scheduler on cpu %d\n",
                    pthread_id, frs->frs_info.cpu);
            counter = NLOOPS_A;
            res = 2;
            /*
             * This is the real-time loop. The first iteration
             * is done right after returning from the join
             */

            do {
                    for (i = 0; i < LOGLOOPS_A; i++) {
                            res = res * log(res) - res * sqrt(res);
                    }
                    /*
                     * After we are done with our computations, we
                     * yield the cpu. The yield call will not return until
                     * it's our turn to execute again.
                     */
                    if ((previous_minor = frs_yield()) < 0) {
                            perror("ThreadA: frs_yield failed");
                            frs_abort(1);
                    }
                    fprintf(stderr,
                    "ThreadA (%x): Return from Yield; previous_minor: %d\n",
                            pthread_id, previous_minor);
            } while (counter--);
            fprintf(stderr, "ThreadA (%x): Exiting\n", pthread_id);
            pthread_exit(0);
    }

    void
    Thread_B(frs_t* frs)
    {
            int counter;
            double res;
            int i;
            int previous_minor;
            pthread_t pthread_id = pthread_self();
            /*
             * Join to the frame scheduler once given the go-ahead
             */
            if (sem_wait(&sem_threads_enqueued)) {
                    perror("ThreadB: sem_wait failed");
                    frs_abort(1);
            }

            if (frs_join(frs) < 0) {
                    perror("ThreadB: frs_join failed");
                    frs_abort(1);
            }

            fprintf(stderr, "ThreadB (%x): Joined Frame Scheduler on cpu %d\n",
                    pthread_id, frs->frs_info.cpu);
            counter = NLOOPS_B;
            res = 2;
            /*
             * This is the real-time loop. The first iteration
             * is done right after returning from the join
             */

            do {
                    for (i = 0; i < LOGLOOPS_B; i++) {
                            res = res * log(res) - res * sqrt(res);
                    }
                    /*
                     * After we are done with our computations, we
                     * yield the cpu. THe yield call will not return until
                     * it's our turn to execute again.
                     */
                    if ((previous_minor = frs_yield()) < 0) {
                            perror("ThreadB: frs_yield failed");
                            frs_abort(1);
                    }
                    fprintf(stderr,
                    "ThreadB (%x): Return from Yield; previous_minor: %d\n",
                            pthread_id, previous_minor);
            } while (counter--);
            fprintf(stderr, "ThreadB (%x): Exiting\n", pthread_id);
            pthread_exit(0);
    }
    /*
     * Error Signal handlers
     */
    void
    underrun_error()
    {
            if ((int)signal(SIGUSR1, underrun_error) == -1) {
                    perror("[underrun_error]: Error while resetting signal");
                    frs_abort(1);
            }
            fprintf(stderr, "[underrun_error], thread %x\n", pthread_self());
            frs_abort(2);
    }
    void
    overrun_error()
    {
            if ((int)signal(SIGUSR2, overrun_error) == -1) {
                    perror("[overrun_error]: Error while resetting signal");
                    frs_abort(1);
            }
            fprintf(stderr, "[overrun_error], thread %d\n", pthread_self());
            frs_abort(2);
    }
    void
    setup_signals()
    {
            if ((int)signal(SIGUSR1, underrun_error) == -1) {
                    perror("[setup_signals]: Error setting underrun_error signal");
                    frs_abort(1);
            }
            if ((int)signal(SIGUSR2, overrun_error) == -1) {
                    perror("[setup_signals]: Error setting overrun_error signal");
                    frs_abort(1);
            }
    }

Prev | Table of Contents | Next
Chapter 7. Managing User-Level Interrupts |  | Glossary