Chapter 2. Creating Video Programs With the Video Library

Video Library (VL) calls let you perform video teleconferencing, blend computer-generated graphics with frames from videotape or any video source, and output the input video source to the graphics monitor, to a video device such as a VCR, or both.

This chapter explains the basics of creating video programs for Indigo2 IMPACT Video:

The VL Programming Model

Syntax elements are as follows:

  • VL types and constants begin with uppercase VL; for example, VLServer

  • VL functions begin with lowercase vl; for example, vlOpenVideo()

Data transfers fall into two categories:

  • transfers involving memory (video to memory, memory to video), which require setting up a VL buffer

  • transfers not involving memory (such as video to screen and graphics to video), which do not require a VL buffer

For the two categories of data transfer, based on the VL programming model, the process of creating a VL application consists of these steps:

  1. Open a connection to the video daemon (vlOpenVideo()); if necessary, determine which device the application will use (vlGetDevice(), vlGetDeviceList()).

  2. Specify nodes on the data path (vlGetNode()).

  3. Create the path (vlCreatePath()).

  4. (Optional step) Add more connections to a path (vlAddNode()).

  5. Set up the hardware for the path (vlSetupPaths()).

  6. Specify path-related events to be captured (vlSelectEvents()).

  7. Set input and output parameters (controls) for the nodes on the path (vlSetControl()).

  8. For transfers involving memory, create a VL buffer to hold data for memory transfers (vlGetTransferSize(), vlCreateBuffer()).

  9. For transfers involving memory, register the buffer (vlRegisterBuffer()).

  10. Set the path topology (vlSetConnection()).

  11. Start the data transfer (vlBeginTransfer()).

  12. For transfers involving memory, get the data (vlGetNextValid() or vlGetLatestValid(), vlGetActiveRegion(), vlPutFree()) to manipulate frame data.

  13. Clean up (vlEndTransfer(), vlDeregisterBuffer(), vlDestroyPath(), vlDestroyBuffer(), vlCloseVideo()).

Table 2-1 lists calls explained in this chapter.

Table 2-1. Video Library Calls for Data Transfer

All Transfers

Transfers Involving Memory

Setting Controls

vlOpenVideo()
vlGetDevice()
vlGetDeviceList()
vlGetNode()
vlCreatePath()
vlSetConnection()
vlGetConnection()
vlAddNode()
vlRemoveNode()
vlSetupPaths()
vlSelectEvents()
vlBeginTransfer()
vlEndTransfer()
vlDestroyPath()
vlCloseVideo()

vlGetTransferSize()
vlCreateBuffer()
vlRegisterBuffer()
vlGetNextValid()
vlGetLatestValid()
vlPutValid()
vlGetNextFree()
vlGetActiveRegion()
vlPutFree()
vlGetDMediaInfo()
vlGetImageInfo()
vlDeregisterBuffer()
vlDestroyBuffer()

vlSetControl()
vlGetControl()
vlControlList()
vlGetControlInfo()


Performing Preliminary Steps

To run VL, you must

  • install the dmedia_dev option

  • link with libvl.so

  • include vl.h and dev_mgv.h

The client library is /usr/lib/libvl.so. The header files for the VL are in
/usr/include/dmedia. The header file for the VL, vl.h, contains the main definition of the VL API and controls. The header files for Indigo2 IMPACT Video are /usr/include/dmedia/dev_mgv.h (linked to /usr/include/vl/vl_mgv.h) and /usr/include/dmedia/dev_impact.h (linked to /usr/include/vl/vl_impact.h).


Note: When building a VL-based program, you must add -lvl to the linking command.


Opening a Connection to the Video Daemon

The first thing a VL application must do is open the device with vlOpenVideo(). Its function prototype is

VLServer vlOpenVideo(const char *sName) 

where sName is the name of the server to which to connect; set it to a NULL string for the local server. For example:

vlSvr = vlOpenVideo("")

Specifying Nodes on the Data Path

Use vlGetNode() to specify nodes; this call returns the node's handle. Its function prototype is

VLNode vlGetNode(VLServer vlSvr, int type, int kind, int number) 

where

VLNode 

is a handle for the node, used when setting controls or setting up paths

vlSvr 

names the server (as returned by vlOpenVideo())

type 

specifies the type of node:

  • VL_SRC: source

  • VL_DRN: drain

  • VL_INTERNAL: internal node, such as the blend node

  • VL_DEVICE: device for device-global controls


    Note: If you are using VL_DEVICE, the kind should be set to 0.


kind 

specifies the kind of node:


Note: Appendix C, “Indigo2 IMPACT Video Nodes and Their Controls,” gives full details of all Indigo2 IMPACT Video nodes.


number 

is the number of the node in cases of two or more identical nodes, such as two video source nodes

To discover which node the default is, use the control VL_DEFAULT_SOURCE after getting the node handle the normal way. The default video source is maintained by the VL. For example:

vlGetControl(vlSvr, path, VL_ANY, VL_DEFAULT_SOURCE, &ctrlval);
nodehandle = vlGetNode(vlSvr, VL_SRC, VL_VIDEO, ctrlval.intVal);

In the first line above, the last argument is a struct that retrieves the value. Corresponding to VL_DEFAULT_SOURCE, the control VL_DEFAULT_DRAIN gets the default VL_SRC node.

Creating and Setting Up the Data Path

Once nodes are specified, use VL calls to

  • create the path

  • get the device ID

  • add nodes (optional step)

  • set up the data path

  • specify the path-related events to be captured

Creating the Path

Use vlCreatePath() to create the data path. Its function prototype is

VLPath vlCreatePath(VLServer vlSvr, VLDev vlDev
    VLNode src, VLNode drn) 

This code fragment creates a path if the device is unknown:

if ((path = vlCreatePath(vlSvr, VL_ANY, src, drn)) < 0) {
    vlPerror(_progName);
    exit(1);
}

This code fragment creates a path that uses a device specified by parsing a devlist:

if ((path = vlCreatePath(vlSvr, devlist[devicenum].dev, src,
    drn)) < 0) {
    vlPerror(_progName);
    exit(1);
}


Note: If the path contains one or more invalid nodes, vlCreatePath() returns VLBadNode.


Getting the Device ID

If you specify VL_ANY as the device when you create the path, use vlGetDevice() to discover the device ID selected. Its function prototype is

VLDev vlGetDevice(VLServer vlSvr, VLPath path)

For example:

devicenum = vlGetDevice(vlSvr, path);
deviceName = devlist.devices[devicenum].name;
printf("Device is: %s/n", deviceName);

Adding a Node

For this optional step, use vlAddNode(). Its function prototype is

int vlAddNode(VLServer vlSvr, VLPath vlPath, VLNodeId node)

where

vlSvr 

names the server to which the path is connected

vlPath 

is the path as defined with vlCreatePath()

node 

is the node ID

This example fragment adds a source node and a blend node:

vlAddNode(vlSvr, vlPath, src_vid);
vlAddNode(vlSvr, vlPath, blend_node);

Setting Up the Data Path

Use vlSetupPaths() to set up the data path. Its function prototype is

int vlSetupPaths(VLServer vlSvr, VLPathList paths,
      u_int count, VLUsageType ctrlusage,
      VLUsageType streamusage) 

where

vlSvr 

names the server to which the path is connected

paths 

specifies a list of paths you are setting up

count 

specifies the number of paths in the path list

ctrlusage 

specifies usage for path controls:

  • VL_SHARE: other paths can set controls on this node; this control is the desired setting for other paths, including vcp, to work


    Note: When using VL_SHARE, pay attention to events. If another user has changed a control, a VLControlChanged event occurs.


  • VL_READ_ONLY: controls cannot be set, only read; for example, this control can be used to monitor controls

  • VL_LOCK: prevents other paths from setting controls on this path; controls cannot be used by another path

  • VL_DONE_USING: the resources are no longer required; the application releases this set of paths for other applications to acquire

streamusage 

specifies usage for the data:

  • VL_SHARE: transfers can be preempted by other users; paths contend for ownership


    Note: When using VL_SHARE, pay attention to events. If another user has taken over the node, a VLStreamPreempted event occurs.


  • VL_READ_ONLY: the path cannot perform transfers, but other resources are not locked; set this value to use the path for controls

  • VL_LOCK: prevents other paths that share data transfer resources with this path from transferring; existing paths that share resources with this path will be preempted

  • VL_DONE_USING: the resources are no longer required; the application releases this set of paths for other applications to acquire

This example fragment sets up a path with shared controls and a locked stream:

if (vlSetupPaths(vlSvr, (VLPathList)&path, 1, VL_SHARE,
    VL_LOCK) < 0)
{
    vlPerror(_progName);
    exit(1);
}


Note: The Video Library infers the connections on a path if vlBeginTransfer() is called and no drain nodes have been connected using vlSetConnection() (implicit routing). To specify a path that does not use the default connections, use vlSetConnection() (explicit routing). Chapter 5, “Managing Connections,” explains the use of this function and related requirements.The following rules are used in determining the connections:


  • For each internal node on the path, all unconnected input ports are connected to the first source node added to the path. Pixel ports are connected to pixel ports and alpha ports are connected to alpha ports.

  • For each drain node on the path, all unconnected input ports are connected to the first internal node placed on the path, if there is an internal node, or to the first source node placed on the path. Pixel ports are connected to pixel ports and alpha ports are connected to alpha ports.


Note: Do not combine implicit and explicit routing.


Specifying the Path-Related Events to Be Captured

Use vlSelectEvents() to specify the events you want to receive. Its function prototype is

int vlSelectEvents(VLServer vlSvr, VLPath path, VLEventMask eventmask)

where

vlSvr 

names the server to which the path is connected

path 

specifies the data path.

eventmask 

specifies the event mask; Table 2-2 lists the possibilities

Table 2-2 lists and describes the VL event masks

Table 2-2. VL Event Masks

Symbol

Meaning

VLStreamBusyMask

Stream is locked

VLStreamPreemptedMask

Stream was grabbed by another path

vlStreamChangedMask

Video routing on this path has been changed by another path

VLAdvanceMissedMask

Time was already reached

VLSyncLostMask

Irregular or interrupted signal

VLSequenceLostMask

Field or frame dropped

VLControlChangedMask

A control has changed

VLControlRangeChangedMask

A control range has changed

VLControlPreemptedMask

Control of a node has been preempted, typically by another user setting VL_LOCK on a path that was previously set with VL_SHARE

VLControlAvailableMask

Access is now available

VLTransferCompleteMask

Transfer of field or frame complete

VLTransferFailedMask

Error; transfer terminated; perform cleanup at this point, including vlEndTransfer()

VLEvenVerticalRetraceMask

Vertical retrace event, even field

VLOddVerticalRetraceMask

Vertical retrace event, odd field

VLFrameVerticalRetraceMask

Frame vertical retrace event

VLDeviceEventMask

Device-specific event, such as a trigger

VLDefaultSourceMask

Default source changed

 

For example:

vlSelectEvents(vlSvr, path, VLTransferCompleteMask); 

Event masks can be Or'ed; for example:

vlSelectEvents(vlSvr, path, VLTransferCompleteMask |
      VLTransferFailedMask);

Setting Parameters for Data Transfer to or From Memory

Transferring data to or from memory requires creating a VL buffer; its size is determined by the size of the frame data you are transferring. This section explains how to set node controls for data transfer.

To set frame data size and to convert from one video format to another, apply controls to the nodes. The use of source node controls and drain node controls is explained separately in this section.

Important data transfer controls for source and drain nodes are summarized in Table 2-3. They should be set in the order in which they appear in the table.

These controls are highly interdependent, so the order in which they are set is important. In most cases, the value being set takes precedence over other values that were previously set.


Note: For drain nodes, VL_PACKING must be set first. Note that changes in one parameter may change the values of other parameters set earlier; for example, clipped size may change if VL_PACKING is set after VL_SIZE.


Table 2-3. Data Transfer Controls

Control

Basic Use

Video Nodes

Memory Nodes

Screen Nodes

VL_FORMAT

Video format on the physical connector

See “Using VL_FORMAT” in this chapter

N/A

N/A

VL_TIMING

Video timing

See Table 2-4 for values

Not applicable

Not applicable

VL_CAP_TYPE

Setting type of field(s) or frame(s) to capture; see “Interlacing” in Appendix A

Not applicable

VL_CAPTURE_NONINTERLEAVED
VL_CAPTURE_INTERLEAVED
VL_CAPTURE_EVEN_FIELDS
VL_CAPTURE_ODD_FIELDS
VL-CAPTURE_FIELDS

Not applicable

VL_PACKING

Pixel packing (conversion) format

Not applicable

Changes pixel format of captured data; see Table 2-6 for values

Not applicable

VL_SIZE

Clipping size

Full size of video; read only

Clipped size

Clipped size

VL_OFFSET

Position within larger area

Position of active region

Offset relative to video offset

Pan within the video

VL_ORIGIN

Position within video

Not applicable

Not applicable

Screen position of first pixel displayed

VL_WINDOW

Setting window ID for video in a window

Not applicable

Not applicable

Window ID

VL_RATE

Field or frame transfer speed

Depends on capture type as specified by VL_CAP_TYPE

Not applicable

Not applicable

To determine default values, use vlGetControl() to query the values on the video source or drain node before setting controls. The initial offset of the video node is the first active line of video.

Similarly, the initial size value on the video source or drain node is the full size of active video being captured by the hardware, beginning at the default offset. Because some hardware can capture more than the size given by the video node, this value should be treated as a default size.

For all these controls, it pays to track return codes. If the value returned is VLValueOutOfRange, the value set is not what you requested.

To specify the controls, use vlSetControl(), for which the function prototype is

int vlSetControl(VLServer vlSvr, VLPath vlPath, VLNode node, 
      VLControlType type, VLControlValue * value) 

The use of VL_TIMING, VL_FORMAT, VL_PACKING, VL_SIZE, VL_OFFSET, VL_CAP_TYPE, and VL_RATE is explained in more detail in the following sections.

Using VL_TIMING

Timing type expresses the timing of video presented to a source or drain. Table 2-4 summarizes dimensions for VL_TIMING.

Table 2-4. Dimensions for Timing Choices


Timing

Maximum Width

Maximum Height

VL_TIMING_525_SQ_PIX (12.27 MHz)

640

486

VL_TIMING_625_SQ_PIX (14.75 MHz)

768

576

VL_TIMING_525_CCIR601 (13.50 MHz)

720

486

VL_TIMING_625_CCIR601 (13.50 MHz)

720

576


Using VL_FORMAT

To specify video input and output formats of the video signal on the physical connector, use VL_FORMAT.

Table 2-5. VL_FORMAT

Format

Explanation

Supported by Node

VL_FORMAT_DIGITAL
_COMPONENT_SERIAL

8- or 10-bit YCrCb

Single-link and dual-link

VL_FORMAT_SMPTE_YUV

Backwards compatibility: 8- or 10-bit YCrCb

Single-link and dual-link

VL_FORMAT_RAW_DATA

Arbitrary 8-bit data (non-video format)

Single-link only

VL_FORMAT_RGB

Full-range 8-bit (0-255) or 10-bit
(0-1023) RGBA

Dual-link only

VL_FORMAT_DIGITAL
_COMPONENT_RGB_SERIAL

RP175 standard RGBA

Dual-link only


Using VL_PACKING

A video packing describes how a video signal is stored in memory, in contrast with a video format, which describes the characteristics of the video signal. For example, the memory source nodes—CC1 and both VGI1 nodes—accept packed video from a VL buffer and output video in a given format.

Packings are specified through the VL_PACKING control on the memory nodes. This control also converts one video output format to another in memory, within the limits of the nodes.


Note: On dual-linked VGI1 memory nodes, only native packings are available; no conversions can be performed.

Packing types for eight bits per component are summarized in Table 2-6. In this table, the Native To column lists the nodes to which the packing is native; no software conversion is required, so these packings are fastest.

Table 2-6. Packing Types for Eight Bits per Component

Type

Native To

63-56

55-48

47-40

39-32

31-24

23-16

15-8

7-0

VL_PACKING_YVYU_422_8
YUV 4:2:2, single-link

All memory nodes

U0

Y0

V0

Y1

U2

Y2

V2

Y3

VL_PACKING_YUVA_4444_8
YUVA 4:4:4:4, dual-link

VGI1 memory nodes

A0

U0

Y0

V0

A1

U1

Y1

V1

VL_PACKING_AUYV_4444_8
AUYV 4:4:4:4, dual-link

VGI1 memory nodes

V0

Y0

U0

A0

V1

Y1

U1

A1

VL_PACKING_RGBA_8
RGBA, dual-link

VGI1 memory nodes

A0

B0

G0

R0

A1

B1

G1

R1

VL_PACKING_ABGR_8
ABGR, dual-link

VGI1 memory nodes

R0

G0

B0

A0

R1

G1

B1

A1

VL_PACKING_RGB_332_P
RGB, single-link
Each 8-bit pixel, Pn,
is shown as BBGGGRRR

None

P7

P6

P5

P4

P3

P2

P1

P0

VL_PACKING_Y_8_P
Grayscale (luminance only),
single-link

None

Y7

Y6

Y5

Y4

Y3

Y2

Y1

Y0

VL_PACKING_RGB_8
RGB, single-link
24-bit word, Xn are ignored

None

X0

B0

G0

R0

X1

B1

G1

R1

Packing types for ten bits per component are summarized in Table 2-7. The ten data bits are left-aligned within a 16-bit word. The hardware sets the lower six bits to zero before it writes them to memory. When reading from memory, the lower six bits are ignored. All are native to VGI1 memory nodes.

Table 2-7. Packing Types for Ten Bits per Component

Type

63-48

47-32

31-16

15-0

VL_PACKING_YVYU_422_10
YUV 4:2:2, single-link

[U0]000000

[Y0]000000

[V0]000000

[Y1]000000

VL_PACKING_YUVA_4444_10
YUVA 4:4:4:4, dual-link

[A0]000000

[U0]000000

[Y0]000000

[V0]000000

VL_PACKING_AUYV_4444_10
AUYV 4:4:4:4, dual-link

[V0]000000

[Y0]000000

[U0]000000

[A0]000000

VL_PACKING_RGBA_10
RGBA, dual-link

[A0]000000

[B0]000000

[G0]000000

[R0]000000

VL_PACKING_ABGR_10
ABGR, dual-link

[R0]000000

[G0]000000

[B0]000000

[A0]000000

In addition, Indigo2 IMPACT Video also supports dual-link AYUAYV, a packed format with three 10-bit components per 32-bit word, with the lowest two bits set to 0. It is native to VGI1 memory nodes. Bits are

  • 63-32: [U0][Y0][A0]00

  • 31-0: [V0][Y1][A1]00


Note: Other libraries may use different packing names.

For example:

VLControlValue val;

val.intVal = VL_PACKING_RGBA_10;
vlSetControl(vlSvr, path, memdrn, VL_PACKING, &val);

If the single-link packings VL_PACKING_RGB_332_P, VL_PACKING_RGB_8, and VL_PACKING_Y_8_P are requested of a memory drain node, the Video Library performs a software conversion to translate the data from a native packing and format. The application receives data in the requested packing and format, although the capture rate may be degraded.

Using VL_ZOOM

VL_ZOOM controls the expansion or decimation of the video image. Values greater than one expand the video; values less than one perform decimation. Figure 2-1 illustrates zooming and decimation.


Note: Indigo2 IMPACT Video screen drain nodes support the full range of VL_ZOOM (7/1, 6/1, 5/1, 4/1, 3/1, 2/1, 1/1, 1/2, 1/3, 1/4, 1/5, 1/6, 1/7, 1/8). Screen source nodes support 1/1 and 1/2. The texture node supports decimation only. The remaining nodes do not support zoom or decimation (the ratio 1/1 only).

Figure 2-1. Zoom and Decimation


VL_ZOOM takes a nonzero fraction as its argument; do not use negative values. For example, this fragment captures half-size decimation video to the screen:

val.fractVal.numerator = 1;
val.fractVal.denominator = 2;
if (vlSetControl(server, screen_path, screen_drain_node, VL_ZOOM, &val)){
   vlPerror("Unable to set zoom");
   exit(1);
}


Note: For a source, zooming takes place before blending; for a drain, blending takes place before zooming.

This fragment captures half-size decimation video to the screen, with clipping to 320 × 243 (NTSC size minus overscan):

val.fractVal.numerator = 1;
val.fractVal.denominator = 2;
if (vlSetControl(server,screen_path, screen_drain_node,
VL_ZOOM, &val))
{
    vlPerror("Unable to set zoom");
    exit(1);
}
val.xyVal.x = 320;
val.xyVal.y = 243;
if (vlSetControl(server, screen_path, screen_drain_node,
VL_SIZE, &val))
{
    vlPerror("Unable to set size");
    exit(1);
}

This fragment captures xsize × ysize video with as much decimation as possible, assuming the size is smaller than the video stream:

if (vlGetControl(server, screen_path, screen_source, VL_SIZE, &val))
{
   vlPerror("Unable to get size");
   exit(1);
}
if (val.xyVal.x/xsize < val.xyVal.y/ysize)
   zoom_denom = (val.xyVal.x + xsize - 1)/xsize;
else
   zoom_denom = (val.xyVal.y + ysize - 1)/ysize;
val.fractVal.numerator = 1;
val.fractVal.denominator = zoom_denom;
if (vlSetControl(server, screen_path, screen_drain_node, VL_ZOOM, &val))
{
   /* allow this error to fall through */
   vlPerror("Unable to set zoom");
}
val.xyVal.x = xsize;
val.xyVal.y = ysize;
if (vlSetControl(server, screen_path, screen_drain_node,
VL_SIZE, &val))
{
   vlPerror("Unable to set size");
   exit(1);
}

Using VL_SIZE

VL_SIZE controls how much of the image sent to the drain is used, that is, how much clipping takes place. This control operates on the zoomed image; for example, when the image is zoomed to half size, the limits on the size control change by a factor of 2. Figure 2-2 illustrates clipping.

Figure 2-2. Clipping an Image


For example, to display PAL video in a 320 × 243 space, clip the image to that size, as shown in the following fragment:

VLControlValue value;

value.xyval.x=320;
value.xyval.y=243;
vlSetControl(vlSvr, path, drn, VL_SIZE, &value); 


Note: Because this control is device-dependent and interacts with other controls, always check the error returns. For example, if offset is set before size and an error is returned, set size before offset.


Using VL_OFFSET

VL_OFFSET puts the upper left corner of the video data at a specific position; it sets the beginning position for the clipping performed by VL_SIZE. The values you enter are relative to the origin.

This example places the data ten pixels down and ten pixels in from the left:

VLControlValue value;

value.xyval.x=10; 
value.xyval.y=10; 
vlSetControl(vlSvr, path, drn, VL_OFFSET, &value); 

To capture the blanking region, set offset to a negative value.

Figure 2-3 shows the relationships between the source and drain size, offset, and origin.


Note: For memory nodes, VL_OFFSET and VL_SIZE in combination define the active region of video that is transferred to or from memory.

Figure 2-3. Zoom, Size, Offset, and Origin


Using VL_CAP_TYPE and VL_RATE

An application can request that Indigo2 IMPACT Video capture or play back a video stream in a number of ways. For example, the application can request that each field be placed in its own buffer, that each buffer contain an interleaved frame, or that only odd or even fields be captured. This section enumerates the capture types that Indigo2 IMPACT Video supports.

A field mask is useful for identifying which fields will be captured and played back and which fields will be dropped. A field mask is a bit mask of 60 bits for NTSC or 50 bits for PAL (two fields per frame). A numeral 1 in the mask indicates that a field is captured or played back, while a zero indicates that no action occurs.

For example, the following field mask indicates that every other field will be captured or played back:

10101010101010101010...

Capture types are as follows:

  • VL_CAPTURE_NONINTERLEAVED

  • VL_CAPTURE_INTERLEAVED (not used for texture node VL_TEX)

  • VL_CAPTURE_EVEN_FIELDS

  • VL_CAPTURE_ODD_FIELDS

  • VL_CAPTURE_FIELDS

VL_RATE determines the data transfer rate by field or frame, depending on the capture type as specified by VL_CAP_TYPE, as shown in Table 2-8.

Table 2-8. VL_RATE Values (Items per Second)

VL_CAP_TYPE Value

VL_RATE Value

VL_CAPTURE_NONINTERLEAVED, VL_CAPTURE_INTERLEAVED

NTSC: 1-30 frames/second
PAL: 1-25 frames/second

VL_CAPTURE_EVEN_FIELDS,
VL_CAPTURE_ODD_FIELDS

NTSC: 1-30 fields/second
PAL: 1-25 fields/second

VL_CAPTURE_FIELDS

NTSC: 1-60 fields/second
PAL: 1-50 fields/second



Note: Not all rates are supported on all memory nodes; see Appendix C, “Indigo2 IMPACT Video Nodes and Their Controls,” for details. The buffer size must be set in accordance with the capture type, as listed in Table 2-9 later in this chapter.


VL_CAPTURE_NONINTERLEAVED

The VL_CAPTURE_NONINTERLEAVED capture type specifies that frame-size units are captured noninterleaved. Each field is placed in its own buffer, with the dominant field in the first buffer. If one of the fields of a frame is dropped, all fields are dropped. Consequently, an application is guaranteed that the field order is maintained; no special synchronization is necessary to ensure that fields from different frames are mixed.

The rate (VL_RATE) for noninterleaved capture is in terms of fields and must be even. For NTSC, the capture rate may be from 2-60 fields per second, and for PAL 2-50 fields per second. Because a frame is always captured as a whole, a rate of 30 fields per second results in the following field mask:

1100110011001100...

The first bit in the field mask corresponds to the dominant field of a frame. Indigo2 IMPACT Video waits for a dominant field before it starts the transfer.

If VL_CAPTURE_NONINTERLEAVED is specified for playback, similar guarantees apply as for capture. If one field is lost during playback, it is not possible to “take back” the field. Indigo2 IMPACT Video resynchronizes on the next frame boundary, although black or “garbage” video might be present between the erring field and the frame boundary.

The rate during playback also follows the rules for capture. For each 1 in the mask above, a field from the VL buffer is output. During the 0 fields, the previous frame is repeated. Note that the previous frame is output, not just the last field. If there are a pair of buffers, the dominant field is placed in the first buffer.

VL_CAPTURE_INTERLEAVED

Interleaved capture interleaves the two fields of a frame and places them in a single buffer; the order of the frames depends on the value set for VL_MGV_DOMINANCE_FIELD (see Table C-5 or Table C-4 in Appendix C for details). Indigo2 IMPACT Video guarantees that the interleaved fields are from the same frame: if one field of a frame is dropped, then both are dropped.

The rate for interleaved frames is in frames per second: 1-30 frames per second for NTSC and 1-25 frames per second for PAL. A rate of 15 frames per second results in every other frame being captured. Expressed as a field mask, the following sequence is captured:

1100110011001100....

As with VL_CAPTURE_NONINTERLEAVED, Indigo2 IMPACT Video begins processing the field mask when a dominant field is encountered.

During playback, a frame is deinterleaved and output as two consecutive fields, with the dominant field output first. If one of the fields is lost, Indigo2 IMPACT Video resynchronizes to a frame boundary before playing the next frame. During the resynchronization period, black or “garbage” data may be displayed.

Rate control follows similar rules as for capture. For each 1 in the mask above, a field from the interleaved frame is output. During 0 periods, the previous frame is repeated.

This option is not applicable to the texture node VL_TEX.

VL_CAPTURE_EVEN_FIELDS

In the VL_CAPTURE_EVEN_FIELDS capture type, only even (F2) fields are captured, with each field placed in its own buffer. Expressed as a field mask, the captured fields are

1010101010101010...

Indigo2 IMPACT Video begins processing this field mask when an even field is encountered.

The rate for this capture type is expressed in even fields. For NTSC, the range is 1-30 fields per second, and for PAL 1-25 fields per second. A rate of 15 fields per second (NTSC) indicates that every other even field is captured, yielding a field mask of

1000100010001000...

During playback, the even field is repeated as both the F1 and F2 fields, until it is time to output the next buffer. If a field is lost during playback, black or “garbage” data might be displayed until the next buffer is scheduled to be displayed.

VL_CAPTURE_ODD_FIELDS

The VL_CAPTURE_ODD_FIELDS capture type works the same way as VL_CAPTURE_EVEN_FIELDS, except that only odd (F1) fields are captured, with each field placed in its own buffer. The rate for this capture type is expressed in odd fields. A rate of 15 fields per second (NTSC) indicates that every other odd field is captured. Field masks are the same as for VL_CAPTURE_EVEN_FIELDS.

VL_CAPTURE_FIELDS

The VL_CAPTURE_FIELDS capture type captures both even and odd fields and places each in its own buffer. Unlike VL_CAPTURE_NONINTERLEAVED, there is no guarantee that fields are dropped in frame units. Field synchronization can be performed by examining the UST, the MSC, or the dmedia info sequence number associated with each field.

The rate for this capture type is expressed in fields. For NTSC, the range is
1-60 fields per second, and for PAL 1-50 fields per second. A rate of 30 fields per second (NTSC) indicates that every other field is captured, resulting in the following field mask:

101010101010101010...

Contrast this with the rate of 30 for VL_CAPTURE_NONINTERLEAVED, which captures every other frame.

Field mask processing begins on the first field after the transfer is started; field dominance, evenness, oddness plays no role in this capture type.

Displaying Video Data Onscreen

To set up a window for live video, follow these steps, as outlined in the example program simplev2s.c.

  1. Open an X display window; for example:

    if (!(dpy = XOpenDisplay("")))
        exit(1);
    

  2. Connect to the video daemon; for example:

    if (!(vlSvr = vlOpenVideo("")))
         exit(1);
    

  3. Create a window to show the video; for example:

    vwin = XCreateSimpleWindow(dpy, RootWindow(dpy, 0), 10,
        10, 640, 486, 0,
        BlackPixel(dpy,DefaultScreen(dpy)),
        BlackPixel(dpy, DefaultScreen(dpy));
    XMapWindow(dpy, vwin);
    XFlush(dpy);
    

  4. Create a source node on a video device and a drain node on the screen; for example:

    src = vlGetNode(vlSvr, VL_SRC, VL_VIDEO, VL_ANY);
    drn = vlGetNode(vlSvr, VL_DRN, VL_SCREEN, VL_ANY);
    

  5. Create a path on the first device that supports it; for example:

    if ((path = vlCreatePath(vlSvr, VL_ANY, src, drn)) < 0)
        exit(1);
    

  6. Set up the hardware for the path and define the path use; for example:

    vlSetupPaths(vlSvr, (VLPathList)&path, 1, VL_SHARE,
        VL_SHARE); 
    

  7. Set the X window to be the drain; for example:

    val.intVal = vwin;
    vlSetControl(vlSvr, path, drn, VL_WINDOW, &val);
    

  8. Get X and VL into the same coordinate system; for example:

    XTranslateCoordinates(dpy, vwin, RootWindow(dpy,
        DefaultScreen(dpy)), 0, 0,&x, &y, &dummyWin);
    

  9. Set the live video to the same location and size as the window; for example:

    val.xyVal.x = x;
    val.xyVal.y = y;
    vlSetControl(vlSvr, path, drn, VL_ORIGIN, &val);
    

    XGetGeometry(dpy, vwin, &dummyWin, &x, &y, &w, &h, &bw,
       &d);
    val.xyVal.x = w;
    val.xyVal.y = h;
    vlSetControl(vlSvr, path, drn, VL_SIZE, &val);
    

  10. Begin the data transfer:

    vlBeginTransfer(vlSvr, path, 0, NULL);
    

  11. Wait until the user finishes; for example:

    printf("Press return to exit.\n");
    c = getc(stdin);
    

  12. End the data transfer, clean up, and exit:

    vlEndTransfer(vlSvr, path);
    vlDestroyPath(vlSvr, path);
    vlCloseVideo(vlSvr);
    

Transferring Video Data to and From Devices

The processes for data transfer are as follows:

  • creating a buffer for video data (for transfers involving memory)

  • registering the VL buffer with the path (for transfers involving memory)

  • starting data transfer

  • reading data from the buffer (for transfers involving memory)

Each process is explained separately.

Creating a Buffer for Video Data

Once you have specified frame parameters in a transfer involving memory (or have determined to use the defaults), create a buffer for the video data. In this case, video data is frames or fields, depending on the capture type:

  • frames if the capture type is VL_CAPTURE_NONINTERLEAVED

  • fields if the capture type is anything else

Like other libraries in the IRIX digital media development environment, the VL uses VL buffers. Vl buffers provide a way to read and write varying sizes of video data. A frame of data consists of the actual frame data and an information structure describing the underlying data, including device-specific information.

When a VL buffer is created, constraints are specified that control the total size of the data segment and the number of frame or field buffers (sectors) to allocate.

A head and a tail flag are automatically set in a VL buffer so that the latest frame can be accessed. A sector is locked down if it is not called; that is, it remains locked until it is read. When the VL buffer is written to and all sectors are occupied, data transfer stops. The sector last written to remains locked down until it is released.

All sectors in a VL buffer must be of the same size, which is the value returned by vlGetTransferSize(). Its function prototype is

long vlGetTransferSize(VLServer vlSvr, VLPath path)

For example:

transfersize = vlGetTransferSize(vlSvr, path); 

where transfersize is the size of the data in bytes.

To create a VL buffer for the frame data, use vlCreateBuffer(). Its function prototype is

VLBuffer vlCreateBuffer(VLServer vlSvr, VLPath path,
      VLNode node, int numFrames)

where

VLBuffer 

is the handle of the buffer to be created

vlSvr 

names the server to which the path is connected

path 

specifies the data path

node 

specifies the memory node containing data to transfer to or from the VL buffer

numFrames 

specifies the number of sectors in the buffer (fields or frames, depending on the capture type)

For example:

buf = vlCreateBuffer(vlSvr, path, src, 1); 

Table 2-9 shows the relationship between capture type and minimum VL buffer size.

Table 2-9. Buffer Size Requirements

Capture Type

Minimum Sectors for Capture

Minimum Sectors for Playback

VL_CAPTURE_NONINTERLEAVED

2

4

VL_CAPTURE_INTERLEAVED

1

2

VL_CAPTURE_EVEN_FIELDS

1

2

VL_CAPTURE_ODD_FIELDS

1

2

VL_CAPTURE_FIELDS

1

2



Note: For VGI1 memory nodes, real-time memory or video transfer can be performed only as long as buffer sectors are available to the Indigo2 IMPACT Video device.


Registering the VL Buffer

Use vlRegisterBuffer() to register the VL buffer with the data path. Its function prototype is

int vlRegisterBuffer(VLServer vlSvr, VLPath path,
     VLNode memnodeid, VLBuffer buffer)

where

vlSvr 

names the server to which the path is connected

path 

specifies the data path

memnodeid 

specifies the memory node ID

buffer 

specifies the VL buffer handle

For example:

vlRegisterBuffer(vlSvr, path, drn, Buffer);

Starting Data Transfer

To begin data transfer, use vlBeginTransfer(). Its function prototype is

int vlBeginTransfer(VLServer vlSvr, VLPath path, int count,
      VLTransferDescriptor* xferDesc) 

where

vlSvr 

names the server to which the path is connected

path 

specifies the data path

count 

specifies the number of transfer descriptors

xferDesc 

specifies an array of transfer descriptors

Tailor the data transfer by means of transfer descriptors. Multiple transfer descriptors are supplied; they are executed in order. The transfer descriptors are

xferDesc.mode 

Transfer method:

  • VL_TRANSFER_MODE_DISCRETE: a specified number of frames are transferred (burst mode)

  • VL_TRANSFER_MODE_CONTINUOUS (default): frames are transferred continuously, beginning immediately or after a trigger event occurs (such as a frame coincidence pulse), and continues until transfer is terminated with vlEndTransfer()

  • VL_TRANSFER_MODE_AUTOTRIGGER: frame transfer takes place each time a trigger event occurs; this mode is a repeating version of VL_TRANSFER_MODE_DISCRETE

xferDesc.count 

Number of frames to transfer; if mode is VL_TRANSFER_MODE_CONTINUOUS, this value is ignored.

xferDesc.delay 

Number of frames from the trigger at which data transfer begins.

xferDesc.trigger 

Set of events to trigger on; an event mask. This transfer descriptor is always required. VLTriggerImmediate specifies that transfer begins immediately, with no pause for a trigger event. VLDeviceEvent specifies an external trigger.

If xferDesc is NULL, then VL_TRIGGER_IMMEDIATE and VL_TRANSFER_CONTINOUS_MODE are assumed and one transfer is performed.

This example fragment transfers the entire contents of the buffer immediately.

xferDesc.mode = VL_TRANSFER_MODE_DISCRETE;

xferDesc.count = imageCount;
xferDesc.delay = 0;
xferDesc.trigger = VLTriggerImmediate;

This fragment shows the default descriptor, which is the same as passing in a null for the descriptor pointer. Transfer begins immediately; count is ignored.

xferDesc.mode = VL_TRANSFER_MODE_CONTINUOUS;

xferDesc.count = 0;
xferDesc.delay = 0;
xferDesc.trigger = VLTriggerImmediate;

Reading Data From the Buffer

If your application uses a buffer, use various VL calls for reading frames, getting pointers to active buffers, freeing buffers, and other operations. Table 2-10 lists the buffer-related calls.

Table 2-10. Buffer-Related Calls

Call

Purpose

vlGetNextValid()

Returns a handle on the next valid frame or field of data

vlGetLatestValid()

Reads only the most current frame or field in the buffer, discarding the rest

vlPutValid()

Puts a frame or field into the valid list (memory to video)

vlPutFree()

Puts a valid frame or field back into the free list (video to memory)

vlGetNextFree()

Gets a free buffer into which to write data (memory to video)

vlBufferDone()

Informs you if the buffer has been vacated

vlBufferReset()

Resets the buffer so that it can be used again

Figure 2-4 illustrates the difference between vlGetNextValid() and vlGetLatestValid(), and their interaction with vlPutFree().

Figure 2-4. vlGetNextValid(), vlGetLatestValid(), and vlPutFree()


Table 2-11 lists the calls that extract information from a buffer.

Table 2-11. Calls for Extracting Data From a Buffer

Call

Purpose

vlGetActiveRegion()

Gets a pointer to the data region of the buffer (video to memory); called after vlGetNextValid() and vlGetLatestValid()

vlGetDMediaInfo()

Gets a pointer to the DMediaInfo structure associated with a frame; this structure contains timestamp and field count information

vlGetImageInfo()

Gets a pointer to the DMImageInfo structure associated with a frame; this structure contains image size information



Caution: None of these calls has count or block arguments; appropriate calls in the application must deal with a NULL return in cases of no data being returned.

In summary, for video-to-memory transfer, use

buffer = vlCreateBuffer(vlSvr, path, memnode1);
vlRegisterBuffer(vlSvr, path, memnode1, buffer); 
vlBeginTransfer(vlSvr, path, 0, NULL); 
info = vlGetNextValid(vlSvr, buffer);
/* OR vlGetLatestValid(vlSvr, buffer); */
dataptr = vlGetActiveRegion(vlSvr, buffer, info); 

/* use data for application */
…
vlPutFree(vlSvr, buffer); 

For memory-to-video transfer, use

buffer = vlCreateBuffer(vlSvr, path, memnode1);
vlRegisterBuffer(vlSvr, path, memnode1, buffer); 
vlBeginTransfer(vlSvr, path, 0, NULL); 
buffer = vlGetNextFree(vlSvr, buffer, bufsize); 
/* fill buffer with data */
…
vlPutValid(vlSvr, buffer); 

These calls are explained in separate sections.

Reading the Frames to Memory From the Buffer

Use vlGetNextValid() to read all the frames in the buffer or get a valid frame of data. Its function prototype is

VLInfoPtr vlGetNextValid(VLServer vlSvr, VLBuffer vlBuffer)

Use vlGetLatestValid() to read only the most current frame in the buffer, discarding the rest. Its function prototype is

VLInfoPtr vlGetLatestValid(VLServer vlSvr, VLBuffer vlBuffer) 

After removing interesting data, return the buffer for use with vlPutFree() (video to memory). Its function prototype is

int vlPutFree(VLServer vlSvr, VLBuffer vlBuffer)

Sending Frames From Memory to Video

Use vlGetNextFree() to get a free buffer to which to write data. Its function prototype is

VLInfoPtr vlGetNextFree(VLServer vlSvr, VLBuffer vlBuffer,
      int size)

After filling the buffer with the data you want to send to video output, use vlPutValid() to put a frame into the valid list for output to video (memory to video). Its function prototype is

int vlPutValid(VLServer vlSvr, VLBuffer vlBuffer)


Caution: These calls do not have count or block arguments; appropriate calls in the application must deal with a NULL return in cases of no data being returned.


Getting DMediaInfo and Image Data From the Buffer

Use vlGetActiveRegion() to get a pointer to the active buffer. Its function prototype is

void * vlGetActiveRegion(VLServer vlSvr, VLBuffer vlBuffer,
     VLInfoPtr ptr)

Use vlGetDMediaInfo() to get a pointer to the DMediaInfo structure associated with a frame. This structure contains timestamp and field count information. The function prototype for this call is

DMediaInfo * vlGetDMediaInfo(VLServer vlSvr,
   VLBuffer vlBuffer, VLInfoPtr ptr)

Use vlGetImageInfo() to get a pointer to the DMImageInfo structure associated with a frame. This structure contains image size information. The function prototype for this call is

DMImageInfo * vlGetImageInfo(VLServer vlSvr,
   VLBuffer vlBuffer, VLInfoPtr ptr)

Ending Data Transfer

To end data transfer, use vlEndTransfer(). Its function prototype is

int vlEndTransfer(VLServer vlSvr, VLPath path) 

A discrete transfer is finished when the last frame of the sequence is output. The two types of memory nodes behave differently at the last frame:

  • The CC1 memory source stops transferring data from main memory to the Indigo2 IMPACT Video device, but continues to output to video the last frame transferred, which is held in a framebuffer associated with the CC1 memory node.

  • The VGI1 memory nodes have no associated framebuffer and consequently emit black video output after a transfer (discrete or continuous) has been completed.

To accomplish the necessary cleanup to exit gracefully, use the following functions:

  • for transfers involving memory: vlDeregisterBuffer(), vlDestroyPath(), vlDestroyBuffer()

  • for all transfers: vlCloseVideo()

The function prototype for vlDeregisterBuffer() is

int vlDeregisterBuffer(VLServer vlSvr, VLPath path,
    VLNode memnodeid, VLBuffer ringbufhandle) 

where

vlSvr 

is the server handle

path 

is the path handle

memnodeid 

is the memory node ID

ringbufhandle 

is the VL buffer handle

The function prototypes for vlDestroyPath(), vlDestroyBuffer() and vlCloseVideo() are, respectively,

int vlDestroyPath(VLServer vlSvr, VLPath path)
int vlDestroyBuffer(VLServer vlSvr, VLBuffer vlBuffer) 
int vlCloseVideo(VLServer vlSvr)

This example ends a data transfer that used a buffer:

vlEndTransfer(vlSvr, path);
vlDeregisterBuffer(vlSvr, path, memnodeid, buffer);
vlDestroyPath(vlSvr, path);
vlDestroyBuffer(vlSvr, buffer);
vlCloseVideo(vlSvr);

Example Programs

The directory /usr/people/4Dgifts/impact/examples/dmedia/video/vl includes a number of example programs. These programs illustrate how to create simple video applications; for example:

  • a simple screen application: simplev2s.c

    This program shows how to send live video to the screen.

  • a video-to-memory frame grab: simplegrab.c

    This program demonstrates video frame grabbing.

  • a memory-to-video frame output simplem2v.c

    This program sends a frame to the video output.

  • a continuous frame capture: simpleccapt.c

    This program demonstrates continuous frame capture.


Note: To simplify the code, these examples do not check returns. However, you should always check returns.

See Chapter 4 for a description of eventex.c and Chapter 7 for descriptions of simpleblend.c and simplewipe.c.

The directory /usr/people/4Dgifts/examples/dmedia/video/vl/OpenGL contains three example OpenGL programs:

  • contcapt.c: performs continuous capture using buffering and sproc

  • mtov.c: uses the Silicon Graphics Movie Library to play a movie on the selected video port

  • vidtomem.c: captures an incoming video stream to memory

Note that these programs differ from the programs with the same names in /usr/people/4Dgifts/impact/examples/dmedia/video/vl.