Introduction to the CM Toolkit


Ketan Patel

1.0 General Introduction

This document is a brief overview of the CM Toolkit from two different perspectives: the application programmer's view and the CM Toolkit developer's view. The application programmer's view is intended for people writing CM applications. The Toolkit developer's view is intended for people extending and building the CM Toolkit.

CMT is implementd in a combination of C, the Tool Command Language (Tcl), the Tcl X Toolkit (Tk), and the Tcl-DP distributed programming extension.

2.0 Application Programmer's View

2.1 The application process.

CMT applications use runtime Tcl libraries that provide procedures which allow connections to be made with CM processes. The location of these libraries should be appended to the application's auto load path. Currently, there are two executables that wll create a CM process: cmwish and cmsh. Cmwish is an extended Tcl/Tk wish process that included all the CM objects and facilities described below. Cmsh is simply a cmwish that does not have a main window and does not connect to an X server. A CM application can either run in a CM process of its own, providing access to objects created within its own process space, or in a process that contains a Tcl-DP enabled Tcl interpreter, utilizing the runtime libraries to make connections to remote CM processes. A simple application structure is shown graphically here

:

In the previous simple example, the user interface and local CM objects run in a CM process on host A, and other CM objects are running in a remote CM process on host B. The objects have been linked togethor and data is flowing from object to object indicated by the arrows. Two types of connections exist between the two CM processes: control and data. Control channels use RPC's implemented on TCP by Tcl-DP. Data channels use CM transport protocols. Current protocos include cyclic-UDP.

A more complicated example is shown next:

In this example, the application runs on host A and creates objects in CM processes on host A, B, and C. Objects in CM processes on host B and C are accessed through a CM process on host A.

2.2 Connecting to and controlling CM processes.

Assuming a application does not contain CM objects (i.e., a plain Tcl/Tk interpreter with Tcl-DP and nothing else), it needs to connect to CM processes to create and link together CM objects. Connecting to CM processes is done using the runtime Tcl library cmt_api.tcl. This library provides an object called a cmt object, the application issues the command:

cmt <parent_name>

where <parent_name> is an arbitrary string that is used as a prefix for the new cmt object's name. This command will return the actual cmt object name which will consist of the <parent_name> provided by the application appended with a unique identifier. For example, the command:

cmt "mycmt."

will return an object with, for example, the name:

mycmt.cmt1

Operations on CMT objects are implemented similar to the way operations are implemented on Tk widgets. An object creation command creaes a new Tcl command with a unique name that is used to specify operations on the object.

CM objects differ from Tcl/Tk widgets which allow the application to specify the command name in that the application has only partial control of the command name. The reason for this difference is that many applications may use the same CM process without knowledge of each other. Thus, names, that is object names, must be kept unique. All CM objects have this creation syntax.

2.2.2 Connecting to a CM process.


Once a cmt object has been created, a connection to a CM process can be opened using the cmt object command "open." This command has three variants. The first opens a connection to a specific machine at a specific port number. For example, using our newly created cmt object "mycmt.cmt1," a connection to the CM process running on the machine hostA at port number 1000 can be created with the following command:

mycmt.cmt1 open -host hostA 1000

Alternatively, a nameserver can be used to locate a particular service name and a connection can be opened to it using the second variant of the open command. For example, a connection to the CM service named "/hostA/cms" can be created with the command:

mycmt.cmt1 open -service /hostA/cms

When the "-service" option is given to the open command, the nameserver is contacted to provide the hostname and port number of the service. At this time the nameserver is NOT distributed with CMT and this variant should not be used.

Finally, a cmt object can create a connection its own CM process. This is done by using the "-local" option. For example:

mycmt.cmt1 open -local

Although CM objects can be created directly in the local process, an application may choose to create and control those objects through a cmt object connected locally to take advantage of certain object management and tagging facilities that are available only if the cmt object is used.

2.2.3 Creating CM objects through a CMT.


Once a connection to a CM process has been established, CM objects can be created with the cmt object command "create_object." This command takes the object type as an argument and returns the new object name. The prefix for the new object is automatically generated by the cmt object. For example, if we wanted to create a logical time system object, the following command is executed:

mycmt.cmt1 create_object lts

This command returns the name of the new lts object which will be something like:

mycmt.cmt1-1.lts0

The actual semantics of why this object name was chosen is not important to the application. In general, the application will assign this name to a local variable and use the local variable instead of the actual object name. This new object is now a command just like any other CM object. It has various configuration options and object commands specific to its type. In reality, however, all commands to the new object are actually sent through the control channel opened by the cmt object through which it was created. This fact is hidden from the application which uses the new object's name as it would any other CM object or Tk widget.

2.2.4 Object management.


The cmt object provides many ways to manage objects. Callbacks to be executed either remotely in the CM process or locally in the application process can be created to respond to events such as an unexpected closing of the connection. Proxie objects can be created which allow one CM process to know about objects actually created in another CM process. In addition, cmt objects can be hierarchical. One cmt object can be created to open a control channel from the application process to a CM process on host A and then another cmt object can be created through the first cmt object to open a control channel from the CM process on host A to a CM process on host B. These features and details are documented more fully in the cmt man page.

2.2.5 RPC and RDO through CMT objects.


Cmt objects also allow applications to issue RPC's and RDO's to the CM process. An RPC is a Remote Procedure Call while an RDO is an asynchronous RPC that does not wait for the procedure call's results. These two operations are analogous to dp_RPC and dp_RDO from the Tcl-DP package. For example, suppose the cmt object "mycmt.cmt1" has been created and has a control channel open to some remote CM process. The following command would return a file listing of the current working directory of the remote CM process:

mycmt.cmt1 rpc ls

More details on issuing RPC's and RDO's and the available options are given in the cmt object man page.

2.2.6 Closing connections.


A connection opened with a cmt object can be closed by issuing the command "close." If a connection is closed, all objects created through the connection are automatically destroyed. The same cmt object can be then used to create a new connection to a different CM process by using the open command again. The following example illustrates the close command:

# Open connection to hostA at port number 1000

mycmt.cmt1 open -host hostA 1000

# Create a remote lts object

set remote_lts [mycmt.cmt1 create_object lts]

# Close the connection.

mycmt.cmt1 close

# Once the connection has been closed, any object created

# through the connection (in this case, the remote lts object

# we created) is destroyed and no longer exists.

# We can open a new connection to some other host (or even

# the same host) with the same cmt object.

mycmt.cmt1 open -host hostB 1000

2.3 CM objects in general.

The discussion about the cmt object above brought to light several attributes that all CM objects share. All CM objects use the same creation, configuration, and destruction syntax.

2.3.1 Creation syntax.


All CM objects have the following creation syntax:

<object_type> <parent_name>

where <object_type> is the creation command for a new object of the desired type, and <parent_name> is used as a prefix for the new object. Note that <parent_name> can be the null string "" which results in an object name consisting of only the unique identifier assigned by the object creation command. The object name returned as a result is a command in and of itself that is used to configure and control the specific object created in the same way as a Tk widget.

Table 1. lists some of the objects supplied with the system. The naming convention we try to follow for objects is formatKind where format specifies the media format (e.e., MJPEG, MPEG, au, etc.) and Kind specifies the kind of object (e.g., File Segment, Play, Camera, etc.). The command to create a Motion JPEG clipfile segment object in the current process is:

mjpegSegment ""

The same command is executed in a separate CM process through a cmt object by executing:

$cmt create_object mjpegSegment

where $cmt is a variable that contains the name of the cmt object.

Objects provided by current CM Toolkit
Object TypeDescription
pktSrcSends data over the network using cyclic-UDP.
pktDestReceives data from the network using cyclic-UDP
udpSrcSends data over the network using plain UDP.
udpDestReceives data from the network using plain UDP
mjpegSegmentReads MJPEG data from a file.
mjpegPriorityPrioritizes MJPEG data to optimize playback smoothness.
mjpegPlayDecodes and displays MJPEG data.
mjpegCaptureCaptures MJPEG frames from a camera.
auSegmentReads Sparc Audio data from a file.
auPlayAccepts audio data and sends it to the speaker.
auPriorityPrioritizes audio data by splitting the data into packets of sub-samples frequencies
auReassemReassembles audio data from frequency subsampled packets created by an Au Priority Object.
mpegSegment Read MPEG data from a file.
mpegPriorityPrioritize MPEG data to optimize playback smoothness in the presence of frame dependencies.
mpegPlayDecodes and display MPEG data.
mpegSysFileReads full interleaved MPEG video and audio data from a file.
mpegDemuxSplits full interleaved MPEG streams into independent video and audio streams.
h261PlayDecodes and displays h261 data.
yuvPlayDisplay YUV format streams.
yuvFilterImplements filter functions like fade, dissolve, picture-in-picture, and text overlay onto YUV fomrat streams.
bufferPrinterPrints buffer types of data passing through it. Used as a debugging tool.
bufferCapturePrintes buffer types and captures buffer data to a file. Used as a debugging tool.

2.3.2 Destruction syntax.


All CM objects have the following destruction syntax:

<object_name> destroy

where <object_name> is the object's name. This command differs from Tk in that the command:

destroy <object_name>

will not necessarily destroy the object. The reason for this difference is that object names used by a CM application may actually refer to objects created in remote CM processes through one or more cmt objects. As a result, the destroy command requires special processing that can not be folded into the general Tk destroy method.

2.3.3 Configuration syntax.


All CM objects have the following configuration syntax:

<object_name> configure ?option-name? ?option-value? \

?option-name option-value?...

This syntax is identical to the Tk widget configuration syntax. If a configuration option name is specified without any subsequent option value, a list is returned consisting of: "option-name default-value current-value." If the configure command is invoked with no arguments, a list of lists is returned giving this information for all configuration option names.

2.4 Packet Source and Packet Dest: The Network Wonder Twins

There are two types of CM objects which almost always come in pairs that are crucial to the flow of CM data over a network: Packet Source (pktSrc) and Packet Destination (pktDest)

All pktDest objects automatically create a udp socket on which the object listens for data. The host name and port number of this socket can be queried with the address command. Similarly, all pktSrc objects automatically create a udp socket through which to send data. The destination address and port number for a pktSrc object must be provided by the application. In short, the steps needed to create a pktSrc/pktDest pair that communicats from host A to host B are:

1) Create a pktDest object in a CM process on host B.

2) Create a pktSrc object in a CM process on host A.

3) Configure the pktSrc object to send data to the pktDest object.

What follows is a specific example of a Tcl script creating a pktSrc/pktDest pair from hostA to hostB:

# Create a cmt object for communicating to hostA

set cmtA [cmt ""]

# Open the connection to a CM process on hostA. Here we

# assume the CM process is listening on port number 1000.

$cmtA open -host hostA 1000

# Create a cmt object for communicating to hostB

set cmtB [cmt ""]

# Open the connection to a CM process on hostB. Here we

# assume the CM process is listening on port number 2000.

$cmtB open -host hostB 2000

# Do step 1: Create a pktDest object on host B

set my_pd [$cmtB create_object pktDest]

# Do step 2: Create a pktSrc object in a CM process on host A

set my_ps [$cmtA create_object pktSrc]

# Do step 3: Configure the packet source object to send

# data to the packet dest object's udp socket.

$my_ps configure -dest [$my_pd address]

# Done.

Details on how pktSrc and pktDest can be configured and used to send data is described in the man pages.

2.5 Linking Objects

Objects are said to be "linked" if they pass data from one to the other. An application using the CM Toolkit sets up chains of objects that operate on data passed to them. An object can be thought of as a data source (e.g., a live camera or a file), a data destination (e.g., a speaker or a file), or a filter (e.g., a media format converter or an audio mixer). This section describs the general architecture of objects and explains how objects are linked together.

2.5.1 Moving data.


Generally, objects can do one or more of the following four actions:

1) Actively pull data from a different object.

If an object can actively pull data, it will have a configuration option "-inCmd" or something like that. This command is called whenever data is needed and should return a scatter buffer list (more on that later).

2) Actively push data to a different object.

If an object can actively push data, it will have a configuration option "-outCmd" or something similar. This command is called with a scatter buffer list given as an argument.

3) Passively accept data from a different object.

If an object can passively accept data, it will have an object command "accept" or something similar. This object command takes a scatter buffer list as an argument.

4) Passively send data to a different object.

If an object can passively send data, it will have an object command "send" or something similar. This command will produce a scatter buffer list.

If an object can actively pull or push data (i.e. has an "-inCmd" or "-outCmd"), it will generally also have a configuration option "-cycleTime" that specifies how often to execute the pull or push.

2.5.2 Scatter buffer lists.


In each of the four styles of data flow (i.e. push, pull, accept, or send), the commands involved either produce or accept as an argument a scatter buffer list. A scatter buffer list specifies "frames" of data by specifying buffers of memory that contain the appropriate data. A scatter buffer list is simply a list of scatter buffers where each scatter buffer contains one frame of data. A frame of data is a logical piece of data given the data type. For example, for a video stream, each frame of data contains one picture. For an audio stream, each frame of data contains some number of seconds of audio. A scatter buffer is simply a list of tuples specifying a buffer, an offset into the buffer where the data starts, and the length of the data.

For example, the following is a scatter buffer list containing 3 frames of data:

{ {{buffer0 0 100} {buffer1 0 200} {buffer0 300 100}}

{{buffer2 0 100} {buffer3 0 500}}

{{buffer0 400 1000}}

}

In this example, the first frame of data consist of the first 100 bytes of buffer0, the first 200 bytes of buffer1, and bytes 300 to 400 of buffer0. Notice that the same buffer, buffer0, is used twice.

The second frame of data in this example, consists of the first 100 bytes of buffer2 and the first 500 bytes of buffer3.

The third frame consists of 1000 bytes from buffer0, starting at byte number 400. Note that buffer0 is used both in the first frame and the last frame.

The buffers are created using the buffer manager (see man page). Typically, the application programmer does not need to know about or create any buffers. Objects that produce data will create and fill these buffers as appropriate and pass them to other objects using the push, pull, send, and accept semantics outlined above.

2.5.3 An example.


To illustrate the passing of buffers from one object to another, the following example creates three objects and links them together. The first object is a MJPEGSegment object that reads MJPEG data from a file. This object creates the buffers, fills them and sends them to the next object. The second object is a MJPEGPriority object which buffers up data sent to it, reorders it according to a priority algorithm, and sends it along to the next object. The last object is a pktSrc object which sends data through the network to some pktDest object. We will assume that the pktDest/pktSrc pair has already been setup.

# Create the MJPEG Segment object

set mjpeg_f [mjpegSegment ""]

# Create the MJPEG Priority object

set mjpeg_p [mjpegPriority ""]

# Create the pktDest object

set pd [pktDest ""]

# Link the Segment object to the Priority object.

# Here the segment object is actively pushing the data to

# the priority object by calling the priority object's

# accept command. CycleTime for the Clipfile object is set

# to 0. For this object, a 0 cycle time means that each

# frame should be sent individually as needed.

$mjpeg_f configure -outCmd "$mjpeg_p accept" \

-cycleTime 0.0;

# Link the priority object to the packet source object.

# The priority object actively pushes the data to the

# packet source object. Here cycleTime is set 0.25 seconds,

# meaning that every quarter of a second, all the data that

# has been buffering up in the Source object gets sent to the

# packet dest object.

$mjpeg_p configure -outCmd "$pd accept" -cycleTime 0.25

Now, every time a frame needs to be sent, the segment object will execute the priority object's accept command with a scatter buffer list containing one frame's data. Every 1/4 second, the priority object will call the packet source object's accept command with a scatter buffer list containing all the frames accumulated in the last 1/4 second.

2.5.4 Linking through C.


One problem with the linking mechanism described above is that Tcl commands can be associated with a lot of overhead. We would like a mechanism that allowed object to pass the scatter buffer lists through C and not Tcl. To this end, a convention has been created that will allow objects to register pointers to their C commands that accept and produce scatter buffer lists. The prototypes for C procedures that deal with scatter buffer lists is not important to the application programmer but is dealt with in the CM developer's guide.

The convention used is this:

If an object registers a pointer to the C procedure equivalent of the accept command, it does so under the name "<object_name>.accept". Similarly, if an object registers a pointer to the C procedure equivalent of the send command, it does so under the name "<object_name>.send". Whether or not an object does this registration is up to the developer who wrote the object and this information should be documented in the man page.

If one object wants to use another object's C procedure (if available), it specifies its "-outCmd" or "-inCmd" as "@<object_name>.accept" or "@<object_name>.send".

Going back to our previous example, the MJPEG Priority object and Packet Dest object both register their accept commands. The previous example can be rewritten to use the C procedures instead of Tcl thusly:

# Create the MJPEG Segment object

set mjpeg_f [mjpegSegment ""]

# Create the MJPEG Priority object

set mjpeg_p [mjpegPriority ""]

# Create the Packet Dest object

set pd [pktDest ""]

# Link the file object to the priority object.

# We want to use the C procedure registered by the priority

# object.

$mjpeg_f configure -outCmd "@$mjpeg_p.accept" \

-cycleTime 0.0;

# Link the priority object to the packet source object.

# We want to use the C procedure registered by the packet

# source object.

$mjpeg_p configure -outCmd "@$pd.accept" -cycleTime 0.25

2.6 Logical Time Systems (LTS's)

Most objects need some sense of what "time" is in order to properly function. In the CM Toolkit, time is told by a logical time system (LTS). An LTS is a CM object like any other and simply maps from "logical" time to system time (i.e. the specific time the machine keeps). There are three components to a LTS: speed, value, and offset. The mapping between logical time and system time is simply:

logical time value = speed * system time + offset.

Generally, offset is not utilized by the application programmer directly. The offset value is calculated by the LTS as a function of speed and value. Value is where in logical time the LTS is now, and speed is how fast time is moving. There is no start or end to logical time. Instead, think of logical time like a timeline infinite in both directions. Objects are set up do things during specified portions of the time line and use the LTS to decide when to do them given the current time and speed. Speed is a ratio between LTS seconds and real seconds. Thus a speed of 1 means that 1 LTS second occurs every real second. A speed of 2 means that 2 LTS seconds occurs every real second, or in other words, LTS is moving twice as fast as real time. Speed and value are set using basic configuration options on a LTS object (see the man page).

Generally, an application sets up one LTS in each CM process that it is connected to in order to control timing for any objects it creates in that CM process. Most applications set up chains of objects across the network requiring that two LTS objects in two different CM processes on two different machines be synchronized. This sort of synchronization is done by making one LTS object the slave of another. Whenever the master LTS object is changed (i.e. speed or value is set), those changes are propagated to all the slave LTS's. Special synchronization code is executed to account for clock drift between different machines.

2.7 Chunk Files

Objects that read data from disk generally require some information about the data to be preparsed. We have created a file format that we call "chunk files" that essentially allow us to preparse this information in advance, storing the data as well as the original file together in a way that makes it very easy for objects that require the data to access it. We have created a number of utilities that will transform various types of files into the corresponding chunk file type. Thus a MJPEG chunk file is a MJPEG file augmented by preparsed tables of information. The specific format of the chunk files is not important for the application programmer. The conversion of files into chunk files is generally NOT necessary. All objects that access data in files should be able to create the preparsed data on the fly. The advantage of using the chunk file formats is to speed the set up time of objects that read data from files.

2.8 CM Bind

Another service that is provided by certain CM objects is the cmBind mechanism. The cmBind command is very much like the Tk "bind" command. It allows the application programer to set up Tcl commands to be executed whenever a particular event occurs. The syntax of the cmBind command is described fully in its man page. Essentially, certain CM objects define "events" and allow callbacks to be associated with those events. Currently, only mpegPlay and mjpegPlay objects register CM events with the cmBind mechanism. Examples of CM events that these objects register include: <FRAME_RECEIVED>, <FRAME_PLAYED>, and <FRAME_DROPPED>. In addition, a script the is bound to these events can use special "%-substitution" parameters. Some substitution parameters are available to every CM event, while others are provided based on the object type. The convention is if the substitution parameter is a capital letter, the parameter is always available, while if the substitution parameter is a lover-case letter, the parameter is specific to the object type. The cmBind man page describes parameters available to all CM events, and object type specific parameters ar described in the object type's man page.

2.9 Some Example Applications

To illustrate how all of this is put together, three example applications are detailed below.

The first example application is simple local playback of an MPEG file and an AU file from a local filesystem. This application does not send data over a network and the application runs inside a CM process.

# Set up variables with the filenames of the files to be

# played.

set videoFile /n/data/some/path/video.mpgc

set audioFile /n/data/some/path/audio.acf

# Create a MPEG Segment object

set mpeg_seg [mpegSegment ""]

# Create a MPEG Priority object

set mpeg_prio [mpegPriority ""]

# Create a MPEG Play object

set mpeg_play [mpegPlay ""]

# Create a AU Segment object

set au_seg [auSegment ""]

# Create a AU Play object

set au_play [auPlay ""]

# Create a Logical Time System

set clock [lts ""]

# Configure the MPEG Segment object in order to :

# use the LTS object,

# read the chosen MPEG file,

# send frames individually (i.e., set cycleTime to 0),

# send frames to the priority object

$mpeg_seg conf -lts $clock -filename $videoFile \

-cycleTime 0.0 -outCmd @$mpeg_prio.accept

# Instruct the MPEG Segment object to play the contents

# of the file starting at logical time 0. By setting the

# logical end time of the segment to be ?, we instruct the

# object to calculate the time value based on the values of

# clipStart, clipEnd, and logicalStart.

$mpeg_file configure -clipStart 0 -clipEnd end \

-logicalStart 0 -logicalEnd ?

# Configure the MPEG Priority object in order to:

# send frames in 1/4 second chunks,

# send prioritized frames to the play object

$mpeg_prio conf -cycleTime 0.25 -outCmd @$mpeg_play.accept

# Create a frame for video output.

frame .f

# Query the MPEG File object for the video's width and

# height

set vid_width [lindex [$mpeg_file info] 3]

set vid_height [lindex [$mpeg_file info] 4]

# Configure the frame to be the right size

.f conf -width $vid_width -height $vid_height

# Pack the frame

pack .f

update;

# Configure the MPEG play object to:

# use the lts object,

# display output in the frame we just created.

$mpeg_play conf -lts $clock -xid [winfo id .f]

# Configure the AU segment object to:

# use the lts object

# read data from the desired file,

# send data in 1/4 second chunks,

# send data to the AU play object.

$au_seg conf -lts $clock -filename $audioFile \

-cycleTime 0.25 -outCmd @$au_play.accept

# Instruct the AU segment object to play the contents of the

# file starting at time 0.

$au_seg configure -clipStart 0 -clipEnd end \

-logicalStart 0-logicalEnd ?

# Configure the AU play object to:

# use the lts object.

$au_play conf -lts $clock

# Ready all the objects.

$au_play ready;

$mpeg_play ready;

$au_seg ready;

$mpeg_prio ready;

$mpeg_seg ready;

# Start playback by setting the lts speed to 1.

$clock conf -speed 1.0

The second example illustrates a simple network playback application. The application runs locally in a CM process, connects to a remote CM process on Host A at port number 5000, and constructs two pipelines of objects for MPEG video and audio data to flow through. In this example, packet source and dest objects will be required to send the data over the network and a cmt object will be required to create a control channel to the remote process.

# Hold the hostname and filename information in local

# variables

set remoteHost hostA

set remotePort 5000

set videoFile /some/path/video.mpgc

set audioFile /some/path/audio.acf

# Create a cmt connection to the remote host.

set conn [cmt ""]

$conn open -host $remoteHost $remotePort

# Create a MPEG Segment object in the remote process

set mpeg_seg [$conn create_object mpegSegment]

# Create an AU Segment object in the remote process

set audio_seg [$conn create_object auSegment]

# Create a MPEG Priority object in the remote process

set mpeg_prio [$conn create_object mpegPriority]

# Create a packet source for video in the remote process

set video_pkt_src [$conn create_object pktSrc]

# Create a packet source for audio in the remote process

set audio_pkt_src [$conn create_object pktSrc]

# Create a packet dest for video in the local process

set video_pkt_dest [pktDest ""]

# Create a packet dest for audio in the local process

set audio_pkt_dest [pktDest ""]

# Create a MPEG Play object in the local process

set mpeg_play [mpegPlay ""]

# Create an AU Play object in the local process

set au_play [auPlay ""]

# Create a local LTS.

set local_clock [lts ""]

# Create a remote LTS.

set remote_clock [$conn create_object lts]

# Now that all the objects we need are created, let's

# configure them properly. First, lets do the video

# pipeline.

# Configure the MPEG Segment object to:

# use the remote clock,

# read the desired file,

# send frames individually (cycleTime = 0.0),

# send frames to the priority object.

$mpeg_seg conf -lts $remote_clock -filename $videoFile \

-cycleTime 0.0 -outCmd @$mpeg_prio.accept

# Instruct the MPEG Segment object to play the contents of

# the file starting at logical time 0.

$mpeg_file configure -clipStart 0 -clipEnd end \

-logicalStart 0 -logicalEnd ?

# Configure the MPEG Priority object to"

# send frames in 1/4 second chunks,

# send frames to the video packet source.

$mpeg_prio conf -cycleTime 0.25 \

-outCmd @$video_pkt_src.accept

# Configure the video packet source object to send frames

# to the video packet dest object.

$video_pkt_src conf -dest [$video_pkt_dest address];

# Configure the video packet dest object to send frames

# to the MPEG Play object.

$video_pkt_dest conf -outCmd @$mpeg_play.accept

# Create a frame for video playback.

frame .f

# Configure the width and height of the frame to match

# the width and height of the video.

.f conf -width [lindex [$mpeg_file info] 3] \

-height [lindex [$mpeg_file info] 4];

# Pack the frame.

pack .f

update;

# Configure the MPEG Play object to

# use the local LTS,

# display frames in .f

$mpeg_play conf -lts $local_clock -xid [winfo id .f]

# Now let's set up the audio pipeline.

# Configure the AU Segment object to:

# use the remote LTS

# read from the desired file

# send data in 1/4 second chunks

# send data to the audio packet source object.

$audio_seg conf -lts $remote_clock -filename $audioFile \

-cycleTime 0.25 -outCmd @$audio_pkt_src.accept

# Instruct the AU Segment object to play the contents of

# the file starting at time 0.

$audio_file configure -clipStart 0 -clipEnd end \

-logicalStart 0 -logicalEnd ?

# Configure the audio packet source object to send data

# to the audio packet dest object.

$audio_pkt_src conf -dest [$audio_pkt_dest address];

# Configure the audio packet dest objec to send data

# to the AU Play object.

$audio_pkt_dest conf -outCmd @$au_play.accept

# Configure the AU Play object to use the local LTS

$au_play conf -lts $local_clock;

# Finally, make the remote LTS a slave of the local LTS.

$local_clock addSlave $remote_clock;

# Ready all the objects.

# Notice that packet source and dest objects do not

# require ready'ing.

$mpeg_play ready;

$mpeg_prio ready;

$mpeg_seg ready;

$au_play ready;

$audio_seg ready;

# Start playback by setting the local LTS speed to 1.

$local_clock conf -speed 1.

The final example is a much more complicated application. This application is written for a simple Tcl/Tk interpreter with Tcl-DP that has NOT been extended with CM objects. This application relies solely on connections made through cmt objects (remember that cmt objects are implemented completely in Tcl and Tcl-DP and can be used by sourcing the file cmt_api.tcl). The application makes a cmt connection to a local CM process to create its local objects as well as to two different remote CM processes which will serve video and audio separately over the network. We assume that the local and remote CM processes are up and running and can be connected to through port 5000. In reality, applications will probably use some sort of name server to avoid well known port numbers like this, but for this example we will assume the simple case of a well known port number. In this example we will be streamin MJPEG and AU data.

# The following information sets the hostname and filename

# of the two files we want to play.

set videoHost hostA

set videoFile /n/data/some/path/video.mjpgc

set audioHost hostB

set audioFile /n/data/some/path/audio.acf

# Put the CM Application runtime library directory

# in the auto load path.

lappend auto_path $env(CMT_ROOT)/cmapp

# Find our local host name

set localHost [exec /usr/bin/hostname]

# Create a cmt connection to our local CM process

set local_cm [cmt ""]

$local_cm open -host $localHost 5000

# Create a cmt connection to the video host

set video_cm [cmt ""]

$video_cm open -host $videoHost 5000

# Create a cmt connection to the audio host.

set audio_cm [cmt ""]

$audio_cm open -host $audioHost 5000

# Create an lts on the video host

set video_lts [$video_cm create_object lts]

# Create an lts on the audio host

set audio_lts [$audio_cm create_object lts]

# Create an lts on the local host

set local_lts [$local_cm create_object lts]

# Now make the video_lts and auiod_lts slaves of the

# local_lts. This part is a bit tricky. Since the video and

# audio lts objects were created through different cmt

# connections than the local lts object, the process where

# the local lts object lives does not know about these

# objects and their object names will mean nothing.

# What we must do is create proxie objects in the local

# CM process that map their object names into commands

# that will go through the proper cmt connections.

# This proxie function is detailed in the cmt man page.

$local_cm create_proxie $audio_lts

$local_cm create_proxie $video_lts

# Now make the audio and video lts's slaves of the local lts.

$local_lts addSlave $audio_lts

$local_lts addSlave $video_lts

# Now let's create all the different objects that we

# will need. Notice that the remote video server objects

# are created through the video_cm object, the remote

# audio server object are create through the audio_cm object

# and that the local objects are created through the

# local_cm object.

set mjpeg_seg [$video_cm create_object mjpegSegment]

set mjpeg_prio [$video_cm create_object mjpegPriority]

set video_pkt_src [$video_cm create_object pktSrc]

set audio_seg [$audio_cm create_object auSegment]

set audio_pkt_src [$audio_cm create_object pktSrc]

set video_pkt_dest [$local_cm create_object pktDest]

set mjpeg_play [$local_cm create_object mjpegPlay]

set audio_pkt_dest [$local_cm create_object pktDest]

set audio_play [$local_cm create_object auPlay]

# Now let's set up the video pipeline

# Configure the MJPEG Segment object to:

# use the video lts,

# read the desired file,

# send frames individually (i.e., cycleTime = 0.0),

# send frames to the MJPEG Priority object.

$mjpeg_seg conf -lts $video_lts -filename $videoFile \

-cycleTime 0.0 -outCmd @$mjpeg_prio.accept

# Instruct the MJPEG Segment object to play the contents of

# the file starting at time 0.

$mjpeg_seg configure -clipStart 0 -clipEnd end \

-logicalStart 0 -logicalEnd ?

# Configure the MJPEG Priority object to:

# send frames in 1/4 second cycles,

# send frames to the video packet source.

$mjpeg_prio conf -cycleTime 0.25 \

-outCmd @$video_pkt_src.accept

# Configure the video packet source object to send

# frames to the video packet dest object.

$video_pkt_src conf -dest [$video_pkt_dest address];

# Configure the video packet dest object to send frames

# to the MJPEG Play object.

$video_pkt_dest conf -outCmd @$mjpeg_play.accept

# Create a frame for the video output, resize it to the

# width and height of the video, and pack it.

frame .f

.f conf -width [lindex [$mjpeg_file info] 3] \

-height [lindex [$mjpeg_file info] 4];

pack .f

update;

# Configure the MJPEG Play object to:

# use the local lts,

# display output in the newly created frame.

$mjpeg_play conf -lts $local_lts -xid [winfo id .f]

# Now let's set up the audio pipeline.

# Configure the AU Segment object to:

# use the audio_lts,

# read the desired file,

# send data in 1/4 second cycles,

# send data to the audio packet source object.

$audio_seg conf -lts $audio_lts -filename $audioFile \

-cycleTime 0.25 -outCmd @$audio_pkt_src.accept

# Instruct the AU Segment bject to play the contents of the

# file starting at time 0.

$audio_seg configure -clipStart 0 -clipEnd end \

-logicalStart 0 -logicalEnd ?;

# Configure the audio packet source object to

# send data to the audio packet dest object.

$audio_pkt_src conf -dest [$audio_pkt_dest address];

# Configure the audio packet dest object to send

# data to the AU Play object.

$audio_pkt_dest conf -outCmd @$audio_play.accept

# Configure the AU Play object to use the local lts.

$audio_play conf -lts $local_lts;

# Ready all the objects.

$mjpeg_play ready;

$mjpeg_prio ready;

$mjpeg_seg ready;

$audio_play ready;

$audio_seg ready;

# Start playback by setting local lts speed to 1.

$local_lts conf -speed 1.0;

This concludes the application programmer's view of the CM Toolkit.

3.0 CM Toolkit Developer's View

This section of the document covers information useful to people extending the CM Toolkit. It assumes that you've read the Application Programmer's View. I've tried to include as much information as a developer would need, but I am sure that I've left things out. Please feel free to ask questions and make comments on how I can improve this document.

3.1 The Source Tree

In this section, the top of the source tree will be refered to as TOP.

3.1.1 Building the Source Tree.


The following are instructions for building the source tree for a local developer at UC Berkeley. If you received a distribution tree from Berkeley but are NOT involved in setting up a CVS revision controlled build, ignore these instructions and follow the instructions found in README.INSTALL at the top of your source tree.

The source tree is setup to easily do multiple architecture builds. Compiling for only one architecture is simply a special case of a multiple architecture build. The first thing you need to do is decide on a architecture extension name. The value returned by uname is usually a pretty good one. In our experience, we use "sun4" for our SunOS systems, "alpha" for our Alpha systems, and "hp" for our HPUX systems. The architecture extension value will be referred to as <ARCH>. In order to build the source tree, you must have the program "tclsh" somewhere in your path. In addition, the makefiles work most smoothly if using gmake.

To build the source tree follow the following steps:

1. Create TOP/config/configure.users.<ARCH> setting the variables according to your system. A blank template for configure.users.<ARCH> exists in configure.users.template. In addition, we have provided several common configurations in configure.users.sun4.default, configure.users.alpha.default, etc.

2. Cd to TOP. Execute the command "make-arch <ARCH>". The shell script makearch will create TOP/config/configure by running autoconf on TOP/config/configure.in. It will then execute the configure script in order to create the file TOP/config.status.<ARCH>. It will also create the subdirectory TOP/LINKS which will have symbolic links to the locations of all the library and executable sources distributed with the CMT. The script will then descend into each library and executable source directory and create a subdirectory named <ARCH>. In each of these subdirectories, makefiles will be generated and the compilation process started.

3. To compile the tree for another architecture, simply follow steps 1 and 2 with a different value for <ARCH>.

One of the configurable variables in configure.users.<ARCH> is the compiled in value for CMT_ROOT. All runtime libraries are by default looked for in subdirectories of this value. In other words, the tk runtime libraries will be looked for in CMT_ROOT/tk, the tcl runtime libraries will be looked for in CMT_ROOT/tcl, the cm process runtime libraries will be looked for in CMT_ROOT/cmp, and the cm application interface and runtime libraries will be looked for in CMT_ROOT/cmapp. The reason for this is due to the fact that the CMT does not use the latest version of Tcl or Tk and makes modifications to both of these. Thus, in order to avoid confusion with standard Tcl/Tk applications and runtime libraries, we've set things up to use the value of CMT_ROOT as a base and then to look for specific libraries under it. Generally, you should set the compiled in value to be the location of where you want the runtime libraries once you install everything (typically /usr/local/lib/cmt). In the meantime, you can set the value of CMT_ROOT at runtime by providing the value desired in the environment variable CMT_ROOT. The directory TOP/runtime has symbolic links to all the runtime libraries and should be used as the value for this environment variable.

Man pages for make-arch exist in TOP/doc/man/man1. There are several options that can be given to make-arch for rebuilding or reconfiguring the source tree.

3.1.2 The executables.


There are basically two executables: cmwish and cmsh. Cmwish is a CM process that creates a main window and has Tk widgets. Cmsh is a CM process that makes a connection to the X server, but does not create a main window or have Tk widgets. If cmsh is run as "cmsh -server" then cmsh does not even make a connection to the X server and objects that require such a connection will not be present

The code and executable for cmwish is in TOP/apps/cmwish and the code and executable for cmsh is in TOP/apps/cmx.

3.1.3 The libraries


The executables are compiled with 5 libraries: libtcl.a, libtk.a, libdp.a, libnv.a, and libcm.a. These libraries are built and kept in TOP/lib/tcl, TOP/lib/tkx, TOP/lib/dp, and TOP/lib/cm.

3.1.4 Coding Standards


The directory TOP/doc/engManual contains a large manual detailing coding standards. Although we have not been following those coding standards very well yet, we are making an effort to move toward them. If you can follow these standards things should be much easier to read in the future.

3.2 CM Objects

It would be very hard to describe exactly how to create a new object in this document. Some basic information is given here, but learning by example will probably be the easiest thing to do. The mjpegSegment object and the mjpegPriority object are good ones to look at.

3.2.1 Adding new CM objects.


New CM objects are to be added to the cm library in TOP/lib/cm. Typically, adding a new object will entail the following steps:

1. Create the .c and .h files necessary for the new object.

2. Create a procedure "<Object>ModuleInit" that registers the new object creation command and does any other initialization of the new object module. This procedure should return TCL_OK on success and TCL_ERROR on failure.

3. Modify the file TOP/lib/cm/cmInit.c to make a call to the new object module's initialization procedure.

4. Modify the file TOP/lib/cm/Makefile.in to add the new object files to the appropriate library.

5. Rebuild.

3.2.2 Object syntax.


The creation, destruction, and configuration syntax for all CM objects must adhere to the syntax outline in the application programmer's view. If it does not, the cmt object and object management routines will NOT work.

3.3 Atom registration.

One facility used very often in CM objects is TOP/doc/man/man3/Atom.3.

3.4 LTS

The LTS object has a C interface that allows objects to access timing information quickly. Every lts registers its object name as an atom pointing to its C data structure. Thus, objects that need to be controlled by an LTS can provide the configuration option "-lts" that the application can set to be the name of the lts to be used. Using this name, the object can look up the atom associated with it using the atom registration facility. The pointer to the lts's C data strucuture will be returned. This pointer can be used with the C interface to the LTS object module to access the lts in question.

Part of the LTS C interface is the ability to register triggers if either speed, value, or offset is changed. The object provides a pointer to a function that as an argument a pointer to arbritrary data and the pointer to pass along as the argument (typically the data structure of the object registering the trigger). This function will then be called if something about the LTS changes. When a trigger is registered with an LTS a trigger number is returned. The object should store the trigger number so that it can tell the LTS to unregister the trigger when it is either no longer needed or when the object itself is destroyed or put in an unready state.

3.5 Chunk Files

The format of chunk files is type specific but they all are organized using the "chunk file" routines found in TOP/lib/cm/cmChunk.c. Basically, each chunk file consists of one or more chunks. Each chunk is typed. The chuck types already defined are all in TOP/lib/cm/cmCookies.h. The chunk file routines allow the object to open a file as a chunk file and then search through the file for a specific chunk type. Then the object is responsible for reading the chunk's data from the point in the file that the chunk file routines have placed the file pointer. Thus, objects that read data from chunk files do NOT open the files and access them in any way other than through the chunk file routines.

The basic model of reading data from a chunk file is:

1. Open the file as a chunk file using Chunk_OpenFile.

2. Search for a particular chunk type using Chunk_FindType. If this routine does not succeed, the chunk in question may be before the pointer where searching started. Use Chunk_ReadRewind and try again if necessary. If the Chunk_FindType fails again, the chunk in question does not exist.

3. Read data from the file. The file descriptor is available through the tChunkFileState pointer that the Chunk_OpenFile routine has returned.

4. Use Chunk_CloseFile to close the file.

If Chunk_OpenFile fails, the file in question is not a properly formatted chunk file and probably represents data that was not pre-processed. In this case, objects should be able to open the file in the normal way and construct the necessary information.

This is probably very confusing, but examining how things are done in the utility vid2mjpgc which is located in the HOME/utilities/vid2mjpgc directory and in the object mjpegFile should be enlightening.

3.6 Buffers and buffer passing.

Objects that accept buffers from other objects or pass buffers to other objects should use the buffer manager routines to create and control these buffers. Creating and configuring buffers is covered in the buffer manager man page.

Passing buffers is a little bit more complicated. A C data structure for scatter buffer lists exists in HOME/lib/cm/bufferlist.h. Routines that operate on this data structure are provided in HOME/lib/cm/bufferlist.c and HOME/lib/cm/tcmBufferUtil.c.

If an object creates a new buffer to be filled with data, the first thing that should be done to each buffer created is an "attach". The routine BufferAttach in HOME/lib/cm/tcmBuffer.c does this attaching. The attach simply increments a reference count. A newly created buffer starts with a reference count of 0, so attaching gives it a count of 1.

Basically, attach and detach increment and decrement a reference count associated with the buffer. If the reference count ever falls to zero, the buffer manager automatically places the buffer back in the "free" state. Thus, objects that accept buffers should attach to them immediately. Objects that push buffers, should detach from them once the pushing is complete. At first glance it would appear that passively sending buffers will be problematic since the sending object wants to detach from the buffers before the receiving object can attach to them, thus allowing the reference count of the buffers to artificially fall to 0 and be collected as free. We have circumvented this problem by delaying detach operations until the Tcl interpreter is idle. Thus, an object can detach from a buffer and send it without error. The receiving object must attach to the buffer immediately. The attach operation will occur immediately, while the detach operation is delayed until the event loop is processed again, ordering the attach and detach operations correctly to prevent error.

If an object can passively accept buffers, it should do so through the object command "accept". The accept object command should be able to parse a Tcl representation of a scatter buffer list. Routines for doing this are in HOME/lib/cm/bufferlist.c.

If an object wishes to register a C interface to the accept routine for other objects to use, it should register a pointer to the C accept procedure with the atom registration facility under the name <object_name>.accept. The prototype for a C accept procedure should be the same as the InFunc procedure type in bufferlist.h. Basically, a C accept procedure should take as arguments a pointer to the object accepting the buffers and a pointer to the scatter buffer list data structure. The scatter buffer list structure passed to a C accept procedure is assumed to belong to the caller of the procedure. As such, the accepting procedure should copy the scatter buffer list structure using the copy buffer list structure utility procedure in tcmBufferUtil.c.

Regardless of how the scatter bufferlist was given to the object (either through the Tcl interface or the C interface) the first thing that the object should do is "attach" to each buffer in the scatter buffer list.

If an object can passively send buffers, it should do so through the object command "send". The send object command should be able to produce a Tcl representation of a scatter buffer list. Routines for doing this are in HOME/lib/cm/bufferlist.c.

If an object wishes to register a C interface to the send routine for other objects to use, it should register a pointer to the C send procedure with the atom registration facility under the name <object_name>.send. The prototype for a C send procedure should be the same as the OutFunc procedure type in bufferlist.h. Basically, a C send procedure should take as arguments a pointer to the object producing the buffers and a pointer to a pointer to the scatter buffer list data structure to be filled.

If an object can actively push buffers, it should do so through a configuration option "-outCmd". In addition, a configuration option "-cycleTime" should control how often this pushing action is taken. Objects that have an "-outCmd" should be able to accept any Tcl command as well as registered names of C procedures. If -outCmd is a Tcl command, the command should be evaluated with a Tcl representation of a scatter buffer list as an argument. In other words, if -outCmd was "puts" then the command evaluated would be "puts <scatter_buffer_list>" where <scatter_buffer_list> is a Tcl scatter buffer list representation of the data being pushed. A registered C procedure will always begin with "@" differentiating it from a regular Tcl command. If the value of -outCmd begins with a "@", the remainder of the value should be taken to be in the format: <object_name>.<cmd_name> where <cmd_name> will generally be "accept". The object can use the value of -outCmd after the initial "@" to look up the register C procedure with the atom registration facility. This procedure will want two arguments: a pointer to the object accepting the data and a pointer to a C scatter buffer list data structure. The pointer to the object accepting the data can be found by parsing <object_name> out of the value of -outCmd and looking up the recieving object's command info using Tcl_GetCommandInfo and using the ClientData pointer of the structure returned.

Regardless of whether -outCmd is a regular Tcl call or a registered C procedure, the object pushing the buffers should "detach" from all buffers after the push is completed (i.e. the Tcl call executed or the C procedure called). The reason for this is that the accepting object will increment the reference count when it attaches to buffers it recieved.

If an object can actively pull buffers, it should do so through a configuration option "-inCmd". Again, "-cycleTime" should be provided to control how often pulling occurs. Again, C procedures can be used instead of regular Tcl commands if the -inCmd begins with a "@". The procedure registered, however, will have a different prototype. Procedures registered that can produce a scatter buffer list takes as arguments a pointer to the object producing the scatter buffer list (again this can be found by parsing off the object pointer name and using Tcl_GetCommandInfo) and a pointer to a pointer to a scatter buffer list. This structure will be filled by the producing object.

The mjpegPriority object is an object that has all four styles of buffer passing and can be used as an example.

3.7 Registering cmBind events.

Each object can register cmBind events and all the possible object type specific substitution parameters by using the C interface to the cmBind mechanism. This interface is described in the man page cmBind.3. Examples of using the cmBind mechanism can be found by looking at the mjpegPlay and mpegPlay objects.

4.0 Caveats

This document presents things as they should be and not necessarily as they are. If you think that things are being done incorrectly, please inform us of any bugs so that we can try to fix them. Buffer passing between objects is a point of particular concern since this complicated convention is critical to avoid data copying. If any questions arise please feel free to ask me.


No Title - 2 FEB 1996
[Next]

Generated with Harlequin WebMaker