Flow-Based Programming - Appendix

THREADS API and Network Specification

This chapter has been excerpted from the book "Flow-Based Programming: A New Approach to Application Development" (van Nostrand Reinhold, 1994), by J.Paul Morrison.

To find out more about FBP, click on FBP . This will also tell you how to order a copy.                            drawflow.ico     

For definitions of FBP terms, see Glossary                                                                                        


[The THREADS package is in process of being converted to use Windows "fibres",  and C++.  As a result of this, there will be a number of changes to this page, so it will no longer match the original material published in my 1994 book.  Anyone contemplating using THREADS should wait until the relevant web pages have been updated - or can work directly with the latest zip file, which, although currently undergoing enhancement, should be internally consistent at any point in time - Jan. 2010.]

Material from first edition of book starts here:

In Chapter 23, we described the THREADS notation for linking components into networks - this will also be repeated below in summary form. However, perhaps even more important are the conventions for writing components, as these are what will allow independently developed components to be combined into a single network without the requirement that they be developed by the same person or even in the same location (apart from the normal requirements for consistency of IP and stream formats).

Here is a simple component written in C++ to copy IPs from IN to OUT. Let's call this rather unoriginally ThCopy.

A THREADS component is a regular C++ subroutine, so, as you might expect, the first few statements are the declares.


#include <string.h>
#include "compsvcs.h"

THRCOMP ThCopy(anchor proc_anchor)
{
void *ptr;
void *ptr2;
int value;
long size;
char *type;
port_ent port_tab[2];

value = dfsdfpt(proc_anchor, 2, port_tab,"IN","OUT");

value = dfsrecv(proc_anchor, &ptr, &port_tab[0], 0, &size, &type);
while (value == 0) {
value = dfscrep(proc_anchor, &ptr2, size, type);
memcpy(ptr2,ptr,size);
value = dfssend(proc_anchor, &ptr2, &port_tab[1], 0);
if (value != 0) {
dfsdrop(proc_anchor, &ptr);
return(1);
}
value = dfsrecv(proc_anchor, &ptr, &port_tab[0], 0, &size, &type);
}

return(0);
}



The first action is always to initialize the port table using a call to dfsdfpt - the 2nd parameter of the call to dfsdfpt must match the dimension declared for the port table. This number must also match the number of port names specified in the parameters to dfsdfpt . Note: dfsdfpt modifies the port table, so this must not be defined as constant.

The return statement causes deactivation (but not necessarily termination, as described above). The return code value on the return statement determines whether the process is willing to be reactivated: a value of 5 or greater forces termination even if data is available.

API Calls:

Here are the API calls available to components in THREADS:

Create a Packet:
value = dfscrep(proc_anchor, &ptr, size, type);

Define ports:
value = dfsdfpt(proc_anchor, port_count, port_tab);

Receive an IP:
value = dfsrecv(proc_anchor, &ptr, &port_tab[port_no],
elem_no, &size, &type);

Send an IP:
value = dfssend(proc_anchor, &ptr, &port_tab[port_no],
elem_no);

Drop an IP:
value = dfsdrop(proc_anchor, &ptr);

Pop an IP off the stack:
value = dfspop(proc_anchor, &ptr, &size, &type);

Push an IP onto the stack:
value = dfspush(proc_anchor, &ptr);

Close a port element:
value = dfsclos(proc_anchor, &port_tab[port_no],
elem_no);

Parameters:

Note that the ptr, size, type and port table element parameters all have &'s attached, except in the case of dfscrep , where only ptr has it. The &'s are required because these parameters may be modified, and C parameters are all passed by value (except for strings and arrays). In some cases a particular service does not set them, but for consistency &'s are used for all of them (except dfscrep ).

port_count and elem_no are all binary integer variables (int). size is binary long. dfscrep is restricted to a maximum of 64000 bytes.

All pointers in the following declarations are "far" pointers.

Declare for port_ent:

  struct _port_ent
{
char port_name[32];
void *reserved;
int elem_count;
int ret_code;
};

typedef struct _port_ent port_ent;

After a call to dfsdfpt, elem_count in each port_ent instance will be set to the number of connected elements for that port, and ret_code will be set to 0 or 2, depending on whether that port was connected or not.

Declare for anchor (thxanch.h):

struct _anchor {
int (* svc_addr) ();
void *proc_ptr;
} ;

typedef struct _anchor anchor;

Service return codes:

dfscrep:
0 OK

dfsdfpt:
0 OK - but some elements of port_tab may have their
ret_code fields set to 2, meaning "port name not
found"; a receive or send from or to such a
port_tab element will result in a return code value
of 2.

dfsrecv:
0 OK
1 port element closed (end of data)
2 port element not defined or not connected

dfssend:
0 OK
1 port element closed
2 port element not defined or not connected

dfsdrop:
0 OK

dfspop:
0 OK
2 stack empty

dfspush:
0 OK

dfsclose:
0 OK
1 port element already closed
2 port element not defined or not connected

Limitations:

There is a limit of 4000 elements per array port.

Working storage for a component should not exceed a few thousand bytes (including the working storage of any subroutines it calls). If the component needs more than this, it should use one of C's dynamic allocation functions (e.g. malloc or calloc) to allocate the additional storage.

Network Notation

Networks are defined initially using a free-form notation, described briefly in Chapter 23. For instance,

'data.fil'->OPT Reader(THFILERD) OUT -> IN Selector(THSPLIT) MATCH -> ...,
Selector NOMATCH -> ...

The general syntax for networks is quite simple, and can be shown as follows (using a variant of the flow notation which has started to become popular for defining syntax):


I have labelled the ports above and below the connection indicator (arrow with optional capacity figure) "up-port" and "down-port" to indicate that they are upstream and downstream, respectively, of the connection.

The main network may be followed by one or more subnets, which have basically the same notation (each one starting with a label and finishing with a semi-colon). However, subnets have to have additional notation describing how their external port names relate to their internal ones. Since this association is like an equivalence, we use the symbol => to indicate this relationship. Thus,

, port-name-1 => port-name-2 process-A port-name-3,

indicates that port-name-1 is an external input port of the subnet, while port-name-2 is the corresponding input port of process-A. Similarly,

, port-name-1 process-A port-name-2 => port-name-3,

indicates that port-name-3 is an external output port of the subnet, while port-name-2 is the corresponding output port of process-A. For an example, see the example of subnets given in Chapter 23.


Other syntactic elements:


Two consecutive quotes are taken to mean a single quote.


element-number does not apply to the external port names of subnets. The asterisk indicates an automatic port (see Chapter 13).


capacity does not apply to the external port names of subnets.


The component name can be specified on any occurrence of the process name. The question mark indicates that tracing is desired.

In Chapter 13 we alluded to the fact that we can specify that a process "must run at least once" - this is specified by means of an attribute file for the component, which has the name of the component, and an extension of atr, e.g. thcount.atr. These files must be provided by the supplier of a component, and must have a specific format. So far, only the "must run" attribute has been defined - it is specified by coding one of the character strings must_run, Must_run or MUST_RUN, with optional preceding blanks, in the attribute file for a given component. If no attribute file is found for a component, default values are assumed to apply (the default for the "must run" attribute is "need not run").

As mentioned in Chapter 23, there is also a compiled (or compilable) format for specifying networks - while this is obviously not appropriate for hand-coding by human users, it is easy enough to generate by means of software. It is a data-only C program, and specifies a network, together with all referenced subnets. Its importance is that this will be the source form for networks owned by customers, so THREADS must guarantee that any enhancements to it will be upwards compatible. These guarantees are essentially encoded in the C headers which it uses: thxiip.h, thxanch.h and thxscan.h.

thxanch.c has been given above. The other two are defined as follows:

thxiip.h

struct _IIP
{
int IIP_len;
char datapart[512];
};

typedef struct _IIP IIP;


thxscan.h

struct _cnxt_ent {
char upstream_name[32]; /* if 1st char is !, */
char upstream_port_name[32]; /* connxn points at IIP */
int upstream_elem_no;
char downstream_name[32];
char downstream_port_name[32];
int downstream_elem_no;
union cnxt_union {IIP *IIPptr; void *connxn;} gen;
int capacity;
};

typedef struct _cnxt_ent cnxt_ent;

struct _label_ent {
char label[32];
char file[10];
struct _cnxt_ent *cnxt_ptr;
struct _proc_ent *proc_ptr;
char ent_type;
int proc_count;
int cnxt_count;
};

typedef struct _label_ent label_ent;

struct _proc_ent {
char proc_name[32];
char comp_name[10];
int (*faddr)();
void *proc_block;
int label_count;
int trace;
int composite;
int must_run;
};

typedef struct _proc_ent proc_ent;

Note: the above structures are not the internal control blocks of THREADS - they are an encoding of the free-form network specification notation. This separation will allow THREADS to be extended in the future without requiring network definitions to be reprocessed.