Tsctl API Conventions

The tsctl API follows certain conventions regarding the C prototypes of member functions, and how these relate to the command line client and to the tsctl network protocol.

Data is passed TO a function through parameters. Each parameter must be one of the following:

1. Primitive type (e.g char x, int y, etc.)
2. Pointer to an Array of constant data (e.g. const char* data)
3. Fixed length array to constant data (e.g. const char data[16])
   Note that this specifically excludes C struct and union constructs. Such data must be passed by breaking it down into individual parameters of allowed tpyes.  It also requires all pointers to be to an Array. All data passed TO the function by pointer (Array) or array must be marked const.
4. The first parameter to any member function is always a pointer to the object instance.  Notationally it appears as an array (Class *), although it is always a single element.

All data returned FROM the function must be returned in one of the following ways:

1. A return value which is primitive (e.g. char, int, etc.)
2. A return value which is an Array (e.g. char*).  This is only allowed if the caller cannot know the length of the Array to be returned.  The Array is allocated by the function and it is the responsibility of the caller to free it as needed.  Unlike pointers in the function parameters, a function return can be a pointer to a C struct.
3. A dynamically allocated Array passed by the caller as a parameter, e.g. char* data.  The lack of a const qualifier denotes that the function will put data in the Array.  The contents of the Array upon entry to the function is undefined, and the caller is not allowed to pass data in it to the function.  The length of the Array indicates the maximum number of elements the caller wants.  The function will resize the Array as needed so that on return its length is equal to the number of elements actually put into the array.  This method of returning data is only allowed when the length of the Array need not be the same for every call.
4. A fixed length array passed by the caller as a parameter, e.g. char data[16].  The lack of a const qualifier denotes that

the function will put data in the array.  The contents of the array upon entry to the function is undefined, and the caller is not allowed to pass data in it to the function.   The array will be completely filled by the function upon return, unless it defines an external signalling mechanism (e.g. a return code) to indicate whether or not the data is valid.  For short arrays it is permissible for efficiency reasons to return that the array was partially filled.  This method of returning data is required when the length of the array must be the same for every call.
Note that the convention for returning a struct is for the caller to pass an array (length 1) of the structure to the function.  If the structure has any pointers, they must be initialized dynamically by the function.

The tsctl calling convention allows for a well-define programatic relationship between each C function and the corresponding tsctl client command and network packet sequence.

Relationship between C function and tsctl client command

A tsctl client command consists of the form:

tsctl @Host Object Command ...

Where Host is the TCP/IP hostname of the server to run the command on. (This parameter is omitted for a direct-access client.)

The Object specification consists of the Class name (e.g. System, Bus, etc.), optionally followed by a colon and the instance number.  If the latter is not specified, instance 0 is used.

Command is the name of the API function in the specified class to be called.  It must be followed by any parameters required by the function in question. These parameters are referred to as "pieces of data" or "elements" in the description below and correspond to parameters in the corresponding C function.

Each piece of data passed TO the function has a corresponding command line element.

When data is to be returned FROM a function via an Array passed by the caller, a corresponding command line element is required to specify the length of the Array passed.

Fixed length arrays passed to the function for returning data do not have corresponding command line elements.
The command line elements are in the same order as the parameters to the C function.

Each C parameter type has a corresponding syntax for the command line element for it:

char - a decimal, hexadecimal (starting with "0x"), or octal string representation (starting with "0").  Only the least significant 8 bits will be considered.

short - a decimal, hexadecimal (starting with "0x"), or octal string representation (starting with "0").  Only the least significant 16 bits will be considered.

int - a decimal, hexadecimal (starting with "0x"), or octal string representation (starting with "0").

unsigned char - same as char, except the value will be treated as unsigned

unsigned short - same as short, except the value will be treated as unsigned

unsigned int - same as int, except the value will be treated as unsigned

const char* - either a literal string (quoted if spaces are present to notify the shell to hold the entire string in a single element), or a sequence of char values separate by colons.

const short* - a sequence of short values separate by colons.

const int* - a sequence of int values separate by colons.

const unsigned char* - either a literal string (quoted if spaces are present to notify the shell to hold the entire string in a single element), or a sequence of unsigned char values separate by colons.

const unsigned short* - a sequence of unsigned short values separate by colons.

const unsigned int* - a sequence of unsigned int values separate by colons.

const char[] - either a literal string (quoted if spaces are present to notify the shell to hold the entire string in a single element), or a sequence of char values separate by colons.  if the number of elements provided are greater than that expected by the function, the excess is silently ignored.

const short[] - a sequence of short values separate by colons. if the number of elements provided are greater than that expected by the function, the excess is silently ignored.

const int[] - a sequence of int values separate by colons. if the number of elements provided are greater than that expected by the function, the excess is silently ignored.

Internally, enums are ints, so this is how they are represented anywhere a binary value is used.  For the command line client, enums can be input as either the int value, or as the (case insensitive) string equivalent.  For enums which are bit-fields, you can specify multiple enums with separating then with a plus character (no spaces!)

Data returned from a function is output one line per element, with a single newline ('\n') character terminating the line.

Data returned from the function is formatted according to the mode.  There are separate modes for primitive types and Arrays.<

Primitive types must be of one of the following modes: Hex, Dec, Oct.  These correspond to hexadecimal (e.g. 0x1234), decimal (e.g. 20), and octal (e.g. 017) output.

Array types can be any of the primitive types, or Str.  This last mode corresponds to an ASCII representation of the value.

In addition to these mutually exclusive modes denoting the base the number of output in, there is an additional flag-like mode called "Assign".  When this flag is set, all outputs are returned formatted as name=value pairs.  This allows the output of the tsctl client to be evaluated by the shell to produce shell variables containing the results of the command(s) invoked.  The name field is equal to the name of the function called, followed by an underscore and a count number appended to it, to distinguish it from other possible calls to the same function.  Some functions such as System MapGet return multiple pieces of data; in these cases the number will be incremented multiple times for the same function. At the end of invoking multiple commands, a summary assignment is produced for each command issues, in the form Count_name=#.  (The italicized portions are replaced by each command name and the number of invocations, respectively)

When the "Assign" flag is not set ("NoAssign"), the output contains only value.

The mode can be changed using the Mode commands.  These resemble tsctl object commands in which the object to be accessed is of the pseudo-class "Mode" followed by the command which is the mode: Assign, NoAssign, Hex, Dec, Oct, AStr, AHex, ADec, and AOct.  The last four are the modes for Arrays.

Enums are always formatted as their string representation.  For enums which are flags, each flag present is separated by a "+" (no spaces).

Relationship between C function and tsctl network packet sequence

The tsctl network API defines a command to consist of a request packet which is sent to the server, followed by a reply packet which is sent back to the client in response to the request.

All parameters to a function which are primitive types or const pointers and/or arrays directly correspond to the request packet.  Each of these elements is packed by the client into the packet in the same order as it occurs in the parameter list, with no padding between elements.  Primitive types are packed in network order.  Arrays are packed as the array length (as int), followed by each element of the Array, while arrays are packed without a preceding length as their length are known constants.  Enums are treated as int.

The return value from the function, followed by all non-const pointers and/or arrays passed to the function correspond to the reply packet.  Each of these elements is packed by the server into the packet, starting with the return value and continuing with any additional data in the same order as it occurs in the parameter list, with no padding between elements. Primitive types are packed in network order.  Arrays are packed as the array length (as int), followed by each element of the Array, while arrays are packed without a preceding length as their length are known constants. Enums are treated as int.

To aid in generic interpretation of the reply values from the tsctl server it is possible to enable "Tagged" mode. In Tagged mode, each element in the reply is tagged as follows:

byte 0:

bit 7 = 1 (enum or void)

bit 6 = 1 (enum)

bit 5:0 = enum# (0-63)

or

bit 7 = 1 (enum or void)

bit 6 = 0 (void)

bit 5:0 = 0 (reserved)

or

bit 7 = 0

bit 6 = array(1) or not array(0)

bit 5:4 = 00 Unsigned Int

bit 3:0 = int size bytes - 1

or

bit 7 = 0

bit 6 = array(1) or not array(0)

bit 5:4 = 01 Int

bit 3:0 = int size bytes - 1

or

bit 7 = 0

bit 6 = array(1) or not array(0)

bit 5:4 = 10 Real

bit 3:0 = real size bytes - 1

or

bit 7 = 0

bit 6 = array(1) or not array(0)

bit 5:4 = 11 Struct

bit 3:0 = number of fields - 2

The tsctl command-line client operates in several stages:

1. All commands are converted to the tsctl binary format.  During this step, any time a numeric value is expected, but a text value is found, the text is stored in a separate look-up table along with the offset in the binary stream at which the value corresponding to the text occurs, and this value in the stream is set to zero.
2. Any values in the look-up table are converted into numbers by calling System function MapLookup, and the results is patched into the binary stream at the location previously specified.
3. The resulting binary protocol stream is fed to the tsctl server interpreter, which dispatches unmarshals the argument to each command and calls the corresponding underlying C function for the specified object.  The data returned by each function is marshalled back into the binary format and tagged.
4. The tsctl client then passes the binary reply to the generic tagged data writing functions, correlating each datum to the class and command name it belongs to.  The tagged data writing functions then generate output based on the current mode settings which including encoding and base formats.

The tsctl client provides command-line options to selectively enable these stages (NOTE: UNIMPLEMENTED).

--compile performs only stage 1. The output is a binary sequence of bytes containing two sections, the first containing the binary format the for requests and the second section containing the look-up table. Support is planned for outputting these sections in a format suitable for reading in subsequent invocations of tsctl, or for putting in your own C, Python, Javascript, etc. program.  Compiling text commands allows you to

--binary reads the output of --compile as its input and performs stages 2-4.

--skip-lookup does not read a look-up table from the input and skips tags 2.  Only used in conjunction with --binary.

--lookup-only is used only in conjunction with --binary.  It only performs stage 2, then outputs the resulting binary protocol for the requests.

--raw-output skips stage 4.  The raw binary reply data from the server is output.  The result is the same as if raw binary output mode is explicitly specified in the request for the entire duration of the requests.

--raw-untagged disables output tagging.  Implies and automatically enables the --raw-output option, as the normal output stages rely on tagging to render output.

The tsctl server listens on two ports: 8000 and 5001.  The only different between the two ports in the default mode for communicating with the server.

port 8000:

all input is ignored until 2 empty lines are received.  This is for receiving an HTTP POST.

input defaults to text commands

output defaults to JSON, AStr, Dec

port 5001:

input defaults to binary format

output defaults to binary format