STO2
Command & Control Computer ACE
To
Instrument Computer Communication
Interface Control Document
0.
Document version Control
Rev #
1
2
3
4
5
6
7
8
Date
12/03/2008
08/18/2009
08/25/2009
09/29/2009
03/12/2011
08/19/2011
10/11/2011
10/20/2011
Issued by
Pietro Bernasconi
Craig Kulesa
Pietro Bernasconi
Pietro Bernasconi
Pietro Bernasconi
Pietro Bernasconi
Pietro Bernasconi
Pietro Bernasconi
9
10
11
10/26/2011
12/20/2011
02/13/2015
Pietro Bernasconi
Pietro Bernasconi
Pietro Bernasconi
12
06/26/2015
Pietro Bernasconi
Description
Initial version
Second draft
Third draft
4th draft
5th draft
Added DATASEND command
Added DACWATCH command
Added GONDOLAALT and SHUTDOWN
commands
Added SOCKETSEND command
Added FILESEND command
Largely updated document with more information.
- Modified syntax for commands:
OPENHOUSEKPCH
KILLHOUSEKPCH
- Added the following commands:
SCIENCESENDOFF
SCIENCESENDON
Modified the STARTSCAN syntax
Table of Contents
1.
0.
Document version Control ...................................................................................................... 2
1.
Table of Contents .................................................................................................................... 3
2.
Scope ....................................................................................................................................... 4
3.
Block Diagram ........................................................................................................................ 4
4.
CCC Processes ........................................................................................................................ 5
5.
Communication channels ........................................................................................................ 5
IP Addresses: .......................................................................................................................... 5
Socket Ports: ........................................................................................................................... 5
5.1
Watchdog Socket.............................................................................................................. 6
Description .............................................................................................................................. 6
Protocol ................................................................................................................................... 6
Commands .............................................................................................................................. 6
5.2
MOC/ACE Command Socket .......................................................................................... 7
Description .............................................................................................................................. 7
Protocol for instrument specific commands ........................................................................... 7
Protocol ................................................................................................................................... 7
ACE Special Commands and Syntax ...................................................................................... 8
5.3
ICI/SOC Command Socket ............................................................................................ 13
Description ............................................................................................................................ 13
Interface Details .................................................................................................................... 14
5.4
Housekeeping Socket ..................................................................................................... 14
Protocoll: ............................................................................................................................... 14
5.5
Science and Quicklook Data Transfer to the Ground .................................................... 15
Description ............................................................................................................................ 15
Downlinking ......................................................................................................................... 15
Ground handling of DAC data .............................................................................................. 15
6.
Parameters values: ................................................................................................................ 15
7.
Command Data Flow Block Diagram................................................................................... 16
2.
Scope
This document defines the communication channels between the Autonomous Control Executive
(ACE) process running on the Command & Control Computer (CCC) the pointing control
computer (MAX3) and the instrument computer commanding the STO instrument. From now on
in this document the instrument computer is simply called the DAC (Digital Acquisition
Computer). Both the physical communication channels and the communication protocols are
described.
3.
Block Diagram
4.
CCC Processes
The CCC runs two separate processes:
 The Instrument Control Interface (ICI) process. It is a simple continuous loop process
that interfaces with the ACE process and the other gondola subsystems, including the miniSIP that handles telecommunications. Data communications between the ICI and other
gondola computers and subsystems are via Ethernet or serial interfaces.
 The Autonomous Control Executive (ACE) process. It manages scheduling of gondola
operations such as telescope deployment, setting track state, setting telescope focus, filter
settings, etc. ACE software implements science mission objectives such as scheduling of
target observations and instrument calibrations. The ACE process is also the main
commanding interface between ground and the gondola subsystems and the two science
instruments. It receives the commands from the ICI process, interprets and executes them. If
the commands are directed to the instrument computers it forwards them to the appropriate
instrument via an Ethernet socket.
5.





Communication channels
The ACE and DAC are connected via an Ethernet line.
The IO is established via sockets.
3 Sockets will be opened between the two computers.
2 Data transfer folder are allocated in the ACE file system to allow transfer of data from the
DAC to the ACE for downlinking
UDP packets are streamed from the MAX3 computer to the DAC
IP Addresses:
ACE IP:
192.168.0.10
MAX3 IP:
192.168.0.13
DAC IP:
192.168.0.11
As a general rule for the gondola network all IP addresses < 192.168.0.100 are reserved to
gondola services. If the science payloads need addresses < 100 permission must be requested to
the Gondola Systems Engineer Harry Eaton.
Socket Ports:
MOC/ACE Command port: .. 6001
SOC/ICI Command port: .... 14201
Watchdog port:...................... 6005
Housekeeping port: ............... 6020
UDP position stream port: .... 1718
5.1
Watchdog Socket
Description
The watchdog socket is used to send a RESET command from the ACE to the DAC. On the
DAC computer the Watchdog is a process separate from the actual SISSI process. If the SISSI
process for some reason dies or hangs then the ACE has the option to kill it and restart the SISSI
process WITHOUT actually rebooting the entire DAC system.
The socket is created by the ace which will be listening until either the DAC establishes the
connection or the DAC_CONNECT_TIMEOUT timer expires.
The default value of DAC_CONNECT_TIMEOUT is set to 40 seconds (Section 0 summarizes
all default values for configurable parameters). If the timer expires a flag in the ACE parameters
space is set indicating a failure to establish the data connection.
Protocol
The commands to the Watchdog process are \n terminated strings.
“<string with command to DAC Watchdog> \n”
The maximum total length of the string is 256 characters.
After sending the command to the DAC Watchdog the ACE will be listening for the reply until
up to DAC_RESET_TIMEOUT seconds. Currently the DAC_RESET_TIMEOUT is set to 30
seconds. The ACE will try to reset the DAC and resend the command to the DAC Watchdog up
to DAC_RETRIES times. DAC_RETRIES is currently set to 5. If after DAC_RETRIES it fails
to receive a positive reply from the Watchdog the ACE will set a flag in its internal parameter
space stating that the DAC is not responding. At this point a manual reboot of the DAC system
will be necessary.
Commands

Simple acknowledgement request from ACE (e.g. to check that the command
channel is still working or that the DAC is still alive or to see what kind of
problems it has):
ACE command:
“OK\n”
WAtchdog reply:
“OK\n”

ACE commands the Watchdog to kill and restart the SISSI process:
ACE command:
“DAC_RESET\n”
WAtchdog reply:
“OK\n”
5.2
MOC/ACE Command Socket
Description
The command socket is used to send commands from the ACE to the DAC. It is also used for
direct commanding from the ground MOC to the DAC via the ACE process on the CCC. Upon
receiving the command the DAC will execute the command and reply to the ACE accordingly.
Either with an acknowledgement or by sending back parameter values as requested by the ACE.
The socket is created by the ace which will be listening until either the DAC establishes the
connection or the DAC_CONNECT_TIMEOUT timer expires. Currently in the ACE the value
of CONNECT_TIMEOUT is set to 120 seconds. If the timer expires the ACE will assume the
DAC process is dead and will send a reset command to restart the process.
Protocol for instrument specific commands
Instrument specific commands are simply passed from the ACE to the DAC without
interpretation by the ACE. The command internal syntax can be determined by the programmers
of the DAC instrument control software. Only the following syntax rules are imposed on those
commands:
The commands are \n terminated strings.
“<string with command to DAC> \n”
The maximum total length of the string is 256 characters.
After sending the command to the DAC the ACE will be listening for the reply until up to
DAC_WAITSECONDS seconds.
The reply must be:
“OK\n”
Acknowledgement or Command performed successfully
“OK string\n” Command performed successfully and returning a string as output
from the command executed.
“ERROR <Error string>\n” DAC is alive but having a problem
The total length of the DAC reply string is limited to 256 characters.
If the timer expires, the ACE will resend the same command to the DAC up to DAC_RETRIES
times. If after DAC_RETRIES times the DAC still has not responded to the command the ACE
will assume the DAC process is dead and will send a reset command to the BIRC watchdog to
restart the process.
Protocol
There are a number of commands (“Special” commands that the ACE also understands and may
either be issued by the ACE itself autonomously or from the ground.
The special commands are also \n terminated strings:
“<string with command to DAC> \n”
The string is composed by first the command in human readable format ALL CAPS, followed by
a series of parameters (if needed) and ending with a \n character.
The reply from the DAC follows the same pattern.
After sending the command to the DAC the ACE will be listening for the reply until up to
DAC_WAITSECONDS seconds. If the timer expires, the ACE will resend the same command
to the DAC up to DAC_RETRIES times. If after DAC_RETRIES times the DAC still has not
responded to the command the ACE will assume the DAC process is dead and will send a reset
command to the DAC watchdog to restart the process.
ACE Special Commands and Syntax

Simple acknowledgement request from ACE (e.g. to check that the command
channel is still working or that the DAC is still alive or to see what kind of
problems it has):
ACE command:
“OK\n”
DAC reply:
“OK\n”
DAC working nominally
“ERROR <string>\n” DAC is alive but having a problem
 ACE sends to the DAC heads up of imminent gondola power shutdown:
ACE command:
“SHUTDOWN\n”
DAC reply:
“OK\n”
Acknowledged and ready for shutdown
“ERROR <string>\n” DAC is alive but having a problem
Note: The DAC will wait to reply until all its subsystems are orderly shutdown.
The ACE will wait until its TIMEOUT timer of 40 seconds expires.

ACE tells the DAC to open the housekeeping socket channel:
ACE command:
“OPENHOUSEKPCH\n”
DAC reply:
“OK \n”
Acknowledged
“ERROR <string>\n” DAC is alive but having a problem

ACE tells the DAC to close the housekeeping channel
ACE command:
“KILLHOUSEKPCH\n”
DAC reply:
“OK \n”
Acknowledged
“ERROR <string>\n” DAC is alive but having a problem

ACE tells the DAC how often a housekeeping scan is required:
ACE command:
“HOUSEKEEPING <INTERVAL>\n”
DAC reply:
“OK \n”
Acknowledged
“ERROR <string>\n” DAC is alive but having a problem
where INTERVAL is an unsigned integer telling DAC the housekeeping interval
in seconds.
 ACE informs the DAC of the path in ACE’s file system where to send the science
data file:
ACE command:
“SCIENCEPATH <PATH>\n”
DAC reply:
“OK\n”
Acknowledged
“ERROR <string>\n” DAC is alive but having a problem
Where PATH is a string containing the path that the DAC will use to store the
science data files on the ACE file system.
 ACE informs the DAC of the path in ACE’s file system where to send the quicklook
file:
ACE command:
“QUICKLOOKPATH <PATH>\n”
DAC reply:
“OK\n”
Acknowledged
“ERROR <string>\n” DAC is alive but having a problem
Where PATH is a string containing the path that the DAC will use to store the
quicklook data files on the ACE file system.
 ACE sends to the DAC information about the gondola altitude:
ACE command:
“GONDOLAALT <ALTITUDE>\n”
DAC reply:
“OK\n”
Acknowledged
“ERROR <string>\n” DAC is alive but having a problem
Where:
ALTITUDE = unsigned int: altitude in METERS
 ACE notifies the DAC to STOP sending science data to the ACE science partition
because the partition is full:
ACE command:
“SCIENCESENDOFF\n”
DAC reply:
“OK\n”
Acknowledged
“ERROR <string>\n” DAC is alive but having a problem
 ACE notifies the DAC that it can resume sending data to the ACE science partition:
ACE command:
“SCIENCESENDON\n”
DAC reply:
“OK\n”
Acknowledged
“ERROR <string>\n” DAC is alive but having a problem

ACE informs the DAC that a scan will start with a certain scan ID:
ACE command:
“STARTSCAN <ID> <TYPE> <VERBOSITY>
<OT_ROW#> <OBJECT_NAME>\n”
DAC reply:
“OK \n”
“ERROR <string>\n” DAC is alive but having a problem
Where:
ID is an unsigned integer indicating the scan number.
TYPE is a string that indicates:
“SRC” for a position-switched on-source scan
“REF” for a position-switched reference scan
“OTF” for an On-The-Fly scan.
“HOT” to acquire a hot load calibration measurement
“COLD” to acquire a cold load calibration measurement
VERBOSITY is an integer that determines how much data is returned and by
which method. It is interpreted by the DAC as the following:
0: Silent mode; return no data to ACE
1: Return quicklook data to ACE
2: Return full science frame as taken by DAC
The exact data returned by the VERBOSITY flag depends on the
observation TYPE.
OT_ROW# is the row number in the Observation Table that commanded this
scan. If the value is -1 then it was not commanded by the OT but was a
manual scan command
OBJECT_NAME is a 15 charcters string describing the type of object observed
by the scan. Typically this is the object name from the OT

ACE informs the DAC that the scan sequence has ended and the DAC replies
whether the scan was recorded successfully or not:
ACE command:
“STOPSCAN\n”
DAC reply:
“OK 0 \n”
Scan recorded successfully
“OK 1 \n”
Scan is bad needs to be redone
“ERROR <ID>\n” DAC is alive but having a problem
 ACE requests the DAC to transmit old data for a particular SCAN IDs range:
ACE command:
“DATASEND <ID_strt> <ID_end> <PRIORITY>\n”
DAC reply:
“OK\n”
Acknowledged
“ERROR <string>\n” DAC is alive but having a problem
Where:
ID_strt = (UINT) indicating the range beginning scan ID.
ID_end = (UINT) indicating the end range scan ID. If ID_end is equal to ID_strt
that means that is one single scan and not a range
PRIORITY = (UCHAR) if 1 then send data to ACE immediately.
If 0 then send it next time verbosity is not 0.
 ACE requests the DAC to transmit old data for a particular FILE IDs range:
ACE command:
“FILESEND <ID_strt> <ID_end> <PRIORITY>\n”
DAC reply:
“OK\n”
Acknowledged
“ERROR <string>\n” DAC is alive but having a problem
Where:
ID_strt = (UINT) indicating the range beginning file ID.
ID_end = (UINT) indicating the end range file ID. If ID_end is equal to ID_strt
that means that is one single file and not a range
PRIORITY = (UCHAR) if 1 then send data to ACE immediately.
If 0 then send it next time verbosity is not 0.
 ACE sends to the DAC a command that the DAC will then send to one of the
instrument computers:
ACE command:
“SOCKETSEND <COMPUTER> <PORT> <CMD>\n”
DAC reply:
“OK <filename>\n” If the server response was multiple
lines and is sent to <filename> in quicklook
rdirectory.
“OK <server response>\n” If the server response was a
single line
“ERROR <string>\n” If input error or some other sort of
problem arose
Where:
COMPUTER = unsigned short: last part of the computer IP address to which the
command is to be sent to: 11, 12, or 14 (for 192.168.0.11,… etc.)
PORT = unsigned short: port number for the socket: 9001, 9002, 9003, or 9734
CMD = string: is the verbatim command that will be sent through the socket.

ACE commands the DAC to perform a system command.:
ACE command:
“SYSTEM <system command string>\n”
DAC reply:
“OK \n”
Command performed successfully
“OK string\n” Command performed successfully and
returning a string as output from the
command executed
“ERROR <string>\n” DAC is alive but having a problem
 CE instructs the DAC which parameter to watch by sending its value to the ground
through the housekeeping stream:
ACE command:
“DACWATCH <PARAMNAME> <SLOT>\n”
DAC reply:
“OK\n”
Acknowledged
“ERROR <string>\n” DAC is alive but having a problem
Where:
PARAMNAME = string: name of the parameter to watch
SLOT = unsigned char: slot number of the parameter to watch. 4 slots available:
0, 1, 2, 3

5.3
ACE informs the DAC as to how much background data processing should be
attempted:
ACE command:
“PROCESS <LEVEL>\n”
DAC reply:
“OK\n”
Acknowledged
“ERROR <string>\n” DAC is alive but having a problem
Where LEVEL indicates the level of processing to be performed on recently
acquired data. The DAC will examine all data taken during the mission as
applicable. These data are eventually sent to the “Science data” handling part of
ACE (either via socket or folder on CCC storage). The spatial extent is assumed
to be 1 square degree.
0: No processing will be performed in the background. (Stop processing.)
1: Return level 2+ data only:
moment maps for OTF
truncated spectra for PS
2: Return FITS cubes of spatial and spectrally-truncated data (OTF)
3: Return FITS cubes of spectrally truncated data (OTF)
4: Return nominal survey data FITS cubes (OTF and PS)
ICI/SOC Command Socket
Description
In addition to the command channel via the MOC/ACE, the STO Science Operation Center
(SOC) command terminal has access to a pseudo-terminal (PTY) on the instrument computer
through functionality provided by the ICI running on the CCC. The STO Instrument computer
will be connected to the CCC over its own dedicated interface.
The STO instrument developer is free to use this path as they see fit for INDEPENDENT
operations on their instrument. I.e. it is not intended for operations that are expected to have
coordinated action with the ACE, such as with the pointing control system
Interface Details
1. The CCC will attempt a TCP-CONNECT on the port once every 10 seconds until a
connection is established.
2. The CCC will send keep-alives to keep the connection open.
3. If the connection is ever broken, the CCC will resume attempts every 10 seconds to reestablish.
4. Once the connection is established, the CCC forwards any characters received (by the
CCC) to the ground SOC terminal screen; this is an ASCII terminal screen with no cursor
control (except LF), and the characters are buffered as strings (thus NULL characters are
not allowed).
5. Characters typed on the SOC ground terminal are sent up to the gondola (once the return
key is hit) and forwarded to the TCP-port, with a LF termination.
6. Upward strings are limited to no more than 64 characters.
7. There is a mechanism on the ground to see if an upward string has been received.
8. Through this path the average downlink data rate should not average more than
3kbits/second (300 characters per second).
9. The uplink data rate is less than 32 bytes per second.
10. The text coming up/down the terminal is locally logged on the SOC.
11. To establish this TCP connection the instrument developers may want to use netcat or
socat to link the socket on their instrument to a pseudo-terminal.
5.4
Housekeeping Socket
This socket is used to send Instrument status data from the DAC to the ACE which will forward
it to the ground. The socket is created by the ace which will be listening until either the DAC
establishes the connection or the DAC_CONNECT_TIMEOUT timer expires. If the timer
expires a flag in the ACE parameters space is set indicating a failure to establish the data
connection.
The ACE spawns a separate thread that constantly listens for new data packets sent by the DAC.
The DAC sends housekeeping updates to the ACE at the rate set with the command
“HOUSEKEEPING”. The ACE retransmits the data to the IO process for downlink to the
ground. On the ground this data will be displayed on the GSEOS console.
The data can be for example: DAC uptime, various temperatures of the instrument, data storage
available, position of different motors, ...
Protocoll:
The protocol is simple. The DAC sends a stream of bytes to the ACE terminated by a ‘\n’
character. This defines a housekeeping packet. The length of the packet shall be constant:
Example (6 bytes of housekeeping data): “a365fe28b28ffa\n”
5.5
Science and Quicklook Data Transfer to the Ground
Description
The transfer of science and quicklook data from the DAC to the ACE is done via trivial file
transfer protocol (TFTP).
The ACE sets up two directories in its file system where the DAC can transfer science and
quicklook data to them.
The ACE communicates the directory path to the DAC using the SCIENCEPATH and
QUICKLOOKPATH commands (see section 5.2). Prior to receiving this information the BIRC
will assume an unknown directory location and not transfer data to the ACE via this system.
After setting up the directories and communicated their paths to the DAC, the ACE will look for
files in these directories and queue them for uploading with priority given to quicklook data.
Downlinking
The ACE before dispatching the data files to the ground performs lossless compression using
bzip2 and error correction encoding. The resulting file is then divided in packets 128 bytes each.
Each packet is sent to the ground independently.
Ground handling of DAC data
The GSEOS on the ground receives the packets. Upon receiving the last packet it attempts to
reconstruct the original file. If packets are lost or corrupted it attempts to use the error correction
encoding to reconstruct the lost data. If successful the GSEOS then proceeds to uncompress the
file. If not sufficient data is available to reconstruct the file, the GSEOS will issue an error
message indicating the packets that were lost. An operator can then manually (for now) send
commands to the ACE to resend those packets so that the reconstruction can be performed
successfully.
Once a file is successfully reconstructed and decompressed the GSEOS looks at its ID (encoded
in a preamble packet) to see whether it is a quicklook or a science data file. Depending on what
it is it will be accordingly dispatched to the correct destination.
6.
Parameters values:
DAC_CONNECT_TIMEOUT ......60
DAC_RESET_TIMEOUT .............30
DAC_WAITSECONDS ................10
DAC_RETRIES ...............................1
seconds
seconds
seconds
times
7.
Command Data Flow Block Diagram
The diagram below shows the command/data flow on each of the ports.